media-import-tooling

Helpers for turning MakeMKV rips into a well-formed media library: identifying what every title on a disc actually is, naming and flagging audio and subtitle tracks, and generating thumbnails for extras.

These are small, single-purpose scripts meant to be run by hand while working a disc. Nothing here is a pipeline — the judgement calls stay with the operator.

Which media server?

Built and used against Jellyfin, and the library-layout advice here reflects that. But most of what these scripts do happens below the media server: they read discs, and they write names and flags into Matroska headers with mkvpropedit. A track labelled Commentary by Ben Affleck and Chris Terrio and flagged flag-commentary reads correctly in Jellyfin, Plex, Emby, Kodi, VLC and anything else that understands Matroska, because the metadata lives in the file rather than in a server's database.

So if you are running something else, the identification and tagging tools should work unchanged. What differs is the layout around the files:

	Jellyfin / Emby	Plex
Extras	extras/, plus typed folders (interviews/, featurettes/, …)	typed folders (Featurettes/, Interviews/, …) or -behindthescenes filename suffixes
Trailers	trailers/	Trailers/ or -trailer suffix
Alternate cuts	labelled files in one folder	Movie (Year) {edition-Extended}.mkv
Extra thumbnails	<basename>.jpg beside the video	handled by the server; sidecars are ignored

Patches that make the layout side configurable for other servers are welcome. The per-file work needs no such accommodation — it is already portable.

Requirements

ffmpeg / ffprobe, mkvtoolnix (mkvpropedit, mkvmerge), Python 3. No third-party Python packages; numpy is deliberately not used so the scripts run anywhere.

The scripts

Script	What it answers
lib/scan_titles.py	What's on this disc? Parses a makemkvcon info log into a title table sorted by runtime.
lib/scan_streams.py	What's in one title? Per-stream detail from the same log, including MakeMKV's stream flags.
lib/rip_disc.py	Rip titles, verifying each output is the title that was actually asked for.
lib/rip_result.py	Did that rip actually save anything? makemkvcon exits 0 even when it saves nothing.
lib/tag_episode.py	Name and flag one episode's tracks: commentaries, companion cores, forced subtitles.
lib/probe_mkv.py	What's in the finished file? Stream list with names, languages and flags.
lib/audio_identify.py	Is this unnamed track a commentary, descriptive audio, a dub, or a lossy companion core?
lib/subtitle_inspect.py	Is this subtitle track full or forced, and is it SDH?
lib/pickthumb.py	Pick a representative thumbnail frame for an extra.

Typical run against a disc:

makemkvcon -r --cache=512 --minlength=15 info disc:0 > scan.log
python3 lib/scan_titles.py scan.log            # pick the feature, spot duplicates
python3 lib/scan_streams.py scan.log 4         # inspect the feature's streams
python3 lib/rip_disc.py scan.log ./out 4 7 9   # rips AND verifies it got those titles
python3 lib/rip_result.py ./out/riplog-t4.txt --expect ./out/*_t04.mkv
python3 lib/probe_mkv.py feature.mkv
python3 lib/audio_identify.py feature.mkv --main 0:a:0 \
        --candidates 0:a:2 0:a:18 --controls 0:a:3 0:a:5
python3 lib/subtitle_inspect.py feature.mkv --survey   # full vs forced, all streams

scan_streams.py decodes MakeMKV's per-stream flags, which name the companion core outright and mark the derived forced-subtitle views. Trust a set bit; a clear bit means nothing, since plenty of discs leave real commentaries unflagged.

Things that have burned us

Recorded because each one cost real time or produced a wrong answer.

MakeMKV subtitle scan counts are not stream counts. The scan adds a derived (forced only) entry for streams containing forced segments, so scan totals exceed real counts — but not by a clean factor. Guardians Vol. 3: 14 → 7 (all paired). The Untouchables 4K: 52 → 34 (only 18 paired). Argo 4K: 64 → 40. Compare against the finished file, never arithmetic on the scan. Never re-rip on a raw count mismatch.

Discs expose the feature twice. Usually a playlist (with chapters) and a bare .m2ts (without). Rip the playlist. But check the streams before assuming: on the Argo Blu-ray the second copy carried a DTS Express track, which is the secondary-audio codec — that copy was the picture-in-picture playlist, not a bare duplicate.

A failed phase-inversion null does not mean unrelated content. Descriptive audio is a different mix of the same film, so it cannot cancel even though it contains the film. Use envelope correlation plus the silent-gap lift, and always run known dubs as controls. See the table in audio_identify.py.

Envelope correlation measures timing, not content. A foreign dub correlates highly with the original because dubbed dialogue lands where the original dialogue was. It is the lift in the film's silent gaps that separates dub from descriptive audio.

Lossy cores are exposed as separate selectable tracks. DTS-HD MA carries a DTS core, TrueHD carries an AC-3 core. Name the core identically to its parent so the player doesn't offer a mix that isn't distinct. When channel counts differ (7.1 TrueHD with a 5.1 AC-3 companion), say "companion" explicitly rather than mislabelling the channel count.

PGS emits two packets per subtitle. A display segment carrying the bitmap, then a small "clear screen" segment ending it. Counting every packet roughly doubles the apparent cue count, and picking timestamps without filtering lands on clear segments, which render nothing at all. Filter to packets over ~1 KB.

Subtitles sit in the letterbox bar. On scope films a crop of the lower third misses them entirely. Render full frames.

Extras open on black. Nearly every disc extra begins with black or an ident, so a player generating its own thumbnail gets nothing useful. pickthumb.py skips the head and tail and scores candidates; write the result as a .jpg sidecar beside the video.

signalstats has no YDEV. It emits YLOW/YHIGH/YDIF. Asking for a metric that doesn't exist silently yields zero and any gate built on it passes nothing.

MakeMKV sanitises the disc name when it builds output filenames. The name it reports can contain characters the name it writes does not. A Monty Python disc reported

MONTY PYTHON’S LIFE OF BRIAN     (U+2019, bytes e2 80 99)

and wrote

MONTY PYTHON'S LIFE OF BRIAN_t01.mkv   (ASCII apostrophe, 0x27)

Anything that predicts the output path by pasting the reported name together with the title number therefore looks for a file that was never created. Four good rips were reported FAILED with "0 read errors" while sitting on disk, and the checker deleted nothing only because there was nothing at the path it guessed. Find the output by its _tNN.mkv suffix instead; the suffix is MakeMKV's own and survives any mangling of the leading name.

A title's container duration can be much longer than its video. A Blu-ray menu asset on the same disc held 20 seconds of video (480 frames at 23.976) over a five-minute audio bed, so the container reported 300s against the scan's 20s. That is not a bad rip and not a wrong title -- it is how menus are authored. Compare durations as advisory only; the playlist is what settles identity.

A child process reading stdin eats the script that is driving it. ffmpeg, whisper and makemkvcon all read standard input, and a subprocess inherits the parent's. When these scripts are driven from a shell heredoc, the script is stdin, so a child that consumes a few bytes deletes them from the commands that have not been read yet. The failure surfaces on a LATER line and looks nothing like its cause: a loop over four titles reported

bash: line 14: rab: command not found

because ffmpeg had swallowed the g of the next grab call, and the third title was silently never processed at all. Nothing errors at the call site, and re-running by hand always works, because an interactive shell is not a heredoc.

Every subprocess.run here now passes stdin=subprocess.DEVNULL. Do the same in any wrapper you write, and pass -nostdin to ffmpeg invoked from a shell.

Bracketed paths break shell globbing. [tmdbid-120] is a character class, so Python's glob.glob, and sed/parameter expansion in shell, all silently return nothing. Use os.listdir and pass paths as arguments.

makemkvcon exits 0 even when it saves nothing. A rip that hit an unreadable sector printed Copy complete. 0 titles saved, 1 failed. and still returned 0. Driving it in a loop over a season, $? will happily report success for an episode that produced no file. Check with rip_result.py, which parses the completion message and confirms the output exists.

Parse the message codes, not the text — and note that success and failure use different codes. A clean run emits 5036/5005 carrying only a saved count; a run with failures emits 5037/5004 carrying saved and failed. A checker that knows only the failure pair calls every successful rip a failure.

Read errors alone do not condemn the output. The pass that recovered a failed episode still logged 5 of them; MakeMKV retried through, the content hash table verified the M2TS, and the file decoded start to finish. Errors mean the media is marginal, not that the result is bad — but do compare the reported byte offsets across runs. A repeat at the same offset is a fixed defect; a moving one is a marginal read worth retrying.

makemkvcon mkv does not always enumerate the same titles as makemkvcon info. With identical flags on the same disc, info offered 29 titles and mkv offered 26 -- it silently dropped the episode's playlists because their first sector was unreadable. Every index then pointed at a DIFFERENT title, one or more positions along. Twenty titles ripped, every completion message said success, every output file existed, and not one was the title requested; four were duplicates of each other and the episode was absent entirely.

Neither "did it save something" nor "does the file exist" detects this, because both were true. The check that does is comparing each output's DURATION against the duration the scan recorded for that title -- rip_disc.py does this and aborts on the first mismatch rather than the twentieth. Treat a divergence as a reason to stop, not an off-by-one to correct: the omissions need not be contiguous.

Title numbers restart on every disc. Each disc in a set has its own t00, t01, t05. Anything keyed on the title tag alone — a cache, a temp file, a results dict — collides across discs of the same set, and because the stream indices also line up, the wrong data reads as entirely plausible. On a Game of Thrones box set this made disc 2's E04 analyse disc 1's E01 subtitle packets, and the only visible symptom was Italian ending up with a forced track and no full track, which no disc ships. Key on the full disc-qualified filename.

mkvpropedit changes only what you name. It has no notion of resetting a track to defaults, so a second pass that renames tracks leaves every flag from the first pass in place. After a wrong pass this yields correct names over stale flag-forced / flag-commentary bits — the hardest kind of error to see, because everything you look at reads right. Write every flag explicitly as 0 or 1 on every track, so the operation is idempotent and self-correcting.

The last audio track is not necessarily a commentary. It usually is when a disc carries one, but episodes and cuts within one set differ: on a Game of Thrones disc, two episodes ended with an English commentary and the third ended with Russian DTS, because that episode has no commentary at all. Require the trailing track to be the expected language and past the known film tracks before labelling it, and fail loudly rather than guessing.

A missing track can identify content. The inverse is useful: when published sources say which episodes carry commentary, the episode whose audio count is one short is identifiable from the scan log alone, before decoding anything. Cheap corroboration for a runtime-based guess — and on that disc the runtimes were not monotonic with episode order, so runtime alone would not have ordered them.

Duplicate titles are not always duplicates. Two menu loops with the same duration and near-identical frames still differed by video-stream MD5 and by 2 MB — two encodes of one montage, not one file twice. Hash the video stream (ffmpeg -map 0:v:0 -c copy -f md5 -) before calling anything a second copy; whole-file hashes are useless here because MakeMKV writes distinct container metadata on every invocation.

Jellyfin notes

These are Jellyfin-specific; skip them if you run something else.

Extras cannot own NFO metadata (upstream closed as not-planned), so descriptions have to be pushed through the API and written as sidecars separately. Images are picked up from <basename>.jpg beside the video via the local image provider.

Jellyfin 12.0 disables legacy API authorization. Only the Authorization header with the MediaBrowser scheme and the ApiKey query parameter (capital A, capital K) survive; the lowercase api_key parameter and the X-Emby-* headers are rejected. Also: auto-resolved alternate versions are dropped on upgrade and a full library scan is required afterwards to rebuild them.

License

MIT — see LICENSE.