x264 "Macroblock Tree Ratecontrol" testing (committed)
Warning: (not really) Highly Experimental
It's done and committed! Grab the latest x264 from x264.nl!
Download RC1 (Windows, old)
Download RC1 (Windows debug build, old)
Download 0.13 (Windows)
Download 0.13 (Windows debug build)
Download 0.11 (Windows, old)
Download 0.11 (Windows debug build, old)
Patch v0.09 (old)
Download 0.09 (Windows, old)
Download 0.09 (Windows debug build, old)
Changes in RC1:
1. Statsfile format changed; now takes only 2 bytes per macroblock. Varying-endianness now supported. Error handling improved.
2. More commandline validation fixes.
3. Clip propagation factors just in case they overflow.
4. Change default lookahead to 40 to save some memory.
5. It's now called --rc-lookahead, not --lookahead, at Akupenguin's request.
6. Various other things, just read the damn commit message in the patch.
Changes in 0.13:
1. Bugs fixed in the VBV patch. It's still separate, as per below.
2. B-adapt 1 now supported.
3. All B-adapt code is now merged into MB-tree code instead of duplicated. B-adapt 1 should be slightly better than it was before (even without MB-tree). In general, all three B-adapt modes now use the same basic framework.
4. MB-tree strength is now adjustable through qcomp. Raising qcomp lowers MB-tree strength; it works the same way that qcomp did before MB-tree, with qcomp 1 meaning constant frame quantizer. This lets you experiment a bit more; there's some possibility that the default qcomp 0.6 is not the psy-optimal value--it might be higher.
5. New --no-psy option, which disables all psy optimizations in x264 that don't improve PSNR or SSIM. This includes psy-RD/trellis, but also the recently-added chroma lambda offset and the magical "num_frames = j" line that causes MB-tree to pretend that non-scenecut I-frames don't exist.
6. "Fast" preset is now slower, "faster" preset added to cover the gap.
7. MB-tree is now default on presets "fast" and higher. Lookahead values are smaller for faster presets.
Changes in 0.11:
1. VBV now supported. Includes a separate VBV patch which fixes most VBV issues that MB-tree has; this will be committed separately, as it doesn't require MB-tree. It should also improve 1-pass VBV (especially CBR) performance on clips with lots of static content, e.g. anime, CGI.
Do note that CBR doesn't benefit much from MB-tree though, especially with small buffer sizes.
2. Lots of trivial parameter-handling fixes, e.g. lookahead automatically disabled on second pass.
How do I use it?
It's on by default. --no-mbtree turns it off.
What doesn't it work with yet?
How can I tweak it?
--lookahead; default is 50, higher takes more memory and is slower, but should give better results. It's probably overkill for most sources. If you get crashes immediately after the lookahead fills up, you probably ran out of memory (max is 2GB for x264 on 32-bit Windows).
--qcomp; works the same as it did before with actual qcomp, higher numbers lower MB-tree strength.
How much does it cost, speed-wise?
A bit. Not too much though. Speed cost is linearly proportional to --lookahead.
How can I help test it?
Make sure you don't get any odd errors or warnings, such as asserts failing, crashes, etc. Try various combinations of the relevant settings (b-adapt, bframes, mbtree, lookahead, pass 1/2, crf, keyint, scenecut) that I might not have thought of.
Try it on various sources; see if you can find places where, with --tune ssim, at the same bitrate as without mbtree, it gives worse SSIM (or, equally, places where Global PSNR goes down with tune psnr).
Make sure it doesn't fall apart visually either.
Also, note that CRF is redefined again by this patch--I set it so that it kept the same bitrate on Blackpearl as before, but it may vary for different videos.
What does it actually do?
It tracks the propagation of information from future blocks to past blocks across motion vectors. It could be described as localizing qcomp to act on individual blocks instead of whole scenes. Thus instead of lowering quality in high-complexity scenes (like x264 currently does), it'll only lower quality on the complex part of the scene, while for example a static background will remain high-quality. It also has many other more subtle effects, some potentially negative, most probably not.
This helps at all bitrates, but can even help at phenomenally low bitrates where the video would otherwise fall apart completely, like in this 67kbps anime clip.
Should I go around distributing this patch and getting everyone I know to use it for their everyday encodes?
No. The commandline options here may change in the future, as may the statsfile format and various other aspects of the patch, so it's not a good idea to make this patch something that gets put in ordinary x264 builds. Furthermore, I might have broken something unrelated to mbtree.
Why does it give different output even without mbtree?
B-adapt (all modes) was slightly modified to work better with mbtree, so the frametype decisions may change somewhat.
What improvement does it give?
I've gotten up to a 70% SSIM increase.
You're joking, right?
Seriously? How the hell can you get a 70% quality increase?
OK, you're just making this u--wait a minute, what's this new statsfile, and why is it using up all my hard disk space?
Oh, that? Kyahahahahahaha.