Fix AMF encoders reported as broken, harden AMF ffmpeg args - #55
Open
StuartCameronCode wants to merge 1 commit into
Open
Fix AMF encoders reported as broken, harden AMF ffmpeg args#55StuartCameronCode wants to merge 1 commit into
StuartCameronCode wants to merge 1 commit into
Conversation
Addresses #51. The availability probe encoded a 64x64 frame. Hardware encoders have minimum dimensions and AMD AMF rejects that size, so VapourBox marked working AMF encoders as broken — the reporter confirmed the same command succeeds at a larger size. Probe at 512x512 (AMF's floor is 192x128 on some ASICs); still one frame, so the cost is unchanged. Verified the larger frame keeps both VideoToolbox encoders passing. The reported ffmpeg crash under CQP (exit -1073741819 = access violation) can't be reproduced without AMD hardware, so this addresses the two things in our invocation that could plausibly cause it: - AMF is now pinned to nv12. Left to negotiate, a >8-bit source reaches the encoder as p010, and 10-bit HEVC encode only exists on some AMD ASICs — where it doesn't, the AMF runtime faults rather than failing cleanly. h264_amf has no 10-bit mode at all. Custom FFmpeg Arguments still override it, so 10-bit HEVC stays reachable where supported. - -qp_b is set alongside -qp_i/-qp_p. B-frames were left on the encoder's default QP while I and P were pinned, so a GOP could mix the chosen quality with an unrelated one. Separately, presets can no longer cross encoder families. Each family has its own vocabulary (x264 ultrafast…placebo, NVENC p1…p7, QSV veryfast…veryslow, AMF speed|balanced|quality) and a saved preset can pair a codec with the wrong one; ffmpeg then rejects the option and fails the encode. normalized_preset() substitutes the family default.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #51. Deliberately not "Fixes" — one half is confirmed, the other isn't (see below), so this shouldn't auto-close the issue.
AMF reported as broken — diagnosed, fixed
The functional availability probe encoded a 64x64 frame. Hardware encoders have minimum dimensions and AMF rejects that size, so VapourBox marked working AMF encoders unavailable. The reporter confirmed the same command succeeds at a larger size.
Probe is now 512x512 (AMF's floor is 192x128 on some ASICs, and NVENC/QSV have their own). Still a single frame, so the cost is unchanged. Verified locally that both VideoToolbox encoders still pass at the larger size.
CQP crash — addressed, not confirmed
Exit
-1073741819is0xC0000005, an access violation inside the AMF runtime. I can't reproduce this — there's no AMD hardware here or in CI. The reporter notes CQP works for them outside VapourBox, which points at our invocation rather than CQP itself, so this fixes the two candidates in it that are independently justifiable:nv12. We forced no pixel format, so a >8-bit source reached the encoder asp010. 10-bit HEVC encode exists only on some AMD ASICs, and where it doesn't the runtime tends to fault rather than fail cleanly;h264_amfhas no 10-bit mode at all. Given the reporter is feeding camera footage, which is frequently 10-bit, this is the main suspect. Trade-off: AMD users lose 10-bit HEVC output by default. Custom FFmpeg Arguments still override it (a later-pix_fmtwins), so it stays reachable on hardware that supports it.-qp_bis now set alongside-qp_i/-qp_p. B-frames were left on the encoder's default QP while I and P were pinned, so a GOP could mix the chosen quality with an unrelated one. Not a crash mechanism by itself, but wrong.If neither lands, the next step needs a reporter-side test build rather than more guessing from here.
Adjacent bug found while reading
Encoder presets could cross families. Each has its own vocabulary — x264
ultrafast…placebo, NVENCp1…p7, QSVveryfast…veryslow, AMFspeed|balanced|quality— and a saved preset can pair a codec with the wrong one. ffmpeg then rejects the option and the whole encode fails.VideoCodec::normalized_preset()substitutes the family default; the Rust preset lists must stay in step withapp/lib/models/video_job.dart.Testing
-qp_b, and preset fallback across three families (plus a check that valid presets pass through untouched)subtitle_integration_testfails locally only because the whisper model isn't downloaded on this machine; CI provisions it.)CLAUDE.md gains a Hardware Encoders section covering the three invariants, since each fails in a way that looks like "this GPU doesn't work" rather than an error.