Problems with reencoding HEVC videos on 14.0.0 and 14.0.1

[dumbledore]

I am observing two problems when reencoding HEVC videos on 14.0.0 and 14.0.1 on Windows 11 (haven't tried on Ubuntu or MacOS so far):

1. Reencoding HEVC videos produces videos with artefacts: They start OK for a second and then they look quite broken and don't recover. I haven't verified if this is actually a decoding issue, but I don't think so.
2. Using the default SLICED threading model very often (but not always) blocks indefinitely on Packet.decode() for HEVC. CTRL+C is ignored. I haven't been able to reproduce the freeze-up with FRAME threading.

Neither issue is observed with H264.

Here is how to reproduce it:

import av
import av.datasets
import os

# I need this on windows as by default it goes to a system folder which needs admin permissions
os.environ["PYAV_TESTDATA_DIR"] = os.getcwd()

av.logging.set_level(av.logging.VERBOSE)


def test(input_filename, output_filename, thread_type=None):
    print(f"{os.path.basename(input_filename)} -> {os.path.basename(output_filename)} thread_type={thread_type}")
    with av.open(input_filename) as in_container:
        with av.open(output_filename, "w") as output_container:
            in_stream = in_container.streams.video[0]

            if thread_type:
                in_stream.thread_type = thread_type

            out_stream = output_container.add_stream_from_template(in_stream)

            print(in_stream)
            print(out_stream)

            for packet in in_container.demux(in_stream):
                print(f"Demuxed {packet}")

                if packet.dts is None:
                    continue

                for frame in packet.decode():
                    print(f"Decoded {frame}")

                    frame = out_stream.encode(frame)
                    if (frame):
                        print(f"Encoded {frame}")
                        output_container.mux(frame)

    print("Done")

h264_filename = av.datasets.curated("pexels/time-lapse-video-of-night-sky-857195.mp4")
hevc_filename = av.datasets.cached_download("http://fate-suite.ffmpeg.org/hevc/dv84.mov", "dv84.mov")

# Re-encoding H264 for the curated video works fine both with sliced and frame threading
test(h264_filename, "h264-sliced.mp4")
test(h264_filename, "h264-frame.mp4", "FRAME")

# Re-encoding HEVC/X265 produces shorter video with artefacts. It is OK only for a second.
test(hevc_filename, "hevc-frame.mov", "FRAME")

# Re-encoding HEVC/X265 using SLICE often blocks at packet.decode() and CTRL+C does not work
test(hevc_filename, "hevc-sliced.mov")

See log.txt for the output.

Actually, to reproduce the blocking issue, it is enough to call Packet.decode() enough times:

import av
import av.datasets
import os

# I need this on windows as by default it goes to a system folder which needs admin permissions
os.environ["PYAV_TESTDATA_DIR"] = os.getcwd()

av.logging.set_level(av.logging.VERBOSE)


def test(input_filename, it):
    with av.open(input_filename) as in_container:
        in_stream = in_container.streams.video[0]

        idx = 1
        for packet in in_container.demux(in_stream):
            print(f"#{it} Demuxed #{idx} {packet}")

            if packet.dts is None:
                continue

            print(f"#{it} Decoding...")
            packet.decode()
            idx += 1


hevc_filename = av.datasets.cached_download("http://fate-suite.ffmpeg.org/hevc/dv84.mov", "dv84.mov")

it = 1
while True:
    print(f"Try {it}...")
    test(hevc_filename, it)
    it += 1

[JoeSchiff]

Similar issues on Linux

I did some testing on Ubuntu 24.04. Both problems occur as you described, but with some differences.

Using the first script: the code will hang when using either FRAME or SLICE threading.

Using the second script: calling Packet.decode() is not enough to reproduce the blocking issue. I ran hundreds of iterations without hanging.

Misc notes

When I use the ffmpeg installed through apt (not bundled with pyav), the code does not hang, but the artifacts still occur.

I noticed the video will always hang at the same spot for a given thread type.
For example, SLICE always hangs here:
Demuxed <av.Packet of #0, dts=59, pts=160; 62043 bytes at 0x7f48eda41170>

FRAME hangs here:
Demuxed <av.Packet of #0, dts=1340, pts=1441; 63442 bytes at 0x77e03110d990>

[nagadomi]

On my Ubuntu, I get the same deadlock problem with the second script (no modification).
It does not occur with av==13.1.0.
I suspect this is a ffmpeg side issue( ? ), but I wonder what the developer's opinion is.

[johan-sightic]

I did not have this problem yesterday but today .decode() hangs all the time. I have not updated ffmpeg or PyAV manually but i did install nvidia-cuda-toolkit and the latest Ubuntu updates.
Setting stream.thread_type = "FRAME" makes the hanging go away

[WyattBlue]

hangs all the time
Ubuntu

many such cases

[nagadomi]

On my Ubuntu, I get the same deadlock problem with the second script (no modification).

I have confirmed that this problem has been fixed in 14.1.0.

[ladaapp]

For me the deadlock problem is still present in 14.1.0 as well as in version 14.0.0 and 14.0.1 as described by OP. Building latest PyAV against my locally installed ffmpeg 7.1 has the same problem.

Previous releases which don't ship with ffmpeg 7.1 work just fine.

The latest commit FFmpeg/FFmpeg@8cabfd9 in release/7.1 branch of ffmpeg seems to fix the issue when building PyAV against this build. With it, the second script never blocks for me anymore.

Commit message mentions

Fixes races and deadlocks in hevdec with slice threading, e.g. some of
those mentioned in #11221.

[nagadomi]

Yes, I think I am mistaken about this.
It seems that av==14.1.0 is shipped ffmpeg-7.1 2024-09-30 version.
However, the deadlock no longer occurs in my environment. Perhaps it is timing dependent.

[ladaapp]

Now that the official release for ffmpeg 7.1.1 is out for a couple weeks i tried to reproduce the issue again to see if its fixed in the new release using the 2nd repro script from OP.
Result
I could not reproduce this issue with ffmpeg 7.1.1 an latest pyav. Only on ffmpeg 7.1 and latest pyav or on recent official binary av releases from PyPi I could reproduce this issue consistently.

@WyattBlue Would be great if you could consider publishing a new pyav release build on ffmpeg 7.1.1 to fix this issue

There is also another duplicate report of this problem in #1736

---

here are my testing notes

export PYAV_LIBRARY=ffmpeg-7.1.1 ; then follow build/install instructions from README
run1 / 200 steps -> no issue
run2 / 200 steps -> no issue
run3 / 200 steps -> no issue
run4 / 200 steps -> no issue
run6 / 1000 steps -> no issue
rm -rf $PYAV_LIBRARY_ROOT ; deactivate ; exit

export PYAV_LIBRARY=ffmpeg-7.1 ; then follow build/install instructions from README
run1 / 200 steps -> no issue
run2 / 81 steps -> hang
run3 / 21 steps -> hang
run4 / 9 steps -> hang
run5 / 38 steps -> hang
rm -rf $PYAV_LIBRARY_ROOT ; deactivate ; exit

export PYAV_LIBRARY=ffmpeg-7.0.2 ; then follow build/install instructions from README
run1 / 200 steps -> no issue
run2 / 200 steps -> no issue
run3 / 200 steps -> no issue
run4 / 200 steps -> no issue
run5 / 200 steps -> no issue
rm -rf $PYAV_LIBRARY_ROOT ; deactivate ; exit