[HELP] RobustVideoMatting

[aklacar1]

I created Hardware Accelerated Robust Video Matting from https://github.com/PeterL1n/RobustVideoMatting in order to make it faster. And while I succeeded in making it much faster compared to CPU and even non Hardware Accelerated GPU. On each movement of people I am getting some strange rainbow glow glitch (mostly red and green, creating double image, one being normal other one being partial with red and green glow/blur effect). In order to create Tensor that is needed for calculations I used this snipped that I got from the samples:

   yuv420 = to_yuv.Execute(nv12_surface, self.cc_ctx)
    if yuv420.Empty():
        print("Can not convert nv12 -> yuv420")
        raise

    yuv_small = to_dim.Execute(yuv420) if is_small else yuv420
    if yuv_small.Empty():
        print("Can not downscale yuv420 surface")
        raise

    rgb24 = to_rgb.Execute(yuv_small, self.cc_ctx)
    if rgb24.Empty():
        print("Can not convert yuv420 -> rgb")
        raise

    rgb24_planar = to_pln.Execute(rgb24, self.cc_ctx)
    if rgb24_planar.Empty():
        print("Can not convert rgb -> rgb planar")
        raise

    surf_plane = rgb24_planar.PlanePtr()
    img_tensor = pnvc.makefromDevicePtrUint8(
        surf_plane.GpuMem(),
        surf_plane.Width(),
        surf_plane.Height(),
        surf_plane.Pitch(),
        surf_plane.ElemSize(),
    )

    img_tensor.resize_(3, height, width)
    img_tensor = img_tensor.type(dtype=torch.cuda.FloatTensor)
    img_tensor = torch.divide(img_tensor, 255.0)

    return img_tensor.unsqueeze(0).to('cuda')

[RomanArzumanyan]

Hi @aklacar1

I need more info to understand what's happening exactly but from the first glance the issue you observe is quite similar to #506
If that doesn't help - please provide more code and input sample required for the bug to reproduce.

[aklacar1]

It looks to be same issue, however it also might be that I managed to fix issue in meantime. Thank you a lot, if I do not see that the issue is fixed I will try everything from related post. But for now I fixed it by changing to pnvc.DptrToTensor instead of pnvc.makefromDevicePtrUint8

[aklacar1]

Your fix was better. Thank you

[aklacar1]

Since the answer seems to be related to another issue, and this looks like I solved it with a fix I posted. I am closing this