VideoFxProcessor does not render properly in some cases

[utdemir]

What happened and what did you expect to happen?

We're trying to use VideoFxProcessor for background replacement in our browser app.

However, for some users, they intermittently get this issue where the background renders fine, but their face doesn't. See the image:

They either describe it as rapid flickering, or:

Mine would go black for a few seconds, then back to not black but flickering, then back to black.. constantly the whole time

Once they end up in this state, it doesn't fix itself unless they reload (we only start one VideoFxProcessor when they log in and it is reused across multiple calls).

Relevant parts of the codebase:

    const isSupported = await VideoFxProcessor.isSupported(
      processorLogger,
      true,
    ).catch((err) => {
      logger.error(`VideoFxProcessorError: ${err}`);
      return false;
    });
    if (isSupported) {
      this.videoFxProcessor = await VideoFxProcessor.create(
        processorLogger,
        {
          backgroundBlur: {
            isEnabled: false,
            strength: 'medium' as const,
          },
          backgroundReplacement: {
            isEnabled: true,
            backgroundImageURL: getBackgroundImageUrl(effect),
            defaultColor: undefined,
          },
        }
      );
    }

   ...


    const transform = new DefaultVideoTransformDevice(processorLogger, device, [
      this.videoFxProcessor,
    ]);

Have you reviewed our existing documentation?

• GitHub issues
• FAQs page
• README page
• Feature-specific guides

Reproduction steps

Unfortunately I do not have a reproducer. I only have reports and screenshots from two other people. Their system is not unusual either - recent Apple Silicon Macbooks with recent versions of Chrome. And that is not consistent for them either - one user reports only seeing it one in three meetings.

Amazon Chime SDK for JavaScript version

3.16.0

What browsers are you seeing the problem on?

Chrome, Electron

Browser version

Chrome 118, Electron 32.2

Meeting and Attendee ID Information.

Browser console logs

[kenpp]

Here we have the same problem with our product.

In our products, this bug may occur if you turn the background image on and off or switch between background images and background blur multiple times.

However, reproducibility has not been achieved.

Amazon Chime SDK for JavaScript version
3.19.0

Thank you.