Class VirtualBackgroundProcessor

The VirtualBackgroundProcessor, when added to a VideoTrack, replaces the background in each video frame with a given image, and leaves the foreground (person(s)) untouched. Each instance of VirtualBackgroundProcessor should be added to only one VideoTrack at a time to prevent overlapping of image data from multiple VideoTracks.

Example

import { createLocalVideoTrack } from 'twilio-video';
import { VirtualBackgroundProcessor } from '@twilio/video-processors';

let virtualBackground: VirtualBackgroundProcessor;
const img = new Image();

img.onload = async () => {
  virtualBackground = new VirtualBackgroundProcessor({
    assetsPath: 'https://my-server-path/assets',
    backgroundImage: img
  });
  await virtualBackground.loadModel();

  const track = await createLocalVideoTrack({
    // Increasing the capture resolution decreases the output FPS
    // especially on browsers that do not support SIMD
    // such as desktop Safari and iOS browsers, or on Chrome
    // with capture resolutions above 640x480 for webgl2.
    width: 640,
    height: 480,

    // Any frame rate above 24 fps on desktop browsers increase CPU
    // usage without noticeable increase in quality.
    frameRate: 24
  });
  track.addProcessor(virtualBackground, {
    inputFrameBufferType: 'videoframe',
    outputFrameBufferContextType: 'bitmaprenderer'
  });
};

img.src = '/background.jpg';
Copy