Skip to content
Cloudflare Docs

Video Effects

Add video background effects and blur to participant video feeds in your RealtimeKit meetings using the Core SDK.

Installation

Terminal window
npm i @cloudflare/realtimekit-virtual-background

Usage

import { useState, useEffect } from "react";
import { useRealtimeKitClient } from "@cloudflare/realtimekit-react";
import RealtimeKitVideoBackgroundTransformer from "@cloudflare/realtimekit-virtual-background";


function App() {
  const [meeting] = useRealtimeKitClient();
  const [videoBackgroundTransformer, setVideoBackgroundTransformer] =
    useState(null);


  useEffect(() => {
    const initializeTransformer = async () => {
      if (!meeting) return;


      // Check browser support
      if (!RealtimeKitVideoBackgroundTransformer.isSupported()) {
        console.warn("Video background not supported in this browser");
        return;
      }


      // Disable default per frame rendering
      await meeting.self.setVideoMiddlewareGlobalConfig({
        disablePerFrameCanvasRendering: true,
      });


      // Initialize transformer
      const transformer = await RealtimeKitVideoBackgroundTransformer.init({
        meeting,
      });


      setVideoBackgroundTransformer(transformer);
    };


    initializeTransformer();
  }, [meeting]);


  const applyStaticBackground = async (imageUrl) => {
    if (!videoBackgroundTransformer) return;


    meeting.self.addVideoMiddleware(
      await videoBackgroundTransformer.createStaticBackgroundVideoMiddleware(
        imageUrl,
      ),
    );
  };


  const applyBlur = async (blurStrength = 50) => {
    if (!videoBackgroundTransformer) return;


    meeting.self.addVideoMiddleware(
      await videoBackgroundTransformer.createBackgroundBlurVideoMiddleware(
        blurStrength,
      ),
    );
  };


  const removeBackground = () => {
    // Remove all video middlewares
    meeting.self.removeVideoMiddleware();
  };


  return (
    <div>
      <button
        onClick={() =>
          applyStaticBackground(
            "https://images.unsplash.com/photo-1487088678257-3a541e6e3922",
          )
        }
      >
        Apply Background
      </button>
      <button onClick={() => applyBlur(50)}>Apply Blur</button>
      <button onClick={removeBackground}>Remove Background</button>
    </div>
  );
}

Advanced configuration

For better, sharper results, pass a custom segmentation configuration:

const transformer = await RealtimeKitVideoBackgroundTransformer.init({
  meeting,
  segmentationConfig: {
    model: "mlkit", // 'meet' | 'mlkit'
    backend: "wasmSimd",
    inputResolution: "256x256", // '256x144' for meet
    pipeline: "webgl2", // 'webgl2' | 'canvas2dCpu'
    // canvas2dCpu gives sharper blur, webgl2 is faster
    targetFps: 35,
  },
});