Can’t see passthrough in WebXR

When I first set up WebXR development to build a mixed reality experiment, I had trouble seeing the passthrough. Everything was fully immersive VR instead of being AR/MR and displaying the surrounding room.

I used BabylonJS, so the code shown is for that, but hopefully it has some universal relevance.

Initialise as Mixed Reality

In BabylonJS, you can initialise the experience as MR with this code. This enables the background to be the devices passthrough (ie. it’s camera feed).

await scene.createDefaultXRExperienceAsync({
    uiOptions: {
        // This tells it to go into augmented reality mode
        sessionMode: "immersive-ar"
    }
});

In order for the passthrough to be visible, however, Babylon needs to not be drawing over the background with it’s own non-transparent values.

Transparent background needed

First, reset the scenes clearColor to a colour that is invisible. The clear colour is the colour that BabylonJS repaints the scene with first for every frame, before it draws everything else on top. If this isn’t transparent, then the passthrough won’t show through.

// A Color4 has an alpha value at the end so it can be transparent.
scene.clearColor = new BABYLON.Color4(0,0,0,0);

createDefaultEnvironment blocks the passthrough

If you were following a tutorial (Or maybe you started off from an AI prompt), you might have added in a function for a default environment. This will mess with your background too —Either by background colour or perhaps by adding a skybox in the background.

// Get rid of the below code if you have it.
// scene.createDefaultEnvironment();

Now you should be able to see the passthrough in your scene.

Thanks…

I also dissect and speculate on design and development.
Digging into subtle details and implications, and exploring broad perspectives and potential paradigm shifts.
Check out my conceptual articles on Substack or find my latest below.

You can also find me on Threads, Bluesky, Mastodon, or xTwitter for more diverse posts about ongoing projects.