For more custom implementations, consider using browserIncompatible function.

`tsx
const incompatible = browserIncompatible(); // true / false
`

Publishing and Hosting Content

Once you've built your site, you have a number of options for hosting it. These include, hosting with ZapWorks and self-hosting. Head over to the ZapWorks Publishing and Hosting article to learn more about these options.

$3

This wrapper library is MIT licensed, but relies on our proprietary computer vision library, @zappar/zappar-cv, for which you must maintain an activate subscription at ZapWorks. To learn more about licensing, click here.

The source code for this wrapper library is available freely for your viewing pleasure over at GitHub:


Setting up the Canvas

The first step when developing a React Three Fibre UAR project is replace any existing Canvas you have in your scene with the ZapparCanvas.

`tsx
import { ZapparCanvas / ... / } from '@zappar/zappar-react-three-fiber';
`

`tsx
{/ YOUR CONTENT HERE /}
`

You may alternatively use the default react-three-fiber Canvas component, with colorManagement toggled off.

`tsx
{/ YOUR CONTENT HERE /}
`

When using @zappar/webgl-snapshot to take screenshots, construct the canvas with drawing buffer preservation enabled:

`tsx

`

or

`tsx
colorManagement={false}
gl={{ preserveDrawingBuffer: true }
>
`

Setting up the Camera

Add or replace any existing camera you have in your scene with the ZapparCamera component:

`tsx
import { ZapparCamera / ... / } from '@zappar/zappar-react-three-fiber';
`

`tsx
// ...
return (



);
`

You don't need to change the position or rotation of the camera yourself - the Zappar library will do this for you, automatically.

Advanced Usage

$3

Custom video device IDs can be provided as options passed into ZapparCamera component:

`ts
sources={{
userCamera:
"csO9c0YpAf274OuCPUA53CNE0YHlIr2yXCi+SqfBZZ8=",
rearCamera:
"RKxXByjnabbADGQNNZqLVLdmXlS0YkETYCIbg+XxnvM=",
}}
/>
`

$3

Use onFirstFrame callback prop to detect when the first frame has been processed:

`tsx
onFirstFrame={() => {
console.log('first frame');
}}
/>
`

$3

When the camera component is mounted, it sets itself as the scene's main camera with render priority of 1. You may change this behavior with the following props:

`tsx
makeDefault={false} // default: true
renderPriority={0} // default: 1
/>
`

To set the camera as your main scene camera yourself, use useThree:

`tsx
const set = useThree(state => state.set)
const cameraRef = useRef()

useLayoutEffect(() => {
set(() => ({ camera: cameraRef.current }))
}, [])


// ...

// ...
`

$3

Some experiences, e.g. face tracked experiences, require the use of the user-facing camera on the device. To activate the user-facing camera, provide the userFacing prop to the ZapparCamera component:

`tsx

`

$3

Users expect user-facing cameras to be shown mirrored, so by default the ZapparCamera will mirror the camera view for the user-facing camera.

Configure this behavior with the following option:

`tsx

`

The values you can pass to userCameraMirrorMode are:

- poses: this option mirrors the camera view and makes sure your content aligns correctly with what you're tracking on screen. Your content itself is not mirrored - so text, for example, is readable. This option is the default.
- css: this option mirrors the entire canvas. With this mode selected, both the camera and your content appear mirrored.
- none: no mirroring of content or camera view is performed

There's also a rearCameraMirrorMode prop that takes the same values should you want to mirror the rear-facing camera. The default rearCameraMirrorMode is none.

$3

The SDK provides an automatically generated environment map that's useful if you're using materials that support reflections (e.g. MeshStandardMaterial, MeshPhysicalMaterial). The map uses the camera feed to create an approximate environment that can add some realism to your scene.

To apply the map to your scene, simply pass environmentMap prop to the ZapparCamera component:

`tsx

`

$3

The Zappar library provides multiple modes for the camera to move around in the scene. You can set this mode with the poseMode prop of the ZapparCamera component. There are the following options:

- default: in this mode the camera stays at the origin of the scene, pointing down the negative Z axis. Any tracked groups will move around in your scene as the user moves the physical camera and real-world tracked objects.
- attitude: the camera stays at the origin of the scene, but rotates as the user rotates the physical device. When the Zappar library initializes, the negative Z axis of world space points forward in front of the user.
- anchor-origin: the origin of the scene is the center of the group specified by the camera's poseAnchorOrigin prop. In this case the camera moves and rotates in world space around the group at the origin.

The correct choice of camera pose will depend on your given use case and content. Here are some examples you might like to consider when choosing which is best for you:

- To have a light that always shines down from above the user, regardless of the angle of the device or anchors, use attitude and place a light shining down the negative Y axis in world space.
- In an application with a physics simulation of stacked blocks, and with gravity pointing down the negative Y axis of world space, using anchor-origin would allow the blocks to rest on a tracked image regardless of how the image is held by the user, while using attitude would allow the user to tip the blocks off the image by tilting it.

Tracking

The Zappar library offers three types of tracking for you to use to build augmented reality experiences:

- _Image Tracking_ can detect and track a flat image in 3D space. This is great for building content that's augmented onto business cards, posters, magazine pages, etc.
- _Face Tracking_ detects and tracks the user's face. You can attach 3D objects to the face itself, or render a 3D mesh that's fit to (and deforms with) the face as the user moves and changes their expression. You could build face-filter experiences to allow users to try on different virtual sunglasses, for example, or to simulate face paint.
- _Instant World Tracking_ lets you tracking 3D content to a point chosen by the user in the room or immediate environment around them. With this tracking type you could build a 3D model viewer that lets users walk around to view the model from different angles, or an experience that places an animated character in their room.

Importing trackers from the package:

`tsx
import { InstantTracker, ImageTracker, FaceTracker } from '@zappar/zappar-react-three-fiber';
`

$3

To track content from a flat image in the camera view, use the ImageTracker component:

`tsx
{/PLACE CONTENT TO APPEAR ON THE IMAGE HERE/}
`

The group provides a coordinate system that has its origin at the center of the image, with positive X axis to the right, the positive Y axis towards the top and the positive Z axis coming up out of the plane of the image. The scale of the coordinate system is such that a Y value of +1 corresponds to the top of the image, and a Y value of -1 corresponds to the bottom of the image. The X axis positions of the left and right edges of the target image therefore depend on the aspect ratio of the image.

#### Target File

ImageTrackers use a special 'target file' that's been generated from the source image you'd like to track. You can generate them using the ZapWorks command-line utility like this:

`bash
zapworks train myImage.png
`

The resulting file can then be passed as a targetFile prop to be loaded:

`tsx
const App = () => {
const targetFile = 'target.zpt';
return (


{/PLACE CONTENT TO APPEAR ON THE IMAGE HERE/}

);
};
`

#### Events

The ImageTracker component will emit the following events on the element it's attached to:

- onVisible - emitted when the image appears in the camera view
- onNotVisible - emitted when the image is no longer visible in the camera view
- onNewAnchor - emitted when a a non-previously seen before anchor appears in the camera view.

Here's an example of using these events:

`tsx


onVisible={anchor => console.log(Visible ${anchor.id})}
onNotVisible={anchor => console.log(Not visible ${anchor.id})}
onNewAnchor={anchor => console.log(New anchor ${anchor.id})}
targetImage={targetFile}
>
{/PLACE CONTENT TO APPEAR ON THE IMAGE HERE/}


`

$3

To place content on or around a user's face, create a new FaceTracker component:

`tsx


{/PLACE CONTENT TO APPEAR ON THE FACE HERE/}

`

The group provides a coordinate system that has its origin at the center of the head, with positive X axis to the right, the positive Y axis towards the top and the positive Z axis coming forward out of the user's head.

Note that users typically expect to see a mirrored view of any user-facing camera feed. Please see the section on mirroring the camera view earlier in this document.

#### Events

The FaceTracker component will emit the following events on the element it's attached to:

- onVisible - emitted when the face appears in the camera view
- onNotVisible - emitted when the face is no longer visible in the camera view

Here's an example of using these events:

`tsx

onNotVisible={(anchor) => console.log(Not visible ${anchor.id})}
onVisible={(anchor) => console.log(Visible ${anchor.id})}
>
{/PLACE CONTENT TO APPEAR ON THE FACE HERE/}

`

$3

In addition to tracking the center of the head, you can use Landmark to track content from various points on the user's face. These landmarks will remain accurate, even as the user's expression changes.

To track a landmark, construct a new Landmark component, passing your camera, face tracker group, and the name of the landmark you'd like to track as props:

`tsx
const App = () => {
return (



{/PLACE CONTENT TO APPEAR ON THE NOSE BRIDGE HERE/}


);
};
`

The following landmarks are available: eye-left, eye-right, ear-left, ear-right, nose-bridge, nose-tip, nose-base, lip-top, lip-bottom, mouth-center, chin, eyebrow-left, and eyebrow-right. Note that 'left' and 'right' here are from the user's perspective.

$3

In addition to tracking the center of the face using FaceTracker, the Zappar library provides a number of meshes that will fit to the face/head and deform as the user's expression changes. These can be used to apply a texture to the user's skin, much like face paint, or to mask out the back of 3D models so the user's head is not occluded where it shouldn't be.

To use a face mesh, create a child mesh component, attaching FaceBufferGeometry within your FaceTracker component, like this:

`tsx
import { FaceBufferGeometry / ... / } from '@zappar/zappar-react-three-fiber';
`

`tsx
// ...
const App = () => {
const faceTrackerGroup = useRef();
return (









);
};
`

At this time there are two meshes included with the library. The default mesh covers the user's face, from the chin at the bottom to the forehead, and from the sideburns on each side. There are optional parameters that determine if the mouth and eyes are filled or not:

`tsx

`

The full head simplified mesh covers the whole of the user's head, including some neck. It's ideal for drawing into the depth buffer in order to mask out the back of 3D models placed on the user's head (see Head Masking below). There are optional parameters that determine if the mouth, eyes and neck are filled or not:

`tsx

`

$3

If you're placing a 3D model around the user's head, such as a helmet, it's important to make sure the camera view of the user's real face is not hidden by the back of the model. To achieve this, the library provides HeadMaskMesh. It's an entity that fits the user's head and fills the depth buffer, ensuring that the camera image shows instead of any 3D elements behind it in the scene.

To use it, add the entity into your HeadMaskMesh entity, before any other 3D content:

`tsx
import { HeadMaskMesh / ... / } from '@zappar/zappar-react-three-fiber';
`

`tsx
// ...



{/OTHER 3D CONTENT GOES HERE/}

`

$3

To track content from a point on a surface in front of the user, use the InstantTracker component:

`tsx
{/PLACE CONTENT TO APPEAR IN THE WORLD HERE/}
`

The group provides a coordinate system that has its origin at the point that's been set, with the positive Y coordinate pointing up out of the surface, and the X and Z coordinates in the plane of the surface.

You can use the placementMode prop to let the user choose a location for their content by moving their camera around the room. While the prop is set, the InstantTracker's position will be updated every frame to be in front of the user at [0, 0, -5] relative to the camera. Once the prop is removed the tracker will keep the content anchored at that position in the user's environment, rather than directly in front of the camera. You can customize the camera-relative position that's set during placement mode using the placementCameraOffset prop.

In typical usage, the tracker starts with placementMode set, and an on-screen UI element enables to the user to 'place' and 'pick-up' the content by toggling that prop, like this:

`tsx
const [placementMode, setPlacementMode] = useState(true);
return (
<>










id="zappar-placement-ui"
onClick={() => {
setPlacementMode(currentPlacementMode => !currentPlacementMode);
}}
>
Tap here to
{placementMode ? ' place ' : ' pick up '}
the object