Needle Engine FaceFilter

Add face tracking to your Needle Engine projects with ease.

Features

- Blendshape mesh face tracking
- Texture face tracking with google/mediapipe canonical or procreate texture layouts (Use the FaceMeshTexture class)
- Video face tracking: Play a video as a face texture (Use the FaceMeshVideo class)
- Custom shader face meshes: Use custom materials on your face mesh (Use the FaceMeshCustomShader class)
- Tracking for multiple faces at once (with smoothing)
- Can be used with Unity to create filters, animations, materials...

Quickstart

Run npm i @needle-tools/facefilter in your web project

Then see the code or examples below:

$3

See full examples in /examples/html/

#### Supported attributes

| | |
| -- | -- |
| face-filter | URL to either a image or model file. Supported formats: glTF, GLB, FBX, OBJ or 2D jpeg, jpg, png, webp |
| face-filter-mask | (optional, 2D only) URL to image file that will be used to mask out the face filter texture
| face-filter-layout | (optional, 2D only) Either procreate, mediapipe or canonical. Default mediapipe
| face-filter-scale | (optional, 3D only) Apply scale to the 3D face filter model (e.g. face-filter-scale=".5")
| face-filter-offset | (optional, 3D only) Offset the 3D face filter model (e.g. face-filter-offset="0.0, 0.1, 0.1")
| face-filter-max-faces | (optional) How many faces should be tracked automatically. Default: 1
| face-filter-show-video | (optional) Should the camera videofeed be rendered in the background? Default: true. Can be set to 0 to hide the videofeed in the background.
| face-filter-video-selector | (optional) HTML selector for a HTMLVideoElement. Useful if you want to render provide your own video element on the website elsewhere. If none is provided a hidden video element will be created automatically.

#### HTML Example
``html








background-color="#ffffdd"
face-filter="https://cdn.needle.tools/static/facefilter/facemask-template-procreate.webp"
face-filter-mask="https://cdn.needle.tools/static/facefilter/facemask-occlusion-procreate.webp"
face-filter-layout="procreate"
>



`
Open 2D Example –
Open 3D Example

$3


`ts
import { onStart } from '@needle-tools/engine';
import { FaceMeshTexture, NeedleFaceFilterTrackingManager } from '@needle-tools/facefilter';

onStart(context => {
const scene = context.scene;

// Create a face filter tracking manager and add it to the scene
const filtermanager = new NeedleFaceFilterTrackingManager();
filtermanager.createMenuButton = true;
scene.addComponent(filtermanager);

// Creating a filter
const filter = new FaceMeshTexture({
layout: 'procreate', // we support both the google/mediapipe canonical layout and procreate/arkit layouts
texture: {
url: './assets/crocodile.webp', // provide a URL to the texture
// texture: // alternatively you can assign an existing texture directly
},
});
// Activate one of your filters
filtermanager.activateFilter(filter);
});
`
Open Example on Stackblitz

$3


`ts
import { onStart } from '@needle-tools/engine';
import { FaceMeshTexture, NeedleFaceFilterTrackingManager } from '@needle-tools/facefilter';

onStart(context => {
const scene = context.scene;

// Create a face filter tracking manager and add it to the scene
const filtermanager = new NeedleFaceFilterTrackingManager();
filtermanager.createMenuButton = false;
scene.addComponent(filtermanager);

// Creating a filter using a GLB/glTF URL model that has blendshapes
const filter = await FaceFilterRoot.create('https://cloud.needle.tools/-/assets/Z23hmXBZWllze-ZWllze/file', {
scale: 0.5,
offset: { x: 0, y: 0.01, z: 0 },
});
if (filter) filtermanager.activateFilter(filter);
});
``
Open Example on Stackblitz

Examples

- Full project & source on Github
- Example with Blendshapes on Stackblitz
- Example with Sunglasses on Stackblitz
- Example with Texture Mesh on Stackblitz
- Example with ShaderToy on Stackblitz
- Example without a bundler on Stackblitz
- Example with HTML only 2D filter on Stackblitz (or github)
- Example with HTML only 3D filter on Stackblitz (or github)
- Demo Video

Contributing

See Github for more information

Package Dependencies

Source files are 75 kB (gzip).
This package contains files for the Unity integration and are not included in web builds.

Contact

needle •
Twitter •
Forum •
Youtube