Reality Accelerator Toolkit


The Reality Accelerator Toolkit is a comprehensive WebXR utilities library that simplifies the integration of mixed reality features in WebXR applications. Compatible with the three.js 3D library, it bridges the gap between the low-level WebXR APIs and the higher-level APIs offered by three.js. Designed for extensibility, it enables the management of detected planes, meshes, anchors, and real-world hit test targets, translating them seamlessly to Object3D instances in the three.js scene.

Features

- 🚀 Seamless integration with three.js for enhanced 3D application development.
- 🌐 Automatic detection and update of planes, meshes and anchors in the scene.
- 🎯 Translation of real-world hit test targets to Object3D instances in three.js.

Getting Started

$3

- Peer dependency: three.js (latest version recommended): three.js Documentation

$3

To install the package via NPM, run:

``bash
$ npm install ratk
`

Usage

Each section below details how to use different features of the toolkit.

$3

1. Setting Up AR Button: Request a WebXR session with specified features.

`js
import { ARButton } from 'ratk';
const renderer = / Three.js WebGLRenderer /;
const arButton = document.getElementById('ar-button');

ARButton.convertToARButton(arButton, renderer, {
sessionInit: {
requiredFeatures: ['hit-test', 'plane-detection', 'mesh-detection', 'anchors'],
optionalFeatures: [],
},
});
`

2. Initializing RATK: Integrate RATK's root object with your scene. Update it in the render loop.

`js
// Import the library
import { RealityAccelerator } from 'ratk';
const ratk = new RealityAccelerator(renderer.xr);
const scene = / Three.js Scene object /;
scene.add(ratk.root);

function render() {
ratk.update();
}
`

$3

- Creating Anchors: Define position and orientation, then attach objects.

`js
const ratk = / RealityAccelerator instance /;
const anchorPosition = new Vector3(1, 2, 3);
const anchorQuaternion = new Quaternion(0, 0, 0, 1);

ratk.createAnchor(anchorPosition, anchorQuaternion, isPersistent)
.then((anchor / RATK Anchor object extends Object3D /) => {
// Attach a new THREE.Mesh to the anchor
anchor.add(new THREE.Mesh(geometry, material));
});
`

- Recovering Anchors: Access persistent anchors from previous sessions.

`js
ratk.restorePersistentAnchors().then(() => {
ratk.anchors.forEach((anchor) => {
// Process each recovered anchor
anchor.add(new THREE.Mesh(geometry, material));
});
});
`

$3

- Processing Planes: Define a hook to handle new planes.

`js
ratk.onPlaneAdded = (plane / extends Object3D /) => {
if (plane.semanticLabel === 'wall art') {
const mesh = plane.planeMesh; // Three.js Plane Mesh
mesh.material = new MeshBasicMaterial(/_ parameters _/);
// Additional plane processing
}
};
`

- Alternatively, access planes directly from the RATK instance:

`js
ratk.planes.forEach((plane) => {
// Process each plane
});
`

$3

- Processing Meshes: Define a hook to handle new meshes.

`js
ratk.onMeshAdded = (rmesh / extends Object3D /) => {
const meshMesh = rmesh.meshMesh; / reconstructed Three.js Mesh /
meshMesh.material = new MeshBasicMaterial(...);
meshMesh.geometry.computeBoundingBox();

// put a text label on top of the mesh
const semanticLabel = new Text();
meshMesh.add(semanticLabel);
semanticLabel.text = mesh.semanticLabel;
semanticLabel.sync();
semanticLabel.position.y = meshMesh.geometry.boundingBox.max.y;
};
`

- Alternatively, access meshes directly from the RATK instance:

`js
ratk.meshes.forEach((rmesh) => {
// Process each mesh
});
`

$3

- Controller Hit-Testing: Create HitTestTarget from controller with offset transform

`js
const ratk = / RealityAccelerator instance /;
const offsetPosition = new Vector3(0, 0, 0);
const offsetQuaternion = new Quaternion(0, 0, 0, 1);

ratk
.createHitTestTargetFromControllerSpace(
handedness,
offsetPosition,
offsetQuaternion,
)
.then((hitTestTarget / extends Object3D /) => {
hitTestTarget.add(
new Mesh(...),
);
});
`

- Alternatively, conduct hit-testing from viewer space:

`js
ratk.createHitTestTargetFromViewerSpace(
offsetPosition,
offsetQuaternion,
).then(...);
``

Documentation

- API Reference: RATK API Documentation.
- Example Application: Example WebXR Application (Source Code).

Contributing

Please read CONTRIBUTING.md for details on how to contribute to the project.

License

This project is licensed under the MIT License - see the LICENSE.md file for details.