MultiSet VPS WebXR

MultiSet VPS WebXR is a TypeScript SDK that enables developers to integrate MultiSet's Visual Positioning System (VPS) capabilities into WebXR applications. It provides precise 6-DOF (6 degrees of freedom) localization by matching camera frames against cloud-hosted maps, allowing AR applications to understand their position and orientation in physical space.

Features

- Core Client (@multisetai/vps/core) - Authentication and API client for MultiSet VPS services
- WebXR Controller (@multisetai/vps/webxr) - Three.js WebXR session management and frame capture
- Framework-agnostic - Works with React, Vue, Angular, or vanilla JavaScript
- TypeScript support - Full type definitions included
- Event-driven architecture - Comprehensive callbacks for all operations
- Precise localization - 6-DOF pose estimation with position and rotation
- Cloud-based mapping - Leverages MultiSet's cloud infrastructure for map storage and matching

Installation

``bash
npm install @multisetai/vps three
`

> Note: three is a peer dependency and must be installed separately.

Requirements

$3

- HTTPS: WebXR requires a secure context. Use HTTPS in production or https://localhost for local development.
- WebXR-capable device: Android device with ARCore or iOS device with ARKit (via WebXR Viewer or Safari)
- Modern browser: Chrome/Edge (Android) or Safari (iOS 15+)
- Three.js: Version 0.176.0 or higher (peer dependency)

$3

- Node.js 16+ and npm
- TypeScript 5.8+ (for TypeScript projects)

Configuration

$3

Since this SDK makes direct API calls to MultiSet servers from browser-based applications, you must configure Cross-Origin Resource Sharing (CORS) by whitelisting your application's domain in the MultiSet dashboard.

Why is this required? Browsers restrict cross-origin HTTP requests for security. By adding your domain to the MultiSet whitelist, you allow your web application to make API requests to MultiSet services.

#### How to Whitelist Your Domain

1. Navigate to the MultiSet Dashboard
2. Go to Credentials → Settings tab
3. Locate the Domains section
4. Click the purple Add + button
5. Enter your application's full origin (protocol + domain + port if applicable)

#### Recommended Setup

Add entries for both your local development and production environments:

Local Development:
- Format: http://localhost:PORT (e.g., http://localhost:3000, http://localhost:5173)
- Note: Use the exact port your development server runs on

Production:
- Format: https://your-domain.com (e.g., https://www.myapp.com)
- Note: Do not include trailing slashes

> Important: Without whitelisting your domain, API requests will be blocked by the browser's CORS policy, and you'll see errors in the browser console.

For more details, see the MultiSet CORS Configuration Documentation.

Quick Start

$3

`typescript
import { MultisetClient, DEFAULT_ENDPOINTS } from '@multisetai/vps/core';
import { WebxrController } from '@multisetai/vps/webxr';
`

$3

`typescript
const client = new MultisetClient({
clientId: 'CLIENT_ID',
clientSecret: 'CLIENT_SECRET',
code: 'MAP_OUR_MAPSET_CODE',
mapType: 'map', // or 'map-set'
endpoints: DEFAULT_ENDPOINTS,
onAuthorize: (token) => console.log('Authorized:', token),
onError: (error) => console.error('Error:', error),
});

await client.authorize();
`

$3

`typescript
const controller = new WebxrController({
client,
canvas: document.querySelector('canvas'),
overlayRoot: document.body,
onSessionStart: () => console.log('AR session started'),
onSessionEnd: () => console.log('AR session ended'),
});

await controller.initialize();
`

$3

`typescript
const result = await controller.captureFrame();
if (result?.localizeData?.poseFound) {
console.log('Position:', result.localizeData.position);
console.log('Rotation:', result.localizeData.rotation);
}
`

Core Client API

$3

The core client handles authentication and API interactions with MultiSet services.

#### Constructor

`typescript
new MultisetClient(config: IMultisetSdkConfig)
`

Configuration Options:

`typescript
interface IMultisetSdkConfig {
clientId: string; // Your MultiSet client ID
clientSecret: string; // Your MultiSet client secret
code: string; // Map code (e.g., 'MAP_XXXXX')
mapType: 'map' | 'map-set'; // Type of map to use
endpoints?: Partial; // Optional custom endpoints
onAuthorize?: (token: string) => void;
onFrameCaptured?: (payload: IFrameCaptureEvent) => void;
onCameraIntrinsics?: (intrinsics: ICameraIntrinsicsEvent) => void;
onPoseResult?: (payload: IPoseResultEvent) => void;
onError?: (error: unknown) => void;
}
`

#### Methods

##### authorize(): Promise

Authenticates with MultiSet services and obtains an access token. Must be called before making any API requests.

`typescript
const token = await client.authorize();
`

Returns: The access token as a string.

#### Events

The client emits events through callback functions:

- onAuthorize: Called when authorization succeeds with the access token
- onFrameCaptured: Called when a camera frame is captured for localization
- onCameraIntrinsics: Called with camera intrinsic parameters
- onPoseResult: Called with localization results (pose found/not found)
- onError: Called when any error occurs

WebXR Controller API

$3

Manages WebXR sessions, Three.js scene, camera, and renderer. Handles AR button creation and frame capture for localization.

#### Constructor

`typescript
new WebxrController(options: IWebxrControllerOptions)
`

Configuration Options:

`typescript
interface IWebxrControllerOptions {
client: MultisetClient; // Required: MultisetClient instance
canvas?: HTMLCanvasElement; // Optional: Canvas element (created if not provided)
overlayRoot?: HTMLElement; // Optional: Root for DOM overlay (default: document.body)
buttonContainer?: HTMLElement; // Optional: Container for AR button
onARButtonCreated?: (button: HTMLButtonElement) => void;
onSessionStart?: () => void;
onSessionEnd?: () => void;
}
`

#### Methods

##### initialize(buttonContainer?: HTMLElement): Promise

Initializes the WebXR controller, sets up Three.js scene/renderer/camera, and creates the AR button.

`typescript
const arButton = await controller.initialize(buttonContainer);
`

Returns: The created AR button element.

##### captureFrame(): Promise

Captures the current camera frame and performs localization.

`typescript
const result = await controller.captureFrame();
if (result?.localizeData?.poseFound) {
const { position, rotation, confidence } = result.localizeData;
console.log('Localized at:', position);
}
`

Returns: Object with localizeData and optional mapDetails, or null if capture fails.

##### getScene(): THREE.Scene

Gets the Three.js scene object for adding 3D models and objects.

`typescript
const scene = controller.getScene();
const cube = new THREE.Mesh(geometry, material);
scene.add(cube);
`

##### getCamera(): THREE.PerspectiveCamera

Gets the Three.js camera object for custom camera configuration.

`typescript
const camera = controller.getCamera();
camera.near = 0.1;
camera.far = 1000;
`

##### getRenderer(): THREE.WebGLRenderer

Gets the Three.js WebGL renderer for advanced rendering configuration.

`typescript
const renderer = controller.getRenderer();
renderer.shadowMap.enabled = true;
`

##### hasActiveSession(): boolean

Checks if an active WebXR session is currently running.

`typescript
if (controller.hasActiveSession()) {
// AR session is active
}
`

##### dispose(): void

Cleans up resources and removes event listeners. Call this when destroying the controller.

`typescript
controller.dispose();
`

Type Definitions

All TypeScript types are exported from the main entry points:

`typescript
// Core types
import type {
IMultisetSdkConfig,
IMultisetSdkEndpoints,
IFrameCaptureEvent,
ICameraIntrinsicsEvent,
IPoseResultEvent,
ILocalizeAndMapDetails,
MapType,
} from '@multisetai/vps/core';

// WebXR types
import type {
IWebxrControllerOptions,
} from '@multisetai/vps/webxr';
`

Examples

> Full working examples: This repository includes complete, runnable example applications in the examples/ directory:
> - React Example - Full React implementation with TypeScript
> - Vanilla JavaScript Example - Vanilla JavaScript implementation
>
> Each example includes setup instructions, configuration, and demonstrates the complete SDK integration workflow.

$3

`javascript
import * as THREE from 'three';
import { MultisetClient, DEFAULT_ENDPOINTS } from '@multisetai/vps/core';
import { WebxrController } from '@multisetai/vps/webxr';

const client = new MultisetClient({
clientId: 'CLIENT_ID',
clientSecret: 'CLIENT_SECRET',
code: 'MAP_OR_MAPSET_CODE',
mapType: 'map',
endpoints: DEFAULT_ENDPOINTS,
onAuthorize: (token) => console.log('Authorized:', token),
onError: (error) => console.error('Error:', error),
});

const controller = new WebxrController({
client,
canvas: document.querySelector('canvas'),
overlayRoot: document.body,
onSessionStart: () => console.log('AR session started'),
onSessionEnd: () => console.log('AR session ended'),
});

// Authorize and initialize
await client.authorize();
await controller.initialize();

// Add 3D objects to scene
const scene = controller.getScene();
const cube = new THREE.Mesh(
new THREE.BoxGeometry(0.1, 0.1, 0.1),
new THREE.MeshBasicMaterial({ color: 0xff0077 })
);
cube.position.set(0, 0, -0.4);
scene.add(cube);

// Capture and localize
const result = await controller.captureFrame();
if (result?.localizeData?.poseFound) {
console.log('Position:', result.localizeData.position);
}
`

$3

`tsx
import { useEffect, useRef, useState } from 'react';
import * as THREE from 'three';
import { MultisetClient, DEFAULT_ENDPOINTS } from '@multisetai/vps/core';
import { WebxrController } from '@multisetai/vps/webxr';

export default function App() {
const [authorized, setAuthorized] = useState(false);
const canvasRef = useRef(null);
const clientRef = useRef(null);
const controllerRef = useRef(null);

useEffect(() => {
clientRef.current = new MultisetClient({
clientId: 'CLIENT_ID',
clientSecret: 'CLIENT_SECRET',
code: 'MAP_OR_MAPSET_CODE',
mapType: 'map',
endpoints: DEFAULT_ENDPOINTS,
});

controllerRef.current = new WebxrController({
client: clientRef.current,
canvas: canvasRef.current!,
});

return () => {
controllerRef.current?.dispose();
};
}, []);

const handleAuthorize = async () => {
await clientRef.current!.authorize();
await controllerRef.current!.initialize();

// Add 3D objects
const scene = controllerRef.current!.getScene();
const cube = new THREE.Mesh(
new THREE.BoxGeometry(0.1, 0.1, 0.1),
new THREE.MeshBasicMaterial({ color: 0xff0077 })
);
scene.add(cube);

setAuthorized(true);
};

const handleCapture = async () => {
const result = await controllerRef.current!.captureFrame();
if (result?.localizeData?.poseFound) {
console.log('Localized!', result.localizeData.position);
}
};

return (



);
}
``

License