XR Blocks

!jsDelivr hits (GitHub)

#### JavaScript library for rapid XR and AI prototyping

Site —
Manual —
Templates —
Demos —
YouTube —
arXiv —
Blog

$3

XR Blocks is a lightweight, cross-platform library for rapidly prototyping
advanced XR and AI experiences. Built upon three.js, it
targets Chrome v136+ with WebXR support on Android XR (e.g.,
Galaxy XR) and also
includes a powerful desktop simulator for development. The framework emphasizes
a user-centric, developer-friendly SDK designed to simplify the creation of
immersive applications with features like:

- Hand Tracking & Gestures: Access advanced hand tracking, custom
gestures with TensorFlow Lite / PyTorch models, and interaction events.
- Gesture Recognition: Opt into pinch, open-palm, fist, thumbs-up, point,
and spread detection with options.enableGestures(), tune providers or
thresholds, and subscribe to gesturestart/gestureupdate/gestureend
events from the shared subsystem.
- World Understanding: Present samples with depth sensing, geometry-aware
physics, and object recognition with Gemini in both XR and desktop simulator.
- AI Integration: Seamlessly connect to Gemini for multimodal
understanding and live conversational experiences.
- Cross-Platform: Write once and deploy to both XR devices and desktop
Chrome browsers.

We welcome all contributors to foster an AI + XR community! Read our
blog post
and white paper for a visionary roadmap.

$3

XR Blocks can be imported directly into a webpage using an importmap. This code
creates a basic XR scene containing a cylinder. When you view the scene, you can
pinch your fingers (in XR) or click (in the desktop simulator) to change the
cylinder's color. Check out
this live demo with simple
code below:

``html





name="viewport"
content="width=device-width, initial-scale=1.0, user-scalable=no"
/>
type="text/css"
rel="stylesheet"
href="https://cdn.jsdelivr.net/gh/google/xrblocks@main/samples/main.css"
/>






`

$3

Follow the steps below to clone the repository and build XR Blocks:

`bash


Clone the repository.

git clone --depth=1 git@github.com:google/xrblocks.git
cd xrblocks

Install dependencies.

npm ci

Build xrblocks.js.

npm run build

After making changes, check ESLint and run Prettier

npm run lint # ESLint check
npm run format # Prettier format
`

XR Blocks uses ESLint for linting and Prettier for formatting.
If coding in VSCode, make sure to install the ESLint extension and the Prettier extension. Then set Prettier as your default formatter.

This is not an officially supported Google product, but will be actively
maintained by the XR Labs team and external collaborators. This project is not
eligible for the
Google Open Source Software Vulnerability Rewards Program.

$3

When using specific features in this SDK (e.g., WebXR, hand tracking, camera),
users will be prompted with permission requests and the application may not
function as expected with denied permissions.

XR Blocks is an open source software development kit that does not handle data
by itself; however, the use of other APIs may collect user data and require user
permissions:

When using WebXR and
LiteRT APIs (e.g., depth sensing, gesture
recognition), all data is stored and processed locally with on-device models.

When using AI features (e.g.,
Gemini Live, Gemini Flash), the
data will be sent to Gemini servers and please follow
Gemini's Privacy & Terms.

$3

This SDK does not require any API keys for non-AI samples. In specific AI use
cases, this SDK provides an interface to use cloud-hosted Gemini services with
XR experiences, requiring an API key from
AI Studio. Please follow
this doc for best
practices to keep your API key secure.

Treat your Gemini API key like a password. If compromised, others can use your
project's quota, incur charges (if billing is enabled), and access your private
data, such as files.

#### Critical Security Rules

Never commit API keys to source control. Do not check your API key into version
control systems like Git.

Never expose API keys on the client-side. Do not use your API key directly in
web or mobile apps in production. Keys in client-side code (including our
JavaScript/TypeScript libraries and REST calls) can be extracted.

$3

To remove XR Blocks from your code, simple remove the lines from your type="importmap"> tag in HTML, or import * from xrblocks in JavaScript, or
use npm uninstall xrblocks from your project directory.

$3

If you find XR Blocks inspiring or useful in your research, please reference it
as:

`bibtex
@misc{Li2025XR,
title={{XR Blocks: Accelerating Human-centered AI + XR Innovation}},
author={Li, David and Numan, Nels and Qian, Xun and Chen, Yanhe and Zhou, Zhongyi and Alekseev, Evgenii and Lee, Geonsun and Cooper, Alex and Xia, Min and Chung, Scott and Nelson, Jeremy and Yuan, Xiuxiu and Dias, Jolica and Bettridge, Tim and Hersh, Benjamin and Huynh, Michelle and Piascik, Konrad and Cabello, Ricardo and Kim, David and Du, Ruofei},
year={2025},
eprint={2509.25504},
archivePrefix={arXiv},
primaryClass={cs.HC},
url={https://arxiv.org/abs/2509.25504},
}
`

#### Key Works Built with XR Blocks

These references are built with XR Blocks:

`bibtex
@inproceedings{Lee2025Sensible,
title = {{Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agent}},
author = {Lee, Geonsun and Xia, Min and Numan, Nels and Qian, Xun and Li, David and Chen, Yanhe and Kulshrestha, Achin and Chatterjee, Ishan and Zhang, Yinda and Manocha, Dinesh and Kim, David and Du, Ruofei},
booktitle = {Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology},
year = {2025},
publisher = {ACM},
numpages = {22},
series = {UIST},
doi = {10.1145/3746059.3747748},
}
`

#### Inspiring Related Works

We call for contributors to integrate our prior art into XR Blocks to enhance
reproducibility and knowledge sharing:

E.g., integrating models from and Transformers.js
to XR Blocks; bringing more
depth-based interaction to
XR Blocks; and add more samples and demos. For large commits, feel free to add
an issue before working on it so that your work won't be duplicated with others.

`bibtex
@inproceedings{Du2023Rapsai,
title = {{Rapsai: Accelerating Machine Learning Prototyping of Multimedia Applications Through Visual Programming}},
author = {Du, Ruofei and Li, Na and Jin, Jing and Carney, Michelle and Miles, Scott and Kleiner, Maria and Yuan, Xiuxiu and Zhang, Yinda and Kulkarni, Anuva and Liu, XingyuBruce and Sabie, Ahmed and Orts-Escolano, Sergio and Kar, Abhishek and Yu, Ping and Iyengar, Ram and Kowdle, Adarsh and Olwal, Alex},
booktitle = {Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems},
year = {2023},
publisher = {ACM},
month = {Apr.},
day = {22-29},
number = {125},
pages = {1--23},
series = {CHI},
doi = {10.1145/3544548.3581338},
}
`

Extending XR Blocks to XR communication:

`bibtex
@inproceedings{Hu2025DialogLab,
title = {{DialogLab: Authoring, Simulating, and Testing Dynamic Group Conversations in Hybrid Human-AI Conversations}},
author = {Hu, Erzhen and Chen, Yanhe and Li, Mingyi and Phadnis, Vrushank and Xu, Pingmei and Qian, Xun and Olwal, Alex and Kim, David and Heo, Seongkook and Du, Ruofei},
booktitle = {Proceedings of the 39th Annual ACM Symposium on User Interface Software and Technology},
year = {2025},
number = {210},
publisher = {ACM},
number = {210},
pages = {1--20},
series = {UIST},
doi = {10.1145/3746059.3747696},
}
``

$3

- Please follow
Google's Privacy & Terms when
using this SDK.

- When using AI features in this SDK, please follow
Gemini's Privacy & Terms.