# Introduction
```bash
npm install three @react-three/fiber @react-three/viverse
```
### What does it look like?
> A prototype map with the `` component and its default model
Dependencies:
```js
{
'three': 'latest',
'@react-three/fiber': '<9',
'@react-three/viverse': 'latest',
'@react-three/drei': '<10'
}
```
Files:
File: /App.tsx
```tsx
import { Sky } from '@react-three/drei'
import { Canvas } from '@react-three/fiber'
import { Viverse, SimpleCharacter, BvhPhysicsBody, PrototypeBox } from '@react-three/viverse'
export default function App() {
return (
)
}
```
## How to get started
> Some familiarity with
> react, threejs, and @react-three/fiber, is recommended.
Get started with **[building a simple game](#doc-tutorials-simple-game)**, take a look at our **[examples](#doc-getting-started-examples)**, or follow one of our **tutorials**:
- [First person controls](#doc-tutorials-first-person)
- [Augmented and virtual reality](#doc-tutorials-augmented-and-virtual-reality)
- [Accessing avatar and profile](#doc-tutorials-access-avatar-and-profile)
- [Equipping the character with items](#doc-tutorials-equipping-items)
- [Using custom animations and models](#doc-tutorials-custom-models-and-animations)
- [Actions](#doc-tutorials-actions)
- [Custom Character Controller](#doc-tutorials-custom-character-controller)
- [How to remove the viverse integrations](#doc-tutorials-remove-viverse-integrations)
- [Publish to VIVERSE](#doc-tutorials-publish-to-viverse)
- [Vibe coding with @react-three/viverse (using AI)](../tutorials/vibe-coding-with-ai.mdx)
## Not into react?
> No Problem
Check out how to build games using @pmndrs/viverse and only [vanilla three.js](#doc-without-react-introduction).
## Acknowledgments
This project would not be possible without the default model and default animations made by [Quaternius](https://quaternius.com/), the prototype texture from [kenney.nl](https://www.kenney.nl/), and the [three-vrm project](https://github.com/pixiv/three-vrm) from the [pixiv team](https://github.com/pixiv)!
# All Components and Hooks
## Components
### ``
The main provider component that sets up VIVERSE authentication and physics context. Must wrap your entire application or the parts that use VIVERSE features.
**Props:**
- `children?: ReactNode` - Child components
- `loginRequired?: boolean` - Forces user to login before playing (default: `false`)
- `clientId?: string` - VIVERSE app client ID. Typically you pass this from your app’s environment (e.g. a `VITE_VIVERSE_APP_ID` env var you manage) into this prop.
- `domain?: string` - Authentication domain (default: `'account.htcvive.com'`)
- `authorizationParams?: object` - Additional authorization parameters
- `cookieDomain?: string` - Cookie domain for authentication
- `httpTimeoutInMS?: number` - HTTP request timeout in milliseconds
> [!WARNING]
> Don't set the `clientId` during local development!
**Example:**
```tsx
```
### ``
Creates a simple character controller with physics based on three-mesh-bvh, walking, running, jumping animations, and camera controls. Automatically uses the active VIVERSE avatar if authenticated.
**Props:** See [SimpleCharacter Options](#simplecharacter-options) section below for complete details.
**Example:**
```tsx
{/* Optional child components */}
```
### ``
Provides physics context for collision detection. Usually wrapped automatically by ``, but can be used standalone.
**Props:**
- `children?: ReactNode` - Child components
### ``
Adds visible children as static (non-moving) or kinematic (moving) objects as obstacles to the physics world.
> [!WARNING]
> Content inside the object can not structurally change.
**Props:**
- `children?: ReactNode` - Static mesh objects for collision
- `kinematic?: boolean` - whether the objects world transformation can change - default: false
**Example:**
```tsx
```
### ``
Adds visible children as sensors that detect player intersection and trigger callbacks (does not add obstacles).
> [!WARNING]
> Content inside the object can not structurally change; Hiding the sensors content requires to wrap it in `...`.
**Props:**
- `children?: ReactNode` - Static mesh objects for collision
- `isStatic?: boolean` - whether the objects world transformation is static - default: true
- `onIntersectedChanged?: (intersected: boolean) => void` - callback that get's called when the player starts or stops intersecting with the sensor
**Example:**
```tsx
console.log('currently intersected', intersected)}>
```
### ``
A quick prototyping component that renders a textured box with the prototype material.
**Props:**
- `color?: ColorRepresentation` - Box color tint
- All standard Three.js Group props (position, rotation, scale, etc.)
**Example:**
```tsx
```
### ``
Provides the active character model context so that animation and bone utilities can target the same model instance. Wrap any content that uses ``, ``, ``, or ``.
**Props:**
- `model: CharacterModel` - Model returned by `useCharacterModelLoader`
- `children?: ReactNode` - Child components
**Example:**
```tsx
const model = useCharacterModelLoader({ url: 'avatar.vrm', castShadow: true })
return (
)
```
### ``
Defines a logical animation layer (e.g., "lower-body", "upper-body"). All nested animation actions inherit this layer unless they provide their own `layer` prop. Layers allow to manage animations when managing e.g. additive animations or animations with masks.
**Props:**
- `name: string` - Layer name
- `children?: ReactNode` - Nested timeline/animation content
**Example:**
```tsx
{/* ...lights, environment... */}
{/* masks can be created with @pmndrs/viverse animation masks */}
```
### ``
Loads and plays a clip on the active character model, integrating with `@react-three/timeline` for lifecycle and transitions. Supports masking, cross-fading, syncing, and layering. The `ref` exposes the underlying Three.js `AnimationAction`.
**Props:**
- Clip options (from `@pmndrs/viverse`):
- `url: string | DefaultUrl` - Source of the animation
- `type?: 'mixamo' | 'gltf' | 'vrma' | 'fbx' | 'bvh'`
- `removeXZMovement?: boolean`
- `trimTime?: { start?: number; end?: number }`
- `boneMap?: Record`
- `scaleTime?: number`
- `mask?: CharacterAnimationMask` - Limit animation to specific bones/regions
- Playback and blending:
- `fadeDuration?: number` - Cross-fade/fade time (default: `0.1`)
- `crossFade?: boolean` - Whether to cross-fade from current layer action (default: `true`)
- `sync?: boolean` - Sync time with current action on same layer (if any)
- `paused?: boolean`
- `loop?: AnimationActionLoopStyles` - Defaults to `LoopRepeat`
- `layer?: string` - Overrides the current ``
- Timeline control (from `@react-three/timeline`):
- `init?(): void | (() => void)` - Called when the action starts; return a cleanup
- `update?(state, delta): void` - Per-frame update
- `until?(): Promise` - Resolve to stop; defaults to when the clip finishes
- `dependencies?: unknown[]` - Re-run when any dependency changes
- Advanced:
- `additiveReferenceClip?: AnimationClip` - Use an additive version of the clip relative to this reference clip (prefer `` for convenience)
**Example:**
```tsx
```
### ``
Convenience wrapper around `` that plays an additive version of the clip, using a provided reference pose/clip (e.g., aim offsets layered over locomotion).
**Props:**
- All `` props, except it uses:
- `referenceClip: CharacterAnimationOptions` - Clip used as the additive reference pose
**Example:**
```tsx
```
### ``
Component for placing content inside the character model at specific bones.
**Props:**
- `bone: VRMHumanBoneName` - The bone name to access
```tsx
```
## Hooks
| Hook | Description | Returns |
| ------------------------------------------------ | ------------------------------------------------------------- | ---------------------------------------------------------------- |
| `useViverseClient()` | Returns the VIVERSE client instance for making API calls | `Client` |
| `useViverseAuth()` | Returns the current authentication state | Auth object with access tokens, or `undefined` |
| `useViverseAvatarClient()` | Returns the avatar client for avatar-related operations | `AvatarClient \| undefined` |
| `useViverseLogin()` | Returns a function to initiate the VIVERSE login flow | Login function |
| `useViverseLogout()` | Returns a function to initiate the VIVERSE logout flow | Logout function |
| `useViverseProfile()` | Fetches the user's profile (name, avatar info) using Suspense | Profile object with `name`, `activeAvatar`, etc., or `undefined` |
| `useViverseActiveAvatar()` | Fetches the user's currently selected avatar using Suspense | Avatar object with `vrmUrl`, `headIconUrl`, etc., or `undefined` |
| `useViverseAvatarList()` | Fetches the user's personal avatar collection using Suspense | Array of avatar objects, or `undefined` |
| `useViversePublicAvatarList()` | Fetches publicly available avatars using Suspense | Array of public avatar objects, or `undefined` |
| `useViversePublicAvatarByID(id)` | Fetches a specific public avatar by ID using Suspense | Avatar object, or `undefined` |
| `useIsMobile()` | Returns `true` on touch-centric/mobile devices (media query) | `boolean` |
| `useCharacterModel()` | Gets the current character model from context | `CharacterModel` |
| `useCharacterModelLoader(options?)` | Loads a character model with Suspense | `CharacterModel` |
| `useCharacterAnimationLoader(model, options)` | Loads an animation clip for a model with Suspense | `AnimationClip` |
| `useBvhPhysicsWorld()` | Accesses the BVH physics world context | `BvhPhysicsWorld` |
| `useBvhCharacterPhysics(modelRef, options?)` | Character controller physics tied to a model ref | `BvhCharacterPhysics` |
| `useCharacterCameraBehavior(modelRef, options?)` | Camera behavior that follows/rotates around model | `RefObject` |
| `useSimpleCharacterActionBindings(...)?` | Deprecated: sets up default action bindings | `void` |
| `useScreenButton(image)` | Create and mount a styled on-screen button element | `HTMLElement` |
> [!NOTE]
> `useViverseClient()` returns `undefined` if not within a `` provider or if no `clientId` is provided. Also all avatar-related hooks return `undefined` when the user is not authenticated.
### useIsMobile
Lightweight media-query based mobile detection. It subscribes to `@media (hover: none) and (pointer: coarse)`.
```tsx
import { useIsMobile } from '@react-three/viverse'
function MobileOnlyUI() {
const isMobile = useIsMobile()
return isMobile ? Shown on mobile
: null
}
```
### Action Binding Hooks
Actions allow to decouple specific user inputs from game/business logic.
One action can be connected to multiple inputs via Action bindings (keyboard, mouse/touch, on-screen controls). For background on actions vs. action bindings and how to create custom ones, see the Actions tutorial: [Create and use actions](#doc-tutorials-actions). We provide several easy to use hooks to setup default action bindings for general use cases such as pressing a button or specific use case such as locomotion using a keyboard.
#### `useKeyboardActionBinding(action, options)`
- **Description:** Binds a `KeyboardEvent` or boolean state action to one or more keys.
- **Options:** `{ keys: string[]; requiresPointerLock?: boolean }`
- **Returns:** `void`
```tsx
useKeyboardActionBinding(jumpAction, { keys: ['Space'] })
```
#### `usePointerButtonActionBinding(action, options)`
- **Description:** Binds a pointer button (mouse/touch) event or state action.
- **Options:** `{ domElement?: HTMLElement | RefObject; buttons?: number[]; requiresPointerLock?: boolean }`
- **Returns:** `void`
```tsx
usePointerButtonActionBinding(fireAction, { buttons: [0] }) // left mouse / primary touch
```
#### `usePointerCaptureRotateZoomActionBindings(options)`
- **Description:** Enables rotate/zoom camera controls using Pointer Capture on the canvas.
- **Options:** `{ rotationSpeed?: number; zoomSpeed?: number }`
- **Returns:** `void`
```tsx
usePointerCaptureRotateZoomActionBindings({ rotationSpeed: 1000, zoomSpeed: 1000 })
```
#### `usePointerLockRotateZoomActionBindings(options)`
- **Description:** Enables rotate/zoom camera controls using Pointer Lock on the canvas.
- **Options:** `{ rotationSpeed?: number; zoomSpeed?: number; lockOnClick?: boolean }`
- **Returns:** `void`
```tsx
usePointerLockRotateZoomActionBindings({ lockOnClick: true })
```
#### `useKeyboardLocomotionActionBindings(options)`
- **Description:** WASD movement, Shift to run, Space to jump.
- **Options:** `{ moveForwardKeys?, moveBackwardKeys?, moveLeftKeys?, moveRightKeys?, runKeys?, jumpKeys?, requiresPointerLock? }` (arrays of key strings)
- **Returns:** `void`
```tsx
useKeyboardLocomotionActionBindings({ requiresPointerLock: false })
```
#### `useScreenJoystickLocomotionActionBindings(options)`
- **Description:** On-screen joystick for movement and run on mobile devices.
- **Options:** `{ runDistancePx?: number; deadZonePx?: number }`
- **Returns:** `void`
```tsx
useScreenJoystickLocomotionActionBindings({ deadZonePx: 8, runDistancePx: 40 })
```
## SimpleCharacter Options
The `SimpleCharacter` component can be configured with a variety of props but also supports all the default group props, such as position, rotation, and scale.
### `useViverseAvatar` flag
Allows to configure whether the users vrm avatar should be displayed as the character model.
- **Default:** `true`
### `movement` Options
- **walk:** `object | boolean` - Enable walking (default: `true`)
- **speed:** Movement speed in units per second (default: `3`)
- Set to `false` to disable walking
- **run:** `object | boolean` - Enable running (default: `true`)
- **speed:** Running speed in units per second (default: `6`)
- Set to `false` to disable running
- **jump:** `object | boolean` - Enable jumping (default: `true`)
- **delay:** Time before jump starts in seconds (default: `0.2`)
- **bufferTime:** Jump input buffer time in seconds (default: `0.1`)
- **speed:** Jump velocity in units per second (default: `8`)
- Set to `false` to disable jumping
### `actionBindings` Options
An array of action binding classes to instantiate for handling controls
- **Default:** `[ScreenJoystickLocomotionActionBindings, ScreenButtonJumpActionBindings, PointerCaptureRotateZoomActionBindings, KeyboardLocomotionActionBindings]`
- Configure action bindings with custom action binding classes
**Available Action Binding Classes provided by @pmndrs/viverse:**
- `KeyboardLocomotionActionBindings` - WASD movement, Space for jump, Shift for run
- `PointerCaptureRotateZoomActionBindings` - Mouse look with pointer capture (requires manual `setPointerCapture`)
- `PointerLockRotateZoomActionBindings` - Mouse look with pointer lock (requires manual `requestPointerLock`)
- `ScreenJoystickLocomotionActionBindings` - On-screen joystick for movement and run (mobile). Options: `{ screenJoystickDeadZonePx?, screenJoystickRunDistancePx? }`
- `ScreenButtonJumpActionBindings` - On-screen jump button (mobile-only). Visible only on mobile.
### `model` Options
- **url:** `string` - URL to VRM or GLTF model file
- **type:** `"gltf" | "vrm"` - the type of file to be loaded (optional)
- **castShadow:** `boolean` - Enable shadow casting (default: `true`)
- **receiveShadow:** `boolean` - Enable shadow receiving (default: `true`)
- **boneRotationOffset:** `Quaternion | undefined` - Allows to apply an rotation offset when placing objects as children of the character's bones (default: `undefined`)
- Set to `false` to disable model loading
- Set to `true` or omit to use default robot model
### `physics` Options
- **capsuleRadius:** `number` - Character collision capsule radius (default: `0.4`)
- **capsuleHeight:** `number` - Character collision capsule height (default: `1.7`)
- **gravity:** `number` - Gravity acceleration in m/s² (default: `-20`)
- **linearDamping:** `number` - Air resistance coefficient (default: `0.1`)
- **maxGroundSlope:** `number` - Max slope for a collider to be detected as walkable (default: `0.5`)
### `cameraBehavior` Options
- **collision:** `object | boolean` - Enable camera collision (default: `true`)
- **offset:** `number` - Collision offset distance (default: `0.2`)
- **characterBaseOffset:** `Vector3 | [number, number, number]` - Camera position relative to character (default: `[0, 1.3, 0]`)
- **rotation:** `object | boolean` - Enable camera rotation (default: `true`)
- **minPitch:** `number` - Minimum pitch angle (default: `-Math.PI/2`)
- **maxPitch:** `number` - Maximum pitch angle (default: `Math.PI/2`)
- **minYaw:** `number` - Minimum yaw angle (default: `-Infinity`)
- **maxYaw:** `number` - Maximum yaw angle (default: `+Infinity`)
- **speed:** `number` - Rotation speed multiplier (default: `1000`)
- **zoom:** `object | boolean` - Enable camera zoom (default: `true`)
- **speed:** `number` - Zoom speed multiplier (default: `1000`)
- **minDistance:** `number` - Minimum camera distance (default: `1`)
- **maxDistance:** `number` - Maximum camera distance (default: `7`)
### `animation` Options
- **yawRotationBasedOn:** `'camera' | 'movement'` - Character rotation basis (default: `'movement'`)
- **maxYawRotationSpeed:** `number` - Maximum rotation speed (default: `10`)
- **crossFadeDuration:** `number` - Animation blend time in seconds (default: `0.1`)
The `SimpleCharacter` uses the following animations `walk`, `run`, `idle`, `jumpForward`, `jumpUp`, `jumpLoop`, `jumpDown` each with the following options:
- **url:** `string` - Animation file URL
- **type:** `'gltf' | 'vrma' | 'fbx' | 'bvh'` - Animation file type (optional)
- **boneMap** - Allows to map the bone names of the animation amature to the standard VRM bone names
- **removeXZMovement:** `boolean` - Remove horizontal movement from animation
- **trimTime:** `{ start?: number; end?: number }` - Trim animation timing
- **scaleTime:** `number` - Scale animation playback speed
## PrototypeMaterial
The `` component provides a textured material for prototyping using kenney.nl's prototype texture.
- **color:** `ColorRepresentation` - Material color tint
- **repeat:** `Vector2` - Texture repeat pattern (accessible as `materialRef.current.repeat`)
- All standard Three.js MeshPhongMaterial properties
```tsx
// As JSX element
```
# Examples
[](https://worlds.viverse.com/wyTQbnB)
Simple Game Example w. a Player Tag
[](https://worlds.viverse.com/kjALCp2)
Simple Game Example using Vanilla Threejs
[](https://worlds.viverse.com/UB6VBmX)
Augemented Reality Example using WebXR
[](https://worlds.viverse.com/asuA4ay)
Virtual Reality Example using WebXR
[](https://worlds.viverse.com/ziWwWno)
Fortnite Character Controller Example
# Building a Simple Game
In this tutorial, we'll build the following simple 3D platformer game using `@react-three/viverse` with:
- Character movement (WASD + mouse look)
- Jumping mechanics
- Physics-based collision detection
- A simple level with platforms to jump on
- Respawn system when falling off the map
*Here's a preview of what we will build in this tutorial:*
Dependencies:
```js
{
'three': 'latest',
'@react-three/fiber': '<9',
'@react-three/viverse': 'latest',
'@react-three/drei': '<10'
}
```
Files:
File: /Scene.tsx
```tsx
import { Sky } from '@react-three/drei'
import { SimpleCharacter, BvhPhysicsBody, PrototypeBox } from '@react-three/viverse'
import { useRef } from 'react'
import { useFrame } from '@react-three/fiber'
import { Group } from 'three'
export function Scene() {
const characterRef = useRef(null)
// Respawn logic - reset character position if they fall off the map
useFrame(() => {
if (characterRef.current == null) {
return
}
if (characterRef.current.position.y < -10) {
characterRef.current.position.set(0, 0, 0)
}
})
return (
<>
{/* Environment */}
{/* Lighting */}
{/* Character */}
{/* Level Geometry */}
{/* Main ground */}
{/* Platforms */}
>
)
}
```
File: /App.tsx
```tsx
import { Suspense } from "react"
import { Canvas } from '@react-three/fiber'
import { Viverse } from '@react-three/viverse'
import { Scene } from "./Scene"
export default function App() {
return (
)
}
```
## Step 0: Prerequisites
Make sure you have the required dependencies installed:
```bash
npm install three @react-three/fiber @react-three/viverse @react-three/drei
```
## Step 1: Setting Up the Canvas
First, let's create the basic Canvas setup with shadows and proper camera settings:
```tsx
import { Canvas } from '@react-three/fiber'
import { Viverse } from '@react-three/viverse'
import { Suspense } from 'react'
export function App() {
return (
)
}
```
## Step 2: Adding the Scene and creating the Sky
Let's create another component called `Scene` and add the sky and basic lighting. Add the `Sky` import from `@react-three/drei`:
```tsx
import { Sky } from '@react-three/drei'
export function Scene() {
return (
<>
{/* Environment */}
{/* Basic lighting */}
>
)
}
```
At this point, you should see a beautiful sky gradient in your scene!
## Step 3: Building the Level
Now add the level geometry. Import `BvhPhysicsBody` and `PrototypeBox` from `@react-three/viverse`, and expand the directional light with shadow properties:
```tsx
import { Sky } from '@react-three/drei'
import { BvhPhysicsBody, PrototypeBox } from '@react-three/viverse'
export function Scene() {
return (
<>
{/* Environment */}
{/* Lighting - expanded with shadow settings */}
{/* Platforms */}
>
)
}
```
Now you should see a colorful platformer level with various platforms at different heights!
## Step 4: Adding the Character
Next we will add the character. Import `SimpleCharacter` from `@react-three/viverse`:
```tsx
import { Sky } from '@react-three/drei'
import { SimpleCharacter, BvhPhysicsBody, PrototypeBox } from '@react-three/viverse'
import { useRef } from 'react'
export function Scene() {
return (
<>
{/* Environment */}
{/* Lighting */}
{/* Level Geometry */}
{/* ... platforms remain the same ... */}
>
)
}
```
Great! Now you can move around with WASD keys, look around with the mouse, and jump with the spacebar. Try jumping between the platforms!
## Step 5: Adding Respawn Logic
Finally, we will add the respawn system. Import `useRef` from `react` and `useFrame` from `@react-three/fiber` and add the respawn logic:
```tsx
import { Sky } from '@react-three/drei'
import { SimpleCharacter, BvhPhysicsBody, PrototypeBox } from '@react-three/viverse'
import { useRef } from 'react'
import { Group } from 'three'
import { useFrame } from '@react-three/fiber' // NEW
export function Scene() {
const characterRef = useRef(null)
// Respawn logic - NEW
useFrame(() => {
if (characterRef.current == null) {
return
}
if (characterRef.current.position.y < -10) {
characterRef.current.position.set(0, 0, 0)
}
})
return (
<>
{/* ... rest remains the same ... */}
>
)
}
```
Perfect! Now if you fall off the map (below y = -10), you'll automatically respawn at the starting position (0, 0, 0).
# First Person Controls
In this tutorial we will configure the `SimpleCharacter` to use first person controls with the following result:
_Here's a preview of this tutorial's result:_
Dependencies:
```js
{
'three': 'latest',
'@react-three/fiber': '<9',
'@react-three/viverse': 'latest',
'@react-three/drei': '<10'
}
```
Files:
File: /App.tsx
```tsx
import { Sky } from '@react-three/drei'
import { Canvas } from '@react-three/fiber'
import {
Viverse,
SimpleCharacter,
BvhPhysicsBody,
PrototypeBox,
FirstPersonCharacterCameraBehavior,
PointerLockRotateZoomActionBindings,
KeyboardLocomotionActionBindings,
} from '@react-three/viverse'
export default function App() {
return (
)
}
```
First, we switch from third-person to first-person camera behavior and hide the character model to prevent the model from occluding the players view.
```tsx
```
**Changes:**
- `model={false}` - Hides the character model since in first-person view, you don't want to see your own character
- `cameraBehavior={FirstPersonCharacterCameraBehavior}` - Switches from the default third-person camera to first-person camera behavior
Next, you need to set up the appropriate action bindings for first-person movement and looking around:
```tsx
```
- `KeyboardLocomotionActionBindings` - Handles WASD movement action bindings for walking around
- `PointerLockRotateZoomActionBindings` - Enables mouse look action bindings for rotating the camera/view direction
# Augmented and Virtual Reality
This tutorial shows how to add Augmented Reality (AR) and Virtual Reality (VR) support to your `@react-three/viverse` games. We'll start with AR and then show the additional changes needed for VR.
## Prerequisites
Make sure you have the `@react-three/xr` package installed:
```bash
npm install @react-three/xr
```
## Augmented Reality (AR)
Let's start by adding AR support to a basic `@react-three/viverse` game.
*Here's the AR app we'll build now:*
Dependencies:
```js
{
'three': 'latest',
'@react-three/fiber': '<9',
'@react-three/viverse': 'latest',
'@react-three/drei': '<10',
'@react-three/xr': 'latest'
}
```
Files:
File: /App.tsx
```tsx
import { Canvas } from '@react-three/fiber'
import { Viverse } from '@react-three/viverse'
import { XR, XROrigin, createXRStore } from '@react-three/xr'
import { Scene } from './Scene'
const store = createXRStore({ offerSession: 'immersive-ar' })
export default function App() {
return (
)
}
```
File: /Scene.tsx
```tsx
import { useFrame } from '@react-three/fiber'
import {
SimpleCharacter,
BvhPhysicsBody,
PrototypeBox,
useXRControllerLocomotionActionBindings,
} from '@react-three/viverse'
import { useRef } from 'react'
import { Group } from 'three'
export function Scene() {
const characterRef = useRef(null)
useFrame(() => {
if (characterRef.current == null) {
return
}
if (characterRef.current.position.y < -10) {
characterRef.current.position.set(0, 0, 0)
}
})
useXRControllerLocomotionActionBindings()
return (
<>
>
)
}
```
### Step 1: Set Up the XR Store
Create an XR store configured for AR. Add these imports and create the store:
```tsx
import { XR, XROrigin, createXRStore } from '@react-three/xr'
const store = createXRStore({ offerSession: 'immersive-ar' })
```
The `offerSession: 'immersive-ar'` option tells the XR system that we want to create an AR experience.
### Step 2: Wrap Your Scene with XR Components
Wrap your scene content with the `XR` component and add an `XROrigin`:
```tsx
export function App() {
return (
)
}
```
**Key points:**
- `XROrigin` defines the coordinate system origin for AR tracking
- `scale={10}` makes the scene 10x larger relative to the real world
- `position-y={-8} position-z={10}` adjusts the initial positioning
### Step 3: Remove Sky Component
For AR, you don't want a sky background since the camera feed should show through. Remove any `Sky` components from your scene:
```tsx
export function Scene() {
return (
<>
{/* Remove for AR */}
{/* ... rest of your scene */}
>
)
}
```
### Step 4: Use XR Controller Action Bindings
Add XR controller action bindings using the `useXRControllerLocomotionActionBindings` hook:
```tsx
import { useXRControllerLocomotionActionBindings } from '@react-three/viverse'
export function Scene() {
useXRControllerLocomotionActionBindings()
return (
<>
{/* ... rest of scene */}
>
)
}
```
**Key changes:**
- `cameraBehavior={false}` - Disables automatic camera control (AR handles this)
- `useXRControllerLocomotionActionBindings()` - Hook that provides controller action bindings for movement
The `useXRControllerLocomotionActionBindings` hook binds XR controller inputs to character locomotion actions:
- **Left thumbstick** controls movement (forward/backward/left/right)
- **Right controller A button** triggers jumping
- **Left trigger** enables running
## Virtual Reality (VR)
Now let's look at how to add VR support to a game building on the knowledge from adding AR support.
*Here's the VR app we'll build now:*
Dependencies:
```js
{
'three': 'latest',
'@react-three/fiber': '<9',
'@react-three/viverse': 'latest',
'@react-three/drei': '<10',
'@react-three/xr': 'latest'
}
```
Files:
File: /App.tsx
```tsx
import { Canvas } from '@react-three/fiber'
import { Viverse } from '@react-three/viverse'
import { XR, createXRStore } from '@react-three/xr'
import { Scene } from './Scene'
const store = createXRStore({ offerSession: 'immersive-vr' })
export default function App() {
return (
)
}
```
File: /Scene.tsx
```tsx
import { useFrame } from '@react-three/fiber'
import {
SimpleCharacter,
BvhPhysicsBody,
PrototypeBox,
useXRControllerLocomotionActionBindings,
} from '@react-three/viverse'
import { Sky } from '@react-three/drei'
import { XROrigin, useXRInputSourceState } from '@react-three/xr'
import { useRef } from 'react'
import { Group } from 'three'
export function Scene() {
const characterRef = useRef(null)
useFrame(() => {
if (characterRef.current == null) {
return
}
if (characterRef.current.position.y < -10) {
characterRef.current.position.set(0, 0, 0)
}
})
useXRControllerLocomotionActionBindings()
return (
<>
>
)
}
function SnapRotateXROrigin() {
const ref = useRef(null)
const rightController = useXRInputSourceState('controller', 'right')
const prev = useRef(0)
useFrame(() => {
if (ref.current == null) return
const current = Math.round(rightController?.gamepad?.['xr-standard-thumbstick']?.xAxis ?? 0)
if (current < 0 && prev.current >= 0) {
// Rotate left
ref.current.rotation.y += Math.PI / 2
}
if (current > 0 && prev.current <= 0) {
// Rotate right
ref.current.rotation.y -= Math.PI / 2
}
prev.current = current
})
return
}
```
### Step 1: Change Session Type to VR
We can update the offer session to show the user a native "VR" enter button.
```tsx
const store = createXRStore({
offerSession: 'immersive-vr',
})
```
### Step 2: Re-add the Sky for VR
Unlike AR, VR needs a sky background since there's no camera feed:
```tsx
import { Sky } from '@react-three/drei'
export function Scene() {
return (
<>
{/* ... rest of scene */}
>
)
}
```
### Step 3: Hide the Character Model
In VR, you typically don't want to see your own character model:
```tsx
```
**Key change:**
- `model={false}` - Hides the character model in VR
### Step 4: Place the XROrigin into the Simple Character and Optionally Add Snap Rotation
As the XROrigin defines the player's position, we need to remove it from outside the Scene and add it into the SimpleCharacter.
```tsx
import { useXRInputSourceState } from '@react-three/xr'
```
For comfort in VR, you can add snap rotation using the right thumbstick by building a SnapRotateXROrigin which replaces the XROrigin component.
```tsx
import { useXRInputSourceState } from '@react-three/xr'
function SnapRotateXROrigin() {
const ref = useRef(null)
const rightController = useXRInputSourceState('controller', 'right')
const prev = useRef(0)
useFrame(() => {
if (ref.current == null) return
const current = Math.round(rightController?.gamepad?.['xr-standard-thumbstick']?.xAxis ?? 0)
if (current < 0 && prev.current >= 0) {
// Rotate left
ref.current.rotation.y += Math.PI / 2
}
if (current > 0 && prev.current <= 0) {
// Rotate right
ref.current.rotation.y -= Math.PI / 2
}
prev.current = current
})
return
}
```
## Summary
**For AR:**
1. Use `offerSession: 'immersive-ar'`
2. Remove `Sky` component
3. Use `useXRControllerLocomotionActionBindings()` for to bind the locomotion actions
4. Set `cameraBehavior={false}` on SimpleCharacter
5. Add `XROrigin` with appropriate scaling/positioning
**Additional changes for VR:**
1. Change to `offerSession: 'immersive-vr'`
2. Re-add `Sky` component
3. Set `model={false}` to hide character
4. Place the XROrigin into the SimpleCharacter and optionally add snap rotation for comfort
# Accessing Avatar and Profile
This tutorial shows you how to display a player tag above the character by accessing the user profile information from VIVERSE.
_Here's a preview of what we'll build in this tutorial:_
Dependencies:
```js
{
'three': 'latest',
'@react-three/fiber': '<9',
'@react-three/viverse': 'latest',
'@react-three/drei': '<10',
"@react-three/uikit": "^1.0.41"
}
```
Files:
File: /Playertag.tsx
```tsx
import { useViverseProfile } from '@react-three/viverse'
import { Container, Image, Text } from '@react-three/uikit'
import { useRef } from 'react'
import { useFrame } from '@react-three/fiber'
import { Group } from 'three'
export function PlayerTag() {
const profile = useViverseProfile() ?? {
name: 'Anonymous',
activeAvatar: { headIconUrl: 'https://picsum.photos/200' },
}
const ref = useRef(null)
// Make the tag always face the camera
useFrame((state) => {
if (ref.current == null) {
return
}
ref.current.quaternion.copy(state.camera.quaternion)
})
return (
{profile.name}
) }
```
File: /App.tsx
```tsx
import {Sky} from '@react-three/drei' import {Canvas} from '@react-three/fiber' import
{(Viverse, SimpleCharacter, BvhPhysicsBody, PrototypeBox)} from '@react-three/viverse' import {PlayerTag} from
"./Playertag"
export default function App() {
return (
)
}
```
First, we use the `useViverseProfile()` hook to fetch the current user's profile from VIVERSE, including their name and avatar information. We provide a fallback for when the user isn't logged in:
```tsx
const profile = useViverseProfile() ?? {
name: 'Anonymous',
activeAvatar: { headIconUrl: 'https://picsum.photos/200' },
}
```
Next, we need a 3D ui library, install it via
```bash
npm install @react-three/uikit
```
UIKit provides HTML-like components (`Container`, `Image`, `Text`) that work in 3D space. We create a card-like layout with flexbox:
```tsx
{profile.name}
```
Next, we use `useFrame` to constantly update the tag's rotation to match the camera:
```tsx
import { Group } from 'three'
const ref = useRef(null)
useFrame((state) => {
if (ref.current == null) {
return
}
ref.current.quaternion.copy(state.camera.quaternion)
})
```
The full `PlayerTag` component looks like this:
```tsx
import { useViverseProfile } from '@react-three/viverse'
import { Container, Image, Text } from '@react-three/uikit'
import { useRef } from 'react'
import { useFrame } from '@react-three/fiber'
import { Group } from 'three'
export function PlayerTag() {
const profile = useViverseProfile() ?? {
name: 'Anonymous',
activeAvatar: { headIconUrl: 'https://picsum.photos/200' },
}
const ref = useRef(null)
// Make the tag always face the camera
useFrame((state) => {
if (ref.current == null) {
return
}
ref.current.quaternion.copy(state.camera.quaternion)
})
return (
{profile.name}
)
}
```
Now finally lets add the PlayerTag as a child of SimpleCharacter to display it
```tsx
```
# Equipping the Character With Items
This tutorial shows you how to equip your character with items by attaching 3D objects to specific bones. We'll create a simple sword using just two meshes and attach it to the character's right hand.
_Here's a preview of what we'll build in this tutorial:_
Dependencies:
```js
{
'three': 'latest',
'@react-three/fiber': '<9',
'@react-three/viverse': 'latest',
'@react-three/drei': '<10'
}
```
Files:
File: /App.tsx
```tsx
import { Sky } from '@react-three/drei'
import { Canvas } from '@react-three/fiber'
import { Viverse, SimpleCharacter, BvhPhysicsBody, PrototypeBox, CharacterModelBone } from '@react-three/viverse'
export default function App() {
return (
)
}
```
## Understanding Bone Attachment
The `CharacterModelBone` component allows you to attach any 3D object to specific bones in the character's skeleton. This is perfect for equipping weapons, accessories, or any items that should move with the character.
### Step 1: Import the CharacterModelBone Component
First, import the `CharacterModelBone` component from `@react-three/viverse`:
```tsx
import { CharacterModelBone } from '@react-three/viverse'
```
### Step 2: Add a Simple Sword to the `"rightHand"`
Next, we place the CharacterModelBone inside the `SimpleCharacter` component and attach it to the `"rightHand"`. We then build a simple sword using two meshes. For better looks, you probably want to import your own 3D model.
```tsx
{/* Blade */}
{/* Handle */}
```
# Custom Models and Animations
## Using Custom Character Models
By default, the `SimpleCharacter` component uses a built-in robot avatar. You can easily replace this with your own 3D model by providing a URL to the model file in any of the following formats:
- **VRM** - The standardized VRM format for avatars requires no additional configuration
- **GLTF** - (or also glb) Standard 3D Model format - make sure to use the standard vrm bone names as shown below
VRM 1.0 humanoid bone names (click to expand)
The following are the standard VRM 1.0 humanoid bone names your GLTF rig should use (aligned with the VRM specification). If your model uses these names, animations and retargeting will work reliably:
| Bone name | Description |
| ------------------------- | ---------------------------------------------- |
| `hips` | Pelvis root; parent of the spine and both legs |
| `spine` | Lower/waist spine segment above hips |
| `chest` | Mid/upper torso segment above spine |
| `upperChest` | Optional highest chest segment below neck |
| `neck` | Neck base; parent of head |
| `head` | Head root; parent of eyes and jaw |
| `leftEye` | Left eyeball transform |
| `rightEye` | Right eyeball transform |
| `jaw` | Jaw/mandible pivot |
| `leftUpperLeg` | Left thigh (upper leg) |
| `leftLowerLeg` | Left shin (lower leg) |
| `leftFoot` | Left foot root/ankle |
| `leftToes` | Left toe base |
| `rightUpperLeg` | Right thigh (upper leg) |
| `rightLowerLeg` | Right shin (lower leg) |
| `rightFoot` | Right foot root/ankle |
| `rightToes` | Right toe base |
| `leftShoulder` | Left clavicle/shoulder pivot |
| `leftUpperArm` | Left upper arm (humerus) |
| `leftLowerArm` | Left forearm |
| `leftHand` | Left hand/wrist root |
| `rightShoulder` | Right clavicle/shoulder pivot |
| `rightUpperArm` | Right upper arm (humerus) |
| `rightLowerArm` | Right forearm |
| `rightHand` | Right hand/wrist root |
| `leftThumbMetacarpal` | Left thumb metacarpal (root of thumb) |
| `leftThumbProximal` | Left thumb proximal phalanx |
| `leftThumbDistal` | Left thumb distal phalanx |
| `leftIndexProximal` | Left index proximal phalanx |
| `leftIndexIntermediate` | Left index intermediate phalanx |
| `leftIndexDistal` | Left index distal phalanx |
| `leftMiddleProximal` | Left middle proximal phalanx |
| `leftMiddleIntermediate` | Left middle intermediate phalanx |
| `leftMiddleDistal` | Left middle distal phalanx |
| `leftRingProximal` | Left ring proximal phalanx |
| `leftRingIntermediate` | Left ring intermediate phalanx |
| `leftRingDistal` | Left ring distal phalanx |
| `leftLittleProximal` | Left little/pinky proximal phalanx |
| `leftLittleIntermediate` | Left little/pinky intermediate phalanx |
| `leftLittleDistal` | Left little/pinky distal phalanx |
| `rightThumbMetacarpal` | Right thumb metacarpal (root of thumb) |
| `rightThumbProximal` | Right thumb proximal phalanx |
| `rightThumbDistal` | Right thumb distal phalanx |
| `rightIndexProximal` | Right index proximal phalanx |
| `rightIndexIntermediate` | Right index intermediate phalanx |
| `rightIndexDistal` | Right index distal phalanx |
| `rightMiddleProximal` | Right middle proximal phalanx |
| `rightMiddleIntermediate` | Right middle intermediate phalanx |
| `rightMiddleDistal` | Right middle distal phalanx |
| `rightRingProximal` | Right ring proximal phalanx |
| `rightRingIntermediate` | Right ring intermediate phalanx |
| `rightRingDistal` | Right ring distal phalanx |
| `rightLittleProximal` | Right little/pinky proximal phalanx |
| `rightLittleIntermediate` | Right little/pinky intermediate phalanx |
| `rightLittleDistal` | Right little/pinky distal phalanx |
```tsx
import { SimpleCharacter } from '@react-three/viverse'
export function MyCharacter() {
return
}
```
## Adding Custom Animations
Your can replace the default animations of the `SimpleCharacter` component with files in any of these three supported animation formats:
- **VRMA** (VRM Animation) - The native VRM animation format
- **FBX** - Popular file format for character animations
- **GLTF** - Standard 3D format with animations
- **Mixamo** - **deprecated** - use remove `type: 'mixamo'` and add `boneMap: mixamoBoneMap` instead
Make sure to either use a bone map, e.g. when your bone names follow the mixamo naming conventions use the `mixamoBoneMap` or use the standard VRM bones for the animation amature as shown above under "VRM 1.0 humanoid bone names".
Each animation type can be configured individually:
```tsx
```
You can customize any of these animation slots:
- `walk` - Walking animation
- `run` - Running animation
- `idle` - Standing idle animation
- `jumpStart` - Beginning of jump
- `jumpUp` - Ascending during jump
- `jumpLoop` - Mid-air loop animation
- `jumpDown` - Landing animation
# Actions and Action Bindings
Actions allow to decouple specific user inputs from game/business logic. Inputs (keyboard, mouse/touch, controllers, on‑screen UI) are converted by action bindings into actions that game systems consume on every frame or whenever an event happens to act upon the action.
- **Input → Action Binding → Action → Effect**
- Input: hardware or UI event (key, mouse move, touch, thumbstick, button)
- Action Binding: translates that input into a domain signal
- Action: a shared signal (event or state) consumed by systems
- Effect: the game changes on frame (e.g., camera rotates, character moves, jumps)
## State vs. Event actions
- **StateAction<T>**
- Represents a continuous state that persists until changed (e.g., movement axes, “is running”).
- Merges multiple writers (e.g., keyboard + joystick) into one value each frame.
- Read anywhere via `.get()`.
- **EventAction<T>**
- Represents instantaneous events (e.g., “jump pressed”, “rotate delta”, “zoom delta”).
- Produces values per frame via a reader; values are combined (sum, etc.) before consumption.
For example, the CharacterCameraBehavior consumes rotation and zoom event actions, which are StateActions, every frame. The StateAction returns the final value as an accumulation of all the inputs since the last frame.
## Built-in actions:
- Movement: `MoveForwardAction`, `MoveBackwardAction`, `MoveLeftAction`, `MoveRightAction`, `RunAction` (State)
- Jumping: `JumpAction` (Event)
- Camera: `RotateYawAction`, `RotatePitchAction`, `ZoomAction` (Event)
## Built-in action bindings
- Keyboard locomotion: `useKeyboardLocomotionActionBindings(...)`
- Mouse/touch camera (pointer capture): `usePointerCaptureRotateZoomActionBindings(...)`
- Mouse camera (pointer lock): `usePointerLockRotateZoomActionBindings(...)`
- Single key/button bindings: `useKeyboardActionBinding(...)`, `usePointerButtonActionBinding(...)`
- Mobile UI: `useScreenJoystickLocomotionActionBindings(...)`, `useScreenButton(...)`
These hooks connect hardware inputs to actions. Multiple bindings can feed the same action; values are safely merged.
## Example: Rotate the camera with the mouse (Pointer Lock)
*This shows the full pipeline: mouse movement → pointer-lock binding → rotation actions → camera rotates on frame.*
Dependencies:
```js
{
'three': 'latest',
'@react-three/fiber': '<9',
'@react-three/viverse': 'latest',
'@react-three/drei': '<10'
}
```
Files:
File: /App.tsx
```tsx
import { Canvas } from '@react-three/fiber'
import { Sky } from '@react-three/drei'
import {
Viverse,
SimpleCharacter,
BvhPhysicsBody,
PrototypeBox,
usePointerLockRotateZoomActionBindings,
} from '@react-three/viverse'
function Bindings() {
// Binds mouse movement (while pointer is locked) to RotateYawAction/RotatePitchAction,
// and mouse wheel to ZoomAction. The camera behavior consumes these every frame.
usePointerLockRotateZoomActionBindings({ lockOnClick: true })
return null
}
export default function App() {
return (
)
}
```
Click the canvas once to lock the pointer. Move the mouse to rotate the camera; use the mouse wheel to zoom. That’s the actions pipeline in action.
## Example: Map Q/E keys to camera yaw rotation
You can also route keyboard inputs into camera rotation by mapping `KeyboardEvent` to yaw deltas and binding them to the same `RotateYawAction`.
```tsx
import { useKeyboardActionBinding } from '@react-three/viverse'
import { RotateYawAction } from '@react-three/viverse'
function KeyboardYawBindings() {
// Map KeyboardEvent → number (negative = left, positive = right)
const rotateFromKeyboard = RotateYawAction.mapFrom((e: KeyboardEvent) => {
if (e.code === 'KeyQ') return -0.02
if (e.code === 'KeyE') return 0.02
return 0
})
// Bind Q/E key presses to the mapped action
useKeyboardActionBinding(rotateFromKeyboard, { keys: ['KeyQ', 'KeyE'] })
return null
}
```
Place `` alongside your other bindings. Now both mouse and keys contribute to `RotateYawAction`; their values are combined before the camera consumes them each frame.
## Example: Keyboard locomotion and jump
To bind classic WASD + Shift + Space to movement and jump, use the built-in locomotion bindings:
```tsx
import { useKeyboardLocomotionActionBindings } from '@react-three/viverse'
function LocomotionBindings() {
useKeyboardLocomotionActionBindings({
requiresPointerLock: false, // set to true if you want movement only when pointer is locked
})
return null
}
```
These bindings write to:
- `MoveForwardAction`, `MoveBackwardAction`, `MoveLeftAction`, `MoveRightAction` (State: 0..1)
- `RunAction` (State: boolean)
- `JumpAction` (Event)
Your character controller (e.g., `SimpleCharacter`) reads these to move and animate on frame, regardless of the input device that produced them.
## Takeaways
- **Decouple input from gameplay**: actions provide a stable interface your systems consume.
- **Compose inputs**: multiple bindings can feed the same action; values merge predictably.
- **Think “signals,” not devices**: code against actions like “move forward” or “yaw delta,” not specific keys or hardware.
# Custom Character Controller
In this tutorial, you’ll build a custom extensible humanoid character controller in a fortnite style with support for aim, shoot, reload, run, and jump, using `@react-three/viverse` and `@react-three/timeline`.
## What you’ll build
- Third-person character with physics and camera behavior
- WASD movement, run, jump, mouse look (pointer lock)
- Aiming up/forward/down with upper-body blending
- Pistol idle/shoot/reload with sound and muzzle flash
- Camera FOV and zoom effects while running/aiming
- Map collisions using a BVH physics body
- Simple HUD with name, health bar, ammo, and crosshair
## Prereqs and install
Use pnpm and ensure these packages are installed. VRM support is optional but recommended for humanoids.
```bash
pnpm add three @react-three/fiber @react-three/drei @react-three/timeline @react-three/viverse zustand
```
Create an environment file for your Viverse app id (replace the value with your own):
```bash
# .env.local
VITE_VIVERSE_APP_ID=your_app_id_here
```
Notes:
- Keep your assets (e.g. `map.glb`, `avatar.vrm`, sounds/textures) under `public/`.
- Pointer lock: click the canvas to lock the mouse; press ESC to release.
## Step 1 — App shell (scene, provider, UI)
First, we start by copying all the assets, we'll need from this repository under `examples/fortnite/public` into your public folder. For animations, we are using the [Universal Animation Library - Pro](https://quaternius.com/packs/universalanimationlibrary.html) from Quaternius. If you want to use these animations for your project, please make sure to buy the Pro version at their website.
Next, we start adding a `