# Introduction ```bash npm install three @react-three/fiber @react-three/viverse ``` ### What does it look like? > A prototype map with the `` component and its default model Dependencies: ```js { 'three': 'latest', '@react-three/fiber': '<9', '@react-three/viverse': 'latest', '@react-three/drei': '<10' } ``` Files: File: /App.tsx ```tsx import { Sky } from '@react-three/drei' import { Canvas } from '@react-three/fiber' import { Viverse, SimpleCharacter, BvhPhysicsBody, PrototypeBox } from '@react-three/viverse' export default function App() { return ( ) } ``` ## How to get started > Some familiarity with > react, threejs, and @react-three/fiber, is recommended. Get started with **[building a simple game](#doc-tutorials-simple-game)**, take a look at our **[examples](#doc-getting-started-examples)**, or follow one of our **tutorials**: - [First person controls](#doc-tutorials-first-person) - [Augmented and virtual reality](#doc-tutorials-augmented-and-virtual-reality) - [Accessing avatar and profile](#doc-tutorials-access-avatar-and-profile) - [Equipping the character with items](#doc-tutorials-equipping-items) - [Using custom animations and models](#doc-tutorials-custom-models-and-animations) - [Actions](#doc-tutorials-actions) - [Custom Character Controller](#doc-tutorials-custom-character-controller) - [How to remove the viverse integrations](#doc-tutorials-remove-viverse-integrations) - [Publish to VIVERSE](#doc-tutorials-publish-to-viverse) - [Vibe coding with @react-three/viverse (using AI)](../tutorials/vibe-coding-with-ai.mdx) ## Not into react? > No Problem Check out how to build games using @pmndrs/viverse and only [vanilla three.js](#doc-without-react-introduction). ## Acknowledgments This project would not be possible without the default model and default animations made by [Quaternius](https://quaternius.com/), the prototype texture from [kenney.nl](https://www.kenney.nl/), and the [three-vrm project](https://github.com/pixiv/three-vrm) from the [pixiv team](https://github.com/pixiv)! # All Components and Hooks ## Components ### `` The main provider component that sets up VIVERSE authentication and physics context. Must wrap your entire application or the parts that use VIVERSE features. **Props:** - `children?: ReactNode` - Child components - `loginRequired?: boolean` - Forces user to login before playing (default: `false`) - `clientId?: string` - VIVERSE app client ID. Typically you pass this from your app’s environment (e.g. a `VITE_VIVERSE_APP_ID` env var you manage) into this prop. - `domain?: string` - Authentication domain (default: `'account.htcvive.com'`) - `authorizationParams?: object` - Additional authorization parameters - `cookieDomain?: string` - Cookie domain for authentication - `httpTimeoutInMS?: number` - HTTP request timeout in milliseconds > [!WARNING] > Don't set the `clientId` during local development! **Example:** ```tsx ``` ### `` Creates a simple character controller with physics based on three-mesh-bvh, walking, running, jumping animations, and camera controls. Automatically uses the active VIVERSE avatar if authenticated. **Props:** See [SimpleCharacter Options](#simplecharacter-options) section below for complete details. **Example:** ```tsx {/* Optional child components */} ``` ### `` Provides physics context for collision detection. Usually wrapped automatically by ``, but can be used standalone. **Props:** - `children?: ReactNode` - Child components ### `` Adds visible children as static (non-moving) or kinematic (moving) objects as obstacles to the physics world. > [!WARNING] > Content inside the object can not structurally change. **Props:** - `children?: ReactNode` - Static mesh objects for collision - `kinematic?: boolean` - whether the objects world transformation can change - default: false **Example:** ```tsx ``` ### `` Adds visible children as sensors that detect player intersection and trigger callbacks (does not add obstacles). > [!WARNING] > Content inside the object can not structurally change; Hiding the sensors content requires to wrap it in `...`. **Props:** - `children?: ReactNode` - Static mesh objects for collision - `isStatic?: boolean` - whether the objects world transformation is static - default: true - `onIntersectedChanged?: (intersected: boolean) => void` - callback that get's called when the player starts or stops intersecting with the sensor **Example:** ```tsx console.log('currently intersected', intersected)}> ``` ### `` A quick prototyping component that renders a textured box with the prototype material. **Props:** - `color?: ColorRepresentation` - Box color tint - All standard Three.js Group props (position, rotation, scale, etc.) **Example:** ```tsx ``` ### `` Provides the active character model context so that animation and bone utilities can target the same model instance. Wrap any content that uses ``, ``, ``, or ``. **Props:** - `model: CharacterModel` - Model returned by `useCharacterModelLoader` - `children?: ReactNode` - Child components **Example:** ```tsx const model = useCharacterModelLoader({ url: 'avatar.vrm', castShadow: true }) return ( ) ``` ### `` Defines a logical animation layer (e.g., "lower-body", "upper-body"). All nested animation actions inherit this layer unless they provide their own `layer` prop. Layers allow to manage animations when managing e.g. additive animations or animations with masks. **Props:** - `name: string` - Layer name - `children?: ReactNode` - Nested timeline/animation content **Example:** ```tsx {/* ...lights, environment... */} {/* masks can be created with @pmndrs/viverse animation masks */} ``` ### `` Loads and plays a clip on the active character model, integrating with `@react-three/timeline` for lifecycle and transitions. Supports masking, cross-fading, syncing, and layering. The `ref` exposes the underlying Three.js `AnimationAction`. **Props:** - Clip options (from `@pmndrs/viverse`): - `url: string | DefaultUrl` - Source of the animation - `type?: 'mixamo' | 'gltf' | 'vrma' | 'fbx' | 'bvh'` - `removeXZMovement?: boolean` - `trimTime?: { start?: number; end?: number }` - `boneMap?: Record` - `scaleTime?: number` - `mask?: CharacterAnimationMask` - Limit animation to specific bones/regions - Playback and blending: - `fadeDuration?: number` - Cross-fade/fade time (default: `0.1`) - `crossFade?: boolean` - Whether to cross-fade from current layer action (default: `true`) - `sync?: boolean` - Sync time with current action on same layer (if any) - `paused?: boolean` - `loop?: AnimationActionLoopStyles` - Defaults to `LoopRepeat` - `layer?: string` - Overrides the current `` - Timeline control (from `@react-three/timeline`): - `init?(): void | (() => void)` - Called when the action starts; return a cleanup - `update?(state, delta): void` - Per-frame update - `until?(): Promise` - Resolve to stop; defaults to when the clip finishes - `dependencies?: unknown[]` - Re-run when any dependency changes - Advanced: - `additiveReferenceClip?: AnimationClip` - Use an additive version of the clip relative to this reference clip (prefer `` for convenience) **Example:** ```tsx ``` ### `` Convenience wrapper around `` that plays an additive version of the clip, using a provided reference pose/clip (e.g., aim offsets layered over locomotion). **Props:** - All `` props, except it uses: - `referenceClip: CharacterAnimationOptions` - Clip used as the additive reference pose **Example:** ```tsx ``` ### `` Component for placing content inside the character model at specific bones. **Props:** - `bone: VRMHumanBoneName` - The bone name to access ```tsx ``` ## Hooks | Hook | Description | Returns | | ------------------------------------------------ | ------------------------------------------------------------- | ---------------------------------------------------------------- | | `useViverseClient()` | Returns the VIVERSE client instance for making API calls | `Client` | | `useViverseAuth()` | Returns the current authentication state | Auth object with access tokens, or `undefined` | | `useViverseAvatarClient()` | Returns the avatar client for avatar-related operations | `AvatarClient \| undefined` | | `useViverseLogin()` | Returns a function to initiate the VIVERSE login flow | Login function | | `useViverseLogout()` | Returns a function to initiate the VIVERSE logout flow | Logout function | | `useViverseProfile()` | Fetches the user's profile (name, avatar info) using Suspense | Profile object with `name`, `activeAvatar`, etc., or `undefined` | | `useViverseActiveAvatar()` | Fetches the user's currently selected avatar using Suspense | Avatar object with `vrmUrl`, `headIconUrl`, etc., or `undefined` | | `useViverseAvatarList()` | Fetches the user's personal avatar collection using Suspense | Array of avatar objects, or `undefined` | | `useViversePublicAvatarList()` | Fetches publicly available avatars using Suspense | Array of public avatar objects, or `undefined` | | `useViversePublicAvatarByID(id)` | Fetches a specific public avatar by ID using Suspense | Avatar object, or `undefined` | | `useIsMobile()` | Returns `true` on touch-centric/mobile devices (media query) | `boolean` | | `useCharacterModel()` | Gets the current character model from context | `CharacterModel` | | `useCharacterModelLoader(options?)` | Loads a character model with Suspense | `CharacterModel` | | `useCharacterAnimationLoader(model, options)` | Loads an animation clip for a model with Suspense | `AnimationClip` | | `useBvhPhysicsWorld()` | Accesses the BVH physics world context | `BvhPhysicsWorld` | | `useBvhCharacterPhysics(modelRef, options?)` | Character controller physics tied to a model ref | `BvhCharacterPhysics` | | `useCharacterCameraBehavior(modelRef, options?)` | Camera behavior that follows/rotates around model | `RefObject` | | `useSimpleCharacterActionBindings(...)?` | Deprecated: sets up default action bindings | `void` | | `useScreenButton(image)` | Create and mount a styled on-screen button element | `HTMLElement` | > [!NOTE] > `useViverseClient()` returns `undefined` if not within a `` provider or if no `clientId` is provided. Also all avatar-related hooks return `undefined` when the user is not authenticated. ### useIsMobile Lightweight media-query based mobile detection. It subscribes to `@media (hover: none) and (pointer: coarse)`. ```tsx import { useIsMobile } from '@react-three/viverse' function MobileOnlyUI() { const isMobile = useIsMobile() return isMobile ?
Shown on mobile
: null } ``` ### Action Binding Hooks Actions allow to decouple specific user inputs from game/business logic. One action can be connected to multiple inputs via Action bindings (keyboard, mouse/touch, on-screen controls). For background on actions vs. action bindings and how to create custom ones, see the Actions tutorial: [Create and use actions](#doc-tutorials-actions). We provide several easy to use hooks to setup default action bindings for general use cases such as pressing a button or specific use case such as locomotion using a keyboard. #### `useKeyboardActionBinding(action, options)` - **Description:** Binds a `KeyboardEvent` or boolean state action to one or more keys. - **Options:** `{ keys: string[]; requiresPointerLock?: boolean }` - **Returns:** `void` ```tsx useKeyboardActionBinding(jumpAction, { keys: ['Space'] }) ``` #### `usePointerButtonActionBinding(action, options)` - **Description:** Binds a pointer button (mouse/touch) event or state action. - **Options:** `{ domElement?: HTMLElement | RefObject; buttons?: number[]; requiresPointerLock?: boolean }` - **Returns:** `void` ```tsx usePointerButtonActionBinding(fireAction, { buttons: [0] }) // left mouse / primary touch ``` #### `usePointerCaptureRotateZoomActionBindings(options)` - **Description:** Enables rotate/zoom camera controls using Pointer Capture on the canvas. - **Options:** `{ rotationSpeed?: number; zoomSpeed?: number }` - **Returns:** `void` ```tsx usePointerCaptureRotateZoomActionBindings({ rotationSpeed: 1000, zoomSpeed: 1000 }) ``` #### `usePointerLockRotateZoomActionBindings(options)` - **Description:** Enables rotate/zoom camera controls using Pointer Lock on the canvas. - **Options:** `{ rotationSpeed?: number; zoomSpeed?: number; lockOnClick?: boolean }` - **Returns:** `void` ```tsx usePointerLockRotateZoomActionBindings({ lockOnClick: true }) ``` #### `useKeyboardLocomotionActionBindings(options)` - **Description:** WASD movement, Shift to run, Space to jump. - **Options:** `{ moveForwardKeys?, moveBackwardKeys?, moveLeftKeys?, moveRightKeys?, runKeys?, jumpKeys?, requiresPointerLock? }` (arrays of key strings) - **Returns:** `void` ```tsx useKeyboardLocomotionActionBindings({ requiresPointerLock: false }) ``` #### `useScreenJoystickLocomotionActionBindings(options)` - **Description:** On-screen joystick for movement and run on mobile devices. - **Options:** `{ runDistancePx?: number; deadZonePx?: number }` - **Returns:** `void` ```tsx useScreenJoystickLocomotionActionBindings({ deadZonePx: 8, runDistancePx: 40 }) ``` ## SimpleCharacter Options The `SimpleCharacter` component can be configured with a variety of props but also supports all the default group props, such as position, rotation, and scale. ### `useViverseAvatar` flag Allows to configure whether the users vrm avatar should be displayed as the character model. - **Default:** `true` ### `movement` Options - **walk:** `object | boolean` - Enable walking (default: `true`) - **speed:** Movement speed in units per second (default: `3`) - Set to `false` to disable walking - **run:** `object | boolean` - Enable running (default: `true`) - **speed:** Running speed in units per second (default: `6`) - Set to `false` to disable running - **jump:** `object | boolean` - Enable jumping (default: `true`) - **delay:** Time before jump starts in seconds (default: `0.2`) - **bufferTime:** Jump input buffer time in seconds (default: `0.1`) - **speed:** Jump velocity in units per second (default: `8`) - Set to `false` to disable jumping ### `actionBindings` Options An array of action binding classes to instantiate for handling controls - **Default:** `[ScreenJoystickLocomotionActionBindings, ScreenButtonJumpActionBindings, PointerCaptureRotateZoomActionBindings, KeyboardLocomotionActionBindings]` - Configure action bindings with custom action binding classes **Available Action Binding Classes provided by @pmndrs/viverse:** - `KeyboardLocomotionActionBindings` - WASD movement, Space for jump, Shift for run - `PointerCaptureRotateZoomActionBindings` - Mouse look with pointer capture (requires manual `setPointerCapture`) - `PointerLockRotateZoomActionBindings` - Mouse look with pointer lock (requires manual `requestPointerLock`) - `ScreenJoystickLocomotionActionBindings` - On-screen joystick for movement and run (mobile). Options: `{ screenJoystickDeadZonePx?, screenJoystickRunDistancePx? }` - `ScreenButtonJumpActionBindings` - On-screen jump button (mobile-only). Visible only on mobile. ### `model` Options - **url:** `string` - URL to VRM or GLTF model file - **type:** `"gltf" | "vrm"` - the type of file to be loaded (optional) - **castShadow:** `boolean` - Enable shadow casting (default: `true`) - **receiveShadow:** `boolean` - Enable shadow receiving (default: `true`) - **boneRotationOffset:** `Quaternion | undefined` - Allows to apply an rotation offset when placing objects as children of the character's bones (default: `undefined`) - Set to `false` to disable model loading - Set to `true` or omit to use default robot model ### `physics` Options - **capsuleRadius:** `number` - Character collision capsule radius (default: `0.4`) - **capsuleHeight:** `number` - Character collision capsule height (default: `1.7`) - **gravity:** `number` - Gravity acceleration in m/s² (default: `-20`) - **linearDamping:** `number` - Air resistance coefficient (default: `0.1`) - **maxGroundSlope:** `number` - Max slope for a collider to be detected as walkable (default: `0.5`) ### `cameraBehavior` Options - **collision:** `object | boolean` - Enable camera collision (default: `true`) - **offset:** `number` - Collision offset distance (default: `0.2`) - **characterBaseOffset:** `Vector3 | [number, number, number]` - Camera position relative to character (default: `[0, 1.3, 0]`) - **rotation:** `object | boolean` - Enable camera rotation (default: `true`) - **minPitch:** `number` - Minimum pitch angle (default: `-Math.PI/2`) - **maxPitch:** `number` - Maximum pitch angle (default: `Math.PI/2`) - **minYaw:** `number` - Minimum yaw angle (default: `-Infinity`) - **maxYaw:** `number` - Maximum yaw angle (default: `+Infinity`) - **speed:** `number` - Rotation speed multiplier (default: `1000`) - **zoom:** `object | boolean` - Enable camera zoom (default: `true`) - **speed:** `number` - Zoom speed multiplier (default: `1000`) - **minDistance:** `number` - Minimum camera distance (default: `1`) - **maxDistance:** `number` - Maximum camera distance (default: `7`) ### `animation` Options - **yawRotationBasedOn:** `'camera' | 'movement'` - Character rotation basis (default: `'movement'`) - **maxYawRotationSpeed:** `number` - Maximum rotation speed (default: `10`) - **crossFadeDuration:** `number` - Animation blend time in seconds (default: `0.1`) The `SimpleCharacter` uses the following animations `walk`, `run`, `idle`, `jumpForward`, `jumpUp`, `jumpLoop`, `jumpDown` each with the following options: - **url:** `string` - Animation file URL - **type:** `'gltf' | 'vrma' | 'fbx' | 'bvh'` - Animation file type (optional) - **boneMap** - Allows to map the bone names of the animation amature to the standard VRM bone names - **removeXZMovement:** `boolean` - Remove horizontal movement from animation - **trimTime:** `{ start?: number; end?: number }` - Trim animation timing - **scaleTime:** `number` - Scale animation playback speed ## PrototypeMaterial The `` component provides a textured material for prototyping using kenney.nl's prototype texture. - **color:** `ColorRepresentation` - Material color tint - **repeat:** `Vector2` - Texture repeat pattern (accessible as `materialRef.current.repeat`) - All standard Three.js MeshPhongMaterial properties ```tsx // As JSX element ``` # Examples
  • [![Screenshot from the react example](./react-example.png)](https://worlds.viverse.com/wyTQbnB) Simple Game Example w. a Player Tag
  • [![Screenshot from the vanilla example](./vanilla-example.png)](https://worlds.viverse.com/kjALCp2) Simple Game Example using Vanilla Threejs
  • [![Screenshot from the AR example](./ar-example.gif)](https://worlds.viverse.com/UB6VBmX) Augemented Reality Example using WebXR
  • [![Screenshot from the VR example](./vr-example.gif)](https://worlds.viverse.com/asuA4ay) Virtual Reality Example using WebXR
  • [![Screenshot from the Fortnite example](./fortnite-example.gif)](https://worlds.viverse.com/ziWwWno) Fortnite Character Controller Example
  • # Building a Simple Game In this tutorial, we'll build the following simple 3D platformer game using `@react-three/viverse` with: - Character movement (WASD + mouse look) - Jumping mechanics - Physics-based collision detection - A simple level with platforms to jump on - Respawn system when falling off the map *Here's a preview of what we will build in this tutorial:* Dependencies: ```js { 'three': 'latest', '@react-three/fiber': '<9', '@react-three/viverse': 'latest', '@react-three/drei': '<10' } ``` Files: File: /Scene.tsx ```tsx import { Sky } from '@react-three/drei' import { SimpleCharacter, BvhPhysicsBody, PrototypeBox } from '@react-three/viverse' import { useRef } from 'react' import { useFrame } from '@react-three/fiber' import { Group } from 'three' export function Scene() { const characterRef = useRef(null) // Respawn logic - reset character position if they fall off the map useFrame(() => { if (characterRef.current == null) { return } if (characterRef.current.position.y < -10) { characterRef.current.position.set(0, 0, 0) } }) return ( <> {/* Environment */} {/* Lighting */} {/* Character */} {/* Level Geometry */} {/* Main ground */} {/* Platforms */} ) } ``` File: /App.tsx ```tsx import { Suspense } from "react" import { Canvas } from '@react-three/fiber' import { Viverse } from '@react-three/viverse' import { Scene } from "./Scene" export default function App() { return ( ) } ``` ## Step 0: Prerequisites Make sure you have the required dependencies installed: ```bash npm install three @react-three/fiber @react-three/viverse @react-three/drei ``` ## Step 1: Setting Up the Canvas First, let's create the basic Canvas setup with shadows and proper camera settings: ```tsx import { Canvas } from '@react-three/fiber' import { Viverse } from '@react-three/viverse' import { Suspense } from 'react' export function App() { return ( ) } ``` ## Step 2: Adding the Scene and creating the Sky Let's create another component called `Scene` and add the sky and basic lighting. Add the `Sky` import from `@react-three/drei`: ```tsx import { Sky } from '@react-three/drei' export function Scene() { return ( <> {/* Environment */} {/* Basic lighting */} ) } ``` At this point, you should see a beautiful sky gradient in your scene! ## Step 3: Building the Level Now add the level geometry. Import `BvhPhysicsBody` and `PrototypeBox` from `@react-three/viverse`, and expand the directional light with shadow properties: ```tsx import { Sky } from '@react-three/drei' import { BvhPhysicsBody, PrototypeBox } from '@react-three/viverse' export function Scene() { return ( <> {/* Environment */} {/* Lighting - expanded with shadow settings */} {/* Platforms */} ) } ``` Now you should see a colorful platformer level with various platforms at different heights! ## Step 4: Adding the Character Next we will add the character. Import `SimpleCharacter` from `@react-three/viverse`: ```tsx import { Sky } from '@react-three/drei' import { SimpleCharacter, BvhPhysicsBody, PrototypeBox } from '@react-three/viverse' import { useRef } from 'react' export function Scene() { return ( <> {/* Environment */} {/* Lighting */} {/* Level Geometry */} {/* ... platforms remain the same ... */} ) } ``` Great! Now you can move around with WASD keys, look around with the mouse, and jump with the spacebar. Try jumping between the platforms! ## Step 5: Adding Respawn Logic Finally, we will add the respawn system. Import `useRef` from `react` and `useFrame` from `@react-three/fiber` and add the respawn logic: ```tsx import { Sky } from '@react-three/drei' import { SimpleCharacter, BvhPhysicsBody, PrototypeBox } from '@react-three/viverse' import { useRef } from 'react' import { Group } from 'three' import { useFrame } from '@react-three/fiber' // NEW export function Scene() { const characterRef = useRef(null) // Respawn logic - NEW useFrame(() => { if (characterRef.current == null) { return } if (characterRef.current.position.y < -10) { characterRef.current.position.set(0, 0, 0) } }) return ( <> {/* ... rest remains the same ... */} ) } ``` Perfect! Now if you fall off the map (below y = -10), you'll automatically respawn at the starting position (0, 0, 0). # First Person Controls In this tutorial we will configure the `SimpleCharacter` to use first person controls with the following result: _Here's a preview of this tutorial's result:_ Dependencies: ```js { 'three': 'latest', '@react-three/fiber': '<9', '@react-three/viverse': 'latest', '@react-three/drei': '<10' } ``` Files: File: /App.tsx ```tsx import { Sky } from '@react-three/drei' import { Canvas } from '@react-three/fiber' import { Viverse, SimpleCharacter, BvhPhysicsBody, PrototypeBox, FirstPersonCharacterCameraBehavior, PointerLockRotateZoomActionBindings, KeyboardLocomotionActionBindings, } from '@react-three/viverse' export default function App() { return ( ) } ``` First, we switch from third-person to first-person camera behavior and hide the character model to prevent the model from occluding the players view. ```tsx ``` **Changes:** - `model={false}` - Hides the character model since in first-person view, you don't want to see your own character - `cameraBehavior={FirstPersonCharacterCameraBehavior}` - Switches from the default third-person camera to first-person camera behavior Next, you need to set up the appropriate action bindings for first-person movement and looking around: ```tsx ``` - `KeyboardLocomotionActionBindings` - Handles WASD movement action bindings for walking around - `PointerLockRotateZoomActionBindings` - Enables mouse look action bindings for rotating the camera/view direction # Augmented and Virtual Reality This tutorial shows how to add Augmented Reality (AR) and Virtual Reality (VR) support to your `@react-three/viverse` games. We'll start with AR and then show the additional changes needed for VR. ## Prerequisites Make sure you have the `@react-three/xr` package installed: ```bash npm install @react-three/xr ``` ## Augmented Reality (AR) Let's start by adding AR support to a basic `@react-three/viverse` game. *Here's the AR app we'll build now:* Dependencies: ```js { 'three': 'latest', '@react-three/fiber': '<9', '@react-three/viverse': 'latest', '@react-three/drei': '<10', '@react-three/xr': 'latest' } ``` Files: File: /App.tsx ```tsx import { Canvas } from '@react-three/fiber' import { Viverse } from '@react-three/viverse' import { XR, XROrigin, createXRStore } from '@react-three/xr' import { Scene } from './Scene' const store = createXRStore({ offerSession: 'immersive-ar' }) export default function App() { return ( ) } ``` File: /Scene.tsx ```tsx import { useFrame } from '@react-three/fiber' import { SimpleCharacter, BvhPhysicsBody, PrototypeBox, useXRControllerLocomotionActionBindings, } from '@react-three/viverse' import { useRef } from 'react' import { Group } from 'three' export function Scene() { const characterRef = useRef(null) useFrame(() => { if (characterRef.current == null) { return } if (characterRef.current.position.y < -10) { characterRef.current.position.set(0, 0, 0) } }) useXRControllerLocomotionActionBindings() return ( <> ) } ``` ### Step 1: Set Up the XR Store Create an XR store configured for AR. Add these imports and create the store: ```tsx import { XR, XROrigin, createXRStore } from '@react-three/xr' const store = createXRStore({ offerSession: 'immersive-ar' }) ``` The `offerSession: 'immersive-ar'` option tells the XR system that we want to create an AR experience. ### Step 2: Wrap Your Scene with XR Components Wrap your scene content with the `XR` component and add an `XROrigin`: ```tsx export function App() { return ( Loading...}> ) } ``` **Key points:** - `XROrigin` defines the coordinate system origin for AR tracking - `scale={10}` makes the scene 10x larger relative to the real world - `position-y={-8} position-z={10}` adjusts the initial positioning ### Step 3: Remove Sky Component For AR, you don't want a sky background since the camera feed should show through. Remove any `Sky` components from your scene: ```tsx export function Scene() { return ( <> {/* Remove for AR */} {/* ... rest of your scene */} ) } ``` ### Step 4: Use XR Controller Action Bindings Add XR controller action bindings using the `useXRControllerLocomotionActionBindings` hook: ```tsx import { useXRControllerLocomotionActionBindings } from '@react-three/viverse' export function Scene() { useXRControllerLocomotionActionBindings() return ( <> {/* ... rest of scene */} ) } ``` **Key changes:** - `cameraBehavior={false}` - Disables automatic camera control (AR handles this) - `useXRControllerLocomotionActionBindings()` - Hook that provides controller action bindings for movement The `useXRControllerLocomotionActionBindings` hook binds XR controller inputs to character locomotion actions: - **Left thumbstick** controls movement (forward/backward/left/right) - **Right controller A button** triggers jumping - **Left trigger** enables running ## Virtual Reality (VR) Now let's look at how to add VR support to a game building on the knowledge from adding AR support. *Here's the VR app we'll build now:* Dependencies: ```js { 'three': 'latest', '@react-three/fiber': '<9', '@react-three/viverse': 'latest', '@react-three/drei': '<10', '@react-three/xr': 'latest' } ``` Files: File: /App.tsx ```tsx import { Canvas } from '@react-three/fiber' import { Viverse } from '@react-three/viverse' import { XR, createXRStore } from '@react-three/xr' import { Scene } from './Scene' const store = createXRStore({ offerSession: 'immersive-vr' }) export default function App() { return ( ) } ``` File: /Scene.tsx ```tsx import { useFrame } from '@react-three/fiber' import { SimpleCharacter, BvhPhysicsBody, PrototypeBox, useXRControllerLocomotionActionBindings, } from '@react-three/viverse' import { Sky } from '@react-three/drei' import { XROrigin, useXRInputSourceState } from '@react-three/xr' import { useRef } from 'react' import { Group } from 'three' export function Scene() { const characterRef = useRef(null) useFrame(() => { if (characterRef.current == null) { return } if (characterRef.current.position.y < -10) { characterRef.current.position.set(0, 0, 0) } }) useXRControllerLocomotionActionBindings() return ( <> ) } function SnapRotateXROrigin() { const ref = useRef(null) const rightController = useXRInputSourceState('controller', 'right') const prev = useRef(0) useFrame(() => { if (ref.current == null) return const current = Math.round(rightController?.gamepad?.['xr-standard-thumbstick']?.xAxis ?? 0) if (current < 0 && prev.current >= 0) { // Rotate left ref.current.rotation.y += Math.PI / 2 } if (current > 0 && prev.current <= 0) { // Rotate right ref.current.rotation.y -= Math.PI / 2 } prev.current = current }) return } ``` ### Step 1: Change Session Type to VR We can update the offer session to show the user a native "VR" enter button. ```tsx const store = createXRStore({ offerSession: 'immersive-vr', }) ``` ### Step 2: Re-add the Sky for VR Unlike AR, VR needs a sky background since there's no camera feed: ```tsx import { Sky } from '@react-three/drei' export function Scene() { return ( <> {/* ... rest of scene */} ) } ``` ### Step 3: Hide the Character Model In VR, you typically don't want to see your own character model: ```tsx ``` **Key change:** - `model={false}` - Hides the character model in VR ### Step 4: Place the XROrigin into the Simple Character and Optionally Add Snap Rotation As the XROrigin defines the player's position, we need to remove it from outside the Scene and add it into the SimpleCharacter. ```tsx import { useXRInputSourceState } from '@react-three/xr' ``` For comfort in VR, you can add snap rotation using the right thumbstick by building a SnapRotateXROrigin which replaces the XROrigin component. ```tsx import { useXRInputSourceState } from '@react-three/xr' function SnapRotateXROrigin() { const ref = useRef(null) const rightController = useXRInputSourceState('controller', 'right') const prev = useRef(0) useFrame(() => { if (ref.current == null) return const current = Math.round(rightController?.gamepad?.['xr-standard-thumbstick']?.xAxis ?? 0) if (current < 0 && prev.current >= 0) { // Rotate left ref.current.rotation.y += Math.PI / 2 } if (current > 0 && prev.current <= 0) { // Rotate right ref.current.rotation.y -= Math.PI / 2 } prev.current = current }) return } ``` ## Summary **For AR:** 1. Use `offerSession: 'immersive-ar'` 2. Remove `Sky` component 3. Use `useXRControllerLocomotionActionBindings()` for to bind the locomotion actions 4. Set `cameraBehavior={false}` on SimpleCharacter 5. Add `XROrigin` with appropriate scaling/positioning **Additional changes for VR:** 1. Change to `offerSession: 'immersive-vr'` 2. Re-add `Sky` component 3. Set `model={false}` to hide character 4. Place the XROrigin into the SimpleCharacter and optionally add snap rotation for comfort # Accessing Avatar and Profile This tutorial shows you how to display a player tag above the character by accessing the user profile information from VIVERSE. _Here's a preview of what we'll build in this tutorial:_ Dependencies: ```js { 'three': 'latest', '@react-three/fiber': '<9', '@react-three/viverse': 'latest', '@react-three/drei': '<10', "@react-three/uikit": "^1.0.41" } ``` Files: File: /Playertag.tsx ```tsx import { useViverseProfile } from '@react-three/viverse' import { Container, Image, Text } from '@react-three/uikit' import { useRef } from 'react' import { useFrame } from '@react-three/fiber' import { Group } from 'three' export function PlayerTag() { const profile = useViverseProfile() ?? { name: 'Anonymous', activeAvatar: { headIconUrl: 'https://picsum.photos/200' }, } const ref = useRef(null) // Make the tag always face the camera useFrame((state) => { if (ref.current == null) { return } ref.current.quaternion.copy(state.camera.quaternion) }) return ( {profile.name} ) } ``` File: /App.tsx ```tsx import {Sky} from '@react-three/drei' import {Canvas} from '@react-three/fiber' import {(Viverse, SimpleCharacter, BvhPhysicsBody, PrototypeBox)} from '@react-three/viverse' import {PlayerTag} from "./Playertag" export default function App() { return ( ) } ``` First, we use the `useViverseProfile()` hook to fetch the current user's profile from VIVERSE, including their name and avatar information. We provide a fallback for when the user isn't logged in: ```tsx const profile = useViverseProfile() ?? { name: 'Anonymous', activeAvatar: { headIconUrl: 'https://picsum.photos/200' }, } ``` Next, we need a 3D ui library, install it via ```bash npm install @react-three/uikit ``` UIKit provides HTML-like components (`Container`, `Image`, `Text`) that work in 3D space. We create a card-like layout with flexbox: ```tsx {profile.name} ``` Next, we use `useFrame` to constantly update the tag's rotation to match the camera: ```tsx import { Group } from 'three' const ref = useRef(null) useFrame((state) => { if (ref.current == null) { return } ref.current.quaternion.copy(state.camera.quaternion) }) ``` The full `PlayerTag` component looks like this: ```tsx import { useViverseProfile } from '@react-three/viverse' import { Container, Image, Text } from '@react-three/uikit' import { useRef } from 'react' import { useFrame } from '@react-three/fiber' import { Group } from 'three' export function PlayerTag() { const profile = useViverseProfile() ?? { name: 'Anonymous', activeAvatar: { headIconUrl: 'https://picsum.photos/200' }, } const ref = useRef(null) // Make the tag always face the camera useFrame((state) => { if (ref.current == null) { return } ref.current.quaternion.copy(state.camera.quaternion) }) return ( {profile.name} ) } ``` Now finally lets add the PlayerTag as a child of SimpleCharacter to display it ```tsx ``` # Equipping the Character With Items This tutorial shows you how to equip your character with items by attaching 3D objects to specific bones. We'll create a simple sword using just two meshes and attach it to the character's right hand. _Here's a preview of what we'll build in this tutorial:_ Dependencies: ```js { 'three': 'latest', '@react-three/fiber': '<9', '@react-three/viverse': 'latest', '@react-three/drei': '<10' } ``` Files: File: /App.tsx ```tsx import { Sky } from '@react-three/drei' import { Canvas } from '@react-three/fiber' import { Viverse, SimpleCharacter, BvhPhysicsBody, PrototypeBox, CharacterModelBone } from '@react-three/viverse' export default function App() { return ( {/* Blade */} {/* Handle */} ) } ``` ## Understanding Bone Attachment The `CharacterModelBone` component allows you to attach any 3D object to specific bones in the character's skeleton. This is perfect for equipping weapons, accessories, or any items that should move with the character. ### Step 1: Import the CharacterModelBone Component First, import the `CharacterModelBone` component from `@react-three/viverse`: ```tsx import { CharacterModelBone } from '@react-three/viverse' ``` ### Step 2: Add a Simple Sword to the `"rightHand"` Next, we place the CharacterModelBone inside the `SimpleCharacter` component and attach it to the `"rightHand"`. We then build a simple sword using two meshes. For better looks, you probably want to import your own 3D model. ```tsx {/* Blade */} {/* Handle */} ``` # Custom Models and Animations ## Using Custom Character Models By default, the `SimpleCharacter` component uses a built-in robot avatar. You can easily replace this with your own 3D model by providing a URL to the model file in any of the following formats: - **VRM** - The standardized VRM format for avatars requires no additional configuration - **GLTF** - (or also glb) Standard 3D Model format - make sure to use the standard vrm bone names as shown below
    VRM 1.0 humanoid bone names (click to expand) The following are the standard VRM 1.0 humanoid bone names your GLTF rig should use (aligned with the VRM specification). If your model uses these names, animations and retargeting will work reliably: | Bone name | Description | | ------------------------- | ---------------------------------------------- | | `hips` | Pelvis root; parent of the spine and both legs | | `spine` | Lower/waist spine segment above hips | | `chest` | Mid/upper torso segment above spine | | `upperChest` | Optional highest chest segment below neck | | `neck` | Neck base; parent of head | | `head` | Head root; parent of eyes and jaw | | `leftEye` | Left eyeball transform | | `rightEye` | Right eyeball transform | | `jaw` | Jaw/mandible pivot | | `leftUpperLeg` | Left thigh (upper leg) | | `leftLowerLeg` | Left shin (lower leg) | | `leftFoot` | Left foot root/ankle | | `leftToes` | Left toe base | | `rightUpperLeg` | Right thigh (upper leg) | | `rightLowerLeg` | Right shin (lower leg) | | `rightFoot` | Right foot root/ankle | | `rightToes` | Right toe base | | `leftShoulder` | Left clavicle/shoulder pivot | | `leftUpperArm` | Left upper arm (humerus) | | `leftLowerArm` | Left forearm | | `leftHand` | Left hand/wrist root | | `rightShoulder` | Right clavicle/shoulder pivot | | `rightUpperArm` | Right upper arm (humerus) | | `rightLowerArm` | Right forearm | | `rightHand` | Right hand/wrist root | | `leftThumbMetacarpal` | Left thumb metacarpal (root of thumb) | | `leftThumbProximal` | Left thumb proximal phalanx | | `leftThumbDistal` | Left thumb distal phalanx | | `leftIndexProximal` | Left index proximal phalanx | | `leftIndexIntermediate` | Left index intermediate phalanx | | `leftIndexDistal` | Left index distal phalanx | | `leftMiddleProximal` | Left middle proximal phalanx | | `leftMiddleIntermediate` | Left middle intermediate phalanx | | `leftMiddleDistal` | Left middle distal phalanx | | `leftRingProximal` | Left ring proximal phalanx | | `leftRingIntermediate` | Left ring intermediate phalanx | | `leftRingDistal` | Left ring distal phalanx | | `leftLittleProximal` | Left little/pinky proximal phalanx | | `leftLittleIntermediate` | Left little/pinky intermediate phalanx | | `leftLittleDistal` | Left little/pinky distal phalanx | | `rightThumbMetacarpal` | Right thumb metacarpal (root of thumb) | | `rightThumbProximal` | Right thumb proximal phalanx | | `rightThumbDistal` | Right thumb distal phalanx | | `rightIndexProximal` | Right index proximal phalanx | | `rightIndexIntermediate` | Right index intermediate phalanx | | `rightIndexDistal` | Right index distal phalanx | | `rightMiddleProximal` | Right middle proximal phalanx | | `rightMiddleIntermediate` | Right middle intermediate phalanx | | `rightMiddleDistal` | Right middle distal phalanx | | `rightRingProximal` | Right ring proximal phalanx | | `rightRingIntermediate` | Right ring intermediate phalanx | | `rightRingDistal` | Right ring distal phalanx | | `rightLittleProximal` | Right little/pinky proximal phalanx | | `rightLittleIntermediate` | Right little/pinky intermediate phalanx | | `rightLittleDistal` | Right little/pinky distal phalanx |
    ```tsx import { SimpleCharacter } from '@react-three/viverse' export function MyCharacter() { return } ``` ## Adding Custom Animations Your can replace the default animations of the `SimpleCharacter` component with files in any of these three supported animation formats: - **VRMA** (VRM Animation) - The native VRM animation format - **FBX** - Popular file format for character animations - **GLTF** - Standard 3D format with animations - **Mixamo** - **deprecated** - use remove `type: 'mixamo'` and add `boneMap: mixamoBoneMap` instead Make sure to either use a bone map, e.g. when your bone names follow the mixamo naming conventions use the `mixamoBoneMap` or use the standard VRM bones for the animation amature as shown above under "VRM 1.0 humanoid bone names". Each animation type can be configured individually: ```tsx ``` You can customize any of these animation slots: - `walk` - Walking animation - `run` - Running animation - `idle` - Standing idle animation - `jumpStart` - Beginning of jump - `jumpUp` - Ascending during jump - `jumpLoop` - Mid-air loop animation - `jumpDown` - Landing animation # Actions and Action Bindings Actions allow to decouple specific user inputs from game/business logic. Inputs (keyboard, mouse/touch, controllers, on‑screen UI) are converted by action bindings into actions that game systems consume on every frame or whenever an event happens to act upon the action. - **Input → Action Binding → Action → Effect** - Input: hardware or UI event (key, mouse move, touch, thumbstick, button) - Action Binding: translates that input into a domain signal - Action: a shared signal (event or state) consumed by systems - Effect: the game changes on frame (e.g., camera rotates, character moves, jumps) ## State vs. Event actions - **StateAction<T>** - Represents a continuous state that persists until changed (e.g., movement axes, “is running”). - Merges multiple writers (e.g., keyboard + joystick) into one value each frame. - Read anywhere via `.get()`. - **EventAction<T>** - Represents instantaneous events (e.g., “jump pressed”, “rotate delta”, “zoom delta”). - Produces values per frame via a reader; values are combined (sum, etc.) before consumption. For example, the CharacterCameraBehavior consumes rotation and zoom event actions, which are StateActions, every frame. The StateAction returns the final value as an accumulation of all the inputs since the last frame. ## Built-in actions: - Movement: `MoveForwardAction`, `MoveBackwardAction`, `MoveLeftAction`, `MoveRightAction`, `RunAction` (State) - Jumping: `JumpAction` (Event) - Camera: `RotateYawAction`, `RotatePitchAction`, `ZoomAction` (Event) ## Built-in action bindings - Keyboard locomotion: `useKeyboardLocomotionActionBindings(...)` - Mouse/touch camera (pointer capture): `usePointerCaptureRotateZoomActionBindings(...)` - Mouse camera (pointer lock): `usePointerLockRotateZoomActionBindings(...)` - Single key/button bindings: `useKeyboardActionBinding(...)`, `usePointerButtonActionBinding(...)` - Mobile UI: `useScreenJoystickLocomotionActionBindings(...)`, `useScreenButton(...)` These hooks connect hardware inputs to actions. Multiple bindings can feed the same action; values are safely merged. ## Example: Rotate the camera with the mouse (Pointer Lock) *This shows the full pipeline: mouse movement → pointer-lock binding → rotation actions → camera rotates on frame.* Dependencies: ```js { 'three': 'latest', '@react-three/fiber': '<9', '@react-three/viverse': 'latest', '@react-three/drei': '<10' } ``` Files: File: /App.tsx ```tsx import { Canvas } from '@react-three/fiber' import { Sky } from '@react-three/drei' import { Viverse, SimpleCharacter, BvhPhysicsBody, PrototypeBox, usePointerLockRotateZoomActionBindings, } from '@react-three/viverse' function Bindings() { // Binds mouse movement (while pointer is locked) to RotateYawAction/RotatePitchAction, // and mouse wheel to ZoomAction. The camera behavior consumes these every frame. usePointerLockRotateZoomActionBindings({ lockOnClick: true }) return null } export default function App() { return ( {/* SimpleCharacter includes a default camera behavior that reads rotation + zoom actions */} ) } ``` Click the canvas once to lock the pointer. Move the mouse to rotate the camera; use the mouse wheel to zoom. That’s the actions pipeline in action. ## Example: Map Q/E keys to camera yaw rotation You can also route keyboard inputs into camera rotation by mapping `KeyboardEvent` to yaw deltas and binding them to the same `RotateYawAction`. ```tsx import { useKeyboardActionBinding } from '@react-three/viverse' import { RotateYawAction } from '@react-three/viverse' function KeyboardYawBindings() { // Map KeyboardEvent → number (negative = left, positive = right) const rotateFromKeyboard = RotateYawAction.mapFrom((e: KeyboardEvent) => { if (e.code === 'KeyQ') return -0.02 if (e.code === 'KeyE') return 0.02 return 0 }) // Bind Q/E key presses to the mapped action useKeyboardActionBinding(rotateFromKeyboard, { keys: ['KeyQ', 'KeyE'] }) return null } ``` Place `` alongside your other bindings. Now both mouse and keys contribute to `RotateYawAction`; their values are combined before the camera consumes them each frame. ## Example: Keyboard locomotion and jump To bind classic WASD + Shift + Space to movement and jump, use the built-in locomotion bindings: ```tsx import { useKeyboardLocomotionActionBindings } from '@react-three/viverse' function LocomotionBindings() { useKeyboardLocomotionActionBindings({ requiresPointerLock: false, // set to true if you want movement only when pointer is locked }) return null } ``` These bindings write to: - `MoveForwardAction`, `MoveBackwardAction`, `MoveLeftAction`, `MoveRightAction` (State: 0..1) - `RunAction` (State: boolean) - `JumpAction` (Event) Your character controller (e.g., `SimpleCharacter`) reads these to move and animate on frame, regardless of the input device that produced them. ## Takeaways - **Decouple input from gameplay**: actions provide a stable interface your systems consume. - **Compose inputs**: multiple bindings can feed the same action; values merge predictably. - **Think “signals,” not devices**: code against actions like “move forward” or “yaw delta,” not specific keys or hardware. # Custom Character Controller In this tutorial, you’ll build a custom extensible humanoid character controller in a fortnite style with support for aim, shoot, reload, run, and jump, using `@react-three/viverse` and `@react-three/timeline`. ## What you’ll build - Third-person character with physics and camera behavior - WASD movement, run, jump, mouse look (pointer lock) - Aiming up/forward/down with upper-body blending - Pistol idle/shoot/reload with sound and muzzle flash - Camera FOV and zoom effects while running/aiming - Map collisions using a BVH physics body - Simple HUD with name, health bar, ammo, and crosshair ## Prereqs and install Use pnpm and ensure these packages are installed. VRM support is optional but recommended for humanoids. ```bash pnpm add three @react-three/fiber @react-three/drei @react-three/timeline @react-three/viverse zustand ``` Create an environment file for your Viverse app id (replace the value with your own): ```bash # .env.local VITE_VIVERSE_APP_ID=your_app_id_here ``` Notes: - Keep your assets (e.g. `map.glb`, `avatar.vrm`, sounds/textures) under `public/`. - Pointer lock: click the canvas to lock the mouse; press ESC to release. ## Step 1 — App shell (scene, provider, UI) First, we start by copying all the assets, we'll need from this repository under `examples/fortnite/public` into your public folder. For animations, we are using the [Universal Animation Library - Pro](https://quaternius.com/packs/universalanimationlibrary.html) from Quaternius. If you want to use these animations for your project, please make sure to buy the Pro version at their website. Next, we start adding a `` and `` component, a environment that includes a sky, clouds, fog, lights, a HUD overlay, the character, and the map. In the next steps, we will create the `Map`, `Character`, and `HUD` components. We use `` up front so profile features, shared actions, and integrations are available everywhere; you could delay it, but then you’d pass more props around later. Key ideas: - **Provider**: `` enables account/profile APIs and shared actions/state. - **Suspense Fallback**: Show a simple “Loading...” UI. - **Environment**: `Sky`, `Clouds`, `directionalLight` with shadows. - **Composition**: `` and `` live inside the canvas. Add the following snippet into your `src/app.tsx`: ```tsx return ( ) ``` ## Step 2 — Map and collisions Create `src/map.tsx` and add the full component below. It loads the map, enables collisions via `BvhPhysicsBody`, and tweaks the ground material to receive shadows properly. ```tsx export function Map() { const [map, setMap] = useState(null) useEffect( () => map?.traverse( (object) => object.name === 'Plane' && object instanceof Mesh && ((object.receiveShadow = true), (object.material = new MeshStandardMaterial({ roughness: 1, metalness: 0, map: (object.material as MeshStandardMaterial).map, }))), ), [map], ) return ( ) } ``` Notes: - Wrapping the map in `BvhPhysicsBody` builds a BVH acceleration structure, necessary for performing collision detections. - We set `receiveShadow = true` on the ground mesh and re-create the material to ensure proper PBR/shadowing with the embedded base color map. - Keep your `map.glb` in `public/` so it’s served statically by Vite. - If you wonder why we load via `src="map.glb"` instead of `import`, it keeps asset paths simple for Vite and mirrors how the rest of the example serves content. ## Step 3 — The Character component Next, we create the `Character` component, which uses a VRM model (`avatar.vrm`), displays the pistol attached to the right hand (`pistol.glb`), adds animations (jogging, aiming, pistol actions), and sets up audio and visual effects. We prefer VRM for humanoids because it standardizes bone names (handy for retargeting); plain glTF works too if your bone names match the `boneMap`. Before creating the component, we need to create several custom actions, specifically reloading, shooting, and aiming actions outside of the character component. ```tsx export const ReloadAction = new EventAction() export const ShootAction = new EventAction() export const AimAction = new StateAction(BooleanOr, false) ``` Next, we create the character component and start by loading the character model and setting its height to the spawn height. ```tsx const model = useCharacterModelLoader({ castShadow: true, url: 'avatar.vrm' }) useEffect(() => void (model.scene.position.y = 70), [model]) ``` We set the spawn height once so the character drops onto the level instead of intersecting it. Next, we set up physics on the character and apply the movement actions to the physics. We use the built-in helper rather than writing raw velocity math to keep input → motion deterministic and consistent with other examples. ```tsx const physics = useBvhCharacterPhysics(model.scene) useFrame((state) => updateSimpleCharacterVelocity(state.camera, physics)) ``` Then, we bind all actions to the inputs (keyboard and mouse). Notice, that we have not yet added a camera behavior or used the custom actions, so even though they are bound, they have no effect yet. ```tsx // action bindings usePointerLockRotateZoomActionBindings() useKeyboardLocomotionActionBindings({ requiresPointerLock: true }) useKeyboardActionBinding(ReloadAction, { keys: ['KeyR'], requiresPointerLock: true }) usePointerButtonActionBinding(ShootAction, { buttons: [0], requiresPointerLock: true }) usePointerButtonActionBinding(AimAction, { buttons: [2], requiresPointerLock: true }) ``` Render the character and attach the pistol so you can already walk around. We will create the animation components, specifically `LowerBodyAnimation`, `SpineAnimation`, `UpperBodyAimAnimation`, `UpperBodyAdditiveAnimation` later. ```tsx {/* pistol and model */} ``` ## Step 3.1 — Create ammo store We’ll track ammo with a tiny Zustand store so reload/shoot actions can update it and the HUD can display it. Add this for example in `src/app.tsx` or in a small `src/state.ts` file: ```tsx import { create } from 'zustand' export const useAmmo = create(() => ({ ammo: 12 })) ``` ## Step 4 — Camera behavior and rotation sync We use the character camera behavior and keep the character’s yaw aligned with the camera. Using the provided behavior avoids re‑implementing orbit/zoom/offset logic; if you need full control later, you can swap it for your own. ```tsx const cameraBehaviorRef = useCharacterCameraBehavior(model.scene, { zoom: { speed: 0 }, characterBaseOffset: [0.5, 1.3, 0], }) // character rotation matches camera Y useFrame((state) => (model.scene.rotation.y = state.camera.rotation.y)) ``` ## Step 5 — Camera effects: FOV while running, zoom while aiming Two small hooks manage the cinematic feel. We apply gentle, framerate‑independent easing to avoid jarring changes: **FOV while running**: When `RunAction` is active, we increase the camera’s field of view (from 60 → 75) to convey speed. We use exponential smoothing (`t = 1 - exp(-k * delta)`) so the transition feels responsive yet stable at any framerate. After changing `fov`, we call `updateProjectionMatrix()` to apply it. ```tsx function useCameraFovControl() { useFrame((state, delta) => { if ('fov' in state.camera) { const targetFov = RunAction.get() ? 75 : 60 const t = 1 - Math.exp(-10 * delta) state.camera.fov += (targetFov - state.camera.fov) * t state.camera.updateProjectionMatrix?.() } }) } ``` **Zoom while aiming**: We adjust the `CharacterCameraBehavior`’s `zoomDistance` (2.0 → 0.7) while the `AimAction` is active. This narrows composition around the crosshair and subtly reduces parallax. We use a slightly faster smoothing constant for snappier aim‑down‑sights behavior. ```tsx function useAimZoomControl(behaviorRef: RefObject) { useFrame((_state, delta) => { const behavior = behaviorRef.current if (behavior == null) return const targetDistance = AimAction.get() ? 0.7 : 2.0 const t = 1 - Math.exp(-20 * delta) behavior.zoomDistance += (targetDistance - behavior.zoomDistance) * t }) } ``` ## Step 6 — Bone map and masks We provide a bone map for retargeting and define a mask for “upper body without spine”. Why exclude the spine? - We manually control the spine rotation in Step 8 to keep the torso aiming consistently forward inline with the cross-hair. If the aim layer also animated the spine, it would fight our manual rotation. - Therefore, the “upper body without spine” mask includes shoulders, arms, hands, chest, etc., but excludes the spine so we can drive it explicitly. - And because rigs vary, the `boneMap` lets us translate from your model’s bone names to VRM’s, so timeline clips target the right joints. ```ts export const upperBodyWithoutSpine = (name: VRMHumanBoneName) => upperBody(name) && name !== 'spine' export const boneMap: Record = { 'DEF-hips': 'hips', 'DEF-spine001': 'spine', // ... full mapping in file } ``` ## Step 7 — Lower-body locomotion and jumping For the lower body, which represents the character’s movement and jumping, we blend eight move directions plus idle, then add a jump state machine. This uses `@react-three/timeline` graphs and viverse helpers. Concepts: - Compute normalized input direction from actions. - Scale animation speed when running. - Jump state machine: start → loop → land → move. Create `src/lower-body-animation.tsx`. We’ll build it in small parts so each piece is clear and easy to place. This timeline approach keeps animation state readable and composable, compared to ad‑hoc `useFrame` toggles. 1. Setup: compute input direction and time scaling We convert the action values into a normalized 2D direction (for selecting locomotion clips) and speed up locomotion when running. ```tsx export function LowerBodyAnimation({ physics }) { const normalizedDirection = useMemo(() => new Vector2(), []) useFrame(() => normalizedDirection .set(MoveRightAction.get() - MoveLeftAction.get(), MoveForwardAction.get() - MoveBackwardAction.get()) .normalize(), ) const forwardRef = useRef(null) const backwardRef = useRef(null) const leftRef = useRef(null) const rightRef = useRef(null) const forwardRightRef = useRef(null) const forwardLeftRef = useRef(null) const backwardRightRef = useRef(null) const backwardLeftRef = useRef(null) useFrame(() => { const timeScale = RunAction.get() ? 2 : 1 for (const ref of [ forwardRef, backwardRef, leftRef, rightRef, forwardRightRef, forwardLeftRef, backwardRightRef, backwardLeftRef, ]) { ref.current && (ref.current.timeScale = timeScale) } }) // ... } ``` 2. Timeline scaffold and movement state (add this inside the return) We use a small timeline graph: - `RunTimeline` evaluates the timeline every frame. - `CharacterAnimationLayer` groups animation actions for a specific body region. - `Graph` declares states and transitions. - `GrapthState` is a named state node with transition rules. ```tsx return ( shouldJump(physics, lastJumpTimeRef.current) }, jumpLoop: { whenUpdate: () => !physics.isGrounded }, }} > {/* Add the directional Switch in substep 2a below */} {/* jump states below */} ) ``` 2a) Directional clip selection (Switch) Inside the `move` state, we select one of eight directional clips plus idle based on the normalized input. The `scaleTime` values are tuned so diagonals feel consistent with straight movement. ```tsx Math.abs(normalizedDirection.x) < 0.5 && normalizedDirection.y > 0.5}> normalizedDirection.x > 0.5 && normalizedDirection.y > 0.5}> normalizedDirection.x > 0.5 && Math.abs(normalizedDirection.y) < 0.5}> normalizedDirection.x > 0.5 && normalizedDirection.y < -0.5}> Math.abs(normalizedDirection.x) < 0.5 && normalizedDirection.y < -0.5}> normalizedDirection.x < -0.5 && normalizedDirection.y < -0.5}> normalizedDirection.x < -0.5 && Math.abs(normalizedDirection.y) < 0.5}> normalizedDirection.x < -0.5 && normalizedDirection.y > 0.5}> ``` 3. Jump states Jumping splits into short phases so we can apply upward velocity once, loop while airborne, and land smoothly when grounded again. ```tsx const lastJumpTimeRef = useRef(0) /* place inside after the "move" state */ !physics.isGrounded }, finally: 'jumpUp' }}> timePassed(0.2, 'seconds')} update={() => void physics.inputVelocity.multiplyScalar(0.3)} mask={lowerBody} paused url={JumpUpAnimationUrl} /> physics.isGrounded } }}> actionTime > 0.3 && physics.isGrounded }, finally: 'jumpLoop', }} > { lastJumpTimeRef.current = performance.now() / 1000 physics.applyVelocity(new Vector3(0, 8, 0)) }} url={JumpUpAnimationUrl} /> timePassed(150, 'milliseconds')} loop={LoopOnce} url={JumpDownAnimationUrl} /> ``` ## Step 8 — Spine: keep upright and match camera yaw Create `src/spine-animation.tsx`. Add these parts: 1. Resolve the spine bone once We cache the lowest upper‑body bone so updates are fast and don’t require repeated lookups. ```tsx export function SpineAnimation() { const model = useCharacterModel() const spineBone = useMemo(() => { // VRM or plain glTF return model instanceof VRM ? model.humanoid.getNormalizedBoneNode('spine') : model.scene.getObjectByName('spine') }, [model]) // ... } ``` 2. Align the spine each frame (keep upright, match camera yaw) The spine should stay upright and rotate only around Y to match camera yaw; this keeps the torso aligned with aim and avoids double transforms. ```tsx const eulerYXZ = new Euler(0, 0, 0, 'YXZ') const qWorld = new Quaternion() const qParentWorldInv = new Quaternion() const qLocal = new Quaternion() const cameraRotationOffsetY = -0.5 useFrame((state) => { if (spineBone == null) return state.camera.getWorldQuaternion(qWorld) eulerYXZ.setFromQuaternion(qWorld, 'YXZ') const cameraYaw = eulerYXZ.y + (model instanceof VRM ? 0 : Math.PI) + cameraRotationOffsetY eulerYXZ.set(0, cameraYaw, 0, 'YXZ') qWorld.setFromEuler(eulerYXZ) const parent = spineBone.parent if (parent != null) { parent.getWorldQuaternion(qParentWorldInv).invert() qLocal.copy(qParentWorldInv).multiply(qWorld) spineBone.quaternion.copy(qLocal) } else { spineBone.quaternion.copy(qWorld) } spineBone.updateMatrixWorld() }) ``` ## Step 9 — Aim up/forward/down blending Create `src/upper-body-aim-animation.tsx`. Add these parts: 1. Weight aim clips by camera pitch We blend “up/forward/down” by the camera’s pitch so the upper body points toward where you look. Using three focused clips keeps pose fidelity better than stretching a single generic clip. We also introduce `Parallel`, which plays its children at the same time—we’ll reuse it for simultaneous clip layers and effects. ```tsx export function UpperBodyAimAnimation() { const aimUpRef = useRef(null) const aimForwardRef = useRef(null) const aimDownRef = useRef(null) useFrame((state) => { if (!aimUpRef.current || !aimForwardRef.current || !aimDownRef.current) return const pitch = -state.camera.rotation.x if (pitch <= 0) { aimUpRef.current.weight = Math.min(1, Math.max(0, -pitch / (Math.PI / 2))) aimForwardRef.current.weight = 1 - aimUpRef.current.weight aimDownRef.current.weight = 0 } else { aimDownRef.current.weight = Math.min(1, Math.max(0, pitch / (Math.PI / 2))) aimForwardRef.current.weight = 1 - aimDownRef.current.weight aimUpRef.current.weight = 0 } }) // ... } ``` 2. Layer the three aim clips (mask excludes the spine) We play all three clips together and only change their weights; the mask excludes the spine to avoid conflicts with our manual spine rotation. ```tsx return ( ) ``` ## Step 10 — Additive upper-body: idle, shoot, reload, audio, muzzle flash Create `src/upper-body-additive-animation.tsx`. Add these parts: 1. Attach muzzle flash and audio under the right hand Audio and flash sit where the muzzle is, so sounds and visuals feel spatially correct. Notice, that the sound only plays when we execute `.play()` on the attached ref. ```tsx ``` 2. Timeline for idle → reload/shoot transitions (additive layer) An additive layer lets us overlay weapon actions on top of locomotion/aim; timeline transitions keep behavior deterministic and easy to expand. ```tsx ReloadAction.waitFor() }, shoot: { whenPromise: async () => { await ShootAction.waitFor() if (useAmmo.getState().ammo === 0) await new Promise(() => {}) }, }, }} > { reloadAudioRef.current?.play(0.3) useAmmo.setState({ ammo: 12 }) }} mask={upperBodyWithoutSpine} scaleTime={0.5} url="pistol-reload.glb" /> { const jitter = 0.01 state.camera.rotation.set( state.camera.rotation.x + (Math.random() - 0.5) * jitter, state.camera.rotation.y + (Math.random() - 0.5) * jitter, 0, ) }} until={() => timePassed(0.11, 'seconds')} /> { useAmmo.setState({ ammo: useAmmo.getState().ammo - 1 }) muzzleFlashAudioRef.current?.stop() muzzleFlashAudioRef.current?.play() if (muzzleFlashVisualRef.current) { muzzleFlashVisualRef.current.visible = true return () => (muzzleFlashVisualRef.current!.visible = false) } }} until={() => timePassed(0.07, 'seconds')} /> ``` The brief “jitter” recoil in the `shoot` state nudges the camera by a tiny random amount only while the state is active. Because the `Action` has an `until={() => timePassed(0.11,'seconds')}`, the shake starts exactly when the state begins and stops automatically, letting the regular camera behavior bring the view back smoothly (we keep roll at 0 to avoid unwanted tilt). ## Step 11 — HUD and crosshair The HUD is plain React DOM absolutely positioned over the canvas. It reads the player profile and ammo, shows a health bar, and renders a minimal crosshair. Highlights from `src/hud.tsx`: ```tsx const { name } = useViverseProfile() ?? { name: 'Anonymous', activeAvatar: null } const ammo = useAmmo((s) => s.ammo) // ... name at top-left, health bar bottom-left, ammo bottom-right ... // ... a simple crosshair (centered) composed of small divs ... ``` Create `src/hud.tsx` and add the full component. It overlays the canvas (no pointer events on the crosshair) and uses the same system font stack as the example. ```tsx export function HUD() { const [health, setHealth] = useState(50) const ammo = useAmmo((s) => s.ammo) const { name } = useViverseProfile() ?? { name: 'Anonymous', activeAvatar: null } const percent = Math.max(0, Math.min(100, health)) return ( <>
    {name}
    +
    {Math.round(health)}
    AMMO
    {ammo} / 12
    {/* Crosshair */}
    {/* center dot */}
    {/* top line */}
    {/* bottom line */}
    {/* left line */}
    {/* right line */}
    ) } ``` At this point your project should behave exactly like `examples/fortnite`. If anything does not work, compare your project with the files in `examples/fortnite`. # Remove VIVERSE Integrations To remove the VIVERSE integrations replace the `` component with `` and remove all VIVERSE-specific hooks from your components as these hooks will no longer work without the VIVERSE context. Dependencies: ```js { 'three': 'latest', '@react-three/fiber': '<9', '@react-three/viverse': 'latest', '@react-three/drei': '<10' } ``` Files: File: /App.tsx ```tsx import { Sky } from '@react-three/drei' import { Canvas } from '@react-three/fiber' import { BvhPhysicsWorld, SimpleCharacter, BvhPhysicsBody, PrototypeBox } from '@react-three/viverse' export default function App() { return ( ) } ``` # Without React `@pmndrs/viverse` offers everything from `@react-three/viverse` (excluding the hooks) in case you don't want to use react. The following tutorial shows how to build a simple game and display a player tag from the [Simple Game](https://pmndrs.github.io/viverse/tutorials/simple-game) and [Access Avatar and Profile](https://pmndrs.github.io/viverse/tutorials/access-avatar-and-profile) tutorials using only Three.js. ## Setting up the project First, create a new project and install the required dependencies: ```bash npm init -y npm install three @pmndrs/viverse @pmndrs/uikit@1.0.41 @viverse/sdk npm install -D vite ``` Create an `index.html` file: ```html VIVERSE Vanilla Game ``` ## Step 1: Basic Three.js Setup Start with the standard Three.js boilerplate in `index.ts`: ```typescript import { PerspectiveCamera, Scene, WebGLRenderer, Clock, } from 'three' // Create camera, scene, and renderer const camera = new PerspectiveCamera(90) camera.position.z = 1 camera.position.y = 1 const scene = new Scene() const canvas = document.getElementById('root') as HTMLCanvasElement const renderer = new WebGLRenderer({ antialias: true, canvas, powerPreference: 'high-performance', alpha: true }) const clock = new Clock() clock.start() // Basic render loop renderer.setAnimationLoop(() => { const delta = clock.getDelta() renderer.render(scene, camera) }) // Handle window resizing function updateSize() { renderer.setSize(window.innerWidth, window.innerHeight) renderer.setPixelRatio(window.devicePixelRatio) camera.aspect = window.innerWidth / window.innerHeight camera.updateProjectionMatrix() } updateSize() window.addEventListener('resize', updateSize) ``` ## Step 2: Adding the Sky Add a realistic sky background using the Three.js Sky class: ```typescript import { MathUtils, Vector3 } from 'three' import { Sky } from 'three/examples/jsm/Addons.js' // Add after scene creation const sky = new Sky() sky.scale.setScalar(450000) const phi = MathUtils.degToRad(40) const theta = MathUtils.degToRad(30) const sunPosition = new Vector3().setFromSphericalCoords(1, phi, theta) sky.material.uniforms.sunPosition.value = sunPosition scene.add(sky) ``` ## Step 3: Loading the Map Load a 3D environment using GLTFLoader: ```typescript import { GLTFLoader } from 'three/examples/jsm/Addons.js' // Add after sky setup const gltfLoader = new GLTFLoader() const ground = await gltfLoader.loadAsync('./map.glb') ground.scene.traverse((object) => { object.castShadow = true object.receiveShadow = true }) scene.add(ground.scene) ``` Make sure to place your `map.glb` file in the `public` folder. ## Step 4: Adding Lighting Create proper lighting with shadows: ```typescript import { AmbientLight, DirectionalLight } from 'three' // Add after sky setup const dirLight = new DirectionalLight('white', 1.0) dirLight.shadow.bias = -0.001 dirLight.shadow.mapSize.set(1024, 1024) dirLight.shadow.camera.left = -100 dirLight.shadow.camera.right = 100 dirLight.shadow.camera.top = 100 dirLight.shadow.camera.bottom = -50 dirLight.castShadow = true dirLight.position.set(10, 10, 10) scene.add(dirLight) scene.add(new AmbientLight('white', 0.3)) // Enable shadows in renderer renderer.shadowMap.enabled = true ``` ## Step 5: Adding Physics and the Character Integrate physics and a controllable character: ```typescript import { BvhPhysicsWorld, SimpleCharacter } from '@pmndrs/viverse' // Add after loading the map const world = new BvhPhysicsWorld() world.addBody(ground.scene, false) const character = new SimpleCharacter(camera, world, canvas, { model: { url: undefined } // We'll add avatar URL later }) scene.add(character) // Update character in render loop renderer.setAnimationLoop(() => { const delta = clock.getDelta() character.update(delta) renderer.render(scene, camera) // Respawn player when they fall down if (character.position.y < -10) { character.position.set(0, 0, 0) } }) ``` ## Step 6: VIVERSE Client Integration Connect to VIVERSE services to access user profiles and avatars: > [!NOTE] > The following assumes your are building your app with vite. If you are using a different build tool, you need to manually insure that the VIVERSE app ID is provided only in production builds. ```typescript import { Client } from '@viverse/sdk' import AvatarClient from '@viverse/sdk/avatar-client' // Add before character setup const client = import.meta.env.VITE_VIVERSE_APP_ID == null ? undefined : new Client({ clientId: import.meta.env.VITE_VIVERSE_APP_ID, domain: 'https://account.htcvive.com/' }) const auth = await client?.checkAuth() const avatarClient = auth == null ? undefined : new AvatarClient({ token: auth?.access_token, baseURL: 'https://sdk-api.viverse.com/' }) const profile = (await avatarClient?.getProfile()) ?? { name: 'Anonymous', activeAvatar: { headIconUrl: 'https://picsum.photos/200', vrmUrl: undefined }, } // Update character creation to use profile avatar const character = new SimpleCharacter(camera, world, canvas, { model: { url: profile.activeAvatar?.vrmUrl, type: "vrm" } }) ``` Create a `.env.production` file for your VIVERSE app ID: ``` VITE_VIVERSE_APP_ID=your_viverse_app_id_here ``` ## Step 7: Player Tag UI Finally, add a floating player tag above the character: ```typescript import { Group } from 'three' import { Container, withOpacity, Image, Text, reversePainterSortStable } from '@pmndrs/uikit' // Configure renderer for UI rendering renderer.setTransparentSort(reversePainterSortStable) renderer.localClippingEnabled = true // Create player tag const playerTag = new Group() character.add(playerTag) playerTag.position.y = 2.15 const container = new Container({ depthTest: false, renderOrder: 1, '*': { depthTest: false, renderOrder: 1, }, borderRadius: 10, paddingX: 2, height: 20, backgroundColor: withOpacity('white', 0.5), flexDirection: 'row', alignItems: 'center', gap: 4, }) playerTag.add(container) const playerImage = new Image({ width: 16, height: 16, borderRadius: 8, src: profile.activeAvatar?.headIconUrl }) container.add(playerImage) const playerText = new Text({ fontWeight: 'bold', fontSize: 12, marginRight: 3, text: profile.name }) container.add(playerText) // Update render loop to include UI updates renderer.setAnimationLoop(() => { const delta = clock.getDelta() character.update(delta) renderer.render(scene, camera) // Update the UI container.update(delta) // Rotate the player tag to face the camera playerTag.quaternion.copy(camera.quaternion) // Respawn player when they fall down if (character.position.y < -10) { character.position.set(0, 0, 0) } }) ``` ## Running the Project Add a dev script to your `package.json`: ```json { "scripts": { "dev": "vite --host" } } ``` Run your project: ```bash npm run dev ``` Your vanilla Three.js game with VIVERSE integration is now complete! You should see a 3D environment with a controllable character that displays a floating player tag with the user's avatar and name. # Publish to VIVERSE ## Prerequisites - Node.js version 22 or higher installed - A VIVERSE account (create one at [viverse.htcvive.com](https://viverse.htcvive.com)) ## Step 1: Install the VIVERSE CLI Install the official VIVERSE command-line interface: ```bash npm install -g @viverse/cli ``` ## Step 2: Authenticate with VIVERSE Before you can create apps or deploy, you need to authenticate with your VIVERSE account: ```bash viverse-cli auth login -e your-email -p your-password ``` ## Step 3: Create a VIVERSE App Create a new app entry in the VIVERSE platform: ```bash viverse-cli app create ``` After creation, **note the App ID** - you'll need this for deployment. ## Step 4: Configure Your App ID Next, we need to provide the App ID to our VIVERSE component. > [!TIP] > Do not include the app ID in your local development environment. Keep it production-only to avoid conflicts during development. Create a production environment file (`.env.production`) in your project root. ```bash # .env.production VITE_VIVERSE_APP_ID=your-app-id-here ``` This allows you to provide the app ID to your VIVERSE component using the environment variable `VITE_VIVERSE_APP_ID` ```tsx ``` This only works when using vite. If you don't use vite you need to manually make sure the `appId` is provided to the VIVERSE `clientId` in the production build. ## Step 5: Build Your Application Build your application for production. The exact command depends on your build tool. For vite you need to run `vite build`. ## Step 6: Deploy to VIVERSE Deploy your built application to the VIVERSE platform: ```bash viverse-cli app publish your-build-output-directly-here --app-id your-app-id-here ``` The CLI now shows you the URL with which you can preview your game in VIVERSE and how to submit it for review.