pathtracer Full documentation content. \[!NOTE] Both packages are at 0.1. Expect breaking changes between minor versions until 1.0. Each release lists them. ## Next - [Installation](/pathtracer/getting-started/installation) - [Your first render](/pathtracer/getting-started/your-first-render): a complete program in about 50 lines. - [Loading glTF](/pathtracer/getting-started/loading-gltf) - [The examples](/pathtracer/getting-started/examples), each one page with its source. ]]> 255 * (v <= 0.0031308 ? 12.92 * v : 1.055 * v ** (1 / 2.4) - 0.055); for (let i = 0; i < image.width * image.height; i++) { for (let c = 0; c < 3; c++) { const v = image.data[i * 4 + c]!; pixels.data[i * 4 + c] = encode(v / (1 + v)); } pixels.data[i * 4 + 3] = 255; } canvas.getContext('2d')!.putImageData(pixels, 0, 0); ``` ## What each step does **`requestDeviceForCaps`** asks for the optional features the library can use (subgroups, `shader-f16`, timestamps) and returns `caps`, which the kernels compile against. **`buildGpuScene`** builds one BVH per mesh and merges the static instances into one tree. It uploads the tree, then builds the scene copy: materials packed into records, a light table with every emissive triangle, the sun and the environment, and the texture arrays. Its `format` and `packing` are what a kernel must be compiled for. **`ptSceneOptions`** reads the scene and returns the kernel options. These cover which OpenPBR lobes the materials use, whether there are textures, alpha cutouts or an environment map, and the BVH format. The kernel text only contains code the scene needs. Add your own settings as overrides: ```ts ptSceneOptions(scene, { integrator: { nee: true, sampler: 'owen-sobol' } }); ``` **`encode`** records sample passes into your encoder. It reads nothing back, so it can go in a frame loop. `readImage` maps a buffer: call it outside timed code, never every frame. ## Showing it every frame For a live view, draw the accumulator directly instead of reading it back: - `pt.accumulator` is an `array` with the radiance sum in `rgb` and the sample count in `w`. - A fullscreen pass divides, tone maps and writes the canvas. - When the camera moves, call `setCamera` and `pt.reset(encoder)` before the next `encode`. The examples' [`present.ts`](https://github.com/pmndrs/pathtracer/blob/main/examples/shared/present.ts) and [`progressive.ts`](https://github.com/pmndrs/pathtracer/blob/main/examples/shared/progressive.ts) are about 150 lines together. Copy them freely. ]]> \[!NOTE] The default decoder draws each image into a 2D canvas, and browsers may premultiply alpha there. The colour of nearly transparent texels can lose precision. Pass an exact PNG decoder if that matters (for example in a reference comparison). ## In Node There is no default image decoder in Node: pass `decodeImage`, or `textures: false`. Everything else works with Node's own `fetch`. File URLs need `fetch` replaced, or pass the bytes. ```ts import { readFile } from 'node:fs/promises'; const input = await loadGltf(await readFile('model.glb'), { textures: false }); ``` ]]> \[!NOTE] The contract stores `−far` for the `viewZ` of a miss. The path tracer's frame planes have no far plane and write 0 there; test the hit flag rather than the depth. In those planes, motion comes from camera motion only: a moving instance reads as static. See [Outputs](/pathtracer/pathtracer/outputs). ## Jitter Sub-pixel jitter is measured in pixels, x right and y down, each in `[−0.5, 0.5]`. The sample position of pixel `(i, j)` is `(i + 0.5 + jx, j + 0.5 + jy)`, and its NDC offset is `(2·jx / width, −2·jy / height)`. A pipeline has exactly **one jitter owner**. By default the path tracer jitters its own camera rays: every sample draws a new offset inside the pixel. With V-buffer primaries (**Experimental**) the engine that rasterises the V-buffer owns the jitter and passes it in. Motion vectors never contain jitter, whoever owns it. ## Hits and materials - A ray is `{ origin, tMin, direction, tMax }` with a unit direction (`Ray`, 32 bytes). A hit's `t < 0` means a miss. - Barycentrics are the weights of vertices 1 and 2; vertex 0 gets `1 − u − v`. - A hit's `normal` is the world-space geometric normal. Shading normals, uvs and tangents come from the scene copy (`rtSurface`). - Material ids are 16 bits (`BVH_MAX_MATERIAL` = 65534) and index the scene's own material table. ## WGSL names WGSL has no namespaces, so each package owns a prefix: `bvh*` for `@pmndrs/bvh`, `rt*` for `@pmndrs/raytracer`, `pt*` for `@pmndrs/pathtracer`. Do not declare names with these prefixes in your own WGSL. See [Inline WGSL](/pathtracer/raytracer/inline-wgsl). ]]> \[!WARNING] The kernel text depends on which material features the scene uses. If an edit adds a feature no material had before (a coat, fuzz, transmission), create the kernel again from `ptSceneOptions`, or the new lobe is not compiled. ]]> 0 | A GGX conductor with complex Fresnel. Its reflectance is `baseColor × baseWeight`, or the explicit eta and k with `conductorEtaK`. A metalness between 0 and 1 picks metal or dielectric at random per hit with probability `metalness`, so the mean is the exact blend. | | Metal with `coatWeight` > 0.5 | A dielectric coat over the conductor. | | `transmissionWeight` > 0.5 | Glass: a smooth or rough dielectric. `thinWalled` makes it a thin sheet tinted by `transmissionColor`. Otherwise `transmissionDepth` > 0 adds Beer absorption inside the mesh (no nesting of media). | | Opaque dielectric with a specular lobe or `coatWeight` > 0.5 | A layered BSDF: the specular (or coat) interface over a Lambert base. A coat replaces the specular interface. | | Anything else | Lambert with `baseColor × baseWeight`, or black. | On top of these: - **Sheen**: `sheenWeight` > 0 with a non-black `sheenColor` wraps the BSDF in OpenPBR's fuzz layer (Zeltner's LTC sheen). - **Thin film**: an uncoated metal or opaque dielectric with `thinFilmWeight` > 0 gets an iridescent Fresnel. It needs `layering: 'albedo'` (below). - **Anisotropy**: `specularAnisotropy` stretches the GGX lobe of metals along the vertex tangent. The mesh needs tangents (glTF `TANGENT`). - **Emission**: from the front face, or both faces with `emissionTwoSided`. An emitter without an emission texture is also a light (see [Lights](/pathtracer/concepts/lights)). - **Alpha mask**: with `alphaMode` mask, a hit where the base colour alpha × `opacity` is below `alphaCutoff` is passed through. The kernel only compiles the cutout test for a textured scene. The coat weight and the transmission weight are thresholded at 0.5: there is no partial coat or partial transmission. These parameters are stored but not rendered yet: coat colour, `alphaMode` blend (partial opacity renders opaque), subsurface, diffuse roughness, and the metal's F82 tint. `dispersion` is only read in spectral mode (**Experimental**). ## Layered BSDF settings The layered BSDF (a dielectric interface over a base) has two ways to evaluate it: | `layering` | What it does | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `'albedo'` (default) | A closed form: the base is scaled by the energy the interface lets through, with light the interface reflects back down. Fast, and adds no noise of its own. | | `'stochastic'` | pbrt-v4's random walk between the layers (`coateddiffuse`, `coatedconductor`). The reference. It costs more and adds noise. | With `'albedo'`, `albedoBase: 'internal'` (default) is the closed form of a diffuse base under a dielectric; `'openpbr'` uses the OpenPBR specification's albedo scaling instead. `albedoCorrection` brings the closed form's energy to the random walk's on rough interfaces: `'texture'` (default, an 8 × 8 lookup texture), `'lut'` (the same table as WGSL constants) or `'none'`. ```ts const options = ptSceneOptions(scene, { layered: { layering: 'stochastic' } }); ``` ## Only the lobes a scene uses Each lobe costs WGSL, and a big kernel compiles slowly (on Windows in particular). `ptSceneOptions(scene)` calls `ptMaterialFeatures` on the scene's material table and compiles only what the table needs: | Feature | Compiled when some material has | | ------------- | --------------------------------------------------------- | | `mix` | `0 < metalness < 1` (a metallic-roughness texture counts) | | `thin` | thin-walled transmission | | `medium` | thick transmission with `transmissionDepth` > 0 | | `layered` | a dielectric specular lobe or a coat | | `coatedMetal` | a coated metal | | `aniso` | an anisotropic metal | | `fuzz` | sheen | | `thinFilm` | a thin film | Every material in the table counts, even one no ray ever hits. > \[!NOTE] If you write kernel options by hand, pass `integrator.materials` (`{ features: ptMaterialFeatures(records) }`) as `ptSceneOptions` does. Without it the kernel uses its legacy material adapter, which renders every non-metal as pure Lambert. See [Integrator options](/pathtracer/pathtracer/integrator-options). ## glTF `loadGltf` maps glTF materials and their `KHR_materials_*` extensions to these parameters. [Loading glTF](/pathtracer/getting-started/loading-gltf) has the table. ]]> \[!NOTE] A material with an `emissionTexture` is not in the light table, because next-event estimation needs a constant radiance per triangle. Rays that hit it by BSDF sampling still see it, so it is correct but noisier as a light source. ## The sun ```ts import type { SunInput } from '@pmndrs/raytracer'; const sun: SunInput = { direction: [-0.3, -1, -0.2], irradiance: [3, 2.9, 2.7], angularRadius: 0.00465 }; ``` | Field | What | | --------------- | -------------------------------------------------------------------------------------------------- | | `direction` | The way the light **travels**: down for a sun overhead. It is normalised for you. | | `irradiance` | W/m² on a surface facing the sun. | | `angularRadius` | Radians. 0 (the default) is a delta light: perfectly sharp shadows. The real sun is about 0.00465. | A delta sun is invisible to camera rays and in mirrors, as in pbrt-v4 and Mitsuba. With an angular radius, set `integrator.sunVisible: true` to see its disk in the image and in reflections. > \[!WARNING] `SunInput.direction` points the way light travels. The decoded `RtSunLight.direction` in the scene copy points toward the sun. A sign error gives a scene lit only by the sky, not an error message. ## The environment ```ts import type { EnvironmentInput } from '@pmndrs/raytracer'; const sky: EnvironmentInput = { radiance: [0.2, 0.25, 0.3] }; // data: width * height * 3 floats, linear radiance, row 0 = +Y const hdri: EnvironmentInput = { map: { width, height, channels: 3, data }, rotationY: Math.PI / 2 }; ``` | Field | What | | ----------- | ----------------------------------------------------------------------------------------------------------------------------------- | | `radiance` | Constant radiance, W/sr/m². With a `map` it defaults to the map's mean. | | `map` | An equirectangular image: `width`, `height`, `channels` (1, 3 or 4; alpha is ignored), `data` as linear float radiance, row 0 = +Y. | | `scale` | Multiplies the map. Default 1. | | `rotationY` | Radians about +Y, counter-clockwise seen from above. | The library has no HDR or EXR file reader: decode the file yourself into a float array. With no rotation, u = 0 of the map looks toward −Z and u = 0.25 toward +X. Lookups are bilinear. The kernel compiles the map code only when it is told there is a map (`integrator.envMap: true`). `ptSceneOptions` sets it from the scene. ## Next-event estimation and MIS At every scattering vertex that is not a perfect mirror or glass, the path tracer: 1. Picks one light from the light table. 2. Samples a point on it (or a direction for the sun and the environment) and traces one shadow ray. 3. Weights that sample against BSDF sampling with the power heuristic (multiple importance sampling). When a BSDF-sampled ray later hits an emitter or escapes to the sky, its contribution gets the matching MIS weight. A delta sun can only be reached by next-event estimation, so its sample is unweighted. `integrator.nee: false` turns next-event estimation off (BSDF sampling only, a debug setting); delta suns then contribute nothing. ### Sampling the environment | `integrator.envSampling` | What it does | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `'importance'` | Samples the map in proportion to its luminance (a row distribution, then a column distribution in the row). Needs `envMap`. `ptSceneOptions` picks it when the scene has a map. | | `'uniform'` | Uniform directions on the sphere. The integrator's own default, and `ptSceneOptions`' choice for a constant sky. | | `'none'` | No next-event estimation for the environment: BSDF sampling only. | ### Light selection Step 1 uses an alias table, so picking a light costs the same for one light or a million. The weights in the table only change the noise, never the mean. Set them with `SceneCopyInput.options.lightSelection`: | Value | Weights | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | | `'pbrt'` (default) | pbrt-v4's rule: each infinite light (sun, environment) gets an equal share, and the emissive triangles together get one more share, split by power. | | `'power'` | Proportional to power. | | `'kind'` | Each kind of light gets an equal share; within a kind, by power. | | `{ capped: c }` | Proportional to power, but the sun and the environment together get at most share `c`. | `options.sunPowerScale` and `options.envPowerScale` multiply the power the sun and the environment are given in these weights (default 1). Large numbers of emitters still cost noise: each vertex samples one light. For scenes with many lights, ReSTIR DI (**Experimental**, real-time only) reuses light samples across pixels and frames. See [Real time](/pathtracer/pathtracer/realtime). ]]> 1 | n/a | no | | Debug counters with alpha MASK | yes | no | | Reduced secondary materials (`materialLod: 'secondary'`) | one kernel, branch per vertex | a second, smaller shade kernel | An unsupported combination throws when the kernel is created, with the reason in the message. ]]> \[!WARNING] `preview` and `realtime` are biased: the cutout shadows, the reduced secondary materials and the texture cone all trade accuracy for speed. Never use them for reference images or for comparing against pbrt-v4 or Mitsuba. The settings that came from measurements are documented with their status in `PT_MODE_NOTES` (`ptModeNotes(mode)` returns one mode's entries), so a page can print what a mode means. ## Your options win A preset only fills keys you left `undefined`. Anything you set explicitly stays: ```ts // Real-time frames, but full materials at every bounce. ptSceneOptions(scene, { mode: 'realtime', blueNoise, integrator: { materialLod: 'full' } }); ``` Two details: - If you set `alphaPolicy` to anything but `'exact'` (the fastest, solid-shadow setting `'opaqueShadow'`, say), the preset does not add its `'transmit'` shadow settings: your policy decides the shadows. - The layered settings are merged only into a kernel whose materials use the layered BSDF, and only into the fields you left out. - With ReSTIR on (**Experimental**), `realtime` also sets `restir.sharedHit` on the megakernel unless you set it. ## Limits and packing - **Limits:** both schedulers start from the preset's limits. A later `setLimits` replaces them. - **Scene packing** is never changed by `create`, because the scene copy is uploaded before the kernel exists. Pass `ptModePreset(mode).scenePacking` (when it is not `null`) to both `buildGpuScene(device, input, { packing })` and the kernel's `scenePacking`. Both presets ask for `'packed'`, which is already the default. ## Building your own `ptModePreset(mode)` returns the preset as data (`integrator`, `layered`, `limits`, `scenePacking`), and `ptApplyMode(options, scheduler)` is the merge `create` runs. Use them to show a mode's settings in a UI, or to start a custom preset from one of ours. ]]> cam: Cam; @group(2) @binding(1) var outImage: texture_storage_2d; @compute @workgroup_size(8, 8) fn main(@builtin(global_invocation_id) id: vec3u) { if (id.x >= cam.width || id.y >= cam.height) { return; } let ndc = (vec2f(id.xy) + 0.5) / vec2f(f32(cam.width), f32(cam.height)) * 2.0 - 1.0; let dir = normalize(cam.forward + cam.right * (ndc.x * cam.tanHalfFov * cam.aspect) - cam.up * (ndc.y * cam.tanHalfFov)); let ray = rtRay(cam.origin, dir, 0.0, 1e30); let hit = rtClosest(ray, RT_MASK_ALL); var color = vec3f(0.0); if (rtIsHit(hit)) { let s = rtSurface(hit, ray, RT_SURF_SHADING_NORMAL); color = s.shadingNormal * 0.5 + 0.5; } textureStore(outImage, id.xy, vec4f(color, 1.0)); }`; const COMPUTE = GPUShaderStage.COMPUTE; const layouts = [ device.createBindGroupLayout({ entries: withBindings(rtWalkEntries(scene.format)) }), device.createBindGroupLayout({ entries: rtSceneCopyLayoutEntries(0, scene.copy.textureLayout, scene.packing) }), device.createBindGroupLayout({ entries: [ { binding: 0, visibility: COMPUTE, buffer: { type: 'uniform' } }, { binding: 1, visibility: COMPUTE, storageTexture: { access: 'write-only', format: 'rgba8unorm' } }, ] }), ]; const pipeline = await rtCreateComputePipelineAsync(device, { layout: device.createPipelineLayout({ bindGroupLayouts: layouts }), code: module + kernel, entryPoint: 'main', }); const [width, height] = [640, 360]; const image = device.createTexture({ size: [width, height], format: 'rgba8unorm', usage: GPUTextureUsage.STORAGE_BINDING | GPUTextureUsage.COPY_SRC }); // Camera at (0, 1.5, 5) looking down -Z, 45 degree vertical field of view (right-handed, +Y up). const cam = new ArrayBuffer(64); const [f, u] = [new Float32Array(cam), new Uint32Array(cam)]; f.set([0, 1.5, 5, Math.tan((45 * Math.PI) / 360), 1, 0, 0, width / height, 0, 1, 0]); u[11] = width; f.set([0, 0, -1], 12); u[15] = height; const camBuffer = device.createBuffer({ size: 64, usage: GPUBufferUsage.UNIFORM | GPUBufferUsage.COPY_DST }); device.queue.writeBuffer(camBuffer, 0, cam); const groups = [ device.createBindGroup({ layout: layouts[0]!, entries: rtSceneBindGroupEntries(scene.trace) }), device.createBindGroup({ layout: layouts[1]!, entries: rtSceneCopyBindGroupEntries(scene.buffers) }), device.createBindGroup({ layout: layouts[2]!, entries: [ { binding: 0, resource: { buffer: camBuffer } }, { binding: 1, resource: image.createView() }, ] }), ]; const encoder = device.createCommandEncoder(); const pass = encoder.beginComputePass(); pass.setPipeline(pipeline); groups.forEach((group, i) => pass.setBindGroup(i, group)); pass.dispatchWorkgroups(Math.ceil(width / 8), Math.ceil(height / 8)); pass.end(); device.queue.submit([encoder.finish()]); ``` `image` now holds the picture. Copy it to a canvas texture or draw it with a fullscreen pass. ## Options | Option | Default | What it does | | ------------------------------------------------ | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `nodeFormat`, `triangleLeaves`, `positionLayout` | bvh's defaults | The scene's buffer formats, compiled into the walk. Spread `scene.format` (or `sceneFormatOf(trace)`) so they always match. | | `traceSlot` | `{ group: 0, firstBinding: 0 }` | Where the walk's bindings go: 3 storage buffers, 4 with indexed leaves. | | `scene` | none | Add the scene copy at `scene.slot`: `rtSurface`, `rtMaterial` and the light functions. See below. | | `shade` | off | Add `rtShade`, one-bounce shading. `true` or `RtShadeOptions`. See [Shading](/pathtracer/raytracer/shading). | | `intervals` | `false` | Add the [interval queries](/pathtracer/raytracer/interval-queries). | | `rayMasks` | `true` | Compile instance masks into the walk. `false` makes the `mask` argument a no-op and the walk a little smaller. | | `stripComments` | `true` | Remove comments, keeping line breaks so compiler line numbers still match. | | `minify` | `false` | Also drop indentation and spaces around punctuation (line breaks stay). Use it with `shade`. | | `anyHitOrder` | `'far'` | Child order of the any-hit walk (bvh's setting). | | `triangleFilter` | every triangle blocks | WGSL for bvh's `fn bvhTrianglePasses(candidate: BvhCandidate) -> bool` hook, called for triangles flagged in bvh; `null` to declare it yourself. See [@pmndrs/bvh](https://github.com/pmndrs/bvh). | | `wideWalk`, `wideDecode` | `wideDecode: 'folded'` | Compile options of the experimental wide formats (`bvh4`, `cwbvh8`). | `scene` takes: | Field | Default | What it does | | -------------------------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `slot` | required | Group and first binding of the scene copy (1 binding packed, 2 split, then textures). | | `layout` | required | `scene.copy.layout`: which vertex attributes the copy stores. | | `packing` | `'packed'` | Must match the upload (`scene.packing`). | | `textures` | none | `scene.copy.textureLayout`: adds the texture arrays and lookups. Pass it whenever the scene has textures, so the layout and the bind group agree. | | `surface`, `materials`, `lights` | `true` | Leave out `rtSurface`, `rtMaterial` or the light functions. | | `sampling` | `false` | Add light point sampling (`rtSampleLightLi`). | ## The WGSL you get ```wgsl struct RtRay { origin: vec3f, tMin: f32, direction: vec3f, tMax: f32 } // direction unit length struct RtCone { width: f32, spread: f32 } struct RtHit { t: f32, instance: u32, triangle: u32, frontFace: bool, bary: vec2f, material: u32, normal: vec3f, coneWidth: f32 } fn rtVisible(ray: RtRay, mask: u32) -> bool // true = nothing blocks [tMin, tMax] fn rtVisibleSegment(a: vec3f, b: vec3f, mask: u32, eps: f32) -> bool fn rtClosest(ray: RtRay, mask: u32) -> RtHit // t == RT_MISS_T (-1) on a miss fn rtClosestCone(ray: RtRay, mask: u32, cone: RtCone) -> RtHit // also sets coneWidth at the hit fn rtSurface(hit: RtHit, ray: RtRay, want: u32) -> RtSurface // needs the scene copy ``` Both ends of `[tMin, tMax]` are inclusive. `RtHit.triangle` is the triangle's index in its original mesh, `bary` are the weights of vertices 1 and 2, and `normal` is the world-space geometric normal of the authored front side. `material` is the instance's or triangle's material id, `RT_NO_MATERIAL` when there is none. Helpers: `rtRay(origin, direction, tMin, tMax)`, `rtSegment(a, b, trimStart, trimEnd)` (a segment as a ray, pulled in at both ends), `rtIsHit(hit)`, `rtConeAt(cone, t)`, `rtConePropagate(cone, t, extraSpread)`. ### rtSurface `rtSurface` decodes what a hit needs from the scene copy. Only the bits in `want` are loaded: | Bit | Field | | -------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | `RT_SURF_POSITION` | `position`: world space. From the vertices when the copy stores positions (`buildGpuScene` does), else `origin + t * direction`. | | `RT_SURF_GEOMETRIC_NORMAL` | `geometricNormal`, `frontFace` (free: they come with the hit). | | `RT_SURF_SHADING_NORMAL` | `shadingNormal`: interpolated, on the geometric normal's side; the geometric normal when the mesh has none. | | `RT_SURF_UV` | `uv`, zero when the mesh has none. | | `RT_SURF_MATERIAL` | `material` (free). | | `RT_SURF_TEXFRAME` | `uvDensity` and `tangent` for texture level of detail and normal maps. Only with `scene.textures`. | | `RT_SURF_TANGENT` | `vertexTangent`. Only when the layout stores tangents. | `RT_SURF_ALL` is the first five. Normals face the authored front side, so flip them with `frontFace` when you need the side facing the ray. ## Masks Each instance has an 8-bit mask (`mask` on the scene's instances, 0 to 255, default 255). A trace skips an instance when `(instanceMask & rayMask) == 0`. `RT_MASK_ALL` hits everything. Use a mask bit to keep shadow rays from hitting emitters, or picking rays from hitting helpers. Masks survive bvh's merging of static instances. ## Compiler errors The module is long and generated, so a raw compiler message is hard to place. `rtCreateComputePipelineAsync(device, { code, layout, entryPoint, map })` compiles normally. If the pipeline fails, it throws an `Error` with the compiler messages, the source line and the lines around it. If you shrink the text yourself with `rtTransformWgsl` (prune unused declarations, shorten names, minify), pass the returned `map`: messages then name the original lines and identifiers. ```ts import { rtTransformWgsl, stripWgslComments } from '@pmndrs/raytracer'; const { code, map } = rtTransformWgsl(module + stripWgslComments(kernel), { keep: ['main'] }); const pipeline = await rtCreateComputePipelineAsync(device, { layout, code, map, entryPoint: 'main' }); ``` To read messages by hand: `rtMapCompilationInfo(await shaderModule.getCompilationInfo(), map)`, or `rtCheckShaderModule(shaderModule, map)`, which throws on an error. ## Things to know - The module starts with `enable` lines for the device's features. Put it first in your shader. To paste it into the middle of another shader, build `caps` with no optional features ([Capabilities](/pathtracer/raytracer/capabilities)). - Do not run the module through `buildWgsl` again: it is already preprocessed. - Keep the whole shader under about 50 KB: Windows' shader compiler slows down sharply above that. `rtModuleStorageBuffers(options)` tells you how many storage buffers the module adds. - With a packed scene copy, your own WGSL that reads `rtSceneTable` or `rtSceneData` directly must go through `rtPackSceneAccess`. Calling the module's functions needs nothing. ]]> ` | Label of the pipeline and buffers. | ## The batch | Field | What | | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `rays` | One buffer (`aos`) or a pair `[a, b]` (`soa`), `STORAGE` usage. | | `results` | At least `count * tracer.resultBytes` bytes, `STORAGE` usage. | | `count` | The ray count, or for an indirect tracer `{ buffer, word, capacity }`: the count is `buffer[word]` (a `u32`, `STORAGE` usage), clamped to `capacity`. | | `mask` | One ray mask for the whole batch. Default all ones. | | `cones` | Cone tracer only: `{ cones, widths }`. `cones` holds 8 bytes per ray (`width`, `spread`), `widths` gets one `f32` per ray (0 on a miss). | | `timestampWrites` | For `encode` only: timestamps around the batch's pass. | Buffers smaller than `capacity` entries throw. The tracer caches a uniform and a bind group per batch shape (its buffers, count and mask), up to 256 shapes. A direct count that changes every frame makes a new shape every frame: use an indirect tracer for that. ## Record layouts A ray is 32 bytes (`RAY_BYTES`), the same layout as bvh's `BvhRay` and WGSL `RtRay`: | Bytes | Field | | ----- | ------------------------- | | 0-11 | `origin` (vec3 f32) | | 12-15 | `tMin` | | 16-27 | `direction` (unit length) | | 28-31 | `tMax` | A full hit record is 40 bytes (`HIT_BYTES`); the compact one is the first 24 (`HIT_COMPACT_BYTES`): | Word | Field | | ---- | ------------------------------------------------- | | 0 | `t` (f32), negative on a miss (`MISS_T` is -1) | | 1 | `instance` | | 2 | `triangle`: index in the instance's original mesh | | 3 | `frontFace` (0 or 1) | | 4-5 | barycentrics of vertices 1 and 2 (f32) | | 6 | `material` (full only) | | 7-9 | world-space geometric normal (full only) | `HIT_WORD` gives these word offsets in code. `resultBytes(mode, hitRecord)` gives the stride. ## CPU helpers | Function | What it does | | ----------------------------------------------------- | --------------------------------------------------------------------------------------- | | `packRays(rays)` | `Ray[]` to a `Float32Array` in the AoS layout. | | `packRaysSoa(rays)` | `Ray[]` to `{ a, b }` for the SoA layout. | | `aosToSoa(packed)` | AoS floats to `{ a, b }`, bit for bit. | | `unpackRay(packed, index)` | One `Ray` back from AoS floats. | | `packCones(cones)` | `RayCone[]` to a `Float32Array`. | | `readHits(data, count, hitRecord)` | Hit records to `Hit[]`. Compact records report material `0xffffffff` and a zero normal. | | `readVisibility(data, count)` | Visibility words to `boolean[]`. | | `dispatchGrid(count, workgroupSize, maxPerDimension)` | The 2D workgroup grid the kernel uses for large counts. | ## Building your own pipeline The tracer is a thin wrapper. If you want the kernel inside your own pipeline setup, `rtBatchKernelWgsl(caps, options)` returns its WGSL (entry point `rtBatch`, the walk in group 0, the batch IO in group 1), `rtBatchEntries(resolveBatchOptions(options))` its group 1 layout entries, and `rtBatchArgsKernelWgsl` the one-thread kernel (entry point `rtBatchArgs`) that writes indirect dispatch arguments. ]]> \[!WARNING] Interval queries are experimental: they work and are checked, but the API can still change. An interval query traces only one segment of a ray: the closest hit with `t` in `(t0, t1]`, or whether anything lies in that range. The bounds are exact in f32, so consecutive intervals `(a, b]` and `(b, c]` split `(a, c]` with no gap and no overlap: every hit lands in exactly one of them. That is what radiance cascades need. Cascade `n` owns the distances `(t_{n-1}, t_n]`, and a hit must belong to exactly one cascade. The same holds for any probe scheme that splits a ray into near and far parts. ## Inline `rtModuleWgsl(caps, { intervals: true, ... })` adds: ```wgsl fn rtClosestInterval(origin: vec3f, direction: vec3f, t0: f32, t1: f32, mask: u32) -> RtHit // (t0, t1] fn rtClosestIntervalLo(origin: vec3f, direction: vec3f, t0: f32, t1: f32, mask: u32) -> RtHit // [t0, t1) fn rtVisibleInterval(origin: vec3f, direction: vec3f, t0: f32, t1: f32, mask: u32) -> bool // (t0, t1], any-hit fn rtVisibleIntervalLo(origin: vec3f, direction: vec3f, t0: f32, t1: f32, mask: u32) -> bool // [t0, t1), any-hit fn rtCascadeEnd(n: u32, t0: f32, growth: f32) -> f32 // t0 * growth^n fn rtCascadeStart(n: u32, t0: f32, growth: f32) -> f32 // 0 for n = 0, else the end of n - 1 fn rtCascadeIndex(t: f32, t0: f32, growth: f32, count: u32) -> u32 // the cascade holding t, or count fn rtOctEqualAreaDecode(uv: vec2f) -> vec3f // unit square to unit sphere, equal area ``` A miss returns `t == RT_MISS_T`. Without `intervals` the module text is unchanged. ```wgsl // Which cascade does this probe ray's first hit belong to? let hit = rtClosestInterval(probe, dir, 0.0, rtCascadeEnd(count - 1u, t0, 4.0), RT_MASK_ALL); let cascade = select(count, rtCascadeIndex(hit.t, t0, 4.0, count), rtIsHit(hit)); ``` ## Boundary rules | Convention | Range | A hit at exactly `t0` | A hit at exactly `t1` | | ----------------------------------------- | ---------- | -------------------------- | -------------------------- | | `open-low` (default, `rtClosestInterval`) | `(t0, t1]` | belongs to the range below | belongs here | | `closed-low` (`rtClosestIntervalLo`) | `[t0, t1)` | belongs here | belongs to the range above | "Exactly" means the f32 `t` the walk computes for the hit, compared with the f32 bounds you pass. That `t` depends only on the ray and the triangle, not on the interval. - With `open-low` and `t0 = 0`, a ray that starts on a surface never reports that surface at `t = 0`. - `t0` must be 0 or more. `t1` is clamped to the largest finite f32 (`RT_T_INF`); pass that for "to infinity". - An empty interval (`t1 <= t0`) is answered without walking. NaN bounds are not supported. > \[!NOTE] The closest-hit forms are exact at both ends. The any-hit forms (`rtVisibleInterval*` and the `visible` output) are not: a blocker within about 1e-6 (relative) of an interval end can be missed. Where a boundary hit matters, use a closest form and test `t`. Two triangles at the same `t` (a ray through a shared edge) may swap with the range. That is a tie, not an error. ## RtIntervalTracer The batched form: one compiled kernel, many dispatches per frame (one per cascade, or per probe batch), each with its own interval and seed. Parameters go into a ring of uniform slots, so a dispatch costs one small `writeBuffer` and nothing is read back. ```ts import { RtIntervalTracer, intervalResultBytes, intervalRayCount } from '@pmndrs/raytracer'; const tracer = await RtIntervalTracer.create(device, caps, { ...scene.format, source: 'probes', output: 'distance' }); tracer.beginFrame(); // once per frame, before the first job const encoder = device.createCommandEncoder(); const pass = encoder.beginComputePass(); for (const c of cascades) { tracer.dispatch(pass, scene.trace, { source: probeBuffer, // vec4f per probe: xyz position, w = interval scale (0 skips the probe) results: c.results, // intervalRayCount('probes', probeCount, c.directions) * tracer.resultBytes bytes count: probeCount, interval: { t0: c.start, t1: c.end }, // scaled per probe by w directions: c.directions, // cells per axis: directions^2 rays per probe jitter: [Math.random(), Math.random()], }); } pass.end(); device.queue.submit([encoder.finish()]); ``` ### Options | Option | Default | What it does | | --------------------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `source` | required | `'rays'`: a buffer of 32-byte rays whose `tMin` / `tMax` are the interval. `'probes'`: a buffer of `vec4f` probe positions and a bundle of directions per probe. | | `output` | required | `'hit'`: the [hit record](/pathtracer/raytracer/batch-tracer#record-layouts). `'distance'`: one `f32` per ray, -1 for nothing in the interval. `'visible'`: one `u32`, 1 = nothing in the interval. `'shaded'`: one `vec4f`, radiance at the hit from `rtShade` in `rgb` and `t` in `w` (`w = -1` and black for no hit). | | `convention` | `'open-low'` | `'open-low'` `(t0, t1]` or `'closed-low'` `[t0, t1)`. | | `workgroupSize` | `64` | `32`, `64`, `128` or `256`. | | `indirect` | `false` | The item count comes from a GPU buffer (`{ buffer, word, capacity }`). | | `hitRecord` | `'full'` | `'compact'` for the 24-byte record (`hit` output). | | `scene` | none | `shaded` only, required there: `{ layout, textures, packing }` of the scene copy. | | `shadeMode`, `shade`, `hooksWgsl` | `'lambert'` | `shaded` only: the [`rtShade`](/pathtracer/raytracer/shading) mode, its options, and WGSL for its hooks. | | `minify` | `true` for `shaded` | Minify the kernel text. | ### Jobs | Field | What | | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | `source`, `results`, `count` | The input buffer, the output buffer and the item count (rays or probes). | | `mask` | One ray mask for the job. Default all ones. | | `interval`, `directions`, `jitter` | `probes` only. `jitter` is the position inside each direction cell, `[0.5, 0.5]` (the centre) by default; change it per frame for coverage. | | `seed` | `shaded` only: seed of the light samples. | | `sceneCopy` | `shaded` only: `scene.buffers`. | | `reuseParams` | Repeat the previous job's parameters without using a new slot. | For `probes`, ray `i` belongs to probe `i / D` and direction cell `i % D`, with `D = directions^2`. Cells come from the equal-area octahedral map, so each covers the same solid angle and the 2 x 2 cells of cascade `n + 1` subdivide a cell of cascade `n` exactly. Results come out probe-major, so a probe's cells are contiguous. `bundleDirection(x, y, directions, jitter)` is the CPU twin of a cell's direction. The ring holds 256 jobs (`INTERVAL_RING_SLOTS`); more between two `beginFrame` calls throws. The `shaded` output does not support the resource-table texture binding. ## CPU twins `intervalBounds(t0, t1, convention)` and `inInterval(t, t0, t1, convention)` reproduce the GPU's membership test. `cascadeEnd`, `cascadeStart`, `cascadeIndex`, `octEqualAreaDecode` and `bundleDirection` match their WGSL versions; `nextUpF32` and `nextDownF32` step one f32. The cascade functions multiply in the same f32 order as the shader, so their boundaries agree bit for bit. GPU `sin` and `cos` are about 3e-5 off, so `rtOctEqualAreaDecode` matches its twin to that tolerance, not exactly. ]]> RtMaterial fn rtMaterialAlpha(m: RtMaterial) -> f32 // GGX alpha = specularRoughness^2 ``` `id` is `hit.material`. An id past the end of the table, including `RT_NO_MATERIAL`, returns the default material (OpenPBR's defaults: 0.8 grey dielectric, roughness 0.3). Each material is a 128-byte record of OpenPBR parameters. `RtMaterial` decodes it: | Group | Fields | | ------------ | --------------------------------------------------------------------------------------------------------------- | | Base | `baseColor`, `baseWeight`, `metalness` | | Specular | `specularColor`, `specularWeight`, `specularRoughness`, `specularAnisotropy`, `anisotropyRotation`, `ior` | | Conductor | `conductorEta`, `conductorK`, `conductorEtaK` (use eta and k instead of the base colour) | | Transmission | `transmissionWeight`, `transmissionColor`, `transmissionDepth`, `thinWalled` | | Coat | `coatWeight`, `coatColor`, `coatRoughness`, `coatIor` | | Sheen | `sheenWeight`, `sheenColor`, `sheenRoughness` | | Emission | `emissionColor`, `emissionLuminance`, `emissionTwoSided` | | Alpha | `opacity`, `alphaCutoff`, `alphaMode` (`RT_ALPHA_OPAQUE`, `RT_ALPHA_MASK`, `RT_ALPHA_BLEND`), `doubleSided` | | Textures | `baseColorTexture`, `roughnessMetalTexture`, `normalTexture`, `emissionTexture` (`RT_TEXTURE_NONE` when unused) | `baseColor` and `emissionColor` are stored as f32; most other weights and colours are 16-bit unorm. `readMaterial(copy, id)` and `unpackMaterial(words)` are the CPU twins. How the parameters combine into a BSDF is up to you; `rtShade` has a simple one, and `@pmndrs/pathtracer` has the full OpenPBR model. ## Lights The light table lists every emissive triangle (each triangle with an emissive material is its own light), then the sun, then the environment. A material whose emission comes from a texture is not in the table: it is still visible when a ray hits it, but it is never sampled as a light. An alias table over it picks a light in constant time. | Function | What it does | | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `rtLightCount()` | Number of lights. | | `rtSampleLight(u)` | Picks a light: `RtLightSample { index, pmf, u }`. `u` is the leftover of the random number, rescaled to `[0, 1)`. `index` is `RT_NONE` when there are no lights. | | `rtLightPmf(index)` | The probability that `rtSampleLight` picks `index`. | | `rtLight(index)` | An emissive triangle: `RtLight { position, edge1, edge2, normal, area, radiance, twoSided, instance, triangle, ... }`. | | `rtSun()`, `rtHasSun()` | `RtSun { direction, angularRadius, solidAngle, irradiance, pmf, ... }`. `direction` points toward the sun. | | `rtEnvironment()`, `rtHasEnvironment()` | `RtEnvironment { radiance, scale, rotationY, ... }`. | | `rtLightIndexOfHit(instance, triangle)` | The light index of an emissive triangle you hit, `RT_NONE` otherwise. | | `rtLightPmfOfHit(instance, triangle)` | Its pick probability (0 for a plain triangle): the light side of an MIS weight at a BSDF-sampled hit. | | `rtSceneInfo()` | Counts, total power and scene radius. | The table's weights follow `options.lightSelection` of the scene copy input (`'pbrt'` by default; also `'power'`, `'kind'`, `{ capped: share }`). The choice changes noise, never the mean. ### Light sampling With `scene.sampling: true` the module also has point sampling, the second half of next-event estimation: ```wgsl fn rtSampleLightLi(index: u32, p: vec3f, u: vec2f) -> RtLightLi struct RtLightLi { wi: vec3f, distance: f32, point: vec3f, pdf: f32, normal: vec3f, delta: bool, radiance: vec3f, valid: bool } ``` ```wgsl let pick = rtSampleLight(u.x); if (pick.index != RT_NONE) { let li = rtSampleLightLi(pick.index, position, u.yz); if (li.valid && rtVisibleSegment(position, position + li.wi * min(li.distance, 1e30), RT_MASK_ALL, 1e-4)) { direct = li.radiance * myBsdf(li.wi) * max(dot(n, li.wi), 0.0) / (pick.pmf * li.pdf); } } ``` `pdf` is a solid-angle density and does not include `pick.pmf`; multiply them yourself. Emissive triangles are sampled uniformly by area (one-sided ones give nothing from behind), the sun uniformly inside its cone (a sun of radius 0 is a delta direction, `delta = true`, `pdf = 1`), and the environment uniformly over the sphere. > \[!NOTE] `scene.sampling` compiles the environment as a constant radiance (for a map, its mean). To sample an environment map by its luminance, leave `sampling` off and append `rtLightSampleWgsl({ envMap: true })` to the module, through `rtPackSceneAccess` when the copy is packed. That is what the path tracer does. ## Textures Textures are grouped into _buckets_: one `texture_2d_array` and one trilinear sampler per (size, colour space, wrap mode). Each texture is resized to a power of two and stored as one layer. A scene uses at most 8 buckets by default (`textures.maxBuckets`), whatever the number of textures; more are merged. ```wgsl fn rtTexSample(slot: u32, uv: vec2f, lod: f32) -> vec4f fn rtApplyMaterialTextures(m: RtMaterial, uv: vec2f, uvFootprint: f32) -> RtMaterial fn rtApplyNormalMap(n: vec3f, tangent: vec4f, texel: vec3f) -> vec3f ``` - `rtTexSample` returns linear values. Colour textures (`colorSpace: 'srgb'`: base colour, emission) are stored as `rgba8unorm-srgb`, so the hardware decodes them before filtering. Data textures (`'linear'`: metallic-roughness, normal maps) are `rgba8unorm`. The scene copy checks that each material field gets the right kind. Alpha is always linear. - `rtApplyMaterialTextures` applies glTF semantics: base colour times `rgb`, opacity times `a`, roughness times `g`, metalness times `b`, emission times `rgb`. `uvFootprint` is the filter width in uv units; 0 reads the full-resolution level. - `rtApplyNormalMap` takes the shading normal, the tangent from `rtSurface(..., RT_SURF_TEXFRAME)` and the normal map texel (`rgb`), and returns the perturbed normal. - Texture coordinates follow glTF: `(0, 0)` is the top-left texel, `v` points down. For mip selection, `rtSurface` with `RT_SURF_TEXFRAME` gives `uvDensity` (uv units per metre). With a ray cone, the footprint is `hit.coneWidth * uvDensity / abs(cosTheta)`; `coneUvFootprint` is the CPU twin, and `textureLod(width, height, footprint)` the level it selects. The path tracer does this with `textureLod: 'cone'`. ### Mips Mip levels 1 and up are made on the GPU by default (`textures.mips: 'gpu'`). `buildSceneCopy` keeps level 0 and `uploadSceneCopy` fills the rest with [`@pmndrs/downsampler`](https://github.com/pmndrs/downsampler), one `Downsampler` per device (`rtMipGenerator(device)`). Pass `uploadSceneCopy(device, copy, { mipGenerator })` to use your own (anything with `encode(encoder, texture)`). `mips: 'cpu'` builds every level on the CPU instead, which is slower. Use it when there is no device, or when the CPU twin `sampleSceneTexture` must read levels above 0. `mips: false` stores one level. ### Bindless textures (experimental) > \[!WARNING] The resource-table binding is experimental: it works and is checked, but the API can still change, and browser support is thin. With `textures.binding: 'table'`, the buckets go into one resource table instead of one binding pair each, so buckets are never merged. It needs a device with `caps.resourceTable !== 'none'` (`chromium-experimental-sampling-resource-table`). - `rtModuleWgsl` adds the `enable` line itself, and `rtSceneCopyLayoutEntries` adds no texture bindings. - The pipeline layout needs `usesResourceTable: true` (`rtUsesResourceTable(scene.copy.textureLayout)` tells you). - Bind the table on each pass: `rtSetResourceTable(pass, scene.buffers.textures!.table!)`. In Chrome Canary on Metal, a compute pass that sets a resource table fails today; render passes work. Use the default `'arrays'` binding unless you are testing this. ]]> RtShaded` | Radiance at the hit toward `-ray.direction`. A miss returns the environment in `emission`. | | `rtShadePoint(hit, ray) -> RtShadePoint` | The surface decode `rtShade` does: position, shading and geometric normals on the viewer's side, `wo`, the material, the diffuse albedo. | | `rtShadeAt(p, mode, u, mask) -> RtShaded` | Shade a point you already decoded, for example with several light samples. | | `rtShadeTotal(s) -> vec3f` | `emission + direct + indirect`. | | `rtShadeMiss(direction) -> vec3f` | Environment radiance, plus the sun disc with `sunVisible`. | | `rtShadeDiffuseFromIrradiance(p, E) -> vec3f` | `albedo * E / pi`, for caches that store irradiance. | | `rtOffsetOrigin(p, n) -> vec3f`, `rtSideNormal(n, w) -> vec3f` | A ray origin that does not hit its own surface, and a normal flipped to the side of `w`. | `RtShaded` keeps the terms apart, for denoisers and cache writers: `{ emission, direct, indirect: vec3f, light: u32, occluded: bool }`. `light` is the sampled light (`RT_NONE` if none contributed) and `occluded` says a shadow ray was traced and blocked. `u` is three independent random numbers in `[0, 1)` from your sampler: `u.x` picks the light, `u.yz` the point on it. The module has no random state of its own. `mask` is the shadow ray's ray mask. ## Modes The mode is a runtime argument, so one pipeline serves all three: | Mode | Shadow rays | Result | | ------------------ | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `RT_SHADE_FLAT` | 0 | `emission + indirect`. With `flat: 'albedo'`, also the unlit base colour in `direct`. | | `RT_SHADE_LAMBERT` | at most 1 | Adds direct light with a Lambert BSDF of the diffuse albedo, `baseColor * baseWeight * (1 - metalness) * (1 - transmissionWeight)`. A pure metal is black here. | | `RT_SHADE_FULL` | at most 1 | As Lambert, but the BSDF is diffuse plus GGX specular (built in), or yours. | The direct term is `Le * f * cos / (pmf * pdf)` for one light chosen by the light table and one point on it. There is no BSDF sample inside `rtShade`, so there is no MIS: your continuation ray is the BSDF sample. The estimate is unbiased, with the noise of one light sample. ## Options | Option | Default | What it does | | ------------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `bsdf` | `'builtin'` | `'builtin'`: Lambert diffuse plus GGX specular (height-correlated Smith, Schlick Fresnel). `'external'`: you declare `fn rtShadeBsdf(p: RtShadePoint, wi: vec3f) -> vec3f`, the BSDF value without the cosine. Used by `RT_SHADE_FULL` only. | | `cache` | `'none'` | `'none'`: the indirect term is black. `'external'`: you declare `fn rtShadeIndirect(p: RtShadePoint) -> vec3f`, the outgoing radiance toward `p.wo`, already weighted by the BSDF. | | `envSampling` | `'uniform'` | How the environment is sampled when the light table picks it: `'uniform'` over the sphere, or `'cosine'` over the shading hemisphere. | | `flat` | `'cache'` | What `RT_SHADE_FLAT` returns: `'cache'` (emission + indirect) or `'albedo'` (also the unlit base colour). | | `sunVisible` | `false` | `rtShadeMiss` adds the sun's radiance inside its disc. | | `minAlpha` | `0.01` | GGX alpha floor of the built-in BSDF. A near-mirror lobe is hard to hit with one light sample. | ## Hooks WGSL has no virtual functions. A hook is a function the module either defines, or leaves for you to define anywhere in the final shader: ```ts shade: { cache: 'external' } ``` ```wgsl // Read your own radiance cache at the shading point. fn rtShadeIndirect(p: RtShadePoint) -> vec3f { let irradiance = myProbeLookup(p.position, p.normal); return rtShadeDiffuseFromIrradiance(p, irradiance); } ``` `RtShadePoint` is `{ position, normal, geometricNormal, wo: vec3f, frontFace: bool, material: RtMaterial, albedo: vec3f }`, all in world space with the normals on the viewer's side. Your hooks bring their own bindings by declaring them in your part of the shader. ## What it does and does not do - Emission counts on the authored front face only, unless the material is `emissionTwoSided`. Back faces are otherwise shaded like front faces, with the normals flipped toward the viewer. - Shadow rays start and end at offset points (`rtOffsetOrigin`), so they do not hit their own surface or the light. - Textures are not sampled: base colour and roughness are the material's constants. - An environment map counts as its mean radiance. Its directions are not importance sampled. - It needs the full 40-byte hit (material and normal). The batch tracer's compact record does not carry them. ## CPU twin `shade(lights, point, mode, u, visible, hooks, options)` in TypeScript computes the same estimate in f64. `lights` can be the CPU `SceneCopy`, `mode` is `'flat'`, `'lambert'` or `'full'`, and `visible` (a `ShadeVisible` function) stands in for the shadow ray. It is what the GPU code is tested against, and it works for CPU-side probes and baking. The other helpers (`makeShadePoint`, `builtinBsdf`, `shadeMiss`, `shadeTotal`, ...) are its parts. ]]> \[!WARNING] Never branch on `tier`. It hides which feature is missing. Test the flag you need (`caps.subgroups`, `caps.shaderF16`, ...). ## Enable lines and the preprocessor Generated WGSL starts with `enable` lines for the features the device has (`enable f16;`, `enable subgroups;`, ...). WGSL only allows them at the top of a shader, so: - Put the output of `rtModuleWgsl` first and append your code after it. - Error line numbers in your part are shifted by the number of `enable` lines. `wgslEnableLines(caps)` returns them, so you can count them. `buildWgsl(source, caps, defines)` is the preprocessor the library uses: it prepends the `enable` lines and resolves `#if NAME`, `#if !NAME`, `#elif`, `#else` and `#endif` against `capsDefines(caps)` (`HAS_SUBGROUPS`, `HAS_F16`, `HAS_TIMESTAMP`, `BINDLESS`, ...) plus your own `defines`. Dropped lines become empty lines, so line numbers still match. `preprocess(source, defines)` does the `#if` part alone. Use them for your own kernels if you like. > \[!NOTE] Output of `rtModuleWgsl` is already preprocessed. Running it through `buildWgsl` again adds a second block of `enable` lines in the middle of the file, which fails to compile. ## Pasting into someone else's shader When the module has to go into the middle of another shader (a three.js node, another library's kernel), it must not carry `enable` lines. Build caps with no optional features: ```ts import { buildCaps, rtModuleWgsl } from '@pmndrs/raytracer'; const plainCaps = buildCaps({ features: new Set(), limits: device.limits as unknown as Record }); const code = rtModuleWgsl(plainCaps, { ...scene.format, traceSlot: { group: 3, firstBinding: 0 } }); ``` The trace, surface, material, light and shading code needs no optional feature, so nothing is lost. The one exception is the experimental resource-table texture binding, which needs `caps.resourceTable`. Choose `traceSlot` and the scene slot so they do not collide with the host shader's bindings, and check that the host does not use names starting with `rt` or `bvh`. ]]> \[!WARNING] A kernel and a scene must agree on the node format, the scene packing and whether there are textures. `setScene` throws on a mismatch. Build the kernel from the scene it will draw, with `ptSceneOptions`, rather than by hand. ### Other create options | Option | Default | What it does | | --------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `mode` | none | `'reference'`, `'preview'` or `'realtime'`. Fills the integrator settings you left undefined and the starting limits. See [Modes](/pathtracer/concepts/modes). | | `blueNoise` | none | The blue-noise texture, needed by `sampler: 'bluenoise'`. See [Real time](/pathtracer/pathtracer/realtime). | | `volumeGrid` | none | The density grid of a grid medium. See [Volumes](/pathtracer/pathtracer/volumes). | | `spectralTable` | fitted on first use | The RGB-to-spectrum table. See [Spectral rendering](/pathtracer/pathtracer/spectral). | | `minify` | `true` | Drops indentation and spaces. Line breaks stay, so line numbers do not move. | | `prune` | `true` | Drops WGSL declarations the entry point cannot reach. | | `shortenNames` | `true` | Renames top-level WGSL names to one or two letters. | | `sourceMap` | `false` | Also passes a Source Map v3 to `createShaderModule`. Browsers may ignore it. | | `label` | `'pt megakernel'` | Label of the pipeline and buffers. | `mode` does not change `scenePacking`. If you want the mode's packing, pass `ptModePreset(mode).scenePacking` to both `buildGpuScene` and the kernel. ## The lifecycle | Call | What it does | | -------------------------- | --------------------------------------------------------------------------------------------- | | `setScene(trace, buffers)` | Binds the BVH and the scene copy. Call again after rebuilding the scene. | | `setFilm(width, height)` | Allocates the accumulator. A new size replaces it and restarts the sample count. | | `setCamera(camera)` | A look-at pinhole: `position`, `target`, `up`, `yFovDeg` (vertical field of view in degrees). | | `setLimits(limits)` | `maxDepth`, optional `rrStartDepth` and `textureConeScale`. | | `setSeed(seed)` | The scrambling seed. A new seed gives an independent noise pattern. | | `reset(encoder)` | Clears the accumulator and the sample count. Records a clear into `encoder`. | | `encode(encoder, passes)` | Records `passes` sample passes into your encoder. | | `readImage()` | Reads back the mean radiance. | | `destroy()` | Frees the kernel's buffers and textures. | `setLimits` follows pbrt's semantics. `maxDepth` counts scattering events: 1 is direct lighting only, and the emission seen by the camera ray is always included. `rrStartDepth` is the first vertex with Russian roulette; omitted or 0 turns it off. Without a mode, the limits start at `{ maxDepth: 5 }` with no roulette. `setLimits` replaces the whole object. `encode(encoder, passes, timestampWrites?)` records one compute pass with `passes` dispatches. It takes at most 256 passes per call. Each pass writes its own record into a ring of 1024 uniform slots, so keep fewer than 1024 passes in flight between submits. `encode` reads nothing back, so it can go in a frame loop. The optional `timestampWrites` go on the compute pass, for GPU timing. ## Progressive accumulation The accumulator is a storage buffer of `width * height` `vec4f`: the radiance sum in `rgb` and the sample count in `w`. Each pass adds one sample. `pt.samples` counts the passes encoded since the last `reset`. ```ts // The camera moved: start over. pt.setCamera(camera); const encoder = device.createCommandEncoder(); pt.reset(encoder); pt.encode(encoder, 4); device.queue.submit([encoder.finish()]); ``` There are two ways to see the image: - `readImage()` copies the accumulator to a staging buffer, maps it and divides by the sample count. It returns `{ width, height, data, samples }` with RGBA `float32` data, linear, row 0 at the top. Call it outside timed code, never every frame. - `pt.accumulator` is the `GPUBuffer` itself (or `null` before `setFilm`). A fullscreen pass can divide by `w`, tone map and draw it, with no readback. With `integrator.aovs` the AOVs live in the tail of the same buffer. See [Outputs](/pathtracer/pathtracer/outputs). ## Compile errors The kernel text is minified, pruned and renamed by default, so a raw compiler message would name `C5` at a shifted line. `create` translates errors back: the message it throws has the original names, the source line numbers and the lines around the error. To see the source yourself: ```ts import { ptMegakernelBuild, ptMegakernelWgsl } from '@pmndrs/pathtracer'; const { code, map } = ptMegakernelBuild(caps, options); // what create compiles, plus its source map const readable = ptMegakernelWgsl(caps, { ...options, minify: false, prune: false, shortenNames: false }); ``` `map` reads a compiler's messages with `rtMapCompilationInfo` from `@pmndrs/raytracer`. To get readable names in your own errors, build with `shortenNames: false`. ## Kernel size `pt.wgslBytes` is the length of the compiled WGSL. The kernel only contains code the scene and the options need, so it ranges from about 40 KB for a plain scene to just over 50 KB with every OpenPBR lobe. Size matters on Windows: the DXC compiler behind Chrome's D3D12 backend gets slow on large shaders, and the project keeps generated kernels near 50 KB or less. `ptwfAutoPolicy` takes `kernelBytes` and sends a megakernel over that size to the wavefront scheduler, whose kernels are smaller. See [Wavefront](/pathtracer/pathtracer/wavefront#choosing-a-scheduler). ## Debug helpers | Member | What | | ------------------------------------------------ | -------------------------------------------------------------------------- | | `options` | The options after the mode preset was applied. | | `instrumented` | `true` when the kernel writes debug counters. Its timings are not results. | | `restir` | The ReSTIR passes, or `null`. See [ReSTIR](/pathtracer/pathtracer/restir). | | `hookBindGroupLayout`, `setHookBindGroup(group)` | The path hooks' own bind group (experimental). | | `setVbuffer(texture)`, `setVbufferJitter(x, y)` | V-buffer primaries (experimental, `integrator.primary: 'vbuffer'`). | The [schedulers example](https://pmndrs.github.io/pathtracer/examples/pathtracer/schedulers/) runs the same scene on both schedulers. ]]> 1` with `aovs`, `counters` or `hooks` | Those buffers are indexed per pixel, not per slot. | | `integrator.frame` with `integrator.aovs` | Both live in the tail of the accumulation buffer. | | `integrator.restir` with alpha mask, with `hooks`, or with `restir.sharedHit` | Not ported yet. | | `integrator.counters` with alpha mask | The cutout loop runs in the intersect kernel, which has no counter buffer. | | `integrator.materialLod: 'secondary'` with `volumes` or `spectral` | The reduced shade kernel has no medium or wavelength code. | | `textures.binding: 'table'` | Bindless textures run on the megakernel only. | `ptwfUnsupported(integratorOptions)` returns the integrator part of this check as a string, or `null`. Volumes, spectral rendering, ReSTIR GI, alpha cutouts, `materialLod`, hooks and the real-time frame planes all run on the wavefront within those limits. ## Choosing a scheduler `ptwfAutoPolicy(input)` picks a scheduler and its settings from facts known before the first frame: ```ts import { PtMegakernel, PtWavefront, ptMegakernelWgsl, ptSceneOptions, ptwfAutoPolicy } from '@pmndrs/pathtracer'; const options = ptSceneOptions(scene); const { scheduler, reason, ...settings } = ptwfAutoPolicy({ width: 1280, height: 720, maxDepth: 8, subgroups: caps.subgroups, kernelBytes: ptMegakernelWgsl(caps, options).length, }); const pt = scheduler === 'wavefront' ? await PtWavefront.create(device, caps, { ...options, ...settings }) : await PtMegakernel.create(device, caps, options); ``` | Input | | | ----------------- | ------------------------------------------------------------------------------------------------------------------------ | | `width`, `height` | The film. | | `maxDepth` | The path length limit. | | `subgroups` | `caps.subgroups`. | | `frame` | `true` for real-time frames (one sample per pixel per encode): the choice is then the megakernel or the plain wavefront. | | `triangles` | Scene size, used with `frame`. | | `kernelBytes` | The megakernel's WGSL size. Above about 50 KB the policy picks the wavefront whatever the other rules say. | The result has `scheduler`, `refill`, `layers`, `appends`, `sort` and a one-line `reason`. In short: short paths (`maxDepth` 2 or less) and small real-time scenes go to the megakernel; big films go to the plain wavefront; mid and small films to the wavefront with inline refill, with layers on small films. Sorting and ordered refill are never picked. Spread the result, then your own overrides. > \[!NOTE] The rules come from measurements on one machine and are provisional. Every field stays an explicit setting: override any of them. ]]> \[!WARNING] The `preview` and `realtime` modes use `alphaShadow: 'transmit'` with `alphaTransmit` 0.45. It only works when the cutout cards are in instances whose `mask` has the `alphaMixedMask` bit. Without that, the kernel compiles and the shadows are those of solid cards. ## Outputs | Option | Default | What it does | | --------------- | ----------------------------- | --------------------------------------------------------------------------------------- | | `accumulate` | `true` (`false` with `frame`) | `false` overwrites the accumulator with each pass's sample. | | `aovs` | `false` | Albedo, normal, view depth and coverage. See [Outputs](/pathtracer/pathtracer/outputs). | | `counters` | `false` | Per-pixel debug counters. Never time a kernel with them. | | `frame` | `false` | Real-time frame planes. See [Real time](/pathtracer/pathtracer/realtime). | | `frameEmission` | `'mirror'` with `frame` | Which light skips the demodulated planes: `'none'`, `'primary'` or `'mirror'`. | ## Experimental These work and are checked, but their options can still change. | Option | Default | What it does | | ------------------ | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `materialLod` | `'full'` | `'secondary'`: vertices at depth `materialLodDepth` and beyond use a reduced material with one lobe picked at random. Noisier, cheaper, for real time. Needs OpenPBR mode to change a BSDF. | | `materialLodDepth` | `1` | First depth that uses the reduced material. | | `primary` | `'trace'` | `'vbuffer'`: the primary hit comes from a visibility buffer your rasteriser wrote (`setVbuffer`, `setVbufferJitter`). One `encode(encoder, 1)` per frame; needs `layout.positions`. | | `hooks` | `null` | Your WGSL called at every path vertex: `ptOnVertex` can end a path, `ptTerminate` adds radiance when it does. | | `restir` | `null` | ReSTIR DI and GI. See [ReSTIR](/pathtracer/pathtracer/restir). | | `volumes` | `null` | Participating media. See [Volumes](/pathtracer/pathtracer/volumes). | | `spectral` | off | Spectral transport. See [Spectral rendering](/pathtracer/pathtracer/spectral). | | `materialx` | `null` | Generated MaterialX WGSL. See [MaterialX](/pathtracer/pathtracer/materialx). | ## Combinations that throw `resolveIntegratorOptions` (called by `create`) refuses these: | Option | Cannot be combined with | | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `volumes` | `restir`, `frame`, `primary: 'vbuffer'`, alpha mask, `hooks`, `counters`, `aovs`, `spectral`, the Beer-law absorber (`materials.features.medium`) | | `spectral` | `hooks`, `frame`, `aovs`, `restir`, `volumes` | | `materialx` | a scene without textures (`create` throws) | | `restir` | see [ReSTIR](/pathtracer/pathtracer/restir#what-it-refuses) | The wavefront has its own list: see [Wavefront](/pathtracer/pathtracer/wavefront#what-the-wavefront-cannot-do). ]]> \[!NOTE] Our normals are in world space. OIDN expects view-space normals in `[-1, 1]`, with 0 on a miss. Rotate them by the camera's view matrix before passing them on. For a moving camera, use the real-time frame planes below instead. ## Counters ```ts const pt = await PtMegakernel.create(device, caps, ptSceneOptions(scene, { integrator: { counters: true } })); // ... encode ... const { summary, raw } = await pt.readCounters(); console.log(summary.depth, summary.shadowRays, summary.reasons.rr); ``` `readCounters()` returns sums over all samples since the last `reset`: | Field | What | | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | `raw` | `PT_COUNTER_WORDS` (12) `u32` per pixel. | | `summary.depth` | Mean scattering events per path. | | `summary.extensionRays`, `summary.shadowRays` | Mean rays per path, cutout pass-throughs included. | | `summary.nodeVisits` | Reserved, always 0. | | `summary.reasons` | Fraction of paths per end reason: `escaped`, `absorbed`, `maxDepth`, `rr`, `hook`, `invalid`, `maskOverflow`. The fractions sum to 1. | `ptCounterImage(raw, width, height, samples, word)` turns one counter into a per-sample image for a heat map. The word indices are in `PT_COUNTER`, the reasons in `PT_TERM`. > \[!WARNING] Counters cost time inside the kernel. `pt.instrumented` is `true` for such a kernel: never report its timings. The wavefront refuses counters with alpha cutouts. ## Frame planes For real-time rendering, `integrator.frame` writes one sample per pixel per frame and seven planes a temporal denoiser and an upscaler read: | Plane | `rgb` | `a` | | ------------------- | ------------------------------------------------ | -------------------------------- | | 0 diffuse | Diffuse radiance divided by its albedo factor | Hit distance of the first bounce | | 1 specular | Specular radiance divided by its specular factor | Hit distance of the first bounce | | 2 normal, viewZ | World-space normal of the primary hit | View-space depth | | 3 motion, roughness | `rg`: motion in NDC, `b`: perceptual roughness | Change in view depth | | 4 albedo factor | Diffuse demodulation factor | 1 for a specular primary hit | | 5 specular factor | Specular demodulation factor | 1 when the primary ray hit | | 6 emission | Light that skips demodulation | 0 | [Real time](/pathtracer/pathtracer/realtime) describes them in full, with the frame loop and the chain to a denoiser and an upscaler. ]]> \[!WARNING] Experimental. It works and is checked against our own plain path tracer, but the API can still change. At one sample per pixel, plain next-event estimation picks one light at random. With many lights, most pixels pick a light that contributes little. ReSTIR (reservoir-based spatiotemporal importance resampling) draws several candidates per pixel, keeps one in proportion to its contribution, and then reuses the kept samples of neighbouring pixels and of the previous frame. The result is far less noise per frame, at the cost of a few extra passes. - **ReSTIR DI** resamples the direct light at the first surface hit. - **ReSTIR GI** resamples the first indirect bounce: a point on another surface and the light it reflects. It runs on top of DI. Use it for real-time frames with many lights or strong indirect light. Reference renders stay on plain next-event estimation. ## Turning it on ```ts const pt = await PtMegakernel.create(device, caps, ptSceneOptions(scene, { mode: 'realtime', blueNoise, integrator: { restir: { bias: 'unbiased', spatial: 2 } }, })); ``` `integrator.restir` adds four passes before every sample dispatch: a G-buffer of the primary hits, the initial candidates, the temporal merge and the spatial merge. The shade kernel then reads the final reservoir at the first vertex and traces one shadow ray for it; deeper vertices use plain next-event estimation. One `encode` pass is one frame. It works in the frame loop of [Real time](/pathtracer/pathtracer/realtime) and in progressive accumulation, where `readImage` is the mean of the frames. Both schedulers run it. The wavefront needs the plain scheduler (no `refill`, no `layers`). ## DI options `resolveRestirOptions(options)` fills these defaults: | Option | Default | What it does | | ------------------- | ----------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `candidates` | `8` | Light candidates per pixel and frame (1 to 64). | | `temporal` | `true` | Merge the previous frame's reservoir, reprojected through the previous camera. | | `spatial` | `2` | Neighbours merged per pixel (0 to 8). | | `spatialRadius` | `16` | Radius of the neighbour disc, in pixels. | | `bias` | `'unbiased'` | How merged reservoirs are weighted. See below. | | `initialVisibility` | `true` for `'biased'`, `false` for `'unbiased'` | Trace a shadow ray for the kept candidate and drop it when occluded. | | `historyCap` | `20` | Cap on the stored history, as a multiple of `candidates`. | | `neighbours` | `'random'` | Neighbour offsets: `'random'` per pixel and frame, or one `'fixed'` spiral. | | `temporalPick` | `'random'` | The reprojected pixel: a random sub-pixel shift, or `'nearest'`. | | `normalCos` | `0.9` | A neighbour's normal must be within this cosine of the pixel's. | | `depthRel` | `0.1` | A neighbour's view depth must be within this fraction of the pixel's. | | `sharedHit` | `false` | The G-buffer pass's camera-ray hit is reused by the shade kernel: one camera ray per frame instead of two. Needs `layout.positions`; megakernel only. The `realtime` mode turns it on when it can. | | `gi` | `null` | ReSTIR GI options. See below. | ### Biased and unbiased | `bias` | How | Trade-off | | ------------ | --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `'unbiased'` | Balance-heuristic weights over every merged reservoir, each evaluated at its own pixel. | Converges to the right answer with `initialVisibility: false`. More target evaluations, more noise in shadowed regions. | | `'biased'` | Each reservoir weighted by its sample count only. | Cheaper and less noisy, but darker where neighbours disagree, and the darkening grows under temporal reuse. | `initialVisibility: true` lowers noise in shadow but makes either mode biased. For a reference-grade result use `'unbiased'` with `initialVisibility: false` (the defaults). ## GI options ```ts integrator: { restir: { gi: { bias: 'biased', spatial: 1 } } } ``` | Option | Default | What it does | | ---------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------ | | `temporal` | `true` | Merge the previous frame's GI reservoir. | | `spatial` | `1` | Neighbours merged per pixel (0 to 4). | | `spatialRadius` | `16` | Radius of the neighbour disc, in pixels. | | `bias` | `'biased'` | `'unbiased'` traces one shadow ray per pair of merged reservoirs; `'biased'` one per reused sample, and darkens where neighbours disagree. | | `historyCap` | `20` | Cap on the stored history. | | `normalCos`, `depthRel` | `0.9`, `0.1` | Neighbour tests, as for DI. | | `minDistance` | `0.05` | A sample closer than this (metres) to its own pixel's hit is not reused elsewhere. | | `neighbours`, `temporalPick` | `'random'` | As for DI. | With GI on, the path ends at its first non-mirror vertex: DI gives the direct light there and the GI sample gives one more bounce. **DI plus GI is a two-bounce estimate**, like the real-time mode's `maxDepth` 2. Compare it against a plain render with `maxDepth: 2`, not the scene's full depth. Only a sample on a Lambert surface can be reused at another pixel; glossy secondary hits stay with their own pixel. ## What it refuses `create` throws with the reason (`ptRestirUnsupported(options)` returns it) for: | Refused | Applies to | | ---------------------------------------------------------------- | -------------------------------------- | | `nee: false` | DI and GI | | `hooks` | DI and GI | | `volumes`, `spectral` | DI and GI (`resolveIntegratorOptions`) | | OpenPBR anisotropy, fuzz, thin film, or `layering: 'stochastic'` | DI and GI. Use `layering: 'albedo'`. | | `materialLodDepth: 0` with OpenPBR materials | DI and GI | | `sharedHit` without `layout.positions` | DI | | Textured scenes, alpha cutouts | GI | | `materialLod: 'secondary'` with OpenPBR materials | GI | | `envSampling: 'none'` | GI | | Alpha cutouts, `hooks`, `sharedHit`, `refill`, `layers` | DI and GI on the wavefront | Lights must not change between frames (reservoirs store light indices), and the temporal merge assumes a static scene with a moving camera. ## At run time `pt.restir` is the `PtRestirPasses` object, or `null`: | Member | What | | -------------------------- | ------------------------------------------------------------------------------------------ | | `settings`, `gi` | The resolved DI and GI options. | | `stats` | `{ bytes, gbufBytes }`: the G-buffer, reservoir and hit buffer memory at the current film. | | `dropHistory()` | After a camera cut: the next frame has no temporal neighbour. | | `reset(encoder)` | Empties the reservoirs. `pt.reset(encoder)` calls it. | | `giOverrides` | `{ temporal?, spatial? }`: switch GI reuse at run time without a new kernel. | | `wgslBytes`, `giWgslBytes` | Sizes of the DI and GI pass modules. | The previous camera is the kernel's: the one `nextFrame` or `setPrevCamera` set, else the camera of the last encoded frame. > \[!NOTE] GI reservoirs take 128 bytes per pixel and are bound as one buffer: at 1280x720 that is close to the 128 MiB storage binding limit. Render GI at a reduced size and upscale. ReSTIR DI pays off on scenes with many lights, like the [lights example](https://pmndrs.github.io/pathtracer/examples/pathtracer/lights/). A scene lit by one sun and a sky gains little. ]]> \[!WARNING] Experimental. It works and is checked against pbrt-v4 and Mitsuba 3, but the API can still change. A participating medium is a volume that absorbs and scatters light: fog, smoke, milk, murky water. `integrator.volumes` adds them to the path tracer. A medium fills the inside of a closed mesh. Light is attenuated along every ray through it, and paths can scatter inside it, with next-event estimation at the scattering point. Two kinds of media are supported: - **Homogeneous**: the same density everywhere. - **Grid**: a dense 3D density grid, looked up with trilinear interpolation. Both schedulers run media. ## Describing media The kernel takes the media as an integrator option, `PtVolumeOptions`: | Field | What | | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `media` | The media, as `PtMedium` records. Medium id = index + 1; id 0 is vacuum. | | `materials` | One `PtMaterialVolume` per scene-copy material id: which media lie on its two sides. Ids past the end have no medium. | | `cameraMedium` | The medium id the camera starts in (a homogeneous medium). Default 0. | | `shadowTracking` | Shadow transmittance through a homogeneous medium: `'analytic'` (default, `exp(-sigma_t d)`, no noise) or `'ratio'` (ratio tracking, as pbrt). Grids always use ratio tracking. | | `majorant` | Grid media: `'global'` (default) or `'brick'`, a coarser grid of local maxima that cuts wasted steps in sparse volumes. | A medium (`PtMedium`) holds `sigmaT` (extinction per metre, RGB), `albedo` (scattering over extinction, per channel), `g` (the Henyey-Greenstein mean cosine, positive scatters forward) and an optional `grid`. `ptMediumFrom(sigmaA, sigmaS, g, scale, grid)` builds one from absorption and scattering coefficients per metre, the way pbrt takes them. ## Medium boundaries A medium is bounded by a closed mesh whose triangles are wound outward. Each scene-copy material says what its surface separates (`PtMaterialVolume`): | Field | What | | ---------- | ------------------------------------------------------------------------------------- | | `interior` | Medium id on the side the normals point away from: inside the mesh. | | `exterior` | Medium id on the outside. | | `boundary` | `true`: the surface is only a medium boundary. Rays pass through it without a bounce. | A boundary surface gives no reflection and costs no path vertex. Its scene-copy material can be anything; a black one with `baseWeight: 0` and `specularWeight: 0` makes the intent clear. A surface with `boundary: false` and a medium pair is an ordinary surface that also switches media, such as a glass sphere full of milk. The medium pair belongs to the material id, so a material that borders different media on different meshes needs one scene-copy material per pair. ```ts import type { SceneCopyInput } from '@pmndrs/raytracer'; import { PtMegakernel, ptMediumFrom, ptSceneOptions, type PtVolumeOptions } from '@pmndrs/pathtracer'; // Scene-copy materials: 0 floor, 1 lamp, 2 the boundary of a fog box. const input: SceneCopyInput = { // ... meshes and instances: the fog box is a closed, outward-wound mesh with material 2 materials: [ { baseColor: [0.8, 0.8, 0.8] }, { baseColor: [0, 0, 0], specularWeight: 0, emissionColor: [20, 18, 15] }, { baseColor: [0, 0, 0], baseWeight: 0, specularWeight: 0 }, ], }; const volumes: PtVolumeOptions = { media: [ptMediumFrom([0.05, 0.05, 0.05], [0.4, 0.4, 0.4], 0.3)], materials: [ { interior: 0, exterior: 0, boundary: false }, { interior: 0, exterior: 0, boundary: false }, { interior: 1, exterior: 0, boundary: true }, ], }; const pt = await PtMegakernel.create(device, caps, ptSceneOptions(scene, { integrator: { volumes } })); ``` The integrator keeps one current medium per path, set by the last boundary crossed. Nested media work as long as each boundary names both of its sides; there is no medium stack. Shadow rays pass boundary surfaces and are blocked by every other surface. A glass sphere therefore shadows the medium inside it, as in pbrt and Mitsuba. ## Grid media A grid medium adds a `PtMediumGrid` to its `PtMedium`, and the density itself goes to `create` as `volumeGrid`: ```ts import { gridWorldToIndex, ptMediumFrom } from '@pmndrs/pathtracer'; const resolution = [64, 64, 64] as const; const density = new Float32Array(64 * 64 * 64); // x fastest, then y, then z // ... fill it ... const densityMax = density.reduce((a, b) => Math.max(a, b), 0); const bounds = [[-1, 0, -1], [1, 2, 1]] as const; // the grid's box in the mesh's object space const grid = { resolution, worldToIndex: gridWorldToIndex(resolution, bounds, objectToWorld), densityMax }; // `materials`: the PtMaterialVolume list, as in the fog box above. const pt = await PtMegakernel.create(device, caps, ptSceneOptions(scene, { integrator: { volumes: { media: [ptMediumFrom([0.1, 0.1, 0.1], [2, 2, 2], 0.5, 1, grid)], materials } }, volumeGrid: { resolution, density }, })); ``` - The density multiplies the medium's `sigmaT`. Voxel centres sit at `(i + 0.5) / n` across the bounds; outside the grid the density is 0. - `worldToIndex` is a row-major 3x4 matrix from world space to voxel index space. `gridWorldToIndex(resolution, bounds, toWorld)` builds it from the bounds in the owning mesh's object space and that mesh's object-to-world matrix (column-major, identity when omitted). - One grid per kernel: the density is one 3D texture. The camera cannot start inside a grid medium. - `majorant: 'brick'` also needs `volumeGrid.majorant = brickMajorants(resolution, density)`. The library takes a dense array only. Our bench converts OpenVDB files to one with WebVDB's `parseVdb` ([`vdb-grid.ts`](https://github.com/pmndrs/pathtracer/blob/main/bench/scenes/volumes/vdb-grid.ts)); a dense grid of a large sparse volume costs a lot of memory. ## Path rules - A scattering event in a medium counts as a path vertex for `maxDepth`, like a surface hit (pbrt's `volpath` rule). - Boundary crossings are free, up to 256 per path. - Free flight uses delta tracking against a single majorant; the phase function is sampled exactly. - Media are RGB. Use grey coefficients when comparing against a spectral renderer such as pbrt. ## What it refuses `integrator.volumes` cannot be combined with `restir`, `frame`, `primary: 'vbuffer'`, alpha cutouts, `hooks`, `counters`, `aovs`, `spectral`, or the Beer-law absorber of glass materials (`materials.features.medium`). Other OpenPBR lobes are fine. The wavefront also refuses `materialLod: 'secondary'` with media. Media add about 3 to 4 KB of WGSL. With every OpenPBR lobe the megakernel passes 50 KB: consider the [wavefront](/pathtracer/pathtracer/wavefront). The helpers behind the WGSL have TypeScript twins for tests: `deltaTrack`, `transmittance`, `gridDensity`, `hgPhase`, `hgSampleCos`, and the closed forms `slabTransmittance` and `slabSingleScatter`. ]]> \[!WARNING] Experimental. It works and is checked against pbrt-v4, but the API can still change. By default the path tracer carries three colour channels, red, green and blue. In spectral mode a path carries three wavelengths instead. Every colour a path meets is turned into a smooth spectrum and read at those wavelengths, and at the end of the path the three values are converted back to RGB through the CIE 1931 colour matching functions. The image you read back is ordinary linear RGB. Use it for two things: - **Dispersion.** Glass whose index of refraction depends on wavelength splits white light into colours. Only spectral mode can render it. - **Matching a spectral renderer.** pbrt-v4 is spectral. Saturated colours under many bounces come out a few percent different in RGB and in spectral transport; spectral mode matches pbrt there. For everything else RGB is the better default: spectral mode adds colour noise, because the three wavelengths of one path see a coloured surface differently. ## Turning it on ```ts import { defaultRgb2SpecTable, PtMegakernel, ptSceneOptions } from '@pmndrs/pathtracer'; const spectralTable = defaultRgb2SpecTable(64); // fitted once, then cached const pt = await PtMegakernel.create(device, caps, ptSceneOptions(scene, { integrator: { spectral: true }, spectralTable, })); ``` | `integrator.spectral` field | Default | What it does | | --------------------------- | ------- | -------------------------------------------------------------------------------------------- | | `res` | `64` | Resolution of the RGB-to-spectrum table per axis (10 to 128). Must match the table you pass. | | `dispersion` | `true` | Compile the wavelength-dependent IOR. `false` leaves it out of the kernel. | `spectral: true` means `{ res: 64, dispersion: true }`. Both schedulers support it. ## Wavelengths A path's first wavelength (the _hero_) is drawn from a density that favours the visible middle of the 360 to 830 nm range (pbrt-v4's `SampleVisibleWavelengths`). The other two are the same random number rotated by a third and by two thirds, so each wavelength alone has the right distribution and together they cover the spectrum evenly. pbrt uses four wavelengths per path; three fit the existing RGB path state. ## RGB to spectrum Every RGB value in the scene becomes a spectrum when a path reads it: | Value | Spectrum | | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------- | | Reflectances: base, specular, coat, sheen and transmission colours | A smooth sigmoid-polynomial spectrum (Jakob and Hanika 2019), looked up in the table. | | Conductor `eta` and `k` | The same, scaled for values above 1. | | Emission, environment, sun | The same, times the D65 illuminant, so white `(1, 1, 1)` is D65 white. | | Greys | A constant spectrum, with no table lookup. | The conversion back to RGB uses linear Rec.709 primaries, scaled so the table's D65 white is exactly `(1, 1, 1)`. A grey stays the same grey and a white furnace stays a furnace. The table is an `Rgb2SpecTable` (`{ res, data }`): | Function | What | | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- | | `buildRgb2SpecTable(res, onProgress)` | Fits a new table. It is a few seconds of CPU work at res 64; `onProgress` reports progress from 0 to 1. | | `defaultRgb2SpecTable(res)` | Fits on first use and caches the table per resolution. | | `rgb2SpecTexture(table)` | The texture layout the kernel uploads. | | `rgbToCoeffs(table, rgb)`, `sigmoidSpectrum(c, lambda)` | CPU lookups, for tests and tools. | Without the `spectralTable` option, `create` fits the default table itself. Fit it once, ideally off the main thread or ahead of time, and pass it to every kernel. `create` throws if its `res` differs from `integrator.spectral.res`. ## Dispersion A dielectric with `dispersion > 0` in its material has an IOR that follows a Cauchy law through its `ior` at 587.56 nm. `dispersion` is the `KHR_materials_dispersion` value, 20 divided by the Abbe number: ordinary glass is 0.3 to 1, dense flint about 2. `loadGltf` reads it from the extension; scene materials take it as `dispersion` (stored up to 8). Outside spectral mode it is ignored. At a dispersive refraction the path keeps only its hero wavelength: the other two are dropped and the hero's weight is tripled. Dispersive glass is therefore noisier than plain glass. | Helper | What | | ------------------------------------------------------------ | ------------------------------------------------- | | `iorAt(ior, dispersion, lambdaNm)` | The IOR at a wavelength. | | `cauchyCoefficients(ior, dispersion)` | The Cauchy `A` and `B` (B in square micrometres). | | `dispersionFromAbbe(abbe)`, `abbeFromDispersion(dispersion)` | Convert between the two conventions. | ## Limits `integrator.spectral` cannot be combined with `hooks`, `frame`, `aovs`, `restir` or `volumes`: their planes, callbacks and media are RGB. On the wavefront it also refuses `materialLod: 'secondary'`. Spectral mode is not a real-time setting. Compare it against a spectral reference, never against an RGB render: a saturated colour can differ from RGB mode by a couple of percent, and that difference is the point. ]]> \[!WARNING] Experimental. It works and is checked against its own TypeScript twin and against equivalent native materials, but the API can still change. [MaterialX](https://materialx.org/) describes a material as a graph: a surface shader whose inputs are driven by pattern nodes such as images, ramps and math. The path tracer reads `.mtlx` documents with `open_pbr_surface`, `standard_surface` or `gltf_pbr` surfaces and renders them as its own OpenPBR materials. The reader is plain TypeScript, with no MaterialX WASM module. There are two ways to render a graph whose inputs vary over the surface: | | Bake | Codegen | | ----------------------- | ------------------------------------------------------------------------------------------------------ | ------------------------------------------- | | How | Evaluate the graph into textures on the CPU | Compile the graph to WGSL inside the kernel | | Kernel | Unchanged | `integrator.materialx`, larger kernel | | What can vary | Base colour and weight, opacity, roughness, metalness, emission, normal map; anything else is averaged | Every material field the scene copy has | | Inputs a graph may read | uv only | uv, position, normal, tangent | | Precision | Limited by texture size and 8 bits | Exact per hit | | Changing a material | Re-bake the texture | Recompile the kernel | A graph whose inputs are all constant needs neither: its parameters become an ordinary material. ## Using a document ```ts import { buildMaterialX, mtlxAddToScene, mtlxKernelOptions, mtlxSyntheticImage, PtMegakernel, ptSceneOptions } from '@pmndrs/pathtracer'; import { buildGpuScene } from '@pmndrs/raytracer'; const text = await (await fetch('/materials/tiles.mtlx')).text(); const built = buildMaterialX(text, { mode: 'codegen', images: (file) => myImages.get(file) }); const use = mtlxAddToScene(input, built); // appends the material and its textures to your SceneCopyInput input.instances[2].material = use.material; // put it on a mesh (the mesh needs uvs) const scene = await buildGpuScene(device, input); const pt = await PtMegakernel.create(device, caps, ptSceneOptions(scene, mtlxKernelOptions([use]))); ``` `buildMaterialX(text, options)` parses the document and builds a scene-copy material (`material`) with the textures it needs (`textures`, decoded RGBA8), in one of three modes: | `mode` | What it builds | | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `'constants'` | The folded parameters as one record. Default when nothing in the graph varies; throws if something does. | | `'bake'` | Bakes the varying inputs into textures over the uv square (`resolution`, default 256). | | `'codegen'` | The parameters at uv (0.5, 0.5) as the record's fallback, plus one texture per image the graph reads (a 1x1 placeholder if none). Default when the graph varies. | `images(file)` returns the RGBA8 pixels of an image file, or `undefined` (the node's default then applies). `mtlxSyntheticImage` reads the procedural `gen:checker8`, `gen:bumps4`, `gen:ramp` and `gen:rings` images our test documents use. `notes` lists what was approximated. `mtlxAddToScene(input, built)` appends the material and its textures to a scene input and returns where they went. `mtlxKernelOptions(uses)` returns the overrides for `ptSceneOptions`: `integrator.materialx` with the generated WGSL for every codegen material, and `materialProbes`, so the kernel compiles any lobe a program can switch on (a graph-driven metalness needs the metal/dielectric mix even if the fallback record is not metallic). For constant and baked materials it returns no kernel options. The [MaterialX example](https://pmndrs.github.io/pathtracer/examples/pathtracer/materialx/) renders a texture graph and a position-driven graph both ways. The sections below describe the layer underneath, for tools and custom pipelines. ## Loading ```ts import { loadMtlxMaterial, mtlxConstantParams } from '@pmndrs/pathtracer'; import { openPbrToRtMaterial, type OpenPbrParams } from '@pmndrs/raytracer'; const mtl = loadMtlxMaterial(await (await fetch('/materials/brass.mtlx')).text()); console.log(Object.keys(mtl.varying)); // inputs that depend on the shading point const material = openPbrToRtMaterial(mtlxConstantParams(mtl) as OpenPbrParams); ``` `loadMtlxMaterial(source, options)` parses the document, picks a material (`options.material`, default the first) and lowers its surface to OpenPBR parameters. The result splits them into `constants` and `varying`; `images` lists the image files the varying part reads. `lengthScale` (default 0.01) converts MaterialX centimetres to our metres for `transmission_depth`. Parameters come out under OpenPBR names (`MtlxParams`), which `openPbrToRtMaterial` from `@pmndrs/raytracer` turns into a scene-copy material. `standard_surface` and `gltf_pbr` defaults that differ from OpenPBR's are applied. Inputs with no counterpart, such as subsurface radius or coat darkening, are listed in `mtl.surface.ignored`. ## Baking ```ts import { bakeMtlxMaterial, imageSampler } from '@pmndrs/pathtracer'; const baked = bakeMtlxMaterial(mtl, { resolution: 1024, sample: imageSampler((file) => images.get(file)) }); ``` `bakeMtlxMaterial(mtl, options)` evaluates the graph at the texel centres of a square over the uv range `[0, 1]`. It returns `params` (constants as authored, baked inputs at a neutral factor) and up to four `BakedTexture`s: `baseColorTexture`, `metallicRoughnessTexture`, `emissiveTexture`, `normalTexture`. Each is RGBA8 with a `colorSpace`, the shape `SceneCopyInput.textures` takes. `approximations` lists what was averaged or dropped. | Option | Default | What | | ------------ | ------- | ---------------------------------------------------------------------------------------------------------------- | | `resolution` | `256` | Side of the baked textures. | | `sample` | none | Image lookups for `image` and `tiledimage` nodes: `imageSampler(load)`, where `load(file)` returns RGBA8 pixels. | | `texels` | `true` | `false` plans only: the same parameters and texture sizes, without the pixel work. | To use the textures, add them to `SceneCopyInput.textures`, name them in `params` (`baseColorTexture`, `metallicRoughnessTexture`, `emissiveTexture`, `normalTexture`) and pass `openPbrToRtMaterial` a second argument that maps those names to texture indices. The mesh's uvs must stay inside `[0, 1]`. ## Codegen ```ts import { mtlxFallbackParams, mtlxProgramsWgsl } from '@pmndrs/pathtracer'; // The record holds the parameters at uv (0.5, 0.5); the generated function overrides them per hit. materials[3] = openPbrToRtMaterial(mtlxFallbackParams(mtl) as OpenPbrParams); // slotOf: your own lookup from an image file to its index in SceneCopyInput.textures. const programs = mtlxProgramsWgsl([{ material: 3, mtl }], { textureSlot: (request) => slotOf(request.file) }); const pt = await PtMegakernel.create(device, caps, ptSceneOptions(scene, { integrator: { materialx: { wgsl: programs.wgsl, normal: programs.normal } }, })); ``` `mtlxProgramsWgsl(programs, { textureSlot })` generates one WGSL function per material and a dispatcher the kernel calls after its own texture lookups. `textureSlot` maps each image the graph reads to its index in `SceneCopyInput.textures`, or -1 for none (the node's default then applies). `notes` lists what could not be generated. - The scene copy must have textures, because uv, position and tangent come with them. A graph that reads no image still needs one: add a 1x1 placeholder texture. - If emission varies, set the record's `emissionLuminance` to 0. The generated function emits at the hit, and an emissive record would add the surface to the light table. - An opacity that varies is seen by shading, not by alpha cutouts or shadow rays. - Both schedulers take the generated text. It composes with spectral mode (the graph runs in RGB, then the uplift). `position`, `normal` and `tangent` in a graph are world-space values at the hit. ## Supported nodes | Group | Nodes | | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Math | `add subtract multiply divide modulo power min max absval floor ceil round fract sign sin cos tan asin acos atan2 sqrt exp ln clamp invert remap smoothstep mix ifgreater ifgreatereq ifequal normalize magnitude dotproduct crossproduct luminance` | | Data | `constant dot convert combine2 combine3 combine4 extract swizzle separate2 separate3 separate4` | | Geometry | `texcoord position normal tangent`, `transformpoint transformvector transformnormal` | | Patterns | `image tiledimage place2d rotate2d ramplr ramptb splitlr splittb normalmap` | | Surfaces | `open_pbr_surface standard_surface gltf_pbr` through `surfacematerial` | Anything else throws `MtlxUnsupportedError` naming the node. Notably missing: the noise family, `checkerboard`, colour-correction and HSV nodes, matrix nodes, `bump`, `heighttonormal`, and time and frame. ## Conventions - uv `(0, 0)` is the top-left texel and v grows downward, as in glTF and the scene textures. - `file` inputs get the document's `fileprefix`. Images with `colorspace="srgb_texture"` are sRGB colour; everything else is linear data. - Address modes `periodic`, `clamp` and `mirror` map to the texture wrap; `constant` reads as clamp. Filtering is always bilinear. - The normal input must be a tangent-space `normalmap` for codegen. ## For tools `parseMaterialX`, `MtlxLowering`, `evalIr`, `evalMtlxParams`, `foldConstants` and `MTLX_NODES` expose the parser, the lowered graph and the TypeScript evaluator that the baker and the code generator share. ]]>