serge/bloodrun-editor
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
bloodrun-editor/analysis.md
serge_shubin 89b825ece5 Add L2 spherical harmonics light grid to q3map2 
Implements a new SH light grid that runs alongside the legacy Q3 light grid,
storing 9 RGB L2 spherical harmonic coefficients per grid point for accurate
directional lighting of dynamic objects from all angles.

BSP format: v47 with 19-lump header (160 bytes) when -sh is used, v46 with
17-lump header (144 bytes) otherwise. SH data stored in LUMP_LIGHTGRID_SH
(index 18) with a header containing grid bounds/size/mins followed by the
coefficient array. Stock Q3 engines read v46 lumps unchanged.

New CLI flags: -sh (enable), -gridscalesh (density multiplier, default 2x),
-gridsh (explicit cell size). SH grid receives bounced light with -bouncegrid.

Also adds libjpeg-turbo as a proper build dependency with its own vcxproj,
fixing the previous external engine path requirement.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 13:28:28 +08:00

351 lines
17 KiB
Markdown
Raw Permalink Blame History

# Source Engine 2013 (VRAD) vs Q3Map2 — Lighting System Analysis
## 1. VRAD Overview
VRAD (Valve RADiosity) is the final stage of the Source Engine map compilation pipeline (after VBSP and VVIS). It takes a compiled BSP file and embeds all precomputed lighting data into it.
### Pipeline Stages
1. **Initialization** — Load BSP, parse light entities, load `lights.rad` / `<mapname>.rad` texture-light definitions
2. **Patch Creation** — Create one patch per face, then recursively subdivide into a binary tree until patches are smaller than the "chop" size on all axes
3. **Macro Texture Setup** — Optional world-space TGA mask for lighting modulation
4. **Visibility Determination** — Per-face visibility using PVS data (or mark all visible in incremental mode)
5. **Direct Lighting** — Faces broken into triangles, partitioned into a KD-tree. For each luxel sample, ray-cast to light sources through the KD-tree. Apply attenuation (constant/linear/quadratic).
6. **Indirect Lighting (Radiosity)** — Compute form factors between patches (Fij), then use **progressive refinement** (shooting from brightest patch each iteration) rather than full matrix inversion. Default 100 bounces.
7. **Post-Processing** — Displacement lighting, detail prop lighting, leaf ambient cubes, static prop per-vertex lighting
8. **Export** — Write lightmaps to BSP lighting lump
### Core Radiosity Equation
```
Bi = Ei + Pi * SUM_j(Bj * Fij)
```
Where Bi = radiosity of patch i, Ei = emissivity, Pi = reflectivity, Fij = form factor (proportion of light from j reaching i).
VRAD uses **progressive refinement** for memory efficiency — it does not store a full NxN form factor matrix.
---
## 2. What Source Engine Bakes
### 2.1 Lightmaps
- **Luxels** (lighting pixels) form a grid on every brush face
- Resolution controlled by "Lightmap Scale" per face (default 16 units per luxel; can be 2, 4, 8, 16, 32, 64, 128)
- **Storage format**: `ColorRGBExp32` — 4 bytes per sample (R, G, B as bytes + signed exponent byte). Actual color = (R, G, B) * 2^exponent. This gives HDR range from the storage format itself.
- Lightmaps are packed into pages in the BSP lighting lump
- Up to **4 light styles** per face (each style gets its own lightmap layer)
- **Bumpmapped surfaces store 4x lightmap samples**: 3 directional basis lightmaps + 1 flat (non-bumped) fallback
### 2.2 Ambient Cubes (CompressedLightCube)
- Stored per **visleaf** (not per-face) — a sparse set of sample points in each leaf
- Each ambient cube = **6 ColorRGBExp32 values** (one per cardinal direction: +X, -X, +Y, -Y, +Z, -Z)
- Used to light **dynamic objects** (players, physics props, prop_dynamic) that don't have lightmaps
- Evaluation: weighted blend of 6 RGB values based on the world-space normal of the receiving surface
- VRAD distributes samples somewhat evenly across leaves; smaller leaves get denser coverage
### 2.3 Light Grid (comparison to Q3)
Source does NOT use a uniform 3D grid like Quake 3. Instead, it uses the **per-leaf ambient cube** system described above. This is more memory-efficient but less granular — dynamic objects in large leaves get coarser lighting approximation.
### 2.4 Static Prop Per-Vertex Lighting
- Enabled via `-StaticPropLighting` VRAD flag
- For each vertex of each `prop_static`, VRAD computes a color+brightness value
- The engine interpolates between vertices at runtime to create smooth gradients
- Without this flag, static props use **origin-based lighting** (single sample at the prop's origin, applied uniformly)
- Static props can also opt into **bounced lighting** via keyvalue "Enable Bounced Lighting"
- `-StaticPropPolys` treats prop triangles as occluders for shadow casting
### 2.5 Bounced/Indirect Lighting (Radiosity)
- Faces subdivided into **patches** (binary tree structure)
- `CPatch` stores: spatial bounds, winding polygon, normal (phong-interpolated), accumulated light (totallight, directlight, samplelight), reflectivity, area
- **Samples** (one per luxel) are distinct from patches — samples map to luxel positions, patches accumulate radiosity
- Form factors computed between patch pairs based on visibility and geometry
- Progressive refinement shoots light from highest-energy patch each iteration
- Default bounce count: 100
- `-final` flag increases quality by casting more rays for sky and indirect light
---
## 3. Light Entity Types in Source Engine
### 3.1 `light` (Point Light)
- Omnidirectional point source
- Static (baked into lightmaps by VRAD)
- Parameters: color, brightness, constant/linear/quadratic attenuation
- Default falloff: pure inverse-square (constant=0, linear=0, quadratic=1)
- Can be named (makes it switchable, adds second lightmap page)
### 3.2 `light_spot` (Spotlight)
- Directional cone light with **inner cone** (full brightness) and **outer cone** (falloff to zero)
- Same attenuation model as `light`
- Static, baked by VRAD
### 3.3 `light_environment` (Sun/Sky)
- Defines **two lighting components**:
- **Brightness**: Direct sunlight (directional, parallel rays, specific color/intensity)
- **Ambient**: Diffuse skylight (omnidirectional fill, separate color/intensity)
- Direction set by entity's pitch/yaw
- Only one per map (last one wins)
- Constant falloff (parallel rays, no distance attenuation)
- Sky light enters through `tools/toolsskybox` surfaces
### 3.4 `light_dynamic`
- Calculated at runtime, NOT baked
- Casts **dlights** on brushes, **elights** on models
- Can be toggled on/off, can track entities
- Limit: **3 dynamic lights per model** simultaneously
- Cannot cast shadows
- Higher performance cost than static lights
### 3.5 `env_projectedtexture`
- Projects a texture as a spotlight with FOV control
- Fully dynamic, can move
- Only **1 active projected texture** at a time (engine limitation in most branches)
- Used for flashlights, dynamic shadows
### 3.6 `env_cascade_light` (CS:GO+)
- Cascaded shadow maps from sky
- 3 detail levels across configurable distance
- Cannot coexist with projected textures
---
## 4. Advanced Features
### 4.1 HDR Lighting Pipeline
- VRAD compiles **separate LDR and HDR lighting data** (flag `-both`, `-hdr`, or `-ldr`)
- With `-both`, VRAD runs **twice** (once for each mode)
- `ColorRGBExp32` format inherently supports HDR via the exponent byte (range far exceeds 0-255)
- At runtime, the engine performs **tone mapping**: bloom on colors above 100% brightness, virtual camera aperture adjustment for over-exposure
- `mat_hdr_level` controls HDR mode at runtime
- Left 4 Dead branch and later: single compile pass for both modes
### 4.2 Bump-Mapped Lightmaps (Radiosity Normal Mapping)
This is one of Source's most distinctive features, and it is **entirely absent from q3map2**.
- For bumpmapped surfaces, VRAD stores **3 directional lightmaps** plus 1 flat fallback (4x memory)
- The 3 lightmaps correspond to the **HL2 basis vectors** in tangent space:
```
basis0 = ( 0.816497, 0.0, 0.57735)
basis1 = (-0.408248, 0.707107, 0.57735)
basis2 = (-0.408248, -0.707107, 0.57735)
```
- These 3 vectors point roughly 120 degrees apart in the tangent plane, tilted 30 degrees up from the surface
- At runtime, the shader combines the 3 lightmaps weighted by the normal map:
```hlsl
diffuse = normal.x * lightmap0 + normal.y * lightmap1 + normal.z * lightmap2
```
(where normal is transformed into the RNM basis)
- This allows **per-pixel lighting variation from baked lightmaps** — bumps catch light directionally
### 4.3 Self-Shadowed Bumpmaps
- Chris Green's SIGGRAPH 2007 paper: "Efficient Self-Shadowed Radiosity Normal Mapping"
- Adds **directional occlusion** to bump maps at no extra texture memory cost
- When bump data comes from height maps, standard RNM only shows orientation effects; self-shadowing adds occlusion cues where bumps block light
- Actually **faster** than the non-shadowing solution due to optimized shader paths
- The directional basis vectors encode both lighting AND shadowing information
### 4.4 Light Styles (Switchable/Animated Lights)
- A light gets a "style" by: giving it a **targetname**, setting a **style index**, or setting a **pattern string**
- Switchable lights compile **two lightmap layers** (on/off states) per affected face
- Engine blends between layers at runtime
- Hard limit: **4 light styles per face**, **32 total lightmap pages**
- Animated patterns use character strings where 'a'=dark, 'z'=bright (e.g., "mmmaaammmaaam" for flickering)
- Exceeding limits causes "Too many light styles on a face" warning
### 4.5 Texture Light Emission (Surface Lights)
- Defined in `lights.rad` or `<mapname>.rad` files
- Format: `<texture_name> <R> <G> <B> <brightness>`
- Every luxel of a brush face using that texture emits light during VRAD compilation
- The emitting surface is NOT self-lit by its own emission (needs `$selfillum` or `UnlitGeneric` shader separately)
- Map-specific `.rad` overrides global `lights.rad`
- Conceptually similar to q3map2's `q3map_surfacelight` shader directive
### 4.6 Lightmap Filtering and Anti-Aliasing
- VRAD applies filtering to lightmap edges
- Supersampling available via quality settings
- `-final` flag dramatically increases ray count for smoother indirect lighting
- No explicit `-samples` or `-filter` flags like q3map2; quality is controlled via `-extra` / `-final` presets
### 4.7 Macro Textures
- Optional world-space TGA file (matching BSP name) modulates lighting globally
- Maps world bounds onto texture coordinates
- Alpha channel controls per-luxel darkening (255 = no effect, 0 = full shadow)
- Used for large-scale ambient occlusion or artistic lighting control
### 4.8 Ambient Occlusion
- Source Engine does NOT have a dedicated AO pass in VRAD (unlike q3map2's `-dirty` flag)
- AO-like effects emerge naturally from radiosity bounces and the self-shadowed RNM system
- Some Source Engine branches add screen-space AO at runtime
---
## 5. Feature Comparison Table
### Features Source Has That Q3Map2 Doesn't
| Feature | Source Engine (VRAD) | Q3Map2 | Gap Severity |
|---------|---------------------|--------|--------------|
| **HDR lightmaps** | `ColorRGBExp32` (4 bytes, HDR via exponent) | 24-bit RGB, clamped 0-255 | **Major** |
| **Bump-mapped lightmaps (RNM)** | 3 directional basis lightmaps + 1 flat per bumped face | Single flat lightmap only | **Major** |
| **Self-shadowed bumpmaps** | Directional occlusion baked into RNM basis | None | **Major** |
| **Ambient cubes** | 6-axis `CompressedLightCube` per leaf for dynamic objects | Uniform 3D grid (direction + ambient) | Medium |
| **True progressive radiosity** | Form-factor based, patch-to-patch energy transfer | Fake radiosity (spawn point lights at hit surfaces) | Medium |
| **Static prop per-vertex lighting** | VRAD traces light to each vertex of static models | No model-aware lighting | Medium |
| **Hard-falloff / smoothstep lights** | Start/end distance with smoothstep fade | No equivalent | Small |
| **CLQ attenuation model** | Full Constant/Linear/Quadratic per light | Simpler falloff | Small |
| **Macro textures** | World-space TGA modulates lighting globally | None | Small |
### Features Q3Map2 Already Has That Match or Exceed Source
| Feature | Q3Map2 | Source |
|---------|--------|--------|
| **Ambient occlusion** | Explicit `-dirty` flag with configurable depth/samples | No dedicated AO — emerges from bounces |
| **Deluxe maps** | Stores average light direction per luxel | Similar concept via RNM basis |
| **Floodlight** | Fills dark areas with ambient fill | No equivalent |
| **Light styles** | Basic support exists | More mature (4 layers, runtime blending) |
| **Surface lights** | `q3map_surfacelight` shader directive | `lights.rad` file — functionally equivalent |
| **Sun/sky system** | `_sun` entity + sky shaders with deviance jitter | `light_environment` — similar capability |
---
## 6. The RNM Basis — Source's Key Innovation
Source stores **3 directional lightmaps** per bumpmapped face using these tangent-space basis vectors:
```
basis0 = ( 0.8165, 0.0000, 0.5774) // 120° apart in tangent plane,
basis1 = (-0.4082, 0.7071, 0.5774) // tilted 30° up from surface
basis2 = (-0.4082, -0.7071, 0.5774)
```
The shader recombines at runtime:
```hlsl
diffuse = dot(n, b0) * lm0 + dot(n, b1) * lm1 + dot(n, b2) * lm2
```
This gives **per-pixel lighting variation from baked data**. This is the single biggest visual quality difference between Source and Q3-engine games.
---
## 7. Q3Map2 Lighting System Architecture (Current State)
### 5 Source Files (~340K total)
| File | Size | Role |
|------|------|------|
| `light.cpp` | 82K | Entry point (`LightMain`), entity lights, core sample tracing |
| `light_ydnar.cpp` | 106K | Lightmap mapping, illumination, dirt, floodlight |
| `lightmaps_ydnar.cpp` | 89K | Lightmap allocation, packing, storage |
| `light_bounce.cpp` | 25K | Radiosity / bounce lighting |
| `light_trace.cpp` | 39K | Ray tracing through BSP (shadow tests) |
### Pipeline Flow (orchestrated by `LightMain()`)
```
1. CreateEntityLights() — parse light entities from BSP
2. CreateSurfaceLights() — create area/sun/sky lights from shaders
3. SetupEnvelopes() — compute bounding boxes for light culling
4. SetupGrid() + TraceGrid() — light the 3D sample grid [threaded]
5. MapRawLightmap() — project surfaces onto lightmap UV [threaded]
6. DirtyRawLightmap() — ambient occlusion / dirt pass [threaded]
7. FloodLightRawLightmap() — fill dark areas with floodlight [threaded]
8. IlluminateRawLightmap() — trace all lights to luxels [threaded]
9. IlluminateVertexes() — light non-lightmapped surfaces [threaded]
10. Bounce loop (if bounce > 0):
└── RadCreateDiffuseLights() → SetupEnvelopes() → re-illuminate
11. StoreSurfaceLightmaps() — pack luxels into final lightmap pages
```
### Key Data Structures (in `q3map2.h`)
- **`light_t`** — light source (type, origin, color, photons, envelope, flags)
- **`ELightType`** — Point, Area, Spotlight, Sun
- **`LightFlags`** — AttenLinear, AttenAngle, Negative, Dark, Grid, Surfaces, Fast...
- **`trace_t`** — ray trace state (origin, direction, hit info, accumulated color)
- **`rawLightmap_t`** — working lightmap with luxel arrays, floodlight, ambient
- **`outLightmap_t`** — final packed lightmap page
- **`sun_t`** / **`skylight_t`** — sun and sky light definitions
### Core Tracing Function
`LightContributionToSample(trace_t*)` at `light.cpp:777` — the inner loop that calculates one light's contribution to one sample point. Handles all light types, attenuation modes, and shadow rays via `TraceLine()`.
### Threading
Uses `std::thread` with work-stealing dispatch (`RunThreadsOnIndividual`). All heavy passes (grid, mapping, dirt, illumination, vertex, floodlight, radiosity) run multi-threaded.
### Post-Processing Pipeline
`ColorToBytes()` at `light_ydnar.cpp:62`: Brightness → Contrast → Gamma → Exposure → Saturation → Clamp
---
## 8. Proposed Feature Roadmap (Priority Order)
### Phase 1: HDR Lightmaps
Foundation for everything else. Switch from `byte[3]` to `ColorRGBExp32` or `half[3]` storage. Requires BSP format extension + renderer support.
### Phase 2: Radiosity Normal Mapping (Bump-mapped Lightmaps)
The biggest visual win. Requires:
- Computing tangent-space basis vectors per face
- Tracing 3 directional lightmaps per bumped surface
- 4x lightmap storage for bumped faces
- Shader-side recombination with normal maps
### Phase 3: True Progressive Refinement Radiosity
Replace q3map2's "fake bounce" (spawning point lights at hit surfaces) with proper patch-based energy transfer. More physically correct indirect lighting.
### Phase 4: Ambient Cubes / Enhanced Light Grid
Upgrade the 3D light grid to store 6-axis directional ambient (one color per cardinal direction) for better dynamic object lighting.
### Phase 5: Hard-Falloff Lights / CLQ Attenuation
Add smoothstep fade between start/end distances, and full constant/linear/quadratic attenuation control per light.
### Phase 6: Static Prop Per-Vertex Lighting
Trace light to model vertices for baked model illumination.
---
## 9. References
- [How VRAD Works (gpurad technical document)](https://github.com/x6herbius/gpurad/blob/master/How%20VRAD%20works.md)
- [Source Lighting Technical Analysis: Part One (Mapcore)](https://www.mapcore.org/articles/development/source-lighting-technical-analysis-part-one-r65/)
- [Source Lighting Technical Analysis: Part Two (Mapcore)](https://www.mapcore.org/articles/development/source-lighting-technical-analysis-part-two-r66/)
- [Shading in Valve's Source Engine (SIGGRAPH 2006 PDF)](https://cdn.cloudflare.steamstatic.com/apps/valve/2006/SIGGRAPH06_Course_ShadingInValvesSourceEngine.pdf)
- [Efficient Self-Shadowed Radiosity Normal Mapping (SIGGRAPH 2007, Chris Green, Valve)](https://cdn.fastly.steamstatic.com/apps/valve/2007/SIGGRAPH2007_EfficientSelfShadowedRadiosityNormalMapping.pdf)
- [VRAD - Valve Developer Community](https://developer.valvesoftware.com/wiki/VRAD)
- [HDR Lighting Basics - Valve Developer Community](https://developer.valvesoftware.com/wiki/HDR_Lighting_Basics)
- [Advanced Lighting - Valve Developer Community](https://developer.valvesoftware.com/wiki/Advanced_Lighting)
- [BSP (Source) format - Valve Developer Community](https://developer.valvesoftware.com/wiki/BSP_(Source))
Reference in a new issue View git blame Copy permalink