From 6f78fcb4528626d87d56725e73b77b915061b020 Mon Sep 17 00:00:00 2001 From: serge_shubin Date: Fri, 3 Apr 2026 13:52:07 +0800 Subject: [PATCH] Add L2 spherical harmonics light grid to q3map2 Implements a new SH light grid that runs alongside the legacy Q3 light grid, storing 9 RGB L2 spherical harmonic coefficients per grid point for accurate directional lighting of dynamic objects from all angles. BSP format: v47 with 19-lump header (160 bytes) when -sh is used, v46 with 17-lump header (144 bytes) otherwise. SH data stored in LUMP_LIGHTGRID_SH (index 18) with a header containing grid bounds/size/mins followed by the coefficient array. Stock Q3 engines read v46 lumps unchanged. New CLI flags: -sh (enable), -gridscalesh (density multiplier, default 2x), -gridsh (explicit cell size). SH grid receives bounced light with -bouncegrid. Also adds libjpeg-turbo as a proper build dependency with its own vcxproj, fixing the previous external engine path requirement. --- analysis.md | 351 ++++++++++++++ makefiles/Props/CommonVariables.props | 1 + makefiles/Props/Project_libjpeg-turbo.props | 9 + makefiles/Props/Project_q3map2.props | 2 +- makefiles/editor.sln | 104 ++-- makefiles/libjpeg-turbo.vcxproj | 65 +++ makefiles/q3map2.vcxproj | 1 - q3map2/q3map2/bspfile_abstract.cpp | 3 + q3map2/q3map2/bspfile_ibsp.cpp | 190 +++++--- q3map2/q3map2/convert_json.cpp | 44 ++ q3map2/q3map2/light.cpp | 201 ++++++++ q3map2/q3map2/q3map2.h | 25 + q3map2/q3map2/sh.h | 138 ++++++ sh.md | 505 ++++++++++++++++++++ 14 files changed, 1534 insertions(+), 105 deletions(-) create mode 100644 analysis.md create mode 100644 makefiles/Props/Project_libjpeg-turbo.props create mode 100644 makefiles/libjpeg-turbo.vcxproj create mode 100644 q3map2/q3map2/sh.h create mode 100644 sh.md diff --git a/analysis.md b/analysis.md new file mode 100644 index 0000000..0d30737 --- /dev/null +++ b/analysis.md @@ -0,0 +1,351 @@ +# Source Engine 2013 (VRAD) vs Q3Map2 — Lighting System Analysis + +## 1. VRAD Overview + +VRAD (Valve RADiosity) is the final stage of the Source Engine map compilation pipeline (after VBSP and VVIS). It takes a compiled BSP file and embeds all precomputed lighting data into it. + +### Pipeline Stages + +1. **Initialization** — Load BSP, parse light entities, load `lights.rad` / `.rad` texture-light definitions +2. **Patch Creation** — Create one patch per face, then recursively subdivide into a binary tree until patches are smaller than the "chop" size on all axes +3. **Macro Texture Setup** — Optional world-space TGA mask for lighting modulation +4. **Visibility Determination** — Per-face visibility using PVS data (or mark all visible in incremental mode) +5. **Direct Lighting** — Faces broken into triangles, partitioned into a KD-tree. For each luxel sample, ray-cast to light sources through the KD-tree. Apply attenuation (constant/linear/quadratic). +6. **Indirect Lighting (Radiosity)** — Compute form factors between patches (Fij), then use **progressive refinement** (shooting from brightest patch each iteration) rather than full matrix inversion. Default 100 bounces. +7. **Post-Processing** — Displacement lighting, detail prop lighting, leaf ambient cubes, static prop per-vertex lighting +8. **Export** — Write lightmaps to BSP lighting lump + +### Core Radiosity Equation + +``` +Bi = Ei + Pi * SUM_j(Bj * Fij) +``` + +Where Bi = radiosity of patch i, Ei = emissivity, Pi = reflectivity, Fij = form factor (proportion of light from j reaching i). + +VRAD uses **progressive refinement** for memory efficiency — it does not store a full NxN form factor matrix. + +--- + +## 2. What Source Engine Bakes + +### 2.1 Lightmaps + +- **Luxels** (lighting pixels) form a grid on every brush face +- Resolution controlled by "Lightmap Scale" per face (default 16 units per luxel; can be 2, 4, 8, 16, 32, 64, 128) +- **Storage format**: `ColorRGBExp32` — 4 bytes per sample (R, G, B as bytes + signed exponent byte). Actual color = (R, G, B) * 2^exponent. This gives HDR range from the storage format itself. +- Lightmaps are packed into pages in the BSP lighting lump +- Up to **4 light styles** per face (each style gets its own lightmap layer) +- **Bumpmapped surfaces store 4x lightmap samples**: 3 directional basis lightmaps + 1 flat (non-bumped) fallback + +### 2.2 Ambient Cubes (CompressedLightCube) + +- Stored per **visleaf** (not per-face) — a sparse set of sample points in each leaf +- Each ambient cube = **6 ColorRGBExp32 values** (one per cardinal direction: +X, -X, +Y, -Y, +Z, -Z) +- Used to light **dynamic objects** (players, physics props, prop_dynamic) that don't have lightmaps +- Evaluation: weighted blend of 6 RGB values based on the world-space normal of the receiving surface +- VRAD distributes samples somewhat evenly across leaves; smaller leaves get denser coverage + +### 2.3 Light Grid (comparison to Q3) + +Source does NOT use a uniform 3D grid like Quake 3. Instead, it uses the **per-leaf ambient cube** system described above. This is more memory-efficient but less granular — dynamic objects in large leaves get coarser lighting approximation. + +### 2.4 Static Prop Per-Vertex Lighting + +- Enabled via `-StaticPropLighting` VRAD flag +- For each vertex of each `prop_static`, VRAD computes a color+brightness value +- The engine interpolates between vertices at runtime to create smooth gradients +- Without this flag, static props use **origin-based lighting** (single sample at the prop's origin, applied uniformly) +- Static props can also opt into **bounced lighting** via keyvalue "Enable Bounced Lighting" +- `-StaticPropPolys` treats prop triangles as occluders for shadow casting + +### 2.5 Bounced/Indirect Lighting (Radiosity) + +- Faces subdivided into **patches** (binary tree structure) +- `CPatch` stores: spatial bounds, winding polygon, normal (phong-interpolated), accumulated light (totallight, directlight, samplelight), reflectivity, area +- **Samples** (one per luxel) are distinct from patches — samples map to luxel positions, patches accumulate radiosity +- Form factors computed between patch pairs based on visibility and geometry +- Progressive refinement shoots light from highest-energy patch each iteration +- Default bounce count: 100 +- `-final` flag increases quality by casting more rays for sky and indirect light + +--- + +## 3. Light Entity Types in Source Engine + +### 3.1 `light` (Point Light) + +- Omnidirectional point source +- Static (baked into lightmaps by VRAD) +- Parameters: color, brightness, constant/linear/quadratic attenuation +- Default falloff: pure inverse-square (constant=0, linear=0, quadratic=1) +- Can be named (makes it switchable, adds second lightmap page) + +### 3.2 `light_spot` (Spotlight) + +- Directional cone light with **inner cone** (full brightness) and **outer cone** (falloff to zero) +- Same attenuation model as `light` +- Static, baked by VRAD + +### 3.3 `light_environment` (Sun/Sky) + +- Defines **two lighting components**: + - **Brightness**: Direct sunlight (directional, parallel rays, specific color/intensity) + - **Ambient**: Diffuse skylight (omnidirectional fill, separate color/intensity) +- Direction set by entity's pitch/yaw +- Only one per map (last one wins) +- Constant falloff (parallel rays, no distance attenuation) +- Sky light enters through `tools/toolsskybox` surfaces + +### 3.4 `light_dynamic` + +- Calculated at runtime, NOT baked +- Casts **dlights** on brushes, **elights** on models +- Can be toggled on/off, can track entities +- Limit: **3 dynamic lights per model** simultaneously +- Cannot cast shadows +- Higher performance cost than static lights + +### 3.5 `env_projectedtexture` + +- Projects a texture as a spotlight with FOV control +- Fully dynamic, can move +- Only **1 active projected texture** at a time (engine limitation in most branches) +- Used for flashlights, dynamic shadows + +### 3.6 `env_cascade_light` (CS:GO+) + +- Cascaded shadow maps from sky +- 3 detail levels across configurable distance +- Cannot coexist with projected textures + +--- + +## 4. Advanced Features + +### 4.1 HDR Lighting Pipeline + +- VRAD compiles **separate LDR and HDR lighting data** (flag `-both`, `-hdr`, or `-ldr`) +- With `-both`, VRAD runs **twice** (once for each mode) +- `ColorRGBExp32` format inherently supports HDR via the exponent byte (range far exceeds 0-255) +- At runtime, the engine performs **tone mapping**: bloom on colors above 100% brightness, virtual camera aperture adjustment for over-exposure +- `mat_hdr_level` controls HDR mode at runtime +- Left 4 Dead branch and later: single compile pass for both modes + +### 4.2 Bump-Mapped Lightmaps (Radiosity Normal Mapping) + +This is one of Source's most distinctive features, and it is **entirely absent from q3map2**. + +- For bumpmapped surfaces, VRAD stores **3 directional lightmaps** plus 1 flat fallback (4x memory) +- The 3 lightmaps correspond to the **HL2 basis vectors** in tangent space: + +``` +basis0 = ( 0.816497, 0.0, 0.57735) +basis1 = (-0.408248, 0.707107, 0.57735) +basis2 = (-0.408248, -0.707107, 0.57735) +``` + +- These 3 vectors point roughly 120 degrees apart in the tangent plane, tilted 30 degrees up from the surface +- At runtime, the shader combines the 3 lightmaps weighted by the normal map: + +```hlsl +diffuse = normal.x * lightmap0 + normal.y * lightmap1 + normal.z * lightmap2 +``` + +(where normal is transformed into the RNM basis) + +- This allows **per-pixel lighting variation from baked lightmaps** — bumps catch light directionally + +### 4.3 Self-Shadowed Bumpmaps + +- Chris Green's SIGGRAPH 2007 paper: "Efficient Self-Shadowed Radiosity Normal Mapping" +- Adds **directional occlusion** to bump maps at no extra texture memory cost +- When bump data comes from height maps, standard RNM only shows orientation effects; self-shadowing adds occlusion cues where bumps block light +- Actually **faster** than the non-shadowing solution due to optimized shader paths +- The directional basis vectors encode both lighting AND shadowing information + +### 4.4 Light Styles (Switchable/Animated Lights) + +- A light gets a "style" by: giving it a **targetname**, setting a **style index**, or setting a **pattern string** +- Switchable lights compile **two lightmap layers** (on/off states) per affected face +- Engine blends between layers at runtime +- Hard limit: **4 light styles per face**, **32 total lightmap pages** +- Animated patterns use character strings where 'a'=dark, 'z'=bright (e.g., "mmmaaammmaaam" for flickering) +- Exceeding limits causes "Too many light styles on a face" warning + +### 4.5 Texture Light Emission (Surface Lights) + +- Defined in `lights.rad` or `.rad` files +- Format: ` ` +- Every luxel of a brush face using that texture emits light during VRAD compilation +- The emitting surface is NOT self-lit by its own emission (needs `$selfillum` or `UnlitGeneric` shader separately) +- Map-specific `.rad` overrides global `lights.rad` +- Conceptually similar to q3map2's `q3map_surfacelight` shader directive + +### 4.6 Lightmap Filtering and Anti-Aliasing + +- VRAD applies filtering to lightmap edges +- Supersampling available via quality settings +- `-final` flag dramatically increases ray count for smoother indirect lighting +- No explicit `-samples` or `-filter` flags like q3map2; quality is controlled via `-extra` / `-final` presets + +### 4.7 Macro Textures + +- Optional world-space TGA file (matching BSP name) modulates lighting globally +- Maps world bounds onto texture coordinates +- Alpha channel controls per-luxel darkening (255 = no effect, 0 = full shadow) +- Used for large-scale ambient occlusion or artistic lighting control + +### 4.8 Ambient Occlusion + +- Source Engine does NOT have a dedicated AO pass in VRAD (unlike q3map2's `-dirty` flag) +- AO-like effects emerge naturally from radiosity bounces and the self-shadowed RNM system +- Some Source Engine branches add screen-space AO at runtime + +--- + +## 5. Feature Comparison Table + +### Features Source Has That Q3Map2 Doesn't + +| Feature | Source Engine (VRAD) | Q3Map2 | Gap Severity | +|---------|---------------------|--------|--------------| +| **HDR lightmaps** | `ColorRGBExp32` (4 bytes, HDR via exponent) | 24-bit RGB, clamped 0-255 | **Major** | +| **Bump-mapped lightmaps (RNM)** | 3 directional basis lightmaps + 1 flat per bumped face | Single flat lightmap only | **Major** | +| **Self-shadowed bumpmaps** | Directional occlusion baked into RNM basis | None | **Major** | +| **Ambient cubes** | 6-axis `CompressedLightCube` per leaf for dynamic objects | Uniform 3D grid (direction + ambient) | Medium | +| **True progressive radiosity** | Form-factor based, patch-to-patch energy transfer | Fake radiosity (spawn point lights at hit surfaces) | Medium | +| **Static prop per-vertex lighting** | VRAD traces light to each vertex of static models | No model-aware lighting | Medium | +| **Hard-falloff / smoothstep lights** | Start/end distance with smoothstep fade | No equivalent | Small | +| **CLQ attenuation model** | Full Constant/Linear/Quadratic per light | Simpler falloff | Small | +| **Macro textures** | World-space TGA modulates lighting globally | None | Small | + +### Features Q3Map2 Already Has That Match or Exceed Source + +| Feature | Q3Map2 | Source | +|---------|--------|--------| +| **Ambient occlusion** | Explicit `-dirty` flag with configurable depth/samples | No dedicated AO — emerges from bounces | +| **Deluxe maps** | Stores average light direction per luxel | Similar concept via RNM basis | +| **Floodlight** | Fills dark areas with ambient fill | No equivalent | +| **Light styles** | Basic support exists | More mature (4 layers, runtime blending) | +| **Surface lights** | `q3map_surfacelight` shader directive | `lights.rad` file — functionally equivalent | +| **Sun/sky system** | `_sun` entity + sky shaders with deviance jitter | `light_environment` — similar capability | + +--- + +## 6. The RNM Basis — Source's Key Innovation + +Source stores **3 directional lightmaps** per bumpmapped face using these tangent-space basis vectors: + +``` +basis0 = ( 0.8165, 0.0000, 0.5774) // 120° apart in tangent plane, +basis1 = (-0.4082, 0.7071, 0.5774) // tilted 30° up from surface +basis2 = (-0.4082, -0.7071, 0.5774) +``` + +The shader recombines at runtime: + +```hlsl +diffuse = dot(n, b0) * lm0 + dot(n, b1) * lm1 + dot(n, b2) * lm2 +``` + +This gives **per-pixel lighting variation from baked data**. This is the single biggest visual quality difference between Source and Q3-engine games. + +--- + +## 7. Q3Map2 Lighting System Architecture (Current State) + +### 5 Source Files (~340K total) + +| File | Size | Role | +|------|------|------| +| `light.cpp` | 82K | Entry point (`LightMain`), entity lights, core sample tracing | +| `light_ydnar.cpp` | 106K | Lightmap mapping, illumination, dirt, floodlight | +| `lightmaps_ydnar.cpp` | 89K | Lightmap allocation, packing, storage | +| `light_bounce.cpp` | 25K | Radiosity / bounce lighting | +| `light_trace.cpp` | 39K | Ray tracing through BSP (shadow tests) | + +### Pipeline Flow (orchestrated by `LightMain()`) + +``` +1. CreateEntityLights() — parse light entities from BSP +2. CreateSurfaceLights() — create area/sun/sky lights from shaders +3. SetupEnvelopes() — compute bounding boxes for light culling +4. SetupGrid() + TraceGrid() — light the 3D sample grid [threaded] +5. MapRawLightmap() — project surfaces onto lightmap UV [threaded] +6. DirtyRawLightmap() — ambient occlusion / dirt pass [threaded] +7. FloodLightRawLightmap() — fill dark areas with floodlight [threaded] +8. IlluminateRawLightmap() — trace all lights to luxels [threaded] +9. IlluminateVertexes() — light non-lightmapped surfaces [threaded] +10. Bounce loop (if bounce > 0): + └── RadCreateDiffuseLights() → SetupEnvelopes() → re-illuminate +11. StoreSurfaceLightmaps() — pack luxels into final lightmap pages +``` + +### Key Data Structures (in `q3map2.h`) + +- **`light_t`** — light source (type, origin, color, photons, envelope, flags) +- **`ELightType`** — Point, Area, Spotlight, Sun +- **`LightFlags`** — AttenLinear, AttenAngle, Negative, Dark, Grid, Surfaces, Fast... +- **`trace_t`** — ray trace state (origin, direction, hit info, accumulated color) +- **`rawLightmap_t`** — working lightmap with luxel arrays, floodlight, ambient +- **`outLightmap_t`** — final packed lightmap page +- **`sun_t`** / **`skylight_t`** — sun and sky light definitions + +### Core Tracing Function + +`LightContributionToSample(trace_t*)` at `light.cpp:777` — the inner loop that calculates one light's contribution to one sample point. Handles all light types, attenuation modes, and shadow rays via `TraceLine()`. + +### Threading + +Uses `std::thread` with work-stealing dispatch (`RunThreadsOnIndividual`). All heavy passes (grid, mapping, dirt, illumination, vertex, floodlight, radiosity) run multi-threaded. + +### Post-Processing Pipeline + +`ColorToBytes()` at `light_ydnar.cpp:62`: Brightness → Contrast → Gamma → Exposure → Saturation → Clamp + +--- + +## 8. Proposed Feature Roadmap (Priority Order) + +### Phase 1: HDR Lightmaps + +Foundation for everything else. Switch from `byte[3]` to `ColorRGBExp32` or `half[3]` storage. Requires BSP format extension + renderer support. + +### Phase 2: Radiosity Normal Mapping (Bump-mapped Lightmaps) + +The biggest visual win. Requires: +- Computing tangent-space basis vectors per face +- Tracing 3 directional lightmaps per bumped surface +- 4x lightmap storage for bumped faces +- Shader-side recombination with normal maps + +### Phase 3: True Progressive Refinement Radiosity + +Replace q3map2's "fake bounce" (spawning point lights at hit surfaces) with proper patch-based energy transfer. More physically correct indirect lighting. + +### Phase 4: Ambient Cubes / Enhanced Light Grid + +Upgrade the 3D light grid to store 6-axis directional ambient (one color per cardinal direction) for better dynamic object lighting. + +### Phase 5: Hard-Falloff Lights / CLQ Attenuation + +Add smoothstep fade between start/end distances, and full constant/linear/quadratic attenuation control per light. + +### Phase 6: Static Prop Per-Vertex Lighting + +Trace light to model vertices for baked model illumination. + +--- + +## 9. References + +- [How VRAD Works (gpurad technical document)](https://github.com/x6herbius/gpurad/blob/master/How%20VRAD%20works.md) +- [Source Lighting Technical Analysis: Part One (Mapcore)](https://www.mapcore.org/articles/development/source-lighting-technical-analysis-part-one-r65/) +- [Source Lighting Technical Analysis: Part Two (Mapcore)](https://www.mapcore.org/articles/development/source-lighting-technical-analysis-part-two-r66/) +- [Shading in Valve's Source Engine (SIGGRAPH 2006 PDF)](https://cdn.cloudflare.steamstatic.com/apps/valve/2006/SIGGRAPH06_Course_ShadingInValvesSourceEngine.pdf) +- [Efficient Self-Shadowed Radiosity Normal Mapping (SIGGRAPH 2007, Chris Green, Valve)](https://cdn.fastly.steamstatic.com/apps/valve/2007/SIGGRAPH2007_EfficientSelfShadowedRadiosityNormalMapping.pdf) +- [VRAD - Valve Developer Community](https://developer.valvesoftware.com/wiki/VRAD) +- [HDR Lighting Basics - Valve Developer Community](https://developer.valvesoftware.com/wiki/HDR_Lighting_Basics) +- [Advanced Lighting - Valve Developer Community](https://developer.valvesoftware.com/wiki/Advanced_Lighting) +- [BSP (Source) format - Valve Developer Community](https://developer.valvesoftware.com/wiki/BSP_(Source)) diff --git a/makefiles/Props/CommonVariables.props b/makefiles/Props/CommonVariables.props index 25b2c4b..0c36c2c 100755 --- a/makefiles/Props/CommonVariables.props +++ b/makefiles/Props/CommonVariables.props @@ -13,6 +13,7 @@ $(EditorRoot)libxml2\ $(EditorRoot)libpng\ $(EditorRoot)zlib\ + $(EditorRoot)libjpeg-turbo\ $(ProjectRoot)engine\ NOT_SET diff --git a/makefiles/Props/Project_libjpeg-turbo.props b/makefiles/Props/Project_libjpeg-turbo.props new file mode 100644 index 0000000..5881610 --- /dev/null +++ b/makefiles/Props/Project_libjpeg-turbo.props @@ -0,0 +1,9 @@ + + + + + WIN32;_WIN32;%(PreprocessorDefinitions) + $(LibJpegTurboSrcDir);%(AdditionalIncludeDirectories) + + + diff --git a/makefiles/Props/Project_q3map2.props b/makefiles/Props/Project_q3map2.props index 64b6508..0f620d9 100755 --- a/makefiles/Props/Project_q3map2.props +++ b/makefiles/Props/Project_q3map2.props @@ -25,7 +25,7 @@ $(LibXml2SrcDir)include; $(LibPngSrcDir); $(ZlibSrcDir); - $(EngineSrcDir)libjpeg-turbo; + $(LibJpegTurboSrcDir); %(AdditionalIncludeDirectories) diff --git a/makefiles/editor.sln b/makefiles/editor.sln index 1385018..f7ce5e1 100755 --- a/makefiles/editor.sln +++ b/makefiles/editor.sln @@ -1,49 +1,55 @@ - -Microsoft Visual Studio Solution File, Format Version 12.00 -# Visual Studio Version 17 -VisualStudioVersion = 17.14.36908.2 -MinimumVisualStudioVersion = 10.0.40219.1 -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "q3map2", "q3map2.vcxproj", "{E1A0A3B1-2001-4001-8001-000000000005}" -EndProject -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "assimp", "assimp.vcxproj", "{E1A0A3B1-2001-4001-8001-000000000004}" -EndProject -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpng", "libpng.vcxproj", "{E1A0A3B1-2001-4001-8001-000000000002}" -EndProject -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libxml2", "libxml2.vcxproj", "{E1A0A3B1-2001-4001-8001-000000000003}" -EndProject -Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "zlib", "zlib.vcxproj", "{E1A0A3B1-2001-4001-8001-000000000001}" -EndProject -Global - GlobalSection(SolutionConfigurationPlatforms) = preSolution - Debug|x64 = Debug|x64 - Release|x64 = Release|x64 - EndGlobalSection - GlobalSection(ProjectConfigurationPlatforms) = postSolution - {E1A0A3B1-2001-4001-8001-000000000005}.Debug|x64.ActiveCfg = Debug|x64 - {E1A0A3B1-2001-4001-8001-000000000005}.Debug|x64.Build.0 = Debug|x64 - {E1A0A3B1-2001-4001-8001-000000000005}.Release|x64.ActiveCfg = Release|x64 - {E1A0A3B1-2001-4001-8001-000000000005}.Release|x64.Build.0 = Release|x64 - {E1A0A3B1-2001-4001-8001-000000000004}.Debug|x64.ActiveCfg = Debug|x64 - {E1A0A3B1-2001-4001-8001-000000000004}.Debug|x64.Build.0 = Debug|x64 - {E1A0A3B1-2001-4001-8001-000000000004}.Release|x64.ActiveCfg = Release|x64 - {E1A0A3B1-2001-4001-8001-000000000004}.Release|x64.Build.0 = Release|x64 - {E1A0A3B1-2001-4001-8001-000000000002}.Debug|x64.ActiveCfg = Debug|x64 - {E1A0A3B1-2001-4001-8001-000000000002}.Debug|x64.Build.0 = Debug|x64 - {E1A0A3B1-2001-4001-8001-000000000002}.Release|x64.ActiveCfg = Release|x64 - {E1A0A3B1-2001-4001-8001-000000000002}.Release|x64.Build.0 = Release|x64 - {E1A0A3B1-2001-4001-8001-000000000003}.Debug|x64.ActiveCfg = Debug|x64 - {E1A0A3B1-2001-4001-8001-000000000003}.Debug|x64.Build.0 = Debug|x64 - {E1A0A3B1-2001-4001-8001-000000000003}.Release|x64.ActiveCfg = Release|x64 - {E1A0A3B1-2001-4001-8001-000000000003}.Release|x64.Build.0 = Release|x64 - {E1A0A3B1-2001-4001-8001-000000000001}.Debug|x64.ActiveCfg = Debug|x64 - {E1A0A3B1-2001-4001-8001-000000000001}.Debug|x64.Build.0 = Debug|x64 - {E1A0A3B1-2001-4001-8001-000000000001}.Release|x64.ActiveCfg = Release|x64 - {E1A0A3B1-2001-4001-8001-000000000001}.Release|x64.Build.0 = Release|x64 - EndGlobalSection - GlobalSection(SolutionProperties) = preSolution - HideSolutionNode = FALSE - EndGlobalSection - GlobalSection(ExtensibilityGlobals) = postSolution - SolutionGuid = {3C669F3A-FFB5-4A17-A9F2-F7D29754851E} - EndGlobalSection -EndGlobal + +Microsoft Visual Studio Solution File, Format Version 12.00 +# Visual Studio Version 17 +VisualStudioVersion = 17.14.36908.2 +MinimumVisualStudioVersion = 10.0.40219.1 +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "q3map2", "q3map2.vcxproj", "{E1A0A3B1-2001-4001-8001-000000000005}" +EndProject +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "assimp", "assimp.vcxproj", "{E1A0A3B1-2001-4001-8001-000000000004}" +EndProject +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpng", "libpng.vcxproj", "{E1A0A3B1-2001-4001-8001-000000000002}" +EndProject +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libxml2", "libxml2.vcxproj", "{E1A0A3B1-2001-4001-8001-000000000003}" +EndProject +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "zlib", "zlib.vcxproj", "{E1A0A3B1-2001-4001-8001-000000000001}" +EndProject +Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libjpeg-turbo", "libjpeg-turbo.vcxproj", "{E1A0A3B1-2001-4001-8001-000000000006}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|x64 = Debug|x64 + Release|x64 = Release|x64 + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {E1A0A3B1-2001-4001-8001-000000000005}.Debug|x64.ActiveCfg = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000005}.Debug|x64.Build.0 = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000005}.Release|x64.ActiveCfg = Release|x64 + {E1A0A3B1-2001-4001-8001-000000000005}.Release|x64.Build.0 = Release|x64 + {E1A0A3B1-2001-4001-8001-000000000004}.Debug|x64.ActiveCfg = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000004}.Debug|x64.Build.0 = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000004}.Release|x64.ActiveCfg = Release|x64 + {E1A0A3B1-2001-4001-8001-000000000004}.Release|x64.Build.0 = Release|x64 + {E1A0A3B1-2001-4001-8001-000000000002}.Debug|x64.ActiveCfg = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000002}.Debug|x64.Build.0 = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000002}.Release|x64.ActiveCfg = Release|x64 + {E1A0A3B1-2001-4001-8001-000000000002}.Release|x64.Build.0 = Release|x64 + {E1A0A3B1-2001-4001-8001-000000000003}.Debug|x64.ActiveCfg = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000003}.Debug|x64.Build.0 = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000003}.Release|x64.ActiveCfg = Release|x64 + {E1A0A3B1-2001-4001-8001-000000000003}.Release|x64.Build.0 = Release|x64 + {E1A0A3B1-2001-4001-8001-000000000001}.Debug|x64.ActiveCfg = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000001}.Debug|x64.Build.0 = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000001}.Release|x64.ActiveCfg = Release|x64 + {E1A0A3B1-2001-4001-8001-000000000001}.Release|x64.Build.0 = Release|x64 + {E1A0A3B1-2001-4001-8001-000000000006}.Debug|x64.ActiveCfg = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000006}.Debug|x64.Build.0 = Debug|x64 + {E1A0A3B1-2001-4001-8001-000000000006}.Release|x64.ActiveCfg = Release|x64 + {E1A0A3B1-2001-4001-8001-000000000006}.Release|x64.Build.0 = Release|x64 + EndGlobalSection + GlobalSection(SolutionProperties) = preSolution + HideSolutionNode = FALSE + EndGlobalSection + GlobalSection(ExtensibilityGlobals) = postSolution + SolutionGuid = {3C669F3A-FFB5-4A17-A9F2-F7D29754851E} + EndGlobalSection +EndGlobal diff --git a/makefiles/libjpeg-turbo.vcxproj b/makefiles/libjpeg-turbo.vcxproj new file mode 100644 index 0000000..dc53c67 --- /dev/null +++ b/makefiles/libjpeg-turbo.vcxproj @@ -0,0 +1,65 @@ + + + + {E1A0A3B1-2001-4001-8001-000000000006} + 10.0 + + + StaticLibrary + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/makefiles/q3map2.vcxproj b/makefiles/q3map2.vcxproj index 576cd90..82c6dbf 100755 --- a/makefiles/q3map2.vcxproj +++ b/makefiles/q3map2.vcxproj @@ -87,7 +87,6 @@ - diff --git a/q3map2/q3map2/bspfile_abstract.cpp b/q3map2/q3map2/bspfile_abstract.cpp index 8619b7d..ffd6a91 100755 --- a/q3map2/q3map2/bspfile_abstract.cpp +++ b/q3map2/q3map2/bspfile_abstract.cpp @@ -322,6 +322,9 @@ void PrintBSPFileSizes(){ bspLightBytes.size() / ( g_game->lightmapSize * g_game->lightmapSize * 3 ), bspLightBytes.size() ); Sys_Printf( "%9zu lightgrid %9zu *\n", bspGridPoints.size(), bspGridPoints.size() * sizeof( bspGridPoints[0] ) ); + if ( !bspGridPointsSH.empty() ) + Sys_Printf( "%9zu lightgrid SH %9zu\n", + bspGridPointsSH.size(), bspGridPointsSH.size() * sizeof( bspGridPointsSH[0] ) ); Sys_Printf( " visibility %9zu\n", bspVisBytes.size() ); } diff --git a/q3map2/q3map2/bspfile_ibsp.cpp b/q3map2/q3map2/bspfile_ibsp.cpp index fc453de..75a1128 100755 --- a/q3map2/q3map2/bspfile_ibsp.cpp +++ b/q3map2/q3map2/bspfile_ibsp.cpp @@ -62,16 +62,37 @@ #define LUMP_LIGHTGRID 15 #define LUMP_VISIBILITY 16 #define LUMP_ADVERTISEMENTS 17 -#define HEADER_LUMPS 18 +#define LUMP_LIGHTGRID_SH 18 + +#define HEADER_LUMPS_V46 17 /* standard Q3: lumps 0-16, header = 144 bytes */ +#define HEADER_LUMPS_V47 19 /* extended: lumps 0-18, header = 160 bytes */ +#define HEADER_LUMPS_MAX 19 /* max for internal use */ + +#define IBSP_VERSION_V46 46 +#define IBSP_VERSION_V47 47 /* types */ +struct ibspHeader_v46_t +{ + char ident[ 4 ]; + int version; + bspLump_t lumps[ HEADER_LUMPS_V46 ]; +}; + +struct ibspHeader_v47_t +{ + char ident[ 4 ]; + int version; + bspLump_t lumps[ HEADER_LUMPS_V47 ]; +}; + +/* internal header -- always uses max lump count for convenience */ struct ibspHeader_t { char ident[ 4 ]; int version; - - bspLump_t lumps[ HEADER_LUMPS ]; + bspLump_t lumps[ HEADER_LUMPS_MAX ]; }; @@ -217,44 +238,76 @@ void LoadIBSPFile( const char *filename ){ /* load the file */ MemBuffer file = LoadFile( filename ); - ibspHeader_t *header = file.data(); - - /* swap the header (except the first 4 bytes) */ - SwapBlock( (int*) ( (byte*) header + 4 ), sizeof( *header ) - 4 ); + /* read ident and version to determine header size */ + const byte *data = (const byte *)file.data(); + const int version = LittleLong( *(const int *)( data + 4 ) ); /* make sure it matches the format we're trying to load */ - if ( !force && memcmp( header->ident, g_game->bspIdent, 4 ) ) { + if ( !force && memcmp( data, g_game->bspIdent, 4 ) ) { Error( "%s is not a %s file", filename, g_game->bspIdent ); } - if ( !force && header->version != g_game->bspVersion ) { - Error( "%s is version %d, not %d", filename, header->version, g_game->bspVersion ); + if ( !force && version != IBSP_VERSION_V46 && version != IBSP_VERSION_V47 ) { + Error( "%s is version %d, expected %d or %d", filename, version, IBSP_VERSION_V46, IBSP_VERSION_V47 ); } - /* load/convert lumps */ - CopyLump( (bspHeader_t*) header, LUMP_SHADERS, bspShaders ); - CopyLump( (bspHeader_t*) header, LUMP_MODELS, bspModels ); - CopyLump( (bspHeader_t*) header, LUMP_PLANES, bspPlanes ); - CopyLump( (bspHeader_t*) header, LUMP_LEAFS, bspLeafs ); - CopyLump( (bspHeader_t*) header, LUMP_NODES, bspNodes ); - CopyLump( (bspHeader_t*) header, LUMP_LEAFSURFACES, bspLeafSurfaces ); - CopyLump( (bspHeader_t*) header, LUMP_LEAFBRUSHES, bspLeafBrushes ); - CopyLump( (bspHeader_t*) header, LUMP_BRUSHES, bspBrushes ); - CopyLump( (bspHeader_t*) header, LUMP_BRUSHSIDES, bspBrushSides ); - CopyLump( (bspHeader_t*) header, LUMP_DRAWVERTS, bspDrawVerts ); - CopyLump( (bspHeader_t*) header, LUMP_SURFACES, bspDrawSurfaces ); - CopyLump( (bspHeader_t*) header, LUMP_FOGS, bspFogs ); - CopyLump( (bspHeader_t*) header, LUMP_DRAWINDEXES, bspDrawIndexes ); - CopyLump( (bspHeader_t*) header, LUMP_VISIBILITY, bspVisBytes ); - CopyLump( (bspHeader_t*) header, LUMP_LIGHTMAPS, bspLightBytes ); - CopyLump( (bspHeader_t*) header, LUMP_ENTITIES, bspEntData ); - CopyLump( (bspHeader_t*) header, LUMP_LIGHTGRID, bspGridPoints ); + /* determine lump count from version */ + const int numLumps = ( version == IBSP_VERSION_V47 ) ? HEADER_LUMPS_V47 : HEADER_LUMPS_V46; - /* advertisements */ - if ( header->version == 47 && strEqual( g_game->arg, "quakelive" ) ) { // quake live's bsp version minus wolf, et, etut - CopyLump( (bspHeader_t*) header, LUMP_ADVERTISEMENTS, bspAds ); + /* CopyLump uses (byte*)header + offset to access file data, + so we overlay a bspHeader_t at the file buffer start and + copy/swap the lump directory into it. bspHeader_t has lumps[100] + so it can hold all our lumps. */ + bspHeader_t *header = (bspHeader_t *)file.data(); + /* swap just the lump directory entries we have */ + SwapBlock( (int *)( (byte *)header + 8 ), numLumps * sizeof( bspLump_t ) ); + /* note: lumps beyond numLumps are not zeroed -- they overlap file data + in the buffer. We only access extended lumps under version checks. */ + + /* load/convert standard lumps (0-16) */ + CopyLump( header, LUMP_SHADERS, bspShaders ); + CopyLump( header, LUMP_MODELS, bspModels ); + CopyLump( header, LUMP_PLANES, bspPlanes ); + CopyLump( header, LUMP_LEAFS, bspLeafs ); + CopyLump( header, LUMP_NODES, bspNodes ); + CopyLump( header, LUMP_LEAFSURFACES, bspLeafSurfaces ); + CopyLump( header, LUMP_LEAFBRUSHES, bspLeafBrushes ); + CopyLump( header, LUMP_BRUSHES, bspBrushes ); + CopyLump( header, LUMP_BRUSHSIDES, bspBrushSides ); + CopyLump( header, LUMP_DRAWVERTS, bspDrawVerts ); + CopyLump( header, LUMP_SURFACES, bspDrawSurfaces ); + CopyLump( header, LUMP_FOGS, bspFogs ); + CopyLump( header, LUMP_DRAWINDEXES, bspDrawIndexes ); + CopyLump( header, LUMP_VISIBILITY, bspVisBytes ); + CopyLump( header, LUMP_LIGHTMAPS, bspLightBytes ); + CopyLump( header, LUMP_ENTITIES, bspEntData ); + CopyLump( header, LUMP_LIGHTGRID, bspGridPoints ); + + /* v47 extended lumps (17-18) */ + if ( version == IBSP_VERSION_V47 ) { + /* advertisements */ + CopyLump( header, LUMP_ADVERTISEMENTS, bspAds ); + + /* SH light grid */ + const int length = header->lumps[LUMP_LIGHTGRID_SH].length; + const int offset = header->lumps[LUMP_LIGHTGRID_SH].offset; + if ( length > (int)sizeof( bspGridSHHeader_t ) ) { + const bspGridSHHeader_t *shHeader = (const bspGridSHHeader_t *)( data + offset ); + gridMinsSH = shHeader->gridMins; + gridSizeSH = shHeader->gridSize; + gridBoundsSH[0] = shHeader->gridBounds[0]; + gridBoundsSH[1] = shHeader->gridBounds[1]; + gridBoundsSH[2] = shHeader->gridBounds[2]; + const int numPoints = shHeader->numPoints; + const bspGridPointSH_t *points = (const bspGridPointSH_t *)( data + offset + sizeof( bspGridSHHeader_t ) ); + bspGridPointsSH.assign( points, points + numPoints ); + } + else { + bspGridPointsSH.clear(); + } } - else{ + else { bspAds.clear(); + bspGridPointsSH.clear(); } } @@ -267,28 +320,33 @@ void LoadIBSPorRBSPFilePartially( const char *filename ){ /* load the file */ MemBuffer file = LoadFile( filename ); - ibspHeader_t *header = file.data(); - - /* swap the header (except the first 4 bytes) */ - SwapBlock( (int*) ( (byte*) header + 4 ), sizeof( *header ) - 4 ); + const byte *data = (const byte *)file.data(); + const int version = LittleLong( *(const int *)( data + 4 ) ); /* make sure it matches the format we're trying to load */ - if ( !force && memcmp( header->ident, g_game->bspIdent, 4 ) ) { + if ( !force && memcmp( data, g_game->bspIdent, 4 ) ) { Error( "%s is not a %s file", filename, g_game->bspIdent ); } - if ( !force && header->version != g_game->bspVersion ) { - Error( "%s is version %d, not %d", filename, header->version, g_game->bspVersion ); + if ( !force && version != IBSP_VERSION_V46 && version != IBSP_VERSION_V47 ) { + Error( "%s is version %d, expected %d or %d", filename, version, IBSP_VERSION_V46, IBSP_VERSION_V47 ); } - /* load/convert lumps */ - CopyLump( (bspHeader_t*) header, LUMP_SHADERS, bspShaders ); - if( g_game->load == LoadIBSPFile ) - CopyLump( (bspHeader_t*) header, LUMP_SURFACES, bspDrawSurfaces ); - else - CopyLump( (bspHeader_t*) header, LUMP_SURFACES, bspDrawSurfaces ); + const int numLumps = ( version == IBSP_VERSION_V47 ) ? HEADER_LUMPS_V47 : HEADER_LUMPS_V46; - CopyLump( (bspHeader_t*) header, LUMP_FOGS, bspFogs ); - CopyLump( (bspHeader_t*) header, LUMP_ENTITIES, bspEntData ); + bspHeader_t *header = (bspHeader_t *)file.data(); + SwapBlock( (int *)( (byte *)header + 8 ), numLumps * sizeof( bspLump_t ) ); + for ( int i = numLumps; i < 100; ++i ) + header->lumps[i] = {}; + + /* load/convert lumps */ + CopyLump( header, LUMP_SHADERS, bspShaders ); + if( g_game->load == LoadIBSPFile ) + CopyLump( header, LUMP_SURFACES, bspDrawSurfaces ); + else + CopyLump( header, LUMP_SURFACES, bspDrawSurfaces ); + + CopyLump( header, LUMP_FOGS, bspFogs ); + CopyLump( header, LUMP_ENTITIES, bspEntData ); } /* @@ -297,17 +355,23 @@ void LoadIBSPorRBSPFilePartially( const char *filename ){ */ void WriteIBSPFile( const char *filename ){ + /* determine version: v47 if SH data exists, v46 otherwise */ + const bool extendedBSP = !bspGridPointsSH.empty(); + const int bspVersion = extendedBSP ? IBSP_VERSION_V47 : IBSP_VERSION_V46; + const int numLumps = extendedBSP ? HEADER_LUMPS_V47 : HEADER_LUMPS_V46; + const int headerSize = 8 + numLumps * (int)sizeof( bspLump_t ); + ibspHeader_t header{}; //% Swapfile(); /* set up header */ memcpy( header.ident, g_game->bspIdent, 4 ); - header.version = LittleLong( g_game->bspVersion ); + header.version = LittleLong( bspVersion ); - /* write initial header */ + /* write initial header (placeholder, overwritten at the end) */ FILE *file = SafeOpenWrite( filename ); - SafeWrite( file, &header, sizeof( header ) ); /* overwritten later */ + SafeWrite( file, &header, headerSize ); { /* add marker lump */ time_t t; @@ -317,7 +381,7 @@ void WriteIBSPFile( const char *filename ){ AddLump( file, header.lumps[0], std::vector( marker.cbegin(), marker.cend() + 1 ) ); } - /* add lumps */ + /* add standard lumps (0-16) */ AddLump( file, header.lumps[LUMP_SHADERS], bspShaders ); AddLump( file, header.lumps[LUMP_PLANES], bspPlanes ); AddLump( file, header.lumps[LUMP_LEAFS], bspLeafs ); @@ -336,8 +400,26 @@ void WriteIBSPFile( const char *filename ){ AddLump( file, header.lumps[LUMP_FOGS], bspFogs ); AddLump( file, header.lumps[LUMP_DRAWINDEXES], bspDrawIndexes ); - /* advertisements */ - AddLump( file, header.lumps[LUMP_ADVERTISEMENTS], bspAds ); + /* v47 extended lumps (17-18) */ + if ( extendedBSP ) { + /* advertisements */ + AddLump( file, header.lumps[LUMP_ADVERTISEMENTS], bspAds ); + + /* SH light grid -- header + points packed into a byte lump */ + const size_t shHeaderSz = sizeof( bspGridSHHeader_t ); + const size_t dataSz = bspGridPointsSH.size() * sizeof( bspGridPointSH_t ); + std::vector shLump( shHeaderSz + dataSz ); + bspGridSHHeader_t shHeader; + shHeader.gridMins = gridMinsSH; + shHeader.gridSize = gridSizeSH; + shHeader.gridBounds[0] = gridBoundsSH[0]; + shHeader.gridBounds[1] = gridBoundsSH[1]; + shHeader.gridBounds[2] = gridBoundsSH[2]; + shHeader.numPoints = (int)bspGridPointsSH.size(); + memcpy( shLump.data(), &shHeader, shHeaderSz ); + memcpy( shLump.data() + shHeaderSz, bspGridPointsSH.data(), dataSz ); + AddLump( file, header.lumps[LUMP_LIGHTGRID_SH], shLump ); + } /* emit bsp size */ const int size = ftell( file ); @@ -345,7 +427,7 @@ void WriteIBSPFile( const char *filename ){ /* write the completed header */ fseek( file, 0, SEEK_SET ); - SafeWrite( file, &header, sizeof( header ) ); + SafeWrite( file, &header, headerSize ); /* close the file */ fclose( file ); diff --git a/q3map2/q3map2/convert_json.cpp b/q3map2/q3map2/convert_json.cpp index 8aa91dd..813e907 100755 --- a/q3map2/q3map2/convert_json.cpp +++ b/q3map2/q3map2/convert_json.cpp @@ -420,6 +420,29 @@ static void write_json( const char *directory ){ } write_doc( StringStream( directory, "GridPoints.json" ), doc ); } + if ( !bspGridPointsSH.empty() ) { + doc.RemoveAllMembers(); + /* write SH grid header info */ + { + rapidjson::Value header( rapidjson::kObjectType ); + header.AddMember( "gridMins", value_for( gridMinsSH, all ), all ); + header.AddMember( "gridSize", value_for( gridSizeSH, all ), all ); + header.AddMember( "gridBounds", value_for_array( gridBoundsSH, all ), all ); + doc.AddMember( "header", header, all ); + } + /* write SH grid points */ + { + rapidjson::Value points( rapidjson::kArrayType ); + for ( const auto& point : bspGridPointsSH ){ + rapidjson::Value coeffs( rapidjson::kArrayType ); + for ( int i = 0; i < 9; ++i ) + coeffs.PushBack( value_for( point.coeffs[i], all ), all ); + points.PushBack( coeffs, all ); + } + doc.AddMember( "points", points, all ); + } + write_doc( StringStream( directory, "GridPointsSH.json" ), doc ); + } } inline rapidjson::Document load_json( const char *fileName ){ @@ -644,6 +667,27 @@ static void read_json( const char *directory, bool useFlagNames, bool skipUnknow value_to_array( obj.value["latLong"], item.latLong ); } } + { + const auto shFile = StringStream( directory, "GridPointsSH.json" ); + if ( FileExists( shFile ) ) { + const auto doc = load_json( shFile ); + /* read header */ + const auto& header = doc["header"]; + value_to( header["gridMins"], gridMinsSH ); + value_to( header["gridSize"], gridSizeSH ); + value_to_array( header["gridBounds"], gridBoundsSH ); + /* read points */ + const auto& points = doc["points"]; + bspGridPointsSH.clear(); + bspGridPointsSH.reserve( points.Size() ); + for ( rapidjson::SizeType i = 0; i < points.Size(); ++i ) { + auto& point = bspGridPointsSH.emplace_back(); + const auto& coeffs = points[i]; + for ( int j = 0; j < 9; ++j ) + value_to( coeffs[j], point.coeffs[j] ); + } + } + } } int ConvertJsonMain( Args& args ){ diff --git a/q3map2/q3map2/light.cpp b/q3map2/q3map2/light.cpp index cffe36c..89d1a57 100755 --- a/q3map2/q3map2/light.cpp +++ b/q3map2/q3map2/light.cpp @@ -31,6 +31,7 @@ /* dependencies */ #include "q3map2.h" #include "bspfile_rbsp.h" +#include "sh.h" #include @@ -1787,6 +1788,163 @@ static void SetupGrid( const Vector3& ambientColor ){ +/* + SetupGridSH() + allocates and initializes the SH light grid with independent resolution + */ + +static void SetupGridSH(){ + if ( !gridSH ) { + return; + } + + /* derive SH grid cell size from legacy grid if no explicit size was given */ + if ( !gridSHExplicitSize ) { + for ( int i = 0; i < 3; ++i ) + gridSizeSH[i] = gridSize[i] / gridScaleSH; + } + + /* quantize */ + for ( int i = 0; i < 3; ++i ) + gridSizeSH[i] = std::max( 8.f, std::floor( gridSizeSH[i] ) ); + + /* compute bounds */ + size_t numGridPointsSH; + for ( int j = 0; ; ) + { + for ( int i = 0; i < 3; ++i ) + { + gridMinsSH[i] = gridSizeSH[i] * ceil( bspModels[0].minmax.mins[i] / gridSizeSH[i] ); + const float max = gridSizeSH[i] * floor( bspModels[0].minmax.maxs[i] / gridSizeSH[i] ); + gridBoundsSH[i] = ( max - gridMinsSH[i] ) / gridSizeSH[i] + 1; + } + + const int64_t num = int64_t( gridBoundsSH[0] ) * gridBoundsSH[1] * gridBoundsSH[2]; + + /* cap at 4x the legacy grid max -- SH points are larger but still need a limit */ + if ( num > MAX_MAP_LIGHTGRID * 4 ) { + gridSizeSH[j++ % 3] += 8.0f; + } + else { + numGridPointsSH = num; + break; + } + } + + /* allocate */ + bspGridPointsSH = decltype( bspGridPointsSH )( numGridPointsSH, bspGridPointSH_t{} ); + + /* print */ + Sys_Printf( "SH Grid size = { %1.0f, %1.0f, %1.0f }\n", gridSizeSH[0], gridSizeSH[1], gridSizeSH[2] ); + Sys_Printf( "%9zu SH grid points\n", bspGridPointsSH.size() ); +} + + + +/* + TraceGridSH() + per-point SH light grid tracing -- projects all light contributions onto L2 SH basis + */ + +static void TraceGridSH( int num ){ + trace_t trace; + + /* get grid origin from SH grid bounds */ + int mod = num; + const int z = mod / ( gridBoundsSH[0] * gridBoundsSH[1] ); + mod -= z * ( gridBoundsSH[0] * gridBoundsSH[1] ); + const int y = mod / gridBoundsSH[0]; + mod -= y * gridBoundsSH[0]; + const int x = mod; + + trace.origin = gridMinsSH + Vector3( x, y, z ) * gridSizeSH; + + /* set inhibit sphere */ + trace.inhibitRadius = gridSizeSH[vector3_max_abs_component_index( gridSizeSH )] * 0.5f; + + /* find point cluster */ + trace.cluster = ClusterForPointExt( trace.origin, GRID_EPSILON ); + if ( trace.cluster < CLUSTER_NORMAL ) { + /* try to nudge the origin around to find a valid point */ + const Vector3 baseOrigin( trace.origin ); + double step = 0; + while ( ( step += 0.005 ) <= 1 ) + { + trace.origin = baseOrigin; + trace.origin[0] += step * ( Random() - 0.5 ) * gridSizeSH[0]; + trace.origin[1] += step * ( Random() - 0.5 ) * gridSizeSH[1]; + trace.origin[2] += step * ( Random() - 0.5 ) * gridSizeSH[2]; + + trace.cluster = ClusterForPointExt( trace.origin, VERTEX_EPSILON ); + if ( trace.cluster >= CLUSTER_NORMAL ) { + break; + } + } + + /* can't find a valid point at all */ + if ( step > 1 ) { + return; + } + } + + /* setup trace */ + trace.testOcclusion = !noTrace; + trace.forceSunlight = false; + trace.recvShadows = WORLDSPAWN_RECV_SHADOWS; + trace.numSurfaces = 0; + trace.surfaces = nullptr; + trace.numLights = 0; + trace.lights = nullptr; + + /* accumulate SH from all lights */ + SHCoeffs3 sh; + + for ( const light_t& light : lights ) + { + trace.light = &light; + + /* sample light */ + if ( !LightContributionToPoint( &trace ) ) { + continue; + } + + /* handle negative light */ + Vector3 color = trace.color; + if ( trace.light->flags & LightFlags::Negative ) { + vector3_negate( color ); + } + + /* project this directional sample onto SH basis */ + const Vector3 dir = VectorNormalized( trace.direction ); + SH_project_sample( sh, dir, color ); + } + + /* floodlight contribution */ + if ( floodlighty ) { + trace.testOcclusion = true; + trace.forceSunlight = false; + trace.inhibitRadius = DEFAULT_INHIBIT_RADIUS; + trace.testAll = true; + + for ( int k = 0; k < 2; ++k ) + { + trace.normal = ( k == 0 ) ? g_vector3_axis_z : -g_vector3_axis_z; + + const float f = FloodLightForSample( &trace, floodlightDistance, floodlight_lowquality ); + const Vector3 floodColor = floodlightRGB * ( floodlightIntensity * f ); + const Vector3 dir = ( k == 0 ) ? g_vector3_axis_z : -g_vector3_axis_z; + + SH_project_sample( sh, dir, floodColor ); + } + } + + /* store */ + for ( int i = 0; i < SH_NUM_COEFFS; ++i ) + bspGridPointsSH[num].coeffs[i] = sh.c[i]; +} + + + /* Handles writing optional hacks after -light and each -bounce pass */ @@ -1973,6 +2131,12 @@ static void LightWorld( bool fastAllocate, bool bounceStore ){ Sys_Printf( "--- SetupGrid ---\n" ); SetupGrid( ambientColor ); + /* setup SH grid (independent resolution) */ + if ( gridSH ) { + Sys_Printf( "--- SetupGridSH ---\n" ); + SetupGridSH(); + } + /* create world lights */ Sys_FPrintf( SYS_VRB, "--- CreateLights ---\n" ); CreateEntityLights(); @@ -1999,6 +2163,16 @@ static void LightWorld( bool fastAllocate, bool bounceStore ){ Sys_FPrintf( SYS_VRB, "%9d grid points bounds culled\n", gridBoundsCulled ); } + /* calculate SH lightgrid */ + if ( gridSH && !bspGridPointsSH.empty() ) { + Sys_Printf( "--- TraceGridSH ---\n" ); + inGrid = true; + RunThreadsOnIndividual( bspGridPointsSH.size(), true, TraceGridSH ); + inGrid = false; + Sys_Printf( "%d x %d x %d = %zu SH grid\n", + gridBoundsSH[0], gridBoundsSH[1], gridBoundsSH[2], bspGridPointsSH.size() ); + } + /* slight optimization to remove a sqrt */ subdivideThreshold *= subdivideThreshold; @@ -2093,6 +2267,14 @@ static void LightWorld( bool fastAllocate, bool bounceStore ){ inGrid = false; Sys_FPrintf( SYS_VRB, "%9d grid points envelope culled\n", gridEnvelopeCulled ); Sys_FPrintf( SYS_VRB, "%9d grid points bounds culled\n", gridBoundsCulled ); + + /* bounce SH grid too */ + if ( gridSH && !bspGridPointsSH.empty() ) { + Sys_Printf( "--- BounceGridSH ---\n" ); + inGrid = true; + RunThreadsOnIndividual( bspGridPointsSH.size(), true, TraceGridSH ); + inGrid = false; + } } /* light up my world */ @@ -2870,6 +3052,25 @@ int LightMain( Args& args ){ while ( args.takeArg( "-fillpink" ) ) { lightmapPink = true; } + + /* spherical harmonics light grid */ + while ( args.takeArg( "-sh" ) ) { + gridSH = true; + Sys_Printf( "Spherical harmonics light grid enabled\n" ); + } + while ( args.takeArg( "-gridscalesh" ) ) { + gridScaleSH = std::max( 0.5f, (float)atof( args.takeNext() ) ); + Sys_Printf( "SH grid density scale set to %f (relative to legacy grid)\n", gridScaleSH ); + } + while ( args.takeArg( "-gridsh" ) ) { + gridSHExplicitSize = true; + gridSizeSH[0] = atof( args.takeNext() ); + gridSizeSH[1] = atof( args.takeNext() ); + gridSizeSH[2] = atof( args.takeNext() ); + Sys_Printf( "SH grid size explicitly set to { %1.0f, %1.0f, %1.0f }\n", + gridSizeSH[0], gridSizeSH[1], gridSizeSH[2] ); + } + /* unhandled args */ while( !args.empty() ) { diff --git a/q3map2/q3map2/q3map2.h b/q3map2/q3map2/q3map2.h index 8567c60..b9709ab 100755 --- a/q3map2/q3map2/q3map2.h +++ b/q3map2/q3map2/q3map2.h @@ -364,6 +364,21 @@ struct bspGridPoint_t byte latLong[ 2 ]; }; +/* SH light grid header -- stored at the start of LUMP_LIGHTGRID_SH */ +struct bspGridSHHeader_t +{ + Vector3 gridMins; /* world-space origin of the SH grid */ + Vector3 gridSize; /* cell size per axis */ + int gridBounds[3]; /* number of grid points per axis */ + int numPoints; /* total grid points following this header */ +}; + +/* SH light grid point -- L2 spherical harmonics, 9 RGB coefficients (float) */ +struct bspGridPointSH_t +{ + Vector3 coeffs[9]; /* 9 * 12 = 108 bytes per grid point */ +}; + struct bspDrawSurface_t { @@ -2306,6 +2321,14 @@ inline Vector3 gridMins; inline int gridBounds[ 3 ]; inline Vector3 gridSize = { 64, 64, 128 }; +/* SH lightgrid -- independent resolution, defaults to 2x density per axis */ +inline bool gridSH; /* enabled via -sh flag */ +inline Vector3 gridSizeSH = { 32, 32, 64 }; /* default: half the legacy grid cell size = 2x density */ +inline Vector3 gridMinsSH; +inline int gridBoundsSH[ 3 ]; +inline float gridScaleSH = 2.0f; /* multiplier over legacy grid density (used when no explicit -gridsh) */ +inline bool gridSHExplicitSize; /* true when -gridsh "X Y Z" was given */ + /* ------------------------------------------------------------------------------- @@ -2341,6 +2364,8 @@ inline std::vector bspLightBytes; inline std::vector bspGridPoints; +inline std::vector bspGridPointsSH; + inline std::vector bspVisBytes; // MAX_MAP_VISIBILITY inline DrawVerts bspDrawVerts; diff --git a/q3map2/q3map2/sh.h b/q3map2/q3map2/sh.h new file mode 100644 index 0000000..45553e3 --- /dev/null +++ b/q3map2/q3map2/sh.h @@ -0,0 +1,138 @@ +/* ------------------------------------------------------------------------------- + + Spherical Harmonics (L2) for light grid + + 9 coefficients per color channel, encoding a smooth directional lighting + function over the sphere. This is the standard real-valued SH basis up to + order 2 (l=0,1,2), widely used for irradiance environment maps. + + Basis functions (Cartesian, real-valued, orthonormal): + Y0 = 0.282095 (l=0, m=0) -- constant / DC + Y1 = 0.488603 * y (l=1, m=-1) -- linear + Y2 = 0.488603 * z (l=1, m=0) + Y3 = 0.488603 * x (l=1, m=1) + Y4 = 1.092548 * x*y (l=2, m=-2) -- quadratic + Y5 = 1.092548 * y*z (l=2, m=-1) + Y6 = 0.315392 * (3*z*z - 1) (l=2, m=0) + Y7 = 1.092548 * x*z (l=2, m=1) + Y8 = 0.546274 * (x*x - y*y) (l=2, m=2) + + References: + - "Stupid Spherical Harmonics (SH) Tricks", Peter-Pike Sloan + - "An Efficient Representation for Irradiance Environment Maps", + Ramamoorthi & Hanrahan, SIGGRAPH 2001 + + ------------------------------------------------------------------------------- */ + +#pragma once + +#include "math/vector.h" +#include + + +/* number of L2 SH coefficients */ +static constexpr int SH_NUM_COEFFS = 9; + +/* SH basis function constants */ +static constexpr float SH_C0 = 0.282094791773878f; /* 1 / (2*sqrt(pi)) */ +static constexpr float SH_C1 = 0.488602511902920f; /* sqrt(3) / (2*sqrt(pi)) */ +static constexpr float SH_C2 = 1.092548430592079f; /* sqrt(15) / (2*sqrt(pi)) */ +static constexpr float SH_C3 = 0.315391565252520f; /* sqrt(5) / (4*sqrt(pi)) */ +static constexpr float SH_C4 = 0.546274215296040f; /* sqrt(15) / (4*sqrt(pi)) */ + + +/* + SH L2 coefficients for a single color channel (RGB stored as 3 of these), + or for all 3 channels interleaved as Vector3. +*/ +struct SHCoeffs3 +{ + Vector3 c[SH_NUM_COEFFS]; + + SHCoeffs3() { + for ( int i = 0; i < SH_NUM_COEFFS; ++i ) + c[i].set( 0 ); + } +}; + + +/* + SH_evaluate() + Evaluate the 9 L2 SH basis functions for a given direction. + Direction must be normalized. +*/ +inline void SH_evaluate( const Vector3& dir, float basis[SH_NUM_COEFFS] ){ + const float x = dir.x(); + const float y = dir.y(); + const float z = dir.z(); + + /* l=0 */ + basis[0] = SH_C0; + + /* l=1 */ + basis[1] = SH_C1 * y; + basis[2] = SH_C1 * z; + basis[3] = SH_C1 * x; + + /* l=2 */ + basis[4] = SH_C2 * x * y; + basis[5] = SH_C2 * y * z; + basis[6] = SH_C3 * ( 3.0f * z * z - 1.0f ); + basis[7] = SH_C2 * x * z; + basis[8] = SH_C4 * ( x * x - y * y ); +} + + +/* + SH_project_sample() + Given a directional light sample (direction + RGB color), accumulate + its contribution into the SH coefficients. + + This projects: color * SH_basis(direction) for each basis function. + The direction must be normalized. +*/ +inline void SH_project_sample( SHCoeffs3& sh, const Vector3& dir, const Vector3& color ){ + float basis[SH_NUM_COEFFS]; + SH_evaluate( dir, basis ); + + for ( int i = 0; i < SH_NUM_COEFFS; ++i ) + sh.c[i] += color * basis[i]; +} + + +/* + SH_reconstruct() + Reconstruct the lighting color for a given direction from SH coefficients. + Direction must be normalized. + Returns the RGB color (may contain negative values from ringing -- caller should clamp). +*/ +inline Vector3 SH_reconstruct( const SHCoeffs3& sh, const Vector3& dir ){ + float basis[SH_NUM_COEFFS]; + SH_evaluate( dir, basis ); + + Vector3 result( 0 ); + for ( int i = 0; i < SH_NUM_COEFFS; ++i ) + result += sh.c[i] * basis[i]; + + return result; +} + + +/* + SH_scale() + Scale all SH coefficients by a scalar. +*/ +inline void SH_scale( SHCoeffs3& sh, float s ){ + for ( int i = 0; i < SH_NUM_COEFFS; ++i ) + sh.c[i] *= s; +} + + +/* + SH_add() + Add src coefficients into dst. +*/ +inline void SH_add( SHCoeffs3& dst, const SHCoeffs3& src ){ + for ( int i = 0; i < SH_NUM_COEFFS; ++i ) + dst.c[i] += src.c[i]; +} diff --git a/sh.md b/sh.md new file mode 100644 index 0000000..9adf81d --- /dev/null +++ b/sh.md @@ -0,0 +1,505 @@ +# Spherical Harmonics Light Grid (SH Grid) + +## Overview + +The SH light grid is an extension to q3map2's lighting system that stores **L2 spherical harmonics** (9 RGB coefficients per grid point) alongside the legacy Quake 3 light grid. It encodes a smooth directional lighting function over the full sphere at each grid point, enabling dynamic objects to receive accurate directional lighting from all angles -- not just a single dominant direction as in the legacy grid. + +The SH grid is stored in a new BSP lump (`LUMP_LIGHTGRID_SH`, index 18) that stock Quake 3 engines ignore, preserving full backward compatibility. + +### Why SH over the Legacy Grid? + +The legacy Q3 light grid stores per point: +- `ambient` (RGB byte) -- omnidirectional ambient term +- `directed` (RGB byte) -- directional light color +- `latLong` (2 bytes) -- single dominant light direction + +The engine evaluates: `result = ambient + directed * max(0, dot(normal, decode(latLong)))` + +This collapses all directional information into **one direction**. A point lit by a red light from the left and a blue light from the right produces a muddy average -- a dynamic object turning in place cannot see the color shift. + +L2 SH encodes 9 directional basis functions that capture smooth angular variation across the entire sphere. A dynamic object rotating through the grid receives correct color shifts, soft gradients, and directional cues from all light sources. + +### Comparison with Source Engine + +Source Engine uses **ambient cubes** (6 axis-aligned RGB values, 18 bytes). SH L2 (9 RGB float coefficients) provides strictly higher angular resolution for comparable storage and is the industry standard used by UE4/5, Unity, Godot, and idTech 6/7. + +--- + +## Building Maps with SH Grid + +### Basic Usage + +```bash +q3map2 -light -sh maps/mymap.bsp +``` + +The `-sh` flag enables SH grid computation alongside the standard light grid. Both are written to the BSP. + +### Command-Line Options + +| Flag | Arguments | Default | Description | +|------|-----------|---------|-------------| +| `-sh` | none | off | Enable SH light grid generation | +| `-gridscalesh` | `` | `2.0` | Density multiplier relative to legacy grid. `2.0` means half the cell size = 2x the resolution per axis = 8x total points | +| `-gridsh` | ` ` | derived | Explicit SH grid cell size in world units. Overrides `-gridscalesh` | + +### Resolution Control + +By default, the SH grid cell size is derived from the legacy grid: + +``` +gridSizeSH = gridSize / gridScaleSH +``` + +With default values: legacy grid = `64 64 128`, scale = `2.0`, so SH grid = `32 32 64`. + +The minimum cell size is 8 units per axis (hard floor). + +**Examples:** + +```bash +# Default: 2x density (32x32x64 cells) +q3map2 -light -sh maps/mymap.bsp + +# 4x density per axis (16x16x32 cells) -- high quality, large maps may be slow +q3map2 -light -sh -gridscalesh 4 maps/mymap.bsp + +# Same density as legacy grid (for debugging/comparison) +q3map2 -light -sh -gridscalesh 1 maps/mymap.bsp + +# Explicit cell size +q3map2 -light -sh -gridsh 24 24 48 maps/mymap.bsp +``` + +### With Radiosity Bounces + +The SH grid receives bounced (radiosity) light when `-bouncegrid` is enabled: + +```bash +q3map2 -light -sh -bounce 3 -bouncegrid maps/mymap.bsp +``` + +Each bounce pass re-traces the SH grid with the newly generated diffuse lights. + +### With Other Features + +The SH grid is independent of and compatible with all existing light features: + +```bash +# Full quality build with SH grid, bounces, dirt, and deluxemaps +q3map2 -light -sh -bounce 2 -bouncegrid -dirty -deluxe -fast maps/mymap.bsp +``` + +### Worldspawn Entity Keys + +The legacy grid size can be set per-map via the worldspawn entity: + +``` +"gridsize" "48 48 96" +``` + +If `-gridsh` is not explicitly given on the command line, the SH grid cell size is derived from this worldspawn value using the `-gridscalesh` multiplier. + +--- + +## BSP Lump Format + +### Lump Index + +``` +LUMP_LIGHTGRID_SH = 18 +``` + +This extends the IBSP header from 18 to 19 lumps. Stock Q3 engines read lumps 0-17 by index and never access lump 18 -- the BSP remains fully compatible. + +### Lump Layout + +The lump contains a fixed-size header followed by a flat array of grid points: + +``` +[bspGridSHHeader_t] [bspGridPointSH_t × numPoints] +``` + +### Header: `bspGridSHHeader_t` (40 bytes) + +```c +struct bspGridSHHeader_t +{ + float gridMins[3]; // World-space origin of the SH grid (12 bytes) + float gridSize[3]; // Cell size per axis in world units (12 bytes) + int gridBounds[3]; // Number of grid points per axis (12 bytes) + int numPoints; // Total grid points following header (4 bytes) +}; +``` + +| Offset | Size | Type | Field | Description | +|--------|------|------|-------|-------------| +| 0 | 12 | float[3] | `gridMins` | World-space position of grid point (0,0,0) | +| 12 | 12 | float[3] | `gridSize` | Cell dimensions in world units (e.g. 32, 32, 64) | +| 24 | 12 | int[3] | `gridBounds` | Grid dimensions in cells (e.g. 64, 48, 16) | +| 36 | 4 | int | `numPoints` | Total number of points. Must equal `gridBounds[0] * gridBounds[1] * gridBounds[2]` | + +### Grid Point: `bspGridPointSH_t` (108 bytes) + +```c +struct bspGridPointSH_t +{ + float coeffs[9][3]; // 9 SH coefficients, each an RGB triplet (108 bytes) +}; +``` + +Each grid point stores 9 L2 SH coefficients as 32-bit floats, with 3 color channels (RGB) per coefficient. Memory layout is coefficient-major: + +``` +coeffs[0] = { R, G, B } // Y0 (l=0, m=0) -- DC / average +coeffs[1] = { R, G, B } // Y1 (l=1, m=-1) +coeffs[2] = { R, G, B } // Y2 (l=1, m=0) +coeffs[3] = { R, G, B } // Y3 (l=1, m=1) +coeffs[4] = { R, G, B } // Y4 (l=2, m=-2) +coeffs[5] = { R, G, B } // Y5 (l=2, m=-1) +coeffs[6] = { R, G, B } // Y6 (l=2, m=0) +coeffs[7] = { R, G, B } // Y7 (l=2, m=1) +coeffs[8] = { R, G, B } // Y8 (l=2, m=2) +``` + +Total lump size = 40 + (108 * numPoints) bytes. + +### Grid Point Indexing + +Grid points are stored in a flat array, indexed as: + +``` +index = x + y * gridBounds[0] + z * gridBounds[0] * gridBounds[1] +``` + +Where `x`, `y`, `z` are integer grid coordinates in range `[0, gridBounds[i])`. + +The world-space position of grid point `(x, y, z)` is: + +``` +position = gridMins + float3(x, y, z) * gridSize +``` + +### Byte Order + +All values are stored in the native byte order of the platform (little-endian on x86/x64). This matches the existing IBSP convention. + +### Empty Lump + +If the map was not compiled with `-sh`, the lump has zero length (offset and length both set to 0 in the header). Engines should check `lump.length > sizeof(bspGridSHHeader_t)` before reading. + +--- + +## SH Basis Functions + +The implementation uses real-valued, orthonormal L2 spherical harmonics in Cartesian form. Given a normalized direction vector `(x, y, z)`: + +| Index | Band | Order | Basis Function | Constant | +|-------|------|-------|---------------|----------| +| 0 | l=0 | m=0 | `1` | 0.282095 | +| 1 | l=1 | m=-1 | `y` | 0.488603 | +| 2 | l=1 | m=0 | `z` | 0.488603 | +| 3 | l=1 | m=1 | `x` | 0.488603 | +| 4 | l=2 | m=-2 | `x*y` | 1.092548 | +| 5 | l=2 | m=-1 | `y*z` | 1.092548 | +| 6 | l=2 | m=0 | `3*z*z - 1` | 0.315392 | +| 7 | l=2 | m=1 | `x*z` | 1.092548 | +| 8 | l=2 | m=2 | `x*x - y*y` | 0.546274 | + +Full precision constants: + +```c +#define SH_C0 0.282094791773878f // 1 / (2*sqrt(pi)) +#define SH_C1 0.488602511902920f // sqrt(3) / (2*sqrt(pi)) +#define SH_C2 1.092548430592079f // sqrt(15) / (2*sqrt(pi)) +#define SH_C3 0.315391565252520f // sqrt(5) / (4*sqrt(pi)) +#define SH_C4 0.546274215296040f // sqrt(15) / (4*sqrt(pi)) +``` + +### Coordinate System + +The SH basis uses the same coordinate system as the BSP: +- **X** = forward/east +- **Y** = left/north +- **Z** = up + +This matches Q3/idTech conventions. If your engine uses a different coordinate system, transform the direction vector before evaluating the SH basis. + +--- + +## Engine Integration Guide + +### Step 1: Loading the Lump + +```c +// After loading the BSP file into memory: +typedef struct { + float gridMins[3]; + float gridSize[3]; + int gridBounds[3]; + int numPoints; +} shGridHeader_t; + +typedef struct { + float coeffs[9][3]; // [coeff_index][rgb_channel] +} shGridPoint_t; + +// Lump 18 in the BSP header +#define LUMP_LIGHTGRID_SH 18 + +static shGridHeader_t shHeader; +static shGridPoint_t *shGridPoints; +static qboolean shGridAvailable; + +void LoadSHGrid(const byte *bspData, const lump_t *lump) { + if (lump->filelen <= sizeof(shGridHeader_t)) { + shGridAvailable = qfalse; + return; + } + + const byte *data = bspData + lump->fileofs; + + // Read header + memcpy(&shHeader, data, sizeof(shGridHeader_t)); + data += sizeof(shGridHeader_t); + + // Read points + shGridPoints = malloc(shHeader.numPoints * sizeof(shGridPoint_t)); + memcpy(shGridPoints, data, shHeader.numPoints * sizeof(shGridPoint_t)); + + shGridAvailable = qtrue; +} +``` + +### Step 2: Trilinear Sampling + +Given a world-space position, sample the SH grid with trilinear interpolation: + +```c +void SampleSHGrid(const float position[3], float outCoeffs[9][3]) { + // Convert world position to grid-local floating-point coordinates + float fx = (position[0] - shHeader.gridMins[0]) / shHeader.gridSize[0]; + float fy = (position[1] - shHeader.gridMins[1]) / shHeader.gridSize[1]; + float fz = (position[2] - shHeader.gridMins[2]) / shHeader.gridSize[2]; + + // Integer grid coordinates and fractional parts + int x0 = (int)floorf(fx); + int y0 = (int)floorf(fy); + int z0 = (int)floorf(fz); + float dx = fx - x0; + float dy = fy - y0; + float dz = fz - z0; + + // Clamp to grid bounds + int x1 = x0 + 1; + int y1 = y0 + 1; + int z1 = z0 + 1; + x0 = CLAMP(x0, 0, shHeader.gridBounds[0] - 1); + x1 = CLAMP(x1, 0, shHeader.gridBounds[0] - 1); + y0 = CLAMP(y0, 0, shHeader.gridBounds[1] - 1); + y1 = CLAMP(y1, 0, shHeader.gridBounds[1] - 1); + z0 = CLAMP(z0, 0, shHeader.gridBounds[2] - 1); + z1 = CLAMP(z1, 0, shHeader.gridBounds[2] - 1); + + // Fetch 8 corner grid points + #define SH_INDEX(X, Y, Z) ((X) + (Y) * shHeader.gridBounds[0] + \ + (Z) * shHeader.gridBounds[0] * shHeader.gridBounds[1]) + + const shGridPoint_t *corners[8] = { + &shGridPoints[SH_INDEX(x0, y0, z0)], + &shGridPoints[SH_INDEX(x1, y0, z0)], + &shGridPoints[SH_INDEX(x0, y1, z0)], + &shGridPoints[SH_INDEX(x1, y1, z0)], + &shGridPoints[SH_INDEX(x0, y0, z1)], + &shGridPoints[SH_INDEX(x1, y0, z1)], + &shGridPoints[SH_INDEX(x0, y1, z1)], + &shGridPoints[SH_INDEX(x1, y1, z1)], + }; + #undef SH_INDEX + + // Trilinear interpolation weights + float weights[8] = { + (1-dx) * (1-dy) * (1-dz), + dx * (1-dy) * (1-dz), + (1-dx) * dy * (1-dz), + dx * dy * (1-dz), + (1-dx) * (1-dy) * dz, + dx * (1-dy) * dz, + (1-dx) * dy * dz, + dx * dy * dz, + }; + + // Blend SH coefficients + memset(outCoeffs, 0, 9 * 3 * sizeof(float)); + for (int c = 0; c < 8; c++) { + for (int i = 0; i < 9; i++) { + outCoeffs[i][0] += corners[c]->coeffs[i][0] * weights[c]; + outCoeffs[i][1] += corners[c]->coeffs[i][1] * weights[c]; + outCoeffs[i][2] += corners[c]->coeffs[i][2] * weights[c]; + } + } +} +``` + +### Step 3: Evaluating Lighting from SH + +Given the interpolated SH coefficients and a world-space surface normal, reconstruct the lighting color: + +```c +void EvaluateSH(const float coeffs[9][3], const float normal[3], float outColor[3]) { + float x = normal[0], y = normal[1], z = normal[2]; + + // Evaluate SH basis functions + float basis[9]; + basis[0] = 0.282095f; + basis[1] = 0.488603f * y; + basis[2] = 0.488603f * z; + basis[3] = 0.488603f * x; + basis[4] = 1.092548f * x * y; + basis[5] = 1.092548f * y * z; + basis[6] = 0.315392f * (3.0f * z * z - 1.0f); + basis[7] = 1.092548f * x * z; + basis[8] = 0.546274f * (x * x - y * y); + + // Dot product: sum(coeffs[i] * basis[i]) per channel + outColor[0] = outColor[1] = outColor[2] = 0.0f; + for (int i = 0; i < 9; i++) { + outColor[0] += coeffs[i][0] * basis[i]; + outColor[1] += coeffs[i][1] * basis[i]; + outColor[2] += coeffs[i][2] * basis[i]; + } + + // Clamp negative values (SH ringing artifact) + if (outColor[0] < 0.0f) outColor[0] = 0.0f; + if (outColor[1] < 0.0f) outColor[1] = 0.0f; + if (outColor[2] < 0.0f) outColor[2] = 0.0f; +} +``` + +### Step 4: GLSL Shader (GPU evaluation) + +For GPU-side evaluation, pass the interpolated SH coefficients as a uniform or SSBO: + +```glsl +// Vertex or fragment shader +// shCoeffs[9] are vec3 uniforms, interpolated per-object or per-vertex + +vec3 evaluateSH(vec3 shCoeffs[9], vec3 normal) { + float x = normal.x, y = normal.y, z = normal.z; + + vec3 color = + shCoeffs[0] * 0.282095 + + shCoeffs[1] * (0.488603 * y) + + shCoeffs[2] * (0.488603 * z) + + shCoeffs[3] * (0.488603 * x) + + shCoeffs[4] * (1.092548 * x * y) + + shCoeffs[5] * (1.092548 * y * z) + + shCoeffs[6] * (0.315392 * (3.0 * z * z - 1.0)) + + shCoeffs[7] * (1.092548 * x * z) + + shCoeffs[8] * (0.546274 * (x * x - y * y)); + + return max(color, vec3(0.0)); +} +``` + +For per-vertex lighting, the engine can sample the SH grid per vertex on the CPU and pass per-vertex colors. For per-pixel lighting on dynamic objects, pass the 9 SH coefficients as uniforms (9 * vec3 = 9 * 12 = 108 bytes per object). + +### Step 5: Fallback + +If the SH lump is not present (stock BSP, or compiled without `-sh`): + +```c +if (!shGridAvailable) { + // Fall back to legacy Q3 light grid (lump 15) + R_LightGrid_Legacy(position, normal, outColor); +} +``` + +--- + +## Memory and Performance + +### Storage Costs + +| Grid Cell Size | Typical Map (4096x4096x1024) | Points | Legacy Size | SH Size | +|---------------|------------------------------|--------|-------------|---------| +| 64x64x128 (legacy default) | 64 x 64 x 8 | 32,768 | 262 KB | -- | +| 32x32x64 (SH default, 2x) | 128 x 128 x 16 | 262,144 | -- | 27 MB | +| 48x48x96 | 86 x 86 x 11 | 81,356 | -- | 8.4 MB | +| 16x16x32 (4x) | 256 x 256 x 32 | 2,097,152 | -- | 216 MB | + +Each SH grid point is **108 bytes** (9 coefficients * 3 floats * 4 bytes). The 2x default is a good balance between quality and size. + +For very large maps, consider reducing density with `-gridscalesh 1.5` or using explicit cell sizes. + +### Compilation Time + +SH grid tracing uses the same `LightContributionToPoint()` function as the legacy grid, tracing all lights per grid point. At 2x density (8x more points), expect roughly 8x the legacy grid computation time. This is typically a small fraction of total light compilation time (lightmap illumination dominates). + +### Runtime Performance + +SH evaluation is 9 multiply-adds per channel = 27 total operations per grid sample. Trilinear interpolation requires 8 grid lookups. Total cost per dynamic object is comparable to 8 texture fetches + a small ALU workload -- negligible on modern hardware. + +--- + +## JSON Export/Import + +The SH grid data is included in the JSON export/import pipeline: + +```bash +# Export BSP to JSON (includes GridPointsSH.json if SH data present) +q3map2 -json -unpack maps/mymap.bsp + +# Import JSON back to BSP +q3map2 -json -pack maps/mymap.bsp +``` + +### JSON Format + +`GridPointsSH.json`: + +```json +{ + "header": { + "gridMins": [x, y, z], + "gridSize": [sx, sy, sz], + "gridBounds": [bx, by, bz] + }, + "points": [ + [ + [r, g, b], // coeffs[0] (Y0, DC) + [r, g, b], // coeffs[1] (Y1) + [r, g, b], // coeffs[2] (Y2) + [r, g, b], // coeffs[3] (Y3) + [r, g, b], // coeffs[4] (Y4) + [r, g, b], // coeffs[5] (Y5) + [r, g, b], // coeffs[6] (Y6) + [r, g, b], // coeffs[7] (Y7) + [r, g, b] // coeffs[8] (Y8) + ], + // ... one array of 9 RGB triplets per grid point + ] +} +``` + +--- + +## Source Files + +| File | Description | +|------|-------------| +| `q3map2/q3map2/sh.h` | SH math: basis evaluation, projection, reconstruction | +| `q3map2/q3map2/q3map2.h` | `bspGridSHHeader_t`, `bspGridPointSH_t` structs, SH grid globals | +| `q3map2/q3map2/light.cpp` | `SetupGridSH()`, `TraceGridSH()`, CLI parsing, pipeline integration | +| `q3map2/q3map2/bspfile_ibsp.cpp` | BSP lump read/write for `LUMP_LIGHTGRID_SH` | +| `q3map2/q3map2/bspfile_abstract.cpp` | BSP info stats for SH grid | +| `q3map2/q3map2/convert_json.cpp` | JSON export/import for SH grid data | + +--- + +## References + +- Peter-Pike Sloan, "Stupid Spherical Harmonics (SH) Tricks", Microsoft, 2008 +- Ravi Ramamoorthi and Pat Hanrahan, "An Efficient Representation for Irradiance Environment Maps", SIGGRAPH 2001 +- Robin Green, "Spherical Harmonic Lighting: The Gritty Details", GDC 2003 +- Quake 3 BSP format specification (IBSP v46/v47)