bloodrun-editor/sh.md
serge_shubin 6f78fcb452 Add L2 spherical harmonics light grid to q3map2
Implements a new SH light grid that runs alongside the legacy Q3 light grid,
storing 9 RGB L2 spherical harmonic coefficients per grid point for accurate
directional lighting of dynamic objects from all angles.

BSP format: v47 with 19-lump header (160 bytes) when -sh is used, v46 with
17-lump header (144 bytes) otherwise. SH data stored in LUMP_LIGHTGRID_SH
(index 18) with a header containing grid bounds/size/mins followed by the
coefficient array. Stock Q3 engines read v46 lumps unchanged.

New CLI flags: -sh (enable), -gridscalesh (density multiplier, default 2x),
-gridsh (explicit cell size). SH grid receives bounced light with -bouncegrid.

Also adds libjpeg-turbo as a proper build dependency with its own vcxproj,
fixing the previous external engine path requirement.
2026-04-03 13:52:07 +08:00

16 KiB
Raw Blame History

Spherical Harmonics Light Grid (SH Grid)

Overview

The SH light grid is an extension to q3map2's lighting system that stores L2 spherical harmonics (9 RGB coefficients per grid point) alongside the legacy Quake 3 light grid. It encodes a smooth directional lighting function over the full sphere at each grid point, enabling dynamic objects to receive accurate directional lighting from all angles -- not just a single dominant direction as in the legacy grid.

The SH grid is stored in a new BSP lump (LUMP_LIGHTGRID_SH, index 18) that stock Quake 3 engines ignore, preserving full backward compatibility.

Why SH over the Legacy Grid?

The legacy Q3 light grid stores per point:

  • ambient (RGB byte) -- omnidirectional ambient term
  • directed (RGB byte) -- directional light color
  • latLong (2 bytes) -- single dominant light direction

The engine evaluates: result = ambient + directed * max(0, dot(normal, decode(latLong)))

This collapses all directional information into one direction. A point lit by a red light from the left and a blue light from the right produces a muddy average -- a dynamic object turning in place cannot see the color shift.

L2 SH encodes 9 directional basis functions that capture smooth angular variation across the entire sphere. A dynamic object rotating through the grid receives correct color shifts, soft gradients, and directional cues from all light sources.

Comparison with Source Engine

Source Engine uses ambient cubes (6 axis-aligned RGB values, 18 bytes). SH L2 (9 RGB float coefficients) provides strictly higher angular resolution for comparable storage and is the industry standard used by UE4/5, Unity, Godot, and idTech 6/7.


Building Maps with SH Grid

Basic Usage

q3map2 -light -sh maps/mymap.bsp

The -sh flag enables SH grid computation alongside the standard light grid. Both are written to the BSP.

Command-Line Options

Flag Arguments Default Description
-sh none off Enable SH light grid generation
-gridscalesh <float> 2.0 Density multiplier relative to legacy grid. 2.0 means half the cell size = 2x the resolution per axis = 8x total points
-gridsh <X> <Y> <Z> derived Explicit SH grid cell size in world units. Overrides -gridscalesh

Resolution Control

By default, the SH grid cell size is derived from the legacy grid:

gridSizeSH = gridSize / gridScaleSH

With default values: legacy grid = 64 64 128, scale = 2.0, so SH grid = 32 32 64.

The minimum cell size is 8 units per axis (hard floor).

Examples:

# Default: 2x density (32x32x64 cells)
q3map2 -light -sh maps/mymap.bsp

# 4x density per axis (16x16x32 cells) -- high quality, large maps may be slow
q3map2 -light -sh -gridscalesh 4 maps/mymap.bsp

# Same density as legacy grid (for debugging/comparison)
q3map2 -light -sh -gridscalesh 1 maps/mymap.bsp

# Explicit cell size
q3map2 -light -sh -gridsh 24 24 48 maps/mymap.bsp

With Radiosity Bounces

The SH grid receives bounced (radiosity) light when -bouncegrid is enabled:

q3map2 -light -sh -bounce 3 -bouncegrid maps/mymap.bsp

Each bounce pass re-traces the SH grid with the newly generated diffuse lights.

With Other Features

The SH grid is independent of and compatible with all existing light features:

# Full quality build with SH grid, bounces, dirt, and deluxemaps
q3map2 -light -sh -bounce 2 -bouncegrid -dirty -deluxe -fast maps/mymap.bsp

Worldspawn Entity Keys

The legacy grid size can be set per-map via the worldspawn entity:

"gridsize" "48 48 96"

If -gridsh is not explicitly given on the command line, the SH grid cell size is derived from this worldspawn value using the -gridscalesh multiplier.


BSP Lump Format

Lump Index

LUMP_LIGHTGRID_SH = 18

This extends the IBSP header from 18 to 19 lumps. Stock Q3 engines read lumps 0-17 by index and never access lump 18 -- the BSP remains fully compatible.

Lump Layout

The lump contains a fixed-size header followed by a flat array of grid points:

[bspGridSHHeader_t] [bspGridPointSH_t × numPoints]

Header: bspGridSHHeader_t (40 bytes)

struct bspGridSHHeader_t
{
    float gridMins[3];      // World-space origin of the SH grid (12 bytes)
    float gridSize[3];      // Cell size per axis in world units   (12 bytes)
    int   gridBounds[3];    // Number of grid points per axis      (12 bytes)
    int   numPoints;        // Total grid points following header   (4 bytes)
};
Offset Size Type Field Description
0 12 float[3] gridMins World-space position of grid point (0,0,0)
12 12 float[3] gridSize Cell dimensions in world units (e.g. 32, 32, 64)
24 12 int[3] gridBounds Grid dimensions in cells (e.g. 64, 48, 16)
36 4 int numPoints Total number of points. Must equal gridBounds[0] * gridBounds[1] * gridBounds[2]

Grid Point: bspGridPointSH_t (108 bytes)

struct bspGridPointSH_t
{
    float coeffs[9][3];     // 9 SH coefficients, each an RGB triplet (108 bytes)
};

Each grid point stores 9 L2 SH coefficients as 32-bit floats, with 3 color channels (RGB) per coefficient. Memory layout is coefficient-major:

coeffs[0] = { R, G, B }    // Y0  (l=0, m=0)  -- DC / average
coeffs[1] = { R, G, B }    // Y1  (l=1, m=-1)
coeffs[2] = { R, G, B }    // Y2  (l=1, m=0)
coeffs[3] = { R, G, B }    // Y3  (l=1, m=1)
coeffs[4] = { R, G, B }    // Y4  (l=2, m=-2)
coeffs[5] = { R, G, B }    // Y5  (l=2, m=-1)
coeffs[6] = { R, G, B }    // Y6  (l=2, m=0)
coeffs[7] = { R, G, B }    // Y7  (l=2, m=1)
coeffs[8] = { R, G, B }    // Y8  (l=2, m=2)

Total lump size = 40 + (108 * numPoints) bytes.

Grid Point Indexing

Grid points are stored in a flat array, indexed as:

index = x + y * gridBounds[0] + z * gridBounds[0] * gridBounds[1]

Where x, y, z are integer grid coordinates in range [0, gridBounds[i]).

The world-space position of grid point (x, y, z) is:

position = gridMins + float3(x, y, z) * gridSize

Byte Order

All values are stored in the native byte order of the platform (little-endian on x86/x64). This matches the existing IBSP convention.

Empty Lump

If the map was not compiled with -sh, the lump has zero length (offset and length both set to 0 in the header). Engines should check lump.length > sizeof(bspGridSHHeader_t) before reading.


SH Basis Functions

The implementation uses real-valued, orthonormal L2 spherical harmonics in Cartesian form. Given a normalized direction vector (x, y, z):

Index Band Order Basis Function Constant
0 l=0 m=0 1 0.282095
1 l=1 m=-1 y 0.488603
2 l=1 m=0 z 0.488603
3 l=1 m=1 x 0.488603
4 l=2 m=-2 x*y 1.092548
5 l=2 m=-1 y*z 1.092548
6 l=2 m=0 3*z*z - 1 0.315392
7 l=2 m=1 x*z 1.092548
8 l=2 m=2 x*x - y*y 0.546274

Full precision constants:

#define SH_C0  0.282094791773878f   // 1 / (2*sqrt(pi))
#define SH_C1  0.488602511902920f   // sqrt(3) / (2*sqrt(pi))
#define SH_C2  1.092548430592079f   // sqrt(15) / (2*sqrt(pi))
#define SH_C3  0.315391565252520f   // sqrt(5) / (4*sqrt(pi))
#define SH_C4  0.546274215296040f   // sqrt(15) / (4*sqrt(pi))

Coordinate System

The SH basis uses the same coordinate system as the BSP:

  • X = forward/east
  • Y = left/north
  • Z = up

This matches Q3/idTech conventions. If your engine uses a different coordinate system, transform the direction vector before evaluating the SH basis.


Engine Integration Guide

Step 1: Loading the Lump

// After loading the BSP file into memory:
typedef struct {
    float gridMins[3];
    float gridSize[3];
    int   gridBounds[3];
    int   numPoints;
} shGridHeader_t;

typedef struct {
    float coeffs[9][3];    // [coeff_index][rgb_channel]
} shGridPoint_t;

// Lump 18 in the BSP header
#define LUMP_LIGHTGRID_SH 18

static shGridHeader_t  shHeader;
static shGridPoint_t  *shGridPoints;
static qboolean        shGridAvailable;

void LoadSHGrid(const byte *bspData, const lump_t *lump) {
    if (lump->filelen <= sizeof(shGridHeader_t)) {
        shGridAvailable = qfalse;
        return;
    }

    const byte *data = bspData + lump->fileofs;

    // Read header
    memcpy(&shHeader, data, sizeof(shGridHeader_t));
    data += sizeof(shGridHeader_t);

    // Read points
    shGridPoints = malloc(shHeader.numPoints * sizeof(shGridPoint_t));
    memcpy(shGridPoints, data, shHeader.numPoints * sizeof(shGridPoint_t));

    shGridAvailable = qtrue;
}

Step 2: Trilinear Sampling

Given a world-space position, sample the SH grid with trilinear interpolation:

void SampleSHGrid(const float position[3], float outCoeffs[9][3]) {
    // Convert world position to grid-local floating-point coordinates
    float fx = (position[0] - shHeader.gridMins[0]) / shHeader.gridSize[0];
    float fy = (position[1] - shHeader.gridMins[1]) / shHeader.gridSize[1];
    float fz = (position[2] - shHeader.gridMins[2]) / shHeader.gridSize[2];

    // Integer grid coordinates and fractional parts
    int x0 = (int)floorf(fx);
    int y0 = (int)floorf(fy);
    int z0 = (int)floorf(fz);
    float dx = fx - x0;
    float dy = fy - y0;
    float dz = fz - z0;

    // Clamp to grid bounds
    int x1 = x0 + 1;
    int y1 = y0 + 1;
    int z1 = z0 + 1;
    x0 = CLAMP(x0, 0, shHeader.gridBounds[0] - 1);
    x1 = CLAMP(x1, 0, shHeader.gridBounds[0] - 1);
    y0 = CLAMP(y0, 0, shHeader.gridBounds[1] - 1);
    y1 = CLAMP(y1, 0, shHeader.gridBounds[1] - 1);
    z0 = CLAMP(z0, 0, shHeader.gridBounds[2] - 1);
    z1 = CLAMP(z1, 0, shHeader.gridBounds[2] - 1);

    // Fetch 8 corner grid points
    #define SH_INDEX(X, Y, Z) ((X) + (Y) * shHeader.gridBounds[0] + \
                               (Z) * shHeader.gridBounds[0] * shHeader.gridBounds[1])

    const shGridPoint_t *corners[8] = {
        &shGridPoints[SH_INDEX(x0, y0, z0)],
        &shGridPoints[SH_INDEX(x1, y0, z0)],
        &shGridPoints[SH_INDEX(x0, y1, z0)],
        &shGridPoints[SH_INDEX(x1, y1, z0)],
        &shGridPoints[SH_INDEX(x0, y0, z1)],
        &shGridPoints[SH_INDEX(x1, y0, z1)],
        &shGridPoints[SH_INDEX(x0, y1, z1)],
        &shGridPoints[SH_INDEX(x1, y1, z1)],
    };
    #undef SH_INDEX

    // Trilinear interpolation weights
    float weights[8] = {
        (1-dx) * (1-dy) * (1-dz),
           dx  * (1-dy) * (1-dz),
        (1-dx) *    dy  * (1-dz),
           dx  *    dy  * (1-dz),
        (1-dx) * (1-dy) *    dz,
           dx  * (1-dy) *    dz,
        (1-dx) *    dy  *    dz,
           dx  *    dy  *    dz,
    };

    // Blend SH coefficients
    memset(outCoeffs, 0, 9 * 3 * sizeof(float));
    for (int c = 0; c < 8; c++) {
        for (int i = 0; i < 9; i++) {
            outCoeffs[i][0] += corners[c]->coeffs[i][0] * weights[c];
            outCoeffs[i][1] += corners[c]->coeffs[i][1] * weights[c];
            outCoeffs[i][2] += corners[c]->coeffs[i][2] * weights[c];
        }
    }
}

Step 3: Evaluating Lighting from SH

Given the interpolated SH coefficients and a world-space surface normal, reconstruct the lighting color:

void EvaluateSH(const float coeffs[9][3], const float normal[3], float outColor[3]) {
    float x = normal[0], y = normal[1], z = normal[2];

    // Evaluate SH basis functions
    float basis[9];
    basis[0] = 0.282095f;
    basis[1] = 0.488603f * y;
    basis[2] = 0.488603f * z;
    basis[3] = 0.488603f * x;
    basis[4] = 1.092548f * x * y;
    basis[5] = 1.092548f * y * z;
    basis[6] = 0.315392f * (3.0f * z * z - 1.0f);
    basis[7] = 1.092548f * x * z;
    basis[8] = 0.546274f * (x * x - y * y);

    // Dot product: sum(coeffs[i] * basis[i]) per channel
    outColor[0] = outColor[1] = outColor[2] = 0.0f;
    for (int i = 0; i < 9; i++) {
        outColor[0] += coeffs[i][0] * basis[i];
        outColor[1] += coeffs[i][1] * basis[i];
        outColor[2] += coeffs[i][2] * basis[i];
    }

    // Clamp negative values (SH ringing artifact)
    if (outColor[0] < 0.0f) outColor[0] = 0.0f;
    if (outColor[1] < 0.0f) outColor[1] = 0.0f;
    if (outColor[2] < 0.0f) outColor[2] = 0.0f;
}

Step 4: GLSL Shader (GPU evaluation)

For GPU-side evaluation, pass the interpolated SH coefficients as a uniform or SSBO:

// Vertex or fragment shader
// shCoeffs[9] are vec3 uniforms, interpolated per-object or per-vertex

vec3 evaluateSH(vec3 shCoeffs[9], vec3 normal) {
    float x = normal.x, y = normal.y, z = normal.z;

    vec3 color =
        shCoeffs[0] * 0.282095 +
        shCoeffs[1] * (0.488603 * y) +
        shCoeffs[2] * (0.488603 * z) +
        shCoeffs[3] * (0.488603 * x) +
        shCoeffs[4] * (1.092548 * x * y) +
        shCoeffs[5] * (1.092548 * y * z) +
        shCoeffs[6] * (0.315392 * (3.0 * z * z - 1.0)) +
        shCoeffs[7] * (1.092548 * x * z) +
        shCoeffs[8] * (0.546274 * (x * x - y * y));

    return max(color, vec3(0.0));
}

For per-vertex lighting, the engine can sample the SH grid per vertex on the CPU and pass per-vertex colors. For per-pixel lighting on dynamic objects, pass the 9 SH coefficients as uniforms (9 * vec3 = 9 * 12 = 108 bytes per object).

Step 5: Fallback

If the SH lump is not present (stock BSP, or compiled without -sh):

if (!shGridAvailable) {
    // Fall back to legacy Q3 light grid (lump 15)
    R_LightGrid_Legacy(position, normal, outColor);
}

Memory and Performance

Storage Costs

Grid Cell Size Typical Map (4096x4096x1024) Points Legacy Size SH Size
64x64x128 (legacy default) 64 x 64 x 8 32,768 262 KB --
32x32x64 (SH default, 2x) 128 x 128 x 16 262,144 -- 27 MB
48x48x96 86 x 86 x 11 81,356 -- 8.4 MB
16x16x32 (4x) 256 x 256 x 32 2,097,152 -- 216 MB

Each SH grid point is 108 bytes (9 coefficients * 3 floats * 4 bytes). The 2x default is a good balance between quality and size.

For very large maps, consider reducing density with -gridscalesh 1.5 or using explicit cell sizes.

Compilation Time

SH grid tracing uses the same LightContributionToPoint() function as the legacy grid, tracing all lights per grid point. At 2x density (8x more points), expect roughly 8x the legacy grid computation time. This is typically a small fraction of total light compilation time (lightmap illumination dominates).

Runtime Performance

SH evaluation is 9 multiply-adds per channel = 27 total operations per grid sample. Trilinear interpolation requires 8 grid lookups. Total cost per dynamic object is comparable to 8 texture fetches + a small ALU workload -- negligible on modern hardware.


JSON Export/Import

The SH grid data is included in the JSON export/import pipeline:

# Export BSP to JSON (includes GridPointsSH.json if SH data present)
q3map2 -json -unpack maps/mymap.bsp

# Import JSON back to BSP
q3map2 -json -pack maps/mymap.bsp

JSON Format

GridPointsSH.json:

{
    "header": {
        "gridMins": [x, y, z],
        "gridSize": [sx, sy, sz],
        "gridBounds": [bx, by, bz]
    },
    "points": [
        [
            [r, g, b],   // coeffs[0] (Y0, DC)
            [r, g, b],   // coeffs[1] (Y1)
            [r, g, b],   // coeffs[2] (Y2)
            [r, g, b],   // coeffs[3] (Y3)
            [r, g, b],   // coeffs[4] (Y4)
            [r, g, b],   // coeffs[5] (Y5)
            [r, g, b],   // coeffs[6] (Y6)
            [r, g, b],   // coeffs[7] (Y7)
            [r, g, b]    // coeffs[8] (Y8)
        ],
        // ... one array of 9 RGB triplets per grid point
    ]
}

Source Files

File Description
q3map2/q3map2/sh.h SH math: basis evaluation, projection, reconstruction
q3map2/q3map2/q3map2.h bspGridSHHeader_t, bspGridPointSH_t structs, SH grid globals
q3map2/q3map2/light.cpp SetupGridSH(), TraceGridSH(), CLI parsing, pipeline integration
q3map2/q3map2/bspfile_ibsp.cpp BSP lump read/write for LUMP_LIGHTGRID_SH
q3map2/q3map2/bspfile_abstract.cpp BSP info stats for SH grid
q3map2/q3map2/convert_json.cpp JSON export/import for SH grid data

References

  • Peter-Pike Sloan, "Stupid Spherical Harmonics (SH) Tricks", Microsoft, 2008
  • Ravi Ramamoorthi and Pat Hanrahan, "An Efficient Representation for Irradiance Environment Maps", SIGGRAPH 2001
  • Robin Green, "Spherical Harmonic Lighting: The Gritty Details", GDC 2003
  • Quake 3 BSP format specification (IBSP v46/v47)