Lighting in 3D

I’ve been experimenting with WebGL recently; my initial ambition was to use it for a 2D interface that needs to be very highly performant, but I very quickly fell down a rabbit-hole of 3D graphics. I’m very bad at it! Despite this, though, it’s much simpler it is than I had expected. 3D rendering has always seemed like literally magic to me, so it’s reassuring to learn that, while it’s complicated and difficult, it’s just about understandable with some reading and my very limited knowledge of linear algebra.

The results are very appealing. Of course, it’s nice that there’s something pretty to look at. I’ve written ray tracers in the past, and while it’s satisfying to make something that produces an image of some orbs from first principles, there’s something neater to me about getting much the same result with smoke, mirrors and triangles. I have enjoyed learning that you can approximate something realistic with a little maths that does a fraction of the work of a “real” simulation.

Lighting is a good example of this. With a ray tracer the basic approach is to model the way light interacts with surfaces in a semi-realistic way, point some lights at it and record the result. With more traditional 3D graphics, we have the fragment shader: a small program that runs on the graphics card and determines the colour of each pixel on each triangle that makes up the mesh. Lighting then becomes a question of deciding how light or dark a pixel should be, based on its position in the scene and the position and intensity of a notional light.

In a real scene, there is generally not a single source of light, so we layer on a few different calculations to simulate different lights, and different kinds of light, as well as reflections for reflective surfaces. You can play with an example below; click and drag on the teapot to rotate the camera around it, scroll to zoom in and out, and use the sliders to change the weighting of the different kinds of light in the overall image.

I used three.js for the demo above, which has a built-in version of the effects above. I implemented them by hand as a custom shader to get a better idea of how it works; I explain the various passes in turn below.

Useful functions

These are used in the code described below and are not native GLSL functions, so they bear some explaining. The first is inverseLerp, or “inverse linear interpolation”:

float inverseLerp(float v, float minValue, float maxValue) {
  return (v - minValue) / (maxValue - minValue);
}

inverseLerp is, predictably enough, the inverse of the built-in lerp function. Lerp takes a fraction and two values min and max, and returns the interpolation at that fraction between the two values. inverseLerp, by contrast, takes a value, a min and a max, and returns a fraction representing how far between min and max the value is.

We can use inverseLerp to also define remap:

float remap(float v, float inMin, float inMax, float outMin, float outMax) {
  float t = inverseLerp(v, inMin, inMax);
  return mix(outMin, outMax, t);
}

remap takes a value and two sets of bounds, inMin/inMax and outMin/outMax. It uses inverseLerp to determine how far between inMin and inMax the input value is, and returns a value that is the same distance between outMin and outMax.

With these functions defined, as well as a few GLSL builtins, we can achieve all of the lighting effects in the demo.

Ambient lighting

This is the simplest pass; it simulates the light in a space that has been so scattered that it doesn’t obviously come from any direction. Each pixel in the fragment shader is illuminated by a constant amount.

void main() {
  /* ... */
  vec3 lighting = 
      (settings.vAmbientColor * settings.fAmbientWeight);
  /* ... */
}

Here settings.vAmbientColor is a vec3, representing the red, blue and green channels of the light’s colour, and settings.fAmbientWeight is a float representing the light’s intensity. A value of 0.0 represents no light, and 1.0 the maximum amount.

Hemispheric lighting

This is similar to ambient lighting, but simulates the tendency for objects to be lit differently from above and from below; ambient light from above will tend to be lighter, and maybe blueish if outside in the daytime. Light scattered from below will be darker as it’s reflected from the ground. We can achieve this effect per-pixel by taking the fragment’s y-normal and using it to mix between the two colours. If a fragment points straight down it will be the floor colour, one pointing straight up will be the sky colour, and those in between will get an interpolation between the two.

varying vec3 vNormal; // Passed in from the vertex shader

void main() {
  /* ... */
  vec3 normal = normalize(vNormal);
  float hemiMix = remap(normal.y, -1.0, 1.0, 0.0, 1.0);
  vec3 hemi = mix(settings.vHemiGroundColour, settings.vHemiSkyColour, hemiMix);
  vec3 lighting = 
      (hemi * settings.fHemiWeight); 
  /* ... */
}

We use remap to map the normal between 0.0 and 1.0 before using it in mix.

Diffuse lighting

Diffuse lighting simulates a broad directional light pointing at our mesh. Fragments pointing directly at the light will be at full brightness, while those pointing away from the light are not lit at all. Those pointing at the light at an angle will be lit but with less intensity. This gives quite a natural looking shaded effect.

The direction of the light is defined by a single vec3. We can find the intensity of the light on a fragment by taking the dot product of the fragment’s normal vector with the light vector. The dot product of two normalised vectors will vary between -1.0 and 1.0; we clamp this value to have a minimum value of 0.0. This is because the dot product for normals pointing away from the light will be less than zero, and it doesn’t really make sense for a surface to be negatively lit; they’re just unlit.

varying vec3 vNormal; // Passed in from the vertex shader

void main() {
  /* ... */
  vec3 lightDir = normalize(settings.vDiffuseDirection);
  vec3 lightColour = settings.vDiffuseColour;
  float dp = max(0.0, dot(lightDir, normal));
  vec3 diffuse = dp * lightColour;

  vec3 lighting = 
      (diffuse * settings.fDiffuseWeight); 
  /* ... */
}

From the dot product we get a weight, which we can then multiply by the colour of the diffuse light. We multiply this by an overall weight representing the brightness of the light to get the finished effect.

Reflections

So far, we have a fairly good approximation of a matte, unreflective surface. If we want to model reflective surfaces, we need to model reflections. In a raytracer, we would do this by having our rays bounce off the surface, and sample the colour of whatever they went on to collide with. This is very computationally expensive, though, and we can approximate the same effect with a few tricks. These are more “artistic” than “physical”. We’re not really in the business of simulating reality; we’re just interested in getting an image that looks broadly right.

Specular highlights

The first of these tricks is specular highlights. Specular highlights mimic the bright spots that are visible on reflective surfaces when lights are shone at them. This concept in computer graphics was invented by Bui Tuong Phong and combines with ambient and diffuse lighting to make up the Phong reflection model.

void main() {
  /* ... */
  vec3 lightDir = normalize(settings.vDiffuseDirection); // Same lightDir as for diffuse above
  /* ... */
  vec3 r = normalize(reflect(-lightDir, normal));
  float phongValue = max(0.0, dot(viewDir, r));
  phongValue = pow(phongValue, 2.0);

  vec3 specular = vec3(phongValue) * settings.fPhongWeight;
  /* ... */

This code takes the lightDir vec3, the same one from diffuse lighting, and calculates a new vector reflected about the normal of the fragment; this represents the light reflecting off the surface. It then compares this vector with viewDir, which represents the direction the camera is facing, using the clamped dot product of the two vectors. When the vectors are aligned, a value of 1.0 is returned, and when they face away from each other, a value of 0.0 is returned. This translates to a bright surface when we are facing the reflected light directly, dropping off to nothing as the angle becomes more oblique. We raise the value by some arbitrary power (in this case 2.0) to increase or reduce the effect.

Image-based lighting

Phong highlights look quite good on their own, but if you look at them closely it does become quite clear that they’re not really reflecting anything. It would be nice, for mirror-like materials, if we could see the details of the reflected objects on the surface.

With a ray-tracer we’d get full reflections for “free”, albeit at the cost of a lot of processing power. We can get a similar effect at a fraction of the cost by sampling the texture of the dominant object in the environment, in this case the skybox, and layering it onto our specular highlight.

uniform samplerCube specMap; // skybox texture

void main() {
  /* ... */
  // IBL Specular
  vec3 iblCoord = normalize(reflect(-viewDir, normal));
  vec3 iblSample = textureCube(specMap, iblCoord).xyz;
  specular += iblSample * settings.fIblWeight;
}

We take the reflection of our camera vector around the normal of the fragment to find which pixel on the texture map should be visible, and then sample the texture at that point. Then we just add that colour to our specular highlight value.

This looks quite realistic in the scene above, although note that there’s nothing in the scene except the teapot and the skybox. In a busier scene, it would be more obvious that only the sky is reflected off the teapot’s surface, and other objects are ignored.

Fresnel

Another interesting fact about reflective surfaces is that they are more reflective at shallow angles than when looked at head-on. An example of this is a still body of water; looking straight down you can see into the water, whereas looking across the pool you see a reflection. This is called the fresnel effect. For our purposes, we can boil it down to the observation that, for reflective surfaces, the brightness of our specular highlights should increase with the shallowness of the angle from which we view it, and decrease the more head-on we are.

void main() {
  float fresnel = 1.0 - max(0.0, dot(viewDir, normal));
  fresnel = pow(fresnel, 2.0);
  specular *= fresnel;
}

As should now be familiar, we can get a measure of the incidence of the camera’s view vector with the fragment’s normal vector using the dot product clamped between 0.0 and 1.0. We then take its inverse by subtracting this value from 1.0. The fresnel value will now be 0.0 when the two vectors are completely aligned, and 1.0 when the angle between them is 90° or greater. We multiply our existing specular highlight value by fresnel to attenuate the effect based on our view angle.

Putting it together

Happily, light is additive. Layering our different lights onto the surface of the model is therefore just a case of adding them together. We then multiply this value by the base colour of the surface and add the highlights to get the overall colour of the fragment.

void main() {
  /* ... */
  lighting = 
      (ambient * settings.fAmbientWeight) 
    + (hemi * settings.fHemiWeight) 
    + (diffuse * settings.fDiffuseWeight);

  vec3 colour = baseColour * lighting;
  if (settings.bShowSpecular) {
    colour += specular;
  }
}

One final thing to note is that the above calculations all assume a linear colour space, whereas web browsers use sRGB by default. This leads to quite muddy colours in the final result. We can fix that by converting to sRGB, which involves a formula with a lot of magic numbers that I won’t pretend to understand well:

vec3 linearTosRGB(vec3 value) {
  vec3 lt = vec3(lessThanEqual(value.rgb, vec3(0.0031308)));
  
  vec3 v1 = value * 12.92;
  vec3 v2 = pow(value.xyz, vec3(0.41666)) * 1.055 - vec3(0.055);

  return mix(v2, v1, lt);
}

void main() {
  /* ... */
  if (settings.bUseSrgb) {
    colour = linearTosRGB(colour);
  }

  gl_FragColor = vec4(colour, 1.0);
}

After all that, we set gl_FragColor to our colour vec3 with a transparency of 1.0, and that’s it!

Links

• Vertex shader
• Fragment shader