There is an updated edition of this book available! View Latest Edition
Home iOS & Swift Books Metal by Tutorials

12
Environment Written by Caroline Begbie

Heads up... You're reading this book for free, with parts of this chapter shown beyond this point as scrambled text.

You can unlock the rest of this book, and our entire catalogue of books and videos, with a raywenderlich.com Professional subscription.

Unlock Now

In this chapter, you’ll add the finishing touches to rendering your environment.

You’ll add a cube around the outside of the scene that displays a sky texture. That sky texture will then light the models within the scene, making them appear as if they belong.

Look at the following comparison of two renders.

This comparison demonstrates how you can use the same shader code, but change the sky image to create different lighting environments.

Getting started

Open the starter project for this chapter. For the most part, this project is similar to the engine you created in Chapter 9, “Scene Graph.” There are, however, a few notable changes:

  • You can add a fragment function string name to the Model initializer, letting you test different rendering styles for different props.
  • The racing car asset and the Textures asset catalog includes metallic and ambient occlusion maps.

Build and run the project, and you’ll see the car rendered using physically based shading, as described in Chapter 7, “Maps and Materials”. Chapter 8, “Character Animation”, and Chapter 9, “Scene Graph” used this same shading, but sneakily used a scaling factor in the fragment shader to lighten the shadows.

Aside from the darkness of the lighting, there are some glaring problems with the render:

  • All metals, such as the metallic wheel hubs, aren’t looking shiny. Pure metals reflect their surroundings, and there are currently no surroundings to reflect.
  • As you move around the car using the keyboard, notice where the light doesn’t directly hit the car, the color is black. This happens because the app doesn’t provide any ambient light. Later on in the chapter, you’ll use the skylight as global ambient light.

Note: If you’re using macOS, use the keyboard keys WASD to move, and use the QE or right and left arrow keys to rotate.

The skybox

Currently, the sky is a single color, which looks unrealistic. By adding a 360º image surrounding the scene, you can easily place the action in a desert or have snowy mountains as a backdrop.

import MetalKit

class Skybox {
  
  let mesh: MTKMesh
  var texture: MTLTexture?
  let pipelineState: MTLRenderPipelineState
  let depthStencilState: MTLDepthStencilState?
  
  init(textureName: String?) {
    
  }
}

let allocator = MTKMeshBufferAllocator(device: Renderer.device)
let cube = MDLMesh(boxWithExtent: [1,1,1], segments: [1, 1, 1],
                   inwardNormals: true, 
                   geometryType: .triangles,
                   allocator: allocator)
do {
  mesh = try MTKMesh(mesh: cube,
                     device: Renderer.device)
} catch {
  fatalError("failed to create skybox mesh")
}

private static func 
    buildPipelineState(vertexDescriptor: MDLVertexDescriptor) 
                              -> MTLRenderPipelineState {
  let descriptor = MTLRenderPipelineDescriptor()
  descriptor.colorAttachments[0].pixelFormat = 
       Renderer.colorPixelFormat
  descriptor.depthAttachmentPixelFormat = .depth32Float
  descriptor.vertexFunction = 
        Renderer.library?.makeFunction(name: "vertexSkybox")
  descriptor.fragmentFunction = 
        Renderer.library?.makeFunction(name: "fragmentSkybox")
  descriptor.vertexDescriptor = 
        MTKMetalVertexDescriptorFromModelIO(vertexDescriptor)
  do {
    return 
      try Renderer.device.makeRenderPipelineState(
          descriptor: descriptor)
  } catch {
    fatalError(error.localizedDescription)
  }
}

private static func buildDepthStencilState() 
            -> MTLDepthStencilState? {
  let descriptor = MTLDepthStencilDescriptor()
  descriptor.depthCompareFunction = .lessEqual
  descriptor.isDepthWriteEnabled = true
  return Renderer.device.makeDepthStencilState(
      descriptor: descriptor)
}

pipelineState = 
    Skybox.buildPipelineState(vertexDescriptor: cube.vertexDescriptor)
depthStencilState = Skybox.buildDepthStencilState()

Rendering the skybox

Still in Skybox.swift, create a new method to perform the skybox rendering:

func render(renderEncoder: MTLRenderCommandEncoder, uniforms: Uniforms) {

}

renderEncoder.pushDebugGroup("Skybox")
renderEncoder.setRenderPipelineState(pipelineState)
// renderEncoder.setDepthStencilState(depthStencilState)
renderEncoder.setVertexBuffer(mesh.vertexBuffers[0].buffer, 
                              offset: 0, index: 0)

var viewMatrix = uniforms.viewMatrix
viewMatrix.columns.3 = [0, 0, 0, 1]
var viewProjectionMatrix = uniforms.projectionMatrix 
                               * viewMatrix
renderEncoder.setVertexBytes(&viewProjectionMatrix, 
                      length: MemoryLayout<float4x4>.stride, 
                      index: 1)

let submesh = mesh.submeshes[0]
renderEncoder.drawIndexedPrimitives(type: .triangle,
  indexCount: submesh.indexCount,
  indexType: submesh.indexType,
  indexBuffer: submesh.indexBuffer.buffer,
  indexBufferOffset: 0)

The skybox shader functions

In the Metal Shaders group, add a new Metal file named Skybox.metal. Again, add this file to both the macOS and iOS targets.

#import "Common.h"

struct VertexIn {
  float4 position [[ attribute(0) ]];
};

struct VertexOut {
  float4 position [[ position ]];
};

vertex VertexOut vertexSkybox(const VertexIn in [[stage_in]],
                         constant float4x4 &vp [[buffer(1)]]) {
  VertexOut out;
  out.position = (vp * in.position).xyww;
  return out;
}

fragment half4 fragmentSkybox(VertexOut in [[stage_in]]) {
  return half4(1, 1, 0, 1);
}

Integrating the skybox into the scene

Open Scene.swift, and add a new property to Scene:

var skybox: Skybox?

skybox = Skybox(textureName: nil)

scene.skybox?.render(renderEncoder: renderEncoder, 
                     uniforms: scene.uniforms)

Procedural skies

Yellow skies might be appropriate on a different planet, but how about a procedural sky? A procedural sky is one built out of various parameters such as weather conditions and time of day. Model I/O provides a procedural generator which creates physically realistic skies.

Cube textures

Cube textures are similar to the 2D textures that you’ve already been using. 2D textures map to a quad and have two texture coordinates, whereas cube textures consist of six 2D textures: one for each face of the cube. You sample the textures with a 3D vector.

Adding the procedural sky

You’ll use these sky textures shortly, but for now, you’ll add a procedural sky to the scene in the starter project. In Skybox.swift, add these properties to Skybox:

struct SkySettings {
  var turbidity: Float = 0.28
  var sunElevation: Float = 0.6
  var upperAtmosphereScattering: Float = 0.1
  var groundAlbedo: Float = 4
}

var skySettings = SkySettings() 

func loadGeneratedSkyboxTexture(dimensions: int2) -> MTLTexture? {
  var texture: MTLTexture?
  let skyTexture = MDLSkyCubeTexture(name: "sky",
        channelEncoding: .uInt8,
        textureDimensions: dimensions,
        turbidity: skySettings.turbidity,
        sunElevation: skySettings.sunElevation,
        upperAtmosphereScattering: 
               skySettings.upperAtmosphereScattering,
        groundAlbedo: skySettings.groundAlbedo)
  do {
    let textureLoader = 
          MTKTextureLoader(device: Renderer.device)
    texture = try textureLoader.newTexture(texture: skyTexture, 
                                           options: nil)
  } catch {
    print(error.localizedDescription)
  }
  return texture
}

if let textureName = textureName {
  
} else {
  texture = loadGeneratedSkyboxTexture(dimensions: [256, 256])
}

renderEncoder.setFragmentTexture(texture, 
                    index: Int(BufferIndexSkybox.rawValue))

float3 textureCoordinates;


out.textureCoordinates = in.position.xyz;

fragment half4 
        fragmentSkybox(VertexOut in [[stage_in]],
                       texturecube<half> cubeTexture 
                           [[texture(BufferIndexSkybox)]]) {
  constexpr sampler default_sampler(filter::linear);
  half4 color = cubeTexture.sample(default_sampler, 
                                   in.textureCoordinates);
  return color;
}

Custom sky textures

As mentioned earlier, you can use your own 360º sky textures. The textures included in the starter project were downloaded from http://hdrihaven.com, a great place to find environment maps. The HDRI has been converted into six tone mapped sky cube textures before adding them to the asset catalog.

extension Skybox: Texturable {}

do {
  texture = try Skybox.loadCubeTexture(imageName: textureName)
} catch {
  fatalError(error.localizedDescription)
}

skybox = Skybox(textureName: "sky")

Reflection

Now that you have something to reflect, you can easily implement reflection of the sky onto the car. When rendering the car, all you have to do is take the camera view direction, reflect it about the surface normal, and sample the skycube along the reflected vector for the fragment color for the car.

let car = Model(name: "racing-car.obj", 
                fragmentFunctionName: "skyboxTest")

fragment float4 skyboxTest(VertexOut in [[stage_in]],
  constant FragmentUniforms &fragmentUniforms
    [[buffer(BufferIndexFragmentUniforms)]],
  texturecube<float> skybox [[texture(BufferIndexSkybox)]]) {
  return float4(0, 1, 1, 1);
}


func update(renderEncoder: MTLRenderCommandEncoder) {  
  renderEncoder.setFragmentTexture(texture,
                      index: Int(BufferIndexSkybox.rawValue))
}

scene.skybox?.update(renderEncoder: renderEncoder)

float3 viewDirection = in.worldPosition.xyz - 
                              fragmentUniforms.cameraPosition;
float3 textureCoordinates = reflect(viewDirection, 
                                    in.worldNormal);

constexpr sampler defaultSampler(filter::linear);
float4 color = skybox.sample(defaultSampler, 
                             textureCoordinates);
float4 copper = float4(0.86, 0.7, 0.48, 1);
color = color * copper;
return color;

Image-based lighting

At the beginning of the chapter, there were two problems with the original car render. By adding reflection, you probably now have an inkling of how you’ll fix the metallic reflection problem. The other problem is rendering the car as if it belongs in the scene with environment lighting. IBL or Image Based Lighting is one way of dealing with this problem.

Diffuse reflection

Light comes from all around us. Sunlight bounces around and colors reflect. When rendering an object, you should take into account the color of the light coming from every direction.

let car = Model(name: "racing-car.obj", 
                fragmentFunctionName: "fragment_IBL")


var diffuseTexture: MTLTexture?

func loadIrradianceMap() {
  // 1
  let skyCube = 
       MDLTexture(cubeWithImagesNamed: ["cube-sky.png"])!
  // 2
  let irradiance = 
       MDLTexture.irradianceTextureCube(with: skyCube, 
                              name: nil, dimensions: [64, 64], 
                              roughness: 0.6)
  // 3                           
  let loader = MTKTextureLoader(device: Renderer.device)
  diffuseTexture = try! loader.newTexture(texture: irradiance, 
                                          options: nil)
}

loadIrradianceMap()

renderEncoder.setFragmentTexture(diffuseTexture,
  index: Int(BufferIndexSkyboxDiffuse.rawValue))

texturecube<float> skybox [[texture(BufferIndexSkybox)]],
texturecube<float> skyboxDiffuse 
                   [[texture(BufferIndexSkyboxDiffuse)]]

float4 diffuse = skyboxDiffuse.sample(textureSampler, normal);
return diffuse * float4(baseColor, 1);


diffuseTexture = 
     try Skybox.loadCubeTexture(imageName: "irradiance.png")

Specular reflection

The irradiance map provides the diffuse and ambient reflection, but the specular reflection is a bit more difficult.

BRDF look-up table

During runtime, you supply a look-up table with the actual roughness of the model and the current viewing angle and receive back the scale and bias for the Fresnel and geometric attenuation contributions to the final color. You can represent this two-dimensional look-up table as a texture that behaves as a two-dimensional array. One axis is the roughness value of the object, and the other is the angle between the normal and the view direction. You input these two values as the UV coordinates and receive back a color. The red value contains the scale, and the green value contains the bias.

var brdfLut: MTLTexture?

brdfLut = Renderer.buildBRDF()

renderEncoder.setFragmentTexture(brdfLut, 
                  index: Int(BufferIndexBRDFLut.rawValue))


texture2d<float> brdfLut [[texture(BufferIndexBRDFLut)]]

// 1
float3 viewDirection = in.worldPosition.xyz -
                           fragmentUniforms.cameraPosition;
float3 textureCoordinates = reflect(viewDirection, normal);
// 2
constexpr sampler s(filter::linear, mip_filter::linear);
float3 prefilteredColor = 
        skybox.sample(s, textureCoordinates, 
                      level(roughness * 10)).rgb;
// 3
float nDotV = saturate(dot(normal, normalize(-viewDirection)));
float2 envBRDF = brdfLut.sample(s, float2(roughness, nDotV)).rg;

Fresnel reflectance

When light hits an object straight on, some of the light is reflected. The amount of reflection is called Fresnel zero, or F0, and you can calculate this from the material’s index of refraction, or IOR.

float3 f0 = mix(0.04, baseColor.rgb, metallic);
float3 specularIBL = f0 * envBRDF.r + envBRDF.g;

float3 specular = prefilteredColor * specularIBL;
float4 color = diffuse * float4(baseColor, 1) 
                  + float4(specular, 1);
return color;


diffuse = mix(pow(diffuse, 0.5), diffuse, metallic);

Ambient occlusion maps

Ambient occlusion is a technique that approximates how much light should fall on a surface. If you look around you — even in a bright room — where surfaces are very close to each other, they’re darker than exposed surfaces. In Chapter 19, “Advanced Shadows”, you’ll learn how to generate global ambient occlusion using ray marching, but assigning pre-built local ambient occlusion maps to models is a fast and effective alternative.

color *= ambientOcclusion;

Challenge

On the first page of this chapter is a comparison of the car rendered in two different lighting situations. Your challenge is to create the red lighting scene.

Where to go from here?

You’ve dipped a toe into the water of the great sea of realistic rendering, and you’ll read about more advanced concepts of lighting and reflectivity in Chapter 20, “Advanced Lighting.” If you want to explore more about realistic rendering, references.markdown for this chapter contains links to interesting articles and videos.

13. Instancing & Procedural Generation 11. Tessellation & Terrains

Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.

© 2022 Razeware LLC

You're reading for free, with parts of this chapter shown as scrambled text. Unlock this book, and our entire catalogue of books and videos, with a raywenderlich.com Professional subscription.

Unlock Now

To highlight or take notes, you’ll need to own this book in a subscription or purchased by itself.

Read for Free with a Subscription

Buy this book for $59.99