There is an updated edition of this book available! View Latest Edition
Home iOS & Swift Books Metal by Tutorials

13
Instancing & Procedural Generation Written by Caroline Begbie

Heads up... You're reading this book for free, with parts of this chapter shown beyond this point as scrambled text.

You can unlock the rest of this book, and our entire catalogue of books and videos, with a raywenderlich.com Professional subscription.

Unlock Now

Now that you have an environment with a sky that you can populate with models, you’ll want to add some randomness to your scene. Trees and grass make the scene more natural, but they can take up a lot of valuable resources without adding any player action to your game.

In this chapter, you’ll first find out how to efficiently render many trees and blades of grass using instancing. You’ll then render instanced rocks, and you’ll use morphing for different shapes. Finally, you’ll create a procedural house system that will create a row of houses whose size and style will change every time you run the app.

As well as all that, you’ll improve your versatility and proficiency in handling GPU resources which will enable you to access any data in MTLBuffers with confidence.

The starter project

Open the starter project for this chapter. This is almost the same project as the previous chapter, with a few skybox tweaks, but it includes a new GameScene that renders 100 trees in random places. Each tree consists of 9,119 vertices, with a color texture of size 8 MB and 2048×2048 pixels.

Note: You generally wouldn’t spend 9,119 vertices just on a tree, unless you wanted some fine detail. Low-poly trees are much more efficient. However, this example will show you just how important instancing is.

The project also contains two other scenes with supporting files for later sections in which you’ll create a rock system and perform procedural house generation.

Build and run the app. If your device can’t handle 100 trees, you might need to reduce instanceCount in GameScene.

In Xcode, check the Debug navigator. This is the result on a 2015 iMac:

Your aim in the first part of this chapter is to reduce that huge memory footprint, and maybe even address that terrible frame rate.

Note: If you’re using faster hardware and are seeing 60 FPS, you might consider increasing the number of trees to 200 or even 300 for the purpose of this exercise – set instanceCount in GameScene accordingly.

Instancing

Note: Instance drawing for iOS is available for GPU Family 3 and up - that’s a minimum hardware device of the iPhone 6s.

private var transforms: [Transform]
let instanceCount: Int

init(name: String,
     vertexFunctionName: String = "vertex_main",
     fragmentFunctionName: String = "fragment_IBL",
     instanceCount: Int = 1) {

static func buildTransforms(instanceCount: Int) -> [Transform] {
  return [Transform](repeatElement(Transform(), 
                     count: instanceCount))
}

self.instanceCount = instanceCount
transforms = Model.buildTransforms(instanceCount: instanceCount)

Create the instance MTLBuffer

The GPU only requires matrices, not the position and rotation data, so instead of sending all the transform data to the GPU, you’ll only send a buffer of matrices.

var instanceBuffer: MTLBuffer

struct Instances {
  matrix_float4x4 modelMatrix;
  matrix_float3x3 normalMatrix;
};

static func buildInstanceBuffer(transforms: [Transform]) -> MTLBuffer {
// 1
  let instances = transforms.map {
      Instances(modelMatrix: $0.modelMatrix,
          normalMatrix: float3x3(normalFrom4x4: $0.modelMatrix))
  }
// 2
  guard let instanceBuffer = 
      Renderer.device.makeBuffer(bytes: instances, 
        length: MemoryLayout<Instances>.stride 
                     * instances.count) else { 
    fatalError("Failed to create instance buffer")
  }
  return instanceBuffer
}

instanceBuffer = 
     Model.buildInstanceBuffer(transforms: transforms)

Accessing MTLBuffer data

An MTLBuffer contains bytes, which can be of any data type. Swift is a strongly typed language, meaning that Swift can only access the data in the buffer if you tell Swift what type the data is. You do this by binding the data to a type. In this case, the type is Instances.

func updateBuffer(instance: Int, transform: Transform) {
  transforms[instance] = transform
}

var pointer = 
    instanceBuffer.contents().bindMemory(to: Instances.self,
                                 capacity: transforms.count)

pointer = pointer.advanced(by: instance)
pointer.pointee.modelMatrix = transforms[instance].modelMatrix
pointer.pointee.normalMatrix = transforms[instance].normalMatrix

renderEncoder.setVertexBuffer(instanceBuffer, offset: 0,
                   index: Int(BufferIndexInstances.rawValue))

renderEncoder.drawIndexedPrimitives(type: .triangle,
        indexCount: mtkSubmesh.indexCount,
        indexType: mtkSubmesh.indexType,
        indexBuffer: mtkSubmesh.indexBuffer.buffer,
        indexBufferOffset: mtkSubmesh.indexBuffer.offset,
        instanceCount: instanceCount)

GPU instances

So you can access the instance array on the GPU side, you’ll change the vertex shader.

constant Instances *instances [[buffer(BufferIndexInstances)]],
uint instanceID [[instance_id]]

Instances instance = instances[instanceID];
VertexOut out {
  .position = uniforms.projectionMatrix * uniforms.viewMatrix 
      * uniforms.modelMatrix * instance.modelMatrix * position,
  .worldPosition = (uniforms.modelMatrix * 
            instance.modelMatrix * position).xyz,
  .worldNormal = uniforms.normalMatrix * 
            instance.normalMatrix * normal.xyz,
  .worldTangent = uniforms.normalMatrix * 
            instance.normalMatrix * tangent.xyz,
  .worldBitangent = uniforms.normalMatrix * 
            instance.normalMatrix * bitangent.xyz,
  .uv = vertexIn.uv
};

let tree = Model(name: "tree.obj", instanceCount: instanceCount)
add(node: tree)
for i in 0..<instanceCount {
  var transform = Transform()
  transform.position.x = .random(in: -10..<10)
  transform.position.z = .random(in: -10..<10)
  let rotationY: Float = .random(in: -.pi..<Float.pi)
  transform.rotation = [0, rotationY, 0]
  tree.updateBuffer(instance: i, transform: transform)
}

Morphing

You rendered multiple instances of the same high poly tree, but your scene will look boring if you render the same model with the same textures all over it. In this section, you’ll render a rock with one of three random textures and one of three random shapes, or morph targets. You’ll hold the vertex information for these three different shapes in a single buffer: an array of vertex buffers. You’ll also learn how to render vertices that you’ve read in using Model I/O without using the stage_in attribute.

let scene = RocksScene(sceneSize: metalView.bounds.size)

Vertex descriptors and stage_in

So far, to render OBJ models, you’ve been using the [[stage_in]] attribute to describe vertex buffers in the vertex shader function.

struct VertexIn {
  float3 position;
  float3 normal;
  float2 uv;
};

constant VertexIn *in [[buffer(0)]],
uint vertexID [[vertex_id]],

VertexIn vertexIn = in[vertexID];

float4 position = float4(vertexIn.position, 1);

Packed floats

In Apple’s Metal Shading Language Specification, available at https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf, you can see the size and alignment of vector data types.

var offset = 0

let packedFloat3Size = MemoryLayout<Float>.stride * 3

offset += MemoryLayout<float3>.stride

offset += packedFloat3Size

struct VertexIn {
  packed_float3 position;
  packed_float3 normal;
  float2 uv;
};

MTLBuffer array

You’re now able to render a single rock shape. In Swift, if you were going to have several rock shapes, you’d probably consider putting each shape into an array. However C++ does not allow variable length arrays, and you want to have the ability to add a variable number of morph targets to your Nature system.

var vertexCount: Int

vertexBuffer = mesh.vertexBuffers[0].buffer

let bufferLength = mesh.vertexBuffers[0].buffer.length
vertexBuffer = Renderer.device.makeBuffer(length: bufferLength 
  * morphTargetNames.count)!

let layout = mesh.vertexDescriptor.layouts[0] 
                 as! MDLVertexBufferLayout
vertexCount = bufferLength / layout.stride

The Blit Command Encoder

You’ll copy each morph target vertex buffer into the single MTLBuffer using a blit operation. You were introduced to a render command encoder in Chapter 1, “Introduction to Metal,” and then briefly to a compute encoder in Chapter 11, “Tessellation”. You are now learning about yet another type of encoder. A blit command encoder does a fast copy between resources such as textures and buffers. Just like with an MTLRenderCommandEncoder, you create an MTLBlitCommandEncoder and issue commands to the command buffer.

let commandBuffer = Renderer.commandQueue.makeCommandBuffer()
let blitEncoder = commandBuffer?.makeBlitCommandEncoder()

for i in 0..<morphTargetNames.count {
  guard let mesh = Nature.loadMesh(name: morphTargetNames[i]) else {
    fatalError("morph target not loaded")
  }
  let buffer = mesh.vertexBuffers[0].buffer
  blitEncoder?.copy(from: buffer, sourceOffset: 0,
                    to: vertexBuffer, 
                    destinationOffset: buffer.length * i,
                    size: buffer.length)
}

blitEncoder?.endEncoding()
commandBuffer?.commit()

Random rock shape

In Nature.swift, change updateBuffer(instance:transform:) to receive a random number for both the morph target and the texture:

func updateBuffer(instance: Int, transform: Transform, 
                  textureID: Int, morphTargetID: Int) 

guard textureID < textureCount 
        && morphTargetID < morphTargetCount else {
  fatalError("ID is too high")
}

pointer.pointee.textureID = UInt32(textureID)
pointer.pointee.morphTargetID = UInt32(morphTargetID)

updateBuffer(instance: 0, transform: Transform(),
             textureID: 0, morphTargetID: 0)

let textureID = Int.random(in: 0..<textureNames.count)
let morphTargetID = Int.random(in: 0..<morphTargetNames.count)
rocks.updateBuffer(instance: i, transform: transform,
                   textureID: textureID, 
                   morphTargetID: morphTargetID)


renderEncoder.setVertexBytes(&vertexCount, 
                             length: MemoryLayout<Int>.stride, 
                             index: 1)

constant int &vertexCount [[buffer(1)]],

VertexIn vertexIn = in[vertexID];
NatureInstance instance = instances[instanceID];

NatureInstance instance = instances[instanceID];
uint offset = instance.morphTargetID * vertexCount;
VertexIn vertexIn = in[vertexID + offset];

Texture arrays

Accessing a random texture is slightly easier than a random morph target since you can load the textures into an MTLTexture with a textureType of type2DArray. All of the textures are held in one MTLTexture, with each element of the array being called a slice.

static func loadTextureArray(textureNames: [String]) -> MTLTexture? {
}

var textures: [MTLTexture] = []
for textureName in textureNames {
  do {
    if let texture = 
         try Nature.loadTexture(imageName: textureName) {
      textures.append(texture)
    }
  }
  catch {
    fatalError(error.localizedDescription)
  }
}
guard textures.count > 0 else { return nil }

let descriptor = MTLTextureDescriptor()
descriptor.textureType = .type2DArray
descriptor.pixelFormat = textures[0].pixelFormat
descriptor.width = textures[0].width
descriptor.height = textures[0].height
descriptor.arrayLength = textures.count
let arrayTexture = 
     Renderer.device.makeTexture(descriptor: descriptor)!

let commandBuffer = Renderer.commandQueue.makeCommandBuffer()!
let blitEncoder = commandBuffer.makeBlitCommandEncoder()!
let origin = MTLOrigin(x: 0, y: 0, z: 0)
let size = MTLSize(width: arrayTexture.width, 
                   height: arrayTexture.height, depth: 1)
for (index, texture) in textures.enumerated() {
  blitEncoder.copy(from: texture, 
                   sourceSlice: 0, sourceLevel: 0,
                   sourceOrigin: origin, sourceSize: size,
                   to: arrayTexture, destinationSlice: index, 
                   destinationLevel: 0, 
                   destinationOrigin: origin)
}
blitEncoder.endEncoding()
commandBuffer.commit()
return arrayTexture

do {
  baseColorTexture = 
       try Nature.loadTexture(imageName: textureNames[0])
} catch {
  fatalError(error.localizedDescription)
}

baseColorTexture = 
   Nature.loadTextureArray(textureNames: textureNames)

texture2d_array<float> baseColorTexture [[texture(0)]]

uint textureID [[flat]];

out.textureID = instance.textureID;

float4 baseColor = baseColorTexture.sample(s, in.uv, 
                                           in.textureID);

Procedural systems

With the techniques you’ve learned so far, you have the power to load random objects into your scene and make each scene unique. However, you’re currently limited to having a single mesh size. In games such as No Man’s Sky, each planet and all of the animals that you meet on the planets are procedurally generated using many different meshes.

Rules

The secret to procedural systems is having a set of rules. For example, if you’re procedurally generating animals out of multiple parts, you don’t want to be attaching heads to leg joints; this isn’t The Island of Doctor Moreau after all.

The Houses scene

Add HousesScene.swift, Houses.swift and Houses.metal to the targets by Cmd-Selecting all three files, and on the File inspector, check Target Membership for both macOS and iOS targets.

let scene = HousesScene(sceneSize: metalView.bounds.size)

Determine the rules

In Houses.swift, add an enum to Houses to lay down the rules with constants:

enum Rule {
  // gap between houses
  static let minGap: Float = 0.3
  static let maxGap: Float = 1.0

  // number of OBJ files for each type
  static let numberOfGroundFloors = 4
  static let numberOfUpperFloors = 4
  static let numberOfRoofs = 2

  // maximum houses
  static let maxHouses: Int = 5
    
  // maximum number of floors in a single house
  static let maxFloors: Int = 6
}

var floorsRoof: Set<Int> = []

Load the OBJ files

Create a new method to read all the house OBJ files and place them into a Swift array.

func loadOBJs() -> [Model] {
  var houses: [Model] = []
  func loadHouse(name: String) {
    houses.append(Model(name: name + ".obj",
                        vertexFunctionName: "vertex_house",
                        fragmentFunctionName: "fragment_house"))
  }
  for i in 1...Rule.numberOfGroundFloors {
    loadHouse(name: String(format: "houseGround%d", i))
  }
  for i in 1...Rule.numberOfUpperFloors {
    loadHouse(name: String(format: "houseFloor%d", i))
  }
  for i in 1...Rule.numberOfRoofs {
    loadHouse(name: String(format: "houseRoof%d", i))
    floorsRoof.insert(houses.count-1)
  }
  return houses
}


houses.append(Model(name: "houseGround1.obj",
                    vertexFunctionName: "vertex_house",
                    fragmentFunctionName: "fragment_house"))

houses = loadOBJs()

Create the first (ground) floors

Create two new properties in Houses:

var remainingHouses: Set<Int> = []
var housefloors: [[Int]] = []

let numberOfHouses = 5
for _ in 0..<numberOfHouses {
  let random = Int.random(in: 0..<Rule.numberOfGroundFloors)
  housefloors.append([random])
  let lastIndex = housefloors.count - 1
  remainingHouses.insert(lastIndex)
}

while remainingHouses.count > 0 {
  for i in 0..<housefloors.count {
    // 1
    if remainingHouses.contains(i) {
      let offset = Rule.numberOfGroundFloors
      let upperBound = 
          offset + Rule.numberOfUpperFloors + Rule.numberOfRoofs
      let random = Int.random(in: offset..<upperBound)
      housefloors[i].append(random)

      // 2
      if floorsRoof.contains(random) ||
        housefloors[i].count >= Rule.maxFloors ||
        Int.random(in: 0...3) == 0 {
        // 3
        remainingHouses.remove(i)
      }
    }
  }
}
// 4
print(housefloors)


struct Floor {
  var houseIndex: Int = 0
  var transform = Transform()
}
var floors: [Floor] = []

var width: Float = 0
var height: Float = 0
var depth: Float = 0
for house in housefloors {
  var houseHeight: Float = 0
  
  // add inner for loop here to process all the floors 
  
  let house = houses[house[0]]
  width += house.size.x
  height = max(houseHeight, height)
  depth = max(house.size.z, depth)
  boundingBox.maxBounds = [width, height, depth]
  width += Float.random(in: Rule.minGap...Rule.maxGap)
}

for floor in house {
  var transform = Transform()
  transform.position.x = width
  transform.position.y = houseHeight
  floors.append(Floor(houseIndex: floor, transform: transform))
  houseHeight += houses[floor].size.y
}

for house in houses {

for floor in floors {
  let house = houses[floor.houseIndex]

uniforms.modelMatrix = modelMatrix * floor.transform.modelMatrix
uniforms.normalMatrix = 
  float3x3(normalFrom4x4: modelMatrix * floor.transform.modelMatrix)

Challenge

Using the Nature system that you set up for the rocks, you can easily create a field full of blades of grass.

Where to go from here?

Procedural systems are fun to create. Later in this book, you’ll encounter Perlin noise, and if you use this noise for your randomness, you can generate infinite terrains, or even dynamic wind or animation.

14. Multipass & Deferred Rendering 12. Environment

Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.

© 2022 Razeware LLC

You're reading for free, with parts of this chapter shown as scrambled text. Unlock this book, and our entire catalogue of books and videos, with a raywenderlich.com Professional subscription.

Unlock Now

To highlight or take notes, you’ll need to own this book in a subscription or purchased by itself.

Read for Free with a Subscription

Buy this book for $59.99