Metal Tutorial with Swift 3 Part 3: Adding Texture

In part 3 of our Metal tutorial series, you will learn how to add textures to 3D objects using Apple’s built-in 3D graphics framework. By Andrew Kharchyshyn.

Leave a rating/review
Save for later
Share

Update: This tutorial has been updated for Xcode 8.2 and Swift 3.

Welcome back to our Swift 3 Metal tutorial series!

In the first part of the series, you learned how to get started with Metal and render a simple 2D triangle.

In the second part of the series, you learned how to set up a series of transformations to move from a triangle to a full 3D cube.

In this third part of the series, you’ll learn how to add a texture to the cube. As you work through this tutorial, you’ll learn:

  • How to reuse uniform buffers
  • How to apply textures to a 3D model
  • How to add touch input to your app
  • How to debug Metal

Dust off your guitars — it’s time to rock Metal!

Getting Started

Previously, ViewController was a heavy lifter. Even though you’d refactored it, it still had more than one responsibility. Now ViewController is split into two classes:

  • MetalViewController: The base class that contains the generic Metal setup code.
  • MySceneViewController: A subclass that contains code specific to this app for creating and rendering the cube model.

The most important part to note is the new protocol MetalViewControllerDelegate:

protocol MetalViewControllerDelegate : class{
  func updateLogic(timeSinceLastUpdate: CFTimeInterval)
  func renderObjects(drawable: CAMetalDrawable)
}

This establishes callbacks from MetalViewController so that your app knows when to update logic and when to render.

In MySceneViewController, you set yourself as a delegate and then implement MetalViewControllerDelegate methods. This is where all the cube rendering and updating action happens.

Now that you’re up to speed on the changes from part two, it’s time to move forward and delve deeper into the world of Metal.

Reusing Uniform Buffers (optional)

Note: This next section is theory-driven and gives you more context about how Metal works under the hood. If you’re eager to move into the exercises, feel free to skip ahead to the “Texturing” section. But reading this will make you at least 70 percent smarter. ;-]

In the previous part of this series, you learned about allocating new uniform buffers for every new frame — and you also learned that it’s not very efficient.

So, the time has come to change your ways and make Metal sing, like an epic hair-band guitar solo. But every great solution starts with identifying the actual problem.

The Problem

In the render method in Node.swift, find:

let uniformBuffer = device.makeBuffer(length: MemoryLayout<Float>.size * Matrix4.numberOfElements() * 2, options: [])

Take a good look at this monster! This method is called 60 times per second, and you create a new buffer each time it’s called.

Since this is a performance issue, you’ll want to compare stats before and after optimization.

Build and run the app, open the Debug Navigator tab and select the FPS row.

Screen Shot 2015-01-08 at 4.35.43 PM

You should have numbers similar to these:

before

You’ll return to those numbers after optimization, so you may want to grab a screencap or simply jot down the stats before you move on.

The Solution

The solution is that instead of allocating a buffer each time, you’ll reuse a pool of buffers.

To keep your code clean, you’ll encapsulate all of the logic to create and reuse buffers into a helper class named BufferProvider.

You can visualize the class as follows:

bufferProvider_Diagram

BufferProvider will be responsible for creating a pool of buffers, and it will have a method to get the next available reusable buffer. This is kind of like UITableViewCell!

Now it’s time to dig in and make some magic happen. Create a new Swift class named BufferProvider, and make it a subclass of NSObject.

First import Metal at the top of the file:

import Metal 

Now, add these properties to the class:

// 1
let inflightBuffersCount: Int 
// 2
private var uniformsBuffers: [MTLBuffer]
// 3
private var avaliableBufferIndex: Int = 0 

You’ll get some errors at the moment due to a missing initializer, but you’ll fix those shortly. For now, review each property you just added:

  1. An Int that will store the number of buffers stored by BufferProvider. In the diagram above, this equals 3.
  2. An array that will store the buffers themselves.
  3. The index of the next available buffer. In your case, it will change like this: 0 -> 1 -> 2 -> 0 -> 1 -> 2 -> 0 -> …

Now add the following initializer:

init(device:MTLDevice, inflightBuffersCount: Int, sizeOfUniformsBuffer: Int) {
    
  self.inflightBuffersCount = inflightBuffersCount
  uniformsBuffers = [MTLBuffer]()
    
  for _ in 0...inflightBuffersCount-1 {
    let uniformsBuffer = device.makeBuffer(length: sizeOfUniformsBuffer, options: [])
    uniformsBuffers.append(uniformsBuffer)
  }
}

Here you create a number of buffers, equal to the inflightBuffersCount parameter passed in to this initializer, and append them to the array.

Now add a method to fetch the next available buffer and copy some data into it:

func nextUniformsBuffer(projectionMatrix: Matrix4, modelViewMatrix: Matrix4) -> MTLBuffer {
    
  // 1
  let buffer = uniformsBuffers[avaliableBufferIndex]
    
  // 2
  let bufferPointer = buffer.contents()
    
  // 3
  memcpy(bufferPointer, modelViewMatrix.raw(), MemoryLayout<Float>.size * Matrix4.numberOfElements())
  memcpy(bufferPointer + MemoryLayout<Float>.size*Matrix4.numberOfElements(), projectionMatrix.raw(), MemoryLayout<Float>.size*Matrix4.numberOfElements())
    
  // 4
  avaliableBufferIndex += 1
  if avaliableBufferIndex == inflightBuffersCount{
    avaliableBufferIndex = 0
  }
    
  return buffer
}

Reviewing each section in turn:

  1. Fetch MTLBuffer from the uniformsBuffers array at avaliableBufferIndex index.
  2. Get void * pointer from MTLBuffer.
  3. Copy the passed-in matrices data into the buffer using memcpy.
  4. Increment avaliableBufferIndex.

You’re almost done: you just need to set up the rest of the code to use this.

To do this, open Node.swift, and add this new property:

var bufferProvider: BufferProvider

Find init and add this at the end of the method:

self.bufferProvider = BufferProvider(device: device, inflightBuffersCount: 3, sizeOfUniformsBuffer: MemoryLayout<Float>.size * Matrix4.numberOfElements() * 2)

Finally, inside render, replace this code:

let uniformBuffer = device.makeBuffer(length: MemoryLayout<Float>.size * Matrix4.numberOfElements() * 2, options: [])
let bufferPointer = uniformBuffer.contents()
memcpy(bufferPointer, nodeModelMatrix.raw(), MemoryLayout<Float>.size * Matrix4.numberOfElements())
memcpy(bufferPointer + MemoryLayout<Float>.size * Matrix4.numberOfElements(), projectionMatrix.raw(), MemoryLayout<Float>.size * Matrix4.numberOfElements())

With this far more elegant code:

let uniformBuffer = bufferProvider.nextUniformsBuffer(projectionMatrix: projectionMatrix, modelViewMatrix: nodeModelMatrix)

Build and run. Everything should work just as well as it did before you added bufferProvider:

IMG_3030

A Wild Race Condition Appears!

Things are running smoothly, but there is a problem that could cause you some major pain later.

Have a look at this graph (and the explanation below):

gifgif

Currently, the CPU gets the “next available buffer”, fills it with data, and then sends it to the GPU for processing.

But since there’s no guarantee about how long the GPU takes to render each frame, there could be a situation where you’re filling buffers on the CPU faster than the GPU can deal with them. In that case, you could find yourself in a scenario where you need a buffer on the CPU, even though it’s in use on the GPU.

On the graph above, the CPU wants to encode the third frame while the GPU draws the first frame, but its uniform buffer is still in use.

So how do you fix this?

The easiest way is to increase the number of buffers in the reuse pool so that it’s unlikely for the CPU to be ahead of the GPU. This would probably fix it, but wouldn’t be 100% safe.

Patience. That’s what you need to solve this problem like a real Metal ninja.

Andrew Kharchyshyn

Contributors

Andrew Kharchyshyn

Author

Over 300 content creators. Join our team.