Moving from OpenGL to Metal

In this Metal tutorial, you’ll learn how to move your project from OpenGL to Apple’s 3D graphics API: Metal.

Version

  • Swift 4.2, iOS 12, Xcode 10

Metal was introduced in 2014 as a general purpose API for GPU-based computation. In 2018, Apple deprecated OpenGL in iOS 12 for both iOS and macOS.

In this tutorial, you’ll learn how to convert an app from using OpenGL to Metal. To complete this tutorial, you’ll need a working OpenGL app. The starter project for this tutorial is the finished project from the OpenGL tutorial. You can download what you need using the Download Materials button found at the top or bottom of this tutorial.

Before getting started, you may want to check out these great resources on Metal and OpenGL.

If you don’t have experience with 3D graphics, don’t worry! You’ll still be able to follow along. If you do have some experience with 3D programming or OpenGL, you may find this tutorial easier. Many of the same concepts apply in Metal.

Note: Metal apps don’t run on the iOS simulator. They require a device with an Apple A7 chip or later. To complete this tutorial, you’ll need an A7 device or newer.

OpenGL ES vs. Metal

OpenGL ES is designed to be a cross-platform framework. That means, with a few small modifications, you can run C++ OpenGL ES code on other platforms, such as Android.

The cross-platform support of OpenGL ES is nice, but Apple realized it was missing the signature integration of the operating system, hardware, and software that all good Apple products have. So, it took a clean-room approach and designed a graphics API specifically for Apple hardware. The goal was to have low overhead and high performance while supporting the latest and greatest features.

The result is Metal, which can provide up to 10✕ the number of draw calls for your app compared to OpenGL ES. The effects are amazing — you may remember them from the Zen Garden example in the WWDC 2014 keynote.

Understanding Conceptual Differences

From a development perspective, OpenGL and Metal are similar. In both, you set up buffers with data to pass to the GPU and specify vertex and fragment shaders. In OpenGL projects, there is a GLKBaseEffect, which is an abstraction on top of shaders. There’s no such API for Metal, so you need to write shaders yourself. But don’t worry – it’s not too complicated!

The biggest difference between OpenGL and Metal is that in Metal, you usually operate with two types of objects:

  1. Descriptor objects.
  2. Compiled-state objects.

The idea is simple. You create a descriptor object and compile it. The compiled-state object is a GPU-optimized resource. The creation and compilation are both expensive operations, so the idea is to do them as rarely as possible, and later to operate with compiled-state objects.

This approach means that when using Metal, you don’t need to do a lot of setup operations on the render loop. This makes it much more efficient than OpenGL which can’t do the same due to architecture restrictions.

Time to explore the differences on your own!

Getting Started

As a reminder, download the files you need using the Download Materials button at the top or bottom of this tutorial. Then, open OpenGLKit.xcodeproj in the Starter folder. You’ll see a project which uses OpenGL.

Build & run.

Metal Square

You should see a colorful square spinning. This square is rendered with OpenGL. However, since the deployment target for this project is iOS 12, there are several OpenGL deprecation warnings. You can see these in Xcode in the Issue navigator.

Now you’re going to draw the same square but with Metal. And get rid of all those pesky warnings!

Integrating Metal

Open ViewController.swift, and change ViewController to be a subclass of UIViewController instead of GLKViewController. In Metal, there’s no such thing as MetalViewController. Instead, you have to use MTKView inside the UIViewController.

MTKView is a part of the MetalKit framework. To access this API,
add the following at the top of the file:

import MetalKit

Switching From OpenGL

Now it’s time to do some OpenGL cleanup. Follow these steps:

1. Rename both occurrences of setupGL() to setupMetal().
2. Remove tearDownGL() and deinit() methods. With Metal, there’s no need for explicit cleanup like this.
3. Find and remove the whole extension GLKViewControllerDelegate, since this view controller is no longer a GLKViewController. Note that glkViewControllerUpdate contains the logic for spinning. This is useful, but for now, remove it.
4. Remove the following code from the top of setupMetal():

context = EAGLContext(api: .openGLES3)
EAGLContext.setCurrent(context)
    
if let view = self.view as? GLKView, let context = context {
  view.context = context
  delegate = self
}

5. Remove the following properties from the top of ViewController:

private var context: EAGLContext?
private var effect = GLKBaseEffect()
private var ebo = GLuint()
private var vbo = GLuint()
private var vao = GLuint()

Finally, at the top of the ViewController class declaration, add an outlet to a MTKView:

@IBOutlet weak var metalView: MTKView!

Setting Up the Storyboard

The ViewController is no longer GLKViewController, so you need to make some changes in the storyboard.

Open Main.storyboard. In this example, the storyboard contains two scenes, both named View Controller Scene. One has a GLKView, and the other one contains a MTKView and a connection to the outlet that you’ve just added to the source code.

All you need to do is set the scene with the MTKView as the initial View Controller. Find the scene which doesn’t currently have the arrow pointing to it. Click on the bar at the top to select the view controller. Alternatively you can select it in the document outline pane. Then open the attributes inspector and check Is Initial View Controller.

Once that’s done, you can delete the first scene. Good work!

Setting Up Metal

Are you ready? It’s time to use some Metal!

In Metal, the main object that you’ll use to access the GPU is MTLDevice. The next most important object is MTLCommandQueue. This object is a queue to which you’ll pass encoded frames.

Open ViewController.swift and add these properties:

private var metalDevice: MTLDevice!
private var metalCommandQueue: MTLCommandQueue!

Now, go to setupMetal(). Replace the contents of it with the following:

metalDevice = MTLCreateSystemDefaultDevice() 
metalCommandQueue = metalDevice.makeCommandQueue()  
metalView.device = metalDevice 
metalView.delegate = self

That’s a lot shorter than what was there before right!

This grabs the system default Metal device, then makes a command queue from the device. Then it assigns the device to the Metal view. Finally it sets the view controller as the view’s delegate to receive callbacks when to draw and resize.

Now you need to to implement the MTKViewDelegate protocol.

At the bottom of ViewController.swift, add this extension:

extension ViewController: MTKViewDelegate {
  // 1
  func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {
  }
  
  // 2
  func draw(in view: MTKView) { 
  }
}

This extension implements two methods.

  1. This method is called when the drawable size changes, such as when the screen rotates.
  2. This method is called to perform the actual drawing.

Basic Drawing

You’re all set to draw! To keep things simple, you’ll draw a gray background first.

For each frame you want to draw in Metal, you must create a command buffer in which you specify what and how you want to draw. Then, this buffer is encoded on the CPU and sent to the GPU through the command queue.

Add the following code inside draw(in:):

// 1
guard let drawable = view.currentDrawable else {
  return
}
    
let renderPassDescriptor = MTLRenderPassDescriptor() // 2
renderPassDescriptor.colorAttachments[0].texture = drawable.texture // 3
renderPassDescriptor.colorAttachments[0].loadAction = .clear // 4
renderPassDescriptor.colorAttachments[0]
  .clearColor = MTLClearColor(red: 0.85, green: 0.85, blue: 0.85, alpha: 1.0) // 5
    
// 6
guard let commandBuffer = metalCommandQueue.makeCommandBuffer() else {
  return
}
    
// 7
guard let renderEncoder = commandBuffer
  .makeRenderCommandEncoder(descriptor: renderPassDescriptor) else {
    return
}
    
// Frame drawing goes here
    
renderEncoder.endEncoding() // 8

commandBuffer.present(drawable) // 9
commandBuffer.commit() // 10

This is a big one. Here’s what is going on in the code above:

  1. Ensure there’s a valid drawable to be used for the current frame.
  2. MTLRenderPassDescriptor contains a collection of attachments that are the rendering destination for pixels generated by a rendering pass.
  3. Set the texture from the view as a destination for drawing.
  4. Clear every pixel at the start of a rendering.
  5. Specify the color to use when the color attachment is cleared. In this case, it’s a lovely 85% gray.
  6. Ask the command queue to create a new command buffer.
  7. Create an encoder object that can encode graphics rendering commands into the command buffer. You’ll add your actual drawing code after this statement later.
  8. Declare that all command generations from this encoder are complete.
  9. Register a drawable presentation to occur as soon as possible.
  10. Commit this command buffer for execution in the command queue.

In summary, you create a command buffer and a command encoder. Then, you do your drawing in the command encoder and commit the command buffer to the GPU through the command queue. These steps are then repeated each time the frame is drawn.

Note: If you’re using the simulator, at this point you’ll get a build error. You need a compatible device to build and run the app.

Build and run the app.

Metal Gray View

What a gorgeous grey color!

Drawing Primitives

It will take some time to draw something more meaningful. So, take a deep breath and dive right in!

To draw something, you must pass data that represents the object to the GPU. You already have Vertex structure to represent the vertices data, but you need to make a small change to use it with Metal.

Open ViewController.swift and change the Indices property from this:

var Indices: [GLubyte]

To this:

var Indices: [UInt32]

You’ll see why this change is required when drawing primitives.

Data Buffers

To pass vertices data to the GPU, you need to create two buffers: one for vertices and one for indices. This is similar to OpenGL’s element buffer object (EBO) and vertex buffer object (VBO).

Add these properties to ViewController:

private var vertexBuffer: MTLBuffer!
private var indicesBuffer: MTLBuffer!

Now, inside setupMetal(), add the following at the bottom:

let vertexBufferSize = Vertices.size()
vertexBuffer = metalDevice
  .makeBuffer(bytes: &Vertices, length: vertexBufferSize, options: .storageModeShared)
    
let indicesBufferSize = Indices.size()
indicesBuffer = metalDevice
  .makeBuffer(bytes: &Indices, length: indicesBufferSize, options: .storageModeShared)

This asks metalDevice to create the vertices and indices buffers initialized with your data.

Now, go to draw(in:), and right before:

renderEncoder.endEncoding()

Add the following:

renderEncoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)  
renderEncoder.drawIndexedPrimitives(
  type: .triangle, 
  indexCount: Indices.count, 
  indexType: .uint32, 
  indexBuffer: indicesBuffer, 
  indexBufferOffset: 0) 

First this passes the vertex buffer to the GPU, setting it at index 0. Then, it draws triangles using indicesBuffer. Note that you need to specify the index type which is uint32. That’s why you changed the indices type earlier.

Build and run the app.

Crash! That’s not good. You passed data from the CPU to the GPU. It crashed because you didn’t specify how the GPU should use this data. You’ll need to add some shaders! Fortunately, that’s the next step.

Adding Shaders

Create a new file. Click File ▸ New ▸ File…, choose iOS ▸ Source ▸ Metal File. Click Next. Name it Shaders.metal and save it wherever you want.

Metal uses C++ to write shaders. In most cases, it’s similar to the GLSL used for OpenGL.

For more on shaders, check the references at the bottom of this tutorial.

Add this to the bottom of the file:

struct VertexIn {
    packed_float3 position;
    packed_float4 color;
};

struct VertexOut {
    float4 computedPosition [[position]];
    float4 color;
};

You’ll use these structures as input and output data for the vertex shader.

Writing a Vertex Shader

A vertex shader is a function that runs for each vertex that you draw.

Below the structs, add the following code:

vertex VertexOut basic_vertex( // 1
  const device VertexIn* vertex_array [[ buffer(0) ]], // 2
  unsigned int vid [[ vertex_id ]]) {                  // 3
    // 4
    VertexIn v = vertex_array[vid];

    // 5
    VertexOut outVertex = VertexOut();
    outVertex.computedPosition = float4(v.position, 1.0);
    outVertex.color = v.color;
    return outVertex;
}
  1. The vertex indicates this is a vertex shader function. The return type for this shader is VertexOut.
  2. Here, you get the vertex buffer that you passed to the command encoder.
  3. This parameter is a vertex id for which this shader was called.
  4. Grab the input vertex for the current vertex id.
  5. Here, you create a VertexOut and pass data from the current VertexIn. This is simply using the same position and color as the input.

At this point, the vertex shader’s job is done.

Writing a Fragment Shader

After the vertex shader finishes, the fragment shader runs for each potential pixel.

Below the vertex shader, add the following code:

fragment float4 basic_fragment(VertexOut interpolated [[stage_in]]) { 
  return float4(interpolated.color);              
}

The fragment shader receives the output from the vertex shader — the VertexOut. Then, the fragment shader returns the color for the current fragment.

Hooking up the Shaders to the Pipeline

The shaders are in place, but you haven’t hooked them to your pipeline yet. In order to do that, go back to ViewController.swift, and add this property to the class:

private var pipelineState: MTLRenderPipelineState!

This property will contain shaders data.

Now, find setupMetal(). Add the following at the bottom of the method:

// 1
let defaultLibrary = metalDevice.makeDefaultLibrary()!
let fragmentProgram = defaultLibrary.makeFunction(name: "basic_fragment")
let vertexProgram = defaultLibrary.makeFunction(name: "basic_vertex")

// 2
let pipelineStateDescriptor = MTLRenderPipelineDescriptor()
pipelineStateDescriptor.vertexFunction = vertexProgram
pipelineStateDescriptor.fragmentFunction = fragmentProgram
pipelineStateDescriptor.colorAttachments[0].pixelFormat = .bgra8Unorm

// 3
pipelineState = try! metalDevice
  .makeRenderPipelineState(descriptor: pipelineStateDescriptor)

This is what the code does:

  1. Finds the vertex and fragment shaders by their name in all .metal files.
  2. Creates a descriptor object with the vertex shader and the fragment shader. The pixel format is set to a standard BGRA (Blue Green Red Alpha) 8-bit unsigned.
  3. Asks the GPU to compile all that into the GPU-optimized object.

Now, the pipeline state is ready. It’s time to use it. For this, go to draw(in:), and right before:

renderEncoder.drawIndexedPrimitives(
  type: .triangle, 
  indexCount: Indices.count, 
  indexType: .uint32, 
  indexBuffer: indicesBuffer, 
  indexBufferOffset: 0)

Add:

renderEncoder.setRenderPipelineState(pipelineState)

Build and run. You should get this colorful screen.

Metal Gradient Background

Matrices

In order to manipulate the scene, you need to pass the projection and model-view matrices to the GPU. The projection matrix allows you to manipulate the perception of the scene to make closer objects appear bigger than farther ones. The model view matrix allows you to manipulate the position, rotation, and scale of an object or the whole scene.

In order to use those matrices, you’ll create a new struct. Open Vertex.swift. At the top of the file, add:

struct SceneMatrices {
    var projectionMatrix: GLKMatrix4 = GLKMatrix4Identity
    var modelviewMatrix: GLKMatrix4 = GLKMatrix4Identity
}

Note that you’ll still use GLKMatrix4. This part of GLKit is not deprecated, so you can use it for matrices in Metal.

Now, open ViewController.swift, and add two new properties:

private var sceneMatrices = SceneMatrices()
private var uniformBuffer: MTLBuffer!

Then, go to draw(in:), and right before:

renderEncoder.setRenderPipelineState(pipelineState)

Add:

// 1
let modelViewMatrix = GLKMatrix4MakeTranslation(0.0, 0.0, -6.0)
sceneMatrices.modelviewMatrix = modelViewMatrix

// 2
let uniformBufferSize = MemoryLayout.size(ofValue: sceneMatrices)
uniformBuffer = metalDevice.makeBuffer(
  bytes: &sceneMatrices, 
  length: uniformBufferSize, 
  options: .storageModeShared)

// 3
renderEncoder.setVertexBuffer(uniformBuffer, offset: 0, index: 1)

Here’s what the code above does:

  1. Creates a matrix to shift the object backwards by 6 units, to make it look smaller.
  2. Creates a uniform buffer like you did with vertex buffer before, but with matrices data.
  3. Hooks the uniform buffer to the pipeline and sets its identifier to 1.

Projection Matrix

While you’re still in ViewController.swift, inside mtkView(_:drawableSizeWillChange:), add the following:

let aspect = fabsf(Float(size.width) / Float(size.height))  
let projectionMatrix = GLKMatrix4MakePerspective(
  GLKMathDegreesToRadians(65.0), 
  aspect, 
  4.0, 
  10.0)
sceneMatrices.projectionMatrix = projectionMatrix 

This code creates a projection matrix based on the aspect ratio of the view. Then, it assigns it to the scene’s projection matrix.

With this in place, your square will now look square and not stretched out to the whole screen. :]

Matrices in Shaders

You’re almost there! Next, you’ll need to receive matrices data in shaders. Open Shaders.metal. At the very top, add a new struct:

struct SceneMatrices {
    float4x4 projectionMatrix;
    float4x4 viewModelMatrix;
};

Now, replace the basic_vertex function with the following:

vertex VertexOut basic_vertex(
  const device VertexIn* vertex_array [[ buffer(0) ]],
  const device SceneMatrices& scene_matrices [[ buffer(1) ]], // 1
  unsigned int vid [[ vertex_id ]]) {
    // 2
    float4x4 viewModelMatrix = scene_matrices.viewModelMatrix;
    float4x4 projectionMatrix = scene_matrices.projectionMatrix;
    
    VertexIn v = vertex_array[vid];

    // 3
    VertexOut outVertex = VertexOut();
    outVertex
      .computedPosition = projectionMatrix * viewModelMatrix * float4(v.position, 1.0);
    outVertex.color = v.color;
    return outVertex;
}

Here’s what has changed:

  1. Receives matrices as a parameter inside the vertex shader.
  2. Extracts the view model and projection matrices.
  3. Multiplies the position by the projection and view model matrices.

Build and run the app. You should see this:

Gradient Static Cube

W00t! A square!

Making it Spin

In the OpenGL implementation, GLViewController provided lastUpdateDate which would tell you when the last render was performed. In Metal, you’ll have to create this yourself.

First, in ViewController, add a new property:

private var lastUpdateDate = Date()

Then, go to draw(in: ), and just before:

commandBuffer.present(drawable)

Add the following code:

commandBuffer.addCompletedHandler { _ in
  self.lastUpdateDate = Date()
}

With this in place, when a frame drawing completes, it updates lastUpdateDate to the current date and time.

Now, it’s time to spin! In draw(in:), replace:

let modelViewMatrix = GLKMatrix4Translate(GLKMatrix4Identity, 0, 0, -6.0)

With:

var modelViewMatrix = GLKMatrix4MakeTranslation(0.0, 0.0, -6.0)
let timeSinceLastUpdate = lastUpdateDate.timeIntervalSince(Date()) 

// 1
rotation += 90 * Float(timeSinceLastUpdate)

// 2
modelViewMatrix = GLKMatrix4Rotate(
  modelViewMatrix, 
  GLKMathDegreesToRadians(rotation), 0, 0, 1)

This increments the rotation property by an amount proportional to the time between the last render and this render. Then, it applies a rotation around the Z-axis to the model-view matrix.

Build and run the app. You’ll see the cube spinning. Success!

Metal Spinning Cube

Where to Go From Here?

Download the final project for this tutorial using the Download Materials button at the top or bottom of the tutorial.

Congrats! You’ve learned a ton about the Metal API! Now you understand some of the most important concepts in Metal, such as shaders, devices, command buffers, and pipelines, and you have some useful insights into the differences between OpenGL and Metal.

For more, be sure to check out these great resources from Apple:

You also might enjoy this Beginning Metal course, which covers these concepts in detail with video tutorials.

If reading is more your style, check out the Metal Getting Started tutorial. For an entire book on the subject, see Metal by Tutorials.

I hope you’ve enjoyed this tutorial! If you have any comments or questions, please join the forum discussion below!

Contributors

Comments