Home iOS & Swift Books Metal by Tutorials

30
Metal Performance Shaders Written by Caroline Begbie & Marius Horga

Heads up... You're reading this book for free, with parts of this chapter shown beyond this point as scrambled text.

You can unlock the rest of this book, and our entire catalogue of books and videos, with a raywenderlich.com Professional subscription.

Unlock Now

In Chapter 19, “Tessellation & Terrains”, you had a brief taste of using the Metal Performance Shaders (MPS) framework. MPS consists of low-level, fine-tuned, high-performance kernels that run off the shelf with minimal configuration. In this chapter, you’ll dive a bit deeper into the world of MPS.

Overview

The MPS kernels make use of data-parallel primitives that are written in such a way that they can take advantage of each GPU family’s characteristics. The developer doesn’t have to care about which GPU the code needs to run on, because the MPS kernels have multiple versions of the same kernel written for every GPU you might use. Think of MPS kernels as convenient black boxes that work efficiently and seamlessly with your command buffer. Simply give it the desired effect, a source and destination resource (buffer or texture), and then encode GPU commands on the fly!

The Sobel Filter

The Sobel filter is a great way to detect edges in an image.

The Sobel filter
Lya Xanag sitnes

let shader = MPSImageSobel(device: device)
shader.encode(
  commandBuffer: commandBuffer,
  sourceTexture: inputImage,
  destinationTexture: drawable.texture)

Image Processing

There are a few dozen MPS image filters, among the most common being:

A Gaussian blur matrix
E Puisroos vxuq voxyiz

Convolution
Wuctawifaip

(6 * 1  +  7 * 2  +  3 * 1  +
 4 * 2  +  9 * 4  +  8 * 2  +
 9 * 1  +  2 * 2  +  3 * 1) / 16 = 6
Convolution applied to border pixels
Woxxovudeed oxxhoim de jocnew bejisw

(0 * 1  +  0 * 2  +  0 * 1  +
 0 * 2  +  6 * 4  +  7 * 2  +
 0 * 1  +  4 * 2  +  9 * 1) / 9 = 6

Bloom

The bloom effect is quite a spectacular one. It amplifies the brightness of objects in the scene and makes them look luminous as if they’re emitting light themselves.

The bloom effect
Wko hhuay eytirk

The Starter Project

➤ In Xcode, open the starter project for this chapter and build and run the app.

Setting Up the Textures

➤ In the Post Processing group, open Bloom.swift, and import the MPS framework:

import MetalPerformanceShaders

var outputTexture: MTLTexture!
var finalTexture: MTLTexture!

outputTexture = TextureController.makeTexture(
  size: size,
  pixelFormat: view.colorPixelFormat,
  label: "Output Texture",
  usage: [.shaderRead, .shaderWrite])
finalTexture = TextureController.makeTexture(
  size: size,
  pixelFormat: view.colorPixelFormat,
  label: "Final Texture",
  usage: [.shaderRead, .shaderWrite])

Image Threshold to Zero

The Metal Performance Shader MPSImageThresholdToZero is a filter that returns either the original value for each pixel having a value greater than a specified brightness threshold or 0. It uses the following test:

destinationColor =
  sourceColor > thresholdValue ? sourceColor : 0

guard
  let drawableTexture =
    view.currentDrawable?.texture else { return }
let brightness = MPSImageThresholdToZero(
  device: Renderer.device,
  thresholdValue: 0.5,
  linearGrayColorTransform: nil)
brightness.label = "MPS brightness"
brightness.encode(
  commandBuffer: commandBuffer,
  sourceTexture: drawableTexture,
  destinationTexture: outputTexture)

metalView.framebufferOnly = false

The Blit Command Encoder

➤ Open Bloom.swift, and add this to the end of postProcess(view:commandBuffer:):

finalTexture = outputTexture
guard let blitEncoder = commandBuffer.makeBlitCommandEncoder()
  else { return }
let origin = MTLOrigin(x: 0, y: 0, z: 0)
let size = MTLSize(
  width: drawableTexture.width,
  height: drawableTexture.height,
  depth: 1)
blitEncoder.copy(
  from: finalTexture,
  sourceSlice: 0,
  sourceLevel: 0,
  sourceOrigin: origin,
  sourceSize: size,
  to: drawableTexture,
  destinationSlice: 0,
  destinationLevel: 0,
  destinationOrigin: origin)
blitEncoder.endEncoding()
Brightness threshold
Zkoyqxwuzk dytujnekj

Gaussian Blur

MPSImageGaussianBlur is a filter that convolves an image with a Gaussian blur with a given sigma value (the amount of blur) in both the X and Y directions.

let blur = MPSImageGaussianBlur(
  device: Renderer.device,
  sigma: 9.0)
blur.label = "MPS blur"
blur.encode(
  commandBuffer: commandBuffer,
  inPlaceTexture: &outputTexture,
  fallbackCopyAllocator: nil)
Brightness and blur
Zqujywqevh icg dsop

Image Add

The final part of creating the bloom effect is to add the pixels of this blurred image to the pixels of the original render.

let add = MPSImageAdd(device: Renderer.device)
add.encode(
  commandBuffer: commandBuffer,
  primaryTexture: drawableTexture,
  secondaryTexture: outputTexture,
  destinationTexture: finalTexture)
Brightness, blur and add
Ptilvyvuyq, dter upj iqd

let brightness = MPSImageThresholdToZero(
  device: Renderer.device,
  thresholdValue: 0.8,
  linearGrayColorTransform: nil)
Glowing skeletons
Hwubuts bgeqewimx

Matrix / Vector Mathematics

You learned in the previous section how you could quickly apply a series of MPS filters that are provided by the framework. But what if you wanted to make your own filters?

import MetalPerformanceShaders

guard let device = MTLCreateSystemDefaultDevice(),
      let commandQueue = device.makeCommandQueue()
else { fatalError() }

let size = 4
let count = size * size

guard let commandBuffer = commandQueue.makeCommandBuffer()
else { fatalError() }

commandBuffer.commit()
commandBuffer.waitUntilCompleted()

func createMPSMatrix(withRepeatingValue: Float) -> MPSMatrix {
  // 1
  let rowBytes = MPSMatrixDescriptor.rowBytes(
    forColumns: size,
    dataType: .float32)
  // 2
  let array = [Float](
    repeating: withRepeatingValue,
    count: count)
  // 3
  guard let buffer = device.makeBuffer(
    bytes: array,
    length: size * rowBytes,
    options: [])
  else { fatalError() }
  // 4
  let matrixDescriptor = MPSMatrixDescriptor(
    rows: size,
    columns: size,
    rowBytes: rowBytes,
    dataType: .float32)

  return MPSMatrix(buffer: buffer, descriptor: matrixDescriptor)
}

let A = createMPSMatrix(withRepeatingValue: 3)
let B = createMPSMatrix(withRepeatingValue: 2)
let C = createMPSMatrix(withRepeatingValue: 1)

let multiplicationKernel = MPSMatrixMultiplication(
  device: device,
  transposeLeft: false,
  transposeRight: false,
  resultRows: size,
  resultColumns: size,
  interiorColumns: size,
  alpha: 1.0,
  beta: 0.0)

multiplicationKernel.encode(
  commandBuffer: commandBuffer,
  leftMatrix: A,
  rightMatrix: B,
  resultMatrix: C)

// 1
let contents = C.data.contents()
let pointer = contents.bindMemory(
  to: Float.self,
  capacity: count)
// 2
(0..<count).map {
  pointer.advanced(by: $0).pointee
}
Show result
Wjar kalaqk

Value history
Qoxuu gedjifs

Challenge

You may have noticed that in the app where you did the bloom post processing, the Outline option does nothing. Your challenge is to fill out Outline.swift so that you have an outline render:

Outline
Iudmicu

Key Points

  • Metal Performance Shaders are compute kernels that are performant and easy to use.
  • The framework has filters for image processing, implementations for neural networks, can solve systems of equations with matrix multiplication, and has optimized intersection testing for ray tracing.
  • Convolution takes a small matrix and applies it to a larger matrix. When applied to an image, you can blur or sharpen or distort the image.
  • Bloom adds a glow effect to an image, replicating real world camera artifacts that show up in bright light.
  • The threshold filter can filter out pixels under a given brightness threshold.
31. Performance Optimization 29. Advanced Lighting

Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.

© 2022 Razeware LLC

You're reading for free, with parts of this chapter shown as scrambled text. Unlock this book, and our entire catalogue of books and videos, with a raywenderlich.com Professional subscription.

Unlock Now

To highlight or take notes, you’ll need to own this book in a subscription or purchased by itself.

Read for Free with a Subscription

Buy this book for $59.99