Metal is a unified application programming interface (API) for the graphics processing unit, or GPU. It’s unified because it applies to both 3D graphics and data-parallel computation paradigms. Metal is a low-level API because it provides programmers near-direct access to the GPU. Finally, Metal is a low-overhead API because it reduces the central processing unit (CPU) cost by multi-threading and pre-compiling of resources.
But beyond the technical definition, Metal is the most appropriate way to use the GPU’s parallel processing power to visualize data or solve numerical challenges. It’s also tailored to be used for machine learning, image/video processing or, as this book describes, graphics rendering.
This book will introduce you to low-level graphics programming in Metal — Apple’s framework for programming on the graphics processing unit (GPU). As you progress through this book, you’ll learn many of the fundamentals that go into making a game engine and gradually put together your own engine. Once your game engine is complete, you’ll be able to put together 3D scenes and program your own simple 3D games. Because you’ll have built your 3D game engine from scratch, you’ll be able to customize every aspect of what you see on your screen.
This book is for intermediate Swift developers interested in learning 3D graphics or gaining a deeper understanding of how game engines work.
Before You Begin
This section tells you a few things you need to know before you get started, such as what you’ll need for hardware and software, where to find the project files for this book, and more.
It takes a wealth of knowledge to render a simple triangle on the screen or animate game characters. This section will guide you through the necessary basics of vertex wrangling, lighting, textures and character animation; and if you’re worried about the math, don’t be! Although computer graphics is highly math-intensive, each chapter explains everything you need, and you’ll get experience creating and rendering models.
To this point, you’ve been working in playgrounds. You’ll go further with Metal by creating a macOS app from scratch, where you’ll learn how the rendering pipeline works to create the graphics on screen.
We'd all love to be math geniuses but some of us lost the opportunity early in life. Fortunately to use math, you don't always have to know exactly what's under the hood. In this chapter you're going to become a matrix master and you will learn what matrices can do for you and how to manipulate them painlessly.
Lighting and beyond! In this chapter you'll learn basic lighting; but more importantly, you'll learn how to craft data in shaders, and be on the path to mastering shader artistry. Lighting, shadows, non-photorealistic rendering - these are all techniques that start with the methods that you'll learn in this chapter.
This is the final chapter on how to render still models. In the previous chapter, you rendered a simple house with a single color texture imported using Model I/O. In this chapter you'll find out how to use material groups to describe a surface, and how to design textures for micro detail.
Rendering still models is a great achievement, but rendering models that move is even more fun. So far your models have been simple inanimate props. You'll now render characters with body movement and give them personality. In this chapter, you'll start off by bouncing a ball, and then move on to rendering a friendly skeleton.
Section II: The Scene
With the basics under your belt, you can move on to creating game scenes with scenery, skies and terrains. You’ll be able to add multiple models to a scene, and move around the scene and explore it.
In this chapter you'll take your rendering engine and put together a simple game engine by abstracting the rendering code away from scene management. This takes you to the next level where you can start creating your own games.
Before embarking on huge features such as tessellation and instancing, you'll learn some simple techniques to improve your render quality. A series of operations run on the GPU after the fragments have been processed in the pipeline, including alpha testing, depth testing, stencil testing, scissor testing, blending, and anti-aliasing. You’ll touch on most of these operations in this chapter.
So far, you’ve used normal map trickery in the fragment function to show the fine details of your low poly models. To achieve a similar level of detail without using normal maps requires a change of model geometry by adding more vertices, which can problematically choke up the pipeline. In this chapter, you’re going to create a detailed terrain using a small number of points. You’ll send a flat ground plane with a grayscale texture describing the height, and the tessellator will create as many vertices as needed.
Many of the best features in games aren’t coded manually; they’re procedurally generated instead. Learn how to generate vertices for grass, how constant and argument buffers work, how to use L-system trees, and then add the grass and random trees to your scene.
Up to this point, you have been running projects and playgrounds that only had one pass. In other words, you used just one render command encoder to submit all your draw calls to the GPU. In more complex apps, you would often need the result of one pass used in a following pass before presenting the texture to the screen. Sometimes you need to render content off-screen and use it in subsequent stages of your app. When you finish this chapter, you will be able to render a scene with shadows and multiple lights.
Get incredibly detailed with the lighting effects in your scene. Learn about the rendering equation, reflection and refraction, the Fresnel effect, lens flare, subsurface scattering and more lighting details!
MPS consists of low-level, fine-tuned, high-performance kernels that run off the shelf with minimal configuration. The MPS kernels make use of data-parallel primitives that are written in such a way that they can take advantage of each GPU family’s characteristics. In this chapter, you’ll dive a bit deeper into the world of MPS.
Whether it’s objects in the scene you seem to have accidentally lost, or maybe more lights and shadows you want to render, you’ll get to use some of Xcode’s excellent tools to find out what’s happening on the GPU, as well as find ways to optimize your game to run at blazing speeds.
In this final chapter, you’ll learn more ways to optimize the performance of your game. You’ll understand how to categorize GPUs, how to manage memory, how to synchronize resources between the CPU and GPU, how to move your game on multiple threads and what are the best practices for a smooth rendering.