Assembly Register Calling Convention Tutorial

Objective-C and Registers

Registers use a specific calling convention. You can take that same knowledge and apply it to other languages as well.

When Objective-C executes a method, a special C function is executed named objc_msgSend. There’s actually several different types of these functions, but more on that later. This is the heart of message dispatch. As the first parameter, objc_msgSend takes the reference of the object upon which the message is being sent. This is followed by a selector, which is simply just a char * specifying the name of the method being called on the object. Finally, objc_msgSend takes a variable amount of arguments within the function if the selector specifies there should be parameters.

Let’s look at a concrete example of this in an iOS context:

[UIApplication sharedApplication];

The compiler will take this code and create the following pseudocode:

id UIApplicationClass = [UIApplication class];
objc_msgSend(UIApplicationClass, "sharedApplication");

The first parameter is a reference to the UIApplication class, followed by the sharedApplication selector. An easy way to tell if there are any parameters is to simply check for colons in the Objective-C selector. Each colon will represent a parameter in a Selector.

Here’s another Objective-C example:

NSString *helloWorldString = [@"Can't Sleep; " stringByAppendingString:@"Clowns will eat me"];

The compiler will create the following (shown below in pseudocode):

NSString *helloWorldString; 
helloWorldString = objc_msgSend(@"Can't Sleep; ", "stringByAppendingString:", @"Clowns will eat me");

The first argument is an instance of an NSString (@"Can't Sleep; "), followed by the selector, followed by a parameter which is also an NSString instance.

Using this knowledge of objc_msgSend, you can use the registers in x64 to help explore content, which you’ll do very shortly.

Putting Theory to Practice

You can download the starter project for this tutorial here.

For this section, you’ll be using a project supplied in this tutorial’s resource bundle called Registers.

Open this project up through Xcode and give it a run.

This is a rather simple application which merely displays the contents of some x64 registers. It’s important to note that this application can’t display the values of registers at any given moment, it can only display the values of registers during a specific function call. This means that you won’t see too many changes to the values of these registers since they’ll likely have the same (or similar) value when the function to grab the register values is called.

Now that you’ve got an understanding of the functionality behind the Registers macOS application, create a symbolic breakpoint for NSViewController’s viewDidLoad method. Remember to use “NS” instead of “UI”, since you’re working on a Cocoa application.

Build and rerun the application. Once the debugger has stopped, type the following into the LLDB console:

(lldb) register read

This will list all of the main registers at the paused state of execution. However, this is too much information. You should selectively print out registers and treat them as Objective-C objects instead.

If you recall, -[NSViewController viewDidLoad] will be translated into the following assembly pseudocode:

RDI = UIViewControllerInstance 
RSI = "viewDidLoad"
objc_msgSend(RDI, RSI)

With the x64 calling convention in mind, and knowing how objc_msgSend works, you can find the specific NSViewController that is being loaded.

Type the following into the LLDB console:

(lldb) po $rdi 

You’ll get output similar to the following:

<Registers.ViewController: 0x6080000c13b0>

This will dump out the NSViewController reference held in the RDI register, which as you now know, is the location of the first argument to the method.

In LLDB, it’s important to prefix registers with the $ character, so LLDB knows you want the value of a register and not a variable related to your scope in the source code. Yes, that’s different than the assembly you see in the disassembly view! Annoying, eh?

Try printing out the RSI register, which will hopefully contain the selector that was called. Type the following into the LLDB console:

(lldb) po $rsi 

Unfortunately, you’ll get garbage output that looks something like this:

140735181830794

Why is this?

An Objective-C selector is basically just a char *. This means, like all C types, LLDB does not know how to format this data. As a result, you must explicitly cast this reference to the data type you want.

Try casting it to the correct type:

(lldb) po (char *)$rsi 

You’ll now get the expected:

"viewDidLoad"

Of course, you can also cast it to the Selector type to produce the same result:

(lldb) po (SEL)$rsi

Now, it’s time to explore an Objective-C method with arguments. Since you’ve stopped on viewDidLoad, you can safely assume the NSView instance has loaded. A method of interest is the mouseUp: selector implemented by NSView’s parent class, NSResponder.

In LLDB, create a breakpoint on NSResponder’s mouseUp: selector and resume execution. If you can’t remember how to do that, here are the commands you need:

(lldb) b -[NSResponder mouseUp:]
(lldb) continue

Now, click on the application’s window. Make sure to click on the outside of the NSScrollView as it will gobble up your click and the -[NSResponder mouseUp:] breakpoint will not get hit.

As soon as you let go of the mouse or the trackpad, LLDB will stop on the mouseUp: breakpoint. Print out the reference of the NSResponder by typing the following into the LLDB console:

(lldb) po $rdi 

You’ll get something similar to the following:

<NSView: 0x608000120140>

However, there’s something interesting with the selector. There’s a colon in it, meaning there’s an argument to explore! Type the following into the LLDB console:

(lldb) po $rdx 

You’ll get the description of the NSEvent:

NSEvent: type=LMouseUp loc=(351.672,137.914) time=175929.4 flags=0 win=0x6100001e0400 winNum=8622 ctxt=0x0 evNum=10956 click=1 buttonNumber=0 pressure=0 deviceID:0x300000014400000 subtype=NSEventSubtypeTouch

How can you tell it’s an NSEvent? Well, you can either look online for documentation on -[NSResponder mouseUp:] or, you can simply use Objective-C to get the type:

(lldb) po [$rdx class]

Pretty cool, eh?

Sometimes it’s useful to use registers and breakpoints in order to get a reference to an object you know is alive in memory.

For example, what if you wanted to change the front NSWindow to red, but you had no reference to this view in your code, and you didn’t want to recompile with any code changes? You can simply create a breakpoint you can easily trip, get the reference from the register and manipulate the instance of the object as you please. You’ll try changing the main window to red now.

First remove any previous breakpoints using the LLDB console:

(lldb) breakpoint delete
About to delete all breakpoints, do you want to do that?: [Y/n] 

Then type the following into the LLDB console:

(lldb) breakpoint set -o -S "-[NSWindow mouseDown:]"
(lldb) continue

This sets a breakpoint which will fire only once — a one-shot breakpoint.

Tap on the application. Immediately after tapping, the breakpoint should trip. Then type the following into the LLDB console:

(lldb) po [$rdi setBackgroundColor:[NSColor redColor]]
(lldb) continue 

Upon resuming, the NSWindow will change to red!