Augmented Reality iOS Tutorial: Marker Tracking

Getting the camera output on the screen

The first step is to remove the following methods:

• update (only the content)
• glkView:drawInRect: (only the content)
• loadShaders
• compileShader:type:file:
• linkProgram:
• validateProgram:

Remove the line [self loadShaders]; from setupGL.
Search update and add the line [self.stringOGL process];
The last step is to add the line [self.stringOGL render]; to glkView:drawInRect:.

process starts the attempt to detect markers on the latest captured frame.
render renders the latest frame to the screen.

Build and run and you see the captured frame sod your camera.

Drawing a cube

In the last step you use the detected marker to draw a cube and rotate it dependent on the marker position.
Add this lines at the end to update:

    [self.stringOGL getProjectionMatrix:_projectionMatrix.m];
    _numMarkers = [self.stringOGL getMarkerInfoMatrixBased:_markerInfoArray maxMarkerCount:1];

The first line gets the projection matrix that is used for rendering your scene. The next line gets the marker information from String. Lets have a look on the parameters for this method.

• markerInfo is a C array that stores the information for each detected marker. Like mentioned before the array must have the of the total marker count you want to track.
• maxMarkerCount this is the maximum count of marker information you want to receive. Let’s say you want to track five markers in your app. If you pass a value of two you will receive only information for two markers even if there were four markers detected.

The return value tells you how many MarkerInfo was written to your array. So if you want to track four markers in this frame and there are only two detected the method will return two.

Now add this to glkView:drawInRect:

    glBindVertexArrayOES(_vertexArray);
    
    //1
    _effect.transform.projectionMatrix = _projectionMatrix;
    
    //2
    for(int i = 0; i < _numMarkers; i++) {
        //3
        _effect.transform.modelviewMatrix = GLKMatrix4MakeWithArray(_markerInfoArray[i].transform);
        
        float diffuse[4] = {0,0,0,1};
        diffuse[_markerInfoArray[i].imageID % 3] = 1;
        _effect.light0.diffuseColor = GLKVector4MakeWithArray(diffuse);
        [_effect prepareToDraw];
        glDrawArrays(GL_TRIANGLES, 0, 36);
    }

Let's have a look on the interesting stuff of this method.

Where To Go From Here?

Build and run the project. You should see the cube only when the marker is on the screen, and the cube will rotate a little when you tilt or move your device. This way, you can have a look at the top, bottom and sides of the cube.

Note: Your cube may flash from time to time, depending on the screen on which you’re displaying your marker (retina vs. standard) and on the lighting of your surroundings. If you have trouble, try changing rooms or monitors.

Here is an example project with the code from this part of the augmented reality iOS tutorial series.

Note: Because of the beta status we can not provide you the whole project. I removed the libString.a, to compile the project you have to add your own copy of the lib.

At this point, you have a great start toward making your own augmented reality app. With a little OpenGL ES knowledge, you could extend this app to display a model of your choice instead of the 3D cube. You might also want to add in additional features like the model responding to your touches, animation, or sound.

Also, I want to mention another SDK for marker-based AR named Metaio that can also track 3D markers. Metaio is C++ based so you have to mix C++ with Objective-C (referred to as Objective-C++). The basic version is free, so be sure to check it out and see which is the best fit for you.

Want to learn more about augmented reality? Check out our location based augmented reality iOS tutorial, where you'll learn how make an app that displays nearby points of interest on your video feed.

In the meantime, if you have any comments or questions, please join the forum discussion below!

1. This sets the projection matrix on your GLKBaseEffect.
2. Here you start a for loop to iterate over all detected markers. Again you track only one marker so the loop is unnecessary, but it shows you how to work with more markers.
3. This line gets the transformation of the detected marker and applies it to your effect.