Beginning Automated Testing With Xcode Part 2/2

Charlie Fulton Charlie Fulton

Learn how to add automated testing into your iOS 6 apps!

Note from Ray: This is the tenth and final iOS 6 tutorial in the iOS 6 Feast! This tutorial comes from our new book iOS 6 By Tutorials. Charlie Fulton wrote this chapter – a friend of mine and one of the newest members of the Tutorial Team. Enjoy!

This is a blog post by iOS Tutorial Team member Charlie Fulton, a full time iOS developer who enjoys hunting, fishing, and hanging out with his family.

Welcome back to our beginning automated testing with Xcode tutorial series!

In the first part of the tutorial, you learned how to add your code to Github, set up a Jenkins continuous integration server, and how to add a unit test to your app.

In this second and final part of the series, you’ll learn a lot more about unit testing, and how to automatically archive your builds and upload them to TestFlight!

Creating a unit test class

Now let’s add a simple unit test class for testing the WowUtils class. WowUtils looks up the string values for the Character class, Character race, and Item quality from the web service JSON data.

It’s best to create a new unit test class for each new class you want to test. Xcode has an Objective-C test case class template just for this purpose.

In the Xcode project navigator, right-click on the GuildBrowserLogicTests group, choose New File\Cocoa Touch\Objective-C test case class, click Next, enter WowUtilsTests for the class name, click Next again, make sure only the GuildBrowserLogicTests target is selected, and then click Create.

Now you have a test suite to which you can add some test cases. Let’s create your first test case. This test case will make sure that you get the correct response when looking up a character’s class.
Here are the methods you are testing from WowUtils.h:

+(NSString *)classFromCharacterType:(CharacterClassType)type;
+(NSString *)raceFromRaceType:(CharacterRaceType)type;
+(NSString *)qualityFromQualityType:(ItemQuality)quality;

Note:These methods are expected to get the correct name based on the data retrieved from Blizzard’s web service. Below you can see the output used by the WowUtils class (Chrome is a great tool for inspecting JSON output from a web service):

Replace the contents of WowUtilsTests.m with:

#import "WowUtilsTests.h"
#import "WowUtils.h"
 
@implementation WowUtilsTests
 
// 1
-(void)testCharacterClassNameLookup
{
    // 2
    STAssertEqualObjects(@"Warrior",
                         [WoWUtils classFromCharacterType:1],
                         @"ClassType should be Warrior");
    // 3
    STAssertFalse([@"Mage" isEqualToString:[WoWUtils classFromCharacterType:2]],
                  nil);
 
    // 4
    STAssertTrue([@"Paladin" isEqualToString:[WoWUtils classFromCharacterType:2]],
                 nil);
    // add the rest as an exercise
}
 
- (void)testRaceTypeLookup
{
    STAssertEqualObjects(@"Human", [WoWUtils raceFromRaceType:1], nil);
    STAssertEqualObjects(@"Orc", [WoWUtils raceFromRaceType:2], nil);
    STAssertFalse([@"Night Elf" isEqualToString:[WoWUtils raceFromRaceType:45]],nil);
    // add the rest as an exercise
}
 
- (void)testQualityLookup
{
    STAssertEquals(@"Grey", [WoWUtils qualityFromQualityType:1], nil);
    STAssertFalse([@"Purple" isEqualToString:[WoWUtils qualityFromQualityType:10]],nil);
    // add the rest as an exercise
}
 
@end

Let’s break this down bit-by-bit.

  1. As I stated earlier, all test cases must start with the name test.
  2. The expectation is that the WowUtils class will give you the correct class name, given an ID. The way you verify this expectation is with the STAssert* macros. Here you are using the STAssertEqualObjects macro.
    The expected result, “Warrior”, is compared to the result from the WowUtils method; if the test fails, you log the message “ClassType should be Warrior”.
  3. It’s always good to include a “failing test” in your test case. This is a test where the result is expected to fail. Again, you are using one of the assertion macros from the SenTestingKit – this time STAssertFalse.

    The expected result, “Mage”, is compared to the result from the WowUtils method; if the test fails, you use the default message, since you passed in nil in this example.

  4. Finally, you have another example test macro to use.
Note: For a complete list of the testing macros, check out the Unit-Test Result Macro Reference (http://bit.ly/Tsi9ES) from the Apple Developer library.

Now you can run your new test suite that presently contains one test case. Go to Product\Test (⌘-U).

Doh! You’ll see a compile error:

Since you added a new target, you need to let it know about the classes you’re trying to test. Every target has its own set of source files available to it. You need to add the source files manually, since you’re running a logic test target without the bundle and test host set.

  1. Switch to the project navigator – if it’s not open, use the View\Navigators\Show Project Navigator menu item (⌘-1).
  2. Click on the project root to bring up the Project and Targets editor in the main editor area.
  3. Select the Build Phases tab along the top of Xcode’s main editing pane.
  4. Now click on the GuildBrowserLogicTests target in the TARGETS section. You should see this:

  5. Expand the disclosure arrow in the Compile Sources section, and click on the + button. In the popup window, choose the following classes: Character.m, Guild.m, Item.m, and WowUtils.m. Then click Add.

  6. Now add the app target as a dependency so that it will get built before your test target runs. Expand the disclosure arrow in the Target Dependencies section, click the + button, choose the GuildBrowser app target and click Add.

When these steps are complete, you should see the following:

Now run Product\Test (⌘-U) and the test should succeed. You can see the output from all the tests by switching to the Log Navigator (go to View\Navigators\Show Log Navigator (⌘-7)).

You should see this when you click on the last build on the left sidebar:

Make sure to commit and push your latest changes to Github.

Testing a class with local JSON data

Let’s take a look at the Character class and add a test suite for it. Ideally, you would have added these tests as you were developing the class.

What if you wanted to test creating your Character class from local data? How would you do that? What if you wanted to share such data with each test case?

So far, you haven’t used two special methods that are available in your unit tests: setUp and tearDown. Every time you run your unit tests, each test case is invoked independently. Before each test case runs, the setUp method is called, and afterwards the tearDown method is called. This is how you can share code between each test case.

When building an app that relies on data from web services, I find it very helpful to create tests using data that matches the payload from the services. Quite often in a project, you will only be working on the client side and waiting for the services from another developer. You can agree on a format and can “stub” out the data in a JSON file. For this app, I downloaded some character and guild data to create tests for the model classes so I could flesh those out before working on the networking code.

First add your test data. Extract this AutoTestData.zip file and drag the resulting folder into your Xcode project. Make sure that Copy items into destination group’s folder (if needed) is checked, Create groups for any added folders is selected, and that the GuildBrowserLogicTests target is checked, and then click Finish.

Update 10/31/12: There were some questions on the forums about the exact command line instructions to use to set up Jenkins, so if you have any troubles check out this handy list of Jenkins shell steps text file I put together for you.

Now add an Objective-C test case class for your Character test cases. In the Xcode project navigator, right-click on the GuildBrowserLogicTests group, choose New File\Cocoa Touch\Objective-C test case class, click Next, enter CharacterTests for the class name, click Next again, make sure only the GuildBrowserLogicTests target is selected, and then click Create.

Replace the contents of CharacterTests.m with:

#import "CharacterTests.h"
#import "Character.h"
#import "Item.h"
 
@implementation CharacterTests
{
    // 1
    NSDictionary *_characterDetailJson;
}
 
// 2
-(void)setUp
{
    // 3
    NSURL *dataServiceURL = [[NSBundle bundleForClass:self.class]
                             URLForResource:@"character" withExtension:@"json"];
 
    // 4
    NSData *sampleData = [NSData dataWithContentsOfURL:dataServiceURL];
    NSError *error;
 
    // 5
    id json = [NSJSONSerialization JSONObjectWithData:sampleData
                                              options:kNilOptions
                                                error:&error];
    STAssertNotNil(json, @"invalid test data");
 
 
    _characterDetailJson = json;
}
 
-(void)tearDown
{
    // 6
    _characterDetailJson = nil;
}
 
@end

Hammer time, break it down! :]

  1. Remember that test classes can have instance variables, just like any other Objective-C class. Here you create the instance variable _characterDetailJson to store your sample JSON data.
  2. Remember that setUp is called before each test case. This is useful because you only have to code up the loading once, and can manipulate this data however you wish in each test case.
  3. To correctly load the data file, remember this is running as a test bundle. You need to send self.class to the NSBundle method for finding bundled resources.
  4. Create NSData from the loaded resource.
  5. Now create the JSON data and store it in your instance variable.
  6. Remember that tearDown is called after each test case. This is a great spot to clean up.

Make sure everything is loading up correctly by running Product\Test (⌘-U). OK, now you have your test class set up to load some sample data.

After the tearDown method, add the following code:

// 1
- (void)testCreateCharacterFromDetailJson
{
    // 2
    Character *testGuy1 = [[Character alloc] initWithCharacterDetailData:_characterDetailJson];
    STAssertNotNil(testGuy1, @"Could not create character from detail json");
 
    // 3
    Character *testGuy2 = [[Character alloc] initWithCharacterDetailData:nil];
    STAssertNotNil(testGuy2, @"Could not create character from nil data");
}

Break it down!

  1. Here you are creating test cases for the Character class designated initializer method, which takes an NSDictionary from the JSON data and sets up the properties in the class. This might seem trivial, but remember that when developing the app, it’s best to add the tests while you are incrementally developing the class.
  2. Here you are just validating that initWithCharacterDetailData does indeed return something, using another STAssert macro to make sure it’s not nil.
  3. This one is more of a negative test, verifying that you still got a Character back even though you passed in a nil NSDictionary of data.

Run Product\Test (⌘-U) to make sure your tests are still passing!

After the testCreateCharacterFromDetailJson method in CharacterTests.m, add the following:

// 1
-(void)testCreateCharacterFromDetailJsonProps
{
    STAssertEqualObjects(_testGuy.thumbnail, @"borean-tundra/171/40508075-avatar.jpg", @"thumbnail url is wrong");
    STAssertEqualObjects(_testGuy.name, @"Hagrel", @"name is wrong");
    STAssertEqualObjects(_testGuy.battleGroup, @"Emberstorm", @"battlegroup is wrong");
    STAssertEqualObjects(_testGuy.realm, @"Borean Tundra", @"realm is wrong");
    STAssertEqualObjects(_testGuy.achievementPoints, @3130, @"achievement points is wrong");
    STAssertEqualObjects(_testGuy.level,@85, @"level is wrong");
 
    STAssertEqualObjects(_testGuy.classType, @"Warrior", @"class type is wrong");
    STAssertEqualObjects(_testGuy.race, @"Human", @"race is wrong");
    STAssertEqualObjects(_testGuy.gender, @"Male", @"gener is wrong");
    STAssertEqualObjects(_testGuy.averageItemLevel, @379, @"avg item level is wrong");
    STAssertEqualObjects(_testGuy.averageItemLevelEquipped, @355, @"avg item level is wrong");
}
 
// 2
-(void)testCreateCharacterFromDetailJsonValidateItems
{
    STAssertEqualObjects(_testGuy.neckItem.name,@"Stoneheart Choker", @"name is wrong");
    STAssertEqualObjects(_testGuy.wristItem.name,@"Vicious Pyrium Bracers", @"name is wrong");
    STAssertEqualObjects(_testGuy.waistItem.name,@"Girdle of the Queen's Champion", @"name is wrong");
    STAssertEqualObjects(_testGuy.handsItem.name,@"Time Strand Gauntlets", @"name is wrong");
    STAssertEqualObjects(_testGuy.shoulderItem.name,@"Temporal Pauldrons", @"name is wrong");
    STAssertEqualObjects(_testGuy.chestItem.name,@"Ruthless Gladiator's Plate Chestpiece", @"name is wrong");
    STAssertEqualObjects(_testGuy.fingerItem1.name,@"Thrall's Gratitude", @"name is wrong");
    STAssertEqualObjects(_testGuy.fingerItem2.name,@"Breathstealer Band", @"name is wrong");
    STAssertEqualObjects(_testGuy.shirtItem.name,@"Black Swashbuckler's Shirt", @"name is wrong");
    STAssertEqualObjects(_testGuy.tabardItem.name,@"Tabard of the Wildhammer Clan", @"nname is wrong");
    STAssertEqualObjects(_testGuy.headItem.name,@"Vicious Pyrium Helm", @"neck name is wrong");
    STAssertEqualObjects(_testGuy.backItem.name,@"Cloak of the Royal Protector", @"neck name is wrong");
    STAssertEqualObjects(_testGuy.legsItem.name,@"Bloodhoof Legguards", @"neck name is wrong");
    STAssertEqualObjects(_testGuy.feetItem.name,@"Treads of the Past", @"neck name is wrong");
    STAssertEqualObjects(_testGuy.mainHandItem.name,@"Axe of the Tauren Chieftains", @"neck name is wrong");
    STAssertEqualObjects(_testGuy.offHandItem.name,nil, @"offhand should be nil");
    STAssertEqualObjects(_testGuy.trinketItem1.name,@"Rosary of Light", @"neck name is wrong");
    STAssertEqualObjects(_testGuy.trinketItem2.name,@"Bone-Link Fetish", @"neck name is wrong");
    STAssertEqualObjects(_testGuy.rangedItem.name,@"Ironfeather Longbow", @"neck name is wrong");
}

You need to make sure that the properties correctly match up with the JSON data that you loaded initially. Just a few comments on what’s going on here:

  1. This tests the information shown in the main screen Character cell.
  2. This tests the information shown in the CharacterDetailViewController when you click on a Character cell from the main screen.

To have a little fun, “forget on purpose” to run Product\Test (⌘-U), and commit and push your changes. You will see what your little friend Jenkins will find, when you update your Jenkins job script to include the tests.

Finalizing your Jenkins job

First off, make sure to commit all changes and push them to your origin/master branch on GitHub.

As a reminder, you will need to do the following:

  1. Go to File\Source Control\Commit (⌥⌘-C), then on the following screen, enter a commit message like “added test target” and click Commit.
  2. Push the local master branch to the remote origin/master branch. Go to File\Source Control\Push and click Push.

Now edit your Jenkins job again to include the tests you just set up. Go to the Jenkins Dashboard\GuildBrowser job\Configure. In the Build section, replace the existing code with this:

export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer/
 
xcodebuild -target GuildBrowserLogicTests \
-sdk iphonesimulator \
-configuration Debug \
TEST_AFTER_BUILD=YES \
clean build

Click Save and then Build Now.

Ummm, RUN! You broke the build and Chuck Norris is going to find you! The gloves are on! This is serious, man. No one can escape him. :]

So what happened? You could scour the output logs of the build that put you on Mr. Norris’s radar, or… you could set up Jenkins to give you some test results reporting! I don’t know about you, but if I’m on that radar I want to be off it FAST! You should have also received an email telling you that you broke the build. Let’s get some reporting, stat!

Getting a unit test report

It would be quite tedious to pour through the logs after each build, and it sure would be nice if there were a handy report you could look at to see what passed and what failed.

Well, it turns out there is a script that does exactly that! Christian Hedin has written an awesome Ruby script to turn the OCUnit output into JUnit-style reports. You can find it on GitHub at https://github.com/ciryon/OCUnit2JUnit.

A copy of the Ruby script is included in the resources for this chapter. Work quickly; remember whose radar you are on!

Copy the ocunit2junit.rb file to somewhere that Jenkins can access it – I placed mine in /usr/local/bin. Make a note of the location, and use it below when updating the build job.

In the GuildBrowser Jenkins job, update the shell script to the following:

export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer/
 
xcodebuild -target GuildBrowserLogicTests \
-sdk iphonesimulator \
-configuration Debug \
TEST_AFTER_BUILD=YES \
clean build | /usr/local/bin/ocunit2junit.rb

The only new bit is on the final line, where the output from the build is piped through to the Ruby script for processing.

Now you need to add another Post-Build Action to your job to capture these reports. At the bottom of the configure screen, click on the Add post-build action button and choose Publish JUnit test result report.

Enter test-reports/*.xml in the Test report XMLs text field.

Click Save and then Build Now. When the job is finished and you go to the main project area for the GuildBrowser job, you should see a Latest Test Result link. Work quickly; get on that link!

Now it’s really obvious what has angered “He who must not be named.”

It looks like a bug has been found by the CharacterTests class. Let’s investigate this further by clicking on the link to the test name:

Error Message
 
"Vicious Pyrium Bracers" should be equal to "Girdle of the Queen"s Champion" name is wrong
 
Stacktrace
 
/Users/charlie/.jenkins/workspace/GuildBrowser/GuildBrowserLogicTests/testclasses/CharacterTests.m:58

OK, let’s go check out the test code. First look at line 76 from CharacterTests.m:

STAssertEqualObjects(hagrel.waistItem.name,@"Girdle of the Queen's Champion", @"name is wrong");

Interesting! Open the file character.json to see what you have; maybe your data was wrong, but surely not your code!

Note: When working with JSON, I highly recommend verifying your test data or any JSON using the awesome website jsonlint.com. Not only does it validate your JSON, but it will format it too!

Remember character.json represents what’s returned from Blizzard’s real character web service. You’ll find this snippet:

{
    "thumbnail": "borean-tundra/171/40508075-avatar.jpg",
    "class": 1,
    "items": {"wrist": {
            "icon": "inv_bracer_plate_dungeonplate_c_04",
            "tooltipParams": {
                "extraSocket": true,
                "enchant": 4089
            },
            "name": "Vicious Pyrium Bracers",
            "id": 75124,
            "quality": 3
        },
        "waist": {
            "icon": "inv_belt_plate_dungeonplate_c_06",
            "tooltipParams": {
                "gem0": 52231
            },
            "name": "Girdle of the Queen's Champion",
            "id": 72832,
            "quality": 4
        },
…

Hmmm, the plot thickens. Your test found “Vicious Pyrium Bracers”, the name of the wrist item, but was looking for “Girdle of the Queen’s Champion”, the name of the waist item. Now it’s looking more like a bug!

You can see in the test in CharacterTest.m that you create a Character like so:

Character *hagrel = [[Character alloc] initWithCharacterDetailData:characterDetailJson];

Open Character.m and let’s take a look at initWithCharacterDetailData:. Can you spot the bug?

_wristItem = [Item initWithData:data[@"items"][@"wrist"]];
_waistItem = [Item initWithData:data[@"items"][@"wrist"]];

You were using the wrong key for the waistItem property. Fix that by changing the last line to:

_waistItem = [Item initWithData:data[@"items"][@"waist"]];

OK, now commit your changes and push them to GitHub. Go to the Jenkins project and click on Build Now.

Test result: no failures! Chuck gives that a thumbs up – what a huge relief!

Drill down into the report by clicking on Latest Test Result\root and you will see all the tests that were run. By drilling further into the CharacterTests, you can see that you fixed the issue (look closely at the status on line three :]):

Polling for changes

Now let’s set up the Jenkins project to look for changes every 10 minutes, and if it finds any, to run the build. Edit your Jenkins job again by going to the Jenkins Dashboard\GuildBrowser job\Configure. In the Build Triggers section, check the box. In the schedule text area, enter:

*/10 * * * *

Click Save. Now builds will happen if there is new code that has been pushed to the origin/master repo. Of course, you can still manually build like you’ve been doing with the Build Now button.

Automate archiving

Let’s update the build script to archive your app after successfully running the test script. Edit your Jenkins job again by, you guessed it, going to the Jenkins Dashboard\GuildBrowser job\Configure.

You are now going to add another shell step, which will happen after the test script step. In the Build section, click on the Add build step dropdown and select Execute shell.

Enter the following, replacing the CODE_SIGN_IDENTITY with your distribution certificate name:

# tests passed archive app
 
export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
 
/usr/bin/xcrun xcodebuild -scheme GuildBrowser clean archive \
CODE_SIGN_IDENTITY="iPhone Distribution: Charles Fulton"

Your build section should now look similar to this:

If you save your changes and build again via Jenkins, you should now see the latest archive in the Xcode organizer for this project. In Xcode, open Window\Organizer and switch to the Archives tab, and you should see the GuildBrowser project listed on the left.

When you click on the project, you will see all of your archived builds. This is great, because now you will know that you are building and testing the same archive that will be submitted to the App Store!

Note: If the archiving fails, you will not see an archive listed in the Xcode Organizer, nor will you see any immediate indication from Jenkins as to the archive failure. You would need to check the build logs to see if the archiving actually succeeded.

Usually, the archive process fails because the CODE_SIGN_IDENTITY is not correctly specified, or because it doesn’t match the Bundle ID for the project. So if you run into any archival failures, those would be the items to check. One solution here would be to set the CODE_SIGN_IDENTITY to iPhone Distribution, since that will match the default distribution profile.

Also remember that if you make any project changes to fix the above, you need to commit to Git and push to GitHub. Otherwise, Jenkins won’t pick up your changes for the next build. :)

Next you will add sending this archived build to TestFlight, and keeping track of the artifact (which is Jenkins terminology for the results of the build) in Jenkins.

Uploading the archive to TestFlight

One of the best things about the iOS development community is the variety of awesome frameworks and services that have emerged over the past few years.

Back in the day, it was quite an effort to get a beta build to your testers. You would have to get your IPA file to them by email, have them drag that to iTunes, then connect their device to sync up with iTunes just to get it on their device. You also had to send them an email asking for the UDID of their device, scribble that down or copy it up to create the new provisioning profile, then create the new build. It was a nightmare to keep track of which device belonged to what user, what iOS version they were running, etc.

Enter TestFlight! This is a website that makes distribution and testing of beta versions of apps a breeze. ☺ Before TestFlight, the only way to distribute builds to your beta testers was via ad hoc builds. The ad hoc builds still remain, since TestFlight works within the ad hoc mechanism, but it makes distribution and management of these builds much simpler.

Testflight will also allow you to set up their TestFlight SDK packaging in your app for crash log analysis, usage analysis, and more!

We are going to focus on the auto upload and distribution pieces that TestFlight offers.

PackageApplication

Here is a sweet little Perl script included by Apple in the Xcode.app bundle. You can take a peek at it (no touching!) by opening:

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/PackageApplication

This is the tool that will allow you to do the following from the latest archived build, the one you just created in the previous step. You will be modifying your Jenkins archive step to include:

  • Creating the GuildBrowser.app bundle;
  • Embedding your ad hoc provisioning profile;
  • Codesign with your distribution certificate.

To make sure you’re ready, download your latest ad hoc provisioning profile from the iOS Provisioning Portal and add it to the top level of your project folder.

After downloading it, your project should look like the image below. Notice my ad hoc provisioning profile named Charles_Fulton_All_Ad_Hoc.mobileprovision:

Note: Your .mobileprovision file can be located anywhere. You just have to make sure to give it the full absolute path – no relative paths. For example:

~/Library/MobileDevice/Provisioning\ Profiles/

Must be:

/Users/charlie/Library/MobileDevice/Provisioning\ Profiles/

I like to keep mine in Git, so that when new devices are added, I just check in a new provisioning profile. Then I can do a manual build in Jenkins and it’s ready to go.

Make sure to commit and push to GitHub after adding the file, so Jenkins can see it.

If you are using Xcode to commit to Git, then note that you would need to add the mobile provisioning profile to your project before Xcode sees the new file. Otherwise, you will not be able to commit it to Git. If you use the command-line Git tools or a separate Git client, this issue should not arise.

Let’s edit your Jenkins job again. Go to the Jenkins Dashboard\GuildBrowser job\Configure. Add a new Build step of type execute shell:

export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
 
#
# Setup 
# 
# 1 
PROJECT="GuildBrowser"
SIGNING_IDENTITY="iPhone Distribution: Charles Fulton"
PROVISIONING_PROFILE="${WORKSPACE}/Charles_Fulton_All_Ad_Hoc.mobileprovision"
 
# 2 
# this is the latest archive from previous build step
ARCHIVE="$(ls -dt ~/Library/Developer/Xcode/Archives/*/${PROJECT}*.xcarchive|head -1)"
# 3
IPA_DIR="${WORKSPACE}"
DSYM="${ARCHIVE}/dSYMs/${PROJECT}.app.dSYM"
APP="${ARCHIVE}/Products/Applications/${PROJECT}.app"
 
 
# 
# PackageApplication
#
 
# package up the latest archived build
/bin/rm -f "${IPA_DIR}/${PROJECT}.ipa"
 
# 4
/usr/bin/xcrun -sdk iphoneos PackageApplication \
-o "${IPA_DIR}/${PROJECT}.ipa" \
-verbose "${APP}" \
-sign "${SIGNING_IDENTITY}" \
--embed "${PROVISIONING_PROFILE}"
 
# zip and ship
/bin/rm -f "${IPA_DIR}/${PROJECT}.dSYM.zip"
 
# 5
/usr/bin/zip -r "${IPA_DIR}/${PROJECT}.dSYM.zip" "${DSYM}"

There is a lot going on in this build script. You know what that means.

Break it down!

  1. These are the settings for what certificate and provisioning profile to use when creating your IPA file. You should change the SIGNING_IDENTITY and PROVISIONING_PROFILE to use your ad hoc distribution profile.
  2. This bit of shell trickery finds the latest archive build location. This gives you a sneaky way to get the latest archive result directory. To make more sense of this one, open a terminal and run the command:

    ls -dt ~/Library/Developer/Xcode/Archives/*/GuildBrowser*.xcarchive|head -1

    You should see output like this:

    /Users/charlie/Library/Developer/Xcode/Archives/2012-08-27/GuildBrowser 8-27-12 10.37 AM.xcarchive/
  3. You can now use the ARCHIVE variable to create the variables APP and DSYM saving the absolute paths to send to PackageApplication. Take a peek inside by trying this command:
    ls –l "/Users/charlie/Library/Developer/Xcode/Archives/2012-08-27/GuildBrowser 8-27-12 10.37 AM.xcarchive/Products/Applications/GuildBrowser.app"
  4. Here you are calling the PackageApplication script. Notice that Jenkins gives you some nice environment variables in $WORKSPACE. The $WORKSPACE variable lets you get an absolute path to the Jenkins job. You can now create artifacts in Jenkins of exactly what gets sent to your users.
  5. Compress the dSYMs from the archive. dSYMs are used to symbolicate crash logs so that you can find out which source file, method, line, etc. had an issue instead of getting memory addresses that would mean nothing to you.

Before saving and building the updated script, let’s add a step to create artifacts of all successfully-created .apps and dSYMs.

Go to the Post-build Actions section, and select Archive the artifacts from the Add post-build action menu.

Enter *.ipa, *.dSYM.zip in the files to archive text field.

Click Save and select Build Now. Once the build completes, you should see this:

If something fails at this point, it usually is because the code signing information wasn’t correct, or because the ad hoc provisioning profile isn’t in the project root folder. So check the build logs to see what is going on.

Mission completion

OK, let’s send those artifacts to TestFlight and notify your users of the new build.

Note: This section assumes you already have a TestFlight (testflightapp.com) account. You will need your TestFlight team and API tokens.

You can get your API token here: https://testflightapp.com/account/#api

Your team token can be found by clicking on the team info button while logged into TestFlight.

Edit your Jenkins job again by going to the Jenkins Dashboard\GuildBrowser job\Configure. You will be editing the script you added in the previous step.

Add this code, filled out with your own TestFlight info, after the export DEVELOP_DIR line:

# testflight stuff
API_TOKEN=<YOUR API TOKEN>
TEAM_TOKEN=<YOUR TEAM TOKEN>

Add this to the end of the existing script:

#
# Send to TestFlight
#
/usr/bin/curl "http://testflightapp.com/api/builds.json" \
  -F file=@"${IPA_DIR}/${PROJECT}.ipa" \
  -F dsym=@"${IPA_DIR}/${PROJECT}.dSYM.zip" \
  -F api_token="${API_TOKEN}" \
  -F team_token="${TEAM_TOKEN}" \
  -F notes="Build ${BUILD_NUMBER} uploaded automatically from Xcode. Tested by Chuck Norris" \
  -F notify=True \
  -F distribution_lists='all'
 
echo "Successfully sent to TestFlight"

Click Save and do another Build Now.

Now when the job completes, your build should have been sent to TestFlight! And your users should have received an email telling them that a new build is available. This will allow them to install your app right from the email and begin testing for you!

Where to go from here?

You should now be equipped to set up an automated building, testing, and distribution system for all of your iOS apps!

Let’s recap what you did in this chapter:

  • First you learned how to set up a remote repo on Github, giving you a spot to share and test your code.
  • After that, you took a look at continuous integration with Jenkins, and created a nice build script step-by-step, first building, then testing, and finally uploading your archived app to Testflight.
  • You also looked at how to include a “bottom up” approach to unit testing your code. If you’re interested in learning more about unit testing in iOS, I highly recommend the book Test-Driven iOS Development by Graham Lee. I also encourage all of you to submit a radar to apple to make it easier to run the application unit tests from scripts, without hacks!

If you enjoyed this tutorial and want to learn more, check out our new book iOS 6 by Tutorials, where you take the same app and do some “top down” unit testing by creating a cool little testing robot. This robot willg use instruments and a UI Automation script to drive some UI interactions in the GuildBrowser app. :]

If you have any questions or comments on this tutorial or have any questions about automated testing in general, please join the forum discussion below!


This is a blog post by iOS Tutorial Team member Charlie Fulton, a full time iOS developer who enjoys hunting, fishing, and hanging out with his family.

Charlie Fulton
Charlie Fulton

Charlie Fulton is a full time iOS developer. He has worked with many languages and technologies in the past 16 years, and is currently specializing in iOS and Cocos2D development. In his spare time, Charlie enjoys hunting, fishing, and hanging out with his family.

You can follow Charlie on Twitter.

User Comments

50 Comments

[ 1 , 2 , 3 , 4 ]
  • Hi Charlie,

    Nice article

    But I have a problem, WowUtilsTests.h is not found, I try to recreate it, but I don't know what put in it.

    Can you help me?

    Thanks
    TimTim
  • This is a great tutorial. Everything worked great for a long time but I ran into a snag that I'd like to pass on the resolution to.

    I realized that the build number displayed on Testflight all of a sudden did not match up with the Xcode build numbers. After troubleshooting this I discovered that in a previous Xcode "Add files to <project>" somehow the "copy items into destination group's folder" became unchecked. The Jenkins job ran successfully but it kept uploading a previous build to Testflight. Apparently, the Jenkins job errored out (though in the console output it stated: "** ARCHIVE SUCCEEDED **". Later, however, in the console output I found "The following build commands failed:" which were a CpResource and a CopyPNGFile.
    I deleted those files from my project, cleaned, checked in to Git, added the files back into the project (using "copy items into destination"), checked in to Git, ran the Jenkins job and the correct build was produced and uploaded to TestFlight.

    Hope this helps anyone avoid this needle in a haystack issue.
    CocosKat
  • Hi, Charlie
    Thank you for your great tutorial, it really helps me to start to writing a build script by myself.
    But one thing I want to mention about this article is it would be more greater if you update this tutorial for Xcode 5 version. Making a target for unit test part looks need to be change.
    Thank you!
    barty82
  • Charlie,
    I'm trying to get through your tutorial. Stuck at the point to set up the saving of test reports. The config screen in Jenkins says:
    test-reports/*.xml doesnt match anything: even test-reports doesnt exist

    Is 'test-reports' relative to the git repo? Jenkins workspace? my file system? Nothing I try seems to fit here. I've read the posts here and there were others that seemed to have the same issue, but nowhere do I see a solution or explanation posted. Can you help?

    Thanks
    rlawson289
  • A good to tutorials! but i have a problem at this time, when i build, some error output: it trouble me long time!

    + echo 'Archive Success'
    Archive Success
    + /usr/bin/xcrun -sdk iphoneos PackageApplication
    error: An application was not specified.
    Build step 'Execute shell' marked build as failure
    Archiving artifacts
    Build step 'Upload to Testflight' marked build as failure
    Sending e-mails to: dengbinhero@163.com
    Finished: FAILURE
    dengbinhero
[ 1 , 2 , 3 , 4 ]

Other Items of Interest

Ray's Monthly Newsletter

Sign up to receive a monthly newsletter with my favorite dev links, and receive a free epic-length tutorial as a bonus!

Advertise with Us!

Vote for Our Next Tutorial!

Every week, we alternate between Gaming and Non-Gaming tutorial votes. This week: Gaming!

    Loading ... Loading ...

Last week's winner: Apple TestFlight Tutorial.

Suggest a Tutorial - Past Results

Hang Out With Us!

Every month, we have a free live Tech Talk - come hang out with us!


Coming up in January: WatchKit.

Sign Up - January

Our Books

Our Team

Tutorial Team

  • Tammy Coron

... 60 total!

Update Team

... 12 total!

Editorial Team

  • Matt Galloway

... 17 total!

Code Team

  • Orta Therox

... 3 total!

Subject Matter Experts

  • Richard Casey

... 4 total!