Beta Test Tutorial for iOS: How To Host a Beta Test for your iOS App

Ray Wenderlich
This dude says: Thank you Battle Map Beta Testers!

This dude says: Thank you Battle Map Beta Testers!

As indie software developers, sometimes the only type of testing we do is developer testing. And we all know that can be less than optimal ;]

But with larger apps, sometimes it can be useful to get some outside help. I recently did that with my latest app (Battle Map for iPad) by hosting a beta test, so thought I’d share some of the knowledge I learned along the way.

This beta test tutorial will address the questions that might arise as you’re setting up your beta test – or deciding whether you should have one in the first place. We will cover how to find beta testers, get them a copy of your app, respond to the feedback, and much more.

So let’s dig into the world of iOS beta testing!

Do I Need a Beta Test?

If you have a simple app or you’re in a rush to get your app out quickly, the answer is probably no. After all, hosting a beta test takes a lot of time and energy, and delays the time till you’re making money!

But you might want to consider hosting a beta test if:

  • Your app has a lot of features. If you have a complicated app with a lot of features, this means there’s a higher chance of bugs! Beta testers can help with this by finding bugs before they’re released.
  • You are making a high quality app. If you are making a high quality app, beta testers can help a lot by requesting features that you may not have thought of. By implementing some of these features before launch, your app can be much higher quality.
  • You are making a game. If you’re making a game, beta testing can be particularly valuable because it’s hard to balance a game’s difficulty and fun properly without having players other than yourself give it a try!

How Can I Find Beta Testers?

If you just need a few beta testers, one good way is to simply write a post requesting beta testers on forums that app fans frequent. I’ve posted on the MacRumors Forums and subreddits of reddit.com in the past and found some great & helpful testers (thanks RPG reddit!).

If you’re on Twitter, that’s another good source to try! There are a lot of great iPhone developers on there willing to help out a fellow developer with their app, often providing an invaluable developer perspective (thanks @GeekAndDad & @wadevondoom!).

And of course, don’t forget family & friends. For games, this can be especially valuable because you can watch the testers use your app in person, which is often worth 1000 words.

Do I Need a Beta Signup Form?

If you just need a couple beta testers for your app, a beta signup form is probably overkill. But there are several advantages of having a beta signup form:

  • It makes it easier to have multiple rounds of testing. Sometimes you might want to go through multiple rounds of testing for your app, which is useful to avoid burnout of beta testers as you make upgrades & improvements. By having a beta signup form, testers can sign up anytime and you can send invites as you’re ready.
  • It allows you to ask questions to the beta testers. When people sign up to beta test, you have a chance to ask them some questions that might help you decide who to invite. For example, in my case I wanted to make sure both Game Masters and Players tried out Battle Map before launch to get both perspectives.
  • It helps you keep your information organized. Since your beta testers will be in a database, you can keep track of their UDIDs, whether they’ve responded with feedback, and other useful bookkeeping information in a central place.
  • It allows people to ask to be notified when your app launches. A lot of times, the people who sign up for your beta test are interested in knowing when your app launches, so give them the chance to sign up to find out! It is a simple way to help spread the word on launch day.

How Do I Make a Beta Signup Form?

Aha, I thought you might ask that! A n00b’s guide to making a beta signup form is coming to this blog soon, so stay tuned! :]

Update: My guide is good if you want to learn how to make a signup form yourself or have your own database on your web site, but if you don’t care about all that and want to just get something working ASAP, Jindrich from the comments section recommends using Google Docs forms, which looks like a quick & easy way to go!

What Do I Do Once I Find Some Beta Testers?

Once you have some volunteers, before you can do anything you need to get them to send you their device’s unique id (UDID) so you can build them a copy for their device.

Here are the instructions I usually send the beta testers on how to send me their UDID:

  1. Search for the free app called “UDID Tool” on the App store
  2. Download it onto your [insert device type here]
  3. Run app, click Send email, email to [insert email address here] (and please put your name in the email so I know who it’s from)

Update: If you don’t want to make your users download an app to get the UDID, there’s a perhaps easier way pointed out by Charles Tidswell in the comments section below. Just check out this tutorial on an alternate method for testers to send you their UDID from iTunes!

How Do I Send the Beta Testers an Ad-Hoc Build?

If you haven’t done an Ad-Hoc build before, you might think that it’s some hugely complicated process (like I once did), but never fear: it’s super easy!

Just take the following steps:

  1. Log on to the iPhone Provisioning Portal inside the iPhone Dev Center.
  2. Set up an App ID for your App if you haven’t already and make sure you set up your app to use it as well.
  3. Go to the Devices tab, and add an entry for each UDID that you receive from your beta testers. Note you’re limited to only 100 per year, so you have to use some discretion in how many people you invite to your beta tests to make sure you don’t run out!
  4. Go to the Provisioning tab, and create a new profile. Choose your App ID and select all of the devices that you want the app to be able to run on.
  5. Follow the instructions in the “Building your Application with Xcode for Distribution” section of the Distribution tab to set your project up to use the new provisioning profile + make an Ad-Hoc build (don’t worry, it’s easy).
  6. Build the project with your new Ad-Hoc configuration, and locate the .app that was built. Next, we need to package it so it’s easily installable by your beta testers:
    1. Make a folder named “Payload”
    2. Copy your app into the “Payload” folder
    3. Zip up the “Payload” folder
    4. Rename the resulting .zip with an .ipa extension. So if it was “MyApp.zip”, name it “MyApp.ipa”.
  7. Take your .ipa and the .mobileprovision, and zip up both of those so there’s just a single file needed by the beta testers.

Update: There’s an even easier way to create the .ipa from the .app than the manual instructions shown above. Jim Murff from the comments section below pointed out that @gaminghorror has written an excellent tutorial series on creating a Cocos2D project template that covers now to tweak the build settings to automatically build an IPA file for you!

And that’s it! Now you just need to send the .zip to the beta testers and they can install your app.

How Can the Beta Testers Install the App?

When I send the beta testers the .zip and .mobileprovision (which I usually do by uploading to my web site and sending them a link rather than clogging their email with a large attachment), I also send them the following instructions:

  1. Download [insert link to zip file here]
  2. Unzip the file. There should be two files inside: a file ending with “.ipa” and a file ending with “.mobileprovision”.
  3. Drag-and-drop the .mobileprovision file and the .ipa file onto Applications in your iTunes library on Windows, or the iTunes icon on a Mac
  4. In iTunes, select your device and open the Applications tab. Sync applications and the new app should be checked.
  5. Sync your device and try out the app!

Also, here’s a link to a useful guide that you can refer the beta testers to if they get stuck.

How Should I Solicit Feedback?

When you send your beta testers the link to the app zip and the instructions, that’s also a good time to explain what sort of feedback you’re looking for.

For example, you could point out particular areas you’re hoping for them to test or particular things you’re uncertain about/are hoping for feedback on. You could even list particular questions you’d like them to answer.

Here are the questions that I ask for my beta test:

  1. Things you like
  2. Things you don’t like, or things that are confusing or frustrating
  3. Ideas for improvements you have
  4. Bugs you may have found (with instructions for how to reproduce if possible)

Basically I try to keep the questions to a minimum in order to encourage free-flow thoughts and responses, which I find the most valuable kind of feedback to get.

What Should I Do With the Feedback?

If all goes well, at this point you should have several emails back from your beta testers with a bunch of feedback. Some feedback will be positive encouragment, some will be bugs they found, and MOST of what you’ll probably get is ideas for new features and improvements.

This is valuable stuff, so the first thing you should do is make sure you keep good track of it! I like using Evernote to keep track of all ideas for improvements people suggest for any of my apps (as well as potential blog posts! :])

Next, you should look over the ideas and implement as many as you have time for before launch. Of course, you probably can’t do everything, so you can prioritize by doing the ones that are requested by several different people, would be especially awesome to add, or are easy low-hanging fruit to implement.

It’s not necessary, but is often a nice touch to let the beta testers who came up with the idea know that you actually implemented their idea, they’re sure to appreciate that their ideas came in handy!

How Many Responses Should I Expect?

As far as I can tell, the number of responses you get varies a lot based on the people you invite and your app. But for a point of reference, in my most recent beta test I had:

  • 45 signups
  • 16 invites sent
  • 12 responded with their UDIDs
  • 9 responded with feedback

So, as you can see not everyone will respond with feedback. So in my case, for future betas I’ll probably plan on inviting around double the amount of people than the amount of feedback emails I’m hoping to end up with.

Conclusion

All in all, having a beta test is a lot of work, but can help lead to a more polished app if you take up some of the beta tester’s advice and suggestions.

But is it worth it given all of the time and effort? I think that is completely debatable, and probably dependent on the app too. What do you guys think – what do you do for your apps?

Ray Wenderlich

Ray is an indie software developer currently focusing on iPhone and iPad development, and the administrator of this site. He’s the founder of a small iPhone development studio called Razeware, and is passionate both about making apps and teaching others the techniques to make them.

When Ray’s not programming, he’s probably playing video games, role playing games, or board games.

User Comments

13 Comments

  • Ray - I saw this the other day and thought you might be interested.

    http://testflightapp.com/ << a distribution service for beta testing iOS apps.
    digitalcassette
  • I just went through this procedure for my new game, Hungry Faces. I sent the file to my sister who I've set up as a beta tester. She's running Windows Vista. I talked her through the procedures but the .ipa and .mobileprovision will not install into iTunes. When she drags the files into iTunes and drops them on the Apps, nothing happens. I sent her the .ipa by itself which was built using XCode 4.2 and it will install in iTunes but won't install on her iPad. She has iPad 1 with iOS 5.0.1. I deleted the app from my iPad 2 running iOS 5.0.1 and double clicked on the .ipa file and it put it in iTunes and then synced it onto my iPad. The big difference is my iPad is connected to my iMac computer whereas her iPad is connected to a Windows Vista computer. So apparently this procedure isn't working on Vista or I'm missing something. We tried a number of things like dragging the zipped up .ipa that has the provisioning file and the app in it but iTunes said it was an invalid file. She can get the .ipa by itself into iTunes but when she loads it to her iPad through sync, it says it failed to install (on the iPad). Any ideas?

    Bob Hunt
    Bob Hunt
  • Use www.testflightapp.com, Bob. Or set up this Over-the-Air distribution by yourself. Then the beta tester can simply browse to a website and install the app from there.

    http://jeffreysambells.com/posts/2010/0 ... tribution/
    Hollance
  • Hi!

    I would say a beta test is more then finding bugs. Our main reason we do beta testing on our apps is that we want to test it on different target groups.
    Different ages, girls!, and on different devices and OS. We use http://www.thebetafamily.com that eases the process a lot. They have registered testers and collects UDIDs and have feedback forms. I have also tried services like ibetatest etc.

    Thank you for sharing

    Elivs
    elvisjoh
  • Thanks for the share guys.
    I am developing a simple game and it is almost done. Some Final check is needed.

    Since my game is very simple, I am not planning to go big on beta Test and I am not going to spend too much time on it. I need some people (5-10) who can beta test my game and give me good feedback. I am planning to include their name on my game as a beta tester :)

    If you want to help me please let me know.

    Thanks
    RabinL
  • If you just want to find a limited numbers of beta testers (5-20) and you have a limited budget (which is the case for all indie devs :mrgreen: ), you can also use http://www.reviewfordev.com
    It's a paid service, but the beta testers are hand-picked among the pool of reviewers and the reviews are manually validated by the admins to control the quality of the feedback.
    Jeoud
  • I know it's an old post, but I thought this might be useful for people who will view the post.
    You can increase of odds of you getting more feedback from your users by integrating a simple feedback mechanism as the one offered by http://instabug.io.

    We've released it in beta right now, but you can signup using this promocode: rwnd4450

    Let me know what you think about it
    msoliman3890
  • This post is outdated. Because I trust the tutorials of Ray Wenderlich, I followed the instructions. It worked, but when I finally found the workflow on the Apple developers pages, a much simpler method was presented. Perhaps an idea to update the tutorial? ;)
    LucasUijlings
  • @LucasUijlings: Sorry about that, yes this tutorial is an older one. What's the link you found with a better workflow for future reference for others?
    rwenderlich
  • Our organization is building an enterporse mobile app that would not use Testflight for beta distribution. We would have the beta testers download the app from our internal servers.
    Since we have a mix of internal and external beta testers (who count more than 100), what are our options

    1. Can we create multiple developer accounts to address this scenario?
    I would presume we would need multiple download url's to address each block of 100 beta testers (UDID's). Can we still use the same app name and create multiple "ipa" files each addressing a block of 100 bate testers?

    2. Can we also use a combination of developer program / developer enterprise program to address the mix of beta users? Would any restrictions apply?
    sundar
  • By chance I am doing this today and went here before checking out Apple's site. Since I saw Ray @rwenderlich asking about the link, I ink I have it for you.

    https://developer.apple.com/library/ios ... OSApp.html

    Hope it works and helps you out.
    Kat
    StealthKat
  • just posted a tutorial request before i found this thread

    @StealthKat, yes the information under that link is all very nice, but assumes the beta tester syncs through a desktop.
    For iPhone users to go through the beta process over-the-air (OTA) it doesn't inspire confidence you're giving the right step by step instruction
    aremvee
  • According to Apple's documentation, this should also work for Mac apps. So far, I have tried to follow the steps, but my app runs happily on computers that do not have the provision profile installed. I can't send out a beta release without some kind of distribution control.

    Any suggestions of how to locate my error with the build settings or code signing would be greatly appreciated. I'm running Xcode 4.6 on OS X 10.7.5.
    robc

Other Items of Interest

Ray's Monthly Newsletter

Sign up to receive a monthly newsletter with my favorite dev links, and receive a free epic-length tutorial as a bonus!

Advertise with Us!

Hang Out With Us!

Every month, we have a free live Tech Talk - come hang out with us!


Coming up in September: iOS 8 App Extensions!

Sign Up - September

RWDevCon Conference?

We are considering having an official raywenderlich.com conference called RWDevCon in DC in early 2015.

The conference would be focused on high quality Swift/iOS 8 technical content, and connecting as a community.

Would this be something you'd be interested in?

    Loading ... Loading ...

Our Books

Our Team

Tutorial Team

... 49 total!

Update Team

  • Riccardo D'Antoni

Editorial Team

  • Matt Galloway

... 23 total!

Code Team

  • Orta Therox

... 1 total!

Translation Team

  • Jiyeon Seo
  • Di Peng
  • Jose De La Roca

... 33 total!

Subject Matter Experts

  • Richard Casey

... 4 total!