Introduction to Google Cardboard for iOS

Lyndsey Scott
Google Cardboard VR

Dive into the world of virtual reality with Google Cardboard VR.

Update 10/3/16: Updated for Swift 3.

Many of the biggest tech firms in the world are investing heavily in virtual reality technologies: Facebook spent $2 billion to acquire Oculus Rift; Disney invested $65 million into a VR film company called Jaunt; Microsoft introduced its HoloLens this year and is selling the device to developers for $3000 apiece.

Apple spent $32 million to acquire the engineers from Metaio and $345 million to acquire PrimeSense, two companies of VR innovators, and pre-orders for Sony’s Playstation VR sold out even quicker than they’d expected, proving there’s a strong consumer market for VR.

Of all the exciting new VR technology being introduced, however, Google Cardboard VR has made virtual reality most accessible to the hobbyist. In fact, with just a low-cost Google Cardboard VR headset, a smartphone and your iOS skills, you can go farther than you ever thought possible.

In this intro to Google Cardboard VR tutorial, fellow jetsetters, you’ll embark on a worldwide 360° vacation by clicking through multiple 360° vacation images and pausing/playing your way through a 360° vacation video.

Note: This Google Cardboard VR tutorial assumes you know the basics of iOS and Swift development. If you’re new to iOS development and Swift, check out our “Learn to Code iOS Apps with Swift Tutorial” series first.

Getting Started

Download the starter project and open Vacation 360.xcodeproj in Xcode. In the 360 images folder in the Navigation Bar on the left-hand side of your Xcode console, you’ll see three 360° panorama images you’ll eventually display in the app. In Main.storyboard, you’ll find a VacationViewController containing a few labels, two empty UIViews and some constraints.

The interface may not look like much yet, but you’ll eventually change these two UIViews into Google Cardboard photo and video VR views. Build and run your app; you won’t see much except for the labels describing the behaviors you’ll implement in this Google Cardboard VR tutorial.

Starter app screenshot

Starter app screenshot

Installing the Google Cardboard VR SDK

Before you start coding your Google Cardboard VR app, the first step is to install the Google Cardboard VR SDK with CocoaPods. Or rather, if you’ve never used CocoaPods before, the first step is to install CocoaPods and the second is to install Google Cardboard VR using CocoaPods.

As described in Joshua Green’s great tutorial on How to Use CocoaPods with Swift, install CocoaPods and the Google VR SDK using the following steps.

Open Terminal, then execute the following command:

sudo gem install cocoapods

Enter your computer’s password when requested. To install the Google VR SDK in the project, navigate to the Vacation_360 starter project folder by using the cd command:

cd ~/ComputerLocation/Vacation_360

Next, create a Podfile for your project:

pod init

Then open the Vacation_360 folder, open the Podfile with a text editor and replace all of its current text with the following:

target "Vacation 360" do
    pod 'GVRSDK'
end

This tells CocoaPods that you want to include the GVRSDK, i.e. the Google VR SDK, as a dependency for your project. Save and close Podfile. Then in Terminal, in the same directory to which you navigated earlier, enter:

pod install

That’s it! You’ve installed Google’s VR SDK. As the log output states, “Please close any current Xcode sessions and use `Vacation 360.xcworkspace` for this project from now on.”

terminal

As with any app in which you install CocoaPods, you’ll be working from the .xcworkspace file instead of the .xcodeproj from this point forward.

Because the Google VR SDK is an Objective-C framework, you’ll have to use an Objective-C bridging header to access it from your Swift code. Go to File\New\File…, select iOS\Source\Header File and then click Next. Name it Vacation 360-Bridging-Header, select the Vacation 360 folder for the Group, then hit Create.

Create Bridging Header

Next, select the Vacation 360 project in the Project Navigator, then select Vacation 360 under TARGETS. Go to Build Settings, look for Objective-C Bridging Header under the Swift Compiler – Code Generation section and enter “Vacation 360/Vacation 360-Bridging-Header.h”.

Bridging Header Build Setting - Google Cardboard VR

As you might have guessed, this tells the compiler where to look for your bridging header.

Replace the contents of Vacation 360-Bridging-Header.h with the following:

#import "GVRPanoramaView.h"
#import "GVRWidgetView.h"
#import "GVRVideoView.h"

Now you have access to the three Google Cardboard VR SDK classes you’ll be using in this tutorial.

Start packing those bags; Vacation 360, here we come! :]

Hula, hula!

Hula, hula!

Coding the Groundwork

The Google Cardboard VR SDK has three VR view types: GVRCardboardView, GVRPanoramaView, and GVRVideoView. GVRCardboardView is by far the most powerful of the three, since in VR mode it lets you determine the user’s head and eye positions, layout 3D audio, and dynamically alter the landscape. Unfortunately, GVRCardboardView requires complex OpenGL rendering beyond the scope of this tutorial; but GVRPanoramaView, Google VR’s image viewer, and GVRVideoView, Google VR’s video viewer, are more than adequate vessels for an international adventure.

Before you start diving into the Google VR methods, you’ll have to lay some groundwork. First, you’ll connect and load the Google Cardboard 360° panoramic photo and video views.

Navigate to Main.storyboard and select the transparent UIView directly under the Click through Sindhu Beach, Grand Canyon, & Underwater Photos label. In the Utilities menu on the right hand side of Xcode, navigate to the Identity Inspector, then in the Custom Class section, enter “GVRPanoramaView” in the Class field. Similarly, select the UIView directly below Play/Pause “Living with Elephants” Safari Video by Photos of Africa and enter “GVRVideoView” in the Class field.

Using the Assistant Editor, control-drag from these two views in Interface Builder to the top of the VacationViewController class in VacationViewController.swift and create Outlets. For GVRPanoramaView, use the name imageVRView and for GVRVideoView use videoVRView. Similarly connect the label outlets to the top of VacationViewController. Name the top label imageLabel and the bottom label videoLabel.

Connecting GVRVideoView outlet - Google Cardboard VR

Next, make the media file URLs accessible by defining them in an enumeration. Directly under the IBOutlets, define enum Media as follows:

enum Media {
  static var photoArray = ["sindhu_beach.jpg", "grand_canyon.jpg", "underwater.jpg"]
  static let videoURL = "https://s3.amazonaws.com/ray.wenderlich/elephant_safari.mp4"
}

In this enumeration, photoArray holds the file names of the 360° images stored in the bundle and videoUrl holds the URL of the 360 video you’ll display. The photoArray is variable and not constant so you can alter the array to easily cycle through the images later on.

Monoscopic 360° of Sindhu Beach, Indonesia by Eggy Sayoga

Monoscopic 360° of Sindhu Beach, Indonesia by Eggy Sayoga

Set the initial photo and video by adding the following in viewDidLoad() just below super.viewDidLoad():

imageVRView.load(UIImage(named: Media.photoArray.first!),
                            of: GVRPanoramaImageType.mono)
videoVRView.load(from: URL(string: Media.videoURL))

You load imageVRView‘s UIImage with Media.photoArray.first! while passing the type GVRPanoramaImageType.mono to indicate that the image is monoscopic.

Then you load videoVRView with an NSURL created from the Media.videoURL string.

There are two format types for 360° images and videos: stereoscopic and monoscopic – stereo and mono for short. Although both formats render 360° media by converting a rectangular panorama into a spherical layout, the stereoscopic format adds some depth to the viewing experience by including two slightly offset images or videos, one stacked above the other, that mimic the viewer’s perspective.

In VR mode, even monoscopic media appears stereoscopic. - Google Cardboard VR

In VR mode, even monoscopic media appears stereoscopic. Photo by Eggy Sayoga

Monoscopic media features the image or video from a single point of view. Google Cardboard VR supports both formats, but displays either format as stereoscopic. When viewing either format in fullscreen VR mode, the viewer shows two side-by-side images or videos slightly offset from one another. You can find the ideal specifications for both mono and stereo here.

Stereoscopic 360° of Sindhi Beach, Indonesia by Eggy Sayoga

Stereoscopic 360° of Sindhi Beach, Indonesia by Eggy Sayoga

Build and run the app without your Google Cardboard VR headset to see what these views provide with minimal configuration. You’ll see the 360° image in the top view and, as long as you’re connected to the internet and thus able to load from the web URL, you’ll see the 360° video in the bottom view.

Rotating the device will rotate both embedded views. Tap the info symbol at the bottom left corner of either VR view and you’ll be directed to a Google Cardboard VR help webpage. Tap elsewhere on the VR views, and nothing will happen.

Initial VR View - Google Cardboard VR

In order to enable the fullscreen mode and fullscreen VR mode, you’ll need to enable the corresponding buttons on the GVRPanoramaView and GVRVideoView. Add the following below the line in viewDidLoad() where you load imageVRView‘s image:

imageVRView.enableCardboardButton = true
imageVRView.enableFullscreenButton = true

Next, add the following below the line where you load videoVRView‘s video:

videoVRView.enableCardboardButton = true
videoVRView.enableFullscreenButton = true

Build and run again; two new buttons should appear on the bottom right of each VR view. Tap the outer right button, i.e. the Fullscreen Button, on either VR view, and the view will resize to fit the full screen. While in full screen, tap the back button on the top left corner to return to the main view.

Next tap the inner right button, i.e. the Cardboard Button, and a view should appear with instructions on how to connect your viewer. To view the media in fullscreen VR mode, insert your device into your Google Cardboard VR headset as instructed such that the top of the device is on the left side of the viewer and the bottom is on the right:

(1) Main embedded view (2) Fullscreen view (3) Google Cardboard instruction view - Google Cardboard VR

(1) Main embedded view (2) Fullscreen view (3) Google Cardboard instruction view

Note: Tapping the Switch button at the bottom left corner of the Google Cardboard instructions screen will allow you to sync your app to your current VR headset by capturing the QR code on your viewer.

To get the full immersive VR experience from this tutorial, I suggest you use a Google Cardboard compatible device. You can purchase a headset on the Google Cardboard website for as little as $14.95.

Or you can even make your own! To do this, just sit back and order a pizza. … What? You’re still VR-less 30 minutes later? Oh right – you also have to cut up that box, add some lenses, a magnet, velcro, and a rubber band to MacGyver your way into your worldwide vacation. The do-it-yourself instructions are available at that same Google Cardboard website link towards the bottom of the page.

You can build your own Google Cardboard! - Google Cardboard VR

You can build your own Google Cardboard!

But if you’re content with viewing a 360° panorama from your device without being fully immersed, this tutorial will still work for you without a viewer.

Making the VR Views Interactive

In Vacation 360, the user can click through images while viewing imageVRView and can play and pause video while viewing videoVRView. The backing classes for each of these inherit from GVRWidgetView, which implements the GVRWidgetViewDelegate protocol. This protocol lets GVRWidgetViewnotify its delegate of various state changes and interactions, which will come in handy while customizing the VR view behaviors.

Directly underneath the closing bracket of the VacationViewController class, add the following:

extension VacationViewController: GVRWidgetViewDelegate {
  func widgetView(_ widgetView: GVRWidgetView!, didLoadContent content: Any!) {
  }

  func widgetView(_ widgetView: GVRWidgetView!, didFailToLoadContent content: Any!, 
    withErrorMessage errorMessage: String!)  {
  }

  func widgetView(_ widgetView: GVRWidgetView!, didChange displayMode: GVRWidgetDisplayMode) {
  }
  
  func widgetViewDidTap(_ widgetView: GVRWidgetView!) {
  }
}

You’ve implemented the GVRWidgetViewDelegate and all of its methods in an extension. It’s time to fill in each method one by one.

widgetView(_:didLoadContent:) is called when a VR view has loaded its content successfully. What’s that you say? How about using this method to only reveal elements of the view once they have loaded? Sounds like a great idea! :]

First, head back to viewDidLoad() and add the following just below super.viewDidLoad():

imageLabel.isHidden = true
imageVRView.isHidden = true
videoLabel.isHidden = true
videoVRView.isHidden = true

This will hide all of the labels and VR views initially, so that they won’t appear until the content loads.

Now, grouped with the other imageVRView code in viewDidLoad(), add the following:

imageVRView.delegate = self 

Similarly, add the following with the videoVRView code:

videoVRView.delegate = self

This sets both view’s GVRWidgetViewDelegates to the VacationViewController.

You can now unhide the labels and views once their corresponding content has loaded. Back in widgetView(_:didLoadContent:), add the following:

if content is UIImage {
 imageVRView.isHidden = false
 imageLabel.isHidden = false
} else if content is NSURL {
 videoVRView.isHidden = false
 videoLabel.isHidden = false
}

widgetView(_:didLoadContent:) is passed the loaded content object – in this case a UIImage for GVRPanoramaViews and an NSURL for GVRVideoViews. For the UIImage, unhide the imageVRView and its corresponding label. For an NSURL, unhide the videoVRView and its corresponding label.

Build and run; you’ll see that each VR view and label now only appears when its corresponding content has loaded:

The VR views now appear after they load. (1) The image VR view has loaded. (2) The video VR view has also loaded.

The VR views now appear after they load. (Left) The image VR view has loaded. (Right) The video VR view has also loaded.

Add the following to widgetView(_:didFailToLoadContent:withErrorMessage:):

print(errorMessage)

This is called when there’s an issue loading the content. In that case, you simply print the passed errorMessage.

Next you’ll use widgetView(_:didChange:) to store the current display mode and selected view, so that tapping on the widgets will trigger the appropriate actions only when a VR view is in fullscreen or VR mode. This method is called each time you switch between embedded, fullscreen, or VR mode and passes the involved widgetView.

First, in the main VacationViewController class below the enum Media declaration block, add the following:

var currentView: UIView?
var currentDisplayMode = GVRWidgetDisplayMode.embedded

currentView will maintain a reference to the view currently displayed in fullscreen or VR mode. currentDisplayMode is used to hold the current display mode. It is initialized to GVRWidgetDisplayMode.embedded which represents the initial display mode when the VR views are embedded in the VacationViewController.

Then in widgetView(_:didChange:), add the following:

currentView = widgetView
currentDisplayMode = displayMode

widgetView(_:didChange:) passes the new displayMode as well as the widgetView where the mode was changed. These are stored in currentDisplayMode and currentView, respectively, for later reference.

Knowing the display mode and displayed view, you can finally start to implement the interactive behaviors while in fullscreen or VR mode. Tapping the magnetic widget button on Cardboard or directly tapping the screen should cycle through the photos in Media.photoArray when the imageVRView is in fullscreen or VR mode. To implement this behavior, add the following to widgetViewDidTap(_:):

// 1
guard currentDisplayMode != GVRWidgetDisplayMode.embedded else {return}
// 2
if currentView == imageVRView {
  Media.photoArray.append(Media.photoArray.removeFirst()) 
  imageVRView?.load(UIImage(named: Media.photoArray.first!), 
    of: GVRPanoramaImageType.mono)
}
  1. Since you only want to handle taps while in fullscreen or VR mode, a currentDisplayMode of .embedded triggers an early return.
  2. If currentView is imageVRView, advance the Media.photoArray queue by removing the first element of the array and appending it to the end. Since Media.photoArray.removeFirst() returns the removed element, you can remove the first element and append it to the end in a single line of code. Then you load imageVRView‘s UIImage using the file now in the first index of the photo array.

Build and run your project; open the image VR view into either fullscreen or VR fullscreen, then tap either the device screen or the widget button respectively. You should be able to click through the images.

Clicking through images in fullscreen - Google Cardboard VR

Clicking through images in fullscreen

You may notice one minor issue: the other elements of the view show through between each click. To prevent that, hide the elements whenever you’re not viewing the embedded display mode by adding the following to the end of widgetView(_:didChange:):

if currentView == imageVRView && currentDisplayMode != GVRWidgetDisplayMode.embedded {
  view.isHidden = true
} else {
  view.isHidden = false
}

If the currentView is the imageVRView and the currentDisplayMode isn’t embedded, this means you’re viewing the image view in fullscreen or VR mode. In that case, hide the view controller’s root view; otherwise, unhide it.

Build and run; click through the images again, and the transition between images should be almost seamless:

Clicking through images in fullscreen

Clicking through images in fullscreen

Note: Even after setting view.isHidden to true, the GVRPanoramaView stays unhidden because it and the GVRVideoView are not subviews of the VacationViewController‘s view when in fullscreen or VR mode.

Congrats on successfully creating a 360 image slideshow!

congrats!

But don’t get too lost in your vacation yet. The interactions are only halfway done!

What? It gets even better??

What? It gets even better??

Video View Interaction

You still have to implement the video view interaction which will let you pause and play the “Living with Elephants” video by Photos of Africa so that each moment of your vacation can last even longer.

Back in the main VacationViewController class, add the following new variable underneath the var currentDisplayMode declaration:

var isPaused = true

There is no property inherent to the GVRVideoView that indicates whether the video is paused or playing, so the variable you’ve just created can serve as a state marker.

Also create a method to set the initial play state behavior dependent on the display mode. Underneath viewDidLoad() add:

func refreshVideoPlayStatus() {
  // 1
  if currentView == videoVRView && currentDisplayMode != GVRWidgetDisplayMode.embedded {
    videoVRView?.resume()
    isPaused = false
  } 
  // 2
  else {
    videoVRView?.pause()
    isPaused = true
  }
}
  1. If videoVRView is in fullscreen or VR mode, calling this method plays the video and appropriately sets isPaused to false.
  2. Otherwise, if the videoVRView‘s display mode is in its embedded state or the imageVRView is in fullscreen, calling this method pauses the video and appropriately sets isPaused to true.

Then within widgetView(_:didLoadContent:)‘s else if block, add:

refreshVideoPlayStatus()

This pauses the video as soon as it loads.

Now in widgetView(_:didChange:), just before the if block and after setting currentDisplayMode, insert the following:

refreshVideoPlayStatus()

By calling refreshVideoPlayStatus() whenever the display mode changes, the video will play when entering fullscreen or VR mode, and pause otherwise.

Now that you’ve set videoVRView‘s initial display mode behaviors, you can implement a play/pause toggle triggered by user interaction. In widgetViewDidTap(_:), after the if block, add:

else {
  if isPaused {
    videoVRView?.resume()
  } else {
    videoVRView?.pause()
  }
  isPaused = !isPaused
}

You hit this else clause when you tap a videoVRView. If the video is paused, resume playing; if the video is playing, pause it. Then you update isPaused to reflect the new play state.

Build and run your app, open the video into fullscreen or VR mode, then tap the screen or widget button and the video should toggle between play and pause states:

Play/pause while watching the video by Photos of Africa - Google Cardboard VR

Play/pause while watching the video by Photos of Africa

Note: There is currently a minor bug in the SDK that causes the video to unpause when the VR mode is selected, but not actually entered. In other words, if you back out of the VR view without actually tilting the device to start playback, the video will unpause.

You don’t want your vacation to stop after a few minutes though, do you? Of course not! So therefore you’ll loop the video to the beginning once it reaches the end.

Add a GVRVideoViewDelegate extension below the final bracket of the GVRWidgetViewDelegate extension to implement the looping behavior:

extension VacationViewController: GVRVideoViewDelegate {
  func videoView(_ videoView: GVRVideoView!, didUpdatePosition position: TimeInterval) {
    if position >= videoView.duration() {
      videoView.seekTo(0)
      videoView.resume()
    }
  }
}

videoView(_:didUpdatePosition:) is called at approximately one-second intervals as the video plays. Once the NSTimeInterval position of the video is greater than or equal to the duration of the video, the video has reached its end. videoView.seekTo(0) then sets the position back to the beginning before resuming the video.

Note: There is no need to set VacationViewController as the GVRVideoViewDelegate because…it’s already set! GVRVideoViewDelegate inherits from GVRWidgetViewDelegate, which you’ve already adopted and set up in viewDidLoad() of VacationViewController.

There you have it! Now you can build, run, sit back and enjoy the worldwide vacation you’ve just created with your bare hands. Isn’t iOS development spectacular?

kick back with cardboard - Google Cardboard VR

Where to Go From Here?

Download the finished project here, and check out Google’s sample projects here.

The Google Cardboard mobile SDK had originally only been available for Android devices until Google graciously opened up its Cardboard SDK to iOS earlier this year. Google VR for iOS is still very new and will evolve over time, so keep an eye on raywenderlich.com for Google Cardboard VR tutorials using the enthralling new features Google will undoubtedly roll out in the near future.

Have comments or questions? Please join the forum discussion below!

Lyndsey Scott

Actress, Model, App Developer -- www.LyndseyScott.com

Other Items of Interest

Save time.
Learn more with our video courses.

raywenderlich.com Weekly

Sign up to receive the latest tutorials from raywenderlich.com each week, and receive a free epic-length tutorial as a bonus!

Advertise with Us!

PragmaConf 2016 Come check out Alt U

Our Books

Our Team

Video Team

... 20 total!

iOS Team

... 74 total!

Android Team

... 30 total!

Unity Team

... 12 total!

Articles Team

... 14 total!

Resident Authors Team

... 25 total!

Podcast Team

... 7 total!

Recruitment Team

... 9 total!