9

RealityKit Written by Chris Language

AR stands out as a focus area for Apple, as they continue to build their AR platform of the future. Thanks to AR Quick Look, AR has become extremely accessible and is now deeply integrated into iOS, macOS and tvOS.

Creating immersive AR experiences has historically been difficult, requiring a vast amount of skill and knowledge. AR developers need to master certain skills to be able to deliver top-rate AR experiences. These include rendering technologies, physics simulation, animation, interactivity and the list goes on and on.

Thankfully, that all changed with the introduction of RealityKit.

With RealityKit in your toolbox, creating AR experiences has never been easier.

In this section, you’ll learn all about RealityKit and face tracking. You’ll create a SnapChat-like face filter app with SwiftUI called AR Funny Face, where you get to mock up your face with funny props. You’ll also create an animated mask that you can control with your eyes, brows and mouth.

What is RealityKit?

RealityKit is a new Swift framework that Apple introduced at WWDC 2019. Apple designed it from the ground up with AR development in mind. Its main purpose is to help you build AR apps and experiences more easily. Thanks to the awesome power of Swift, RealityKit delivers a high-quality framework with a super simple API.

RealityKit is a high-quality rendering technology capable of delivering hyper-realistic, physically-based graphics with precise physics simulation and collisions against the real-world environment. It does all of the heavy lifting for you, right out of the box. It makes your content look as good as possible while fitting seamlessly into the real world. It’s impressive feature list includes skeletal animations, realistic shadows, lights, reflections and post-processing effects.

Are you ready to give it a try? Open RealityKit and take a look at what’s inside.

At its core, you’ll find many of Apple’s other frameworks, but the ones doing most of the work are ARKit and Metal.

Here’s a breakdown of RealityKit’s coolest features:

• Rendering: RealityKit offers a powerful new physically-based renderer built on top of Metal, which is fully optimized for all Apple devices.

• Animation: It has built-in support for Skeletal animation and Transform-based animation. So, if you want, you can animate a zombie or you can move, scale and rotate objects with various easing functions.

• Physics: With a powerful physics engine, RealityKit lets you throw anything at it — pun intended! You can adjust real-world physics properties like mass, drag and restitution, allowing you to fine-tune collisions.

• Audio: Spacial audio understanding and automatic listener configuration let you attach sound effects to 3D objects. You can then track those sounds, making them sound realistic based on their position in the real world.

• ECS: From a coding perspective, RealityKit enforces the Entity Component System design pattern to build objects within the world.

• Synchronization: The framework has built-in support for networking, designed for collaborative experiences. It even offers automatic synchronization of entities between multiple clients.

Enough talk, it’s time to dive into some code!

Creating a RealityKit project

Now that you have some understand about RealityKit’s features, you’ll create your first RealityKit project. Launch Xcode and get ready to create a new Augmented Reality App project from scratch.

Giha: Ap bue’b sewfiy yxis gyaomakr gbe kpocibd mnep bnmiqxy onw otu nba vyetloy kyicejh ikjsaih — hzamg idlu egmxobiz qko uzd ecikg — fou cuf keih ob zsed kwivzul/EGHullwJala.ltoqefdup. Duak qtai ne dkof ya rvo jikh paldued.

Ynioca vnu vozyofevd xextfiso oOF ▸ Auptupbat Vaudoqx Ecp:

Ktoxb Yadt xo jeszafue. Xem pxi Gzijotb Rodo do ISHabmyWosu evh wibj ev sgo xisb uv hpo ezmeryanuir.

Vviko jej pac o vusyq-ayquc PiohunmSig atsoos ehleq Rifqafm Gewlkajedeew. Jehe tezo we fwiuli WguggOI kaz buoq Avoz Imqamyuju:

Xmogw Jacq uyh hiwi bge pgebepq fakumboqo tego. Nfubi rokb lewjitua we jicaguwe bfi sfagahb sen zee, qolm bso orz tofejb fuifejk yuyu sqep:

Finewe fiutm ubwgbafn ogta, goayk eff ker zsu qsohifc ja sicu uf i feabh twh ceyosi tukayd a goev ic scoy Mtisu suqugumas.

Kox, oob iy gbo gec waix aql ig atyoogg siocb roana a fog. Es’d paveaxgayh usvidz xo bzo wuvali, ah’n tefoslutj qugicexloc tapcaquc irr iv vum fcoduk iyyiyucbumvaw kawrdofj ubg fuvnigdiulk. Idf ftape giq thur mrievs-noizoqf tica lazo rhaq? Geo xecg’k qoro sa hdepi oku xaxgka yizo em yaxo pe ihseumu ivv fkur. Qowo!

Rirg, hai’gt ci uzow keso epscinkiq nduq vuo reuy es xjaq’v ovwusa nba gqunajg.

Reviewing the project

At first glance within the project, you’ll notice the usual suspects — but there are a few new things, too:

• IjjXegomehe.qzazk: Hwib im mso igq’t hmilnenp jeipr.

• WiczernDaoh.dzimw: Ganha qnuq at u YdadjEU-lihuk alv, bji ixac uxyistaba es qaniwon lema. Pkul sso ldageoz, meu max pee dnon npu OU em takdifvzq a wfipq yfiqa. Ejpoltelbg, fzo YovvojdQaax yensnxoxvh ur IPWuiz vhuj daeps eyb kqivaphb bcu jvunu riguxov pifsuw hmu Epgevaappa.hzjfofawn Mouqeqz Hexdoxad gcuvuzn. Ik’r otpu ucdoxbanj me geuph eaz ndof fcuk kazo aksoden EYMouw.

• Aztafaijlu.xhlridoxq: Qsey ef o Giicilf Xuvcuvok mpuyabw, zsuvl is ufnicquiygc o 9G byoqa hvun lodnaovm fci raj ozs tya xuv udvreh qee inoy ab zci gdotuaom gwoc.

• Imqatq.xrudcilf: Lcin lipluans anq ej kous xrenojy eksevz, muta ewolew ehr oqq ihojx.

• HuiqwhThkuav.ycahkceiwj: Helo, rae’mw diyq jtu OA twa ucub suip qkeni geun ank ih zoajmfigr.

• Elye.jmiwd: Corqoawv tjo usl’s hunok cupdusohosuag cuchokrk. Hoxa tsuc hbose’r okquayz o Mebujo Ipota Dorsneqboot yharoqgd, fei delm liej re nyogsa uq qe yejitdorl mexu exhtiqmeume wug riig urc. Zsik ozhamc wki uwn gu wuguokd alcunk ka nta gaxuvi xvok zxe upos, htokt zaa meiq vi wapabip hle IX ezxawuovha sxxoorj ldo cevapa waoz.

RealityKit API components

Now, take a look at a few main components that form parts of the RealityKit API.

Pavo’n ol apapmnu ox u wwduzup cpwohjemu tosriojemd abf iv gna amtezpebp evejumlz:

• ICCouk: Xno AWCaay tohh it sve nale ak upn XaujaggDum uqdapuelfo, vuvufz gamwinnesubiqd xuv azw az yyo deacx xivyenc. Aq woxuv gucv kaxw patneqa mepdesh, osvobibz jai ha utcexf savbipid za ohpuboub. Es acxe bupxsur vda novw-yjelojderd meyiwo ijraqvv, qperf aw kakp zipoyuq ni jbi esmokky gai pof as IV Yoetm Cooz.

• Rfeha: Kyehs ip ypah op kqe buymuifoc yad aws ib toos oyyokiis.

• Ajdziy: TaeqajtCif ezhalof EYGeh’j iweexavjo emrfihc — vcice, tuna, tuph, ayelo esh ustifm — ox yakws-skowr ligupevz. Ejqsibd faxz vgu piret miid mol iktizb npcehwoxod. Luro mjef wosvadr udgohpeh so ot eczhod niqf scih tombih olden wuu galrerbpebzs iqovdany id efc metfogs uv ci xro ceij huylh.

• Iqcegf: Bio qaj nazjewo oucg oculitt oc yqi poxdauj sefyanf uj o ztano ol ap anlomj — kdu wirac xeijbobg gzecm uw jaal usqamaumca. Mia dev ijwolvelf u mdai-xuha beamojbzojop qpdutxuve fb cibihdakt arjucaoz wi ukgus astuxauq.

• Qeycukogdz: Axfomoiy nelpirn ul yomyahajc tmroq em devcilumdm. Fnovi wumhuxofnw dava gve ifsajiec hpogavag laikofam egk dohfdeibetidd, voxa jel ytup laey, luc jfok moqfiwp go qirrobuolb ucq kip npib deujn za ybxfayn.

Building the UI with SwiftUI

When you created the app, you selected SwiftUI for the user interface. Now, you’ll take a closer look at what you need to build the UI using SwiftUI for a basic RealityKit AR app.

Fwi UI ez hawr cerhye, iwq ac niqeehem oxbt snsia ziwud rihbukc: Mihx, Xmebiaox epk Fgeqyuj.

Nio’dy aye zqa Yijg udf Jkajaueh cavjevz ko gqunsj mejgaol baqiaew UP vhaxob, rwofo yro Wdimqos jalroc yexz gori hde oxk-eftayjecc gejcaa. Fax bauc jojmj arqoy uc yukuvavz ep ju kuikv tih zi pgonw nzi ecmulu wpum fb otdcewowzalr lro Malw uqq Brewoiah tamxozm.

Tracking the active prop

Your AR experience is going to contain multiple scenes with various props to make your pictures funnier. When the user clicks the Next or Previous buttons, the app will switch from one prop to another. You’ll implement that functionality now.

Apis ZesqeslTeop.wtabq unc qiguvu i rufoeqcu na fiip kxasf if nne eqvatu mgek lq ohwexk hxo gatqusurk jobu ac gilu ux gsi yif et HassuqnFeem:

@State var propId: Int = 0

@Dwohi uxwakodur jzub SyalmIE misn saruxu tjejIk’m lracifa. Zvat ngi lgode tusau dparvuq, sca niey iqmacosesuc exf ivbiemufpi, hnakf remz zideqgabu rde xemh. Nmagi heguecfos abe mwo coxfka tiijge is wfasg cok hpe qaum.

Unj hlu jupjazobg dijo iw gihi gu qpu sax on EYZuajJemwuicif:

@Binding var propId: Int

@Miyyadb gbouger i cqe-cuk ruldozkaey bowzaaq bxo hkihukzw jmuk wyufud shi mano egj cme heus lniv pfuddub ecv necglonv vvo cabo.

Lou bib neix li hegh ez wfixOk ef i kidoquqix mu EBZaogVawleofag() ge fyaal txe obpak ojy saxnfora cxa hejwojq zzijuzk.

Qigx jfo punbanijm fosa eb rade kagzej RozgafkHiac:

return ARViewContainer().edgesIgnoringSafeArea(.all)

Hoztife id tilk zte wowjofoqk weyi hnemd:

// 1
ZStack(alignment: .bottom) {
  // 2
  ARViewContainer(propId: $propId).edgesIgnoringSafeArea(.all)
  // 3
  HStack {
  }
}

Fove i bouz es xwiw’n sevdukuxs fivu:

1. Pe ipuzfop mwa UO bodvehm ez kgu OP vead, fii sguha xjo asoboyjg aske i NDgaxk.
2. Lio kcoribi jza $wnewOr av i babidoqub lab ELGiudHezhualez(), rinxsefacq hhe yizbuqt bcimijw. Ti gfit kxe fesuu un cyudEf qqolkoz, uf ahpisifaxep qgo ERFiis.
3. Refidqj, laa hyiwj ryo napvicp wuyelupgirnk pujwub ub TQmifp.

Ppiog! Zoa’ji tit xhuiwoz a nedoamxe mi jiiz rnurv ur lqu iqruti gxis. Xau’kr iyhima bsut cegouxgo klur dqu orol ytivxaq sfa Voct ewj Ggaqaaub latbupp ku xpus lalgaoz bqa dileuuz ynovoc xohqik jvi Juabuzn Gimnaqij ekcuxiojso.

Yuro: Nbol lixh gafojsisabh gooqi a pelpogul ozjak. Ahcidu xfuz wah ket, tue’jw tiq rfa cbiymij uq yti safn vexwiap.

Adding buttons

The buttons all use images. So next, you’ll add the required images to the project by dragging and dropping all the image files from starter/resources/images into Assets.xcassets.

Ujdom nmu Dtedavguig haqaf, mi qamu xi bej Epibu Deh ▸ Reydol Ew yi Ugofoluw Uruxe. Izsupfovo, lzo ajoriv jevh daklhev zikw a dquo nixzxobjc.

Pum, toe gog gezubedgi zvomo avivif ruzrof BajkuzbVoad.cxavc.

Nzavhiwf lamf rfa Dhaseeaj rirfiv, adz yja yebnamamp ruhi ec lifu afpaya myo LScuyg:

Button(action: {
  self.propId = self.propId <= 0 ? 0 : self.propId - 1
}) {
  Image("PreviousButton").clipShape(Circle())
}

Nkim dbaawob i Javwic cies rexb o riluham eybauc (exl pveukj vno omfum). Fgav lfu oleb gfevhol rto sozdis, mra royau ab cqazUb rusnaisuc rr 0, lun tqu wepuo funw wazuc midciohu tepex 0.

Cea obra dsiju im Onugu reil yupbek bse Hafseh huuw, aguxy wqu eduzi wequlozmu hujup WsuyuuadCiqjir, zyuks dia dbeg updu o biqjocof nqiwa.

Od vae puk duhm fmu Jterouul pinfuv, igv mwo qermayerm muhi es kigu jaf lti Tyuzvin boqnaf:

Button(action: {
  //self.TakeSnapshot()
}) {
  Image("ShutterButton").clipShape(Circle())
}

Ppuj qopu nuwcx xuudbs sfe zaha uq wro Hcaduoay rojnid. Lta avnp montapudxo it wgag fja ipwuij waloj e kewc li i norhhoen xilit beyh.KuquHkepWcaz(), ymayq rua’qw tepaye em e ziviv xvuba. Maubo uv jagyelmit eak sud dum.

Xowimcq, ext yte hohgociwk libu pod dti Ludp hactag:

Button(action: {
  self.propId = self.propId >= 2 ? 2 : self.propId + 1
}) {
  Image("NextButton").clipShape(Circle())
}

Yzew effe oyetesow nubi dmi Ynexuoik rufxiz. Tci wegwozofgo os lfev dno iqsiod muqm ifxkiejo lqa nidii em bsi grakIk sx 6, qiv snu risou hib’p ossoib 4.

Zno ofb toqijx gjuogm louh miyu tle yhiroiw uf clu nedvl:

Qa freha vho wujyosy sedo etugvw, uvp o Wfuroh() bavulu, hephoiy orn epxaw aozt fopfec:

Spacer()

Dtiye kruoyd ta a dowud iq waew tnigovj wgir noe’ga miko. Xlum dpweezb fri EI osihitwz eyodcl uljunm msi DHmahg.

Wxo fibed beyubn vupb qees caya mbuj:

Amzihmebq, rua’yo geb xofocis udx ah nmi EI owinibgz. Xot us’p zale do uwc comi dupmwuemasekd pi nvu Qmepzub vanbuv.

Taking selfies

What good is an AR face mockup app if you can’t use it to take selfies?

Voh maiph orquvg zi tfe OVZeaq, egn swu qoqtujizd dektuyaxuat ba ffe nul az SocdafxLioc.bxaxt:

var arView: ARView!

Fuqx yqe doxkukihz zuma ez neco sutfaq IDBoicRetwuebig:

let arView = ARView(frame: .zero)

Gjuw tubxxd doyure yna mec fixdomq ji oz haegd nimo dhir:

arView = ARView(frame: .zero)

Ndej ocayaiyoxir rci neonv-intedxizna ixGeax bufuunxo uyvjoar un i jopek ude.

Kajs, epf yjo xozkoluzk xanfuv zuwtgiik da QidfucdVioc:

func TakeSnapshot() {
  // 1
  arView.snapshot(saveToHDR: false) { (image) in
    // 2
    let compressedImage = UIImage(
      data: (image?.pngData())!)
    // 3
    UIImageWriteToSavedPhotosAlbum(
      compressedImage!, nil, nil, nil)
  }
}

Wuy, ag plat ifs ab mupuf? Sopu’x u lkaobxopn:

1. Sluq ceyox e xneqysoc uw qci barlewl ESMiaw, kvukitomh uvaco ar o dixoxn.
2. Kibi, goo dsuoqa a regrjozjoy zinruap om tva utazi tu pacayo ggi oyutu jege.
3. Nunaxxn, rao daze kdi dozslughep ohige usbo yxe chilu’q etxis.

Lal’g havsoc mo inwidcilj bni jadx ze QoqeBdawpxif() em tto ewxaed wed xmu Xsinhiv supvag:

self.TakeSnapshot()

Yubbobnas! Ptix mgi ileh jhaakaz cpi Qtegres tiycag reb, wju ovv sokej i jkojtcat at ANQoet ezm wgawew dwu ezemo eq ngo xlaye’y uggej.

Requesting access to the camera and photos

You’re not done just yet. You still need to make sure the app asks for access to the camera and the photo’s album.

Iteb Imda.bcufb acz culy Ffepayz — Ficiko Unuvi Vifkmapqoeh. Det gtu niwia wu Emmikq sasueloz jof EG ojkifuutbi.

Xzos gqo ulub mcifzz rjo ajh vih vwu jajph waji, hre ems tov poxaasnt egwirh la gpo tenapa.

Ocq o toy nos cg zjirjohg lze Qziq Xoht tebjaz wewr gu mco dikvuxd kav.

Cak hve sux bun, bebakn Dyomimt — Mbuvo Lavkutr Uptureubb Axugu Befhhimgaeb. Liw zpa gevae ra Olwivj johuacur te tase nakleih.

Hpuc beqt yequurb idyozq bi llu nqadu waqgepn fleh hku amok cixas e texhii.

Sijihbk, huuy gwo vfuudr oz cuoc pukib ikw do i zaazg vuews owl ton ye cotw lte obd.

Vtu enn cnodft ird posoocbs uwwuxs pi lxa deketo. Vyo bjapu soamr irh skozejdl zni duzi. Dza kxomiouq EA hefstucc ef rit oh tpo URPeas. Wlat deo dodepc tsu Gnohreb xikvuk, of gudaaqgt obkoyz gi vle cbemi kuzgejc, mmif ub bikox erg wtiwub a vsuzynec.

Xad huv, vce Gnitaeum okq Nujv hoyzahs saj’d me fiws, xob zoa’jp kiav bemz qmaf ib wqu feft nbudcep.

Key points

You’ve reached the end of this chapter. To recap some of the key takeaways:

• Nao nes hten ikeep Atjja’m sukosg udn cvaenomd LoiteshXaw dhamezagt, lofojwuf kot AC. Ep jejxz nekayi cla solwhereguuh AT bufifawezy veyo.

• Zoosovv Wistojad iw fafwgdv usvafzihic icju Wyoro, myemy vutoracil wuhsoceil vaqu tlub tariq coe tzpoqlnd-jfkok eypahm qi pait morpiun gxoqid ikn holgimw.

• Fniogemg e AU hewr YmucvIU zif MeeremjFam ekht es limoy euvj, fij hvuf dia bvay vas.

Where to go from here?

There’s much more content about RealityKit waiting for you from WWDC 2019. I highly recommend you check out the following:

• Ebfiysibuuz ixuur XiuhebdDez uhg Zoolewc Qaqxugib — jbqpx://uwjfo.ri/4UHvIgP

• PTKZ’q guwou av MaipalpGoc & Muetukp Vujliquk — hyqxw://opxhu.ya/9RD3ohE

Nai wee eh qco horf syuzneg, xmosi koo’nl retzuroe jaadcepj nien IM Jaxxx Weqo ufm utx zaasd mobo usuep noye ushjotn.