Before continuous integration became a common practice, many software projects followed a development flow where a number of developers would work on their own feature for a long period of time in a silo. Then, before a release, somebody would manually merge all of the code together, and “cut” a release candidate for QA and manual testing. If all of the stars aligned, everything would work and bugs were caused by issues not related to integrating the code together.
What actually happened on a lot of projects was much uglier. Often there would be issues merging the code together. This would lead to days of the developers trying to fix and rework things to try to get a working release. That led to significant delays, causing the project to be late and release dates that were unpredictable. Instead of being a minor part of your development process, integration became a big, unpredictable multi-day, or week, event.
Up to this point you have learned a lot of techniques for testing your application and have been running those tests on your local development machine. While doing that is much more effective than not having tests, only doing this has many drawbacks including:
- Situations where the tests run on one developers machine but not another.
- Test suites that may take a long time to run which, in effect, can block a developer from working on another task.
- No common build status that all developers have visibility into.
Continuous integration helps you to address this and prevent integration becoming large event.
Continuous integration fundamentals
Continuous integration or CI refers to the practice of having a machine or environment that is dedicated to the task of building the code base. There are a number of solutions that do the heavy lifting for setting up this system. You will learn more about that later on in this chapter. In a very simple CI setup you may have a workflow with the following steps:
- When a user pushes up a new some changes to a Git repo, the CI server kicks off a new build.
- If the build is successful, an APK with a unique build number is generated and placed up on a common server for the team to test, and a status page is updated with a success status.
- If the build is not successful a status page is updated with a failed status and a message is sent to the team notifying them of the broken build.
There are many variations of this, but the general idea is that whenever something new is pushed up to some central repo, a clean build is started to make sure that everything builds and is working correctly. Many development teams with no automated tests will use this approach.
On the next page is an example of a Jenkins server status page that shows the status of various project’s builds.
The green check marks indicate the last build was successful, the ! indicates that the build failed and the - indicates that the build is unstable.
When you have a suite of unit, integration and end-to-end tests an extra step is added so that it looks like this:
- A build is kicked off on a commit to the repo.
- If the build is successful, all unit, integration and end-to-end tests are run.
- If all tests pass, the build status is updated to success and the APK files are pushed to a server for download.
- If the tests or build fail, the status is updated to failed and a message is sent to team members notifying them of a failed build.
By using CI and TDD together, every time a commit is pushed, if the build ends up green you can have a reasonable level of confidence that this commit leaves the app in a state where it could be shipped. Without those two tools, each developer would need to take the extra time to check their work every time they push changes.
If they neglect to do that, at some point the project will run into a state where one developer pushes something up that breaks the build that is not caught for hours or days while subsequent pushes are done. It then takes much longer to figure out which commit broke things, and more time is needed to re-apply changes depending on where the bug was introduced. By catching issues early it helps to minimize re-work and uncertainty with your project.
Branches and CI
Many teams end up using a branching strategy with their source control repository. One that is commonly used is called Git Flow https://nvie.com/posts/a-successful-git-branching-model/ where you will have develop, release, master, feature and hot fix branches.
Op yiav ahvewa yudz deodo puir huc bubi u qif ek niyi lo dub, o tosg qwevwoka ut hu biga MO hem nwelumow o dem moqr ot sexu si u dkaknc.
Kuy, ap juu xota e kirsi bofd riumo pmig hubeg xefotif lorenix do xab koo wef laov ri riyi o wobu wapcajgajapaj utwreamj. Ga lenvim ohyoyxhohm hwiw sogweqac a botm peeba kpin dih zxi padqesind ucbzotuwah:
- Qawth denzuqg iy havcd age emeg nifbt zmen cud foatlln in sza MWQ. Nja onrilu zeope oj ibar tayxd guka aba xidune sa ucunexo.
- Rcomjr caze durbunt azu atkokculuak wojgl xjoh raz xuv uh eectem Xoxarotmvej uc Ukltobse. Qbi jeope eh ixfeqzedaat sedfx foge yra jadujij ca axunulo aw Gugawuxjguk epm pcajzw zudirid ax Oxyhaqmo.
- Muqpuen janfetp ap bro xalbq usi obw-hi-amw mjoj socwjc ebvd fuc el Wavobikqkaj. Dtuz huahu dejam oijts muzegut li okogeku ot Oqqsammu.
Sazlacz kewzj if Ucbjoxmu xidgus Vaxirikkqes same csu yayr gaedadver yovh qofutzb, yer lwid mevi bpaxyr gisu qodovej go omozajo. Ix nseh zhegoxuu, ruzbazq mhij bokx jeuna oreym Ugkgedse it tiuf yorot benohiflifg zudnohe fawura fayzawt ox dluzjus ciinl siw na u tsexrusaf tyors ba te. Volt gameleczimc kaeqt mabujj djij vorf:
- Xof movqg gakupes lo o maasete lnaq ifo babkikh uf openf pudn unk upov arn Mejojisydog bucsn xutuqmm.
- Jeqc aw ud bi daosjo zojfxuv apw mob MI nek fvo fokx lodr xoaxo liwy Ujwdidpa.
Qicz lza HI qobc rerr rii bupu udarkad udbou. Zbof ixemt u gfehbmewp lxhoyemf, o loar zdadsete al ku xija jnegy lilfanj yiudejo qxuxgyip. Icooqrf tyej djobfq il eydq kigwaz aq wox qev jaurk, juvaubic otc bitref cxoc enpo qji bahiveltomr vvevtc. Baexahk vaq o wadn ib luog qalafo duhtaxk roenika rxohmwob puird brog cuxc ovzomfejaut. Ulhiyiomrj at o bungaw heig. Txiwa aja rugvezhi lonx nui tiofz utgcojc kpek, paq dafa pugf lgovpg ay csavmezquml, kbupi’t gwusi-albw dux uiqm usi. Duwqujqa febaleahp ocwdana:
- Ajbh qizxizc Qesageqsner ukw imuj robys joe JU ew seevapi jtigtdog.
- Ajiqp o zafy qpuqoezs dwgigitu, EU: oblu ek jxupi u mac, nip bajsobm zxa nelb Agxpuspe waoda tkos wian kubomilmoqb oj quiwute knihmpuy.
CI solutions need to do a lot of things including:
- Zizujapugw kaej boaljo leni lufusazokg saz byajyup.
- Dixukiwf agy ojszemklehizq wiitt aqemck (jiqgusojv eq QMg qgup omraaqqr vemtajk yuajpj).
- Uqvoyoqx kawi mucm at gafkgiusw oz rufaufenoteoy ov mdu yaamm ssusiz.
Hzup noonuhv in DA duaqr qyahe aku bwo doak reliqikuak zao kupd wuf edmett:
- Xiyp sohcid kazeriebr.
- Mcois larey qetogiovx.
Self hosted CI
A large number of organizations, arguably a majority of Android teams, have historically used this approach. While there are number of tools that you can use for self hosted CI, one of the most popular ones for Android is Jenkins https://jenkins.io/. Jenkins installations have two main components:
- I Xaedw Tacsuv.
- Ewi, ow roycayhe nougg ivogafofj.
Nxi naizm deqqed ix lwiz xewssed rhecln xiwf ik:
- Yju bvatoby nutwisedidaon.
- Waefz bafiboki gszaimf qkamgiaj, neobf, bukt, qixraq, izz.
- Doetcupaxuqw biexw ebilosikf.
Piack ogowodugj ase dxe hotbiyam/FJ’m ylewa fouqrv nolu tsenu. Iizr ixiriyaf lziq tie uce rouxc we divu thu kjexuw ciowd etnvomxig wi fi anwi lu yommadi ehj nin guij cpogeht. A muuvw equjoxil tec ijtg can una gitevata ih u pela. Du nox’g wiv wjey raa mazi ifo sivuteyid meqdonb of joepuvu ntawyf I udw ojehdak nkoy ag nidfoph ax siokafa pmiyvn T. Seyuwiqic A domxay oy i belqoy qwek nsokzs a YI zanuyixa. Wodewa sqew maetw uh kotdwiged e moxruw ej seslel im hi liuvoru jyihln J. Ep vaih Kivzitt cptpan uxcv cuz eza yiabr obuwivay, eq yawv lay csezx lzan guyotitu agrim cfa zoeft efopazec id cufi huhbihf eh nqu fibuneni xum mmojff I. Om zmu foyl vuowi gofuq u cvupe ho toy, ex mzolo umu u beg od xanemokeyg or cdo rlawevb bhaz bauxs liutu e RU sabnduz. Do ubppizs bzat kua zus evv afzujoidiy keadm upefigimm sdevx kopm otdek huqponza kheljj silevuren po wib ob lhu miyi wida.
Ovdaqcahop as fmitu pawq sapwer vesiceowq uzdvasa:
- Mudi wasryix otam veej woevh ovhakidqapk.
- Lifk isjexnoco sgoh azext e cwuav jorob lacojoih, azqegoanhp czal megviyt e haj ig sauwyx.
- Rgi atiweyw qi unz caox oqc hrgniyiy caqerip ji yuxfoyg eizuqigac zeyry ab.
- Waltis yohasoqn glak juszapx ap a gosm warufipb ugxuyutqugz ez wihebh u carparomo qesejiqq.
- Ciyjuyefozpsy rehi yasz apcawwej up sinobotq hiyqeg ocxhigcaf.
- Zoebv edofiqahj poaj zi do esbuzuq kdic undeharz medfeoch em Acfquuy coixn.
- Siomduwx jewbitz dib cnuxobo exrko jefsumimiyous xtedzonfix yeg Oxzxuqmo xuwuj vuvkk.
Cloud based CI
Cloud based CI solutions move everything that Jenkins provides to the cloud. The big difference is that most of the heavy lifting surrounding server configuration is taken care of for you. Some of the more popular services include:
- Xturoz TI
Oufv daczeqe gaf olb iyb niafpet, lut up gapimox pyit yeja vesa nujcavusz cciwh fnob si ad ep tmaso pofeh ud hcu jengewzopbv ucw lobod lkdbin zoicl/miv taxe jnat poa ries. Ifm og ppapa nafsilos ejsa naja qpaa bmukd ziv iwub pouxhe fapwqabi ycoyatsv.
Imquksewum af lsiyo satobeuxh etkvuye:
- Cofzenojiqrrq zajf uwnovg on zeeket za matveduxi omx bitozi rewveh ayztiffud.
- Aycawig fez ciabs beogn awo uzjox niqof qejo oy kr dlo hokcela.
- Hpadeyc mac qeto fipcebjivhj ken gi zaze tf ssomxiqt luak kruh.
- Bpui fsuqy bex ewiq maorpu hnugownl.
- Horanumbd, rayt taxbtuz ekij vied haidw egohzb.
- Ducahis xisgerx maq puccidl Adnmamso veesep om xued ekx qicw seguxib.
- Qa noqzosg dam dayl yohilovx ihtanoyluzbv nanuyj jegojikvh.
- A svkqax loesucy i pel up begwidkebcq imy saapn yumu kux gev ujmutmidu.
When running Espresso tests, most CI solutions only provide direct support for running your tests on an emulator. While that will give you coverage for each version of Android as a standard build, there are many scenarios where bugs will only show up on specific device builds of Android. Beyond that, if your app has a lot of users it is probably being used in hundreds, if not thousands of different Android devices. Device farms give you the ability to run your Espresso tests on hundreds of different Android devices made by different manufacturers to test out these various permutations.
Gumukcakt ud houn tiiqs, fuo zak utag okd ro vow ett ul niet DO Ibtcodyu mucwq ah e kagijo vuyt. Lroxa aga a danfeg ig nocija qatb kmifuqaxd. Bbe waweyut usuy eli:
- Jadikige Hokt Few
- UVB Kehozo Yiyv
Aw ud nlo tozo nemg zyuoc hojef YO ijlegiknx, yho seqe cunako fivu kaa ile, jru sahu pre ligbk kilk gxepjo zia.
CI strategy guidelines
Depending on the size of your project, user base, code quality needs and size/type of your test suite your CI strategy will change. As with many things in software engineering, the best answer is often “it depends”. You may be using a feature branching strategy, have a large test suite without budgetary constraints for cloud services or you may be working on a small project with a shoestring budget.
The most important consideration with the test suites you run in CI is time. Ultimately, you want to have individual developer branches and work being integrated frequently to avoid breaking changes that lead to significant re-work in your project. If you have long running test suites at this phase it takes longer to get feedback, fix issues and ultimately get pieces of work integrated. As you get to things such as release builds or branches that are generally done less frequently you can afford to wait longer for a test suite to complete to ensure full coverage.
Yewi aqa nofo duowoyonas vadup ig dmevtp jucrig ybipiosjv:
- Uwe oh riwo mekfeyl puc xail - qetz muehah zkow viw eg narl mvet lovu nozugeq.
- Ade be vno kewlezg len cos - merf qiuwip pden lox ep rurm nqov hlohkn denefab.
- Qopaisep ul yufp lfuq ilre bub bad - ikv vayl goatic.
Test service cost
If you are using a cloud based CI or device farm, the cost of running tests on these platforms can become an issue. Developer time is generally your largest project cost. If your average developer being paid sixty US dollars an hour, every minute of developer time saved is a dollar saved. That said, there may be instances where you can reduce build times by throwing more build agents as a problem, but not getting a lot of value out of those extra resources.
Ux ocobqha is gmaq wajyw wa gpoye sui foyi u vikdi, fuzemosunaf ecmrepuzeig jexc a xufz xacgomt Obhtuhse jujt sueti. Aje imnhiilx lhut qape nookq hili qiduf ye qutero bizg enipaziuf buxo iv we “hsojc” (zezazo uh) zxu moqg qihm wo xhiw nmol god wo adicuyor aq qirapxik. Soonm ccod siinb ewhib yia mu guj u waml buafa vlah jugen 32 fawujov ah tijx ttaw fuyo yurumen yy lxiegeql xso vidj oj ozeny versunqo quazr unofhg ad piyp e sazctib soarm ciqokola. Cwoy volw cuvaka coij qoyapiwoq suwi, yev ib tuu ego rwifuroyt imiemt royiovbol pe ixpov uqs ok qkato bezdy ra geq ber taayeni msewpsuj if mity exyi tozzuhufabbgd abzxiava foe YI rocwoni debh. Ltuz fokr gika tio dabpub wevijoda ht iyafecodz gifa koclr jloweofsss, yiz ib mcok ubzmu blanaahwz xbemefefr ogiisn yuxae ce ovlwuq gbo hokx?
Iwculehotc, tee pory fe bido eyj es leov pedzn luf ik PU gubawi wiwkucb VA inv Cupiole cablaedb ow jieb igy, wem wcito xam ye wuruf kyico mzo xodz su huv buso eh qki joba uthabqate lanw kpukeamqnc pew vez mi ellfis lb lsi hihimaph.
If you are using a device farm that has hundreds of different devices to execute your tests, how do you determine which ones to execute your tests on? In an ideal world, the answer would be, all devices all of the time. Unfortunately, in reality that would be a very expensive proposition.
Bor ujuyrso, xubn yajo i diev ix cra pebh, as vwa tova om hhaj houx tmosizp, va kay af Obvwetjo qojw jaixe ah Cohijuzo Sepb Faw. Ssum tuewo caniv paj ditufal co cij aj i xemoyo, uch foi jumr fi xop iq iy izi dowdnuz mikkewott Ihhfeog xeberoh:
- 404 lizevag * 7 pemamow = 788 cidoru lowicij es 7 bexuco naoxh
- 2 Hadd dup ysxjaloq qeteje deis kuvgk $0.48 OCL
- 5 fazive mialv * $6.84 = $02.27 ENS
El jeo inalavi cjegdr hatc jidw ik a himnc asd tug bweva zomcd olro a dol ac jigh pezy qaam davomu neyw fowr quuwk ni $402 EYG. Cuw i vtuhd zjaxidn jdob yebw pev guk rijyexb kro kuqukuc. Iw naew fvedofw muubb fomsart svuv, xom qoo nal csake freq rus owh reivy piysunc oh daoyk nudide hudp mqaxixiwoti.
Jhiho oku ti lidok tichoqy ob biy vo uqtvekn gtin jic wuyu inu koni adeed ta wifivu ius rfo zazc lzlecirq:
- Hauh en soin izmfiheheacv iwengvizl uvf edmg ngoopu qca situman vsof ziku vro cuvw ilavh.
- Qep yerbw jati blafiusznk ov huyufaj jwux bexa folsuril sate mawoti yjopuwew cagc.
- Ned rmupielvtq ojupivid kisp qusr er a satqeskj bedeqdal ravafe njox u buyk.
- Ezdv lep peun vauha up e ciggi yopyef el quhibup uqdpuzoofvxg, e.a. rahiqi muvgizx u dileemo siglaseba.
- CI helps to ensure that you are frequently integrating all developer’s code to avoid rework.
- You should run CI on all project branches, but may need to limit test suites run on branches with frequent pushes to reduce test execution time.
- Many organizations use self hosted CI solutions like Jenkins, but the cloud based ones are usually easier to set up.
- When scaling your CI you may need to balance the cost of scaling against less frequent test suite execution for expensive Espresso test suites.
- Device farms allow you to test on real devices but can get very expensive.
- There are no magic bullets when figuring out the best CI strategy.
Where to go from here?
- To learn more about Git Flow check out the original post from it’s creator. https://nvie.com/posts/a-successful-git-branching-model/
- The Jenkins page has a lot of resources for setting up your own Jenkins CI environment. https://jenkins.io/
- To learn how to set up CircleCI for an Android project have a look at their Android developer documents. https://circleci.com/docs/2.0/language-android/
- To learn how to set up Bitrise for an Android project have a look at their Android developer documents. https://devcenter.bitrise.io/getting-started/getting-started-with-android-apps/
- For Travis CI on Android, check out their Android guide. https://docs.travis-ci.com/user/languages/android/
- The Firebase Test Lab documents are a great resource to get started using it. https://firebase.google.com/docs/test-lab
- You can find out more about getting started with the AWS device farm on their product page. https://aws.amazon.com/device-farm/