Build an API with Kotlin on Google Cloud Platform

Google Cloud Platform has made building Kotlin-based server applications more accessible than ever. If you’ve always done mobile development, creating a server application in Kotlin for your Android app might look intimidating. Fear not, thanks to Google and Google Cloud Platform, you’ll be creating APIs in no time!

Picture this situation: the boss gives you an Android application for a conference. They also give you a Google Sheets document with the list of talks that you need to show, and ask you to create an API for the mobile app.

Since you’re a willing coder, you’ll of course take on this project with vim and vigor. As you build out this theoretical API, you’ll learn how to:

• Configure Google Cloud Console and AppEngine
• Create a server-side application, using Ktor and Kotlin
• Access APIs for Google services, like Spreadsheets
• Use data persistency using DataStore
• Deploy to a scalable service

Getting Started

Download the starter project from the link at the top or bottom of this tutorial. In there, you can find a module with an Android app, a common module for shared code, and a server module for the API. In this tutorial, you’ll be working on the server module.

Before moving into the code part, you need to set up Google Cloud and create a new project. If this is your first time working with it, make sure you follow these steps in detail. It can be a long process, so grab a drink and get comfortable. Fortunately, you only have to do this once! :]

Setting up Google Cloud

To manage Google Cloud, you need to have an internet browser and be signed in with a Google account. If you don’t have a Google account already, make sure to create one before continuing.

Creating a Google Cloud Project

First, you need to create a new Google Cloud project. To do so navigate to the following URL:

• https://console.cloud.google.com/projectcreate

If it’s your first time using Google Cloud, you’ll have to accept their Terms of Service and select a country.

Once you’ve accepted the terms, you can name the project something meaningful. Name it “RayConf”:

Then click Create, which will forward you to the project’s dashboard. There you can see a summary of the resources used.

Creating an AppEngine Application

Now that you have a project, you need to create a new AppEngine Application. AppEngine is one of the many services that Google Cloud provides. It removes all the complications of creating a scalable service. You’ll be deploying the API there.

A project can have many Server Applications, but for this application, you only need one. Click the hamburger menu in the top-left corner, scroll down to the Compute section and click App Engine.

You will be taken to the AppEngine Dashboard view, with the below card already in place.

Select Create Application. Next, the setup process will ask you to select a region and zone. For this tutorial, you can use the defaults and select Create App.

Next, you need to select your language and environment. For the language, you have to select Java although you are going to use Kotlin, it will run on the Java Virtual Machine (JVM). You can leave the Environment as Standard and select Next.

You will see a confirmation message on the screen.

Installing Google Cloud

Now, follow the instructions to install by selecting the Download the SDK button. Once installed, you can verify that it works by running the following command in the terminal:

gcloud --version

You may need to restart the IDE for it to recognize the changes.

Logging into Google Cloud

Brilliant, now that you have the SDK installed, you need to log in. To do so, run the following command in the terminal:

gcloud auth login

The browser will open and ask you to authorize the SDK with your Google account. Make sure this is the same account under which you created the AppEngine Project.

Once logged in, the project ID will be set to the only project you have on your account.

gcloud projects list

Once you have the PROJECT_ID, you can switch to the project by running the following command:

gcloud config set project PROJECT_ID

Also, to make sure you have the latest version installed, run the following command:

gcloud components update

Enabling Google Sheets API

Now, you want to use the Google Sheets document your boss gave you to generate the API. Therefore, you have to permit AppEngine to access Sheets. To do so, you need first to enable the API here:

• https://console.developers.google.com/apis/api/sheets.googleapis.com/overview

Authorizing Service Account

Once the API is enabled, you need to create an account for the server to use. These accounts, called service accounts, are useful when there is no option for a user to log in, as is the case in the cloud. Go to the following URL to manage service accounts:

Note: Make sure the project is selected, if not then go ahead and select it.

• https://console.developers.google.com/iam-admin/serviceaccounts and click on Create service account.

  

  Give it a meaningful name and select Create.

  

  Choose Editor as the role so you can edit resources, such as persistent data. No spoilers, but it’ll be handy later.

  

  Select Continue. On the next screen, you have the option to grant extra permissions and to create a key. It’s safe to ignore the first part, but you need to create a key so the server can authenticate itself.

  Select Create key, make sure you have JSON ticked, and select the Create button.

  Doing so generates the key and downloads a JSON file. This key is important as it grants your app access to resources, so make sure you keep it safe. To install it in the app, rename it to credential.json and place it in the following folder:

  

  The sample project has a convenience object named AppCredentials. It loads the key file into memory when needed.

  I’m sure you will be glad to hear that this was the last thing you need to set up on Google Cloud.

  Now it’s time to get to the fun part, actual code! :]

  Creating your First Endpoint

  You are going to create an endpoint that returns the classic Hello World! message.

  Start by adding a new Kotlin file to the server module and name it RayConfApp.kt:

  

  Inside this file, create an extension function named main on Application class:

  import io.ktor.application.Application
  import io.ktor.application.call
  import io.ktor.response.respond
  import io.ktor.routing.get
  import io.ktor.routing.routing
  
  fun Application.main() {
      routing {                             // 1
          get("/") {                        // 2
              call.respond("Hello World!!") // 3
          }
      }
  }
  

  Note:You can learn more about extension functions here: Programming in Kotlin · Extension

  This function, once registered, is run when the server creates the Application. At which point, you will be:

  1. Defining a group of routes for the server
  2. Letting the server Application know that you want to handle a GET request on the root path
  3. Then, every time that request is made you respond to the call with a given response

  You can now run the app by executing the following command in the terminal:

  ./gradlew appengineRun
  

  After some time the server will be accessible at the following URL:

  • http://localhost:8080

  Open it in a browser to see the result.

  

  Note: To stop the server, use the keyboard shortcut Control + C.

  The main function is registered with the application in the application.conf file:

  

  Ktor provides a great DSL (Domain Specific Language) for defining clear and descriptive servers. You’ve named the function main, but you can use any name and define different ones. This way, it’s easy to structure an application into separate parts for easy management.

  Note: Always remember to update the application.conf when adding new ones.

  Defining a Server Endpoint for Talks

  It’s time to create something a bit more useful than a “Hello World!” example. The common module contains the model that the Android app is using. Since you are also writing the server application in Kotlin, you can use it in the server-side. Isn’t that efficient?

  Start by adding another get("talks") block, right below the existing get("/") block inside RayConfApp.kt file:

  fun Application.main() {
    routing {
      get("/") {
        ...
      }
      get("talks") {
        // Add response here
      }
    }
  }
  

  Next, you need a list of events to return. For now, you can add it on the same file:

  import com.raywenderlich.common.AgendaEntry
  ...
  private val talks = listOf(
      AgendaEntry(
          id = 0,
          title = "This is the first talk",
          date = "28/09/2019",
          startTime = "09:00:00",
          endTime = "10:00:00",
          description = "Description for something very interesting, we hope.",
          speaker = "TBC"
      )
  )
  

  Now, you need to return a String that represents the contents of this list. For this, you need a way to convert the model into a String. Kotlin provides a simple way to do such serialization with kotlinx.serialization.

  To set it up in the project, you will need to first add the plugin’s classpath. To do so navigate to build.gradle file at the root of the project. Then replace // Todo: Add kotlin-serialization plugin's classpath here with below:

  classpath "org.jetbrains.kotlin:kotlin-serialization:$kotlin_version"
  

  Next, navigate to build.gradle file for the common module and apply the kotlin-serialization plugin by replacing // Todo: Add kotlin-serialization plugin here with below:

  apply plugin: 'kotlinx-serialization'
  

  Lastly, add the kotlin-serialization dependency by replacing // Todo: Add kotlin-serialization dependency here with below:

  implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime:0.13.0"
  

  All done, sync the project with Gradle to download all dependencies and configure plugins.

  After adding the kotlinx serialization dependency to the project, adding the @Serializable annotation on AgendaEntry class is all that is needed:

import kotlinx.serialization.Serializable

@Serializable // <-- added
data class AgendaEntry(
    ...
)

Then, hide the fact that you’re using Kotlin serialization by adding parser functions inside the AgendaEntry.kt file:

import kotlinx.serialization.json.Json
import kotlinx.serialization.list

fun AgendaEntry.toJsonString(): String = 
  Json.stringify(AgendaEntry.serializer(), this)

fun List<AgendaEntry>.toJsonString(): String = 
  Json.stringify(AgendaEntry.serializer().list, this)

Hiding away your toJsonString() implementations like this is helpful for maintenance. It means that future changes to the serialization won't affect other parts of the code.

The first function uses the generated serializer, AgendaEntry.serializer(), to convert the entity into a string. The second one does the same but using the equivalent for lists: AgendaEntry.serializer().list. More information about this can be found in the official documentation here.

Now you can go back to the main function inside the RayConfApp.kt file. In there, add the code for the response, inside the get("talks") block, making sure all imports are resolved:

call.respondText(contentType = ContentType.Application.Json) {
  talks.toJsonString()
}

Here, you are telling Ktor:

1. To respond with some text
2. To set the content type to application/json
3. And, finally, the lambda at the end returns a string which is a representation of the list of talks

Time to see it in action. Run the server again using ./gradlew appengineRun. Once deployed locally, verify in your browser that the result from http://localhost:8080/talks has a list with a single entry:

Deploying to Google Cloud

Now that you have something non-trivial, you can take it to the next level. You can deploy it to an actual server in the cloud! :]

This is where all the hard work you did setting Google Cloud pays off. Run the below command in the terminal of the IDE.

./gradlew appengineDeploy

When it starts building, you should see something similar to this:

The output tells you where the SDK deploys the app. So, once it’s finished building, visit https://[your_appengine_project_name].appspot.com/talks and see that it shows the same list with one entry on your browser. Remember to replace the project name with yours.

Next, go to MainActivity.kt and replace [your_appengine_project_name] with the name that you got from the deployment. It doesn't change on each deployment, so this is the only time that you’ll have to change it.

Now the Android app is ready to start consuming the service. Run the app by selecting the app configuration and clicking the play button next to it:

The app runs and displays just one event:

Adding Dynamic Data with Google Drive

Now, it's time to load some data from Google Sheets.

Reading Data From a Spreadsheet

Create a new Kotlin class named SpreadSheetDataSource, right next to RayConfApp.kt. This class is going to be in charge of loading data from a Google Sheet.

In this file, add the below code:

class SpreadSheetDataSource {

  // 1
  private val transport = GoogleNetHttpTransport.newTrustedTransport()
  // 2
  private val jacksonFactory = JacksonFactory.getDefaultInstance()
  // 3
  private val sheets = Sheets.Builder(transport, jacksonFactory, AppCredentials.local).build()                        
  // 4
  private val getAll = sheets.spreadsheets()
    .values().get("10mOp9WSGCZbMmlreLwy0GJiMBeTN_t05TaXXmFmBnVI", "A:G")                           
  // 5
  fun listAll() = getAll.execute().getValues()                                                       

}

1. transport provides the Google API the means to generate and execute network calls.
2. jacksonFactory provides the Google API the means to deserialize entities.
3. Create a new Sheets Builder with required arguments such as the credentials that you included in the resources as credential.json.
4. Using sheetsinstance to define the getAll query. It fetches all the available rows for the first six columns: A:G. The first parameter is the ID of the Spreadsheet.
5. listAll() method executes the query when asked and returns the values.

You may have noticed that the return type for that last function is of type:

(Mutable)List<(Mutable)List<Any!>!>!

That scary sausage of code means that it returns a list containing lists of objects of Any type. Not extremely useful. So, you’ll have to convert it into something to present to the user.

Combining Headers and Rows

Start by creating a function inside SpreadSheetDataSource.kt to convert the raw columns to a list of maps:

private fun List<List<Any>>.toRawEntries(): List<Map<String, Any>> {
  val headers = first().map { it as String }           // 1
  val rows = drop(1)                                   // 2
  return rows.map { row -> headers.zip(row).toMap() }  // 3
}

The toRawEntries function doesn't change the state of the data source so you can define it in the same file, as a top-level function.

In this function, you are:

1. Extracting the first list and casting the values to Strings.
2. Then, you drop the first row to get the data rows.
3. Here you are converting each row to a map. The zip function creates a
   List<Pair<String, Any>>. zip pairs the elements with the same index. Finally, there is a convenience function called toMap() that converts a list of pairs into a Map.

Building Agenda Entries

Next, you need a function to convert a Map into a AgendaEntry.

private fun Map<String, Any>.toAgendaEntry() = AgendaEntry(
    id = longFor("Id"),
    title = stringFor("Title"),
    date = stringFor("Date"),
    startTime = stringFor("Start Time"),
    endTime = stringFor("End Time"),
    description = stringFor("Description"),
    speaker = stringFor("Speaker")
)

private fun Map<String, Any>.stringFor(key: String) = (this[key] as? String) ?: ""
private fun Map<String, Any>.longFor(key: String) = stringFor(key).toLongOrNull() ?: -1

stringFor(key) looks up a value in the receiver Map and tries to cast it to a String using as?. This conditional cast operator returns null if the cast is not successful, in which case you return an empty String.

longFor(key) uses the previous function to look up a value, and after toLongOrNull(), tries to parse the string into a Long. A null result means it can’t parse it into a Long, so you return a default value of -1.

toAgendaEntry() uses these two functions to build a AgendaEntry matching the headers from the following spreadsheet:

• https://docs.google.com/spreadsheets/d/10mOp9WSGCZbMmlreLwy0GJiMBeTN_t05TaXXmFmBnVI

Putting the Transformations Together

Now, update listAll to return a list of AgendaEntry entities:

fun listAll(): List<AgendaEntry> =
    getAll.execute().getValues()
        .toRawEntries()
        .map { it.toAgendaEntry() }

The return value now is a list of AgendaEntry.

Integrating the SpreadSheetDataSource

Now, go to the main() function inside RayConfApp.kt file and create a property named entries with the data source:

val entries = SpreadSheetDataSource()

Next replace talks.toJsonString() with entries.listAll().toJsonString() inside get("talks") block:

get("talks") {
   call.respondText(contentType = ContentType.Application.Json) {
     entries.listAll().toJsonString() // <--- replaced here
   }
 }

Run the server locally using the command ./gradlew appengineRun in the terminal. In the browser, the result from http://localhost:8080/talks now has all the entries from the Spreadsheet.

Now, deploy the server to the cloud using ./gradlew appengineDeploy. Once deployed, all the events are also visible on the Android app when relaunched/refreshed.

Caching Response

One thing you may have noticed is that the request takes a long time to return, anywhere between 300ms and 2s. Such a delay is understandable since the server is talking to Google Drive each time. Google Cloud provides several ways to persist data, making it more readily available. For this API, you are going to use Data Storage which provides basic key-value storage. You can see other storage options here.

Mapping Entities

Before writing a Local data source, you need a couple of mapper functions to convert from and to Entity instances. So, create a file named LocalDataSource.kt under the server module and add the following code:

import com.google.appengine.api.datastore.*
import com.raywenderlich.common.AgendaEntry

private const val AGENDA_ENTRY_NAME = "AgendaEntry"

private fun AgendaEntry.toEntity() = Entity(AGENDA_ENTRY_NAME, id).apply {
  setProperty("id", id)
  setProperty("title", title)
  setProperty("date", date)
  setProperty("startTime", startTime)
  setProperty("endTime", endTime)
  setProperty("description", description)
  setProperty("speaker", speaker)
  setProperty("votes", votes)
  setProperty("updated", updated)
}

private fun Entity.toEntry() = AgendaEntry(
    id = getProperty("id") as Long,
    title = getProperty("title") as String,
    date = getProperty("date") as String,
    startTime = getProperty("startTime") as String,
    endTime = getProperty("endTime") as String,
    description = getProperty("description") as String,
    speaker = getProperty("speaker") as String,
    votes = getProperty("votes") as Long,
    updated = getProperty("updated") as Long
)

Entity is a type from DataStore. It is similar to a Map. You can read and write values with the following two methods:

• getProperty(key: String): Any? tries to find a value for the provided key or returns null if not found.
• setProperty(key: String, value: Any?) creates or updates a value for a given key.

For the toEntity() mapper, you are creating a new Entity with "AgendaEntry" as the name and the AgendaEntry's id as the ID for the Entity. The ID is used to find and/or update entities. Then, you set the properties with the relevant keys.

In toEntry(), you do the reverse. Using the same keys used for the previous function, you read the relevant value and cast it to either a String or a Long.

Creating a Local Data Source

Now that you can transform your application models into DataStore Entities, and vice versa, you can start building a local data source. Add the class LocalDataSource inside the LocalDataSource.kt file:

class LocalDataSource {

  private val dataStore = DatastoreServiceFactory.getDatastoreService()
  private val agendaEntryQuery = dataStore.prepare(Query(AGENDA_ENTRY_NAME))

  fun listAll(): List<AgendaEntry> = 
      agendaEntryQuery.asList(FetchOptions.Builder.withDefaults()).map { it.toEntry() }

  fun save(entries: List<AgendaEntry>): List<Key> {
      entries.map { it.toEntity() }.also { return dataStore.put(it) }
  }

}

The two properties in this component are a DataStore instance and a query to get all values for entities with the name "AgendaEntry". There are also two entry points:

• listAll() is similar to what you had in SpreadSheetDataSource, but this time you are executing agendaEntryQuery on dataStore as a list and mapping it to Entries using toEntry.
• save(entries) lets you put a list of entries into the dataStore. The Data Store overrides any entities which have the same ID.

Composing Data Sources

Now you have a new data source to read and write instances of AgendaEntry in the local cache. So now you need to combine it with your other data store and do some "cache balancing".

Create a file named AgendaEntryRepository.kt under the server module and add the following code:

import com.raywenderlich.common.AgendaEntry

private const val HOUR = 3_600_000L

class AgendaEntryRepository {

  private val spreadSheetDataSource = SpreadSheetDataSource()
  private val cacheDataSource = LocalDataSource()
  private val cachePeriod: Long = HOUR

  fun listAll(): List<AgendaEntry> = cacheDataSource.listAll().let { cached ->
    cached.takeUnless { it.requiresUpdate(cachePeriod) }
        ?: spreadSheetDataSource.listAll()
            .also { entries -> cacheDataSource.save(entries) }
  }

}

private fun List<AgendaEntry>.requiresUpdate(cachePeriod: Long) =
    isEmpty() || any { System.currentTimeMillis() - it.updated > cachePeriod }

In this component, you have three properties: two for the different data sources and a third one for how long the entities are kept in memory. listAll() first tries to retrieve the data from the LocalDataSource. Then, you return the cached entities unless they need to be reloaded from the spreadsheet. The decision to load data from the spreadsheet comes down to whether the cache has no entries, or whether the last time they were loaded was over an hour ago.

If an update is required, you handle it using the Elvis operator. There, you fetch the values from the spreadsheet and tell the cacheDataSource to save the values for next time.

Integrating the Cache

Go to the main() function under the RayConfApp.kt file and instead of using SpreadSheetDataSource, use AgendaEntryRepository.

val entries = AgendaEntryRepository()

Deploy the server again by executing ./gradlew appengineDeploy in the terminal, and you will see that the first request takes about 2-3 seconds, but any consecutive calls take around 10-20ms. That's a 10x improvement!

Implementing Votes

A read-only API is not the most interesting, as you could replace it with a static JSON file somewhere in the cloud and no one would know the difference.

Next, you are going to add the option to increment votes for a given talk, so you know which one is more popular.

Updating LocalDataStore to save votes

Go to LocalDataSource class and add the following methods:

fun vote(id: Long) =
  find(byId = id)?.also { it.incrementVotes() }

private fun find(byId: Long): AgendaEntry? = try {
  dataStore.get(KeyFactory.createKey(AGENDA_ENTRY_NAME, byId)).toEntry()
} catch (e: EntityNotFoundException) {
  null
}

private fun AgendaEntry.incrementVotes() {
  copy(votes = votes + 1).toEntity().also { dataStore.put(it) }
}

The private method find(byId: Long) looks for an entity in dataStore and converts it to an AgendaEntry. It returns null if the DataStore throws an exception because it couldn't find the expected value.

In the other private method, incrementVotes() you make a copy using the copy constructor, incrementing the votes.

And, finally, vote() combines these two, returning the mutated AgendaEntry if successful or null otherwise.

Creating the Voting Endpoint

The main function doesn't have direct access to the LocalDataSource class, so you’ll have to add a new method to the AgendaEntryRepository class to be able to vote for an entry.

fun vote(id: Long) = cacheDataSource.vote(id)

Now you have everything you need to create the relevant vote endpoint. Go to the main function and add a new route in the routing block:

post("vote") {
// 1
  when (val id = call.request.queryParameters["id"]?.toLong()) {       
// 2             
    null -> call.respond(HttpStatusCode.BadRequest, "Missing id parameter")  
// 3             
    else -> when (entries.vote(id)) {                                
// 4               
      null -> call.respond(HttpStatusCode.NotFound, "No entry found for id: $id") 
// 5
      else -> call.respondText("Added vote for id: $id")                            
    }
  }
}

1. Retriev the id URL parameter and parse it to a Long.
2. If there is no ID provided, you respond with a Bad Request error.
3. If you have a valid ID, you place the vote for the user.
4. If the vote didn't happen, you respond with a Not Found error.
5. Otherwise, return a success message.

This time, to see this last change in action you’ll have to deploy to the cloud again. Open up the Android app and you’ll be able to see how you can now vote for talks:

Storing Votes After Updates

When the server updates the cache from the spreadsheet, the vote counts will be lost. The loss occurs because the entries that you get from the spreadsheet have no information about the votes.

One way to replicate this behavior is to temporarily deploy a version of the server with the cachePeriod inside AgendaEntryRepository set to 0:

The vote count remains at zero, as the server overwrites it when updating.

One way to fix this is to make sure you are keeping the votes when updating the cache from the spreadsheet. Add the following function to the AgendaEntryRepository.kt file to do this:

private fun List<AgendaEntry>.updateVotes(old: List<AgendaEntry>): List<AgendaEntry> {
  // 1
  val votes = old.map { it.id to it.votes }.toMap()
  // 2
  return map { entry -> votes[entry.id]?.let { entry.copy(votes = it) } ?: entry }
}

1. First, you create a Map of IDs to votes from the old list of entries.
2. Then, if there are votes for a given entry, you create a copy with the votes; otherwise you can just return the current entry.

Add a call to this function by updating the listAll() function in the AgendaEntryRepository class, just before the spot where you save the entries into the dataStore, so the server will persist the votes:

fun listAll(): List<AgendaEntry> = cacheDataSource.listAll().let { cached ->
  cached.takeUnless { it.requiresUpdate(cachePeriod) }
      ?: spreadSheetDataSource.listAll()
          .updateVotes(cached) // <-- added call to new function
          .also { entries -> cacheDataSource.save(entries) }
}

Deploy the server again, keeping cachePeriod as 0. The mobile app now increments the votes.

Now that you have verified that it works, you can restore the one hour cachePeriod. The server code is ready for consumers.

Where to Go From Here?

You can download the final project at the top or bottom of this tutorial.

Checkout the official documentation to explore more about Ktor.

You can also find lots of examples to learn from at Ktor’s collection of samples. Our video course Server-Side Kotlin with Ktor covers using the Ktor framework to build a Kotlin web app and API, and shows how to deploy the app to Heroku and also run the app in a Docker container.

As a challenge, let's say that your event is super-successful, so now your company has the money to have a multi-day conference. You need to add multi-day support in the app.

How would you modify both the server-side and the mobile app to support such changes? Leave your solutions in the comments below!

I hope you enjoyed the tutorial, and please leave any questions or comments below!