Home · Server-Side Swift Tutorials

File Handling Tutorial for Server-Side Swift Part 1

In this two-part file handling tutorial, we’ll take a close look at Server-side Swift file handling and distribution by building a MadLibs clone.

4.8/5 4 Ratings

Version

  • Swift 5, macOS 10.15, Xcode 11

In this two-part file-handling tutorial, you’ll take a close look at how Server-Side Swift handles and distributes files. Part One focuses on the internal aspects of file handling: How to persist data to file, and how to access data stored within a server’s file system.

Part Two follows up by exploring best practices for serving up data between a Server-Side Swift app and its consumers.

If you’ve worked with Server-Side Swift, you’ve likely used tools like Vapor’s Fluent or Kitura’s SwiftKuery to integrate databases and other structured data into your app.

But what if you need to interact directly with files and their contents? Do familiar iOS file management tools work as expected on a server and at web scale? You’ll find out in this tutorial.

Your sample project for this file-handling tutorial is a Vapor app called RetroLibs. It’s a nostalgic homage to the classic MadLibs text-based game. You’ll find its starting page brimming with the vintage colors, fonts and styling of Web 1.0.

Start page of the RetroLis app

For this retro game, you’ll use the equally-classic method of reading and writing your story templates directly to and from the file system. You’ll enable basic gameplay by reading and parsing stock templates from disk.

Finally, since RetroLibs is best with your own stories, you’ll accept custom story templates and store them on the file system.

While doing this, you’ll learn both synchronous and asynchronous file read and write techniques.

So go ahead and dive in!

For a quick look at Vapor, see Getting Started with Server-Side Swift with Vapor

Getting Started

Use the Download Materials button at the top or bottom of this tutorial to download the starter project. Then open Terminal, navigate to the begin folder, and run this command:

open Package.swift

This command opens the project in Xcode. It will take some for Xcode to download the dependencies but when it’s finished, Xcode will populate the schemes and show the dependencies in the project navigator.

As with any Server-Side Swift project, your sample project uses MVC – but with a special twist! Take a look at the three files in your Models directory. Notice anything unusual?

Xcode Model Directory

Your project has no database support! Everything is driven by stored file content. For this reason, your Model objects aren’t classes as usual, but simple Codable structs.

Take your app for a quick spin. In Xcode, ensure you’ve selected the Run schema with My Mac as the target device.

Set Xcode’s working directory: Edit the Run▸My Mac scheme, and check the Run▸Options▸Working Directory checkbox. Click the folder icon to locate your begin folder.

Build and run, then open your browser to localhost:8080. You might see a prompt to allow access to the directory: Be sure to grant access so the app can read and write files.

Documents Folder Access

Note: When you build and run in Xcode, you might get mysterious build errors. Congratulations! You get to swift run in Terminal instead! You can still edit code in the Xcode editor, but you’ll have to vapor build then swift run in Terminal to run the server. You probably won’t see the access prompt, but everything will still work OK.

You’ll see the RetroLibs opening page. Other than proudly displaying its retro colors, it doesn’t do anything yet. It’s time to get to work!

Using Templates

To get underway, you’ll first assemble a list of available RetroLibs templates. You’ll find sample template files in the RetroLibsTemplates subdirectory.

Template Syntax

Take a look at these sample templates. Each template is a simple text file. Templates let users add any number of fill-in-the-blank-with-a-word-of-this-description elements by surrounding inline descriptors with curly braces. For example:

I like {type of food}, because it is {adjective}!

As a bonus(?!) – and in the spirit of the web’s earlier Wild West days — templates can include ad hoc HTML tags. After all, what’s a vintage Web 1.0 page without a bit of freeform inline styling? :]

The <b>{adjective} robot</b> blinked 
its <blink>{adjective} {color} eyes</blink>.

The pre-supplied sample templates are <blink>-free, we promise!

Getting a List of Templates

To keep things simple, each template’s filename is the title you’ll present to users. So you’ll just need to get a list of this directory’s contents.

To get the directory’s contents, you’ll use FileManager, which you likely know from iOS and other Apple-native platforms. Even though FileManager isn’t optimized for server-side use, it’s common to use it to quickly grab the contents of a directory.

Back in Xcode, open FileController.swift, and locate its getTemplateNames() stub method. Replace its contents with the following:

do {
  return try fileManager.contentsOfDirectory(atPath: workingDir)
} catch {
  return nil
}

Here, you leverage two pre-provided static vars. fileManager provides an instance of FileManager.default, while workingDir uses Vapor’s DirectorConfig type to detect the local environment’s working directory, and build a reference to the RetroLibsTemplates directory, which contains your RetroLibs templates.

From there, you simply ask fileManager to return an array of filenames, or nil if there’s an error.

Build and run, then open your browser to localhost:8080 again.

Getting a List of Templates

You’ll now see a list of the sample templates. Nice! But if you click one, you’ll get an error because you haven’t wired up the read-file functionality yet. You’ll also see an unwanted .DS_Store tag-along in the list of templates. You’ll address these issues next.

Reading Template Files

To teach your sample app to read template files, you’ll first add that functionality to FileController.swift. Open it in Xcode, find the static method readFileSync(_:) and replace its contents with:

 
fileManager.contents(atPath: workingDir + filename) 

This uses fileManager again. Note that you wouldn’t want to do this in production. You’ll return to this a bit later to make the read operations production-worthy.

Next, open Models/TemplateCollection.swift, the file tasked with assembling raw template files into an array of structured Template objects. Find its init() method.

Currently, it creates an array of empty Template objects with only a name, but you’ll need it to read each template body from the file system.

To do this, replace the existing for name in templateNames { ... } loop with the following:

for name in templateNames {
  if let data = FileController.readFileSync(name), 
    let content = String(data: data, encoding: .utf8) {
    let (templateParts, tags) = content.toRetroLibsClips()
    let template = Template(name: name, templateParts: templateParts, tags: tags)
    templates.append( template )
  }
}

Here, you use the just-updated readFileSync(_:) to read each template file from disk. You then parse each template’s raw file contents into paired arrays of templateParts, the static parts of a template, and tags, the parts you’ll fill in to create a unique story.

Persisting the Selected Template

Next, you need to add a route-handler pair to present a selected template. But first, you need to solve a problem: Since there’s no database, you need a way to persist a selected template as users navigate between pages.

Your solution will take advantage of Vapor’s SessionsMiddleware, which provides a session-specific dictionary for caching small bits of String data.

You can’t store structs directly here, and your model objects are structs. What you can do is to “freeze-dry” them into JSON strings before presenting a new page, then “rehydrate” them back into native structs later on.

Now, open WebController.swift. Locate templatesHandler(_:), and insert this line just before the return statement:

try req.session()["templates"] = context.toJSONString()

This caches the template selected by the user in the session dictionary as JSON. Downstream route handlers can now retrieve it.

Next, add this handler method at the bottom of the Route Handlers section:

func selectedTemplateGetHandler(_ req: Request) throws -> Future<View> {
  guard let name = req.query[String.self, at: "name"] 
    else { throw Abort(.badRequest) }

  let context = try getCachedTemplate(named: name, on: req)
  try req.session()["selected-template"] = context.toJSONString()

  return try req.view().render("selected-template", context)
}

Here, you make sure you’ve received a template name. Then, with the help of getCachedTemplate(named:on:), you locate the desired template from an array of templates cached earlier, and place it in a context struct. Finally, you cache just the selected template for downstream use, and present the user with a RetroLibs entry form via the selected-template.

Last, register your route by appending this to boot(router:):

sessions.get("selected", use: selectedTemplateGetHandler)

Nice! Build and run, and open localhost:8080 in your browser.

Template and Result

This time, you see just your templates, no unwanted tag-alongs as before. Even better, clicking a template takes you to a retro-chic page presenting you with a dynamic set of RetroLib words to fill in.

You have one last task to complete before you can fill in the form and see your custom story — you’ll quickly handle that now.

Submitting the Completed Form

When a user completes a form and clicks the Retro-ify Me! button, they submit a form via an HTTP POST request. You’ll need to take the form data and zip it into a complete story in a new POST route and handler. Make it so.

Back in WebController.swift, append this method to the struct’s extension:

private func getStory(from data: EntryFormData, 
  on req: Request) throws -> Future<(String,String)> {
  let tagValues = data.valuesJoined.split(separator: "¶")
    .map { String($0) }
  guard let template = try req.session()["selected-template"]?
    .toRetroLibsTemplate(), template.tags.count == tagValues.count 
  else {
      throw Abort(.internalServerError)
  }

  let story: String = {
    var str = ""
    for (index, templatePart) in template.templateParts.enumerated() {
      str += templatePart
      str += index < tagValues.count ? tagValues[index] : ""
    }
    return str
  }()
  let title = template.name

  return req.future((title, story))
}

This method addresses an interesting problem you'll occasionally encounter in Server-Side Swift. When you created this form, you looped through each template tag and created an <input> object.

That's great for presentation, but when you accept form data via Codable, a core requirement is that you return data in predictable, fully-defined Codable – or in Vapor, wrapping Content – structs.

How do you reconcile the unpredictable numbers of RetroLibs template inputs with Codable requirements?

In this case, you handle this by gathering all your input values, which are all set to required, into a delimited string you place into a single hidden <input> field.

You then ingest just this single, hidden form input into your app, split the string into an array, then zip the story snippets and user-entered values into a single story string.

Handling the Completed Form

Now, you just need to write and register a POST handler. First, append this method at the bottom of the Route Handlers section:

func selectedTemplatePostHandler(_ req: Request, data: EntryFormData)
  throws -> Future<View> {
  try getStory(from: data, on: req).flatMap(to: View.self) {
    (title, story) in
    let context = CompletedStory(title: title, story: story)
    return try req.view().render("completed-story", context)
  }
}

Here, you parse form data sent with the POST request. Leveraging the just-added getStory(from:on:), you convert the entry values into a future complete story. In the flatMap closure, you create a context struct from the story components, and pass this to the user.

Last, register this route at the bottom of boot(router:):

sessions.post(EntryFormData.self, at: "selected",
  use: selectedTemplatePostHandler)

Ready? Build and run, and open localhost:8080. This time, select a template, enter your most creative word choices, click Retro-ify Me!, aaaaaand... stare in wonder at your first RetroLibs story.

RetroLibs story

Creating Your Own Template

RetroLibs now hums along nicely with canned templates. But why settle for the same old pre-packaged stories when any modern-retro app worth its salt ought to let users create their own homegrown goodies?

Back in Xcode, open FileController.swift once again. Locate the stub method writeFileSync(named:with:overwrite:), and replace its contents with:

guard overwrite || !fileExists(filename) else { return false } 
return fileManager.createFile(
  atPath: workingDir + filename, contents: data)

This method simply confirms that you're not overwriting an existing file if you don't want to, then writes the file to disk.

As before, you're using FileManager to get up and running initially, but you wouldn't use it in production. You'll make this production-worthy a bit later.

Next, open WebController.swift, and append the following method to the WebController extension:

private func saveTemplate(from data: UploadFormData, on req: Request)
  throws -> Future<String> {
  guard !data.title.isEmpty, !data.content.isEmpty
    else { throw Abort(.badRequest) }

  let title = data.title.trimmingCharacters(in: CharacterSet.whitespaces)
  let filename = String(title.compactMap({ $0.isWhitespace ? "_" : $0 }))
  let contentStr = data.content.trimmingCharacters(
    in: CharacterSet.whitespaces)

  guard let data = contentStr.data(using: .utf8), 
    FileController.writeFileSync(named: filename, with: data) == true 
  else {
    throw Abort(.internalServerError)
  }

  return req.future(filename)
}

This method handles the details of accepting a user-submitted story from a form, converting it to structured title and content strings, saving the template to disk and returning its filename. You now have the pieces you need to wire up the routes and handlers for entering and saving custom templates.

Adding Routes for Your Template

You'll now add a pair of routes and matching handlers: one for the GET entry route and a second for the POST route.

Append these two handler methods to the Route Handlers section:

func addTemplateGetHandler(_ req: Request) throws -> Future<View> {
  try req.view().render("add-template", VoidContent())
}

func addTemplatePostHandler(_ req: Request, data: UploadFormData)
  throws -> Future<View> {
  try saveTemplate(from: data, on: req).flatMap(to: View.self) {
    filename in
    let content = UploadSuccess(filename: filename)
    return try req.view().render("template-uploaded", content)
  }
}

The GET route handler doesn't need to package up any data, so you simply present the add-template Leaf template with an empty content struct to satisfy Codable requirements.

Your POST handler has more work to do, but the saveTemplate(from:on:) helper method you added earlier makes this an easy job. Since this helper passes back a future-wrapped filename, you use flatMap(to:) to access the filename, create an UploadSuccess struct, and present the results via the template-uploaded template.

Putting It All Together

Now, add the routes for these handlers by appending the following to boot(router:):

sessions.get("add-template", use: addTemplateGetHandler)
sessions.post(UploadFormData.self, at: "add-template", use: addTemplatePostHandler)

The last thing you need is to add a button to direct users to the custom template entry page. Open Resources/Views/templates.leaf, and add the following to its content declaration, immediately after the closing curly brace of its else clause:

<h2 class="or"> Or... Make Your Own!</h2>
<form method="get" action="/templates/add-template">
  <button type="submit"> Let Me Make My Own Retro-Awesome Template! </button>
</form>

OK! Build and run and, once again, open your browser to localhost:8080.

Make Your Own Button.

Click the new button at the bottom of this page to navigate to the custom template entry form. Feel free to enter your own template using the syntax described above, or use the following values:

Title: My Big Day
Template Contents: I woke up and made myself a {container} of {breakfast food}. It tasted {adjective}.

Click the submit button, and you'll see a just-slightly over-the-top confirmation that your new creation was saved to disk.

Added My Big Day Template

Take a quick peek in Xcode, and you'll see your template has also been added to the RetroLibsTemplates directory.

My Big Day in Xcode folder.

To complete the loop, return to the landing page, select your new template, fill in your custom words and enjoy your very own timeless masterpiece. Ah... the wonders of technology!

My Big Day story

Adding Concurrency

RetroLibs is now functionally complete. But it's hiding a lurking problem: Its core read and write methods are currently both synchronous — they block the main thread. This isn't a big deal in a trial app running locally for one user. At scale, however, it could bring a web app to its knees.

The backstory here is that FileManager was written long before modern Server-Side Swift was a glimmer in Apple's eye, with Apple-native-app needs in mind. It's generally entirely reasonable for reads and writes to be synchronous within the context of a mobile or desktop app, since apps have a single user.

But in a server-side context, web apps can scale to large numbers of concurrent users. And although threads often save the day in native app contexts, there are only a finite number of threads available — often, one per CPU core.

For this reason, threads aren't the primary answer to performance at scale in a scalable web app. What you need is a library written from the ground up to support the unique needs of the server.

Fortunately, Apple offers precisely this feature with SwiftNIO.

Using SwiftNIO

If you've worked with Swift on the server, you know that SwiftNIO provides the high-performance engine that underlies both Kitura and Vapor. So it'll come as no surprise to use it to address your thread-blocking issues.

Return to FileController.swift, and take a look at the signature of readFileSync(_:). Notice that it directly returns an optional Data type. It's of course expected – and very convenient – for a blocking, synchronous method to return a result directly.

But with SwiftNIO, processes like these are asynchronous. They return EventLoopFuture-wrapped results that aren't guaranteed to be ready when downstream methods receive them.

This means that your asynchronous file writing method needs to return Future<Data>, and you need to adjust your receiving functions to accommodate Future-wrapped results.

Note: SwiftNIO is a high-performance, low-level library not intended for direct use. It's common — and generally desirable — to work with framework-specific convenience wrappers, rather than with SwiftNIO directly.

This approach lets frameworks handle the heavy lifting of working with SwiftNIO so developers can focus on simpler, more immediately-useful APIs. You'll take that approach here.

Enough talk. Make it so!

Reading a File Asynchronously

Locate FileController's readFileAsync(_:on:). Replace its contents with:

try req.fileio().read(file: workingDir + filename)

If you think that after all this buildup, the solution was almost trivial, you're right! This deceptively simple line of code does a lot of heavy lifting on your behalf. It's one of a great many things that are easy to overlook when considering the value of frameworks like Kitura and Vapor.

This code asynchronously reads the contents of a file in a manner guaranteed not to block your web app. It throws an error if it encounters any problems, and returns the file contents within a Future.

Notice that its signature differs from its synchronous counterpart in that it takes a Request object: You need this to attach EventLoopFutures to the request's thread, as well as to return a Future-wrapped result.

Getting a List of Templates Asynchronously

Open TemplateCollection.swift. See how its init() needs to read the content of each template in real time? That's natural for synchronous methods, but it doesn't square with your new asynchronous read method.

Quite simply, if you call the read method within the context of the initializer, you don't have a way to "reach inside" the Future-wrapped Data. You'll need to add a layer of indirection to solve this.

First, add this new initializer below the current one:

init(withTemplates templates: [Template]) {
  self.templates = templates
}

Here, you quite simply pass the buck back to the calling context, in effect saying "You take care of handing me a set of unwrapped templates, buddy." Here's how you'll make this happen.

Immediately below the new initializer, add this static method:

static func makeFutureSelf(on req: Request) throws -> Future<TemplateCollection> {
  // 1
  guard let templateNames = FileController.getTemplateNames() else {
      throw Abort(.preconditionFailed)
  }
  // 2
  var futureFileData: [Future<Data>] = []
  for name in templateNames {
    try? futureFileData.append(FileController.readFileAsync(name, on: req))
  }
  // 3
  return futureFileData.flatten(on: req).map(to: TemplateCollection.self) { 
    dataArray in
    var templates = [Template]()
    for (index, data) in dataArray.enumerated() {
      guard let content = String(data: data, encoding: .utf8)
        else { continue }
      let (templateParts, tags) = content.toRetroLibsClips()
      let template = Template(name: templateNames[index],
        templateParts: templateParts, tags: tags)
      templates.append( template )
    }
    guard templates.count > 0 else { throw Abort(.preconditionFailed) }
    return TemplateCollection(withTemplates: templates)
  }
}

Here's what you're doing in this method:

  1. Get an array of template names from FileController.
  2. Loop through the template names, and call readFileAsync(_:on:) to create an array of Future.
  3. Flatten the array of Future to a single Future containing a TemplateCollection. Within the map closure, you do have direct access to the unwrapped templates SwiftNIO has read from disk. This gives you the platform you need to call your new initializer with an already-assembled set of templates. W00t!

Now, you just need to wire this new functionality to your routes. Open WebController.swift, and replace the contents of templatesHandler(_:) with:

try TemplateCollection.makeFutureSelf(on: req)
  .flatMap(to: View.self) { templateCollection in
  let context = templateCollection
  try req.session()["templates"] = context.toJSONString()
  return try req.view().render("templates", context)
}

Here, you rely on flatMap(to:) to "step inside" the Future (a heady concept!), and work directly with the Future-nested Data. You do your work within this closure, and Vapor and SwiftNIO take care of resolving the chain of Futures before presenting the results in your view.

Build and run your app, then open your browser to localhost:8080 again. You'll notice that... well, everything works exactly the same as before. Exciting, right?

Actually, of course, if you're building web apps, it really is exciting to know your app stands ready to scale as needed! :]

Writing a File Asynchronously

The last piece of your puzzle is to make your write operations asynchronous. Here, you hit a minor hiccup.

As it turns out, Apple initially released SwiftNIO without support for asynchronous write functionality. That's less surprising than it might seem since in production, you typically do any serious file management via APIs. You likely won't store large files on a production web server.

Nonetheless, SwiftNIO added this functionality last year.

At the time of writing, the major Swift frameworks have not yet added convenience wrappers for SwiftNIO's write functionality. When that happens, using a framework's wrapper will become a best practice.

Until then, one common solution is to use Grand Central Dispatch (GCD) to wrap FileManager's synchronous write functionality within an asynchronous DispatchQueue task. This ensures that write operations are non-blocking. Many production sites currently use this method, and you'll follow suit here.

Using Grand Central Dispatch

It's easy enough to wrap a synchronous method call within an asynchronous DispatchQueue task. The challenge is ensuring that the route handler waits for the task to complete and sees its result.

Open FileController.swift, and find writeFileAsync(named:with:on:queue:overwrite:). Replace its contents with:

// 1
guard overwrite || !fileExists(filename) else {
  return req.future(false)
}
// 2
let promise = req.eventLoop.newPromise(of: Bool.self)
// 3
queue.async {
  let result = fileManager.createFile(
    atPath: workingDir + filename, contents: data)
  // 4
  promise.succeed(result: result)
}
// 5
return promise.futureResult

Here's what you've done:

  1. As with the synchronous variant, you ensure you're not about to accidentally overwrite an existing file.
  2. You register a new promise on the Request object's EventLoop. This promise must either fail or be fulfilled before the route handler completes.
  3. You then call the synchronous createFile(atPath:contents:) within a DispatchQueue task, and dispatch it to execute asynchronously.
  4. Importantly, you bind the results of the call to the promise, and call its succeed promise method.
  5. Finally, you return the promise's futureResult, effectively binding the asynchronous DispatchQueue task to the route's EventLoop. This ensures that the method returns the Future-wrapped results to its calling context.

With this in place, open WebController.swift, and locate saveTemplate(from:on:) at the bottom of the file.

You'll update its signature to return a Future, and replace the current guard clause, which relies on synchronous writes, with code that works with asynchronous writes.

Replace the entire method with this:

private func saveTemplate(from data: UploadFormData, on req: Request) 
  throws -> Future<String> {
  // 1
  guard !data.title.isEmpty, !data.content.isEmpty 
    else { throw Abort(.badRequest) }
  // 2
  let title = data.title.trimmingCharacters(in: CharacterSet.whitespaces)
  let filename = String(title.compactMap({ $0.isWhitespace ? "_" : $0 }))
  let contentStr = data.content.trimmingCharacters(in: CharacterSet.whitespaces)

  guard let data = contentStr.data(using: .utf8) 
    else { throw Abort(.internalServerError) }

  // 3
  return try FileController
    .writeFileAsync(named: filename, with: data, on: req, queue: dispatchQueue)
    .map(to: String.self) { success in
      return success ? filename : ""
  }
}

With this code you:

  1. Ensure the title is not empty: If it is, throw a 400 bad request error.
  2. Build the content string by trimming unwanted characters.
  3. Asynchronously write the file data to disk, and return a Future.

And now your app handles both read and write functionality without blocking the main thread. Bring on the web-scale, baby!

Build and run, and open your browser to localhost:8080 one final time. You haven't changed any functionality, but you can enjoy the satisfaction of knowing that your app is now ready to read and write files and to scale gracefully without blocking!

Where to Go From Here

You can download the completed project files by clicking the Download Materials button at the top or bottom of the tutorial.

Be sure to check out Part 2 of this file handling tutorial when it's published. It focuses on how to serve up files from your Server-Side Swift app, including important nuances of doing this in the Linux and Docker environments, where most Server-Side Swift apps live.

If you're curious about SwiftNIO, Futures and Promises, check out Creating an API Helper Library for SwiftNIO.

Meanwhile, what do you think about reading and writing files in Server-Side Swift? Have you run into difficulties doing this, or have you found any helpful techniques you'd like to share? Let us know in the comments section, and Happy File-ing!

Average Rating

4.8/5

Add a rating for this content

4 ratings

More like this

Contributors

Comments