The evolution of asynchronous programming in JavaScript

I want to talk about something I built recently, but first I want to provide context so that it can be understood by a wider audience. I’ll cover years of background and try to explain terms so that this article can be understood by a novice JavaScript developer.

Callback Hell

JavaScript is asynchronous by nature, but what does that mean? An asynchronous function is just one whose result is not ready when the function returns. You can think of calling an asynchronous function as scheduling some work to be done, which may or may not eventually yield some value in the future. How do we get that value? The general mechanism is to pass a function that will receive it as an argument, i.e. a callback.

Chaining asynchronous functions, such that the results of one feed into another by calling an asynchronous function from within a callback to another asynchronous function, can lead to what is called callback hell:

step1(function (error, result1) {
  if (error) throw error
  step2(result1, function (error, result2) {
    if (error) throw error
    step3(result2, function (error, result3) {
      if (error) throw error
      step4(result3, function (error, result4) {
        if (error) throw error
        console.log(result4)
      })
    })
  })
})

Promises

The first advancement came as a library solution: promises. Instead of accepting a callback, an asynchronous function returns a promise object. Eventually, the promise either resolves with a value or rejects with an error. Either way, it completes. To know when that happens, you can attach success and/or failure callbacks with the methods then and catch, respectively.^[1] These methods return a new promise to which further callbacks can be attached. If a callback returns a value, it is passed to the next then callback in the chain. If it throws an exception, it is passed to the next catch callback. If it returns a promise, it is inserted at that point in the chain. Promises provided relief by replacing nested callbacks with chained callbacks, which are generally considered easier to read:

step1()
  .then(result1 => step2(result1))
  .then(result2 => step3(result2))
  .then(result3 => step4(result3))
  .then(result4 => console.log(result4))
  .catch(error => throw error)

(I have switched the example to use arrow functions which arrived in the language standard at the same time as promises. I want these examples to show the state-of-the-art as it evolved.)

Further, promises let us consolidate error handlers for multiple asynchronous functions, in the same way that multiple function calls can be handled by a single try-catch statement.

Promises come with their own problems. We’re still using callbacks, they’re just chained instead of nested. The most popular way to share one asynchronous result between multiple callbacks is to save it to a variable kept outside the callbacks. Exceptions are handled with a callback instead of the familiar try-catch structure. Compared to synchronous code, it is still less readable.

Generators

Generator functions are JavaScript functions that may yield multiple values before finally returning. They are defined in JavaScript with a special function* syntax:

function* g() {
  yield 1
  yield 2
  return (yield 3) + 1
}

Calling a generator function does not start execution of the function. Instead, it returns a generator object that represents the function call. You may step through the function by calling its next method. The first time you call next, it executes the function from its beginning to its first yield, returning the value that was yielded.^[2] The second time you call next, it re-enters the function at the point where it yielded, replaces the yield expression with whatever argument you passed to next, and continues to the next yield. This process is repeated each time you call next, until the function finally returns or throws (REPL):

const generator = g()
generator.next() // { value: 1, done: false }
generator.next() // { value: 2, done: false }
generator.next() // { value: 3, done: false }
generator.next(10) // { value: 11, done: true }

Instead of next, you may call throw to throw an exception from the yield expression. A try-catch block within the generator function can catch it, but if it is uncaught, it will pass up the call stack to the scope where you called throw (REPL):

function* g() {
  try {
    yield 1 // throws 'first error'
  } catch (error) { // catches 'first error'
    console.error('inside generator', error)
  }
  yield 2 // throws but does not catch 'second error'
  return 3 // never reached because exception thrown
}

const generator = g()
generator.next() // { value: 1, done: false }
generator.throw('first error') // { value: 2, done: false }
try {
  generator.throw('second error') // exception escapes generator
} catch (error) { // catches 'second error'
  console.error('outside generator', error)
}
generator.next() // { value: undefined, done: true }

There are a few related terms you may encounter when studying generators. The way the stepping code (by calling methods next and throw) and the generator (with yield, throw, and return) pass control back and forth between each other is called cooperative multitasking. This “bouncing” is why the stepping code is called a trampoline. Lastly, generators are a type of coroutine.

Imagine if we write a generator that yields promises. We can pair it with a special trampoline that intercepts each yielded promise and adds success and failure callbacks that pass their argument back to the generator by calling either next or throw, respectively. The trampoline itself returns a promise that resolves with the return value of the generator (or rejects with its only uncaught exception). This lets us write asynchronous code in a synchronous way:

const getFullName = async(function* (username, password) {
  let token
  try {
    token = yield logIn(username, password);
  } catch (error) {
    console.error('wrong username or password')
    return
  }
  const user = yield getUser(token)
  return user.fullName
})

Asynchronous functions

This pattern proved so popular that it was enshrined in the language with native syntax as async functions. The only differences are that async(function* (...) {...}) becomes async function (...) {...} and yield becomes await:

async function getFullName(username, password) {
  let token
  try {
    token = await logIn(username, password);
  } catch (error) {
    console.error('wrong username or password')
    return
  }
  const user = await getUser(token)
  return user.fullName
})

Footnotes