I started streaming-iterables a few months ago to learn how to use async-generators and it was hard. The concepts all sound very similar but there wasn't a great resource that spelled it all out, even MDN left me wanting more. In this post, I will layout the terminologies and show how they work, and then I'll follow up in another post with some examples using streaming-iterables to taking advantage of how of it helps manage workflows.

Before I get too far, streaming-iterables is the Swiss Army Knife I've always wanted for working with data input over time. With the release of v3, it's stable and faster than your streams. It has zero dependencies and is hardly any code. The magic is in the generator functions built into your runtime - and they're only getting faster.

If you want more details on sync Iterables MDN has a pretty complete article. I'm going to assume some familiarity but lets start with naming the Iterable building blocks; 

• IteratorResult gives you the data and lets you know if the iterable is finished. It has done and value properties. 
• Iterator This object has a next() function that returns an IteratorResult object. This is the part that does all the work.
• Iterable This object is required to have a function at the symbol Symbol.iterator that returns an Iterator. It expects that each time you get a fresh Iterator, it will start giving values from the start of the collection. For example, a new Iterator for an array will always start from the first element and end with the last.

This last part confused the heck out of me;

• IterableIterator Some objects give values but aren't necessarily a collection. It doesn't make sense to be able to get a fresh Iterator that starts from the beginning.
• GeneratorFunction for example starts executing when called, and can't be restarted. It returns an Iterator object to traverse its values. However it's much easier to work with an Iterable (I'll explain why) so it provides an iterator symbol that returns itself.

MDN describes for...of as

"a loop iterating over iterable objects [...] It invokes a custom iteration hook with statements to be executed for the value of each distinct property of the object.

// an example for loop for an iterable array
const values = [1, 2, 3]
for (const value of values) {
  console.log(value)
}

Under the hood the loop calls values[Symbol.iterator]() to get an iterator object and then calls next() on that object to get values until the done property is true. We can show how it works with a while loop.

const values = [1, 2, 3]
// arrays being an Iterable always have an Symbol.iterator
const iterator = values[Symbol.iterator]()
while (true) {
  const { value, done } = iterator.next()
  if (done) {
    break
  }
  console.log(value)
}

The for...of loop is a bit nicer.

We can do this same exercise with AsyncIterables;

• IteratorResult an object with the done and value properties. 
• AsyncIterator This object has a next() function that returns a Promise for an IteratorResult object.
• AsyncIterable This object is requited to have a function at the symbol Symbol.asyncIterator that returns an AsyncIterator.
• AsyncIterableIterator is an AsyncIterator that returns itself when it's Symbol.asyncIterator function is called. For example the result of an AsyncGeneratorFunction.

Using one in a for...await...of loop.

// an example for loop for an async iterable
// getPokemon is an async generator that returns pokemon
// objects once per `setImmediate`
const { getPokemon } = require('iterable-pokedex')
const values = getPokemon()
for await (const pokemon of values) {
  console.log(pokemon)
}

This loop looks about the same as our sync example. Unsurprisingly so will our while loop.

const { getPokemon } = require('iterable-pokedex')
const values = getPokemon()
const iterator = values[Symbol.asyncIterator]()
while (true) {
  const { value, done } = await iterator.next()
  if (done) {
    break
  }
  console.log(value)
}

This is straightforward and I'm happy to write it down all in once place.

Look for my upcoming post with some examples of using streaming-iterables to make stream based workflows a little faster and easier to understand!