Fibers are more flexible(?)

[gunar]

The README states:

node-fibers: This implementation of coroutines is unfortunately limited to Node.js. ES6 generators may be simpler, but fibers are more flexible and support a far broader space of design possibilities. It would be great if ES6 generators were this open and flexible.

and I got intrigued. What kind of things can Fibers do that Coroutines can't?

Thank you!

[bjouhier]

Fibers and coroutines are equivalent. The difference is between fibers and generators.

Fibers and coroutines support deep continuations. This means that you can yield at any depth in the call stack and resume there later.

Generators only support single frame continuations. This means that yielding only saves 1 stack frame. This is less powerful and it explains why you need to yield (or await) at all levels when you use generators (or ES7 async/await) to streamline async call graphs.

[yortus]

As an example of yielding at any stack depth as mentioned by @bjouhier, with this fiber-based implementation of async/await you can easily combine async and functional programming:

let testFn = async (() => {

    someArray = [...];

    let results = someArray
        .filter(el => await (someAsyncTest())
        .map(el => await (someAsyncMapping());

    return results;
});

Note the two awaits are not directly within the async function, they are actually inside the anonymous lambda functions passed to the filter and map methods. ES2017 async/await and ES6 generators cannot do that. They only support 'shallow' await/yield which must be directly within the body of an async function/generator.

[gunar]

So neither ES207 async/await nor generators+co are proper coroutines? Never thought of that.

@yortus that's an amazing README-worthy example of the benefits! This could get me to switch over to Fibers/asyncawait.

Next, I'll read on the Downsides of Fibers. Thank you.

[bjouhier]

There is also a performance benefit with some code patterns.

• If you have many layers of calls on top of async APIs, all the intermediate calls are vanilla JS function calls with fibers. So you don't pay the overhead of allocating a generator or a promise at every call and you benefit from all the optims, like inlining, that the JS engine applies to vanilla calls.
• If, on the other hand, you only have a thin layer of logic on top of the async APIs, fibers are less efficient because of the Fiber.yield calls and because each fiber allocates a stack (lazily, with mmap).

The global balance may vary but with our application fibers are a clear winner.

I had not seen the Downsides of Fibers SO entry before. The first answer is a nice piece of disinformation 👎 :

• Unfamiliarity: coding on top of fiberized APIs feels just like coding on top of sync APIs. Sounds pretty familiar, doesn't it?
• Fibers does not work on windows. See Small fixes/workaround in preparation for Windows support laverdet/node-fibers#67. I've been using fibers on windows (with streamline.js) since mid 2012. Always worked like a charm.
• Browser incompatible: true
• More difficult debugging: nothing can be more wrong than this. As calls are vanilla JS calls, the debugging experience is awesome. Stepping through the code is just like stepping through sync code. And stack traces just work. No need for long-stack-trace hacks.

Coroutines are very popular in go-land but have always been challenged in js-land.

[gunar]

That SO answer is at least one year old so yeah.
But debugging is even better than when using co? That's nice!

[bjouhier]

I haven't done debugging with co but I've written a library called galaxy, which is similar to co. Debugging experience is not great because you have to step through the library at every call. Also there is no API to find out where a generator has been suspended. So it is impossible to generate a precise stack trace of async call chains that go through generators.

For an (abandoned) attempt of providing the missing API, see https://github.com/bjouhier/galaxy-stack.

[gunar]

because you have to step through the library at every call

So you're saying this doesn't happen with Fibers? Dang I'll just stop talking and give this a try.

[bjouhier]

because you have to step through the library at every call

So you're saying this doesn't happen with Fibers? Dang I'll just stop talking and give this a try.

Depends how you write your code. Let's take the example above:

let testFn = async (() => {
    someArray = [...];
    return someArray
        .filter(el => await (someAsyncTest())
        .map(el => await (someAsyncMapping());
});

If you write it this way, you will have to call testFn as await(testFn()) so you will go through the library at every call.

But, if instead you write:

let testFn = () => {
    someArray = [...];
    return someArray
        .filter(el => await (someAsyncTest())
        .map(el => await (someAsyncMapping());
};

Then you can call it as testFn() and you can write things like:

function foo() { return testFn(); }
function bar() { return foo(); }
function zoo() { return bar(); }

You don't need wrappers in these calls. You just need await whenever your code calls async node APIs, and async whenever your code is being called from node APIs. Typical code:

http.createServer((request, response) => {
  async(zoo)().catch(err => console.error(err.stack));
}).listen(port);

[bjouhier]

I forgot to mention parallelizing. If you write all your code as described above, all I/O operations will be serialized and you won't get the parallelizing benefits that node naturally gives you.

But parallelizing is easy:

const [p1, p2] = [async(zoo)(),  async(zoo)()];
// here the two zoo calls are running concurrently (still single-threaded, but interleaved)
doSomethingElse();
// now we can wait on the two results
const [r1, r2] = [await(p1), await(p2)];

Welcome to the wonderful world of coroutines 😄 !

[gunar]

Oh man... I finally realized the need for this and it sooo sucks to be stuck on this side :-(

Thank you for enlightening me.
I'll be looking more into asyncawait.
I sure wish there was a way to "transpile" asyncawait code for the browser.

[gunar]

In this example, is map blocked by finishing filter first?
It must be since this is native to javascript.
But if I were to have that, what name would it have? I'm guess the only way out is Reactive Programming which I am not thrilled by.

let testFn = () => {
    someArray = [...];
    return someArray
        .filter(el => await (someAsyncTest())
        .map(el => await (someAsyncMapping());
};

[bjouhier]

Yes, map is blocked by filter.

Not great if the different steps perform I/O because map will only start after filter has completed all its I/O. It would be better to pipeline the steps.

There are libraries that help with this. My own: https://github.com/Sage/f-streams. At the cross-roads between reactive programming and node.js streams.

[mikermcneil]

Great discussion y'all. Just as a side note, seems like .map() etc could expect an async function for its iteratee. Imagine:

someArray = await async2.mapSeries(someArray, async (item)=>{
  if (!item.lastName) {
    throw new Error('I\'m sorry, I haven\'t had the pleasure, Mr/Ms..?');
  }
  item.fullName = item.firstName+' '+item.lastName;
  item.id = (await User.create(item)).id;
  return item;
});

Or to start processing each item simultaneously, then wait for them all to finish:

someArray = await async2.map(someArray, async (item)=>{
  //...same as above, then
  return item;
});

Or to use either approach, but without waiting for them all to finish, just omit the outer "await" keyword. For example, here's what it would look like to start processing one-at-a-time, in order, but without waiting for the result (note that without waiting for the result, there's no reason to use "map" over "each"):

var promise = async2.eachSeries(someArray, async (item)=>{
  if (!item.lastName) {
    throw new Error('I\'m sorry, I haven\'t had the pleasure, Mr/Ms..?');
  }
  item.fullName = item.firstName+' '+item.lastName;
  await User.create(item);
});

in this case, since we're not using await at the top level, if the iteratee throws, the implementation of async2.eachSeries() would fire the custom function passed in to promise.catch(), if there was one (otherwise it'd cause an unhandled promise rejection)

[mikermcneil]

I do wish that es8 included a restriction such that invoking an async function would always mandate a keyword prefix. That way, you'd be sure that intent was captured and that teams new to Node.js weren't writing fire-and-forget-code by mistake.

Since async functions are new to JS, we had an opportunity to force userland code to be more clear without breaking backwards compatibility. For example, we might have had the JS interpreter demand one of the following two syntaxes:

• var result = await foo()
• var promise = begin foo()

And thus just attempting to call foo() normally would throw an error (because foo() is an async function, and it's important to make intent clear).

Currently in JavaScript, calling foo() naively (unpreceded by the special await keyword) causes the latter fire-and-forget usage to be assumed. But unfortunately, that usage is much less common in everyday apps, and also conceptually unexpected for someone new to asynchronous code.

Oh well. It's still a hell of a lot easier to remind someone that they have to use "await" than it is to try and explain how to do asynchronous if statements with self-calling functions that accept callbacks; or the nuances of your favorite promise library; or the pros and cons of external named functions versus inlining asynchronous callbacks.

So I'll take what I can get ☺️

[dtipson]

Having to call begin would feel awful similar to, but less powerful than, functional & lazy Futures and .fork...

[erobwen]

Actually, there is another aspect of why Fiber is a superior solution to async/await. I am currently building a web server with a transparent loading functionality, meaning that objects will load what they need lazily. Given that we want to call a method of an object A:

A.someFunction()

Then there is a breach in encapsulation if we have to declare someFunction to be "async". The caller of someFunction has to know if A can return a value by it self, or if it needs to load B and C in the process.

A very crude solution to this problem is to declare ALL methods of the system as async, since EVERYHERE there is a potential loading taking place, but then it will just clutter down your code with a lot of "await" and "async", even when they are not needed.

To properly encapsulate the loading taking place inside methods of A, we need a way to yield somewhere inside those methods, without having to declare all methods in the call chain up to that point as "async".

[cztomsik]

Yep, it's impossible to do lazy loading with coroutines but it is possible to do with fibers. Fibers enable true OOP (implementation hiding) and so they are in my opinion superior to (current implementation of) coroutines.

It would be great, if await was allowed in regular functions which would effectively block current coroutine (and throw exception if it is not run in any). I'd love to file proposal to tc39 but I have zero experience with this.

BTW @erobwen I'm currently using something like this:

const Fiber = require('fibers')

Promise.prototype.yieldFiber = function() {
  const f = Fiber.current

  this.then(res => f.run(res)).catch(e => f. throwInto(e))

  return Fiber.yield()
}

and then I can have

class X {
  get lazyField() {
    // do anything async
    const promise = ...

    return promise.yieldFiber()
  }
}

new Fiber(() => {
  const x = new X()
  console.log(x.lazyField)
}).run()

[bjouhier]

@erobwen. So true.

@cztomsik. I'd rather say: "it is impossible to do lazy loading with stackless coroutines". Coroutines come in 2 flavors: stackful (fibers) and stackless (async/await).