How do coroutines in Python compare to those in Lua?

Question

Support for coroutines in Lua is provided by functions in the coroutine table, primarily create, resume and yield. The developers describe these coroutines as stackful, first-class and asymmetric.

Coroutines are also available in Python, either using enhanced generators (and yield from) or, added in version 3.5, async and await.

How do coroutines in Python compare to those in Lua? Are they also stackful, first-class and asymmetric?

Why does Python require so many constructs (async def, async with, async for, asynchronous comprehensions, ...) for coroutines, while Lua can provide them with just three built-in functions?

Answer 1

The simple answer is that they are different languages. Yes, Python coroutines are stackful, first-class and asymmetric. See this answer: Coroutine vs Continuation vs Generator

From the Lua documentation:

Some people call asymmetric coroutine semi-coroutines (because they are not symmetrical, they are not really co). However, other people use the same term semi-coroutine to denote a restricted implementation of coroutines, where a coroutine can only suspend its execution when it is not inside any auxiliary function, that is, when it has no pending calls in its control stack. In other words, only the main body of such semi-coroutines can yield. A generator in Python is an example of this meaning of semi-coroutines.

Unlike the difference between symmetric and asymmetric coroutines, the difference between coroutines and generators (as presented in Python) is a deep one; generators are simply not powerful enough to implement several interesting constructions that we can write with true coroutines. Lua offers true, asymmetric coroutines. Those that prefer symmetric coroutines can implement them on top of the asymmetric facilities of Lua. It is an easy task. (Basically, each transfer does a yield followed by a resume.)

Also, see this discussion on Python's developer mail list: PEP 492: What is the real goal?

Answer 2

Are [Python coroutines] also stackful, first-class and asymmetric?

Yes, they are.

How do coroutines in Python compare to those in Lua?

A taxonomy that actually separates Python and Lua with respect to their implementations of coroutines is: can it suspend the caller?

Python coroutines cannot propagate the “suspension event” higher up in the call stack. The functions can only suspend themselves. Therefore you end up with viral flow control keywords as Christopher Kohlhoff would describe. The callee must annotate itself as an “async function” so the calling convention can recognize the return type. And the caller must redundantly re-propagate the suspension request higher up in the call stack at every call site (the await keyword).

Bob Nystrom describes this phenomena as the two colors problem: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/.

Here's a blog post from a Lua user who gets it: https://leafo.net/posts/itchio-and-coroutines.html.

And here's someone from the Java community: https://www.infoq.com/presentations/continuations-java/.

Why does Python require so many constructs (async def, async with, async for, asynchronous comprehensions, ...) for coroutines, while Lua can provide them with just three built-in functions?

Python coroutines cannot suspend the caller, so they come up with a different syntax sugar for every control flow construct to propagate the “suspension event” from callee to caller. Lua doesn't need this as Lua coroutines can just suspend whole call stacks directly.

Answer 3

I just had my first look at lua, which included the sieve.lua live demo. It is an implementation of the sieve of Erathostenes using coroutines. My immediate thought was: This would look much cleaner in python:

#!/usr/bin/env python3

# sieve.py
# the sieve of Eratosthenes programmed with a generator functions
# typical usage: ./sieve.py 500 | column

import sys

# generate all the numbers from 2 to n
def gen(n):
    for i in range(2,n):
        yield i

# filter the numbers generated by `g', removing multiples of `p'
def filter(p, g):
    for n in g:
        if n%p !=0:
            yield n

N=int(sys.argv[1]) if len(sys.argv)>1 else 500 # from command line
x=gen(N)                     # generate primes up to N
while True:
    try:
        n = next(x)          # pick a number until done
    except StopIteration:
        break
    print(n)                 # must be a prime number
    x = filter(n, x)         # now remove its multiples

This does not have much to do with the question, but on my machine using Python 3.4.3 a stack overflow happens somewhere for N>7500. Using sieve.lua with Lua 5.2.3 the stack overflow happens already at N>530.

Generator objects (which represent a suspended coroutine) can be passed around like any other object, and the next() built-in can be applied to it in any place, so coroutines in python are first-class. Please correct me if I am wrong.