[HN Gopher] Asyncio, twisted, tornado, gevent walk into a bar
___________________________________________________________________
Asyncio, twisted, tornado, gevent walk into a bar
Author : BiteCode_dev
Score : 73 points
Date : 2023-08-22 17:56 UTC (5 hours ago)
(HTM) web link (www.bitecode.dev)
(TXT) w3m dump (www.bitecode.dev)
| westurner wrote:
| Eventlet (Linden Labs) actually owns the bar, then.
|
| Edit
|
| Green threads > Green threads in other languages:
| https://en.wikipedia.org/wiki/Green_thread#Green_threads_in_...
|
| Coroutine > Comparison with > Threads, Generators:
| https://en.wikipedia.org/wiki/Coroutine#Implementations_for_... :
|
| > _Generators, also known as_ semicoroutines _, [8] are a subset
| of coroutines. Specifically, while both can yield multiple times,
| suspending their execution and allowing re-entry at multiple
| entry points, they differ in coroutines ' ability to control
| where execution continues immediately after they yield, while
| generators cannot, instead transferring control back to the
| generator's caller.[9] That is, since generators are primarily
| used to simplify the writing of iterators, the yield statement in
| a generator does not specify a coroutine to jump to, but rather
| passes a value back to a parent routine._
|
| > ( _However, it is still possible to implement coroutines on top
| of a generator facility_ )
|
| Asynchronous I/O > Forms > Light-weight processes or threads:
| https://en.wikipedia.org/wiki/Asynchronous_I/O#Light-weight_...
|
| Async/Await > History, Benefits and criticisms:
| https://en.wikipedia.org/wiki/Async/await
| wahern wrote:
| > That is, since generators are primarily used to simplify the
| writing of iterators, the yield statement in a generator does
| not specify a coroutine to jump to, but rather passes a value
| back to a parent routine.
|
| Another term for this asymmetric coroutine, as opposed to
| symmetric coroutine. Coroutines in Lua are asymmetric, but
| they're not generally referred to as generators as they're much
| more capable than what are called generators in other
| languages, like Python. This is largely because Lua's
| coroutines are stackful rather than stackless, which is an
| orthogonal, more pertinent dimension when implementing
| concurrency frameworks.
|
| You can implement symmetric coroutines using asymmetric
| coroutines, and vice versa, by implementing a higher-level
| library that implements one using the other. So in principle
| they have equivalent expressive power, formally speaking.
| Ultimately the distinction between these terms, including
| generator, comes down to implementation details and your
| objective. And just because a language name-drops one of these
| terms doesn't mean what they provide will be as useful or
| convenient in practice as a similarly named feature in another
| language. Other dimensions--stack semantics, type system
| integration, etc--can easily prove the determining factor in
| how useful they are.
| Waterluvian wrote:
| I love how I read the setup, then clicked on the comments and got
| the punchline delivered perfectly by Hacker News:
|
| "We're having some trouble serving your request. Sorry!"
| butz wrote:
| Took me a while to figure out that real content of the website
| hides under whole page covering popup.
| BiteCode_dev wrote:
| Damn, I explicitly disabled the subscribe wall in the settings.
| WTF?
| shanemhansen wrote:
| Great article. I had the misfortune of writing a bunch of twisted
| code over a decade ago and I wanted to remind the author that
| twisted has a feature called "inline callbacks" that allow you to
| use yield. https://twisted.org/documents/16.4.1/core/howto/defer-
| intro....
|
| So twisted code can actually look like:
| @inlineCallbacks def doIt(): responseBody =
| yield makeRequest("GET", "/users")
| returnValue(json.loads(responseBody))
|
| iirc `returnValue` throws an exception of a specific type. It's
| ugly, but it's also the logical implementation of async on top of
| yield/generators.
| BiteCode_dev wrote:
| Also, asyncio started like this as well.
| [deleted]
| plq wrote:
| returnValue raises a specific exception that inlineCallbacks
| understands and translates to a normal return value. It was a
| hack that was only needed for python versions that couldn't
| have generators return non-None values. With modern versions,
| you don't need it anymore -- just use return.
| zokier wrote:
| While inlineCallbacks exists, and indeed works, I would
| recommend not using it. I recall it not interacting with mypy
| particularly well, and in general being such a hack it had some
| annoying sharp edges. Explicit callbacks might be annoying but
| at least they are extremely clear on what's happening.
| Fizzadar wrote:
| Really good write up. Been using gevent in pyinfra[1] for years
| and swear by it. Had some pains with setup, and am usually very
| wary of such magic, but it's just really solid. Mostly write go
| these days though which has taken the shine off for sure!
|
| Twisted, however, is a different beast. Have spent s decent chunk
| of time working on Matrix synapse homeserver[2], written in
| twisted, and oh my it just sucks.
|
| [1] https://github.com/Fizzadar/pyinfra
|
| [2] https://github.com/matrix-org/synapse
| zokier wrote:
| I struggle to call fastapi a microframework, or even bundle it in
| the same category as flask. Starlette, the actually small
| framework at the core of fastapi, fits the description better and
| would have deserved a mention imho.
| throwmeout123 wrote:
| But its also not a framework as it offers barely any facilities
| for mid+ size projects, which immediately start to crack under
| the weight of trying to reimplement Django
| BiteCode_dev wrote:
| _Summary_ :
|
| Concurrency has a lot to do with sharing one resource, and Python
| has dedicated tools to deal with that depending on the resource
| you must share.
|
| If you have to share one CPU while waiting on the network, then
| the specialized tools for this are asyncio, twisted, trio,
| gevent, etc.
|
| Asyncio is the current standard to do this, but tornado, gevent
| and twisted solved this problem more than a decade ago. While
| trio and curio are showing us what the future could look like.
|
| But chances are, you should use none of them.
| bch wrote:
| > But chances are, you should use none of them.
|
| So, no event-oriented programming in Python, or by some other
| mechanism, or...?
| js2 wrote:
| From the conclusion of the article:
|
| > If you need to get a few URLS fast, a ThreadPoolExecutor is
| likely the Pareto solution, 99% of the time.
|
| I agree with this. ThreadPoolExecutor is easy to use. In
| general, using threads in Python to handle concurrent I/O is
| pretty simple and plenty performant.
| jononor wrote:
| I picked gevent for a greenfield backend (in IoT/ML space) back
| in 2019. No regrets. The main contender was asyncio, which was
| the hot-new-thing at the time (still is, I guess). At the time, I
| did not feel that the ecosystem and programming practices for
| asyncio were good enough. That has of course matured a lot now,
| so it is no longer an issue. But I would actually still consider
| gevent again for a new project today. Possibly the asyncio web
| frameworks (ie FastAPI) have the edge over gevent-compatible (ie
| Flask) though, which might tip the scales.
|
| I remember using Twisted some 10 years ago. Never again...
| whalesalad wrote:
| gevent is the way to go these days tbh. asyncio is close but the
| dependency on async-aware libraries can be problematic.
|
| surprised the author ended on the note that none of these are
| relevant - i use a lot of these tools on a weekly basis.
| plq wrote:
| Here's the said black magic (for amd64/unix) that the gevent
| project relies on:
|
| https://github.com/python-greenlet/greenlet/blob/master/src/...
|
| I still think it's too much for a Python project -- if you think
| you need this in 2023, maybe you need to reevaluate? I'd
| seriously have another look at Go.
| Blackthorn wrote:
| That's surprisingly readable and understandable. Nice work by
| the authors for encapsulating it so well.
| d1l wrote:
| All greenlet does is allow the Python frames to managed
| separately from the C call stack - it is very similar to what
| Pypy is also doing. I don't think your characterization is at
| all accurate.
| [deleted]
| DishyDev wrote:
| Great write up on something I depend on a lot but don't
| understand well.
|
| This also answers something I've always wondered about which is
| how Twisted and Tornado fit into the Python web framework
| landscape and whether I should use them. Tornado always seemed
| popular but slow and less intuitive than Flask/Django. And then
| Twisted was a far lower level library but people were still
| building APIs on it.
|
| Where does Eventlet fit into the picture? Is it a similar box of
| magic monkey patches like GEvent?
| sigmonsays wrote:
| ever look at a twisted stack trace?
| BiteCode_dev wrote:
| And worse, when you do, something loop back.
___________________________________________________________________
(page generated 2023-08-22 23:00 UTC)