[HN Gopher] See Python, See Python Go, Go Python Go (2016)
___________________________________________________________________
See Python, See Python Go, Go Python Go (2016)
Author : amzans
Score : 65 points
Date : 2020-01-23 20:07 UTC (2 hours ago)
(HTM) web link (blog.heroku.com)
(TXT) w3m dump (blog.heroku.com)
| tech234a wrote:
| Title should mention this is from 2016.
| dang wrote:
| Added. Thanks!
| alimoeeny wrote:
| I had not used python for like 5 year, since when I migrated all
| my work to go. Recently I went back to python and was shocked to
| see how hard (relatively) it is to setup a (moderately) high
| performing web server in python. I mean in my case I had a "data
| science" type application and sometimes a request would block and
| take a second to finish, and this meant a handful of users would
| bring the server to it's knees (due to extremely high mem usage I
| could not have a lot of indepentend worker processes running at
| the same time),
|
| I wish I could call python code from within a go web server with
| some ease and safety.
| throw_14JAS wrote:
| Wouldn't a job queue solve both issues? You could either make
| the data science stuff async (and spin up more servers as
| necessary). Or, with a job queue, you could use a go webserver
| to push requests into the queue for processing in python.
| whalesalad wrote:
| > I mean in my case I had a "data science" type application and
| sometimes a request would block and take a second to finish,
| and this meant a handful of users would bring the server to
| it's knees
|
| This sounds like an issue with the design of your program, not
| Python.
| 0xCMP wrote:
| We run python at work and I have to agree with this however
| with Python 3 and ASGI servers now existing this should be
| significantly less of an issue. Even just Python 3 WSGI using
| Gunicorn + gevent should be fairly performant for Python.
|
| There is no way around the high memory usage, but a large
| number of the problems with Python concurrency is not loading
| nginx (or load balancer) in front and not switching to gevent
| from PreFork which uses a considerably higher amount of memory
| per "node" for higher concurrency. That said, gevent is only
| "performant" if what you're doing is IO bound. Same thing with
| any AsyncIO based server.
|
| "data science" sounds like DB or NoSQL heavy so should fit this
| case, but of course all of this is just general advice and
| depends on the app/code like others said.
| Something1234 wrote:
| Your claims are without merit unless you post details about
| your setup and what you were trying to do.
|
| Especially considering that there's quite a few sites which run
| entirely on Python.
| alimoeeny wrote:
| This claim is specifically related to blocking webserver in
| python vs concurrent webservers in go. I assumed this is
| common knowledge, that scaling python webservers is a
| complicated thing, while in go you get a concurrent web
| server in standard library,
| infecto wrote:
| And this is the trouble with the OP imo. While its true, I
| find it an unfair comparison to make.
| throw_14JAS wrote:
| Were you deploying using a Python webserver (e.g.
| SimpleHTTPServer), or was it nginx routed to an app server?
| eloff wrote:
| That you can run a site entirely on Python does not mean it's
| efficient to do so.
| whalesalad wrote:
| Efficiency is measured in a lot of different ways.
| Development speed? Team familiarity? Existing
| infrastructure (private pypi etc...)? Existing libraries?
|
| If we go with what I would assume is your definition --
| speed of execution versus resources used -- it is certainly
| possible to build fast Python applications that are
| efficient.
|
| But use what makes you and your team happy, life is too
| short for anything else.
| toyg wrote:
| Is there anything about doing it the other way around, i.e. embed
| python code in Go...? I wish we could get the easy-deployment
| story of Go around the easy-development story of Python.
| zaptheimpaler wrote:
| Python imports modules dynamically so even if we embedded
| python in Go, all the deployment woes of Python would still be
| around. Its kind of a fundamental problem with dynamically
| linked libraries and python being an interpreted language.
| hopia wrote:
| I'm also curious to know how people who use Python for data
| science and combine it with some other platform for web facing
| endpoints architect their apps in the cloud.
| rajamaka wrote:
| Python web server triggering lambda functions to process jobs
| hopia wrote:
| Where do you put the web server? What's the purpose of it?
|
| Or you mean to use Python as the client facing API server?
| I meant how to architect if you use some other platform for
| the publicly exposed API. Would you build an internal
| Python API at the lambdas that run the actual data science
| computation, and invoke that from the client facing
| API/app?
| maxmcd wrote:
| You might be interested in this:
| https://github.com/google/grumpy
|
| (although it looks like it's not close to supporting the needed
| functionality to import a typical python library)
| lostmsu wrote:
| There is a similar project for .NET:
| https://github.com/pythonnet/pythonnet
|
| It makes calling C# as easy as: import clr
| import System uri = System.Uri('http://python.org')
|
| And also works the other way around. In both cases you have to
| mind GIL though.
| raziel2p wrote:
| The gohttp library seems to do a lot less than what
| gunicorn/werkzeug/flask does. I would guess this is what makes
| the drastic performance difference, not the fact that the HTTP
| handler is written in go. I'm surprised that it was so close to
| go-net/http performance, though.
| jchw wrote:
| Actually it's probably concurrency:
|
| > Keep in mind that this is with 10 concurrent requests, so
| werkzeug-flask probably chokes more on the concurrency than the
| response time being slow.
|
| I am not sure though. I'd imagine Go can beat Python
| performance enough to make up for the (clearly not very
| egregious) CGo penalties.
___________________________________________________________________
(page generated 2020-01-23 23:00 UTC)