[HN Gopher] Learning Go as a Python Developer: The Good and the Bad
___________________________________________________________________
Learning Go as a Python Developer: The Good and the Bad
Author : shantnutiwari
Score : 137 points
Date : 2022-07-18 18:22 UTC (4 hours ago)
(HTM) web link (new.pythonforengineers.com)
(TXT) w3m dump (new.pythonforengineers.com)
| 120photo wrote:
| I tried porting a Python utility I wrote to Go specifically
| because I did not want people to have to install any 3rd party
| libraries (in this case just one). If anything I had a much
| deeper appreciation as to how much Python does for you and just
| lets you work. I would still like to port someday. For now I can
| containerize my app but that would still require people to
| install docker and learn how to use docker.
| zozbot234 wrote:
| You don't need docker to bundle an app with custom libraries.
| You can use AppImage which is pretty much just an ordinary app
| install rolled into a single file.
| LtWorf wrote:
| i guess he's not using linux, otherwise python dependencies
| would be no problem
| zozbot234 wrote:
| Docker is linux specific. It can only run in a VM on non-
| linux platforms.
| wizofaus wrote:
| It requires a linux subsystem, true, but not a fully
| fledged VM. The subsystem still shares the file system
| with the primary OS, for a start. And while I haven't
| tried it, the instructions for installing AppImage on WSL
| (for example) don't look trivial.
| DenseComet wrote:
| WSL is a fully fledged VM under the hood, it's just well
| integrated.
| wizofaus wrote:
| MS call it a "lightweight utility virtual machine", and
| "not a traditional VM experience", whatever that means.
| Docker also works on MacOS, with a basic linux distro
| running under hypervisor that again, I'm not sure
| qualifies as a 'fully fledged VM' (at least in terms of
| user access to it).
| deivid wrote:
| The easiest way to do this is to use shiv[0] if you don't mind
| asking people to have Python itself installed; if you want a
| truly "one file bundle" you should use PyInstaller[1] (which
| bundles a Python interpreter)
|
| [0]: https://shiv.readthedocs.io/en/latest/
|
| [1]: https://pyinstaller.org/en/stable/
| [deleted]
| cookiengineer wrote:
| I've been reimplementing a tool that was done in nodejs before
| and ported it to golang.
|
| I have to say that a lot of programming paradigms are different
| in golang, a few parts are annoying, and some parts are "getting
| there" with generics.
|
| The most annoying part if you do a lot of parsing is the golang
| mandated rule of what is public and what is private in a package.
| If you want to parse lots of JSON into a struct, mess is on you,
| always have to implement a custom marshal/unmarshal for it.
|
| If you always have to implement a custom JSON marshalling method,
| and abuse annotations for mapping upper/lowercase
| properties...why was the uppercase rule mandated in the first
| place?
|
| I wish golang had gone for an export keyword instead. Structs
| would have been so much cleaner.
|
| The second issue I have with golang is that struct properties (as
| all data types) aren't nullable except via pointers, which makes
| consuming JSON APIs a fustercluck to deal with some times. I wish
| there was a nullable string data type instead of having to use
| dozens of helper functions.
|
| The last issue is lack of functional ideas (as of now) but they
| are coming. lo is a lodash inspired library that makes a lot of
| things easier. [1] I hope that a lot of these helpers will at
| some point be part of the core libraries, so that datatypes can
| be easily mapped, casted and filtered.
|
| [1] https://github.com/samber/lo
| binwiederhier wrote:
| If you do a lot of JSON parsing for unknown or large structs,
| you can use gjson: https://github.com/tidwall/gjson
| usmannk wrote:
| Personally I find struct field tags a nice way to encode the
| json fields. They also allow you to use upper case names for
| all struct members. I've never had an issue with the way json
| is done in golang, and I'm kind of confused by this rebuke.
| What's wrong with using string pointers? What helper methods
| are necessary to write? This kind of feels the same as reading
| a rant against strict types.
| returningfory2 wrote:
| > I had heard of Go for many years, but never stuck with it; it
| gets constant negative press on Hacker News/Reddit
|
| I think on Hacker News it's really fashionable to criticize Go,
| and this has led to a culture where Go gets significantly more
| negativity than it deserves.
|
| As technologists, one of our most important jobs is to see
| through such fashions and judge languages/tools/technologies
| based on their actual merit.
|
| While gauging sentiment about things on a forum like Hacker News
| is generally really helpful, it should not be the main basis of
| our decisions.
| tempest_ wrote:
| Go is getting used in places where Java would have been used in
| the past.
|
| Java complaints go down, Go complaints go up.
| Xeoncross wrote:
| If Go hadn't come along, I would be using and complaining
| about C# or Java for sure.
| erik_seaberg wrote:
| Some of the HN audience had already been reading PG's essays
| for years, and our expectations of new languages were shaped by
| http://www.paulgraham.com/avg.html.
| 37ef_ced3 wrote:
| As an engineer who uses Go and appreciates it as a replacement
| for most purposes where you might use C or C++, the complaints
| just seem bizarre.
|
| You've got people who argue against explicit error checking and
| people who don't understand the important role of nil pointers
| and people who don't see the massive net win of garbage
| collection in concurrent programs, and so on.
|
| Endlessly.
|
| It feels like a waste of time defending the language when the
| people criticizing it seem to hold such a vastly different
| point of view. It's like trying to convince people that Natural
| Born Killers is a good movie or that Primus makes good music.
| There's an unbridgeable chasm between you and the people you're
| trying to convince.
| frazbin wrote:
| > people who.. don't understand the important role of nil
| pointers
|
| ... I guess you're right; I really do not understand
| LtWorf wrote:
| But C++ had generics ages ago already... You never used
| those?
| obviouslynotme wrote:
| Go has always been a Java, C#, Python, and Ruby killer. It
| has not and never will be a C or C++ killer. C is the
| language of libraries and embedded, neither of which Go is
| good for. C++ is the language of large systems that need
| extreme control over resource allocation. Go is not good for
| this either.
| wizofaus wrote:
| Is Go really killing C# though? I've rewritten Go
| components in C# (the initial version only had basic
| functionality requirements, but as they expanded, Go made
| less and less sense given the rest of our codebase), but
| can't readily imagine wanting to rewrite C# in Go. Maybe
| once they add proper exceptions...
| obviouslynotme wrote:
| No. C# is usually used by Microsoft shops, and half the
| reason to use C# is so you can use Visual Studio.
| wizofaus wrote:
| That implies that if a better alternative to VS came
| along, C# developers wouldn't choose it! I certainly
| would, for one, and I'm a long-term VS user, well aware
| of its failings (but also yet to come across another IDE
| that's noticeably superior. And coding with a text editor
| and command line compiler is something I spent 10+ years
| doing when it was all that was available, and nothing I
| have any great hankering to go back to - even if it's
| something I still resort to as an emergency measure, e.g.
| when on some remote system with text-only access etc.)
| vsnf wrote:
| My current project was switched from C# to Go almost 2
| years ago. It was a management decision, and while the
| team all generally has found something or another to like
| about Go, we all miss C#.
| tptacek wrote:
| It is too easy for people to forget that C was not _always_
| the language of "libraries and embedded", and that writing
| large systems programs in C, of the sort that nobody would
| realistically consider writing that way in 2022, was a
| norm. Displacing C doesn't just mean rewriting existing C
| dependencies, but also changing the balance of decisions
| about which _new_ programs to write in C. Go has had a
| tremendous amount of impact there. Without Go, Docker would
| almost certainly be a large ball of C code.
| ledauphin wrote:
| I agree with you that Go does not replace C or C++, but I
| strongly disagree that Go is an obvious replacement for
| most uses of Python, C#, Java or Ruby, all of which are,
| frankly, _much_ higher level languages than Go.
|
| Go gets used for two main reasons that I've been able to
| observe:
|
| 1) a desire for a very specific type of concurrency 2) a
| desire for a fast compiled language with a minimalistic
| feature set that scales well to large teams.
|
| Switching to Go from one of those languages is very much
| giving up a kitchen sink for a purpose-built tool. It may
| be the right decision under lots of different
| circumstances, but it doesn't directly compete with any of
| the languages you've mentioned because of how minimalistic
| it tries to be.
|
| Overall, in fact, I think Go does something far more
| interesting: it's legitimately an attempt to carve out a
| whole separate niche for software development. Whether it's
| ultimately been successful there is for a different comment
| thread, though. :)
| obviouslynotme wrote:
| Go has always been a high level language, and is even
| more so now with generics. I would even argue that Go,
| correctly done, is halfway between the Java tier and the
| Lisp tier. Due to Go's quick and easy parsability,
| generation, and compilation. It is very easy to write
| tools for Go. Code generation and struct tags are even
| part of the standard, but very people use them beyond the
| basic serialization libraries. It's not quite lisp, but
| it is very close in many ways.
|
| Very few people rewrite projects. Most change happens by
| new projects adopting one language over another. The fact
| that I hear of Python rewrites to Go is honestly amazing.
| adra wrote:
| Languages generally aren't stuck in time. If we actually want
| languages to evolve and in ways that are actually valuable to
| real people (not a small subset of those developers writing
| it), then these pieces should be an invaluable resource to
| help steer the language in the direction that will most
| reward adoption, longevity, etc.
| bogeholm wrote:
| > Natural Born Killers ... Primus ...
|
| We should have a beer sometime!
| devonkim wrote:
| I think there's a lot of disagreements about what people
| think is desirable in a language in the first place and then
| start complaining without understanding the design concepts
| that shaped its creation nor even agree the problems are even
| problems to begin with. Many even seem to think that bad
| ergonomics are to be accepted and want to create a higher
| barrier for programming productivity (there are some merits
| to this attitude honestly but that's not the point I'm
| interested in).
|
| I think it's more akin to trying to talk to people about
| Quentin Tarantino movies. Lots of people enjoy them, many
| critics like his stuff, and then there's a relatively small
| group of people that disparage everything he does because he
| isn't ascribed to any artistic school of filmmaking and under
| a lot of such critical analysis his films are pop fodder.
| It's not like he ever makes his movies to please critics is
| the thing, he's basically a super fan that just winged it
| all. Many fairly sane and measured, learned critics "get"
| Tarantino and appreciate him for what he tries to do and all
| is fine there. But it won't matter what Tarantino does going
| forward with the other crowd - it's because they have
| diametrically opposed ideas of wtf movies even are supposed
| to be.
| [deleted]
| notduncansmith wrote:
| "There are only two kinds of languages: the ones people
| complain about and the ones nobody uses."
|
| - Bjarne Stroustrup
| zozbot234 wrote:
| Go is a vastly preferable language to Python or Ruby for most
| things. The "negative press" is quite overstated, it does get
| plenty of favorable advocacy as well.
| metadat wrote:
| Use an appropriate tool for the job, Python is useful for
| tons of things that would be a nightmare to pull off in
| golang, and vice versa.
| suremarc wrote:
| Indeed, try doing any sort of data analysis in Go, it's
| maddeningly verbose.
| mountainriver wrote:
| > The libraries for Go aren't as good as for Python- certainly,
| the documentation is lacking
|
| Huh, Go has some of the best autodoc features of any language.
| Also the library assertion is insane to me. I switch back and
| forth from Go to Python all day and generally Go has more stable
| better maintained libs
| Zopieux wrote:
| Sadly I agree with the author on the general feel of Go
| libraries. I can feel the language design itself and non-
| written philosophical identity of Go strongly affects what APIs
| people write in it. Most often, they suck. They feel
| monolithic, unforgiving, and impossible to extend. I don't
| belive we can only blame the lack of generics for that, it runs
| deeper.
|
| On documentation, I've never seen a language community that
| comes with so little examples most of the time. Take any random
| Python library RTD, it's full of it. You can skim through said
| examples to understand the overall features and capabilities
| before you dive into the API reference. No such luck in Go:
| here is the bare-bones godoc, good luck!
| maerF0x0 wrote:
| All the docs are easily found here: https://pkg.go.dev/
|
| The complaint about aws-sdk-go (presumably v1) is legitimate, it
| seems to me that code was generated and therefore the repo can be
| less than ideal w/ Go. I suspect that is improved/ing in v2.
|
| using goimports as your formatter solves the complaint about
| unused imports
|
| There was a significant portion of time where python 2.x and
| earlier releases of Go overlapped, and finding UTF-8 safe python
| libraries was non-trivial. Go makes utf-8 the default. As I
| understand it many companies still have python2 w/o a EoL plan.
|
| One thing I really miss about working with python is list/dict
| comprehensions and lambdas
| kelvie wrote:
| It's still valid for v2. While it's a bit easier to use in
| almost all respects, the documentation is still kind of crazy
| because it's mostly autogenerated.
|
| In Go, the first thing you run into when making an API call is
| figuring out how to handle the errors, and the aws-sdk-go docs
| still don't give you much help there --you have to still dive
| into the official REST docs, or worse, the Java docs via
| google, and sort of guess how each exception/error code gets
| mapped into what Go error.
|
| Jumping into the library code works for _most_ Go libraries,
| but with the autogenerated REST bindings that is the AWS SDK,
| it usually isn 't much more illuminating.
| benhoyt wrote:
| This roughly matches my experience, though he doesn't really
| discuss the language differences in this article. Here's a
| similar kind of article I wrote a few years ago, "Learning Go by
| porting a medium-sized web backend from Python":
| https://benhoyt.com/writings/learning-go/ -- mine focuses more on
| language and standard library differences.
| atwood22 wrote:
| I don't understand how the author struggled to find a good
| DynamoDB library. The official AWS SDK has a very useable
| DynamoDB client and provides utility classes for marshaling and
| unmarshaling Go types: https://docs.aws.amazon.com/sdk-for-
| go/api/service/dynamodb/...
| nathants wrote:
| i dragged my feet on go for a long time. i also thought that
| skipping go and moving to rust was the play. a few years later, i
| still write python often, but i don't build systems with it.
| python i now use like bash, to glue things together and automate
| random things. it's a fantastic language and i will never drop
| it.
|
| the verbosity of go is the biggest hurdle for a pythonista. the
| thought of giving up context managers, decorators, iterators,
| comprehensions, exceptions, coroutines, it's unthinkable. in
| comparison go is ugly. your aesthetic mind screams in protest.
|
| write go full time. dive in. as months pass, not only will those
| aesthetic objections fade, your mental model from python cleanly
| transforms to go. go is what mypy tried to be. the cost was
| aesthetic changes. the benefit is worth it.
|
| the zen of python says if it's easy to explain it might be a good
| idea. this is go, and it is.
|
| i rebuilt a reasonably sized project from python[1] to go[2] over
| the last few years. i also have a system that i maintain both
| python[3] and go[4] implementations for, sharing a test suite in
| python. squint at the implementations. consciously ignore
| aesthetic objections. they are basically the same, not very
| different from a python codebase with and without type
| annotations.
|
| go, like python, is fantastic. use both in whatever amount works
| for you. don't read about them, build with them. you won't regret
| it.
|
| 1. https://github.com/nathants/cli-
| aws/tree/bb78e529e7d1d3f95ac...
|
| 2. https://github.com/nathants/libaws
|
| 3. https://github.com/nathants/s4/tree/python
|
| 4. https://github.com/nathants/s4
| jerf wrote:
| Obviously the closest thing in the general sense to Python is
| Ruby. But I think a case can be made that in practice, the
| closest thing to Python on the static language side is Go. The
| list of superficial differences is a mile long, I don't deny,
| nor will I exhaust my audience's patience with actually writing
| them out. But the interface orientation of Go captures the
| _essence_ of how you design in Python surprisingly well. Of all
| the static languages, Go has the closest thing to duck typing,
| in particular the ability to declare a "duck type" for an
| existing third-party thing that you don't want to or can't
| change, and integrate it into your code as a compile-time
| checked member of that "duck type".
| nathants wrote:
| very true! to be a WriteCloser[1], you have to be able to
| Write[2] and to Close[3]. quacking confirmed.
|
| 1. https://pkg.go.dev/io#WriteCloser
|
| 2. https://pkg.go.dev/io#Writer
|
| 3. https://pkg.go.dev/io#Closer
| vonseel wrote:
| Do Go programmers also not like capitalization? Just a tease, I
| noticed the readme and your comment don't have any Capitalized
| Letters. Well, the readme has a few but you know what I mean.
| nathants wrote:
| i can only speak for myself. i reserve raising my voice for
| where it's APPROPRIATE. as with the python/go debate,
| aesthetics are subjective.
| corrral wrote:
| Breaking basic language norms in this way is the equivalent
| of CONSTANTLY RAISING YOUR VOICE. It's distracting and
| attention-getting. Which may be your goal, but if your goal
| is to be easily understood and sympathetic to your reader,
| it's counterproductive.
| nathants wrote:
| this is exactly correct.
|
| i sincerely apologize to any who have been harmed, even
| slightly, by my oddity.
| tptacek wrote:
| I hated the "damnable use requirement" (the error you get on
| unused imports) for years, but I've been keeping a count of how
| many bugs they've caught (I was surprised the first time this
| happened) and I'm up to 3-4 now. What people who code in Go
| seriously do is just hook `goimports` up to their editor, and
| then never think about this again.
| bobbylarrybobby wrote:
| How is this better than downgrading the error into a warning?
| You'd be able to build but would still know something was
| wrong-ish. Certainly a warning would be sufficient to chase
| down any bugs, and is very freeing compared to an error.
| tptacek wrote:
| I'm not offering any opinion about the use requirement other
| than to say that it has caught 3-4 bugs for me.
| xboxnolifes wrote:
| Turned around, what's the benefit of allowing someone to
| build something that may potentially break, when the
| alternative is an easy to fix error instead?
| bobbylarrybobby wrote:
| The benefit is not needing to make any additional source
| changes when you go from using an import to not using it to
| using it again.
| morelisp wrote:
| goimports seems an endless source of absolutely bizarre bugs
| unless your packages' terminal path element exactly matches the
| package name (even sometimes if it does e.g.
| https://github.com/golang/go/issues/29041).
|
| These do get addressed over time but also seems to break in a
| new way at least a couple times a year.
| ActorNightly wrote:
| >sharing your code is even pain with colleagues even if they are
| using the same operating system, mainly because the Python
| requirement file doesn't pin dependencies,
|
| wat? Pretty sure you can use == in requirements.txt
|
| Also, its very possible, and quite easy to just include the code
| for the library in your package, which effectively "locks in" the
| version. We did this all the time when building AWS lambda
| deployments.
| pletnes wrote:
| Sure can pin versions, it's easy too, if you just use <<pip
| freeze>>.
| pid-1 wrote:
| Now you lost track of which are your top level dependencies
| and which are dependencies of dependencies.
| anyfactor wrote:
| Better option is to use pipreqs.
|
| Virtual Environment gets on my way. Looking at you Jupyter
| notebook.
| ska wrote:
| That only solves part of the problem.
|
| something like poetry's approach is the right one here; you
| need a list of core dependencies (not derived ones), you need
| a solver for when anything changes to find a new set of
| viable versions, and you need a lock file of some sort to
| uniquely & reproducible construct an environment.
| paulsef11 wrote:
| There's lots of cases where you wouldn't want to pin your
| requirements.txt, the main one being if you're authoring a
| package. You need to leave the versions unpinned, preferably
| just bound to a major version, allowing some variability for
| the users of your package in case there's a shared dependency.
| I have a feeling that's what the author is describing here,
| because Poetry solves this dilemma by introducing a poetry.lock
| file which pins the dev versions of all the dependencies, but
| publishes a package with unpinned deps.
| giantrobot wrote:
| The issue is transitive dependencies. A dependency you pin
| isn't guaranteed to pin its own dependencies. A bug somewhere
| in a grandchild dependency can manifest for you even if you
| have a version pinned but the dependency did not.
|
| It's not automatically a problem but it certainly can become
| one.
| linsomniac wrote:
| I'm a huge, long time, python fan. I don't tend to have a lot
| of problems with dependencies, but it clearly is something that
| certain situations have problems with.
|
| As I understand the post, the author is saying "It sure is nice
| to be able to just hand off a go executable and be done with
| it." And I think we can agree that the Python runtime situation
| is far from this.
|
| I largely control my work environment, so this isn't a huge
| issue for me. But I'm right at this very moment converting a
| "python" to "python3" in a wrapper script because the
| underlying code got converted to py3 and can't "import
| configparser". (Actually, I'm removing "python" entirely and
| adding a shebamg).
|
| I've been looking at writing a small tool in Python and then
| porting it to Go (I don't know go, so seems like a reasonable
| way to approach it), because the main target of the app is my
| developers, who probably don't have the right Python
| environment set up, and I just want them to be able to use it,
| not give them a chore.
| alexanderh wrote:
| This is what I came to the comment section to say... You
| absolutely can pin dependencies.... da fudge?
|
| Sounds like this guy needs to finish learning Python before he
| learns something else.
|
| From what you suggested, to containerizing things with
| something like Docker, there are ways to make Python more
| easily distributable.
| anacoluthe wrote:
| What if the depedencies you pinned have non-pinned
| depedencies?
|
| packageA==1.0.0 depends itself on packageB
|
| Therefore, you can find yourself with a different set of
| deps. Had a bug like this once.
| im3w1l wrote:
| Pip freeze will pin explicit as well as transitive
| dependencies
| fnord123 wrote:
| It's a hassle to do this correctly and upgrade the
| dependencies. Use poetry.
| robertlagrant wrote:
| pip freeze > requirements.txt
| shrimpx wrote:
| > Pretty sure you can use == in requirements.txt
|
| You can, but one of those packages that you depend on will have
| a loose version spec for one of _its_ dependencies, making your
| `pip install -r requirements.txt` non-deterministic.
|
| Poetry and Pipenv solve this, though, by pinning all
| dependencies in a lock file.
| ActorNightly wrote:
| Pip freeze will grab all of the versions that are installed.
| AlphaSite wrote:
| I recommend using poetry and just moving on with your life,
| it's far easier than trying to use pip (which is totally
| doable, but not clean).
| LtWorf wrote:
| But poetry requires pip... you can't use it without using pip
| biorach wrote:
| They clearly mean using pip on its own, presumably by `pip
| freeze > requirements.txt`
| gen220 wrote:
| Poetry requires pip in the way that `go mod` requires `go
| get`, i.e. Poetry allows one to operate at a higher level
| of abstraction, where it's harder to make mistakes and
| generally easier to manage your dependency tree.
| ryanianian wrote:
| Kind of. It's intended to be a system package/tool. In that
| regard you can use, wait for it, yet another python tool -
| pipx. So instead of installing poetry to a venv or whatever
| using pip, you can use `pipx run poetry`. Now you have to
| install pipx...
| jeremydw wrote:
| This comment section itself clearly shows how crazy dependency
| and environment management is in Python. In this thread alone,
| we've received instructions to...
|
| - poetry
|
| - "Just pin the dependencies and use Docker"
|
| - pip freeze
|
| - Vendoring in dependency code
|
| - pipreqs
|
| - virtualenv
|
| This is simply a mess and it's handled much better in other
| languages. I manage a small agency team and there are some
| weeks where I feel like we need a full-time devops person to
| just help resolve environment issues with Python projects
| around the team.
| okasaki wrote:
| Sometimes I feel people are using Python very differently
| than me. I just use pip freeze and virtualenv (these are
| Python basics, not some exotic tools) and I feel it works
| great.
|
| Granted, you don't get a nice executable, but it's still
| miles ahead of C++ (people literally put their code into
| header files so you don't have to link to a library), and
| even modern languages like rust (stuff is always broken, or I
| have some incompatible version, even when it builds it
| doesn't work)
|
| By the way if you're a Python user, Nim is worth checking
| out. It's compiled, fast and very low fuss kind of language
| that looks a lot like Python.
| fiddlerwoaroof wrote:
| When I was a Python dev, I never saw that happen in ten
| years or so of work. Pip freeze and virtualenv just worked
| for me.
|
| I will say, though, that this only accounts for times where
| you're not upgrading dependencies. Where I've always run
| into issues in Python was when I decided to upgrade a
| dependency and eventually trigger some impossible mess.
| pdimitar wrote:
| > _and even modern languages like rust (stuff is always
| broken, or I have some incompatible version, even when it
| builds it doesn 't work)_
|
| Been working on and off with Rust for the last 3 years,
| never happened to me once -- with the exception of the
| Tokio async runtime that has breaking changes between
| versions. Everything else always worked on the first try
| (checked with tests, too).
|
| Comparing Python with C++ doesn't make do argument any
| favours, and neither does stretching a single Rust accident
| to mean the ecosystem is bad.
| mcronce wrote:
| This is consistent with my experience. Semantic
| versioning is very very widely used in the Rust
| ecosystem, so you're not looking at breaking changes
| unless you select a different major version (or different
| minor version, for 0.x crates) - which you have to do
| manually, cargo will only automatically update
| dependencies to versions which semver specifies should be
| compatible.
|
| For crates that don't follow semver (which I'm fairly
| certain I've encountered zero times) you can pin a
| specific exact version.
| tempest_ wrote:
| Comparing anything to C++ is a very low bar.
|
| I have rarely encountered issues in rust. Most rust crates
| stick to semver so you know when there will be a breaking
| change. My rust experience with Cargo can only be described
| as problem free(though I only develop for x86 linux).
|
| As for pip freeze and virtualenv things start to fall apart
| especially quickly when you require various C/C++
| dependencies (which in various parts of the ecosystem is a
| lot) as well as different python versions (if you are
| supporting any kind of legacy software). This is also
| assuming other people working on the same project have the
| same python yadda yadda the list goes on, its not great.
| fiddlerwoaroof wrote:
| The issue I've encountered in rust is looking at a
| library and seeing that it requires a nightly build of
| the compiler.
| ksdnjweusdnkl21 wrote:
| I agree that it is handled better in many other languages.
| However, Go has some weird thing with imports going on. When
| I tried to learn it I just could not import a function from
| another file. Some env variable making the program not find
| the path. Many stackoverflow/reddit threads condecendenly
| pointed to some setup guide in official docs which did not
| fix or explain the situation.
|
| After an few hours or so of not making much progress in AOC
| day 1 I just gave up and never continued learning Go.
| Bullfight2Cond wrote:
| one more that supports the latest standards:
| https://pdm.fming.dev/
| SloopJon wrote:
| This has indeed been eye opening. We got bit by a dependency
| problem in which TensorFlow started pulling in an
| incompatible version of protobuf. After reading these
| comments, I don't think that pip freeze is quite what we
| want, but poetry sounds promising. We have a relatively small
| set of core dependencies, and a bunch of transitive
| dependencies that just need to work, and which we sometimes
| need to update for security fixes.
| chpmrc wrote:
| - Poetry is a 3rd party package manager, I'm sure it's great
| but it's not widely used (yet)
|
| - Pip freeze just pins all dependencies at once to
| requirements.txt
|
| - I don't know what "vendoring in dependency code" means
|
| - I've never used pipreqs in my life (and 80% of my work has
| been in Python)
|
| - Virtualenvs are just a convenient way to keep project
| runtimes separated
|
| And for 90% of Python projects in existence the following is
| sufficient (assuming Python3 is installed):
|
| - python -m venv .venv
|
| - source .venv/bin/activate
|
| - pip install -r requirements.txt
|
| That's it. And all of that requires a single dependency:
| Python. Could it be better? Sure. But to call that a "mess"
| is an exaggeration.
| linsomniac wrote:
| "Vendoring" means including the library in your package. So
| instead of listing it in requirements.txt, you copy the
| code of the library.
| korijn wrote:
| Is it a mess? Yes. But, is the problem to be solved perhaps
| much simpler "in other languages"? Do you interface with C++
| libraries, system-managed dependencies, and perhaps your GPU
| in these other languages? Or are all your dependencies purely
| coded in these other languages, making everything simpler?
|
| Of course the answer to these questions could be anything but
| to me it feels like attacks on Python's package management
| are usually cheap shots on a much much more complicated
| problem domain than the "complainers" are typically aware of.
| pdimitar wrote:
| Or the "complainers" work with Rust and Elixir and giggle
| at Python's last-century dependency-management woes, while
| they run a command or two and can upgrade and/or pin their
| dependencies and put that in version control and have
| builds identical [to those on their dev machines] in their
| CI/CD environment.
|
| -\\_(tsu)_/-
|
| Your comment hints that you are feeling personally attacked
| when Python is criticized. Friendly unsolicited advice:
| don't do that, it's not healthy for you.
|
| Python is a relic. Its popularity and integration with
| super-strong C/C++ libraries has been carrying it for at
| least the last 5 years, if not 10. There's no mystery: it's
| a network effect. Quality is irrelevant when something is
| popular.
|
| And yes I used Python. Hated it every time. I guess I have
| to thank Python for learning bash scripting well. I still
| ended up wasting less time.
| Zopieux wrote:
| It's a bit ironic to pick someone's up on "taking things
| personally upon criticism", then proceeding to display a
| deep, manichean, unfounded hatered for a language that,
| despite its numerous flaws, remains a popular and useful
| tool.
| pdimitar wrote:
| Hanging people upon dawn was popular as well; people even
| got their kids for the event and it was happening
| regularly. Popularity says nothing about quality or even
| viability.
|
| Use Python if it's useful for you, obviously. To me
| though the writing is on the wall -- it's on its loooong
| and painful (due to people being in denial) way out.
|
| EDIT: I don't "hate"; it was a figure of speech. Our work
| has no place for such emotions. I simply get "sick of"
| (read: become weary of) something being preached as good
| when it clearly is not, at least in purely technical
| terms. And the "hate" is not at all unfounded. Choosing
| to ignore what doesn't conform to your view is not an
| argument.
| ActorNightly wrote:
| Its not a mess, people just make it a mess because of the
| lack of understanding around it, and getting lazy with using
| a combination of pip install, apt install, and whatever else.
| Also, the problem is compounded by people using Mac to
| develop, which have a different way of handling system wide
| python installs from brew, and then trying to port that to
| Linux.
| takeda wrote:
| Keep in mind that Python is 31 year old (it's even older than
| Java) it was created around the same time as world wide web.
| So it started when no one even knew they would need
| dependency management and evolved over time from people
| posting packages on their home pages, to a central website to
| what we now call PyPI. Similarly the tooling and way of
| packaging the code evolved.
|
| What you described are multiple tools that also target
| different areas:
|
| > - poetry
|
| from what you listed this seems like the only tool that
| actually takes care of dependency management
|
| > - "Just pin the dependencies and use Docker"
|
| this is standard fallback for all languages when people are
| lazy and don't want to figure out how to handle the
| dependencies
|
| > - pip freeze
|
| all this does it just lists currently installed packages in a
| form that can be automatically read by pip
|
| > - Vendoring in dependency code
|
| this again is just a way that applies to all languages, and
| it is still necessary even if there's a robust dependency
| management as there are some cases where bundling everything
| together is preferred
|
| > - pipreqs
|
| this is just a tool that scans your code and tells you what
| dependencies you are using. You are really lost if you need a
| tool to tell you what packages is your application is using,
| but I suppose it can be useful for one offs if you inherit
| some python code that wasn't using any dependence management.
|
| > - virtualenv
|
| this is just a method to have dependencies installed locally
| in project directory instead per system. This was created
| especially for development (although it can be used for
| deployment as well) as people started working on multiple
| services with different dependencies. It's now included in
| python so it's more like a feature of the language.
| feet wrote:
| Even using conda to manage reqs is an absolute nightmare. Did
| a subreq get updated? Did the author of the library pin that
| subreq? No? Have fun hunting down which library needs to be
| downgraded manually
| MonkeyMalarky wrote:
| I used a couple of tricks to solve this. First, make cond
| env export a build step and environment.yml an artifact so
| you've got a nice summary of what got installed. Second,
| nightly builds so you aren't surprised by random package
| upgrade errors the next time you commit code to your
| project.
| MonkeyMalarky wrote:
| Don't forget Anaconda because you're on windows and have no
| idea how to compile random packages that are really C++ code
| with Python bindings!
| jjoonathan wrote:
| Solving environment | Solving environment /
| Solving environment - Solving environment \
| Solving environment | ...
|
| One day it takes 10 seconds, next month it takes 10
| minutes, and the month after that it takes 30 minutes and
| then fails entirely.
| darkarmani wrote:
| Are you using conda-forge? Solving from over 6TBs of
| packages can take quite a while. Conda-forge builds
| _everything_. This isn 't a criticism, but because of
| that the number of packages is massive.
| MonkeyMalarky wrote:
| That's half of why from 2017 to 2021 I had a yearly
| "uninstall Anaconda and start fresh" routine. The other
| half is because I'd eventually corrupt my environments
| and have no choice but to start over.
| jjoonathan wrote:
| Me too. In 2021 it got so egregious that I finally jumped
| ship. Pip or bust.
|
| Since you specifically mentioned 2021 instead of 2022, I
| half suspect you had the same experience.
| punnerud wrote:
| Not a mess, but options. That's what's encourage innovation
| and open up for new ideas.
|
| This Fortran vs Python example is worth reading:
| https://cerfacs.fr/coop/fortran-vs-python
| throwawaymaths wrote:
| Now try installing tensorflow. Treat yourself to ice cream if
| you get it to install without having to reinstall Linux and
| without borking the active project you're on.
| derac wrote:
| The major issues you'll see involve library version
| mismatches. It's a very good idea to use venv with these
| tools since there are often mismatches between projects.
| raffraffraff wrote:
| And unless things have gotten a lot better in the 2 years
| since I last did `pip install numpy` on ARM, prepare for a
| very long wait because you'll be building it from source.
| Gordonjcp wrote:
| It's not crazy at all. You use requirements.txt to keep a
| track of the dependencies needed for development, and you put
| the dependencies needed to build and install a package into
| setup.py where the packaging script can get it.
|
| These are two different things, because they do two different
| jobs.
|
| It's really very simple.
| dagw wrote:
| _Also, its very possible, and quite easy to just include the
| code for the library in your package_
|
| This only works if installing on exactly the same os and
| architecture. It can also make the installer for your quick
| little command line tool hundreds of megabytes.
|
| That being said packing up the python interpreter and all
| dependencies is the approach I ended up using when shipping
| non-trivial python applications in the past.
| iasay wrote:
| That's fine until it wont compile on someone else's machine.
| Had that many times before with python.
|
| SciPy is a bastard on macs for example.
| geraneum wrote:
| On the other hand tools on par with SciPy are not common in
| other languages.
| jossclimb wrote:
| Yep, I have no idea how they do not even understand the basics
| of python dependency management. pip freeze > requirements.txt
| will do it all for you. No wonder they found rust to hard.
| shrimpx wrote:
| Your dismissal of the author is unwarranted.
|
| You can do you want you suggest, but it's an operational pain
| in the ass. You need to maintain two files, the actual
| requirements.txt and the `pip freeze` one that locks the
| environment. And you better never `pip install` anything by
| hand or you'll capture random packages in your frozen file,
| or else always take care to create your frozen file in a
| fresh virtualenv. And if you don't want to install your dev
| packages into the production environment, then you need to
| maintain two requirements.txt and two of those frozen files.
|
| The author mentions Poetry which does solve these issues with
| a nice interface.
| vietthan wrote:
| recommend pipreqs
| fnord123 wrote:
| `pip freeze > requirements.txt` will generate a lock file.
| You want a requirements listing as well as a lock file (like
| rust, poetry, golang, npm, ruby).
| [deleted]
| syassami wrote:
| You can go even further now with PEP508
| https://peps.python.org/pep-0508/
| Areibman wrote:
| > I especially struggled with the DynamoDb library for Go; so
| much so I wrote a Python script to query Dynamodb, and called it
| from Go.
|
| This seems like a compelling enough reason to stick with Python
| atwood22 wrote:
| The official AWS SDK contains a perfectly good DynamoDB client,
| especially when combined with this utility library (also part
| of the AWS SDK): https://docs.aws.amazon.com/sdk-for-
| go/api/service/dynamodb/...
| wodenokoto wrote:
| I was kinda hoping an article with a headline like that would
| present some code and tooling and explain it in terms of how a
| Python programmer sees the world.
| barefeg wrote:
| How to combine both languages as he suggests? I.e. creating
| python binds for the go binary? What about the opposite?
| arccy wrote:
| you could do something like
| https://www.datadoghq.com/blog/engineering/cgo-and-python/
| kitd wrote:
| Nim operates in the same space as Go, but with a syntax very
| much in the spirit of Python.
|
| https://nim-lang.org/
| pojzon wrote:
| About unused imports, cannot you simply use formatter on build
| hook ?
| travisgriggs wrote:
| I realize that Go's original release is later than Python's, but
| they still feel roughly the same vintage to me. Usually, these
| articles have a "old to newer" feel, but these two ecosystems
| feel just different to me. Which is not bad.
|
| What would be fascinating is to see some HN links to articles
| where the journeyman programmer went backwards in time, e.g.
| "Learning Fortran as a Python developer" or something like that.
| linsomniac wrote:
| TL;DR: Python is hard because of library dependencies, go is
| better about that because of compiling to an executable, but is
| pickier about code quality ("I just want to try something"), and
| has more issues with library quality.
| LtWorf wrote:
| I've encountered input libraries in go tha for some reason just
| grab all the signals and whatnot, so you can't ctrl+c, sighup,
| sigterm, use ctrl+z to return to the shell... the only option
| is to do a kill -9 from a different shell.
|
| Great library indeed :D
| linsomniac wrote:
| Maybe there's a go library that implements that "kill but
| make it look like an accident" functionality on HN a few days
| ago? :-)
| bumper_crop wrote:
| > I hate how it forces to you create a module everytime.
|
| This affected me recently, so I have sympathy for the author.
| Trying to upgrade an older project I had to the module system
| meant trying to find out how to import modules which don't have
| reachable URLs and were only on the GOPATH. At I hate how it
| forces to you create a module everytime. some point programming
| in Go stopped being for fun.
| brodouevencode wrote:
| Organizing a golang-based project is also something that, while
| documented, is not front of mind for most new golang users, nor
| does the "beginning go" posts out there do a good job of how to
| lay out a project for success.
| morelisp wrote:
| Is Python significantly better in this regard? I don't think
| so, especially with the differentiation between modules which
| are directories with an __init__.py and modules which are files
| you import directly but if not in the same directory _also_
| still need an __init__.py which has tripped up probably 80% of
| the people I try to teach Python to.
|
| In Go you can at least get pretty far with a totally flat
| namespace and _there 's nothing wrong with that_, up to at
| least 50kloc or so. That's less true in any language where
| files become a unit of modularization.
| DandyDev wrote:
| Directories with an __init__.py are not modules, they are
| packages. Modules are .py files
| morelisp wrote:
| The __init__.py is also itself a module, which you import
| by importing the package. That's confusing, and not easily
| avoided!
| LtWorf wrote:
| Make it an empty file?
| Scarblac wrote:
| > modules which are files you import directly but if not in
| the same directory also still need an __init__.py which has
| tripped up probably 80% of the people I try to teach Python
| to.
|
| Not necessary anymore since Python 3.3 (released in 2012).
| tandav wrote:
| Still necessary for mypy (if you want type checks) and
| setuptools (if you want to publish library to pypi)
| morelisp wrote:
| Thanks, I went from a lifetime of Python 2.7 straight to
| 3.8 and missed this.
| kccqzy wrote:
| With python you can also go pretty far without __init__.py
| IMO. Just put everything in a single directory.
| morelisp wrote:
| Beginners trying this are still liable to make import
| cycles, which... sort of work? Depending on what you do
| with them? I feel like Go and Java have much better (and
| different) answers here; I wonder what a Python equivalent
| of the Java style would look like.
| jamal-kumar wrote:
| I always felt that the official documentation is pretty
| excellent (Worth a re-read if you haven't in a while):
|
| Code organization: https://go.dev/doc/code
|
| Pretty much everything else you should know:
| https://go.dev/doc/effective_go
|
| There's one thing I think that people getting started with this
| language should know and that's in the pursuit of getting the
| stuff you mentioned right, is making sure the reference
| material (blog post, book, whatever it is) you're reading is
| something written within the past two or three years. The
| official site really has enough that you need to know in order
| to have good knowledge coverage, but the module system beyond
| version 1.16 especially is something you need to get right and
| which a ton of old information is out of date on:
|
| https://go.dev/doc/modules/developing
| Uptrenda wrote:
| I don't think Python packaging is as bad as people make out (if
| packages only stick to the basic features of Python!) A far
| bigger issue I see is despite Python supposedly being 'cross-
| platform': the language introduces so many small, backwards-
| compatibility breaking changes that you really need to use the
| most up to date interpreter you can.
|
| For instance: there is now a really cool operator that lets you
| do expressions where you assign a value and evaluate the result
| in one line. So you can write the C equivalent of something like
| while(buf = recv(...)) ... and it will work. Well, any program
| that uses this new feature won't run on anything but the most
| bleeding edge Python versions. Despite a program only needing a
| small fix to remove such expressions. It wouldn't run at all.
|
| I think the Python interpreter needs to be 'smarter' and add the
| capability to automatically install parallel Python versions if
| it detects a package using a more recent interpreter. Would
| honestly solve so many issues.
| [deleted]
| loosescrews wrote:
| While the documentation of many open source Go libraries is
| definitely lacking, I find that the combination of types and
| links to readable source in the generated docs more than make up
| for it. While the documentation for Python libraries is often
| better, it is generally much harder to answer questions not
| answered by the docs yourself.
| akpa1 wrote:
| I agree with this. I also derive quite a lot of value from
| having a consistent way of reading and navigating documentation
| for every library too. I can have an applet in my browser's
| toolbar and hit it when I'm looking at the Git repo and it'll
| almost always take me to the right place to ser the docs.
| JustSomeNobody wrote:
| On the subject: What is a good, idiomatic, Go project to study?
| Something big enough to be a good example of code organization,
| yet small enough not to become a new hobby.
| erik_seaberg wrote:
| Tooling aside, Python is a more powerful language, so I'm
| wondering whether he started generating code to work around stuff
| like try/except and defaultdict and itertools and decorators.
| vippy wrote:
| aidenn0 wrote:
| I only use python casually, but it seems to me that python
| library authors are always refactoring and breaking interfaces.
|
| Recently, there was a bug in a second order dependency, and the
| version that fixed this bug was after a version that moved a
| function into a submodule.
|
| So I had to make a local patch to my dependency that changed all
| of the imports.
|
| Was the new interface more consistent? Yes, but could they have
| left a shim for backwards compatibility, or just lived with the
| old slightly less consistent interface? Seems to me like they
| could.
| takeda wrote:
| I think that depends on library. The ones I use seem to use
| appear to not do that. In fact I think I actually have seen
| that more in Go. What goes for Go though is that you won't
| compile your code until you fix it. In Python you could use
| mypy, but unfortunately is optional and will work only if you
| and library author uses annotations.
| aidenn0 wrote:
| I don't use Go at all, but TFA was concerned with pinning
| dependencies, and stuff like this makes dependency pinning
| much more important
___________________________________________________________________
(page generated 2022-07-18 23:00 UTC)