[HN Gopher] Keeping your Go modules compatible
___________________________________________________________________
Keeping your Go modules compatible
Author : stargrave
Score : 88 points
Date : 2020-07-07 17:15 UTC (5 hours ago)
(HTM) web link (blog.golang.org)
(TXT) w3m dump (blog.golang.org)
| poorman wrote:
| > When you run into a case where you want to add a method to an
| existing interface, you may be able to follow this strategy.
| Start by creating a new interface with your new method, or
| identify an existing interface with the new method. Next,
| identify the relevant functions that need to support it, type
| check for the second interface, and add code that uses it.
|
| I like this pattern, and I have used it. The issue I have is when
| I look at the generated godoc, the original interface is in the
| signature. I don't know the function accepts the new interface as
| well without either digging into the code or a comment for the
| function.
| candiddevmike wrote:
| Reading these blog posts, you get see Go's simplicity start to
| unravel with modules. Uncomparable structs? Type checking
| workarounds? You start to wonder if these bandaids would be
| necessary in a generics world.
|
| For the record, I'm indifferent on the chosen module solution
| being right or wrong. My ideal module tool is something like npm,
| but it shows me the impact: diff the module changes, look at my
| code, and tell me what's impacted by the upgrade. Giving
| developers more information when upgrading libraries will go a
| long way towards making "update your libraries" a regular
| housekeeping task.
| merb wrote:
| btw. this blog is about binary/source compatibility, which is a
| fucking hard problem in nearly EVERY language. (maybe not in
| javascript tough)
| Ericson2314 wrote:
| Gah in every language I want that interface comparison tool,
| and I keep on being disappointed.
| Ar-Curunir wrote:
| rust-semverver does what you want for Rust, I think?
| jbamsterdam wrote:
| Until gorelease is ready, you can try apidiff (https://go.goo
| glesource.com/exp/+/refs/heads/master/cmd/apid...).
| jrockway wrote:
| Modules have nothing to do with comparable structs. It's just
| that making a comparable struct uncomparable could break users
| of your API, so it's something to be on the lookout for if you
| value backcompat. (The module system ensures that users will
| get the version of the code they want; you only have to worry
| about backwards compatibility if you expect users to upgrade to
| your new version.)
|
| There are millions of ways to break users with an update that
| machines can't detect. For example:
|
| Old version:
|
| func Lt(a, b YourType) bool { return a < b }
|
| New version:
|
| func Lt(a, b YourType) bool { return a >= b }
|
| This is not specific to Go.
| jbamsterdam wrote:
| Despite the title, the problems that this article discusses
| could have happened even before modules were introduced. Before
| modules, your only choices were to stay compatible or break
| users. Modules give us the freedom to make breaking changes in
| a safe, documented way.
| mseepgood wrote:
| > Reading these blog posts, you get see Go's simplicity start
| to unravel with modules. Uncomparable structs? Type checking
| workarounds? You start to wonder if these bandaids would be
| necessary in a generics world.
|
| Nothing of this has to do with or would be better with
| generics.
| naringas wrote:
| there's something that I just don't get about golang modules, I
| don't know what it is but they don't click with me
|
| it might well be something tacit that golangers learn from each
| other directly? or otherwise it's so obvious and simple and I
| just haven't realized?
|
| it feels like it's the kind of thing that just clicks one day
| possibly after a few years
| ArmandGrillet wrote:
| IMO the documentation isn't there yet, it lacks a Q&A webpage
| or something short and giving the main how-tos in a succinct
| manner: How to create a module? How to bump a package version?
| How to use a specific package's branch? What to do if there is
| a conflict because a package used by another package messed up
| its compatibility?
|
| The current first Google result when looking for "go modules"
| is https://blog.golang.org/using-go-modules which is a very
| verbose post starting with "This post is part 1 in a series of
| 5 parts". Not very engaging.
|
| In one year I've accumulated a few snippets that I use
| regularly and get used to `go mod` but it was painful at the
| beginning. I wish I could have contributed to this topic but Go
| is most of the time a language that only requires to read
| pkg.go.dev and the main blog posts to know what's next and this
| Go modules workflow is, like gopls, something that doesn't
| currently have a good place to be documented AFAIK.
|
| Edit: Thaxll's comment gives a very nice URL to learn about Go
| modules, still too much for a beginner IMHO but it has a Q&A
| part that goes very deep.
| [deleted]
| m0th87 wrote:
| You're not alone. I've been writing go since before 1.0, and
| can handle modules/packages in _any other language_ fine, yet
| go's regularly leaves me stumped.
|
| A little while ago we were hitting yet another issue with
| dependencies related to version pinning at work. I tried to
| divine what the disparate versions mean, and came up with this
| flow chart from a legalistic reading of stack overflow issues
| [1] and the go modules doc [2]: has the package
| opted into go modules (i.e. is there a go.mod?) yes
| does the repository use semver? yes
| is the major version <= 1? yes
| => normal versioning, e.g. github.com/stretchr/testify v1.3.0
| no => when importing, we need add /vN at the
| end, e.g., import "github.com/my/mod/v4", and in go.mod it will
| behave like github.com/my/mod/v4 v4.1.0 no
| => pseudo-versioning will be used, e.g. github.com/kolo/xmlrpc
| v0.0.0-20190717152603-07c4ee3fd181 no does
| the repository use semver? yes is the
| major version <= 1? yes =>
| pseudo-versioning will be used, e.g. github.com/kolo/xmlrpc
| v0.0.0-20190717152603-07c4ee3fd181 no
| => version will be marked as incompatible, e.g.
| github.com/zeromq/goczmq v4.1.0+incompatible no
| => ???
|
| I still don't know if this is right. But that's not really the
| point. I think the go authors painted themselves into a corner
| with such an anemic initial release. Now that they're trying to
| address shortcomings, the complexity is blowing up to handle
| the very large/diverse ecosystem. It reminds me a lot of java
| around 1.4 when they added in generics. In the long-run it was
| the right move, but man was it painful.
|
| 1: https://stackoverflow.com/a/57372286 2:
| https://github.com/golang/go/wiki/Modules
| fourseventy wrote:
| I agree. I used godep before and I thought it was intuitive and
| had no issues with it. But I still struggle with gomodules
| Thaxll wrote:
| It's pretty clear and well explained there:
| https://github.com/golang/go/wiki/Modules
|
| On a day to day operation, you need to know 2/3 commands that's
| it.
|
| - go mod init
|
| - go mod tidy
|
| - go get xxx
| opqpo wrote:
| Go modules and the entire package management and versioning
| system is just embarrassingly horrible. It looks like it was
| designed by people who have no background in software engineering
| at all. The language as a whole despite being now universally
| used as the backbone of cloud native, it honestly looks like it's
| intentionally designed to be horrible and insulting for anyone
| with sensible mind. Every obvious right decision looks like it's
| intentionally being replaced by a bad one if it's the right path
| is easier.
| hinkley wrote:
| Something I knew starting out, forgot, and learned again, is
| that there's a huge liability to spending your entire career
| working for one company.
|
| There will be things they do so well they aren't even on your
| radar, and you will have a huge blindspot there. And there are
| things they tolerate that pretty much nobody else will. So you
| will either suffer in ignorance or learn some nasty, nasty
| habits.
|
| Without moving you see no diversity, and without seeing
| diversity you shouldn't be put in charge of building _anything_
| for 'everyone'. You are only qualified for niche solutions,
| which unfortunately outsiders might not clue in on until
| they've invested heavily in your ideas.
| Jtsummers wrote:
| In college we referred to this same idea (among my peers who
| were going on through graduate studies, particularly PhDs) as
| "academic incest", where you aren't sufficiently exposed to a
| diverse enough set of ideas and end up too narrowly focused
| or clouded in your judgment.
|
| Ways to overcome it include: moving (in academics, accepting
| a fellowship at other institutions; as a professional, change
| jobs); observation (keeping abreast of the latest trends
| outside your immediate environment and field); community
| participation (attending conferences, symposia, and similar
| things; or even forums like this).
|
| But ultimately it requires a recognition that your views,
| knowledge, and beliefs are, necessarily, constrained by your
| experiences and that you sometimes have to seek out novel
| experiences or the experiences of others to change yourself.
| If you lack the drive for either, or too great a confidence
| in your own present state, then you're going to be stuck.
| Ericson2314 wrote:
| That's true, but Google is big and they use many languages,
| and the amount of ignorance needed to make Go cannot be
| explained by that alone.
| mseepgood wrote:
| > Go modules and the entire package management and versioning
| system is just embarrassingly horrible. It looks like it was
| designed by people who have no background in software
| engineering at all.
|
| Go modules is probably the most well-wrought versioning system
| for a programming language. The amount of thought that went
| into it is insane.
|
| https://research.swtch.com/vgo
| littlestymaar wrote:
| It's the most _over-though_ versioning system ever actually.
| It 's a rare mixture of NIH syndrome and obsession to a
| specific theoretical concern. Yes semver dependency
| management is NP-hard[1], but lawn mowing is NP-hard too, and
| yet that doesn't seem to stop anyone else!
|
| [1] in fact it's only NP-hard if you add an optional
| aditionnal constraint: that every dependency must have a
| single instance even if it's a transitive dependency from
| several dependencies with conflicting requirements. If you
| allow different major version of the same dependency to
| coexist in the dependency tree, it becomes a straightforward
| linear problem while still behaving like every single other
| package manager in existence for the end-user. Now that's a
| well thought system! (And there at least one language, pretty
| popular these days on Hacker News, which does exactly this)
| LukeShu wrote:
| Go does allow coexisting major versions, just not
| coexisting minor versions.
| jacques_chester wrote:
| Go mod is a gigantic pig's breakfast. It would work _OK_ if
| everyone was on it already, but everyone is _not_ on it
| already.
|
| If I took a survey of every go.mod file and looked at the
| versions, more than half would still be that absurd, un-
| eyeballable, merge-review-hostile, another-arbitrary-c-
| hacker-mini-language, shas-upon-shas version string that the
| "insane amount of thought" landed on because it assumes
| everyone is already conformant. The ugly corner case for
| "unversioned" dependencies is the _main_ case.
|
| The "insane amount of thought", as is the norm for the Golang
| team, involved a gymnastic demonstration to evade (1) any
| serious consideration of prior art _in use_ whatsoever, and
| (2) entertaining not even the slightest concept of how shitty
| it would be to impose massive externalities on everyone else.
|
| Even an apologetic tone would have made the pill less bitter.
| Or better yet, blessing godep. Or copying Maven wholesale. Or
| Bundler. I don't care, because what happened has vapourised
| millions of developer hours chasing down completely
| unnecessary versioning issues.
| ithkuil wrote:
| It's a delicious breakfast but, served for dinner.
|
| I love the mechanics of it. Most of what you describe is
| the result of having waited ages before implementing it,
| and the resulting prolonged migration period, exacerbated
| by the relatively decent (but not perfect) backward
| compatibility with pre-modules code.
|
| Prior art isn't perfect either: I'm happy somebody tried
| something different from centralized naming systems or
| almost-but-not-quite federated artifact collections (such
| as the various maven repos). There is already a global
| system to get hold to unique names: the domain name system.
| For better or worse, that's a solved problem, let's not
| create new naming arbiters
| jacques_chester wrote:
| > _Most of what you describe is the result of having
| waited ages before implementing it_
|
| Except we already had godep. It worked. It was in
| widespread use.
|
| > _exacerbated by the relatively decent (but not perfect)
| backward compatibility with pre-modules code._
|
| My experience has been that the backwards compatibility
| has been a nightmare. It forces itself on any downstream
| repository, whether or not that repository wishes to
| participate. Plus it does very poorly on diamond
| dependencies, giving error messages about as scrutable as
| the I Ching. If your answer is "they're doing it wrong",
| take it up with the Kubernetes and go-client maintainers,
| because I can't change what they are doing.
|
| > _There is already a global system to get hold to unique
| names: the domain name system. For better or worse, that
| 's a solved problem, let's not create new naming
| arbiters_
|
| Versions are not addresses. Addresses are not versions.
| You may find a version _at_ an address, but addresses are
| not versions. Docker did the same thing with image
| references and they too created a pig 's breakfast. The
| difference was that they didn't have prior art to compare
| to and it was an oversight obvious largely in hindsight.
| I do not extend the same credit to Russ Cox.
| ithkuil wrote:
| > Except we already had godep. It worked. It was in
| widespread use.
|
| > un-eyeballable, merge-review-hostile, another-
| arbitrary-c-hacker-mini-language, shas-upon-shas version
|
| godep shared the same problem. We can argue that godep
| was good enough and the Go maintainers handled the
| situation poorly with this whole "ask community and then
| ignore the community" trick. But you're focusing on the
| transition; if this system had been part of Go 1.0, I
| would have had nothing bad to say about it.
|
| I'm not arguing the the transition process wasn't a mess.
| I'm arguing that it became a mess precisely because they
| wanted to attempt backward compat. They could have said
| "this is go 2; if you want to use your library with go 2
| you have to use this standard, tag your release, or
| upload it to some central index"
| jbamsterdam wrote:
| Do you have any specific examples in mind?
___________________________________________________________________
(page generated 2020-07-07 23:00 UTC)