[HN Gopher] A new Go API for Protocol Buffers
___________________________________________________________________
A new Go API for Protocol Buffers
Author : zdw
Score : 87 points
Date : 2020-03-02 20:21 UTC (2 hours ago)
(HTM) web link (blog.golang.org)
(TXT) w3m dump (blog.golang.org)
| cube2222 wrote:
| Congrats on the big release!
|
| One thing which saddens me a lot though, is returning types you
| have to type assert because they had to avoid an import cycle. I
| don't know if it could be avoided (probably not) but it's still
| unergonomic.
|
| > (Why the type assertion? Since the generated descriptorpb
| package depends on protoreflect, the protoreflect package can't
| return the concrete options type without causing an import
| cycle.)
| chairmanwow1 wrote:
| This article doesn't do a great job explaining exactly what new
| features are a part of this release. Seems to me like it's
| finally enabling some of the useful type iteration stuff that is
| possible in other versions of protobufs.
| sigzero wrote:
| I thought the general consensus on protocol buffers was "Meh"?
| myko wrote:
| I think for large organizations they probably make sense.
| They're potentially a lot less data per request than JSON.
|
| For the average app? Overkill, sure.
| xmprt wrote:
| I also recently learned about grpcurl that makes using
| protobufs a lot easier. Still not as easy as json but I think
| protobufs are the future.
| untog wrote:
| You thought wrong?
| capableweb wrote:
| I'm wondering if the whole idea behind "general consensus"
| really work on technical problems. There are so many factors
| involved with the choices engineers make that you can't say
| that general consensus says it's "meh" as it really depends on
| your use case. No tool will ever fit all general problems, so
| will always be "meh" for someone.
| dgellow wrote:
| > The github.com/golang/protobuf module is APIv1.
|
| > The google.golang.org/protobuf module is APIv2. We have taken
| advantage of the need to change the import path to switch to one
| that is not tied to a specific hosting provider.
|
| That's such a weird choice, and will be quite confusing. Now
| every time you see protobuf being imported, you have to be sure
| to pay extra attention to the domain used, and correctly remember
| which one is v1 or v2. If only go modules established a clear way
| to differentiate API versions using the import path!
| icholy wrote:
| > If only go modules established a clear way to differentiate
| API versions using the import path!
|
| It did, they just chose not to follow it :/ I wonder what rsc
| thinks about this decisions
| giancarlostoro wrote:
| Would be nice if they would just make packages.golang.org sorta
| how Python has PyPI, Rust has Crates.IO and so on... They can
| just redirect to relevant repos...
| [deleted]
| alkonaut wrote:
| Then you see the version imported being v1.X where X>20 and of
| course that makes it obvious that it's v2?
|
| Are version numbers 2.0 and higher explicitly banned?
|
| Edit: found an explanation further down (spoiler: not a very
| good one but a very go one).
| kardianos wrote:
| The goal of issuing a major version is to create a separate
| package namespace so it can be imported concurrently to other
| major versions. By treating major versions as different
| packages, it creates efficiencies in VCS history and common
| namespace.
|
| However, the goal is to create a new package name that can be
| concurrently imported with previous versions. Everything else
| is just method. Don't confuse the goal with the method.
| zdw wrote:
| Interesting that even Google decided to do complex versioning
| gymnastics and introduce a new namespace in order to avoid how
| `go mod` adds the version as a path component.
| neild wrote:
| Nah, we just really wanted to stop using an import path tied to
| a specific hosting provider. Once we decided to change to an
| entirely different path, we waffled a lot over whether to tag
| it v1 or v2. There were arguments for and against either, so we
| eventually picked one and went with it.
| endorphone wrote:
| "we just really wanted to stop using an import path tied to a
| specific hosting provider"
|
| e.g. github is now owned by Microsoft and it's a little
| awkward.
|
| If Google won out on their attempt to buy Github, they
| wouldn't wouldn't be talking about a "specific hosting
| provider".
|
| Though it's a little weird that people take this at face
| value, when the majority of the userbase was questioning this
| since the first release.
| jzwinck wrote:
| Sure, but why version the new package as v1.20 instead of
| v2.x?
| farslan wrote:
| Because they would had to add a `/v2` suffix to the import
| path and seems like they didn't wanted to add that. The
| question is, why is this not released as v1.0.0 then? Still
| trying to understand the reasoning for this.
| jzwinck wrote:
| Oh I didn't realize that was mandatory. I thought
| omitting v2 from the path was sort of OK until you
| explained this. Now it just seems wrong.
| neild wrote:
| Reasoning went something like this:
|
| We could tag the new API v2:
|
| + Makes the "v1" and "v2" distinction very clear in the
| import path.
|
| - Confusing: google.golang.org/protobuf@v1 doesn't exist,
| but v2 does.
|
| - In ten years, hopefully nobody cares about the old
| github.com/golang/protobuf and the confusion is gone.
|
| We could tag the new API v1:
|
| - Less visually distinct in the import path.
|
| + Seems to make sense for the first version of
| google.golang.org/protobuf to be a v1.
|
| + If we decide it was a terrible idea, it's easier to go
| from v1 to v2 than to roll back from v2 to v1.
|
| We waffled back and forth for quite a while on this, and
| eventually decided that the first version of
| google.golang.org/protobuf would be a v1. Then as we got
| closer to release (but with a certain amount of usage of
| v0 in the wild), we decided not to second-guess that
| decision but to start with a version that wouldn't
| overlap with any version of github.com/golang/protobuf to
| avoid confusion when someone reports a bug in "v1.0.1".
|
| Maybe it was the wrong choice. If it was the worst choice
| we've made in the new API, I'll be happy!
| bouk wrote:
| You could give google.golang.org/protobuf@v1 an
| //importpath to the github package, so go would warn
| people about it.
| farslan wrote:
| Thanks for the all the work. And I hope my rant doesn't
| come something as a bad. But I truly believe the
| versioning along with the import path is confusing. I
| hope the team goes over the versioning again and come up
| with something that makes more sense.
| tsimionescu wrote:
| > The google.golang.org/protobuf module is APIv2. We have
| taken advantage of the need to change the import path to
| switch to one that is not tied to a specific hosting
| provider.
|
| This seems like a good example why tying the name of the
| package in the code with the place where it can be
| downloaded was a very bad idea. Simply changing hosting
| providers is now a backwards compatibility break for your
| clients.
|
| If Go imports didn't work this way, you could have simply
| offered the old v1 from both github.com and
| google.golang.org, and used a clear v2 for the v2. Sure ,
| you would have similar problems if you wanted to change
| the logical package structure for some reason (say, when
| an open-source package moves to a different
| organization), but that is a much rarer case than
| switching code repos.
|
| However, given that that ship has probably long sailed,
| you probably picked the right choice.
|
| I would also note that v2 in the import path feels a bit
| strange, since it makes users of the newest code need to
| know about the history of the package, but there are also
| clear advantages which probably out-weigh this (I believe
| this is absolutely not the case for the decision to
| include the hosting provider in the import path).
| neild wrote:
| Indeed, tying the name of a package to a hosting provider
| is a bad idea! That's why we changed to an import path
| which isn't tied to one. google.golang.org/protobuf is
| not tied to any particular provider, and we can redirect
| it wherever we want.
|
| See "go help importpath" for details on how this works.
| rolleiflex wrote:
| I think you're overestimating how much people pay
| attention to the body part of the import paths. Since
| there have been a quite few projects that implemented a
| sort of redirect (i.e. 'vanity' imports [0]) most
| people's eyes just glaze over the small change in the
| URL.
|
| I would strongly suggest using /v2 at the end. /v1 not
| existing on *.golang.org is not a problem in practice,
| but if it ends up one you could just set that up as a
| vanity import pointing to github.com for the old version.
|
| [0] https://sagikazarmark.hu/blog/vanity-import-paths-in-
| go/
| llimllib wrote:
| When I read the article, my brain passed right over the
| different domain names; the two are even the exact same
| number of characters, set in a monospace font, and my
| brain passes right over the domain of golang imports to
| the end of the path. That's where the interesting bit is
| (usually!)
|
| I find this a super confusing and unintuitive choice, but
| I suppose it's too late for changing it now.
|
| Sincerely, an othwerwise mostly happy protobuf user
| rverghese wrote:
| What are you going to do for APIv3?
|
| Version it as google.golang.org@v2? google.golang.org@v3
| (but @v2 doesn't exist)? Release an @v2 which is
| identical to @v1?
| neild wrote:
| We used up our breaking change budget for the next decade
| with this release, so we'll think about it in 2030.
| ifoundthetao wrote:
| That makes the most sense, and throws a great seasoning of
| context on that decision. Thanks for providing that here.
___________________________________________________________________
(page generated 2020-03-02 23:00 UTC)