[HN Gopher] How to start a Go project in 2023
___________________________________________________________________
How to start a Go project in 2023
Author : taylorbuley
Score : 262 points
Date : 2023-05-23 16:31 UTC (6 hours ago)
(HTM) web link (boyter.org)
(TXT) w3m dump (boyter.org)
| kortex wrote:
| I see $GOPATH is no longer strictly needed, which is nice. But
| what's the deal with $GOROOT? I seem to always need it set but I
| don't know if that's just my workflow or force of habit.
| typical182 wrote:
| People should essentially never set GOROOT. It's mostly a
| holdover. For example, Ian Lance Taylor on the core Go team
| wrote [0] somewhat recently:
|
| "It's a special purpose hook that is almost never needed".
|
| [0] https://groups.google.com/g/golang-
| nuts/c/qDhJbkE1QeY/m/JoV2...
| liampulles wrote:
| I use Makefiles for my projects, and right at the top I do:
|
| GOBIN := $(shell go env GOBIN)
|
| And then use that var where needed.
| BarkMore wrote:
| It's either habbit or you are doing something out of the
| ordinary with your Go installation. The standard installation
| of Go has not required GOROOT for a very long time.
| tptacek wrote:
| Go is pretty mainstream now, so if your editor is mainstream too,
| just keep using it and add its Go support; you don't need to pick
| a Go-optimized editor. The basic thing I think you really want is
| something like `goimports` (which you can just install), which
| automatically manages your imports for you and takes out 80-90%
| of the pain of Go's library usage policy.
| cratermoon wrote:
| Note that statically linked go binaries work in a docker image
| from scratch. This can be created with a multi-stage build where
| the builder image uses whatever OS you prefer, with any required
| packages. A build line such as RUN
| CGO_ENABLED=0 go build -o mybin -ldflags '-extldflags "-static"'
| -tags timetzdata
|
| And the second stage like FROM scratch
| COPY --from=app-builder mybin mybin ENTRYPOINT ["/mybin"]
|
| The builder can create users and groups, and the final image can
| import necessary certs like so: COPY
| --from=alpine:latest /etc/ssl/certs/ca-certificates.crt
| /etc/ssl/certs/
| rollulus wrote:
| Pretty good article. One comment: it recommends zerolog for
| logging, but recently slog [1] has started to become part of the
| standard lib. I guess it's the future.
|
| [1]: https://pkg.go.dev/golang.org/x/exp/slog
| arp242 wrote:
| I used slog for a project a few months ago; then I stopped
| working on it and continued on it a few weeks ago and there
| were all sorts of incompatible changes.
|
| That's completely fair; it's still in development so this isn't
| a complaint! But just saying, at this point you need to be
| prepared to have to deal with that.
| xh-dude wrote:
| FWIW: slog has been pretty stable for a month or two, and
| should be officially standard library in go1.21
|
| There was a last round of changes mostly revisiting use of
| contexts a few months ago - hats off to jba for taking a lot
| of time to work out the best fit
| liampulles wrote:
| I use zerolog at work - I can vouch that it does the job just
| fine.
| jzelinskie wrote:
| Things I can't live without in a new Go project in no particular
| order:
|
| - https://github.com/golangci/golangci-lint - meta-linter
|
| - https://goreleaser.com - automate release workflows
|
| - https://magefile.org - build tool that can version your tools
|
| - https://godoc.org/github.com/ory/dockertest/v3 - run containers
| for e2e testing
|
| - https://github.com/ecordell/optgen - generate functional
| options
|
| - https://golang.org/x/tools/cmd/stringer - generate String()
|
| - https://mvdan.cc/gofumpt - stricter gofmt
|
| - https://github.com/stretchr/testify - test assertion library
|
| - https://github.com/rs/zerolog - logging
|
| - https://github.com/spf13/cobra - CLI framework
|
| FWIW, I just lifted all the tools we use for
| https://github.com/authzed/spicedb
|
| We've also written some custom linters that might be useful for
| other folks:
| https://github.com/authzed/spicedb/tree/main/tools/analyzers
| donatj wrote:
| Cobra is just so... intense and complicated. It has it's place
| but Google's Subcommands is enough for 99.9% of projects I've
| worked on
|
| - https://github.com/google/subcommands
| azemetre wrote:
| hmm I recently just finish making a CLT with cobra and it
| wasn't too bad. Granted it's extremely basic but I like it
| for my use case (cloning all the repos in an org):
|
| https://github.com/shimman-dev/piscator
|
| What did you find complicated about it? What I struggle with
| is creating man pages automatically, although I did just find
| where in the repo this is explained.
|
| subcommands does look neat tho, I'll likely use it for
| another tool I have in mind.
|
| One thing I did not realize is that cobra and
| charmbracelet/bubbletea are not compatible. It may be my
| inexperience in making CLTs vs TUIs but I was disappointed I
| could use say the loading spinner from bubbles easily in my
| tool (opted for briandowns/spinner instead).
| icholy wrote:
| I like https://github.com/urfave/cli
| jacurtis wrote:
| I've built many CLIs with Cobra and haven't found it all that
| intense. I've built incredibly simple, single function CLIs
| up to some incredibly advanced CLIs that the entire company
| relies on daily.
|
| I like Cobra because it gives you a great place to start,
| with tons of stuff "for free". Things like spellcheck on
| commands like if you type "mycli statr", you might get a
| response "mycli statr command not found. Did you mean mycli
| start?". This is out of the box. I don't do a single thing to
| create this functionality. Really nice help pages, automatic
| --help flags with autodocumentation all comes for free, by
| just running a simple init script to start the project. It
| speeds up my ability to make a reliable CLI because I don't
| need to worry about alot of the little things, I basically
| just create the command I want, choose the flags, and then
| start writing the actual code that performs the task and
| don't have to write very much code to manage the tool.
|
| I usually organize my project so all the Cobra stuff is in
| the main module. Then I write my own custom module that
| contains the application code that I am building. It builds
| great seperation between the two. The main module just has
| cobra setup, flags, configuration, documentation, etc.. Then
| for each command, all I do is call a function from my module,
| where all the logic resides.
|
| This makes it easy for me to switch between a "Cobra" context
| and my "Application" context depending on the module. It also
| makes it portable. If i want to use a different CLI framework
| or make this into a backend job, I can pull my module into a
| different project and then call the functions from that other
| project that reside in my module. The module is unaware of
| Cobra entirely, but it performs all my logic. The cobra
| module (the main module) contains only Cobra stuff and then
| offloads all the logic to my application module.
|
| Cobra has all the power you could want from even the most
| advanced CLIs (I think github's CLI and Kubectl, kubernetes
| cli are both built on it for example). But you don't need to
| use any of the advanced stuff if you don't want. It means
| there is a lot of confidence to build a project in cobra and
| if it grows you won't be limited, but it also abstracts the
| complexity away when the project is simple.
|
| I don't have a dog in this fight, just a fan. It is a tool I
| really appreciate. I will check out subcommands though, it
| looks like a good project. Reminds me of "click" for python.
| nvy wrote:
| >spellcheck on commands
|
| I prefer to just type "fuck":
|
| https://github.com/nvbn/thefuck
| andrenth wrote:
| > - https://godoc.org/github.com/ory/dockertest/v3 - run
| containers for e2e testing
|
| For this there is also the testcontainers project, which has
| support for Go and other languages.
|
| https://golang.testcontainers.org/
| etxm wrote:
| You've gotta check out Charm's tools for CLI/TUI dev. They are
| doing some great work.
|
| https://charm.sh/
| tester457 wrote:
| Charm tools I can't live without:
|
| - https://github.com/charmbracelet/wish - Golang SSH server
| that makes building SSH apps easy
|
| - https://github.com/charmbracelet/vhs - terminal GIF demos
|
| - https://github.com/charmbracelet/log - minimal and colorful
| go logs
|
| - https://github.com/charmbracelet/gum - leverage charm's
| bubbles to write useful TUI shell scripts in any language
| omneity wrote:
| Are there boilerplates that include things like this? (I'm
| coming from the JS/TS world)
| emmelaich wrote:
| Perhaps https://github.com/lacion/cookiecutter-golang ?
|
| Might be a little out of date.
| cyri wrote:
| well written post!
|
| one minor thing: I've skipped using build tags for integration
| tests because those tests will be out of sync one day with your
| main code, even with Goland (?).
|
| Instead I use the usual test and check if an environment variable
| is set, if not, then t.Skipf("env var %q not
| set, skipping integration test",envVarName)
|
| or you can use an additional CLI flag, e.g. in `feature_test.go`
| write func init() { flagIntegration :=
| flag.Bool("test.integration",false,"run int tests") }
|
| then $ go test -v -test.integration
| Crowberry wrote:
| I usually make use of the long and short testmodes that are
| supported https://stackoverflow.com/questions/55180613/how-do-
| i-write-...
|
| I used to use buildflags before this, but my linter ignored
| those files so they were hard to maintain
| maerF0x0 wrote:
| > integration tests because those tests will be out of sync one
| day with your main code
|
| What do you mean here? What would be out of sync, and what
| would happen if it were?
| syngrog66 wrote:
| just start writing some Golang. same as any other programming
| language for decades now.
|
| Really thats all you need to get started. OK: a text editor, the
| Golang toolchain and ideally a terminal/shell.
|
| Worry about other things if/when they come up, right in your
| face. Assume by default: YAGNI.
| mparnisari wrote:
| If your code uses goroutines i recommend https://github.com/uber-
| go/goleak to detect leaks
|
| And then for GUID generation we use https://github.com/oklog/ulid
| instead of https://github.com/google/uuid/
| [deleted]
| bitwize wrote:
| [flagged]
| steveklabnik wrote:
| Please don't do this.
| kardianos wrote:
| Package search: https://pkg.go.dev/
| maxmcd wrote:
| Please don't use build tags for integration tests:
| https://peter.bourgon.org/blog/2021/04/02/dont-use-build-tag...
|
| Along with the issues listed here you will run into issues with
| editors not building/linting your tests files because they have
| build tags that the editor is unaware of.
|
| You can also put the environment variable in a TestMain[1] to
| cover an entire package of integration tests:
| func TestMain(m *testing.M) { if
| os.Getenv("RUN_INTEGRATION_TESTS") == "" {
| fmt.Println("Skipping integration tests") return
| } os.Exit(m.Run()) }
|
| [1] https://pkg.go.dev/testing#hdr-Main
| eweise wrote:
| Looks like more boilerplate to me.
| UGIForAll wrote:
| which languages are the best
| sureglymop wrote:
| Genuine question.. I'll often see Golang pretty heavily
| criticized here on hacker news. Either that or people say it's a
| boring language and not worth learning when there is something
| more interesting (usually referring to Rust or Zig). Why does it
| have such a bad image? Personally i like it as an alternative for
| python because: 1. It can build binaries that just work for most
| architectures quickly. 2. It has nice c-like syntax without some
| of the headaches. 3. It seems to be really nice for creating apis
| or backends, especially for web projects. Lately i use it as an
| alternative to php, to build MPAs which are enhanced with htmx.
| 4. It seems very beginner friendly and easy to start with and has
| a non-gatekeepy community. There are also some things i don't
| like so much such as: 1. Goroutines and other go specific stuff
| 2. The dependency system requiring full import paths with urls.
| 3. The strictness about syntax etc. The fact that saving a file
| with an unused import will remove it in the ide.
|
| But it overall seems much nicer than running node/js or python on
| the server side, no?
| dboreham wrote:
| It's just not a shiny squirrel thing in a tree, that's all.
| opportune wrote:
| Golang is IMO the best applications language. Most of my
| criticisms of it would be that it makes some systemsy things
| clunky, and because of garbage collection it just isn't ideal
| for some systemsy stuff.
|
| I personally hate the empty interface and definition shadowing
| of Go but that could be just me not "getting it". Fortunately
| at work we don't use that too often
|
| I think most of the criticism is from people like me coming
| from C++. I am continually baffled that people write web
| backends in Python and Node at all, to me they seem so
| inappropriate that criticizing them would be a waste of time. I
| would consider Go to be much much better overall, and thus
| worthy of actual criticism
| tomcam wrote:
| Don't worry about it. Your assessments are exactly correct.
|
| I'm not putting down any other development environments, but
| everything you say is absolutely true in your circumstances
| (mine too).
| bitwize wrote:
| Yeah, it's pretty much optimized for junior programmers to
| write babby's first enterprise network service in. It's got a
| lot of features junior programmers think are nice and easy to
| work with, but as you mature as a developer its verbosity
| becomes annoying and its shortcomings become apparent.
|
| Using Go as a PHP alternative is pretty much the use case most
| aligned with its niche. So go nuts if you like doing that. But
| stray too far from that use case and Go will start to provide
| pain without adequate justification, especially when compared
| against Zig, Rust, or even TypeScript.
| avgcorrection wrote:
| > Genuine question..
|
| There have been about a hundred "why Go"/"why not Go" threads
| on Hacker News already.
| Patrickmi wrote:
| It's simple when you have WhatsApp vs BlackBerry Messenger and
| we know that WhatsApp wins, you begin to question yourself
| what's making blackberry win even tho it's inferior, then
| hearing their secret weapon "Simplicity" trying to mimic the
| secret weapon but yet still wondering why isn't going as
| expected, this is what is happening No matter how go improves
| people will still talk about its past failures with present
| failures, so you don't have to worry people will still use the
| language and its competition
| hombre_fatal wrote:
| > But it overall seems much nicer than running node/js or
| python on the server side, no?
|
| No, it's just trade-offs.
|
| I think you are making the same mistake by looking for
| validation on HN that you're making some sort of Better Choice,
| but you're just making a normal choice. You just don't yet have
| the experience to see all the trade-offs nor how they compare
| to, say, Node or Python.
|
| For example, there are various ways Node is "nicer" than Go on
| the server. Just compare things like Promise.all or a
| concurrency-limited Promise.map to Go's WaitGroups.
| sureglymop wrote:
| But I'm not really making a choice. I'm open to anything, i
| don't want to restrict myself. I have and do use the things i
| mentioned a lot (node/python). I was just curious what the
| people have to say since i noticed this recently. I don't
| really need the validation since I'll try out all available
| options due to curiosity anyway.
| liampulles wrote:
| It is a boring, no-magic language on purpose.
|
| I think it appeals to cynical devs who have seen projects
| misuse more powerful languages, and who don't want to debate
| style guidelines or linter settings for any more than 5
| minutes. I count myself among them.
| gadflyinyoureye wrote:
| It's verbose. Errors everywhere.
|
| There are a lot of foot guns. nil slice? Fine. nil map?
| Segfault. Loop variables with closures. The list goes on.
|
| Generics seemingly split the community. May be some libraries
| won't get used because they picked the wrong side.
|
| It's surprisingly weak at modeling data. Union types would
| really help out.
|
| The community is so anti-design that it's hard to play with
| them. Most want to make a big ball of mud and call it agile.
| When you point out simple patterns, they call you an Architect
| Astronaut. Checkout r/golang. Also look out for people telling
| you how dumb you are for wanting generics.
|
| In many cases it's a step backwards but it has the positives
| you posted. That is often a reason to grin and bear it.
| Eventually Stockholm Syndrome kicks in.
| 0zemp2c wrote:
| > Loop variables with closures
|
| indeed a brutal footgun but will be fixed soon
| sapiogram wrote:
| Have they actually agreed on how to fix this now?
| typical182 wrote:
| > Generics seemingly split the community. May be some
| libraries won't get used because they picked the wrong side.
|
| I haven't really observed that at all.
|
| One thing that is going on is there hasn't been a massive
| disruption while everyone stops to rewrite the world in
| generics, and generics are not suddenly everywhere, which is
| what some people had predicted would happen. I think part of
| the reason is that in some cases another solution (closures
| or interfaces or whatever) can be a better fit, and the
| evolutionary approach to generics that Go took means you can
| use generics in conjunction with non-generic libraries or
| other pre-existing approaches without suffering from an
| ecosystem split.
| 0zemp2c wrote:
| HN loves playing "obligatory contrarian" so often commenters
| will go to lengths to find faults
|
| there is nothing wrong with Go; it delivers on its promise, you
| don't need to be a genius to use it, has good community
| support, and you can get access to a large and decent job
| market
|
| Rust is a great tool but isn't as purpose-suited to network
| services as Go
|
| Zig is even less purpose-suited to writing network services and
| won't be at Go's level of maturity for years, if ever
|
| If a backend dev could only know one language in 2023, it would
| be hard to go wrong with Go
| shakow wrote:
| > but isn't as purpose-suited to network services as Go
|
| Why?
| sapiogram wrote:
| +1. In terms of development speed, Node + Axios is
| lightyears ahead. It's like 5 lines of code to send a JSON
| payload via http, vs 15+ in Go. The Javascript version is
| much likely to be correct as well, since it doesn't let you
| forget to check any of the three errors, or forget to check
| the http status of the response.
| Daegalus wrote:
| I would like to mention Magefile. We recently have been using it
| over makefiles and it has been amazing. Removes more non-go
| dependencies. You write your files in Go, and it all works really
| well.
|
| https://magefile.org
|
| Between that, goreleaser, 2-stage dockerfiles, static binaries,
| etc. It all just works so well and only needs Go for most things.
|
| Recently for actual stuff we have been using Exho, Zerolog, and
| fairly common libraries for most tools.
| j3s wrote:
| i don't like this post because it makes golang feel overwhelming
| when the stdlib + default tooling is plenty good for most use-
| cases. it's as if someone made a post called "how to go hiking in
| 2023" and spent 10 pages linking to gear on amazon.
|
| how should you actually start hiking? grab a water bottle, get
| outside, and hike.
|
| here is how you should _actually_ start a go project in 2023:
| $EDITOR main.go go run .
|
| everything else you should add as needed. don't overcomplicate
| things.
| opportune wrote:
| Yeah, for a basic Go service or tool I don't think you usually
| need anything besides the standard lib. Maybe you will want to
| use a client library but most of the time it's only thinly
| wrapping various http functionality. I work on some go binaries
| used at massive scale that have little/no dependencies.
| hintymad wrote:
| Any de-facto service framework in Go to recommend, something like
| Dropwizard but supports both gRPC and HTTP APIs? Such framework
| should also has ready-to-use integrations things like metrics,
| logging, tracing and etc. And God please don't just support
| Prometheus. A pulling-based `/metrics` is really not the best
| solution, at least not always.
| liampulles wrote:
| go-micro is a well known one (though I've never used it):
| https://github.com/go-micro/go-micro
|
| In my own experience coming from a Java background, I find Go
| much easier to build from scratch with since the control flow
| is so plain and the standard library API is simple and well
| designed - worth trying.
| vaunt_dev wrote:
| We create a Generic HTTP Handlers enable developers to create
| functions, methods, and data structures that can operate on any
| type, rather than being limited to a specific type.
| https://blog.vaunt.dev/generic-http-handlers
| jasonthorsness wrote:
| I see the note in the article around using -ldflags="-s -w" - is
| there any other useful tool for binary size analysis/reduction? I
| was surprised when my binary size doubled when incorporating the
| K8s client package to get a secret; just using the HTTP secrets
| API manually without referencing the client package shrank the
| size by many MB. It would be nice to find similar opportunities
| for size reduction that aren't as obvious.
| typical182 wrote:
| goda[0] is _excellent_. You do do things like:
| # show the impact of cutting any package goda cut
| ./...:all
|
| which prints a sorted ASCII table with stats like 'size:4.4MB
| loc:134171' for each package, which is an estimate the savings
| you'd get if you eliminated that package from your binary. That
| is a great way to see what is unexpectedly large compared to
| its value.
|
| goda has a bunch of other capabilities around dependency
| analysis, and was written by long-time Go contributor Egon
| Elbre. The examples in the README are the best way to get
| started after 'go install github.com/loov/goda@latest'.
|
| [0] https://github.com/loov/goda
| liampulles wrote:
| The go k8s packages are pretty bloated - this may also just be
| a niche case. If you are looking to get secrets with hot
| reloading, you might also consider mounting a file or setting
| env vars and coupling it with this reloading operator:
| https://github.com/stakater/Reloader
| dhagz wrote:
| One thing for profiling HTTP services specifically, you can
| attach handlers for pprof data easily [0]. I usually only mount
| the routes if I've set a flag for it, usually something to
| indicate I want to run in debug mode. This does everything "for
| free", i.e. it starts profiling memory and CPU and then exposes
| the data on routes for you to visualize in the browser.
|
| [0]: https://pkg.go.dev/net/http/pprof
| nologic01 wrote:
| Looks like a good resource for go beginners.
|
| Would there something similar for a next, second, step that
| focuses on go concurrency aspects?
|
| Go (and erlang/elixir) as touted as concurrency-ready platforms
| and good overviews of setups, tools and best practices might help
| more people benefit.
| lima wrote:
| How to start a new Go project: go mod init
| mymodule
|
| Go's default toolchain is fine, everything else is optional. Some
| questionable advice in the article:
|
| - Vendoring dependencies using "go mod vendor" is not a good
| default workflow - it bloats the repo, the checked in code is
| impossible to review, and is generally a pain to keep up to date.
| Don't, unless you really have to.
|
| - There's no point in stripping a binary or even using UPX on it
| unless you're targeting extremely low memory environments (in
| which case Go is the wrong tool anyways), all it'll do it make it
| harder to debug.
| latchkey wrote:
| > There's no point in stripping a binary or even using UPX on
| it unless you're targeting extremely low memory environments
|
| I really dislike absolutes like this.
|
| My target is 30,000+ servers and distributing a binary to all
| of them is a lot easier when it is 3m than when it is 26m.
| nemothekid wrote:
| If the problem is distribution, what's wrong with gzip? All
| the upsize of UPX and none of the downsides. If your
| distribution method is http, then you don't even have to
| write any code other than setting a Content-Encoding header.
| latchkey wrote:
| gzip doesn't make it small enough.
|
| 3mb is after `xz -z -9e`.
|
| But, if you start with something smaller, you generally get
| something even smaller.
|
| I tried UPX, but ended up with just `-s -w` (and xz),
| simply because UPX was taking too long to build the binary
| in CI.
|
| More importantly though, I was responding to OP's absolute.
| Thaxll wrote:
| I don't really believe that, at the speed of nic it makes
| pretty much 0 difference even on 30k servers. Shaving couple
| of ms at worse few seconds vs modifing a binary, def not
| worth it.
| latchkey wrote:
| The servers are not all on gige. Many are on 100mbit and
| yes, that saturates the network when they are all updating.
| I learned through trial and error.
|
| The updates are not pushed, they are pulled. Why? Because
| the machines might be in some sort of rebooting state at
| any point. So trying to first communicate with the machine
| and timeouts from that, would just screw everything up.
|
| So, the machines check for an update on a somewhat random
| schedule and then update if they need to. This means that a
| lot of them updating at the same time would also saturate
| the network.
|
| Smaller binaries matter.
| hnlmorg wrote:
| I'm curious why you've got servers on 100Mb. Last time I
| ran a server on 100Mb was more than 20 years ago. I
| remember the experience well because we needed AppleTalk
| support which wasn't trivial on GbE (for reasons
| unrelated to GbE -- but that's another topic entirely).
|
| What's your use case for having machines on 100Mb? Are
| you using GbE hardware but dropping down to 100Mb, and if
| not, where are you getting the hardware from?
|
| Sounds like you might work in a really interesting domain
| :)
| pdmccormick wrote:
| Sounds like an interesting problem to have. Would
| something peer-to-peer like BitTorrent work to spread the
| load? Utilize more of the networks' bisectional
| bandwidth, as opposed to just saturating a smaller number
| of server uplinks. I recall reading many years ago that
| Facebook did this (I think it was them?)
| sethammons wrote:
| I'm on the vendor bandwagon; always have been. I don't want a
| github outage to dictate when I can build/deploy. Yes, that
| happened. That is why we vendor :).
|
| Now you can set up a proxy server; however, I don't want to do
| that. I'm pretty sure I have a few vendored packages that no
| longer exist at their original import path. For code reviews,
| we put off checking in the vendor path til the end if possible.
| jasonwatkinspdx wrote:
| I like vendoring in most languages as it means I can follow
| all the code flow easily in my editor when debugging
| something.
| djbusby wrote:
| I do all my vendor in a "cache-proxy" thing (for lots of
| vendors). That box always runs, I just need upstream the
| first time I get the package. Doesn't bloat my code, makes
| sure package is available and makes audits of vendor stuff
| easy.
| attentive wrote:
| also increases startup time
| didip wrote:
| There is a benefit to using "go mod vendor". Some corporate
| environments lock down their CI/CD pipelines. By vendoring
| everything, the CI/CD does not need to make external HTTP
| calls.
| 0zemp2c wrote:
| vendoring is a bit of project smell, but for large teams it
| removes the confusion of who has what version of a dependency
|
| unfortunately most teams don't schedule a periodic `go mod
| tidy` so you just end up with ancient deps
|
| most people never read the code of the deps they pull in, so I
| don't think vendoring provides any security assurances
| dilyevsky wrote:
| > it removes the confusion of who has what version of a
| dependency
|
| go.mod/sum files already remove that confusion as it's their
| intended purpose
| icholy wrote:
| That's not what `go mod tidy` does ...
| Paul-Craft wrote:
| I'd go way farther than "a bit of a project smell." I
| literally cannot think of a single instance in which
| vendoring a dependency for any reason other than, say,
| caching it for CI so you don't have to worry that the
| maintainer pulls a `left-pad` on you, has gone well.
|
| If the package has bugs, you're far better off either waiting
| for upstream fixes, working around the bug in your
| application code, or just switching to a different library.
| That goes double if the library you're using is missing a
| feature you need, even if it's scheduled for the next version
| release.
|
| Unless you're prepared to maintain a full-on fork of the
| dependency (and, if you do, _please_ make it public),
| everything about vendoring for these reasons is 100% bad for
| you for very little incremental benefit. It 's like the joke
| about regular expressions ("You have a problem and think
| 'I'll use regexes to solve it.' Now you have two problems"),
| except it's not a joke, and it sucks way more.
|
| TL;DR: Vendoring to cache for CI/build servers, yes. Any
| other reason, just don't; it's not worth the headaches.
| ownagefool wrote:
| If you work on code that introduces threat to life, you
| might be prepared to own all the code, even if you don't
| write it all from scratch.
| pdmccormick wrote:
| A Go vendoring pattern that I've found very useful is to use
| two repositories, the first for the main "project" repository,
| then a second "vendoring" repository that imports the first as
| a module, and also vendors everything.
|
| This may require a few extra tricks to plumb through, for
| example, to make all cmd's be externally importable (i.e. in
| the project repository, transform "cmd/foo/%.go" from being an
| unimportable "package main" into an importable
| "cmd/foo/cmdfoo/%.go", then have a parallel "cmd/foo/main.go"
| in the vendoring repository that is just "func main() {
| cmdfoo.Main() }", same as you have in the project repository in
| fact).
|
| Vendoring aside, this is also a useful pattern if you're
| "go:embed"ing a collection of build artefacts coming from
| another source, like a frontend HTML/JS/CSS project.
| vardump wrote:
| UPX only means smaller files on the disk, but it comes with a
| cost: it tends to _increase_ memory requirements, because the
| binary on the disk cannot be mapped to memory anymore. Unless
| it 's uncompressed somewhere in the filesystem.
|
| Worse, if you run multiple instances of the same binary, none
| of them can be shared.
|
| A bit simplified, without UPX, 100 processes of 100 MB binaries
| requires only 100 MB RAM for the code, but with UPX 10 GB.
|
| Edit: In reality, likely only a fraction of that 100 MB needs
| actually to be mapped into memory, so without UPX true memory
| consumption is even less than 100 MB.
| cced wrote:
| Can you expand on this a bit? I use upx at work to ship
| binaries. Are you saying these binaries have different memory
| usage upx'd than they do otherwise?
| Karrot_Kream wrote:
| I'm curious, was the practice of using upx there before you
| got there? We generally A/B test changes like this pretty
| thoroughly by running load tests against our traffic and
| looking at things like CPU and Memory pressure in our
| deploys.
| themerone wrote:
| Every instance of a program will use an amount of ram equal
| to the uncompressed size.
|
| If the application is uncompressed, the uncompressed
| executable will be loaded into ram 1 time and be reused by
| every instance of the application.
| ashishb wrote:
| Any source that you can cite for this? I'm not saying you
| are wrong. I'm just curious to see more proof of this.
| themerone wrote:
| https://stackoverflow.com/questions/9219244/why-does-my-
| appl...
| mikepurvis wrote:
| Also impacts startup time. Really it's only appropriate
| for situations like games where you're very confident
| there will be just one instance of it, and it'll be long-
| running.
|
| And even then, it's of dubious value when game install
| footprints are overwhelmingly dominated by assets rather
| than executable code.
| vardump wrote:
| Normally operating system simply maps binaries, executables
| and loadable libraries (.dylib, .so, .dll, etc.) into
| memory. The cost is approximately same whether you do this
| once or 1000 times. The code is executed from the mapped
| area as-is.
|
| However, when a binary is compressed, this cannot work,
| because in the file the binary is represented as a
| compressed data. The only way you can work around is to
| allocate some memory, decompress the binary there, map the
| region as executable and run it from there. This results a
| non-shareable copy of the data for each running instance.
|
| A random link about this issue in practice:
| https://github.com/nushell/nushell/issues/4131
| ownagefool wrote:
| So, I don't bother with vendoring my dependencies ( usually ),
| but you have it the wrong way round.
|
| Vendoring would make it more likely you're gonna review the
| changes, be ause you can quickly eyeball whether or not changes
| look significant, which is something you often won't get out of
| a go.sum change.
| fsociety wrote:
| Unless you import a dependency which totals several hundred
| thousand lines of code.
| tonyhb wrote:
| Make your git commit history good? `go mod vendor` in a
| separate commit to your PR changes. Review the commit with
| local code changes. Easy.
| bakoo wrote:
| > There's no point in stripping a binary or even using UPX on
| it unless you're targeting extremely low memory environments
|
| Deploing at a large enough scale, perhaps where other
| optimization options aren't as good or even available, could
| also be a target.
| shellac wrote:
| [flagged]
| tptacek wrote:
| I don't know, it's a complicated question, and it has nothing
| to do with the article and asks for a big language fight.
| kortex wrote:
| What's "most modern languages"? Go is better than Python's
| native tooling (only Poetry and other similar 3rd party tools
| compare). Javascript I find unruly and fragmented. "Modern" C++
| is still a nightmare.
|
| Java I haven't touched but I don't exactly hear rave reviews or
| angry rants about, so I expect it's middling.
|
| Rust, scala, and haskell are definitely better experiences, but
| they are definitely in the minority in terms of industry usage.
|
| Go is not "quite bad", in fact far from it. I'd say it's better
| than average.
| shellac wrote:
| > What's "most modern languages"? Go is better than Python's
| native tooling (only Poetry and other similar 3rd party tools
| compare). Javascript I find unruly and fragmented.
|
| Exactly the scale I had in mind, thanks. When I saw 'go get
| <package>' rather than dependencies added to the equivalent
| of a Gemfile / cargo / pom file it had concerns.
| kortex wrote:
| There's nothing stopping you from adding to go.mod though,
| you just have to update the sumfile, no different than
| using pyproject.toml directly vs adding with CLI.
| bborud wrote:
| What are you comparing to and what do you think the shortfalls
| are? What is a "modern" language?
| icholy wrote:
| Go has better dependency management than most modern languages.
| cpuguy83 wrote:
| "go mod init <stare at the terminal endlessly trying to decide on
| a name>".
|
| Minor improvement over "mkdir -p
| $GOPATH/src/path/in/vcs/remote/<stare at the terminal endlessly
| trying to decide on a name>"
|
| /s
| SergeAx wrote:
| > The solution is to upgrade [Go language] after a week or so
|
| I personally have a habit of updating only when the next patch
| version is out. It saved be countless hours of debugging and
| frustration.
| arriu wrote:
| One thing I wish there was better support for is live debugging
| and stepping through code.
| poo-yie wrote:
| You know about VS Code and GoLand?
| puika wrote:
| I absolutely love the integration in VSCode with the official
| Go extension. I can debug a running web server with delve with
| minimal config. Same for tests. Just experiment with the
| options, there are quite a lot, and unfortunately some not very
| well documented like gopls ones, at least last time i checked.
| maerF0x0 wrote:
| I'd add "What's the best web framework?" and answer it with "No."
|
| (Go comes batteries included)
| rollulus wrote:
| That's quite true, although for routing / muxing I do tend to
| use a third party one. The numerous Go web frameworks are
| solely created for the glory of the authors.
| LVB wrote:
| A better answer would be some recommendations for component
| pieces. e.g., I will need most of the essential things in
| https://github.com/go-chi/chi, so why bother rolling a version
| myself? The same goes for things like sqlx. I'm averse to
| leaning on a "framework," but do find good value in targeted
| libraries.
| liampulles wrote:
| And in general, one does not need frameworks in Go to the same
| degree as say Java or C# - it is an easier language to build
| things from scratch with I think.
| baby wrote:
| I've left Golang for a while, then when I came back it felt a bit
| complicated to figure out how to get started. I feel rustup.rs,
| really got Rust to a much better spot than Golang in such a short
| amount of time.
| revskill wrote:
| Working with Go is boring but productive. Go libraries are
| reliable, fast. It's more than enough to write good software.
| athorax wrote:
| The article mentions GOW[0] for a file watcher. If anyone is
| looking for a non-go specific one, I've really enjoyed reflex[1].
| Makes it super easy to reload different parts of a project based
| on what type of file has changed.
|
| [0]https://github.com/mitranim/gow
| [1]https://github.com/cespare/reflex
| agotterer wrote:
| I've been very happy with Air - https://github.com/cosmtrek/air
| puika wrote:
| Just to add to the list there is also
| https://github.com/cosmtrek/air
| leetrout wrote:
| Mentioned in reflex's competition section I am a huge fan of
| entr
|
| https://github.com/eradman/entr
| synergy20 wrote:
| how about watchman? I use that one.
| https://manpages.ubuntu.com/manpages/jammy/man1/watchman.1.h...
| jweir wrote:
| Since we are mentioning these, I use entr
|
| # Makefile test: `find * -name "*.go" | entr bash -c "clear; go
| test ./..."`
|
| https://eradman.com/entrproject/
___________________________________________________________________
(page generated 2023-05-23 23:00 UTC)