[HN Gopher] The simplicity of single-file Golang deployments
___________________________________________________________________
The simplicity of single-file Golang deployments
Author : KingOfCoders
Score : 143 points
Date : 2023-03-22 13:07 UTC (9 hours ago)
(HTM) web link (www.amazingcto.com)
(TXT) w3m dump (www.amazingcto.com)
| mastax wrote:
| > Systemd holding connections and restarting the new binary.
|
| How does this work?
|
| Or does it just mean it stops new connections while it's
| restarting?
| christophilus wrote:
| Systemd effectively acts as a proxy. I don't know that it's
| _actually_ a proxy, but it keeps accepting connections from
| what I 've seen. I use it for zero-downtime single-binary
| deploys, and it's great.
| yencabulator wrote:
| No, it's not a proxy, and it's not accepting connections
| (unless you're using the inetd emulation, but that's rare and
| inefficient).
|
| It's merely passing the listening socket as an already-open
| file descriptor to the spawned process.
|
| The "keeps accepting" part is just the listening socket
| backlog.
|
| Last I looked, systemd socket passing couldn't be used to do
| graceful shutdown, serving existing connections with the old
| version while having the new version receive new connections.
| Outside of that, it's very nice.
| hnarn wrote:
| Could be what's described in this post:
| https://vincent.bernat.ch/en/blog/2018-systemd-golang-socket...
| sneak wrote:
| I still put the single file in a docker container because docker
| isn't complex.
| hsn915 wrote:
| This makes zero sense.
|
| I don't like cargo culting.
| jerf wrote:
| There are reasons. It's a reasonable security boundary. It
| integrates with other things that use Docker as the primary
| abstraction, and there's good odds I've got other docker
| things that aren't single binaries like databases and other
| tools. It puts it into a uniform control interface that works
| with other things as well. It doesn't cost much additional
| resources over simply running the binary directly because the
| real runtime cost of a docker container is the mini-OS they
| often bring up, not the target executable.
|
| It isn't necessary, but it's not nonsense.
| sneak wrote:
| I deploy everything everywhere with docker, because then the
| only thing installed on the system is dockerd. I can deploy
| identically on different distributions; I don't need to know
| anything about the host or keep track of files on the host.
|
| I can keep all of my build artifacts in a docker image
| repository with versions. I can deploy any version on any
| host without worrying about copying the version to the host.
|
| Whether your deploy is 1000000 files or 1, this system has
| clear advantages to copying things to the base server OS and
| turning it into a snowflake.
| marcrosoft wrote:
| How is this news. Welcome to 10 years ago.
| adql wrote:
| It become easier few years ago as tools to embed files are now
| nicely builtin into Go instead of external packages.
| danwee wrote:
| How does one handle zero downtime deployments with single-file
| golang binaries? I remember I tried this setup some time ago and
| I couldn't successfully manage cleanly to accomplish no downtime
| when deploying a new version of my service. The reason was mainly
| port reuse. I couldn't have the old and the new version of my
| service running on the same port... so I started to hack together
| something and it became dirty pretty quickly. I'm talking about
| deployment of new version of service on the same machine/server
| as the old version was running.
| bojanz wrote:
| Socket activation via systemd[0] is an option, assuming you are
| fine with certain requests taking a longer time to complete (if
| they arrive while the service is being restarted). Otherwise
| using a proxy in front of your app is your best bet (which has
| other benefits too, as you can offload TLS and request
| logging/instrumentation).
|
| - https://github.com/bojanz/httpx#systemd-setup
| marcosdumay wrote:
| You can always share ports.
|
| But the one way to do no downtime deployments is to have more
| than one server.
| SkyPuncher wrote:
| I've always run services behind a proxy. Spin up a new server
| with the code (works for any type of deployment). Validate it's
| up. Switch the proxy from the old to new server.
| klodolph wrote:
| Some of this is solved by using e.g. systemd, depending on your
| needs.
|
| > I couldn't have the old and the new version of my service
| running on the same port...
|
| You can, actually! You just can't open the port twice by
| default. So one or both of the processes needs to inherit the
| port from a parent process, get passed the port over a socket
| (Unix sockets can transmit file descriptors), or use
| SO_REUSEADDR.
|
| There are some libraries that abstract this, and some of this
| is provided by tools like systemd.
|
| Some of this is probably going to have to be done in your
| application--like, once your new version starts, the old
| version should stop accepting new connections and finish the
| requests it has already started.
| joncfoo wrote:
| > Some of this is probably going to have to be done in your
| application...
|
| FWICT tableflip does exactly this:
| https://github.com/cloudflare/tableflip
| mattbillenstein wrote:
| So can I just start another process with SO_REUSEADDR and
| gracefully shutdown the old process?
|
| The master/worker thing that nginx / gunicorn et al do is
| pretty neat, but relies on signals; so seems pretty messy and
| error prone to write yourself.
| klodolph wrote:
| You may have to start both processes with SO_REUSEADDR, I
| don't remember the exact semantics.
|
| People have a healthy skepticism of signals from the C
| days, but if we're talking about Go, you'd just call
| signal.Notify. Any way of signaling your app to shut down
| works, though.
|
| https://pkg.go.dev/os/signal@go1.20.2#Notify
| stasmo wrote:
| This is where the simplicity of single-file golang deployments
| falls short.
|
| Just make sure you're not slowly recreating bad, homebrew
| versions of all of the nice things that Kubernetes does in an
| attempt to turn a simple deployment into a production ready
| deployment.
| iamjackg wrote:
| If you really don't want to use different ports you can handle
| it with Docker. Since each container has its own IP, they can
| all expose the same port. Otherwise, for non-containerized
| deployments you'll have to resort to two different ports.
|
| In either case, you will need a reverse proxy like
| Traefik/Nginx in front to smartly "balance" incoming requests
| to the two instances of the service.
| [deleted]
| adql wrote:
| Same way you do with any other app not specifically designed
| for it; you start 2 copies of it and put loadbalancer in front
| of it. I did that via some systemd voodoo
|
| But _TECHNICALLY_ to do that in one without external proxy you
| 'd need to figure out how to set SO_REUSEPORT for the web
| socket handler, then start the second one before the first.
|
| Haven't actually tried it but someone apparently did:
| https://iximiuz.com/en/posts/go-net-http-setsockopt-example/
|
| You'd still have any ongoing connections cut unless you unbind
| socket and then finish any existing connection, which would be
| pretty hard with default http server.
|
| I just put HAProxy instance on my VPS that does all of that,
| including only allowing traffic once app says "yes I am ok" in
| healthcheck. Then the app can have "shutting down" phase, where
| it reports "I am down" on healthcheck but still finished any
| active connections to the client.
| dividuum wrote:
| Can't help with how to implement this, but just to be sure: You
| should be able to use the same port in multiple instances if
| you bind those with SO_REUSEPORT. A quick search points to
| https://github.com/libp2p/go-reuseport for an implementation.
| Now you just need a mechanism to drain the old process.
| joncfoo wrote:
| Rough psuedocode to do this with the built-in http.Server
| where startServer(..) would use the reuseport library to
| create the listener so multiple servers can listen within the
| same process: func reloadConfig(config) {
| if newServer, err := startServer(config); err != nil {
| // gracefully shutdown previous server // no new
| connections will go to old server
| oldServer.Shutdown(...) oldServer = newServer
| } }
| hnarn wrote:
| This doesn't sound Go-specific, if you use something like
| haproxy targeting multiple nodes you can take them down one by
| one to perform a rolling upgrade.
| Edd314159 wrote:
| I guess this is a problem inherent not just in a single-file go
| app, but in any deployment where the whole stack is contained
| within a single process.
|
| The post says the process starts up quick enough that the
| process being temporarily unavailable isn't noticeable - but
| what if the process _doesn't come back_? It's also impossible
| to do blue/green deployments this way.
|
| It's clearly not a solution suitable to large-scale
| deployments. The simplicity has its trade-offs.
| InitialBP wrote:
| If you want to do deployments with single-file apps or other
| "whole stack in a single process" type of deployments there
| are other options to do it with zero-downtime.
|
| One good option would be to spin up a second
| server/instance/container, run binary on new system, ensure
| it's good, once comfortable then swap DNS entry to the new
| system.
| andrewfromx wrote:
| Interesting. Are you saying for a big project run something to
| minimize the .go files into just one? Assuming there is just one
| package.
| neverartful wrote:
| Probably referring to the fact that when you build a go
| executable there's just a single file to deploy (the
| executable).
| jjtheblunt wrote:
| Go has a facility for embedding build time files within the
| resulting binary such that they can be read as if in a
| runtime file system, because the Go file access routines know
| about this file system type.
| PaulHoule wrote:
| You can definitely pack a Java application into a single
| JAR file and skip the Docker. Java's xenophobia (allergy to
| linking libraries) is the real root of "write once run
| everywhere" so often all you need is the Java runtime.
| thefounder wrote:
| You still need jre so it's not really a "single file"
| like on Go
| erichocean wrote:
| You still need a bespoke systemd configuration for TFA's
| Go deploys so it's not really a "single file" deploy
| their either.
|
| And like the sibling comment noted, once you're allowed
| to set up the machine to support easy depolyment (e.g.
| JRE, Tomcat), a WAR becomes a single file deploy.
| logistark wrote:
| You can use JLink to embed only the needed modules and
| classes of the JRE for your application to run
| creshal wrote:
| Eh. From an ops perspective there isn't much difference
| between an executable file that Golang statically
| compiled all dependencies into and embedded a file system
| into, and a WAR archive that the Java compiler embedded a
| file system including dependencies into.
|
| Both are self-contained single files you can give to a
| completely different organization and expect to run on
| the first attempt with no complications.
|
| It's just that the latter needs Tomcat (not an issue,
| realistically) and _has_ to be written as EnterpriseFacto
| ryPatternFactorySingletonAbstractBaseFactorySingletonProv
| ider that makes you feel dead inside just from looking at
| the documentation; while Golang (and similar newer
| languages) give you a lot more flexibility and better
| ergonomics on the developer side.
| PaulHoule wrote:
| Try Guava or Spring. In either case the framework
| supplies you with a Factory,
| FactoryFactory, FactoryFactoryFactory, ...
|
| that does the transitive closure so you can just get a
| "single" object injected into your app where you need it.
| jjtheblunt wrote:
| "lasciate ogne speranza, voi ch'intrate" (~ lose all
| hope, who enters) from Dante's Inferno is what comes to
| mind whenever someone mentions Spring.
| jjtheblunt wrote:
| class loaders in java are jvm's dynamic linker/loaders,
| wouldn't you say?
| twic wrote:
| I feel like i'm taking crazy pills (at a low dose) when i read
| this stuff.
|
| I deploy Java applications. In a runnable condition, they aren't
| a single file, but they aren't many - maybe a dozen jars plus
| some scripts. Our build process puts all that in a tarball.
| Deployment comprises copying the tarball to the server, then
| unpacking it [1].
|
| That is one step more than deploying a single binary, but it's a
| trivial step, and both steps are done by a release script, so
| there is a single user-visible step.
|
| The additional pain associated with deploying a tarball rather
| than a single binary is negligible. It simply is not worth
| worrying about [2].
|
| But Go enjoyers make such a big deal of this single binary! What
| am i missing?
|
| Now, this post does talk about Docker. If you use Docker to
| deploy, then yes, that is more of a headache. But Docker is not
| the only alternative to a single binary! You can just deploy a
| tarball!
|
| [1] We do deploy the JDK separately. We have a script which takes
| a local path to a JDK tarball and a hostname, and installs the
| JDK in the right place on the target machine. This is a bit
| caveman, and it might be better to use something like Ansible, or
| make custom OS packages for specific JDKs, or even use something
| like asdf. But we don't need to deploy JDKs very often, so the
| script works for us.
|
| [2] Although if you insist, it's pretty easy to make a self-
| expanding-and-running zip, so you could have a single file if you
| really want: https://github.com/vmware-archive/executable-dist-
| plugin
| simiones wrote:
| Related to [1], I thought modern Java deployment style is to
| bundle the required modules of the JDK with your app, rather
| than any concept of a "deployed JDK".
|
| As it is, the difficulty of deploying a JDK + your app is much
| more than a single static binary, Go-style.
| 5e92cb50239222b wrote:
| It depends on your requirements and environment. With most
| things I am working on the commands to deploy both look
| exactly the same: $ rsync -a foo
| server:/opt/foo $ rsync -a bar server:/opt/bar
|
| Can you guess which one is a static binary written in Go, and
| which one is a directory with a slimmed down JRE produced by
| jlink + an application jar?
|
| For those using containers, there's no practical difference
| between the two.
| lenkite wrote:
| There are also some advantages towards the tarball jar
| encapsulating multiple jar approach - some cloud platform Java
| buildpacks superbly optimize the deployment process by only
| sending the differential jars - sometimes just the differential
| classes - which makes deployment 2x-3x faster than Golang
| single big bang executable approaches.
|
| In our company which leverage both Java microservices and
| Golang microservices - the Java app deployment is much faster!
| x0x0 wrote:
| I'm taking the same crazy pills.
|
| We did this deployment pattern with a jar in, like... the early
| 2000s? It's trivial (well, maybe an annoying couple hours to
| config, but it's a config once and then done) in maven to build
| a megajar and add every single thing you need into one large
| jar. All resources, dependencies, etc.
|
| And then deployment is, indeed, an rsync.
| stanleydrew wrote:
| Your footnotes basically invalidate your argument. You aren't
| just deploying a tarball, you also have to deploy the java
| runtime and make sure it's compatible with your application.
|
| I agree that Go fans make too much of the single binary
| feature, but it does seem easier than your deployment process.
|
| Of course your process is easy _for you_ because you built it
| to fit your needs. But if you imagine a new developer who has
| no experience deploying either Java or Go applications, and
| consider what 's easier to deploy without any previous
| knowledge or automation, I think you might agree the Go
| deployment options are simpler.
| smallerfish wrote:
| We also install the JRE separately, but each app is a single
| executable (by Java) jar, which stands up a jetty instance when
| run. We also add a yml for configuration.
|
| It's a much better packaging & deploy story than frontend code
| or python.
| ardit33 wrote:
| twic, you just made the op's point.
|
| I too miss the days where you can just ssh/ftp a file, and
| boom, it was live. (this was usually a php file back then).
|
| It is such a great feeling to be able to know whats going on at
| every step. With increased complexity, so has the deployment
| process in general. The java steps you described where the
| beginning of more complex deployments back then (1999-2001)
|
| And, yes, I agree with the author in this case. Golang, makes
| it super simple to deploy a web service.
| solatic wrote:
| > Deployment comprises copying the tarball to the server
|
| You must not scale servers up and down very frequently then.
|
| > both steps are done by a release script, so there is a single
| user-visible step... we do deploy the JDK separately
|
| Wait, so it's not really a single user-visible step. You have
| one user-visible step to deploy the application server, and a
| different user-visible step to deploy the JDK.
|
| Look, there's a reason why this way is old-fashioned. If you
| bought the server outright (i.e. running on-prem/colo), and so
| it represents a sunk cost, and the usage is all well within the
| ceiling of what that server is capable of providing, then sure,
| that's an eminently reasonable setup. If that server is humming
| along for several years, and electricity/data center costs are
| cheap, you're probably even saving money.
|
| But in most cloud-first architectures, if you're not scaling
| down on low-usage times, you're wasting money. Scaling up and
| down is much, much simpler with immutable infrastructure
| patterns, and it's much simpler to just replace the entire
| image - whether that's a VM, or a container, or something else
| - rather than replacing just the application.
| twic wrote:
| > You must not scale servers up and down very frequently
| then.
|
| Indeed we don't. But i don't see why it would be a problem if
| we did. If you can run a script to copy a Go binary to a VM
| when you scale up, you can run a script to copy a tarball and
| unpack it. If you're scaling based on a prepared image, then
| you can prepare the image by unpacking a tarball, rather than
| copying in one file.
|
| > Wait, so it's not really a single user-visible step. You
| have one user-visible step to deploy the application server,
| and a different user-visible step to deploy the JDK.
|
| Oh come on! If that matters to you, change the app deployment
| script to run the JDK deployment script first. Bam, one step.
|
| > Scaling up and down is much, much simpler with immutable
| infrastructure patterns
|
| Sure, and as far as i can see, this is completely orthogonal
| to having a single-file deployment. You haven't made any case
| at all for why having _single-file_ deployment is valuable
| here.
| drewg123 wrote:
| The fact that it produces a single static binary is one of the
| nicest things about golang.
|
| This used to be easy with C (on BSD & Linux) a long time ago, but
| then everything started to depend on various shared libs, who
| then depend on other libs, then things started to even dlopen
| libs behind your back so they didn't even show up in ldd, etc.
| Sigh.
| synergy20 wrote:
| very true, for Go though if you need CGO it's hard to make a
| single executable, otherwise it is great.
|
| I run a few Go apps, all are single executables, upgrading to
| new releases has never been easier.
|
| if you have a network oriented application, nothing beats
| golang as far as release|maintenance is concerned.
| dilyevsky wrote:
| > CGO_ENABLED=1 CC=musl-gcc go build --ldflags '-linkmode
| external -extldflags=-static'
|
| Unless you cant use musl for some reason, but that's a glibc
| problem, not Go
| aatd86 wrote:
| Some people are writing alternative via WebAssembly.
|
| For instance wazero which I'm really excited about.
|
| https://tetrate.io/blog/introducing-wazero-from-
| tetrate/?utm...
| traceroute66 wrote:
| > The fact that it produces a single static binary is one of
| the nicest things about golang.
|
| _AND_ unlike Rust, Go has an eminently usable and
| comprehensive stdlib.
|
| It means that for most projects you can just Get Stuff Done
| (TM) instead of deciding which of 24 "reinventing the wheel"
| Rust Crates you want to use for X, Y and Z functions of your
| Rust binary.
| tptacek wrote:
| Please don't start random language fights.
| tomcam wrote:
| Directly addressing the topic of a upvoted article is
| starting random language fights?
| doodpants wrote:
| The article is about Go, and while the article mentions
| other languages for comparison, Rust is not one of them.
| So traceroute66's comment comes across as starting a Rust
| vs. Go fight where not appropriate.
| tomcam wrote:
| Doesn't come across as starting a fight to me. Not even
| close.
| [deleted]
| djbusby wrote:
| Not even, they just expressing an opinion.
| vore wrote:
| Where does the word Rust appear in the article?
| benatkin wrote:
| That's no language fight. Rust doesn't want batteries
| included. That's highlighting a difference between the
| languages.
| earthling8118 wrote:
| From another point of view "usable" isn't a word I'd
| associate with Go, especially not when there is an "unlike
| Rust" in the phrase. I've not had any issues of the sort when
| building out Rust applications. At this point the only time I
| touch Go is if I need to modify something else someone has
| made-- which I avoid at all costs
| aatd86 wrote:
| Maybe AI will make it more palatable but I just can't get
| into Rust.
|
| I can probably easily understand borrowing. It's mostly an
| issue of controlling pointer aliasing wrt mutability,
| especially in a multithreaded context I guess.
|
| But that gottdarn syntax... And goroutines are too nice for
| the type of code I use.
|
| I'm too spoiled with Go I guess.
| pjmlp wrote:
| If only they would decide what to do with plugin package, not
| use SCM paths for packages and decide to eventually support
| enumerations instead of iota dance (Pascal style would be
| enough.
|
| Maybe 10 more years.
| p5v wrote:
| Regarding the "iota dance," I wrote my 2 cents on what
| could be a slightly more robust approach a couple of days
| ago: https://preslav.me/2023/03/17/create-robust-enums-in-
| golang/
|
| P.S generic type sets make it even better. I'll write an
| update to my post these days.
| Joker_vD wrote:
| type Kind enum { Simple, Complex,
| Emacs, } const kindStrings
| [Kind]string = { "simple",
| "complex", "emacs", } func
| (k Kind) String() string { return
| kindStrings[k] } func t() {
| var a = Kind.Emacs - Kind.Simple // a has type int and
| value 2 var b = Kind.Simple + Kind.Emacs //
| type error var c = Kind.Simple + 1 //
| type error var d = len(Kind) //
| d has type int and value 3 for k := range
| Kind { fmt.Printf("%v\n", k) // prints what
| you expect it to do } // not
| sure about legality or runtime behaviour of the followng
| var t = Kind.Emacs t++ t = Kind(42)
| }
|
| Would be nice, not gonna lie.
| Asdrubalini wrote:
| Rust's philosophy (unlike Python's, for example) consists in
| not including tons of stuff in the standard library, this way
| you can choose the best implementation for your specific use
| case from one of the many crates (which are super easy to
| install, by the way). There is no "always the best"
| implementation for a specific functionality, nor a legacy-
| but-still-officially-supported-in-std implementation that
| nobody uses anymore but still needs to be maintained with its
| own namespace.
|
| I don't see this as negative or "reinventing the wheel".
| Reinventing the wheel would be writing your own
| implementation, which doesn't happen if you can choose from
| many high-quality crates.
| lenkite wrote:
| "a legacy-but-still-officially-supported-in-std
| implementation that nobody uses anymore but still needs to
| be maintained with its own namespace."
|
| The cardinality of the set of "nobody uses anymore" is
| usually in tens of millions.
| Asdrubalini wrote:
| If something is used by tens of millions, be sure that it
| will be updated even if legacy. Just not officially by
| the people who maintain the language.
| abtinf wrote:
| > legacy-but-still-officially-supported-in-std
| implementation
|
| There is _massive_ value in this.
| Asdrubalini wrote:
| Not officially supported doesn't mean not updated or not
| supported in general.
| brigadier132 wrote:
| Using a crate is the opposite if reinventing the wheel. Also
| are we talking about go? The language without a set
| implementation?
| asalahli wrote:
| Parent is saying those crates themselves are reinventing
| the wheel, not talking about using a crate
| brigadier132 wrote:
| So by that logic Golang reinvented the wheel by creating
| a new programming language when java already existed
| 1vuio0pswjnm7 wrote:
| IME, using musl, compiling static binaries written in C is as
| easy as it was before glibc changes and as it has always has
| been on NetBSD. I compile static binaries written in C every
| day on Linux. I never encountered any problems compiling static
| binaries on BSD.
| candiddevmike wrote:
| From what I have seen in major OSS projects like systemd and
| PostgreSQL, nothing seems to support static linking, to the
| point where some contributors get annoyed when you ask for it.
|
| Seems like the C/C++ ecosystem will stay dynamically linked,
| even with a lot of the industry shifting towards statically
| linked, fat binaries as disk space is pretty cheap.
|
| I wonder how much simpler Linux packaging would be if
| everything was statically linked...
| bzzzt wrote:
| Packaging would be simpler, but you probably have to update
| the whole OS and all your installed apps when a security
| update for a common library is released.
| taeric wrote:
| Not wrong; at the same time, I'm becoming less convinced
| this is relevant. Seems many security updates have to do
| with code paths that are not used in a large number of
| applications. Similarly, many of them are fixes on features
| that even more apps didn't want/need, but they got when
| they updated to get the last round of security fixes. :(
|
| The docker world is a neat example of this. The stories of
| ${absurdly high percent} of containers having
| vulnerabilities only gets to claim that because of
| "unpatched libraries." If you reduce it to the number of
| containers with exploitable vulnerabilities, it is still
| non-zero, but not nearly as high.
| nightfly wrote:
| The problem is having to wait for all those applications
| to push updates when a vulnerability is found in a common
| library
| rangerelf wrote:
| Simpler but small updates, like say openssl, become massive
| distro updates. There's a reason why everyone went with
| shared libs when they became stable.
| iambvk wrote:
| A system with shared libraries also needs an _update_ for
| the security fixes. There is no avoiding the _update_ step.
| However, the only difference is the size of the update. If
| update process is robust, size of the update shouldn 't
| matter, isn't it?
| dahfizz wrote:
| It does matter. If there is a problem with openssl, just
| the openssl maintainers have to push an update and
| everything on your system is secure.
|
| If everything is statically linked, you need to wait for
| every maintainer of every single program on your system
| to rebuild and push an update. You're basically
| guaranteed that there is always _something_ missing
| important patches
| rvense wrote:
| Docker images are static linking for the modern era.
| shp0ngle wrote:
| You can still do that with musl, can't you? Can make a static c
| binary with that
| quaintdev wrote:
| > The fact that it produces a single static binary is one of
| the nicest things about golang.
|
| Not only that it can also cross compile for different
| architecture and operating systems.
| lagniappe wrote:
| I made a cool program for Go projects that will compile all
| the supported OS and ARCH combos for your code. Please try it
| :) I use it for everything I make now.
|
| Just do `release --name "mycoolprogram" --version "0.1.0"`
|
| and it will output all of your labeled release binaries for
| every platform your code supports.
|
| check it out! [0] You can see it at work here for this simple
| markdown blog generator I made, which sports 40 different
| platform combos [1]
|
| [0] - https://github.com/donuts-are-good/release.sh
|
| [1] - https://github.com/donuts-are-
| good/bearclaw/releases/latest
| adql wrote:
| We used that to make a simple "VPN diagnostics" app for our
| helpdesk (app checked connectivity and config of the machine
| then displayed the summary page). Only thing that needed to
| be written per-os is how to call a browser to display the
| report
| sconi wrote:
| interesting. could you share what specifically the report
| had?
| mirekrusin wrote:
| We're bundling b/e services from typescript monorepo in
| production as single bundle files - it works very well. Main
| reason was simply enforcing lockfile from monorepo.
| neilv wrote:
| I'm often a fan of single _source_ files, at the package level,
| including inline embedded API docs and unit tests.
| avgcorrection wrote:
| > Fast-forward and we were automatically deploying Scala
| applications from CI bundled in Docker in the startup of my wife.
|
| > Last forward and I have deployed a Golang application to a
| cloud server.
|
| Editor hello?
| ethicalsmacker wrote:
| It's not always a static binary, if you use any os/config stdlib
| function calls or DNS look ups. In that case you need to specify
| CGO_ENABLED=0 to force static builds.
|
| I have been doing single binary full website deploys for about
| ~16 months in production. That includes all html, css and js
| embedded. It has been wonderful.
| adql wrote:
| And with a little bit of code you can do switching between "use
| embedded files/use local files" on the app start easily and
| have convenience of not having to re-compile app to change some
| static files.
| lenkite wrote:
| Is this article from 2016? You can do all this with Java
| nowadays. I have observed a lot of folks on HN whose last
| knowledge about Java was from a decade plus ago pontificating
| about Java deficiencies that no longer exist today.
|
| Use the GraalVM native build tools
| https://graalvm.github.io/native-build-tools/latest/index.ht....
|
| "Use Maven to Build a Native Executable from a Java Application"
|
| https://www.graalvm.org/22.2/reference-manual/native-image/g...
| Rapzid wrote:
| .Net too, also same prob with decade old worldview about .Net
|
| One thing that hasn't changed is that C# > Java ;P
| nathants wrote:
| (swirls fancy wine)
|
| pairs well with single file frontends.
| masto wrote:
| When I was stuck doing a web application in Java 15 years ago, I
| hated everything about it except for the deployment story, which
| boiled down to a single .war file being pushed to the server.
|
| When we upgraded to Perl, I liked that system so we designed
| deployment around "PAR" files in a similar way, bundling all of
| the dependencies together with the application in the CI build
| process, and I wrote a tiny bit of infrastructure that
| essentially moved a symlink to make the new version live.
|
| Google uses MPM and hermetic packaging of configuration with
| binaries: https://sre.google/sre-book/release-
| engineering/#packaging-8...
|
| The way I see it, Docker is basically this same thing,
| generalized to be independent of the
| language/application/platform. As a practical matter, it still
| fundamentally has the "one file" nature.
|
| I don't see what's special or better about compiling everything
| into a single binary, apart from fetishizing the executable
| format. In any system at scale, you still have to solve for the
| more important problems of managing the infrastructure. "I can
| deploy by scping the file from my workstation to the server" is
| kind of a late 90s throwback, but golang is a 70s throwback, so I
| guess it fits?
| pjc50 wrote:
| > I don't see what's special or better about compiling
| everything into a single binary, apart from fetishizing the
| executable format.
|
| Indeed. If you think of the docker image itself as an
| executable format like PE or ELF, this becomes clearer. Rather
| than targeting the OS API, which has completely the wrong set
| of security abstractions because it's built around "users", it
| defines a new API layer.
|
| > "I can deploy by scping the file from my workstation to the
| server"
|
| I kind of miss cgi-bin. If we're ever to get back to a place
| where random "power users" can knock up a quick server to meet
| some computing need they have, easy deployment has to be a big
| part of that. Can we make it as easy to deploy as to post on
| Instagram?
| brian_cunnie wrote:
| AWS Lambda is basically cgi-bin. Except, of course, they re-
| branded it as an exciting new technology, which I think was a
| clever move on their part.
| mrkeen wrote:
| What does cgi-bin get you? What does it run on? If I already
| have a single-file web server, do I need to introduce extra
| cgi-bin technology also?
| pjc50 wrote:
| Apache supported it, back in the day before nginix existed.
| It gives you a single-file web _page_. You can then have
| multiple pages on the same server run by different users
| under different userids.
|
| It also operates on a model of one execution run per
| request. So CGIs that aren't currently being served consume
| no resources.
| mike_hearn wrote:
| It still does support it. In fact I use it on our company
| website to handle the contact form submission. It invokes
| a Kotlin script which reads the form, handles the
| recaptcha call and sends an email. Old school but it
| works and doesn't require any resources except when
| running.
| giantrobot wrote:
| The CGI mechanism let the web server call a separate local
| binary passing parameters of the request as envars/stdin in
| a specified manner. The "cgi-bin" was just a server side
| directory where those CGI binaries lived.
|
| If I understand the GP's point, they like the idea of
| dropping a singular binary in a directory on a server and
| then it's magically available as an endpoint off the cgi-
| bin/ path.
| jacobr1 wrote:
| For anybody comfortable with the compiling the single-file
| binary ... it doesn't gain you anything. For a class of
| "power-end-users" it provides a mechanism to build
| sandboxed apps on a multi-tenant system. I think the
| spiritual successors split though between PAAS/heroku-like
| systems and "low-code" platforms.
| candiodari wrote:
| It gains you multi-tenancy on the serving end, making
| infrastructure _very_ cheap.
| tomcam wrote:
| > Indeed. If you think of the docker image itself as an
| executable format like PE or ELF, this becomes clearer.
|
| But I don't, because a docker image will not run without
| docker. A standalone, executable file can be distributed and
| deployed all by itself. A docker image cannot.
| crote wrote:
| A standalone executable doesn't remain a standalone
| executable for very long, though.
|
| You need something to handle its lifecycle and restart it
| when it dies. You need something to handle logging. You
| need something to jail it and prevent it from owning your
| system when it has a bug. You need something to pass it
| database credentials. You need something to put a cpu/mem
| limit on it. Not to mention that most executables _aren 't_
| standalone but depend on system libraries.
|
| A lot of that can be handled by systemd these days. But now
| you have a single standalone executable, its mandatory
| companion config files, and all its dependencies. Docker
| was designed to create a platform where the _only_
| dependency is Docker itself, and it does that job
| reasonably well.
| candiodari wrote:
| That used to be apache and mod_* (mod_php, mod_python,
| mod_ruby, ...)
|
| I've always wondered if it is possible to extend this
| system. Mod_docker?
| justinsaccount wrote:
| Have you ever read https://medium.com/@gtrevorjay/you-could-
| have-invented-conta... ?
|
| You were halfway there :-)
| marcosdumay wrote:
| > I don't see what's special or better about compiling
| everything into a single binary, apart from fetishizing the
| executable format.
|
| When you distribute your software to other people, it cuts the
| step of installing the correct interpreter... at the cost of
| requiring the correct computer architecture.
|
| It is very likely a gain.
| somehnguy wrote:
| Exactly - I primarily write Java and fat-jars are great when
| developing apps for environments I control. But if I want to
| send an app to a friend it's a few additional steps to make
| sure they have the correct version of Java, paths are setup
| correct, etc. This isn't always trivial if they already have
| a different version of Java and want things to play nice side
| by side.
|
| Just bundle everything into a native executable, so many
| little annoyances just disappear. From what I understand Java
| does have facilities to bundle the runtime now but I haven't
| had the opportunity to really play with it yet.
| ihateolives wrote:
| > But if I want to send an app to a friend it's a few
| additional steps to make sure they have the correct version
| of Java, paths are setup correct, etc
|
| My friends would do full stop and reverse at "install Java"
| step. It will just not fly with 99% of people. It's not
| 1999 anymore.
| lenkite wrote:
| You can use Graal Native Image
| https://www.graalvm.org/22.0/reference-manual/native-image/
| to produce a single native executable. Example of a Java
| Micro-service framework that has first class support for
| this is Quarkus (https://quarkus.io/). See
| https://quarkus.io/guides/building-native-image
|
| For a plain Java app:
|
| "Use Maven to Build a Native Executable from a Java
| Application" https://www.graalvm.org/22.2/reference-
| manual/native-image/g...
| vips7L wrote:
| > From what I understand Java does have facilities to
| bundle
|
| jlink, jpackage, and if you need something complex you can
| use conveyor: https://hydraulic.software/index.html
| mike_hearn wrote:
| Or even if you need something simple :-) Sending little
| apps to friends is easy now just like it once was, the
| difference is that the friends don't need to know what
| runtime you use. Also not only for Java: Electron,
| Flutter and anything else works too.
|
| We have an internal version of Conveyor that can be used
| to push servers as well. It bundles the jvm, makes debs
| with auto-scanned dependencies, systemd integration is
| automatically set up, it's easy to run the server with
| the DynamicUser feature for sandboxing and it
| uploads/installs the packages for you via ssh. We use it
| for our own servers. There seems to be more interest
| these days in running servers without big cloud
| overheads, so maybe we should launch it? It's hard to
| know how much demand there is for this sort of thing,
| though it gets rid of the hassle of manually configuring
| systemd and copying files around.
| pjmlp wrote:
| For 20 years that AOT compilers for Java exist, even if
| only available in commercial JDKs at enterprise prices
| (PTC, Aonix, Aicas, Excelsior, J/Rockit, Websphere RT).
|
| That alternative would be jlink, and GraalVM / OpenJ9 as
| free beer AOT.
| coldtea wrote:
| > _When you distribute your software to other people, it cuts
| the step of installing the correct interpreter... at the cost
| of requiring the correct computer architecture_
|
| Not even, as it's trivial to cross compile on Golang. Then
| you just offer 3-4 arch binaries, and they download the one
| that matches their platform.
| JCWasmx86 wrote:
| You could take it a step further and make the user download a
| small stub Actually Portable Executable
| (https://justine.lol/ape.html) which downloads the real
| binaries.
| busterarm wrote:
| Also if you had some catastrophe where you're no longer able
| to build overnight or if you have to replace 100% of your
| infrastructure, you're still able to operate because you have
| a single compiled binary to ship.
|
| It eliminates whole classes of business risk. The more hosts
| in your fleet the more risk eliminated as well.
| eddsh1994 wrote:
| > it cuts the step of installing the correct interpreter...
| at the cost of requiring the correct computer architecture.
|
| Obviously this depends on the product but I'd give anything
| to worry about interpreters over the correct computer
| architecture in the M1/M2 Intel Embedded world.
| adql wrote:
| Well, system-wise Go app is just a binary that only needs
| network access, could be run directly from systemd and just
| have all permissions set there.
|
| Docker is a bunch of file mounts and app running in separate
| namespaces. So extra daemon, extra layers of complexity. Of
| course if you're already deploying other docker apps it doesn't
| really matter, as you'd want to have that one binary in docker
| container anyway just to manage everything from same place.
| BrandoElFollito wrote:
| There is also the deployment part that is easier (au least
| for an amateur dev such as myself).
|
| I have a CI/CD template, Erin all my web stuff via a
| dockerized caddy reverse proxy, do not need to touch the
| configuration of the host (to create .service &co. files)
|
| I find deploying to docker just simpler.
| dicroce wrote:
| A single binary is nice...
|
| Perhaps second place is using the rpath origin linker option to
| create a relocatable application.
| RcouF1uZ4gsC wrote:
| I feel docker in many cases is a hack for languages and runtimes
| that don't support single file static linked binaries.
|
| Often a single binary is a simpler and better option instead of a
| docker container.
| mariusmg wrote:
| Often ?
|
| It's 100% scenario. Complex apps (like Gitea for example)
| delivered as a single binary is basically the pinnacle of
| deployment.
| dizhn wrote:
| gitea, caddy (which can update itself even with the same
| modules included), restic (again in-place updates), adguard
| home which embeds a dhcp and dns service etc etc. I really
| like the stuff the golang developers can put out.
|
| I even asked someone to produce a fresbsd binary please and
| they added one line to their github ci to make it available
| that day.
| ye-olde-sysrq wrote:
| I more view it as us recognizing that there's more to "a
| system" than a binary. Kubernetes is this concept taken to its
| conclusion (since it defines everything in code, literally
| everything). But docker is often a super convenient middle
| ground where it's not nearly as stupidly verbose to just get a
| simple thing running, but still checks a lot of the boxes.
|
| I used to feel similarly with Java. "Why," I asked, "would you
| need this docker thing? Just build the shaded JAR and off you
| go."
|
| And to be sure, there are some systems - especially the kind
| people seem to build in go (network-only APIs that never touch
| the fs and use few libraries) - that do not need much more than
| their binary to work. But what of systems that call other CLI
| utilities? What of systems that create data locally that you'd
| like to scoot around or back up?
|
| Eventually nearly every system grows at least a few weird
| little things you need to do to set it up and make it comfy.
| Docker accommodates that.
|
| I do think there's a big kernel of truth to your sentiment
| though - I loved rails as a framework but hated, just hated
| deploying it, especially if you wanted 2 sites to share a linux
| box. Maybe I was just bad at it but it was really easy to break
| BOTH sites. Docker has totally solved this problem. Same for
| python stuff.
|
| I do think docker is also useful as a way to make deploying
| ~anything all look exactly the same. "Pull image, run container
| with these args". I actually think this is what I like the most
| about it - I wrote my own thing with the python docker SDK,
| basically a shitty puppet/ansible, except it's shitty in the
| exact way I want it to be. And this has been the best side
| effect - I pay very little in resource overhead and suddenly
| now all my software uses the exact same deployment system.
| [deleted]
| anderspitman wrote:
| If you're looking for a similar deployment experience, but can't
| use Golang, we've been using Apptainer (previously Singularity)
| for a couple years at work. It's really nice to be able to get
| the benefits of containers while retaining the simplicity of
| copying and running a single file. Only dependency is installing
| Apptainer, which is easy as well.
|
| [0]: https://apptainer.org/
| zer00eyz wrote:
| From the article:
|
| "Standing here it looks like Docker was invented to manage
| dependencies for Python, Javascript and Java. It looks strange
| from a platform that deploys as one single binary."
|
| Let me say the quiet part out loud: Docker is covering up the
| fact that we don't write deployable software any more.
|
| Go isn't perfect either. The author isn't dealing with assets
| (images anyone?).
|
| I think there is plenty of room for innovation here, and were
| over due for some change.
| ElevenLathe wrote:
| Anymore? When were we writing "deployable" software in the
| past?
| zer00eyz wrote:
| I'll bite!
|
| We used to ship software, on discs, and we didn't have the
| Internet to update it.
|
| There is plenty of software out there that works via your
| linux distress package manager. Note that this isn't
| everything you can get from your package manager. Plenty of
| things that are available are broken or miersable to get
| working unless you get the container/vm version.
| tomcam wrote:
| I remember those days well. It really meant you had to be
| careful with bugs and documentation. It is not clear to me
| that the we are winning with daily or multiple times daily
| release schedules at this point.
| jrockway wrote:
| Were people careful with bugs and documentation? I
| remember the Internet blowing up one day because every
| Windows install was sending every IP on the Internet a
| virus, and there was nothing anyone could do about it.
| (And yes, Unix also had similar worms, though they
| largely predate me!) Word used to crash and corrupt your
| entire novel. There was no online banking. I'm not sure
| the rose-colored glasses are a realistic take on changing
| software quality.
|
| Today, the tools are available to move quickly and
| maintain quality. You probably do what was months of
| manual testing every time you save a file, and certainly
| every time you commit a set of changes. There are fuzz
| testers to find the craziest bugs that no human could
| even imagine. There are robots that read your PR and
| point out common errors. HN really likes to pan on
| software quality, but "I don't like this feature" is not
| a bug per se, just a company you don't like. There are a
| lot of those, but there are more lines of code than ever,
| and a lot more stuff works than 30 years ago. I think we,
| as a field, are getting better.
| marcosdumay wrote:
| You used to buy a disk, put it on your computer and run the
| thing there.
| ElevenLathe wrote:
| Isn't that just "running" the software? To me, "deployment"
| implies some repeated process that actually wasn't
| especially valuable to automate or be very careful about
| when software was released once a year (and hence the
| reason it wasn't).
|
| Also, disks are a horrible way to deploy software. They
| have all the same problems of just distributing a random
| tarball: What operating system is it for? What version?
| Where do I copy the files? How do I get the OS to
| automatically start the service on startup? What version of
| make does it use? How about which libc and cc? You can say
| this stuff in the README (or printed docs) but isn't
| something more "deployable" when it's all machine-readable
| and can be reasoned about automatically? This is what
| package managers were invented for.
| xyzzy4747 wrote:
| My father told me a story about when he was in college, he
| had to find a book in the library catalogue with the
| program he wanted, order it and check it out, and then type
| it up and test/debug it. After that all his colleagues and
| professors wanted to borrow it as well. This was in the
| 70s.
| Jtsummers wrote:
| A better question is what _is_ deployable software? How does
| it contrast from non-deployable so we can understand what we
| 're even talking about. Software gets "deployed" all the time
| so in what way is it currently non-deployable versus some
| rose tinted view of yesteryear's software?
| [deleted]
| bojanz wrote:
| _Go isn 't perfect either. The author isn't dealing with assets
| (images anyone?)._
|
| Go supports embedding assets since 1.16, the author mentions
| embedfs in the post.
| divan wrote:
| And people are using go-bindata since Go<1.0, so Go supports
| embedding assets since forever.
| acatton wrote:
| > Go isn't perfect either. The author isn't dealing with assets
| (images anyone?).
|
| From the article: "The Go web application had all files like
| configurations (no credentials), static css and html templates
| embedded with embedfs (and proxied through a CDN)."
|
| See https://pkg.go.dev/embed
| zer00eyz wrote:
| OP here,
|
| How are you doing caching without a mod time?
|
| https://github.com/golang/go/issues/44854
|
| Are you re-naming or hashing for cache clearing?
| acatton wrote:
| I'm not the author of the post, so I can't tell you what
| the author does.
|
| What I do in my projects is that I tell varnish-cache to
| cache assets in "/static/..." forever. And I have a "curl
| -X PURGE <varnish_endpoint>" as part of the "ExecStartPre="
| of my go binary.
|
| * https://varnish-cache.org/docs/trunk/users-
| guide/purging.htm...
|
| * https://www.freedesktop.org/software/systemd/man/systemd.
| ser...
| galdor wrote:
| I'm unsatisfied with the current situation too, but it's a hard
| problem. You can either go full container (which means no BSD,
| and having to deal with Docker or Kubernetes and all the
| associated woes), or fallback to native packages which are a
| huge PITA to build, deploy and use.
|
| I think that Nix and Guix have part of the solution: have a way
| to build fully independent packages that can be easily
| installed. But I'm not comfortable with the complexity of Nix,
| and Guix does not run on FreeBSD. And ultimately you still have
| to handle distribution and configuration of the base system you
| deploy on.
|
| Innovation is possible, but there are a lot of expectations for
| any system dealing with building and deploying software. I feel
| that there are fundamental limitations inherited from the way
| UNIX OS work, and I wish we had lower level operating systems
| focused on executing services on multiple machines in a way
| similar to how mainframes work. One can dream.
| xyzzy4747 wrote:
| Just curious, what's the benefit of using FreeBSD versus
| Debian/Ubuntu?
| indymike wrote:
| I'm really growing tired of significant effort development
| effort going to dealing with deployment on the part of our
| stack that isn't written in Go. The Go side, deployment is
| replace binary, restart app, done. The Python and Javascript
| code we maintain takes significant effort to deploy, and
| builds can be brittle due to dependency issues.
|
| > I feel that there are fundamental limitations inherited
| from the way UNIX OS work
|
| There are, but the way go does deployments plays to Unix's
| strengths.
| pjmlp wrote:
| Java never needed Docker, in fact the Docker / Kubernetes is
| now re-inventing Java Application Servers with WASM containers.
| Rapzid wrote:
| > we don't write deployable software any more
|
| What did it use to look like exactly, this "deployable"
| software? Going back to the birth of web 2.0 we had Perl, PHP,
| Java(?), .Net Framework a few years later. These all required
| tons of pre-configured infrastructure on the servers to run..
|
| > It looks strange from a platform that deploys as one single
| binary
|
| It's just a tool with many uses. I CAN deploy my Asp.Net app as
| a self-contained(even single) file.. But the size of updates is
| smaller between images if I copy the app into an image that
| already has the .Net and Asp.Net library code in the base
| layers.
| joncfoo wrote:
| Assets of any kind can be embedded in the executable and
| accessed via the embed.FS interface. This makes it trivial to
| bundle up all dependencies if desired.
| candiodari wrote:
| Of course Visual Basic 2.0 and Delphi 1.0 both had embeddable
| filesystems. Even updateable embedded filesystems (which
| worked because the exe would really be a zip files. Zip files
| are indexed from the _end_ of the file. So you can prepend
| the actual executable code and that would work. Zip files are
| updateable ...)
|
| I believe after a while you also had sqlite-inside-the-exe
| things.
| cytzol wrote:
| Embedding your assets like this isn't always an improvement.
| For example, I work on a site with a Go server and static
| content pages, and I like that I can update one of the pages
| and see the change instantly without having to re-compile the
| entire server binary just to get the new files included.
___________________________________________________________________
(page generated 2023-03-22 23:00 UTC)