[HN Gopher] Life without line numbers for 6% smaller Go binaries
___________________________________________________________________
Life without line numbers for 6% smaller Go binaries
Author : caust1c
Score : 43 points
Date : 2020-07-21 15:35 UTC (7 hours ago)
(HTM) web link (commaok.xyz)
(TXT) w3m dump (commaok.xyz)
| ottoboney wrote:
| Your HTTP cert is invalid. Just a heads up.
| msbarnett wrote:
| If it's showing up as invalid for you, you may have locally
| revoked trust of the DST Root CA X3 IdenTrust root certificate.
| donmcronald wrote:
| It's fine for me and was issued May 26. The URL when I visited
| was https://commaok.xyz
| rnotaro wrote:
| His browser might have oppened the following link:
| https://www.commaok.xyz/post/no-line-numbers/ somehow.
|
| It's serving a certificate for github.com.
| jeffbee wrote:
| Since the Go compiler is so primitive you can strongly influence
| the size of the program by just restructuring it. An example: if
| you have a function that converts integer return codes to errors,
| and you have another function that calls it a lot:
| func codeToError(i int) error { switch i {
| case ... ... } } func stuff()
| error { if rv := callJunk(); rv < 0 { return
| codeToError(rv) } if rv := moreJunk(); rv <
| 0 { return codeToError(rv) } ...
| return nil }
|
| You're going to get multiple whole copies of the inner function
| inlined into the outer one. This can add up, if the outer
| function is long and/or if the inner switch statement is huge.
|
| You can save a lot of code by jumping through an adapter function
| instead, like: func callWithErrorsConverted(f
| func() int, g func(int) error) error { return g(f())
| }
|
| ... because now the compiler doesn't get to inline g into f.
| These things make a big difference in whole-program size. I wrote
| about it in the context of tailscale and protobufs here:
|
| https://paper.dropbox.com/doc/Bloaty-Puffs-and-the-Go-Compil...
| rowanG077 wrote:
| Pretty insane that the compiler for such a popular language is
| so shitty.
| pantalaimon wrote:
| I'm pretty happy I don't have to worry about such details in C
| and can rely on the compiler to do the easy optimizations for
| me.
| ascar wrote:
| Wait, isn't that trick basically about avoiding optimization
| (inlining the function to avoid the context switch) to reduce
| code size?
|
| The C compiler would do something similar here and also
| increase code size for performance unless you explicitly use
| `-Os`
| monocasa wrote:
| The better C compilers use better heuristics in these
| cases, knowing that binary size can mean I$ pressure and
| therefore be a detriment to performance.
| jeffbee wrote:
| It's not really a language problem, it's a compiler problem,
| so at least in the future we might get a compiler that
| recognizes error-handling idioms. Or we just get llvm? The
| language problems, like the performance-hostile calling
| convention, are harder to fix.
| varbhat wrote:
| https://blog.filippo.io/shrink-your-go-binaries-with-this-on...
|
| This is great article instructing how to shrink the Go Binaries.
| Author is lead of cryptography and security team of Go Team at
| Google.
| ed25519FUUU wrote:
| tl;dr for those wondering: strip debugging and then compress
| the binary with upx: go build -ldflags="-s
| -w" main.go upx --brute main
|
| Just tried that on one of my binaries and went from 12 mB down
| to 3.6 mB.
| distortedsignal wrote:
| Completely agree.
|
| I performed the steps in the article and reduced the output
| binary size from ~35 MB for raw go build to around 29 MB for go
| build with the ldflags all the way down to 8.3 MB for
| compressed with upx.
|
| Really impressive results.
| Karunamon wrote:
| Does UPX still trigger virus scanners, though? As much as I
| like the idea of tinier binaries, the chance that your user's
| endpoint security is going to freak out makes it not worth
| the hassle.
| kingnothing wrote:
| 35MB seems relatively small already. What do you get from an
| 8MB binary that you don't from one 4x that size?
| Rebelgecko wrote:
| I'm not familiar with UPX compression so it might
| invalidate what I'm about to say. However it can be
| beneficial for performance if you fit more of your
| executable into CPU cache (actual results may vary based on
| access patterns and many other factors)
| makapuf wrote:
| UPX is just a (nice) compresser for binaries, where
| running the executable really runs a small decompressor
| on the binary, gets a binary in ram and runs it. So your
| binary is not smaller in cache when executed.
| smabie wrote:
| 35mb seems small to you? That's an absolutely massive
| binary.
|
| But to answer your question, you're going to see some
| performance improvements if your binary can fit into lower
| levels of the CPU cache.
|
| Kdb+/q for example is less than 700kb, which mean that the
| entire thing can fit into the L1 instruction cache on high
| end server CPUs. And that size is even more impressive
| considering its only dynamically linked to libc and
| libpthread.
|
| This small size contributes to the out of this world
| performance the system has. And remember that's the
| language and the time series db.
| qmmmur wrote:
| 27MB. Where that is important is context specific
| pantalaimon wrote:
| > 35MB seems relatively small already
|
| 35 MB is huge! Are there any assets inside the binary?
| cesarb wrote:
| > 35 MB is huge!
|
| For context, 35MB is two and a half _boxes_ of high-
| density floppy disks. There were whole operating systems,
| complete with applications, smaller than that.
| pantalaimon wrote:
| _Were?_ My initrd + vmlinuz is still smaller than that
| today on a stock Ubuntu kernel.
| birdyrooster wrote:
| I wonder how much more from javascript minifying can be reused in
| this space.
| coldtea wrote:
| Probably not much? Javascript minifying is about raw source
| code, whereas those are binary instructions + debug info.
| kardos wrote:
| How much better does a 6% smaller go binary perform?
| jeffbee wrote:
| Shrinking the line table will be irrelevant to performance. As
| I understand it, the tailscale binary has a hard size
| requirement to run as a VPN under iOS.
| svnpenn wrote:
| This didnt seem to change anything for me. I am currently using
| Go 1.14.4 (Windows), and after trying go1.15beta1, the resultant
| executables are actually larger.
|
| Another thing Ive always found strange is people always recommend
| (-ldflags -w), when (-ldflags -s) gives smaller result, and
| (-ldflags '-s -w') is identical to (-ldflags -s).
| skykooler wrote:
| Given the article is talking about DWARF debugging formats, I
| believe the numbers in the article are for Linux binaries. I'm
| not sure if Windows binaries store line numbers in the same
| way.
| ygra wrote:
| Doesn't Windows usually just use a .pdb file next to the
| exe/dll, containing the debugging information? So stripping
| debug information on Windows is just deleting (or not
| distributing) that file.
| mehrdadn wrote:
| Not entirely, there's also a debug directory embedded
| inside executables that gets stripped.
| kjksf wrote:
| Yes, .pdb files are used by most toolchains for debug
| information on Windows.
|
| No, Go doesn't use .pdb files.
|
| On all platforms Go stores a significant amount of meta-
| data in the binary, in the same format so that they can
| access it using the same code on all platforms.
|
| This information is used by the runtime. Precise garbage
| collector needs to know the layout of structures and stack
| to know which fields are pointers; To generate readable
| callstack Go needs info about function and where they come
| from (source code file name, position). etc.
|
| A sidenote: you can convert DWARF debug info to .pdb using
| external tool which allows debugging Go programs with
| Windows native debuggers like WinDBG.
___________________________________________________________________
(page generated 2020-07-21 23:00 UTC)