[HN Gopher] A Way Out for A.out
___________________________________________________________________
A Way Out for A.out
Author : bitcharmer
Score : 196 points
Date : 2022-03-24 16:23 UTC (6 hours ago)
(HTM) web link (lwn.net)
(TXT) w3m dump (lwn.net)
| flakiness wrote:
| This is very good example how engineering discussion should
| proceed:
|
| The project lead (Linus) pushed back first with reasonable-
| looking point, but it is hit back by a thorough research. And
| finally a clever workaround broke it through.
|
| It was very energizing to me who has habit to rely on non-
| engineering tools to push through these kind of discussions.
| __turbobrew__ wrote:
| If only the python devs had a similar attitude.
| zamalek wrote:
| And engineering in general, it is a great day when someone
| notices that something is unused and pushes for its removal.
| bityard wrote:
| Speaking as someone who maintains a fair bit of legacy
| infrastructure for a living, the hard part is knowing what is
| unused and what isn't. That's the whole rationale behind
| Linus' "don't break user space" policy.
| copperx wrote:
| A quick trivia question: when you compile a program using GCC and
| do not specify the name of the output binary, an A.out file gets
| created. Is that A.out binary using the A.out format, or is it an
| ELF binary?
| gumby wrote:
| The name and the format are distinct. The linker output (the
| a.out file) will be in whatever is considered native by the
| compiler, which for most people is the format of the system
| they are running the compiler on.
|
| "a.out" is barely a format: just a block of executable, a block
| of data initialization and an integer to say how large the
| (uninitialized) heap should be. Later some symbols and a bit of
| debugging info was added. But remember back then a lot of stuff
| was still written in assembler and machines were not that
| powerful, so something simple was not only sufficient but
| better.
|
| Separating code and data wasn't even really needed but back
| then it wasn't clear that (almost) every machine would be a von
| Neumann architecture. There were still plenty of Harvard
| architecture machines (split I/ D) space. Of course these days
| we've partially circled back with write protected executable
| blocks and execute permission forbidden to data and stack.
|
| When I started designing bfd in 89 or 90 I started with a.out.
| There were already other formats (predominantly forms of COFF)
| because the limitations of a.out had started to be a problem.
| GauntletWizard wrote:
| I was just thinking about this in the shower. The death of
| the Harvard architecture combined with the slow adoption of
| w^x seems to vindicate the problem, but I think there's a
| deeper truth to it: Turing complete machines can emulate any
| other Turing complete machine. Harvard architecture is immune
| to stack-smashing, but it's not immune to privilege
| escalations - Fundamentally, there's a usability tradeoff of
| "Loading new executable code from the data segment" that
| makes true Harvard architectures inoperable. We've settled on
| a happy if imperfect medium.
| kibwen wrote:
| WASM is a Harvard architecture, so it's not as dead as one
| might think. :)
| gumby wrote:
| I believe this is for the same reason some
| microcontrollers do it, to whit there is no page table
| and much less page access bits. These days the browser is
| a greater thread vector than ordinary application code.
| gumby wrote:
| Van neumann is more general and provides more flexibility
| (what if you need more data space and don't need all that
| code space?).
|
| There are still new harvard architecture microcontrollers
| being designed. It provides some security in the field as
| well as simplifying the chip layout.
| ludamad wrote:
| It'd be pretty annoying for a JIT compiler to have to
| manage too. Java can do a garbage collection with barely
| any memory left - cool! But, it'd be a shame if similar
| tricky stuff was written just to e.g. decompile code just
| to make way for more code space. Pages with only one of
| 'write' or 'execute' privileges work well enough
| samus wrote:
| These use cases could be served by asking the operating
| system or the hardware to copy data to the code memory.
| Weakens absolute protection, but it would be an escape
| hatch to enable JIT compilers.
| GauntletWizard wrote:
| That's half the definitional difference between Harvard
| and Von Neumann architectures, though - That never the
| twain shall meet.
|
| There's a second insight on the tip of my tongue, here:
| Harvard architectures can emulate Von Neumann ones, and
| vice versa. And problems, specific problems, can be
| solved by either. But it seems to me that there's
| something interesting about emulation too - That Harvard
| architectures can open themselves up to precisely the
| same vectors of attacks if you enable "Copy this data
| block to code memory".
| gnufx wrote:
| An aside on BFD, which doesn't seem to have come up, and I
| suspect relatively few people have read the History node of
| the 'Binary File Descriptor' Library manual on the back-
| constructed name: "The name came from a conversation David
| Wallace [gumby] was having with Richard Stallman about the
| library: RMS said that it would be quite hard--David said
| "BFD". Stallman was right, but the name stuck.". [I gather
| that's 'Big', 'Deal', and something like an adjective.]
| gumby wrote:
| That's basically correct. The first time I gave a talk on
| the subject, I was asked what the acronym stood for. I had
| to come up with an explanation on the spot and casually
| said "Oh, it's Binary File Descriptors" as if that had
| always been the name. And that's how it got an official
| name.
| hoten wrote:
| That also was my question. The wikipedia article on the format
| addresses it in the second paragraph
|
| > "a.out" remains the default output file name for executables
| created by certain compilers and linkers when no output name is
| specified, even though the created files actually are not in
| the a.out format.
|
| https://en.wikipedia.org/wiki/A.out
| necheffa wrote:
| It has been ELF for a while now.
| jandrese wrote:
| Since around 1995 to be specific.
| icedchai wrote:
| I remember manually migrating my Slackware install from
| a.out to ELF around that time. Maybe it was 1996.
| anyfoo wrote:
| Me too, then later the same again with libc5 to glibc.
| Obviously learned a _ton_ about the system by manually
| reconstructing it that way (using only source tarballs,
| no packages).
| deckard1 wrote:
| '96 feels about right. I, too, was on a.out briefly on
| Slackware before the big ELF migration. Was actually
| rather smooth. Not nearly as bad as the lib32 to lib64
| pain train
| flatiron wrote:
| On Linux. Did FreeBSD switch?
| toast0 wrote:
| Yeah; looks like in the FreeBSD 3.0 release (October 1998),
| mentioned in old versions of the FreeBSD handbook[1]:
|
| > FreeBSD comes from the ``classic'' camp and used the
| a.out(5) format, a technology tried and proven through many
| generations of BSD releases, until the beginning of the 3.X
| branch. Though it was possible to build and run native ELF
| binaries (and kernels) on a FreeBSD system for some time
| before that, FreeBSD initially resisted the ``push'' to
| switch to ELF as the default format.
|
| [...]
|
| > So ELF had to wait until it was more painful to remain
| with a.out than it was to migrate to ELF.
|
| > However, as time passed, the build tools that FreeBSD
| derived their build tools from (the assembler and loader
| especially) evolved in two parallel trees. The FreeBSD tree
| added shared libraries and fixed some bugs. The GNU folks
| that originally write these programs rewrote them and added
| simpler support for building cross compilers, plugging in
| different formats at will, and so on. Since many people
| wanted to build cross compilers targeting FreeBSD, they
| were out of luck since the older sources that FreeBSD had
| for as and ld were not up to the task. The new GNU tools
| chain (binutils) does support cross compiling, ELF, shared
| libraries, C++ extensions, etc. In addition, many vendors
| are releasing ELF binaries, and it is a good thing for
| FreeBSD to run them.
|
| > ELF is more expressive than a.out and allows more
| extensibility in the base system. The ELF tools are better
| maintained, and offer cross compilation support, which is
| important to many people. ELF may be a little slower than
| a.out, but trying to measure it can be difficult. There are
| also numerous details that are different between the two in
| how they map pages, handle init code, etc. None of these
| are very important, but they are differences. In time
| support for a.out will be moved out of the GENERIC kernel,
| and eventually removed from the kernel once the need to run
| legacy a.out programs is past.
|
| It looks like default support of a.out was removed in 5.0
| (January 2003), but you can still load a kernel module
| (shipped with the generic kernel) or compile a custom
| kernel with support built in. At least on i386/amd64.
|
| [1] https://docs.freebsd.org/doc/4.9-RELEASE/usr/share/doc/
| handb...
| bell-cot wrote:
| From a quick look at the FreeBSD 13.0-RELEASE Release Notes
| ( https://www.freebsd.org/releases/13.0R/relnotes/ ) and
| the a.out-related commits that it links to, I'd say that
| FreeBSD is close to having a.out fully walked to the door.
| (They suggest installing a ~2-decade-old version of FreeBSD
| if you still need to work with a.out...)
| toast0 wrote:
| I see that in the release notes, but they still have the
| kernel module available in GENERIC, and the config
| available. Maybe they don't do anything though :)
| sedatk wrote:
| Is it even possible to create a.out format binaries on the
| latest GCC? "objdump -i" doesn't show it as an option on my
| Ubuntu.
| mueddib wrote:
| yes. please learn -g3 and -Wl and --oformat parameter (gcc).
| objdump -i only display (info).
| mueddib wrote:
| Wl (linker) oformat (output format try a.out)
| technothrasher wrote:
| Why a.out format, of course... on my Sun 3/80.
| jandrese wrote:
| Halfway through the article I was wondering if it wouldn't be
| possible to write an ELF wrapper for a.out binaries. I was
| pleasantly surprised to not be the first person to consider that,
| and that it turned out to be easy to do.
| pavon wrote:
| I like the binfmt_misc suggestion in the LWN comments. The fact
| that I could just run a jar file like it was an executable made
| me unreasonably happy when I first learned about it. It was one
| of those things (like X11 being network transparent) that made
| Linux seem almost magical to me as a 90's teen. Anyway, this
| seems like a perfect use case for that functionality.
| duskwuff wrote:
| Even crazier: you can use binfmt_misc with qemu-user to
| transparently run Linux executables compiled for other
| architectures. Debian (and Ubuntu) packages the required
| configuration as "qemu-user-binfmt".
| gnufx wrote:
| For what it's worth, qemu-user-static is more general, e.g.
| for a foreign container image that won't find the shared
| library. (Wheeled out when it was said people needed to
| build ppc64le images on x86 boxes and it was impossible,
| rather than trivial.)
| lloeki wrote:
| That's exactly what Docker does behind the scenes to run
| foreign arch images, see
| https://github.com/tonistiigi/binfmt
| flatiron wrote:
| Are you referring to
| https://www.freebsd.org/cgi/man.cgi?query=brandelf&sektion=1 ?
| jabyess wrote:
| Once again, XKCD[0] proved to be right
|
| [0] https://xkcd.com/1172/
| larusso wrote:
| Thank you sir. I had to laugh so damn hard. It's such a nice
| parody about these discussions in the kernel which I actually
| love. I would appreciate if more software producers would stop
| breaking or removing features because they think in percentages
| of users who will be pissed etc. on the other hand it can
| become quite silly.
|
| I had a good laugh thanks again :)
| linkdd wrote:
| I wrote a french translation to this article[0] without noticing
| I was in violation of their copyright (my bad).
|
| Quickly sent a mail to LWN and got their authorization in the
| next 10 minutes.
|
| Big up to the team and awesome content! This motivated me to buy
| a subscription to support them. [0] -
| https://linuxfr.org/users/linkdd/journaux/lwn-une-porte-de-
| sortie-pour-a-out-63380a9d-20e0-44de-979d-74afcb6f3910
| genr8 wrote:
| Kees Cook is a legend. Actually wrote a new ELF wrapper for a.out
| binaries !
| 1vuio0pswjnm7 wrote:
| The NetBSD kernel still has support for a.out. On BSD, a.out is
| something simpler than ELF that works without any problems.^1 On
| Linux, a.out may have caused problems when building shared
| libraries thus motivating the switch to ELF.^2
|
| 1. ELF is OK, but times have changed since ELF was created. The
| same constraints no longer exist. Personally, I prefer static
| binaries and can afford the space for them. It is good to have
| options for people who prefer shared libraries as well as those
| who prefer static binaries.
|
| 2. https://people.freebsd.org/~nik/advocacy/myths.html
| TeeMassive wrote:
| > It seems to be a universal rule that somebody always has to
| come along to ruin the party. In this case, just as it seemed
| like there were no further obstacles to the removal of a.out,
| James Jones showed up to let it be known that he was still using
| a.out:
|
| I find it surprising that the Kernel doesn't have a formal and
| easy way to know who is using what for the purpose of dropping
| support except to go fishing for complaints and hoping by chance
| nobody bites.
|
| There should be something like a list of legacy features on
| kernel.org where people can easily subscribe to a mailing list
| which will ask for them to know if they still need the feature
| when there are plans to phase it out. A phase-out list so to
| speak.
| lloydatkinson wrote:
| Windows and macOS solves this by the much hated "user tracking"
| (aka analytics).
| Zababa wrote:
| I'm not sure if they "solve" this. In this current case, this
| was solved by working with the person that need the feature,
| rather than just tracking their usage of said feature.
| anyfoo wrote:
| Anyone can start using Linux as part of their product or
| environment, and so today there are literally billions of
| resulting end users effectively using Linux systems with
| uncountable configurations.
|
| Unless you want at least every developer and system
| administrator in the world to subscribe to that mailing list
| and regularly read it, I don't know how this can really solve
| the problem.
|
| Add to that the fact that the classification of a "feature" is
| rather fuzzy...
|
| There are already announcements and discussions through various
| channels, but at the end you can never be sure. Fortunately
| systems don't have to always be updated immediately (aside from
| for security concerns).
| TeeMassive wrote:
| I see the workflow something like this:
|
| 1) Some old feature, like an obscure filesystem, is causing
| some pain to maintain.
|
| 2) Therefore people suggest to add it to the phase-out list.
|
| 3) After some discussion it is decided it is a worthy
| candidate for removal and so it is placed on the phase-out
| list.
|
| 4) Some people starts subscribing, but not too much. Turns
| out it's some obscure distribution for an Atari emulator.
|
| 5) Time pass and years later these people have moved on to
| better things, they reply to the reminder email saying it's
| fine now.
|
| 6) Feature gets removed safely.
| donatj wrote:
| I'm curious how much overhead and developer-time keeping a.out
| support adds?
|
| I'm generally of the mindset to keep something until there's
| reason to remove it.
___________________________________________________________________
(page generated 2022-03-24 23:00 UTC)