[HN Gopher] Running Intel Binaries in Linux VMs with Rosetta
___________________________________________________________________
Running Intel Binaries in Linux VMs with Rosetta
Author : gok
Score : 214 points
Date : 2022-06-06 19:35 UTC (3 hours ago)
(HTM) web link (developer.apple.com)
(TXT) w3m dump (developer.apple.com)
| rudolph9 wrote:
| How would this affect running an x86 container on a M1/M2 Mac?
| therealmarv wrote:
| really good question... is a x86 container already
| bootstrapping or can we put it through Rosetta (instead through
| QEMU) to get it run?
| olliej wrote:
| It explicitly states it doesn't support bootstrapping - you
| still need your OS to actually run on arm64
| speedgoose wrote:
| I think it's already the case. The kernel is arm64 and the
| amd64 containers are running with qemu. Most containers are
| fine but I experienced some issues, with maintainers refusing
| to support this setup.
|
| I understand that you can replace qemu by rosetta now.
| [deleted]
| saagarjha wrote:
| Containers are virtualized on macOS. x86-64 containers have
| used qemu to run the whole system under emulation, which is
| pretty slow. This workflow continues to require QEMU;
| however, now you can create an arm64 container and run
| x86-64 code in that using this technology.
| dilyevsky wrote:
| What is arm64 container? Container is just a namespaced
| process so if your binary is x86 it is still technically
| x86 container just run using emulation. And qemu user
| static already does translation a la rosetta 2 so there
| probably wont be much difference in speed
| saagarjha wrote:
| Docker contains a Linux kernel, does it not?
| dilyevsky wrote:
| Nope, docker just sets up linux cgroups/namespaces and
| spawns a new linux process, no kernel of its own.
| cpuguy83 wrote:
| Docker Desktop (ie Docker on Mac) absolutely sets up a
| VM.
| dilyevsky wrote:
| Correct, on macs it spawns a (usually aarch64) vm using
| apples hypervisor framework api (hw virt) and runs docker
| daemon inside of it. Then it's what I described above -
| just an process running arm instructions. If you're using
| qemu binfmt_misc runner which uses "user mode emulation"
| not to be confused with system emulation which does full
| blown hw emulation. So it's aready doing similar things
| to rosetta 2
| cpuguy83 wrote:
| Yes qemu and Rosetta perform the same basic role here
| (bintfmt handler for x86 bins), but Rosetta should
| perform significantly better (otherwise there would be
| virtually no reason at all to build this).
| cpuguy83 wrote:
| The whole point in providing Rosetta is because it
| (should) be significantly faster than plain qemu.
| dilyevsky wrote:
| If by plain qemu you mean its user mode emulation then i
| think it remains to be seen what the performance
| improvement will be. User mode already can be ok if it's
| not doing endianness translation
| cpuguy83 wrote:
| Instead of running in qemu (as it currently would), it would
| run under Rosetta.
| siggen wrote:
| If I cannot use Apple Boot Camp to boot into another OS and avoid
| macOS, I'm not really interested in non-Intel macs.
| bfgoodrich wrote:
| 58028641 wrote:
| The effort to support Linux has made decent progress.
| https://asahilinux.org Apple said Microsoft would have to
| cooperate to get Window working.
| https://www.macrumors.com/2020/11/20/craig-federighi-on-wind...
| Retr0id wrote:
| Do any hypervisors support this yet? (i.e. can I try it out right
| now?)
| saagarjha wrote:
| No, this requires changes to use new APIs. I expect the major
| VM developers to adopt it in the coming days and weeks.
| garaetjjte wrote:
| Virtualization.framework is the hypervisor itself, you only
| need simple tool to launch it. You probably could just copy-
| paste provided code into eg. https://github.com/gyf304/vmcli.
| However macOS 13 beta currently seems to be only available for
| registered developers.
| foodstances wrote:
| You're asking about something that was just released in a beta
| version of an OS announced a couple hours ago.
| easton wrote:
| There will be a session posted tomorrow from Apple on how to
| use their sample to run a Linux VM with Rosetta support. I
| wonder if the sample on this page will do the trick by itself?
|
| https://developer.apple.com/documentation/virtualization/run...
| PeopleB4Cars wrote:
| Can Apple please just make a simple Virtualization front end ala
| libvirts Virtual Machine Manager. They seem to have all the bits,
| just need to bring them all together.
| zaptrem wrote:
| Seems like they're leaving that part up to the dev community
| (which I'm fine with). They'll never be able to make every
| person and every use case happy, so better to just give us the
| tools to do so ourselves. Check out UTM if you want a GUI for
| hypervisor.framework powered VMs.
| a-dub wrote:
| interesting. so this fixes the major performance issue by
| allowing users to install a user installable rosetta inside user
| created vms.
|
| could still end up with some "works for me" but still broken on
| prod issues resulting from the underlying architecture being
| different (for intel backing infra), and also some questions
| around how it would work in practice in terms of integrating with
| production container workflows, but seems like a boon for anyone
| who is struggling today with intel vm or container performance
| issues on apple silicon.
|
| nice!
| MBCook wrote:
| There is a specific piece of x86-64 software I need for my job,
| a back end service. It meant there was no way for me to use an
| M1 Mac for my job. I wish this was available last year, maybe I
| could've upgraded to an M1 based Mac.
| a-dub wrote:
| when the m1 initially shipped, i was watching this issue like
| a hawk. rosetta was really cool technology, but they seemed
| to really miss the mark in terms of many developer workflows
| that involve intel vms and containers.
|
| it was particularly interesting to me as i always got the
| sense that a lot of the success of the intel era macs was
| carried by their popularity and (and recommendability)
| amongst the internet development and engineering crowd. it
| seemed to me like a mistake to potentially alienate that
| group.
| MBCook wrote:
| I figured it was largely a time management thing. They
| could only do so much (I mean they moved the whole base OS
| architecture... again).
|
| Glad to see it.
|
| Hope that by the time I get my next work laptop it's still
| there, or better yet we've moved to a newer version of the
| software where this isn't a problem.
| dilyevsky wrote:
| Ime podman (or docker) + qemu-user-static covers nearly
| everything. There are some issues with older go binaries
| emulation but most of c/c++ x86 code works as is
| a-dub wrote:
| what's the performance cost vs. intel code on the host
| under rosetta2 and native arm code on the host?
| dilyevsky wrote:
| I think it depends greatly on what instructions are being
| used (sse, avx, etc). Some benches I've seen boast
| rosetta2 is only 30% hit but that's doubtful to me based
| on my own experience running factorio =)
| cpuguy83 wrote:
| I don't know about "production container workflows" on a Mac,
| however the only thing that needs to be done is to use the new
| Rosetta binfmt handlers instead of qemu (which Docker Desktop
| already sets up for you). I imagine Docker will have an option
| to use Rosetta instead of qemu.
| jellyksong wrote:
| Basic question: Why is this faster than running Intel Linux
| apps in an emulated Intel Linux VM? Because Rosetta is faster
| than QEMU, and you only need to emulate one application rather
| than the entire kernel?
| nicoburns wrote:
| It's because Rosetta is able to do AOT compilation for the
| most part. So it actually converts the binary rather than
| emulating it at run time.
| saagarjha wrote:
| Correct, plus Rosetta is substantially faster than QEMU
| because of AOT (as others mentioned) as well as a greater
| focus on performance.
| [deleted]
| a-dub wrote:
| a lot of the performance gains in rosetta 2 come from load
| time translation of executables and libraries. so when you
| run a binary on the host mac, rosetta jumps in, does a one
| time translation to something that can run natively on the mx
| processor and then (probably) caches the result on the
| filesystem. next time you run it, it runs the cached
| translation. if you're running a vm without support inside
| the guest for this, then you're just running a big intel blob
| on the mx processor and having to do realtime translation
| which is really slow. (worst case, you have an interrupt for
| every instruction, as you have to translate it... although i
| assume it must be better than that. either way you're
| constantly interrupting and context switching between the
| target code you're trying to run and the translation
| environment, context switches are expensive in and of
| themselves, and they also defeat a lot of caching)
| brigade wrote:
| Emulation of an x86 kernel level means you lose the hardware-
| assisted virtualization support you'd get with an ARM kernel,
| and emulating an MMU is _slow_ (among other things.)
|
| Technically this would be replacing QEMU user-mode emulation.
| Which isn't fast in a large part because QEMU being portable
| to all host architectures was more important than speed.
| taejavu wrote:
| Does this mean I can run MSSQL Server on an M1 Mac?
| cassandratt wrote:
| I mean, you can do that now. So, yes, but not because of this.
| eckza wrote:
| Soooo does this mean that Docker will suck less on macOS, Real
| Soon Now?
| samwillis wrote:
| I believe the main issue is file syncing between macOS and the
| container host OS.
|
| I don't have a M1 mac, but on my Mac a standard Linux VM has no
| impact on performance or battery life. The minute I use Docker
| the machine become sluggish and halves (literally) the batter
| life, I now avoids is as much as possible.
| zamalek wrote:
| Try out Colima with the experimental 9p mount type. Network
| performance is improving. It's as good as it will likely get
| on the platform.
| laurencerowe wrote:
| Have you tried enabling "Use the new virtualization
| framework" in experimental features? This removed the
| background CPU usage for me.
| saagarjha wrote:
| This can't boot x86-64 containers, but you can create an arm64
| container and run x86-64 code in it.
| WatchDog wrote:
| Docker on macOS already has some kind of qemu based solution
| for running x86_64 containers on m1.
|
| Potentially this could be used as an alternative?
| zamalek wrote:
| It's QEMU emulation. It's slow. It also segfaults a lot. I
| am able to run a select few containers on it, but I needed
| to sort out native arm64 for the vast majority.
| akdor1154 wrote:
| I don't think that's right - containers are just userspace
| code, nothing to boot.
|
| By my reading of TFA, your container runtime (eg Docker for
| Mac) should be able to provide an ARM VM, and use this
| Rosetta feature to run amd64 containers directly.
| duskwuff wrote:
| I have to admit to a little bit of trepidation about the
| Rosetta mount point. Will this work for Docker without
| explicitly passing that mount point through to the
| containers?
| cpuguy83 wrote:
| The way this would work is Docker would be setting this up
| for you (maybe through some option to switch between qemu
| and Rosetta).
|
| The containers should not need the mount. What happens is
| Rosetta gets registered with the kernel (binfmt_misc) to
| execute x86 binaries. The is the same mechanism that allows
| seamless qemu support.
| saagarjha wrote:
| Yeah I'm not entirely sure how they're going to make this
| work, but I kind of understand why it's so weird, because
| there's really not any very good way to communicate between
| macOS userspace and some arbitrary kernel running inside
| the VM...
___________________________________________________________________
(page generated 2022-06-06 23:00 UTC)