[HN Gopher] FreeBSD on Firecracker
___________________________________________________________________
FreeBSD on Firecracker
Author : jmmv
Score : 184 points
Date : 2023-08-24 18:52 UTC (4 hours ago)
(HTM) web link (www.usenix.org)
(TXT) w3m dump (www.usenix.org)
| laurencerowe wrote:
| It's a shame neither AWS nor macOS on ARM support nested
| virtualization. It would would make it far easier to develop and
| deploy Firecracker based tech.
| saurik wrote:
| FWIW, AWS a1.metal instances are pretty small and thereby
| reasonably cost effective for working with virtualization tech.
| Rapzid wrote:
| Their metal offerings are puny in general though(as in, not a
| ton of options).
| znpy wrote:
| Afaik you can do virtualisation on the .metal variants.
|
| Actually you can do virtualisation on any instance type afaik,
| but only with .metal instances you can use hardware
| acceleration.
| tedunangst wrote:
| It's funny how many one second pauses turn out to be less than
| necessary. How many sysadmins took meaningful action because the
| system paused when they had an invalid machine uuid?
| cperciva wrote:
| Probably a significant proportion of the sysadmins who
| experienced that one-second pause.
|
| The "print a message telling the user that we're rebooting,
| then wait a second to let them read the console before we go
| ahead and reboot", on the other hand...
| mnsc wrote:
| Here's the recent BSDCan talk from Colin that was posted a couple
| of days ago.
|
| https://youtu.be/MT3cdeuRTzs?si=l6baNriUjcvy0ZOE
| cperciva wrote:
| FWIW, this is basically the same material -- after my BSDCan
| talk the FreeBSD Journal said "hey, that was a great talk, can
| you turn it into an article for us", and after the FreeBSD
| Journal article was published ;login: asked if they could
| republish it.
| getcrunk wrote:
| So firecracker vs v8 isolates if only doing js or wasm?
| doublerabbit wrote:
| I'm not wanting to sound snoody. What use-cases do firecracker
| instances and the likes chime?
|
| I use FreeBSD for everything from my colocated servers, to my own
| PC. By no means am I developer; seasoned Unix Admin at best.
| Bare-metal forever but welcome to the future. Especially anything
| that contributes to the OS.
|
| However I hear buzz words like Lambda and Firecracker and really
| have no idea where the usage is. I get docker, containers, barely
| understand k8s but why do you need to spin up a VM only to tear
| it down compared to where you could just spin up a VM and use it
| when you really need to. Always there, always when.
|
| Is it purely a cloud experience, cost saving exercise?
| jedberg wrote:
| The main use case if for seldom used APIs. If I run a service
| where the API isn't used often, but I need it quick when it is,
| Lambdas or something like it are perfect.
|
| As it turns out, a lot of APIs for phone apps fit this
| category. You don't want a machine sitting around idle 99% of
| the time to answer those API calls.
| artificial wrote:
| FaaS, function as a service. Depending on how software is
| packaged and the expectations the richness a VM, like
| Firecraker, provides may be useful. Many of these tradeoffs are
| for velocity, I can run X easily on Y.
| znpy wrote:
| > However I hear buzz words like Lambda and Firecracker and and
| really have no idea where the usage is.
|
| Sometimes you just want to slap some lines pf code together and
| run them from time to time, and don't need a whole server
| (physical or virtual) for that.
|
| Sometimes you have no idea if you'll have to run a piece of
| code 100 times a day or 10'000'000 times a day.
|
| Sometimes you don't feel like paying a whole month for and
| maintaining a whole instance for a cronjob that lasts 20
| seconds, and maybe it runs once a week.
| Datagenerator wrote:
| IoT devices can execute short lived actions by calling remote
| Functions. The provider wants complete isolation and wipes
| these micro VMs after every few seconds and let's the user pay
| for use. The response from these can be anything, voice, data
| or API responses.
| wnolens wrote:
| > just spin up a VM and use it when you really need to. Always
| there, always when
|
| and always charging you :)
| r3trohack3r wrote:
| Instances of an application are created as part of the
| request/response lifecycle.
|
| Allows you to build a compute plane where any node in the plane
| can service the traffic for any application.
|
| Any one application can dynamically grow to consume the
| available free compute of the plane as needed in response to
| changes in traffic patterns.
|
| Applications use no resources when they aren't handling
| traffic.
|
| Growing the capacity of the compute plane means bringing more
| nodes online.
|
| Can't come up with a use case for this beyond managing many
| large-scale deployments. If you aren't working "at scale" this
| is something that would sit below a vendor boundary for you.
| sangnoir wrote:
| > why do you need to spin up a VM only to tear it down compared
| to where you could just spin up a VM and use it when you really
| need to. Always there, always when
|
| Firecracker has a much amaller overhead compared to regular VMs
| - which makes the (time and compute) costs of spinning up new
| VMs really low. This can be an advantage, depending on how
| chunky your workloads are - the less chunky they are - the more
| they can take advantage of finer-grained scaling.
| lproven wrote:
| ... "snooty"?
|
| https://www.merriam-webster.com/dictionary/snooty
| willsmith72 wrote:
| Instead of (or alongside) a CDN, you can deploy mini services
| around the world at the "edge"
| turtlebits wrote:
| Just about every single company can benefit from scaling as
| traffic is never consistent 24/7. Most don't bother as the
| effort outweighs the savings, but the potential is there.
| Things like lambda and firecracker make it much easier.
| garganzol wrote:
| I never really realized that Firecracker VM is a full-blown
| machine and not just some sort of a Linux container tech. At
| first, it may sound like an ineffective approach, but if you take
| a closer look on a real-world usage example such as fly.io, you
| will be surprised: micro-VMs are very small and capable.
| NovemberWhiskey wrote:
| There's no way an "enterprise grade" cloud vendor like AWS
| would allow co-tenancy of containers (for ECS, Lambda etc) from
| different customers within a single VM - it's the reason
| Firecracker exists.
| rewmie wrote:
| > There's no way an "enterprise grade" cloud vendor like AWS
| would allow co-tenancy of containers (...)
|
| I don't think your beliefs are well founded. AWS's EC2 by
| default only supoprts shared tenancy, and dedicated instances
| are a premium service.
| tptacek wrote:
| I take them to mean shared kernels.
| monocasa wrote:
| Does google? I know they use gvisor in production, which is
| ultimately enforced by a normal kernel (with a ton of
| sandboxing on top of it).
| eddythompson80 wrote:
| Google is moving away from gvisor as well.
|
| The "process sandbox" wars are over. Everybody lost,
| hypervisors won. That's it. It feels incredibly wasteful
| after all. Hypervisors don't share mm, scheduler, etc. It's
| a lot of wasted resources. Google came in with gvisor at
| the last minute to try to say "no, sandboxes aren't dead.
| Look at our approach with gvisor". They lost too and are
| now moving away from it.
| RainbowFriends wrote:
| Citation needed. gvisor seems to be under active
| development and just added support for the systrap
| platform, deprecating ptrace:
| https://gvisor.dev/blog/2023/04/28/systrap-release/
| znpy wrote:
| Everything google does "seems under active development"
| until they kill it
|
| ... stadia anyone?
| lima wrote:
| > _Google is moving away from gvisor as well._
|
| I've been wondering about this - are they really?
| beardedwizard wrote:
| I have seen zero evidence of this; but if it's true I
| would love to learn more. The real action is in side
| channel vulnerabilities bypassing all manner of
| protections.
| Rapzid wrote:
| Really? Has gvisor ever been popped? Has there ever even
| been a single high-profile compromise caused by a
| container escape? Shared hosting was a thing and
| considered "safe enough" for decades and that's all
| process isolation.
|
| Can't help but feel the security concerns are overblown.
| To support my claim; Well, Google IS using gvisor as part
| of their GKE sandboxing security..
| tptacek wrote:
| I don't know what "popped" means here, but so far as I
| know there's never been a major incident caused by a flaw
| in gvisor. But gvisor is a much more intricate and
| carefully controlled system than standard Linux
| containers. Obviously, there have been tons of container
| escape compromises.
| pjmlp wrote:
| In a way, it feels like a sweet revenge for microkernels.
| [deleted]
| basique wrote:
| Aren't Cloudflare Workers multitenant? Although, if you want
| to be cynical, that could be a reason they aren't 'enterprise
| grade(tm)'.
| tyingq wrote:
| They are using v8 isolates, which is maybe easier to do in
| a sound way than the whole broad space of containers.
| Previous discussion:
| https://news.ycombinator.com/item?id=31740885
| mochomocha wrote:
| Both Lambda firecracker VMs and t2 instances are multi-tenant
| and oversubscribed.
| tptacek wrote:
| I take them to mean "multiple tenants sharing a kernel"; I
| think everyone understands that AWS and GCP have
| multitenant hypervisor hosts.
| nilptr wrote:
| > There's no way an "enterprise grade" cloud vendor like AWS
| would allow co-tenancy of containers (for ECS, Lambda etc)
| from different customers within a single VM - it's the reason
| Firecracker exists.
|
| I won't speak for AWS, but your assumption about what
| "enterprise grade" cloud vendors do is dead wrong. I know,
| because I'm working on maintaining one of these systems.
| lima wrote:
| Lots of enterprise grade cloud vendors trust the Linux
| kernel boundary WAY too much...
| jen20 wrote:
| "Enterprise grade" deserves scare quotes for those people
| of course!
| seabrookmx wrote:
| Well they're not a "full-blown" machine, in that they do cut
| out a lot of things unnecessary for Lambda's (and incidentally,
| fly.io's) use case. ACPI is one example given in the article.
|
| But yes, they do virtualize hardware not the kernel. I'm
| willing to bet you could swap out vanilla containerd with
| firecracker-containerd for most users and they wouldn't notice
| a difference given they initialize so fast.
| Thaxll wrote:
| It's not a full blown VM, it has limitations.
| tptacek wrote:
| It is a full blown VM, it has limitations.
| mjb wrote:
| If you're interested in learning more about that, check out our
| NSDI'20 paper on how we chose this direction
| (https://www.usenix.org/conference/nsdi20/presentation/agache)
| and the Firecracker source and docs
| (https://github.com/firecracker-microvm/firecracker).
|
| Thanks to KVM, and to the minimal hardware support (no PCI, no
| ACPI, etc), Firecracker's source is rather simple and even
| relatively readable for non-experts.
| cperciva wrote:
| _Firecracker 's source is rather simple and even relatively
| readable for non-experts._
|
| ... as long as they're experienced at writing Rust. As a Rust
| newbie it took me a long time to figure out simple things
| like "where is _foo_ implemented ", due to the twisty maze of
| crates and uses directives.
|
| I totally get why this code is written in Rust, but it would
| have made my life much easier if it were written in C. ;-)
| packetlost wrote:
| I'm very surprised the standard isn't to build a microkernel
| that emulates Linux userspace (or *NIX userspace) and is
| tailored towards the subset of virtual hardware that
| Firecracker and QEMU provide. I don't get the impression that
| implementing a new target for a PL is all that difficult, so if
| you create a psuedo-OS like WASI/WASM and send PRs to the
| supported languages you could cut out most of the overhead.
|
| The "hardest" part is probably sufficiently emulating Linux
| userspace accurately: it's a big surface area. That's why I
| think creating a pseudo-OS target is the best route.
| tptacek wrote:
| You're describing gvisor.
| packetlost wrote:
| No, I'm not. Gvisor is a security layer around Linux
| containers that emulates and constrains syscalls. It
| specifically runs on top of a container platform and
| kernel. What I'm suggesting is a stripped down Linux-like
| kernel that is really good at running exactly one process.
| I'm describing a microkernel.
| andrewstuart wrote:
| qemu has microvm, inspired by firecracker
|
| https://qemu.readthedocs.io/en/latest/system/i386/microvm.ht...
| bonzini wrote:
| I wonder how many of these workarounds are needed with QEMU!
| Some of course will be needed because they are fixes for bugs
| in FreeBSD.
___________________________________________________________________
(page generated 2023-08-24 23:00 UTC)