[HN Gopher] Containers vs. Zones vs. Jails vs. VMs (2017)
___________________________________________________________________
Containers vs. Zones vs. Jails vs. VMs (2017)
Author : gullyfur
Score : 79 points
Date : 2020-04-06 20:47 UTC (2 hours ago)
(HTM) web link (blog.jessfraz.com)
(TXT) w3m dump (blog.jessfraz.com)
| mooreds wrote:
| Note this is from 2017. Previous discussion:
| https://news.ycombinator.com/item?id=13982620
| dang wrote:
| Year added. Thanks!
| [deleted]
| outworlder wrote:
| > A "container" is just a term people use to describe a
| combination of Linux namespaces and cgroups. Linux namespaces and
| cgroups ARE first class objects. NOT containers.
|
| Amen.
|
| Somewhat tangential note: most developers I have met do not
| understand what a 'container' is. There's an aura of magic and
| mystique around them. And a heavy emphasis on Docker.
|
| A sizable fraction will be concerned about 'container overhead'
| (and "scalability issues") when asked to move workloads to
| containers. They are usually not able to explain what the
| overhead would be, and what could potentially be causing it. No
| mention to storage, or how networking would be impacted, just
| CPU. That's usually said without measuring the actual performance
| first.
|
| When I press further, what I most commonly get is the sense that
| they believe that containers are "like VMs, but lighter"(also,
| I've been told that, literally, a few times, specially when
| interviewing candidates). To this day, I've heard CGroups being
| mentioned only once.
|
| I wonder if I'm stuck in the wrong bubble, or if this is
| widespread.
| loeg wrote:
| > Somewhat tangential note: most developers I have met do not
| understand what a 'container' is.
|
| The problem is at least in part that the term is basically
| meaningless. It's too broad and flexible to be descriptive. As
| you quoted:
|
| > A "container" is just a term people use to describe a
| combination of Linux namespaces and cgroups.
| sanderjd wrote:
| I'm more sympathetic to the mainstream usage. People are
| (attempting to) use an abstraction: a "container" is an
| isolated process runtime environment without virtualization
| overhead. That abstraction seems useful to me. Ideally it
| would be usable without too much leakiness, in which case its
| users would not need to be aware of implementation details
| like cgroups and namespaces. In practice, all abstractions
| are leaky to some degree and effective use of an abstraction
| often eventually requires a more sophisticated understanding
| of the details beneath the veil of the abstraction. But that
| doesn't mean the abstraction is totally useless or completely
| a mirage or anything, it's just a leaky abstraction like all
| others.
|
| If you say that a container is not a first class object but
| cgroups and namespaces are, I can just as easily say that
| cgroups and namespaces aren't first class objects, they are
| just terms people use to describe a combination of system
| calls. It's just abstractions with different amounts of
| leakage the whole way down.
| tasogare wrote:
| The "containers are just XX Linux technology" comes regularly
| in the comments but it's untrue: Windows containers are
| obviously not based on any Linux tech proper.
|
| Also the overhead intuition exists for a reason: on both macOS
| and Windows when Linux containers are used, there is actually a
| whole VM running Linux underneath. And Windows containers come
| in two flavors, one being Hyper-V based, so again a VM tech
| comes in play.
|
| So there are technical reasons why containers are
| "misunderstood", it's because most people don't run Linux
| natively, and on their stack containers are more than just
| cgroups and namespaces.
| matharmin wrote:
| I feel nowadays containers generally refer to the concept, and
| cgroups and namespaces are the implementation details of a
| specific container runtime. These are very important
| implementation details for security and performance, but it
| doesn't fundamentally impact how you structure your
| containerized application.
|
| You can take the same container image, and run using Docker,
| Firecracker, gVisor, or many other container runtimes. Some of
| them are exactly "like a VM, but lighter".
| cmckn wrote:
| Agreed. The post feels a bit pedantic; I don't know any dev
| doing "cool things" with the underlying namespaces/cgroups.
| They're just using Docker. De-mystifying containers has
| value, but so does the abstraction.
| taf2 wrote:
| Thanks I'd have fallen into your category of developers because
| in large part I never bothered with containers since we have
| everything running on vms and we've already isolated things so
| I've had only partial interest in exploring them ... but now
| that I know in linux it's CGroups and namespaces that helps a
| lot in understanding thanks
| jjtheblunt wrote:
| widespread in my experience; it reminds me to users of npm not
| seeming to understand what node is, for example.
| k__ wrote:
| Doesn't surprise me if I think about all the people who
| suddendly need a K8s cluster...
| CBLT wrote:
| > To this day, I've heard CGroups being mentioned only once.
|
| See https://www.kernel.org/doc/Documentation/cgroup-v2.txt
|
| > "cgroup" stands for "control group" and is never capitalized.
| The singular form is used to designate the whole feature and
| also as a qualifier as in "cgroup controllers". When explicitly
| referring to multiple individual control groups, the plural
| form "cgroups" is used.
|
| To this day, I've heard cgroup mentioned only once...
|
| To put forth a more substantive argument, everybody has a layer
| of abstraction they do not peek under. You interviewed people
| that didn't peek under container. You went a layer deeper, but
| never peeked at the source tree to learn what cgroup really is.
| Does it really feel that much better to be one level above
| others?
| MuffinFlavored wrote:
| > A sizable fraction will be concerned about 'container
| overhead' (and "scalability issues") when asked to move
| workloads to containers. They are usually not able to explain
| what the overhead would be, and what could potentially be
| causing it.
|
| For what it's worth, one of the biggest "containerization"
| recommendations is to _not_ run your database (example:
| Postgres) in a container, correct? Due to I /O performance
| decrease?
| wmf wrote:
| No. Docker volumes aka bind mounts have little or no
| overhead. You don't want to run a database in an ephemeral
| "cattle" container without some kind of HA because you'd lose
| data.
| aprdm wrote:
| I think it is more because containers should be stateless and
| you cannot make a database stateless.
|
| We do run databases that do 100k`s ops/s in containers but we
| don't run them in kubernetes. We just mount the VM hard drive
| in it.
| pojzon wrote:
| This is widespread unfortunately. Developers changed to
| ,,users" and no longer pursue the details of solutions. Im not
| gonna say its the dominant behaviour in the field right now,
| but I see it more and more often on various experience levels.
| justanotherc wrote:
| You talk as if that's a bad thing.
|
| As a developer I don't want to wade into the details of
| systems I'm using, I want to spend my time writing code that
| solves the business problems I'm tasked with solving.
|
| If there is a system that allows me to do that by abstracting
| away the details I don't care about, why wouldn't I use that
| system?
| ben509 wrote:
| I do want to wade into the details, so I'd put it more that
| I need to try many things, and by necessity I have to be a
| user before I can become an expert.
|
| I know a good deal about how Docker works from having
| hammered at it. I'm not remotely an expert in it yet, but
| as I go I'm learning more details about cgroups and
| namespaces, and this is stuff I've been able to fit in
| while solving problems.
| jschwartzi wrote:
| Because frequently the devil is in the details.
| bluGill wrote:
| But only some of the details. I still need to ignore most
| of the details even while I need control of the ones that
| matter.
|
| This is easier said than done.
| jagged-chisel wrote:
| An experienced software engineer (yeah, developer) has
| experience engineering software. It's been less than 10yrs
| since the advent of containerized deployments, and the space
| has been fraught with change nearly on par with the front end
| JavaScript ecosystem. Might as well just stick to writing
| code. OK, that's the perception of my own peers, but I assume
| it scales.
|
| DevOps is a recent advent, too, and sounds to me like it
| should be populated with folks who can participate in
| development and operations. Most developers I've ever known
| aren't interested in operations.
| dielectrikboog wrote:
| SRE was born of putting software engineers to work building
| operational software and automation tailored to an
| organization and application. In contrast, no matter what
| anyone says, DevOps was objectively born of replacing the
| operations discipline and career track with a poorly-
| understood tool economy and ongoing opex to a cloud
| provider. As you say, typical JavaScript engineers can't be
| bothered to understand network capacity planning yet feel
| they are more qualified to take their application to
| production by deferring all decisions to cloud providers.
| Who all employ SREs/PEs, not DevOps Engineers, by the way,
| and there is a big distinction.
|
| We have people who can handle the operational stuff.
| They're called systems administrators, network engineers,
| yes, even SREs, and other folks who are really good at
| understanding how computers and the Internet actually work,
| and a webdev bootcamp gives zero context into exactly what
| they do. None. Then, your ten-head startup suddenly scales
| to needing a physical footprint because it will literally
| save 80% of opex, and all your DevOps Engineers say "but
| why? There's AWS," and you're in Medium hell for weeks
| arguing about it.
|
| Apropos, if I interview you and find you have written a
| thoughtpiece on Medium about how "sysadmin is evolving" and
| it's "time for the gray beards to learn code," you do not
| get a call back. That has actually happened, and no,
| sysadmin is not evolving. I know staff-level "full stack
| engineers" who can't tell me what an ASN is. The motions in
| the industry have merely made those people more in demand
| at a few companies and served to cement their position as
| "where computing gets done".
|
| Expect serious, existential operational threats and
| breaches to rise dramatically as DevOps continues to "win,"
| and consider it a smart strategic move to avoid DevOps
| culture in the long term. If you write a req called "DevOps
| Engineer," I don't even know what to say to you.
| delusional wrote:
| Has sysadmin not evolved? if I found some sysadmin
| logging into a production system and editing the config
| file in nano today, I'd be downright depressed.
| dielectrikboog wrote:
| Sounds like you're going to be depressed when you learn
| how the entire Internet plane, all software engineering
| outside of "SV", all IT, all government, and basically
| everything except your GitHub CI/CD adventure works,
| then. Sorry.
| aprdm wrote:
| Even within SF. Having talked with a bunch of folks from
| Amazon and Netflix they're far from having most of their
| workflows running in containers... imagine is the same
| for google.
| icedchai wrote:
| Most "DevOps" folks I know are actually former sysadmins
| who _evolved_ to work more with cloud technologies. To
| say sysadmin hasn 't evolved in a bit of an exaggeration.
| Titles follow the trend. What people actually do is often
| similar.
| dielectrikboog wrote:
| No, it isn't an exaggeration. They ceded one particular
| competency, systems administrator, and now pay cloud
| providers to do it instead. The job didn't go anywhere.
| Capacity planning, change management, peering, supply
| chain management, all of that stuff is still happening,
| they just willingly tapped out of it and _took another
| job_ (probably because the DevOps people came in with a
| slide deck and hand waved them out of a job at the SVP
| level).
|
| That is not evolution (nor an indictment of those people,
| importantly). The side effect, which literally nobody is
| paying attention to, is a future where computing as a
| concept is concentrating on a few major companies. Every
| every every every single person who says "why would I buy
| a rack? There's AWS" is furthering that outcome.
| icedchai wrote:
| You still need to do capacity planning, change
| management, monitoring, etc. within your cloud
| environment. Those AWS instances and the software their
| running doesn't manage itself. For _some subset_ of
| "cloud", such as PaaS providers like Heroku, etc., you
| are absolutely correct. For another subset of "cloud",
| you still need sysadmin / ops skills to manage it.
| dielectrikboog wrote:
| Yeah, you do. It's a shame pretty much every single
| cloud-native shop in existence, you know, doesn't bother,
| and pushes out the people arguing for bothering. I've
| been at this nearly two decades, and I have yet to find
| engineers even running an _Excel notebook_ of _inventory_
| , much less capacity planning. You know, because
| describe-instances and monitoring and Ansible and Chef
| and blah blah.
|
| My role right now is telling a major government agency
| how much they're wasting on Azure. You know, because
| describe-instances. It's a lot, and I think there might
| be a business model in "let me at your API for a week and
| give me 10%." I'd be retired by Labor Day.
|
| Reminder: They're sending Congressionally appropriated
| funds. To Redmond. And they're not entirely sure why, in
| $millions of cases. Line up fifty startups that have had
| a Demo Day and I'd bet you'd find the same thing in
| fifty-one of them.
|
| That's the DevOps legacy: don't mind the budget, because
| AWS, Azure, and GCP have our financial interests in mind
| and APIs are cheaper to staff than fiber. Parades of
| like-minded individuals came to D.C. and said "DevOps! Do
| it!" and the agencies are now increasingly beholden to
| organizations _incentivized by profit_ and those
| contractors took their Innovation Award check and don't
| return the "um, what now?" calls. That's the mess I'm
| trying to help clean up, and it's happening across every
| major agency in the United States.
| twic wrote:
| > no matter what anyone says, DevOps was objectively born
| of replacing the operations discipline and career track
| with a poorly-understood tool economy and ongoing opex to
| a cloud provider
|
| This isn't the origin of DevOps.
| aprdm wrote:
| This is a great post but I want to say that I feel
| there's space for both. IMO a DevOps Engineer would sit
| between the sysadmin/network folks and the developers who
| wants to be users of a system.
|
| In my current gig we've moved from the DevOps department
| to the Platform department as it aligns more with what we
| are trying to provide. A Platform for developers.
|
| That said we essentially can speak sysadmin and can speak
| developers. We trust sysadmins with network, linux image
| and more specialist topics. We make tools for both sides
| and try to make them work together often sitting in the
| middle and negotiating.
| dielectrikboog wrote:
| Call them SREs and cross-train SWEs into it. It's not a
| toothless distinction even though it seems like one. You
| absolutely, positively will hire better staff with better
| deliverables if you frame the work as "a software
| engineer focused on operational integration," which SRE
| understands more.
|
| SREs like to build platforms for exactly the same reasons
| you're touching. You sound like you're halfway there
| already. I strongly suggest the Google book, with "I am
| not Google scale" written in Sharpie on the cover for
| help digesting it.
| aprdm wrote:
| In my understanding SRE is more related to "keep the
| lights on and systems running", it might be just a
| different understanding of the nomenclature.
|
| E.g: In my current case the software teams own their ops,
| my team doesn't ops for them.
|
| We give them a platform of centralized logging,
| monitoring and etc. so that they can easily ops their
| services but is not my phone that rings and I am not on
| call. I am on call if some component of the platform
| itself fails.
|
| At least my perception of SRE is that they're on call for
| products.
|
| That said I would frame the work we do as "a software
| engineer focused on operational integration".
|
| That does sound like a good book and I will add to my to
| read list.
| nunez wrote:
| This is extremely widespread amongst the really large companies
| I've consulted for (with equally large development teams to
| boot). "Containers are VMs, but smaller and/or faster" is an
| extremely common school of thought. I would like to think that
| this viewpoint is dying somewhat, especially now that many
| large organizations have at least experimented with
| orchestrators like Kubernetes or ECS.
|
| I can't blame them, however.
|
| If you're a dev at a company where such misunderstandings are
| pervasive, and you're interested in trying to get your app
| running in this "Docker thing that you've heard of", you will,
| probably:
|
| - need to seek out a Linux dev server because you probably
| don't have admin rights on your machine, and getting Docker
| installed onto your machine in a way that doesn't suck is more
| trouble than its worth,
|
| - have engineering management that are being told that
| containers are like VMs, but smaller, likely from magazines or
| sales/pre-sales consultants,
|
| - Have to wait days/weeks to get SSH credentials to a dev
| server that has RHEL 7 on it, hoping that it has Docker (a
| thing you've heard of at this point, but don't really know much
| about it otherwise),
|
| - Have to wait even more time to get Docker installed for you
| on the dev server by a sysadmin that dislikes Docker because
| "it's insecure" or something, and
|
| - be constantly reminded that your job is shipping features
| before anything else, usually at the cost of learning new
| things that make those features more stable
|
| The point here is that environments like this are barely
| conducive for getting engineering done, let alone learning
| about new things. Moreover, the people that will do anything to
| scratch that itch usually find themselves out of companies like
| that eventually. It's a circle of suck for everyone involved.
|
| So when I introduce containers (which Docker is, by far, the
| most common runtime, so I introduce it as "Docker" to avoid
| confusion) to someone who doesn't know what cgroups or
| namespaces are, or someone who responds with something about
| containers being VMs or whatever, I happily meet them where
| they are at and do my best to show them otherwise.
| wmf wrote:
| People love to bring this up, but if Linux did have first-class
| containers, how would the developer's experience be different?
| mulmen wrote:
| I think it is widespread because containers are (seemingly)
| marketed as being some kind of magic. The impression I get is
| that the benefit to containers is that you _don 't_ have to
| think about them. This may be more a product of Docker but I
| think containers and Docker have become synonymous.
|
| I'm not sure this is a fair comparison but that is my
| impression. I could be in a bubble too.
| erjiang wrote:
| This is true for many new waves of popular technologies.
|
| 1) A new technology or method becomes popular. 2) Developers
| find new advantages in using the technology. 3) Understanding
| of tech and original advantage is somewhat lost.
|
| For example: containers are now widely used as part of a
| scriptable application build process, e.g. the Dockerfile.
| There are probably many developers out there who care about
| this and not about how containers are run and how they interact
| with the kernel. And for their use cases, that is probably the
| thing that matters most anyways.
___________________________________________________________________
(page generated 2020-04-06 23:00 UTC)