[HN Gopher] ContainerSSH: Launch containers on demand
___________________________________________________________________
ContainerSSH: Launch containers on demand
Author : gaocegege
Score : 212 points
Date : 2022-09-10 08:17 UTC (14 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| toastal wrote:
| I would rather have a Nix devShell.
| viraptor wrote:
| Is there a good solution for devshell with associated services?
| The app itself is fine that way, but what about the database,
| message queue, seed data, etc.?
| qbasic_forever wrote:
| You can install any services in your nix dev shell
| environment. Run them however you desire--a bash script that
| backgrounds their tasks, systemd user services, supervisord,
| or any other process manager.
|
| edit: A really simple and effective process manager is
| foreman and its procfile format. I like the golang version of
| it (simple single binary with zero dependencies):
| https://github.com/ddollar/forego or
| https://github.com/mattn/goreman
| viraptor wrote:
| So I'm aware you can script anything you want. I meant a
| more integrated version. Now I'm tempted to write one
| myself where you provide the (for example) mysql you want
| to run and it automatically inserts the right scripts...
| _hl_ wrote:
| I usually write a script that idempotently spins these things
| up locally, and use the shellHook to call that script. That
| way whenever I cd into the dir the first time, direnv loads
| the nix-shell, which starts the db, message broker, etc. I
| also have a shutdown script that I run when I manually want
| to shut things down. Works quite well.
|
| For more advanced use cases, such as k8s with tons of
| services, you're better off using something like tilt.dev.
| thenonameguy wrote:
| I've been using https://github.com/F1bonacc1/process-compose
| with great success.
|
| It's a userspace process orchestrator/scheduler that works
| across all relevant platforms, supporting daemon processes
| and k8s style readiness/health checks.
|
| In combination with nix flakes, it quickly reduced my
| projects docker-compose usage for easy-to-configure services.
|
| This gave huge performance benefits for the M1 Mac folks on
| my team especially for CPU intensive processes thanks to
| native binaries.
|
| For maximal ease of use, the remaining docker-compose
| containers are started/stopped as a process-compose task.
| Quite meta :)
| evol262 wrote:
| I would, too, but the barrier to entry is much higher.
|
| This has been discussed ad-nauseum, but "getting started" with
| nix (or home-manager, or devshell) is an experience worse than
| trying to do anything with Linux in 2005.
|
| Cobbling together a bunch of blogposts and docs which kind of,
| sort of, might let you piece together a config which works
| after enough iterations and unreadable backtraces is far more
| difficult than it needs to be, and that's before you get into
| cases like "I want Python linters in my environment, so I need
| Python, but I also need access to some modules from Python on
| the system, which aren't installable (let's say `python-apt`,
| which cannot be installed from pypi), and..."
|
| For probably the 500th time someone has said it on HN this
| week, nix will never be really usable or gain wider adoption
| until someone puts together solid documentation, or even a
| solid "getting started -- intermediate" guide which can bridge
| between "let me set up a trivial env" and "I'm running nixos
| and it's nix all the way down"
| cod3rboy wrote:
| I like the idea. With this, I can do something like - a Golang
| IDE container for go projects, a Javascript IDE container for
| frontend projects and so on.
| goodpoint wrote:
| Or just use SystemD with nspawn.
| janosdebugs wrote:
| You can, but it be comes quite difficult to secure and manage
| very quickly. Nikos talks about the SSH use case at CERN in
| this video [1] and mentions that they tried quite a few things
| before ending up with ContainerSSH.
|
| [1] https://youtu.be/d6aCBiInMbg?t=525
| 0xbadcafebee wrote:
| Thanks for this! This explains the project better than the
| GitHub site
| janosdebugs wrote:
| Yeah, we've gone through a few iterations with the README
| on GH, but it's really hard to explain the project in the
| length available there without going into only a very
| specific use case.
| JoyrexJ9 wrote:
| Sounds a lot like VS Code devcontainers
| https://code.visualstudio.com/docs/remote/containers
| janosdebugs wrote:
| Not exactly, VS Code devcontainers work by directly interfacing
| with the container environment. If you don't trust the
| connecting party with direct access, you can use ContainerSSH.
| One use case would be a shared development environment.
| sascha_sl wrote:
| For those that prefer a local environment that's disposable but
| still supports most graphical apps as well, there's Fedora's
| Toolbox for pet containers.
|
| https://docs.fedoraproject.org/en-US/fedora-silverblue/toolb...
| razemio wrote:
| Nice! Will give this a try. My current setup is a vscode docker
| container with every dependency and plugin I need for
| development. It runs on a powerful server.
|
| It is SO nice to come back where you left from multiple machines.
| Also it is very nice to have a 10 gig connection. It works
| incredible well. With Scala and the metals plug-ins it is very
| close to a full blown intellij experience.
| scoopertrooper wrote:
| It seems (from the material on the site) that the motivating
| use case for this isn't development, but rather running
| computer labs for students.
| razemio wrote:
| If did understand it correctly, it can be used to auto deploy
| a container on an ssh host, start the cli and once you exit,
| it will clean everything up. This would be very usable for
| developing certain things.
| janosdebugs wrote:
| You did understand it correctly. ;)
| scoopertrooper wrote:
| For sure, it could be used that way. But, a lot of people
| were comparing this unfavourably to how they currently
| manage their dev workflows. My point was that they weren't
| really the target market.
| 0xbadcafebee wrote:
| I think this is the workflow most people should be moving
| to: immutable development.
|
| If your dev environment's state mutates and deviates from
| the rest of a team's (or away from production), the
| deviation introduces bugs. This tool lets you "start
| fresh" with the correct environment every time, but (I
| imagine) also lets you update that shared environment by
| pushing a new container.
|
| In this way you ensure everybody is developing off
| exactly the same thing, and can easily update that shared
| environment, but nobody clobbers anybody else's WIP, and
| you can test new changes in your own container, even
| persisting your own configuration using persistent
| volumes.
|
| Sadly, this does not support port forwarding, which is
| the final feature necessary to do remote development (run
| your web app remotely, see it in your browser locally).
| janosdebugs wrote:
| If you build from the libcontainerssh sources on GitHub
| it does. We just haven't released it yet. :)
| rmetzler wrote:
| Your setup sounds a lot like gitpod.io
| razemio wrote:
| Yes but everything "self-made". Been working like this for
| years now.
|
| Also have arch + i3 + openrdp dev containers which work
| awesome. Even with multi-monitor support. The only thing
| missing is gpu acceleration. Then it would be perfect.
| yjftsjthsd-h wrote:
| > arch + i3 + openrdp dev containers
|
| Any chance you've published the build files somewhere? This
| sounds fantastic
| gingerlime wrote:
| sounds intriguing. is there a blog post with your setup?
| can this be (easily) used between several developers? each
| needs their own server? can the server spin on-demand?
| janosdebugs wrote:
| For VS Code you will need to build your own binary because it
| needs port forwarding which isn't released yet.
|
| Edit: also, ContainerSSH tears down the container when you
| disconnect, so that may not exactly be what you are looking
| for.
| janosdebugs wrote:
| Hey folks, one of the authors here. The website contains a lot
| more information than the GitHub page: https://containerssh.io
|
| If you have any questions, I'd be happy to answer.
| e12e wrote:
| From a quick glance it's unclear if there are any ready-made
| auth servers for this? Eg, ldap, sso for o365 or Google?
| janosdebugs wrote:
| Currently, there is only the test server that's available. In
| the next version we will support direct oAuth2 / OIDC
| integration. It's a little ways out, but testable with Google
| at the moment. Here's an early demo from when we started
| working on it: https://youtu.be/SGHee9cV_rA
| judge2020 wrote:
| Slightly related: Is there a similar ssh auth system for
| regular openssh? One where no client software is needed and
| to login you use a browser to authorize the login based on
| Google/GitHub identity?
| janosdebugs wrote:
| Yes, there are several PAM integrations, some more
| rudimentary than others. The most developed seems to be
| this one [1], but I haven't tested it. However, it only
| supports the device flow, so it's limited to 50 logins
| per hour on GitHub.
|
| [1] https://github.com/slaclab/pam_oauth2_device
| gaocegege wrote:
| Sorry I did not ask you to post the project to HN. I just found
| it interesting.
| janosdebugs wrote:
| Oh, no worries, thank you for posting it, we are all very
| excited it made it on the front page. :) I just wanted to
| point to the website as it contains a lot more info than the
| GitHub repository. :)
| michaelsalim wrote:
| Love the concept of the explainer video. But am I the only one
| that didn't understand what this does? I had to watch the video
| twice and really read the readme to finally get it.
|
| Am I understanding that this is a fancy way of doing: - SSH to
| server - Create docker container (or Kubernetes) - docker exec
| -it container /bin/bash But in 1 command & auto cleanup?
|
| Pretty neat. Not sure I have a use for it though. Would've loved
| to see more use case in the docs (along with making the video
| explain the project better since it's a great concept!)
| janosdebugs wrote:
| In a nutshell, yes. However, the main use case is when you
| don't want to give people access to the container environment,
| instead you want to drop them in a restricted shell. Think jump
| host. :)
| honkler wrote:
| there is no use for this, or most of the stuff around
| docker/k8s that's posted here. It's all just a fad. And people
| keep mindlessly pumping out more code.
| honkler wrote:
| you may downvote my comment, but you will never juice out
| anything useful from most new age "technologies".
| joshmanders wrote:
| Why are you even in this industry if you hate it so much?
| scrappyjoe wrote:
| Awesome! I wonder if I can use this to replace jupyterhub for my
| small data science team. The jhub kubernetes architecture is
| (roughly) -
|
| Proxy --> hub --> spawner based on selected image --> container
| with /user persistent PV
|
| But the image needs to be spawnable by jhub (i/e it needs to have
| some variant of jupyter-server-proxy installed), and currently
| that limits you to JupyterLab, RStudio and openrefine.
|
| If you can port-forward with containerssh you could bypass the
| whole jhub proxy / hub / spawner rigmarole and simply
|
| Select image -> containerssh w/port forward -> start the desired
| dev environment from inside the container -> get to work.
|
| Then you're not limited to jhub-compatible environments, and you
| don't have to manage the complexity of jhub.
| paulgb wrote:
| For the "jhub but for any container that speaks HTTP" use case,
| you might find our Spawner project interesting:
| https://github.com/drifting-in-space/spawner
|
| We don't have support for volumes yet, but I'm open to ideas.
| urcyanide wrote:
| Recently, I found another tool built for machine learning
| environment: https://github.com/tensorchord/envd. It integrates
| with buildkit and can share the config like Python functions
| cross the team. This can reduce a lot complexity for data
| scientists.
| janosdebugs wrote:
| Yes you can and you wouldn't be the only person doing this.
| However, port forwarding is only supported in the in-
| development version, so you have to build your own binary for
| that.
| Demiurge wrote:
| If this works with Docker, it can work in AWS ECS with a bit of
| wrapping, right?
| janosdebugs wrote:
| Yes, your config server can pass the Docker connection
| parameters directly, so you can scale it. I believe you could
| scale AWS ECS up and down as needed and then expose the Docker
| socket with TLS to ContainerSSH.
| drdaeman wrote:
| Would this work with Fargate? I don't think I've ever seen
| any Docker (or alike) API there that can spawn new tasks.
|
| I wonder how hard it is to make a custom backend...
| tambourine_man wrote:
| The one minute video doesn't load for me
| janosdebugs wrote:
| It should link to this YouTube video:
| https://www.youtube.com/watch?v=Cs9OrnPi2IM
| tambourine_man wrote:
| Yeah, but I get this error:
|
| https://ibb.co/TYjY8Y0
| janosdebugs wrote:
| I just went through the YouTube settings, nothing there
| indicates anything that we can do about it. I also checked
| in several browsers and devices to no avail.
|
| I'm afraid, this may be something where YouTube isn't
| working as intended. (For example, I had problems with
| YouTube in Firefox on Linux for quite a while and needed to
| disable tracking protection entirely to get it working
| despite paying for Premium.)
| tambourine_man wrote:
| Thanks for the trouble. I'll try again later
| janosdebugs wrote:
| I uploaded the video to Google Drive, maybe this works
| for you: https://drive.google.com/file/d/1tIf86ba68Bc1RbW
| kVgrXlXeIFMD...
| lapser wrote:
| I'm struggling to understand why this is useful. How is this
| useful from doing a docker exec, or kubectl exec?
| janosdebugs wrote:
| It was born from a need in the webhosting sector. I wanted to
| give users shell access so they can do things like git pull
| directly to their website. Without containers it's difficult to
| constrain a user. The first SSH server I build along these
| lines involved the components what make containers today:
| chroots, cgroups, etc. (That was more than 10 years ago.)
|
| Over time, more use cases have developed, most of them around
| the need for jump host or lab access. There has also been some
| research done into SSH attack patterns using this as a
| honeypot, which will hopefully be published soon. See:
| https://containerssh.io/usecases/lab/
| alerighi wrote:
| It is difficult? It is something most web hosting did way
| before container existed. You just create the user on the
| system (well, probably have the user in LDAP) and give the
| user the permission to access only its files and possibly
| execute only a number of trusted programs (or not execute
| programs at all, and dive only SFTP/SCP/git access and not a
| shell).
|
| Containers may give some false sense of security tough, a lot
| of people doesn't understand really that escaping a container
| is not such a difficult thing, since it pretty new stuff and
| bugs are being discovered, also the container may be
| configured badly, while the UNIX permission mechanism is
| around since forever and it's pretty solid (not that in the
| past there weren't bug that bypassed it, but the same bugs
| may as well be used in a container anyway).
| janosdebugs wrote:
| Most webhosting services barely managed to offer encrypted
| FTP/SFTP before containers existed, most went with plain
| text FTP. When I worked in the sector back 2011 we had a
| customer with a hacked/stolen FTP password every other
| week.
|
| As far as features are concerned, yes, you can make a git
| server you can push to, but allowing users to get a full
| shell and pull from their git server is a whole other
| comfort level. More modern alternatives would include on-
| server development with VS Code, which we aim to support in
| the next version with port forwarding supported.
|
| As far as container security is concerned, if properly
| configured, these are still a sight more secure than trying
| to give users shell access without them. The UNIX
| permission mechanism is woefully inadequate for keeping
| users from messing with each other's stuff. This is
| obviously not a problem if you don't want to provide a
| shell service to users, but some services, like the LX Plus
| service at CERN mentioned in the other thread [1] is
| specifically that: a shell service for users to access.
|
| One of the problems containers (or more accurately, network
| namespaces) solve are the language or development servers
| users may start for their development needs. These often
| don't contain any extra security beyond binding to
| 127.0.0.1. However, on a shared server this is obviously
| not enough.
|
| The other problem with using purely UNIX permissions to
| isolate users in the webhosting sector is users messing up
| their permissions. Back in the day this was also a constant
| problem, so nowadays all webhosters run the webserver /
| PHP-FPM instance with the same user ID as the user uses to
| upload their code, often having multiple websites for the
| same user using the same user ID. This lends itself to
| cross-contamination between sites if one is breached. If
| the sites run on a different user ID, however, it becomes
| more difficult for a user to move data between them.
|
| [1] https://news.ycombinator.com/item?id=32790856
| rkeene2 wrote:
| Indeed, I let anonymous people run arbitrary code on my
| system... though I haven't bothered to make it available
| over SSH.
|
| https://rkeene.dev/js-repl/?arg=bash
|
| It creates a secure environment on every connection, though
| it doesn't use cgroups, just chroot, resource limits, and
| other boundary protection mechanisms, etc.
___________________________________________________________________
(page generated 2022-09-10 23:00 UTC)