[HN Gopher] The Twelve-Factor App (2011)
___________________________________________________________________
The Twelve-Factor App (2011)
Author : edward
Score : 317 points
Date : 2023-10-12 14:19 UTC (6 hours ago)
(HTM) web link (12factor.net)
(TXT) w3m dump (12factor.net)
| ris58h wrote:
| You should add 2017 in the title.
| wesleytodd wrote:
| yeah I got confused seeing this on the top. in my head said
| "what is this, 2012?". Am I wrong this existed way before 2017?
|
| edit: I am correct, it is much more old.
| https://github.com/heroku/12factor/graphs/contributors
| larsnystrom wrote:
| First commit was 2011.
| webmaven wrote:
| The 12 factors go back even further, to Joel Spolsky's
| "Joel Test" in 2000:
|
| https://www.joelonsoftware.com/2000/08/09/the-joel-
| test-12-s...
| richardwhiuk wrote:
| That's a rather different set of 12 factors.
| screamingninja wrote:
| Previous discussion-
| https://news.ycombinator.com/item?id=31198956 (102 comments)
| didntcheck wrote:
| Another interesting thread about how these practices have held
| up or changed over the past decade:
|
| "Twelve-factor app anno 2022"
|
| Article: https://xenitab.github.io/blog/2022/02/23/12factor/
|
| Discussion: https://news.ycombinator.com/item?id=31225921
| songshu wrote:
| I love this generally but some of it has leaked out to the wrong
| people and there are strong cargo cults around such things as one
| codebase===one app.
| notoverthere wrote:
| Back in the early days of Docker, I did a whole bunch of work to
| make WordPress behave as a Twelve-Factor App.
|
| It traditionally hasn't behaved as one - which sort of makes
| sense, because WordPress grew-up in a world where long-lived
| servers with writable and persistent local disk storage was
| commonplace.
|
| I'm sure things have moved on since those days. This was back in
| 2016. But it sure was a fun challenge!
| DanielHB wrote:
| I remember when I learned about making servers stateless by
| storing session information in a database and just not writing
| to disk and stuff. I was amazed at how much simpler it made
| things and added the capacity to load-balance multiple nodes
| without having to bother with session stickness.
|
| Of course it made things harder in other ways like having a
| separate DB for sessions.
| frankfrank13 wrote:
| I like this in general, but the number of times I've had to delay
| a release because some non/psuedo-technical person has pulled out
| "12 factor" like a universal yellow card has been enough for me,
| overall, to ignore it almost completely. I guess the same has
| happened to me for "agile" honestly. I recognize the intent of
| guidelines like these, but they seem to be much more useful to
| people whose only "value" is to provide ivory tower technical
| leadership.
| Waterluvian wrote:
| I've had issues like that before. The solution was to have a
| clear, documented process, and point them to that.
|
| I've learned to take issues like that in good faith. Why are
| they raising an issue? Is the existing process not clear? Is
| there a deficiency in it? Is there a lack of reliability? If
| reasonable concerns are found, the process can change (and the
| documentation is to be updated).
| ChrisMarshallNY wrote:
| _> The solution was to have a clear, documented process, and
| point them to that._
|
| This has its own drawbacks. I worked for a company, in which
| Process Was God, and Thou Shalt Do Only That Which God Has
| Proclaimed Good.
|
| It worked most of the time, but made it very, very difficult
| to introduce flexibility and orthogonal approaches.
|
| There's really no way to duplicate the results of
| experienced, cohesive teams of high-functioning engineers,
| with process. We can get similar results, in many ways, but
| there's always a price to be paid.
|
| T. A. N. S. T. A. A. F. L.
| willseth wrote:
| Give an example. Like most things, it's context dependent and
| there are grey areas, but I think most devs use 12 factor like
| a north star. I wouldn't block a release due to some minor
| deviation, but generally lacking alignment should be treated as
| tech debt as a minimum. If a release regresses significantly in
| one of the factors, I think it's fair to block or at least
| force more detailed review to understand why the dev felt the
| tradeoff was worth it for that release.
| cduzz wrote:
| There's the "Plumber Problem[1]" by John Siracusa where you
| watch a story and see a problem in your specific domain that
| leads you to mistrust the whole story.
|
| With the 12 factor nonsense, I see the guidance of "Dump
| unformatted logs into stdout and make it someone else's
| problem; they'll hadoop or splunk it and everyone wins."
| There is no magic splunk that absolve you (the developer) of
| generating useful data to logs, if you expect useful data out
| of your logs. You can't reverse "cow -> burger -> sewage" and
| shouldn't expect some magic hadoop to reverse entropy without
| a ton of work.
|
| So -- I guess the 12 factor app dictum is fine for some
| trivial "I rewrote twitter in 80 lines of erlang" app, but in
| my opinion it is a "if we can land a man on the moon we can
| land a man on the sun" misstatement of complexity in actually
| getting things done in the real world.
|
| [1] https://hypercritical.co/2023/08/18/the-plumber-problem
| morelisp wrote:
| Spoken like someone who has never had to work on a Java
| application with 22 custom log files, not including stdout,
| stderr, and two separate syslogs.
|
| Yes, if you want useful data from logs you'll need to log
| useful data, but this is a case where 12 factor is so
| successful / dominant today I don't think you even know
| what it's arguing against anymore.
| cduzz wrote:
| No, writing files is nearly as bad as writing to stdout
| -- both imply someone else has to parse the output.
|
| If the logs matter -- write them to an API that retains
| log structure by serializing it.
|
| If the logs don't matter, then they don't matter, but
| typically the logs are used for all sorts of analytics
| processes after the fact.
|
| Asking someone else to parse your data is rude.
| nightpool wrote:
| Again, "this is a case where 12 factor is so successful /
| dominant today I don't think you even know what it's
| arguing against anymore"
| cduzz wrote:
| The twelve factor app says: A twelve-factor app never
| concerns itself with routing or storage of its output
| stream. It should not attempt to write to or manage
| logfiles. Instead, each running process writes its event
| stream, unbuffered, to stdout. During local development,
| the developer will view this stream in the foreground of
| their terminal to observe the app's behavior.
|
| That guidance is deeply foolish; I have to assume the
| rest of the guidance is just as silly for anything but a
| toy application.
| lexicality wrote:
| Where does it say "unformatted"? All it says is that your
| logs should be sent to stdout.
|
| You write your logs in whatever structure you want and
| have something in the runtime environment (k8s, systemd,
| file redirection in bash whatever) forward them to the
| eventual storage location. I don't see the issue.
| cduzz wrote:
| So -- there's a file, that's created by some something in
| a container emitting a string of bytes. A file must go
| through a parsing and serialization phase to be useful to
| a logging infrastructure. Just because you decorate the
| contents of that file with { and } and other punctuation
| doesn't make that file a string of json records, until
| something's read and verified it.
|
| I'm supposed to just load it into memory hoping it's json
| or avro or whatever? Nope, errors happen; there's a limit
| to the number of bytes that can be sent from one place to
| another before the kernel may interrupt it (you're
| writing an application log and your runtime emits it's
| own debugging message; they get intermingled because
| they're both being written to a fifo).
|
| Maybe this is fine (usually it's fine) but eventually it
| isn't fine. Use a logging library, categories your logs,
| maybe start off with stdout but eventually put on some
| big person pants and use a logging api and use the
| stderr/stdout for high cardinality exceptions, not "all
| my logs"
| jameshart wrote:
| 12 factor does not say 'don't use a logging library'
|
| Use a logging library. Configure it to output well formed
| and categorized log messages to stdout. Let the runtime
| environment take that stream of data from there.
| lexicality wrote:
| Just so I understand, you are saying that the entire
| concept of the 12 factor app is bogus because it tells
| you to write unformatted logs to stdout because anything
| you write to stdout is by its very nature unformatted
| because even if you're outputting formatted data
| something else in your program might decide to
| concurrently write into your stdout buffer while you're
| writing a log line and that will corrupt your entire data
| stream irreparably?
| jameshart wrote:
| What 12 factor is telling you to do is to _not build a
| log file rotator into your application_.
|
| It is the least controversial and most broadly beneficial
| of the 12 factors.
|
| Why do you think it's foolish?
| cduzz wrote:
| It says "send it to stdout and let someone else figure it
| out."
|
| As that someone else, who often has bug reports like "why
| aren't my log events all in one record in my thing?
| Because you filled your stdout with newlines!" "Why did
| my message vanish? It was important!" "because it's
| malformed json because your runtime had an error; it's
| over there instead of over here."
|
| Programming's a stack of details; all of which matter.
| Log using a logging API where available; use
| stdout/stderr for exception logging only, if at all
| possible. If you want your logging to be useful, think
| carefully about how and what you log.
| jameshart wrote:
| That's a total misreading of what 12 factor is about.
|
| All it means is to let someone else (other than the
| program itself) figure out how to get the log messages
| (whatever you decide they should be) from your process
| (running wherever it is) to wherever you want to have
| those logs for inspection.
|
| Do not hardcode into your application the assumption that
| they are written to a file or sent to a port.
|
| In your application, write them to stdout.
|
| In local dev, stdout will probably go to an interactive
| console. In a container running in a cloud hosted cluster
| it will probably go to a networked log aggregator that
| collects it all into an indexed time oriented data store
| along with other logs from other running instances.
|
| Because you used stdout, that's easy to do.
|
| If you wrote them to a file whose name your application
| manages and changes itself to manage log rotation...
| either of those is _hard_ to do.
| jeremyjh wrote:
| The reason people pay outrageous prices for Splunk is you
| can get some pretty decent burgers out of sewage if you
| know what you are doing, and from personal experience the
| results can be totally game-changing if you are stuck with
| a mishmash of different log formats and have to support a
| large distributed application.
| Aurornis wrote:
| > Give an example.
|
| I can give an example. The part about sticky sessions is good
| generic advice for someone building generic web services, but
| there are situations where sticky session-like functionality
| or sharding-like behavior might be a better choice. Anyone
| adhering literally to the 12-factor guide would get hung up
| on trying to move that data to redis or memcached:
|
| > Some web systems rely on "sticky sessions" - that is,
| caching user session data in memory of the app's process and
| expecting future requests from the same visitor to be routed
| to the same process. Sticky sessions are a violation of
| twelve-factor and should never be used or relied upon.
| Session state data is a good candidate for a datastore that
| offers time-expiration, such as Memcached or Redis.
|
| The guide also has a strong preference for languages like
| Ruby, Perl, and Python. There's even a quip about how
| 12-factor prefers languages with a REPL, which in the wrong
| hands becomes an argument against using Rust or other
| languages.
|
| People who think the entire world is simple CRUD web apps
| tend to think that guides like this are completely universal.
| It's a good guide for people making common web apps, but it's
| not a perfect methodology for every backend service.
| brodouevencode wrote:
| > People who think the entire world is simple CRUD web apps
| tend to think that guides like this are completely
| universal
|
| That's the very first point they make:
|
| > In the modern era, software is commonly delivered as a
| service: called web apps, or software-as-a-service. The
| twelve-factor app is a methodology for building software-
| as-a-service apps that...
|
| Anything not web app/SaaS they express no opinion on.
| willseth wrote:
| I think it depends on how you interpret "rely". If your app
| can gracefully recover to a reasonable state from a failure
| to resume the expected session, then you could argue it's
| not that reliant on it. Many systems use sticky sessions as
| simply as an optimization, which is fine.
|
| You didn't actually give an complete example. When are
| sticky sessions a better choice than persistence?
| robertlagrant wrote:
| > but they seem to be much more useful to people whose only
| "value" is to provide ivory tower technical leadership.
|
| I've found it pretty useful in practice. I don't see why it's
| much more useful to those people than it has been for me.
| flakes wrote:
| I call that weaponized dogmas. It's at least better than people
| who insist on being DRY by writing an inscrutable generic
| helper function which will only save 3 or 4 lines of code
| duplication.
| laurent_du wrote:
| Nothing I hate more than a useless "helper" function that
| wraps a built-in feature of the language and gets buried in a
| "common lib" package that you need to inspect when you want
| to understand what is going on, instead of just relying on
| your knowledge of the language. Do we really need a function
| to map a dictionary and a list of keys to the corresponding
| values?
| BossingAround wrote:
| Oh my god I got a flashback to Groovy's missing method [0]
| which caused us so many issues, and made our codebase so
| unreadable.
|
| When you discover it for the first time, you think, "huh,
| neat, why not use it". Before you know it, nobody can
| reason about your code and if you leave, nobody wants to
| maintain it (esp. if your test suite is not extensive).
|
| [0] https://groovy-lang.org/metaprogramming.html
| booleandilemma wrote:
| I'm a huge hater of these and I think they almost always
| decrease readability of the code and slow down the
| development and review process.
|
| I want to read and write the language I'm developing with,
| not the helper methods of some overeager developer who
| wanted to leave his mark on the codebase.
| brodouevencode wrote:
| The absolute counter to weaponized dogma is itself weaponized
| dogma. There are times when implementing DRY, 12FA make sense
| and times when it doesn't. Be careful with hard-lined
| stances.
| samtho wrote:
| This is more a reflection of the organizational structure that
| allows or tolerates such behavior rather than anything to do
| with the 12 factor app methodology. I think most reasonable
| people would agree that if you were able to incorporate most of
| the list, you're doing alright.
| Aurornis wrote:
| > some non/psuedo-technical person has pulled out "12 factor"
| like a universal yellow card has been enough for me, overall,
| to ignore it almost completely.
|
| Ignoring it completely on principal is just as bad as the
| people who treat it like mandatory dogma.
|
| I've also had over-eager junior engineers or wannabe architects
| use the 12-factor article as mandatory requirements for any
| launch. It takes a firm and unified explanation that these are
| good goals to strive for, but in the real world we need to make
| compromises for launch and choose which aspects to delay or
| defer.
| lopatin wrote:
| Ignoring it completely is a viable strategy. A lot of 12
| factor sounds like common sense to people who regularly ship
| stuff. For example, if I fail to put the config in the
| environment, the risk that introduces will be immediately
| obvious in the PR and someone would be able to raise that
| concern, with real justification, not because it's declared
| in a 12 factor manifesto.
| i-use-nixos-btw wrote:
| I think ignoring it completely is a viable strategy once
| you've already gone through the stages - when your mind is
| in the right place.
|
| I have moved on a long way from the days that 12 factor was
| my bible, but those days completely redefined my way of
| developing software. I'm not exaggerating here - it
| completely shifted my approach.
|
| I guess it's similar to Scrum in that way - it isn't a
| great end point, but if you're stuck in the old ways (i.e.
| the methods used before it was even a thing) then it's at
| least a good approach to sink your teeth into, try it out,
| shape your approach, and move on from there.
|
| Basically, I'd rather see a replacement that improves on 12
| factor than throwing out the book entirely.
| ascendantlogic wrote:
| > A lot of 12 factor sounds like common sense to people who
| regularly ship stuff.
|
| In 2023, yes. A lot of the 12 factor methodologies became
| table stakes in the modern cloud era. Back when this was
| written "cloud" was not nearly as ubiquitous as it is now,
| although we were certainly headed in that direction. A lot
| of the comments here are viewing this through a 2023 lens
| and forgetting it was written in 2011.
| darkwater wrote:
| Well, if your app log directly to files on disk (extra bonus if
| it doesn't respect any verbosity setting) or will only works if
| secrets are hard coded in config files (even if this could be
| work-arounded at deploy time probably), I for one will do all
| that I can to avoid your app reaches production.
|
| Yeah, I work in ops^W platform.
| brodouevencode wrote:
| No amount of dogma should interfere with a deployment within
| the scope of an MVP. Once it's out of MVP then anything not
| addressed best practice-wise is tech debt. If your organization
| had adopted 12FA as a set of best practices then those should
| be met, but they should never interfere with a deployment.
|
| 12FA is not a single box that is to be checked. It can and in
| most cases probably should be implemented over time as the
| product matures, taking each point, point by point (and maybe
| even breaking those up) and adding that into the product. If
| you've done good engineering up front (abstractions and
| interfaces where it makes sense, not hard coding everything)
| this should not be an issue.
|
| inb4 YAGNI: this is just as abused as 12FA.
|
| EDIT: the biggest thing 12FA is missing are concrete examples
| that junior engineers can reference.
| jsight wrote:
| TBH, most of the value in things like this is for consultants
| who go into shops that really don't know better.
|
| Being able to point management at this can be really useful.
|
| For most of us here, it is 99% common sense.
| bunderbunder wrote:
| Indeed.
|
| Hard to believe that treating best practices as strict rules
| would not turn out to be best practice, isn't it?
| ireallywantthat wrote:
| Nowadays, I see many articles titled "Beyond 12 factor Apps"
| which proposes even more factors for App prominent among them
| being one from IBM. What's your opinion on that?
|
| https://developer.ibm.com/articles/15-factor-applications/
| Moto7451 wrote:
| This is a small but real trend. My experience at a company that
| went deep into expanding beyond/removing some of the 12 Factors
| is that it breaks portability of containerized software built
| with those assumptions. There were a few times I needed to
| install a vendor product or open source project that was built
| to the 12 factors but spent a month or more just building a
| shim to make it conform to our particular bend on it. I'm sure
| there are good reasons to deviate but I wasn't particularly
| impressed with our decisions and more importantly why we made
| them.
| aj_g wrote:
| For sure an influential engineering codex. Feels almost strange
| thinking about all the easy abstractions we have now for hosting,
| like Render or Vercel or that this was written in 2012, when web
| apps were much much more of a wild west in terms of
| accepted/shared practices.
| RugnirViking wrote:
| I was kinda expecting this from the title to be some sort of
| commentary on two factor authentication, like an app that needs
| passport photo, face scan, drivers licence, sms text, google
| authenticator, email link, password, fingerprint, all together to
| log in just once.
| patrickaljord wrote:
| Your average centralized crypto exchange.
| moffkalast wrote:
| Sounds like roughly what the average stock or crypto exchange
| requires before they let you trade haha.
| rando_dfad wrote:
| Thank KYC/AML for that.
|
| Money is not data, it is a complex social construct. Moving
| money requires interacting with that social construct, i.e.
| government regulations.
|
| Yes, crypto was supposed to get around that. No, government
| didn't agree.
| moffkalast wrote:
| Well I'm glad they at least aren't collecting skin samples
| yet.
| revskill wrote:
| Sorry, i put all secrets in my code. I don't need 12 factors at
| all.
|
| It's super easy to deploy to multiple environments, because code
| is single source of truth.
| Aurornis wrote:
| Secrets in code is bad practice because it puts the secrets in
| source control and gives them to anyone with access to the
| code.
|
| You separate out the secrets to keep them secret.
| revskill wrote:
| Next evolution of Git should automatically introspect the
| code base for comment to leaving out all "secrets".
|
| My point is, make the code easy to clone in a runnable shape
| wihtout any hacking on host system.
| gdprrrr wrote:
| You can configure GitLab to reject the push if you comitted
| secrets, based on regex marching with common formats for
| tokens.
| pjs_ wrote:
| Hell yeah brother. Move fast and break things philosophy
| porsager wrote:
| It's not a problem. Here you go: https://github.com/AGWA/git-
| crypt
| samus wrote:
| This only works if secrets are frequently rotated and contain
| enough entropy. Else, all secrets will eventually be
| decrypted from leaked repositories and distribution
| artifacts.
| brodouevencode wrote:
| What company do you work for? I'd like to cancel my
| subscription.
|
| Seriously...bad idea. One repo leak or even one bad copy/paste
| in StackOverflow and you're done.
| ACV001 wrote:
| Boring. This is already happening in any sane modern application.
| seanhunter wrote:
| It might be worth you reading
| https://news.ycombinator.com/newsguidelines.html
|
| In particular: > Be kind. Don't be snarky.
| Converse curiously; don't cross-examine. Edit out swipes.
| > Please don't post shallow dismissals, especially of other
| people's work. A good critical comment teaches us something.
| gouggoug wrote:
| > This is already happening in any sane modern application.
|
| Probably in part thanks to 12factor.net
| sbjs wrote:
| I genuinely feel like there's two camps of professionals: those
| who know how to do something and do it extremely well; and those
| who know how to describe something and break it down very well.
| The former are the ones making the magic happen, like Fabrice
| Bellard. The latter become very influential teachers like Bob
| Martin. Both are needed (although the former more so), but the
| latter usually get all the attention and credit.
| webscalist wrote:
| I prefer to have known configuration presets (groups) version
| controlled. Documents known shared environments (db host, ...
| etc) well.
| vilunov wrote:
| I feel like each of this points can be reasonably disputed.
|
| 1. One app - one repo. There is nothing fundamentally wrong with
| multiple apps (as in separate pods/containers/processes) being
| developed in a single repo, as long as they are tightly coupled
| functionally, share the release cycle, but still need to be
| deployed separately to get advantages of separate processes and
| independent scaling. The first example that comes to mind is
| separation of public api and worker processes, e.g. ruby's
| sidekiq, python's celery or a generic kafka consumer.
|
| 2. > A twelve-factor app never relies on implicit existence of
| system-wide packages.
|
| This is incredibly hard to achieve in practice if you're not
| running something nix, and even then there are leaks of
| dependencies on kernel syscall APIs. Every Rust app implicitly
| depends on a glibc (except musl ones), which is present in major
| Linux distros except slim ones such as Alpine. In fact, the whole
| Docker crutch IMO is needed specifically to alleviate this
| problem.
|
| 3. > The twelve-factor app stores config in environment variables
|
| It seems to me much more brittle, as it forces you to store
| secrets in envs, which could be less secure, and to abandon
| structured file configs, which may enjoy type safety, IDE
| autocomplete and automatic parsing. With envs you are forced to
| implement parses for non-trivial and non-string config vars
| yourself. In fact, envs are often committed into repos as well in
| form of dot-env files, so the point about commit-safety is moot
| as well.
| twic wrote:
| > This is incredibly hard to achieve in practice if you're not
| running something nix, and even then there are leaks of
| dependencies on kernel syscall APIs.
|
| You're misinterpreting that guideline. The text says (my
| emphasis):
|
| > Most programming languages offer a packaging system for
| distributing support libraries, such as CPAN for Perl or
| Rubygems for Ruby. Libraries installed through a packaging
| system can be installed _system-wide_ (known as "site
| packages")
|
| It's not saying not to depend on the operating system (of which
| glibc is part), it's saying not to require particular language
| package manager packages to be installed on the machine.
|
| Agreed that the other ones you mention are silly, though.
| Especially the one about environment variables - that isn't
| actually reasoned advice, it was just the authors writing down
| what was most common in Ruby development at the time, on the
| assumption that that must be the best thing to do.
| vilunov wrote:
| Well, glibc could be considered part of the operating system,
| although this too is arguably not correct, because some
| technologies go out of the way to not depend on anything
| present in the system and be completely self-contained, e.g.
| fully statically linked Go binary with no Cgo usage.
|
| Nevertheless, glibc is not the only system dependency which
| is often used by apps, another such example is JVM of a
| specific version, which is not tracked by any of the popular
| JVM build tools and package managers; and OpenSSL, which is
| too installed by the system package manager.
|
| Dockerfiles exists to fix this by defining all the necessary
| installation steps as a sequence of instructions, but this is
| far from a proper package manager.
| aniforprez wrote:
| In my experience, sourcing config from env variables seems
| like the best thing to do because you can completely ignore
| where the app is running in favor of injecting whatever you
| want in any environment of your choosing and your app no
| longer needs to care about that. The implication is that
| business logic must not include the content of your config or
| where it is sourced from. It must simply use the values in
| the config. This way my apps could give less of a fart if I
| run it as a container in kube, in nomad, in local on
| localhost, in a devcontainer, in a directory with vars loaded
| using direnv, in vercel/netlify and so on. This also allows
| incredible flexibility in how that config is sourced as long
| as you simply dump them into the env vars. Want to load
| secrets from a .env file on local? Sure. Want to load secrets
| from Vault on production? Run a simple script to load vault
| secrets onto the env and run your app.
| dalyons wrote:
| yeah this is the actual point - it makes the code/app
| totally portable & decoupled from config. I find it
| significantly better than having to have the app code
| choose different config based on interrogating facts about
| its runtime (eg am i in prod? use this value. am i in test?
| use this)
| jjice wrote:
| I heavily agree. I used to work on a LAMP stack where the
| deployment was just dumping the code into an EC2 instance.
| All is fine until there's a weird system dependency error
| that isn't documented because the AMI being used has just
| been hand built over the last three years. Have fun tracking
| down that specific PHP system dependency when AL2 goes EOL. I
| don't want to write code that doesn't have _documented_
| dependencies on system packages. Even just a Dockerfile
| serves as solid documentation on what we're using.
| damianh wrote:
| "One app - one repo"
|
| Where did you see that being expressed?
| vilunov wrote:
| https://12factor.net/codebase
| ulizzle wrote:
| It says there that a codebase can contain multiple repos.
| They don't mean a monorepo when they say codebase
| vilunov wrote:
| > A codebase is any single repo (in a centralized
| revision control system like Subversion), or any set of
| repos who share a root commit (in a decentralized
| revision control system like Git).
|
| They do mean that a repo = a codebase. A remark about
| "set of repos" is just a technicality of distributed
| VCSs, meaning that each copy cloned from a single source
| (or from other copies) is a single repo.
| samus wrote:
| I agree with your view of the first rule and would gladly store
| multiple related apps in one repo. No rationale for why one
| repo should only containing one app is given, therefore that
| rule can be disregarded.
|
| Dynamic or managed languages can often ignore most of the
| hassle with system packages.
| brodouevencode wrote:
| > 1. One app - one repo.
|
| There is a U-shaped utility to monorepos. Most projects fall in
| the middle somewhere, largely ones managed by independent
| engineering teams that have no centralized
| platform/devops/whatever-we-are-calling-it-these-days team.
| kagakuninja wrote:
| They also say every server should be a separate process, which
| is debatable. When using languages that lack real threads
| (Ruby, Python, etc) you have no choice.
| talent_deprived wrote:
| > There is nothing fundamentally wrong with multiple apps (as
| in separate pods/containers/processes) being developed in a
| single repo, as long as they are tightly coupled functionally,
| share the release cycle, but still need to be deployed
| separately to get advantages of separate processes and
| independent scaling.
|
| We disagree on fundamentals then and I think more of the world
| agrees with me and with the 12 factors.
| twic wrote:
| The big thing missing from this document is justifications for
| the rules. It's almost all just rules. Are the rules good? Hard
| to say, and this document won't help you find out.
| brodouevencode wrote:
| If you click on the rule it links to a page that is what you're
| asking for. For example: https://12factor.net/backing-services
| samus wrote:
| Not all the rules are explained on. For example, a detailed
| reasoning why there should be only one app per repository is
| not provided. It is only stated that it is not the 12factor
| way.
| twic wrote:
| The page you link to does not contain a justification for the
| rule.
| moondev wrote:
| Kubernetes basically forces you to deploy 12fa, using these
| guidelines is a pretty good playbook for migrating an existing
| application as well.
| sergioisidoro wrote:
| Twelve factor app: use environment for config
|
| Docker: Don't use environment for config, it's insecure
|
| I like and use many of the patterns of 12 factor, but indeed some
| of these were written in the context of VPS, where environment is
| stable, secure and more constant (unlike containers, where
| environment might end up shipped in one of the layers). In the
| age of containers, this particular point has been a bit of a
| struggle for me. Docker secrets don't always play well, and you
| need to do some gymnastics to make it work.
| samus wrote:
| The basic idea is still valid. You might just have to use a
| more appropriate mechanism when using containers. For example,
| the explanation of the Config factor explains that a config
| file can also be used.
| 3by7 wrote:
| In retrospect, a better name for that section could have been
| "Separate config from code" as it says in one of the first
| paragraphs.
| chrissoundz wrote:
| How are environment variables insecure?
| dharmab wrote:
| Envvars have much weaker access controls than files;
| basically anything in the same PID namespace can read your
| envvars. Poke around in /proc/*/environ to see for yourself.
| Files can use user permissions and MAC (AppArmor, SELinux,
| etc) to secure them against unauthorized processes.
| wutwutwat wrote:
| If running in a containerized environment ENV is injected
| only to the container, at container runtime. If you're
| running your containers with a single process, you are
| doing the exact access control you just outlined. At that
| point what is the difference between a file on disk with
| read permissions for the proc owner vs only the proc having
| the ENV injected to its shell at runtime? I'd maybe even
| say the ENV at runtime is _more_ secure than the chown'd
| file on disk because the ENV injected variant is ephemeral
| and when that container /proc dies, so does the ENV sitting
| in its memory, whereas the other leaves a file on disk
| potentially depending on where it was written to,
| regardless of who can access it. Then there's the fact that
| a lot more people than you think run containers as root so
| all file access control goes out the window.
|
| From a security perspective it is always more secure to
| have something only in memory of the running process than
| it is to have a file on disk regardless of file permissions
| dharmab wrote:
| Containers don't enforce that isolation. Another process
| can nsenter the container's PID namespace. You also have
| issues if your container's PID1 creates subprocesses.
|
| File are not necessarily written to disk. e.g. the
| Secrets CSI Driver loads secrets directly from a secrets
| store as virtual files within ephemeral volumes.
| jcrites wrote:
| Could you elaborate? How are environment variables insecure? If
| the secret will be present in the application throughout it's
| lifetime, then what's a better alternative?
|
| Just because a secret is injected into the application as an
| environment variable does not mean it's handled insecurely
| elsewhere. For example, AWS ECS containers have built-in
| support for fetching secrets from Secret Manager at startup and
| passing them as environment variables:
| https://docs.aws.amazon.com/AmazonECS/latest/developerguide/...
|
| This fetches the secret at the time the container starts up
| (IIRC), and uses the IAM credentials of the running application
| to fetch them from Secret Manager (so it must have permission
| to the secret).
|
| The merit of using environment variables seems to depend
| entirely on how they are, well, configured. With a mechanism
| like this I don't see much disadvantage. (The main disadvantage
| I see is that generic malware that attempts to dump environment
| variables might capture them, but defending against that threat
| is largely obfuscation, unless you aren't storing the secret at
| all permanently in memory.)
| dathinab wrote:
| - any sub process has full unlimited access to env variables
|
| - env variables are prone to be dumped in various contexts
| and might leak secrets to e.g. logs
|
| - env variables might leak, e.g. in context of docker build
| stages etc. (but then using building a docker image to build
| both the image and the software is often anyway not a good
| idea)
|
| - env variables are often easily accessible by other
| processes
|
| the method you mentioned of ad-hoc injecting a secret as env
| by some form of security manager for containers avoid many
| such problems
| lazyant wrote:
| > - env variables are often easily accessible by other
| processes
|
| A process would need to have the same permission (same
| user) or root to read /proc/$PID/environ no? or what
| mechanism is there for a process to read another process
| env vars (not children processes, that was already
| mentioned)? unless of course, a process dumped them in some
| world-readable file (like logs as mentioned) or some other
| silly way
| ezekg wrote:
| Exactly. The operating system already offers security for
| environment variables: use it. And you get the benefit of
| increased security in other places by not sharing users
| between applications.
| yjftsjthsd-h wrote:
| > env variables might leak, e.g. in context of docker build
| stages etc. (but then using building a docker image to
| build both the image and the software is often anyway not a
| good idea)
|
| Since the rule describes runtime configuration, it
| shouldn't be involved with the build process at all
| sergioisidoro wrote:
| My comment is specific to docker, although environment as a
| broader sense can be discussed.
|
| I remember reading that it is possible to end up in
| situations where your environments can end up in the layers
| of the docker image. Also, you can read the secrets of a
| container with inspect, and apparently linked containers
| might be able to access other containers's secrets.
|
| I think it was this article: https://snyk.io/blog/keeping-
| docker-secrets-secure/
| derefr wrote:
| Presumably because /proc/$PID/environ exists and can be read
| by the user that owns the process -- which means that any
| attacker that can figure out how to spawn an arbitrary
| subprocess running as the daemon user (a pretty common
| vulnerability class!) can exfiltrate all your env-vars.
|
| It's similar to the reason that you're not supposed to supply
| passwords on the command-line (/proc/$PID/cmdline exists),
| but instead either read them from stdin, or do the big dance
| of
|
| 1. creating a file that contains the password, is owned by
| root, and is 0400;
|
| 2. having your daemon start off running as root for just long
| enough to read the password into process memory;
|
| 3. then having your daemon drop privileges before doing any
| work, so that someone exploiting the daemon _won 't_ have the
| privilege as the "do the work" user to read the file with the
| password in it.
| dathinab wrote:
| it's both true
|
| putting any form of secrets into a env var is deeply insecure
|
| using env is the most simple uniformly available way to have
| configs
|
| through I would take it a step further, anything which must not
| be leaked probably should not be in any config, weather env or
| file or similar, it should be injected through other means if
| viable
| 0cf8612b2e1e wrote:
| On the other hand, without platform support there are not a lot
| of easy alternatives. It at least encourages separation of
| config and code, keeping secrets out of vcs.
| wutwutwat wrote:
| One could argue that you should never consider any input as
| "safe to consume", and if you write defensive app code, would
| never consider the ENV to ever be secure or what you expect,
| and would therefor sanitize that input before consuming. If
| you're blindly trusting any input just because you're single
| tenant, you're in for some hard to track down bugs and long
| nights when those circumstances one day change due to random
| business pivots.
| jsight wrote:
| Yeah, I've never been a fan of secrets in env vars for this
| reason. It is shocking how frequent dumb debug consoles have
| accidentally exposed env vars. You really don't want that to
| accidentally expose your private keys.
|
| There's really nothing wrong with secrets mounted on the file
| system via k8s. But, of course, all of this depends on your
| deployment environment.
|
| Sometimes env vars are the least bad option, because secrets
| really are always hard.
| berkes wrote:
| > dumb debug consoles have accidentally exposed env vars
|
| Huh? If the app can read the secret, then the app can... read
| the secret. Maybe I'm misunderstanding what you call a "debug
| console", but if this is anything running in the app, it can
| ... read the secrets. Regardless of their source.
|
| Or, to put it differently: it matters nothing if your secrets
| at rest are stored in a quantum-resistent-encrypted vault,
| secrets.yml or ENV vars, if your logging, backtracing or
| debug consoles dump them somewhere, you are compromised
| regardless.
| jsight wrote:
| This is absolutely true, but it is far more common for
| someone to accidentally leave open a display of env vars
| than a display of files.
|
| I've got this problem to debug in prod, I know, I'll embed
| the env vars in the source code of the error page. No big
| deal and it might help me debug that! For a dev used to not
| using secrets in env vars (for example, when testing
| locally), this is an easy mistake. He may even use a third
| party library that embeds features like this without
| knowing it! Even things like the jmx console can be risky,
| since access to that may not relate 1:1 with access to
| secrets.
|
| I've never seen anyone make a similar mistake and
| accidentally embed all_my_passwords.ini in their output.
|
| I must admit that some of what I've said above has become
| outdated, since it is now a much more common practice to
| embed passwords in env vars than it used to be.
| zemo wrote:
| the twelve-factor app is a set of recommendations from 2011 that
| are based less on engineering principles and more on the
| capabilities of Heroku and containerized infrastructure in 2011.
| For example:
|
| > Another approach to config is the use of config files which are
| not checked into revision control, such as config/database.yml in
| Rails. This is a huge improvement over using constants which are
| checked into the code repo, but still has weaknesses: it's easy
| to mistakenly check in a config file to the repo; there is a
| tendency for config files to be scattered about in different
| places and different formats, making it hard to see and manage
| all the config in one place. Further, these formats tend to be
| language- or framework-specific.
|
| > The twelve-factor app stores config in environment variables
| (often shortened to env vars or env).
|
| yeah the reason they argue for this is that the people that wrote
| this worked on Heroku, and the way that Heroku worked is you
| populated the environment variables from some fields in a web
| app. If you want your config history tracked in version control,
| or you do gitops, or you have k8s configmaps, or you want to have
| your configuration files on a mounted volume ... those things are
| all broadly fine; they keep the configuration state separate from
| the app deploy state. This document really confuses the forest
| with the trees and recommends things based less on actual
| engineering principles and more on the product capabilities of
| the corporation that produced it. It is an actively harmful set
| of guidelines.
| morelisp wrote:
| The 12 factor app that stored config in env vars found it
| trivial to migrate to ConfigMaps. The LOB apps today that
| expect complex ConfigMap layouts (at worst case, directly from
| the k8s API) are going to be hell to migrate to whatever comes
| after k8s.
| throw1234651234 wrote:
| The whole .env file holding all your configs thing is a
| disaster. People forget to update the template, it has to be
| passed around, etc.
|
| Configure your secrets in Azure Key Vault (AWS Secrets Manager,
| GCP-something-I-forget) or whatever the non-cloud Vault
| software is called.
|
| Devs only set client_id, client_secret, and env defaulted to
| dev. On startup, the app fetches configs for the secret store.
| The needed secrets are in source control. If the needed secret
| is missing from the secret store, there is a clear error on
| startup.
|
| You are welcome, I just saved you weeks of onboarding,
| confusion, and devs passing around .envs on Slack.
| campbel wrote:
| I agree, sops / ejson / gitcrypt or something else that
| encrypts a file in vcs, then use your cloud or shared secret
| vault to disperse the decryption key.
| danwee wrote:
| > You are welcome, I just saved you weeks of onboarding,
| confusion, and devs passing around .envs on Slack.
|
| Don't get it. At one company I worked, .env files for the DEV
| environment were open (the file itself was stored on Vault,
| so only engineers had access to it), and one could only
| access DEV resources via VPN. So, starting a service using
| the DEV .env file was rather easy an uncomplicated.
| throw1234651234 wrote:
| Here is the problem with that. A dev pushes a change that
| requires a change to the env file. Another dev pulls down
| the code, but doesn't change the env file. Everything
| breaks. At really bad companies, DevOps sets env vars from
| somewhere other than that file passed around by devs -
| another failure point.
|
| My proposal:
|
| 1. Key Vault | HashiCorp Vault | whatever is the source of
| truth for secrets.
|
| 2. When a developer adds a key, they add it to Key Vault
| and update code to use it.
|
| 3. On app startup, code checks the dev's local for 2 env
| vars ONLY, which allow access to key vault.
|
| 4. The app then tries to pull in all the env vars it needs
| by secret name.
|
| 5. If any are missing, it fails.
|
| This keeps devs from having to pass around .env files
| outside of source control. Btw, Azure (and all other cloud)
| builds can be configured to pull env vars directly from Key
| Vault, so there's that benefit too.
| parasti wrote:
| "5. If any are missing, it fails."
|
| That's literally all you need to solve the scenario you
| described with .env files.
| meowtimemania wrote:
| The difference is any time the .env config is updated,
| all engineers have to update their .env file.
|
| So with .env file:
|
| - developer adds to .env
|
| - pushes code
|
| - all developers see failures in dev env due to missing
| env var
|
| - all developers search slack/email to figure out what
| went wrong
|
| - all developers update their .env file and things are
| working again
|
| --------- In gp's comment -------
|
| - one person works on feature that introduces new config
|
| - failure happens in their local environment
|
| - they update config backend and push code
|
| - all other developers get updates with no changes needed
| throw1234651234 wrote:
| Exactly, thank you for phrasing it much better than I
| did!
| Rapzid wrote:
| You can also just supply defaults that can be locally
| overriden.. Benefit is they are reviewable in the same PR
| that adds the code depending on them..
| nickzelei wrote:
| I've resulted into a pretty similar setup.
|
| We use SOPS and the encrypted .env file is stored in VCS. The
| key is stored in 1password with instructions on where to
| stuff it on your machine. Then you just decrypt the .env file
| and you're off to the races.
| edwinbalani wrote:
| For anyone new to SOPS like I was -
| https://github.com/getsops/sops
| victorNicollet wrote:
| Hard agree. I think it's easy to get stuck on the mental
| model that "the configuration value is the secret" (which
| means the configuration itself needs special handling), and
| the correct mental model should be "the configuration value
| is the identifier of the secret", with another system (Azure
| Key Vault, etc) being responsible for converting the secret-
| identifier into the secret-value.
|
| My company open sourced the .NET library we use for this:
| https://github.com/Lokad/Secrets
| geraldwhen wrote:
| The corporate VPN and proxy setups tend to fail or be slow.
| This would add non-trivial startup time to any app, and
| catastrophic startup time to integration tests.
| jbotdev wrote:
| It is true that this is somewhat influenced by how Heroku
| works, but ConfigMaps and GitOps do not meet the same security
| and usability requirements as Heroku config/env vars.
|
| If you want secure config storage on Kubernetes, you end up
| using Secrets, which ends up being key-value like env vars
| anyway. If you want similar security with Git you need a layer
| of encryption, which breaks diffs and requires additional
| tooling. This all leads back to why high-level deployment tools
| like Heroku were created.
| campbel wrote:
| > If you want secure config storage on Kubernetes, you end up
| using Secrets, which ends up being key-value like env vars
| anyway.
|
| You can load the secret file directly into the app, no need
| to load it as env vars or keep it strictly as key-value
| pairs.
|
| > If you want similar security with Git you need a layer of
| encryption, which breaks diffs and requires additional
| tooling. This all leads back to why high-level deployment
| tools like Heroku were created.
|
| You can use tools like ejson[1] or sops[2] to get encrypted
| files checked into Git that have key level granularity on
| diffs.
|
| There is definitely a place for higher level abstractions
| than Kubernetes. Mostly it gives operators a standard
| platform to build from when teams outgrow the PaaS sandbox.
|
| [1] https://github.com/Shopify/ejson [2]
| https://github.com/getsops/sops
| Spivak wrote:
| > You can load the secret file directly into the app
|
| Now you're getting away from the spirit of 12factor and
| hard-coupling them again. The intent is for the app to
| consume the secrets but have no knowledge or care where
| they came from.
|
| Edit: misread as "load the secrets directly into the app."
| Yeah, this is just env vars but different.
| yjftsjthsd-h wrote:
| I'm not really kubernetes expert, but don't they come
| from /my-secrets and the application doesn't need to care
| how that got mounted?
| derefr wrote:
| My trick for this with k8s Secrets, when there's some
| sort of config file that absolutely must be consumed from
| disk, is to
|
| 1. create a Secret that defines the file as a key
|
| 2. mount that Secret into the container
| (.secret.secretName in the volume definition) under an
| arbitrary conventional path, e.g. /secrets
|
| 3. define an env-var _that gives the full path to the
| file_ ; where the fallback when this env-var _isn 't_
| defined is to find the file in the working directory with
| a conventional filename.
|
| If your own code uses the file, it can read the path from
| this env-var; while if the core of your container is some
| opaque tool that doesn't know about the env-var but
| _does_ take the file 's path on the command-line, then
| you can wrap the tool in an entrypoint script that reads
| the env-var and passes it as a command-line arg to the
| tool.
|
| IMHO, this approach is flexible but principle-of-least-
| surprise obeying for both development and production
| deployment.
| inferiorhuman wrote:
| That sounds an awful lot like a .env file to me.
| pojzon wrote:
| yup thats exactly what .env is, and tbh that doesn't
| change anything in the bigger picture
| kelnos wrote:
| > _Now you 're getting away from the spirit of 12factor_
|
| The entire point here is that 12-factor isn't the be-all,
| end-all, so this is irrelevant.
|
| > _The intent is for the app to consume the secrets but
| have no knowledge or care where they came from._
|
| This doesn't really make sense, and is an impossible
| requirement. The app will always have to know where
| configuration or secrets come from. Environment variables
| is just one option of "knowing where they come from".
| Choosing file storage is just as valid.
| kagakuninja wrote:
| I don't use kubernates. We have JVM servers running on
| AWS. We load secrets from Secrets Manager. Configs are
| mostly HOCON files stored in S3. The knowledge of where
| to find them is configured via env vars.
|
| We use a custom setup in which the config values are
| loaded from multiple sources actually, so we could put
| everything into secrets manager, or load them all from
| env vars or HOCON files.
|
| If we had to set every config value in ECS / lambda using
| env vars, it would be a major pain in the ass, and error
| prone.
| pojzon wrote:
| > We load secrets from Secrets Manager
|
| And how your apps authenticate to Secrets Manager ? Did
| you ever call `env` on a pod that has IRSA configured ?
|
| This is just a middle step to do exactly the same thing
| but instead of using envFromSecret you use envFromSM
| maccard wrote:
| Not OP but an IAM role with scoped access to secrets.
| Better again, using secretsFrom in the task definition
| which injects the secret as an environment variable for
| you.
| redeux wrote:
| > somewhat influenced by how Heroku works
|
| The Twelve-Factor App is marketing content produced by the
| former Founder & CTO of Heroku. It's a great piece of content
| no doubt, but it should be viewed with that lens.
| probotect0r wrote:
| If it's marketing content then it's not doing a very good
| job. I have known about the 12factor app principles for a
| long time, and only found out today that it has ties to
| Heroku.
| xwdv wrote:
| If it is truly harmful, it shouldn't be sitting at the top of
| Hackernews. Let's flag it.
| zemo wrote:
| I... don't think the "flag" functionality is intended to
| communicate "this is bad advice".
| xwdv wrote:
| It is "harmful" advice, not bad. Regardless, why should it
| sit on the front page of hackernews?
| zemo wrote:
| it's really hard for me to express just how much content
| hits the front page of HN that, if you follow it, will
| make you worse at programming.
| v2223943777435 wrote:
| i got a ton of value out of reading the comments though
| wutwutwat wrote:
| Heroku didn't invent ENV variables nor did it dream up the idea
| of using them for app config. This has been in use long before
| Heroku existed. I'll give Heroku credit for making it common
| knowledge and spreading the use of these concepts, though.
| zemo wrote:
| yeah I never suggested that heroku invented environment
| variables, and the idea that environment variables weren't
| common knowledge before heroku came along is ... let's say at
| best revisionist.
| wutwutwat wrote:
| True.
|
| If you remember the world before 12 factor you know that
| even if motivated by self serving incentives, Heroku making
| this mainstream has been an overall win for the entire
| industry and we are generally better and more secure for
| them having done so. We are still benefitting from all of
| this today, and Heroku who championed it is making death
| rattles.
| hirako2000 wrote:
| While valid points in many contexts, I've seen many other
| contexts where following this principle/pattern a/ yields
| none of its benefits b/ results in very significant
| business impact.
|
| Here is some obvious case observed many times.
|
| A small and very fast pace team builds a server
| application having limited to no external dependencies.
| No db even.
|
| Just a server doing stuff.
|
| It does have some config yes, many many key/value pairs
| even, a config file ended up with a dozen properties just
| to control Req per sec limits and such.
|
| Now just to deploy the thing, since of course env defined
| configurations is where things belong in best practice,
| the dev team setup a process to peer review those env
| values and despite all of that, fail to get the app
| running.
|
| Human errors happen but since more mistakes and release
| delays were caused by misconfiguration than any other
| reasons, the keen and bright enginnering team all agreed
| it would be better to have a config repo. One genius in
| the group argued that having a config service would be
| even neater than have the app read from a git repo! The
| whole got so hyped and of course spend the whole next
| sprint building and testing that. Brilliant, it worked!
|
| Except that of course outages happened, of course.
| Outages happen.
|
| Bringing down all environments one by one, hence blocking
| pretty much everyone. Everything went down because hey,
| since we made that effort which took for longer than we
| thought to build a config service and have our app reads
| from it, one of the dev minds thought it was a genius
| idea to have the app reload the config every 5 mins at
| runtime so that if a configuration change was made, no
| need to even restart anything. Of course error handling
| wasn't robust and well tested since the boss reminded the
| team we've got to ship a product by the end of the
| quarter and solving the universe can maybe wait 5 or 10
| years after the IPO.
|
| Team resorted to simply track the config in the repo.
| Kept the runtime reload of the config and things worked
| just fine.
|
| Things can be done right and impacts could have been
| avoided. The example may seem like a twisted case since
| these were all configurations, not environment specific
| values. But I've just seen so many occurences of env
| variables being separated from the application repo
| costing so much and distracting contributors for no
| benefit that I have grown very cautious of it.
| ElevenLathe wrote:
| The intent of this part of 12 Factor comes from the Bad
| Old Days when devs didn't run their own stuff in
| production. There was a dev team who could push code and
| cut releases, and an ops team who would then deploy the
| releases, and who was paged when things blew up.
|
| If you store the database hostname/credentials, as an
| example, in the "code" (either literally in the app code,
| or in a config.ini that lives in the repo and gets baked
| into a release), then the ops people can't quickly
| repoint the app at a hot-spare database during an
| incident: they'll need to edit the repo, push it, cut a
| new release, etc. If there is a long build step (as for a
| C++ app, for example), then this isn't really tenable
| during an outage. If this is in an environment variable,
| their normal deployment scripts can make the change for
| them.
|
| I agree that this is probably not necessary anymore if
| your team is all wearing both hats (dev and ops), and its
| just as quick to make an edit to the repo and cut/deploy
| a new release as it would have been for the Ops team to
| run their deployment script in the old days.
| phendrenad2 wrote:
| History repeats itself. Netlify tried this with the MACH
| Alliance.
| donatj wrote:
| The invention of dotenv files just makes me chuckle and seems
| to completely miss the point of not putting them in a config
| file.
| bunderbunder wrote:
| dotenv files are great as a piece of developer ergonomics.
| And that's fine. I use them for that purpose all the time. I
| would not willingly consent to using them to manage
| environment variables in production. But I would also not
| willingly consent to doing it the way that's best for
| production during development.
|
| I think that's fine. 12-factor is a set of guidelines about
| what you do _in production_. There 's a whole universe of
| things that are smart ideas for development and bad ideas for
| production. Leaving debug symbols in your binaries, leaving
| ports for connecting a debugger open on your containers, etc.
| scooble wrote:
| I've read a bunch of tutorials promoting env vars to avoid
| secrets being accidentally checked into git. Step 1 is
| usually to create a .env file and add it to your gitignore.
| rbanffy wrote:
| "Store config in the environment" does not necessarily mean
| configs are in environment variables - it only means
| configuration comes from the hosting environment, not the
| application itself - you are not adding a "settings.json" file
| to your machine before starting the app.
|
| This way, the same source deployed in, i.e. your AKS cluster in
| Azure EU north will take the configuration you set for that
| cluster while the one you run out of an Docker Swarm of RPi
| Zero's in an Ikea photo frame (I have one cluster like that,
| called Gibson), will take the configuration from that cluster.
| Kwpolska wrote:
| The page https://12factor.net/config disagrees with you:
|
| > The twelve-factor app stores config in environment
| variables (often shortened to env vars or env).
| rbanffy wrote:
| I would assume that is the interpretation back in 2011.
| While the mechanism changed, the principle still holds.
| lucideer wrote:
| > _or you have k8s configmaps [...] those things are all
| broadly fine; they keep the configuration state separate from
| the app deploy state_
|
| Why are they fine? This seems to be an argument grounded in
| "it's a newer norm therefore it's better" but I don't see the
| justification. I've never used Heroku but ConfigMaps are a bane
| to manage. Even as a general programming principle within
| application code, colocality is a well-known, long-discussed
| topic around improving maintainability. If you're not using
| monorepos, that's currently not a reasonable option with
| current orchestration conventions.
|
| Even if you are using monorepos, you're highly likely to either
| get into writing & automating DSLs to feed into your configmaps
| or do weird things like using Jinja to do file includes in your
| mounts. That's all before you run into the etcd 1MB size limit
| & realise that actually this whole system is a hack.
|
| TL;DR newer ain't intrinsically better
| throwaway4good wrote:
| As far as I can tell this is the standard way of doing things
| also in a kubernetes setup: The application gets its
| configuration from the environment variables.
|
| Then in the case of a kubernetes setup. The application is
| wrapped inside a mechanism that gets the configuration from
| some where else.
| ljm wrote:
| Nobody calls it '12 factor' anymore but we still adhere to a
| generic set of principles as a result (since 12-factor predated
| docker and kubernetes in the mainstream).
|
| - Logs as a stream: yep, just write your logs to STDOUT instead
| of a file. Let the orchestrator deal with reading and storing
| it.
|
| - Configuration in the environment. Apps tend to pull config
| from different sources depending on the deploy - maybe a .env
| locally and a secret store in production.
|
| - Port binding - your app listens on a port and you put nginx
| or whatever in front of it and set up a reverse proxy. K8S
| services and ingress do that.
|
| I think the biggest criticism you can lay down on 12 Factor
| these days is that it's not actually written very well and
| assumes you know exactly what it is talking about.
| talent_deprived wrote:
| > Port binding - your app listens on a port and you put nginx
| or whatever in front of it
|
| Apache for my personal projects, in front of the docker
| containers.
| morelisp wrote:
| It's written very well but the audience is server application
| programmers in 2011, not someone who has written a dozen
| services yet never seen a log rotator, "production mode", or
| an AF_UNIX socket in their whole career.
| rbanffy wrote:
| > - Logs as a stream: yep, just write your logs to STDOUT
| instead of a file. Let the orchestrator deal with reading and
| storing it.
|
| This is one that always feels odd to me: what happened to
| rsyslogd?
| zoogeny wrote:
| Seems good as a general heuristic but bad as a requirements
| checklist.
| lhnz wrote:
| It's pretty bad that, more than a decade after this was released,
| there are detractors for some of the guidelines but yet nobody is
| able to point to a singular more modern set of principles. A lot
| of our advice is now nuanced and spread out all over the
| internet...
| qwerty4343 wrote:
| It's good to see it coming up again. I hope there will be a
| platform among the new PAAS players that is 100% compatible with
| the 12-factor app principles.
| PaulKeeble wrote:
| I have always disagreed most with the config advice. Config ends
| up getting defined by a lot of different parties and quite often
| by developers and its often best to ship the application with
| reasonable defaults and then allow overrides from environment
| specific files and then environment variables. This is the most
| flexible for most server side applications because you often know
| what a setup should be and its best source controlled but secrets
| need to be injected at run time. Configuration needs to have
| cascading levels to make life easiest in dev, test and production
| without enormous setup time.
| Kinrany wrote:
| You can, and probably should, have reasonable defaults.
|
| Configuration should be version controlled, but separately from
| source code because it describes deployments, not images used
| for deployments.
| willseth wrote:
| Why not also express the defaults as config, or is that what
| you meant? I don't think that runs afoul of 12 factor.
| schneems wrote:
| > Configuration needs to have cascading levels to make life
| easiest in dev, test and production without enormous setup
| time.
|
| The backdrop of when the 12factor app was written is that you
| can toggle modes like that with RAILS_ENV=test (an environment
| variable).
|
| One way I think of the config section is "does your config
| strategy play well with containers?" Once you build an image
| you have a static disk state that doesn't persist changes
| unless you make a new image. If your config is file based
| (only), then to toggle between test and production behavior you
| would have to build a whole new image.
|
| By allowing config to change without the underlying disk, it
| helps to isolate changes. Did the app break because something
| in the deploy (image generation) broke? Or is it a bad config?
| If you decouple image generation from configuration changes it
| eliminates that question.
| ahtihn wrote:
| > Once you build an image you have a static disk state that
| doesn't persist changes unless you make a new image. If your
| config is file based (only), then to toggle between test and
| production behavior you would have to build a whole new
| image.
|
| You could mount a volume with the config file. The nice thing
| is that this lets you change the config at runtime (assuming
| the app watches for file changes). Env vars cannot be changed
| at runtime.
| CraigJPerry wrote:
| So long as you are systemically protecting against someone
| spinning up a QA instance against some Prod resource then it's
| probably alright to ship most config with the app. You don't
| want to allow a (entirely predictable) human error to cause
| 100k auth denied retries against a prod job submission queue
| from QA for example. Much config isn't about infra though, it's
| often what kind of ResolverStrategy (made up but plausible
| sounding name) bean do I want to wire in each env for example.
| sethammons wrote:
| > application with reasonable defaults
|
| defaults for dev or defaults for prod :)
|
| If you say dev, you will eventually break prod. And prod
| defaults may not make _any_ sense for dev.
| berkes wrote:
| > And prod defaults may not make _any_ sense for dev
|
| They are actually rather dangerous even. You certainly don't
| want your dev (or test CI) to reset the database if you are
| accidentally connected to prod. Same for email-endpoints,
| CDN, a message-bus, etc.
|
| I've had my share of near-heart-attacks where I send out 10k
| emails or thousands of payments _on actual production
| services_ because of a spagetti config loaders with stuff
| that boils down to `if (getHostname() === "localhost") {
| include("dev.json") } else { include("prod.json") }`.
|
| I'd prefer a hundred .envs and whatnots to avoid just one of
| these scares.
| porsager wrote:
| Would be nice to have a simple term for all these absolutist
| ideas about how to solve a Problem. I nominate the term "The Holy
| Grail Jail".
|
| I've actually had great success doing the opposite to most of
| these points.
| dathinab wrote:
| every time I read Twelve-Factor App I'm like who they set up
| twelve factor authentication, that's crazy isn't 2FA enough
|
| then I remember that it's about something else
| theusus wrote:
| I hate things like these. They just make people architects and
| SEs create forceful indirection that creates nothing but burden.
| bdcravens wrote:
| Perhaps in 2023, but in 2011, we were still at the edge of when
| people would deploy by moving folders and individual files via
| FTP, hard-coding credentials, and dropping in extracted zip
| files as a method of dependency management.
|
| Of course even then added layers were often considered
| "burden", but in general, we need some degree of order lest we
| create an unmanageable mess of index.old.3 and files edited
| directly on production.
| adolph wrote:
| _The 12 factor app guidelines are still relevant today since many
| software teams are building distributed systems, which require
| careful attention to detail as to how services are coupled,
| configured, and deployed. The 12 factor app is a wonderful
| starting point for best practices. In my experience though they
| omit a few facets and leave room for antipatterns in certain
| facets._
|
| https://smallbatches.fm/5/transcript
|
| _[00:03:52] Joe Kuttner: But today the infrastructure that we
| run on is disposable, right? It 's much more like livestock or
| cattle where we can buy new ones when we need one at market and
| we can dispose of, you know, something's not working. Right. We
| get. Yeah. So it's a very, it's a very different model._
|
| _[00:04:07] Joe Kuttner: And that was just emerging when the 12
| factor app was first launched. I think it did a great job of
| helping bring people into that state of mind._
|
| https://smallbatches.fm/20/transcript
| nopurpose wrote:
| I wish there was a generic control plane for generating and
| shipping config to apps. Kinda what Envoy proxy has, but capable
| of shipping arbitrary schema.
|
| Many complications of config management will go away then. Every
| app instances knows just its ID, then everything else (region,
| cluster, stage, current experiment, maintenance, anything else)
| is discovered by "configd" service and config suitable for that
| very instance is returned. Not unlike feature flags, but for
| backend services.
|
| OPA (Open Policy Agent) comes close: it has data sources (so it
| can fetch inventory from cloud providers, kubernetes, any other
| internal system) and Rego language to program config generation,
| but it doesn't have means to notify instances of config updates,
| so they have to resort to polling.
| wly_cdgr wrote:
| "last updated 2017" if you look at the footer
| RyeCombinator wrote:
| Thought we are up to 15 now?
| https://domenicoluciani.com/2021/10/30/15-factor-app.html
| freedomben wrote:
| Over the years I've had a lot of discussions around 12 factor
| apps and encountered a lot of confusion. I find that the
| 12-factor site is great, but mainly for people that already
| understand _why_ those things are important.
|
| For people that don't already understand the _why_ behind the
| rules, it took more in-depth explanation. For that purpose, I
| made the video: "What are 12 Factor Apps and Why Should You
| Care?"[1]. I know of a couple of companies that use that for new-
| hire orientation for engineers/devops that have said it's been
| really helpful.
|
| Regardless where you get the information, it's worth taking an
| hour or two to learn about 12-factor apps. Most of these "rules"
| are just things that take awareness and aren't immediately
| obvious unless you've learned them all the hard way by making the
| mistakes and feeling the pain.
|
| [1] https://youtu.be/REbM4BDeua0
| commandlinefan wrote:
| It's like unit testing. The idea of writing code to test your
| code was so blindingly obvious to me from the time I started
| programming that I was shocked that it took so long for other
| people to pick up on it, but the concept of "unit testing"
| started to gain some traction around the late 90's and I was
| relieved that somebody with some authority was advocating it.
|
| But the reason I didn't just do it myself before Kent Beck
| published JUnit was because the code I worked on didn't lend
| itself well to being controlled by other code. Global variables
| everywhere, widely distributed state, dependence on external
| systems and specific file system layouts and general disregard
| for modularity made it impossible to run anything outside of
| the context it was designed in. All of this was "bad design",
| but it met deadlines, so everybody did it anyway. I was
| _hoping_ that if unit testing gained some traction, programmers
| would get away from monolithic design.
|
| After 25 years of observing unit tests for getters and setters
| and then one big unit test that creates an in-memory database
| because every function in the app needs a live database just to
| run and then gets commented out because it fails, I've lost any
| faith that unit testing will ever be more than a meaningless
| checkmark that everybody fills out because it's a "best
| practice" but nobody actually stops to think why.
| berkes wrote:
| I'd like to believe that "everybody" and "nobody" in your
| story aren't as black-and-white.
|
| Many people see the value of unit testing. And see how TDD
| actually shapes your code to be testable and (accidentally,
| well, not really) therefore also cleaner and better
| organized.
|
| I've always said, and will continue to do so: good,
| maintainable code, is accidentally also very well testable
| code. And vice-versa. And it's not a real accident, if you
| think about it.
| freedomben wrote:
| I've seen similar as well. Aggressive scrum has also made
| this much worse by laying all the incentives at "ship now and
| let the poor sap later who has to work this figure it out. I
| got my 'story points' and shipped so I won't be under the
| metrics microscope." Having a general requirement of "each
| code change should have tests in order to pass code review"
| can definitely help, but it doesn't incentivize _good_ tests,
| and IMHO the only thing worse than no tests are bad tests.
| throwaway4good wrote:
| Holds up well 12 years later.
| lakomen wrote:
| Not this again. Let the horse die already
| commandlinefan wrote:
| This horse or any other horse. I read this and I think "this is
| good advice, we should follow it." You read it and you think
| "this is good advice, we should follow it." And then we both
| do, but our interpretations conflict completely so we end up
| arguing about the true definition of "implicit existence of
| system-wide packages".
| robbywashere_ wrote:
| Fun to compare this to with how insane deployments are now
| featuring kubernetes manifests which only big brains can
| understand
| mplewis wrote:
| What part of the Kubernetes manifest do you think makes
| deployments insane?
| xdennis wrote:
| Not GP, but here's an example I've seen. Kubernetes files are
| generated by Jsonnet. It is invoked from makefiles. Makefiles
| are spread across different git modules. The scripts require
| various Go binaries. Most are installed through a command,
| some have to be installed manually. If the version is wrong,
| mysterious errors happen.
|
| I once had a problem with the generated files so I deleted
| the generated directory and regenerated everything. Turns out
| you can't do that. The generated files are edited, and if you
| generate from scratch you've wiped out the manual edits.
|
| Note that I'm not criticizing Kubernetes here. But when you
| have a non-simple system with some missing aspects, some
| people take the opportunity to make it even more complex.
| pizzaknife wrote:
| makefile cackles as it observes the exchange
| dang wrote:
| Related:
|
| _Ask HN: Is 12factor.net Still Relevant?_ -
| https://news.ycombinator.com/item?id=36283702 - June 2023 (6
| comments)
|
| _12 Factor App Revisited_ -
| https://news.ycombinator.com/item?id=33164407 - Oct 2022 (7
| comments)
|
| _Twelve-factor app anno 2022_ -
| https://news.ycombinator.com/item?id=31225921 - May 2022 (35
| comments)
|
| _The Twelve-Factor App (2011)_ -
| https://news.ycombinator.com/item?id=31198956 - April 2022 (102
| comments)
|
| _Twelve-factor app development on Google Cloud_ -
| https://news.ycombinator.com/item?id=21415488 - Nov 2019 (63
| comments)
|
| _The Twelve-Factor App_ -
| https://news.ycombinator.com/item?id=19947507 - May 2019 (3
| comments)
|
| _12 Factor CLI Apps_ -
| https://news.ycombinator.com/item?id=18172689 - Oct 2018 (247
| comments)
|
| _12 factor app configuration vs. leaking environment variables
| (2014)_ - https://news.ycombinator.com/item?id=15869436 - Dec
| 2017 (2 comments)
|
| _Ask HN: Alternative to Heroku that doesn 't enforce 12-factor_
| - https://news.ycombinator.com/item?id=10628961 - Nov 2015 (1
| comment)
|
| _The Twelve-Factor App_ -
| https://news.ycombinator.com/item?id=10288216 - Sept 2015 (3
| comments)
|
| _The Twelve-Factor App_ -
| https://news.ycombinator.com/item?id=9492120 - May 2015 (2
| comments)
|
| _Twelve-Factor Applications with Consul_ -
| https://news.ycombinator.com/item?id=7780249 - May 2014 (2
| comments)
|
| _The Twelve-Factor App_ -
| https://news.ycombinator.com/item?id=7547687 - April 2014 (1
| comment)
|
| _Building Twelve Factor Apps on Heroku_ -
| https://news.ycombinator.com/item?id=6219444 - Aug 2013 (1
| comment)
|
| _12 Factor model for architecting SaaS applications_ -
| https://news.ycombinator.com/item?id=6060381 - July 2013 (1
| comment)
|
| _The Twelve-Factor App_ -
| https://news.ycombinator.com/item?id=5979452 - July 2013 (1
| comment)
|
| _12factor: Methodology for Building Software-as-a-Service Apps_
| - https://news.ycombinator.com/item?id=4027026 - May 2012 (1
| comment)
|
| _Twelve Factors of Web Application Development_ -
| https://news.ycombinator.com/item?id=3267187 - Nov 2011 (37
| comments)
___________________________________________________________________
(page generated 2023-10-12 21:00 UTC)