[HN Gopher] A Month of Terraform
___________________________________________________________________
A Month of Terraform
Author : ingve
Score : 26 points
Date : 2020-11-22 19:39 UTC (3 hours ago)
(HTM) web link (jeremywsherman.com)
(TXT) w3m dump (jeremywsherman.com)
| bird_monster wrote:
| If your only comparison is "I used to work with Heroku",
| Terraform might seem not great. I would argue this is a blub
| example, in that this person's lens does not contain enough
| context or experience to be able to assess value.
|
| Terraform's value becomes clear when moving in scenarios like
| moving from CloudFormation to Terraform, or when trying to
| integrate a second cloud's resources into your infrastructure
| workflow. Without experience in anything other than Heroku, of
| course a tool designed to do many many things is going to seem
| complex and at times frustrating.
|
| > Iteration times are way longer than with even mobile apps.
| Like, "you're liable to task-switch while waiting to see plan
| output" longer.
|
| I'm over here laughing in my "over an hour of CloudFormation only
| to have a run fail, and then a rollback fail, and have to contact
| AWS support" life.
| wgjordan wrote:
| > I'm over here laughing in my "over an hour of CloudFormation
| only to have a run fail, and then a rollback fail, and have to
| contact AWS support" life.
|
| Your experience must be over four years out of date, because
| CloudFormation has supported the ability to continue updating a
| rollback since February 2016 [1].
|
| [1] https://aws.amazon.com/blogs/devops/continue-rolling-back-
| an...
| bird_monster wrote:
| My experience is from 2019, friend.
| wgjordan wrote:
| Needing to contact support for this issue in 2019 only
| means that the user didn't know how to use the 'continue
| update rollback' feature properly, which was a feature
| added back in 2016 specifically to support the rollback-
| failed scenarios.
| bird_monster wrote:
| It's weird the frequency with which the AWS Support Reps
| agree the issues we find aren't those that would've been
| resolvable on our own, as each time we file a ticket we
| follow up with our reps for appropriate training and
| support.
|
| I understand that you like this tool, but you do not need
| to fight everyone suggesting it doesn't work as well as
| you want it to.
| pqb wrote:
| Yes, it is true. However, I am hesitant to say the AWS
| CloudFormation is a bulletproof tool that does not need help
| from the support side. If somebody was too impatient and
| manually removed a resource that was managed by CFN, then he
| might end with a deadlock, where only help was actually a
| support team member. Been there, don't repeat that error -
| please do not create (not auto-detectable) drift changes on a
| purpose (it could be hard to modify/remove the stack in
| future).
|
| Edit: My memories are from Q1 2020.
| pm90 wrote:
| > I'm over here laughing in my "over an hour of CloudFormation
| only to have a run fail, and then a rollback fail, and have to
| contact AWS support" life.
|
| Heh, same. Any of the cloud-provided orchestration tools
| (Cloudformation, Openstack Heat etc.) are great only for the
| most basic tasks; using them to provision complex
| infrastructure is just begging for a world of hurt.
|
| That being said. I think terraform could do better. I use
| terraform a lot, and yet I agree with the authors complaints
| that the syntax can be super confusing, is not documented very
| well, and the providers have their own idiosyncrasies. The last
| one isn't strictly an issue with terraform, its with the
| provider implementation. But... if terraform aims to be the
| Swiss army knife of infrastructure provisioning, I think
| criticism that its hard to standardize even within the same
| provider is fair.
| wgjordan wrote:
| > Heh, same. Any of the cloud-provided orchestration tools
| (Cloudformation, Openstack Heat etc.) are great only for the
| most basic tasks; using them to provision complex
| infrastructure is just begging for a world of hurt.
|
| I disagree- I can't speak to OpenStack Heat (and I have _no_
| idea what you 're referring to by 'cloud-provided
| orchestration tools' beyond these two specifically), but my
| own experience using CloudFormation to provision complex
| infrastructure is that it is in fact great for _all but_ the
| most basic tasks (where any orchestration tool would just add
| unnecessary overhead).
| pqb wrote:
| > I'm over here laughing in my "over an hour of CloudFormation
| only to have a run fail, and then a rollback fail, and have to
| contact AWS support" life.
|
| Oh, I see I am not alone in having the same experience with the
| AWS CloudFormation (CF) update flow. I have started feeling
| "good enough" in CF after about 6-8 months to write templates
| from my memory (accompanied with a documentation page). The
| funny thing about the CF is fact, the all people I know had
| working flow based on searching GitHub/Gist over an example,
| edit, refine, deploy, push fixes, deploy, rinse & repeat.
|
| In the blogpost I have noticed mentioned Pulumi. To be honest,
| I have not used it yet, but I have tried AWS CDK [0]. I suppose
| it is AWS's go-to solution for everybody who wants to use the
| CloudFormation for saving deployment stages and also have
| programmed templates in languages of their choice (like a
| TypeScript or Python). It is interesting solution that I
| suggest investigating if you haven't done already. It supports
| "importing" CF templates as-is, so they can be incrementally
| translated to CDK.
|
| [0]: https://github.com/aws/aws-cdk
| pqb wrote:
| > terraform-lsp is supposed to provide autocomplete, but it
| mostly doesn't, in my experience.
|
| I find Terraform for Jetbrains IDEs to work hundred times better
| than the VSCode one. I really can recommend it, does magic to
| autocompletion, peek definition, snippets insertion and linting
| TF/HCL files.
|
| Author of this blogpost have not mentioned Terragrunt [0], while
| I think it is worth mentioning. It is nice tool, especially if
| you work on bigger projects having multiple modules and per-
| environment variables.
|
| Also, a tip from a person, who works in team using terraform -
| use brew/apt repositories to keep the binaries up-to-date or at
| least on the same version as your team mates. I remember at least
| few situations, when patch update of a terraform binary was
| crucial to make some issues disappear.
|
| [0]: https://terragrunt.gruntwork.io/
|
| Psst... I am also using Terraform to bootstrap Pop_OS! with my
| dotfiles and it is surprisingly nice enough to mimic declarative
| and atomic NixOS configuration :)
| ciceryadam wrote:
| For distributed work on environments you ideally you want to
| have something like a gitlab pipeline that does the tf plan for
| you automatically, with a manual approve button of sorts. That
| way you a consistent tf version defined in .gitlab-ci.yml, plus
| due to centralized logging it's easier to find when and what
| changed in the envs.
| pqb wrote:
| I assume you are talking about a similar tool to the Atlantis
| [0] / Terraform SaaS offering [1].
|
| [0]: https://www.runatlantis.io/
|
| [1}: https://www.hashicorp.com/blog/terraform-collaboration-
| for-e...
| joobus wrote:
| But he did mention Terragrunt:
|
| > Terraform: Up & Running is excellent, and Terragrunt makes it
| even easier. Huge thanks to their team for providing the duct
| tape we need.
| pqb wrote:
| Ouch, what a shame on my side. Sorry, I must have missed it.
| Thanks for pointing it out.
| arkadiyt wrote:
| Some secrets (like IAM access keys) can be encrypted with a
| public PGP key.
|
| Some other secrets (like RDS instance master passwords) can't be
| encrypted, but I like to use a trick where you have a local
| provisioner that runs after the instance gets created, which
| updates the just created DB and sets the password to a random
| value and prints it to stdout so you can save it in your secrets
| management tool of choice. The value saved in the terraform state
| is no longer valid.
|
| Some other other secrets, like elasticache auth tokens - there's
| no solution, they'll always be available as plaintext in the
| statefile.
|
| Overall agreed that Terraform needs to do better here.
| gtsteve wrote:
| I have a lambda that runs every day to cycle the RDS master
| password. I create the password using the random provider, and
| save it to a secrets manager secret - the first time the
| environment is created, the secret is in plaintext in the
| state, but it will not be valid the next time the lambda runs
| (less than 24 hours).
|
| You can do the same with ElastiCache auth tokens, and ensure
| your application reads the token from a value in secrets
| manager.
___________________________________________________________________
(page generated 2020-11-22 23:00 UTC)