[HN Gopher] Release of AWS CDKv2 and Construct Hub
___________________________________________________________________
Release of AWS CDKv2 and Construct Hub
Author : cgarvis
Score : 60 points
Date : 2021-12-02 18:03 UTC (4 hours ago)
(HTM) web link (aws.amazon.com)
(TXT) w3m dump (aws.amazon.com)
| qaq wrote:
| CDK is convenient but CloudFormation is horribly slow.
| eranation wrote:
| There is also CDK for Terraform now actually. Hope they'll keep
| it up to date.
| qaq wrote:
| and for k8s
| oneplane wrote:
| This is great if you only target AWS and only have a small scope,
| but for anything else it sucks (just like CloudFormation itself).
|
| This is one the main problems with most of the CDK-abstracted
| SDKs for clouds in general where you're essentially just going to
| re-implement Terraform or SaltStack or Ansible but with your own
| code that doesn't have the same portability in technical and
| human terms.
|
| That knowledge about the in-house system is useless elsewhere,
| and anyone coming in from the outside can't use any pre-existing
| knowledge. This is of course only a problem in larger scopes, say
| a larger company with an internal team that does the Ops-leaning
| side of DevOps.
|
| A company that is larger might simply delegate an entire set of
| accounts and infrastructure to individual teams where they have
| to sort everything out themselves, and a company that is smaller
| is essentially the same as a small division in a large company.
|
| And then you still have the problem if the glue between your AWS
| cloud, Google cloud, Cloudflare and whatever Git provider you
| use. No CDK covers that the way something like Terraform with
| delegation to providers does where you have a standard data
| format where you can transport information between providers. If
| you want to create a repo in GitHub, preset some configuration
| and contents, add that repo to a CD solution that you run on
| Kubernetes on EKS in AWS with delegated accounts per EKS workload
| and then connect Cloudflare to ingress ALBs, that's at least 4
| different APIs you're talking to with incompatible interfaces.
| Most of them have CDK's so your interface becomes your own
| implementation that you now have to maintain. Delegating that to
| a specialised tool works much better.
| habitue wrote:
| I just started using CDK for a project, and after having fought
| with terraform and trying pulumi, I have to say CDK is a breath
| of fresh air. Really well thought out, makes doing things the
| right way easy as well.
|
| The biggest difference I can see between CDK and pulumi (other
| than CDK only being for AWS) is that the CDK is more opinionated.
| When you spawn a new database, it'll automatically create a
| secret in secretsmanager, and set up rotation etc. And since it
| can assume IAM, it generates granular policies for you easily
| with calls like `dbInstance.grantRead(lambdaInstance)` etc,
| instead of you having to manually construct a JSON policy.
|
| I really think the pulumi / CDK method of "Use a real programming
| language to generate a declarative spec" is the right way to go.
|
| For those keeping score:
|
| - chef/puppet: imperative language, imperative effects
|
| - ansible: declarative language, imperative effects
|
| - terraform: declarative language, declarative effects
|
| - CDK/Pulumi: imperative language, declarative effects
|
| Not to mention, CloudFormation actually allows ~transactions,
| which is something you can't really get without cooperation from
| the cloud provider
|
| Edit: I incorrectly mentioned that terraform uses cloudformation
| to get transactions, but it does not
| fitzoh wrote:
| >The biggest difference I can see between CDK and pulumi (other
| than CDK only being for AWS) is that the CDK is more
| opinionated. When you spawn a new database, it'll automatically
| create a secret in secretsmanager, and set up rotation etc. And
| since it can assume IAM, it generates granular policies for you
| easily with calls like `dbInstance.grantRead(lambdaInstance)`
| etc, instead of you having to manually construct a JSON policy.
|
| Hopefully this will get better soonish on the pulumi side.
| [awsx](https://github.com/pulumi/pulumi-awsx) has existed for a
| while which is sort of takes the CFN higher level construct
| approach, but it's currently typescript only.
|
| They just finished some foundational work to enable multi-
| language components, and I expect we'll see some
| opinionated/higher level components from them for all languages
| in the next 6 months or so.
| throwaway894345 wrote:
| > Not to mention, CloudFormation actually allows ~transactions,
| which is something you can't really get without cooperation
| from the cloud provider (terraform and pulumi both use
| cloudformation on AWS for exactly this reason)
|
| Terraform _doesn 't_ use CloudFormation on AWS (and I thought
| Pulumi used Terraform under the covers in some capacity?). I've
| also seen a lot of CloudFormation stacks get into completely
| unrecoverable states because AWS was trying to roll back a
| transaction, but the rollback failed. If you have a premium
| support contract, someone can un-stick it for you, but for the
| rest of us we just had to create a new stack. I've been off AWS
| for a year and change, so maybe this has improved?
|
| In whichever case, I've only dabbled with CDK, but I was
| disappointed. What I really want is a better
| Troposphere[0]--sort of an AST library for CloudFormation,
| ideally type-safe. I don't care that the backend is
| CloudFormation in particular, but the idea is that we should
| have a clean separation between the backend diff engine and the
| abstraction layer that humans use to DRY the input to the
| backend diff engine.
|
| [0]: https://github.com/cloudtools/troposphere
| fitzoh wrote:
| >Not to mention, CloudFormation actually allows ~transactions,
| which is something you can't really get without cooperation
| from the cloud provider (terraform and pulumi both use
| cloudformation on AWS for exactly this reason)
|
| Terraform and Pulumi use Cloudformation?
| dastbe wrote:
| aws recently launched cloud control, which more or less
| decouples the individual resource management aspect of
| cloudformation from everything else. providers like
| terraform/pulumi can use it in lieu of manual integrations
| with every aws api.
| habitue wrote:
| Sorry, I was mistaken about this. I was under the impression
| terraform would create transactions with cloudformation, but
| it only does this if you manually create cloudformation
| resources.
| fdgsdfogijq wrote:
| Honestly, always surprised I don't see more about CDK. As far as
| I am concerned, this is the future of software development.
| Object oriented infrastructure, where you create abstractions
| over pieces of infrastructure, put them into a library, and then
| use them as composable components is incredibly powerful. We are
| just scratching the surface, and through libraries like
| constructs, large swaths of development will be automated.
|
| Again, this is incredibly powerful.
|
| For those that don't know, cloudformation is now often thought as
| the assembly language of cloud development, with CDK the higher
| level language.
|
| I work at Amazon, and here we have a growing library of internal
| CDK constructs that make creating internally facing
| infrastructure, that works with other infra, incredibly easy.
| Even the databases that other teams have, their queues, etc, can
| be vended as common infrastructure packages, and then consumed,
| attaching your own AWS resources to theirs via library imports.
| walls wrote:
| It would be even better if most of your new features were
| actually supported by CloudFormation.
| timclark wrote:
| I wish I could upvote this more - if I can't do it in
| CloudFormation I won't use it.
| taeric wrote:
| I guess the main question I have is what makes this different
| from any other abstraction? Specifically, is this that much
| better than a ton of vm orchestration that has existed for a
| while?
|
| I'm also concerned about it throwing out the lines of
| infrastructure lifecycle versus software lifecycle. Runs
| heavily afoul of the common mono repo criticism that, just
| because you can commit the infra update with the software, does
| not mean they magically deploy together.
| fdgsdfogijq wrote:
| "Runs heavily afoul of the common mono repo criticism that,
| just because you can commit the infra update with the
| software, does not mean they magically deploy together."
|
| Yeah funny you mention this, as the idea of vending infra
| resources as a common package runs into the issue of how to
| know if every consumer will be able to successfully ingest
| updates. From there you have the issue of "blast radius", how
| to handle rollbacks, etc. The complexity goes up, and now you
| have a problem. I think the answer is very sophisticated
| build systems, as well as CI/CD pipelines, which Amazon has.
| Upfront the team pushing changes can know if the deployment
| will work, before it gets to production. But this requires a
| huge amount of tooling, which I don't think most companies
| have. It also requires alot of cloud expertise.
|
| "I guess the main question I have is what makes this
| different from any other abstraction?"
|
| For one its all unit testable, with type completion (CDK is
| generally done in Typescript). The code has to compile, you
| can diff the new infra with the current infra and see whats
| changing. You have version control. You can use other design
| patterns from other teams, much like you would for OOP code,
| except now its infrastructure. A queue is just a variable in
| a CDK code base. It demotes infrastructure from a complex
| thing to manage, into something like a code variable.
| taeric wrote:
| The amount of tooling needed is impressive.
|
| But my concern was more in the teams that try and get it
| all into one code source and pipeline.
|
| Don't get me wrong. One team having to deal with dozens of
| deployment orchestrations is a straw man just as much as
| the complication that would be forced into a single
| pipeline. I would love finding that sweet spot without too
| much swinging between the fires.
| manojlds wrote:
| Why should one use CDK vs Pulumi?
| shepherdjerred wrote:
| In my experience CDK has much nicer documentation. It also has
| a lot of high-level constructs, e.g. I can very easily say give
| me an EC2 instance on the public internet, whereas with pulumi
| you have to set up all of the security groups, vpc, internet
| gateway, etc.
| fitzoh wrote:
| Pulumi's biggest benefit is if you want to manage resources
| outside of AWS. There are providers for Azure, GCP, GitHub,
| Postgres, Datadog, etc.
|
| It's also nice if you want to be able to manage resources in
| multiple AWS regions or accounts from a single stack.
| cconstantine wrote:
| I have a few years of experience with terraform (pulumi is
| built on terraform), and moved to a place using CDK about a
| year ago. From what I've seen, CDK is worse than terraform, and
| I would expect it to be strictly worse than pulumi. I haven't
| seen a problem that CDK solves better than terraform, and it
| fails pretty hard at the core of its job. I genuinely want to
| find something to like (because I'm stuck with it), but I'm
| really struggling.
|
| CDK is bad at the core of its job; managing the state of
| resources in AWS. The two big ways this manifests with CDK are
| that you can't completely import existing resources into CDK,
| and CDK is bad at detecting changes made out-of-band.
|
| CDK has a way to "import" resources, but unlike terraform, it's
| not something you do once then forget about. When you "import"
| a resource in CDK you are doing that as an alternative way to
| define the resource, so the code will always need to 'remember'
| which resources to create and which ones to import. Imported
| resources can be modified, but there are restrictions on what
| modifications can be made so they aren't really interchangeable
| with native cdk built resources. Our CDK stacks are littered
| with "if STAGE" blocks that will never go away until we
| recreate those resources with CDK, and there is no clean way to
| do that.
|
| In general I would say making changes to CDK/terraform created
| resources out-of-band (ie, in the aws console) is a bad idea,
| but it happens sometimes. Maybe you aren't exactly sure how a
| parameter maps to a feature in the aws console, or there is an
| active outage and you need to make the change _now_ , or the
| guy in IT that owns the root account decided to "fix" something
| for you. Whatever the cause, it's important to be able to run
| your IaC tool and 1) see the changes, 2) remediate them. CDK
| can detect some changes but not everything, and that's left me
| with very little confidence that things are actually setup
| correctly. No one's complaining, so I guess it's fine?
|
| There are a few other quality-of-life things I don't like about
| CDK, but I don't think they are as important. It's slow. I
| don't like the diff output format. We always see diffs and
| different devs see different diffs, and we have no way to
| debug/resolve it. I'm boggled that a bunch of resource types
| (it's not clear which ones) default to not being deleted when
| you delete them from cdk. Overall, I get the feeling that "it
| works" and "no one's complaining" is the quality bar CDK is
| trying to meet, and that just isn't good enough for me.
|
| The only reason I don't say terraform is strictly better than
| CDK is that terraform doesn't allow you to define your
| infrastructure with a 'real' programming language. I don't
| think that's an important feature, and it might tempt you into
| doing horrible things (like littering your codebase with "if
| STAGE" blocks). In general I think keeping infrastructure stuff
| simple and separate from application stuff is important, and
| HCL really encourages that. I've been bothered by the lack of
| programable logic in terraform, but honestly, I think that
| constraint has ultimately resulted in cleaner logic. If it's
| important to you, then pulumi is always there.
|
| Ugh, I took way to long writing this. To answer your question,
| IMHO, don't use CDK.
| jaxxstorm wrote:
| Pulumi isn't "built on terraform", that's a common
| misconception. See here:
| https://www.leebriggs.co.uk/blog/2021/11/06/pulumi-faqs.html
| cconstantine wrote:
| Thanks for the correction! I've never used Pulumi directly,
| and thought it was just a wrapper around terraform
| providers.
|
| Do you think it changes anything about how I've
| characterized it above?
| cyberpunk wrote:
| I'm pretty happy to see this. I just inherited a bunch of
| projects that all have entire tf stacks inside them, and I'm
| itching to just replace it all with some actual code.
|
| Terraform was great, once, but over time as general cloud
| complexity got above a handful of some ec2 machines and some
| networking rules, it becomes a real burden to manage. Now all our
| devs are struggling to manage kenesis or ecs/fargate stuff with
| terraform. HCL is _so close_ yet so far from an actual language
| that it 's infuriating to use as a developer.
|
| I think I'll bite the bullet and go all in on this, since the
| rest of our codebase is all ts anyway. Pulumi would be the other
| option, but at this point we're so sucked in to aws anyway, the
| only reason to use that over this (not locked to aws) is moot for
| us.
| avl999 wrote:
| CDK is absolutely brilliant. My eyes glaze over when I see lines
| and lines and pages and pages of CloudFormation/Terraform
| config... sometimes spread over multiple files and having to
| manually join them in them head to get the full view of the
| infrastructure.
|
| But CDK reads so beautifully and gets rid of so much noise in
| these templates that I don't care about. Unfortunately... the
| devops people and those who hold they keys to cloud resources at
| my current company (and many other companies) are so all in on
| Terraform that most won't even consider CDK/Pulumi as an option
| despite the CDK/Pulumi paradigm being objectively better than
| CloudFormation/Terrform paradigm.
| [deleted]
___________________________________________________________________
(page generated 2021-12-02 23:01 UTC)