[HN Gopher] Amazon Redshift Re-Invented ___________________________________________________________________ Amazon Redshift Re-Invented Author : belter Score : 47 points Date : 2022-05-20 15:32 UTC (3 days ago) (HTM) web link (www.amazon.science) (TXT) w3m dump (www.amazon.science) | belter wrote: | PDF: | https://assets.amazon.science/93/e0/a347021a4c6fbbccd5a05658... | mountainriver wrote: | Amazon is losing their edge, the toxic company culture is pushing | away good engineers and we are seeing more and more 3rd party | companies begin to dominate over spaces amazon previously did | e.g. snowflake | | I welcome this | coderintherye wrote: | Does anyone choose Redshift these days, asides from those | completely tied into AWS? Feels like Redshift is playing catch up | but moving so slow it will never catch up. | | As another commenter noted, Redshift in my experience was an | operational hassle. | | Snowflake and BigQuery just work. | | Why choose Redshift at this point? | websap wrote: | How does this compare to offerings from Snowflake and Databricks? | oxfordmale wrote: | In the last two years I have used both Redshift (AQUA) and | Snowflake, and I prefer Snowflake by a mile. Snowflake is just | a lot easier to use, scales better and has a much better | permission model. | yveezy wrote: | I've worked with Redshift for about 5-ish years and BigQuery for | about a year. IMO BigQuery wins hands down. | | From an operational perspective we've had almost 0 issues with | BQ, whereas with Redshift we had to constantly keep giving it | TLC. Right from creating users /schemas to WLM tuning, | structuring files as Parquet for Spectrum access, understanding | why and how spectrum performs in different scenarios, etc. | everything was a chore. All this redshift specific specialization | I was learning was not really contributing to the product in a | meaningful way. | | Switched to Bq, a year ago, and it's been mostly self driving. | the only thing we had to tend to was the slots and bit of a | learning curve for the org about partitioning keys (there is a | setting in BQ that fails your query IF partition key is not | specified) | | Having switched to BQ it's really hard for me to imagine going | back to Redshift. It almost feels antiquated. | dehrmann wrote: | I briefly worked on Redshift and have used Athena/Presto and | Bigquery. Redshift felt like an architectural middle ground. | Presto can query almost anything, and Bigquery requires storing | data in Bigquery, but like Presto, you don't have to pay for | inactive compute use. Redshift's scaling story is more | complicated, and paying for inactive compute wasn't ideal. It | sounds like it might have improved, but you're still | essentially building Bigquery at that point. There might be | some use cases that need a fast, columnar store that's already | online, so queries take 3s, not 10s with Bigquery. | | I generally prefer Bigquery, and between it and Bigtable, I | actually prefer GCP over AWS because their offerings for hard- | to-do things are really good. I'd honestly pick GCP just for | those two products. | beckingz wrote: | Bigquery alone is the reason I recommend GCP. | | That said, both AWS and GCP have some real rough edges. | contol-m wrote: | I work on BigQuery. All of these are great points: just | wanted to point out that BigQuery can federate into external | data sources as well: e.g. files on cloud storage and | BigTable. Relevant feature is BigLake: | https://cloud.google.com/bigquery/docs/biglake-intro | TuringNYC wrote: | Our most shocking discovery on Redshift was that primary key | constraints are not honored. (Not sure why they even have PK | identifiers given this, it just adds more confusion.) | halotrope wrote: | I am glad there is some innovation happening. If you compare | redshift to BigQuery it feels quite archaic in the current | production version. I understand that Google commands a metric | ton of engineering excellence, yet the difference should not be | so extreme especially when you consider AWS still being the king | of cloud. BigQuery is downright magic. | | I would love to see more competition in this space as having | large amounts of data with Google always makes me feel uneasy for | all kinds of reasons. | advisedwang wrote: | Sounds like BigQuery! ___________________________________________________________________ (page generated 2022-05-23 23:00 UTC)