[HN Gopher] AWS Babelfish: The Elephant in the PostgreSQL Room?
___________________________________________________________________
AWS Babelfish: The Elephant in the PostgreSQL Room?
Author : ahachete
Score : 101 points
Date : 2021-02-12 14:58 UTC (8 hours ago)
(HTM) web link (postgresql.fund)
(TXT) w3m dump (postgresql.fund)
| api wrote:
| More freedom as long as you run on AWS. Or did they release this?
| severino wrote:
| They plan to release it under the Apache License this year,
| according to:
|
| https://babelfish-for-postgresql.github.io/babelfish-for-pos...
| jjjeii3 wrote:
| I worked with SQL Server, Oracle, MySQL and many other DBMSes.
| Nothing can beat PostgreSQL.
|
| Oracle's SQL Developer is the worst IDE for databases I ever
| seen. Oracle DB doesn't support "TOP N" or "LIMIT N" notations
| and many other useful features...
|
| I like SQL Server more than Oracle and Microsoft provides 2 good
| IDEs for it (SQL Management Studio and Azure Data Studio). But it
| has some bad limitations - you can't define ON ... CASCADE on
| more than one path. You also can't create CTEs with
| inserts/updates.
|
| PostgreSQL doesn't have a good IDE out of the box. I heard that
| Azure Data Studio now support it, but didn't try it yet, because
| I already paid for a commercial IDE that supports PostgreSQL.
| PostgreSQL also doesn't have a good high availability tooling,
| but all major cloud providers do this for you. Considering that
| you get it for free, it is a much better choice than Oracle or
| SQL Server.
| NDizzle wrote:
| Devart makes good SQL IDEs. They came out with one for Postgres
| ... recently? In the last year or two. If it's similar quality
| to their mysql ide, I'd recommend it.
|
| https://www.devart.com/dbforge/postgresql/studio/
| bdcravens wrote:
| Working with the big 3 on a regular basis, DataGrip is the best
| tool I've found. (It lacks some management features, but it's
| been years since I needed those)
| katnegermis wrote:
| I really love using dbeaver[1] as a Postgres IDE. It requires a
| small bit of configuration for me to use it (why are the
| defaults to automatically lowercase all of my SQL?!). Other
| than that it's excellent!
|
| [1]: https://dbeaver.io/
| Quarrelsome wrote:
| because fuck the shift key. RSI sufferers unite!
|
| but in all seriousness, I don't know.
| gonational wrote:
| Embrace, extend, extinguish, Amazon edition.
|
| In my opinion, this is the inevitable conclusion of Amazon's
| sudden increase in direct involvement in OSS projects. With cases
| like elastic search close in the rearview mirror (I know that
| there were reasons for this, but it was a start), It's hard not
| to come to this conclusion.
|
| Once the Amazon tentacles have reached down into every major
| staple of the OSS economy, it's difficult to predict the end
| result, but one thing is for certain, it will benefit Amazon more
| than it benefits anyone else, lest Amazon breaks its promise to
| its shareholders.
|
| Amazon has demonstrated a pattern of controversial vertical
| integration, by using internal analytics to determine which of
| their own merchants' products they should undermine. To believe
| they would not do the same thing in their largest and most
| important division would require tragic ignorance.
|
| This is all just my opinion
| _msw_ wrote:
| Disclosure: I work for Amazon, where I build AWS
| infrastructure. But these are my own opinions and experiences
| at work, and I am not speaking for my employer.
|
| "Embrace, extend, extinguish" has never been uttered in any
| conversation I've been part of at Amazon. It simply is not part
| of the philosophy, or culture. We obsess about customers, and
| customers do _not_ like the things that they depend on, like
| open source software that they 've built their business on, to
| be damaged.
|
| Rather, they want to do more with open source software all the
| time. And many want to find ways to get off of software that is
| constraining them, or causing unplanned stress or surprises
| when their costs suddenly change due to arbitrary license
| changes.
|
| The promise here is to customers, and customers tell us that
| community-led open source software development is better for
| them. And this is no surprise. As a software developer, and a
| long time FOSS advocate, I believe it too.
|
| By listening attentively to customers, I believe we will
| continue to build a business that we, and our shareholders,
| will be proud to be a part of, and will provide long term
| growth.
| gonational wrote:
| With no disrespect to you, this comes across like a pre-
| canned response from the Amazon Anti-Anti-Trust legal team...
|
| I mean, your second and third paragraph could be the
| introduction to an internal presentation about "why we at
| Amazon should embrace, extend, and extinguish OSS". The
| things you listed are precisely why Amazon would want to
| destroy OSS, Because OSS is a threat to anyone offering
| proprietary software, and anyone who has built their empire
| on OSS will inevitably need to shift to proprietary software
| to retain relevance in a world where OSS software "eats the
| world" by, for instance, commoditizing the infrastructure
| layer...
|
| In other words, just in time for AWS to become less relevant
| due to the enormous gains in open source infrastructure as a
| service tools, Amazon will swoop in and help to destroy the
| most useful parts of the open source economy.
|
| It's easy to look at this with a naive optimism and think
| "but these are the exact kinds of things that helped AWS grow
| to begin with", To which the obvious reply is "yes, at the
| bottom end, but Amazon has been focused on enterprise more
| and more over the past five years, and licensing agreements
| with other big tech companies have changed the offering and
| the prospects of future offerings". All you have to do is
| look at YouTube to see another example of this. YouTube built
| its user base on small contributors, but as soon as it
| reached a plateau it began to focus on the old guard, main
| stream media, Hollywood, etc., unfortunately for the little
| guys. This is the same pattern every company takes once they
| get big enough to play competitively with the big boys.
|
| This is all, of course, only my opinion.
| _msw_ wrote:
| I take it as no disrespect. I can understand why you would
| have this point of view. :-)
| gonational wrote:
| If you understand _why_ I have this viewpoint, then have
| I convinced you to have the same viewpoint? Or, have I
| given away some thing about my internal reasoning
| mechanics to suggest that I would have erroneously come
| to my conclusion, perhaps based largely on blind
| cynicism?
| _msw_ wrote:
| Personally, I try to keep a "curious" outlook, and to
| regularly try to disprove my beliefs. To me, one way to
| do this is to listening closely to criticism from
| viewpoints and perspectives that are different than mine.
|
| Based on my own experiences, including engaging in very
| helpful dialog with critics so far, I am not convinced
| that the same kind of anti-FOSS campaigns that we saw,
| with ample public evidence, from actors like Microsoft
| and SCO of the past, would be in the interest of Amazon
| or its customers. It would be the opposite.
|
| But I think it is right, and it should be expected, to
| continue to scrutinize Amazon, like any other large
| organization. I try to judge based on actions and
| evidence, rather than what-if speculation.
| johncena33 wrote:
| > "Embrace, extend, extinguish" has never been uttered in any
| conversation I've been part of at Amazon.
|
| We are not talking about internal corporate PR. Enron also
| had great set of values [1] that were preached to the
| employees. It's about the actions, not the words uttered.
|
| [1] https://hbr.org/2002/07/make-your-values-mean-something
| _msw_ wrote:
| I am not reciting corporate PR. As I said, I'm not speaking
| for AWS. I only speak for myself, and I'm not saying
| anything that I don't believe.
|
| You'll find elsewhere in the comments [1] that on actions,
| not words, I agree with you. "I try to judge based on
| actions and evidence, rather than what-if speculation."
|
| [1] https://news.ycombinator.com/item?id=26116542
| linuxhiker wrote:
| Except that they haven't extinguished anything.
|
| They certainly embrace and extend which is the very nature and
| purpose of Open Source software.
| sradman wrote:
| The elephant in the room is that AWS Babelfish could be a viable
| alternative to PostgreSQL. Babelfish is currently a fork of
| PostgreSQL that is compatible with Microsoft SQL Server's TDS
| wire protocol and T-SQL language. The article proposes adding
| hooks for wire protocol extensions eliminating the need for
| separate forks. Adding new languages is already supported (EDIT:
| incorrect, see mattashii's comment).
| ahachete wrote:
| OP.
|
| One of my fears. That unless we, as the PostgreSQL Community,
| do something about it, and embrace the efforts that AWS started
| doing and I hope will continue, it would effectively end up
| being a fork. And a fork with a collaboration on GitHub, which
| many would prefer to contribute to rather than current's
| Postgres contribution mechanism.
|
| And if that's the case, I wonder if it could become the
| MariaDB, or the LibreOffice, of Postgres. I hope we will be
| able to do the best thing and have the contribution that
| Postgres is receiving integrated and merged into Postgres.
| _msw_ wrote:
| Disclosure: I work for Amazon where I build AWS
| infrastructure, but these opinions are mine alone.
|
| Personally, I think that the word "fork" has a lot of
| historical meaning in the world of Free and Open Source
| Software. To me, "fork" means that there is a "vote of no
| confidence" in the direction of the people who are making
| decisions about open source software. That clearly is not the
| case here. The PostgreSQL core team has a very good
| reputation of being collaborative, and thoughtful, in their
| work to make PostgreSQL the best database it can be for the
| community at large, and not build it to benefit any
| particular commercial interest.
|
| Often software development happens on branches. These are
| experiments, not forks. Over time, good ideas are propagated,
| and grafted, between the branches. These are not "forks" to
| me.
| ahachete wrote:
| Indeed.
|
| I see this as a very open and collaborative approach. To
| announce such a big contribution to Postgres, and be it
| open sourced when published, is very welcome, and I believe
| one of the boosts, as I mentioned in the post, that
| Postgres needs.
|
| I'm also convinced that Babelfish and PostgreSQL will be
| properly integrated, and this will be a great win. I
| consider it too a development branch more than a fork,
| which certainly has "no confidence" connotations. But
| there's still some worry that it would become one, one day,
| if there's no sufficient effort towards its integration
| --as the conversation that I wanted to spark here was
| almost non-existent up until recently.
|
| This actually brings the topic of more explicitly branch
| development in Postgres, but it's a different story ;)
| linuxhiker wrote:
| This indeed is the concern. I don't think most people are
| going to care if it is PostgreSQL.Org PostgreSQL, or
| Babelfish for PostgreSQL in the long run. What they will care
| about is modernity of the development process and who has the
| "cool" features.
|
| When you look at something that has nothing to do with the
| Babelfish functionality but would absolutely help PostgreSQL,
| consider PHOT. PHOT (Partial HOT) is a great opportunity for
| PostgreSQL that Amazon is trying to give back. However, I
| wouldn't be surprised if the .Org community were to not
| accept the patch, the feature would end up in Babelfish. If
| that happens, we now have a viable fork which has 100%
| compatibility++.
| aargh_aargh wrote:
| This is the only reference I found for the exact query
| "partial heap only tuple". It's hard to find anything using
| "partial HOT".
|
| https://www.talkend.net/post/268380.html
| mattashii wrote:
| > Adding new languages is already supported.
|
| This is true only to a degree: Adding new _procedural_
| languages is supported, but support for new query dialects
| (i.e. the main contact surface of most applications with an SQL
| RDBMS) is not currently a supported feature.
|
| As mentioned in the psql-hackers thread from the article,
| currently the postgresql-specific parsed syntax tree is passed
| through many components, and to replace the syntax you'd have
| to alter and recompile the core application.
| _msw_ wrote:
| NB: it is not "AWS Babelfish". It is (currently) "Babelfish for
| PostgreSQL".
|
| https://babelfish-for-postgresql.github.io/babelfish-for-pos...
|
| Can the title of this post be changed to remove "AWS" please?
| dang wrote:
| The opening paragraph of the article says " Amazon AWS
| announced Babelfish" and links to
| https://aws.amazon.com/blogs/opensource/want-more-
| postgresql... from the AWS blog. This all seems accurate to
| me. Is there some other reason why we should take AWS out of
| the title? Branding is not a reason that HN readers would
| support, and I think if we took AWS out for that reason, the
| community would accuse us of trying to obscure the origins of
| the project. Internet people are trigger-happy about this
| kind of thing, and too quick to accuse us of serving $BigCo
| agenda as it is.
| _msw_ wrote:
| I think we are trying to establish the brand of this
| project in these early days, and to have that brand
| distinct from AWS. I see this like "Mozilla" was separate
| from "Netscape" in their early days [1] (note, this link
| will not work when the referrer is from HN. Archive link
| provided for convenience [2]). It is a project that AWS is
| sponsoring and stewarding, but from what I currently know,
| I think the intent is for it to be community-centric in the
| long term.
|
| If successful, Babelfish will grow a community, and work
| harmoniously as an extension of PostgreSQL. And including
| "AWS" in the brand of the project may hamper the efforts to
| build a community around it (in addition to complications
| of using a brand / trademark like "AWS" in an open-source
| project).
|
| The TL;DR is: "AWS" is not part of the branding of
| Babelfish for PostgreSQL, and that is intentional. The
| reasons for this aren't about obscuring the origins or
| initial sponsorship of the work, but are rather about
| establishing a separate identity.
|
| [1] https://www.jwz.org/blog/2016/10/they-live-and-the-
| secret-hi...
|
| [2] http://archive.is/UMMCx
| andred14 wrote:
| Time to get off AWS or prepare to.
|
| The destruction of Parler, a company that did nothing wrong,
| should be alarming and motivating for you to ditch AWS and all
| big tech censorship companies.
| thinkx wrote:
| Postgres has a core design philosophy of being "extensible". This
| should be embraced..
| lukaseder wrote:
| For those who can't wait, there's https://www.jooq.org/translate
| notenoughbeans wrote:
| Hi Lukas! We use JOOQ at our company and it has been a great
| experience. Thanks for all your hard work.
| eternalban wrote:
| juicy little bit: "Apparently AWS has around 200 open job
| positions (!!) for developers working on Elasticsearch"
| itg wrote:
| According to my friends who are in AWS, the attrition rate is
| really high with many people leaving within less than 2 years.
| JulianRaphael wrote:
| I've been working for AWS across different teams for some
| time now and can't confirm that. There are a LOT of folks
| with +5 years tenure. In the US, the average tenure of
| workers aged 25-34 is 2.8 years, so you'll obviously see some
| turnover and some teams might see more than others - but the
| attrition rate is not really high across the company. In my
| opinion, we're hiring mainly because we are still growing
| really fast!
| myWindoonn wrote:
| Mm, but you're in management. All the Amazonians I've
| talked to have indicated that it's a very rough environment
| and they were glad to get out. Maybe you get a distorted
| view due to the power imbalance between you and your peers?
| JulianRaphael wrote:
| I'm an individual contributor in product management, so
| not in people management. Amazon / AWS is a challenging
| work environment, just like most other tech companies. In
| my opinion, that is also why so many people want to work
| for FAANG & Co, as you generally get to work on really
| interesting but also challenging things. I'd say that
| bursts of intense working periods are par for the course
| in FAANG & Co, but it also depends on your own ability to
| handle our self-service culture (which I find
| empowering). The culture is certainly peculiar but each
| large tech company has their own culture that works
| really well for some and doesn't jive with others. Just
| speaking for myself here, but I have learned a ton
| professionally and personally, work(ed) with inspiring
| people and had the best work-life balance of my career so
| far. Don't want to sound like a shill here - I'm
| certainly aware that in a company the size of Amazon
| there will be good and bad pockets, I just happened to
| work with good teams so far.
| eternalban wrote:
| That was my read of it as well. Amazon is dead serious
| about the fork.
| jorblumesea wrote:
| You might be on a solid team, but AWS is widely regarded as
| being a complete meat grinder. We've hired so many devs
| from AWS in the past 18 months (even pre covid) and they
| all have the same story. 12-14 hour days at a minimum,
| toxic leadership and managers, incredible pressure, tons of
| tech debt and mostly doing ops work.
|
| I'd say the trend for AWS is leaning "do not join".
| Anecdotal? Of course, but with enough stories it becomes a
| statistic and not an anomaly.
| JulianRaphael wrote:
| Hm. I will obviously appear to be biased, but isn't the
| collection of anecdotes that you've gathered a
| manifestation of some kind of inverse survivorship bias?
| Within the total population of people who work or have
| worked at AWS, the largest group of people with a
| negative experience will be in the subset of the
| population that has left the company? I really believe
| that companies the size of Amazon will inevitably have a
| certain percentage of people who had negative experiences
| - just like the company has negative experiences with
| individuals.
|
| Like I said in my other comment in this thread, I'm sure
| that there are some teams that don't have a good culture
| but it also comes down to the individual in many cases. I
| guess it comes down to individual preferences, some folks
| really don't want to work at Google, others dislike
| Facebook, some don't like AWS, etc.. But at the end of
| the day, all of these companies still attract a ton of
| incredibly talented people and I personally don't see
| that trend ending soon.
| jorblumesea wrote:
| Few companies have entire websites dedicated to how poor
| their experience was at said company. Amazon is the only
| company I've ever heard of in tech where people have
| devoted their spare time and money building resources to
| warn others about how bad and toxic their experience was.
| Microsoft doesn't have this. Facebook or Google don't
| have this. It's actually mind blowing when you think
| about it.
|
| Imo, Amazon is a special level of toxic and abusive and
| it shows by their reputation in the industry and how high
| their turnover is. That's what I mean by anomaly vs
| statistic. When thousands (tens of thousands?) of your
| former employees have horror stories, it starts to build
| a strong narrative than a few bad teams or a bad manager.
| Or you start to question if the ratio of bad teams is
| that high, what is wrong the the company as a whole?
|
| Does it mean your experience at Amazon is wrong or
| incorrect? Absolutely not. But your experience isn't the
| norm.
| JulianRaphael wrote:
| Out of curiosity, I just did a few searches along the
| lines of "is MSFT/FB/GOOG bad place to work" and for each
| of them there were several results/websites about how
| each of these companies is the worst place to work. If
| you look at the largest publicly available data set,
| Glassdoor, AWS does quite well - better than Microsoft
| and as well as Google. I have not worked for Amazon
| outside of AWS, so I can't really comment on that.
| However, as Amazon has more than 1,000,000 employees
| (400,000 of which were hired in 2020), I'd argue that you
| get more negative stories simply because of the larger
| population? Comparing Amazon to MSFT (163,000), FB
| (50,000) or GOOG (135,000) without taking its ginormous
| scale into account seems to be comparing apples to
| oranges. In terms of scale and typology, it would
| probably be better to compare AWS to these companies and
| I believe that the publicly available data paints a
| different picture than what you describe.
|
| I'm not trying to argue with you here, I've just had a
| quite few discussions in recent times with people who had
| very strong negative opinions about MSFT being a place
| for the lazy, Google being a place for the special
| snowflakes, Facebook being a place for the cultish, AWS
| being a place for those looking for a stepping stone in
| their career, etc....and I think that all of these
| companies actually are great places to work but there
| will always be a few percent who absolutely hated their
| time there.
| jorblumesea wrote:
| Nothing like this exists for any other company in tech:
| https://sites.google.com/site/thefaceofamazon/
|
| Just filled with stories of people that felt extremely
| wronged. Are there stories of people that with terrible
| experiences at other companies? Of course. Are there
| people that have had good experiences at Amazon?
| Absolutely. Does anything like the above exist for any
| other company? Not that I can find.
|
| That's crazy and really speaks to the difference. If you
| believe that Amazon is "just another tech company" in how
| they treat their employees, I'd probably say you're
| turning a bit of a blind eye.
| JulianRaphael wrote:
| Most of these stories are about Amazon and again, at the
| scale that Amazon operates I think that this would be
| expected? With 1,000,000 employees, if you assume a
| yearly turnover of just 1% that's 10,000 people leaving
| the company and I'm sure there are quite a few people in
| that group who had a bad experience - particularly as
| this includes the retail operation, which definitely had
| a bad reputation (most stories on the website that you
| linked are about retail teams). Post the NYT story in
| 2015 a LOT of things changed in retail, but I'm sure
| there are still issues.
|
| Not trying to be pedantic here, but this thread was about
| AWS and not Amazon, so I will just reiterate that I can't
| personally comment on Amazon as a whole, I can only
| comment on AWS and I think most people would agree that
| AWS _can_ be put into the same category as FB, MSFT,
| GOOG, etc. I have highlighted this distinction several
| times, but you seem to not want to engage in a
| conversation about AWS and rather discuss Amazon as a
| whole. That 's fine, but as AWS and Amazon are really two
| different companies, you are then comparing apples to
| oranges and in that case you should compare Amazon to
| e.g. WalMart?
| aynyc wrote:
| Ha! I guess hardware resources aren't the only thing
| Elasticsearch suck up!
| wiremine wrote:
| > The trends for the last 8 years are clear: Oracle and SQL
| Server are constantly declining in popularity; MySQL is slightly
| declining; and PostgreSQL is clearly growing in popularity. But
| while PostgreSQL almost tripled in popularity in these eight
| years, it is still far behind the other three.
|
| It's great to see Postgres growing, but I often wonder why it
| isn't growing faster, especially when compared to MySQL. Without
| getting into a religious flame war, I'm curious what the HN
| thinks about that?
| wayneftw wrote:
| Postgres never gave me the ability to quickly and easily build
| stored procedures that return multiple heterogeneous resultsets
| with ease. In SQL Server you do it like this:
| CREATE PROCEDURE get_customer_and_orders @id int
| BEGIN SELECT id, first_name, last_name, email, etc
| FROM customers WHERE id = @id; SELECT id, store_id,
| created_at, etc FROM customer_orders WHERE customer_id = @id;
| END
|
| I've quickly built entire applications with this tactic as the
| centerpiece. You can argue that it moves business logic into
| the database layer and to that I'd say "good", at least for
| apps that are maintained by IT departments with many strong SQL
| people and not so many developers. If you know TSQL, you know
| that you can also do branching, looping and other logic
| operations within this same procedure - you can even decide to
| send back 3 resultsets instead of 2 if you want to and the
| client API allows you to handle whatever it received quite
| elegantly.
|
| I think that features like this are why many businesses will
| stay on SQL Server. Also the high quality of tools for SQL
| Server that have no match in Postgres such as SQL Server
| Management Studio, SQL Server Data Tools, SQL Server Profiler,
| SQL Server Integration Services among many other such tools
| that are extremely well integrated.
| dvaun wrote:
| My limited experience with smaller companies (and now local
| government) shares this sentiment. For "small" databases
| (say, not hundreds or more of terabytes) I've experienced
| using stored procedures as a core of the ETL process.
|
| That being said, haven't some recent versions of Postgres
| added support for stored procedures or some variant? I'm
| curious if we'd seen any changes in performance if we
| experimented with switching over.
| Tostino wrote:
| They did, but there is still not a good way to return
| multiple result sets... I mean you can return cursors, but
| it's...strange.
| ubercore wrote:
| This isn't meant to detract from your point, but just a fun
| anecdote -- my first job out of college was working on a
| product that had its entire business logic layer built in
| stored procedures. All of it. Hundreds and hundreds of lines
| per procedure. It "worked", but man was maintenance a bear.
| In a way, it was good; ever since I've been very comfortable
| with SQL. But I can't recommend it.
| intrasight wrote:
| The example you gave makes no sense. It would be better
| expressed as a join. There may be examples where it makes
| sense, but I can't think of any. In SQL Server, I think of a
| stored proc as a logical operation that takes action and
| returns one object. If I want sub-structure within that
| object, I have the stored proc return one json or xml object.
| drdec wrote:
| The example given allows the application layer to
| instantiate a customer object with a list of order objects
| in one database query. Yes, you could do that by returning
| JSON or XML, but why have the extra step of parsing the
| text? Directly accessing the result set and acting on the
| object instances will be more efficient.
| hobs wrote:
| And yet, MARS is not great for troubleshooting or testing
| (for database people writing SQL) and I have worked at many
| places where the complexity of many sets doesn't really
| justify not making two procedures.
|
| Also the freetds guys hate it :)
| https://www.freetds.org/mars.html
|
| (I am currently converting a MARS app explicitly so it never
| does utilizes this approach)
| mrweasel wrote:
| MySQL/MariaDB have several relatively easy ways of building
| multi-master setups. Postgresql doesn't really have a that
| great solutions for master-master replication. Our customers go
| for MariaDB to get master-master replication
|
| We see some customers building new solutions on Postgresql, but
| for large setups, it's still MariaDB/Galera or Oracle and there
| isn't a big push to drop Oracle.
| otabdeveloper4 wrote:
| Postgres doesn't have a great solution for any replication,
| not just master-master.
|
| Lots of things are just half-baked and very clumsy in
| Postgres compared to MySQL (MariaDB).
| Tostino wrote:
| Streaming replication works great in Postgres.
| __s wrote:
| I worked at a place that was running SQL Server 2012 in
| compatibility mode with an older version. The place had a
| pretty high turn over rate, so a lot of the expertise is in the
| existing tech rather than the people. It was hard enough to try
| convince them to upgrade SQL Server (I wanted to leverage row
| level security to get away from having a dozen copies of what
| was essentially the same app, but that required SQL Server
| 2016)
|
| Also, when I brought up postgres for a new project & mentioned
| it being open source, response was "that means it could vanish
| any day"
|
| Inertia is real
| DaiPlusPlus wrote:
| > I'm curious what the HN thinks about that?
|
| People don't need those white-label commodity cPanel or Plesk
| shared-hosting or VPS services to run SaaS applications anymore
| - it's far better for everyone[1] when they switch to a major
| cloud provider, namely AWS, Azure, GCP, etc.
|
| I argue that most web/saas devs today grew up with - and
| tinkered-with - said white-label services: which means they got
| their experience with MySQL because it was part of the stock
| default configuration for all those web-host accounts: with
| PHP, maybe Perl, and many preinstalled web-applications like
| phpBB, Wordpress, Coppermine, phpNuke, and so on... and that's
| probably a solid 15+ years of commonplace web-hosting market
| saturation (thinking 2001 through 2016, with 2014 being the
| tipping-point for AWS/Az/GC being the home of SaaS).
|
| You'd be in high-school age-range (14-18?) and get bored of
| simply running other peoples' programs on your web-space, and
| you saw these web-applications were written in PHP so you'd
| follow some series of online tutorials for PHP and when it
| inevitably leads to databases they assume you have MySQL - and
| MySQL (at the time) had not only the widest installbase, but
| also had a far more forgiving SQL engine than anyone else.
| While PostgreSQL was often only a few clicks away via quick
| installers built-in to cPanel/Plesk/etc, MySQL was a safer-bet
| by everyone involved.
|
| ...but those days are past: as a personal anecdote: at no-point
| in the past 10 years have I come across any "serious" PHP web-
| application software intended for private or on-prem usage
| since vBulletin or SVN.
|
| [1]Except free-as-in-freedom software advocates...
| rubyskills wrote:
| I think around that time, postgres was handicapped because of
| performance reasons. It was considered slow because of ACID
| compliance and MySQL was all the rage with MyISAM.
|
| If only postgres won the marketing battle back then...
| DaiPlusPlus wrote:
| > If only postgres won the marketing battle back then...
|
| Be glad they didn't - otherwise their support dept. would
| be buried under questions from web-hosting newbs.
| sofixa wrote:
| People go with what they know, the the LAMP stack was a classic
| for multiple years. And even today PostgreSQL's HA tooling
| isn't that great ( one can set up MySQL/MariaDB in
| active/active in a few minutes, and that's impossible with
| PgSQL, and even active/passive with easy failover requires more
| work and oftentimes external tooling).
| hospadar wrote:
| Personal guess: because switching DBs is expensive and if it
| works, why bother?
|
| I personally LOVE pg over all other choices, but I've got some
| 5-year-old apps sitting on a little mysql DB and no intention
| to move them. Dbs are small, performance is good enough, cost
| of instances wouldn't be very different on PG, but cost of
| moving everything over is probably a week or two of work that
| would be pretty hard to justify given the current situation.
| linuxhiker wrote:
| In terms of overall adoption, PostgreSQL doesn't need to catch
| Oracle or MySQL. The people that enjoy legacy platforms are not
| going to change (necessarily).
|
| Real work gets done with PostgreSQL. You can't buy something
| from Amazon.com without using PostgreSQL. The more Amazon talks
| about that in their ecosystem, the more PostgreSQL becomes the
| default choice for all new developers within that (very large)
| ecosystem.
|
| When you combine that with the loud presence of another 800lb
| Gorilla promoting PostgreSQL (MSFT), the market has spoken.
| runako wrote:
| The popularity of PG relative to MySQL is going to be
| unavoidably skewed by the popularity of WordPress and its tight
| linkage to MySQL.
| aynyc wrote:
| In my little world, MySQL was popular because we could get
| enterprise level support directly from MySQL. It was the
| default choice from that point on. I don't know if that has
| changed. AWS changed that pretty quickly.
| ahachete wrote:
| OP.
|
| This is a very good and fair question. Postgres is growing a
| lot, but is it growing enough, according to its true potential?
| It has everything: incredibly robust and trusted; very large
| feature set; not under any company's direction; extremely
| liberal license. Should be conquering the database market, and
| by far is not!
|
| I don't have an answer. I have potentially, many. Possibly I
| will blog about this at some time. But I believe that
| definitely it should be growing more and becoming more relevant
| than it is right now.
|
| This is no detriment at all to all the fantastic work done by
| everybody; but just the ambition that Postgres can and should
| go farther.
| throwaway_dcnt wrote:
| Enterprise dba shops have deep expertise with oracle (and
| sometimes sql server). I have personally run into serious
| resistance from these teams when I explicitly ask for postgres.
| They try to warn me that oracle is the recommended option for
| real production use cases because of the deep in-house
| expertise and that I am pretty much on my own if I go with
| postgres. Most people back off at that point.
| derekp7 wrote:
| I've seen this so much, and can empathize with them. When you
| have a product that took you years to master, the value that
| you bring to your employer is wrapped up in that product. So
| it isn't so much that someone is a DBA, they are an Oracle
| Database software specialist. You've learned all the quirks
| so that when something goes south, you "have the answer". So
| naturally you will have a number of reasons why another
| product isn't as good.
|
| I feel the same way with Unix/Linux. At one point I was told
| that my organization may be migrating away from Linux, and
| redeploy everything under Windows. The management layer
| really couldn't understand that they would lose nearly every
| Linux admin on staff (that happened to also have deep
| understanding of their apps and infrastructure). "Why would
| someone have a problem with switching platforms -- we'll
| provide you the training" was their answer. Fortunately we
| were able to come up with a number of technical (and cost
| related) issues that the project was shot down.
| lumost wrote:
| At sufficiently large companies, deep in-house experience
| with a given DB technology may be roughly equivalent to a
| startup engineers deep experience on their companies
| current monolith.
|
| Companies with large and complex db installations often
| need the complexity for various reasons that are difficult
| to escape, even if you're working on a green field project.
| matwood wrote:
| PG over mysql is obvious IMO. PG over mssql, not so obvious.
| People often forget the 'management system' part of RDBMS.
| MSSQL provides an enormous amount of tooling in the typical
| enterprise install. Analysis services, data tools, etc...
| provide a lot of value beyond a data store.
|
| Of course if you don't need any of these additional tools or
| can't afford them, then yeah pg is a great choice.
| runako wrote:
| Cost is a really solid argument against MSSQL. Even at
| Azure, MSSQL costs ~6x as much as the open-source
| databases. For even relatively small workloads, it's not
| hard for the MSSQL license to cost as much as an FTE.
|
| Aside from the raw cost, the licensing model and cost
| considerations can drive design decisions. In my
| experience, most shops that use MSSQL do so because nobody
| on staff has experience with other databases.
| matwood wrote:
| Sure, but we're talking about enterprises. Back when I
| could walk down the hall to the datacenter, paying $30k
| for the hardware and $60k for licenses was normal. Every
| time I bought a new server I tried to make a case for pg
| or mysql, but the cost to replace SSIS, SSAS, or SSRS
| either with another product or to build a replacement was
| always more.
| tomnipotent wrote:
| > Cost is a really solid argument against MSSQL
|
| If you're a small 2-5 person team, sure. But once dev
| payroll breaks $1M/yr, paying ~$100k for SQL Server
| doesn't seem so bad anymore.
| vladvasiliu wrote:
| Are there no equivalents to these tools for PG?
|
| I've only interacted with Analysis Services and Reporting
| Services for maybe five minutes, but the impression I was
| left with is that they're extremely clunky, even by Windows
| standards. I also remember a weird situation with SSL
| wildcard certificate configuration, where the interface
| said the configuration wasn't applied, but it actually was.
|
| My client subcontracts this to a supposedly "expert DBA"
| shop, and they always seem to take ages to do what look
| like simple things on the surface. (I rarely if ever
| interact with them, so I don't know whether they're
| actually competent or not.)
| matwood wrote:
| Tools like SSIS, SSAS, SSRS were hard to replace for less
| money for a very long time. There are products out there
| now, but they are not cheap either. I still think SSIS
| was one of the easiest and most flexible to use ETL tools
| out, and it was included with MSSQLs enterprise license.
| Spivak wrote:
| Or the same issue form the other side. It's damn near
| impossible to hire a good DBA in some areas and so if you
| find someone who has deep magics with MySQL then you go with
| MySQL. In a huge complex app they're all good options except
| for Mongo and so going with what your team knows is far more
| productive.
| ryandvm wrote:
| As a contractor, I'm glad that MySQL is still around. It's an
| extremely useful red flag for potential clients.
|
| It is almost guaranteed that if a client is running on MySQL,
| you can be certain their entire code base is going to be
| plagued with less-than-best-practices. Not to be too harsh, but
| MySQL's most prominent use case is for projects that start with
| "let's just stand-up a DB real quick and we'll sort out the
| hard stuff later".
| pyrophane wrote:
| Have used both extensively. I generally prefer MySQL because I
| find it easier to manage for a few reasons:
|
| 1. MySQL has a simpler permission model, so user management is
| less of a headache.
|
| 2. Connections are cheaper in MySQL so you don't have to use an
| external connection pooler like pgbouncer.
|
| 3. There's more/better documentation on MySQL performance
| tuning, especially from Percona.
|
| That's really it for me.
| gkop wrote:
| Permissions management in Postgres is indeed a pain.
| xyzzy_plugh wrote:
| I agree, but can you enumerate a bit more what specifically
| is painful?
| tpetry wrote:
| Try the lock down e.g. a user to only allow reads. Not
| easy, when testing if it's working you will see so many
| ,,strange" behaviours e.g. you can still create tables in
| public namespace, databases created before/after (dont
| remember which one) your user are accessible etc...
|
| Postgresql is following a blacklist access model which is
| really hard to get right, a whitelist approach like mysql
| would be much easier.
| knodi wrote:
| I think a lot of people just don't know the new capabilities of
| PostgreSQL. PostgreSQL's competition isn't MySQL, its the new
| hybrid DBs like CockroachDb.
|
| DBAs and Engineers that know MySQL will always use MySQL. In
| corps MySQL is a safe choice (sadly).
| Macha wrote:
| 1. MySQL is offered as the default "open source" database in
| most discussions. This puts the onus on Postgres to people who
| go "I need a better database", rather than people going "I need
| a database" in a "Nobody got fired for choosing MySQL" way.
|
| 2. Companies looking for enterprise support don't see anyone as
| high profile as Microsoft, Oracle, and err, Oracle supporting
| Postgres.
|
| (Whether ongoing interaction with Oracle is a net positive is
| of course debatable. At a previous company our only interaction
| with Oracle was them being inflexible and expensive that
| ultimate meant we had a Oracle DB on prem serving a AWS hosted
| service because it would have cost too much to put it in AWS.
| Note, this was many years ago, before AWS had a compatible
| offering, and apparently Oracle even have loosened up a bit
| here since.
| sl1ck731 wrote:
| Wordpress is probably the elephant in the room.
| xmodem wrote:
| At my current gig, we've been burned pretty badly by
| scalability problems with MySQL, and the prevailing wisdom is
| that new functionality and services that have to deal with data
| at scale should be built on Cassandra. My cries of "we wouldn't
| have this problem with Postgres" have mostly fallen on death
| ears.
| pizza234 wrote:
| Can you elaborate on the specifics of the scalability
| problems, and how the different databases do differently on
| them? It's an interesting subject.
|
| We're having scalability problems with our RDBMS, but it's
| something RDBMSs in general don't solve.
| xmodem wrote:
| We have a few large (not large in the scheme of things, but
| large for MySQL) tables that we can't add columns to
| without locking the table for an extended period. In
| Postgres this operation is instant no matter the size of
| the table.
|
| This not the only problem, but it is the most pernicious
| Thaxll wrote:
| MySQL 8 aleviate that problem, although it's not perfect
| it's better than before:
| https://mysqlserverteam.com/mysql-8-0-innodb-now-
| supports-in...
| pizza234 wrote:
| I confirm the sibling post that adding a column is
| instaneous on MySQL 8 (although there are limitation),
| and point to the reference document:
| https://dev.mysql.com/doc/refman/8.0/en/innodb-online-
| ddl-op....
|
| Outside the context of db comparisons, and in relation to
| the specific case, if you don't have triggers/foreign
| keys on a given MySQL table, Gh-ost1 solves the DDL
| locking issues.
|
| 1=https://github.com/github/gh-ost
| tomnipotent wrote:
| > adding a column is instaneous on MySQL 8
|
| Has been since MySQL 5.6 (released Feb `13).
| Thaxll wrote:
| I don't see where PG would scale better than MySQL since
| InnoDB has historically been more performant than PG.
| Shish2k wrote:
| In my cases I found that mysql was faster for "select *
| from <table> where <unique key>=<blah>" -- but for anything
| more complicated than that, postgres won (admittedly I
| haven't checked recently - it was around 10 years ago that
| my 500-concurrent-user site was melting mysql, so I
| switched to postgres, which is now happy with 15,000
| concurrent users - I'm mostly just responding to
| "historically more performant")
| kstrauser wrote:
| Historically, MySQL has been better at single concurrency
| setups, like your average Wordpress site whose main reader
| is the person posting to it. PostgreSQL is usually much
| better in high volume scenarios.
|
| For instance, suppose you have a query that requires a full
| table scan. In PostgreSQL, that query comes in and the DB
| starts the scan. Now, a second such query comes in.
| PostgreSQL notes "I'm on row 10,000" and continues running
| one single scan, sending results to both queries. When it
| reaches the end of the table, it marks the first query as
| complete, then goes back to the beginning of the table and
| starts scanning again, sending results to the second query
| up until it reaches row 10,000. Now the second query has
| seen every row in the table and it's finished.
|
| Now imagine 100 such queries arrive. Rather than doing 100
| full table scans, PostgreSQL satisfies all of them
| concurrently, taking at most 2 full scans (assuming the
| 100th queries joins in on the very last row of the first
| scan).
|
| PostgreSQL has a million such optimizations under its hood
| that make it happily chug away even when it's getting
| slammed. You can vertically scale a single-instance
| PostgreSQL server a lot higher than many people would
| believe.
| tomnipotent wrote:
| > PostgreSQL has a million such optimizations under its
| hood that make it happily chug away even when it's
| getting slammed
|
| So does MySQL. Facebook, Google, and Percona have
| contributed many upstream patches to harden and improve
| MySQL over the years. MySQL also has much better
| observability to find issues causing problems, from
| index_statistics to hunting down individual queries
| causing locks.
|
| I agree that Postgres is by far the better engineered
| product, but it's not necessarily the better RDBMS
| product because of that. MySQL is battle tested in a way
| that Postgres isn't (yet).
| fire7000 wrote:
| MySQL is a bit simpler even though pgadmin is a wonderful tool
| for learning postgres advantages. For me the difference became
| clear when needing to plot locations on a globe. PostGIS is
| super helpful
| sam_lowry_ wrote:
| MySQL has way better UX for command line users. Postgres
| can't get away from legacy Ingres design, with a postgres
| user and a horrible command line client. Also,
| database/schema/tables hierarchy is a penalty over
| database/tables in MySQL.
| mattashii wrote:
| I fail to see how Postgres' hierarchy is any worse than
| MySQL. You have one more layer of hierarchy which you do
| not _need_ to consume, with the added feature of atomic
| renaming any of the layers.
|
| If I'd compare this to MySQL, where you must move tables
| one-by-one to another schema (nay database) with the new
| name to 'rename' your database [0], then I'm very happy
| with postgres' hierarchy structure.
|
| [0] That is, if you're even allowed to, because if you have
| a trigger on your table, good luck renaming your database.
| See https://dev.mysql.com/doc/refman/8.0/en/rename-
| table.html
| jasonpbecker wrote:
| psql is easily one of my favorite tools. What makes the CLI
| "horrible" in your mind?
|
| Also, database/schema/tables is great when doing things
| like multi-tenancy where you split customers by schema
| (which also makes horizontal scaling super easy if needed).
| syspec wrote:
| I would say the commands are just not intuitive.
|
| For example, when you type quit instead of quitting it
| instead prints out that you must type a different command
| to quit
| fire7000 wrote:
| One complaint is that it doesnt like camelCase
| identifiers
| mattashii wrote:
| Last time I tested (literally seconds ago) this works.
|
| All of \q, quit, and exit work fine starting psql 11?
| syspec wrote:
| Oh good to know!
| majewsky wrote:
| Ctrl-D works fine for me.
| specialist wrote:
| Tangentially. From the OC:
|
| > _I don't have an MBA, but this to me is a clear case of The
| Innovator's Dilemma. PostgreSQL protocol v3 is the incumbent, and
| protocol v4 and /or other protocols like HTTP are the potential
| disruptive innovators._
|
| I have no idea what Christensen means by "disruptive technology".
| https://en.wikipedia.org/wiki/Disruptive_innovation
|
| The fifth qualifier "New firm's business model differs
| significantly from incumbent" seems important.
|
| The rest feel like Just So stories. Especially "Originate in low-
| end (less demanding customers) or new market (where none existed)
| footholds".
|
| I think time has shown Peter Drucker to be more right. First
| target affluent customers and then ride the cost curve down
| towards market success.
|
| I just reread Innovator's Dilemma, and searched the rest. Nothing
| about cost curves.
|
| I've been trying to think of how "disruption" would apply to
| Tesla vehicles and Apple Silicon. Neither products came in as
| "toys".
|
| Rather, I think both initially targeted very high end niches
| (novel use cases), and the underlying enabling technology was
| just starting their cost trajectories, whereas the incumbent mass
| market products were at the end of their trajectories. Plough
| metric tons of cash into R&D and wait for the new to overtake the
| old.
|
| --
|
| Back to AWS Babelfish.
|
| AWS adds TDS and T-SQL compatibility to steal business from
| Microsoft Azure. Just some more bullet list items for middle
| management purchase bots to check off.
|
| It'll surely improve, but switchover costs for non-trivial
| projects would still be pretty high. Back when I was warehousing
| medical records on both MSSQL and Oracle, all the hard work was
| tweaking the physical design, eg codesigning the indices and
| queries, to make stuff performant. I'm sure that's still true.
|
| I think adding these features to Postgres is good for everyone
| but Microsoft.
|
| --
|
| I guess I object to the common use of "disruption". As though
| there's an element of surprise.
|
| The meaning of "disruptive innovation" wonderfully Agile.
|
| I'd suggest the OP framed AWS Babelfish in terms of competitive
| threats (aka insurmountable opportunities). And maybe predict
| some unforeseen consequences.
___________________________________________________________________
(page generated 2021-02-12 23:00 UTC)