[HN Gopher] Migrating Netflix to GraphQL safely
___________________________________________________________________
Migrating Netflix to GraphQL safely
Author : theptip
Score : 146 points
Date : 2023-06-18 16:33 UTC (6 hours ago)
(HTM) web link (netflixtechblog.com)
(TXT) w3m dump (netflixtechblog.com)
| nf17 wrote:
| Is JS/Express/Apollo still the goto graphQL server implementation
| or C#(HotChocolate), Java, go etc are widely used too? On the
| client side(for mobile), AFAIK iOS(Swift) and Android(Kotlin)
| does not support GraphQL natively? are there 3rd party libraries
| that can be used?
|
| Edit: Added Apollo
| asabla wrote:
| We've been using Hotchocolate for over a year now I n a
| project. And the experience has mostly been pretty good.
|
| And pair this with their Blazor client called StrawberryShake.
| And your production levels are pretty nuts.
|
| So to answer your question, yes people do use other
| alternatives
| gavinray wrote:
| The "go-to" GraphQL server implementation varies based on
| language.
|
| Here are the popular ones to my knowledge:
|
| - Node.js: GraphQL Yoga + Fastify, Apollo Server
|
| - Java/Kotlin: Netflix DGS (built on top of Spring Boot)
|
| - Scala: Caliban, Sangria
|
| - Python: Strawberry, Graphene Django
|
| - C#: HotChocolate > "On the client side(for
| mobile), AFAIK iOS(Swift) and Android(Kotlin) does not support
| GraphQL natively?"
|
| GraphQL queries are just HTTP POST queries with a JSON body.
| They're supported everywhere.
|
| If you want specialized tooling for them, Kotlin and Swift both
| have great strongly-typed GraphQL libraries.
|
| Apollo publishes libraries for both:
|
| - https://www.apollographql.com/docs/kotlin/
|
| - https://www.apollographql.com/docs/ios/
| a_wild_dandan wrote:
| It appears that fastify can't keep their website up.
| Justsignedup wrote:
| I've been using json-api with graphiti in rails. And I gotta
| say the massive gain from it is that it's all rest and a
| trivial client impl from scratch if necessary.
|
| Couldn't recommend it more.
|
| Have to say. Most json-api impls suck, so graphiti has been a
| game changer for me.
|
| N+1 are a problem buuuut it forces you into good model designs
| and good tools like graphiti and the latest rails show you the
| cause of the n+1s so it's not so bad to debug.
| latchkey wrote:
| I wonder how many hidden n+1 problems there are. Reading the blog
| post, it looks like they mostly only tested existing pathways as
| part of the migration. Who knows what random requests could do.
| rand_r wrote:
| Yes, this is the biggest problem with GraphQL that I haven't
| addressed anywhere.
|
| The architectural beauty of GraphQL is that you can write
| isolated, functional nodes that only have to know about their
| object type. For example, the User node only has to know how to
| get the user's last_login, active status, and return a
| reference to another node type like a profile Image.
|
| Inside the User node, you are free to write code (e.g. Java or
| Python) that can compute which Image node is the right one to
| return. i.e. you can write: class UserNode:
| def get_user_profile_image() -> ImageNode: return
| some_arbitrary_function()
|
| There is no way I've seen to square this kind of arbitrary node
| computation with the desire to write an optimized SQL query
| that selects all needed information in one big-ass statement.
| valenterry wrote:
| The exact same problem exists in an typical REST api as well.
| In both cases you have to add a specialized endpoint to
| retrieve multiple results.
| striking wrote:
| You could use https://github.com/join-monster/join-monster,
| but setting that aside it's usually enough to use Dataloaders
| that resolve based on some source ID, with some server-local
| caching on top if latency becomes an issue.
| gavinray wrote:
| This is Netflix, not some amateur-hour implementation copied
| from a blogpost.
|
| The de-facto GraphQL Java library comes with dataloaders built-
| in anyways:
|
| https://www.graphql-java.com/documentation/batching/
| voz_ wrote:
| While Netflix has a good reputation, we should not give
| blanket benefit of the doubt, especially since the parent
| comment was a rather interesting and inquisitive one.
| 727374 wrote:
| Dataloader (https://github.com/graphql/dataloader) eliminates
| many n+1 issues. Basically, entity requests are queued up
| during evaluation of the query and then you can batch them up
| for fulfillment. It's a great addition to other asynchronous
| contexts as well.
|
| WRT random requests, there are libraries for query cost
| estimation you can use as a gate.
| gumballindie wrote:
| Curious how they handle GQL injections. Most devs using graphql
| have no clue what they are as the majority just cargo cults new
| frameworks to keep themselves busy.
| valenterry wrote:
| What are GQL injections? You mean something like SQL injections
| that come through the graphql API?
| saurik wrote:
| Why? What problem did this solve for them? I use Netflix _a lot_
| , and I use it on AppleTV where I get automatic updates to the
| device and the app... the app has never changed except to get
| slower over the years (and I mean a _lot_ slower). There is
| absolutely no new functionality they have ever deployed... isn 't
| the app "done"? Is there some serious scaling issue on their
| backend that is causing the perceived frontend slowness that this
| will fix? Is this going to massively decrease the costs of the
| backend? Or is it just that the engineers are all bored and can't
| accept the idea that the product is either 1) done or 2) needs a
| new direction (instead of some new tech)?
| thephyber wrote:
| > isn't the app "done"?
|
| Netflix is an ecosystem, not just the frontend app for Apple
| TV. Netflix is pretty close to a vertical monopoly for some of
| its content. Fund-raising, planning, development, post-
| production, translations/dubbing, preparation for distribution,
| and distribution. Your idea of viewing a video content on the
| app is only the last 20%.
|
| Also worth reminding people that the AppleTV app is not the
| only app Netflix supports for viewing. There have been
| dozens/hundreds of different apps and most are still actively
| supported.
|
| Also, they are a company, so they are always looking to be able
| to optimize and cut costs.
|
| It's worth staying curious and asking why before assuming you
| know more than the engineers who have to work "behind the
| curtain".
| mkwarman wrote:
| I have definitely noticed and been frustrated by what you're
| calling out here. There always has to be something to work on,
| even if that thing might invalidate a significant amount of
| previous effort or cause regression. As a developer myself, I
| guess it is good for me, because it means job security. And
| maybe that's why there's always something to be added or
| changed or re-created on the products I've helped develop - to
| ensure all the individual contributors have something to keep
| them busy. But it definitely is frustrating that it seems like
| an app being called "done" is extremely rare nowadays. Though I
| suppose developers didn't really have the choice to continue
| development indefinitely back before the internet allowed
| "miscellaneous bug fixes and feature updates" patches to be
| pushed weekly.
|
| Edit: I just noticed who I was replying to. Thanks for your
| work on Cydia and in the iOS jailbraking scene at large. I have
| lots of fond memories jailbraking and installing all kinds of
| weird tweaks on my iPhone 4 back in the day.
| Dudester230602 wrote:
| It is surely solving a promotion problem and a bonus-getting
| problem for someone in their hierarchy. Plus it is making the
| world a better place through constructing elegant hierarchies
| for maximum code reuse and extensibility and through minimal
| message-oriented transport layers.
| revskill wrote:
| How to avoid N+1 query ? You don't join SQL.
| lolive wrote:
| I am not sure I understand that n+1 select issue with graphQL.
|
| On my project, we generate SPARQL queries [to simplify, it is
| not so different from a GraphQL query] and we send it to a SQL
| translator that then sends that SQL to a Spark cluster. It is
| reasonably efficient, and extremely flexible.
|
| Of course, this is a single database use case, but I don't see
| any n+1 select involved here. And I do not understand how
| single database graphQL servers cannot work exactly the same.
| x86x87 wrote:
| brave to assume they are avoiding this problem :/
| toomim wrote:
| Falcor was cool. I'm sad seeing it go away.
| jFriedensreich wrote:
| I would not say it is more going away now than in the last 2
| years of not getting upstream support from netflix. If anything
| this step at least makes clear this is a dead end at netflix
| and we can think about how to go forward with what is/was great
| about falcor!
| itake wrote:
| I don't see the value ins Graphql. Most apis rarely change and
| the complexity of supporting queries is not worth the time saved.
|
| Graphql was a cool experiment, but I'm sticking with
| rest/protobufs
| rand_r wrote:
| The original motivation was to reduce the number of network
| requests made on cellular network-connected mobile devices,
| both to save on power consumption and reduce latency for the
| user. It off-loads a lot of complexity and computation from the
| client to the server, and make sense if want to create as good
| a mobile experience as you can.
|
| In terms of code architecture, it drastically simplifies
| frontend code, and overall reduces backend code and
| boilerplate. It's extremely elegant and nice to use over
| creating custom endpoints for every use-case, but the N+1 SQL
| query problem is a giant thorn.
| andrewingram wrote:
| Kinda, by all accounts the original motivation was to solve
| the proliferation of endpoints. Everyone was already writing
| per-screen endpoints to solve the network requests problem
| (which, contrary to what many believe, wasn't magically solve
| by http2), but they were doing it in a way which played out
| in several ways:
|
| 1. Having to make a new endpoint every time your frontend
| changes
|
| 2. Not making a new endpoint, and just augmenting the
| existing one for a screen. Which would cause them to
| accumulate dead data over time (especially if multiple app
| versions had to be supported concurrently -- in the case of
| Facebook this was ~3000 versions).
|
| 3. Being lazy and finding a similar endpoint meant for
| another use case and piggy-backing it for your new usage.
|
| All of these would create maintenance and organisational
| problems.
|
| GraphQL is essentially a system that lets you write endpoint
| specs (in the form of GraphQL query documents) that the
| backend knows how to support. It's not a free lunch, you
| still have to do a lot of work on the backend to allow it to
| do this efficiently and securely (nowhere near as difficult
| as people make it out to be though), but it does a really
| good job at solving the endpoint maintenance problem.
| itake wrote:
| You can still design an immutable REST api to match the
| client's needs. Apollo has really nice out-of-the-box caching
| that can be tricky to replicate with rest.
|
| I see your point about reducing network requests (graphql
| lets you co-mingle mutations and queries), but I don't think
| that feature is used often enough, b/c what if one query
| fails, then everyone has to fail).
| troupo wrote:
| > Apollo has really nice out-of-the-box caching that can be
| tricky to replicate with rest.
|
| Caching is _trivial_ to do with REST.
|
| The "really nice caching" that Apollo does literally
| deserializes and deep inspects _both_ the request _and_ the
| response to try and figure out caching on specific fields.
| Also, POST requests are not cacheable according to HTTP
| spec.
|
| With REST you set an ETag and/or Cache-Control headers and
| let your existing infra and client browsers handle that.
| evangow wrote:
| You can make a REST endpoint match a client's needs, but if
| you have multiple clients (for example, 1 for mobile and 1
| for desktop) that all need different amounts of data (for
| example, the mobile client shows a simplified view with
| less data), then you would need to write 2 different REST
| endpoints to handle each clients. Multiply that by the
| number of pages with a difference.
| troupo wrote:
| > then you would need to write 2 different REST endpoints
| to handle each clients.
|
| Or you use Accept header for its intended purpose
|
| Accept: com.company.user-aggregated/json
|
| Accept: com.company.user-aggregated-lite/json
|
| etc.
|
| And let the server return the fields defined by the
| relevant schemas.
|
| I mean, the advise given for GraphQL in prod is to
| literally call predefined queries only.
| troupo wrote:
| > but the N+1 SQL query problem is a giant thorn.
|
| It's worse because in a big system this becomes N+1 API calls
| to other systems (each with their own permissions, latencies,
| caching etc.)
| devmor wrote:
| For an API that centers around filters and nested subfilters to
| retrieve exponentially specific data, graphql can end up saving
| you a lot of work.
|
| For anything else, it seems pointless.
| valenterry wrote:
| I have a lot of experience with GraphQL and I love it. It saves a
| ton of development time.
|
| However, for those who wonder what the disadvantages are, here
| are some. They are from real world production experience and they
| are _not_ what people usually claim wrongly (like graphql would
| be harder to secure /version, ...)
|
| No, some real pain points are:
|
| 1. Lack of a Map-type. There is no native type to return a
| list/array of objects but not a map with unique keys or even a
| set instead of a list. You can fallback to "convention" or custom
| types (like plain json) but it sucks.
|
| 2. Uploading big files (in chunks) is much more tricky than
| expected
|
| 3. There are union-types for the returned data. For example, I
| can query for an animal and get either a cat _or_ a dolphin and
| if I use typescript or another capable language then the compiler
| will tell me that. But I cannot _send_ such a request to the
| server, saying "here I send you an animal, the data is this
| ...". The server will have to provide a separate query/endpoint
| per animal. Very very annoying.
|
| 4. While graphql is awesome when it comes to batching requests,
| what it is not so awesome at is to improve performance by
| allowing the server to return the part of the data is already
| resolved. So while reducing the number http requests, now the
| previous slowest http request will become the bottleneck for all.
| This means the page might load subjectively faster because some
| parts are rendered later than possible. There is work going on
| here (like @defer) but it's not fully spec'd and also the
| solution doesn't work for some cases
|
| There are more points but this gives you some idea of the ACTUAL
| problems. Otherwise, from development and setup time over
| maintenance, analytics, versioning/deprecation, tooling and so
| on, graphql is much better than your standard REST api in my
| opinion.
| scotty79 wrote:
| > Lack of a Map-type.
|
| Why do you need special map types for transferring maps?
| Doesn't list of key-value pairs suffice?
|
| Also, why am I wrong to even ask the above questions?
| valenterry wrote:
| It's a fair question. Having a map-type has several benefits:
|
| 1. I can signal to my api's users that there won't be any
| repeating keys without having to write documentation
|
| 2. The type can be mapped correctly into the target language.
| Most languages have a map-type and so it's nice if you don't
| have to convert it yourself
|
| 3. No manual conversion also saves time in most cases, since
| libraries can do it efficiently in one step. Otherwise you
| get a list<X> from your library and have to turn it into a
| map. Not good for bigger lists in performance sensitive
| applications
|
| 4. The same is also true the other way around when sending
| data to the server
|
| 5. It saves space in many serialization formats like json.
| Instead of [{"key": "key1", value: "value1"}, ...] you get
| {"key1": "value1", ...}
|
| 6. It's nicer to read and debug
|
| 7. It's nicer to manually create/write those values in tools
| like graphiql to test queries by hand and those tools can
| even warn you if you repeat a key by accident
|
| Just some of the things out of my head, I'm sure I've missed
| a bunch.
| tadfisher wrote:
| A (single-level) Map is generally understood to have unique
| keys, which allows for some invariants derived from this
| property, such as a one-to-one or one-to-many object
| relationship, without runtime checks in client code.
|
| I do think it's worthwhile to ask the question, though. I
| have a strong personal preference to avoid encoding
| invariants like this in wire representations, because they
| are properties of the _schema_ and not the wire type (JSON
| object, in my most commonly-encountered case).
|
| Of course, the entire point of GraphQL is to encode
| invariants in the data schema, so it's entirely reasonable to
| ask for a built-in Map type instead of building your own for
| every project.
| sibeliuss wrote:
| Definitely not working anywhere that doesn't have a GraphQL api.
| Once you've used it at scale there's no going back.
| runeks wrote:
| Go on...
| x86x87 wrote:
| I never understood why people go for graphql.
|
| In my experience, it's a nightmare to 1/ secure 2/ version 3/
| ensure qos
|
| Securing it properly should make it a no-go in like 95% of cases.
| you get amazing flexibility on the front end with a heavy heavy
| cost on the backend. Also, in general being able to say "i want
| to do whatever" and request everything at once is an anti-pattern
| IMHO, especially after HTTP2 became mainstream and doing multiple
| requests is reasonably fast.
|
| Versioning? Forget about it. Now you have "all the versions" and
| good luck figuring out what is used, what is not used or god
| forbid deprecate something. you're going to have a bad time.
|
| As far as qos goes, if any client can range from "i'm asking a
| simple thing" to "i would like the whole world please - and btw
| you're only going to figure things out as you pull them in" qos
| becomes a pipe dream.
|
| Last, I don't understand what Netflix does that is so complex
| that would warrant something like gql. I just don't. To the naive
| developer in me it seems that they 1) need to have a basic api to
| get whatever catalog they have + a few apps build on top of that
| 2) have figured out how to do streaming exceptionally well in
| order to scale to all the people watching. That's is and although
| a gross oversimplification I cannot think about a scenario where
| gql is needed.
|
| I think in their desire to reinvent the wheel Netflix has jumped
| the shark. A lot of really good ideas have come out of the
| innovation they did in the past but lately it seems like they are
| doing things for the sake of doing them or they are pissing
| people off with password household policies and whatnot.
| jdauriemma wrote:
| The state of the art of GraphQL tooling around securing and
| versioning has evolved significantly. You may want to take
| another look at what's out there.
| rblatz wrote:
| I was just researching graphql versioning and the
| recommendation is not to version because the graph is
| supposed to be an evolving entity.
| x86x87 wrote:
| [flagged]
| Dudester230602 wrote:
| Put this into the _" my code is self-documenting"_ bin.
| x86x87 wrote:
| my self-documenting code is evolving!
| itsmeste wrote:
| mutation createUser-v2
|
| Seen that in some productive APIs, always fun.
| tnzk wrote:
| Could you please provide some pointers?
| thrashh wrote:
| GraphQL is like SQL. It's an extremely leaky abstraction but in
| the hands of someone experienced, you can do amazing things
| with a fraction of the effort required.
|
| But just like SQL, you have people who happened to learn SQL in
| passing and it's like a table saw in the hands of someone who
| doesn't know what to do.
| geodel wrote:
| Netflix is large company with huge IT/software workforce. And
| as I have seem in large companies tech managers and engineers
| don't usually keep waiting for actual problem to appear and
| develop solution for it. They are going to take initiative to
| migrate _legacy monolith_ to next generation micro services,
| GraphQL architecture and in process save their jobs.
| x86x87 wrote:
| The timing of this is hilarious. Here is something that's on
| the front page at the same time:
| https://news.ycombinator.com/item?id=36380711
| downWidOutaFite wrote:
| My main issue with it is client-side caching doesn't map
| cleanly to graphql. I want well-defined objects to cache.
| gareve wrote:
| that's solved with Relay's graphstore
| sibeliuss wrote:
| It's 2023. There are _so many tools and patterns now_.
| x86x87 wrote:
| [flagged]
| devjab wrote:
| Have you tried the alternatives? We went with Odata, and it's
| hell. It's fine on the client side, but it's terrible to work
| with on the server-side. We went down the C# path because
| everyone else is either using RPC or GraphQL, but even
| Microsofts own libaries don't work together. The model creatirs
| for Odata and Entity Framework sort of work the same, but are
| also different enough to cause some real annoyances and once
| you add asp.versioning it becomes even worse. We went with
| Odata because we had a hefty client that consumed Odata through
| a client library that we build ourselves, and at the time it
| seemed like it would be fine. A year later it turns out that it
| would've been better for us to rewrite everything to work with
| GraphQL, and, we may actually end up doing that eventually
| because the Odata .NET support is so terrible that we're
| constantly having to rewrite parts of it to get it do things
| that are trivial in GraphQL.
|
| You can of course go other routes, we took a serius look at
| tRPC, but it seemed like it would end up being a lot of work.
| Actually quite similar to how doing the things you want from a
| frontend perspective is a lot of work if your backend is "just"
| rest.
|
| > Last, I don't understand what Netflix does that is so complex
| that would warrant something like gql. I just don't.
|
| If you dig into the other blog points that the OP link to you
| can see that it's because they needed better performance, more
| security and better control of the flow. I think you may
| underestimate how much you get for "free" when using graphQL.
| It has so much adoption, that there are so many tools that you
| won't have to build from scracth, even if you're Netflix. You
| can see our Odata library as an example of this, there are
| client libraries that we could've used, but they're bad. So we
| had to build our own. With GraphQL we wouldn't need to do that.
| x86x87 wrote:
| IMHO, Rest works just fine. If you are serious about
| performance I don't really get how you do that with GQL.
| maybe I'm just ignorant but the apps I've worked on were
| medium to large size and it was a fricking nightmare (apollo-
| based, a couple of years back).
|
| Here:
|
| https://hackerone.com/hacktivity?querystring=graphql&filter=.
| ..
| valenterry wrote:
| I think it's simple. You can get the best performance with
| REST, but you have to really put lots of resources into it.
|
| On the other hand, with constraint resources, you likely
| end up with better performance when using graphql because
| of the reduction of requests and decreased latency.
|
| In most cases, graphql hits the sweet spot.
| buro9 wrote:
| > I never understood why people go for graphql.
|
| It allows a full separation of the frontend dev velocity from
| the backend velocity.
|
| The frontend is no longer dependant on backend engineers for
| API creation, nor negotiations on how those should look.
|
| The backend can choose to observe the hot path of GraphQL and
| optimise those for your #3, but #1 and #2 don't differ from,
| say, REST.
| physicsguy wrote:
| Except when it's not performant, and then suddenly you need
| those backend guys again!
| x86x87 wrote:
| So you're solving an organizational problem by using gql? Who
| needs proper authz/authn, security, scalability, sanity, qos
| if we can have frontend dev velocity? Checks out.
| pests wrote:
| Don't we solve organizational problems with software all
| the time?
|
| Git Monorepo vs Polyrepo Microservices vs Monolith
| Kubernetes
| dasil003 wrote:
| > _So you 're solving an organizational problem by using
| gql?_
|
| Yes, of course! All problems are organizational problems at
| scale. I have no great love for gql, but painting it as a
| black and white decision pitting FE velocity against all
| other engineering concerns (including hyperbole like
| "sanity") is childish. There are legit tradeoffs
| discussions to be had, but if you think it's impossible to
| secure or scale gql then you're not a very imaginative or
| resourceful engineer.
| seanp2k2 wrote:
| GraphQL really needs a gateway like Envoy to give better
| observability / control / auth, and indeed that's how lots
| of people deploy it.
| politelemon wrote:
| Is it not possible without Envoy? IMO this feels like a
| failing if I'm relying on a separate piece of software.
| I'm comparing it with RESTish services which are easily
| scaleable in many ways either software or infra. But that
| might be my misunderstanding of what you just said
| x86x87 wrote:
| Okay. Tell me how you do the control/auth and allow
| certain users to access a resource while others do not
| access it while they may formulate complex queries where
| they ask for the kitchen sink.
| pravus wrote:
| I honestly don't understand the complaints here. In our
| deployment it's implemented as an HTTP handler right next
| to all the REST ones and gets all of the standard
| authz/authn/obervability wrappers. It's allowed us to
| focus more on core development because the schema was
| exported over a year ago and hasn't changed much. It
| functions almost identically to the REST components but
| is easier to develop against. No issues with scale or
| performance that aren't apparent in the REST components
| either.
| tomtheelder wrote:
| Honestly there's a few good ways to do that. Schema
| directives, regular old permission system at the resolver
| level, etc.
|
| I'm not a big fan of GraphQL, but I it's not particularly
| difficult to secure.
| andrewingram wrote:
| By wrapping the code that loads data with code that
| checks whether the current (i.e. requesting) user is
| allowed to load that data. I don't really understand the
| problem, how is this any more difficult than any other
| API paradigm?
| foota wrote:
| I think the additional issues come when you want to
| define access by a relationship instead of just the
| entity, which means you need to check access based on the
| edges, and not the nodes. You'd have to do this anyway
| with REST, but I think it's easier in some ways to make
| mistakes and allow more access than intended.
| eddythompson80 wrote:
| Solving organizational problems is a core component of any
| software you design in any organization that's not a 100%
| flat startup unfortunately.
|
| > "Any organization that designs a system (defined broadly)
| will produce a design whose structure is a copy of the
| organization's communication structure." - Melvin E. Conway
|
| It's unavoidable. You can bury your head in the sand about
| it and demand that the technical structure only match your
| technical requirement, but then the organizational
| structure of the company will grind to a hault. People will
| always be people. Just because you give some of them a
| title like "Engineer" doesn't mean that we won't conflict
| on people problems.
|
| Entire concepts like event driven architectures or
| microservices, while might have plethora of technical
| pros/cons are mainly pushed by management for their non-
| technical aspects. One of their main non-technical pro is
| organizational. If your component just emits events,
| handles events, and is implemented as a separate
| microservice then you can always just insert a team, or
| move a team in an organizational structure while minimizing
| the overall impact to a project.
| jameshart wrote:
| > frontend is no longer dependant on backend engineers for
| API creation
|
| So long as the backend engineers at least anticipated all the
| relationships, properties and objects that the frontend might
| ever need?
| dan-robertson wrote:
| I feel like this is a bad take: I'm not sure what your
| experience with graphql is but your comments about Netflix
| strike me as naive. I think there is a simplified version of
| Netflix that doesn't do eg recommendation or history or
| different contracts in different countries and so on, which is
| maybe just a few api calls, but I don't think that's what
| Netflix is. And are they actually reinventing the wheel here? I
| thought graphql was a Facebook thing. Maybe you meant to say
| that they are rewriting systems pointlessly but it (also) feels
| like a bit of a generic criticism that is easy to apply to
| companies for which in-house systems make sense.
|
| (Edit: it's not even like this is a graphql vs rest change as
| Netflix have had graphql for years and they have a custom api
| system that seems a lot more like graphql than it does rest)
|
| I'm curious what you think the best advantages of graphql are,
| particularly for a larger company like Netflix?
|
| I would guess:
|
| - reduces coordination/contention between different
| frontend/backend teams (eg there are many clients - desktop,
| mobile, various smart devices - but maybe other front ends too
| like something customer support might use).
|
| - cuts down on 'boilerplate' api work. For example, maybe you
| have some property X of each item Y in your catalog, but which
| you don't get on the request for the catalog, and that you want
| to display this now (maybe it's some new experimental feature
| you need to thread through from some database table to the
| front end). You need to choose between adding X to the catalog
| (potential version woes...) or managing a bunch of requests for
| X for each Y (risks being slow or bad in a few ways), or add
| some non-restful api endpoint that takes a list of Ys and gives
| you their X properties. With graphql you need to teach the
| backend how to get X of Y but the frontend only needs to change
| the query.
|
| - it potentially lets you apply access controls in a more
| straightforward way. If someone is from the US, maybe there are
| some shows they can't see. In that case we probably want to
| hide all records in results about shows they can't see.
| (Alternative example: various properties of other people's
| accounts). If you have many separate api endpoints, it feels
| easier to me to accidentally screw up on one of them.
|
| I'm not really very familiar with graphql but the middle option
| is the most compelling one for some of the things I work on
| (which are nothing like Netflix). One assumes that some people
| from Netflix had reasons for graphql and I'm interested in what
| they were rather than some imagined Venn diagram with the
| current api in the 'not invented here' area and graphql in the
| intersection of 'invented here' and 'not yet a thing we have'.
| I'd also be curious to see how well it works out for them over
| time - perhaps they'll regret it but maybe not for the reasons
| you oppose graphql; if there are organisational problems (like
| some team becoming a big new bottleneck or some teams being
| unhappy about it being 'forced on them'), they can be hard to
| write about.
| pmoriarty wrote:
| GraphQL is such complete and utter garbage for anything but the
| simplest of queries. I pity anyone who has to use it for
| anything more complicated.
| x86x87 wrote:
| i would not go as far as saying complete and utter garbage,
| but it 95% of cases it brings with it a can of worms that you
| don't need and don't want. All in the name of "frontend
| developer velocity"!
| hn_throwaway_99 wrote:
| Going to reply here because, as a huge fan of GraphQL, I
| strongly disagree with every single one of your points. To
| start, I think the biggest misconception I see about the value
| of GraphQL is people see it as some sort of generic "query
| language", which is unsurprising, given the name. If anything I
| think the biggest problem with GraphQL is that, if people just
| saw it is a simple alternative to REST, with a well defined set
| of query (GET) and mutation (PUT/POST/DELETE) endpoints, they
| wouldn't get caught up in all this "infinite object graph"
| stuff that I never hit in my day-to-day.
|
| To your points:
|
| 1. I find GraphQL to be _fantastic_ for security, primarily
| because all calls are strongly typed, so you can ensure by the
| time a request gets to your resolver it is guaranteed to match
| the defined input types. Furthermore, I always make heavy use
| of custom scalar types instead of String, which allows me to
| get rid of a whole host of potential security issues by
| limiting potential injection attacks before my code even sees
| it. As far as ensuring object ownership /permissions, this is
| trivially easy, or in any case just as easy as with REST.
|
| 2. Versioning is one of GraphQL's greatest strengths. The
| tooling makes it easy to deprecate some fields while adding
| replacement ones in a way that guarantees backwards
| compatibility. Far from "good luck figuring out what is used",
| the tooling e.g. in Apollo makes it very easy to see which
| clients are making use of which fields. Furthermore, if you do
| need to "hard version" a new endpoint, you can do it similarly
| to REST by including a version parameter in your URL (e.g.
| /v1/myGraphQLEndpoint).
|
| 3. Regarding qos, this again just gets to my original point
| about having well defined queries and mutations. Your queries
| don't need to say "allow me to see the world" any more than any
| other API framework. It's not hard to just have queries that
| only say "give me X by ID" and also a single bulk retrieval
| endpoint where you define the boundaries of what you want to
| return.
|
| If anything, I think perhaps a lot of misconceptions around the
| problems of GraphQL have to do with these tools that basically
| say "expose my whole DB with GraphQL", which in my experience
| usually leads to tears in the end.
|
| But starting with a thoughtful definition of your API surface
| area with GraphQL's type definition language can make it a joy
| to use and provides lots of benefits over a straight RESTful
| interface.
| politelemon wrote:
| If I'm understanding correctly, there is a lot more work a
| team has to do to build and maintain a GraphQL API and should
| only do it if they understand what they're getting into and
| what its advantages are. It's not for everyone, and everyone
| shouldn't assume it's for them.
| hn_throwaway_99 wrote:
| > there is a lot more work a team has to do to build and
| maintain a GraphQL API
|
| I wouldn't say that. It's simple and straightforward to get
| around some pitfalls of GraphQL, if you treat it as an
| enumerated set of RPC-style endpoints.
|
| The commenter I was replying to stated "Thr fact that
| you're arguing that it should not be treated as a generic
| query language when it's in the name and this is how it's
| sold is hilaruous". Perhaps so. _By far_ the biggest
| mistake I think the original designers of GraphQL made was
| putting "QL" in the name, as many people think it's in the
| same category of tools as SQL (I heard someone ask once on
| HN "Can you do joins in GraphQL?", which isn't even a
| question that makes sense). It should be treated as a
| competitor to OpenAPI, and there are reasons why I would
| prefer using GraphQL in many scenarios.
|
| I think I'll probably write some longer form blog posts at
| some point about how I think GraphQL is easy to set up and
| get running quickly, and how to avoid some common
| misconceptions.
| troupo wrote:
| > It's simple and straightforward to get around some
| pitfalls of GraphQL
|
| It isn't
|
| > if you treat it as an enumerated set of RPC-style
| endpoints.
|
| It's not that.
|
| > I think the original designers of GraphQL made was
| putting "QL" in the name, as many people think it's in
| the same category of tools as SQL
|
| "GraphQL is a query language for APIs" is literally the
| tagline at https://graphql.org/
|
| And the frankly insane complexity of GraphQL comes from
| the fact that you now have to implement that query
| language, often over completely different sources of
| data.
|
| Which leads to frankly inane solutions like the
| requirement to inspect the full request _and_ the full
| response on the fly just to be able to cache responses.
| [deleted]
| x86x87 wrote:
| [flagged]
| runeks wrote:
| > Falling back to arguing about typing show me that maybe
| you are not aware of the many interseting ways you can get
| screwed or maybe you just worked on simple straightforward
| things.
|
| Please provide a few examples of this.
| x86x87 wrote:
| Since I'm not going to type a novel here is an example: h
| ttps://wundergraph.com/blog/the_complete_graphql_security
| _g...
|
| Also look at hackerone:
|
| https://hackerone.com/hacktivity?querystring=graphql
| zeroxfe wrote:
| > So in conclusion, you're moving the goalposts and
| asserting that everyone that has issues with the tech must
| be an idiot since it works for you.
|
| That's not what GP asserted at all. You made some vague
| statements about gql, and they refuted with very concrete
| arguments.
| hn_throwaway_99 wrote:
| I didn't call anyone else an idiot. You don't actually seem
| to be open to hear about how people can use GraphQL, in
| production, to solve a host of thorny issues that "plain
| REST" APIs commonly have to deal with. Congrats on your
| intellectual superiority.
| dcre wrote:
| This is helpful, but it confirms my sense that a REST API
| with a typed spec like OpenAPI is going to get you basically
| all the benefits of GraphQL.
| magic_quotes wrote:
| Pretty much, they are fundamentally both RPC flavors, so
| it's impossible for them to have different properties
| beyond more or less convenient tooling. It's always jarring
| to see those heated GraphQL vs REST APIs (as implemented in
| practice) vs gRPC discussions as if they are as different
| as, say, client-side CRDTs.
| jakubmazanec wrote:
| > if people just saw it is a simple alternative to REST, with
| a well defined set of query (GET) and mutation
| (PUT/POST/DELETE) endpoints, they wouldn't get caught up in
| all this "infinite object graph" stuff that I never hit in my
| day-to-day.
|
| Can you explain this more? How can you avoid infinite graphs?
| If I have User {things: Thing[]} and Thing {owner: User}, you
| have to deal with this issue.
| vinnymac wrote:
| I have been writing GraphQL APIs on and off for the last 5
| years, and in practice haven't had a scenario where this
| was an issue.
|
| In GraphQL the query for the example you gave could look
| like this
|
| ```gql query { user { things { owner { id } } } } ```
|
| When resolving an array of type `Thing` for the `things`
| field, you would query for the user that the `owner` field
| represents. Rather than assuming all relations should be
| loaded for an owner by default, the client would specify
| the fields for the `owner` that it needs. Such as I have
| above for the owner id. Even if no fields are specified, it
| would be weird to assume all relations should be loaded by
| default.
|
| Now if your developers are intentionally creating
| infinitely deep queries, then you'd solve this the same way
| you'd solve an infinite loop in your code today. An issue
| you would catch during development or CI/CD. This can be
| easy to catch using a linter or during review.
|
| ```gql query { user { things { owner { things { owner {
| things { } # etc } } } } } } ```
|
| To simply protect from bad parties / developers, you might
| use something like https://github.com/slicknode/graphql-
| query-complexity
|
| In addition you could introduce CI tools to enforce your
| devs stop writing such complex queries. Also see the @skip
| and @include directives that can further be used to control
| what data is queried. In practice, however, this isn't
| something that comes up too much. In cases where I have
| seen this happen, it's usually because a developer is
| trying to reuse fragments without considering what data
| they are querying, and whether they should be reusing those
| fragments.
|
| https://graphql.org/learn/queries/#fragments
| Philip-J-Fry wrote:
| >If anything I think the biggest problem with GraphQL is
| that, if people just saw it is a simple alternative to REST,
| with a well defined set of query (GET) and mutation
| (PUT/POST/DELETE) endpoints, they wouldn't get caught up in
| all this "infinite object graph" stuff that I never hit in my
| day-to-day.
|
| But isn't this "infinite object graph" like the only real
| valuable part of GraphQL? People get caught up on that
| because all the other things you mention aren't really an
| issue with normal REST API.
|
| If you're saying you use GraphQL but don't use the key
| defining feature of GraphQL. Then what's the point of using
| GraphQL if you're just using a strict predefined set of
| things you can query/mutate?
| scotty79 wrote:
| Value of GraphQL imho is the ability for the api to be
| driven by frontend requirements without bothering the
| backend guys "to expose that one field in that one query".
|
| Once the app is developed I think you might put up
| reasonable general limits to what queries are responded to
| in a general manner over the whole api.
| Hippocrates wrote:
| This is how I have interpreted its value, and frontend
| devs have very plainly advocated for its use with this
| argument. I think this is a really silly value prop.
|
| Behind the elegance of a query language that lets you get
| your perfect little payload, there is a complex server
| that could need to resolve and block on requests to
| multiple backends, touch different caches and databases,
| and do joins and sorts and projections. Front end devs
| will add the videoRating field and have no idea that this
| comes from a separate API and causes the response time to
| double.
|
| Developing some gragantuan do-it-all meta API as a
| sandbox for frontend engineers is a terrible idea. You
| don't just slap a graph over your APIs and call it done.
| The GraphQL server is orders of magnitude more complex
| than a purpose-built API.
|
| Ultimately once the FE devs have crafted their query to
| satisfaction it will hardly ever change. It's better to
| work together and decide whats needed and how to best get
| that data to your app.
|
| I really favor collaboration on purpose built APIs than
| gargantuan do-it-all systems.
| swyx wrote:
| > Ultimately once the FE devs have crafted their query to
| satisfaction it will hardly ever change
|
| this makes the fallacy that it is FE devs driving change.
| it is not. it is product management. and to them (and
| indirectly to you), making it easy to make performant
| changes without 3 committee meetings is a Good Thing
| jameshart wrote:
| So this completely contradicts your GP post who says the
| fact that it supports generic queries is _not_ the point
| of GraphQL.
|
| So which is it?
| OJFord wrote:
| So you move the issue to the resolver instead? IMO
| graphql is trying to allow the frontend to be the
| 'driver', the first bit; otherwise you just have the
| backend publish an API and then that is what it is, if
| that one field is available from a different endpoint
| than the one you're currently using, that's the one you
| need to hit (as well), tough.
|
| Not to say either way around is inherently better, I just
| see it as a switch from backend-driven to frontend-
| driven.
|
| (Personally I'd love to use something like OpenAPI, with
| tooling more mature than probably exists, to be properly
| _schema_ -driven, with both backend and frontend
| stubs/clients derived from that.)
| scotty79 wrote:
| > So you move the issue to the resolver instead?
|
| Yes. Where it can be logged, inspected and dealt with
| wholesale.
|
| It's moveing a problem from intra-team verbal
| communication into software where it can be dealt with
| more efficiently with software tools and methods.
|
| > Not to say either way around is inherently better, I
| just see it as a switch from backend-driven to frontend-
| driven.
|
| Yes. I think that's the whole point and main benefit of
| GraphQL which makes a lot of sense since apps that evolve
| through their development usually have most changes UI
| driven.
| OJFord wrote:
| > Where it can be logged, inspected and dealt with
| wholesale.
|
| Just as it could be in the backend? I don't follow that
| point.
|
| > apps that evolve through their development usually have
| most changes UI driven.
|
| That's a fair point certainly, but isn't completely
| generalisable - if you offer a third-party API at all (or
| think you might) then maybe don't special-case your own
| frontend.
| 8note wrote:
| How does http2 skip the speed of light to speed up sequential
| requests?
|
| Isn't the back and forth across the network the big problem?
| Rapzid wrote:
| I hate integrating with public GraphQL APIs as well. Give me a
| REST API pls for the love of god. It makes me sad GitHub's new
| projects API is GraphQL-Only.
| bbbbzzz1 wrote:
| In Apollo server you can add directives to your scehma for
| authz. It's dead simple.
|
| I'm surprised how so many HN posts about graphql just slander
| it. If you go to the website and look at the use case of why it
| was made, it makes it pretty clear when it could be a good to
| use. My experience with it has been very straight forward and a
| joy
| withinboredom wrote:
| GraphQL (as in the language spec) offers no way to perform
| authz. Just because there are non-standardized extensions out
| there for some languages, doesn't mean it is available in
| whatever language you're using.
| tomtheelder wrote:
| Ok, but that's true whether or not you use GQL.
| withinboredom wrote:
| Sure. I could respond with a 4xx code because you're not
| authorized for that field. Ah, but maybe you want to know
| which field? Too bad GQL doesn't really support that.
| cameronh90 wrote:
| Authorization errors can go in the errors array which has
| a path reference in the standard.
|
| You generally don't want to use HTTP error codes with
| field level errors in GQL.
| x86x87 wrote:
| [flagged]
| theturtletalks wrote:
| My SaaS backend is a GraphQL Server that has single-
| handedly saved me tons of hours for integrations and
| coding. The first iteration of the app was using rest and
| that was a pain. Hitting multiple endpoints to get all the
| details of an order is not something I miss. On top of
| that, I get a beautiful playground that acts as a query
| builder and documentation.
|
| Have you actually used GraphQL in production or for a wide-
| scale project? A lot of these criticisms for GraphQL comes
| from people who have played with it for an hour and
| determined it's not good. Give it a real chance and you'll
| see why big companies are using it and understand why the
| ecosystem is growing so fast.
| bbbbzzz1 wrote:
| Feel free to read how to do it
|
| https://www.apollographql.com/docs/apollo-
| server/schema/dire...
| xwowsersx wrote:
| Kong seems to have some offerings on the API gateway security
| front for GraphQL https://konghq.com/blog/engineering/graphql-
| authorization-at... I haven't looked at it very closely yet,
| but I'm wondering Kong for other reasons (unrelated to GQL)
| cmgriffing wrote:
| I think the important thing for people to recognize is that
| Falcor, the tool used in their mobile apps, was already GraphQL-
| like. So, this isn't a giant change.
| jFriedensreich wrote:
| This is a sad but expected day. For years i seem to have been the
| last lonely falcor user who is still excited about this and
| thinks this is one of the best abstractions for a key class of
| state management applications, that are not a great fit to
| graphQL at all.
|
| - Falcor can be schemaless with optional json schema where it
| makes sense
|
| - Falcor supports simpler to reason about and implement features
| that map perfectly to javascript objects so you can do zero
| boilerplate databinding with ES proxies without writing query and
| mutation wrappers
|
| - Does not try to be a qyery language but only expose things you
| need to request the data that needs to be rendered in the UI
|
| I think not even the creators of falcor fully understood how
| perfect it is if you just add in the last missing piece of fully
| automating the connection from the falcor model to the place in
| the UI templates where the data is needed. Most people tried
| doing this manually in component boilderplate which lead
| initially to preloading tens of json paths eg
| falcor.preload([`todos.done.length`,`todos.done[0..10].name`,
| `todos.done[0..10].date`, ...etc.])
|
| and then people developing GraphQl like syntax for these eg.
| somthing like load(` todos { done
| 0...10 {name, data} } etc. `)
|
| at which point devs thought they might as well be using graphQL
| and get more features
|
| The key mistake was that you need to instead think about your UI
| components ITSELF as the data quueries eg you can automatically
| derive all falcor quieries from a component like this without ANY
| boilerplate, and this is only possible with falcor, as everything
| else has so many features that it would not be possible to map
| the UI code to the intended data queries {#each
| todos as todo} <div>{todo.name} ({todo.date})</div>
| {/each}
|
| It does not look like netflix will hand over falcor to a
| community and the codebase needs lots of work especially around
| tooling and overuse of some patterns like a custom observable
| implementation but i am still on the fence about building a new
| minimal derived library or forking
| CSDude wrote:
| Maybe now GraphQL api can stop auto completing movies that does
| not exist in their catalog.
| paulddraper wrote:
| ?
|
| It autocompletes so you find out more quickly that Netflix
| doesn't carry it, and what similar options are
|
| I get that you are annoyed Netflix didn't have the movie you
| wanted, but autocomplete doesn't make that worse.
| 9dev wrote:
| You're right, but it's annoying for sure. Instead of telling
| me they don't have a title, they complete its full name (as
| if to say ,,I know what you want, but I'm not gonna give it
| to ya"), yet show me some arbitrary stuff named similar.
|
| It's just a poor solution.
___________________________________________________________________
(page generated 2023-06-18 23:00 UTC)