[HN Gopher] How should we build the APIs of tomorrow?
___________________________________________________________________
How should we build the APIs of tomorrow?
Author : mooreds
Score : 87 points
Date : 2020-08-31 15:27 UTC (7 hours ago)
(HTM) web link (increment.com)
(TXT) w3m dump (increment.com)
| anderspitman wrote:
| > To work around these complexities, Google built Spanner, a
| database that provides strong consistency with wide replication
| across global scales. Achieved using a custom API called
| TrueTime, Spanner leverages custom hardware that uses GPS, atomic
| clocks, and fiber-optic networking to provide both consistency
| and high availability for data replication.
|
| I find it interesting that most of this complexity simply falls
| away if users host their own data. In my estimation, most
| people's computing needs would best be satisfied with a
| smartphone + a raspberry pi in their house hosting their data,
| protected by a simple auth scheme, and accessed using simple
| protocols built on HTTP. That would be more than enough to access
| all their photos and videos, documents, and social feed for their
| few hundred friends to consume. Things like email would probably
| still best be handled by the one cousin in the family who works
| in IT, to manage spam etc.
|
| If only the technical side were the actual problem.
| [deleted]
| forbiddenvoid wrote:
| Users don't want to host their own data. I think this has borne
| out time and time again where hosted options win out over self-
| hosting repeatedly.
| zimbatm wrote:
| Users want convenience. Hosted is usually more convenient to
| use (at least in the short term) as all the maintenance is
| taken care of transparently.
| anderspitman wrote:
| I think users don't like hosting their data using poor
| solutions. A lot of folks seem to be happy with products like
| Synology that just work.
|
| Self hosting leans toward simpler (and thus more reliable)
| software, lower latencies, and higher speeds, due to improved
| locality.
| nine_k wrote:
| This depends on the user; tech-savvy users may prefer a self-
| hosted version, especially if it installs in a few clicks.
| But they are outnumbered by IT-naive users whose only
| realistic option is the hosting by a vendor.
| rhn_mk1 wrote:
| Seems it's about client-server APIs, not API design in general.
| ChrisMarshallNY wrote:
| Yup. It's an important topic, but I've been designing APIs for
| decades; just not the kind he talks about. I do things like
| device control APIs, these days.
|
| The ones that I do have some issues that he doesn't cover, and
| a whole lot of issues that he discusses are of no concern to
| me.
| jayd16 wrote:
| Enjoyable read but not much here. Strange that there's no concept
| of edge computing or caching referenced in the article, from what
| I could tell.
| alonsonic wrote:
| Agree, the author doesn't paint the picture of what might be
| coming. Would love to hear from the HN crowd on what's coming
| next in API design. I see a lot of buzz around grpc, will this
| be the next standard?
| crispyporkbites wrote:
| It boils down to:
|
| - optimise the amount of data you send (see graphql/this
| persons agent idea)
|
| - optimise where you send it to/from
|
| The latter has a hard limit of c, which we'll always try to
| move towards. Distributed computation helps, but trades off
| speed/consistency. The question then becomes whether you can
| have an inconsistent model for those few seconds.
| rumanator wrote:
| It's a pretty good article but It's odd that auth wasn't
| mentioned in the article. If chatty clients are a concern then I
| would expect that token-based auth to be bundled with the
| problem.
| [deleted]
| AriaMinaei wrote:
| A few ideas to put out there:
|
| 1. Emphasize synchronization over imperative API calls.
| _Imperative APIs encourage data silos._ They are the underlying
| technical part of the problem that Zapier and ITTT try to solve.
| See [0] and [1] for some ideas.
|
| 2. Allow users to submit "agents" rather than "requests." An
| agent is a small program given extremely limited resources
| (limited process/memory/runtime budget) that you can safely run
| on your server. It is a possible answer to many of the
| standards/protocols that wouldn't exist if frontends were running
| on the same machines as backends. [2]
|
| 3. Emphasize composition over integration. Functions compose
| (nest, recurse). Event emitters don't. As long as APIs are built
| to be integrated rather than composed, making them work together
| is a full-time or full-company job (eg. Zapier).
|
| 4. Make things immutable. Immutability allows APIs to "play" with
| one another without fear of setting off the nukes (ie. side
| effects). It's possible that this approach would make it so that
| integrating two APIs becomes a job for ML/AI rather than humans.
|
| [0] https://github.com/braid-work/braid-spec
|
| [1] https://writings.quilt.org/2014/05/12/distributed-systems-
| an...
|
| [2] https://news.ycombinator.com/item?id=23900749
| humanfromearth wrote:
| While these ideas (imperative vs sync, immutability +
| idempotency) a great in theory, I have yet to see anyone
| putting them in practice while still keeping a great developer
| experience - that includes Zapier. I guess Stripe is doing this
| more or less well?
|
| I just wish people implemented their webhook systems well (a
| popular standard would be nice as well) - REST is fine, it
| mostly works and it's pretty standard.
| AriaMinaei wrote:
| Synchronization is still an open problem. CRDTs for example
| are still evolving fast [0]. I'd imagine the idea would be
| more doable when the underlying sync primitives mature.
|
| Immutability is a different beast, but it appears that
| decentralization/p2p is working on it through necessity.
|
| [0] https://crdt.tech
| unilynx wrote:
| This is also something that lacks in openapi - a standard way
| to describe websockets/SSE would be nice. Being able to
| listen for remote changes cuts down on API calls and leads to
| more responsive changes.
|
| (and ideally the webhook and websocket pushes and websockets
| event pulls look as much alike as possible, so you can switch
| between them easily as the situation warrants)
| stingraycharles wrote:
| > Allow users to submit "agents" rather than "requests." An
| agent is a small program given extremely limited resources
| (limited process/memory/runtime budget) that you can safely run
| on your server. It is a possible answer to many of the
| standards/protocols that wouldn't exist if frontends were
| running on the same machines as backends.
|
| Forgive my ignorance, but how would this work in practice?
| Would agents be arbitrary code that a client would submit to an
| API, or would the agent be something that the server itself
| implements and manages in the backend?
|
| EDIT: after some further reading of the mentioned threads, it
| appears to me like the idea is that we ship data with their own
| interpreters of some kind? I can help but fear this will cause
| even more JavaScript code to be pushed to the server. :)
| AriaMinaei wrote:
| I think at a minimum, agents are programs that clients submit
| to the server. The server runs the agent with a very limited
| API and likely as a pure function (lang could be JS or wasm
| maybe). The agent could run in response to events on the
| server (like webhooks), or in response to requests coming in
| from the client.
|
| Here is a scribble: https://gist.github.com/AriaMinaei/69e1a9
| 166e7ffbd61e7f6709d...
| cortesoft wrote:
| I agree things that can be immutable should be, but so much
| functionality requires mutability. State has to change, or the
| internet will be a very boring place. If I am shopping, I need
| to be able to buy something... Facebook users need to be able
| to post their updates, upload photos,etc.
|
| Am I missing something, or do all of these use cases preclude
| immutability?
| [deleted]
| AriaMinaei wrote:
| I think it's about delaying the side-effect until
| `transaction.commit()`, and implementing as much of the
| business-logic as possible to happen inside the transaction.
|
| Virtual worlds (a model of the real world) could also be
| useful. Like, let the side effect run in the virtual world
| and let me inspect that virtual world. Here, "me" could be a
| programmer figuring out how to code against your API, or some
| ML thing trying to figure out how to make two APIs talk to
| one another.
| neeleshs wrote:
| As a founder of a startup whose Mantra is sync, I couldn't
| agree more. No API developer looks at API design from this
| angle. In fact, in a few cases, the design actively prohibited
| these use cases (for obvious reasons).
|
| Another aspect is intimating error conditions. One would be
| amazed on the myriad different ways an API fails and many do a
| poor job of conveying what went wrong, and how to recover, if
| possible.
| AriaMinaei wrote:
| I'm curious, what startup that is? FWIW, I think there is a
| large opportunity here (make sync+collab easy => ecosystem of
| products that tightly sync) that can be disruptive to many
| SaaS businesses.
|
| I'm keeping an eye on Croquet: https://www.croquet.io
| MichaelApproved wrote:
| As an end user of APIs, agents sound like a good idea.
|
| As a creator of APIs, agents seem scary from an
| optimization/caching point of view.
|
| I'd rather serve the same request 10,000 (which could be
| cached) than have 1,000 different agents making requests (which
| would be hard to cache).
| carry_bit wrote:
| Is it really much different? In the data flow you're just
| conceptually moving the network from being after the agent to
| being before the agent. You can still have the cache in the
| same spot, it's just that the agent is hitting it directly
| now instead of over the network.
| tux1968 wrote:
| As far as the actual data access goes, it is equivalent,
| but you can't cache the agent process itself. So every
| agent is potentially repeating work centrally, rather than
| spreading that load out to each client. That may still be a
| worthwhile win, but it is a new burden on the data centre.
| AriaMinaei wrote:
| Maybe agents could create rest endpoints of their own
| with its own cache?
|
| Edit: Just wrote some pseudo code: https://gist.github.co
| m/AriaMinaei/69e1a9166e7ffbd61e7f6709d...
| nine_k wrote:
| Agents should use a limited, non-Turing-complete model of
| evaluation. They could combine API calls locally and run
| simple FSMs which are easy to formally check for e.g. absence
| if loops. This could save round-trips without needing to
| sacrifice general orthogonal APIs.
|
| Imagine that you could give an app server a _formula_ to
| combine several API calls, much like you give an SQL server a
| formula to join and filter tables.
|
| Also, you can easily cache agents by calculating a hash of
| their source, and call them repeatedly without resending
| their bodies, much like "prepared statements" work in SQL
| databases.
| chabad360 wrote:
| You sound like your describing (to an extent) GraphQL
| rumanator wrote:
| > Agents should use a limited, non-Turing-complete model of
| evaluation.
|
| That doesn't solve any of the caching worries. If it's not
| a resource then it's not cacheable.
|
| > This could save round-trips without needing to sacrifice
| general orthogonal APIs.
|
| Since the inception of HTTP2, a relative large number of
| requests of cached resources easily beat a relatively small
| number of requests that are computationally demanding or
| hit a database.
| AriaMinaei wrote:
| I agree. Though not sure about the non-Turing-complete-
| ness. How about each agent has a budget. Agents that blow
| the budget get dropped.
|
| Like this: https://gist.github.com/AriaMinaei/69e1a9166e7ff
| bd61e7f6709d...
| xcambar wrote:
| At the risk of being considered non-constructive, I wish the
| author made a leap of faith and shared their educated guess
| regarding where the API engineering is moving to.
|
| I appreciate the effort of sharing a perspective, but I would
| have warmly welcome a bit of prospective.
|
| That being said, nice prose and interesting article nonetheless.
___________________________________________________________________
(page generated 2020-08-31 23:01 UTC)