[HN Gopher] Amazon Ion
___________________________________________________________________
Amazon Ion
Author : BerislavLopac
Score : 93 points
Date : 2020-07-22 21:09 UTC (1 hours ago)
(HTM) web link (amzn.github.io)
(TXT) w3m dump (amzn.github.io)
| gavinray wrote:
| I think that talking about Ion, without talking about PartiQL, is
| not setting people up with proper context.
|
| PartiQL is AWS's specification for a parser/query language that
| is compatible with standard SQL, but can query semi-structured or
| unstructured data (think JSON, Parquet, CSV/TSV etc)
|
| https://aws.amazon.com/blogs/opensource/announcing-partiql-o...
|
| PartiQL uses Ion as it's backbone and data format:
|
| https://partiql.org/faqs.html#why-do-you-choose-ion-to-exten...
|
| https://github.com/partiql/partiql-lang-kotlin/blob/master/e...
|
| I looked pretty deeply into this, but failed a bit short of
| understanding what they meant when "if your query engine supports
| PartiQL." Does that mean writing a new DB that delegates incoming
| queries to PartiQL? Not sure.
|
| Anyways, they use it in Quantum Ledger DB, and a few other
| internal projects:
|
| https://docs.aws.amazon.com/qldb/latest/developerguide/ql-re...
|
| So maybe that can give some more context around "what the hell is
| this, why does it exist, how would you use it?"
| whoevercares wrote:
| Yeah It's more like a spec, they expect you to write the query
| optimizer, analyzer basically a new query engine. They offered
| a document based "reference" implementation on Kotlin though so
| I think expert could follow it.
| willvarfar wrote:
| How does this compare to protobuf, thrift, msgpack etc?
|
| It's roughly the same vintage as protobuf and thrift, from google
| and Facebook respectively, so perhaps it's just Amazon's
| equivalent, which they just never released as quick as the others
| did?
|
| Obvious pros and cons, or yet another serialization format with
| no obvious benefits over anything else?
| habitue wrote:
| Just from reading their page and being familiar with the
| formats you mentioned:
|
| vs. protobuf: ion is self describing, vs needing a schema
|
| vs. thrift: similar, thrift needs a schema to interpret a
| binary file
|
| both thrift and protobuf are really binary formats, though they
| have a canonical textual representation, it's not actually used
| to serialize. Sounds like ion supports serializing as text as a
| first class concept.
|
| vs. msgpack: ion has a corresponding text format, whereas
| msgpack is only binary. Additionally, ion has a symbol type,
| msgpack doesn't.
|
| I think the biggest benefit here is that it's a new chance for
| a format that fixes some of json's rough edges to gain critical
| mass. There's probably nothing ultra special about it that
| hasn't been solved in other formats, but maybe the timing will
| be right and everyone will just adopt it as a json replacement
| (sort of how people just gave up on xml and switch to json
| seemingly overnight). It's impossible to predict stuff like
| that.
|
| Edit: upon noticing that it was released in 2016, it seems less
| likely everyone will jump on the ion bandwagon ...
| Scaevolus wrote:
| Protobuf/Thrift are schema-based serialization systems.
|
| Ion is directly comparable to JSON/MessagePack/BSON/CBOR.
|
| I would expect Ion will have slightly different time/space
| tradeoffs than the other binary schemaless formats.
| b203 wrote:
| There are some painpoints that are being addressed:
|
| 1) timestamp : I have had issues with a round-tripping
| timestamp representation quite a bit 2) decimal : currency is
| denoted in decimal rather than float and shows the Amazon
| retail heritage. This is very useful. 3) symbols : I've had
| cases where symbol table/dictionary would have made big
| difference in serialized size
| geeio wrote:
| It is probably closer to something like avro, but with more
| features and less java focus.
| whoevercares wrote:
| I'll start use it when AWS adopt it :) This is only used in
| retail orgs... the ecosystem is the biggest issue
| NegativeLatency wrote:
| embrace extend extinguish
|
| also
|
| https://xkcd.com/927/
| Scaevolus wrote:
| Previously (2016, 165 comments):
| https://news.ycombinator.com/item?id=11546098
| k__ wrote:
| So it didn't really catch on?
| novok wrote:
| Interesting they don't have a kotlin or swift version. Do their
| iOS clients just communicate with plain json? Are they all
| secretly written in javascript?
| moltar wrote:
| iOS shopping app is a web app and it uses plain JSON.
| roenxi wrote:
| An interesting point - I browse with JavaScript disabled. The
| example at the bottom of the page rendered for me without
| newlines, in a manner that meant the thing rendered in a
| completely unparsable way due to comments like:
| // Field names
|
| This experience has reminded me why JSON is such a great format.
|
| And having a whinge while I'm writing, "superset of JSON" is
| basically false advertising even though it is true; JSONs refusal
| to admit that line breaks are a thing is a major feature. I don't
| care it if it is technically correct and useful to some
| customers, if line breaks matter it is inappropriate to talk
| about a format's relation to JSON because people will get the
| wrong idea. The JSON brand is so strong because it is nigh-
| impossible to get wrong. This format gets screwed up - eg, for
| people who don't like JS.
| skywhopper wrote:
| I think "superset" is a clear relationship. It means "legal
| JSON is legal Ion", just like "legal JSON is legal YAML". I
| don't think it's inappropriate to point that out. In fact, it's
| an excellent feature.
| leetrout wrote:
| It's not listed but there is a Go library
|
| https://github.com/amzn/ion-go
| 101008 wrote:
| Looks nice. I saw that there is no PHP implementation yet. Doing
| it and publish it on Github would give me something, besides a
| "kudos" from Amazon? I am not asking for a position at Amazon,
| but maybe an interview?
| moltar wrote:
| I've published two libraries for Amazon products. They didn't
| care.
| urda wrote:
| PHP is a banned language at Amazon and Amazon subsidiaries, so
| they will not care.
| bowmessage wrote:
| untrue for client libraries, they would care.
|
| https://aws.amazon.com/sdk-for-php/
| ajkjk wrote:
| That's not really how that works. Just don't make it the only
| language you know.
| Godel_unicode wrote:
| As it should be be.
| miohtama wrote:
| What has bugged me a lot with JavaScript that it lacked standard
| presentation of dates and decimals (like money), making it feel
| inferior for application development. Happy to finally seeing
| this addressed on both JavaScript tself and then also in
| serialisation formats.
|
| (Though looks like Ion is not solely targeting JS, but I make an
| assumption it is nice to consume Ion data in frontend)
| imglorp wrote:
| Nope, backend if anything. For example, their new QLDB product
| uses it to get consistent hashing of documents on account of
| Ion being a canonical format.
___________________________________________________________________
(page generated 2020-07-22 23:00 UTC)