[HN Gopher] The Serde Rust Framework
___________________________________________________________________
The Serde Rust Framework
Author : ur-whale
Score : 90 points
Date : 2021-10-14 20:56 UTC (2 hours ago)
(HTM) web link (serde.rs)
(TXT) w3m dump (serde.rs)
| elktea wrote:
| Serde is great. I wish it did XML as well but that's a trickier
| proposition
| pornel wrote:
| Because Serde is a common API for pretty much all serialisation
| formats supported in Rust, it's very easy to try multiple formats
| to choose the best size/speed you need. JSON too big? You can
| make it CBOR or Msgpack with a couple of lines. Want faster? Call
| bincode instead.
| AaronO wrote:
| Whilst not classical de/serialization I wrote serde_v8
| (https://github.com/denoland/serde_v8), an expressive and
| ~maximally efficient bijection between v8 & rust.
|
| It has powered Deno's op-layer since 1.9
| (https://deno.com/blog/v1.9#faster-calls-into-rust-with-serde...)
| and has enabled significant improvements in opcall overhead
| (close to 100x) whilst also simplifying said op-layer.
| apendleton wrote:
| Honestly using serde for language interop is one of my favorite
| things about serde, whether it's "classical de/serialization"
| or not. I've recently had the very-pleasant experience of
| writing some code that needs to pass geospatial data back and
| forth between Python and Rust, and found that the geojson
| crate, even though it's nominally for JSON, actually works with
| other serde-compatible things, including (something I found
| kind of miraculous) Python objects, using the pythonize crate,
| which can walk them with serde visitors. So as long as I can
| get my data into a roughly-geojson-shaped thing on the python
| side, I can consume it on the Rust side, without having to ever
| actually produce json.
| roca wrote:
| serde is one of the best things about Rust in practice.
|
| It is more convenient to use serde to serialize/deserialize to
| some standard format like JSON or YAML than it is to write your
| own half-baked format and serialization/deserialization code ---
| even for the simplest tasks --- so you just stop doing the
| latter. This is a fundamental shift, and very good for your
| software.
|
| "Convenience" here actually covers a lot of ground. The APIs are
| convenient. Boilerplate code is minimal (#[derive(Serialize)]).
| serde's attributes give you lots of control over the
| serialization/deserialization, while still being convenient to
| use. cargo makes importing serde into your project effortless.
| All of these are necessary to achieve that shift away from custom
| formats.
|
| Before Rust I used to write a lot of one-off half-baked formats.
| Pernosco uses Rust and serde and it uses no half-baked formats.
| Everything's JSON, or YAML if it needs to be more human-editable,
| or bincode if it needs to be fast and compact and not human-
| readable or extensible.
| vvanders wrote:
| It's also blazing-fast.
|
| I've done some benchmarking on JSON -> in-memory borrow structs
| and it was pretty darn impressive while being fairly ergonomic
| to use compared to a streaming parser.
| the_duke wrote:
| The only downside is compile time bloat.
|
| Serde generates heaps and heaps of generic code. This all gets
| optimized away to be very efficient, but only once it reaches
| LLVM.
|
| Ever tried working on a crate with hundreds or thousands of
| de/serializable types? Compile times shoot through the roof
| really quickly.
|
| The maintainer of serde also created `miniserde` [1] to tackle
| this problem, which uses dynamic dispatch instead of generics
| and can have 4x compile time improvements.
|
| Due to Rusts lack of orphan instances you really depend on a
| pervasive standard for serialization which is used by all
| libraries though, so the ecosystem is pretty locked in to serde
| by now.
|
| [1] https://github.com/dtolnay/miniserde
| UK-Al05 wrote:
| I think in most ecosystems it's easier to use a prebuilt
| serializer/deserializer than your own format.
| roca wrote:
| Fair enough. I have not found that to be true in C/C++, which
| is my main point of comparison. For very simple cases it's
| generally easier to write a few lines to a text file, maybe
| with some whitespace separators in the lines, than to use an
| "real" format.
|
| For JS and Python, JSON.stringify/parse and json.loads/dumps
| are a step up, but you still end up with an untyped mess with
| no schema validation, which makes them only halfway solutions
| to me. I'm a static typing guy at heart, sue me.
| eptcyka wrote:
| Yea, but with serde, you don't get an untyped mess. You
| either get the type you expected or you get an error.
| dexwiz wrote:
| Yeah I don't know why you would try to write your own
| textbased format. GSON and Jackson have been around for over
| a decade in Java, and equivalents in other languages. Unless
| you are in some perf critical application, a library is
| almost always better for this problem.
| pornel wrote:
| The trick with Serde is that it decodes straight into a
| native Rust struct. You get most of validation for free (it
| also nicely takes advantage of Rust enums with data), and
| struct access is maximally fast.
|
| A generic JSON decoder would give you a dynamic structure
| that can contain anything, and then you'd have to pick it
| apart.
| UK-Al05 wrote:
| Most deserializers in typed languages allow you to
| deserialize straight into structs or typed objects...
|
| Serde just doesn't use reflection to do it.
| Taywee wrote:
| C++ isn't super easy about it, though. The best C++ json
| library (in my opinion), nlohmann's json, still requires
| you to define to_json and from_json for your types to use
| them. It's not too bad as a one-off, but when you have
| dozens of types, it gets really tedious compared to
| Serde, and managing std::variant with it is way more
| annoying than Serde with Rust enums.
| danobi wrote:
| Completely agree. FWIW, I think procedural macros make a huge
| difference for ergonomics. For example, a "similar" library in
| C++ (libcereal) is still a PITA to use and debug.
| xet7 wrote:
| Is there any Rust web framework that does not have problem with
| Slow-Loris ?
|
| https://github.com/SergioBenitez/Rocket/issues/1405
| dbrgn wrote:
| How is that related to serde?
| pornel wrote:
| Because of the way Rust's async works, you can put a timeout on
| anything (e.g. you don't need your http parser support
| timeouts, you can kill it at any await point). So it's only a
| matter of choosing timeout policy for your app. One person's
| DoS attack is another person's long polling API.
|
| Anyway, it has nothing to do with Serde, which doesn't have a
| network component. For Serde you'd typically buffer the input
| first, or use external framing (like line-oriented JSON).
| the_duke wrote:
| None of the current Rust async frameworks will prevent your
| synchronous code from going haywire.
|
| Tokio or async-std timeouts only cancel futures. The
| dedicated spawn_blocking threadpools also do not support
| cancellation.
|
| To prevent such cases you would have to spawn the work into a
| dedicated thread pool and then kill the thread on timeout.
|
| Terminating threads is a very complicated topic though and
| not generally possible, especially without introducing memory
| unsafety.
|
| The only real workaround would be to have the synchronous
| code regularly check for cancellation via an atomic bool or
| similar, and terminate if required.
| eptcyka wrote:
| I think this should be fixed in hyper.
| ttymck wrote:
| This is interesting, and news to me. Your comment suggests
| something about Rust imposes this problem on all web
| frameworks, is that true? If so could you share the root cause?
| xet7 wrote:
| I'm just newbie in Rust. Does some web framework not use
| hyper? I'm trying to find web framework that does not have
| problem with Slow-Loris.
| Ericson2314 wrote:
| I don't think anyone will want to avoid hyper, tbh, that's
| a lot of work to hand-roll instead. so you better advertise
| the issue there instead, and if they refuse to do anything
| about it _then_ make a big stink downstream.
| yazaddaruvala wrote:
| Frankly Serde's continued success has made it basically necessary
| for most Rust development. When the Bazzar organically builds and
| loves a mini-Cathedral it's probably time to upgrade it to
| standard.
|
| The increased complie times are the only real issue with Serde.
|
| Especially if upgrading this crate into std would allow for a way
| to reduce the compile time "penalty" of very common formats like
| JSON, etc.
| paulgb wrote:
| To me, Serde is one of the killer features of Rust. Once you
| realize how much of programming is (for better or worse)
| shuttling data between different representations, it's hard for
| me to program in a language where I don't have a tool like Serde.
|
| A big part of its flexibility is how modular it is. Most common
| Rust libraries support serde serialization (at least as an
| optional feature), so if you use crates that do, you can plug any
| backend in and serialize those data structures to it. It doesn't
| even have to be string-based; I've been using Serde to store
| arbitrary data structures as objects on Google Cloud Firestore.
| [deleted]
| the_gipsy wrote:
| I just came from rust to go and... what a disappointment.
|
| Missing fields default to some kind of default "zero value" - for
| any type, even full blown structs. You can't tell the difference
| of a missing field or the field having the default value.
|
| So if a field has a validation of "must be greater than zero",
| you can't really give a proper error message. If user puts in "0"
| or omits the field, you always get a value of "0".
| X6S1x6Okd1st wrote:
| I've usually seen this handled by having everything be a
| pointer, the default value of the pointer is nil. You can also
| use the SQL values.
| the_gipsy wrote:
| Isn't it insane to change your types to accommodate this? You
| have to change all fields accesses, it behaves differently,
| you lose non-nullability...
| apendleton wrote:
| If you want a field to be able to be not-set, you just need to
| specify that by making it an `Option<_>` type. So like, if the
| field is a number, make it an `Option<u64>` -- that way you can
| distinguish between not-set (`None`) and set to zero
| (`Some(0)`).
|
| Edit: whoops, misread which language it was you were frustrated
| with. Nevermind!
| tylerhou wrote:
| I know you're giving helpful advice, but the author is
| talking (complaining) about Golang, which has no option
| typing (unless you want to use a pointer).
| the_duke wrote:
| You misunderstood, parent is complaining about Go, not about
| Rust. In Go you would have to use a pointer to represent
| Option<T>, which makes code really awkward.
|
| Go serialization really has quite a few issues apart from
| default initialization. Configuration by somewhat weird
| struct tag strings which are only evaluated/validated at
| runtime, de/serialization is all done via reflection (unless
| you want to use code generation), ...
___________________________________________________________________
(page generated 2021-10-14 23:00 UTC)