[HN Gopher] Sorbet: Stripe's Type Checker for Ruby
___________________________________________________________________
Sorbet: Stripe's Type Checker for Ruby
Author : joeyespo
Score : 93 points
Date : 2022-03-28 18:04 UTC (1 days ago)
(HTM) web link (stripe.com)
(TXT) w3m dump (stripe.com)
| areichert wrote:
| I remember having mixed feelings about Sorbet when I first joined
| Stripe in late 2018, but by the time I left, I found it
| indispensable. Especially after the VS Code extension was
| released internally... holy crap, that made such a huge
| difference (vs having CI fail 20 mins after pushing up a PR
| because you forgot to run the typechecker script ahead of time,
| ugh).
|
| This article also made me laugh, because it reminded me of one of
| my small pet peeves about the Ruby codebase at Stripe: the fact
| that you would often find `merchant`, `account`, `invoice`, etc
| used as method parameters that represented the _ID_ of the
| resource rather than the resource itself. So Sorbet definitely
| helped with that, but it also could've been nice to just write
| `invoice_id` instead... :P
|
| Makes me nostalgic though, good times!
| clintonb wrote:
| I also joined Stripe in 2018, and thought Sorbet was a waste of
| time. I quickly changed my mind when I realized how many
| incidents it prevented. Now I want types for :allthethings!
| itslennysfault wrote:
| This is exactly how I felt when I was first forced to use
| TypeScript instead of JavaScript, but I can't even tell you the
| number of hours it has saved me. Now, years later, I can't
| stand using regular JavaScript, and would never recommend it
| for any project that will be beyond a toy.
| hardwaresofton wrote:
| Is there anything you can think to say to convince the old you?
| I have a few friends who haven't yet seen the typing light.
|
| I also think Stripe's API (external) should not be moving ids
| and objects. Given some payload in which 'account_id' is always
| present and 'account' may be the object (using 'expand' IIRC?)
| or not makes a lot more sense to me.
| jez wrote:
| My experience has been that the people opposed to types won't
| be convinced to like types by anything you can say or have
| them read. In all of the cases where I've seen Sorbet be
| adopted, the process looked like this:
|
| 1. Ambitious team who wants types does work to get the
| initial version passing in CI. Importantly, it's only
| checking at `# typed: false`, which basically only checks for
| missing constants and syntax errors.
|
| 2. That initial version sits silently in the codebase over a
| period of days or weeks. If new errors are introduced, it
| pings the enthusiastic Sorbet adoption team, they figure out
| whether it caught a real bug or whether the tooling could be
| improved. It does _not_ ping the unsuspecting user yet.
|
| 3. Repeat until the pings are only high-signal pings
|
| 4. Turn Sorbet on in enforcing mode in CI. It's still only
| checking at `# typed: false` everywhere, but now individual
| teams can start to put `# typed: true` or higher in the files
| they care about.
|
| 5. Double check that at this point it's easy to configure
| whatever editor(s) your team uses to have Sorbet in the
| editor. Sorbet exposes an LSP server behind the `--lsp` flag,
| and publishes a VS Code extension for people who want a one-
| click solution.
|
| 6. Now the important part: show them how good Sorbet, don't
| tell them. Fire up Sorbet on your codebase, delete something,
| and watch as the error list populates instantly. Jump to
| definition on a constant. Try autocompleting something.
|
| In my experience trying to bring static types to Ruby users,
| seeing is really believing, and I've seen the same story play
| out in just about every case.
|
| One final note: be supportive. Advertise one place for people
| to ask questions and get quick responses. Admit that you will
| likely be overworked for a bit until it takes off. But in the
| long run as it spreads, other teammates will start to help
| out with the evangelism as the benefits spread outward.
| brandonbloom wrote:
| > the fact that you would often find `merchant`, `account`,
| `invoice`, etc used as method parameters that represented the
| _ID_ of the resource rather than the resource itself
|
| I've encountered a few Rails projects in the wild that do this.
| One solution is to make liberal use of the `to_param` method.
| This method converts objects to strings that are intended for
| use in URLs. Of particular note, it's the identity function for
| strings and numbers, but returns `.id.to_s` for ActiveRecord
| models. Using this within definitions makes your function
| polymorphic for whether it accepts a model or an id.
|
| If you do this widely, would probably be best to monkey-patch
| in your own `to_id` method.
| hiphipjorge wrote:
| We've been starting to use Sorbet at Figma and honestly it's been
| pretty cool! Sorbet is definitely not as good at TypeScript
| (yet?). It's more verbose, doesn't support things like recursive
| types and records (shapes are experimental), and it doesn't
| inspire the same confidence TS does but it's definitely worth it
| to add it to your codebase if it's big enough!
|
| Also, it's fast! I'm in total agreement with the point made in
| the article. That makes a huge difference in developer UX.
| jez wrote:
| I have some concrete ideas for how to fix shape types to make
| them not incremental. Just a matter of finding the time to push
| the prototype over the line, and do a migration on Stripe's
| codebase to fix or silence the ensuing errors. It's one of the
| most requested features for sure, and I think once we implement
| it Sorbet will feel much better to use, especially in smaller
| projects and scripts where you don't want to have to define
| `T::Struct` for one-off data structures.
| weaksauce wrote:
| huh... didn't expect figma to be using any ruby. what do you
| all use it for there? I'm mainly a ruby programmer lately but I
| used figma for my last project design and it was really lovely
| to use so good work!
| flyingswift wrote:
| Most of the backend is written with Ruby
| ffggvv wrote:
| kinda funny how stripe is hyped so much yet they still use
| ruby...
| clintonb wrote:
| Is the implication that if Stripe started with another
| language, the company would be worth more? What's the problem
| with Ruby?
| henning wrote:
| tootie wrote:
| Types for Ruby, types for JavaScript, types for Python. Why
| didn't we all just stick with Java?
| xtracto wrote:
| Young people are coming around ... I used to write code in C,
| C++ then Java, C# (.NET v1). I could never understand how
| people could implement large systems with dynamically typed
| languages such as Ruby or Python. They are great for smallish
| scripts, but once your codebase (and team) grows, they become a
| nightmare to maintain.
|
| In my experience, large codebases of those types of languages
| have a lot of "magic" thing happen. There's a lot of implicit
| stuff that one has to guess or spend time "following the code"
| to understand what it is doing.
|
| And I say this after having built a major lending platform from
| scratch in Ruby, including a major Machine Learning scoring
| system in Python, having to maintain with a good sized payment
| system in pure JavaScript, and nowadays dealing with a major
| trading/liquidity system in Ruby.
|
| They are fun languages, but once the code and systems start to
| scale, static typing really helps. For that reason I've seen a
| lot of these endeavours try to move to TypeScript or other
| typed languages.
| klibertp wrote:
| I was asked a version of this question by a colleague at work,
| namely: "if types are so great, why didn't Python/Ruby/JS
| include them from the start (ie. early '90s)?"
|
| That's because the theory of gradual type systems was only
| worked out in the '00s. Before that, you could have a static or
| dynamic type system, not anything in between. Common Lisp did
| have type annotations, but they were hints for optimization,
| without any guarantees. They were also local to subroutines
| only. Dylan[3] is an example of an early implementation of the
| idea, but Dylan was several years late and, without being able
| to compete with Java, died without ever being widely used.
|
| The proper theory was first established by J. Siek[1] and W.
| Taha in 2006. It's distinct from nominal static typing which
| uses a single top type (like Object in Java) or generics, and
| obviously it's different from both purely static and dynamic
| typing. It took almost a decade for the idea to start gaining
| practical implementations - I think the original was a made for
| Scheme, and one of the first implementations was Typed Scheme
| for PLT Scheme, which continues on as Typed Racket[2] today.
| Typed Racket is unique in that it enforces the types even on
| the untyped side, by wrapping values and exports in contracts.
|
| The idea proved to be useful in practice, and started being
| adopted in various (non-Scheme) dynamically typed languages,
| starting with TypeScript for JS and Hack for PHP. On the other
| hand, some statically typed languages also became gradually
| typed, most notably C#. The implementations continued to
| improve, shrinking the parts of their respective languages that
| could not be statically typed. In dynamic languages there are
| still features that cannot be practically expressed in static
| type systems - most metaprogramming and code generation falls
| into this category - but they are generally "good enough" for
| day to day coding.
|
| Gradual typing is useful in the same way static type systems
| are useful: it can prevent certain kinds of errors by marking
| known-invalid expressions without the need to run the code (so,
| for example, can help you find errors even in code that's not
| covered by tests); it helps in writing tooling for the language
| (eg. go to definition, find references); it helps make the code
| clearer for the reader (no need to break into a debugger to see
| what kind of value a given identifier refers to); in some
| implementations it may also help in optimizing the runtime
| performance, but that's rare. The "gradual" aspect makes it
| easier to adopt when the codebase grows larger - the bigger the
| codebase, the more useful static types are, but by the time the
| codebase grows large enough to justify static typing it's too
| big to rewrite in a different, statically typed language.
|
| In short: writing small projects or prototypes in a dynamically
| typed language is faster while maintenance and expansion of
| large projects is easier in statically typed one. Gradual
| typing lets you go from one to the other without a huge cost of
| a full rewrite.
|
| [1] https://wphomes.soic.indiana.edu/jsiek/what-is-gradual-
| typin...
|
| [2] https://docs.racket-lang.org/ts-reference/index.html
|
| [3] https://opendylan.org/index.html
| rco8786 wrote:
| It's not like Java is the first or only language with types...
| ecshafer wrote:
| I've worked with Ruby + Sorbet, and also with Java. I would
| rather write Ruby + Sorbet than Java right now. Ruby is a
| really nice language.
|
| Though Java still has some great strengths, especially the 8+
| functional programming features and the concurrency library is
| great. If I could use Rails with Java it might be a different
| story though, since I hate Spring.
| jez wrote:
| Hey! I wrote this article. If you have any questions about Sorbet
| or Stripe, please don't hesitate to ask!
| cmer wrote:
| I added Sorbet to my codebase right after reading the article,
| but it seems to be expecting that I annotate every single one
| of my gems. Is this accurate? Is there a way around this?
| jez wrote:
| Somewhat. They don't all need super specific types for every
| method they've defined, but Sorbet does at least need to know
| all the classes, modules, and constants in use in your
| codebase, whether those come from code you've written or code
| inside gems.
|
| But there's tooling (first-party and third-party) that will
| either download or generate RBI files defining constants that
| come from gems. `srb init` is the first party solution, and
| Shopify's `tapioca` gem is the most popular third-party
| solution[1].
|
| Unfortunately, because Ruby doesn't have import statements at
| the top of every file, Sorbet can't just do something like
| silently treat unknown imports as not having a type (like
| TypeScript and Flow can do), because then it would never be
| able to tell between "exists but unknown" vs "typo; does not
| exist" for constant definitions. This definitely makes the
| adoption process a little tricker compared to other
| languages, but it's generally a one-time thing once you've
| got the tooling set up.
|
| Also if you're ever having trouble getting the tooling to
| work, there's a lot of people chatting about Sorbet daily at
| https://sorbet.org/slack
|
| [1] https://github.com/Shopify/tapioca
| sankha93 wrote:
| Hey, nice work with Sorbet! I am one of the grad students who
| worked on RDL, one of the early research projects related to
| Ruby type systems. What are the next set of challenges that a
| tool like Sorbet needs to solve? I see you mentioned meta-
| programming in the blog post, is that something that is handled
| well by Sorbet? Sorry if this is already handled, I haven't
| been up to date with the latest features of Sorbet.
| jez wrote:
| Thanks for your work on RDL! The post didn't mention it, but
| Sorbet still owes most of its type definitions for the Ruby
| standard library to RDL's original annotations. We just
| borrowed them and changed the syntax.
|
| Our general approach to metaprogramming at the moment has
| been two-fold:
|
| - Use ahead-of-time code generation powered either by runtime
| reflection or ad-hoc static analysis to generate RBI files
| declaring things that have been metaprogrammed. - Build type
| system features, errors, and autocorrects that encourage
| people to structure their code in ways that doesn't require
| metaprogramming to solve.
|
| Metaprogramming is definitely still a sticking point, but the
| existing solutions work ~okay and the rest of the upside
| Sorbet provides make it worthwile to power through.
|
| Next challenges:
|
| - Make it faster. While the post was talking about how fast
| it is, it wasn't telling the whole truth. Turns out some type
| checking operations in a 15 million line codebase are still
| slow, and we're working on making those faster.
|
| - Add more IDE features. At the beginning of this year I put
| a lot of work into making Sorbet's parser more tolerant of
| syntax errors, which helps things like autocompletion work
| better. We also want to make more code actions, autocorrects,
| and refactoring tools, to bring Ruby in line with what you'd
| expect from other typed languages in the IDE experience
|
| - Add more type system features. Shapes and tuples are a huge
| unimplemented feature still, and people ask about it all the
| time. There are a handful of other type system features
| (happy to list them if you're curious) that would also let
| people write idiomatic Ruby and still have good typing.
|
| Lots left to do!
| mwint wrote:
| I write a lot of Ruby for work, but I'm not sure what
| "shapes" are - do you have any good reference where I could
| start reading?
| jez wrote:
| It's what TypeScript calls object types:
|
| https://www.typescriptlang.org/docs/handbook/2/objects.ht
| ml
|
| (Ruby and JavaScript mean slightly different things by
| the word "object" so we chose a different word.)
|
| Flow also has a distinction between exact and inexact
| object types:
|
| https://flow.org/en/docs/types/objects/
|
| where the difference is whether other, unspecified fields
| are allowed to hide in the object, or whether values of
| type `{foo: number}` must have _only_ the `foo` field,
| and no other fields.
| burlesona wrote:
| `srb init` has had a lot of problems since Ruby 3.x, and while
| I haven't tried in a few months it looks like there's recent
| issues that it still doesn't work
| (https://github.com/sorbet/sorbet/issues/5332). Is the advice
| just to use Tapioca instead of `sub init` at this point?
| jez wrote:
| Hoping to have this specific issue fixed either today (or by
| the end of the week at the latest). So sorry for the delay in
| getting around to this!
| vhodges wrote:
| What is the status of the AOT compiler? Still moving forward?
| Stuck for lack of resources? Release planned soon? :).
| chucke wrote:
| When is RBS support coming?
| felipeccastro wrote:
| I tried adding this to a new Rails project with no luck. Is
| there a sample Rails app with Sorbet fully configured (i.e.
| most gems typed) available on github for reference?
| jez wrote:
| Unfortunately I don't know of an example repo, but I do know
| that most projects (except Stripe) who use Sorbet use it with
| Rails. There's a #rails channel on Slack--maybe you'd like to
| try asking there!
|
| https://sorbet.org/slack
| lasvad wrote:
| With Ruby 3 releasing with RBS, for new projects, whats the
| current advised path? Native RBS or Sorbet? Can they co-exist
| and if so, is there a point to using both?
|
| Sorbet is something I've been interested in using for a couple
| years and finally got a round to actually trying it out. I
| tried to use Sorbet with ruby 3.1.1 but unfortunately it didn't
| "just work" which I think is crucial for mass adoption. I want
| to give the benefit of the doubt and say its my local env that
| causing issues with Sorbet but in a fresh `rails new test_app
| --api` project, I'd expect `srb init` to work without errors...
| maybe I need to give it another go, curious on your thoughts
| above tho! :)
| jez wrote:
| We discovered a bug in `srb init` for Ruby 3.1 recently that
| a teammate of mine is working on fixing at the moment. It's
| likely that if you tried again in a few days it'll have been
| fixed. Sorry about that, totally agree that the out-of-box
| experience should just work.
|
| I wrote up an FAQ about the state of Ruby 3 and RBS here:
|
| https://sorbet.org/docs/faq#when-ruby-3-gets-types-what-
| will...
|
| The tl;dr is that RBI files (not RBS files) will probably
| always be the preferred way to declare types for third party
| code (because it will always support exactly the same set of
| features that Sorbet does). We have some people in the
| community look into teaching Sorbet to read the RBS format,
| but the existing parsers for RBS files are written in Ruby
| and are very slow, and there are some ambiguities in the spec
| that make writing a third party parser that compiles to
| native code tricky. You can see an attempt to write a fast
| RBS parser in C++ here[1], but again given that RBI files do
| everything we need them to right now and we have other
| features people are asking us for, we haven't prioritized RBS
| support incredibly highly.
|
| Sorbet works completely fine without RBS files!
|
| [1] https://github.com/Shopify/rbs_parser
| Fire-Dragon-DoL wrote:
| I tried using sorbet on our project, but the type system it
| supports is way too poor. The most glaring problem is the lack of
| support for duck typing, only nominal typing is supported, no
| structural typing.
|
| For key parts of the code, there was no type safety where we
| expected it.
|
| In the end, it felt far from being like Typescript, we opted for
| removing it, instead we added some runtime type checking and we
| document with YARD. Far from ideal, but that's the tooling
| available.
|
| The gem integration is terrible currently: we wrote the gem,
| fully typed with sorbet, but for some reason the type checking
| was completely ignored in the main project where we referred it
| bbrree66 wrote:
___________________________________________________________________
(page generated 2022-03-29 23:00 UTC)