[HN Gopher] The data model behind Notion's flexibility
___________________________________________________________________
The data model behind Notion's flexibility
Author : pspeter3
Score : 127 points
Date : 2021-05-18 19:55 UTC (3 hours ago)
(HTM) web link (www.notion.so)
(TXT) w3m dump (www.notion.so)
| rkangel wrote:
| Notion clicked for me when I realised that everything is a "page
| with some attributes". Blocks are interesting at the engineering
| level, but as a user I found I was normally thinking at the level
| of pages.
|
| Each page is some content (optional) with a set of arbitrary
| attributes of different types, e.g. date, string, number etc.
| When you create a table and populate it, you are creating a set
| of pages all of whom have the same attributes and the table is
| just a view of those pages. You can then create a calendar that's
| a different way of presenting those same pages. You can also deal
| with those pages directly and put content in, and reference them
| from elsewhere.
|
| It's a really simple data model, and it's pretty flexible. We
| built all our processes on Notion initially and then moved them
| off when they got to an appropriate scale. I'm glad the API exist
| now, because then that approach can be incremental rather than a
| wholesale move. All the ways of displaying things - tables,
| calendars etc. are just displaying collections of pages.
| colinplamondon wrote:
| This!
|
| The design decision to feel "default-text" is awesome - text is
| a great foundation for a "custom workspace" product. But, it's
| only when the bulk of content is in Tables that things get
| extremely powerful.
|
| Once Notion has in-line text references, ala Coda, the sky is
| really the limit. I think Notion nailed the UX and speed /
| performance.
|
| Even if it's slow compared to Apple Notes, it's lightning fast
| compared to Coda, where "Documents" are extremely heavy, and
| relatively isolated.
|
| Notion has a clean route towards eating Coda's most powerful
| functionality ([an incredible charting and
| formula](https://coda.io/formulas)). That's the kind of thing
| that a company as well-financed as Notion can dedicate a team
| to, and make incredible progress on.
|
| Similarly, They can eat Coda's in-line text references, so your
| text can live-link to information in tables. A matter of
| resource allocation.
|
| Coda, on the other hand, started from an "easier application
| builder" approach, like Retool for Excel experts. That's
| brilliant and powerful, but the UX creates a bigger dividing
| line between consumers and creators.
|
| Notion feels enough like a normal word processor that consumers
| start doing crazy things without realizing it.
| jitl wrote:
| Author here. I'm happy to answer any questions you might have.
| RyanGoosling wrote:
| Why not extend this to create forms/surveys as well in your
| application? You can render a checkboxInput block type as a
| <input type="checkbox" /> in a form. Your backend would match
| up the form POST parameter names with the associated form input
| names and that would be your "Submission".
| Madeindjs wrote:
| Really interessing.
|
| This seem to be a really good use case for a NoSQL database. Am
| I wrong ?
| thawab wrote:
| PostgreSQL has Ltree
|
| http://patshaughnessy.net/2017/12/13/saving-a-tree-in-
| postgr...
| jitl wrote:
| Ltree is interesting, but if I understand correctly, to
| move a parent block, I'd also need to update the path
| column in all the child blocks -- at our scale such write
| amplification is a non-starter.
| RyanGoosling wrote:
| Yes, you're wrong. You're wrong because you need to JOIN a
| massive tree of blocks, to form the graph the author is
| referring to.
|
| You can break out the "block" model into several tables and
| represent it in a relational database that way.
|
| NoSQL = NO JOIN?
|
| Hope that helps.
| jitl wrote:
| We don't use JOIN for the content tree; I don't think I've
| seen one in any of our queries.
| RyanGoosling wrote:
| What do your queries look like? Are you using an ORM?
| setr wrote:
| I don't actually see a graph represented anywhere in the
| article; the author references wanting a graph at the
| start, but the only thing I'm seeing described are trees of
| nested blocks. Even the properties list seems to be a grab-
| bag of KV pairs that gets permanently attached to a block
| once initialized, to support roundtripping
|
| Which is pretty much the ideal scenario for a document
| store. The article describes Notion as being very strictly
| hierarchal
| RyanGoosling wrote:
| A block has many properties. A property has a name, and a
| value.
|
| The underlying persisted data doesn't necessarily have to
| be a bag of KV pairs.
|
| A block is related to its parent and descendant blocks.
|
| These relations are suitably represented in a relational
| database, not a document store.
|
| EDIT: In graph theory, a tree is an undirected, connected
| and acyclic graph.
| ellimilial wrote:
| @setr explained it really well. A side note, NoSQL also
| includes graph databases, dedicated to this type of
| node/relationship traversal.
| johnknowles wrote:
| I'm curious if Notion has any plans to make the "type" property
| user-extensible. Given the current data-structure, which
| decouples the block data from the way its rendered through the
| type property, a user has to define only one template for
| rendering arrangements of UI components (boxes, bullets, etc),
| titles and children. Extension could operate even at the level
| of derivation, where users could extend current base types with
| custom styling (color, font, size, border, etc) and child
| layout. As a plus, derivation would allow for blocks to be
| shared, with a fallback default rendering if users don't share
| custom types. Given the multi-dimensional nature of the uses of
| Notion (for work, personal projects, life management, etc),
| having types that were specific to their domain (grocery list,
| monthly budget table, contact card) would be a useful tool to
| semantically separate blocks by their presentation.
| DarraghBurke wrote:
| I'm going to be writing a blog post for my engineering team
| soon and I'm really impressed by the storytelling and
| presentation here. How did you plan out your blog? Were there
| any resources or examples you found useful during the writing
| process? Thank you!
| jitl wrote:
| I'm glad you enjoyed it! It took a long time to put together.
| I started out by writing down how the entire system works
| with enough explanation for a technically-minded non-
| engineer, and enough detail to satisfy a newly hired infra
| engineer. That rough draft was about 10,000 words.
|
| From there we started looking for a narrative. We extracted
| out the sections you see in the final post, and removed a lot
| of the superfluous technical detail so we didn't end up with
| technology buzzword soup; for example we cut discussion of
| Postgres, Memcached, etc etc, how we host the web servers;
| the kind of details that don't actually matter to the
| narrative.
|
| The illustrations were in the post from the beginning as
| Mermaid diagrams (https://mermaid-js.github.io/mermaid-live-
| editor/). As we got close to publication we polished them up
| in Figma.
|
| This is really the first engineering blog post we've put out,
| there was a fair amount of figuring-out-how-to-do-it going
| on. Now that we've had the experience, we're starting to
| write up our playbook internally.
| burlesona wrote:
| Very interested in your data store. How do you store, query,
| and search across documents?
|
| Also, are you adding presentational tables any time soon? :)
| louisvgchi wrote:
| I liked the part where they talked about the blocks with the
| blocks and the block blocks and how they all block together.
|
| I thought the reference to field pioneers was a bit much, I
| wouldn't have the blocks to do that with a straight face. I wish
| these app makers (see also Figma, AirTable) would refrain from
| making their self-aggrandizing "we're bringing computing to the
| masses"; what a load of blocks. What you're doing is creating a
| product (another silo in waiting) serving a market that presently
| is interested. This block model is also not unique (see WordPress
| and Drupal): Notion may be refining some of that in a more
| seamless experience, but this is a different kind of statement.
| Perhaps "we added collaboration to blocks" would be a possible
| claim for uniqueness.
|
| Aside from this, I found the technical decisions interesting and
| worth a read.
| colinplamondon wrote:
| I think it really is that unique - Notion blending read/write
| is pretty revolutionary for an app like this. Wiki's are
| obsessed with toggles between read/write mode.
|
| Those apps you mention - Drupal, Wordpress, etc - have write
| modes that exist in backends with alien experiences to the read
| modes.
|
| By unifying read/write into a single continuous UI, with built-
| in relational databases usable by non-technical users, Notion
| really does let people create damned powerful custom apps.
|
| If you squint, you can see how Notion looks a lot like the next
| version of the Internet. By hyper-focusing on company intranets
| they miss out on the power of connected workspaces ("domains"),
| but the thinking is there.
|
| I'll be seriously surprised if Notion isn't looked back on as a
| breakthrough paradigm shift.
| smoldesu wrote:
| > I'll be seriously surprised if Notion isn't looked back on
| as a breakthrough paradigm shift.
|
| To that end, I'd be surprised if they're even remembered much
| at all. I cancelled my subscription after being seriously
| disappointed in the feature-set.
| colinplamondon wrote:
| If you want to set the line, I'll take the over on "Notion
| succeeds"
| wpietri wrote:
| I hadn't heard of Notion before this recent HN discussion, in
| which many, many people make the argument that it's slow, often
| unusably so: https://news.ycombinator.com/item?id=27144566
|
| So it's a bit of a surprise to see them bragging about their
| technology today.
| fellowniusmonk wrote:
| I've not used notion before and this is a great write up...
|
| Having built my own private-ish collaborative
| editor/note/todo/sharing/pm tool back in 2012 (that uses indent
| level permissions) I don't see how you could have a non "block"
| implementation?
|
| I also don't see how they are having so much performance
| difficulty moving up the inheritance tree at the client level?
|
| Does anyone else who has worked on these kinds of problems take a
| different approach? This being HN and a space I develop in I am
| actively curious if any other architecture are in use generally?
|
| Are there approaches outside of blocks for tools with this level
| of granularity?
|
| Really cool of Notion to share these kinds of "inside ball" deep
| dives.
| polote wrote:
| > I don't see how you could have a non "block" implementation?
|
| I mean, documentation is mostly text, image and code. You can
| easily be document-based instead of block-based, like
| Confluence, Google docs ... you don't always need to be able to
| insert everything into everything.
|
| I'm actually building a Confluence/Notion competitor, and it is
| document based. Being block-based makes a lot of things much
| more complex
| fellowniusmonk wrote:
| Interesting, how do you handle line level or "intra-document"
| permissions then?
| polote wrote:
| The permission is at the document level. What are the use
| case of intra document level permission?
| emteycz wrote:
| E.g. managers having different information about tasks
| diplayed in a table than programmers. You could do a
| different page for each group, but it's not as nice -
| different URLs, duplication of surrounding content, etc.
| polote wrote:
| Well in my case each row is a document and the table is
| just a a specific king of structure, but is not
| embeddable in a document. But the thing is how generic
| you want your tool to be. In the notion case, everything
| is just an instance of something. So that's super
| flexible, but also super complex. You can simplify things
| by enforcing a structure while still doing 95% of the use
| cases
|
| You will never need to give a permission to a line of
| text
| burlesona wrote:
| I don't think you can do this without a block-based approach.
| That part isn't super novel, for example I wrote textbook
| editing software circa 2014 which worked largely the same way.
|
| What notion has done that is impressive IMO is made the editing
| experience enough like the MS-Word "inline" editing that people
| seem to understand it and use it without _minding_ that it's a
| block editor. The project I worked on back in the day wasn't
| quite as slick, and there were a lot of users who got
| frustrated when things like copying text across multiple
| paragraphs didn't always do what they had in mind (imagine a
| text selection from the middle of one paragraph (block) to the
| middle of another - that's complicated to deal with).
|
| Nice to see Notion take this model mainstream with such a nice
| UX.
| erjiang wrote:
| I built a no-code web-app creator that also has a concept of
| blocks and I can relate to this. Each paragraph is a
| different block, and I still haven't quite figured out the
| best way of handling cross-block interactions like your
| example of cross-block text selection.
|
| I think I would need to redo the browser-based editor,
| because even if I solved e.g. cross-block selection across
| blocks of the same type, it's unclear how it should work
| across, say, a text block and a video-embed block.
| polote wrote:
| Notion is clearly a feat of engineering. The first time that you
| modify a link name, and it is automatically modified in all
| others connected users whether it is in the menu, inside a
| document or wherever it is. You are just impressed. You can also
| drag and drop links from a document to a menu.
|
| I'm just worried that so much engineering complexity is going to
| make things slow to move. We will see in a few years if that was
| a good idea or not
| fredley wrote:
| I think the power of Notion became apparent to me when I created
| a Page, accidentally changed it to some other type, then
| ctrl+z-ed it back to a Page, with all its contents intact.
| Seriously impressive. I wish search wasn't so crap though.
| ternaryoperator wrote:
| > accidentally changed it to some other type
|
| _This_ is the problem with Notion, in my experience at least.
| The docs are so limited and the maneuverability so unusual that
| it 's very easy to do something like completely change the type
| of document accidentally.
| gumby wrote:
| Not sure how "unique" this model is; for example Claris Works was
| built out of an even more powerful block model (they called them
| frames) back in the late 1980s:
|
| > We came up with a frame-based approach. Most of the
| functionality particular to the various application types was
| packaged up into "frames": word processing frames, graphics
| frames, etc. These frames were then used as building blocks to
| make documents of the appropriate types, in a unified programming
| framework. E.g., a word processing document was essentially a
| bunch of text frames, one per page, linked together. (Doing this
| neatly was a big challenge - many subsequent efforts at building
| a component-based architecture (e.g. OpenDoc) have failed to take
| into account the top-level user interface requirements.) The
| result was that not only was most of the code shared across the
| document types, but the application was also truly integrated -
| the frames could be embedded in each other. E.g., you could plop
| a spreadsheet frame right into your word processing document.
| Text objects in a graphics document had a full-featured word
| processing engine behind them. The database form editor used the
| built-in graphics environment. Etc.
|
| > One related cool thing we had was a "shared graphical context"
| mechanism: sometimes, stuff would wind up being displayed in
| multiple frames at once. E.g., maybe you're dragging an object
| across a page break in a document with multiple pages (like
| this). We developed a general architecture for displaying actions
| live in multiple contexts. Of course, a lot of this kind of stuff
| is old hat today, but it was new and exciting in 1989. Some
| creative programming was required to do these things efficiently
| on the hardware of the time.
|
| > There were some cool features that didn't make it into the
| shipping product. For example, originally spreadsheet formulas
| were much more powerful: you could relate, e.g., graphical object
| positions and document properties to spreadsheet cells. So you
| could have the result of a calculation move objects around
| graphically, or vice-versa. (Further work in this direction led
| to a novel constraint-based programming paradigm called MOOSE,
| which I may resurrect some day...)
|
| (From
| http://groups.csail.mit.edu/mac/users/bob/clarisworks.php#on... )
|
| I've been kicking around the idea of writing a CRDT-based editor
| using this model.
| jonpurdy wrote:
| This is a great explainer on how and why Notion is so powerful,
| and certainly helps explain why it doesn't feel like the fastest
| tool (because it's inherently more complicated and more powerful
| than most).
|
| I've tried using it for personal notes but it's speed and lack of
| offline prevented me from migrating to it. But I think it's the
| best choice for any organization that needs a collaborative
| docuement/wiki/Confluence-esque tool.
|
| Not to mention that I think it has the best default fonts,
| spacing, and general aesthetic of any tool I can think of.
| wafflez wrote:
| May I ask what you _do_ use for your personal notes instead?
| I'm always on the lookout for a new tool or workflow for my
| notes.
| imilk wrote:
| Google Keep has been pretty sufficient for my needs. Nice
| search feature, able to capture things quickly when on a call
| or out and about. Not the most fully featured, but good for
| remembering things a few days or a week from now
| nxc18 wrote:
| I'm not the person you asked, but I evaluated and passed on
| Notion (and Coda, and Airtable) for similar reasons.
|
| I need a system that syncs, works offline, helps capture
| structured-ish data, is quick+flexible, and has fabulous
| linking.
|
| I discovered and am currently using Obsidian.Md. I use
| Ulysses to access and edit on iPhone+iPad.
|
| I love that it is markdown-based, linking is super easy, and
| everything is fast.
|
| Cons: doesn't work super well with native macOS features
| (e.g. services), native app for iOS/Android is still in
| private beta, and I don't like the separation between edit
| and view modes. Also it could probably use a stronger UI for
| database-like features (e.g. tags). I wish there were better
| options for inking on mobile, but that's not really their
| thing.
|
| Previously was a OneNote devotee, but as I get into more
| complicated community engagement work, I needed something
| that could cross-link and find patterns.
| scubbo wrote:
| I appreciate this write-up! I've been considering a
| migration from Evernote to Obsidian/Notion/other-competitor
| for a while as Evernote's UI gets steadily worse, and this
| reconfirms my perspective that Obsidian is leading the
| pack. Thanks!
| Karunamon wrote:
| I used Trilium Notes for a long time before shifting to
| Notion. It checks all the freedom boxes and is very
| customizable, without being alien like emacs.
| spamizbad wrote:
| Emacs org mode is worth a shot.
| posharma wrote:
| Am I the only one who thinks the data model is very complex? How
| does it compare to a document based data model? What are the pros
| and cons of each?
| mattwad wrote:
| Been working with multiple document API's recently for my start-
| up. It's been a real challenge to understand how different
| document stores compare -- Confluence uses undocumented XML
| markup that "mostly" mirrors HTML, Google Docs gives you
| sometimes semantically-incorrect HTML with lots of inline styles,
| Slack has its own funny Markdown, etc. Then there's Notion
| blocks. I had to write my own HTML-to-Notion block parser, but I
| think I prefer working with it over everything else because
| structured data is just easier. One interesting gotcha: if you
| want to completely overwrite a page you currently have to remove
| every content block one at a time, and then add each new line one
| at a time. This is using their undocumented API, but the new API
| doesn't even offer deleting content yet. I'm eager to switch over
| but it may be a while...
___________________________________________________________________
(page generated 2021-05-18 23:00 UTC)