[HN Gopher] Design Docs at Google
___________________________________________________________________
Design Docs at Google
Author : thesephist
Score : 362 points
Date : 2020-07-22 10:12 UTC (12 hours ago)
(HTM) web link (www.industrialempathy.com)
(TXT) w3m dump (www.industrialempathy.com)
| cramforce wrote:
| Author here. Let me know if you have questions or feedback.
| drewlander wrote:
| I thought it was very helpful. I am sending it out to my
| manager and others on our team. We have gone through a
| transition where I work, and a new product owner likes to send
| us implementation manuals as design docs because that's how
| they did things many years ago. We have been trying to get them
| to work in our process, and I am hoping that this article will
| help reinforce a change in thinking or at least help us to come
| to an understanding. Thank you for writing it!
| noselasd wrote:
| Actual real design docs would be extremely helpful
| streblo wrote:
| My question is how do you avoid bikeshedding during this
| process?
| lhorie wrote:
| The alternatives considered section is a good place to deal
| with it. There you can enumerate all the arguments for and
| against any particular thing, so you don't end up talking in
| circles.
|
| Often times, the person writing the document has the most
| context/expertise and can provide a short explanation for why
| one option might be a better trade-off than another even
| though there are clear and logical arguments against it.
|
| Having data also helps. IMHO, a lot of bikeshedding is
| uneducated conjecture, which can be put to rest with proof-
| of-concepts, benchmarks, level headed comparison tables,
| discussion notes with others in the industry etc. At my
| company, large reaching technological decisions often involve
| meeting with people with relevant experience from
| FAANG/others to gather information.
| 010101010101 wrote:
| This is great. At my company we normally create design docs,
| and I've seen each of the anti-patterns described here, along
| with their more informative/useful counterparts, but not seen a
| place that succinctly enumerates "do/don't do" very well.
|
| A piece of feedback - I like to include sequence diagrams along
| side system diagrams to detail interactions a bit. I think this
| serves a similar purpose to documenting the API in an informal
| manner, and gives a good amount of information density (I
| _don't_ expect everyone to read and digest all 5-15 pages of a
| document, pictures help people retain what's important and also
| have a small thing to refer to in the future, IMO).
| altgoogler wrote:
| Hi Malte!
|
| Great article. I've agree that design docs are great part of
| what defines engineering culture at Google. I'd recommend to
| anybody who will listen that their company should do it also.
|
| Lots of people in this thread were burned by waterfall-esque
| requirements documents or formal specification, and I'd like to
| point out why I think design documents (at Google at least) are
| different and more effective than those things.
|
| I like to think of design docs as not an artifact produced by a
| project but as a communication mechanism of a team. You have an
| idea and you write down:
|
| * What are you trying to do?
|
| * Why are you doing to do it?
|
| * How are you going to do it?
|
| * What considerations have you made?
|
| Then you socialize it to one or two reviewers, who ask a lot of
| questions, then you take it to your team, who ask (relatively
| fewer) questions. The socialization part is key to the entire
| system. As you note:
|
| "The primary value that such reviews add is that they form an
| opportunity for the combined experience of the organization to
| be incorporated into a design. Most consistently, ensuring that
| designs take cross-cutting concerns such as observability,
| security and privacy into account is something that can be
| ensured in a review stage. The primary value of the review
| isn't that issues get discovered per-se, but rather that this
| happens relatively early in the development lifecycle when it
| is still relatively cheap to make changes."
|
| If I printed out your article to show people, this is the part
| I would highlight in yellow.
|
| Contrast this with formal doc systems that rarely capture the
| "why" and "why nots". Future maintainers have a document that
| captures the thinking at the time, rather than trying to
| document the full implementation plan of a system.
|
| Hopefully this adds some context to those who've been burnt by
| more traditional approaches.
|
| edit: typos
| fbunau wrote:
| How do you get teams that have knowledge on a particular piece
| put in the work when it is not one of their goals / focusing on
| other things ?
| cramforce wrote:
| I know this isn't a satisfying answer, but tools like design
| docs or any $SoftwareDevelopmentMethodology do not help fix
| broken corporate governance.
|
| Concretely here, I'd try to make solving my problem the other
| team's goal. E.g. by inviting them to a summit during
| planning season and agree on common OKRs.
| numbsafari wrote:
| What really helps here is adopting a culture of shared
| ownership. If a team has knowledge, your best bet is to work
| with them to share it with you. But if they are too busy, or
| otherwise unwilling, then you will be forced to move ahead
| without them. You can't let teams like that become a
| bottleneck to progress.
|
| Similarly, if you are on a team that has important knowledge,
| it's really important to share that knowledge widely. Prepare
| lots of good resources to help spread that knowledge. Don't
| try to operate as gatekeepers or a cabal, instead, it's up to
| you to be an advocate and an activist for your knowledge. If
| you want other teams to respect your team's knowledge, then
| you need to make sure that they recognize that you have it,
| and that you are willing to share it. Lastly, it's best to
| adopt a strategy of empowerment, rather than ownership.
| Encourage and support consumers of your knowledge to help
| themselves, rather than requiring you to opine on every
| single question, or participate in every single design
| review.
|
| All of this, of course, takes leadership, because it's a
| cultural practice. Leadership has to invest in having teams
| document and share knowledge. Leadership has to reward and
| recognize knowledge sharers while similarly recognizing and
| working with knowledge hoarders to change their ways.
| Leadership has to identify when a team has become a blocker
| on progress and either add resources, or as noted above,
| encourage teams to work around them. "So and so is the
| networking expert but he won't help us fix this problem."
| "Okay, I'll work on getting his time, meanwhile let me find
| this outside consultant _or_ I 'll give you cover to do the
| work yourself since they are blocking."
|
| That last thing is your last resort, but you need to not be
| afraid to use it. I actually get the impression that Google
| suffers from that quite a bit (the existence of Principal
| Engineers who "squat" on problems is one I've seen discussed
| repeatedly by former employees, and something I've witnessed
| on OSS projects).
| gowld wrote:
| Pay people for producing value. If you believe design docs
| have value, pay people more for writing more better ones.
| blargmaster33 wrote:
| Properly done MIL-STD-498 is a joy to behold.
|
| Operational Concept Description (OCD) - Used to sync with the
| marketing/sales non technical people.
|
| System/Subsystem Specification (SSS) - Software/System architect
| and agile team leads sync here
|
| System/Subsystem Design Description (SSDD) - Developers documents
| a rough outline of how stuff was done.
| kcudrevelc wrote:
| I've worked at Google for a while, and a few years ago I wanted
| to release an open-source project. Since the code was going on
| Git, it made sense to throw the design there too, so I just added
| a DESIGN.md to the top-level of the project. This has had a
| remarkably nice effect: sometimes, when people make large changes
| to the project, they update the design to reflect the changes!
|
| See this lovely commit as an example:
| https://github.com/google/stenographer/commit/d678531a3e5a87...
| juanbyrge wrote:
| It's a little ironic that the author of that post created AMP...
| nkingsy wrote:
| The emphasis on trade offs is the most important part for me,
| because it formalizes the search for alternative approaches. In
| the majority of my design docs, I end up swapping an
| "alternative" into the doc itself as I find myself more convinced
| by the alternative solution.
| maps7 wrote:
| This is awesome! Thanks for posting - I think I can use this
| where I work. I am wondering though.. how do you allocate time to
| work on the design doc? I'm sure sometimes it takes a day but
| other times it can take longer and if there's a review needed
| then it would take even longer. I work in a agile/scrum team -
| how would this work fit in there?
| the_arun wrote:
| Do we have any templates for design docs created by Google,
| Facebook, Twitter etc.,?
|
| Not to be negative - My problem is there is no spec/standard
| template for design docs (like someone compared it with
| Blueprints). If we have a template approved by industry leaders -
| it will be convenient. Otherwise everyone creates them in their
| own way - convenient for their use cases. But this will miss the
| consistency and all necessary ingredients. This ends up these
| docs fading out with time.
|
| Also, when design changes - these docs need to be updated. People
| don't do this. After a while, design docs become obsolete &
| overhead.
| packetslave wrote:
| The canonical Google design doc template is called the Bluedoc.
|
| An example is here:
| https://docs.google.com/document/d/18hYAQCTsDgaFUo-VJGhT0Uqy...
|
| Delete everything but the headers and you have a bluedoc.
| cameronbrown wrote:
| There are also a few domain-specific headers in-use: Privacy,
| Ethics and Security are very common to see.
| packetslave wrote:
| yeah, Monitoring and Logging are also common to see.
| thebrokencube wrote:
| I think if there was a rigid spec, it would work against one of
| the points made early on in this doc (which I think is worth
| taking note of):
|
| > Rule #1 is: Write them in whatever form makes the most sense
| for the particular project.
|
| This is an unsatisfying rule, but I think it's important
| because each team/problem space/etc. is different and too
| strict of rules can often lead to documents that may end up
| being shallow.
|
| However this admittedly doesn't help with this other problem
| you brought up:
|
| > Also, when design changes - these docs need to be updated.
|
| ... but that may be okay given a shared understanding of what
| the goal of said document is. If the goal is to gain consensus
| around an implementation (as opposed to it being a living
| document), the document may have served its purpose by the time
| that consensus is reached, even if later on the design changes.
| If the goal is to have living documentation of a system, then
| maybe the rough format specified in this article isn't correct.
|
| One term I've heard for design documents is that they end up
| being a piece in a "Decision Log", which has helped me feel
| less bad if the document falls out of date. These documents end
| up being more point-in-time representations of the collective
| understanding/opinions of how a specific project/system was
| being thought of, and that has its own advantages even if it
| falls out of sync.
| brown9-2 wrote:
| The first points here are absolutely correct. These documents
| are a way to communicate and get feedback (and iterate) on an
| idea with a group of people in your organization. Different
| organizations will communicate differently, and have
| different needs.
| enumjorge wrote:
| > If we have a template approved by industry leaders - it will
| be convenient. Otherwise everyone creates them in their own way
| - convenient for their use cases.
|
| One of the more common mistakes I see in software engineering
| is adopting tools and processes of companies/teams that don't
| look like yours. Sure, get an idea for what people in the
| industry are doing, but at some point you'll need to figure out
| what works best for you. The needs of a spec for a company like
| Google are probably pretty different for a team of 10 people
| who are in constant communication.
| cborenstein wrote:
| Design docs are great because writing them is aligned with the
| author's best interest, not just a future reader. (This is
| different from docs that are written after the project is
| complete).
|
| Design docs help you understand the project, identify
| dependencies, and avoid repeating work. Then they leave a trail
| for future engineers of the thought process that went into the
| code.
| jeffbee wrote:
| A fun activity if you're a Googler is to go read ancient design
| docs, like the original pitch for bigtable. They are pretty short
| for the most part, and they were written by the legends. These
| are the Federalist Papers of your company, they give real context
| to how the company arrived here. In particular I always enjoyed
| reading the jarring parts were they describe something that
| definitely did not make it into the implementation, or got
| removed later. It's interesting to think about why they
| considered those things important enough to write down at the
| time.
|
| Now that I wrote the above, it's a bit sad that there aren't many
| design docs for open source software, even ones that originated
| from companies with good design doc culture. Where's a doc that
| discusses alternatives considered and rejected for kubernetes or
| grpc?
| jbeda wrote:
| I wrote a design doc/PRD for k8s that outlined the basic API,
| distributed structure and why/how it related to open source and
| the marketplace. I lost it when I left google. It was a google
| doc IIRC and wasn't checked into source control.
| spelunker wrote:
| Great advice, jeffbee! hey wait a minute.
| Areibman wrote:
| >Finally, the overhead of creating and reviewing a design doc may
| not be compatible with prototyping and rapid iteration. However,
| most software projects do have a set of actually known problems.
| Subscribing to agile methodologies is not an excuse for not
| taking the time to get solutions to actually known problems
| right.
|
| Here lies the biggest problem with the article. Most software
| projects _do not_ have a set of actually known problems. Agile
| exists precisely because it's hard to pin down what problems are,
| and you learn much more about your product with gradual
| experiments than with some grand, centralized design.
|
| Unless you are totally certain about your solution, design
| documents don't have a place for most projects.
| theptip wrote:
| I don't agree with this viewpoint. Agile is about just-in-time
| design, not no-design.
|
| At some point, you need to decide what the next increment
| you're going to build is. Before you write the code for that
| increment, you have by definition picked an actually known
| problem to solve, and for that you should write a design doc.
|
| I think what you're objecting to is the waterfall concept that
| you would write a design doc that covers the whole project,
| rather than just designing what you're going to work on next. I
| can wholeheartedly agree with that objection.
|
| However everything in the article is consistent with epic-level
| or story-level design. As the complexity and cost of change
| increase, more up-front design is appropriate. None of that is
| incompatible with the agile tenets of building small increments
| and delivering them often to make sure that they add value.
|
| If you're lucky enough to be working in the early stages of a
| project with a few thousand lines of code, and little to no
| production data on which to maintain integrity, then the value
| of formal design docs goes way down. But it's important to know
| that you'll need them later.
| nawitus wrote:
| Are you suggesting to write a design doc every sprint, e.g.
| two weeks?
| theptip wrote:
| In some sprints my team will write multiple design docs,
| though that's uncommon. For example, if we're building two
| new features, and each is self-contained. Those design docs
| are probably small.
|
| In other sprints, we won't author any new docs, for example
| if we're implementing and shipping/demo'ing chunks of a
| large, complex feature for which we've already completed
| the technical design.
|
| I'd suggest coupling design docs to stories/epics, not
| sprints.
| bdefore wrote:
| While I agree, I find this common argument to have a faulty
| rhetorical approach. Leadership can often be allergic to
| allegations that there are problems that they don't know about.
| They can be outright defensive towards suspicions of
| uncertainty.
|
| I feel there may be a better way to express the utility of
| rapid iteration and design-as-you-go without triggering a fight
| response.
| cameronbrown wrote:
| Design docs are one of my favourite things about software
| engineering. If code is the bricks and mortar, then these docs
| are the blueprints.
|
| I know this is fairly controversial, but our jobs isn't just to
| write code. Navigating organisations and achieving consensus
| between a lot of teams/technologies is a huge part of it.
|
| Design docs are a way to get all of that out of the way _before_
| writing thousands of lines of code. The review process nets
| significantly different feedback to code reviews too.
| taeric wrote:
| This often betrays an understanding of blueprints to builds
| that just isn't true. The blueprint of simple things is how
| they are built. Even complicated things, this will be roughly
| true for _new_ things.
|
| For old things and complicated things, though, they were how
| things could be attempted to be built. And with the builder
| being someone else, there had to be an audit from the build to
| the blueprint, if you really want confidence in that statement.
| draw_down wrote:
| What engineers actually do is determined by the incentive
| structure they work in. If shipping is prioritized all the time
| above everything else, one of the things that will suffer is
| documentation.
|
| If people get promoted in spite of not doing a good job of
| documentation, now you know why it doesn't happen.
| maps7 wrote:
| I work in an agile/scrum team. Do you have any experience in
| tracking the design doc work in this type of environment? It's
| hard to estimate how long a design doc would take since
| investigation into the solution and conversations with
| stakeholders could expand it.
| gowld wrote:
| If your estimates matter, you are doing agile wrong. Reserve
| some time, do some work, repeat.
| sethammons wrote:
| At work, we call them blueprints :). A fantastic way to get
| early feedback before writing any code, especially important
| for junior teams who may forget to think about failure cases
| and operability.
| thehappypm wrote:
| In a much more real sense, the code itself is the blueprint.
| The compilers/interpreters lay the bricks and mortar for us.
| learc83 wrote:
| >If code is the bricks and mortar, then these docs are the
| blueprints.
|
| That analogy falls apart quickly. Design docs aren't specific
| enough to be analogous to blueprints.
|
| You can give a set of blueprints to 3 different construction
| firms and get fundamentally the same building. Try giving
| design docs to 3 different development shops and see what
| happens.
|
| The problem is that the only way to get to that level of
| specificity is with code.
|
| Design docs are closer to something a city planner would
| produce than to something an architect or civil engineer would,
| and they should be treated accordingly.
| nickbauman wrote:
| This is something that should be taught to folks while
| they're still in school. The "blueprint analogy" has baked in
| the idea that there's a "design phase" and a "construction
| phase" and that these are often discreet parties.
|
| Nothing could be more incorrect when it comes to software. In
| software, _the design is the code._ The compiler /interpreter
| are the construction of the system.
| underdeserver wrote:
| "The design is the code" does not follow from the idea that
| design and construction are not separate things in
| software.
|
| The thing to take away from that idea is that the design
| doc is not law written in stone, but a base for changes. It
| keeps your design honest on the high level. If you end up
| clashing with reality, and your design doesn't work out,
| you can backtrack to the design doc and refine it as
| necessary.
|
| In short, the design doc should be a living document.
| nickbauman wrote:
| I think what you're talking about here is an "intention
| document" not a design document. Design is about
| _planning._ Which implies you define things in detail
| with steps and techniques. Intentions are about what you
| want to achieve. Very different.
| maps7 wrote:
| A design is an outline, sketch, or plan. So it doesn't
| specify the level of detail.
| supahfly_remix wrote:
| If the design is the code, then what is a bug? There is a
| separate model, whether written down or not, of what the
| code must do. That is the deaign.
| learc83 wrote:
| What happens when there is a flaw in the design? There's
| another higher level of design on top of that, with a
| platonic ideal of the design unknown to humans at the
| top? Turtles all the way down.
|
| Each stakeholder has a different "design" in mind, and
| until you actually get specific there is no design,
| there's just a nebulous, incomplete list of requirements.
| And if you do try to get specific enough to be
| reproducible--you're writing code.
|
| None of this is to say that design docs are worthless,
| just that they can never be specific enough to function
| as the actual reproducible design the way blueprints
| would. Thinking of them that way is harmful.
| Kinrany wrote:
| It's true that they're not reproducible. Are there other
| noticeable differences?
| learc83 wrote:
| Reproducibility is just a side effect of lack of
| specificity, which is the major difference and the entire
| point of my critique. Design docs don't tell you enough
| to actually build the thing you want to build.
| cameronh90 wrote:
| Never lived in a house with a design flaw?
|
| My first flat had a toilet that blocked the door from
| opening fully. It was correct according to the
| blueprints.
| supahfly_remix wrote:
| Yup, that's a validation issue, as NASA would term it:
|
| https://en.wikipedia.org/wiki/Verification_and_validation
| sanderjd wrote:
| I think they were using "design" as akin to the blueprint
| to the building. I totally agree with them that in that
| sense, the code is the blueprint, not the building. The
| execution of the program is the analogy to the actual
| construction: the builder (computer) takes the blueprint
| (code) and builds the building (executes the program)
| based on it. The power of computers is that the
| "building" (execution) part is essentially free and
| infinitely reproducible. But the creation of the
| "blueprint" (code) is still labor intensive.
|
| The right analogy for design artifacts really is to the
| higher-level vision definition for a project. Engineers
| don't just show up and start drawing blueprints; one or
| more people come up with a purpose and concept for a
| building project, brainstorm approaches, evaluate trade-
| offs, come to some consensus on the right direction, and
| only then start working with engineers to start creating
| and iterating on blueprints. This vision phase is where
| design docs fit.
| nickbauman wrote:
| Much of this is captured extremely well in Fowler's _The
| New Methodology._
|
| https://martinfowler.com/articles/newMethodology.html
| javier10e6 wrote:
| The Code is the Design and the design uncovers good or bad
| written code. GoF design patterns were modeled after well
| written code. Anti-patterns (the swiss army knife is my
| favorite) are bad written code.
| throwaway_pdp09 wrote:
| > In software, the design is the code. The
| compiler/interpreter are the construction of the system.
|
| That's IMO a pedantic view. Yes, ok, but when I design
| software I describe it's high-level characteristics, more
| like describing the plot of a book than the specific words
| in a book and the what the characters say to each other.
|
| I'd say there are many levels of design. Code is the last.
| javier10e6 wrote:
| I agree wholeheartedly. The best software documentation is
| the an actually well written software module + test that
| explains its functionality.
| TeMPOraL wrote:
| Let's not forget about comments.
|
| A well written piece of code will only tell you what it
| does, a good suite of tests will teach you how to use it,
| but only natural language documentation - be it in comments
| or separate documents - will explain to you _why_ the code
| exists in the first place, and why it looks the way it
| does.
|
| (As for "self-documenting code", unless a lot of your
| functions contain the word "because" in their name, the
| code isn't really self-documenting.)
| fsociety wrote:
| The analogy actually makes sense if you've worked
| manufacturing things from blueprints.
|
| If you hand one complex blueprints to three random
| construction firms, from a distance you'd get the same
| result, nearly, but up close a lot would've changed during
| the project.
|
| There is a reason why engineers are required to inspect the
| project at some interval and perform quality checks.
| learc83 wrote:
| At some level sure there will be differences, but for the
| end users, unless something went wrong, they won't be able
| to tell much of a difference.
|
| Blueprints for a house are much, much closer to being
| reproducible than design docs for a software project.
|
| If you give design docs to 3 separate dev shops, the end
| results will be wildly different.
|
| >There is a reason why engineers are required to inspect
| the project at some interval and perform quality checks.
|
| It's true that there is still some room for interpretation
| because blueprints are still a model. But most of what your
| talking about is because people will cut corners and not
| follow the specifications, not because the specifications
| aren't there.
| cameronbrown wrote:
| I mean, you're right... but the goal of an analogy isn't to
| be perfect, but to be a rough mental model to quickly express
| a concept. I think it's fine.
|
| Different jobs require different levels of abstraction and
| design docs fulfil that role for software engineers.
| learc83 wrote:
| Yes models can be helpful even if they aren't perfect. But
| not only is this one too far off to be useful, it does more
| harm than good. The expectations that happen when people
| (specifically managers) start thinking of design docs as
| blueprints, software architects as architects, and
| developers as builders are downright dangerous.
| sgt-serverless wrote:
| Dangerous seems a bit extreme. If developers aren't
| builders, then what are they?
| DEADBEEFC0FFEE wrote:
| Models are always imperfect.
| shuntress wrote:
| That is true but if the analogy is too imperfect then it
| won't express the right concept.
|
| A bad analogy is like a leaky screwdriver.
| gowld wrote:
| An analogy is a blueprint for an explanation.
| everlost wrote:
| My experience has been that writing the design doc is not as
| controversial, as much as the review process.
|
| Does anyone have any insights on how to make the reviews more
| constructive + quicker?
| solidasparagus wrote:
| Some things I've found useful.
|
| - Have a single person (engineer) who is responsible for
| making final decisions. Gather input from others but there
| should be one individual who is responsible for making sure
| that the design has a consistent vision that meets the goal.
| If a non-critical decision is taking too long, just make a
| choice and run with it.
|
| - Be clear about the goal of the project. At Amazon they use
| the PRFAQ process where you need to begin by writing a
| costumer-centric press release that basically details why
| customers should be excited about the project. This helps
| keep design discussions focused on the customer.
|
| - Be clear about the non-goals or things you will not do.
|
| - At the beginning of a review meeting, have everyone read
| the document. The document should describe all relevant
| context. At the end of reading the doc, everyone should have
| enough context to understand and discuss the design -
| everyone should have the same understanding of the design
| (i.e. no confusion about which version of the doc we're
| talking about, no one who didn't have enough time to read the
| doc beforehand and just skimmed it).
|
| - Have people add comments as they read through it, the
| authors can respond to the comments in real-time and once
| everyone is done reading the doc, the comments act as a list
| of things that need to be discussed.
|
| - Have the doc be the absolute source of truth. If it's not
| in the doc, we're not planning on doing it. This prevents
| cases where you have a chat with someone about an idea, look
| into it further and decide to not do it/do it differently,
| but the reviewer thinks you are still going to do it, but you
| just haven't updated the document yet.
| whoevercares wrote:
| The review process has a human factor in it IMO. I've seen
| many designs get scratched just because "this should be done
| and designed by Team B rather than you". Problem is that
| could come late. Sometimes it could be a career development
| bummer for the engineers who invest a lot into the design.
| etse wrote:
| But isn't that better than code (especially if working)
| that gets scratched? I think designs that are scratched
| indicate you don't understand the value you contribute to
| your company (or that management sees in you) which should
| be the root concern for the career development.
| whoevercares wrote:
| Yea, designs are cheaper then actual implementation. It's
| not always the case that the engineer doesn't understand
| the value but most times it's about the management/PM
| can't get things finalized over a long period of time.
| Sometimes I feel it's just politics in some cases. Don't
| you ever need to fight to do some high visibility work?
| etse wrote:
| This is a good question. It's funny, I'm the technical
| co-founder at my startup, so I hadn't thought to make the
| distinction between high and low visibility work. I guess
| we're small enough that I think about it as high
| impact/cost and low impact/cost. My non-technical co-
| founder will praise an engineer for their work when the
| product metrics are moved, which has happened for mundane
| tasks, too. I tend to notice the craftsmanship more, but
| I think it's right for attention to be given to the
| impact.
|
| I have also been at a larger company, so you bring up a
| good point about fighting for work. I also think this is
| right.
|
| 1. If you win the fight for high visibility work, that's
| a good sign that you're the right person for the job. It
| wouldn't make sense for the company to have you work on
| something for your own benefit over the company.
|
| 2. You may opt not to fight and discover other problems
| to work on that have strong potential for being high
| visibility. Maybe that's something relegated to PMs or
| eng management, where the engineer might feel helpless to
| the given assignments, but that's more of a culture
| problem. I think politics increases when there's less
| work/budget/praise than there are people, which is more
| situational than cultural, in my opinion.
| whoevercares wrote:
| Agreed. On the second point, it's definitely a bit
| cultural thing of the group/org. Worth note that at the
| higher level of the Corp ladder, there's indeed fewer
| work(with the wanted scope) then the people (that's
| looking to do next level work)
| eckesicle wrote:
| My thoughts on this matter:
|
| 1. Start with defining the problem you want to solve and
| identify the stakeholders. Then meet each stakeholder 1-1 and
| get approval.
|
| 2. Propose an API. Then meet 1-1 again for approval.
|
| 3. Make a high level arch diagram. Seek approval again.
|
| 4. Proceed with actually writing the design doc and begin the
| formal and official review.
|
| Points 1-3 could be done in a week if planned well and will
| make the design doc a lot less controversial and get you
| through the review process quicker
| jakevoytko wrote:
| > Navigating organisations and achieving consensus between a
| lot of teams/technologies is a huge part of it
|
| I think this is a major benefit of design docs - they are a way
| to extend your engineering influence beyond your own individual
| output. If you write a design, and your design allows you and
| three other engineers to coordinate your efforts, then your
| engineering output is now "I coordinated a team to build
| something that any of us couldn't have written individually."
|
| This dovetails nicely with your next point - uncovering
| blockers as early as possible is critical when coordinating a
| bunch of entities. Project plans are usually written on the
| assumption that every task will succeed, but there may be extra
| tasks added. If a task cannot be completed / you need to
| redesign something / etc, this will suddenly bring the work of
| N engineers to a halt. The earlier you successfully split the
| work into a series of known unknowns and implementation tasks,
| the better the project will go in general.
| mikepurvis wrote:
| > ... uncovering blockers as early as possible is critical
| when coordinating a bunch of entities.
|
| I think the challenge for me has always been that the
| "uncover blockers" piece means building one or more small
| prototypes to prove out the capabilities of the dependencies,
| check feasibility, etc. So the building of these prototypes
| occurs prior to or in parallel with the authoring of the
| design doc, but then at a certain point they get paused so
| that the design doc can be completed and reviewed, and then
| picked up again when it's time for the "real" implementation
| to occur.
|
| But pausing there takes discipline, since it ideally happens
| at the exact moment when all the main blockers have been
| cleared away and it is maximally tempting to just step on the
| gas and start into the work of cobbling the prototypes
| together into the project.
| mLuby wrote:
| It's also important to set clear expectations with
| stakeholders who have seen the prototype and may think the
| project is 90% done, when in fact there's still 90% more to
| go in making that prototype production-ready.
| Aperocky wrote:
| > I know this is fairly controversial, but our jobs isn't just
| to write code.
|
| I don't know if this is actually controversial, but I would not
| want to work at a place where this is controversial.
| eitally wrote:
| I think -- from experience -- that in traditional IT
| organizations this is absolutely not true. The PMO is
| responsible for business requirements, and almost not time is
| invested in the creation of detailed DDs. You're essentially
| left trying to move from a PRD to writing code ... and if you
| have the luxury of an in-house SQA team and a relatively
| effective CI/CD process, you depend on quick feedback from
| stakeholders to certify whether the guesses and assumptions
| made by developers are correct or not. Since in many cases
| the business owner(s) had never actually considered many of
| the edge cases (stupid, basic example: previous company
| required CFO approval of all international travel requests.
| Tool was built with this logic. No one considered how to
| handle it when the CFO was unavailable for whatever reason,
| so the first time the CFO went on vacation it created all
| kinds of stupid chaos while a manual process was created.),
| flexible logic paths aren't designed into workflow apps.
| Similarly, for reporting/BI tools, there are frequently gaps
| between the answers executives want to glean from the data,
| and the business processes the ultimately result in the data
| creation. Because of this, nearly all reports are faulty, but
| unless you're close enough to the processes you don't have
| explainability and uninformed business decisions can result.
| Ditto from CRUD apps, where form validation rules can be
| insanely complex for stupid reasons, with the result being
| the data entered is impossible to use.
|
| Apologies for the diatribe. Big fan of design docs, but an
| even bigger fan of software engineering cultures that focus
| on simplicity and usability, rather than being everything to
| everyone (or worse, being a pet project of an exec who
| changes their mind every quarter about how things should
| work).
| sanderjd wrote:
| I think this myth comes about because early in peoples'
| careers, the expectations of the job are a lot more focused
| on writing code to execute a vision defined by someone else.
| It is easy to get the impression from this that writing code
| is viewed as the most important part of the job. But in
| reality, it works this way because the opposite is actually
| true, the more important non-coding parts of the job are
| being entrusted to more experienced people (often to their
| chagrin, because writing code is way more satisfying and
| fun).
| rogy wrote:
| One thing I've noticed as I've got more experienced was that
| when I started out, a PM or senior engineer would give me a
| task to do.
|
| I achieved the outcome, mostly with bad to average code that
| slowly improved over time.
|
| When I finally understood the domain to make bigger scoped
| decisions, I started doing design docs beforehand and my code
| continued to improve greatly.
|
| Now when assigning out work to more junior engineers I find
| myself giving them a high-level design doc, with some detail
| missing, they deliver higher quality work than I did at that
| stage, and they also seem to upskill faster.
|
| This however depends on me making the right decisions at this
| stage, which is not always the case, so not fool-proof but an
| overall software quality improvement has definitely occured.
| musingsole wrote:
| Blessed are those who provide fully (or reasonably) spec'ed
| work to juniors trying to piece everything together against a
| book they read once.
| choppaface wrote:
| It would be great to see some actual design docs in a blog post
| versus a summary article with zero primary references. (A
| postmortem of some design docs would be incredible!).
|
| One of the hardest things about introducing a design doc process
| to a team of engineers is providing concrete examples as seeds
| and ensuring feedback to make the process actually work. It can
| be really jarring to introduce the design doc philosophy to a
| larger team if all they've been doing is Scrum every week.
| Moreover, typically design docs happen at a quarterly rhythm and
| feedback periods can take weeks-- this slow cycle is a major
| impediment to consensus and learning.
|
| All this calls for material that speaks to differentiated
| learning. Extolling "we do this at Google" is a leaf in the wind.
| floil wrote:
| Here you go: https://www.chromium.org/developers/design-
| documents
| rckoepke wrote:
| I'd love some examples of real-world design docs.
| mbakke wrote:
| The Ganeti virtualization toolkit which was developed
| internally at Google and released as free software is full of
| such design docs. I always enjoyed reading those as an external
| Ganeti user:
|
| http://docs.ganeti.org/ganeti/master/html/#implemented-desig...
| lowiqengineer wrote:
| I wonder why the "at Google" is necessary for the title. I'd be
| very surprised to see an organization not use some sort of design
| doc before building things - the only thing that would be
| different is the scale and scope of them.
|
| Of course, you won't see a "Design Docs at Amazon" on the front
| page of HN for...some reason. Gee, I wonder why?
| AnonC wrote:
| > I'd be very surprised to see an organization not use some
| sort of design doc before building things - the only thing that
| would be different is the scale and scope of them.
|
| There are companies that use agile methodologies and eschew any
| kind of design documentation (they kneel at the altar of
| working code, and working code alone). I've seen teams just
| discuss things, get a few inputs and start writing (working)
| code and pass it through testing. The design process exists,
| but there's no document or documents for anyone to refer to.
| There may be high level architectural documents, but not what
| many people understand as "design documents".
| sateesh wrote:
| I always wonder how such an approach works towards building
| maintainable software whose scale is not trivial. Even
| assuming the code is reviewed unit tested, but still without
| some kind of design doc guiding the overall structure, and
| interactions wouldn't it lead to hard to maintain software ?
| WeNeedDemocracy wrote:
| At my place of work we probably wouldn't follow this simply
| because we don't love the design of Google's products
| vngzs wrote:
| IMO the ideal format for conveying this information is to show,
| not tell. Give me an example document and annotate it with
| sidenotes that describe the purpose and importance of each
| section.
|
| Then we get to see how well it works, not leave it up to
| imagination.
| time0ut wrote:
| Great article. I spend a lot of my time writing design docs these
| days. We derive a ton of value from it.
|
| The projects I work on are typically collaborations with several
| developers. It is important to all have the same vision for what
| we are building. A good design doc is a vehicle for a team to
| sync their mental models of the thing they are building.
|
| We also often have many other teams (security, infrastructure,
| operations, other business units, etc) who all have a stake in
| what we are doing. A good design doc helps build consensus with
| them. The review process gives them a voice in calling out
| concerns. It is much easier to solve a foundational problem
| before any code is written.
|
| One addition that isn't explicitly called out by the article, but
| I have found very valuable, is the inclusion of references. In my
| experience, good design docs typically include some references.
| Links to the docs for a piece of software we want to use, links
| to an AWS blog post explaining part of the architecture we want
| to use, links to academic papers explaining the algorithms we
| want to implement, etc. Good references support the design doc
| and help readers fill in knowledge gaps they may have.
| bigbizisverywyz wrote:
| >It is much easier to solve a foundational problem before any
| code is written.
|
| This brings to mind the 1-10-100 rule. It takes 1 unit of
| effort to fix a problem in design It takes 10 units of effort
| to fix a problem during development. It takes 100 units of
| effort to fix a problem after going live.
| Ozzie_osman wrote:
| Honestly I sometimes write a design doc even if no one else will
| read it. It can be as short as a few sentences, if needed.
|
| The main benefit is it forces you to:
|
| 1. Think strategically, not tactically, about what you're about
| to build. Beyond just "I need to get this working now".
|
| 2. It makes sure you can describe what you're about to do in
| plain English (or plain English plus some diagrams). If you're
| struggling to do that it's probably a sign you need to rethink
| your approach.
|
| Obviously, the organizational benefits described in the link are
| really important, too, but if all you've done is just 1 and 2
| above that's already worth your effort.
| dgb23 wrote:
| Related: https://www.joelonsoftware.com/2000/10/02/painless-
| functiona...
|
| My opinion:
|
| If you don't specify things in a separate document in the form of
| text and pictures, then you end up doing it anyway but ad-hoc.
|
| Your "non-spec" now leaks into multiple tools and channels that
| likely don't talk to one another: emails, instant messages, file
| storage, notes, phone calls & discussions (AKA in your
| head/memory), Trello/Asana/Jira etc. and in peoples imaginations
| and expectations.
|
| A specification is just the things you do _anyway_ but instead
| written down in a single maintained place. You can source control
| it and iteratively change it. All the parties can pull it and
| look at it before /during/after iterations and discussions.
|
| A spec doesn't need to be set in stone at all. It can
| grow/shrink/change iteratively with the thing you build. It can
| also define things that you specifically shouldn't do or worry
| about.
|
| A "design doc" seems to be something like that with a different
| name?
|
| Also a specification can help to communicate much more clearly,
| since you end up defining a _vocabulary_ for processes, UI
| elements, technical components and so on.
|
| Tools I find useful to do this:
|
| - markdown + css + some tool to convert it to html/pdf
|
| - graphviz, especially for state-machines and decision trees, I
| prefer output as SVG rather than PNG.
|
| - UML diagrams or similar if applicable
| bigbizisverywyz wrote:
| A really nice framework for specifying a system, I've found, is
| the vision document from RUP. It's great for putting the user
| front and center, and ensuring that you're building a system
| that people actually want and have a realistic business case.
| H8crilA wrote:
| A design doc is not exactly a spec. It is a medium for
| discussion and a snapshot of what people thought at the time.
| They do get old pretty quickly and there's no expectation that
| anyone will update them (always put dates on such documents!).
|
| I think that specs should be checked into the codebase, either
| through some kind of code annotation, or next to it in text
| files, depending on what is it that you're doing.
| dgb23 wrote:
| Thank you for clarifying. My question is: when do I need a
| design doc, given I already have a maintained spec? Would
| they be used for as drafts or are they a complete separate
| thing?
| maps7 wrote:
| I think a design doc is for when you don't know enough to
| spec a solution.
| joatmon-snoo wrote:
| I tend to reach for a design doc when I have (1) an
| ambiguous problem, (2) no obvious solution but N possible
| candidates, and (3) need feedback from multiple parties.
|
| The primary functions (as people have mentioned elsewhere)
| are to (1) drive consensus and (2) record thought processes
| at the specific point in time.
| petejames wrote:
| Writing helps you think. In the process of creating a design doc
| you get clarity of thought and refinements happen.
| eximius wrote:
| Half the time these are written after the fact (or updated after
| major changes), so they're often used as shitty documentation. I
| don't disagree with them, in principle, but I've yet to really
| see the benefit of them. The adage "In preparing for battle I
| have always found that plans are useless, but planning is
| indispensable" seems to apply.
| gregdoesit wrote:
| "Unstructured text, like in the form of a design doc, may be the
| better tool for solving problems early in a project lifecycle, as
| it may be more concise and easier to comprehend, and communicates
| the problems and solutions at a higher level than code."
|
| I could not agree more. I came across the concept of design docs
| consistently used for all projects at Uber. They were called RFCs
| here and they were introduced very early on. I wrote in more
| detail about my experience using these and how I am pretty sure
| they helped scale the engineering culture [1]. The most
| surprising thing I've observed is how sending these design docs
| out to all of engineering worked really well, until the org was
| over a thousand engineers.
|
| [1] https://blog.pragmaticengineer.com/scaling-engineering-
| teams...
| ChrisMarshallNY wrote:
| I'm of a mind to reduce documentation as much as possible, or
| keep it as vague as possible.
|
| That's because I came from an environment that insisted on
| incredibly detailed, formal, approved-by-everyone-including-the-
| mens-room-attendant, documents.
|
| These became "concrete galoshes"[0] that turned what should have
| been an agile, iterative project into a waterfall behemoth that
| cost a mint, took forever to make, and delivered a result that no
| one wanted.
|
| I suspect that my current process[1] would not be acceptable for
| many organizations, as it basically requires that everyone
| involved be extremely experienced, and kept together as a team
| for years.
|
| I've actually come to realize that "tribal knowledge" is not the
| boogeyman that its made out to be; but is a way to get very high-
| quality, rapid, adaptive, development done. The main issue is
| that it does require good, empathetic management, long-term
| retention of experienced, capable people, and an "apprenticeship"
| model. These are not popular concepts in today's corporate world
| (at least, in the US).
|
| [0] https://medium.com/chrismarshallny/concrete-
| galoshes-a5798a5...
|
| [1] https://medium.com/chrismarshallny/forensic-design-
| documenta...
| smichel17 wrote:
| Meta: Why do you link to medium when the articles are also
| posted on your website, which I assume is the canonical version
| if there were a discrepancy?
|
| Also, I reviewed your privacy policy and while I am not a
| lawyer, I believe your cookie popup is not unnecessary. As far
| as I understand GDPR, it is not necessary to ask for consent
| _separately_ if the cookies you 're setting are strictly used
| to fulfil an explicit request. Eg, if a user checks "stay
| logged in", you don't need to ask separately to set a
| "wordpress_logged_in_xxxxx" cookie; the user already gave their
| consent by checking the box. Same with a session cookie when
| they log in, or a cookie for persisting settings. Not to
| mention, I there's no prominent log in link, so I'm guessing
| you are the only one who does that, in which case you'll never
| be setting cookies on other people's computers anyway :)
|
| These things still need to be described in your privacy policy,
| but you don't need to bother users with them; you only need the
| consent box when you're using cookies for something the user
| doesn't explicitly request, like tracking (regardless of
| whether it's a discrete tracking cookie or you're using their
| login cookie to do it). I didn't see any of these things in
| your website privacy policy.
|
| Not needing to bother your readers with consent popups is a
| great advantage that the GDPR gives privacy-respecting websites
| like yours!
| ChrisMarshallNY wrote:
| Thanks! That tip about the popup is good. I'll remove it.
|
| I would be happy to link to my regular site, but I assume
| that people prefer Medium; I guess because of the
| standardized layout (which I don't love), and the ability to
| use apps and semantic API hooks.
|
| There are a few series on my site that are not available on
| Medium [0][1][2][3][4][5]
|
| [0] https://littlegreenviper.com/series/swiftwater/ (I
| republished one of these on Medium).
|
| [1] https://littlegreenviper.com/series/spm/
|
| [2] https://littlegreenviper.com/series/bluetooth/ (Which was
| actually #1 on HN for a little while -St. Paddy's Day)
|
| [3] https://littlegreenviper.com/series/bluetooth-2/
|
| [4] https://littlegreenviper.com/series/streaming/
|
| [5] https://littlegreenviper.com/series/stylist/ (Which is a
| decade old)
| lhorie wrote:
| I think something people need to be aware of is that the
| engineering culture at places like Google is extremely
| different than, say, a bank.
|
| Design docs work well for Google because it has decentralized,
| independent engineering teams composed mostly of reasonable
| people, with few or no mandates from outside their teams. They
| tend to fail at companies with underperforming, micro-managing,
| and/or overreaching teams (e.g. unionized project managers that
| have literally nothing to do, bizdev telling developers what
| features to implement (down to specifics), QA teams who are
| incentivized to find "flaws" ad-infinitum to look busy, etc).
|
| But I see this as a symptom of a fundamental dissonance in
| goals. Design documents are used by organizations whose goal is
| to build/change things. Many organizations are averse to
| change, and their processes reflect that, through overly
| microscopic documentation, convoluted approval chains, etc.
| ChrisMarshallNY wrote:
| Yeah, you are correct.
|
| But I have become rather cynical about the motivations of a
| lot of organizations, these days.
|
| Everyone seems to be promoting an environment where there's a
| constant circulation of relatively inexperienced (not always
| low-paid, but inexperienced) developers, staying at companies
| for short periods of time, then moving on.
|
| This requires a codified, ingrained structure that needs to
| be documented and supported. It applies to modern, "agile"
| companies, as much as it does to more traditional,
| "hidebound" corporations. People that come in need to be
| onboarded quickly, squeezed for every ounce of productivity
| possible, then let go, when they are no longer useful.
|
| This is supported by the employees, as much as by the
| managers. The salaries can be quite high; especially for
| people that are quick to adapt and come up to speed.
|
| But there is absolutely _no substitute_ for an experienced,
| cohesive team that has developed a shared vocabulary and
| focus.
|
| I was fortunate to be a member of such a team. The person
| with the least seniority on the team had a decade. Everyone
| had over 30 years' experience in software development, and we
| got some fairly intense stuff done. Sadly, we were still
| subject to the kind of structure I mentioned above, and a lot
| of our product ended up being housed in substandard wrappers
| (we did "engine" code).
| bigbizisverywyz wrote:
| >I think something people need to be aware of is that the
| engineering culture at places like Google is extremely
| different than, say, a bank.
|
| I have to say that this is not uniformly true, I've worked
| for a few financial institutions, and usually follow some
| sort of design process before embarking on any sizable work
| (as mentioned above, I find the RUP process to be a really
| useful framework). More often that not the design process is
| usually welcomed because people are happy that someone is
| willing to create a holistic view of the problem (from
| stakeholders down to db schema and class diagrams). The
| process helps with project management, identifies
| stakeholders and describes the support requirements after
| transition to prod.
|
| Banks actually love somebody to detail all that.
|
| In a few cases I'm aware of, my design docs are still
| referred to years after I've moved on since they capture the
| most important thing that everybody forgets - basically the
| 'why' that certain decisions were made, and the alternatives
| rejected.
|
| A good architecture diagram never goes out of date either :)
| bo1024 wrote:
| That environment reminds me of Conway's law[1] - code structure
| reflects organizational structure.
|
| It sounds like the problem was decisionmaking power was spread
| out among far too many people. Detailed documentation was a
| side effect. Removing the documentation likely wouldn't have
| fixed the problem.
|
| [1] https://en.wikipedia.org/wiki/Conway%27s_law
| skipants wrote:
| > The main issue is that it does require good, empathetic
| management, long-term retention of experienced, capable people,
| and an "apprenticeship" model.
|
| To me this sounds like an idealistic environment. I doubt this
| environment could fail with any sort of process (or lack
| thereof).
|
| Documents don't have to be "incredibly detailed, formal,
| approved-by-everyone-including-the-mens-room-attendant". Those
| are probably the worst kind. I do think that OP's/Googles
| design docs does seem to lean towards that, unfortunately. I
| wonder if that's how it is in practice? Especially given the
| length -- 10 to 20 pages sounds like it's too detailed for me.
| Though I suppose on bigger projects the sum of all the
| implementations of the moving parts can get quite large.
|
| One criticism I especially have of OP's "Review" section is
| they mention opening it up to a large audience. I am skeptical
| whether that is how it actually goes down. In my experience,
| opening something up to review by many people just ends up
| being a too-many-cooks situation. Pretty much what you're
| afraid of with "approved-by-everyone-including-the-mens-room-
| attendant". Those situations just turn into bikeshedding. In
| reality, I bet it gets opened up to everyone but only a couple
| people are actually expected to be the ones who give feedback
| on it.
|
| I still think I'd take the over-formalized process over tribal
| knowledge. What you don't see behind the scenes of tribal
| knowledge are people with a lot of it who constantly get
| interrupted to answer those questions. It ends up being very
| inefficient at scale. Not to mention the overhead that comes
| with the person trying to answer the question -- like finding
| old breadcrumbs about the problem like code or emails. Along
| with the worst enemy of work for a developer: context
| switching. I've been the person with tribal knowledge and it
| basically becomes your full time job. It was great for job
| security but I lament the amount of time I lost because I
| wasn't more proactive about writing documentation.
| ChrisMarshallNY wrote:
| _> What you don 't see behind the scenes of tribal knowledge
| are people with a lot of it who constantly get interrupted to
| answer those questions. It ends up being very inefficient at
| scale. Not to mention the overhead that comes with the person
| trying to answer the question -- like finding old breadcrumbs
| about the problem like code or emails._
|
| Absolutely correct, which is where that "good management"
| comes in.
|
| I managed the team that I mentioned, earlier, and I'd like to
| think that I did it well.
|
| My single biggest task was to shield people from corporate
| shenanigans, take on as much of the structural overhead as
| possible, and be a "gatekeeper."
|
| I managed the team for 25 years. We got a lot done, in that
| time, but also suffered a lot of "lessons learned." I
| certainly made my share of mistakes; most of which were about
| me deprecating my "management shield" responsibilities, in
| misguided attempts to be a working team member.
|
| I kept my tech chops up in extracurricular open-source work.
| No one wanted to pay me for my tech skills.
| shadowgovt wrote:
| > basically requires that everyone involved be extremely
| experienced, and kept together as a team for years
|
| Yep, that's the issue. Businesses can't codify that approach
| into a process because they can't make any guarantees around
| that.
|
| Your approach is "conclave of wizards." It's a high-output
| approach that is the right thing for some problem domains, but
| (a) it isn't repeatable and (b) it isn't sustainable (if a
| critical mass of wizards leave the conclave, what is the
| company to do? Give up and shut down?)
| ChrisMarshallNY wrote:
| _> Give up and shut down?_
|
| In some cases, yes. That's _exactly_ what needs to happen.
|
| If the business model is dependent upon the "conclave of
| wizards," then they are a critical path resource, just like a
| major supplier or business partner, which, if withdrawn,
| could doom the company just as certainly.
|
| If the corporation refuses to treat that resource as a
| valuable, critical resource, then they don't deserve to stay
| in business.
|
| Managing that type of team is not something that is really
| taught in school. It comes from experience, and also requires
| a level of empathy that is, quite honestly, almost
| nonexistent in today's business culture.
| shadowgovt wrote:
| Indeed. So the correct thing for a business to do for both
| its short- and long-term survival (since we can't know
| a-priori whether the conclave of wizards will be stable) is
| to minimize the odds of forming one in the first place.
|
| Businesses are structures that minimize risk. A core set of
| specialists who can't be replaced without tanking the
| business's performance is a risk. And a hard-to-quantify
| one at that.
|
| _Unless,_ of course, the business is _not_ setting out to
| minimize risk, but instead to shield some less-risky
| structure from outsized risk exposure. Like, say, a startup
| company funded by angel investors trying out experimental
| technologies in the hopes of hitting on a jackpot new
| system that a big corp could then buy. If they fail, only
| the angel investor money is gone. If they succeed, a risk-
| averse business can buy a valuable asset.
|
| But the first thing that business will want to do to
| maintain its long-term risk profile is fire the wizards. ;)
| ChrisMarshallNY wrote:
| But...those "wizards" can do some unique stuff. Most of
| the time, the only reason to have them, is to do stuff
| that a "generic" team can't do.
| bruckie wrote:
| I find design docs really useful, even if no one else reads them,
| because they force me to clarify my thinking before I start the
| (more expensive) process of implementation.
|
| However, I've also noticed a couple of very common problems with
| design docs:
|
| - Many design docs don't clearly state the problem that the
| design is intended to solve. Often the best response to a design
| is "that problem isn't important to solve", or "there's another
| system that already solves this problem", or "there's a totally
| different approach that you should consider", but without a clear
| statement of the problem and why it's important to solve, it's
| hard to determine those kinds of things.
|
| - People often focus too much on the details of a design, vs. the
| important decisions that need(ed) to be made, the alternatives
| and rationale for those decisions, and the assumptions that they
| were based on. In a year, you probably won't care that much about
| the details of the parameters of some API, but you probably will
| care a lot about why you chose that API over some other one, and
| it's useful to have a record of that decision.
|
| It's useful to view a design doc as a distillation of the thought
| process behind a system, vs. a high-level description of how to
| build something.
| whoevercares wrote:
| Frankly I think Amazon has a even heavier design culture. The
| designs for a new project (e.g to be launched at reinvent) could
| go as long as more than half year.
| koheripbal wrote:
| Does duration really measure depth of design?
| whoevercares wrote:
| Yeah some are churns from reviewers, partner teams or even
| politics. The depth of the design are commonly pretty deep to
| the ground, because Amazon has many SDE1s with relatively low
| hiring bars and the design need to be super detailed for a
| safe execution
| akhilcacharya wrote:
| > SDE1s with relatively low hiring bars
|
| as an SDE1...yikes dude.
| whoevercares wrote:
| Sorry I mean compared to Google/FB... But that doesn't
| translate to productivity though
| jkaptur wrote:
| It's important draw a distinction between a "design doc" and
| "documentation". tl;dr: "ought" vs "is".
|
| A "design doc", as described in the article, is a tool to help
| _make a decision_. It 's a proposal. It is "ought". It describes
| the state of the world at a point in time, the desired change,
| and how to make it. Once the decision is made and the work is
| begun, it's rarely updated.
|
| "Documentation" is just _the state of the world_ (generally
| "now" or "at release x.y"). It describes the design and use of
| the system, but has very little digression into ways it could be
| different. If it links to a design doc, it should be for
|
| Once you draw this distinction, it's clear that design docs
| should be in something like Google Docs, while documentation
| should be in something like a wiki or git. It's clear why it's
| fine for a design doc to be out of date, but it's a big problem
| if documentation is. etc.
| justicezyx wrote:
| Hah, design docs at Google...
|
| I read probably hundreds of deigns docs during my 6 years at
| Google.
|
| I think among them, only a handful of design docs, like < 10, are
| considered clear, clean, and succinct. And all of them are more
| or less retrospective ones summarizing a system after it's
| already being implemented and running for a while.
|
| The newer design docs, which are served as actual working
| documents for an active projects, are no doubt primarily a
| _process mechanism_ to solicit design inputs, and garner supports
| from stakeholders.
|
| It is a formality to show that the project owner is willing to
| _waste_ his time and to convince so-called "reviewers" about the
| validity of the project. The reviewers usually do not read the
| design doc in earnest. Only the stakeholders, like the TL of the
| team, potential customers, partners, would spend serious time on
| them. It's more or less a debate ground for stakeholders to
| pointing fingers at the design process. That's also why the
| design doc is written in Google Doc in the first place.
|
| Once a design is "approved", it usually means from most major
| share holders are satisfied, to everyone is wearied down enough
| and the project has enough importance to actually push forward,
| they are OKed for implementation.
|
| After that the design doc is largely useless, the designs laid
| out in the doc is usually 1) too simple that there is no need to
| consult design doc anyway, one can explain it in a few minutes
| chat; 2) too complicated that the design diverges from the actual
| code that the design doc offers little guidance. And very few
| design docs sit in between.
|
| All in all, anything produced during a software engineering
| project seems all is about being a tool for organizing the human
| interaction process, the artifact, aside from a very high-level
| motivational piece, and the end result code + docs, whatever in
| between seems largely bears little value after the work is done.
| jp42 wrote:
| After reading this article I can relate it event trace (Chrome
| tracing) design doc form google, very succinct to the point:
| https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQt...!
| jeffbee wrote:
| That is not a design doc, that is a format specification. It
| jumps directly to the implementation details without discussing
| the motivation. The reader has no way to evaluate the utility
| of the example JSON because the doc has not stated any use
| cases.
| cmrdporcupine wrote:
| And frankly that's how most of the design docs at Google
| really are. They're post-facto descriptions put together for
| the purpose of gaining promotion.
| jeffbee wrote:
| Yes and the way you can tell is the doc title is "FooBar
| Design Doc" but the shortlink is "foobarforperf".
| britbull wrote:
| > Are you unsure about the right software design, and would it
| make sense to spend upfront time to gain certainty?
|
| This is a great question to ask. It's human nature to lack
| foresight, especially when you are under pressure from deadlines,
| but in the long run it saves so much time.
| commandlinefan wrote:
| I like the idea of design documentation, but in 30 years of
| software development experience, I've never seen one done well.
| TFA does a reasonably good job of outlining what you'd want from
| an ideal design document but doesn't offer any advice on creating
| one that actually meets these criteria.
| [deleted]
| AgloeDreams wrote:
| Basecamp pitches and shaping are kinda like design docs but
| with a lot of what I found to be lacking here? You might want a
| look, they they refreshingly human-readable concepts.
| theptip wrote:
| OP links to a design doc template that implements the ideas it
| lays out:
|
| https://www.industrialempathy.com/posts/design-doc-a-design-...
| colonelxc wrote:
| One trick google uses is that there are 'standard' templates to
| start your design doc from. So it has all the sections
| mentioned in the post, and each section is initially filled
| with a description of what is needed there (and sometimes links
| to other supporting materials).
|
| Not everyone will be diligent and fill out all applicable
| sections, but it does help bring some uniformity to the
| process.
| thomascgalvin wrote:
| I think forcing a team to ask these questions is more important
| than making the document perfect. I've had too many
| conversations that amounted to "a React front-end, Node
| backend, Mondo data store is the perfect solution to this and
| every problem," with no other options chosen.
|
| There's also value on having some this kind of documentation
| when you're bringing on new people. The project I'm currently
| working on started two years before I got here, and I would
| love to have a document that explained why, for example, we're
| using Active Directory instead of some other authentication
| mechanism.
| gowld wrote:
| Same could be said for code, right? It's a great idea, but in
| 30 years we've never seen code done well.
|
| "Well" is a slippery word.
| commandlinefan wrote:
| But code actually works (eventually, usually). If you don't
| have code, you don't have software. If you don't have a
| design document... it's usually the same as if you do have a
| design document.
| etse wrote:
| In my experience, it's not the same. Design docs allow a
| dialogue about the resulting code that code reviews are
| poorer at or less efficient at. Those conversations have
| saved us time and gives us confidence that the solution is
| the best for the problem.
| jhallenworld wrote:
| Design docs should be written in Markdown and stored in git. The
| advantage here is that the git hash can be used as a legal
| reference, for example in the case where a contractor is expected
| to implement the design.
| bbeekley wrote:
| We did this at a previous company in the healthcare space. It
| made audits easy since we could point to every instance of the
| design changing, who approved it, and the code that implements
| the change (all wrapped up in PR(s)).
|
| In general I like checking docs into source control, for all
| the reasons we use git and code reviews for the code itself.
| dilyevsky wrote:
| I've tried this and I tried just gdocs like everyone at google
| did. Gdocs seems better for iterating because github review ui
| sucks (and everyone else sucks even more). If google critique
| was publicly available I'd much prefer markdown for designs
| pmb wrote:
| The best is to get agreement on the design is gdocs and then
| export it to Github so that the design and code live next to
| each other.
| staplung wrote:
| I worked at Google for 4 years. One thing that has always
| surprised me about documentation at Google is that they use
| Google Docs (just like everyone else) but have never seemed
| interested in making it less of a terrible tool for the task.
| It's relentlessly print oriented, which makes almost no sense
| these days (when was the last time you printed out a Google
| doc?), has no affordances for dealing with pre-formatted text
| (e.g. code), has basically no support for mathematical formulas,
| etc. In fact its actually evolved to be less suited to the task
| of technical documentation since it began given that earlier
| versions at least allowed you to define your own paragraph
| styles. I was hoping that Google Colab might evolve into a
| better/more general tool for writing tech specs but it doesn't
| seem to be moving in that direction really.
| maxwellg wrote:
| Comment discovery is also horribly broken. So often,
| conversations _about_ a document contain information that don't
| necessarily _belong in_ the document, but still have value for
| future readers. Resolving comments makes it much harder to find
| them.
| xnx wrote:
| Exactly correct about the print-oriented interface. Writely,
| and earlier versions of Google Docs didn't force you to pretend
| you were writing on a piece of paper. I would love a "Google
| Docs for text files"!
| akhilcacharya wrote:
| At my company we use the internal Wikis or Quip for this.
| Versioned and effectively markdown formatted with builtin live
| charts or spreadsheets, respectively.
| wsetchell wrote:
| The article does not point out how the system can go astray.
|
| Design docs are great when they can in 1-2 pages describe a how
| an important/large/complex system will work and the tradeoffs
| made when designing it.
|
| Some groups in Google confuse design docs conflated with the
| promotion process, and thus engineers start create long/time
| consuming docs for every little thing as part of building a case
| for promotion.
| mediocreguy wrote:
| Any good examplars of docs that use this format?
| Slump wrote:
| Great article, generating consistent design docs that are both
| useful and not too time-consuming to create is something I am
| constantly working on. One thing I struggle with is finding
| examples of what others are doing in the industry. Does anyone
| have any resources they've found that provide examples of real-
| world design documents? Obviously each project is different but
| I'd be curious if others out there are publishing their
| templates/outlines of what they typically start with similar to
| this article.
|
| I feel like blog posts seem to serve this niche for a lot of open
| source projects but it would be interesting to see if anyone has
| found more structured examples.
| PascLeRasc wrote:
| Does anyone know where to find real examples of design docs,
| possibly following this guide? I don't mean toy examples but
| something that's actually used by real people/orgs. My workplace
| has design docs but they're full of made-up words, so I'm pretty
| burned out by this kind of thing.
| yegle wrote:
| The author of Beancount (a text based accounting software) is a
| Googler and he recently shared
| https://docs.google.com/document/d/1qPdNXaz5zuDQ8M9uoZFyyFis...
| which is a pretty standard Google style design doc.
| Offpics wrote:
| Deno design doc for example
| https://docs.google.com/document/u/0/d/1_WvwHl7BXUPmoiSeD8G8...
| PascLeRasc wrote:
| Thanks! That was interesting to read and gives me hope for a
| better workplace. So it's basically a much more thought-out
| Github issue/corresponding PR, would you agree?
| mbakke wrote:
| I've been a long-time user of a Google virtualization toolkit
| that was developed in the open. It is full of such design docs:
|
| http://docs.ganeti.org/ganeti/master/html/#implemented-desig...
|
| As an end user I always enjoyed reading those.
| vially wrote:
| This is one I follow related to adding multi-window support to
| Flutter: https://flutter.dev/go/desktop-multi-window-support
|
| I also found some of the chromium design docs quite
| interesting: https://www.chromium.org/developers/design-
| documents
| cpeterso wrote:
| Python's PEPs (Python Enhancement Proposals) are pretty good
| examples of design docs that describe the problem, proposed
| solution, and rationale: https://www.python.org/dev/peps/
___________________________________________________________________
(page generated 2020-07-22 23:00 UTC)