[HN Gopher] A Senior Engineer's Guide to the System Design Inter...
___________________________________________________________________
A Senior Engineer's Guide to the System Design Interview
Author : leeny
Score : 365 points
Date : 2023-03-02 18:13 UTC (4 hours ago)
(HTM) web link (interviewing.io)
(TXT) w3m dump (interviewing.io)
| FlyingSnake wrote:
| "When a measure becomes a target, it ceases to be a good
| measure".
|
| The interviewing cottage industry has become a sad joke at this
| point.
| debacle wrote:
| I remember once I was interviewing for a senior position where
| the ask was something like "design a general purpose system for a
| REST layer on top of an ORM."
|
| I had just finished implementing an OSS solution that did exactly
| that, including some upstream changes we made to improve the
| system, so I walked through exactly what we did, challenges we
| faced, etc.
|
| I walked out of the interview feeling as though the interviewers
| and I didn't speak the same language; they might've said the
| same. Not only did I not get the job, I never got another
| communication from the company.
|
| To me, senior level interviewing, especially at smaller
| companies, is fraught. A full 1/3rd of the companies I've
| interviewed at would not have been a good "culture fit," and I'm
| a picky interviewer. I expect, like coding tests etc, these
| interviews are not aligned with the real travails of the position
| - growing talent, managing time, triaging, and ensuring
| stability/capability.
| Aperocky wrote:
| The interviewer usually can only rise to the level of
| themselves, especially when assessing skills.
|
| System design is hard, necessary skills can only be earned, and
| then there are many avenue down to bad system design
| masquerading as good that people who implemented it just don't
| realize.
|
| It takes a combination of humility and a degree of the skill in
| question to recognize the interviewee is more proficient at the
| said skill, and many are lacking in either or both.
| nfw2 wrote:
| I've got an interview coming up for a senior role on a UX
| engineering team (design system, component libraries, etc). The
| interview is just the standard set of backend system design and
| leetcode problems.
|
| It's a pretty big red flag to me that the skills being
| evaluated are so tangential to the actual job. It doesn't give
| me any confidence that the people I will be working with have
| the skillset I consider important, or that the team and I will
| approach problems in a compatible way.
| jahewson wrote:
| I suspect your fears are well-placed. Whoever is hiring this
| team has no idea how to hire for those roles and that
| suggests that those roles might not be valued within the
| larger company. Though sometimes the hiring manager has just
| lucked out and hired a great team so far. Still, will that
| team be listened to? Will that team be respected?
| agumonkey wrote:
| I struggle to read this. It looks like a self-help ebook.
| omniglottal wrote:
| I got such fatigue from scrolling through so very much prologue,
| introduction, expectation-setting, and general "here is what we
| are going to say and here is how you may expect we will say it"
| fluff that I gave up, pages in, before getting to any apparent
| content.
| modzu wrote:
| its an ad, and they nailed the seo
| leeny wrote:
| Hey, founder of interviewing.io here. We actually did zero
| thinking about SEO. We had someone look at this guide at the
| end through an SEO lens, but then we decided that that would
| make the content not as good and would make the guide less
| readable.
|
| (As an aside, I think in recent years, the spirit of SEO has
| become more about just making good stuff and less about
| hacking...)
|
| Anyway, it wasn't for SEO. It's hard to edit stuff well
| because we're so passionate about what we wrote here, and we
| probably should do another pass.
| chrisabrams wrote:
| I'm very excited to see a guide that focuses on the process of
| the interview itself related to system design. There are plenty
| of "how to design twitter" resources out there, but for many of
| us, we really need to better understand the system design
| interview process itself. Thanks so much for putting this
| together, can't wait to see parts 3 & 4!
| newaccount2021 wrote:
| [dead]
| John23832 wrote:
| I think to be frank, this gamification of interviews is BS.
|
| I've built actual production systems at scale. The fact that I
| need to follow a guide to "demonstrate" my ability to someone who
| 9/10 hasn't built (or couldn't build) anything at scale in
| production shows where we are in the absurdity matrix.
| blue039 wrote:
| Yeah, I don't think I want a job that asks such questions. I've
| done the webscale BS. I've done actual architecture that exists
| in reality. At this stage in my career there's nothing an
| interviewer can do but insult me.
|
| You might say that this is hubris about my own skill but it has
| nothing to do with how good I am. My resume has a LOOONG track
| record of consistent work in the industry. Call my references,
| do some actual DD on me, then ask me real questions related to
| actual things.
|
| If leetcode and all these other service went straight to hell
| tomorrow the world would be a better place. This entire
| "interview" industry is propped up by a bunch of leeches
| capitalizing on a recently _extremely_ popular field. Then,
| injecting their BS to make the process harder thereby earning
| them dollars. The linked article is a perfect example. It 's
| actually an advertisement if you look close enough designed by
| these exact leeches. This is just a reimagining of the
| bloodsuckers who run SAT/GRE/GMAT prep services and "ex-ivy-
| league recruiter consultant" bullshit. They are the same people
| and the _only_ place they belong is all the way at the back of
| the breadline. That may be too generous for them anyway. There
| are better people that deserve the bread.
|
| There's a good chance at my career phase I have more experience
| than the interviewer. They "level the playing field" by asking
| me these stupid things. That is why it is insulting. I almost
| want to leave the industry entirely than have to do the process
| one more time. I don't need a 7 phase 360 interview with
| everyone including the CEO's cousin to insure I am a "culture
| fit". It was never like this before. It needs to stop.
| nonethewiser wrote:
| I guess I'm not sure if you're suggesting otherwise, but
| gamification is not unique to interviews. It's a natural
| response to any system that relies on metrics or measurements.
| The request to design a theoretical system is gamification as
| much the guide for answering it is. These games exist because
| it elicits answers that can be used to learn about someone's
| real experience level.
|
| It's great that you feel confident relying on your real world
| application but it takes a long time to get repetition on
| building things at scale in production. Practicing system
| design is a way to shorten that learning loop.
| eternalban wrote:
| _" You can pass system design interviews even if you've never
| designed distributed systems before. If you have copied files
| between machines with drag-and-drop, you are halfway there. If
| you implemented clients or servers or have opened network
| connections, you've got this. This guide will teach you the most
| important 20% of information that will appear 80% of the time in
| system design interviews. By the end of this guide you won't be
| an expert, but you'll be well on your way to being a better
| engineer and a much better interview candidate."_
|
| A better engineer, no less! Wow, this must be some magical guide.
| I have to read it now. Also what does this say about our
| industry. "You want to be a surgeon but don't have the
| experience? Ever cut a tomato? You're halfway there. If you've
| ever made a sandwich, you've got this. Nurse!"
| outworlder wrote:
| > "You want to be a surgeon but don't have the experience?"
|
| Well, what do surgeons have to do beforehand? Study. This is no
| different. Most of tech doesn't require fine coordination motor
| skills that need to be trained in practice, so this comparison
| is way off.
|
| Also, like the excerpt you quoted says, "you won't be an
| expert" after reading the guide. A surgeon is an expert by any
| definition.
| debacle wrote:
| Study, train, shadow, and intern for 11-13 years.
| siva7 wrote:
| That's in our terms a Director of Engineering. In medicine
| it's just a fresh grad.
| rednerrus wrote:
| It's kind of true if you just think about the beginning of the
| long tail as the end point.
| gabereiser wrote:
| The reality is you can become an engineer with no degree whilst
| you can't become a surgeon with no degree. The criticism of the
| writing is dismissing that a lot of people want to go for that
| promotion or higher tier but due to imposter syndrome or lack
| of confidence, abstain. This guide is to help them overcome
| those fears and level up. If you have an issue with people
| leveling up, good luck to you.
| eternalban wrote:
| That trivializes our profession and hard earned experience
| simply because a degree is not a requirement. It is
| ridiculous to say drag'n'droping something is halfway to
| being a systems designer. And the anecdote about "experience
| at facebook" was such nonsense. "I worked at facebook doing
| something other than distributed systems, and geez, I learned
| nothing about distributed systems". "QED!"
|
| p.s. I missed this ad hom bit (no, not the issue here)
|
| > If you have an issue with people leveling up, good luck to
| you.
| gabereiser wrote:
| Just because you work somewhere doesn't make you an expert
| at what they do. My Home Depot cashier is incapable of
| installing the carpet I bought from them. There are levels.
| Everyone wants to pretend they have FB or Google scale when
| they are building only to find out they have Ruby on Rails
| scale after year 4. They don't need a Stanford grad. They
| need a feature developer or someone who can break apart the
| monolith so they can scale beyond a single server or group
| of servers.
| eternalban wrote:
| We're in mild agreement, so somewhere there has been a
| misreading/miscommunication on either my part or yours :O
| richardwhiuk wrote:
| Our profession is trivial and full of smoke and mirrors,
| and barely deserving of profession. It's full of people who
| have literally no clue about what they are doing creating
| systems which screw over other people.
| alfalfasprout wrote:
| Just because you're all smoke and mirrors doesn't mean
| everyone else is.
|
| I agree with you that there's lots of impostors that have
| no business being called an "engineer". But that hardly
| means it's not deserving of a profession.
| shahsyed wrote:
| I've only read up to Part Two so far but find the guide is
| extremely helpful and resonates with other resources (i.e
| Grokking the System Design Interview) and with my own experiences
| for what I believe is the very broken interview process that we
| face today.
|
| Highly recommend. Interviewing.io is one of the best resources
| for those doing prep today and the team behind it is awesome.
| Samit5174 wrote:
| A lot of system design guides do enunciate on the fundamental
| concepts of system design such as scaling, design patterns etc.
| But explaining how to actually approach the problem before the
| interviewer and explaining what they're looking for and not
| looking for is what makes this guide unique. Otherwise, by the
| time the candidate gets to the meat of the problem, the
| interviewer might have already lost interest in them. So great
| job guys!
| tootie wrote:
| I think it's funny how they say you don't need experience to pass
| one of the interviews. Last time I went in to one of these as an
| experienced engineer I probably said all sorts of things they
| didn't want to hear like there's no point building a scalable
| system until you're sure your product has traction. That and I
| kept trying to extract imaginary requirements. I still have no
| idea what they wanted to hear so I just name-dropped architecture
| paradigms. Didn't get the job.
| siva7 wrote:
| You're wondering why you didn't get the job after letting the
| interviewer feel like their assessment tasks weren't perfectly
| designed?
| twelve40 wrote:
| Well, the sad truth is that a lot of _interviewers_ actually
| suck at interviewing, so sometimes an interview devolves into
| a bewildering game of what this particular person wants to
| hear today based on their mood, especially with the open-
| ended questions. It is not always possible to suffer through
| this process politely and constructively.
| alex_sf wrote:
| > Last time I went in to one of these as an experienced
| engineer I probably said all sorts of things they didn't want
| to hear like there's no point building a scalable system until
| you're sure your product has traction.
|
| So.. you argued with the interviewer about their question being
| 'invalid'. That's not going to be a winning strategy. If you
| have to insist on making comments like these, a better way to
| express it would be:
|
| _" Well, I know the context here is that we're designing a
| new, small-scale system. In cases like these, I think it's
| normally most helpful to get an MVP out-the-door, and not worry
| about scalability. However, in a situation where we _were_
| concerned about scalability, I'd start by..."_
|
| > That and I kept trying to extract imaginary requirements.
|
| This is more on the interviewer. If you ask for requirements,
| they should provide them. If they've done it a while, they
| might have a generic set, but they should at least be willing
| to make them up on the spot. In lieu of that, there's no rule
| that says you can't make them up on your own. E.g.:
|
| _You: "How many concurrent clients do we need to support?"
|
| Them: "I dunno, just make it scalable."
|
| You: "Ok, let's assume it's 10k and we want to make sure we can
| scale that horizontally in a roughly linear fashion up to
| 100k.."_
| tootie wrote:
| I don't know why you assume I was rude about it. I phrased it
| all equivalently to how you said. A lot of my background was
| in tech services so I'm very well practiced in asking these
| kind of questions. They pretty much waved off all of them
| saying it wasn't relevant.
| jefQuery wrote:
| I read all 4 parts in the pre-released version on the
| interviewing.io discord. Here's my feedback:
|
| Loved how much they go into the actual interview dynamics and
| phrases to say, where as other resources wave away interacting
| with the interviewer as an implementation detail. Haven't seen
| that anywhere else on the web and it's super helpful. The
| framework is also much more fleshed out than, for instance, the
| HiredInTech system design guide.
|
| However, the 12 tech ideas section was a little too dumbed down
| for me, though it might be helpful for someone with less
| experience. I also noticed a few typos (MangoDB, cache vs hash)
| and told them, and they said they'll fix it in the next version
| of the guide.
|
| 3.5/4 stars; would read again
| rickreynoldssf wrote:
| First off, I'm not disparaging the content, I'm sure its good and
| useful in interviewing at most companies.
|
| ...that said. This kind of thing annoys me. You shouldn't have to
| study for an interview. Either you meet the requirement and have
| the experience or you don't. Reading books and stuff like this is
| really a cheat by the person being interviewed. You many have
| studied enough to "pass the test" but you really don't have a
| deep usable knowledge of the material and will likely forget much
| of what was studied.
|
| When I do interviews I make sure the candidate is aware that
| studying will be a pointless endeavor and if I can tell your
| answers come from one of the interview prep books I'll end the
| interview early. My interviews ask questions that will tell me if
| you REALLY know the subject. For example if JavaScript is a
| requirement I'll throw this at you
|
| const a = [1,2,3,4,5,6,7,8,9];
|
| const b = a.filter(x=>1);
|
| if(!!1 && b.some(e=>e>6)){
|
| foo();
|
| }
|
| and ask you to tell me what it does. Its totally weird code and
| seeing that in actual code is highly unacceptable BUT if you
| really know JavaScript you will be able to tell me what this does
| in less than 30 seconds. Immediately if you're going for a Senior
| position.
|
| I'm known for hiring the best engineers.
| ZephyrBlu wrote:
| Wait, what?
|
| You're saying "you shouldn't have to study for an interview
| because you either have the experience or you don't" and then
| you're equating being able to answer a trivia question with
| experience.
|
| The irony is extremely rich. Being able to answer any
| particular question cannot measure if someone has the requisite
| experience. The only thing it tells you is that they can answer
| that question.
|
| Your judgement of them based on your question is extremely
| subjective and doesn't tell you whether they "REALLY know the
| subject" or not. You are just testing your own biases which may
| yield good results, but call a spade a spade.
|
| > _When I do interviews I make sure the candidate is aware that
| studying will be a pointless endeavor and if I can tell your
| answers come from one of the interview prep books I 'll end the
| interview early. My interviews ask questions that will tell me
| if you REALLY know the subject. For example if JavaScript is a
| requirement I'll throw this at you_
|
| How can you tell if someone had simply memorized a few pieces
| of JS trivia and is regurgitating it to answer your question?
| The code is not that complex or "totally weird".
| xapata wrote:
| > best engineers
|
| Precision is different from recall.
|
| You're asking trivia. I'd rather ask questions that can't be
| studied for, only practiced through years of experience.
|
| I also sometimes hire people who don't know JavaScript, yet.
| candiddevmike wrote:
| I think it's a combination of a lot of interviewers priding
| themselves when they "stump the chump" and a possibly
| subconscious optimization to find folks who won't upset the
| status quo.
| steve76 wrote:
| [dead]
| Ozzie_osman wrote:
| I love a few things about this guide. Firstly, that is
| acknowledges the fact that interviewing for a thing is different
| than actually doing the thing. Secondly, that it takes the
| position that system design interviews are (or should be) more
| about _how_ you approach a problem than about knowing the details
| of distributed systems.
| [deleted]
| ed25519FUUU wrote:
| > _Interviewers want to engage you in a back-and-forth
| conversation about problem constraints and parameters, so avoid
| making assumptions about the prompt._
|
| As someone who has conducted countless interviews at a FANG, I
| can not stress this point enough. The ability to just talk about
| a problem makes a big difference between a mediocre interview and
| a great interview. Besides, it's what makes an interview fun or
| lame.
| dakiol wrote:
| Problem is: as candidate you don't know if "engaging in a back-
| and-forth conversation about problem constraints" is something
| the interviewers want. Some interviewers want it, but it's not
| always like that. And, no, usually discarding an entire company
| just because the interviewer on duty doesn't like "engaging" is
| BS.
| rednerrus wrote:
| We've gone away from system design interview questions on my
| team. We ask people to diagram something technical they
| understand well and the team digs in and asks questions to
| understand depth and breadth of the candidates understanding. For
| us it works much better. It's a chance to see how well candidates
| do in following instructions. It gives you a chance to explore
| depth and breadth of their knowledge on something technical that
| they claim to understand. Our philosophy is how well you
| understand something you claim to know well is indicative of the
| depth and breadth of your technical knowledge in general. There
| are plenty of opportunities in this to ask about design and get
| to know how people think when it comes to design. I think it
| helps to eliminate false negatives and false positives.
| jahewson wrote:
| I don't like this approach because it doesn't provide
| measurable or repeatable criteria for comparing candidates. It
| also suffers from a sort of Gell-Mann amnesia where a candidate
| who spouts good-sounding BS about a topic you're not familiar
| with is assumed to be competent across the board.
| gautamdivgi wrote:
| You're ruling out all candidates who work on classified systems
| and those under an NDA
| rednerrus wrote:
| We assume that good engineers will know something about
| technical systems that are not directly related to the exact
| thing they're working on now.
| zbuf wrote:
| Yup, the key thing is let the candidate pick -- any
| technical system they understand.
| azornathogron wrote:
| I like this... in theory. But I don't think I'm allowed to give
| you a system diagram and detailed explanation of systems I've
| worked on, because obviously they're proprietary. That seems
| like a problem.
|
| How do your candidates normally work around this? Does everyone
| just talk about hobby or open-source systems they've worked on?
| ok_dad wrote:
| The only really important details to keep secret are
| business-specific algorithms, so talking about the overall
| architecture of a system is usually okay, without those
| details. If you're building very special architectures where
| that actually is the business detail, then I suppose you need
| to figure out how to talk about something else similar or
| something. I just talk about the systems I worked on in
| generalities, so I would talk about how we get some telemetry
| from some devices, process it, store it, and etc., but only
| the mechanical details and not the meaty algorithmic parts
| (we use a JSON API to submit the telemetry, then we store it
| in a table, then we take that data in our other component and
| process it through our "algorithm", etc.).
| [deleted]
| sroussey wrote:
| Your ideas of what should be secret or not may not at all
| be what the candidate agreed to in their employment
| contract and NDA.
| cyc116 wrote:
| Exactly. Not to mention high level business logic could
| be inferred from system architecture.
|
| Sure sounds like a good way to perform recon on a target
| company.
| rednerrus wrote:
| We say specifically "diagram a technical system you
| understand well." It's going to be challenging to work in
| technology if the only technical system you understand
| well is the one you're currently working on and only the
| parts that are under NDA.
| cauthon wrote:
| I have the opposite question from other people in the
| thread - this seems pretty open-ended.
|
| Just as an example, I do bioinformatics work, and
| "technical system" for someone I'm interviewing (as I
| understand your question) might encompass any of:
|
| - a workflow manager: how it works under the hood, common
| patterns for modularity/reuse or configuration
|
| - a specific pipeline: the stages of cleaning and
| quantifying raw data, the rationale for various stages
| and the tools chosen to execute them (including how to
| benchmark various methods against each other)
|
| - a specific third-party tool: theoretical details of the
| algorithm, practical considerations for when to apply one
| over another
|
| - familiarity with common general-purpose packages or
| APIs (e.g. pandas, numpy, scikit-learn)
|
| - QC: common metrics for QC'ing various experimental
| data, how to decide if data is "good enough", how to
| troubleshoot biological vs technical (lab) vs technical
| (computational) sources of error
|
| - biology: technical details of an experiment, the
| technologies generating the data, or the underlying
| biology
|
| - data analysis: how to choose and make relevant figures,
| any of many data-science/ML topics (e.g. clustering),
| connecting data to relevant domain questions
|
| - devops: data storage/management on HPC or cloud, HPC
| job schedulers, any of many cloud topics (e.g. IAC,
| setting up a database or cloud execution of a pipeline)
|
| I'm more curious about how you guide a candidate towards
| selecting a system that gives you the most relevant
| perspective on their thinking and skillset - do you
| provide any suggestions, or are there typically pretty
| evident choices based on the role?
| rednerrus wrote:
| For our more junior roles, we leave it more open ended
| because candidates may not have a similar knowledge bases
| to the panel. For more senior roles we advise that if
| they pick technologies that are pertinent to the role and
| likely to be understood by senior/lead/staff engineers,
| the process will go better for candidates and panel
| members.
|
| If I were in your position, one of understanding many
| complex systems, I would advise you to pick where you
| felt both strong and the panel was likely to have some
| entry point into.
| rednerrus wrote:
| We call this out in our pre-interview handout. If you've got
| an NDA, choose something that isn't included in your NDA or
| figure out a way to describe it at a high level. We caution
| candidates that if we dig in and we run into something
| blocked by the NDA it creates challenges with our evaluation.
|
| It doesn't necessarily have to be something that you're
| working on. It just has to be something that you understand
| well enough to describe the design of and how things are
| inter-connected.
| operatingthetan wrote:
| >We caution candidates that if we dig in and we run into
| something blocked by the NDA it creates challenges with our
| evaluation.
|
| So you're penalizing those under NDA and unecessarily
| limiting your candidate pool.
| rednerrus wrote:
| There's no penalty. There are plenty of technologies that
| you're working with that aren't under NDA. We're not
| asking you to diagram what you're working on, we asking
| you to diagram something technical that you understand
| well.
| _visgean wrote:
| When I interviewed with apple they said pretty much
| everything they do is proprietary and considered secret.
| I imagine for some of these people working there for
| years it would be hard to avoid breaking nda
| rednerrus wrote:
| There's no place where we are asking you to diagram the
| system that you're working on. We ask you to diagram a
| system you understand well. Are you using messages
| queues, container orchestration, network components,
| cloud infrastructure, Java/rust/go/Python/etc.? Tell me
| about one of those components and how they connect
| together in theory vs the specifics of your system.
| Everyone who works in technology works with, mostly,
| complicated interconnected systems. The point of this
| exercise isn't to design a system or tell me about a
| system that you've designed as much as it is about
| helping the team to understand the breadth and depth of
| your technology knowledge. Good engineers typically find
| creative ways help us understand their knowledge without
| disclosing NDA material.
| fiddlerwoaroof wrote:
| But, if you're a web developer you should be able to
| diagram what happens when a browser makes a request
| operatingthetan wrote:
| Exactly, I work in consulting and under a blanket NDA for
| all work I do. In addition clients often put us under
| theirs as well. The other poster doesn't seem to have
| much experience with NDAs and seems to think the 'secret
| parts' are made explicit, they are not.
| Salgat wrote:
| Being able to use something you've intimately worked on
| as part of your job is a huge advantage since you're so
| familiar with it.
| rednerrus wrote:
| There have to be parts of what you work on that are not
| covered by NDA. Solving problems like this would be
| indicative of someone we would want on our team.
| zbuf wrote:
| It's a reasonable hurdle; an employee who previously
| signed an NDA (or any one, for that matter) needs to be
| able to design new systems without regurgitating verbatim
| parts of an old one under NDA.
|
| So it's really not a big ask to navigate their own NDA
| and come up with something interesting to talk about at
| an interview.
| eschneider wrote:
| 98% of what you'd be inclined to diagram out and explain for
| a system design interview isn't remotely proprietary. People
| are generally looking for a high-level overview of a system
| you've worked on to see a) that you've really worked on it
| and b) you can explain it such that it makes sense.
|
| Sure, folks will ask you to drill down here and there for
| more detailed info, but if someone pokes on something where
| the details really _ARE_ proprietary, it's fine to just say
| that and offer to drill down somewhere else.
| toolz wrote:
| I'm not a lawyer, but wouldn't anything you or anyone else
| developed on company time at another company be owned by
| that company and thus proprietary?
|
| In practice I can't imagine the spirit of any law would be
| violated by doing an architecture diagram, but it seems
| likely (to me, at least) the word of the law would be
| violated.
| Swizec wrote:
| > I'm not a lawyer, but wouldn't anything you or anyone
| else developed on company time at another company be
| owned by that company and thus proprietary?
|
| In theory yes, in practice ... if that's true, why would
| anyone ever hire you for your experience? You wouldn't be
| allowed to use it.
| ozim wrote:
| So no one else can never build todo app or blogging
| system?
|
| There is term called prior art an for me if you write
| bunch of props to database like text/numbers it is
| basically done in every other system.
|
| Unless you build novel db system of course.
| jfarina wrote:
| How could you ever interview anywhere if you couldn't
| provide details of the projects that you worked on?
| [deleted]
| eschneider wrote:
| Sure. But the level of description that you're giving in
| an interview can be pretty generic/non-proprietary. If
| that wasn't the case, then you really can't reuse any
| knowledge from job to job and that's just not so.
|
| Sure, every company is going to have their 'special
| sauce' components, that ARE proprietary, but nobody's
| going to expect you to unpack those.
| rco8786 wrote:
| > 98% of what you'd be inclined to diagram out and explain
| for a system design interview isn't remotely proprietary
|
| Maybe we're just on different wavelengths, but literally
| every single thing I work on that would be a good candidate
| for this sort of interview is extremely proprietary...and
| I'm just working on regular ol' backend services at
| [generic big tech].
| hadlock wrote:
| System diagram of a CRUD SAAS isn't going to be a critical
| business secret in the same way it might be for a company
| that does stream processing of marketing/advertising data off
| of twitter or video/radar analytics. Or a real-time fintech
| trading company. Many companies are a Python/Django monolith
| on Postgres with redis & || memcached. That system diagram
| isn't going to let you disrupt the online medical services
| industry but it's what a lot of them use.
| bluetomcat wrote:
| Few people at software companies are designing systems from
| scratch nowadays. If they are, they'll be gluing together
| third-party middleware applications like database systems,
| queues, proxies, load balancers, etc.
|
| A general understanding of computer architecture, programming
| language theory, networking and common protocols and formats is
| more valuable. At that fundamental level, they won't be using
| recent fancy buzzwords to sound cool, but could be using their
| brain to come up with something original.
| rednerrus wrote:
| This is exactly why we do this. It gives candidates a chance
| to choose what they want to diagram for us. What do you know
| well?
| fdgsdfogijq wrote:
| If you havent designed a system from scratch, or designed a
| major change, you arent a senior engineer.
| jhp123 wrote:
| these "system design interviews" often focus on systems at
| the scale of major consumer internet companies, e.g.
| "design youtube", "design twitter".
|
| How many systems of that scale even exist? 100? Only the
| core developers of those systems would be considered
| "senior engineer"?
| specialp wrote:
| Someone isn't going to design one of these system or has
| designed some of these systems themselves or during an
| interview. However for something like YouTube one could
| say Ok to start simply I would have a frontend form to
| submit videos, which sends it to a video conversion API
| queue that calls back when it is completed or could be
| polled for progress. Now obviously what YouTube really
| does is much more complicated. But the follow up
| questions could be like OK your form works and now your
| video site becomes enormously successful and your video
| transcoder is overloaded, what would you do? Well I could
| parallelize the consumers of the queue to some large
| number, and scale based on load...
|
| So isn't going to be something like I would architect my
| own massively parallel converter database and binaries
| written from scratch to process all the various formats
| which is probably closer to what is done in reality. But
| senior and lead engineers should indeed understand
| tradeoffs, parallelization, queues, data storage
| concerns. They are given as questions because people know
| from a use case view how they generally work.
| squeaky-clean wrote:
| When I interviewed at Facebook my system design question
| was for a Yelp-ish clone that would be rolled out to
| Facebook as a whole. The interviewer told me the design
| had to be scalable to 1 billion daily users on day 1.
| p1necone wrote:
| > database systems
|
| When was the last time building a database from scratch was a
| sane approach to building something?
| onlyrealcuzzo wrote:
| > It gives you a chance to explore depth and breadth of their
| knowledge on something technical that they claim to understand.
|
| Assuming you know enough about what they're doing to actually
| dig in - which, granted, maybe you're only considering
| interviewing people who are working things you're somewhat
| familiar with.
|
| Ultimately, I don't see how this is different from a technical
| design interview - in that you can be susceptible to hiring
| charlatans, and pass on people who'd rather say they don't know
| something than try to BS most of it on the spot to sound
| impressive.
|
| The two people could have the same knowledge. You're more
| likely to hire the charlatan. Maybe that's what you want
| :shrug:
| Scubabear68 wrote:
| In my own experience interviewing people about systems they
| have worked on (as opposed to hypothetical on-the-spot
| design), we have been highly successful in routing out
| charlatans, as you put it. They very rapidly get hand-wavy
| and are only able to give the most shallow of answers. In a
| few cases they will go in-depth, but reveal truly bizarre
| decisions or lack of understanding of the platform.
|
| Good senior engineers can usually go fairly deep, they are
| honest about where they don't have as much knowledge, and are
| usually candid about what was good and maybe what in
| retrospect was a hack or a bad decision in retrospect.
| rednerrus wrote:
| We've had very few false positives. It's a conversation and
| we understand and appreciate when people say they don't know.
| We understand that really good engineers say "I don't know
| how that works." If all of your answers are "I don't know how
| that works.", it'll be red flags enough for the team. The
| team of interviewers has a very wide and deep knowledge of
| technologies (it's kind of a virtuous cycle). We call it out
| in the pre-interview handout that if you pick something super
| obscure it'll hurt your chances as the team will struggle to
| ask good probing questions.
|
| The nice thing about this is you're giving the candidate a
| choice about what they want to describe. Good engineers will
| choose appropriate topics.
| posharma wrote:
| I don't know if you work in FAANG or no, but most FAANG
| interviews just don't work this way. So, as much as I like this
| style, it's just 1 company and definitely not the norm. The
| rest of us are stuck with design Uber, Whatsapp, crap...
| nothrowaways wrote:
| It still looks design interview
| mdm12 wrote:
| I have seen this process described elsewhere as 'reverse system
| design', and it is my preferred approach to evaluating senior
| candidates as well.
| jstx1 wrote:
| Still sounds like a system design interview to me, just a bit
| less structured.
| rednerrus wrote:
| We've found that candidates who better understand how things
| are connected make better candidates and operators. We need
| some way to gauge how well they understand how things are
| connected. This is as fair of way as we've come up with.
| zekrioca wrote:
| I totally relate to that. In the university setting, I see
| many students more interested in learning keywords instead
| of "key concepts" behind such keywords. Think of preferring
| to learn "k8s" instead of "resource manager". Today, very
| few of them knows that etcd is one of the key components in
| "k8s".
|
| I guess this behavior is similar to the transformation that
| happened in other non-CS fields.
| [deleted]
| xen2xen1 wrote:
| When doing IT people often say "Well, I don't know anything
| about computers", which I generally found pointless, as with
| about 30 seconds of talking I can tell if you know anything or
| not. Your story passes my smell test, as getting someone
| talking is 90% of telling what they know.
| rednerrus wrote:
| People with broad and deep understanding of technology can
| typically gauge if someone knows there stuff or not with 30
| minutes of them describing a system to you and answering
| questions about it.
| spike021 wrote:
| One place I interviewed with last year did something similar in
| that it was more of a BYOSD (bring your own system design). So
| it meant I got a chance to think through complex systems I've
| worked on, mock up a diagram beforehand, and then present it to
| the interviewer. They then drilled down on a lot of components,
| why decisions may have been made, etc. Sort of like what you're
| saying.
|
| Out of all the similar interviews I did last year for that type
| of round, I enjoyed that style the most. Rather than your
| typical "build me an {ecommerce site, social network, video
| streaming site, url shortener}".
| zinclozenge wrote:
| Square/Block (used to?) ask only 1 system design question and
| the recruiter would let you know ahead of time so you could
| prepare for it. Probably because it allows for deeper
| questioning and answering of relevant technical skills.
| blowski wrote:
| It's hard to tell between those who know their stuff, and
| those who are good at memorising all the various courses
| that tell you what to say. By telling the candidate what
| system to design you make the latter's job easier.
| pcthrowaway wrote:
| If someone can quickly learn enough about a system that
| they can convince a domain expert they understand it well
| in a deep technical discussion, they're probably a good
| candidate for hiring, since your business likely consists
| of many complex systems, and the candidate will
| theoretically be able to onboard faster.
| blowski wrote:
| I've seen first hand people hired who've aced the system
| design interview but then fail to get much done. Building
| systems in the real world is much more than a few high-
| level components and algorithms - it's about all the edge
| edge cases, tricky stakeholders, ambiguous and changing
| requirements, sequencing and coordinating small pieces of
| work.
| namaria wrote:
| Attaining high level understanding and speaking
| convincingly about it are entirely different skills from
| knowing how to build it. Interviews are always a proxy
| anyway, your job will never be solely comprise of
| convincing people of your expertise. Sooner of later you
| have to do exercise it somehow. Unless you're the CEO,
| that is. The higher you go the easier it is to make a
| career out of being in the right places at the right
| times.
| [deleted]
| emmanueloga_ wrote:
| where do you work?
| jason-phillips wrote:
| > We ask people to [discuss] something technical they
| understand well and the team digs in and asks questions to
| understand depth and breadth of the candidates understanding.
|
| This is how I conduct all of my interviews. I can't stand the
| gotcha-centric, pitfall-laden Jeopardy contest format of
| interview interrogations.
| rednerrus wrote:
| We think this is the fairest way to evaluate candidates. I
| had one standout interview that was very gotcha centric. I
| didn't think the way the interview was conducted was fair or
| respectful of my time. I also don't think it did a good job
| of assessing how well or how poorly I would have done in the
| job. It seems indicative of poor management. I think it's
| very important as managers that we respect people and their
| time, always. This feels like as fair of a way as we've come
| up with so far to do that. We do two technical exercises,
| diagramming being one of them. We've had really great results
| and it takes ~1 hour. That seems fair.
| bhvangoo wrote:
| hmm interesting approach, really like it. Where do you work?
| and are you hiring now?
| vsareto wrote:
| Feels like an Architect interview more than a Senior interview
| though
| rednerrus wrote:
| The nice thing about it is that it scales to whatever level
| you need it to. We've had junior candidates with no real Ops
| experience diagram synthesizers that they've created. Then we
| ask questions to gauge their level of understanding. We have
| a general idea of what it means to be at each of the clearly
| delineated levels on the team and we ask questions to what
| seems like the appropriate level. We've had candidates come
| in to interview for senior positions and during the interview
| the team drilled down to what we felt like were lead
| candidate levels and we offered the candidate a lead role
| instead of a senior.
|
| We call out in the pre-interview handout exactly what we are
| going to do and what we expect and we let the candidates
| bring what they think is best. For senior, lead, and staff
| candidates we call out that bringing something that the team
| is familiar with as well, will be beneficial to the candidate
| and to the team.
|
| It's really allowed us to do our best evaluation of
| candidates in a reasonable amount of time. We have two
| technical exercises and they take ~1 hour total. We may still
| be getting false negatives, but we're not getting false
| positives. Hire hard, manage easy, right?
| pc86 wrote:
| As a former (reformed?) non-coding architect, non-coding
| architects are a scourge on the industry. I know you didn't
| say "non-coding" but if it's a separate job, they're very
| likely not spending enough time coding.
|
| Seniors should be able to build large, interconnected
| systems. IMO that's a basic skillset required to reach that
| level, and part of why I roll my eyes when I see people with
| 5, 4, _3_ years of experience claiming to be seniors.
| vsareto wrote:
| People can claim to be seniors because the gates for
| getting that title are pretty permissive. But even if you
| had the conviction to re-evaluate yourself ("do I deserve
| to be a senior engineer?"), you'll find tons of opinions on
| what it means to be senior, what knowledge they should
| know, and what they should bring to the table before
| claiming that. It's difficult to build yourself a widely
| accepted roadmap.
|
| You have to watch out for people being too exclusionary
| though, almost getting into no-true-scotsman territory.
|
| For instance: you can be a valuable contributor to a large,
| interconnected system, explain each part of it, but maybe
| not have the skills to build it from scratch. I would call
| that senior, but I wouldn't say architecting it from
| scratch is a basic skillset of this position. My opinion is
| if you could build large systems from scratch for a
| company, I'd say you probably need a title and pay higher
| than senior.
|
| If you did want to roll architecture into a senior job, I'd
| imagine the pay is higher than non-architecting seniors,
| but it's hard to quantify the architecture contribution to
| the salary's bottom-line. If you took the non-coding
| architect's salary and added it to the senior's most of us
| would be clearing like 300k.
|
| Personally, I'm pessimistic around job duties because
| you'll find so many companies looking for do-everything
| one-man armies at lower-end wages.
| eternalban wrote:
| I somewhat take issue with this. (3+ decades of very hands
| on (read: coding) architecting, including some learning
| experiences in orbit. I've been coding code-doodling since
| teenage years.)
|
| A lot of what non-coding architects traditionally brought
| to the table has been taken over by experts designing OSS
| protocols, data formats, etc. Take things like data frames
| that are now exploding in ML space. Seniors today, agreed,
| should be able to integrate (sub)systems and that may in
| fact be enough. But a competent systems architect (who have
| never touched code) should be able to also define data
| formats, patterns of movement of data between sub-systems
| (for say optimal performance), the actual computing
| platform considerations, etc. Also, sometimes when being
| too close to code, things degenerate to debates about
| tools, etc.
|
| Naturally my points here gain more validity as system size
| (or its open-ness requirements) increase.
| monsieurbanana wrote:
| I always thought that non-coding architects still had at
| least some background in coding. What does the path of a
| never-had-coded architect looks like?
| siva7 wrote:
| Architecture and coding are two distinct skillsets. I know
| that most senior developers haven't touched a book about
| architecture for a long time if ever but still feel
| competent about such topics - falling in the "You don't
| know what you don't know" trap. This is as problematic as
| the architect with rusty coding skills or who hasn't worked
| as a professional developer beforehand. Coding should be a
| small part of the architect role but this role is usually
| reserved for very complex systems where the majority of
| your work isn't anymore about coding because there is
| enough to fill a full time job otherwise there is no need
| for an architect at your company.
| Arainach wrote:
| Why are books important? Senior engineers should be
| seeing design documents and code reviews that involve
| architecture and design patterns all the time.
|
| Sure, there's room for external learning, especially at
| companies where software isn't the primary focus, but
| almost none of the best software architects I know spend
| much of their time reading about architecture.
|
| Speaking personally, I've learned more about architecture
| from reading code and system design interview prep
| materials than I ever did from any book. The closest I
| found to a useful book was the "architecure of open
| source applications" series which is essentially reading
| code with a senior eng to walk you through it.
| siva7 wrote:
| I hear the "Why are books important" pretty often when
| guiding developers. All of the best architects i know of
| spend much time honing their skills. That won't happen if
| all you see is just the company you're working at. The
| worst architects i've worked with stopped learning
| outside of their job tasks. It's also a step to
| understand that design patterns and system design are all
| within the core competency of the senior developer and
| that the role of the architect has many other
| responsibilities.
| wholinator2 wrote:
| While I'm sure that the best employees make their entire
| lives about work skills, I just can't imagine that being
| a fulfilling life personally. If i have to spend my
| nights reading books about work, outside of work, then I
| don't want to be the best. I have other hobbies I'm more
| interested in than, "becoming a better employee".
|
| Now, I do feel different about academia though. I could
| understand doing fundamental research and dedicating most
| of your living hours to those problems. But i guess I
| just feel like the research mission is infinitely more
| valuable than the corporate one. I say this having never
| had the chance to really contribute anything meaningful
| to any company I've worked for, and never having worked
| for a company who's mission felt personally fulfilling to
| me.
|
| I'm glad that there are people who work their butts off
| for work in important places but I just can't imagine
| doing that myself and not just disintegrating with regret
| when I turned 65.
| dakiol wrote:
| > I just can't imagine that being a fulfilling life
| personally. If i have to spend my nights reading books
| about work, outside of work, then I don't want to be the
| best. I have other hobbies I'm more interested in than,
| "becoming a better employee".
|
| I'm the guy who reads books about my career (not about my
| work) outside of work (in my free-time) and during
| working hours as well. I love my career (computer
| science) and I love reading well-known tech/cs books. I
| don't do it to be the "best employee". I couldn't care
| less about what I do at work (I, like 99% of the people
| here, work for a totally useless tech company that nobody
| would care about if it disappeared tomorrow). What I do
| at work is stupid distributed systems in Go + Kafka +
| postgres + k8s (totally uninteresting stuff, but pays
| very well though). I genuinely enjoy readings books from
| Stevens, Kerrisk, Kleppmann, etc., just like I enjoy
| reading sci-fi/drama/etc. novels. I usually end up
| learning stuff that's actually useful at work, and once
| in a while I apply such knowledge at work, but I do it
| only for the raises and promotions.
|
| There are people out there who doesn't give a fuck about
| tech companies, but care deeply about tech.
| zabzonk wrote:
| best way of doing interviews imho. apart from anything else, it
| gives you a chance to show that you know how to
| design/implement a system that is perhaps more complex than the
| one you are being interviewed for
| slytherone wrote:
| Hey, Creator of the guide here
|
| Reverse system design interviews have a a lot of untapped
| potential. They just started getting a bit more popular in the
| last year or so (however, most companies haven't caught on to
| the trend yet.)
|
| You're one of the trendsetters @rednerrus
| moosedev wrote:
| I had a "reverse" system design interview at Amazon back in
| the 2000s. Possibly before the modern "forward" system design
| interview became so popular.
|
| I also remember circa 2012-2014 being required to conduct
| what are now considered typical modern system design
| interviews (i.e. as an interviewer) while employed at Amazon,
| and 90% of (even senior) candidates had no idea how to even
| begin to approach the "design a system to do X"-type
| questions. I think this is partly because far fewer people in
| typical software jobs were building distributed systems back
| then anyway, and partly because all the YouTube videos and
| guides like yours didn't exist yet, so nobody was doing the
| kind of dedicated prep and rote-learning that seems
| increasingly expected nowadays.
|
| Back then, when the problem was too far from the candidate's
| real work experience, and they weren't willing to make
| educated guesses and ask clarifying questions in order to
| move forward, it was often a struggle (for both of us) to
| pivot the interview towards something the candidate could
| tackle and demonstrate their design experience. ("Tell me
| about a system you designed", i.e. the "reverse" approach,
| wasn't an acceptable alternative in our rubric.)
|
| Now, just like what happened with coding interviews vs.
| Leetcode, it seems that a more common challenge for the
| interviewer is telling the difference between a candidate who
| is applying real experience and understanding vs.
| regurgitating/performing what they read in a system design
| interview prep guide. But that's assuming one is actually
| more valuable than the other, and I don't have proof of that.
| brailsafe wrote:
| Super good content. Now for a guide on getting a response back at
| all for any job application in any industry
| groffee wrote:
| There's very little content here, especially nothing that would
| be considered 'senior'.
| joshSzep wrote:
| I love the content that interviewing.io puts in this category on
| their YouTube channel. What was especially eye opening is seeing
| two senior engineers that normally conduct systems design
| interviews taking the driver's seat. https://youtu.be/Zi0pPkiFemE
|
| TLDW; they struggle! big time!
| mgraczyk wrote:
| I am on the interviewer side of ~1 system design interview per
| week and have done probably around 100 of them at Google and
| other companies.
|
| Most of the advice here is good, although it could be condensed
| quite a bit.
|
| The biggest red flag for me as an interviewer is when candidates
| list off concepts or technologies they don't understand. DO NOT
| say things like "we can't do this because of the CAP theorem" or
| "we should add a cache to reduce latency" unless what you are
| saying is true and you can explain why this is true (the guide
| mentions this).
|
| The biggest green flag is when the candidate obviously has
| thought deeply about related problems and made design decisions
| based on real world constraints and requirements. If you bring up
| a lot of related things you have worked on, explain why you made
| the decisions you did in those scenarios, if its obviously
| related to the question, and if you don't spend too much doing
| it, then you will definitely get a positive recommendation from
| me.
|
| The purpose of the system design interview is to convince me that
| you could build, or lead a team to build, the thing that we are
| discussing at a company like the one where you are interviewing.
| I feel like I can easily see through interview practice or
| coaching, so I don't recommend spending more than an hour or so
| "preparing" for system design interviews. Unless you are
| interviewing somewhere with very inexperienced interviewers...
| yodsanklai wrote:
| > I feel like I can easily see through interview practice or
| coaching, so I don't recommend spending more than an hour or so
| "preparing" for system design interviews.
|
| I'm very surprised by this recommendation which doesn't match
| my experience at all. First, lots of candidates don't have
| experience building such systems, so they need to learn by
| reading books and articles. Second, the candidates need to be
| familiar with the structure of the interview and what's
| expected from them, and that takes practice too.
| leonidasv wrote:
| >I feel like I can easily see through interview practice or
| coaching
|
| How do you spot it? I've had candidates with stellar
| performance on system design interviews, and I've never had any
| suspect of them (and the resume also seemed pretty strong in
| those cases). But now I'm curious if I missed something.
| moandcompany wrote:
| One of the "heavy practice/coaching/leetcode grinding"
| signals is when the interviewee immediately dives into
| proposing a solution and discusses facets of that solution in
| great detail without bothering to ask clarifying questions or
| discussing assumptions and the risks/consequences of those
| assumptions.
| ditonal wrote:
| I think the opposite is true. An experienced engineer not
| as versed in interview prep might hear the question to
| design something, and go ahead and show how they would
| design it based on their experience doing it in the real
| world.
|
| The leetcode grinder would almost certainly ask clarifying
| questions since it's heavily emphasized as part of the
| rubric in pretty much all system design interview prep
| material.
|
| Of course, the non leetcode grinder might ask those
| questions, but even the social cues that you're allowed to
| ask clarifying questions might not be understood.
|
| In my experience, I've seen a lot of people say that
| systems design test the seniority of a candidate, but then
| stick to a rubric that the leetcode grinder has memorized
| and the senior engineer might miss several important points
| on despite having the experience because they didn't
| understand all the unwritten rules of the song and dance.
| mgraczyk wrote:
| It's possible in those cases that the candidates were really
| good.
|
| The main tells are a mismatch between apparent practical
| experience and apparent knowledge of the design space. I
| always dig into a few specific technical aspects of the
| design as far as possible to see how deep the candidate can
| go.
|
| For example, if there's a queue I'll ask what would happen if
| in production the queue starts to fill, and how to mitigate.
| Most candidates reflexively suggest making the queue bigger,
| some suggest adding more downstream capacity to drain faster.
| People with real experience will have encountered this
| problem and know that you have to spend time investigating to
| find the root cause of the backpressure, otherwise you could
| add capacity to random things without reducing the queue
| occupancy. Are your queue consumers getting throttled by some
| downstream service? Are your workers CPU bound? The "right"
| answer here is basically "we should figure out why the queue
| is filling up before changing anything", but very few
| candidates answer that way.
|
| Another kind of mismatch is the red flag I listed, where
| candidates will basically list buzzwords or ideas regardless
| of whether or not they are related to the question I asked.
| For example if I ask a question where the user-facing API is
| very simple (ie, write opaque data to a log) then they start
| talking about the tradeoffs between graphql vs REST, that's a
| bad sign.
| yunwal wrote:
| > People with real experience will have encountered this
| problem and know that you have to spend time investigating
| to find the root cause of the backpressure, otherwise you
| could add capacity to random things without reducing the
| queue occupancy. Are your queue consumers getting throttled
| by some downstream service? Are your workers CPU bound? The
| "right" answer here is basically "we should figure out why
| the queue is filling up before changing anything", but very
| few candidates answer that way.
|
| This is the opposite of what "people with real experience"
| do, at least in production-down situations. The first step
| is to mitigate, the second step is to root cause. If
| there's some no-brainer step that has a chance of
| alleviating the issue while you root-cause, and is unlikely
| to make things worse, you should take it.
| apsurd wrote:
| Your point is good, but in making it, you sound like an
| adversarial interviewer.
|
| You and OP are actually agreeing with "the correct answer
| is it depends and now let's discuss context". This echoes
| my experience as interviewer too. It's a red flag when
| the candidate responds with "the correct answer". That's
| what OP is calling out.
|
| I'm replying here because I get the impression you're
| looking for "the right answer" as you see it: "the first
| step is to mitigate then do root cause". You're right!
| But it also could be too adversarial.
| jefQuery wrote:
| It feels cool to me that my nontraditional degree for
| software (ChemE) drilled this into us and now it comes up
| in my current job: Rate of input - rate of output = rate of
| accumulation
|
| It'd be fine to just up the queue capacity if the input is
| just inconsistent/spikey but averages to the input, but
| under steady state (or steadily increasing growth) higher
| input, you _must_ increase the output rate or you will
| outgrow whatever capacity the system has
| coolspot wrote:
| > DO NOT say things like "we should add a cache to reduce
| latency"
|
| We should add a cache to reduce latency tho...
| gilbetron wrote:
| If you need to reduce average latency and the cache would
| actually do that, sure. Cache doesn't magically make all your
| network latencies drop, though.
| jeron wrote:
| yeah, not sure what is too hard to understand about that.
| Perhaps it depends on the context but I highly doubt you can
| go wrong by saying a cache should be added to reduce latency.
| N_A_T_E wrote:
| Only two hard things in computer science, blah blah.
| lhorie wrote:
| The way it typically fall aparts for my candidates is I ask
| about optimizations, they say X could be cached, then I ask
| about cache invalidation and lo and behold, it didn't even
| occur to them that the cache needs to be invalidated at
| some point, so they blurt something along the lines of
| doing the entire expensive operation again and checking it
| against the cache, or something similarly nonsensical.
|
| More common than I'd like to admit.
| ThalesX wrote:
| > then I ask about cache invalidation
|
| No doubt with a follow up on how the variable would be
| best named.
| old_hat wrote:
| That's obvious. It's called "x".
___________________________________________________________________
(page generated 2023-03-02 23:00 UTC)