[HN Gopher] How to maintain engineering velocity as you scale
___________________________________________________________________
How to maintain engineering velocity as you scale
Author : sandslash
Score : 109 points
Date : 2022-10-25 16:00 UTC (7 hours ago)
(HTM) web link (www.ycombinator.com)
(TXT) w3m dump (www.ycombinator.com)
| jahewson wrote:
| > From the beginning, we built a simple but solid foundation that
| allowed us to maintain both velocity and quality. When we found
| product-market fit later that year and started bringing on lots
| of new customers, instead of spending engineering resources on
| re-architecturing our platform to scale, we were able to double
| down on product engineering to accelerate the growth.
|
| This is the gold standard. It takes exceptional talent to put
| together an organization like this while searching for product-
| market fit. Bravo!
| falcolas wrote:
| I'm going to be contrary and say "that's not possible". At least,
| it's impossible to maintain feature velocity, even if you
| maintain development velocity.
|
| The reason is simple, in the beginning you're starting with no
| legacy code. Features are simple to add, because there is no
| history to work around. However, even a month in, features are
| going to start blending together, and new features will have to
| work around existing features.
|
| Your databases will get new tables, and the existing tables will
| get more columns. The response code is going to fork as the API
| versions increment. The API space is going to grow.
|
| Technology which handled 1-1000 requests per minute will fall
| over when they start getting hit with 10,000 requests per minute,
| and require optimizations, indexes, caches, and other
| complexities in the stack _which aren 't new features_.
|
| Startups are fun to work for early on, when complications like
| these don't exist. It requires more than "the best engineers" to
| keep up development velocity, let alone feature velocity.
| shadowtree wrote:
| When we started in 2009, we did 3 major releases a year
| (enterprise software, deep vertical SaaS).
|
| We now do monthly releases into production, with 80+%
| marketshare and 500mil rev just in my product line.
|
| Scaling is hard, but doable. Takes a lot of willpower and
| fighting against gravity.
| narag wrote:
| They started with "best practices" that maybe slowed them down
| at the start, but surely made the scaling smoother.
|
| Of course scale will have some effect, but the recommendations
| seem very sensible like keeping teams small and independent.
| JJMcJ wrote:
| To use Frederick Brooks's term, the optimizations, the special
| cases, the interactions with other parts of the system, start
| to become essential complexities as the system grows.
|
| The skill of your developers is all that stands between the
| code base and the total chaos of insurmountable technical debt.
| chrismarlow9 wrote:
| If companies took the concept of an 'error budget' (every SRE
| principle seems to be important to companies except this one)
| seriously and saw it as a signal instead of an annoyance there
| would be some ebb and flow in this realm instead of a continued
| compounding increase where stability sits on the backs of a few
| people with the tribal knowledge. Just my 2 cents.
| twblalock wrote:
| I agree that you can't maintain the same velocity you started
| with as the company grows, but there are definitely ways to
| keep as much of it as you can. Some companies slow down a lot
| more than others.
|
| In my experience, it ultimately comes down to SDLC. Teams need
| to be able to work (mostly) independently of each other. This
| means being strict about using API boundaries rather than
| sharing database tables between multiple apps, and avoiding
| breaking API changes that force you to deploy software written
| by multiple teams in lockstep. That is the real reason
| microservices are valuable: they decouple _teams_.
| jahewson wrote:
| Most of the time, yeah that's how it plays out; it's easy to
| start from a blank slate. But where I differ is that I'd claim
| that most early startups don't even have much velocity. They
| don't actually move with speed, but with haste. All possible
| corners are cut. As the team scales, the debt comes due and
| forward progress stalls. What strikes me about this article is
| the forethought that was put into the engineering from the
| start - this is a company that really does have velocity and
| wants to maintain as much of it as possible.
| int0x2e wrote:
| One thing I think can really help is to have two different
| modes within the same team. You move fast and cut all corners
| and rapidly iterate when you have an idea by building a
| prototype, with the only goal being maximizing the speed at
| which you're learning if/what works and what does. Then you
| basically throw away all that prototype code and build the
| feature properly with careful attention to code quality and
| complexity, testing (++) - since now you actually know what
| you're building and that it's likely a good idea.
|
| Rinse and repeat, so that you only have prototype code where
| you're actively learning, to be immediately replaced by
| properly engineered solutions or wiped away (with the same
| haste you used to create it initially - no "rarely used,
| badly engineered" features should stay in your codebase due
| to priority / resources / attention constraints).
|
| The biggest challenge I've had with this approach has been
| building and fostering the trust and culture needed to have
| everyone truly go "all in" for both aspects/modes of working.
| This breaks down quickly if you have managers or PMs drive
| engineering to keep the prototype code in for longer, or if
| engineering insists on not cutting corners during the
| exploration/prototyping phase...
| winphone1974 wrote:
| This sounds great on paper but I have a hard enough time
| getting buy-in to finish the prototype. I've never seen
| anyone throw away the prototype and rebuild from the ground
| up before waiting years and then calling it "V2"
| jasondigitized wrote:
| From https://www.stickyminds.com/aricle/hudsons-bay-start
|
| England's King Charles II chartered the Hudson's Bay
| Company in 1670. It is considered by many to be the world's
| oldest continuing commercial enterprise. The company
| outfitted fur traders for expeditions out of Hudson Bay in
| Canada. In this part of Canada, referred to as "north of
| summer," having the right supplies in the correct amounts
| was literally a matter of survival. And, of course, the
| company's survival also depended on productive--and
| completed--expeditions. So the company provisioned the
| expedition's canoes with the necessary supplies for the
| trip. Then they sent the expedition team a short distance
| in their canoes to camp overnight. This was a test. The
| first camp was merely a "pull out," commonly called for
| many years a "Hudson's Bay Start" (HBS). Said Sir Samuel
| Benfield Steele in 1874, "[This was] very necessary so that
| before finally launching into the unknown one could see
| that nothing has been forgotten, or that if one had taken
| too much, being so near to the base, the mistake could be
| easily corrected." This risk reduction technique was not
| free. It cost the team precious time in a short trading
| season. They had shipping deadlines to meet. I can imagine
| them saying "Yes, but we really have to be on our way or
| we'll miss our shipping schedule."
| thedudeabides5 wrote:
| Step 1. Set up data pipelines that feed into a data warehouse _.
|
| Amendment if you are using financial data...
|
| _Step 1a. Rather then building this stuff yourself, go to rose.ai
| and use our financial data warehouse, pipelines*, and pre-built
| models to save yourself months.
|
| *If you are in tradfi, we have Bloomberg, Refinitiv, FRED, CapIQ,
| FactSet etc.
|
| If you are in crypto, we have integrations with the blockchain,
| Dune, coinmarketcap, coingecko etc.
| WarOnPrivacy wrote:
| > Faire's engineering team grew from five to over 100 engineers
| in three years.
|
| If I'm reading the (below) image correctly: They did this by
| staying as far away from older engineers as they possibly could.
|
| ref:
| https://www.ycombinator.com/blog/content/images/2022/10/3.jp...
| myvoiceismypass wrote:
| Yikes. Not a single grey hair to be found
| mescortes wrote:
| This photo was the whole company at the end of 2018!
|
| Engineering at that time was about 20 or so people.
| lupire wrote:
| They seem to have avoided Asians too.
| reillyse wrote:
| It's interesting to see CI wait times as core engineering metric.
| I 100% agree, so much so that I'm building a product specifically
| to speed up CI. We have redesigned how hosted CI works focusing
| on speed as our north star. We don't have CI wait time - your
| test starts immediately and we run your tests super fast. How do
| we do it? We have many workers with your test environment pre-
| built so we can split your tests and start running your tests in
| seconds. If anyone is interested you can check it out at
| https://brisktest.com/, I'd love to hear any feedback from the
| community.
| andreareina wrote:
| It looks like you're focused on ruby and javascript at the
| moment, is python on the roadmap and how far down?
| 0xbadcafebee wrote:
| _" Hiring the best engineers"_
|
| Every company can't have the best engineers. They cost too much
| and they're a finite resource. I would argue that the quality of
| engineers is just going down, too, as the hiring pool is flooded
| with people who've had zero tech experience, and then took a
| bootcamp and "churned on algorithms" to finally land a tech job.
| You probably will end up with poor to average engineers, and
| you'll have to deal with that.
|
| _" Building solid long-term foundations from day one"_
|
| You have average engineers. The foundations they make are not
| going to be solid. And even if they could make solid foundations,
| your founders won't care. They just want to ship ship ship ship
| ship. So your foundations aren't going to be solid... and you'll
| have to deal with that.
|
| _" Tracking metrics to guide decision-making"_
|
| Never seen it done well. If you have the time and staff to
| properly do this, you're either crazy lucky or are swimming in
| cash. Those days are probably gone... and you'll have to deal
| with that.
|
| _" Keeping teams small and independent"_
|
| Remember Conway's Law? The more teams you have, the more
| fractured your architecture becomes. Obviously you can't have one
| infinitely big team, so the key is to have as few teams as you
| can get away with. Integrate them tightly - same standards, same
| methods of communication and work. Eliminate monopolies of
| knowledge. Enforce workflows that naturally leads to everyone
| being more informed. (For example, every team having stand-up on
| a video call means none of the teams have any idea what the other
| teams are doing)
|
| There's actually a lot of evidence-based research and practical
| experience out there, that's been around for _decades_ , that
| shows how to maintain the productivity of organizations,
| regardless of whether they're growing or not. But they're not
| trendy, so nobody working today gives a crap. Actual productivity
| gets ignored while managers pat themselves on the back and start-
| ups churn out banal blog posts with the same lame advice that
| _thousands_ of startups have used and still completely failed to
| maintain velocity with. We all know how startups today really get
| by: burning through staff and money, and luck.
| robocat wrote:
| > Every company can't have the best engineers. They cost too
| much and they're a finite resource.
|
| Your argument assumes there is an efficient market for
| engineers, and it also assumes that an engineer's productivity
| is independent of the company environment.
|
| There are multiple counterexamples of startups that have built
| great teams for cheap money. There are many reasons why great
| engineers can be underpriced or under-productive, and many
| reasons why mediocre engineers can be overpriced.
|
| You _can_ get the best engineers if you have the smarts to do
| so. Currently on the front page are two relevant links:
|
| (1) a CEO's opinion on what that takes for the funnel (not
| engineering specific):
| https://www.ycombinator.com/blog/learnings-of-a-ceo-matt-sch...
|
| (2) a CTO's? opinion on how companies fail to catch the best
| engineers: https://blog.southparkcommons.com/startup-
| engineering-hiring...
|
| It definitely is not easy - and it takes unusual skill to
| succeed - but that doesn't make the advice wrong but only
| extremely difficult. If you lack the skill, then partner up
| with somebody who does, or find a mitigating strategy, or don't
| become a C*O for a startup. Edit: I am an engineer and I hate
| rah-rah thinking, but if you don't have enough belief, and you
| poohpooh all the advice of others, then perhaps you lack the
| temperament to be a founder. I was a successful founder, but
| only by joining with others with the skills I lacked as a
| cynical engineer-type. I think it is amazing that engineers are
| now regarded as critical - I am from a decade when MBAs were
| the hot thing!
| [deleted]
| diceduckmonk wrote:
| > We use Redshift to store data. As user events are happening,
| our relational database (MySQL) replicates them in Redshift.
| Within minutes, the data is available for queries and reports.
|
| Is data engineering really this simple?
| Fiahil wrote:
| No, making the data available is only the tip of the iceberg.
|
| Next in line is how to version your data so you can create
| consistent "snapshots" at given point in time with no leakages.
| itsmemattchung wrote:
| > For us, pods generally include 5 to 7 engineers (including an
| engineering manager), a product manager, a designer, and a data
| scientist.
|
| Similar approach to Amazon's internal two pizza team
| philosphy[0]: every internal team should be small enough that it
| can be fed with two pizzas.
|
| [0] https://www.theguardian.com/technology/2018/apr/24/the-
| two-p...
| beckingz wrote:
| Having a full team that can operate autonomously is the only
| way to scale effectively.
| cjblomqvist wrote:
| Each pod should have a clear leader. We have an engineering
| manager and a product manager co-lead each pod.
|
| So, 2 leaders is "a clear leader"? How does that work? Sounds
| contradictory?
| whakim wrote:
| I can't express how much I dislike the advice to be "data-driven"
| and collect all the data you possibly can because it could be
| useful someday. While this may or may not be sound business
| advice, it's deeply unsettling to see such profound disregard for
| user privacy trumpeted as a key to scaling quickly.
| ta5765765 wrote:
| Optus and Medicare in Australia have recently had major
| breaches and the fallout in public opinion is still going on.
|
| A panel on the news was interesting; one person likened data to
| uranium - dangerous to hold and difficult to dispose of. The
| other likened it to the new gold.
|
| It'll be nice when legislation tightens up to minimize the
| latter feeling.
| hotpotamus wrote:
| Huh, I fairly recently started working in healthcare records
| and I actually told people that they way they talk about PHI
| (personal health information), you'd think the hard drives
| were radioactive. In fact we had to come up with and follow
| some very detailed procedures to dispose of it.
| kondro wrote:
| Medibank (private health insurer), not Medicare (public
| health service). :)
| claudiulodro wrote:
| I would say it's not necessarily ideal to keep shipping a bunch
| of features and changes every week as you scale. Once you have an
| established customer base, change needs to be managed and
| customers need a heads-up on things that will affect their
| workflow: you'll want to do deprecation periods, beta periods,
| have backwards-compatibility, etc. No customer is grumpier than
| the one whos critical workflow you just totally changed without
| warning!
|
| I suppose as long as those sorts of tasks count as maintaining
| engineering velocity, it's all good though.
| vasco wrote:
| This is bullshit advice. Hire the best engineers? Might as well
| say "don't make mistakes" in a list of advice about how to
| minimize mistakes. Same thing with building solid foundations
| from day one, maybe you do it, but more likely is that the person
| that comes in to make sure velocity can keep up is different from
| the core group that was just iterating to find PMF.
|
| The other two points are more relevant, but I feel like half of
| this advice is not really useful. Way more to learn from
| descriptions of turning around a codebase of CI/CD pipelines that
| were struggling with slowness, flakeyness, contention and how to
| dig yourself out. Those stories at least you can learn from and
| adapt to your situation.
|
| If "hire the best engineers" is your advice for anything, that is
| only a tenable strategy for VC backed startups willing to burn
| 200k/person from day 0, but I guess this is on a VC blog, so what
| can you expect. More useful advice is "how to do X with normal
| people".
| marcinzm wrote:
| The thing is I've met founders who implicitly don't want to
| hire the best anything. The best are driven, intelligent and
| experienced which to some founders comes off as a threat. So
| they hire yes people instead who won't push back.
| _hl_ wrote:
| "Hire the best engineers?" doesn't read like bullshit to me. A
| lot of startuos start out with scrappy teams who only half know
| what they're doing in their domain. That's partly because the
| domain changes every couple of weeks.
|
| When you scale, that changes. It's _crucial_ that you hire
| competent people for whatever your startup ended up doing. Have
| a lot of frontend? Hire UX designers and senior frontend
| engineers with 10+ years experience. Doing advanced AI? Hire
| someone who 's been in that space for 5-10+ years. Etc. They're
| worth every penny.
|
| The logic of fake-it-till-you-make it doesn't scale, but many
| don't realize it early enough.
| hammock wrote:
| What is missing is the tradeoff. Hire the best engineers at
| the cost of what? It's not a strategy until that's
| articulated.
|
| For example you say "they're worth every penny" but are they?
| Would you borrow from your 401k to hire them? Pull your kids
| out of college to afford them? They might be worth a lot-
| what are you suggesting gets traded off?
| [deleted]
| [deleted]
| jasondigitized wrote:
| This is like a poorly written mission statement like "Win the
| enterprise market". Of course we want to win the enterprise
| market and hire the best engineers. Any statement whose
| converse would be an absurd business tactic should be treated
| as not particularly useful. "Hire the worst engineers!" This is
| like a college football coach saying "Recruit every 5-star
| player in the U.S."
|
| We all want to hire the best engineers from day one. The
| execution of that strategy requires a combination of early
| wins, interesting problems, technology stack, compensation,
| leadership, culture, vision and geography to attract top
| talent. Faire may be able to check most of the boxes, yet some
| rock star is going the find the business model boring and the
| leadership uninspiring and go work for an org that is more
| aligned to their values and ambition.
|
| Hiring the best engineers is table stakes for a company
| centered around a love and deep appreciation for building and
| shipping magical software.
|
| Even from day one, no competent founder says "I should
| deliberately not hire the best engineer I can find based on the
| comp I can offer". You may not be able to get them, but that
| shouldn't change your hiring strategy.
| Beltalowda wrote:
| "Hire the best engineers" is somewhat simplistic, and I don't
| think the article offered great insights on that, but ... all
| business where I worked that had problems actually Getting
| Stuff Done were due to either bad hires or excessively bad code
| (which is often the same as "bad hires", or rather, the result
| of it).
|
| I've never seen tooling really help. I mean, it's nice, saves
| time and effort, ensures some degree of quality, and all of
| that, but it doesn't really move the needle all that much in
| getting stuff done compared to having a team that works well
| together. Actually, I've seen desperate adoption of tooling in
| futile attempts to fix these more fundamental issues, which is
| how some companies end up with 40 linters that moan and whine
| about every little thing so everyone is adding //nolint
| statements every 20 lines.
|
| In short, I do think that it's good advice, although personally
| I'd phrase it as "hire the right people". Someone who is a good
| engineer - maybe even among the best - will not necessarily fit
| in well with any team, or any job, and there are other
| important qualities too aside from engineering skill. A junior
| can be a good hire for your team, even in a startup scenario.
|
| How you actually hire the "right people" is of course not easy.
| jgulbronson wrote:
| I was the first backend engineer hired at Faire joining 2 of
| the cofounders in early 2018. I can assure you I wasn't getting
| paid anywhere close to $200k when I started, and neither were
| my extremely talented colleagues.
| vasco wrote:
| This is going to sound harsh, but based on this, you weren't
| the "best engineers", I'm sorry to say. If you think the best
| engineering talent in the world is working on some random
| ecommerce website for less than 200k/yr, I have news for you.
| And you know what, that's fine, because people who care about
| "best engineers" usually suck. What a vapid thing to waste
| time wondering about. I'd bet you have hardcore impostor
| syndrome in all your new hires if this is the messaging you
| put out.
|
| What you want is hard working people that are cool to work
| alongside and fix problems without much drama or bike
| shedding, not "the best". This circle jerk of "best
| engineers" is only useful to pay you less and keep you happy.
|
| Instead of calling each other best, focus on the work and
| show the work, if it's good, people will judge your
| organisation's engineering skills appropriately, otherwise
| you're just patting your own back for no reason.
| jgulbronson wrote:
| All good! I agree with you, at the time I wasn't one of
| "the best". But I think there's a couple key things, one of
| which you touched on.
|
| The first is that "the best" is subjective. To me, the best
| engineers _are_ the ones who you enjoy working with and
| don't cause drama or bikeshedding. Some 10x programmer who
| everyone hates working with isn't "the best". So perhaps it
| all depends on what traits others are looking for.
|
| The other is that as a startup you can try to hire "the
| best" engineers today, _or_ hire people who you see
| potential in and can grow with the company. I'd put myself
| in that latter group, along some of my extremely talented
| coworkers I mentioned. Having that growth opportunity is
| part of what attracted me to Faire in the first place, and
| is a way to compete for talent on more than just comp like
| the original comment implied.
| vasco wrote:
| Thanks for sharing! And sorry again for being harsh, I'm
| keen on reading some more shares about actual strategies
| in engineering tooling or others that you worked on to
| keep things chugging along. The focus on metrics I think
| is very under-appreciated in our industry and the more we
| can standardize to compare velocity and easily identify
| companies that don't give a shit about developer
| experience and velocity, the better in my opinion.
|
| I would say that I believe it's a much happier life the
| less you think about how good you or your colleagues are
| and the more you think about the systems and the tools
| you develop. I really meant what I said about impostor
| syndrome, I'd be curious what answers you'd get from your
| engineering team regarding that, if so far you've focused
| a lot of your internal messaging / culture on "being
| good", you might find some opportunities for honest
| feedback and sharing which might get a more tight-knit
| empowered team, which does wonders for velocity.
| drewcoo wrote:
| There's a nice "you did it wrong" pre-built into "hire the best
| engineers."
|
| What? That didn't work? Well, they weren't the best.
| cheschire wrote:
| My reaction to the table of contents was similar to yours, and
| I think it was a poor choice of words to summarize the first
| section.
|
| Maybe something like "Hire customer-oriented engineers"
| would've been a better title for section one.
|
| I think the "grit" and expertise in core technology are also
| important points, but it doesn't hook a reader the same way as
| saying "don't hire an engineer that can't adapt to the
| customer" (gloss over the double negative)
| hatware wrote:
| > Hire the best engineers?
|
| Whenever I see this, I read it as "Hire the most loyal drones."
| hayst4ck wrote:
| Here is what I think are several root causes of poor velocity
| 1. too much focus on hiring 2. lack of clear
| responsibilities 3. lack of management <-> line worker
| interaction 4. bad mentor <-> new grad ratios 5. bad
| product development to infra (build infra/infra infra/dev tools
| etc) ratios 6. mistaking prolific senior engineers for good
| senior engineers 7. letting senior engineers off the hook
| for maintenance 8. lack of some process 9. hiring
| specialists
|
| One can ask what sacrifices you make to hire good engineers. You
| might choose to make exciting infrastructure investments rather
| than a necessary investment. You might promise that the "good
| engineer" won't have to do incredibly boring work. You might hire
| people who have made a career out of avoiding the real high risk
| pain centers of a company and instead working on high visibility
| low risk problems. How much of which engineer's days will be
| sacrificed to interviews? The engineering concessions made
| towards the goal of hiring are likely an underrated root cause of
| poor velocity.
|
| I watched the most festering pile of code at a company be hot
| potato-d between the vp of infra and vp of product. The CTO was
| checked out and not in touch with what was happening enough to
| know this was a problem. Neither VPs brought it up as a problem,
| because neither wanted responsibility and therefore the likely
| black mark by their names for the uphill battle that would
| result. The company deeply suffered because there was no advocate
| for the companies highest pain area because everyone with power,
| clout, or authority avoided responsibility for it.
|
| When management gets insular, and management fails to solicit
| direct feedback for line workers, they can't be sure the picture
| they have in their head is what matches reality. This creates
| management level delusions about the state of their engineering
| org. We can see this played out in US vs Russian military
| structure. Management sends goals down and expects them adhered
| to. Failure results in punishment. This creates rigid planning
| and low agility. The US military instead gives lower levels large
| leeway to achieve higher level goals. It is the lower levels
| responsibility to communicate feasibility or feedback, and more
| importantly it is upper managements responsibility to adapt plans
| based on feedback. I was absolutely part of an "e-4 mafia"
| (https://www.urbandictionary.com/define.php?term=E4-Mafia) and I
| knew much better than my superiors what was happening, why it was
| happening, who was doing it, who could help doing it, and its
| likelihood of success because I was in the weeds. When I laughed
| directly at managers who told me their plans, they thought it was
| something wrong with me, not something wrong with their plans.
| That was half management failing, and half my inexperience in
| leading upwards.
|
| Every new grad needs one mentor to prevent them from doing
| absolutely insane overly complicated things. If you do not have a
| level of oversight, complexity will bloom until it festers. A
| good mentor preventing new grad over complications can save an
| incredible amount of headaches. New grads should not be doing
| other new grads code reviews (for substantial work). Teams should
| not be comprised entirely of new grads and an inexperienced tech
| lead. New grads are consistently the largest generators of
| complexity.
|
| I worked at a place where there was 1 person working on build
| infra. .2% of the company was devoted to making sure we had clean
| reliable builds. I estimate 5-15% of the engineering org quit due
| to pain developing software, which meant there was a lot of time
| spent interviewing people and catching them up rather than fixing
| problems. I don't know what the right ratio is, but I can say for
| sure that if you don't invest in dev tools/build infra etc, early
| enough, you will hit a wall and it will be damaging if not a
| mortal wound.
|
| There are lots of engineers who code things to be interesting.
| They write overly complex code. They lay down traps in their
| code. It's rare for there to be a senior engineer who writes
| boring, effective, and, most importantly, simple code. Some of
| the code I've seen senior engineers write violates every
| principle of low coupling, no global state, being easy to test,
| etc. These people are then given new grads who learn to cargo
| cult their complexity until it gets to the point where someone
| says 'we have to re-write this from scratch.'
|
| There is an anti-pattern where senior engineers get to create a
| service with no oversight, then give it to other people to
| maintain and build upon or "finish." Those teams seem to have low
| morale and high turnover. The people left on those teams aren't
| impressive and so it gets harder to hire good engineers for those
| teams. If a team is the lowest rung on the ladder, clearly
| evidenced by being given problems and being told to "deal with
| it," that will show to new hires only exacerbating the problem.
|
| Some people hate process, it slows them down. Bureaucracy is
| (debate-ably) terrible. One design doc with a review can save
| quarters of work. Some process slows progress down now, for less
| road blocks later. If process is not growing at a rate of
| O(log(n)) or growing at a rate greater than O(log(n)), then
| there's probably gonna be some problems.
|
| Lastly, while it's important to hire good people, it's also
| important to hire some specialists. Databases, infra, dev tools,
| build infra, platform/framework infra, various front-end things,
| traffic infrastructure. There are all types of specializations,
| and if you have a good "full stack" product engineer in the room
| without say a platform/framework specialist, you will get the
| product development perspective without the product maintenance
| perspective, and that has exactly the consequences you might
| expect. The earlier you get an advocate for say "build
| infrastructure," the more you are able to address future problems
| before they are major problems.
| bentlegen wrote:
| Genuine question that isn't answered in the article: what is it
| that Faire is doing that suggests they're maintaining engineering
| velocity as they scale in a way that is equal or better to anyone
| else out there? Alternatively, what are the engineering
| challenges of running Faire's storefront/marketplace makes them
| more qualified to write about their experiences scaling vs. other
| organizations?
|
| This article opens with an unsupported assumption, "we are really
| good at this", and doesn't really elaborate on what that means.
| I'd genuinely like to know what great engineering at scale looks
| like, not just some suggested ways to do it.
| kevmo314 wrote:
| Adding on all that, is a 100 person team really necessary to
| run a storefront? That seems like the antithesis of maintaining
| engineering velocity.
| winphone1974 wrote:
| From my experience scaling to a 100 is also pretty easy. You
| don't get consistent results across all your teams but you're
| small enough to mask this with direct communication and adhoc
| process. The foundational stuff really pays off when you go
| to 200, 300 plus, and you will definitely slow down.
| rubyist5eva wrote:
| lol this feels like exactly the opposite of what's going on at my
| company now:
|
| 1. Hire the best engineers: fire half the dev team and replace
| them with offshore devs for pennies on the dollar 2. build solid
| foundations: cut every corner possible to get whatever crazy deal
| our sales team made yesterday 3. tracking metrics: uptime? who
| cares, CI taking too long? whatever
|
| well, I suppose our teams are small when they literally fired
| everyone and made the "team" so small it literally couldn't be
| smaller or it wouldn't be a team (there is 2 of us now). Hardly
| independent though since we're shackled to the whims of clueless
| sales drones that have zero clue how building software works.
| winphone1974 wrote:
| So they value "grit" which is defined as the ability to code and
| push features in near real time, as told to the CEO at a multi
| day trade show, then follow that up with explaining the
| importance of building a solid foundation.
|
| Pick a lane.
| sitkack wrote:
| Doesn't mention Conway's Law or the Scientific Method, Continuous
| Feedback or Composition.
|
| Scaling an organization uses the same techniques as creating
| scalable (computer) systems. Systems are systems.
|
| https://en.wikipedia.org/wiki/Conway%27s_law
|
| If your systems relies on "hiring the best engineers", it is
| operating Open Loop. All Open Loop systems will suffer
| catastrophic failure at some point.
|
| Grit is a dog whistle for grind. You can be tough and resilient
| and flexible w/o being gritty.
| arunc wrote:
| This is my practical experience. I couldn't have put it better
| in words! Thanks!
| hrpnk wrote:
| > Pods should operate like small startups
|
| This only works if a pod is equal to a domain that's fairly
| independent of the other domains, technically, and business-wise.
|
| If there are N pods per domain, each being their own startup
| without additional coordination results in chaos and duplicated
| work. Business complexity not included (two pods from different
| domains can unknowingly work against each other due to having
| conflicting goals/targets).
| ThalesX wrote:
| I wish I'd see advice where they prioritize having a roadmap,
| milestones, an actual plan of execution, after Product Market Fit
| (PMF) has been found. And before you find PMF, any concern for
| hiring the 'best engineers', building a 'solid foundation' is
| moot.
|
| I feel I'm going insane expecting product people to put the time
| in to define the requirements and context; I get weird looks
| asking startups about the plan for the next week. "That's how
| startups do it" is just the most bullshit excuse I keep hearing
| constantly regarding lack of planning.
| hef19898 wrote:
| "That's how startups do it" is the startup equivalent to
| "That's how we always did it" at big companies. And both
| approaches suck equally.
| lifeisstillgood wrote:
| - Have devleads as the central crux for every major decision.
| Screw "senior" management - make sure that the people who are at
| the codeface every day are part of the major discussions - that
| those people need to be persuaded of the need for product X or
| pivot Y. Because they do whether you treat them like it or not.
|
| - the above means you are "surfacing" politics. Do not keep the
| backbiting and infighting amoung a cabal of senior managers who
| talk to each other a lot. Make it public
|
| - Have _one_ place to have these discussions - an email list
| probably, and the only way to get your project signed off is to
| get agreement on this list (well, Ok the CTO can have some veto
| power - this is not a democracy of course)
|
| - Analysis really works. Publish one-page analysis of the
| proposed project. Watch it get destroyed with evidence and
| laughter and then watch _even better ideas_ get proposed.
|
| In short scaling is a political problem - treat it like one. And
| engineer the horror out of politics. Democracy and transparency
| are pretty good at that.
|
| Edit: Buidling a business IMO has strategic, operational and
| tactical levels. Strategy should be obvious, be PMF and be well
| known in the company (a PC on every Desktop). Most of the article
| is _tactics_ - hiring, metrics, stack etc. The hard part is often
| operational - and that is almost always orgabisation design and
| that is about communication, alignment of resources, trade offs,
| etc etc. That 's hard. Dysfunctional organisations blow this.
| Open politics fights dysfunction
___________________________________________________________________
(page generated 2022-10-25 23:00 UTC)