[HN Gopher] Build versus Buy ___________________________________________________________________ Build versus Buy Author : sciurus Score : 159 points Date : 2020-04-13 16:12 UTC (6 hours ago) (HTM) web link (lethain.com) (TXT) w3m dump (lethain.com) | ensignavenger wrote: | When you are looking at paying for an expiring license or SaaS | product, it is build versus Rent. And one cost of renting a | product is that your vendor's business may change. Perhaps they | get bought by Apple and the product is shut down, or maybe they | just decide to go in a different direction. Or they raise their | prices. Or the service isn't as good as you thought it was going | to be. If you have done a lot of integration work (either in | software or organizationally) you are either locked in or all | that work is wasted. | | This is addressed in the article as "risk". | | The best way to balance this is to prefer buying/renting Open | Source software. That way, if your vendor changes direction, or | their service level drops unacceptably, you can choose a new | vendor without losing your integrations. Or even switch to a | self-supported model. | | And any improvements you need, you can contribute to the project | so that others can take advantage of them and share the | maintenance burden. | gk1 wrote: | As someone who helps software vendors overhaul their messaging to | attract and convert buyers, I've noticed the following things | about the "build vs buy" dilemma (or, from the vendor's point of | view: the "build vs buy" objection): | | - More senior buyers (CTOs, VPs) are less likely to pit the | vendor against a DIY option. I suspect it's because they've been | through too many failed DIY projects, or they have great clarity | on what is and isn't a good use of their people's time, or they | are better at estimating the (usually huge) amount of effort that | would go into DIY. | | - As the OP points out, a company with huge engineering resources | is more likely to consider the DIY option, because, hey, they | have all these engineers that need something to do. So if you're | a software vendor and are tired of the "build vs buy" | conversation, stop trying to sell to other software vendors or | VC-backed tech companies. | | - If someone chooses the DIY route, just set a calendar reminder | to check in two months from now. Chances are they'll be | underwhelmed with their progress, and an out-of-the-box solution | might sound pretty good by then. (Don't assume they'd reach out | on their own in that scenario--they've probably spoken with | dozens of other vendors since your last conversation, and forgot | about you.) | | === | | Separately, the part about vendor management is gold: | | > To get the full value from vendors, you have to invest in | managing vendors well. | | I've spent _so_ much time helping evaluate and implement | different vendors (usually marketing platforms and similar), only | to find out three months later that the software barely gets | used. | | On the flip side, if you're a software vendor, don't wait until | renewal periods to check in on your customers and make sure | they're getting the maximum value from your product. | teddyh wrote: | "If it's a core business function -- do it yourself, no matter | what." | | -- Joel Spolsky, _In Defense of Not-Invented-Here Syndrome_ : | https://www.joelonsoftware.com/2001/10/14/in-defense-of-not-... | ternaryoperator wrote: | Except encryption. You don't want to roll your own at all. | yjftsjthsd-h wrote: | I think the advice holds. Cryptographic primitives are | vanishingly unlikely to be your core business, even if the | things built on them are. Therefore, almost nobody should be | building them. | ska wrote: | I don't think that's an exception. It's just that encryption | isn't actually core functionality for most businesses (even | some that think it is). | | And if it is core functionality, you should have staff | qualified to roll their own, and you should all know why you | are doing this (i.e. what differentiation you are going | after). This really isn't limited to encryption, it's just a | hard domain that bumps up against a lot of other domains, so | you see more failures than some others. | | Basically if you are going to build a startup or new program | in a hard domain, you need to have the resources to do it | properly, and you need to know what those are before you | start. This is where being a team of smart and motivated | individuals may just not be enough. | kstrauser wrote: | Super important exception. Thanks for mentioning it! | | Unless you've spent the last 10+ years working on encryption | full time, _please_ don 't try it yourself. | | I used to think I could write crypto. Then I learned more and | realized it needs specialized knowledge I don't have the time | to acquire. Then I spent some time trying to acquire it and | figured I was OK to try again. Then I watched many other more | experienced organizations crash and burn. My current position | is that I know enough about crypto to know I never want | anything to do with authoring it. | jedberg wrote: | Don't take the wrong message from Joel here though. Make sure | it's actually a core business function. Is it something that | will actually give you a competitive advantage if you do it | better than everyone else? | jpdb wrote: | > Is it something that will actually give you a competitive | advantage if you do it better than everyone else? | | I think you also need to evaluate if it can even be done | better than everyone else, and the effort required to do so. | ska wrote: | Sure, but that's sort of the entire point of your business. | | Joel's quote is more narrow, which is a subset of the | dangers of being completely reliant on someone else for | your business. | kstrauser wrote: | I've worked at Not-Invented-Here shops and it was annoying that | the Powers That Be didn't want to use a COTS solution. As much | fun as it would be to reinvent DynamoDB, that's not actually | going to help our business. | | However, even more annoying is working at a Never-Invented-Here | organization. I've been at a place which hired brilliant | people, but then didn't let them innovate on pretty much | anything. Write a simple TCP listener in Python to read a | stream of data and then reply to it? That's crazy talk! We need | to write a full Django app and deploy it behind Nginx behind a | load balancer so that we can pretend it's highly available! | | Don't get me wrong: the off the shelf approach is _usually_ the | right thing, but sometimes it just adds complication where it | didn 't need to be. It's insulting to be told you were hired | for your expertise, but then not be allowed to actually use it. | Ididntdothis wrote: | "Write a simple TCP listener in Python to read a stream of | data and then reply to it? That's crazy talk!" | | This has infected my company quite a bit. People get really | nervous as soon as you write something that's little novel. | Writing everything from scratch is bad but you also need room | for people to innovate. | aledalgrande wrote: | I'd argue that in a startup, at a very early stage, you can use | external services even for core functions, if they work well | enough. | | Look at Intercom: now it's a juggernaut, but it started as an | API mashup with a UI in front of it. If you can hit the market | and start getting revenue faster you should. Many services have | pay as you go and/or startup discounts. | | That's your launch platform, then you start working from there. | DrNuke wrote: | You are fundamentally buying your time, so buy as much as you can | and stay focused on your own business? | KaiserPro wrote: | Tangent: | | We had the same issue with splunk. We were blowing through our | licenses like a champ. | | However the thing that managed to reduce our dependency on splunk | wasnt ELK or greylog (they are all poor comparisons to splunk in | terms of speed or chopping and changing data) it was grafana(with | cloudwatch and graphite, later promethius) | | What we realised was that the primary way we were consuming | splunk was in graphs. Why spend millions of pounds on generating | graphs indirectly, when we could make them directly, in near real | time! | | Not Tangent: | | I think like platform choice, it depends on your company. The | company that I was with was utterly incompetent, and unable to | see a project through to completion. Therefore if we build a | logging system inhouse, we'd get 40% of the way there and rebuild | it. Leaving two incomplete systems. | | Migrating to splunk cloud was done to save costs, and it actually | worked. Had we tried to go inhouse, we'd still have expensive | local splunk _and_ a shitty splunk replacement used by two team | on a legacy critical system with no dev hours. | HolaDonPepito wrote: | :D | rachelbythebay wrote: | Is it really buy? Or is it "rent"? | | I'm guessing for most people, it's actually "rent". | hauschildt wrote: | Nice article. I had published my thoughts from a CTOs | perspectives on this topic a while ago. Maybe it's a good | addition to your article for anyone: | | https://blog.photoeditorsdk.com/build-or-buy-f09785ce1138 | ChefboyOG wrote: | This is the core of the whole debate, in my experience: | | "When it comes to build versus buy, the frequently repeated but | rarely followed wisdom is good advice: if you're a technology | company, vendors usually generate significant value if they're | outside your company's core competency; within your core | competency, they generally slow you down." | | Every startup I've ever worked with has had engineers who | advocated "build" for tooling that fell both outside the eng | team's workflow and our company's core competencies. | | I think part of it is that things like marketing automation or | customer support software--things that are vital for most teams | past a certain scale--don't seem technically "hard" enough to | justify the price-tags, particularly to engineers who've been | isolated from other parts of the business. | | Without fail, every time I've been part of a team building an | internal tool that we/another engineering team won't use, it goes | poorly. I've spent way too long learning about the ins and outs | of email deliverability so that I could fix a bug in a hacky | email platform, when we could have just bought a platform that | works. | Swizec wrote: | > I think part of it is that things like marketing automation | or customer support software, don't seem technically "hard" | enough to justify the price-tags | | As someone who's built far too many of those because "Buying is | just too expensive" ... good god do I never want to do that | ever again. Ever. It's so fucking hard. You _will_ shoot | yourself in the foot. You _will_ suffer. And 3 years later you | still won 't have something half as good as the purpose-built | SaaS. | | If you do that and enjoy it, just spin it off into its own | product. Who knows, you might even get more customers with more | expensive problems than the original startup. | viklove wrote: | This only makes sense if you have a significant equity stake | in the company you're working for. For the vast majority of | engineers, they really don't care about the long-term | profitability/success of whatever company they're working | for. They just want to get paid, and hopefully grow as a dev | in the process. | | They'll learn more by building something in-house than by | bringing in a saas and just writing glue for a living. If you | want your engineers to make the decisions a founder would | make, give them a significant equity stake. | Swizec wrote: | It depends. Significant stake helps of course and despite | that, I'd prefer solving problems that need solving than | spending a bunch of time on problems that don't :) | | There's a balance. | softfalcon wrote: | "A company that does this extraordinarily well is Amazon, who | issue their vendors quarterly report cards grading their | performance against the contract and expectations. Getting great | results from vendors requires managing them. If you neglect them | and get bad results, that's on you." | | - Gives JIRA a report card on them as a vendor to your business. | | - Is largely ignored because my business is not Amazon scale with | all the power that holds. | | tldr; this only really works if you're one of the whales for a | vendor. | trimbo wrote: | I'm super surprised to see this article not mention the phrase | "opportunity cost". | | That's what someone pays to save with something like Splunk. Pay | cash to make SWEs more effective towards solving the problems | that make up the business's value prop. Cash is much easier to | get than the opportunity to grow the business with the people | already there. That time never comes back. | tschellenbach wrote: | I think it's all about focus. There is a limit to how many | goals/projects you can focus on at once. Every new thing you | build distracts your focus. You should minimize what you build | in-house to just the things that make you unique. I like | Unsplash' blogpost about this: | https://medium.com/unsplash/scaling-unsplash-with-a-small-te... | rconti wrote: | One thing overlooked here (or not explicitly called out) is | talent pool. My last boss didn't want to use anything that he | couldn't plug into Dice Or Similar and get dozens of hits for. | | If it's not part of your core business/core competency, and | you're just reinventing the wheel so you don't have to pay a | vendor, you have to weigh against the cost of the off-the-shelf | solution NOT just the cost of developing it yourself, but | developing it, maintaining it, documenting it, replacing your | "irreplaceable engineer" when s/he quits, etc. | tschellenbach wrote: | One thing that's commonly overlooked is the risk/vendor lock in | associated with building things in-house. Sure external vendors | sometimes raise prices or shutdown their product. Those events | are pretty rare though and you can typically move to a competing | solution. The real risk is building in-house and having the | people that create the in-house solution leaving. | TAForObvReasons wrote: | > raise prices | | Many SaaS have 10x'd their prices over the last few years and | it is a significant risk for any tool that you are "renting". | adamc wrote: | Moving to another "competing solution" can be extremely | expensive. | [deleted] | tensor wrote: | I'm surprised "pricing transparency" isn't included here. A | vender that requires you to call a salesperson and do the sort of | negotiation on price that is mentioned here is always going to | take advantage of you. | | My policy is: no pricing on the website, requirement to call a | salesperson to talk price? No buy. I'm even willing to pay more | to a company with transparent pricing that doesn't require | negotiation. It gives me predictability, rather than have to | worry that once they've "locked me in" they're going to jack the | price and crank all the gears. | Ididntdothis wrote: | I think it's good to buy something if you can use it immediately | without much customization. If you have to do a lot of | customization it's not that clear. For example we have bought a | system that requires extensive customization and a lot of people | are starting to wonder if the total effort of building from | scratch that does wha we need and not more would have been less | than the never ending customizations of a system that can do way | more than we need. | | There is also the question of brain drain. The IT department in | my company has outsourced a lot of the tech work to the point | that there aren't many people left who are capable of making | informed purchasing decisions. They mostly read sales | presentations, white papers and Gardner reports but they have no | ability to really assess things. This shows in them repeatedly | buying expensive systems that never get really used. | marcosdumay wrote: | That is integration and evolution costs on the article. Those | tend to be very hard to estimate, and are the ones that most | grow out of control. The article really doesn't do them justice | by passing through them like if they were like the others. | | There is also an undereported gain that is freeing your team to | work on higher impact work. | A4ET8a8uTh0 wrote: | It is a really hard question. | | I can attest that the sales presentations tend to be overly | optimistic and cheery and if the head guy really wants a given | solution, from office politics perspective, you might as well | just nod your head, point out the issues you think should be | addressed just so that you are covered and roll with it. In | practical terms, the decision is not up to you. | | In one of my previous roles, manager simply said it is | happening ( so it did ), spent some time with the system not | fully understanding the underlying logic and was somewhat | surprised, when it did not work as advertised. IT had to get | involved to smooth some edges. The roll out was still painful. | But the contract was already signed and the advertised feature | shown during presentation was now extra money, so we were | forced to use incomplete product. Annoying. | | But then, the real answer is that it varies. Everyone needs to | do their due diligence. | | What I did not notice is that managers see pretty dashboard and | fail to consider the use of the rank and file users. This is | the one thing I have learned from all this. | | Never underestimate the impact of a pretty dashboard. | davidy123 wrote: | Build, buy, and engage if it's part of a larger project that | involves any innovation. Engagement falls under "vendor | management," but that seems like a very reduced description. | jedberg wrote: | My preference in these situations is always to start with buy, | and then try to justify build. Unless it's a core competency to | your own business, which it almost always isn't, it makes sense | to let someone else do it better. | | Their profit margin is usually in their efficiency in doing it | better than you, and at the same time you get the benefit of all | the features they add for other customers. | | Oftentimes the decision to build usually came down to the | difficulty in integration of a commercial product, not the cost. | In most cases the cost is set to be competitive against building | yourself. | | Except in the case of Splunk. Their pricing is ridiculous. | snewman wrote: | > Their profit margin is usually in their efficiency in doing | it better than you | | This is a very nice way of framing the question. | | I think there may be at least two scenarios to consider. For | some products, cost is primarily driven by R&D; for others, | cost of operating the service (COGS) dominates. Of course it's | a continuum, but I suspect there are more services near the | ends than the middle. | | For an R&D-dominated service, if your needs are nontrivial and | there is an off-the-shelf product that is a decent fit, you | probably want to go off-the-shelf. A commercial provider can | amortize the R&D over a large customer base. | | For a COGS-dominated service, the commercial provider may still | be more efficient but it is less of a slam dunk. Log management | (e.g. Splunk) involves a substantial operational component, | because you're dealing with large volumes of data. | | Disclaimer: I am the founder of Scalyr, a Splunk competitor. | Early on we realized that, even though log management is COGS- | heavy, we could achieve huge economies of scale, because query | workloads are highly bursty. By carefully managing a central | pool of resources shared by many clients, we're able to run | much more efficiently than a single-tenant homegrown solution | [1]. This is one of the key points that we highlight when | having build vs. buy discussions with potential customers. | | [1] https://www.scalyr.com/blog/searching-1tb-sec-systems- | engine... | dzimine wrote: | A story of Netflix adopting StackStorm for autoremediation is a | nice illustration of this approach. They began to build a tool, | learned enough to be dangerous, discovered StackStorm and used | it, experimented with it like with a breadboard, until finally | figured out exactly what works - and reimplemented it. All | these time learning the system was working and delivering vale. | | Part of this story is here | https://netflixtechblog.com/introducing-winston-event-driven... | | Disclamer: StackStorm founder here (but not pitching). | GiorgioG wrote: | And Splunk is horrendously slow in my experience. | sethammons wrote: | We self host splunk and it can plow through petabytes of high | cardinality data pretty dang fast. If the fields are not | indexed and the search is complex, it can take minutes or | hours. But usually, I can get live and historic data in a few | seconds. | | As an example, we have a pipeline of services. I can compute | the time spent in each service with multiple levels of | percentiles and group that data by high cardinality fields | (as in, hundreds of thousands or more values). I just did a | search for 4 hours of data across thousands of nodes for half | a dozen or so services with multiple eval statements all | piped to a timechart doing over a dozen stats operations. | Half a billion events. It got done in under a minute. | | Splunk charges so much because they are just so dang | powerful. | wenc wrote: | This has not been true in my experience. I run a Splunk | server in production and at my data volumes it has been very | performant. It's also much easier to setup and maintain than | ELK clusters. | | In the early days Splunk pricing was exorbitant (we evaluated | Splunk 7 years ago and dismissed it), but licensing has | changed in recent years and it is now priced by volume | ingested (the pricing is transparent and listed on their | website now). At low volumes, the pricing is similar to | Sumologic, and is pretty accessible now to smaller dev shops. | Open-source collectors like _fluentd_ also help to | intelligently reduce the ingest volume. | | At high volumes, the TCO changes quite a bit. | GiorgioG wrote: | My experience is solely at work where we use Splunk Cloud | and it's slow as molasses. | m0xte wrote: | Same experience. Also splunk forwarders appear to be | universally unreliable. | dzimine wrote: | Does the speed matter? How exactly? I am genuinely | curious: at Scalyr we _can_ be very fast but it is a | balance with the cost that we want to pass on as price | savings. Same with self-hosted Elastic: one can fine-tune | it to be fast but minding the cost constraints gets it | slower. WDYT? | wenc wrote: | Ours is self-hosted and is plenty fast. This might be | something to bring up with the Splunk folks. Maybe their | cloud side of things needs tuning. | hinkley wrote: | It's Pareto principle for me. >80% of our code should be off | the shelf. The special sauce is 10% of the code. What should we | spend that on? What will make us stand out? | | If you can answer that, it's all the justification you need for | building it. Anything you build that doesn't differentiate you | can be bought by a competitor. | turbinerneiter wrote: | I agree with the idea, but I also think there are limits. | | Let's say you build some kind of product based on some | Azure/AWS product. The negative reading of it is: you spend | money on innovation and have to pay Azure/AWS for | infrastructure. Your product will constantly have to improve | to keep up with the competitors, Azure/AWS keeps getting your | money. And if your product becomes stable enough, they will | make it part of their offering. | | Another example would be Apps, with Apple later folding their | functionality into the OS and kicking your app off the store. | | But judging by my own examples, this migth be more of a | platform problem. | NomDePlum wrote: | Surely that very much depends on whether your product fits | within the current or near future offering of Azure/AWS. | | The other side of that coin is that you use a build | horizontal integration strategy to replace elements of the | products you buy from Azure/AWS if (and only if) it can be | done in a way that increase your profit/value. | | It can work both ways. Moving from buy to build also has | elements of de-risking and economies of scale if you do it | at the point where there is measurable success/product to | market fit. | | Building something you can buy too early runs the risk of | premature optimisation if not done for the right reasons. | HolaDonPepito wrote: | :O | DanHulton wrote: | Absolutely. This is largely what I'm betting on with my | latest project, that the time you spend spinning up all the | "standard stuff" for a SaaS project is duplicated - and thus | wasted - effort throughout the industry. Why write another | user login system? | | Though it sets up an interesting situation for me, where the | "special sauce" of _my product_ is the 80% "off the shelf" | for my customers. | BiteCode_dev wrote: | As usual, it depends. I know some friends that have | profitable websites because they run on a private server and | they handcrafted everything. If they had to use cloud | services for serving stuff, host the db, provide search and | cache things, it would cost too much and they couldn't make a | living out of it. | rconti wrote: | To me, this shows that the actual service/content they're | providing has _relatively_ low value, and so they 're | "making" money not just on the content but also on the | services side; eg, they're a cheaper sysadmin than an | outsourced one. And they're also possibly ignoring risk; | eg, they don't have the data durability that a cloud | provider does, but it Probably Won't Matter (TM). | | There's absolutely nothing wrong with this way of making | money; the lower the margins, the more watching your costs | is crucially important, and the difference between making a | living and not even breaking even! | IanCal wrote: | Possibly, it may also be that the cloud providers are | offering things they don't need. Maybe that's uptime | guarantees, potential scalability or integration with | other services. | BiteCode_dev wrote: | Sure. | | I just mention it because most people in the world have a | job that make them less money than my friends web | services. | | And I'm assuming most startups, unless heavily founded, | may have to think about this trade off as well. | | The cost in infra can easily be a factor of ten. | aledalgrande wrote: | If you take SaaS, they are skewed towards buy, because | their margin is ridiculously high. Better to hit the | market now by using stable products than optimizing on | spend. | | The bigger you are and the more you can think about | optimizing spend. | weinzierl wrote: | > _Except in the case of Splunk. Their pricing is ridiculous._ | | Their pricing is ridiculous and what is even stranger is that | with the ELK stack there is a good and free alternative. Now | Splunk is good, no doubt, but I still wonder how they can be | that successful with that pricing. | m0xte wrote: | Lock in and one trick pony trained staff. I'm going to kill | it in the next 12 months in our org if it doesn't kill me | first. | dmitriid wrote: | Splunk is insanely good. To the point that people forget that | other tools exist, or that it makes sense to build | specialized solutions when needed. | | So people start running anything -- from dashboards to | analytics -- from Splunk alone. Their ability to combine and | extract data from almost anything in very small amounts of | time is unparalleled IMO. | mumblemumble wrote: | Another reason I like to start with buy: It makes scope creep a | lot harder. You have a specific feature set, and it lets you do | certain things, and not others. You may eventually find there | are certain things it has that you can't live without, at which | point you may still have to build, but that's offset by the | fact that you'll probably also discover there are features you | only thought you needed. | | By contrast, whenever I've been on a project where the company | went straight to building their own, it ended up being a | quagmire of scope creep, and the development team would end up | stuck on an endless treadmill of implementing features that | stakeholders vehemently insist on, but never actually use. | | Even for dev tooling and libraries. Some of the worst piles of | technical debt I've encountered happened when some rockstar | decided that they knew better than everyone else, and dived | straight into building what they thought would be a better | mouse trap. It's not that the in-house option isn't any more or | less likely to solve the problem well (no comment), so much as | that the in-house option is going to end up being more deeply | tangled with the rest of the codebase. So you're stuck with it, | because, even if it doesn't turn out to be all it's cracked up | to be, migrating away from it may be next to impossible. | Homegrown ORMs seem to always turn out this way. | | The one exception is when you know that a quick-and-dirty DIY | solution will be cheaper and easier, because it needs to do | only one specific thing while the off-the-shelf option has to | take on a bunch of extra complexity in order to be general | enough for everyone's needs. But even there, I'm only likely to | trust that argument if it's being made by someone who's been | burned by the complexities of the off-the-shelf solution in the | past, and who has a proven track record of obeying the KISS | principle. | hyperpallium wrote: | Modularity by purchase. Information hiding by ownership. | Interfaces by corporation. | | Brooks' software system products (9x the work) is usually | done this way _in practice_. In most usage cases, I would | also include open source projects. | | It's so hard to get interfaces right and it's helpful to have | many users. But I considered your point, of it helping to | enforce boundaries. ___________________________________________________________________ (page generated 2020-04-13 23:00 UTC)