[HN Gopher] Software effort estimation is mostly fake research
___________________________________________________________________
Software effort estimation is mostly fake research
Author : walterclifford
Score : 369 points
Date : 2021-01-18 19:02 UTC (3 hours ago)
(HTM) web link (shape-of-code.coding-guidelines.com)
(TXT) w3m dump (shape-of-code.coding-guidelines.com)
| mr_tristan wrote:
| I wish as much attention was paid to perform post mortems
| regularly then to only do estimation. You know, actually look at
| "hey, this is what we guessed and this is what actually
| happened".
|
| I've had to fight to actually hold post mortems, and every time
| I've done this, the manager ends up asking, "hey, can I share
| this?"
|
| So clearly, there's value, at least when we've done them.
|
| I'm amazed at how few places even perform a complete feedback
| loop. It's just, "when can you get this done?" and, "is it done
| yet?".
| mr_tristan wrote:
| To tie this a bit more into the actual article: It might be
| even more accurate to just ask people in a post mortem for some
| feedback, instead of trying to build some data set based on
| estimates and SLOC. Like an exit poll.
| yibg wrote:
| To me the difficulty of estimating software projects isn't about
| estimating the time / effort to build the thing. The biggest
| uncertainty is trying to estimate what it is we're trying to
| build in the first place. i.e. the details of the spec. Often
| times where I've seen estimates really off is when the scope
| changes, or more details are discovered etc.
|
| To borrow the usual building a house analogy, if during software
| estimation we're actually given how many rooms, how many floors,
| where are the doors and windows etc, then I think estimates will
| be a hell of a lot more accurate. Instead, what we typically get
| is, build a house that can be used by the family of 4, and we'll
| discover exactly what the family wants as we build.
| gpmcadam wrote:
| Cache/mirror
|
| http://webcache.googleusercontent.com/search?q=cache:http://...
| jakubp wrote:
| If someone can conclusively teach inexperienced programmers good
| approach to estimates (methodology) + help embed this into sales
| process of a software house-type company, I know some folks who'd
| love to have this :)
|
| My own experience has been this: people make estimates, client
| has expectations based on some variant of those, and something
| later happens but so much change is introduced during the actual
| software development, that there seems to be no sane way to
| compare what happened with original estimates. (new features,
| changed features, lots of new information, market shifts/product
| vision shifts, quality assumptions change, etc. etc.)
|
| But at that point nobody cares! People go on with further
| projects, and no data is ever collected.
|
| Nobody learns.
|
| When f*ckups happen, i.e. a gross over- or under-estimate, this
| is often not shared with the broader organization (ashamed/afraid
| sales people/PMs/devs say nothing/hide it/sugarcoat it). Client
| walks away.
|
| Sometimes project is severly underestimated but usually not just
| because of the software part. Again, no decoupling and estimation
| of contributing factors is done.
|
| It's insane.
| crispyambulance wrote:
| > ...nobody cares! > Nobody learns. > It's insane.
|
| Estimation is really hard, especially if you're dealing with
| new challenges, new people, and new expectations. Throwing
| moving targets (unplanned changes) into the mix, as you
| noticed, makes estimation even more difficult.
|
| The thing is, it very rarely is the case that a wrong estimate
| will sink a project. Things get delivered late all the time. No
| one dies. Most stakeholders aren't like two year olds throwing
| a tantrum because an unforeseen problem delays a deliverable by
| a few days (admittedly some are close).
|
| The actual definition of failure also needs to be considered.
| Is it considered a failure because a project manager ended a
| colored gantt bar at a particular date and the actual time was
| longer? Or is it a failure because the customer cancelled the
| contract because a deliverable was not available on the agreed-
| upon date? Or is it something far more nebulous, like a slow
| death march towards a misguided goal that burns everyone out
| and compels them to sip artisanal coffee and daydream about
| leaving instead of grinding onwards and hitting targets?
|
| Ultimately, people "don't care" because accurate estimates are
| not the most critical problem on their mind. Most of the time
| an accurate answer to "When will it [the deliverable] be done?"
| has about as much gravitas as guessing the number of jellybeans
| in a jar at a kid's birthday party.
|
| Doing a retrospective and analyzing what went wrong when a
| project delivery slipped expectations is certainly reasonable
| but it so easily slips into a blame storm and/or people making
| commitments that aren't realistic. Getting straight answers
| from folks for this kind of analysis requires a level of
| psychological safety that just isn't present in most
| workplaces.
| kylecordes wrote:
| Failure is a tricky thing.
|
| There is a story that Microsoft Word was a staggering failure
| in terms of actual time and cost to implement versus its
| estimates.
|
| It then went on to be among the most lucrative ($ billions)
| pieces of software ever created.
|
| Was Microsoft Word a failure?
| corty wrote:
| Internal estimates are a political tool, not a technical,
| financial or scientific one: First, decide if you want to
| do it or not. Then do the estimate, high if you don't want
| to do it, low if you do want to do it.
|
| Any kind of reality doesn't factor into it.
|
| In the Word example, somebody really thought this project
| was worthwhile for some reason, so they gave a very low
| estimate to get things started. As soon as money has been
| spent on a project, it isn't likely to be cancelled as
| easily, so you are safe to continue even if you exceed your
| original estimate. In the worst case, you need a few
| excuses.
|
| External estimates, i.e. estimates you use for writing up a
| contract with an external customer, are different. Those
| actually do matter, depending on the wording of the
| contract. But a smart contractor will try to get the
| contract into a shape such that the above for the internal
| estimate does apply.
| bcrosby95 wrote:
| I think it's because inexperienced programmers don't think
| about the "soft" costs of programming.
|
| Requirements gathering, general communication, testing,
| production rollout, coordination, research, final approval - it
| all adds up. For a 2 week project you might only actually spend
| a day of ass-in-seat coding. And that ass-in-seat coding is
| what they tend to estimate.
| ghaff wrote:
| I do a fair bit of writing at work of various different types
| of things.
|
| For stuff that I'm doing on my own? I can estimate pretty
| well. I know more or less how long the writing will take
| based on length. I'll allocate more time for research if I
| need to do more than a base amount. And then I add some slack
| in case I'm struggling with something, priority interrupts
| happen, or I'm just not into it on some given day.
| (Underpromise. Overdeliver.) It only gets harder with bigger
| projects. It helps that I'm quite fast compared to most
| people so even with significantly underpromising people are
| usually very happy with the estimate.
|
| But where it's really hard is work that involves getting buy-
| in and input from a lot of different people. Those always
| take forever. (And at this point I pretty much know they'll
| take forever, a lot of which is just latency rather than
| actual active work.)
| munk-a wrote:
| I think this is a big factor in estimation inaccuracy for
| really new programmers - but it is pretty well fixed by
| project managers following estimation rules of thumb - like
| double it and add five days... The bigger issue that I am
| less certain about being teachable is the experience side of
| things. I've worked in industry for a while and I know how
| long different sorts of patterns will take to write and what
| sorts of complications different sorts of add-ons will add to
| those numbers. Some projects are about where my naive brain
| fifteen years ago would put them and other ones are wildly
| different due to complexities that aren't visible in a
| surface level examination. Learning to sniff out those
| complexities seems to just take experience (not with every
| individual scenario - but it's a pattern analysis thing).
| jacques_chester wrote:
| > _I think it 's because inexperienced programmers don't
| think about the "soft" costs of programming._
|
| That's what the research has previously found[0] decades ago
| -- engineers will generally correctly estimate the part they
| are best at, but will either neglect or under-estimate other
| work. The old-school ways of dealing with this are canned
| Work Breakdown Structures and checklists.
|
| This is not unique to software.
|
| [0] van Genuchten, "Why is Software Late? An Empirical Study
| of Reasons For Delay in Software Development" https://web.arc
| hive.org/web/20160914040151/https://lab.cs.ru...
| llampx wrote:
| Programmers can also try to come up with reasonable
| estimates, only for management (product, middle or upper - or
| often all of them) to come back and assert that the estimates
| are off base and to "reconsider." Not much to do at that
| point but to give a lower estimate and hope it works out. If
| you make that your hill to die on as a programmer you will
| find yourself out of a job with only your integrity to keep
| you warm at night.
| durandal1 wrote:
| Apple executes massive feature releases on a yearly waterfall-
| like schedule across hundreds of teams. It requires honesty,
| transparency, strong leadership (ruthless prioritization),
| strong cross-team goodwill, strong cross-team collaboration as
| well as a solid body of experienced top-tier engineers. Other
| than that there's no magic. While there are teams within apple
| that does scrum/agile, none of the core OS/frameworks teams
| does it AFAIK, and I do think it's incompatible with shipping
| on a hardware-aligned schedule.
| aqme28 wrote:
| That doesn't imply anything about estimates.
|
| Do they tell you at the beginning of that cycle what features
| to expect? Or do they constantly work on features, and then
| at presentation time do a freeze?
| robertlagrant wrote:
| How do you know it's waterfall-like? If you ship working
| software every 2 weeks and every 26th one you release,
| hooray, you have yearly software that's super likely to work
| well.
| nightski wrote:
| When you have hundreds of teams you can cut a bunch of
| features and it still looks like you are massively productive
| each release :) Not everyone has that luxury unfortunately.
| Some of us are a single team of a handful of developers.
| monadic3 wrote:
| I'm sure Apple devotes time to research as well, even if it
| is under a different budget and/or department.
| trimbo wrote:
| Have you noticed their declining OS release quality in recent
| years? I have, and I now wait 6 months to upgrade.
|
| The triangle is real[1]. Something's gotta give when a
| deadline can't move, and as we know from Brooks, software
| development speed doesn't increase linearly with more people.
| So it's primarily quality or scope that must suffer to hit a
| software deadline.
|
| [1] -
| https://en.wikipedia.org/wiki/Project_management_triangle
| alisonkisk wrote:
| People have been saying that since OS X first came out, and
| OS 9 was so buggy it had a famous cute bomb icon for when
| it crashed.
| virgilp wrote:
| Adobe executes (executed) massive releases on a well-defined
| cadence (1.5 yr per "creative suite"). Sure, I won't say they
| don't have collaboration, or top-tier engineers etc. But in
| the end, the success of this well defined cadence boiled down
| to the simple strategy of "we release what is ready".
| Everybody knew the deadlines well in advance, and everybody
| strives to 'make it' - but, (not so) occasionally, some teams
| didn't and you could see entire features get ruthlessly cut
| out. Simply no other way of synchronizing across the entire
| company... you cut scope to make it in time, and if you can't
| make it, that's it, your feature gets cut. With enough teams
| working, you still get features to showcase at each release.
| vinay_ys wrote:
| That's pretty much how Linux kernel releases work. Just
| having a regularly scheduled release train that always runs
| on time - that you can catch (or not) is great.
| AuthorizedCust wrote:
| How do you know they are thoroughly waterfall? How do you
| know they don't use agile, time-boxing, etc.? Apple is so
| secretive, it feels like we know of little other than what
| they are confident about releasing. The things that weren't
| done on a schedule compatible with a hardware release, how
| often are they simply postponed? And for those postponements
| that they do admit, how many are waved away with a creative
| explanation? I think you need a good deal more information to
| back up your point about Apple's work management.
| calpaterson wrote:
| > Apple executes massive feature releases on a yearly
| waterfall-like schedule across hundreds of teams. It requires
| [...]
|
| And no doubt a great deal of crunch time, their fair share of
| delays, some major messes and the occasional outright
| cancellation. And if I'm not wrong their roadmap is not
| public information.
|
| I personally find it hard to believe that a large company can
| exhibit all those adjectives you list. Everyone finds this
| hard.
| scotty79 wrote:
| Estimates don't give you the time when something will be done.
| You can just be almost sure that the thing will not be ready
| before this time. And that's still something.
| wefarrell wrote:
| More important than estimating is knowing how to sequence the
| work. I often see teams try to build the entire DB schema
| first, or built the interface out one wire at a time. The agile
| approach of building small, vertical parts of each feature
| tends to work much better in my experience.
|
| For example say you have a CRUD application with a page for
| editing and searching complex objects. Rather than building the
| entire search page with all of its filters, and then building
| the edit page with all of its fields, you build a simplified
| version of the search page with only a few filters and a
| simplified version of the edit page with fields that correspond
| to those filters. You also make sure the functionality
| integrates with the backend to form a cohesive, vertical
| feature that can go through QA, and possibly the client. Once
| that is working you expand the list of fields and filters.
|
| This approach tends to surface design issues much faster, and
| is more adaptable to change requests from the client.
| catwind7 wrote:
| agree with the adaptability of change requests from clients,
| but i would say only insofar as the design issues you're
| mentioning are actually addressed as quickly as they are
| surfaced. Otherwise this type of building eventually grinds
| to a halt.
|
| another commenter sort of hinted at this, but this type of
| process has a downside of prioritizing short term design
| decisions that lead to working code. It takes time and
| sometimes a lot of thought to think about the system and the
| problems deeply and ask yourself if you've created the right
| primitives.
| ConnorLeet wrote:
| Although when working like this there can be a "Good enough"
| inertia
| 1337shadow wrote:
| Make a complete list of things to develop in a spreadsheet,
| "complete" means _every_ _single_ _item_ the customer wants to
| see in the product, not only every single button but also every
| single label, that should be definable by reading the project
| specs or mockups. I think "forgetting things" is the first big
| mistake leading to under estimations.
|
| Add an estimate that you multiply by three in front of each,
| ie. if the dev thinks 1 hour then they should put 3. From my
| experiments: no multiplying factor turned to nightly and
| weekend work, multiplying by 2 turned to long (10 hours) work
| days, multiplying by 3 turned into comfortable work hours and
| quality work.
|
| I've had success delivering quality code and respecting
| deadlines since 2012 or so with this system, but YMMV
|
| Only problem: the customer might not like what he gets, even if
| it works exactly as planned.
| cmroanirgo wrote:
| I forgot to add this to my other comment. Yes, a multiplier
| works wonders. (I typically added 30% to my own estimates,
| which covers all the little incidentals that trip you up, and
| coffee breaks). I've not been forced to work longer than an
| 8hr day since the 90's.
| 1337shadow wrote:
| I thought your comment stating that time should be added
| for both research & design was actually a reasonable
| explanation for why I need the *3 multiplier!
| ghaff wrote:
| Back when I was a product manager, there was an engineering
| manager who would drive me crazy. He had this idea of a 90%
| schedule, by which he meant a 90% chance of hitting if
| everything went as planned. Everything never went as planned.
| Jtsummers wrote:
| He probably read Goldratt's follow-on to _The Goal_ ,
| _Critical Chain_. Critical Chain Project Management is the
| specific term. Goldratt tried to sell software to companies
| that would help them make schedules like that, but a large
| component (often ignored) was collecting historic data, and
| running models. Most companies might do one of those, but
| rarely both, and of course models with only speculative
| numbers are absolute garbage. In my experience, places that
| tried to apply CCPM never actually grokked it, and it was
| some manager 's obsession because they were buzzword crazed
| (like many big-A Agile offices).
| ghaff wrote:
| I think this was a little before that (mid-90s) but maybe
| something related.
| jakubp wrote:
| I've done backlogs before; both very high-level and very
| detailed; with and without mockups (even final graphic
| designs) and detailed description in the form of multi-page
| spec or user stories (depending on the fashion of the decade
| I was doing it in :).
|
| I don't think having the detailed requirements made estimates
| much more accurate. However it made creating those estimates
| take an order of magnitude longer - developers needed much
| much more time to absorb all the details.
|
| At that point I decided I only ever want to work with senior
| developers who have good imagination and can give reasonable
| triplets (solution, # of devs to do it, approximate time) for
| requested pair (idea, quality level).
| composer wrote:
| ReRe's Law of _Re_ petition and _Re_ dundancy [4] somehow
| does not apply to you: A programmer can
| accurately estimate the schedule for only the repeated and
| the redundant. Yet, A programmer's job is to automate
| the repeated and the redundant. Thus, A programmer
| delivering to an estimated or predictable schedule is...
| Not doing their job (or is redundant).
|
| [4] https://news.ycombinator.com/item?id=25797519
| 1337shadow wrote:
| I think the explanation was made in another comment where
| they say "don't forget to estimate time for research &
| design", which explains why the *3 multiplier works for me.
| And yes, I feel that most of the work is redundant, there's
| always like network services of some sort or web pages,
| databases, linux servers, a humain interface, other
| services depending on the project like a message queue or a
| caching server or both, background workers and scheduled
| code execution... things I had been doing for ~10 years
| back in 2012. Data structures, workflows, they change
| between gigs, and each gig has its own particularity, but
| 90% is the redundant from a gig to another:
| HTML/CSS/JS/Python/SQL/Networking/Storage/Linux stuff
| devdas wrote:
| Software development is the process of writing a complete
| specification, including every single button and label. If
| you have a formalised complete design, you could write a
| compiler to do the work for you.
| csours wrote:
| > "complete" means every single item the customer wants to
| see in the product,
|
| Ah, but the customer does not know everything they want, due
| to the fractal nature of reality. The closer you get to the
| end product, the more detail is resolved, and more work is
| added.
| jakubp wrote:
| Oh man, thank you for the fractal nature of reality. That
| is the perfect term to describe what I saw about ever-
| changing requirements. No matter how closely you define
| something, things will always keep changing on all levels
| when closer to actually "delivering" / launching. Adding
| time, costs, confusion, frustration. Fractal, yes, that's
| it :)
| KineticLensman wrote:
| > No matter how closely you define something, things will
| always keep changing on all levels when closer to
| actually "delivering" / launching
|
| Xeno's paradox [0] doesn't apply here. Some projects do
| actually finish.
|
| [0] https://en.wikipedia.org/wiki/Zeno%27s_paradoxes#Achi
| lles_an...
| cmroanirgo wrote:
| Although this sounds flippant, the design process is where
| unknowns should be uncovered. It is easier to change design
| before a line of code is written. Design is precisely what
| is needed in order for the dev team to break down the
| projects into tiny chunks where nothing is unknown.
|
| Your time estimates should always include time for research
| and design. The customer can sign off on what you're about
| to write at the end of that (whereas beforehand you give
| them a wild ballpark figure made by senior devs - not
| management!)
| ubercore wrote:
| Yeah but too many customers change their mind even after
| they've signed off. Then you're left with strictly
| enforcing a contract or making adaptations.
|
| You're right, design is very very necessary and should be
| given plenty of time. But it's not a magic bullet.
| 1337shadow wrote:
| In terms of contract, I have found this works best
| (recommended by my lawyer (a former ASF lawyer)): 30%
| upfront, 50% at first delivery, then they have 2 months
| to find bugs, final delivery is when the bugs are fixed
| that's when you take the last 20%. This way, you're
| certain to always get 80%.
|
| If they want the liberty to "change their mind on the
| fly" then they should buy man-days which is a completely
| other thing. Or, they can buy supplementary mandays for
| whatever they want at the daily rate without changing the
| contract, or, they can just wait to finish this contract
| and start another one.
| chasd00 wrote:
| it's pretty a well known saying that the last 10% of any
| project takes 90% of the time.
| KineticLensman wrote:
| Another way of looking at this is that (unless there is a
| complete disaster) as you get closer to the end the
| uncertainty in the estimate decreases. When you get to the
| end, you know precisely how long it took.
| sound1 wrote:
| As long as humans are involved in defining requirements,
| inaccuracy of estimate will be proportional to the cusomters'
| incompetence and level of bad politics of the organisation as a
| whole ;-)
| cmroanirgo wrote:
| The problem is knowing how fast they learn. For a n00b this is
| basically impossible, and takes cost monitoring to notice their
| rate of learning. For a person with the basics of the language
| that they'll be using, and a tiny bit of experience in the
| libraries they'll be using, things get easier.
|
| For everyone (n00b to senior) I would always recommend
| designing & breaking things down into no smaller 1/2 day
| blocks, where nothing went longer than 3 days (for
| inexperienced), but longer for more experienced. Senior devs
| should be able to be able to be trusted (to be accurate) for
| week-long blocks.
|
| Use a Gannt chart (ms project or similar) and monitor the
| inexperienced, gauging their real-life times with their
| estimates, adjusting accordingly. Give semi regular feedback at
| the accuracy so that they (& you, the manager) are working
| with.
|
| In one of my companies, the most inaccurate we were for _any_
| project was that we were late 2 days over 2 years. Build on
| trust, build on meetings and other time sinks. Include testing,
| sign offs, etc.
|
| So, absolute n00bs need close mentoring so that the (senior)
| mentor can begin to gauge the growth rate... Which will be
| steep... But different for everyone.
| Zigurd wrote:
| The principles behind the Manifesto for Agile Software
| Development mention only retrospectives, not estimates. You
| could say they value learning over guessing.
|
| When people ask me how can they learn, I tell them to find old
| project schedules and other project documents. Looking at past
| projects makes the learning more objective, and it is not
| happening under time pressure.
| taeric wrote:
| My problem with this is all estimation is hard. Period.
| Quantitative discussions of something that has been done? Fairly
| easy, if still not accurate.
|
| Discussing something that hasn't been rehearsed before? You can
| really only discuss how long you are willing to work on it. Not
| how long it will take to get done.
|
| Fun examples. How long would it take you to clean out your
| fridge? How long would it take you to learn to play a song on
| piano?
| edelans wrote:
| I like this analogy. But I was picturing myself asking my sales
| manager "How long would it take you to learn to play a song on
| piano?", and I'm pretty sure he would reply "but you touched a
| piano before, you are supposed to be a professional piano
| player ! A professional piano player surely knows how long it
| would take him to learn how to play a song". So I guess he
| would miss the point totally :/
| taeric wrote:
| Ironically missing that some folks will never be able to
| learn some songs.
|
| And, learning to play an existing song doesn't necessarily
| translate to being able to write one.
| deandree_ wrote:
| Exactly, you can't really predict what can't be predicted
| Londane wrote:
| The site is down for me, so :
| https://web.archive.org/web/20210118200327/http://shape-of-c...
| distalx wrote:
| Broken Link....
| vincentmarle wrote:
| I've been using the Rumsfeld Matrix to classify software
| estimations into 4 categories:
|
| Known knowns: familiar tasks that can be reliably estimated from
| past experience. You can improve these estimates by better
| knowledge sharing (internal wiki, adding comments).
|
| Unknown knowns: tasks that can be increasingly better estimated
| the more time you spend on estimating (for example creating Draft
| PRs with pseudo code).
|
| Known unknowns: tasks that haven't been done before, so
| estimations need a decent buffer (30-50%) to account for research
| and potential blockers. You can improve these estimates by
| benchmarking your previous efforts combined with the team's skill
| levels (aka sprint story point velocity).
|
| Unknown unknowns: the unforeseen blockers that come out of
| nowhere, can't really be estimated, and can really disrupt a
| project's schedule. Improving these estimates is really hard. The
| key to improving these is by improving team/company communication
| and building agile feedback loops so you can identify these
| issues early and reprioritize as needed.
| djoldman wrote:
| >Those estimation datasets that were flogged to death in the
| 1990s using non-machine learning techniques, e.g., regression.
|
| >...non-machine learning techniques, e.g., regression.
|
| Is this where we are now? The connotation of "machine learning"
| doesn't include regression? Wow.
| disgruntledphd2 wrote:
| They (almost certainly) mean linear regression, which many
| people appear to regard as statistics and boring, as opposed to
| single layer neural networks, which are cool.
|
| To be fair, this is a pretty common perspective in computer
| science academia and adjacent fields (and probably occurs in a
| lot of fields where people don't focus on statistical
| learning..ahem I meant ML).
| BlargMcLarg wrote:
| I seriously don't understand the obsession with estimates in the
| paradigm it is pushed in. Want to be steady at a reliable pace,
| work for a few months and figure out the average and deviations.
| Need a priority on tasks, use a priority queue using whatever
| method you assign it by, often a combination of deadline, value
| added, complexity and more.
|
| Yes, I get on a high level, one needs to be able to say "yes we
| can make this before [date]", when using big deadlines. How often
| do people _really_ have to finish something before a critical
| deadline or decide to drop it right then and there? I 'd wager
| most software devs do not, let alone biweekly deadlines. Isn't
| agile methodology supposed to help us fight against artificially
| tight deadlines (customer collaboration)? Isn't the SaaS model
| combined with "deploy any time" designed to be profitable in
| accordance to features implemented? Then, why push estimates so
| strongly in whatever flavor of the month Scrum version BigCorp
| wishes to use today? What do they even add at that point? If they
| are really so important, why do we always feel the need to
| introduce human error when we can extrapolate former experiences
| with computer models?
|
| Maybe it is just me being cynical. The entire need to hold so
| fiercely onto an estimate reeks of micromanagement and desire to
| push responsibility entire onto the lower ranks.
| captain_price7 wrote:
| I have been involved in Software engineering research a bit, even
| have a first-author short paper. I was struck by just how much
| pointless, low-effort papers there are in this domain. People
| have been researching about bug prediction for over two decades
| now, and judging by the paper quantity, this isn't a niche area.
| Yet how many organizations do actually employ those systems in
| real-world? Can't comment on industry, but I haven't found a
| single open-source program that does that [1].
|
| Now I know "most papers are pointless" is a common complaint in
| science, specially in my area of focus- machine learning. But I
| can't shake the feeling that the situation is particularly worse
| in software engineering related academic research.
|
| [1] I saw Mozilla attempt it, but not sure if it's currently in
| use.
| randomsearch wrote:
| Couldn't agree more. Software engineering is a very practical,
| fast-moving subject, and that makes most academic work even
| more irrelevant - software engineering academics often don't
| have the first clue about what real software engineering is
| about, because they don't have recent industrial experience.
| mcguire wrote:
| Would software effort estimation work better if software
| developers were held to fixed-price bids?
| ThomPete wrote:
| In his book "Code Complete", Steven McConnel speaks about
| metaphors. He reasons that metaphors are necessary to be a good
| developer as it helps visualize the act of coding. The metaphor
| he prefers is the "construction" metaphor. This metaphor he
| argues best explain the act (some would say art) of programming
| and gives developers a language to speak in that brings clarity
| to the development process.
|
| When in construction you prepare the building site, lay a
| foundation, frame the house, put siding and a roof on it, and
| plumb and wire it. This is equivalent to programming. In other
| words through the lens of the construction metaphor, the
| developer is someone who ultimately build someones house by
| working together with different disciplines (Architects,
| designers, contractors etc)
|
| The problem with that metaphor IMO is that it's not actually what
| software development is.
|
| The proper metaphor for software development is more _"
| engineering and development of construction site equipment and
| material"_ i.e. a developer is not building buildings they are
| building the things necessary for building the building.
|
| And so the developer will often find themselves in a situation
| where the material doesn't exist and they have to invent it it or
| the material exist but we dont know how it's going to work with
| some other material needed or the equipment used for that
| material isn't made yet or doesn't work with that material even
| though it normally does.
|
| I.e. a developer is inventing, engineering and building all at
| the same time and that is what makes it impossible to estimate
| development regardless of process or metaphor.
| randomsearch wrote:
| The reality is that software development is nothing like
| engineering or construction, it's totally different. You don't
| build a quick house, let people live in it and start building
| the walls whilst they live there.
|
| Humans like to think via metaphor because it's a least-effort
| mode of thought but sometimes there just isn't one and it's
| just tough luck and start thinking from first principles
| instead.
| ThomPete wrote:
| My point was more that the software developer have to not
| just build with existing material and equipment but have to
| invent and build those on the fly.
| tootie wrote:
| Scrum dogma is that estimates are for complexity, not effort or
| timing. The points you track in JIRA are meant to reflect how
| much of the current backlog is complete and how much is
| remaining. That can be extrapolated into timing but can't be done
| up front.
| wccrawford wrote:
| My understanding is that for a team and project that's been
| going for a while, scrum points can be roughly turned into time
| estimates by looking at the history of stories that were rated
| for that many points and averaging.
|
| But that obviously won't work from the start, and it won't be
| accurate... Just better than nothing and (hopefully) better
| than what a programmer will estimate in their head.
|
| And IME, it's a lot less stressful on the programmer to
| estimate points rather than time.
| dragonwriter wrote:
| > Scrum dogma is that estimates are for complexity, not effort
| or timing
|
| The theory of story points (which originate outside of Scrum
| and are not part of Scrum proper) is that task-specific time
| estimates in creative intellectual work are extraordinarily
| unreliable and expending more effort on them doesn't improve
| them, but broad-banded complexity class evaluation mixed with
| empirical observation of the teams velocity produces time
| estimates that are (while still _extremely fuzzy_ ) both better
| and much lower effort, once you have the basic tracking in
| place, than task-specific direct estimates.
|
| The "dogma" you report seems like something that might be a
| derivative of that that has lost track of rationale and
| purpose, reducing it to a cargo cult practice.
| cuillevel3 wrote:
| Right, the estimates are not even in the guide:
| https://www.scrumguides.org/scrum-guide.html#sprint-planning
|
| Planning poker and such are useful as they encourage the team
| to discuss differences and help identifying stories which are
| too large.
| jakubp wrote:
| If it can't be done upfront it's largely useless for business
| folks ("so you have an estimate but can't translate it into
| time? so why do it? go away" :).
|
| If it assumes remaining work is equivalent in time to done work
| (notion of velocity), it's (in my view) very optimistic. People
| learn the nature of the beast as time progresses. Collective
| learning happens slowly. A lot of important work is usually
| only discovered and planned in the second half of the actual
| (not original) timeline. A lot of important stakeholders are
| naturally only introduced into projects close to its
| finalization, which creates a flurry of new activity and
| discoveries. This makes projects late.
|
| But where that real halftime is - most people I worked with
| rarely know. I was no better, I also rarely knew.
|
| The only way to manage this I found was the basic rule: build,
| demo, decide what to do next. Don't get attached to original
| backlog and grand plans for the future. It won't work that way.
| Just build and make it work, launch ASAP, get decisions on a
| ~weekly basis done. Ignore long term planning. It's not
| helpful.
|
| Sadly, few business folks want that of course :)
| tootie wrote:
| For one, it doesn't assume pace is consistent. Tracking
| velocity is part of the game too and it can vary with
| capacity, talent and maturity.
|
| Secondly, telling business that we can't estimate accurately
| is more useful than saying we can estimate accurately and
| being wrong (which is guaranteed). It's a tough sell but it's
| necessary.
| jakubp wrote:
| Agree on the second point. I tried to convince some
| managers and sales guys to stop giving estimates at all and
| build trust otherwise, didn't work :)
|
| As for consistent, my point was if you have 100 SP in
| backlog, delivered 50, then I contend you are not in the
| mid-point of the project. This is a dangerous assumption.
| You don't know where you are in my experience.
| tootie wrote:
| It is if you're following agile principles. By the time
| you've completed a few stories you should be delivering
| every feature at production quality with all the
| automation pipeline and deployments ready. The question
| from that point is always did we build enough features
| for a launch.
| jonpurdy wrote:
| I agree that building and demoing something small is key. The
| larger and more complex a project will be, the harder it is
| to estimate.
|
| I've been trying to distill this into a framework that is
| accessible to both development and business folks [0]. I've
| used it a few times and it has resulted in a few projects
| getting cancelled due to a massive difference in initially
| expected cost and post-analysis expected cost.
|
| 0 - It's basically just Scrum oriented towards providing
| upfront estimates. https://jonpurdy.com/pm-framework/how-to-
| quickly-plan-and-es...
| mbesto wrote:
| I'm gonna play the devil's advocate here.
|
| > Sadly, few business folks want that of course :)
|
| Business folks want this usually because it's _their_ money
| you 're playing with.
|
| > Ignore long term planning. It's not helpful.
|
| So, no don't tell people to ignore long term planning
| otherwise people who give you money are going to stop giving
| you money to "build, demo and decide". Unless you're on your
| own dime (e.g. you pay yourself or are self funded), then I
| wouldn't take this advice.
| SKILNER wrote:
| Not only do we not know how to predict how long a software
| project will take, we don't even know how to predict what the end
| product will look like.
|
| So who are we kidding?
|
| Another way to look at it: take a small one-person project and
| assign it to three different developers. You may get wildly
| different results. How could you have predicted those differences
| in advance? Let alone apply that type of prediction across a
| large team.
|
| About a dozen years ago I gave a presentation to the Silicon
| Valley Software Process Improvement Network (does it still
| exist?) My presentation: "Unsolved Problems of Software
| Maintenance." You think predicting greenfield development is
| difficult? Try predicting maintenance work, where figuring out
| what to do can be more than half the work.
| MattGaiser wrote:
| Estimation seems like a paperwork generation exercise anyway, so
| why not take it all the way to research?
| jacques_chester wrote:
| I think a better title would have been "most effort estimation
| literature relies on small datasets". The use of "fake" can be
| read as insinuating deliberate malpractice.
| meesles wrote:
| It's unfortunate that this HN thread has been reduced to the
| generic discussion about software estimates when the article is
| specifically talking about research done on the topic of software
| estimates.
|
| According to the article, proper research remains a struggle due
| to outdated datasets from before modern agile methodologies, and
| that the modern datasets from industry are hard if not impossible
| to gather.
|
| If industry is truly interested in improving software development
| and estimation, their data should be anonymized and made
| available to researchers for analysis.
| robertlagrant wrote:
| What data would be useful here? There are so many confounding
| factors.
| froh wrote:
| > How large are these datasets that have attracted so many
| research papers?
|
| > The NASA dataset contains 93 rows (that is not a typo, there
| is no power-of-ten missing), COCOMO 63 rows, Desharnais 81
| rows, and ISBSG is licensed by the International Software
| Benchmarking Standards Group (academics can apply for a limited
| time use for research purposes, i.e., not pay the $3,000 annual
| subscription). The China dataset contains 499 rows, and is
| sometimes used (there is no mention of a supercomputer being
| required for this amount of data ;-).
|
| > Why are researchers involved in software effort estimation
| feeding tiny datasets from the 1980s-1990s into machine
| learning algorithms?
|
| > Grant money. Research projects are more likely to be funded
| if they use a trendy technique, and for the last decade machine
| learning has been the trendiest technique in software
| engineering research. What data is available to learn from?
| Those estimation datasets that were flogged to death in the
| 1990s using non-machine learning techniques, e.g., regression.
|
| Is this telling me that most theories about "sw estimation best
| practices" are cargo cults o-O ?
| randomsearch wrote:
| I'd say the problem is more from the academic side. If good
| data isn't available, then academics should not be publishing
| papers on toy data. It's meaningless. The goal is not to
| publish papers but to advance science.
| [deleted]
| dboreham wrote:
| Redirect loop on TFA?
|
| Update: working now.
| snidane wrote:
| Software development which is a repeatable and already defined
| process is totally possible to predict and estimate. Most tasks
| of repeatable processes follow normal distribution and is
| predictable. Deviations from expected mean will be due to
| predictable factors of the environment such as failed disk or
| sleepy programmer. You can apply arbitrary six sigma methodology
| to measure such process with accuracy.
|
| The problem in software though is that such a repeatable process
| would be immediately automated away by writing a function,
| library, framework or any such tools that programmers use on a
| daily basis without much thinking. Unlike in building
| construction, to which programming discipline is often wrongly
| likened to, where construction companies simply cannot "write a
| function" to deploy cookie cutter houses or bridges one after
| another.
|
| Therefore software engineering is never a repeatable process,
| unless crappy tools are used, which don't allow for sufficient
| abstraction of repeatable parts.
|
| Tasks in software disciplines therefore don't follow a normal
| distribution. They follow exponential distribution most often.
| Most issues go unnoticed. Majority are just so tiny and ofthen
| considered business as usual. Every time you get stuck and have
| to look up a solution in docs or stackoverflow technically is an
| issue, but never gets reported in an issue tracker for its
| triviality. There are however issues which are orders of
| magnitude larger than what management expects when they
| occassionally sampling issue trackers. Some issues lead to
| critical design flaws which could need a full blown rewrite for
| example, or ever lasting and expensive hackery in case the
| executive decision is to work around the critical design flaw.
| These issues can take years to accomplish or take massive amount
| of pain endurance.
|
| Trying to estimate a process with such exponential distribution
| and making sense of averages or other statistics of such
| distribution is borderline insanity.
|
| Why not just go to physics department and ask when the next
| Theory of Relativity will be invented and how much budget and
| story points those guys need.
| k__ wrote:
| _" a repeatable process would be immediately automated away by
| writing a function, library, framework or any such tools"_
|
| This is the crucial point here.
|
| Source code is a (almost) self-assembling blueprint.
|
| The actual product that will be build is the software and
| software is a configuration of matter, in this case of a
| computer.
|
| The source code/blueprint for a house is not self-assembling.
| Compiling such a blueprint requires you to configure the
| building materials in a way that they become a house.
|
| With better robotics, we will probably get there at some point
| in the future.
|
| And with software we will always be in a place where you either
| do new stuff the first time manually or with crappy tools the
| 100th time.
| lugu wrote:
| I bet it is possible to give precise estimates, the problem is
| the amount of time and effort to produce such estimation compares
| with the actual execution.
| Aldipower wrote:
| I am 20 years in development business now. This simple rule of
| thumb works for me and the team: (Your honest and concise
| estimation) * 3
|
| There are just to many unknowns you cannot foresee. Software
| development is complex.
| nikolaj wrote:
| same experience level, similar formula. Take your gut estimate,
| double it, double that new value, then add another "increment".
|
| e.g. say you think it will take a day, so 2 days, 4 days, add
| another day, likely estimate is 5 days.
|
| My pet theory is that when we estimate, we typically think of
| how long it will take to figure out a working solution to the
| problem, but forget about how long it takes to debug it, add
| tests, rework for changed requirements and unexpected nuances,
| and then roll it out and do any training, etc.
| elwell wrote:
| That's a recursive definition (estimate * 3) = ((estimate * 3)
| * 3) = (((estimate * 3) * 3) * 3)... But if you do that for a
| few years, "your honest and concise estimation" starts to grow
| because you've seen how it usually takes longer than expected,
| and your coefficient can approach 1.
| matwood wrote:
| I use a similar methodology.
|
| The part you didn't mention is that for most businesses it's
| better to be over than under in the estimation. I also explain
| this thought process to the various stake holders. We can
| certainly try to tighten up an estimate, but that runs a higher
| risk of being under which is usually a worse outcome (promised
| launch dates are missed, marketing is missed/happening,
| customers are told, etc...).
| kilroy123 wrote:
| I couldn't agree more. This is what I do.
| OneGuy123 wrote:
| This, the famous "prediction* Pi" rule seems to be correct also
| in my experience.
|
| I make the most optimistic prediction and then * PI.
| TehShrike wrote:
| I multiply by 4. Specifically, giving estimates on when
| features will be deployed/usable by end users.
| chasd00 wrote:
| yes, going from identification of the bug/idea to the
| solution in end users hands is a lot different than just the
| coding estimate. On teams with multiple levels of management
| both engineering side and customer side the actual code
| required to fix/implement the solution is such a small part
| of the overall effort you could, ironically, consider it
| immaterial.
| curiousllama wrote:
| I've been hearing this rule for years now, and I think the
| coefficient is increasing. It started at 1.5x, now it's at
| 3x... Is this just me?
| impy wrote:
| I'd say this is Parkinson's Law at play
| Aldipower wrote:
| Maybe the coefficient is related to your age. You got older,
| so there is more wisdom around. ;)
| kthejoker2 wrote:
| I'm such an optimist! I multiply by 2.2.
| encoderer wrote:
| Don't do an estimate, build a prototype.
|
| Don't do an estimate, build a prototype.
|
| Don't do an estimate... seriously... build a prototype.
|
| And then throw it out.
|
| And THEN do an estimate. It will probably be pretty accurate.
|
| Not always easy to do this in a tech company but as an eng
| director I was able to get it done and it changed a seriously
| broken process based on multi-week scoping and estimation
| futility.
| tobyhinloopen wrote:
| I feel like I can get pretty good estimates on the following
| conditions:
|
| - the application is thoroughly specced. You might need WEEKS for
| this. - all variables are taken care of. Stack is known, and
| you've experience with all parts involved. If you don't, get
| familiar with the parts first. Again, might take weeks. - there
| is no implicit functionality. It is either explicit or not
| included. - there are clear boundaries and rules to prevent
| feature creep. - you cannot estimate an estimate
|
| Now the problem is, this estimate is really expensive, because
| it's actual work. It takes about 10-25% of the total project time
| to estimate the project.
| gregors wrote:
| It's worth watching Software Schedules by Joel Spolsky has some
| of the best insight regarding evidence based scheduling. It's
| extremely interesting -
| https://www.youtube.com/watch?v=EUS4ktQJOSY
| Kaze404 wrote:
| I had a conversation about estimates during a recent interview. I
| asked about how the company deals with those, and the interviewer
| said they don't do estimates because there's never been a time
| where something productive came out of one, and I think it makes
| sense.
|
| In my experience, when an estimate is spot on the world goes on
| as if nothing happened. When it's incorrect, all hell breaks
| loose and it's every man for himself. And at the end of the day,
| all of the blame ends up on the person who guessed wrong. I'm
| glad I don't have to deal with that anymore.
| snidane wrote:
| When a company uses software estimation, it suggests strong
| distrust towards the software people and is looking for
| justification of those huge costs. Most often it means some
| shitshow happened or is still going on in there.
| ChrisMarshallNY wrote:
| I find this amusing.
|
| Know how much a Honda Accord costs?
|
| About 25 Grand.
|
| Know how much a Mercedes S450 costs?
|
| About three times as much.
|
| They are both great cars, that will be rewarding to own.
|
| The Mercedes doesn't have 3 times more parts, but it probably
| took four times longer to make, and they paid the folks that make
| it, a lot more than the Honda. It's actually, probably better
| "bang for the buck," although it won't seem like it, on the
| surface.
|
| The reason is that all those _little_ things that go into high
| quality take _lots_ of time.
|
| I can write a pretty complete app that does some cool stuff in a
| day or two. I do it often, when I write test harnesses for my
| libraries and whatnot.
|
| However, if you want that app to be ship quality, and highly
| usable, you're looking at over a month.
|
| The thing is, many folks would consider my test harness lash-ups
| to be their "shipping" product, and will use that as a basis for
| estimation.
| darkerside wrote:
| Your base assumption seems to include that quality is valuable
| for its own sake. I don't totally disagree, but I'm wary of
| assigning value based on effort rather than output.
|
| Depending on why you are buying a car, the Accord is very
| likely much better bang for the buck than the S-class Mercedes.
| And depending on the situation, the prototype is often better
| value than the shippable product.
| ChrisMarshallNY wrote:
| You are exactly correct.
|
| That is why Honda is a bigger company than Mercedes. They do
| decent quality (Hondas cost more than Kias), but at scale.
|
| I worked for a "Mercedes-level" photographic equipment
| company for years. People were often quite surprised, when I
| told them the size of the company. It was like _Roadhouse_ "I
| thought you'd be bigger."
|
| Our prototypes were _incredibly_ expensive. A $2,000 (retail)
| body prototype would be insured at half a million bucks.
|
| I just find, in my experience, managers expect Mercedes-level
| quality, for Kia (not even Honda) prices, and (if you are
| lucky) mature Honda assembly line development speed.
|
| That can lead to real Jurassic-scale disasters. If you want
| that level of quality, you need to plan for it.
|
| I'm working on the app that I'm doing, because I was helping
| a friend of mine evaluate contractors for his dream.
|
| The promises they made were _absurd_. After a couple of
| these, I just said "Screw it. I'll do it for you."
|
| He'll be getting Mercedes quality, but he'll need to wait a
| bit longer for it. I am pretty good at doing damn good; damn
| fast.
|
| But he also won't be paying a dime for it, so I think he's
| good with that.
| mekoka wrote:
| > It's actually, probably better "bang for the buck," although
| it won't seem like it, on the surface.
|
| A car's primary function is to move its passengers from point A
| to point B, safely and timely. Both cars do this very well, but
| the Mercedes will probably be costlier over their respective
| lifetime (fuel, parts, service).
|
| The only way for it to be a better bang for buck would be to
| turn its secondary functions (comfort, prestige, signalling)
| into a primary tool. For instance, if a broker or salesperson
| finds that owning the car gives them the added confidence to
| project the image they want during negotiations and thus
| contributes to their overall success, then it becomes a worthy
| investment.
| RicoElectrico wrote:
| We should not conflate quality with value. [1]
|
| [1]
| https://moznainaczej.com.pl/Download/Teal%20Doctrine/A.Blikl...
| (13.10 Quality and value)
| winrid wrote:
| High quality != high feature count. You are conflating the two
| with your anology.
|
| You don't want to daily drive a 30 year old performance
| oriented Mercedes.
|
| An Accord however, most generations can hit 300k miles easily.
|
| The Mercedes is more expensive because they put more
| investement into _refinement_ and _luxury_. Make the doors feel
| right. Ensure the torque curve is flat and starts at a low RPM,
| usually using turbo chargers that will fail by 200k miles. Lots
| of carefully placed sound deadening.
|
| This isn't higher quality, it's extra features.
| maerF0x0 wrote:
| > but it probably took four times longer to make,
|
| I doubt it. Luxury goods don't proportionately more cost much
| more to produce than commodity goods.
|
| Instead they often derive their value from perceptions and
| exclusivity. That is people perceive the object to be more
| value for their own reasons, or they are made more expensive as
| a proof point of exclusivity / conspicuous consumption
|
| https://en.wikipedia.org/wiki/Conspicuous_consumption
| someguydave wrote:
| Mercades cars do often have more features, but I also am
| skeptical of claims that they are of higher quality.
| someguydave wrote:
| My experience is that most customers are unwilling to wait for
| and pay for quality, especially if someone in the market sells
| a competing product (which itself is of low quality)
| kfk wrote:
| At least in a business setting I think the whole concept of a
| project needs serious reconsideration. We end up more often than
| not trying to fit developing a digital product into an enormously
| stupid gantt chart to execute some poorly thought "business
| requirements". I prefer to talk products and not projects, I
| deliver the full thing including "growth" as adoption doesn't
| come "if you build it" even within a Company setting. If you are
| building a product you can also get closer to those with the real
| problem willing to fund you with real budgets. On top of
| everything else if you are making users happy they will not chase
| you on fake estimates but rather work with you to get stuff done.
| snidane wrote:
| Funny when you look at the actual definition of a project
| straight from PMP.
|
| "a temporary endeavor undertaken to create a unique project
| service or result."
|
| No "projects" in software fhat I know of are actually
| temporary. They only end when management fires the people
| behind them, it gets cut off or there is no adoption.
|
| We in software think of projects really as things which we
| create and which need maintenance in order to live. There is
| never some "end" to it.
|
| Because it doesn't even conform to its own definitions, we
| could therefore conclude that the whole PMP project management
| discipline, as applied to software, is a scam.
| wyldfire wrote:
| How do other disciplines estimate NRE? Do they have the same
| problems with missed predictions?
| ghaff wrote:
| Sure. A lawyer may have a pretty good idea of how a particular
| task or case is likely to play out but there's a lot that isn't
| under their control. I imagine something like drafting a will
| is relatively straightforward but I also imagine a criminal
| case has a huge number of variables.
| alkonaut wrote:
| You only quote the fixed bits and charge by the hour for the
| rest. If you need to give a quote for an unknown you multiply
| the quote by N to cover a worst case.
| solumos wrote:
| Well, lawyers take a retainer. Doctors are paid per-diagnostic-
| visit and per-procedure. Accountants bill hourly. Most
| professions have this - electricians, plumbers, locksmiths,
| etc.
|
| This business model is normalized for other professions, and it
| should be for software engineering too. As a profession, we
| should move more towards partnering with organizations to
| realize business value through software rather than being
| simple "feature factories" (see also: Developer Hegemony[0]).
|
| [0] https://daedtech.com/developer-hegemony-the-crazy-idea-
| that-...
| Netcob wrote:
| Integrations for time-tracking software with issue tracking
| software exists, but I've never actually seen them used in
| combination before. You'd think that issue trackers would always
| include a very user-friendly time tracker by default, and
| advertise integrations with popular time trackers just to get
| that "actual time spent" data. I've also never looked at a
| burndown chart with any particular interest after knowing you can
| close an 8h issue after .5h and a .5h issue after 8h of work and
| the issue tracker will just keep pretending its charts mean
| anything.
|
| Yet any company that does a lot of the same work could probably
| use that. You'd still need to do some extra work and tag your
| issues by things that you want to observe (and maybe predict),
| but then you could say things like "after we switched to that
| fancy new API client generator, our client development times went
| way down but debugging times went up a bit". And the system could
| display some time range while you're writing the issue. You could
| also look at the issues at the end of a sprint and then merge
| them with other items from the time tracker to see how meetings
| and other distractions played into the outcome.
| Groxx wrote:
| One important thing I've managed to get a couple managers to
| track for quarter/half planning purposes: _uncertainty_. Prior to
| that, we 'd of course hemmed and hawed and _verbalized_ the
| fuzziness of our estimates... but only the final decided-on not-
| super-pessimistic number was written down, and decisions were
| made based on those. _That_ was a major source of our estimating
| problems.
|
| Everyone wants estimates. For decent reasons. But until we've
| done a similar thing before, they're utter fabrication. We can
| take a semi-educated guess, but we _know_ they 're still
| guesses... so for things we don't have concrete answers for, we
| give small/medium/large/extreme markers for how uncertain we are.
|
| It's fine to take on a big unknown or two. You might even get
| them both done in a half. But they better be worth it (or you
| have to decide when to cut your losses), because completing those
| two _could_ consume _all_ of your resources... and if that
| happens and you didn 't commit everyone to it up-front, you won't
| get it done this half. Making that tradeoff more explicit managed
| to get us signed up for fewer low-impact-but-highly-uncertain
| projects that would inevitably balloon out of control but never
| be cut.
| 458aperta wrote:
| I disagree with the premise of this article. Software development
| is largely a waterfall process even if you use TDD (which was
| using a machine gun to chop down a tree) and the effort
| estimation is both predictable and consistent. At least with the
| model that we use in-house developed over the past 10 years that
| has resulted in several exits.
|
| Don't rely on HN submissions to shape your view of the world.
| Pedantry, pendants waste your time. Only real world results and
| data matter. The people who can't do simply teach or publish
| papers so all you are left are people who grab a leg or an arm
| and think its an elephant in complete darkness. You simply will
| never ever find us publishing these information nor do others
| that have figured it out.
|
| Transparency when it comes to winning trust is a must.
| Disclosures in the name of generosity or some collective good is
| lust; You might feel good but it ultimately results in increased
| competition and higher operating cost until your edge disappears.
| We are not in the business of helping our competitors, we use our
| unique edge to push them out of the market when we can, we use
| capital to buy them out when we can't.
| csours wrote:
| "Estimates" are for things you've done before - like you can
| estimate building a house, because people have built houses
| before. The more like an existing house, the better you can
| estimate it.
|
| Software is invention and construction. The construction part is
| pretty easy to estimate. The invention part is ... very very
| hard. I'd like to say it's impossible. I'd like to see the
| software industry use a different word than estimate.
| chaz72 wrote:
| Yes! I try to always phrase it as "the part I can see from here
| will take at least X time".
| Netcob wrote:
| Very true, I think those estimates are actually two ideas/types
| crammed into one value.
|
| 1. The construction part, as you said, can be estimated.
|
| 2. I'd just call the other thing "allocated time" instead of
| "estimated time".
|
| Any time someone asks me how long it will take me to fix a bug
| that I haven't really looked at yet, or to plan some new
| feature or something like that, and they badly need a number, I
| ask them how much time I should allocate to that. I can't
| promise to have something like that done by that time, but it
| gives us both an idea about how to treat that problem.
|
| For example, we could allocate two hours to fix a bug, with the
| understanding that if that turns out to not be enough then
| we'll need to talk about workarounds. Or we can allocate two
| days to plan a new feature, and the best solution we can think
| of in that time shall be the one we use.
| vinay_ys wrote:
| This is brilliantly put. That's pretty much what engineering
| is - constraints and tradeoffs - giving software project
| planning/execution a healthy vocabulary to talk about makes
| it so much better. Much of the crappy feeling that developers
| go through by putting themselves into a corner can be avoided
| if everyone had better mental/language models to think/talk
| about it.
| Ericson2314 wrote:
| Yes, and if you find yourself getting better at estimation,
| that's probably because you have failed to build proper
| abstractions. With proper abstractions, the cost of the same
| stuff should be minimal, so the part with no good priors
| predominates.
| [deleted]
| a_wild_dandan wrote:
| You've perfectly nailed the fundamental issue here.
|
| For many software projects, you simply cannot make meaningful
| granular estimations for parts of them. It doesn't matter how
| many story-point poker sessions you hold. _Some software work
| cannot be reduced to a positive integer._ In our business
| scopes creep, bugs plague us, and myriad issues make precise
| estimations pointless. It 's engineering Numberwang.
|
| I get that folks want to tame a chaotic world. But sometimes
| you can't. And the software engineering field in particular
| feels wildly neurotic about pretending otherwise. Sometimes I
| imagine applying our project management tools to other
| technical fields and laugh at how insane we must look.
| cgrealy wrote:
| > It's engineering Numberwang.
|
| Can I just say that is the greatest description of software
| estimation I have ever read? I am 100% stealing that.
| aidos wrote:
| We use it internally ourselves. It's from That Mitchell and
| Webb Look.
| jacques_chester wrote:
| That estimation is hard does not make it impossible. That it
| may be inaccurate does not make it worthless. Special
| pleading that our industry is somehow the only one that faces
| uncertainty and novelty doesn't really hold water.
| csours wrote:
| > "Special pleading that our industry is somehow the only
| one that faces uncertainty and novelty doesn't really hold
| water."
|
| I'm sure you're right. I don't think we should abandon all
| hope. I just think we should be more honest. I also think
| that software has fewer constraints than other industries,
| which is a blessing and a curse for estimation
| btbuildem wrote:
| I'd beg to differ. There is very little invention going on.
| Most software solutions tackle well-known problems, customized
| to a particular business need. Akin to building a house, but
| with specific owner requirements (three-car garage, etc).
|
| It gets a little complicated partly because of the industry's
| penchant for reinventing its tools on a rolling basis. In the
| trades, technology remains largely unchanged over decades, and
| only truly useful new advances are integrated into workflows
| (for example, laser level vs plumb bob or spirit level).
|
| In software, new technological approaches are adopted on a
| continuous basis, and so a lot of effort is spent on learning
| how to solve old problems with new tools.
| csours wrote:
| > "There is very little invention going on."
|
| If you said there is very little fundamental research going
| on, I would agree with you; but even putting together
| building blocks in a new way is invention. If you look at
| patents (arguably a unit of invention), most of them don't
| have fundamentally new building blocks, just existing blocks
| put together in new ways.
| riazrizvi wrote:
| Yes. So you can more accurately estimate replicating components
| you've written before, eg another database client, but
| estimating new software is a successive sequence of uncertainty
| reduction. The first pass being highly intuitive, the second
| pass, with more of the details worked out, less so... until
| you've completed the first working version, and you can finally
| say in hindsight, it actually took this long.
| domano wrote:
| I noticed that my estimates usually end up 3x as high as the next
| one and i feel pretty confident about them
| alisaus6 wrote:
| Hottie hangout pics with nude babes - https://adultlove.life
| samsk wrote:
| My bulletproof formula for doing estimations in corporate
| environments:
|
| InternalEstimation = estimate the work as truly as possible (ie.
| in MD) ExternalEstimation = double the value of
| InternalEstimation, and increase units by one order
|
| Example: 1 day (MD) = 2 weeks 2 weeks = 4 months
| kylecordes wrote:
| Sometimes a request for an "estimate" is really a request for a
| promise, quotation, a guarantee that something will be delivered
| by X time or cost.
|
| It's easy to detect this:
|
| Gently begin a discussion of how much uncertainty is tolerable,
| do they want to know the number we are 50% likely to hit? 80%?
|
| If you get emotional pushback to discussing uncertainty, they are
| looking for a promise.
| yters wrote:
| It is provably impossible to predict development effort.
| neartheplain wrote:
| Reminds me of the excellent 2010 talk, "What We Actually Know
| About Software Development, and Why We Believe It's True":
|
| https://vimeo.com/9270320
|
| Bit long, but well-presented and worth a listen for any
| practicing software developer (or person who manages developers).
| k__ wrote:
| I think, software development works best without deadlines, but
| then you have to find ways to finance yourself while people wait.
| jadams3 wrote:
| The most useful piece of advice I have gotten for estimates is
| that they are all junk until someone sits down and tries to do
| the work. For big jobs, 20% of it, small jobs pushing 50% of the
| work.
|
| In every single case when you've done that much work, I seem to
| wind up with a reasonable estimate.
|
| If we've done the job before and have data on it, also
| reasonable.
|
| Double or triple anything else.
| didibus wrote:
| The issue with estimates are expectations. While nobody
| acknowledges it, you're not actually asked for an estimate,
| you're being asked for a quote.
|
| The difference is when you're asked for a quote, you're asked how
| much you will be charging, with the expectations that you'll be
| willing to eat into your own margins to give a lower quote.
| That's why it's a negotiation, where you negotiate how much extra
| effort, time and headcount you're willing to give, how much tech
| dept you're willing to take, etc., for the privilege of getting
| their business.
|
| If you see it for what it really is, you'll see that it works
| pretty well actually. The business gets more out of you for less
| to them. It was never about having an accurate timeline or
| helping with planning or prioritizing, and always about
| negotiating a better contract with the dev team.
|
| Now keep in mind that the "business" in this case is a person who
| need to report that through their amazing prowess of
| administration and management, they personally managed to get X
| feature out during their last review cycle at Y cost with impact
| Z. This person will not need to deal with developer satisfaction,
| retention and performance. They will not need to deal with the
| impact the lower margins they pushed for had on the next feature
| delivery, or the continued maintainance of the systems. And if
| the dev team had to lower the quality too much in order to meet
| the quote they put out, that will be 100% their fault, the
| "business" will know not to use them for their next contract, or
| they'll expect the dev team to take on fixing all the issues at
| their own expense once more.
| quickthrower2 wrote:
| That is the degenerate scenario, but it's not always true.
|
| Often I see this haggling down of estimates and then
| microagression if estimates are not met but no need to work for
| free to make up for it.
|
| And I've also seen wise use of estimation but that is rarer!
| didibus wrote:
| > And I've also seen wise use of estimation but that is rarer
|
| I've got a question for you here, what is a wise use of
| estimation?
|
| What problems are estimates the ideal solution for? In the
| context of software development?
|
| I can think of only two: planning headcount and priorities
| (aka planning the order in which you do things).
|
| When you focus on those problems, you realize there's no need
| for an accurate estimate. All you need is relative sizing
| between requests. This seems twice as much work as that but
| gives us half the benefit. Ok probably we shouldn't do this
| one first. And even then, often time impact is all you need
| for priorities. Like who cares what takes you longer, if you
| just focus on building the next most impactful thing you'll
| very often succeed.
|
| The question of: "When are we going to be done this?" Is
| almost always irrelevant except for what I mentioned "asking
| for a quote". Like, who cares when it's done? What else you'd
| rather we do? I can tell you if your alternative will be any
| faster or not. And is this not the most impactful thing? If
| so, let's finish it and stop slowing us down with these
| irrelevant questions.
|
| Now for headcount, it's the same thing. Look how much you did
| last year with what headcount. Ask yourself if what you're
| hoping to do this year is of the same relative size, twice as
| big, half as big, or smaller? And then plan headcount
| accordingly. If twice as big, double head count. Overtime
| you'll learn if twice as big requires doubling headcount or
| 1.5x or 1.2x or 2.3x etc.
| jeffreygoesto wrote:
| Get something ready for Christmas business? Get a car on
| the road? Finish the cruise ship in time? ...
| didibus wrote:
| > In the context of software development?
|
| While your examples I'm not sure are super relevant from
| a software development context. I think you're pointing
| out that the estimate would help you decide when you need
| to start so you are done before the "real life events
| that needs the thing".
|
| Is that correct?
|
| Like for example I'm not sure it's useful to learn that
| no, you will not be ready for the Christmas event. Or no
| the cruise ship will not be ready in time. But it might
| be useful to know: "If I want to be ready for the
| Christmas event when should I get started and how many
| developers do I need?"
|
| That's a good example I guess, though I haven't seen this
| issue arise very often in software. I could imagine
| someone needing to know when to start feature freeze and
| prep work for scaling up leading up to a big sale event
| if you run an online store for example.
|
| My take on this is that asking devs to guess that time
| would obviously not work. And if you did this type of
| forecasting for any other discipline, you'd instead
| gather historical data about how much time things took in
| the past and plot that to forecast how much you'd think
| it takes in the future. Which is a process that can be
| done without needing to ask a dev for their personal not
| data based guesstimate.
| jeena wrote:
| Who cares when it's done? Often there are external
| deadlines like a start of production of a car for example
| which can not realistically be moved because too much
| planning would need to be redone and too much disruption
| would cause so much overhead that it could break a company.
|
| Also when price and time are fixed, there is only one thing
| left for negotiation and it is the amount of features being
| delivered. Sometimes people have a hard time to cope with
| it but if you do it correctly it's a great thing for
| everyone. The most important features could be delivered
| early and the closer to the deadline the non critical
| features can be implemented which are nice to have but
| won't break the product if they're not in place. That's a
| great way to bring as much value as possible without
| breaking anyone.
| loopz wrote:
| Wise use of estimation is projection. You don't need to do
| much more than count stories. It would be honest and
| without agenda.
|
| I've never seen it either.
| didibus wrote:
| Yes exactly! When you ask someone for their estimate,
| you're getting a quote from them.
|
| When you actually want to predict the length of time
| something will take, you rely on historical data and
| create a projected forecast. It never involves asking
| somebody for their best guess. That's how every single
| "other thing" we predict is handled, so why is software
| done differently.
|
| Actually, I've seen some of these ideas being talked
| about in KanBan circles, like tracking lead time and
| cycle time, and then using monte carlo simulations to
| forecast, but I've never actually seen them in practice.
| daanlo wrote:
| Software doesn't exist in a vacuum. The marketing team needs to
| know when the landing page / qr code (whatever) is implemented,
| because they need to plan their marketing budget. The customer
| care agent that gets shouted at because feature xyz is broken /
| not working as the customer expects, would like to give the
| customer an answer when it will be fixed. The customer care
| manager is also worried about her team members' retention.
|
| So having a rough or concrete time line / roadmap is helpful
| for these parts of the business.
|
| I also believe that a feature request is incomplete without an
| estimate. It is like a friend asking you to go buy him a car.
| What does he need a 3000$ subaru from 2003 or a 2021 Ferrari?
| Of course the friend can give you an insanely accurate list of
| specifications, but giving you a budget + some guidelines will
| probably be more effective for your friend and more fun for
| you. Especially considering that you know much more about cars
| (read software) than your friend. Knowing the budget you will
| be able to make much better choices.
|
| If you take the allegory of a quote (which I think is correct)
| then you need to consider that the haggling process always also
| includes scope of the product. ,,I'll give you 5000$ for the
| car, but only if you include a free car wash". And you can
| always use an estimate as defense against feature creep. ,,We
| are running low on budget, I suggest descoping xyz into the
| next iteration. Otherwise we won't be able to ship on time". If
| on time is not defined then you can never have this
| conversation.
| cheschire wrote:
| And to continue your metaphor, most software estimation comes
| from someone describing a car they like, and based on your
| mock ups you and others decide on a price, how long is
| acceptable to build, etc.
|
| So then you goto build the car and everything's going great
| till you find out that the latest research shows the specific
| design of engine you're using doesn't work with the eco
| friendly fuel that is now being mandated by the government.
| And suddenly killing cows is uncool so you have to change all
| your chairs to fabric or source fake leather, but half of
| your company thinks that endorsing fake leather is hide-
| appropriation and you're now spending a third of your time in
| HR going through training.
|
| And eventually you get about a week out from delivering the
| car and the customer calls your PM and says they want the car
| to have a top speed 30mph over the original specs while
| maintaining the fuel efficiency rating. And the doors should
| swing up, instead of out. Like billionaires'.
| snarf21 wrote:
| The issue is that you are being asked to estimate something
| that has never been done before. Even houses always go over
| time and money and that is fairly straight forward.
|
| These days, I only give estimates in terms of _units_ but
| without numbers. Hours, days, weeks, months, quarters or years.
| Some relatively small number of those units. If you want a
| quote it will take an extra 1 /4 of the estimate worth of time
| for an exact timeline. I really wish we would treat sales
| people the same way. "How much money is this contract for? What
| date will it be signed?"
| corry wrote:
| 'I really wish we would treat sales people the same way. "How
| much money is this contract for? What date will it be
| signed?"'.
|
| In well-run sales teams, that is exactly what happens. There
| is a constant move towards more accuracy and tightness in
| forecasting.
|
| In poorly run sales teams, the reps basically set their owns
| quotas (padded, of course, to mitigate against massive
| uncertainty) and then are very imprecise at managing the WIP.
| Which sounds like a poorly run engineering team too. In
| sales, though, at least there is a very objective measure
| ($$$ closed) at the end of the period. Also, the period is
| fixed typically (months, quarters, etc).
|
| Engineering is obviously different... what's the measure?
| LOCs? Obviously not...
|
| Edit: just to be clear, I'm saying that well-run sales teams
| and well-run engineering teams probably have a lot in common,
| not that they are exactly the same.
| chasd00 wrote:
| i think well run teams in general have a lot in common. Not
| just sales or engineering but all well run teams.
| didibus wrote:
| I think you're talking about goals, which isn't the same as
| estimates.
|
| A well run sales team, like a well run dev team do benefit
| from goals. Like increase X by Y. That's because it can
| guide your decision process of what you'll spend your time
| on and what you won't. It can also push for focused
| innovation around a particular area.
|
| But I think OP meant what if you asked the sales team: How
| much time will it take you to close the deal with X? Would
| they be able to estimate that? Which is different from
| asking them: "Try and close X Y% faster than it took us to
| close Z last time." The former would be asking for an
| estimate, the latter would be setting a goal.
| xyzelement wrote:
| Sales team certainly understand the sales cycle and
| different velocities involved.
|
| Eg - two jobs we ago sold to finance playera from small
| hedge funds to huge asset managers to central banks and
| finance ministries. You could sell more small hedge funds
| by knocking on more doors but you understood that the big
| players took months and years to decide. And of course we
| always tried to hone the sales org and process to arm the
| sales people with tools and data to move faster. We
| certainly held them much more accountable than devs.
|
| It's actually not a fair comparison - software
| development is often like golf - the only limit to your
| velocity is you while sales can't really move faster than
| customer's process in many cases.
|
| (Yes I obviously know dev processes can have external
| dependencies too)
| didibus wrote:
| Moving faster and increasing impact of deliveries, those
| are goals you can work towards, that do not involve
| needing to estimate anything.
|
| The question is: do you for every sale provide an ETA of
| when you think the sale will close, with projected key
| milestones along the way and their dates, all before even
| beginning engaging the would be customer?
| dkarl wrote:
| > Even houses always go over time and money and that is
| fairly straight forward.
|
| This is an incredibly important point. I was "raised" in
| commercial software at a time when there was a lot of
| condescension internally towards ourselves as an industry
| because we couldn't do estimation and planning, and building
| was the favorite comparison. A $100 million building couldn't
| run late or over budget. That would be unimaginable, and it
| doesn't happen because those people are too serious and
| professional to let it happen, not like us unserious,
| immature software doofuses. Turns out that was a complete
| lie, but at one time it warped the profession, because it
| made people believe the answers already existed, right around
| the corner, something you would figure out pretty soon just
| like last year you didn't know regular expressions and now
| you did.
|
| The problem with that was that when somebody walked in with a
| button-down shirt, oozing confidence and saying they knew
| exactly how to consistently deliver software on time and bug-
| free, those people weren't laughed out of the room. Elaborate
| heavyweight processes, and their "lightweight" (hah!)
| cousins, could sell themselves as proven, established
| solutions. We could believe that bullshit because we had been
| fed the lie that other industries had conquered these
| problems.
| meheleventyone wrote:
| It doesn't even have to be a big construction project to
| hit budget and schedule issues. Rebuilding our house had us
| going right back to the drawing board having to get new
| plans drafted and approved. It worked out much better in
| the end but the unforeseen cost and time overrun was quite
| anxiety inducing at the time.
|
| But yeah the idea anyone has magically solved accurately
| predicting the future is completely bonkers.
| einpoklum wrote:
| > These days, I only give estimates in terms of units but
| without numbers. Hours, days, weeks, months, quarters or
| years.
|
| Then you must be either an independent contractor or have
| some magical source of job security. I couldn't get away with
| this kind of estimate.
| koonsolo wrote:
| I once had a manager that thought he could negotiate estimates.
| That is like negotiating with the weatherman about the weather.
| Sure, maybe in the end you can convince them it will be
| finished sooner. But the fact is that it really doesn't change
| reality.
| lixtra wrote:
| Typically there are many dimensions to a solution, the
| negotiation can be about trading one dimension for another.
|
| - does it have to be instant or can we run it as a batch
|
| - maybe it's enough to have this tool in English only
|
| Etc.
| benhurmarcel wrote:
| I find it much less stressful to deal with since I've made
| peace with the fact that some managers want to be lied to.
| xyzelement wrote:
| I never experienced the dynamic you are talking about. Sounds
| like your workplace is terrible.
|
| I've almost always had business partners that were just that -
| partners - invested in the product and the team long term. I am
| the same way now that I am "the business."
|
| Estimation is a useful forcing exercise for thinking through
| what something takes, identifying risks and hard parts and
| deciding upfront what to do about them.
|
| It also becomes a good measure of people's integrity - what
| they do once it's apparent the estimate is off. 100% of the
| time, when engineering came back saying "we underestimated X,
| it's much harder" - it was totally fine.
___________________________________________________________________
(page generated 2021-01-18 23:00 UTC)