[HN Gopher] The joys of maintenance programming (2011)
___________________________________________________________________
The joys of maintenance programming (2011)
Author : pramodbiligiri
Score : 69 points
Date : 2023-09-18 17:17 UTC (5 hours ago)
(HTM) web link (typicalprogrammer.com)
(TXT) w3m dump (typicalprogrammer.com)
| hiAndrewQuinn wrote:
| I love maintenance programming. I love legacy programming too. I
| have a story where the two worlds collide.
|
| At my last job I had to work with a PowerShell codebase that
| `cloc`ed in at over 100,000 lines, and was absolutely ancient to
| boot, having been added to in bits and pieces by sysadmin after
| sysadmin over the course of 12 years total, in 3 different human
| variables-and-comments languages, all of whom were Windows power
| users with very clear ideas of what needed to be done, but also
| all of whom were very clearly learning to program for the first
| time, at least in the wonky PoSH paradigm. AKA: Classic big ball
| of mud situation. Every new customer had to have their details
| run through this codebase, however, and I saw the potential to
| eventually boil this down to a simple 6- or 7-parameter cmdlet
| (PoSH for "shell script"). I got my undersized chisel and toy
| hammer and got to work.
|
| In some ways starting work on the proj felt more like beginning
| to dig into an ancient COBOL database in a System/360 mainframe
| than anything resembling the state of the art in 2023, or even
| 2011. There was a single VM that everything had to run on, and
| the script couldn't even be called remotely via SSH because the
| only way in was via a web browser based remoting session. The
| connection was spotty and would often drop halfway through coding
| in there in my dinky little Notepad++ session, requiring minutes
| of reauthentication just to get back to work. Unit testing (which
| PowerShell had a surprisingly good third party library for,
| called Pester) was out of the question. And: There was no version
| control! No wonder it got to 100k lines - most of the code was
| being entirely unused! But what?
|
| The first few Saturdays I spent didn't go very far. I was mostly
| trying to figure out where the logic even began and ended in this
| mess of spaghetti. But eventually I started to piece together an
| overall structure for the thing. I made a pact to myself not to
| delete any of the many, many files that had accrued over the
| years until I knew every file that the code would touch first. I
| started doing some lightweight encapsulation with the -WhatIf
| flag, and with firebreaks (sticking `return` after each tiny
| function) so I could work on one linear piece at a time, and
| gradually tiptoe my way forward to the finish line. Eventually I
| got there, and managed to strip the whole thing down to about
| 1500 lines.
|
| The joy of this kind of programming is that, even if you don't
| start with regression tests or modern practices, you do know from
| having ran through the original process a million times _exactly_
| what the end product is supposed to look like. Since no one is
| paying attention to your refactoring work in the first place, you
| 're allowed to take your sweet time. To me it was obvious this
| mess of code was going to become a blocker for the business at
| the pace we were going: We were a SaaS, and yet we had to
| manually provision customers behind the scenes like this, in a
| process which took days to weeks? And we didn't even have a
| proper CLI we could call for this? Jackpot!
|
| I left the company about 2 months after wrapping this project up
| and demoing it to my coworkers in search of better pay, but I'm
| really glad I got the chance to take such a fearsome codebase and
| tidy it up like that. There is just something satisfying about
| that kind of work, it's like gardening. I also got really, really
| good with Notepad++'s find and replace across all files
| functionality. There is in fact a skill to it! (Mostly that you
| should always, always find and replace the longest strings
| first!) Although I think these days I'd probably reach for a
| similar function in vanilla vim first, or even in bare `sed` if I
| was feeling gutsy.
|
| (I'd end with "If you, dear reader, are in a similar PowerShell
| quagmire you need someone to pull you out of, I'm available for
| consulting at $X00/weekend", but alas. My weekends for the next
| few months are already sworn to another equally wonky project for
| my current employer. At least I'm getting overtime this time. ;)
| )
| windock wrote:
| I felt so much joy just reading this! Yeah, it is truly great
| to just simplify.
| [deleted]
| stringtoint wrote:
| Nice post. In the past I've had colleagues (also senior people)
| who disliked touching "legacy" parts (e.g. Java based
| microservices compared to newer Express.js or Kotlin based
| microservices which don't provide half the value that the legacy
| parts do). I also think that maintenance programming can be
| rewarding. However, I do believe that it does not make sense to
| throw entry-level engineers into the maintenance mess alone: they
| still need some guidance in my experience. Then you'll be able to
| reap the benefits such as getting better in debugging and
| understanding more about the business problems that the software
| is solving.
| x1f604 wrote:
| Do people understand code by line-by-line reverse-engineering
| what the code is doing, or do they understand it by relating it
| to what they've written before?
|
| If the latter is the case, then you get better at reading code by
| writing code. Writing lots of code puts those code-patterns into
| your long term memory, and then when you see those code-patterns
| again you'll recognize them.
|
| For system design too - if you've designed lots of systems
| yourself, then when you see a new system, you'll be able to
| relate it to the systems that you've designed before.
|
| So maybe building greenfield projects also makes you better at
| maintaining existing projects?
|
| It'd be great if someone could point me to some existing
| literature on this topic. I've looked around and can't find any.
| fatnoah wrote:
| I haven't had to do it a lot, but in previous cases where I
| came into a codebase blind and had to fix bugs or figure out
| what's going on, I've always started by mapping the flow first
| and then digging into the details of what's going on. When
| looking at the flow, I go breadth-first through everything and
| then dig in from there.
|
| I maintain two major maps, which are what is the code logically
| trying to do, but also what exactly it's doing. (i.e. Sanitize
| inputs, check error conditions, do work, return result as a
| logical model and then "ensure foo is < 0 and > -10" etc. as
| the specific checks).
| tivert wrote:
| > So maybe building greenfield projects also makes you better
| at maintaining existing projects?
|
| I think it does, because it builds a higher-level sense of how
| something "could" or "should" be and familiarity with thinking
| at the system level.
|
| I've had a lot of problems with people who (seemingly) only
| have experience maintaining projects. They seem to have a
| tendency to focus narrowly on "fixing" a bug close to its
| source, and often lack understanding of a lot of fundamental
| technologies (because they only attend to things that have
| broken), and get stuck by a kind streetlight fallacy where they
| fix things in more familiar areas. The end result is ugly
| kludges and whack-a-mole bugs where a superficial "fix" in one
| area causes a new bug to pop up in another.
| ye-olde-sysrq wrote:
| >> So maybe building greenfield projects also makes you
| better at maintaining existing projects?
|
| Only to an extent, i think.
|
| > I've had a lot of problems with people who (seemingly) only
| have experience maintaining projects.
|
| And similarly I've had problems with people who (seemingly)
| only have experience with starting a new project and then
| simultaneously over-engineering and piling on tech debt.
|
| I think "i've never bootstrapped a project before" is easier
| to cure than "I don't have a good sense of what's expensive
| vs what's cheap tech debt."
|
| Some tech debt is basically 0% interest. Was this small hack
| bad? Yeah. Will it need to be fixed, _eventually_? Yes, for
| sure. But does it compound every day, or does it just sit
| there being a little yucky? Very easy to determine in
| hindsight. Very hard to predict as you write the hack. The
| end result being the simultaneity in my example. People will
| over-engineer things that won 't end up being problems and
| under-engineer things that will turn out to require
| compounding hacks and it would've been cheaper to just get it
| right the first time.
| darkhorse222 wrote:
| >If the latter is the case, then you get better at reading code
| by writing code. Writing lots of code puts those code-patterns
| into your long term memory, and then when you see those code-
| patterns again you'll recognize them.
|
| Let's hope what you write the first time is a good practice.
| rvba wrote:
| There are lots of greenfield-only programmers who build
| something (by glueing stuff together), then they proceed to
| ship their barely working software to get that promotion /
| bonus and they run away to the next project
|
| They never have to maintain anything, some poor soul has to fix
| the broken software.
|
| They also dont even know what mistakes they made, since they
| arent there to solve it.
| swader999 wrote:
| The joy of Greenfield is short lived. At some point, it becomes
| too costly to fully refactor and you are stuck with what you
| chose and edging closer and closer to legacy dev.
|
| Some projects though just keep on trucking along and doing the
| next part with the newest shiny thing. You end up with five ways
| to access data, three ways to send emails and fifteen different
| methods of report generation. Frankenstein architecture! This
| architecture also emerges when you have leaders that don't make
| decisions and peeps debating all day on slack and then doing
| their own thing anyway because they didn't really want to offend
| anyone and everyone's ideas are great.
| jebarker wrote:
| I'm currently on a sort of sabbatical from greenfield research
| projects. I'm doing maintenance programming on a large widely
| used open source project. It's been really refreshing. I didn't
| realize how much I was feeling the pressures of 15+ years of
| greenfield research. By contrast the maintenance programming is
| clearly defined, easier to put down at the end of the day and
| unambiguously useful. I'm also rapidly improving in my debugging,
| code understanding, profiling and optimizing skills which will
| make me a better programmer when I do return to writing new code.
| aidos wrote:
| I've described this to other programmers before. Greenfield
| projects are fatiguing because you still have options--too many
| options. Every layer is still open for you to rethink and
| rework. Let the hand wringing commence!
|
| As the months and years wear on though, things change. Each bug
| has an obvious solution, and you'll take the path of least
| disruption. Implementing new features is a matter of figuring
| out which trade offs you're most ok with. Maybe you need to
| refactor something first, but that's an obvious path itself.
|
| You're painted in a corner. A beautiful corner where the only
| safe thing to do is to tread carefully. But you don't mind,
| because you know where you are and what you need to do to get
| to where you want to go.
| jebarker wrote:
| Nicely put. I'd also add that for me being a researcher
| became my identity and so my self-worth became tied to the
| perception and adoption of the research. With an established
| project (especially one I didn't write) there's already a
| stable acceptance of it's value and so I'm free to just
| maintain and improve it.
| dang wrote:
| Discussed at the time (of the article):
|
| _The joys of maintenance programming_ -
| https://news.ycombinator.com/item?id=2267261 - Feb 2011 (29
| comments)
| vsareto wrote:
| >Skilled programmers recognize what makes good code good, and
| what makes bad code bad
|
| You're still somewhat biased, so you can get put on projects
| where the style is different from yours. If you happen to think
| that style is bad, it won't be fun to work on.
|
| You shouldn't want to change the style. Chances are you'll do it
| incompletely (job change) or the time spent won't be worth it. If
| someone else comes along and doesn't like your style, we're just
| going to repeat this process.
|
| There's been way too many different approaches that work for
| developers to be unified on what styles are best.
| JohnBooty wrote:
| Maintenance programming for pay can be rewarding
| _iffffffffffffffff_ management has a deep understanding of and
| appreciation for the task. There can be a lot of joy and
| satisfaction in adding tests, fixing edge cases, fulfilling user
| needs, "quality of life" improvements, optimizing. _Honing._
|
| My experience is that this is almost never the case.
|
| Generally, management does not understand or appreciate why it's
| a slog, why you might want to refactor this or that, etc.
|
| Obviously a lot of this falls upon the engineer. It's your job to
| communicate and demonstrate. Helps if you have other engineers
| backing you up and presenting a unified message. Also helps if
| management is technical enough to understand what you're saying
| instead of simply believing -- or _not_ believing -- you.
|
| But maaaan. Uphill battle! I'm burnt out on it. I've decided I
| need to seek greenfield stuff. Maybe forever. But definitely for
| now.
| jckahn wrote:
| Agreed. Imagine you're a CEO: Why on earth would you want your
| expensive engineers reworking something that already makes
| money? It seems much more sensible to leave the working code
| alone and focus on new revenue-generating opportunities.
| riku_iki wrote:
| > Imagine you're a CEO: Why on earth would you want your
| expensive engineers reworking something that already makes
| money?
|
| CEO will start looking at this topic if app
| performance/stability metrics will start making impact on
| revenue stream, aka app is slow and stop working every day
| and customers go away because of that.
| esafak wrote:
| Which happens if the CEO is conscientious enough to insist
| on collecting and looking at those metrics.
| pixelrevision wrote:
| Heh. Going to be a lot less maintenance work if they are.
| JohnBooty wrote:
| True, but app performance/stability metrics are (often
| extremely laggy) trailing indicators of code quality.
| juliangamble wrote:
| Security and risk. You can run several different
| code/pom/package.json scanner tools that will articulate a
| risk that will send your IT security department into a panic
| attack, which you can sell as time you need to spend fixing
| things.
| bbbobbb wrote:
| That's not what the post is about. They describe the process
| as mostly fixing issues with existing software, so the
| motivation is that something is broken or not working as
| required anymore.
|
| Refactoring it is just one of possible parts of the process,
| much opposed to 'yeah this legacy code is broken dogshit and
| needs to be rewritten even though the current software is
| already written, proven and works for 99% of the stuff we
| need' approach.
| JohnBooty wrote:
| [Refactoring] is not what the post is about.
|
| Not directly, but kinda, at least in my experience.
|
| Clients and management generally understand that old code
| needs to be maintained. They don't love that reality, but
| that's not generally a tough sell unless they're completely
| clueless. And they're usually not _completely_ clueless.
|
| The tension in my experience comes when an engineer wants
| to perform something other than the fastest, sloppiest,
| filthiest duct-tape post imaginable.
|
| Management accepts the need for fixes, but often does not
| have the technical/domain knowledge to understand the
| short- and long-term merits of Proposed Fix A versus
| Proposed Fix B. And naturally, incentives heavily push
| management toward the quick fixes by default. Time is
| money, after all. And managers are expected to meet
| quarterly and annual OKRs. We don't have a good way to
| quantify the long-term value of a "proper" fix versus a
| "duct tape" fix. And so on.
|
| So for me maintenance work is difficult to separate from
| notions of longer-term software maintenance such as
| refactors, dependency upgrades, etc.
| whateveracct wrote:
| CEOs tend to be short-sighted when it comes to engineering
| management, that's true.
| dagw wrote:
| It is not at all obvious that trying to develop new revenue-
| generating opportunities will actually bring in more revenue
| long term compared to optimising and improving existing
| revenue generating sources. And it's not like the old code
| stops working once you start reworking it. Give your
| engineers N months to rework the old code, and if it's not
| better you don't have to deploy it.
|
| Every larger company I've worked at had enough engineering
| overhead that they could afford to have small teams working
| on both reworking old stuff and creating new stuff, while the
| main teams kept the working stuff generating revenue.
| throw239487 wrote:
| Does anyone have any tips on this beyond the additional "sales"
| type stuff where you have to put in a ton of extra effort to
| measure and sell your work?
|
| I've found this to be true... to the point where some places
| even prefer manual maintenance to spending (potentially) a
| little more time on automation, even if that time would be paid
| back over the next couple of years.
|
| I'm not adept at planning, selling my work, doing the manual
| maintenance, and doing the extra automation, as eventually that
| work becomes homework unless you get a boss that's willing to
| put in some effort here, which I haven't found yet.
___________________________________________________________________
(page generated 2023-09-18 23:00 UTC)