[HN Gopher] Happiness is a freshly organized codebase
___________________________________________________________________
Happiness is a freshly organized codebase
Author : felixrieseberg
Score : 68 points
Date : 2020-05-07 19:07 UTC (3 hours ago)
(HTM) web link (slack.engineering)
(TXT) w3m dump (slack.engineering)
| dangwu wrote:
| Interesting read!
|
| > No more dumping ground folders like "Helper" or "Utility"
|
| If you're organizing by feature and you have one of these
| helper/utility classes that support multiple features, where does
| it live? Would you consider each utility to be its own "feature"?
| drawkbox wrote:
| It depends on how many projects you have. A place like Slack
| has only one client/product really when it comes down to it, so
| they can get pedantic about it and look down on smart reuse
| across projects with robust tested libs on top of standard libs
| that are necessary to increase production and
| stability/security etc.
|
| If you work at an agency, or game company shipping many games,
| there will always be a "core" library or "base" library that
| has Helpers and Utility. Good consistent projects wanted tested
| and solid parts. Every game has a common lib of core tools like
| maths, vector tools, prediction, data structures and many more.
|
| There is such a broad difference in coding on one platform for
| one company for one product compared to shipping many companies
| (or many internal projects) on many platforms for many
| products.
|
| Even standard libraries for platforms are essentially a
| Helper/Utility really when it comes down to it: .NET Core,
| Python standard lib, standard node, C++ distributable etc.
| There will be common core tasks across a project, product,
| company or platform for most teams with many projects to
| manage.
|
| If you don't have common libs with common helpers/utilities in
| large sets of products, technical debt and maintenance become a
| nightmare. In projects we work on these are core/base libs that
| are submodules in git and are essentially the 'tech' across
| projects where the project/product itself is the unique
| implementation for that product. Anything common or generic
| gets put in the core 'tech'. Every single game studio will have
| these as well as agencies if they are organized and produce
| quality relatively fast and consistent.
|
| If "Happiness is a freshly organized codebase" they must have
| common tested parts that are ship tested.
|
| This is really just Slack talking specifically about mature
| products at a company that only has one product. It isn't
| reality at places that have many projects, products, companies,
| clients etc. Pretending they are the same is a bit elitist.
| ericaengle wrote:
| We discovered a lot of cases would be unique, so we do try to
| evaluate a "best" case when it comes to each move.
|
| Keeping that in mind, if a feature has a shared helper across
| components within the /Feature folder, we'll have a /Shared
| folder within to capture these files. At a top level, we would
| encourage a new /Feature folder since most likely it will have
| tests or be important enough to merit a folder rather than
| squishing it into another /Feature folder. We're optimizing for
| visibility too, so having it's own /Feature folder helps with
| that.
| dangwu wrote:
| Thanks! And am I correct in assuming Slack just has a single
| target for the iOS app (App), then one for each App
| Extension? So the root source folders map 1:1 to build
| targets?
| ericaengle wrote:
| Yes! that's the idea, or at least what we've worked to
| achieve
| brundolf wrote:
| I actually think it's vitally important to have "dumping
| ground" locations for things that aren't fully figured-out yet.
| If I'm working on a new thing and I have something that's
| relevant in multiple files but I'm not sure where it should go,
| or even whether it will stick around, I don't want to have to
| come to a screeching halt just to make a taxonomic decision
| about something that's still a work in progress. The key, of
| course, is going back and categorizing those things later once
| you _do_ have an idea of where they should go.
| ryanianian wrote:
| > The key, of course, is going back and categorizing those
| things later once you do have an idea of where they should
| go.
|
| Yes, but in my experience this is rarely done. It's put off
| until it gets so convoluted that it introduces bugs. The same
| is true even if you do avoid dumping-grounds: you need to
| examine the project's state _somewhat_ regularly to ensure it
| still makes sense. Where code lives can be as important as
| the code itself and thus deserves at least occasional review
| and change.
| allenu wrote:
| Totally agree. So much of programming and designing is
| figuring out what pattern your code is falling into and what
| stuff "is" or "means".
|
| I believe it's totally okay to name things utilities when you
| haven't yet established a common pattern or understanding of
| it. Naming things is hard and spending so much time on it and
| organizing things can often block you from progressing to a
| place where you DO have more information and can
| intelligently name things.
|
| You'll always be juggling unknowns, so it's okay to have
| dumping grounds here and there, so long as over time you
| clean them up as you gain more info.
| brundolf wrote:
| I recently started working within a Python codebase, and
| one of the things I really like about it (not sure whether
| this is standard Python practice) is that most directories
| have a "common.py" file in them. So if you just want to put
| something somewhere real quick, you can elevate it to
| exactly the appropriate directory-level instead of going to
| a single, global "utils" file. It's a neat pattern.
| honkycat wrote:
| I always argue adding "helper" or "util" adds nothing to the
| description of a file/module/class and is better left off
| nilkn wrote:
| I feel like these names convey a lot, though. When I see
| something like this, I immediately expect it to not do any of
| the following: (1) contain core logic or define primary
| classes/types/records; (2) have dramatic side effects; (3) be
| hard to test; (4) reference or depend on other parts of the
| codebase; (5) do anything controversial in general.
|
| I'd say I'd expect a "utility" to adhere to these conditions
| more strongly than a "helper," which might be a bit more
| entangled with application logic.
| seph-reed wrote:
| Utility - few or no dependencies. Absolutely no dependencies
| specific to the project. (array flattener, 2-way map, etc)
|
| Helper - function which makes some really common project
| specific code more DRY (createApiError, genCommonConfig, etc)
| ravenstine wrote:
| I'd also consider a Helper to be something that may have
| application-awareness. Whereas, as you stated, a Utility
| would have no serious dependencies and is otherwise "dumb".
| [deleted]
| rukittenme wrote:
| I've been wondering a lot recently if folders make projects
| better or worse.
|
| For example, when I write a library its usually very simple.
| There's a single directory which contains all of the source
| files. When people use the library they:
|
| """
|
| import lib
|
| lib.run()
|
| """"
|
| Dead simple, no complex module paths to remember, no hierarchical
| folder structure forcing you to code based on a pattern rather
| than functionality. Pure bliss.
|
| But on the other hand, I have projects that contain 100k lines of
| source code. I can't just leave it out in the cold. So poor baby
| gets a couple of folders.
|
| But I do hate it. I hate writing the code. I hate reading it. I
| hate finding it 6 months after the fact.
|
| That's probably just the nature of the job. It is work at the end
| of the day. Maybe its just doomed to be hard.
| osener wrote:
| I'm not plugging the language, but I've come to appreciate
| OCaml's module system with no imports and (mostly) globally
| unique module names. No circular dependencies allowed either.
| You can have multiple modules within a file (which is also a
| module named after the file name).
|
| I structure larger projects as libraries with minimal
| dependencies that depend on one another, and dump all my
| modules with descriptive file names under the same directory
| within the library.
|
| I vaguely remember reading something that hinted at Facebook
| doing something similar with their React components.
| ldd wrote:
| There is this wonderful utility: dependency-cruiser[0] for
| javascript / typescript projects.
|
| It visualizes dependencies in a project. I found it so so easy to
| refactor and move files around after I started using it. I am not
| usually a visually-oriented person, but for this usecase, and to
| be happy, `dep-cruiser` surely helps.
|
| [0]: https://github.com/sverweij/dependency-cruiser
| kccqzy wrote:
| I sometimes wish Git properly records copy and move information.
| It turns out Git's heuristics for detecting copies and moves work
| about 95% of the time, and for the remaining 5% it's mildly
| annoying to read a git blame with every line from a move. You can
| blame further across that move, but that's manual. If copy and
| move information is perfectly recorded, I'd have more incentive
| to do these kinds of code reorganization.
| ryanianian wrote:
| This article mentioned a while ago had relevant info:
| https://news.ycombinator.com/item?id=22689301
|
| In my experience, ensure you don't change the code as you move
| it, don't rename files and move content at the same time, and
| don't move "too much" code in a single commit. Around 2k lines
| at a time seems to be a good number. Maybe some
| languages/structures are easier for git to analyze when doing
| blames.
| kccqzy wrote:
| How do you not change the code as it's being moved, when you
| need to update #include directives, imports, and other file
| paths?
| ryanianian wrote:
| Sometimes it is unavoidable, but in general minimize it.
| Don't refactor a class and rename it at the same time.
| Hopefully moves just result in imports and paths changing,
| something that is not likely to confuse a blame (and rarely
| do I care about the blame lines on #includes/imports)
| derefr wrote:
| It's easy if not every commit needs to compile. :)
| rudi-c wrote:
| I've encountered similar difficulties around codebases with a
| lack of file hierarchy structure. But one major difficulty in
| fixing the issue is that moving a lot of files around tends to
| trash `git blame`, which is often more valuable than knowing what
| folder to put a new file in. Is that something you've
| encountered?
| smallnamespace wrote:
| Try putting the refactoring commit in .git-blame-ignore-revs
| Lammy wrote:
| Use Mercurial? :)
| ryanianian wrote:
| There are workarounds to get git to search harder or to commit
| things in a way that's helpful for large-scale file-shuffles,
| but to be honest I've rarely found this to be enough of a
| reason to not move things around. The end result is a much more
| productive and purposeful place for code to live and grow. This
| said, moving files and moving code within those files at the
| same time is a recipe for confusing yourself and git. Move
| files first, then content.
| [deleted]
| ryanianian wrote:
| The concept of linting repo structure is an excellent one.
|
| I cannot even get started in projects without a sane repo layout.
| Source files scattered everywhere unrelated to each other, utils
| junk-drawers with dozens of files, multiple top-level source
| directories without explicit rationale, tests and source totally
| disjoint, hacks to modify build and run paths, implicit
| dependencies between directories, awful convoluted build-system
| configuration to match all of these idiosyncrasies, and
| impossible or very difficult editor/ide integration as a result.
| Even rails-eque apps (which come with a reasonable structure) get
| really messy really quickly if somebody doesn't have the
| diligence to stay on top of it.
|
| I want there to be a {sane, polyglot, large-ish-scale} project
| layout convention that plays well with
| {intuition/discoverability, build-systems, editors, project
| size}, but it seems there's an inherent tension between these.
| Maven tried. Kinda. I wonder if there's a fundamental problem we
| aren't solving somewhere.
| DelightOne wrote:
| In maven I don't like the test-separation from the code itself
| because its much simpler to find and update them this way.
| ryanianian wrote:
| There's a lot to not like about maven. I think that
| separation was motivated by java's classpath rather than
| making a 'good'/dev-friendly convention. After 10 years in
| maven-land, I changed a recent project to put `foo.ts` and
| `foo.test.ts` in the same directory and its wonderful, but it
| confuses some build-systems and makes certain kinds of test-
| helpers harder to write.
| bob1029 wrote:
| I feel like a lot of the pain in codebase organization boils down
| to having a technical project structure (i.e. layout on disk)
| that does not align well with the business. Obviously, it's
| impossible to force something to directly align with such
| disparate and abstract requirements, so you have to create
| abstractions (layers) to enable a hypothetically-pure realm.
|
| In our architecture, we've created roughly 2 different kinds of
| abstraction: Platform and Business.
|
| The Platform abstractions are only intended to support themselves
| and the Business abstractions. These are not supposed to be
| easily-accessible by non-technical folks. Their entire purpose is
| to make the Business abstractions as elegant as possible. The
| idea is these should be slowly-changing and stable across all use
| cases. Developers are only expected to visit this realm maybe
| once every other week. We effectively treat ourselves as our own
| customer with this layer.
|
| The Business abstractions are built using only first-party
| primitives and our Platform layer. Most of this is very clean
| because we explicitly force all non-business concerns out into
| the Platform layer. Things like validation logic are written
| using purely functional code in this realm, which makes it very
| easy to reason with. Developers are expected to live in this area
| most of the time. Organization of business abstractions typically
| falls along lines of usage or logical business activity. We
| explicitly duplicate some code in order to maintain very clean
| separation of differing business activities. I've watched first
| hand as the subtle variances of a single combined Customer model
| across hundreds of contexts eventually turned it into a boat
| anchor.
|
| As a consequence of all this, our project managers and other non-
| code-experts are actually able to review large parts of the
| codebase and derive meaningful insights without a developer
| babysitting them. We are at a point where project managers can
| open PRs for adjusting basic validation and parameter items in
| our customers' biz code. Developers also enjoy not having to do
| massive altitude shifts on abstractions while reading through 1
| source file. You either spend a day working platform concerns, or
| you are in the business layer. Context switching sucks and we
| built to avoid it as much as possible. IMO, having a bad project
| structure is one direct side-effect of being forced to context
| switch all the time.
| ajsharma wrote:
| Wow, I love this idea. I'm curious how you have that structured
| within the code base. Is it literally a platform folder and a
| business folder?
___________________________________________________________________
(page generated 2020-05-07 23:00 UTC)