[HN Gopher] Ncdu - NCurses Disk Usage ___________________________________________________________________ Ncdu - NCurses Disk Usage Author : thecosmicfrog Score : 358 points Date : 2022-12-07 12:57 UTC (10 hours ago) (HTM) web link (dev.yorhel.nl) (TXT) w3m dump (dev.yorhel.nl) | user3939382 wrote: | If someone could make a CLI DaisyDisk that would be amazing. If | anyone's aware of it let me know please! | fomine3 wrote: | `rclone ncdu` is also a jewel | ghewgill wrote: | Yes, this. I actually discovered ncdu through first seeing it | in rclone. I thought "that is awesome, why isn't this tool a | thing?" And it was. | | I type `du -sk *|sort -n` a lot less frequently now. | tjoff wrote: | Really nice tool, didn't know of the rewrite in Zig, the linked | blog post discussing the rewrite it was good: | https://dev.yorhel.nl/doc/ncdu2 | esaym wrote: | I don't really see the point of a gfx ui for this. I just live on | the edge: find / -type f -mtime +200 -exec rm -f | {} \; | jstimpfle wrote: | du -hs * | sort -h | unixhero wrote: | I use this system all the time! | | ncdu -x to instruct it not to traverse foreign filesystems | TruthWillHurt wrote: | The page mentions FSV - a 3d fs viewer. Classic! | | https://fsv.sourceforge.net/ | alt227 wrote: | The trick with ncdu is to install it BEFORE your disk gets full | from some random error log going haywire. | banana_giraffe wrote: | I've saved the day once or twice by having a built of ncdu that | I can just copy to /dev/shm to figure out what went haywire. | ndsipa_pomu wrote: | Or a "journalctl --vacuum-time=1d" will usually clear a lot of | space | sdflhasjd wrote: | $ sudo apt install ncdu > Not enough free space | | Fuck! | | Actually, one trick I learned was to create a dummy file full | of zeroes or rand, maybe 200MB or so, and place it in an | obvious location (e.g /) for easy deleting in such a crisis. | herpderperator wrote: | Ext3/ext4 systems do this by default and reserve 5% of space | for root.[0] | | [0] https://docs.cloudera.com/cloudera- | manager/7.4.2/managing-cl... | cmg wrote: | One of the first things I did in my new position was to add | ncdu (and mysqltuner) to our Ansible playbooks. It's that | useful for me - I probably use it every other day at least. | ranting-moth wrote: | Another trick is to have a 1GB file called | DeleteMeWhenDiskIsFull. | | Then you can delete that file and ponder your life choices that | led to that situation. | | Just remember to create it again :) | doubled112 wrote: | > ponder your life choices that led to that situation. | | I always remember that VMware vCenter appliance didn't come | with a working logrotate config for years and feel better. | dottedmag wrote: | That's the reason why typical Linux (or Unix) filesystem comes | with a setting to set reserve some space for root account. | | Doesn't help if the disk is filled by processes running as | root, of course. | wjdp wrote: | While ncdu does the job I've found gdu (similar tool written in | Go) significantly faster for larger directories. | | https://github.com/dundee/gdu | thatwasunusual wrote: | Still only pretty fast, though. | the_gipsy wrote: | Someone's gotta write rdu. | arnarbi wrote: | OP's link has a list at the bottom with alternatives (that in | itself is very cool). It lists both gdu and a Rust | alternative called dua. | aesh2Xa1 wrote: | See also broot, dust, and pdu for Rust alternatives. | unixhero wrote: | The realest reply | agleason wrote: | you're probably more likely to see jsdu first and then I'll | have my buddy explaining to me why it's actually faster | [deleted] | rwky wrote: | Love ncdu. I probably use it at least once a week somewhere | debugging a disk that's getting full. | mritzmann wrote: | If you use a Btrfs filesystem with snapshots, I can recommend | Btdu as an alternative. Advantage: Can handle duplicate files | (snapshots), which however only occupy 1x disk space. | | https://github.com/CyberShadow/btdu | aendruk wrote: | More interesting than its support of Btrfs features is its | unusual statistical approach: | | > btdu is a _sampling_ disk usage profiler [...] Pick a random | point on the disk then find what is located at that point [...] | btdu starts showing results instantly. Though wildly inaccurate | at first, they become progressively more accurate the longer | btdu is allowed to run. | thecosmicfrog wrote: | Have used btdu - very useful on Btrfs systems with lots of | snapshots. | aw1cks wrote: | This looks very useful! I wish there were something similar for | ZFS. | belthesar wrote: | At first blush, this beats the tar out of using other | deduplicator strategies on btrfs. I'm looking forward to | checking this out more thoroughly. | salmo wrote: | Interesting. I've just used df-m | sort -n and walked the tree | from the CLI for so long. | | I've also used windirstat (I think that's what it was called) | years ago on Windows and Disk Inventory X on MacOS graphically. | | This seems like a nice in between that could replace both methods | for me. I'll have to try. | | The GUI bunch of squares is really only useful when you can hover | with a mouse and would be clunky in a TUI even if you could | render it. And the FS structure isn't immediately obvious, so I | find myself wasting time wandering hovering over big files and | globs of little ones. | | I really like to avoid a mouse when I can unless it's really | useful. | mandarax8 wrote: | You mean `du` right? | salmo wrote: | Yes, just dumb fingers typing the comment. :( | smcameron wrote: | I always used "du -S | sort -n" ... I think... it's been a | long time since I've needed to worry about disk space. | lathiat wrote: | FYI I recently discovered sort ---human is a thing so you can | keep the K/M/G units if you want to. | | However I use ncdu all the time. | salmo wrote: | Yeah, my habits come from using GNU, BSD, SysV and having to | use the lowest common denominator. | Sohcahtoa82 wrote: | > I've also used windirstat (I think that's what it was called) | years ago on Windows and Disk Inventory X on MacOS graphically. | | On Windows, the new hotness is WizTree, which rather than | recursively calling directory listing functions, it directly | reads and parses the file tables itself. This makes it orders | of magnitude faster. I have a 2 TB hard drive full of a million | files, and WizTree reads and parses it all in under a minute, | whereas I can expect WinDirStat to take half an hour. | salmo wrote: | That's pretty brilliant. | | My windows experience is really out of date. I knew NT4 and | 2000 the best. Then I picked up 2008 for a while supporting | small businesses. I don't hate it or anything, but am | definitely deeper on Unixes and about equal on VMS that I | supported as my NT4/2000 time. I'll work on whatever pays :). | michaelsbradley wrote: | growlight may be of interest as well: | | https://github.com/dankamongmen/growlight#readme | Cieplak wrote: | Have been using ncdu for more than a decade, and recently started | using diskonaut for similar purposes. Was looking for a terminal- | based treemap visualization for analyzing disk usage and stumbled | upon diskonaut, which is exactly that. | | https://github.com/imsnif/diskonaut | valdect wrote: | i replaced gnome Baobab with ncdu for a while. it's significantly | faster | lasermike026 wrote: | Oldie but a goodie. | ktzar wrote: | ncdu is one of the most useful tools I've come across to quickly | get rid of unneeded files in remote servers. Top program. | jrm4 wrote: | Broadly, is anyone aware of a generalized list of "new versions | of classic tools?" | | There are so many now that are better than the old stuff; I | almost feel like a unified round up of these, maybe even in a | _distro_ form, might be good for linux enthusiasts, newcomers, | old-timers, etc. | seedie wrote: | What came to my mind is a blog post[0] by Julia Evans | | Not exactly what you had in mind but might still be | interesting. | | [0] https://jvns.ca/blog/2022/04/12/a-list-of-new-ish-- | command-l... | charlie0 wrote: | One thing that comes to mind as someone who is not in the know | of these new tools would be, are they safe? | | The old tools have been there forever and used everywhere. My | assumption would be these are safe and don't change often. For | better or for worse, I would be concerned about using the newer | tools unless they are backed and/or approved by a large open | source org. | t-3 wrote: | If the tool is "$x, but with pretty colors", there's a good | chance they are not safe to use in pipelines. It's | distressingly common for colored output to be sent even when | piped. | jrm4 wrote: | I really respect this take -- and also kind of don't like it | at the same time? | | Basically, I don't do "mission critical" Linux things. I | teach IT and I hack around on my own boxes with scripts and | stuff because it's fun and useful to me. I'm always on the | lookout for the hooks and such that can get more and | different people into Linux. | charlie0 wrote: | I understand. The truth can be bitter. My first instinct is | to install most of these tools. Then I remember all the | recent supply chain attacks and malicious packages that | have been snuck into software. | | I use 1 daily driver for everything, including my finances | and crypto. So trust me, I'm bummed out about this. I'll | still check some of these out. As long as they seem safe, I | will install a few. | capableweb wrote: | https://github.com/ibraheemdev/modern-unix | | > A collection of modern/faster/saner alternatives to common | unix commands. | carlhjerpe wrote: | Zellij instead of tmux (not necessarily better, but it's easier | to use) | | Xonsh instead of bash (because you already know Python, why | learn a new horrible language?) | | bat instead of cat (syntax highlights and other nice things) | | exa instead of ls (just nicer) | | neovim instead of vim (just better) | | helix instead of neovim (just tested it, seems promising | though) | | nix instead of your normal package manager (it works on Mac, | and essentially every Linux dist. And it's got superpowers with | devshells and home-manager to bring your configuration with you | everywhere) | | rmtrash instead of rm (because you haven't configured btrfs | snapshots yet) | | starship instead of your current prompt (is fast and displays a | lot of useful information in a compact way, very customizable) | | mcfly instead of your current ctrl+r (search history in a nice | ncurses tui) | | dogdns instead of dig (nicer colors, doesn't display useless | information) | | amp, kakoune (more alternative text editors) | | ripgrep instead of grep (it's just better yo) | | htop instead of top (displays stuff nicer) | | gitui/lazygit instead of git cli (at least for staging, nice | with file, hunk and line staging when you have ADHD) | | gron + ripgrep instead of jq when searching through JSON in the | shell (so much easier) | | keychain instead of ssh-agent (better cli imo) | | Wrote this on the train with my phone by checking | https://github.com/Lillecarl/nixos/blob/master/common/defaul... | for which packages I have installed myself :) | kwhitefoot wrote: | > Xonsh instead of bash (because you already know Python, why | learn a new horrible language?) | | Exactly, one horrible language is enough! | colordrops wrote: | ranger and vifm instead of midnight commander (vim key | bindings) | | lsd instead of exa (better formatting, icons) | | mosh instead of ssh for interactive sessions (maintains | session even with bad connectivity) | | hyprland instead of sway instead of i3 instead of XMonad | dbdoskey wrote: | nushell instead of Xonsh/bash/fish. A bit of a learning curve | but worth it Also tig as a git ui | zorr wrote: | Tig also belongs in this list. An ncurses git repository | browser that I use all the time. | riquito wrote: | tuc instead of cut (cut text better than `cut`, or lines like | head/tails - eg cut first and last line at once) (but I'm | biased, I'm the author) | | https://github.com/riquito/tuc/ | beanaroo wrote: | btop instead of htop instead of top | mtlmtlmtlmtl wrote: | htop has much better support for more obscure unices, fwiw. | Supports every BSD, whatever Solaris is called these days. | | Btop seems to only support Macos, Linux and FreeBSD. | daemoens wrote: | An annoying thing about btop is that it only uses the snap | package manager and nothing else. You can still install it | yourself easily but I don't understand why they'd stick | stick with snap alone. | pluc wrote: | bpytop, ncmpcpp | thecosmicfrog wrote: | I like iotop. Pretty much exactly what it sounds like - a top- | like program for I/O operations. It's also just an apt/yum/dnf | install away on most distros. | xorcist wrote: | htop also includes columns for read/write. | | There's also more detailed CPU usage you can turn on, | including I/O wait. | | Use it, it's great. | chasil wrote: | If you are just looking for the largest files in a directory | hierarchy, try this: find . -type f -print0 | | xargs -0 ls -s | sort -n | | If your find/xargs don't support null delimiters, then the guy | who wrote musl has some tricks: | | http://www.etalabs.net/sh_tricks.html | kqr wrote: | One thing ncdu does _not_ improve on over du | sort is that it | still needs to scan the full directory structure before even | displaying any result. | | I would like something that starts estimating sizes immediately, | and then refines those estimations as it is able to spend more | time. I tried writing it myself, but I ended up not quite knowing | how to go about it, because just getting the count of files in a | directory takes about as long as getting the total size of said | files... | | (Another problem is that file sizes are notoriously fat tailed, | so any estimation based on a subset of files is likely to | underestimate the true size. Maybe by looking at how the | estimation grows with more data one can infer something about the | tail exponent and use that to de-bias the estimation?) | uzername wrote: | I recently discovered ncdu while troubleshooting a logging issue | in one of servers. Something broke and syslog began to inflate | unchecked and consumed the entire disk's worth of space. ncdu | helped explore the system and find out where the file was. Great | tool. | progman32 wrote: | I use this tool all the time! On my Windows boxes, I use an old | tool called "Scanner" that presents the results in an interactive | pie chart: http://steffengerlach.de/freeware/ | | I've used it on everything from Windows 10 to my Win98SE | oldschool gaming box. | WorldPeas wrote: | NCDU is one of the first things I install on any system. Glad | it's finally getting some recognition | emmelaich wrote: | Fantastic tool. One improvement I'd like to see is support for | .Trash instead of straight out deletion. | | I had to delete a lot of stuff so I turned off the 'Are you | sure?' prompt. | | Sure enough, 4 minutes later I fat fingered some wanted files | into oblivion. | topspin wrote: | Used the zig static build in offline mode yesterday to hunt down | a excessive storage consumer. Highly effective. | | Another tool I've used is jdiskreport[1]. It's a java app with a | straightforward GUI. File system scanning is multi-threaded and | quite efficient. | | 1. http://www.jgoodies.com/freeware/jdiskreport/ | vlowrian wrote: | Wow! Someone really outdid himself on todays (7th) advent of code | challenge! ;) | desro wrote: | I use ncdu almost daily across many different systems. Especially | handy when running macOS on a <512GB volume and you need to keep | an eye on ballooning caches and other "helpful" cruft. | | `rmlint` and `jdupes` are also seeing a lot of use here lately, | reclaiming terabytes of space from years of sloppy media | organization (or the lack thereof!) | lanstin wrote: | Yeah I prune stuff with ncdu weekly on my work 0.5T machine. | Oddly I don't have to use it so much on my 2T personal machine. | chrisallenlane wrote: | TIL that my `.xsession-errors` file is ~16GB. Thanks, `ncdu`! | | (I should probably look into this...) | andrewstuart2 wrote: | > This is fine. | staunch wrote: | Despite trying these tools many times, I still find myself just | running `du -sm * | sort -g`. It works and it's easy to pipe to | grep, etc. | luke-stanley wrote: | I have used ncdu for a long time, and it's good if you have time | to wait for it to get sizes, if you can't, Duc AKA "Dude, where | are my bytes" has caching and is ready to apt install on Debian | and Ubuntu repos https://duc.zevv.nl and it has different GUI | options. It's slightly more complex but for larger volumes I use | it instead of ncdu. | patwolf wrote: | I never knew about this tool, but it looks quite useful--much | better than just using df. | | The only thing I wish it had was a squarified treemap view of | disk space. There was an old graphical tool for Windows called | SequoiaView that I used to use years ago, and I've never found a | worthy replacement for it on Linux or MacOS. | warp wrote: | GrandPerspective is in the mac app store and at least for my | needs was similar enough to SequoiaView. | ruined wrote: | sequoiaview looks like a clone of kdirstat, which is available | on all platforms. | flir wrote: | Try https://www.derlien.com/ - GPL. Might fit your needs. | macote wrote: | The best tool I've used on Windows for that is "Scanner" by | Steffen Gerlach: | | http://www.steffengerlach.de/freeware/ | thecosmicfrog wrote: | I find myself using WinDirStat on Windows systems, but often | use ncdu in WSL. | Sohcahtoa82 wrote: | On Windows, you should switch to WizTree. Rather than | recursively calling directory listing functions, it directly | reads and parses the file tables itself. This makes it orders | of magnitude faster. I have a 2 TB hard drive full of a | million files, and WizTree reads and parses it all in under a | minute, whereas I can expect WinDirStat to take half an hour. | | On an SSD, WizTree only takes a couple seconds. | thecosmicfrog wrote: | Thanks for the suggestion! | wereallterrrist wrote: | `dua`, discussed elsewhere in this thread also works on | windows, just for another option. | aftbit wrote: | I use WinDirStat on Windows | | https://windirstat.net/ | Sohcahtoa82 wrote: | You should switch to WizTree. Rather than recursively calling | directory listing functions, it directly reads and parses the | file tables itself. This makes it orders of magnitude faster. | I have a 2 TB hard drive full of a million files, and WizTree | reads and parses it all in under a minute, whereas I can | expect WinDirStat to take half an hour. | | On an SSD, WizTree only takes a couple seconds. | electroly wrote: | WizTree is vastly faster than the other options presented here | (Scanner, SpaceSniffer, WinDirStat). | antisthenes wrote: | It's also the one that's much more regularly updated. | Sakos wrote: | Switched to this a few years ago. Far better than anything | else I've tried. | password4321 wrote: | WizTree v3.33 is the last 'donationware' version vs. free for | personal use now. | | https://diskanalyzer.com/wiztree-old-versions | Farow wrote: | I'm personally a fan of SpaceSniffer: | http://www.uderzo.it/main_products/space_sniffer/index.html | macote wrote: | Thanks for sharing! I really like the real time update during | progress. You sometimes have to wait a very long time in | Scanner before you see the result. | flir wrote: | While we're listing them, Disk Inventory X | (https://www.derlien.com/) | Havoc wrote: | It's a neat tool. Not quite clear why it's #1 on hn though? | aftbit wrote: | One I don't see mentioned here is xdiskusage (for Linux) which | gives a nice 20th century style X11 tree view of your disk usage. | | xdiskusage -qa / | mtooth wrote: | ncdu is one of the most useful CLI tools out there! Been using it | for many years as well. | | Another disk scanner worth plugging that I came across for some | use cases where I needed to generate single-view reports is pdu - | it has the same concurrency implementation that other ncdu | alternatives use so the performance is much better too. | | https://github.com/KSXGitHub/parallel-disk-usage | jwr wrote: | Such a great tool. Thank you! | | On a Mac, I've been using OmniDiskSweeper for years, but this can | be run in a single directory and on my Linux machines as well. | Fantastic! | | And look, it didn't ask me to sign up for an account, and it | didn't require me to consent to usage data collection with a huge | Privacy Policy attached! How is that possible? (still dealing | with this morning's scars from trying, and failing to run the | warp terminal) | gfxgirl wrote: | now if it was just an executable file and didn't require a | package manager or compiling to install it would be flawless. | zoobab wrote: | Love the tool. | HPsquared wrote: | Named after Enkidu, the ancient Sumerian mythological figure? | | https://en.m.wikipedia.org/wiki/Enkidu | mhitza wrote: | Most likely just a coincidence due to abbreviating _NC_ urses | _D_ isk _U_ sage | gchamonlive wrote: | Dense | | Edit, to spell it out: op was surely joking | teddyh wrote: | Old-school alternative: du -ax / | xdu -n -c 9 | mahemola wrote: | Nice GUI to `du -h --max-depth=1 .` or `du --max-depth=1 . | sort | -n -r` | pilif wrote: | The other big advantage over just being a gui is that it Scans | the filesystem ahead of time and keeps the data across your | navigation into directories. | | du will have to rescan as you traverse down the tree | netsharc wrote: | The subsequent scans feel a lot faster, I suspect due to | caching. | LeSaucy wrote: | ncdu and /var/lib/docker, name a better combo. | wereallterrrist wrote: | `docker system prune -af` ? | SSchick wrote: | This tool has been pretty invaluable to me over the years in | diagnosing/tracing high disk usage, particularly relevant for | pypi and npm packages that were needlessly shipping huge | artifacts. | rwmj wrote: | Nice tool - although with remote machines my go-to tool is KDE | filelight, the joy of remote X11. | milderworkacc wrote: | I love ncdu and install it on all of my machines. But at the risk | of sounding like a broken record - why isn't its functionality | baked into stock file managers on windows and Linux? | | Why can't either of these systems do what the Mac has been able | to do since the 90s, and display the recursive size of a | directory in bytes in the file manager, allowing one to sort | directories by recursive size? | | I am not exaggerating to say this is the single biggest roadblock | to my permanent migration to Linux! | | (I would love nothing more than to hear I'm wrong and "you fool, | Dolphin can do that with flag: foo"!) | dotancohen wrote: | The Bash CLI is my file manager. So I've got ncdu built right | in. Try it, you'll love it. I almost never touch the rodent. | tetris11 wrote: | Except that running "ls" doesn't show you the directory | content size, and "ncdu" requires the user to make a tea | first. The above poster is right in saying that having this | built-in to the filesystem metrics would be a huge win. | j33zusjuice wrote: | But `du -h -d1` does, though, or `tree ---du -h`. | jinnko wrote: | ranger has this built in | ShinTakuya wrote: | Yeah but I'm not aware of any repos that use it as a stock | file manager.. | COGlory wrote: | I'm not sure what exactly you're asking for, but Dolphin shows | me the size of a directory. You may have to right click and | update it from time to time. | Doctor_Fegg wrote: | ncdu is fabulous. I used it a while back on my 2009 Mac mini and | recovered 40GB of disk space. 8GB of this was in just four | folders: ~/Library/Logs/CrashReporter/MobileDevice, | /private/var/db/systemstats, | /System/Library/Caches/com.apple.coresymbolicationd, and | /Library/Caches/com.apple.iconservices.store. | crawsome wrote: | justusw wrote: | This program is extremely useful. I have recently performed a | recovery on a bricked Microsoft Surface and was able to extract | all the useful files from a Windows home directory and quickly | discard all the libraries, caches, and so on. By seeing the | folders that take up the most space you can easily determine what | the juicy bits in a folder hierarchy are. Huge time saver. | | Using GrandPerspective on macOS was a similar revelation, but | ncdu being keyboard driven and allowing you to quickly launch a | shell inside each folder and then apply find -exec there quickly | is a productivity boost on yet another level. | farresito wrote: | If you like ncdu, you might also like dua[0]. You can run `dua i` | to get an interface similar to ncdu, and can also run `dua` to | list file sizes in the current directory, similar to `du`. Or | `dua _filename_ ` to get the size of a given file. | | [0] https://github.com/Byron/dua-cli | lathiat wrote: | The UI is not as good but it's multi threaded and much faster. | f1refly wrote: | Not in any repo, I'll pass, thanks | wereallterrrist wrote: | Diistinctly not true. `nix run github:nixos/nixpkgs/nixos- | unstable#dua` | BlackLotus89 wrote: | Actually it is in the arch community repositories and seems | to be quiet a bit faster than ncdu so I will keep it in my | toolbox for now. | | Painpoints are that there seems to be no progress bar in | interactive mode, the ui is (imho) ugly/unintuitive (for | instance the usage bar seems to be relative? and the | shortcuts look like glyphs) and there are functions missing | (like exclude patterns, you can exclude dirs though!). | | So it won't replace ncdu, but if it get a interactive | progressbar maybe it will be on all my machines (with arch) | 0x00101010 wrote: | dua: https://github.com/Byron/dua-cli dust: | https://github.com/bootandy/dust | LeoPanthera wrote: | See also tkdu for a graphical version: https://github.com/daniel- | beck/tkdu | | It's abandoned but it still works, and I like it because you can | pipe the output of du into it, which is useful for visualizing | remote systems. | brazzledazzle wrote: | I use ncdu a lot. It's not strictly necessary but it's really | nice to have in the toolbox. Also IIRC there is a standalone | binary available which comes in handy when you can't install | packages on a full filesystem. | XiS wrote: | Best tool I've ever used for finding where I could free up some | disk space ___________________________________________________________________ (page generated 2022-12-07 23:00 UTC)