EXTRACTING 'TODO'S FROM A LOG Hello gophers! In one of my earliest rambling phlog posts, I mentioned that I keep a personal "log", which is my version of a journal or diary. [1] In short, here's how the system works: 1. I always have a small notebook and mechanical pencil on my person in which I record the happenings of the day, ideas, things I've learned and...TODOs! 2. The next day, I transcribe the previous day's entry into a text file named with the day's date (example: "2019-05-17-Fri"). What works well ================================================================= For the most part, this has worked really well for years because: * Paper notebook and mechanical pencil are (nearly) bomb-proof * As long as I remember to jot things into the notebook, I never lose information or ideas * The transcription process means I never forget about a TODO note because I'm guaranteed to see the note again This last point is especially important because _knowing you'll see the note again_ develops trust in the system. Trust means that I'm not tempted to put important things somewhere else "just in case" and have the whole thing fall apart into a series of post-its, emails to myself, etc. like I used to do before this system. The other awesome super-power of this system (over just writing in a paper journal) is that text files are searchable! The problem ================================================================= Transcribing is a bit of a chore, but it only takes a little over five minutes for me to type up the _average_ day's log. The problem is the TODO entries. In the best case scenario, I can accomplish the TODO immediately - often just a matter of filing away a note in another text file (or VimWiki, which I use for most of my general notes). But some things would be more involved - even taking me away from the keyboard entirely. If a TODO was big enough that I couldn't just stop and get it done right away, I would need to make sure I didn't forget it. So I would then have to stop transcribing anyway, flip forward to the current day in my paper notebook and *re-write* the entry as a safe-guard. Sometimes I ended up re-writing the same TODO over and over for a week before it got done. I don't enjoy hand writing things. So this was truly a painful burden (bordering on punishment), so I usually tried to complete the TODO as soon as I could. Well, I was finding that the so-called task of "transcribing my log" could easily take over an hour for a single day's entries because it actually encompassed a half dozen other tasks. You can imagine how this problem could snowball out of control when I got a little behind in my transcription! (And let's not even speak of being behind on the raw _written_ entries other than to say I am human and it has happened.) The solution ================================================================= I love it when a solution is technological. But I'm shy of sys- tems which are: complicated, fragile, or proprietary. So how could I: 1. Transcribe a day's entry as fast as I could type the words on the page 2. Trust the system to capture TODOs 3. Be reminded of any outstanding TODOs until they are done 4. Make it easy to track the completion of TODOs Whatever it was would also have to be simple, free, and entirely under my control. The solution to the first three items is an amazingly simple Perl script called "extract-todos": #!/usr/bin/perl # CPAN package: use Mail::Sendmail; my $todos = `ag '^todo:' /home/ratfactor/log/`; my $count = $todos =~ tr/\n//; my %mail = ( To => 'me@example.com', From => 'my-computer@example.com', Subject => "$count TODOs", Message => $todos, ); sendmail(%mail) or die "Could not send mail."; It matches any line which starts with "todo:" (case insensitive, so "TODO:" works too) and sends the results in the body of an email. Side-notes: I use "ag" - The Silver Searcher, but grep would be almost a drop-in replacement - just a couple command line changes. The CPAN Sendmail module is completely self-contained and requires no additional configuration to send email. As I expected, I also needed to whitelist the email from my todo script with my email provider based on the "From" address to keep it from being marked as Spam. This script runs every night at 02:00. Here's the cron entry: 0 2 * * * /home/ratfactor/bin/extract-todos Every morning, there's an email in my inbox with a subject like "3 TODOs". Open it up and there they are! I love my little email report. Additional Vim Sugar ================================================================= So you might be wondering how I complete a TODO item so it doesn't keep getting emailed to me over and over. What I've decided to do is complete an entry by changing "todo:" into "done :" so that an entry like todo: Buy some milk becomes done 2019-05-19: Buy some milk There are lots of possible future applications for having a date of an item being complete. It has also occurred to me that there are neat things I could do with the TODOs themselves, such as append- ing a future "due" date. But with added complexity comes a reduction in trust in my system. For now, if I need to schedule something for the fu- ture, I'll use a proper calendar. What I *can* do in my log system, though, is make a TODO entry to *remind me to make* the calendar entry for a specific future date. But wouldn't be a total pain to have to open up each file with a TODO entry, find the entry, and update the text? Yes it would be horrible. I didn't do even a single entry manually. Instead, I've got this incredibly cool Vim "one-liner" in a shell script called "todos": #!/bin/bash vim -c "cd /home/ratfactor/log/" \ -c "vimgrep /^todo:/ *" \ -c "copen" Each "-c" option runs a command in Vim (as if you'd typed ":" and then the ex command yourself). What this does is open Vim, change to my log directory, use the vimgrep command to search for TODOs and then open up the "quick- fix" window in a split window with all of the results. Neat! But to make it even better, I've got these keyboard short- cuts in my .vimrc: nnoremap :cp nnoremap :cn nnoremap :s/^todo:/done =strftime("%F"):/i If you've not dealt with a lot of Vimscript, this may appear to be sheer nonsense, but here's what each keyboard shortcut does: Ctrl+Up Go to previous TODO entry Ctrl+Down Go to next TODO entry F6 Mark and datestamp a TODO as done The previous/next shortcuts work no matter if the entries are in the same day's file or if they span multiple months of files. I used this new setup yesterday afternoon and this morning to blast through a backlog of 32 TODOs in record time. It's not merely less typing, it's less *thinking*. Conclusion ================================================================= I'm really happy with this setup. This system means I can concentrate on the task at hand without the constant annoying and inefficient context-switching required by my old method. If I'm transcribing an entry, that's all I'm doing: typing. If I'm clearing TODO's, all I have to worry about is either com- pleting the TODO or skipping it and going on to the next one. After just a handful of entries, the keyboard shortcuts become completely automatic. I don't lay awake at night wondering if I'm forgetting to do something because I know there's going to be an email waiting for me in my inbox in the morning. I doubt I'll even be reading the email the vast majority of the time. It's more like a piece of string tied to my finger: just seeing it there is enough of a re- minder. This entire system is so minimalistic that I could easily re-cre- ate it if I had to. -- Isn't it amazing how you can ponder something for *years* and when you finally come up with a solution, it can turn out to be so astonishingly *quick and easy* to implement? Of course, it's only because of the years of pondering that the simple solution becomes "obvious". :-) [1] gopher://sdf.org/0/users/ratfactor/phlog/2018-08-10-The-Log- ging-Habit