Migrated from Joplin to Org Attach
==================================

Joplin runs on Linux and MacOS, but is harder to install on
BSD, if possible at all. So, for some time I wanted to move
away from it.

I had some positive experiences with Org Attach, and decided
to see if a migration from Joplin to Org Attach is doable.

Joplin notes
------------
My notes in Joplin have a long history.

In 2012 I got an iPad Mini at work. In order to be able to
share notes between that iPad and my laptop, I installed
Evernote.

After some time I added a notebook for private notes in
Evernote and used that to store PDF files from invoices,
receipts, and stuff like that.

Some years later I abandoned the iPad Mini and stopped
using Evernote, except for the invoices.

When Joplin became available, I exported my private notebook
and imported that into Joplin. After this, I stopped using
Evernote. When we buy something at a store and I want to
keep the receipt, I still put that in Joplin.

Currently, most Joplin notes are just imported PDF files,
and photos of receipts. Also, I have kept notes that I made
before the conversion to Joplin, during some workshops I
attended.

I run a FreeBSD jail with a small webdav daemon, to
let Joplin synchronize the data.

Org Attach
----------
Org Attach is method to link attachments to org mode files.

Org Attach supports several ways to link and store the
attachments. I prefer the default method, which creates a
local data directory and copies the attachments into this
data directory. The big advantage of this method is, that it
creates a completely portable set, consisting of the org
mode text file and the data directory.

When you copy this as a complete set to another system, the
links on that system still work and can be opened from the
org mode text file.

Automatic commit to Git
-----------------------

Org Attach can commit the attachments automatic to Git.

To set this up, you have to do two things:

* Add a line to $HOME/.emacs:
  (require 'org-attach-git)
* Create a Git repository in the data directory:
  git init

After this, new attachments will be committed to Git.

Export from Joplin
------------------
Choose a notebook in Joplin and export all notes to a
directory. This creates a lot of files with the extension
"md", and creates a subdirectory called "resources", with
the PDF files and image files.

The file names of the export are in the format
"checksum.extension", like:

    "199d72f2161d4d5fba1b73ac8a89722d.md".

I tried some scripting to get a more meaningful filename. 

I started with "ls > filelist" in the resource directory, and with
some ed foo I created a file that looks like this:

echo -n "mv 16273d2d78d9a074a60465bf91a04774.pdf "  && grep -A2 "id: 16273d2d78d9a074a60465bf91a04774" *md | grep filename | cut -d' ' -f 2-6 | sed "s/ /_/g"
echo -n "mv 1650fc0df1234e1eb0c715527aab0dc9.pdf "  && grep -A2 "id: 1650fc0df1234e1eb0c715527aab0dc9" *md | grep filename | cut -d' ' -f 2-6 | sed "s/ /_/g"
echo -n "mv 1db93fe90e70f58adb67442578daf105.jpg "  && grep -A2 "id: 1db93fe90e70f58adb67442578daf105" *md | grep filename | cut -d' ' -f 2-6 | sed "s/ /_/g"

For every file in the resources directory there is a line
like the ones above.

I added a she-bang line at the top of the file, made
it executable and run it to create a new file:

   ./filelist > mv-script.sh

This creates a file that looks like this:

mv 0b09593ac85a27142f9d89185405756e.pdf 4493999457.pdf
mv 0bc269d0295edbd1c9c51ad5c4132cab.pdf Fiets_bon_maart_2012.pdf

Unfortunately, for a lot of files the line with the filename in the
Joplin export md-file had an empty file name.

In the created script, I removed all the lines where the
meaningful file name was missing, and run the script, which
remamed lot of files.

After this, I opened the resources directory in Emacs Dired,
and looked at every not-renamed file, to rename it a
more meaningful name.

When I opened such a file and saw it was an old invoice or an
old receipt, I didn't bother and marked it for deletion.

After I had gone through all the not-renamed files, I hit
'x' to let Dired delete all the marked files.

After this, the directory contained files that were
renamed by the script, and files that I had manually
renamed using Dired.

Prepare attach from Dired
-------------------------
In order to attach files with Org Attach from a Dired
buffer, add the following to $HOME/.emacs:

    (add-hook 'dired-mode-hook
              (lambda ()
                (define-key dired-mode-map
                  (kbd "C-c C-x a")
                  #'org-attach-dired-to-subtree)))

Create org text file
--------------------
In Emacs I created a new directory in my $HOME and
opened a new org mode file in that directory.

I created a new header, something like:

   * Invoices and receipts

and put the point (a.k.a. cursor) below it.

With "C-x 3" I opened a new window, and in it opened
in Dired the resources directory. This directory now
contains only useful files, with reasonable meaningful
file names.

I marked all the files, and with the command "C-c C-x a" let
Org Attach attach all the files to the org file. It created an
PROPERTIES drawer with an ID, to link the attachments
to the org mode header..

Next, I issued the command "C-c C-l" a lot of times, once
for each attached file. This command creates the
links to the attachments.
When all the links are created, save the org mode file.

Markdown notes
--------------
For the workshop notes I created a new header in the
org mode text file and moved point below it.

The workshop notes where among the markdown files, with
the format "checksum.md". I grepped to find the specific
files, and opened those in Emacs.

At the bottom of the buffer I removed the lines that had
been added by either Evernote or Joplin (don't know), and
used "C-c C-a b" to create an attacment file from the
buffer, using another brilliant feature from Org Attach.

Result
------
The result is a directory, containing a org mode text file,
and a data subdirectory, together in a Git repository.

This is a portable set, that can be copied to other systems.

So now my invoices and receipts, and my workshop notes, are
in an OS-independent system (unless you want to call Emacs
an OS :), and I don't have to keep a VM running Linux just
for Joplin anymore.

Hindsight, or lessons learned
-----------------------------
In hindsight it was smarter to insert the individual
links with some scripting, and not type many times "C-c C-l"
for each link. It is all plain text after all, the links have
the format:

    [[attachment:filename][readable name]]

First I created a Git repo in the data directory. It is
better to do that one level higher, so the org mode text
file is in the same repository.

Plain text format rocks !


Last edited: $Date: 2023/08/18 15:02:59 $