[HN Gopher] Show HN: Apple Notes Liberator - Extract Notes.app D...
___________________________________________________________________
Show HN: Apple Notes Liberator - Extract Notes.app Data and Save It
as JSON
Hey there! I just released the first version of a project I've
been working on solves a very specific problem that perhaps only I
have. I welcome any and all feedback, even if you just want to drop
in to say that this is a hot piece of garbage!
Author : kello
Score : 218 points
Date : 2023-03-26 16:59 UTC (6 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| ChintanGhate wrote:
| I have published an app [Exporter](https://apps.apple.com/us/app/
| exporter/id1099120373?mt=12) that does something similar -
| exports Apple Notes to HTML and markdown with the attachments.
| Give it a try, let me know if it solves your issue.
|
| Also, why did you select JSON as the output format?
| daydream wrote:
| Thanks for making your app. I've used it more than once and it
| works well. Many export solutions don't handle attachments but
| yours does. So kudos for that.
|
| A few years ago I tried to write my own exporter using
| AppleScript but got stuck by a showstopping bug in Notes
| AppleScript implementation. Or so it seemed. I hope that bug is
| fixed now. I just moved on to use your app instead.
| mkmk wrote:
| I consistently struggle with Notes and iMessage sync issues, and
| I suspect it's due to being a customer for so many years and
| making my way through many devices over those years.
|
| The UX of the Apple ecosystem is so nice but when it goes
| sideways you're kind of out of luck, even if you have applecare.
|
| I wish they had some sort of premium "fix my esoteric appleID
| account issues" service you could pay them for once a decade or
| so.
| bwbmr wrote:
| I agree- long standing issues that have popped up and are never
| fixed over the years:
|
| 1) My iPhone has never properly synced screen time with my
| other devices (presently my other devices: 2x iPads, MBP) sync
| fine together. This has persisted across multiple iPhones and
| major releases of iOS.
|
| 2) Apple News has a permanently saved "story" of a specific
| magazine issue's table of contents. No idea how it got saved,
| but since it isn't actually a story (it's a ToC instead) the UI
| option to "unsave" it is greyed out / unavailable. I reported
| it to a CSR about four years ago, he took down bug reports, I
| even sent a screen capture demonstrating the bug... it still
| persists and Apple provides no mechanism to manually clear all
| saved story data. I can remove it locally from the device from
| disabling iCloud sync for Apple News -> selecting yes when
| prompted to remove local data, but it will still persist in
| iCloud and there is no way to wipe it there... it's back as
| soon as I reenable sync.
| oneearedrabbit wrote:
| I am excited to see more solutions that work with Apple Notes
| database directly. This is my go-to application to capture
| unstructured memos while researching various topics. Last year I
| got carried away and built a simple site generator for Notes [0].
| As a fun exercise I implemented ORM models to query Sqlite
| database in Ruby: User::Account.server(database:
| "tmp/test.sqlite").first. folders_dataset.offset(1).first
| notes. map(&:attachments).flatten. find { |el|
| el.is_a?(User::ObjectTypes::EmbeddedPublicJpeg) &&
| el.height > 700 && el.summary[/animal/i] }.
| media_filepath => "Accounts/<uuid>/Media/F669B83A-0C47-408E-
| 9645-015737F3B11F/Pasted Graphic.jpg"
|
| Apple Notes' database structure is a fascinating product story.
| Notes started as a simple note-taking application that stored its
| content as raw HTML in a few simple tables. However, as the
| product grew and needed more features like collaboration, they
| had to make changes and add new columns and tables. They also
| implemented two flavors of CRDT-like algorithms that evolved over
| time: one for plaintext chunks and the other one for embedded
| objects like tables. On top of that, every change that Apple
| makes must be backward compatible to support legacy devices and
| systems; but backward-compatibility was not always possible and
| users had to go through "Migrate your Notes" steps, but it is
| again a product story.
|
| I wish that one day, someone from the development team would
| write an article about it.
|
| [0] https://devlog.notespub.com/2022/08/site-generator-for-
| apple...
| dunham wrote:
| Yeah, I'd like to know the story behind it. topotext is weird
| and unlike anything I've found in the literature. It is kind of
| like RGA, but with more links. And I haven't seen anyone else
| layer attributes on top of text.
|
| It looked like the other stuff came later and was somewhat
| orthogonal. (Although their ordered set construct was abusing
| topotext to represent ordered lists of ids.)
|
| I don't know how much has changed since 2018. I think my table
| decoding may be broken now.
| codetrotter wrote:
| That is awesome! Thank you for sharing your notes on this :)
| wslh wrote:
| I would love to have a tool to retrieve the hyperlink of the
| current note so you can organize the information as a wiki. I
| know it uses URIs.
| dmd wrote:
| I don't understand what value this tool adds over the library it
| uses under the hood (which includes a command line tool for
| exporting your notes; I've been using it for years). What is it
| for?
| reaperducer wrote:
| I built something similar a while back.
|
| But instead of pulling the data out of Notes, I took it from SMTP
| mail.
|
| I don't know if it still works this way, but when I did it, if
| you used a non-iCloud email address to sync Notes, it saved the
| data as pain text in email messages that could be easily parsed.
| css wrote:
| If anyone ever wanted to do something similar for iMessage data,
| I built a similar tool for that:
| https://github.com/ReagentX/imessage-exporter
| vszakats wrote:
| Here's a Python script that takes the JSON output from
| https://github.com/threeplanetssoftware/apple_cloud_notes_pa...
| (needs master), and rebuilds Notes in Markdown format (or HTML),
| with links, attachments, lists, etc.:
|
| https://gist.github.com/vszakats/5a3bd939721d1dde6142d9ea3b2...
|
| And a short, standalone JXA script, that outputs HTML, but loses
| some data, such as links:
| https://gist.github.com/vszakats/f24e7700428c1e694e20cee0b1c...
| samwillis wrote:
| Super interesting, particularly the link through to the blog with
| all the research into how Notes works.
|
| A particularly interesting thing that I don't see mentioned
| anywhere is that Apple Notes uses CRDTs (conflict free replicated
| data types) to enable both shared notes and multiple concurrent
| edits. With the CRDTs resolving the conflicts.
|
| It looks like the info on the page may be a good place to start
| looking to how that works:
| https://www.ciofecaforensics.com/2020/10/20/apple-notes-clou...
| eddyg wrote:
| CRDTs are definitely an "unsung hero" of Apple Notes... and a
| big reason why it is one Apple's best "built-in" apps as of
| late.
| howinteresting wrote:
| A notes app that only works with one 2-trillion-dollar
| company's hardware and does not have builtin functionality to
| export notes (e.g. a menu option to do so) cannot reasonably
| be described as good. One's expectations have to be through
| the floor for that to be an honest appraisal.
|
| Compare it to Obsidian, which just stores data in a bunch of
| markdown files on disk.
| Kerrick wrote:
| I can export to Text and/or PDF on my iPhone from the Notes
| app.
| howinteresting wrote:
| We're commenting on the website for a project called
| "Apple Notes Liberator". Obsidian does not _need_ a
| "liberator".
| macintux wrote:
| You're moving the goalposts.
|
| Your first comment:
|
| > and does not have builtin functionality to export notes
| (e.g. a menu option to do so)
|
| That's incorrect: you _can_ export to PDF. Certainly not
| as handy as Markdown, but it 's there.
|
| Additionally, Apple offers AppleScript as a way to do
| even more.
|
| No, this isn't an open source piece of software. If
| that's your sole criteria for "good" then you're right,
| it's bad.
| Gigachad wrote:
| I don't actually care about getting some data dump out of
| Notes. I use it as a way to quickly store temporary
| information. Any note older than a month is most likely
| useless. The most important feature is that it effortlessly
| syncs between my phone, work laptop, and home laptop. And
| that it does. Unlike a bunch of markdown files on disk
| where you have to create some janky syncing setup yourself
| which has no conflict resolution system.
| starbugs wrote:
| If they could now implement table resizing and tables that do
| not render with graphical errors, that would be so nice.
| munk-a wrote:
| As a windows user, while we don't have an OS built-in equivalent,
| the standard for such tools (in the form of Sublime and the like)
| is just to operate with temporary buffers that get written out as
| files on user request. Sublime will write internal data to some
| persistent storage somewhere and be able to survive sudden
| crashes and restarts but when you'd like to save the data out to
| a fixed filepath it's pretty trivial to do so. I do find it quite
| weird that Apple has managed to cultivate a system where storage
| in separate files isn't taken as the default.
| simonw wrote:
| I wrote a similar tool in Python a few weeks ago, which uses
| AppleScript to liberate the data and saves it to a SQLite
| database: https://datasette.io/tools/apple-notes-to-sqlite
|
| I didn't actually know AppleScript before writing this tool...
| but it turned out ChatGPT knew it well enough to unblock me and
| provide me with exactly the code I needed to build the rest of
| the project! https://til.simonwillison.net/gpt3/chatgpt-
| applescript
|
| Apple Notes Liberator creates a copy of the SQLite database and
| then runs queries against that directly to extract the data.
|
| I chose not to do that (despite being all-in on SQLite for
| everything) because I worry about future changes to the software
| baking my script - I figured the AppleScript interface was less
| likely to have breaking changes in the future.
| frizlab wrote:
| A bit out of context, but did you know you can actually write
| OSA scripts using JS? In Script Editor there's a menu where you
| can change the language.
| tzs wrote:
| From your article on how you wrote that:
|
| > I've been stubbornly refusing to learn AppleScript for nearly
| twenty years at this point. Thanks to ChatGPT I can continue
| not to learn it, but I can now use it to solve problems
|
| The Apple scripting stuff can also use JavaScript. It might be
| better to continue not to learn in JavaScript instead of
| AppleScript, if ChatGPT is as adept at Apple scripting in
| JavaScript as it seems to be in AppleScript.
|
| That way if you want to tweak what ChatGPT gives you it will
| probably be easier. With AppleScript both interacting with
| other things and control flow or computation can be weird and
| confusing. With JavaScript at least the control flow and
| computation will likely be more normal.
| aidos wrote:
| I did a bit of AppleScript recently (scripted changing
| between draw and erase in zoom annotations from the buttons
| on my tablet pen). In my reading I got the impression that
| not everything is implemented in js so you might get stuck.
| whatch wrote:
| "Unblock" is the perfect term for the effect of all these GPT
| things on me. Thank you for coining that!
|
| Recently I wanted to have something like that but for Safari
| Reading list. The main difference is I tried to find a solution
| not for one time liberation, but for continuous 2-way exchange
| with some web service. It appears you HAVE to run everything
| locally for that. I like "privacy", but I like having options
| even more.
|
| By the way it was ChatGPT that helped me to remove this several
| month old item from my TODO list.
| jasonriddle wrote:
| Interesting. I was curious and wanted to achieve something
| similar with voice memos. I used the prompt "Write applescript
| to loop through all of my Apple Voice Memos" and the code it
| produced couldn't run in applescript.
| pjot wrote:
| I loved your post about the pelican!
| https://simonwillison.net/2020/May/21/dogsheep-photos/
| jamesu wrote:
| For a long while I just copied everything to an imap server and
| dumped the mailbox, but this looks like an even better solution.
| kepano wrote:
| I am excited to see more ways to export data out of Apple Notes.
| Long term it would be amazing to see a solution that allows more
| interoperability with other apps.
|
| Depending on what you are looking to achieve there are a few ways
| to export Apple Notes to Markdown files, but it's only a one-way
| out solution.
|
| From the Obsidian forum: https://forum.obsidian.md/t/import-from-
| apple-notes-to-obsid...
| rco8786 wrote:
| Oh neat. I've been using Notes extensively since the beginning of
| this year. Daily journaling. Personal TODO lists, grocery list
| (shared w my wife), and gym/workout tracking.
|
| It works _really well_ and has way more features than people
| realize. If you're looking for like a "Notion-lite", the answer
| might already be on your phone /MacBook.
| podviaznikov wrote:
| instead of extracting notes data as JSON my simple tool allows
| you to liberate your notes by publishing them to the web.
|
| https://montaigne.io/
|
| I use the same amazing library linked in the blog post.
| pjot wrote:
| This is great! How do you account for updates to a note?
|
| I've been building an archiver tool as well to convert notes into
| markdown and post to git using Apple Shortcuts.
|
| The interop between almost all iOS data formats (notes, safari,
| calendar, etc.) the content graph[0] provides is really
| impressive.
|
| [0]: https://support.apple.com/en-
| me/guide/shortcuts/apd4618db957...
| Alifatisk wrote:
| So now we know how to export data from the notes.app,
| messages.app & reminders.app.
|
| What's next?
| CharlesW wrote:
| At https://support.apple.com/en-us/HT204055, Apple provides
| instructions for a bunch of stuff you might want to export:
| * iCloud Drive files * Photos and videos * Mail
| * Contacts * Calendar * Reminders * Safari
| * Notes * Voice Memos
|
| And/or you can use https://privacy.apple.com/, which is
| basically Apple's version of Google Takeout.
| dangoor wrote:
| Cool! Thanks for sharing. Also worth noting that there's a Mac
| app in the app store to export to markdown or html:
|
| https://apps.apple.com/us/app/exporter/id1099120373?mt=12
|
| (I haven't had a need yet, so I haven't tried it)
| voltaireodactyl wrote:
| I have used it several times and it's fantastic. Exports
| attachments into properly named media folders and everything.
___________________________________________________________________
(page generated 2023-03-26 23:00 UTC)