[HN Gopher] How I recovered a lost email from my email client's ...
___________________________________________________________________
How I recovered a lost email from my email client's memory
Author : weinzierl
Score : 127 points
Date : 2020-05-03 12:09 UTC (10 hours ago)
(HTM) web link (www.ctrl.blog)
(TXT) w3m dump (www.ctrl.blog)
| lixtra wrote:
| Of course recoverin an email is an innocent disguise.
|
| The same approach works for recovering any secret information
| that people used on a computer that an attacker can access. Of
| course there are plenty of possibilities. But it's eye opening to
| see them in action.
| xfitm3 wrote:
| Yes, encryption keys can persist in memory, too. That's why
| many law enforcement agencies use something like a HotPlug[1] +
| mouse jiggler to keep machines powered on when executing a
| search warrant.
|
| [1] https://www.cru-
| inc.com/products/wiebetech/hotplug_field_kit...
| userbinator wrote:
| If you let an attacker have physical access, it's game over
| anyway.
| WrtCdEvrydy wrote:
| If you don't want to pay for HotPlug, you can also grab a
| full memory dump using FTK Imager or Belkasoft...
| segfaultbuserr wrote:
| It's why operating systems should implement a lockdown
| option to restrict users from performing arbitrary access
| to memory or kernel, even if the user is root. I mentioned
| before that, on one of my computer, I completely disabled
| dynamic kernel modules, hotpatching, /dev/mem, no ptrace()
| to arbitrary process, etc., making it difficult for root to
| do any low-level access to memory or kernel. I also enabled
| IOMMU, it isolates the address spaces of different hardware
| from each other, so no external hardware cannot have
| arbitrary RAM access via DMA, hardware-based memory
| capturer won't work. The only way to attack is either an
| 0day or a cold-boot attack, the 0day threat can be reduced
| by using a security-minded kernel, like PaX/grsec (not
| available to the public anymore), OpenBSD, or HardenedBSD.
| As for cold-boot attack, future hardware may support full
| memory encryption [0] at the hardware level and fix this
| vulnerability. Mouse jiggler is a problem, but USB
| firewalls already exist [1], if proper policies is enforced
| by the firewall, unauthorized hardware cannot register as
| an input device.
|
| There may be still some exploits, especially when you
| consider that Linux kernel is not designed with security as
| its first priority, and over the last 20 years a lot of
| black magic has been developed to insert bad things into
| the kernel, but at least doing the countermeasures I
| mentioned will make it difficult. Hence, it's impossible to
| do any low-level changing or debugging on the system
| without rebooting it - which will immediately revert the
| system back to a "at rest" state, and triggers full-disk
| encryption. Other people may choose to do the opposite,
| it's a tradeoff between uptime and security.
|
| Unfortunately, any attempt to introduce such a lockdown
| will be accused of being an evil technology that enables
| DRM. However, ultimately, the question is _not_ whether a
| computer is locked down, but who is in control of the
| computer and it 's locked down to protect whom.
|
| [0] Don't confuse "memory scrambling" and "memory
| encryption". The vast majority of PCs today already use
| memory scrambling - the memory controller will "scramble"
| the data in RAM to a seemingly-random pattern using a
| Linear Feedback Shift Register, but it's done for
| electrical considerations - if there are too many 1s or 0s
| in a row, excessive current spike (di/dt) is produced, and
| it reduces signal integrity and creating excessive
| electromagnetic interference - LFSR-based scrambling is not
| for cryptography purposes and trivial to decode. On the
| other hand, memory encryption is a true solution that
| provides cryptographic protection to the RAM, and many
| hardware vendors have roadmap to implement it. Currently,
| it seems that there are two types, the first type is a
| "full memory encryption" - protecting RAM from physical
| access, the second type is "per-application memory
| encryption", which allows an application to request a
| segment of encrypted memory with an unique key - protect
| sensitive data of one application from accidental access by
| other programs. Both are helpful.
|
| [1] https://lwn.net/Articles/738306/
| weinzierl wrote:
| Yes, eye opening. Non IT folks often believe they are safe
| because no one will find their secrets in a fast sea of
| information anyway. Nothing can be farther from truth. Most of
| the time key material and other secrets can be extracted
| automatically using widely and freely available tools.
| lucb1e wrote:
| This reminded me I was still going to report this bug, but
| anyone else should feel free to do so before me: gpg-agent
| stores your password/-phrase in plain text in memory
| indefinitely. You can clear the cache with some command and
| they also expire after some time, and gpg-agent will pretend to
| have forgotten them and prompt you again for the password, but
| the memory that contains the password is not overwritten and
| you can still dump the process' memory and retrieve it. Email
| contents seem relatively benign by comparison...
| upofadown wrote:
| That's huge and definitely should be reported. Getting rid of
| the private key is one of gpg-agents primary jobs.
| [deleted]
| dredmorbius wrote:
| I used to (when this was still possible) dump /proc/memory (or
| kmem?) to file and rummage through it looking for partially
| composed website submissions when Netscape decided to eat itself,
| back in the 1990s. Remarkably successsful.
| mellow2020 wrote:
| Using Process Explorer at least, it's still just a right click
| on any process away.
| haddr wrote:
| I was using Evolution on daily basis at work (around 2015-17) but
| I switched quickly to Thunderbird due to stability issues. I was
| using Evolution primarily for its support for Exchange Server,
| but it wasn't very stable at the time. On the other hand, the
| same was possible in Thunderbird through a very solid proprietary
| plugin (exQuilla).
| weinzierl wrote:
| > There are specialized tools you can use to analyze this data
| blob.
|
| These are called _file carving_ tools and two better known ones
| are _foremost_ and it 's successor _scalpel_ [1].
|
| [1] https://github.com/sleuthkit/scalpel
| dr_zoidberg wrote:
| Scalpel, as good as it was back in its time, sadly has been
| stalled. Carrier and/or the folks in charge of The Sleuth Kit
| have taken it into their github repo[0] but there haven't been
| commits for ~7 years now.
|
| I did a thesis on file carving some 10 years ago, and scalpels
| ideas where very good back then. Photorec[1], however, has been
| the gold standard for a long time on (open source) file
| carving. It can handle text based formats way better (scalpel
| is severely limited in this aspect due to the "header/footer"
| paradigm), and is a wonder with stream based formats (that can
| have boundaries on the bit level).
|
| And it's not because they authors weren't good[2], I think what
| mainly happened is that they didn't have the time to keep
| maintaining the software they created (I know that has happened
| to me more than once).
|
| There are also some commercial file carving tools, though most
| are aimed at having better integration with forensics software
| (like Encase, FTK, Oxygen, etc) or automate parts of the
| process, like document analysis. Still, if you just want to
| compare them by their ability to recover files, I'm pretty sure
| Photorec makes it to the top.
|
| [0] https://github.com/sleuthkit/scalpel
|
| [1] https://www.cgsecurity.org/wiki/TestDisk_Download (PhotoRec
| is part of TestDisk)
|
| [2] They're some of the best in the field of digital forensics
| weinzierl wrote:
| Good to know! I always had ignored Photorec because I thought
| it is only for image formats.
|
| To add to your list of options there is also YARA when used
| with appropriate rules. I don't know how it stacks up against
| specialized tools though.
| dr_zoidberg wrote:
| Photorec supports a crazy ammount of file types (about 400
| I think, but since they keeep adding it may well be over).
| Fun thing: Diablo II savefiles (and other games!) are
| carve-able with Photorec.
|
| And it can also handle fragmentation (though I haven't
| tested the later versions to see how strong that is).
| WrtCdEvrydy wrote:
| You can also manually carve Photorec using the qt ui.
| [deleted]
| thulecitizen wrote:
| I misread this as: "I recovered a lost email from my client's
| memory". It made me think of the Black Mirror episode
| 'Crocodile', and I was quite amazed. Then I saw 'email' client...
| haha
| d2wa wrote:
| Author here. That was actually the working title up until two
| minutes before publishing.
| jaclaz wrote:
| Yep, and the title, once re-parsed is fine, but I also had -
| initially - the wrong impression, it flashed before me how
| you hypnotized your client (customer)and managed to retrieve
| from his/her memory the contents of an e-mail message he/she
| ddn't rememeber anything about.
| d2wa wrote:
| That's pretty much exactly how I thought the previous title
| might have been misunderstood!
| thedanbob wrote:
| Does anyone know of a good Linux email client that isn't crippled
| by show-stopping bugs of this sort? I used Mailspring for a
| while, which has a nice modern interface, but quit after
| discovering that my drafts were only saved locally, not on the
| server. This has been an open bug for at least two years.
|
| In the end I've always fallen back to Thunderbird as the least
| bad option.
| petepete wrote:
| Have you tried Geary? I helped a colleague get set up with
| ElementaryOS a while back and it looked really nice and,
| compared to Evolution, was much lighter and faster.
|
| I haven't used it in anger though, I tend to stick to email in
| the browser these days.
| thedanbob wrote:
| I did use Geary for a while, I can't remember now why I
| stopped. I appreciated how lightweight it was but I think it
| was just a little _too_ lightweight, missing one or two
| features that I 'd rather put up with thunderbird than live
| without.
| brnt wrote:
| I live in the KDE ecosysteem so I gave Kmail a serious shot,
| but it is such a pain to setup with multiple accounts that I
| gave up. It is very easy to misconfigure sth that I can't think
| of being a use case anybody needs, but then things like reply
| to all by default were removed because it ahem a certain group
| of FLOSS Devs is against such shenanigans. Pity, because apart
| from these issues I really like this mua.
|
| Thunderbird is and will be for some time the only realistic
| least bad option. Once they replaced xul, perhaps updating its
| visuals will get easier.
| axaxs wrote:
| Past security issues aside, Bluemail.
| trombonechamp wrote:
| I highly recommend claws-mail. It is lightweight, very stable,
| and easily scriptable with Python.
| weinzierl wrote:
| I'm afraid the genre is pretty much dead as everyone seems to
| do email in the browser nowadays. Apart from TUI MUAs like Pine
| and Mutt - that may not be everyone's cup of tea - I found
| Sylpheed quite useful for a while. It is not as bloated as
| Evolution or Thunderbird but still has all the useful features
| I need and like. Its development seems to have slowed down
| though, so I don't know if it has any future...
| dhosek wrote:
| Man, hearing this is a reminder of why I don't use Linux as
| my desktop OS and likely never will. My default mix of
| applications has shifted over the last 30+ years, but e-mail
| has always been one of the most fundamental apps for me,
| going back to the VM/CMS mail client (and before that VMS's
| mail program which was comparatively a disaster).
| throwaway2048 wrote:
| I'm not sure why you think this issue is specific to linux,
| it isn't.
| fsckboy wrote:
| seamonkey continues the Netscape Communicator legacy of having
| a browser and MUA together in one product. I use it for doing
| IMAP and POP actions to reorganize and back up my mail. I don't
| compose email in it much, but it does give you the choice of
| HTML or plaintext.
|
| also, emacs can be used as an email client
| brnt wrote:
| Doesnt seamonkey basically bundle an old version of
| Thunderbird? Which itself, while very functional, isn't
| exactly slick and modern ;)
| warent wrote:
| Emulate macOS in a VM and download Spark ;) I recently
| discovered this email client and it completely revolutionized
| how I do email.
| hollander wrote:
| A 5GB limit, and the $6/month (?!?!) has a 10GB limit??????
| warent wrote:
| For their file storage/sharing service unrelated to
| emailing.
| leephillips wrote:
| I've used several, on linux and MacOS, and gmail. Now I use
| mutt, and am happy. Nothing else comes close. But the author
| writes that he needs integration with cardDAV, or something. I
| don't know if mutt can do that.
| joyj2nd wrote:
| Because of some reasons I use evolution and thunderbird. The
| search function in Evolution is abysmal.
|
| May try out InScribe
| therealmarv wrote:
| nano ftw.
|
| Modern web email clients have this since a long time and Gmail
| never crashed internally for me ;)
| annoyingnoob wrote:
| Something tells me that if you are bent on using FOSS and/or
| care about privacy then Gmail isn't going work.
|
| Can't argue that there must be easier ways to handle email.
| lucb1e wrote:
| > Gmail never crashed internally for me
|
| Some google services seem to be crashing the tab in my Firefox
| (but me only, since this bug has been there for many months now
| and I reported the crash with URL a bunch of times), I remember
| street view for sure but recently there was another, don't
| remember the name. (I don't use google services that frequently
| aside from pulling youtube content.)
|
| Anyway, point is, browsers aren't infallible either and google
| is known to make their software work only in google-branded
| browsers. I'm not sure that a really stupid bug in
| $someSoftware is a good argument for why we should move
| everything into the browser, or a Google product in particular
| (why not bring up roundcube or runbox or something?).
| paulpauper wrote:
| Yup..this is also how the fbi recovers stuff too. After Ross
| Ulbright was caught they did this to his laptop.
| cranekam wrote:
| > Evolution [..] has a bad data loss bug. It sometimes deletes
| the email body text in the compose window after changing the
| signature [..] it has bitten me about twice a month for the last
| two years.
|
| and
|
| > I might have been able to partially recover the message from
| the Draft folder if I'd retained my cool and acted immediately.
| It had been overwritten by an empty message instead. I must look
| into versioning my email draft folder at a later time.
|
| This person has a much greater tolerance for shitty software than
| I do. I'm certainly not a perfectionist and appreciate that
| almost all software has bugs, but come on! Arbitrarily deleting
| draft emails twice a month _for two years_? Requiring convoluted
| versioned draft folders to work around this glaring issue? Why
| are they punishing themselves like this?! They must find
| something really awesome about Evolution to deal with this level
| of annoying.
| raheemm wrote:
| It reminds me that we humans are capable of going to great
| lengths to resist change, even when it creates shitty outcomes.
| annoyingnoob wrote:
| Had the same thought. Did this person consider using different
| software? Why hurt yourself like that?
|
| Its no wonder that it will never be the year of the Linux
| Desktop.
| b212 wrote:
| I love Linux for what it is, for the whole idea, being open,
| free and "democratic" but I tried using it while my Macbok
| was in service and oh boy, it's like having a Hackintosh 10
| years ago, for work it was bearable but for "personal use"?
| I'd rather pay 5x more for something that "just works". And I
| did.
| d2wa wrote:
| The main difference for me is that there are more ways to
| rescue, recover, fix, and work-around issues. I'm typing
| this on macOS now. I run into about the same amount of
| issues on macOS as I do on Linux. The big difference is
| that I'm just f*ed on macOS whereas Linux leaves me with
| multiple paths to save myself out of troublesome
| situations.
| iso1210 wrote:
| And that's exactly why I run linux, because it "just
| works", and has done for 20 years.
| OJFord wrote:
| My personal machine runs Linux because my professional
| experience is that macOS doesn't 'just work', and Linux is
| easier and less opaque to fix.
|
| I don't want to trawl through 'have you tried turning it
| off and on again' on Apple support forums, I want to find
| the text-based config solution in the Arch wiki, a man
| page, or unix.SE.
|
| (Yes that order, not `man` first, typically. 'Sue me'.
| Other than for executables I don't find it that
| 'discoverable' for what's available or might be relevant. I
| only recently discovered `man [7] hier` - but how was I
| supposed to know the page is called 'hier' (for hierarchy
| of course, but even that)? I got it from a unix.SE answer.)
| d2wa wrote:
| Person/author here. Yes, I have considered other email
| clients. Evolution is the least bad of the available options
| and still sees active development. Most other clients don't
| support CardDAV or LDAP so I can't sync my address book. An
| email client is mostly useless without the contact details of
| the people want to contact.
|
| Newscommers to the email client market have all been
| proprietary subscription-based middleware instead of actual
| email clients. The market has been standing mostly still for
| the last 10 years.
| the_pwner224 wrote:
| As others have mentioned, Thunderbird is the least worst
| option (in our opinions :). I use it with CalDAV/CardDAV
| synced to my Nextcloud instance.
|
| The three good options are KMail, Thunderbird, and
| Evolution. Everything else is CLI or lacks features like
| DAV.
|
| KMail and Evolution both bring in the entire KDE/Gnome PIM
| suite with daemons and other programs, making them not
| great unless you are using Gnome or KDE - as another HN
| commenter said about Java web applications, you wanted a
| banana and instead got the entire jungle and an angry
| gorilla. But they do integrate very well. Thunderbird is
| standalone.
|
| I tried Evolution for a week, about a month ago, and have
| used KMail and Thunderbird a lot.
|
| KMail is fully-featured, with native support for
| CalDAV/CardDAV and 'send later.' But it's incredibly
| complex and easy to misconfigure. When upgrading to a new
| computer recently, I tried doing an export => import to
| transfer data, but it apparently permanently borked the
| KMail installation on the new computer. Tried
| uninstalling/reinstalling and deleting _all_ KDEPIM-related
| files in ~, and it still would not work... Even on my old
| computer I still kept Thunderbird installed along with
| KMail sometimes didn 't work properly.
|
| Thunderbird is very straightforward to use and is quite
| stable. I use the TbSync add-on (Thunderbird has an
| official add-on repository like Chrome/Firefox). You'll
| also need the 'Provider for CalDAV & CardDAV', which adds
| that functionality to TbSync but is distributed as a
| separate add on.
|
| Set up cal/carddav account, then go to calendar, right
| click the toolbar => Customize, and drag the 'Synchronize'
| button onto the toolbar so you can force a synchronization
| if needed (in addition to the timer-based background sync).
|
| There's also a 'send later' add-on available. Aside from
| that, I only have a few minor issues with Thunderbird:
|
| To switch between HTML and plain-text emails, you need to
| shift-click the 'compose new email' button. Can't switch in
| the middle of composing; you'll need to make a new email
| and copy-paste over. Changing the default from HTML to
| plain text requires going into about:config. And you can't
| enable/disable text wrapping on the fly for plain text
| emails; it's an about:config pref.
|
| With Gmail accounts, it incorrectly lists the
| inbox/sent/etc. folders in a subfolder of the account
| (functions properly, but ugly). You have to right click the
| account, go to Settings => Server Settings => Advanced and
| set the IMAP Server Directory to '[Gmail]'.
|
| Finally, TB has its own Spam filtering mechanism. You can't
| fully disable it. Even if you go to account settings and
| disable junk filtering for that account, it still shows a
| button to mark as junk and overrides the J key for that.
| Annoying if you are used to vim controls and press J often.
| Also J has the homing nub on the keycap so I like pressing
| it a lot...
|
| KMail has all of this stuff built in / fixed, but is just
| way too complex and brittle.
|
| Actually, after seeing your article's screenshot of
| Evolution with KDE titlebars, and a person replying
| suggesting that Evolution is good for this purpose, I'm
| trying it out on KDE. Never thought that would happen! I
| don't use signatures so hopefully this bug doesn't affect
| me... I did have to separately install the gnome-keyring
| package to get it to remember the IMAP/SMTP password but
| aside from that it appears to work fine.
| d2wa wrote:
| (replying where I have something semi-intelligent to
| add.)
|
| > Thunderbird is the least worst option (in our opinions
| :).
|
| I use TB when I use macOS. TB does weird thing to plain-
| text email formatting, though. E.g. it sometimes refuses
| to let me delete lines that contain "> ", and it
| sometimes freaks out when I try to insert al line break
| in a section of quoted text. (I reply inline, like a
| civilized emailer.)
|
| > I tried Evolution for a week, about a month ago, and
| have used KMail and Thunderbird a lot.
|
| I've used all three for years. KMail would be my
| preferred option if it was way more stable and less
| buggy. It has great features and I feel at home in it.
| But it works way less reliably than Evolution.
|
| The version of KMail shipping on Flathub doesn't even
| start. --and that's when it's running in a sandboxed
| environment that's identical on everyone's systems!
|
| > There's also a 'send later' add-on available.
|
| I know. https://www.ctrl.blog/entry/kmail-
| cve-2017-9604-openpgp.html On a related note, I couldn't
| login to my IMAP account with KMail maybe ten years ago.
| My password back then contained an apostrophe. KMail
| didn't encode it properly and would crash every time it
| tried to submit the password to the server. The bug also
| made it impossible to overwrite the saved password with a
| new one from the UI.
|
| > Actually, after seeing your article's screenshot of
| Evolution with KDE titlebars.
|
| The screenshot is manipulated, see disclaimer at the
| bottom of the article. It's indeed running under Plasma,
| though.
| neltnerb wrote:
| I'm not saying Thunderbird isn't right for you, but
| evolution had all those things and sounds no harder to
| use... this is just one bug. It just reminds me of Eudora
| mail and outlook more I think.
| neltnerb wrote:
| I agree and also use evolution because it feels more
| comfortable and well thought out than the alternatives.
|
| Thunderbird feels wrong for some reason, and webmail
| doesn't let me have ten accounts in one place... I love
| being about to readily move an entire email folder to a
| different imap amount entirely with drag and drop.
|
| I think it mostly just reminds me the most of Eudora so I
| just like it for being familiar.
|
| (I've never encountered this bug, I guess I just don't have
| changing my signature in the workflow.)
| axaxs wrote:
| Did you open a bug? I've opened 3 Evolution bugs over the
| past few years, and the devs are always helpful. They even
| changed some behavior that was annoying to me, which made
| me wonder if I was in fact the only user...
| d2wa wrote:
| https://gitlab.gnome.org/GNOME/evolution/-/issues/905
| Sebb767 wrote:
| > which made me wonder if I was in fact the only user...
|
| Nope, there's at least three of us :)
|
| It's by far my favorite client. Not only is the UI nice
| (for some reason I can't stand Thunderbirds UI), but it
| works well with integrations and multiple accounts.
| Wowfunhappy wrote:
| Have you looked at Geary?
|
| I haven't personally used it, I just know that it shares a
| lineage (?) with the elementaryOS project, and they seem to
| be making great stuff.
| chungy wrote:
| Alternative take: I've been using Evolution as my main email
| client since 2008-ish, mostly on Arch and Debian systems. I
| have never encountered this bug, and I'm willing to bet most
| people have not either.
|
| The bug probably exists, and maybe with the magic set of
| configuration options, I could make it trigger too. But bugs
| can be finicky like that -- developers certainly don't like
| them, and it'll probably vanish pretty fast if they are able to
| reproduce it.
| Joeri wrote:
| Maybe they have an exchange mailbox? If you want exchange
| support on linux with full syncing of mail, calendar and tasks
| there are two options I know of: evolution and hiri. And hiri
| is paid abandonware. I've set up both, both are not good.
| Lately I've been using outlook web access, which is still bad,
| just not as bad.
|
| Really I guess I miss outlook for windows. There I've said it.
| Judge me if you will. Its search feature is broken, but
| everything else worked well.
| orf wrote:
| You can use Thunderbird with a paid add on. It's well worth
| it, because evolution is just _bad_.
| d2wa wrote:
| Author here. My problem is related. However, I have a
| IMAP/CardDAV open-standards setup instead of
| ActiveSync/Exchange.
| deadbunny wrote:
| Option 3: davmail[1] acts as a translator between Microsoft's
| proprietary protocols and open protocols.
|
| 1. http://davmail.sourceforge.net/
| yipbub wrote:
| Not quite arbitrary.
|
| He new that he could avoid it by not changing the signature
| afterwards.
| dhosek wrote:
| And yet it still bit him twice a month? After the second time
| that happened, I would have deleted that program and smashed
| the hard drive to keep it from returning.
| renewiltord wrote:
| I tolerated a lot of crap from Evolution to get sync to my
| Gnome calendar stuff and notifications. Eventually it was too
| much, though. Sad. Overall I liked the software but data loss
| is hard to stomach.
| Stierlitz wrote:
| I've noticed the same with web-able apps, you spend time typing-
| up some missive and then it freezes and refreshing the page loses
| it all. Could linux write everything to a file every 30 seconds.
| A bit like a keylogger, only you know it is there.
| code_duck wrote:
| There used to be a Firefox add-on called Lazarus that did this.
|
| https://www.pcworld.com/article/227948/Firefox.html
|
| From Tom's Hardware:
|
| "Lazarus: Form Recovery is a free downloadable Add-On for the
| Firefox web browser that automatically saves everything you
| type into forms of web pages you visit.
|
| With Lazarus: Form Recovery, you will never lose what you write
| after a crash the browser or other technical problems. In the
| case when a problem, simply right click and select "recover
| form" to retrieve data previously typed."
|
| However, that was for web forms, not an email client or other
| applications.
| teddyh wrote:
| I've found that this can replace Lazarus in modern Firefox:
|
| https://addons.mozilla.org/en-US/firefox/addon/form-
| history-...
| OrgNet wrote:
| > Could linux write everything to a file every 30 seconds. A
| bit like a keylogger
|
| yes, but probably not a good idea?
| d2wa wrote:
| Yeah, but it would require an enormous amount of fast disk
| space. An actual keylogger would probably be more useful.
___________________________________________________________________
(page generated 2020-05-03 23:00 UTC)