[HN Gopher] Imaging mounted disk volumes under duress
___________________________________________________________________
Imaging mounted disk volumes under duress
Author : todsacerdoti
Score : 71 points
Date : 2021-09-20 12:07 UTC (1 days ago)
(HTM) web link (blog.benjojo.co.uk)
(TXT) w3m dump (blog.benjojo.co.uk)
| benmmurphy wrote:
| I don't understand how this works. If the trace API gave you the
| data in the writes then I can see how it works. You run your
| copy, then just run the writes over your copy and you have a
| snapshot that is consistent at some point in time.
|
| However, if you just have a page modification flag then if you
| try and recopy the data that has been modified then it seems like
| you could just end up in a loop where you make no progress
| because the disk is continually being modified. If none of the
| modified pages have been modified during your second pass then
| everything is ok but if some have been modified then that could
| invalidate other pages that have weren't modified during the
| first pass but we're modified during the second pass.
| rwmj wrote:
| It's a shame that Linux doesn't have APIs as comprehensive as
| Windows VSS. One of the several features is something called
| Volume Shadow Copy Service which lets you take backups of block
| devices which are in use. It's kind of similar to this but more
| supported and I think it also interacts with user processes like
| databases, quiescing them so you can take a consistent snapshot.
|
| [Also if you are playing with custom block devices in Linux --
| mounted or otherwise but not /dev/sda --
| https://gitlab.com/nbdkit/nbdkit
| https://libguestfs.org/nbdkit.1.html One filter we have which is
| kind of similar to blktrace is called the nbdkit-stats-filter
| https://libguestfs.org/nbdkit-stats-filter.1.html]
| Rygian wrote:
| I've been doing my full disk backups with zfs snapshots for a
| while now and zfs sending them offsite. Granted it's a freebsd
| box, but same tools should be available on Linux.
| makr17 wrote:
| Definitely possible on Linux. In the not distant past I had a
| large on-prem PostgreSQL DB that had a hot standby that we
| used for backups. Since ZFS snapshots are atomic you can just
| snapshot the dataset under the DB without worrying about
| quiescing the database. And then zfs send backups to files
| that are then backed up offsite.
|
| If you ever cared it's easy enough to clone one of those
| snapshots and bring it up as another DB instance to poke
| around at how things used to be...
| emj wrote:
| That can't be supported by many databases, it must be insanely
| difficult to get that right, is it really used?
| namibj wrote:
| It just needs to listen to explicit fsync commands and choose
| a matching point-in-time. Beyond that, no, it's not hard as a
| database to tell the OS about moments where a block-level
| disk image would not need repairing, after the OS asks you.
|
| I assume this is to prevent situations that would need time-
| intensive WAL-replay or such, and there it'd only be
| opportunistic with the "sudden power loss" recovery as a
| fall-back path.
| binwiederhier wrote:
| The dattobd [1] block device driver fills this gap. It
| basically implements point in timr like behavior on Linux, with
| hooks similar to the ones that VSS provides. It is used in the
| Datto Linux backup agent in combination with the backup
| appliance [2], but it can be used stand alone as well (see
| readme).
|
| Disclaimer: I'm a software engineer at Datto.
|
| [1] https://github.com/datto/dattobd
|
| [2] https://www.datto.com/products/siris
| mappu wrote:
| This is just the coolest thing in the world, thank you so
| much for open sourcing it.
|
| Have you asked upstream what they think of it, at all?
___________________________________________________________________
(page generated 2021-09-21 23:00 UTC)