REPORTING FOR DELIVERY, SIR

Setting up DKIM and DMARC, as documented in 
2022-09-11Sacrifices_to_the_Email_Gods.txt, does seem to have put 
me in better stead with the email overlords. One thing that has 
quite litterally been building up though are DMARC reports.

Now you don't have to tell other mail servers to email reports to 
you about successful verifications, but it seemed like potentially 
useful information (see whether anyone's successfully spoofing me 
because I stuffed something up), so I did. The trouble is that this 
results in a hail of compressed XML-format reports, the manual 
reading of which is a good way to send yourself cross-eyed.

Obviously DMARC has been around for ages now, so there's sure to be 
some simple command-line tool that you can point at an inbox 
directory and it'll generate a human-readable web or text page from 
it (ignoring the fact that they could have just put that in the 
email in the first place!). Well no, to my amazement most search 
reults were services expecting you to point your DMARC reports at 
them and _they'd_ generate the readable reports - probably for a 
fee. The rest were complicated systems using a database back-end, 
and with all sorts of dependencies so that they could read in mail 
via IMAP etc., plotting all the data into glossy interactive 
graphs. It's incredible - for the sake of some information that 
could easily have been sent as good old-fashioned plain-text lists 
in the first place, I'm looking at paid services, databases, and 
Javascript graphs, just for an easy way to read the damn info.

Eventually, after much web searching, I found this page which shows 
an XSLT stylesheet for converting the XML reports into HTML using 
xsltproc:
https://www.tana.it/sw/dmarc-xsl/

Then I found UUDeview, which allows extracting email attachments 
from files, with an option to output only those attachments with 
specific filename extensions.
http://www.fpx.de/fp/Software/UUDeview/

So with a mailbox (not in MBOX format) that just gets DMARC 
reports, I can do:

# Extract All zip and gz attachments from DMARC report mail 
# directory
uudeview -p "$DMARCDIR"/zip +e .zip -i -m $MAILDIR/*
uudeview -p "$DMARCDIR" +e .gz -i -m $MAILDIR/*

Then decompress everything:

# Decompress XML
gunzip "$DMARCDIR"/*.gz
unzip "$DMARCDIR"/zip/'*.zip' -d "$DMARCDIR"/

Then, turn it into some nice readable HTML (lazily joining the 
separate HTML pages into one because it works in Dillo and is 
quicker than clicking lots of separate links):

# Generate combined HTML report page
for report in `ls -t "$DMARCDIR"/*.xml`
do
  xsltproc "$DMARCDIR"/htm/dmarc.xsl "$DMARCDIR/$report" >> "$DMARCDIR"/htm/DMARC_reports_`date +%F`.htm
done

$DMARCDIR/htm is a password-protected directory served by the 
web server, so then I can easily view the report with Dillo.

But why did I have to work this out for myself? Sure it wasn't 
hard, but only after I found that XSLT stylesheet and UUDeview. If 
everyone running an email server now has to deal with DMARC, then 
surely I should have been tripping over simple guides for reading 
DMARC reports all over the place, instead of just those 
over-complicated solutions. I suppose large volume servers need to 
deal with vastly more data than I do, and that would require a 
slightly smarter solution, but it still feels there's a bit of 
people making work for themselves going on here.

 - The Free Thinker