Subj : Clearing Houz update... To : All From : deon Date : Tue Jan 04 2022 05:19 pm Howdy, So as some of you may be aware, I've been (slowly) creating a new mailer to manage FTN mail. I've been chipping away at this for the last few months or more, when I havent been distracted with work, parenting or other activities that have caught my attention. You'll have seen the fruits of my efforts, if you have ever seen a test message in FSX_TST or sent a ping message to the system that I've dubbed "Clearing Houz". I started this as a use case (or more correctly an excuse) to learn some (new to me) technologies that are available, or just to learn a different side of programming that I dont normally do. It's worked, I've learnt some new things that I've been able to apply outside of FTN. These Chrissy holidays have been great. We bought my kids a backyard pool (a temporary above ground pool), and with the awesome weather we've had since Christmas, the kids have been in it daily and for hours at a time. Normally, being the on duty lifeguard is boring, but I've been able to position myself at my daughter's arts desk (with a good view to the pool), and supervise and hack at the same time. Who said men cant do two things at once? So this last week, I decided to swap out MongoDB/postgres and switch over to CockroachDB. I started out using postgres (mostly for "config") because it gave me the option to switch to CockroachDB fairly easily if I chose to do so (and I have) - since CoackroachDB uses the postgres protocols. I also started with Mongo (to store the mail), because it was one of the technologies that I wanted to understand better. I work with clients and peers who talk about it, and this was a great way for me to get some hands on experience with it. I could have kept it, but I have stronger skills with RDBMS' and always had CockroachDB in the back of my mind. And I wanted to try out its geo dispersed setup. So, I've switched over to CockroachDB, to understand it better, and I'm planning on deploying it across multiple servers. One node will be at home, and hopefully another node will be on Avon's shiny new server, and a 3rd one could be somewhere else - I have a spot for it in mind. The reason I've gone CockroachDB (and why I was also playing with Mongo), is to build a highly available hub. Too many times I've seen a hub go offline (sometimes with no apparent reason) with the downstream nodes wondering why things are not working. With a highly available hub, the plan is (if I can get it to work), is provide a facility for downstream nodes to connect to an alternate hub with no disruption to mail. Also, i'm hoping it will enable a replacement to be promoted with little to no effort and no disruption to mail. So, with CockroachDB I should only need 51% of the nodes online for it to continue to answer queries and it seems adding/retiring nodes is pretty easy. I dont now how many nodes it can have in total - but starting with the 3 will hopefully be a good indication that it can deliver on my idea. It's secured by SSL (a plus), so in theory nobody should be able to connect to it unless they have a SSL cert signed by my CA (since it will be on the public internet). For end FTN nodes, my mailer code sits in front of it, and talks both BINKD (not fully 1.1) and EMSI. So Clearing Houz will be down for a little bit - I've written a migration tool to migrate the data out of Mongo into CoachroachDB (about 200,000 mail items - which does seem too many), and then I need to double check that my mailer/tosser is still working. In theory, any mail to it should queue up and if you see a burst of replies, hopefully that's only the responses to what was in the queue, and not to the test messages hidden in the 200,000 that get moved over. Once I've got it stable, I'm going to ask if anybody wants to break it - I'm sure there are things that I still need to fix... ....лоеп --- SBBSecho 3.14-Linux * Origin: I'm playing with ANSI+videotex - wanna play too? (21:2/116) .