[HN Gopher] My Overkill Home Network - Complete Details 2023
___________________________________________________________________
My Overkill Home Network - Complete Details 2023
Author : monstermunch
Score : 68 points
Date : 2023-08-10 21:05 UTC (1 hours ago)
(HTM) web link (blog.networkprofile.org)
(TXT) w3m dump (blog.networkprofile.org)
| m3kw9 wrote:
| I could do all that with a single ddwrt router behind my tv
| pm2222 wrote:
| I didn't read in details. Why esxi is not incorporating whatever
| that's on the rpis?
| greggyb wrote:
| The Pis are using GPS receivers to serve as NTP servers on the
| local network. The specific appear to use GPIO pins. You would
| not be able to wire these to the ESXI machines as readily. You
| typically also want time-sensitive workloads running on metal,
| as there are all sorts of potential complications with
| virtualized time.
|
| I am sure there are PCIe solutions that could easily go into a
| more standard form-factor machine, but perhaps these are more
| expensive or less readily available; this is just conjecture.
| The ESXi machines have minimal PCIe connectivity. Perhaps such
| a card could be put into one of the Supermicro chassis.
| Dagger2 wrote:
| All that and no mention of IPv6? At least get the basics down
| first.
| hqsolomo wrote:
| Lol I get sym 1G/1G from my ISP and a free static IP address
| for less than Comcrash offers for their 25/10 service in my
| area...
|
| The catch? _No IPv6 support whatsoever_
| systems_glitch wrote:
| Yeah, even with Comcast/Xfinity I run a HE.net tunnel for
| IPv6, just so I can have truly static allocations and not
| goof around with their DHCPv6 prefix request system.
| jonathantf2 wrote:
| Probably not supplied, I've got no IPv6 on any connection I
| have access to other than a few Azure VMs (and it's a PITA to
| set up so we haven't bothered)
| m463 wrote:
| Do you need ipv6 on a home network?
|
| Honestly I just use ipv4 and turn off ipv6 everywhere. Then I
| have just one configuration and one set of firewall rules (in
| and out).
| systems_glitch wrote:
| To say "couldn't live without it" would be an exaggeration,
| but we do use it extensively for work, and having it at home
| makes WFH much easier. End-to-end connectivity, like in the
| old days, is very nice!
| lxe wrote:
| I'm curious what the total cost of all this is. My reservation of
| rack mounted setup has been cost of hardware.
| dheera wrote:
| It can be reasonable if you wait for good 2nd hand deals on
| equipment, especially from startups shutting down.
|
| My 10G switch with PoE and 4 SFP+ ports cost only $100, for
| example, while many new products with similar specs cost $600+.
| vdfs wrote:
| Cost of electricity is an other factor to consider
| wredue wrote:
| It's a black hole for your wallet. Once you start. You can't
| stop.
|
| But I also now have a bunch of electrician tools for running
| wires, some private cameras that aren't google or Amazon spying
| on everyone that walks by, and a network that isn't bogged down
| by a couple people watching Netflix.
|
| And I have plans for MORE.
| KnobbleMcKnees wrote:
| What cameras do you use?
| hqsolomo wrote:
| If you're crafty you can work around this- three of my servers
| (2 NAS boxes and a Proxmox box) were custom built from
| commodity hardware. I literally bought plain ol uATX and ITX
| decommed business gear from my local uni surplus store and a
| P-Link chassis for each used from eBay. I spent more on drives
| than I did machines (Y'ALL WANNA KNOW HOW MUCH IT COSTS TO BUY
| 16TB WORTH OF SSDS FOR ZFS MIRRORS? TOO. DAMN. MUCH.)
|
| You really gotta do some due diligence to make sure you're not
| buying lemon parts but it's very possible to get a beefy
| homelab at fair prices!
| physhster wrote:
| ...and I thought mine was overkill! Great job!
|
| You might however want to read about the grocer's apostrophe:
| https://www.grammar-monster.com/lessons/apostrophe_error_wit...
| s09dfhks wrote:
| I'll take that Lenovo M73 Tiny off your hands ;)
| thecosas wrote:
| "This is also how you are probably getting to this blog, which is
| hosted at home."
|
| Curious to know how hitting HN impacted your setup (if at all)!
| tdhz77 wrote:
| Reading this post makes me happy we are in the world of cloud
| providers, but realize they don't magically work. People build
| them. I'm glad I don't.
| wredue wrote:
| It's really not all that complicated. Although I still haven't
| figured out if there's some secret way to properly creating
| working Ethernet cables.
|
| I get blisters on blisters on my finger tips when making lots
| of cables.
| LouisvilleGeek wrote:
| Really nice setup! the pfSense part made me smile!
| m463 wrote:
| as long as you're good that pfsense is not open and phones
| home.
| systems_glitch wrote:
| Plus the domain squatting with hitler and goatse thing... :/
| hqsolomo wrote:
| I'm sorry- when did they start doing this? I've been mulling
| making the switch to OPNSense. If pfSense is phoning home
| aside from the update check I might have a busy weekend ahead
| of me
| comprev wrote:
| Have a quick search about the bad activity undertaken by
| pfSense against OPNSense and you'll soon change your
| opinion on the company.
| hqsolomo wrote:
| Welp, that was... A thing they did. Reminds me of middle
| school antics, except these are grown folks. Guess it's
| time to jump ship after all!
| woleium wrote:
| and that BSD still cannot do line speed gigabit on an Intel
| gigabit nic.
| hqsolomo wrote:
| I dunno if this is entirely true- my current pfSense router
| gets as close to line speed as I'll ever be able to get
| with my setup (~900 both ways to a test server in Chicago
| and on LAN) and I'm running an (overkill) i5 in my generic
| Chinese minipc. It's possible that I don't have Intel NICs
| but I swear I do
|
| Got anything more I can read? Dunno if I'm just blind but I
| couldn't find anything on this. I'd like to learn more!
| systems_glitch wrote:
| [citation needed]
|
| We had no problem routing gigabit on an Intel Atom D525
| years and years ago at a previous job. Consumer Mini-ITX
| board, onboard gigabit NIC, and an Intel gigabit NIC in the
| one expansion slot. It did require minor tuning, but
| nothing that couldn't be done thru the web UI.
|
| Everything I have to manage runs plain OpenBSD managed with
| Ansible now, so I don't know what the current state of
| pfSense/OPNsense throughput is.
| whalesalad wrote:
| I keep meaning to produce a post like this. My setup is not
| nearly as pretty though. Love to see the cannabis grow operation
| exhaust fan, haha. Very well done!
| f-securus wrote:
| I didn't see a filter on the exhaust fan. He is so thorough I'm
| sure he thought of it but I think he needs a filter to keep the
| fan running long term unless he is filtering the air into the
| room and that is good enough?
| jolux wrote:
| I just bought a Ubiquiti Dream Machine SE along with a U6
| Enterprise a few weeks ago and so far it's my favorite tech
| purchase ever. The management interface is years if not decades
| ahead of everything else I've used before.
|
| My only complaint is that there's no public API and thus no
| official Terraform providers.
| syntaxing wrote:
| I actually moved away from ubiquiti stuff to OPNsense + TP link
| AP. The firewalls rules on OPNsense makes a ton more sense and
| the plugins are pretty awesome.
| tw04 wrote:
| I would take Ruckus Unleashed over unifi all day long.
| Ubiquiti, unfortunately, feels like everything is constantly in
| beta, both hardware and software. Wait until they release a UDM
| SE v2 and abandon firmware on the UDM SE.
|
| The UDM in particular is a masterclass in how to upset all your
| customers. (coming from a previous all-in UBNT customer that
| had a first gen UDM Pro).
|
| Ebay Ruckus + OPNsense and my network has never been more
| stable and performant.
| whalesalad wrote:
| The Unifi software is pretty incredible. I am not using their
| router though (well... an ER-4 but it does not share the same
| management mechanism) so I am missing out on a lot of the
| goodies.
| donutshop wrote:
| ERX here. The EdgeMax line is still rock solid and have
| incredible value for the price.
| stn8188 wrote:
| Wow, and my wife says my network is complicated! :)
|
| In all seriousness, thanks for sharing, this is really
| incredible. I see a few similarities (fellow Harbor Freight
| shopper, ADS-B receiver)... but I took the mostly lazy way out
| and just use the TP Link Omada router, controller and access
| points. Works great for well over 50 wireless clients that we had
| at a recent BBQ. I particularly love your note about encrypted
| LoRa networks at the bottom there, I'll be interested in a
| follow-up on that topic. Thanks again!
| xpe wrote:
| Exactly. Those 50 wireless clients must be fed Internet while
| their 50 subjugated humans eat BBQ and prioritize device
| interactions over human ones. (I'm exaggerating of course! At
| least you are having gatherings -- better than many of us!)
| godman_8 wrote:
| Nice setup! I have a very similar Homelab minus the Generac (I
| regret not getting one before inflation kicked in, especially
| since I already have LNG to the home.)
|
| My only recommendation would be switching your virtualization
| over to Proxmox (LXC / KVM) and setting up an HA cluster with
| Ceph and MLAG. It's relatively easy and free and will give you a
| lot more features than plain ESXi and even free vSphere/vCenter.
| bazmattaz wrote:
| This is incredible. I still can't believe some people get 1gb
| symmetrical in their home. In the UK I'm stuck with 70mbs down
| and 6mbps up. Pitiful
| toast0 wrote:
| I had 1g symmetric in the bay area (thanks at&t) and it was
| nice, but am now around your speeds (85m/13m) and it's clearly
| worse, but not really terrible. Certainly not terrible enough
| to pay $50k+ install to get munifiber, even though I'd enjoy it
| a lot. Maybe if one of the ISPs on munifiber starts offering
| 10g to residences. Not that I need it, but it'd be fun.
| rjsw wrote:
| I'm in the UK, getting 500MB symmetrical installed tomorrow,
| could have ordered 900MB for PS2 per month more.
| tdfirth wrote:
| I'm in the UK (Oxfordshire) and have a 1gb symmetrical
| connection at home. It's provided by Gigaclear - there's a
| handful of other similar operators that do fibre in more
| "rural" areas. It costs PS79 a month, so it's not cheap to be
| honest, but I love it.
| jonathantf2 wrote:
| Ring em up and tell them you're leaving, they'll drop you
| down to the new customer pricing. I've got the 1G up and down
| for something like PS38 a month now?
| PaulKeeble wrote:
| Should hopefully change in the next few years, about 54% of
| houses have fibre to the property and the plan is over the next
| five years to expand that to nearly 100%. I have had 100/1000
| for a while now but symmetric is still a rare product that only
| smaller competing fibre companies are rolling out.
| hqsolomo wrote:
| Nice and clean, great work!
|
| If you don't mind me asking, does your energy bill take a huge
| blow because of this? I had a modest homelab set up and had to
| start shutting things off due to how much it costs to keep it
| running
|
| I apologize if I missed this info in the blog!
| pm2222 wrote:
| Perhaps there's solar.
| hqsolomo wrote:
| That would definitely be a big help, lol. I also imagine part
| of my problem is my house wiring- the previous owner had the
| place reno'd in '96 and clearly got the landlord special
| dheera wrote:
| Having a rack alone doesn't consume anything, it's what you put
| on the rack.
|
| I have a rack setup and most of the time it consumes around 200
| watts during the daytime and 100 watts at night, but can spike
| upto 600 watts if I put a heavy CPU+GPU load on it.
|
| I also put my desktop into suspend at night, something which I
| think a lot more people with desktops could do. Don't run 24/7
| services (e.g. Home Assistant) on your massive desktop with an
| i7/i9 and a GPU. Run that stuff on a NUC or Pi4 or anything
| that has low power consumption. Then turn your desktop on only
| when you're actually using it.
| hqsolomo wrote:
| I get that but the blog discussed a pretty beefy setup. My
| whole rack still used less power than my gaming PC at load
| but after doing the math I ultimately saved more money by
| going serverless for my apps and dumping (non-critical) data
| into a B2 bucket on paper. In reality I just started shutting
| things off and only turn them on when I need them.
|
| I'm curious as to what others are doing to save costs if
| anything. I love the hobby but we're in a recession, lol!
___________________________________________________________________
(page generated 2023-08-10 23:00 UTC)