[HN Gopher] Over 20k servers have their iLO interfaces exposed t...
___________________________________________________________________
Over 20k servers have their iLO interfaces exposed to the internet
Author : caaqil
Score : 85 points
Date : 2022-01-28 18:28 UTC (4 hours ago)
(HTM) web link (isc.sans.edu)
(TXT) w3m dump (isc.sans.edu)
| johnklos wrote:
| I reported a security issue in Supermicro's IPMI implementation
| to them. They dismissed the issue and never fixed it.
|
| These companies don't care if there's no way to stop a remote
| control service (iLo, iDRAC, IPMI) from binding to a motherboard
| ethernet port. In the case of Supermicro, you can't configure via
| the built-in BIOS, there are no jumpers you can use, and you must
| have a full network setup and installed tools to configure a new
| machine. This makes deploying in the field much, much more
| problematic.
|
| If the BIOS battery dies and settings set are lost, the
| motherboard defaults to joining whichever ethernet is active,
| with default credentials. It's incredibly stupid and insecure.
|
| I will never buy Supermicro again, but for the hardware that was
| already bought, I mandated loopback plugs for all IPMI ports so
| the IPMI wouldn't switch to the system ethernet.
| zamadatix wrote:
| Supermicro switched to unique default passwords (same as
| Dell/HPE) a few years back. Silly to not have day one but
| really when you look at any of these they are all a security
| nightmare.
| stingraycharles wrote:
| Honest question: do people actually connect their ipmi / bmc to
| the public internet? Or is it something different that's being
| exposed here?
| mirceal wrote:
| if you want to be really scared read:
|
| http://fish2.com/ipmi/itrain.pdf
|
| all servers in a datacenter have this management interface (iLO
| is just one type).
|
| if the management network these sit on is poorly secured (like
| here) your servers are literally powned.
| danpalmer wrote:
| BMCs are slightly different I think (more powerful!) but at my
| previous workplace we once had a server provided to us by our
| hosting provider that had its BMC exposed on the internet with
| default credentials. They never told us the machine even had one
| (most of our servers from them didn't). We only figured it out
| once we found a Monero miner on the machine.
|
| People on here love promoting dedicated servers over cloud VMs,
| but it's so much easier for this sort of thing to go wrong with
| dedicated hosting.
| vel0city wrote:
| Integrated Lights-Out (iLO) is the HP flavor of baseboard
| management controller (BMC) platform. Dell calls theirs Dell
| Remote Access Controller (DRAC), other vendors have their own
| branding. They all do more or less the same thing.
| mirceal wrote:
| BMC is the hardware, iLO is the interface. you can also have
| other types of interfaces but in the end it's still a BMC.
|
| a chip that sits on the southbridge of the server and has
| management capabilities (power operations and access to the
| underlying os being the 2 big ones).
| throwawayboise wrote:
| The latest generations of servers ship with randomized BMC
| passwords. This was indeed a problem in the past when they
| shipped with credientials such as ADMIN / ADMIN or no password
| at all.
| dnautics wrote:
| "Latest generation" = last 2 years. Three years ago I was
| buying SMC servers which were the first round configured with
| randomized passwords to comply with CA law (some batches had
| ADMIN/ADMIN) and gigabyte servers still were out of
| compliance.
| dijit wrote:
| I bought my last batch of Dell machines in 2017 (so, 5
| years? wow), and they had randomised passwords.
|
| But yes, some providers put the BMC on the internet because
| it's easier, a provider I used once did this and I was
| quite displeased as iDRAC's are quite weak and suffer under
| the weight of bot-spam. -- even if there were no security
| issues.
| GekkePrutser wrote:
| This is actually really much lower than I expected..
|
| Though what helps is that most servers have a dedicated iLO
| interface and you really have to choose to configure it on the
| regular ones along with normal traffic. So out of band is
| default.
|
| So in this case it's only people who have deliberately configured
| this. I think this is why it's not hundreds of thousands.
| madjam002 wrote:
| I've only dabbled with iLO but even when operating on the same
| interface I'm not sure how this gets onto the internet? From
| what I remember you configure it with a static IP, so these
| people surely are configuring their iLO to listen on a publicly
| routable IP?
| GekkePrutser wrote:
| Yeah if your server is DMZ they could have configured it with
| another ip in the same range.. I guess it's something like
| that.
|
| Some of them could also be intentional, in order to provide
| management capabilities.
|
| At least iLO has no default password (each server comes with
| a label with the password)
| wongarsu wrote:
| We once (~2 years ago) had an Asus server with their BMC, which
| is much easier to expose: while it has a dedicated physical
| interface by default it is also exposed on the first LAN port
| and configured to use DHCP. I can easily see how those boards
| leave you accidentally exposed in a colocation setup. But I'm
| still struggling to come up with a way to accidentally expose
| that on the internet. Public IPv4 isn't usually handed out via
| DHCP.
|
| My best guess is that the vast majority of these 20k servers
| have their management interface deliberately exposed. Only
| takes 2000 people with 10 servers each to think this is a good
| idea.
| assttoasstmgr wrote:
| Not sure I agree with you 100% on your police work there, Lou.
|
| I worked with a good number of SuperMicro servers that - to my
| surprise - the IPMI interface was by default set to "fail over
| to bridge to primary interface". That is to say if the IPMI
| port was disconnected, it would bridge onto your primary NIC
| with a second MAC address. The way to disable it was to
| download an obscure utility from SuperMicro's website, which
| only ran under Windows, and to pass an equally obscure
| hexadecimal command line flag.
|
| Fortunately in my case, these ran on a network with no DHCP
| server, but I can't assume that's true in every case.
| throwawayboise wrote:
| Yes SuperMicro machines are configured that way. Some don't
| even have a dedicated separate IPMI interface.
| dnautics wrote:
| Yeah those supermicros scared the bejeezus out of me when I
| saw them in the catalog. I mean, yeah, you _can_ configure
| the switch to listen onto different vlan ids, but what if
| something weird happens and that stateful information is
| lost or corrupted?
| GekkePrutser wrote:
| iLO as the remote management tech from HP doesn't do that
| though, at least not on any of the servers I've seen.
|
| IPMI is a very different thing.
| trebligdivad wrote:
| Note that the search was only for HP iLO's; not everyone elses
| model.
| GekkePrutser wrote:
| Yeah I got that. Other brands don't actually have iLO as it's
| an HP trademark.
| ggm wrote:
| I love how iDrac shipped with a password everyone knows. I would
| have thought it was trivial to make it a 1-time passthrough to
| change it, to stop this being the cartoon character we know and
| love.
| gengelbro wrote:
| I've had the bad fortune of dealing with iLO in the past. There's
| absolutely nothing surprising to me that it would be remotely
| exploitable, as well as default remote accessible.
| cma wrote:
| Some could be honeypots
| dlsa wrote:
| Likely a lot more aren't. Having tripwires is still a good idea
| though.
| serverCurios wrote:
| That was a great deep dive in to iLO. I wonder how much of that
| would apply to Dell's iDRAC?
| cesaref wrote:
| The older iLO2 and 3 were from the Gen7/8 HP rack mount servers,
| which were released around 2013 or something like that. I'd have
| thought that the majority of commercial uses have long passed, so
| I was wondering if these generation of machines are really being
| used for home labs, that sort of thing, as they are passed their
| use by date (I think the gen7 machines were EOSL'd in 2018 so
| they'll have been chucked out of datacentres).
|
| Actually, I can imagine a fair few are still in use as small
| enterprise servers, with iLO being visible for remote admin,
| which is a shame, as that suggests they are not behind a tunnel,
| so those addresses probably indicate larger problems than a
| visible iLO.
| jabart wrote:
| iLO2 is G6 iLO3 is G7 iLO4 is G8
|
| Walking around a DC, no they have not been "chucked out of
| datacenters"
___________________________________________________________________
(page generated 2022-01-28 23:00 UTC)