[HN Gopher] Second large Hetzner outage in a week caused by DDoS...
___________________________________________________________________
Second large Hetzner outage in a week caused by DDoS attack
Author : xmpir
Score : 52 points
Date : 2022-05-06 21:15 UTC (1 hours ago)
(HTM) web link (status.hetzner.com)
(TXT) w3m dump (status.hetzner.com)
| ricardobeat wrote:
| At their size, don't they have some kind of hardware-level packet
| filtering ability like cloudflare to protect against these
| attacks?
| rstupek wrote:
| Not if the level of incoming bandwidth exceeds the available
| bandwidth of the circuits involved.. you can't filter it when
| the link is saturated. Cloudflare uses other techniques like
| global distribution so aggregate bandwidth is higher than the
| attack bandwidth
| booi wrote:
| This depends on what type of attack it is. If it's volumetric,
| no amount of packet filtering is going to help you. If it's
| protocol-level attack then yes, some form of high performance
| WAF will be helpful if you have the filtering capacity.
|
| Likely the attack isn't an overwhelming volumetric attack as I
| assume they have some fat pipes and big routers, but there's
| likely a bottleneck somewhere in their network.
| xmpir wrote:
| They state using hardware DDoS protection but it seems not to
| be sufficient: https://www.hetzner.com/unternehmen/ddos-schutz
| xmpir wrote:
| Last time it took about 9 hours:
| https://status.hetzner.com/incident/129728ce-ba25-49b6-96cc-...
| _-david-_ wrote:
| >This concerns UDP traffic on port 9000-65535.
|
| Does anybody know what usually runs on those ports?
| melolife wrote:
| It's interesting that 9000 is the starting port for Ethereum
| consensus clients, although the participation rate does not
| seem to be affected.
| rozenmd wrote:
| Online games (MMOs, shooters, etc) come to mind
| baisq wrote:
| MMOs over UDP?
| Retr0id wrote:
| Absolutely
| sascha_sl wrote:
| MMOs often have TCP connections for things like chat and
| services like auction house (often even HTTP
| microservices), but most of the gameplay is still UDP.
| sodality2 wrote:
| That's the preferred protocol for ultra-real-time games
| because a few ms ago is not helpful information to spend
| time recovering. A sufficiently fast-moving MMO could apply
| baisq wrote:
| What MMOs use UDP? Asking sincerely because I have never
| seen one.
| koolba wrote:
| Anything with real time communications like an FPS would
| use UDP as stale action data is mostly useless. The
| latest state of is all that matters.
|
| Most such games will either layer their own streaming
| channel atop UDP for guaranteed ordered delivery of
| important messages or use a separate TCP socket as well.
| xnyanta wrote:
| You must not be looking very hard, pretty much every game
| engine uses UDP as the network transport. There are some
| notable exceptions like Java Minecraft.
| xmpir wrote:
| I fear e.g. wireguard is affected.
| deathanatos wrote:
| That's 56,536 different ports. Half of everything (that uses
| UDP), more or less.
| ascar wrote:
| I would expect 95%+ of TCP traffic to run on 22 (ssh),
| 25(smtp), 53(dns), 80(http), 443(https) plus another handful
| of lower than 1000 ports. Even common dev ports
| (3000,5000,8080) are below 9000. I don't think that's much
| different for UDP. Even most games probably rely on something
| <10,000.
| danachow wrote:
| > I don't think that's much different for UDP.
|
| It is, because of the way that UDP is typically used for
| different applications than TCP. While there are a few old,
| well known TCP/UDP pairs like 53, UDP is more often used
| with a dynamic port assignment scheme sometimes with a
| coordinating TCP protocol - such as SIP/RTP for VoIP that
| uses >16k, WebRTC, etc. A lot of games uses ports above
| 10k. https://help.generationesports.com/hc/en-
| us/articles/3600611...
| lordnacho wrote:
| Isn't it most things that aren't a well-known service?
| sascha_sl wrote:
| Source ports of DNS reflection attacks, presumably.
| [deleted]
| tiffanyh wrote:
| I think games.
|
| Hetzner is a popular host for game servers.
| scottlamb wrote:
| Besides games, I think many AV things, including VOIP and
| perhaps WebRTC. Possibly also HTTP/3; the server picks the UDP
| port number IIUC.
| xmpir wrote:
| I am wondering what the attacker's intent is
| walrus01 wrote:
| [spiderman-pointing-at-spiderman.gif]
|
| seriously, aren't they commonly the SOURCE of many DoS attacks...
|
| any hosting provider where some random person on the internet and
| $5 of credit on a prepaid visa card will have this problem.
| unnouinceput wrote:
| Maybe, just maybe, rely less on embedded framework on embedded
| framework that spit JavaScript that gets 95% unused. If for a
| simple outage apology page the output was 1.7MB, I can only
| imagine for their normal pages how much it is. At this size I
| feel only like 10k legit users would unwillingly do the outage
| anyway. But hey, Kubernetes and Node.js is all the rage nowadays.
| danuker wrote:
| "I have only made this letter longer because I have not had the
| time to make it shorter." - Blaise Pascal
| tempnow987 wrote:
| I thought OVH and Hetzner were the source of a ton of these DDoS
| attacks. Their IP ranges always seem to be in abuse logs.
|
| Cloudflare write in a recent attack:
|
| The top networks included the German provider Hetzner Online GmbH
| (Autonomous System Number 24940), Azteca Comunicaciones Colombia
| (ASN 262186), OVH in France (ASN 16276), as well as other cloud
| providers.
|
| https://blog.cloudflare.com/15m-rps-ddos-attack/
| CircleSpokes wrote:
| I mean that makes sense no? Attacks like that rely on
| compromised servers so it shouldn't be a big surprise large
| hosting provides are among the biggest attackers. Other large
| ISPs like digital ocean and Alibaba are among the top attackers
| in that attack also.
|
| I assume this attack is UDP based unlike the one you linked
| too.
| ffhhj wrote:
| Excuse the ignorance, but couldn't ISPs block the attacks?
| vardagsnyttt wrote:
| That would make sense, but its hard:
|
| - You need to identify the traffic to be filtered and the post
| states: "Due to always different destinations (IPs, ports,
| packet size) (..)"
|
| - You need to maintain some agreement with a large number of
| ISPs
|
| - You need to maintain some gossiping infrastructure to these
| ISPs
|
| - ISPs may not care about your DDoS attack
___________________________________________________________________
(page generated 2022-05-06 23:00 UTC)