[HN Gopher] Masscan: TCP port scanner, scanning entire Internet ...
___________________________________________________________________
Masscan: TCP port scanner, scanning entire Internet in under 5
minutes
Author : ducktective
Score : 346 points
Date : 2021-09-28 14:11 UTC (8 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| marcodiego wrote:
| Makes me wonder if there are still some netbus servers alive.
| jihadjihad wrote:
| If masscan is of interest to you, be sure to check out zmap [0]
| as well. It can scan the entire IPv4 address space in around 45
| minutes.
|
| 0: https://github.com/zmap/zmap
| danilonc wrote:
| Old discussing about the tool:
|
| https://news.ycombinator.com/item?id=8803498
| dang wrote:
| Thanks! Expanded:
|
| _Some idiot is using your tool to mass scan our network_ -
| https://news.ycombinator.com/item?id=24728123 - Oct 2020 (182
| comments)
|
| _MASSCAN: Mass IP port scanner_ -
| https://news.ycombinator.com/item?id=12260809 - Aug 2016 (33
| comments)
|
| _Masscan: Scan the entire Internet in under 5 minutes_ -
| https://news.ycombinator.com/item?id=8803498 - Dec 2014 (29
| comments)
|
| _Masscan: scan the entire Internet in under 6 minutes, 10
| million packets /second_ -
| https://news.ycombinator.com/item?id=6391266 - Sept 2013 (30
| comments)
|
| _Masscan: The entire internet in 3 minutes_ -
| https://news.ycombinator.com/item?id=6388222 - Sept 2013 (12
| comments)
| 1-6 wrote:
| Seems like ports can use a bit of updating. Can't an encoded
| message be sent to a port before it opens up and exchanges its
| presence?
| mcpherrinm wrote:
| The Wireguard VPN protocol doesn't reply to any packets that
| aren't cryptographically authenticated, so that's a good option
| if you want something "unscannable" on the internet.
|
| For a regular, publicly accessible server, you do want people
| to be able to connect to it, so it must be scannable.
|
| I don't know offhand, but with HTTP/3's udp based protocol, it
| ought to be possible to at least make scanning a bit trickier.
| By requiring a valid QUIC client hello packet, with a valid SNI
| header for that server, the scanner must know the name of the
| server it's trying to talk to. I don't have any experience with
| HTTP/3 yet so I am probably wrong.
| EamonnMR wrote:
| https://en.m.wikipedia.org/wiki/Port_knocking
| sneak wrote:
| FWIW I think the late Dan Kaminsky is the person who originally
| invented this two process userspace TCP scan technique
| 2001-2002ish in his tool Paketto Keiretsu.
|
| It's a good one.
|
| I sometimes wonder if it inspired some of the TCP-in-userspace
| stuff that is done in go (gvisor lib, I think it was).
|
| I'm glad Rob is continuing the development of this idea and
| continues to scan the whole internet. Scanning the whole internet
| is cool. It's a shame it's de facto illegal these days (and will
| get most internet connections in the USA terminated near-
| instantly).
| samstave wrote:
| > _the late Dan Kaminsky_
|
| Ehhh... I forgot he was the "the late Dan Kaminsky"
|
| :-(
| redis_mlc wrote:
| FYI: He died at 42 from diabetes.
|
| https://en.wikipedia.org/wiki/Dan_Kaminsky
| sneak wrote:
| Yeah, it's a real bummer.
|
| 2021 has not been a great year. Too many, too young.
| xinniethepooh wrote:
| Source on it being de-facto illegal?
|
| I've not seen anything that says simply scanning for open ports
| is illegal, doing vulnerability scans may be though.
| xenadu02 wrote:
| They mean a lot of ISPs and VPS providers will flag &
| terminate your account for launching port scans so it is
| difficult to find somewhere to run such a scan at any kind of
| speed.
| tptacek wrote:
| Decoupled sender/receiver scanners date back to the 1990s (we
| did one in CASL, our misbegotten packet programming language
| from 1997, the script for which I think shipped with Ballista
| --- and we stole the scanner design from someone else, probably
| from CORE SDI), as do userland TCP libraries (LWIP is a famous
| one, or at least a library that got used for that purpose, that
| predates all this stuff).
| OrvalWintermute wrote:
| The other userland discreteTCP/IP stack mass-scanner of note
| is Unicornscan [1] from Jack C Louis (RIP)
|
| [1] https://defcon.org/images/defcon-13/dc13-presentations/DC
| _13...
|
| I know this came a few years later, but it really advanced
| the state of the art for widescale scanning at that time,
| particularly once it made its way into Kali.
| tptacek wrote:
| For a long time, unicornscan was the preferred scanner for
| pentests (nmap is venerable and impressive but it has a lot
| of weird failure modes on real networks). I had no idea the
| author had died.
| londons_explore wrote:
| Impressive that the bitcoin donation address has received $20k!
| dmos62 wrote:
| I like that you checked it!
| londons_explore wrote:
| I'm always interested how much revenue peoples hobby projects
| earn.
|
| The vast majority of projects seem to take someone months of
| work, get 50 github stars, and $25 worth of donations...
| Which is really sad - it basically means the vast majority of
| opensource authors have to have another job to pay the bills.
| shadycuz wrote:
| I just started to try and monetize. I'm a bit early, less
| than 50 stars. But my expectations are pretty much what you
| stated. Though my actual plan is to try and build a
| following like Jeff feeling and then sell training courses
| and ebooks.
| slim wrote:
| That adress could be used for different transactions
| unrelated to this project
| quickthrower2 wrote:
| And bitcoin has increased in value in the last 2 years
| sva_ wrote:
| Most of it was sent when Bitcoin was worth a lot less.
| mike_d wrote:
| A quick plug for my friends over at GreyNoise... they have
| honeypots all over the internet and identify various scanners as
| well as their observed intentions.
|
| Here is everyone running masscan against the internet:
| https://www.greynoise.io/viz/query/?gnql=tags%3A%22Masscan%2...
| [deleted]
| mod wrote:
| All the results on the first page (as much as it would show me)
| are just VPS providers.
| Aissen wrote:
| Awesome tool. Rob added IPv6 support last year, which can be
| really useful if you know what you're doing (ex: want to scan a
| single subnet for given OUI without privacy extensions).
| dheera wrote:
| > This increases the rate to 100,000 packets/second, which will
| scan the entire Internet (minus excludes) in about 10 hours per
| port (or 655,360 hours if scanning all ports).
|
| So 655360 hours, not 5 minutes
| [deleted]
| wfn wrote:
| Well, keep reading the README, then :) see PF_RING:
| https://github.com/robertdavidgraham/masscan#pf_ring
| unixhero wrote:
| Warning. Masscan is the best scanner I have ever seen.
|
| We have also managed to take down the entire corporate network by
| using it with a too high rate limit. So tread lightly around
| massscan and its power. Our pentesters did the same 6 months
| later. The managed service provider is not able to solve the
| routing table loop which causes the firewall to DOS when a rapid
| masscan is triggered.
| Socketier wrote:
| Yeah I managed to do the same, it filled up the firewall state
| table and nobody was able to initiate new connections, only
| existing ones continued to work. And that's how I learned about
| stateful firewalls!
| trutannus wrote:
| Funny this would show up here. I keep seeing this on a bot
| profiling project of mine. Infrequent requests, but persistent.
| ABraidotti wrote:
| I enjoy Rob Graham on Twitter too: https://twitter.com/ErrataRob
|
| He often discusses current events in infosec if you're into that.
| amatecha wrote:
| Yeah, I appreciate his willingness to share "inconvenient
| truths" that may run counter to the running narrative of
| whatever tech/security story of the moment. Lots of interesting
| perspectives, even if maybe I didn't "want" to hear it ;)
| bogomipz wrote:
| The author states:
|
| >"A mutex on the fast path of a program severely limits
| scalability. Instead, Masscan uses "rings" to synchronize things,
| such as when the user-mode TCP stack in the receive thread needs
| to transmit a packet without interfering with the transmit
| thread."
|
| Is "rings" here referring to PF_RING mentioned in the preceding
| paragraph or is it referring to a specific synchronization
| primitive?
| chaz6 wrote:
| I am not sure any computer is capable of sending and processing
| 2^126 packets (assuming global unicast, 1 packet out, 1 packet
| in) in under 5 minutes.
| OnlyMortal wrote:
| Amateur!
| dmw_ng wrote:
| The v6 space has structure and is much smaller than that
| kalleboo wrote:
| Aside from Shodan's neat trick of infiltrating the NTP pool,
| are there any papers/blogs on successfully exploiting known
| properties of IPv6 (network prefixes, MAC vendors etc)?
| However I think about it it seems too massive
| lima wrote:
| Many providers sell netflow data.
| pjf wrote:
| https://www.entropy-ip.com/
| https://arxiv.org/abs/1606.04327
| tux3 wrote:
| ....infiltrating the NTP pool!
|
| Oh my, that is clever =)
| dmw_ng wrote:
| I was mostly thinking about the sparse structure of upper
| bits (which are easily enumerable e.g. by downloading a
| RIPE database dump). As for the lower 64, MAC address OUIs
| contribute at most around 15 bits, and the distribution is
| likely strongly skewed towards only a handful of vendors,
| so actual randomness is probably lower still.
| helge9210 wrote:
| You can do it in parallel from several computers. Masscan has
| capability of dividing workload across multiple processes.
| martini333 wrote:
| > It can scan the entire Internet in under 5 minutes,
| transmitting 10 million packets per second, from a single
| machine
| birdyrooster wrote:
| Definitely but then click bait wouldn't be click bait without
| some misleading information
| bmicraft wrote:
| I would argue that most interesting (and unpatched?) servers
| do at least have an ipv4 address as well, if not exclusively
| zamadatix wrote:
| "Only" about a /15 worth of v6 is actively advertised on the
| internet. Still, good luck sending that many packets.
| jerf wrote:
| Well, 5 minutes is about 2^8 seconds, you can get about 2^32
| cycles per core per second nowadays, and you can have about 2^6
| cores maximum, so with an incredibly, unspeakably generous
| "scan one IP in one cycle" a 64-core computer should be able to
| scan about 2^(8+32+6) = 2^46 addresses in a little under five
| minutes.
|
| We seem to be a wee bit short of 2^126, yes.
| lol768 wrote:
| Lots of people like to pretend the "entire internet" is
| synonymous with the IPv4 address space...
| birdyrooster wrote:
| Well you can understand why in the case of masscan, it
| would be a pretty boring claim to say that masscan is
| "scanning the entire internet in 100 years from a single
| machine!"
| ehPReth wrote:
| My ISP for one...
| gitfan86 wrote:
| Large blocks of IPs are not part of the internet. 10. 172.
| Etc...
| signa11 wrote:
| all non-routeable addresses, multicast etc. etc.
| [deleted]
| jerf wrote:
| The IPv4 internet is merely 2^32, and people have been
| scanning that whole thing for years now. The /112 is
| referring to IPv6 addresses.
|
| Plus the copy about not hammering other networks won't
| matter when you're trying to scan entire /64s that are
| behind one home router or something. That's gonna get
| noticed. You can't really scan IPv6 like you can scan IPv4,
| the math I gave is part of why. The code to do it is
| trivial, but the hardware just isn't there.
| sva_ wrote:
| I'm pretty sure the IPv4 internet is constantly being
| scanned. I blocked ICMP echo request (ping) in my
| firewall, and saw it throws a warning of an attempted
| ping every other minute.
| londons_explore wrote:
| I'm really surprised shady figures aren't publishing
| lists of active IPv6 addresses to allow scanning.
|
| For example, someone with access to a backbone internet
| router could easily log src and destination ipv6
| addresses, and sell the complete list sorted and
| compressed. Malware authors could then use the list to
| portscan for badly firewalled stuff.
| pjf wrote:
| https://ipv6hitlist.github.io/
|
| First time to be called shady, though ;)
| ryanlol wrote:
| It's not surprising. Malware authors aren't going to pay
| for something that'll have no meaningful effect on their
| infection rates.
|
| There's more than enough stuff on IPv4, IPv6 isn't worth
| the effort.
|
| FWIW shodan was setting up their own public ntp servers
| to track down v6 users.
| dadrian wrote:
| People absolutely do this, they're just not going to post
| it for free.
| pixl97 wrote:
| With IPv6 privacy extensions this gets really big really
| fast and is mostly empty.
| hsbauauvhabzb wrote:
| Scanning 2^32 wasn't exactly viable 30 years ago, so
| never say never.
| nunez wrote:
| Perfect for finding publicly-accessible Kubernetes API servers.
| 1-6 wrote:
| I created a honeypot once and they started crypto mining on
| mine.
| dropalltables wrote:
| I love this project and Robert is one of the most awesome,
| thoughtful people in the security world.
| [deleted]
| johnnyApplePRNG wrote:
| Was curious the other day and couldn't find a real answer... does
| AWS allow portscanning from their infrastructure?
|
| All I could find was statements from them that they do not allow
| port scanning OF their infrastructure.
| mindcrime wrote:
| And if Amazon doesn't, are there any hosting providers that do
| allow port-scanning from their infrastructure? I'm assuming the
| big providers like AWS, Google and Azure all have these limits
| in place, but I wonder if you can do it from some of the
| smaller providers? Even if means going down to some fly-by-
| night outfit in Belize or something...
| throwaway39489 wrote:
| Smaller ones allow yes, they only frown when scanning ports
| relating to infrastructure stuff
| helge9210 wrote:
| Practically, no. Unsure about actual implementation, but looks
| like packets disappear right after leaving the interface.
| dadrian wrote:
| If you bring your own IPs they don't care. However, the
| bandwidth fees are egregious.
| samstave wrote:
| Sort-of.
|
| You can port scan your own infra/vpcs etc -- but YOU MUST tell
| them you are doing so and why you are doing it, else they will
| block it.
|
| again, as with anything AWS (and other providers) have a good
| rapport with your rep, and SEs in AWS and you have a lot more
| freedom than you expect just from boilerplate ULA TOS stuff.
|
| You may not scan anything other than your own infra. And you
| can get your external monitors whitelisted as well...
|
| Just talk to your rep.
| Cantinflas wrote:
| They don't, I got a company acc banned scanning my own local
| infra from an aws instance
| bowmessage wrote:
| oof, how was that ultimately resolved? Hopefully a warning
| from support and a re-enabled account?
| Cantinflas wrote:
| Nope! Afaik they did not get the account re-enabled, at
| least for the next few months!
| slenk wrote:
| You used to be able to let them know you were going to do
| certain kinds of pen-tests, unless port scanning is just one
| of those things never allowed
| nkellenicki wrote:
| You still can, but that's for pentests _targetting_ AWS
| hosted infrastructure. They've always frowned upon using
| AWS to target _other_ services, however.
| [deleted]
| _wldu wrote:
| I wrote netscan years ago and still use it. It's pretty fast too:
| https://github.com/62726164/netscan
| wfn wrote:
| A very nice and well-built tool. Excellent use of sequence
| numbers for stateless send/receive (SYN cookies):
| https://github.com/robertdavidgraham/masscan/blob/master/src...
| (very well-documented source too, a pleasure to read).
|
| I once (2016) used it to scan port 22 on the whole ipv4 (had to
| experiment with rate limits to not trigger alarms and get
| complaints forwarded by my VPS provider; the clever ip+port
| randomization technique helps a lot with that). Then took the
| ~22m (iirc) IPs which responded and ran ssh-keyscan on them to
| extract and analyze some ~15m ssh banners and public keys (a
| bunch of them broken, through debianized PRNG etc.) I think most
| of the scanning + extraction was done overnight, via ~13 VPS
| rented hourly (whole thing cost < $10, and very few complaints).
| Fun times :) I should write it up some time, and do it again.
| dosshell wrote:
| What is "22m" ?
|
| 22 meter?
|
| 22 millies of something.
|
| I do not follow?
| howenterprisey wrote:
| 22 million.
| sigg3 wrote:
| 22 million meters of IP.
| btown wrote:
| Internet pipe!
| RobRivera wrote:
| at the cost of a rod per hogshead
| peakaboo wrote:
| The irony of saving 6 characters in the word million, but
| people don't understand his post and use many more
| keystrokes asking what he means.
| jasonwatkinspdx wrote:
| "mm" is a common abbreviation for million in the context
| of accounting. It refers back to the latin "mille mille"
| which means "thousand thousand" literally. "Mille" is
| still used for thousand in French and Italian. The
| abbreviation is reasonably common in english, though I
| also would not be surprised by anyone not having run into
| it before.
| jodrellblank wrote:
| What is "22 million"? I don't understand. Is it length?
| Thousandths? An ion grinder? An exotic big cat? A tiny
| On?
|
| "People don't understand", none of those were at all
| plausible even without the context of scanning the entire
| IPv4 address space.
| lazyeye wrote:
| Im sure the primary use for this tool will be ethical /s
| SavantIdiot wrote:
| Rather than have everyone who can run this (without getting
| banned) actually run it, couldn't someone just post the results
| to a file for us once per day or week? Seems inefficient to have
| millions of people run this when it results in (mostly) the same
| data.
| nbk_2000 wrote:
| Not as a file but as a searchable index, yes. Here's a few:
| shodan.io spyse.com zoomeye.org
| palebluedot wrote:
| try https://search.censys.io
| LeonidBugaev wrote:
| Question: how do you actually run it without getting banned?
|
| I've got banned pretty hard by both my local home ISP and using
| Linode servers, when tried such scanners. Mass port scanning is
| easy to track, and it usually forbidden all ISP ToC.
| Ms-J wrote:
| Use a no-logs, anonymous VPN. I do it all the time.
| michaelbuckbee wrote:
| Follow up question: can you recommend a no-logs, anonymous
| VPN?
| [deleted]
| eurasiantiger wrote:
| Next time you're visiting any place, install a proxy on
| their network.
| doubled112 wrote:
| Raspberry Pi on their wall, you say?
| leadingthenet wrote:
| IVPN
| prox wrote:
| Question: why would you run this?
| Godel_unicode wrote:
| Research. Ever wonder how many telnet servers are exposed to
| the internet (hint: way too many)?
| ansible wrote:
| I wonder how many of those are just MUDs (text-mode dungeon
| games) where it was common to use telnet.
| loeg wrote:
| MUDs usually run on a non-standard port (not 23).
| prox wrote:
| Those would be running for a while now then
| itslennysfault wrote:
| To hack a gibson, baby.
| nebulous1 wrote:
| I did a CTF a while back and it was used to find the real IP
| of a missconfigured .onion site. I didn't actually do it as I
| was worried about burning a VPS provider.
| captn3m0 wrote:
| My ISP had a misconfiguration that let all the customers ping
| each other (Routers usually). The address space was too large
| to scan any other way except masscan, which worked very well
| (and fast).
|
| https://medium.com/@captn3m0/i-scanned-all-of-act-
| bangalore-...
| jonaslejon wrote:
| You need to have a really good relationship with your ISP and
| get their acknowledge prior to the scanning
| heywherelogingo wrote:
| What do ISPs consider an acceptable rate?
| nothis wrote:
| Hyper-naive question: If something can run in 5 minutes, how
| can it bother an ISP?
| pantulis wrote:
| Because it sends a lot of traffic in such a small time frame.
| Also, it has obvious nefarious purposes and the ISP may face
| legal consecuences in case someone lawyers up.
| jandrese wrote:
| It triggers bans of your netblocks for bad behavior. 5
| minutes to generate hundreds of complaints.
___________________________________________________________________
(page generated 2021-09-28 23:00 UTC)