[HN Gopher] WebWormHole: Send files quickly using WebRTC
___________________________________________________________________
WebWormHole: Send files quickly using WebRTC
Author : pvsukale3
Score : 174 points
Date : 2020-04-29 20:15 UTC (2 hours ago)
(HTM) web link (webwormhole.io)
(TXT) w3m dump (webwormhole.io)
| nerdbaggy wrote:
| I love that it uses chunks and streaming to transfer the file. So
| many of these just try and load the entire file at once so you
| can't transfer much.
| api wrote:
| It's 2020, and people are elated to discover that it is possible
| to transfer a file directly between two systems on the Internet.
|
| True story: I was giving a guest lecture on network
| virtualization at UCI and demoing ZeroTier. One student came up
| afterwords and asked me how traffic could flow between systems
| without "a cloud." Evidently the idea that data could just go
| directly from point A to point B was utterly, completely foreign
| to the point that they weren't aware that the Internet could be
| used this way.
| Jhsto wrote:
| Similar to one of my family members: she thought the Internet
| is a "thing" you just put stuff to, and it was available to
| all. She had no idea Facebook has an actual computer somewhere
| receiving her queries.
| _jal wrote:
| I blame NAT.
|
| Treating end-users like second-class netizen consumers trained
| people to "need" the cloud to do perfectly normal peer-to-peer
| things.
| mr_toad wrote:
| I blame Windows taking 20 years to include ssh.
| codethief wrote:
| I'm not sure I would blame NAT. You can easily disable that.
|
| What you can't disable is the asymmetry of consumer internet
| connections (upload << download) and the fact that most
| consumer devices are not running (or connected to the
| internet) 24/7.
| pjc50 wrote:
| Also security: if consumer PCs were open to the internet,
| they would be constantly getting breached in even bigger
| numbers.
| zozbot234 wrote:
| Consumer PCs are not the real issue by and large, IoT
| crap that can't even be updated is way more problematic.
| Having NAT by default helps screen out attacks to those
| devices.
| kaetemi wrote:
| Carrier grade NAT...
| codethief wrote:
| Sure, when you're talking about mobile devices. (Or has
| it already become a thing with DSL, too?) In any case,
| the issue started much earlier, I'd say.
| 0xdeadbeefbabe wrote:
| "Drop.io was nominated for the Technical Achievement Award at
| the South By Southwest 11th Annual Web Awards in 2007."
| https://en.wikipedia.org/wiki/Drop.io
|
| It keeps happening.
| coopsmgoops wrote:
| Nice, I made one of these a few years ago http://passfiles.com
|
| Yours is a bit more polished than mine though. I didn't use QR
| codes either just good old fashioned urls.
| gccxsse wrote:
| There seem to be hundreds of these sites. They all do the same
| thing. Off the top of my head I can remember https://file.pizza
| rakoo wrote:
| file.pizza and friends work on webtorrent, which doesn't do
| end-to-end encryption
|
| EDIT: just learned that WebRTC actually does end-to-end
| encryption by default, so I'm wrong
| mmm_grayons wrote:
| Interesting note that this guy's choice of PAKE, Cpace, was
| chosen about a week ago by the CFRG for use in IETF protocols.
| Cpace is new, but that's a big vote of confidence for it.
| aspenmayer wrote:
| Nice spot, here's a link to the IETF draft spec for CPace
| mentioned.
|
| https://tools.ietf.org/id/draft-haase-cpace-01.html
|
| IETF post announcing the chosen candidates
|
| https://mailarchive.ietf.org/arch/msg/cfrg/LKbwodpa5yXo6VuND...
|
| Candidate selection process
|
| https://github.com/cfrg/pake-selection
| jedisct1 wrote:
| Implementation using libsodium
| https://github.com/jedisct1/cpace
| crazygringo wrote:
| > _...it uses WebRTC to make the direct peer connections. This
| allows us to make use of WebRTC 's NAT traversal tricks, as well
| as the fact that it can be used in browsers._
|
| But I'm assuming it can't break through all NAT routers, right? A
| good portion of people still won't be able to use this?
|
| A service usable by everyone would require STUN and TURN servers
| to be set up, no?
|
| Or has WebRTC made advances I'm unaware of?
| bscphil wrote:
| > But I'm assuming it can't break through all NAT routers,
| right? A good portion of people still won't be able to use
| this?
|
| > A service usable by everyone would require STUN and TURN
| servers to be set up, no?
|
| Anecdata, of course, but I haven't been able to reliably use
| any of these WebRTC based file transfer services (file.pizza,
| instant.io, etc etc). Testing mostly between two computers on
| the same subnet. Sometimes they work for a little while, at
| surprisingly low speeds (for two computers connected to the
| same wireless access point), sometimes I can let them sit for
| an hour and never get a connection. I've learned to not even
| bother trying them, it just wastes time.
|
| That said, magic-wormhole (the original) works fine between the
| same devices, so maybe I'll see if something is somehow
| different about this implementation.
|
| Edit: ah yes, this service hangs indefinitely on "connecting".
| You love to see it. (Firefox on Linux - firewall disabled
| specifically for this test - and Safari on macOS)
|
| Edit: seems to be working in Chrome (Linux) to Firefox
| (Android). Not sure what the difference is.
| Sean-Der wrote:
| I don't have any hard numbers, but I have heard ~85% ICE
| success rate with out TURN. But you are right, in some cases
| WebRTC will fail without TURN. Just no one wants to pay to run
| those servers :)
|
| I would love to see TCP hole punching in ICE, but it sounds
| like it is super hard to get right.
|
| Consumer internet does a lot better, lots of those failures
| come from Government/Military/Medical I bet.
| folmar wrote:
| > Just no one wants to pay to run those servers :)
|
| No, it's the users that don't want their (meta)data inspected
| by a random third party in transit. This is why I don't use
| filepizza
| jackewiehose wrote:
| You always need a at least a STUN server and in my experience
| that 85% isn't remotely true. For example STUN-only never
| worked from mobile internet (only tested some german
| providers).
| algesten wrote:
| how would TCP hole punching even work? TCP has state and a
| handshake. UDP doesn't.
| zozbot234 wrote:
| UPnP can be used to setup port forwarding if the NAT
| gateway is configured correctly.
| algesten wrote:
| sure. I think it's a bit different though. upnp is like
| some remote control of your fw.
| saljam wrote:
| This uses STUN servers to help it poke through NATs. (That's
| what I mean by "WebRTC's NAT traversal tricks")
|
| There's no TURN server set for this, but it shouldn't be hard
| to add one. There are NATs where you'd need one to relay all
| the traffic, but these seem to be relatively rare nowadays. If
| anyone has any actual statistics on these I'd appreciate it!
| scruffups wrote:
| " but these seem to be relatively rare nowadays "
|
| AT&T 5G uses Symmetric NAT. It's not rare if you have an
| iPhone or iPad with cellular. No way to do P2P without
| relaying traffic unless you want to "guess" the randomized
| port number, and, on that front, there are NAT-device-aware
| algorithms that can make that process faster.
|
| We were promised IPv6 will make NAT's not necessary but I
| believe service providers use NATs not simply to conserve the
| IPv4 space but to actively discourage using the service to
| host your own servers.
| majke wrote:
| Which one? Which STUN server are you using?
| adrianpike wrote:
| Looks like Google's: https://github.com/saljam/webwormhole/
| blob/f542b26fb2fa32820...
| saljam wrote:
| The website uses Google's.
|
| On command line it's an option and Google's is default. I'd
| like to make the signalling server also a STUN server at
| some point.
| crazygringo wrote:
| Oh that's interesting... I had no idea there were
| publicly available STUN servers like that.
|
| But way back in 2014 a Google employee does seem to have
| confirmed it's free to use, but comes without guarantees.
|
| [1] https://groups.google.com/d/msg/discuss-
| webrtc/shcPIaPxwo8/F...
| lioeters wrote:
| stun:stun.l.google.com:19302
|
| Source: https://github.com/saljam/webwormhole/blob/8f707583
| d68173de3...
| jchook wrote:
| Why not generate the QR code client-side?
|
| Folks that care about P2P want to see zero HTTP requests to your
| server after loading the basic resources.
| pkulak wrote:
| Very nice! I'm assuming this is based on the wonderful "Magic
| Wormhole"? Is it actually using that program under the hood?
| gramakri wrote:
| file.pizza is another similar project
| st0le wrote:
| https://drop.lol/ too
| grishka wrote:
| I once tried sending a 3 GB file through it, then kept
| wondering why my entire system became so sluggish. Turns out it
| loads the entire thing into RAM... The file didn't go through
| either.
|
| I hope this one isn't like that.
| f1refly wrote:
| Same experience here. Every website I tried so far died when
| sending large files because it apparently loaded the whole
| thing into ram and then some. That might be fine on a system
| with 32gb memory, but my laptop with 8gb dies when trying to
| send a 7gb file.
| h-cobordism wrote:
| Does HTML5 even let you read a file without loading it all
| into memory?
|
| Edit: Looks like this is it.
| https://developer.mozilla.org/en-
| US/docs/Web/API/ReadableStr...
|
| Edit 2: And yes, this is using it. https://github.com/saljam/
| webwormhole/blob/master/web/main.j...
| mr_toad wrote:
| The JavaScript implementation is creating a blob with a URL
| pointing to it so the user can save the file. I might be
| wrong, but I think that the all the data is in browser memory
| before it is saved.
|
| Browsers are pretty restrictive about writing to the file
| system.
| sairamkunala wrote:
| http://portal.pushbullet.com/
| ignoramous wrote:
| An incomplete list of such projects:
| https://news.ycombinator.com/item?id=22274981
| Sean-Der wrote:
| Do these other ones have native clients? WebWormHole has a Go
| client, so you can run everywhere Go works!
| Unix/Windows/Mobile/Web covers a decent amount of platforms :)
| Sean-Der wrote:
| This is fantastic! Really nice work :)
|
| The nice thing about WebRTC is this works (pretty much)
| everywhere! Someone could throw up Python/Android/iOS/Go/Web/C++
| Clients really easily. That is really exciting.
|
| Also just a HUGE fan of NAT Traversal/P2P in general. The less
| dependence we can have on others for sharing our data the better.
| gigatexal wrote:
| Was it you on the GoTime podcast? If so the enthusiasm for
| webrtc and go just motivated me a ton. Kudos.
| jackewiehose wrote:
| > The nice thing about WebRTC is this works (pretty much)
| everywhere! Someone could throw up
| Python/Android/iOS/Go/Web/C++ Clients really easily.
|
| Huh, what did I miss?! How do you make a WebRTC-client in
| language-of-your-choice? The last time I checked for C++ I only
| found answers like "look at the chrome source". lol.
| Sean-Der wrote:
| Things have changed (alot!) the last two years have seen a
| couple implementations come around.
|
| * https://github.com/pion/webrtc
|
| * https://github.com/aiortc/aiortc
|
| * https://gstreamer.freedesktop.org/documentation/webrtc/inde
| x...
|
| * https://github.com/rawrtc/rawrtc
| wpietri wrote:
| Ooh, these are very cool. Do you know what sorts of things
| people are building with them?
| dicknuckle wrote:
| I'm also curious about what's being built.
| saljam wrote:
| Thanks! There's already a Go client:
| https://github.com/saljam/webwormhole
| Sean-Der wrote:
| I was really excited to see you are using Pion (I am the
| creator) if there is anything I can do to make it better I am
| happy to help :) You should also join https://pion.ly/slack
| and share your project! I posted WebWormHole on
| https://twitter.com/_pion also
|
| I see you are a Recurser (me as well!) I was Summer 2012 I
| think? Are you doing this as a project during your batch? I
| would really love to do a talk/deep dive on WebRTC as a talk
| sometime for Recursers, you should float the idea to Sonali
| and get me in :p
| saljam wrote:
| Pion is great. Thanks for making it!
|
| Definitely give a talk if you can. It's all remote
| currently!
| Naac wrote:
| Neat, looks like a different backend and frontend implementation
| of the very similiar magic-wormhole[0]
|
| Now I wonder if anyone has made a web frontend of the original.
|
| [0] https://github.com/warner/magic-wormhole
| tptacek wrote:
| This is neat, but it seems like unlike with "real" Magic
| Wormhole, the server here can capture files by surreptitiously
| manipulating JS.
| saljam wrote:
| Absolutely true for the web interface if loaded from
| https://webwormhole.io. I'm open for any more suggestions here!
| https://github.com/saljam/webwormhole/issues/13
|
| Someone mentioned the command line client. One can also build
| and serve the html/js/wasm from anywhere and it should still
| work, even with the same signalling server. It has pretty lax
| CORS for this reason.
| StavrosK wrote:
| IPFS would be a solution here, since the files are content-
| addressed. You'd have to fetch them locally, since a gateway
| could still manipulate the content, but it's easier to find a
| gateway you trust.
| mr_toad wrote:
| You can host the code on on your own server if you don't trust
| someone else's. The code is BSD licensed. It should work on a
| static website like GitHub pages.
| Sean-Der wrote:
| There is a native client as well, you don't have to use JS!
| tptacek wrote:
| Neat!
| 0xdeadbeefbabe wrote:
| Who pays for the stun and turn servers?
| inglor wrote:
| Turn servers are expensive but there are plenty of free SatUN
| servers. For example Google offers free stun servers.
| 0xdeadbeefbabe wrote:
| Has the pandemic made ipv6 more popular somehow?
| weaksauce wrote:
| i like the approach of encrypting locally, uploading to the cloud
| and sending the decryption key via a link.
|
| that's the way firefox send does it
|
| https://send.firefox.com
|
| it's open source so you could run an instance of it if you wanted
| to.
| timvisee wrote:
| For the tech savy that like to use the CLI, checkout:
| https://github.com/timvisee/ffsend
| kevincox wrote:
| It is annoying that it removes the ID from the URL. It would be
| nice to bookmark my own code and I can just open it on multiple
| devices whenever I want to transfer a file. However I need to do
| a bit of gymnastics with the QR code to grab the URL.
| saljam wrote:
| Codes are intentionally single use, to limit the bruteforce
| vector. And only two peers can connect any given time
| currently. It would be interesting to figure out how to make it
| work with more than 2 peers!
| jjice wrote:
| I'm not as familiar with WebRTCPeerConnection as I'd like to be.
| Does it use the STUN server to get it's real IP and after that we
| can establish a completely peer to peer connection and now the
| webserver has no interaction with WebRTCPeer stream?
|
| If any of that is wrong, please enlighten me, I didn't realize
| peer to peer connections could be as simple as this.
| Sean-Der wrote:
| That is right!
|
| The only extra thing that STUN does is establish a hole punch.
| It isn't enough to just get your public hole, but you also do a
| temporary 'port forward' to the person that made the STUN
| request.
| algesten wrote:
| that's right. when establishing connection each side enumerates
| a bunch of "ice candidates". which is everything from your
| local LAN to outside NAT ip discovered via TURN servers (some
| restrictions due to privacy reasons).
|
| once the ice candidates are exchanged each side starts spraying
| the other with STUN messages to addresses ranked by "candidate
| pairs" that potentially could make a connection until one is
| found.
|
| this is simplified. there are mechanics like "trickle ice" and
| fallbacks to proxying via TURN servers.
|
| then there's the alleged idea this is all part of a webrtc
| "standard", which is laughable cause no browser follows the
| wild collection of RFCs that supposedly makes up the standard
| and the only reason any of it works is because there's a non
| written down general consensus of what's required.
| waynenilsen wrote:
| We need a distributed db based on webrtc
| dezmou wrote:
| Checkout gundb
| folmar wrote:
| IPFS probably has everything you need and a lot more
| nomel wrote:
| Just curious, why do we need this?
___________________________________________________________________
(page generated 2020-04-29 23:00 UTC)