[HN Gopher] Libgen Storage Decentralization on IPFS
       ___________________________________________________________________
        
       Libgen Storage Decentralization on IPFS
        
       Author : jerheinze
       Score  : 214 points
       Date   : 2020-11-25 13:58 UTC (9 hours ago)
        
 (HTM) web link (freeread.org)
 (TXT) w3m dump (freeread.org)
        
       | Vespasian wrote:
       | I'm hesitant to participate in IPFS in the context of libgen.
       | 
       | Torrenting copyrighted materials (especially if you are
       | distributing) can be quite expensive in some jurisdictions and I
       | do not believe that IPFS is treated different than regular
       | BitTorrent.
        
         | whimsicalism wrote:
         | > Torrenting copyrighted materials (especially if you are
         | distributing) can be quite expensive in some jurisdictions
         | 
         | Why? Your ISP charges for BitTorrent traffic?
        
           | dewey wrote:
           | In case you are serious: Lawyer agencies are joining the
           | swarm, recording IPs, write letters to ISPs and send you
           | fines. That's very common for example in Germany.
           | 
           | Edit: I didn't downvote you if it looks like that :)
        
             | whimsicalism wrote:
             | > write letters to ISPs and send you fines. That's very
             | common for example in Germany.
             | 
             | And you have to pay the fines? That's nuts.
             | 
             | In the US, you are pretty much as likely to get fined for
             | torrenting as you are to get arrested for jaywalking
             | (although without the racial disparity).
        
               | dewey wrote:
               | > And you have to pay the fines? That's nuts.
               | 
               | Yes, conveniently there's a second group of lawyers that
               | focuses on getting people out of these fines...for a fee
               | that's not so much lower. Unfortunately it's a bit of a
               | business model in some countries.
        
               | nemo wrote:
               | In the US the (very) common scenario is to get a couple
               | warnings from your ISP and if you continue torrenting
               | copyrighted material you will be dropped as a customer
               | and banned.
        
               | bboygravity wrote:
               | France was pressured by the US diplomatically to do the
               | same in France. It's called HADOPI.
               | 
               | Source: wikileaks
        
               | whimsicalism wrote:
               | > the (very) common scenario is to get a couple warnings
               | from your ISP and if you continue torrenting copyrighted
               | material you will be dropped as a customer and banned.
               | 
               | This is not "very" common - the ISPs don't really ban
               | you, they just threaten you. I no longer do this, but I
               | extensively torrented over multiple home ISPs and they
               | would send me warnings but they would never actually do
               | anything.
        
               | nemo wrote:
               | AT&T has banned a friend permanently so I would be
               | careful about over-broad generalizations. Folks in my my
               | household were torrenting and we got our service cut with
               | the second offense until we went through some rigmarole
               | promising we'd never do it again at which point they
               | stopped. The cost to an ISP's lawyers to field these
               | things isn't free and at least some really do respond
               | with action.
        
               | whimsicalism wrote:
               | I'm curious if the action taken depends on broadband
               | competition in the area - do you live in a city?
        
         | djeiasbsbo wrote:
         | Check out https://gnunet.org/
         | 
         | It works well for file sharing as well. Don't know about
         | copyrighted material though, have only shared my own files.
        
           | andrius4669 wrote:
           | Does it actually work well? I tried to play with it a while
           | back and while sharing files appeared to work, actually
           | downloading them from other node didn't go that well..
        
             | djeiasbsbo wrote:
             | The thing that doesn't work well yet in my experience is
             | the NAT traversal and finding peers in general. But I
             | tested the same thing with two peers and I could download
             | with no problems. It also seems to get updated quite often,
             | I tested very recently.
        
       | Cactus2018 wrote:
       | Wikipedia
       | 
       | > The InterPlanetary File System (IPFS) is a protocol and peer-
       | to-peer network for storing and sharing data in a distributed
       | file system. IPFS uses content-addressing to uniquely identify
       | each file in a global namespace connecting all computing devices.
        
       | fsflover wrote:
       | I think such materials should better be placed to darknets
       | protecting the privacy of users and seeders, e.g., to I2P.
        
         | LockAndLol wrote:
         | It'd be nice for IPFS to support an I2P proxy. That would be
         | quite an important change and would save anybody downloading
         | from LibGen over IPFS.
        
         | bboygravity wrote:
         | I agree. Actually in this day and age I think all information
         | sharing should better be placed on darknets. I don't see the
         | net advantage to anybody of being profiled constantly on
         | clearnet.
         | 
         | Naive question: is there such a thing as Bittorrent over I2P or
         | otherwise anonymized decentralized file sharing?
        
           | [deleted]
        
           | RealityVoid wrote:
           | I see it as 2 separate problems: data availability and
           | anonymous transfer. IMO, you need 2 separate solutions
           | because they are 2 different problems.
           | 
           | IPFS solved the data availability problem because you can
           | incentivize storage miners to make your data available. This
           | is something new, IMO.
           | 
           | Accessing it anonymously is possible with an extra protocol
           | layer.
        
             | apples_oranges wrote:
             | For instance, using TOR to download books from libgen
             | should be pretty safe, no?
        
           | segfaultbuserr wrote:
           | > _Bittorrent over I2P_
           | 
           | Yes. I2P is designed with P2P in mind. The default Java I2P
           | client already comes with a bundled BitTorrent client, there
           | is also a BitTorrent client called "Vuze" and is used by some
           | non-anonymous users to help seeding a torrent across the
           | clearnet and I2P simultaneously. Because of the nature of
           | I2P, don't expect great speed. No seeder is "usual", 10 KiB/s
           | is "normal" and 50 KiB/s with only 3 or 5 seeders is "good",
           | overall, it feels as if it's still the early 2000s, but when
           | you get used to it and adopt a "download and forget, come
           | back in 3 days" mentality, it's not actually that bad. The
           | fastest download I've ever seen was the movie _Snowden_ , 20+
           | seeders and over 200 KiB/s+...
        
         | derefr wrote:
         | If you take IPFS's model and add inherent anonymity, that's
         | exactly what Freenet is. (To this day I'm confused about why
         | IPFS is popular while Freenet isn't.)
        
           | chriswarbo wrote:
           | I've not tried Freenet for a few years, but every time I did
           | it was _very_ slow; e.g. browsing basic HTML sites hosted on
           | Freenet was quite painful. IPFS can serve streaming video,
           | and host massive archives like Wikipedia.
           | 
           | Also, Freenet actively seeks out data from the network to
           | store locally, to give plausible deniability to users
           | (there's no way to tell if something is locally cached
           | because the user requested it, or whether the client fetched
           | it automatically). IIRC this can be disabled by limiting
           | oneself to a whitelist of peers (preventing network chatter
           | that might reveal Freenet's presence). IPFS will only cache
           | what's requested, so it's more predictable and useful for
           | building infrastructure (e.g. hosting assets for a normal Web
           | site); and presumably leaner on data requirements too.
           | 
           | IPFS isn't really competing with Freenet IMHO; it's competing
           | with HTTP and BitTorrent.
        
             | eMGm4D0zgUAVXc7 wrote:
             | > I've not tried Freenet for a few years, but every time I
             | did it was very slow
             | 
             | It has become a lot faster recently, try again :) When I
             | look at its statistics during active usage it can easily
             | show 1 MiB/s traffic.
             | 
             | You still can't expect HTML to load in sub-second delays,
             | for large multi-MiB sites it may take a low single-digit
             | amount of seconds, but there is an inherent cost to
             | anonymity which probably induces a boundary on how fast
             | things can be:
             | 
             | To achieve anonymity, data needs to be _redirected_ across
             | multiple people so the sender and recipient can 't
             | determine who each other of them is.
             | 
             | Redirecting stuff across a longer path than necessary slows
             | things down.
        
               | bawolff wrote:
               | > To achieve anonymity, data needs to be redirected
               | across multiple people so the sender and recipient can't
               | determine who each other of them is.
               | 
               | >
               | 
               | > Redirecting stuff across a longer path than necessary
               | slows things down.
               | 
               | That's skipping past a lot of details. There are still
               | choices there. Freenet traded speed, efficiency and
               | scalsbility for a specific security model. There are
               | anonoyminity systems (e.g. tor) that are much more
               | efficient but have different trade-offs.
        
           | Mediterraneo10 wrote:
           | Freenet didn't take off in part because there one hosts all
           | kinds of distributed files you don't know the content of,
           | because they are encrypted, but if someday that encryption
           | algorithm is broken, there is a strong chance many people
           | were (unwittingly, but still) hosting child pornography.
        
           | fsflover wrote:
           | What is the point of Freenet when we have I2P, which allows
           | any protocol on top of it, not just file sharing?
        
       | dooglius wrote:
       | A strange choice given that distributing copyrighted materials
       | violates the IPFS Code of Conduct [0]. Why not use BitTorrent
       | whose attitude toward copyright is much more in sync with
       | libgen's?
       | 
       | [0] https://github.com/ipfs/community/blob/master/code-of-
       | conduc...
        
         | toomuchtodo wrote:
         | > given that distributing copyrighted materials violates the
         | IPFS Code of Conduct
         | 
         | Everyone has wishes, but wishes aren't enforceable. It's the
         | equivalent of "Stop! Or I'll say stop again!"
        
         | josu wrote:
         | Yeah, and the fact that they've already decided to blacklist
         | sci-hub.
         | 
         | https://discuss.ipfs.io/t/mirror-of-sci-hub-in-ipfs/1613
        
           | arsome wrote:
           | Am I missing something? I don't see any mention of a
           | blacklist in that thread. They did close it because of
           | course, they don't want to be in Limewire's position, where
           | they get held responsible for the tech they built because
           | they supported, even in a minor way, its use in piracy.
        
         | m-p-3 wrote:
         | > distributing copyrighted materials violates the IPFS Code of
         | Conduct
         | 
         | Even if so, what can they (the IPFS devs) even do about it?
        
         | segfaultbuserr wrote:
         | LibGen already has "torrent per 1000 files" for a long time,
         | some books also have their individual torrents but it's not
         | common. I'm not sure why, but I think the problem is that files
         | on BitTorrent cannot be directly addressed by hashes, it had to
         | go through an intermediate step called a "torrent", which does
         | not uniquely identify individual files directly. You can always
         | select the one file you need from a huge torrent, but it's not
         | as convenient as directly addressing files by their hashes.
        
           | dooglius wrote:
           | That's fair, BitTorrent v2 does solve this problem, but I
           | can't say I've seen any v2 torrents around (not that I'm
           | specifically looking)
        
         | elvis70 wrote:
         | Ah, thanks for mentioning that. If I understand well, it's a
         | distributed filesystem but there is still an authority able to
         | ban users or contents?
        
           | diggan wrote:
           | The answer for both BitTorrent and IPFS is: No.
        
             | m-p-3 wrote:
             | They can blacklist hashes on the ipfs.io reverse proxy, but
             | they cannot block you from accessing those through a
             | different node like your own.
        
         | diggan wrote:
         | Protocols don't have attitudes, everything you're using IPFS
         | for you could also use BitTorrent for (almost). The protocols
         | don't really care about what data you send, and no one can stop
         | you from using them, BT or IPFS.
         | 
         | The CoC you've linked covers the IPFS Community and other
         | Protocol Labs initiatives, it doesn't cover usage of IPFS
         | itself. freeread.org is not bound by that CoC.
        
           | josu wrote:
           | The protocol can't, but the network can.
           | 
           | https://discuss.ipfs.io/t/mirror-of-sci-hub-in-ipfs/1613
        
             | diggan wrote:
             | Your link is to a forum hosted by Protocol Labs for the
             | IPFS Community. Of course they are trying to follow their
             | own CoC on their own properties, especially when it comes
             | to copyright infringement. As a US company, they're bound
             | to follow US law. Hardly surprising.
             | 
             | What does that have to do with the network? Again, the
             | network nor the protocol currently has nothing in place
             | that describes a "attitude" of being against copyright
             | infringement.
             | 
             | I remember that we (I'm a ex-Protocol Labs employee) used
             | to have plenty of discussions about adding allow/blocklists
             | to the protocol that people could opt-in to, but don't
             | think that was never added (yet?). If it was, I'd
             | understand where you're coming from.
        
             | georgyo wrote:
             | They are specifically asking for help and guidance to break
             | the law. Any public community has to avoid those questions
             | like the plague.
             | 
             | Remember youtube-dl got taken down because it's README file
             | referenced copyrighted material.
             | 
             | It doesn't matter if a tool _could_ be used distribiting
             | copyright material. The tool (and communities) cannot
             | promote anything related to that use case at all.
             | 
             | The same is true with bittorrent. Bittorrent _could_ be
             | used to download copyrighted music and movies, but you
             | won't find a single reference to anything like that use
             | case on their web page because that is illegal:
             | https://www.bittorrent.com/
             | 
             | In fact the bittorrent terms of service says you may not
             | use the tool for any illegal purposes:
             | https://www.bittorrent.com/legal/terms-of-use/
        
           | dooglius wrote:
           | It's more that just the protocol though. TFA links directly
           | to the IPFS Desktop Client, which certainly can have an
           | attitude e.g. if a future update blocks this project. That
           | risk doesn't really exist for any mainstream torrent client.
        
       | donpdonp wrote:
       | The linked article does not introduce the project, nor does the
       | project site have an about page. wikipedia to the rescue!
       | 
       | https://en.wikipedia.org/wiki/Library_Genesis
        
       | MichaelMoser123 wrote:
       | does anyone know why libgen is still running over http? (But
       | sincerely thanks for the site! it saved me on several occasions!)
        
         | generationP wrote:
         | They don't want the extra weak spot of certificate revocation
         | being abused against them?
         | 
         | Though I'm not sure how big of an issue this is; sci-hub is
         | using https.
        
           | jdu9 wrote:
           | Unless they activate HSTS, it shouldn't be a problem. They
           | can always go back to HTTP if they want.
           | 
           | Also, Let's Encrypt doesn't even revoke certificates for
           | malware and phishing sites. [1]
           | 
           | Maybe they're doing this for better caching performance? It
           | makes SSL flood attacks impossible, but is that really worth
           | it?
           | 
           | [1] https://community.letsencrypt.org/t/how-to-report-
           | abuse/4110...
        
         | warmwaffles wrote:
         | Probably because they don't care enough to add https.
        
         | dublinben wrote:
         | One of the new IPFS-backed mirror sites (https://libgen.fun/)
         | is on HTTPS.
        
       | segfaultbuserr wrote:
       | The IPFS mirror really gives a substantial performance boost. In
       | the past, the files had to be fetched from a tired server far
       | away in Russia, the speed can be as low as 0.5 Mbps in my
       | experience (depends on your ISP and connectivity), now it can be
       | lightning fast thanks to P2P, 2 Mbps+, more than adequate to
       | download a 100 MiB+ file. You don't even have to install IPFS
       | (but you probably should, to maintain decentralization), books
       | can be downloaded straight via HTTPS from Cloudflare's IPFS
       | reverse proxy (called a "gateway"), cloudflare-ipfs.com.
       | 
       | The IPFS network itself cannot be taken down, but I wonder when
       | will the publishers discover IPFS web gateways, and start filing
       | DMCA takedown requests against them, and what are the gateways
       | gonna do? Will Cloudflare implement a huge copyright blocklist or
       | even terminate its service (or worse, send a list of IP addresses
       | to Prentice Hall or Elsevier)? What about the semi-official
       | ipfs.io gateway? And the smaller independent IPFS gateways (there
       | are currently 10+ on the public web)? Will they be DMCAed out of
       | existence?
        
         | an_opabinia wrote:
         | > send a list of IP addresses to Prentice Hall or Elsevier
         | 
         | I don't know, do you think nobody has noticed LibGen by now? Do
         | you think the reason they're still running is because of some
         | jurisdiction issue? Even the lawyers would be wrong on that
         | one.
        
           | segfaultbuserr wrote:
           | > _do you think nobody has noticed LibGen by now? Do you
           | think the reason they're still running is because of some
           | jurisdiction issue?_
           | 
           | I'm not sure about the implication of your words. But,
           | clearly, Elsevier has already tried to take legal actions
           | against LibGen previously, and once in a while, LibGen's
           | domain names are still being blocked and routinely replaced,
           | just like Sci-Hub. I'd still say the reason it's still
           | running is a jurisdiction issue, or let's say it's a
           | geopolitical issue, it's using Russia to shield itself away
           | from US influences.
           | 
           | But an IPFS gateway hosted in the United States by Cloudflare
           | does not enjoy these legal and political benefits.
        
         | tutfbhuf wrote:
         | IPFS gateways are similar to Tor exit relays, they just route
         | the traffic to the public internet. Tor exit relays still
         | exist, and haven't been DCMAed out of existence so far.
         | 
         | One could also imagine IPFS over Tor.
        
           | georgyo wrote:
           | You're almost right. The IPFS gateway nodes also cache the
           | content. IE, they are both storing, hosting, and sharing to
           | other IPFS nodes.
        
             | mpfundstein wrote:
             | how can one validate that the cached version is indeed bit
             | equal to the one stored within ipfs?
        
               | openfuture wrote:
               | You hash it, everything is content addressed.
        
               | georgyo wrote:
               | The address of the file is the hash of the file. (Really
               | the hash of the DAG of the file)
               | 
               | So you could validate it if you wanted to. Because the
               | URL _is_ the hash, if you did validate it you could be
               | very sure that it is correct.
               | 
               | If a webserver hosts a file, how can you be sure that
               | file is correct? Sometimes the ship a sums file next to
               | it. At best this can find corruption, but it would not
               | find malicious files.
        
           | mintplant wrote:
           | IPFS-over-Tor has been trapped in development hell for 5
           | years.
           | 
           | https://github.com/ipfs/notes/issues/37
        
             | segfaultbuserr wrote:
             | If IPFS-over-Tor is in the development hell, IPFS-over-I2P
             | is in the development limbo.
             | 
             | https://github.com/ipfs/notes/issues/124
        
           | segfaultbuserr wrote:
           | Tor exit nodes are forward proxies, they are used by the
           | client-side to initiate connections to the public Internet
           | but do not accept connections from the public Internet. You
           | cannot host an Internet-facing webserver on a Tor exit (Onion
           | Service exists solely within the Tor network), DMCA is not an
           | issue, they do not act as a point of content distribution,
           | there is nothing to take down. The main issue is the abusive
           | outgoing traffic, which is seen as unavoidable and outweighed
           | by Tor's greater benefits, so the existence of Tor exits is
           | justified.
           | 
           | IPFS gateways (and Tor-to-Web) are reverse proxies, they are
           | configured at the server-side to accept connections from the
           | public Internet and route traffic to IPFS (and Tor), and they
           | distribute content to the public Web. As far as I can see,
           | this is a problem, someone can send DMCA notices, asking the
           | gateway operators to take files down from the Web. The only
           | protection I see is DMCA Section 512 (a.k.a Safe Harbor
           | Provision), the operators are not liable for things solely
           | hosted on IPFS, which is good, but gateways must comply with
           | the takedown notices. So I think it's entirely possible for
           | Cloudflare to put a huge blocklist on its IPFS gateway, and
           | for smaller gateways that don't have the necessary resources
           | to handle the requests, be DMCAed out of existence entirely.
           | In the 2000s, RIAA launched campaigns against eD2k servers
           | and BT trackers, including the use of honeypot servers to
           | collect information. RIAA was not ultimately successful, but
           | it created a major short-term disruption (eventually new
           | servers would always appear in a different jurisdiction). A
           | similar campaign against IPFS gateways sound possible if the
           | publishers decided to launch an aggressive crackdown like the
           | RIAA. The question is only a matter of whether the decision
           | is made.
           | 
           | I'm not saying that IPFS requires the use of IPFS gateways,
           | they don't, and in a sense, gateways are counterproductive
           | for the decentralization goal. But they do currently provide
           | an useful service for the Web, and I'm just speculating
           | whether a major disruption is possible.
        
         | traverseda wrote:
         | In my experience IPFS is _only_ fast when you 're using it
         | through something like Cloudflare's IPFS proxy (which is
         | basically a caching proxy). I haven't found IPFS to be actually
         | usable any time I've tried running a node myself.
         | 
         | Is that unusual?
        
           | magila wrote:
           | This has been the state of IPFS for years now. The protocol
           | and client haven't been developed to where they can work
           | reliably in a real P2P scenario. IPFS only appears to work
           | because most requests go strait to one of the big gateways
           | which already has the content cached.
           | 
           | Development of IPFS slowed to a crawl when most of Protocol
           | Labs switched to working on Filecoin. It remains to be seen
           | if either Filecoin or IPFS will ever be viable for their
           | intended purpose.
        
             | dsun179 wrote:
             | Yes, this is still the case. For a file which is on exactly
             | one other computer, you have about 45 seconds lookup time
             | before the download can start.
        
             | georgyo wrote:
             | I'm not sure you are right about this. The following link
             | will stream big buck bunny directly from the network with
             | full P2P support. It is not using a gateway node for the
             | video content.
             | 
             | https://bbb.fu.io/
             | 
             | You can run the following in the console to see all your
             | peers                   for await (const peer of await
             | node.swarm.peers()) { console.log(peer.addr.toString()) }
             | 
             | The console currently get's spammed pretty hard because
             | they haven't implemented filtering out plain ws (non-wss)
             | peers if you are connecting https. But it will use webrtc
             | to find peers, and find all wss nodes in the network.
        
               | magila wrote:
               | For me at least the vast majority of the data is being
               | downloaded from various servers under bootstrap.libp2p.io
               | or preload.ipfs.io. These are clearly dedicated servers
               | not true peers.
               | 
               | The contact info for these servers appears to be
               | retrieved via the DNS rather than the DHT. This isn't
               | surprising because browser clients can't participate in
               | the DHT, but the DHT is currently one of the biggest weak
               | points of IPFS. IPFS's DHT is so chatty that it consumes
               | over a megabit of throughput just to maintain an idle
               | node. Lookups are excruciatingly slow and often fail
               | completely.
        
               | georgyo wrote:
               | Sorta, a peer can advertise dns addresses, and the
               | browser can participate in the dht as a client. The
               | example above uses webrtc-star for browser to browser
               | discovery.                   $ ipfs dht findpeer
               | 12D3KooWJxNHY6zE1KnzFdBJrTMCbhCVNtSLvjp7qR5Wsp49DnUC
               | /dnsaddr/ipfs.scalable.io
               | /dns4/ipfs.scalable.io/udp/4001/quic
               | /dns6/ipfs.scalable.io/tcp/443/wss
               | /dns6/ipfs.scalable.io/udp/4001/quic
               | /dns4/ipfs.scalable.io/tcp/443/wss
        
             | derefr wrote:
             | IPFS is the engine of Filecoin, though. Filecoin just
             | incentivizes IPFS nodes to store things. So further IPFS
             | development _should_ still be necessary from the
             | perspective of making Filecoin development /adoption
             | happen.
             | 
             | (I say this, and yet I know it isn't true in practice. But
             | _why_ isn 't it true?)
        
               | nwah1 wrote:
               | Pump up Filecoin price. Sell Filecoin. Move to Fiji.
               | Don't do any coding while in Fiji
        
           | the_duke wrote:
           | No.
           | 
           | The last time I tried it (maybe 9-12 months ago) it was a
           | real resource hog (including choking the network) and really
           | slow.
        
           | swills wrote:
           | Are you opening the required ports (4001, 8080)? I found it
           | was like that too until I did opened the ports.
        
           | segfaultbuserr wrote:
           | Interesting, didn't notice that before. I believed my
           | downloads are too obscure to be cached by Cloudflare, but I
           | may be wrong. I'll do a comparison once I fixed my server at
           | home...
        
         | mathnmusic wrote:
         | And what about when browsers (like Brave) start supporting IPFS
         | natively, and gateways are not needed any more?
        
         | jakobdabo wrote:
         | Am I understanding it right that DMCA takedown requests apply
         | to a URL?
         | 
         | Maybe IPFS can introduce a feature to generate one-time URLs,
         | e.g. `oneTimeURL = f(realURL);`.
         | 
         | Websites then can generate new URLs for example every hour or
         | even for each client request. Copyright bots will file a DMCA
         | request to a URL which only they have.
        
           | judge2020 wrote:
           | In general DMCA takedown requests reference a URL to locate
           | infringing content, but the actual requests ask that the
           | content itself be removed and not just the URL.
        
       | frrad wrote:
       | I wrote this https://github.com/frrad/skyhub the other day to
       | provide a nicer interface on top of the scihub torrents.
        
       | kristianpaul wrote:
       | "- Recommended at least 16GB RAM and Intel i5 or equivalent
       | processor - Recommended at least 100 mbps, gigabit connection
       | preferred"
       | 
       | And I was going to try this one of my embedded computers :(
        
       | petra wrote:
       | What's missing from this system(and P2P systems in general), is
       | automatic pinning - i.e. the system automatically choses which
       | files to pin on which user, to ensure every file is available on
       | the net.
       | 
       | It's a bit of tricky problem, because people may not like
       | unknown, random files on their pc, and probably, because such
       | algorithm is distributed and will need to resist attacks.
       | 
       | But i think that the payoff is big: what Bitorrent did to popular
       | files, such an algorithm could do to rare files.
        
         | rudolph9 wrote:
         | Ipfs has something called "bitswap" which solves some of what
         | you're describing but is lacking in areas such as long term
         | file availability. That's where filecoin sort of finds sort
         | fills in the gap.
         | 
         | https://docs.ipfs.io/concepts/bitswap/
        
         | dboreham wrote:
         | We built a simple pinning system for a project I'm involved
         | with: each node queries a blockchain-based database for a set
         | of resources to pin locally. Resource publishers pay blockchain
         | fees to advertise their data for pinning.
         | 
         | We also found it necessary to build a kind of "overlay" to the
         | p2p network (how each peer selects its peers) because the
         | default algorithm produced too sparse a network. The
         | probability of one node ever being connected to another node
         | that holds content it would like to sync approaches zero. The
         | topology for the overlay network is also fetched from said
         | blockchain.
        
         | corey_moncure wrote:
         | If the pinning came with encryption where the key is not known
         | to the pinned system, I think most would be OK with it. There's
         | no way to know, or demonstrate that anyone could know, the
         | contents of any given file. Furthermore, the file could be
         | broken into encrypted chunks and distributed redundantly to
         | many users. Kind of like BT turned inside out.
        
           | arsome wrote:
           | This is pretty much exactly how Freenet works.
        
           | derefr wrote:
           | https://en.wikipedia.org/wiki/Perfect_Dark_(P2P) is this kind
           | of system. You allocate a slice of your disk for the program
           | to use as a global cache, and random encrypted chunks from
           | the network end up downloaded into it.
        
         | chriswarbo wrote:
         | See the comment above about Freenet, which does something like
         | this; although according to
         | https://freenetproject.org/pages/documentation.html#content it
         | seems to prioritise popular files rather than rare ones.
         | 
         | IIRC eDonkey prioritised rare files, although I don't know if
         | this was a property of the network or just a common choice in
         | clients.
        
           | eMGm4D0zgUAVXc7 wrote:
           | There is a Freenet "KeepAlive" plugin which can be used to
           | selectively pin files which you want to preserve, even if
           | they are completely unpopular.
        
         | crest wrote:
         | https://cluster.ipfs.io/
        
           | rudolph9 wrote:
           | This doesn't automatically pin, rather distributes a pin set
           | across a set of trusted nodes.
           | 
           | It's good stuff nonetheless.
        
       | fareesh wrote:
       | Is book piracy a major concern for the publishing industry? Many
       | avid readers that I know are happier buying the physical book
       | because they like the feel of reading from a book. Have others
       | found this to be true as well? I wonder if younger folks are less
       | susceptible to this feeling.
        
         | Synaesthesia wrote:
         | Most people don't know how to pirate books but yeah it is. I
         | got a lot of university textbooks this way.
        
         | contravariant wrote:
         | Many avid readers I know are happy with physical books but also
         | happy to use an e-reader for purely pragmatic reasons (easier
         | to carry, custom font settings).
         | 
         | As for piracy if the places I see libgen and it's kind
         | referenced are any indication then this isn't about just any
         | readers but about students, professors etc. who want to access
         | scientific publications.
        
       | justarandomq wrote:
       | Is LibGen safe? I remember testing some of its mirror links on
       | virustotal and saw some red flags, so I decided not to risk
       | downloading.
        
         | Synaesthesia wrote:
         | Yes it's just ebooks.
        
           | mpfundstein wrote:
           | and scientific papers
        
         | mpfundstein wrote:
         | what red flags did you see?
        
           | justarandomq wrote:
           | It was some book, and when submitting the download link, 3 of
           | the antivirus results were red. Perhaps they were false
           | positives.
        
       ___________________________________________________________________
       (page generated 2020-11-25 23:00 UTC)