[HN Gopher] Spacedrive - a cross-platform file explorer, powered...
___________________________________________________________________
Spacedrive - a cross-platform file explorer, powered by a
distributed filesystem
Author : mmmmkay
Score : 219 points
Date : 2022-04-27 21:56 UTC (2 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| anderspitman wrote:
| I love all the cool backend remote storage projects we have, but
| what I really want is a simple, well-specified app protocol for
| filesystem operations. WebDAV is too crufty. I'm bullish on Solid
| but I think there's room for something much simpler. I've done a
| bit of work[0] in this space but keep getting distracted with
| other parts of the self-hosted storage stack.
|
| [0]: https://gemdrive.io/
| spicybright wrote:
| What's crufty about WebDAV? I've never used it, but it seems
| good enough to just use a useful subset of features to avoid
| the cruft.
| anderspitman wrote:
| It uses XML, which requires a library for browser JavaScript
| (EDIT: not true, see comment below), when filesystem
| operations don't really need to be more complicated than
| vanilla fetch requests. Also uses special HTTP methods, which
| isn't ideal. I've heard people say that various
| implementations are often incompatible with each other in
| some ways. I'm not sure how big of a problem this actually is
| in practice, but see for example this readme[0] which
| explicitly says the library doesn't seek to be compatible
| with RFCs, but rather with implementations in the wild.
| That's a red flag to me. See also [1].
|
| Trying to use a subset is an interesting idea, but the
| technical problem being solved is simple enough that I don't
| see a strong reason not to just greenfield it and make a
| bunch of nice improvements along with the critical ones.
|
| [0]: https://github.com/perry-mitchell/webdav-client#about
|
| [1]: https://news.ycombinator.com/item?id=10213657
| hunterb123 wrote:
| > It uses XML, which requires a library for browser
| JavaScript,
|
| What? XMLHTTPRequest (XHR)
|
| People forget XML parsing is baked into JS as is XPath
|
| But I do agree with your premise otherwise.
| anderspitman wrote:
| Ha. Honestly, you're totally right. I've used XHR for
| years but never actually for XML. When looking at WebDAV
| in the past I couldn't find a way to
| serialize/deserialize XML the same way you can with JSON
| (is there a way?), but I didn't realize XHR handles that
| natively. TIL, thanks.
| easrng wrote:
| You can use a DOMParser to parse XML (or HTML) from a
| string. const doc = new
| DOMParser().parseFromString('<xml><tag attr="value"
| /></xml>', "text/xml");
| doc.querySelector("tag").getAttribute("attr") // returns
| "value"
| vineyardmike wrote:
| > what I really want is a simple, well-specified app protocol
| for filesystem operations.
|
| Genuine question, what's wrong with existing Operating System
| FS + FUSE with any remote protocol? Eg there exists S3fuse if
| you want object/bucket storage, sshfs for using ssh to a host,
| and the fuse api is easy enough to port to custom backends (in
| memory, db, etc).
|
| Don't want to be the guy saying "who needs Dropbox we have
| sftp" but you're asking for a technical spec and protocol...
| and one exists? I haven't worked too much in this space so I
| don't know what tasks are hard to do, but it seems like leaning
| on existing file system tools/api at the app level makes sense?
| What's missing, does "over the network" need to be a primary
| consideration for these uses?
| anderspitman wrote:
| The page I linked lists quite a uses cases that would become
| possible/easier with a new protocol.
|
| To your specific concerns, object storage systems like S3
| don't support partial writes to files, which makes FUSE
| mounts a leaky abstraction. SSHFS is much better, but
| obviously can't be used in web apps.
|
| I'm not saying we need to replace existing protocols, but we
| need one that can work in a more diverse set of use cases.
| qbasic_forever wrote:
| 9p? https://en.wikipedia.org/wiki/9P_(protocol)
|
| I believe most VMs use 9p as a low level protocol for syncing
| filesystems between host and VM or container.
| anderspitman wrote:
| 9p won't work for web apps. Personally I think we just need a
| small layer on top of standard HTTP for most use cases. It's
| really not a very hard technical problem, we just need to
| agree on a protocol.
|
| But it's a chicken-egg situation. App developers aren't going
| to develop against a protocol that no one uses, and no one's
| going to use a protocol that doesn't have any apps.
| monocasa wrote:
| Plan 9 (well, 9 front at least) natively supports 9p over
| websockets.
|
| https://p9f.org/magic/man2html/8/websocket
| anderspitman wrote:
| That's pretty sweet, but it's an entire additional layer
| of unnecessary complexity. Plus it still requires
| everyone to implement a new protocol into their apps. Why
| not use one purpose-built for it?
|
| Also, WebSockets bring some nontrivial complexity. This
| can bite you with things like proxies and custom
| backends, which have to implement a special code path
| from normal HTTP to handle WebSockets.
| mike_d wrote:
| Yeah... computers are hard sometimes.
|
| If you want to read and write files, there are plenty of
| operating system APIs to do it super simple. If you want
| low level access to file systems and to get into the
| complex parts... well WebDAV sucks because the underlying
| problems of file systems suck.
| monocasa wrote:
| I basically view WebDAV as the weird one off protocol
| here. 9p runs over just about any stream oriented
| transport (including web technologies), while WebDAV is
| planted firmly in web technologies.
| qbasic_forever wrote:
| I'd tunnel it over a websocket, or there's no reason why
| you couldn't translate its verbs into a REST API. It's a
| super simple protocol with not a lot of complexity.
|
| It sounds like you're after a federated storage protocol
| though, i.e. something that two totally different apps can
| use to allow a user to transfer data between them. This is
| a much, much bigger leap in complexity and requires a very
| strong authentication and authorization story. IMHO look
| into the fediverse and whatever is going on with storage
| and files there.
| mike_hearn wrote:
| Isn't translating filesystem operation verbs into an HTTP
| based protocol what WebDAV does?
| anderspitman wrote:
| Yes, and it's almost what we need. I think it's just too
| ossified to adapt to the few remaining shortcomings.
| Here's a couple:
|
| * Doesn't specify a federated
| authentication/authorization mechanism to allow self-
| hosted instances to work with each other. This includes
| things like how to have an app delegate permissions to
| other apps, ie for things like launcher apps which can
| open external apps and give them access to only specific
| files.
|
| * Doesn't specify a way to implement image
| thumbnails/previews, which is necessary for any sort of
| remote file explorer or gallery app.
| hunterb123 wrote:
| Dupe: https://news.ycombinator.com/item?id=31170815
| xoa wrote:
| One feature a few of us toyed with for years that I'd love to see
| tried seriously someday (or does it already exist?) would be
| intelligent tag trees or maps that stuff could be plugged into,
| ideally that could also be shared and mixed too. More specific
| tags would have parent/child/sibling relationships with more
| generic tags, and a single tag would then pull in all the others
| as far as searching. So for example if I have a photo of one of
| my cats, and I tag it Cat Name having set that up, "calico" or
| "tabby", breed, plain old "cat" and anything else associated with
| the specific tag would all be hits. And up the stack, if I tag a
| picture "calico" it should show up under the "cat" tag as well
| automatically with no further effort. Conditionals could be
| incorporated too, like if there is a character who in a series
| has long hair, tagging "short hair" could automatically pull in
| "alternative hairstyle".
|
| My big problem with typical tagging is that it's a lot of grunt
| work to be comprehensive. AI can help but has its own issues.
| While getting the UI/UX right would take some work, a universal
| system-wide multi-dimensional tag map could allow a lot more
| metadata a lot more easily. Even more so with sharing,
| communities could curate tag maps around areas of interest and
| one could stay synced with tag maps of "aircraft" or "trees" or
| the like which would bring in lots of technical depth to explore
| in a distributed fashion. Eventually maybe community AI
| recognition models could be an optional part of that too to help
| identify specific given subjects (and one would be free to then
| not have other kinds of things identified or shared).
| jewel wrote:
| My homemade photo organization software has the tags as trees
| and it works great for exactly the example you've given, as
| well as "plant" being a parent of "flower" which is a parent of
| "rose".
|
| In addition to better searches, it also allows for better
| tagging, because a lazy tagger might tag a photo as "food" and
| I can come back later and fix it to be "fruit pie".
|
| There was an open ontology database somewhere I might integrate
| so that it has a lot of prefilled tags. That'd be the place to
| start for photos or anything dealing with the physical world.
| rufugee wrote:
| _Spacedrive accounts for every file you own, uniquely
| fingerprinting and extracting metadata so you can sort, tag,
| backup_
|
| Oh...how I've longed for this. How I've wanted to carve out the
| time to do exactly this to service my own needs (as a hoarder). I
| am so excited to use this one it's stable enough to trust
| fingerprinting to the extent that you can dedupe across devices!
| poetril wrote:
| The creator streams his development over on twitch[0].
|
| 0: https://twitch.tv/jamiepinelive
| InCityDreams wrote:
| ...streams his mealtime, too. No thanks - misophonic, me.
| [deleted]
| LeoPanthera wrote:
| Previous thread: https://news.ycombinator.com/item?id=31170815
|
| My comment from the previous thread: I'm not sure if it actually
| does work this way, but this would be amazing if I could install
| a small daemon on my NAS/fileserver to do the scanning and
| indexing, which the graphical client could then pull the results
| of.
| user_7832 wrote:
| Sightly tangential, but does anyone know of any good system that
| integrates everything across devices? I'm talking backups/data
| management, clipboards, maybe notifications too etc. Bonus if
| music/video can be streamed and mouse/keyboards can be used cross
| device.
|
| KDE Connect is nice and can do some of these, and many backup
| utilities like rclone or duplicati (which I've used in the past)
| can do some, but many features are not great. The _closest_ I can
| think of is Apple 's walled garden where you store everything on
| iCloud, but a) the massive restrictions on iOS, and more recently
| on MacOS (see: kext restrictions) are forced and b) it still
| isn't perfect and requires an iCloud subscription if you have
| more than 5gb of data you want synchronized.
|
| I'm curious to know if any HN person knows of such software
| (assuming I can run a server to host the backend). (In real life
| I still have an "unlimited" GDrive account from my university
| that I use to sync but it's far from perfect.)
| smm11 wrote:
| This is potentially awesome, but if I could just get a decent
| front-end to OneDrive on everything, I'd be happy.
| jbverschoor wrote:
| https://mountainduck.io/ oh.. front end.. well it's ok as a
| backend
| [deleted]
| obphuscate wrote:
| Thanks goodness I clicked on the link out of shear curiosity with
| this title - I might not have known it was written in Rust
| otherwise.
| samstave wrote:
| FEATURE REQUEST PLEASE:
|
| As an (ahem) "hoarder" as you so kindly put in the .MD...
|
| Here is a feature I would like: Tag-routing...
|
| I want to create a folder with content tags.
|
| Then, when I go to download something, instead/in addition, I can
| just put in the tag for the media type and it lands in the right
| bucket.
|
| So /media/docs vs media/movies media/pdfs etc... When I download
| a thing - just let me calss tag it as movie and it will be placed
| in movie. etc...
|
| Make a browser extension that takes on downloading, and allows
| for more precise downloading, tagging, sorting of content before
| it ever hits a disk I own...
|
| Like this
|
| https://youtu.be/c0eTHm8NEqE?t=79
| contr-error wrote:
| git-annex has metadata driven views[0], which might be what
| you're looking for. When you're looking at a view, copying or
| moving a file to a folder "tagname", adds the tag "tagname" to
| that file. That seems easily scriptable to tag by media
| type/extension...
|
| [0] https://git-
| annex.branchable.com/tips/metadata_driven_views/
| danieltanfh95 wrote:
| ikr, me too, which is why I built http://ikomi.space/ , wip tho
| AndrewUnmuted wrote:
| Fauntleroy wrote:
| Can you explain how simply having a mask on in his profile
| picture is "propaganda"? Furthermore, how is having a github
| profile picture "anti-scientific"?
| Melatonic wrote:
| Hilariously thats not even the owner of the Github. Just a
| contributor !
| AndrewUnmuted wrote:
| Yeah it's super-funny! I'm having a ball right now!
| [deleted]
| indiv0 wrote:
| Are you boycotting operating rooms while you're at it? I've
| never seen such an overreaction to such a non-issue before. How
| do people get to a state where they can unironically say things
| like this with a straight face?
| AndrewUnmuted wrote:
| Been in an OR since after you were born, OP? The OR
| physicians are in literal spacesuits these days.
| neatze wrote:
| Facebook people invading hackernews :(
| AndrewUnmuted wrote:
| lazyier wrote:
| I don't get it.
|
| You don't like that he is wearing a mask in the github profile?
|
| Are you serious? Am I missing something?
| AndrewUnmuted wrote:
| shaunregenbaum wrote:
| This is genius. Why didn't Dropbox do this?
| api wrote:
| All economic incentives for the past 20+ years have been away
| from "personal" computing because people flatly refuse to pay
| for software but for whatever reason have less aversion to
| paying for services.
|
| I have to repeat just to emphasize how irrational this is.
|
| People will refuse to pay $50 _once_ for software but will
| happily pay $20-$50 _per month_ for a service that offers less
| performance, no privacy, worse security, no continuity if you
| stop using it, fuzzy ownership of your own data, and a system
| that utterly vaporizes if the company goes away.
|
| The aversion to paying for software is intense too. People will
| happily sign up for SaaS but will go through insane contortions
| to pirate software or just flatly refuse to use it if it's not
| capital-F FOSS. Even OSS that you pay for is not okay. The most
| important thing about FOSS is not freedom, openness, or source,
| but "free" as in beer... but this same aversion to paying does
| not transfer to services.
|
| People will also pay $10+ dollars for a single cup of coffee at
| Starbucks every day. But god forbid they pay that much _once_
| for an app they use every day.
|
| Markets are incredibly irrational.
| haswell wrote:
| Is a cup of coffee at Starbucks really $10+ these days? I
| know that place is expensive, but has it gotten _that_
| expensive without me noticing?
|
| The thing I would primarily challenge is the assertion that
| this is fundamentally irrational behavior.
|
| Software is a means to an end. People don't spend money to
| acquire software, they spend money to solve a particular
| problem.
|
| There are two things that come to mind about subscriptions
| vs. the standalone pricing models of the recent past:
|
| 1) Before I'm willing to pay, you need to prove to me that
| your software will solve my problem. That proof doesn't come
| by installing the software, it comes after I've used it long
| enough to see if it does what I need it to do. Time or
| feature-limited trials attempted to solve this, but in a
| time-limited trial, I might solve my problem within the time
| period and never need to buy it (think unzipping a file). Or
| in a feature limited trial, I might get the general sense
| that the thing will work for me, but the software now holds
| my problem hostage, and I think there's a psychological
| component to why that feels "bad".
|
| 2) Sufficiently advanced software will be proportionally
| expensive, and out of reach for the average person. For this,
| I think about apps meant for writers, media and content
| creation, password management apps, etc. Apps that provide
| tremendous value, and therefore could command a high price,
| but now the average user is priced out.
|
| Subscriptions provide two things:
|
| 1) A feeling of safety coming from the fact that I can both
| use all aspects of the software, but also choose to just
| cancel the thing if it doesn't work or me.
|
| 2) The ability to spread out the cost over time. Maybe I
| can't afford $1,000 for <media creation software of choice>,
| but I can afford $20/month right now.
|
| You might argue that someone good with their finances should
| realize that they will eventually pay $1,000 either way, and
| eventually more, and they should just drop $1,000 now, but
| that is also a somewhat privileged position that most of us
| in the SW industry get to enjoy.
|
| It's also rational to opt for a solution that is lower risk,
| i.e. spending $1,000 for something that might _not_ solve my
| problem could be considered irrational.
|
| I generally agree that people continue to struggle with the
| idea of paying for software. But I don't think that's
| fundamentally _the_ problem. What people really want is a
| service that just handles their problems for them, and the
| software is just a means through which they interact with
| that service. The fact that people are willing to pay for
| services at least tells us that people are willing to pay for
| something. This is a hint that the model might be the
| problem, not the fact that money is involved.
|
| And I think this is what has ultimately led to the change in
| licensing models for apps like 1Password.
|
| I think that we as a development community need to accept the
| maybe uncomfortable truth that continuing to try to sell
| software in the traditional ways is itself irrational. It's
| the ultimate "the user is using it wrong".
|
| This is not a phenomenon that is unique to software either.
| Just look at the popularity of low-friction installment plans
| for purchases of many physical products. It's common across
| most major retailers to either plop down your $1,000 now, or
| sign up for monthly payments to spread it out over time.
| avgcorrection wrote:
| We can always trust HN to blame the (l)user rather than the
| businessmen. Even to blame "markets" rather than to blames
| businessmen.
|
| It is of course unfathomable that any businessmen might have
| _leaned into_ and _promoted_ this current model. They could
| have never have _sought out_ to leverage free product+network
| effect in order to build their business on user data. They
| would never. They are after all just businessmen, mere
| puppets of the depraved luser.
| api wrote:
| You are right but both are right. The market and the user
| pushed the idea of free as in beer (confusing it with
| freedom) _and_ businesspeople were happy to do everything
| you describe.
| avgcorrection wrote:
| Alright. I'll take it :)
| nittanymount wrote:
| good points !
| vineyardmike wrote:
| > People will refuse to pay $50 once for software but will
| happily pay $20-$50 per month for a service that offers less
| performance, no privacy, worse security, no continuity if you
| stop using it, fuzzy ownership of your own data, and a system
| that utterly vaporizes if the company goes away.
|
| Daily reminder for HN that if the service you buy is an
| ongoing service then paying once doesn't work. If the company
| has a monthly expense related to servicing you, it makes
| sense to have a monthly payment to them. Dropbox et al. can't
| be "buy once" because that's not how the cost of data centers
| work.
| jl6 wrote:
| Commercial, perpetually licensed software isn't always free
| of future cost. Even if it's only $50 today, you might also
| be on the hook for:
|
| * Mandatory upgrades when v1 ceases to receive security
| patches.
|
| * Mandatory upgrade of dependencies when a new version drops
| support for, e.g. an old OS version.
|
| * Having to buy a replacement anyway when the vendor walks
| away from supporting the software (and why wouldn't they?
| They're not being paid any more).
|
| * Or alternatively, the vendor doesn't walk away from
| support, but charges you for it (effectively becoming SaaS
| anyway).
|
| * The cost of running your own hardware to run the software.
| Dropbox is the obvious example of this - redundant offsite
| storage hardware is a hassle that you can outsource with the
| SaaS option.
| blooalien wrote:
| I was just about to upvote you until I got to the part where
| you start trashing FOSS users as unwilling to pay for
| software. This is a long-time lie of the Linux-hating Win-
| Troll crowd, since at least the Steve Ballmer days of
| Microsoft, and patently bullshit. I've personally paid for
| more (FOSS _and_ closed-source) software since switching to
| using mostly FOSS software than during my many years as a
| proprietary software and MS Windows user, and I 'm not alone
| in that. Humble Bundle _proved_ (before they removed public
| bundle purchase statistics from their website) that Linux
| users are not only _willing_ to _pay_ for software, but are
| often willing to pay _more_ for _decent_ software than any
| typical Windows user will (and most Mac users, too). We also
| tend to understand and respect software licenses more, and
| are generally unwilling to freely sign away our freedoms as
| easily. Problem is that most folks outside of the FOSS
| community too easily confuse free of cost and free as in
| freedom, and assume that the only reason people would use
| FOSS is because it 's often free to download and use. Freedom
| _is_ the reason I use it, and the fact that it 's often also
| free of cost (even though I still pay for some of it) is just
| "icing on the cake" as the old saying goes.
| bachmeier wrote:
| Yep. The move to free software was about much more than the
| price. Pay for MS Office. Use it on the one machine it was
| authorized. Need to reinstall? Hope you kept the
| authorization code. Want to access your data outside of MS
| Office? Sucker. You stuck it all in their (at that time)
| proprietary formats. You bought a laptop? Great. Now you
| can buy another copy of MS Office to go with it.
| api wrote:
| I've been a FOSS user since 1993 and am totally willing to
| pay for software.
|
| Here's the problem: any license that attempt to require
| _businesses_ to pay for software or that restricts for-
| profit SaaSification tends to be rejected by the FOSS
| community. The AGPL is the closest we have and it doesn 't
| really do that even it gets a lot of hate in FOSS circles.
|
| Without such concerns being addressed in the license it's
| impossible to put any kind of direct simple business model
| behind open source software. This means that almost all
| businesses opt for a SaaS model where it's easy to charge
| or find other roundabout ways to "charge" such as
| harvesting user data.
|
| TL;DR: the software market has structurally organized
| itself so as to almost rule out business models other than
| SaaS and surveillance.
| spogbiper wrote:
| > People will also pay $10+ dollars for a single cup of
| coffee at Starbucks every day. But god forbid they pay that
| much once for an app they use every day.
|
| This is why I no longer write mobile apps
| Filligree wrote:
| If it's any comfort, I made that calculation and decided
| not to sweat it, but purchase any app costing less than ~10
| with about as much thought as I'd give to buying coffee.
| Scramblejams wrote:
| I feel similarly, if I see a mobile app that I think
| might solve my problem and it's low single digits, it's
| practically an impulse buy. If it doesn't work out, I
| don't feel used.
|
| I've also happily spent $15 on an iOS app before because
| it was really good at what it did.
|
| It made me wonder if there's a really great market out
| there for an incredibly well done niche mobile app you
| could charge $99 or more for. A whole lot of customer
| service problems disappear when you preemptively
| disqualify the uncommitted...
| teawrecks wrote:
| For the same reason all streaming services and game storefronts
| don't provide a generalized API to be used with 3rd party front
| ends. They all want to pretend that they are the only service
| you'll ever need/use, and their UX isn't designed for
| practicality, it's designed for marketing.
___________________________________________________________________
(page generated 2022-04-29 23:00 UTC)