[HN Gopher] Engineering Dropbox Transfer: Making simple even sim...
___________________________________________________________________
Engineering Dropbox Transfer: Making simple even simpler
Author : nicksundin
Score : 110 points
Date : 2020-06-26 16:06 UTC (6 hours ago)
(HTM) web link (dropbox.tech)
(TXT) w3m dump (dropbox.tech)
| gaogao wrote:
| This was a great writeup! I loved start to finish view, that
| didn't get bogged down in technical details.
| Eric_WVGG wrote:
| wow, that was a very exhaustively informative explanation of how
| to clone WeTransfer
| [deleted]
| jrochkind1 wrote:
| > helped us break away from preconceived notions based on what is
| easy and incremental to build on top of the Dropbox stack.
|
| In my experience, this is one of the biggest challenges for
| developers involved in product design, expressed succinctly
| there.
|
| And to try to step out of this is why it's so important to have
| product owners/managers who are NOT technical staff, and who
| technical staff doesn't unduly influence. When you've invested
| your time and energy in building a hammer, you really want to
| figure out how to treat anything as a nail.
| yjftsjthsd-h wrote:
| > When you've invested your time and energy in building a
| hammer, you really want to figure out how to treat anything as
| a nail.
|
| And to be fair, this is reasonable, right up until it isn't.
| When every tool costs onboarding and maintenance effort, it
| makes sense to leverage as few tools as possible to the
| greatest effect - but much as "everything should be made as
| simple as possible, but not simpler", you want the fewest tools
| that will do the job well, but sometimes that _is_ N+1, and a
| new tool _will_ pay for itself.
| jrochkind1 wrote:
| Absolutely.
|
| But it's really hard to tell when it is and isn't reasonable,
| when it comes to UX.
|
| I find that the only thing you can do is try to have peopel
| who aren't developers and don't even _know_ what 's "easy to
| build incrementally on top of the existing stack" determining
| the appropriate UX for the customer/market/business need.
|
| Then the developers can say "OK, but if we did it like THIS,
| it would be a lot cheaper to implement because we can
| incrementally add to what we've got", and the product
| manager/owner can push back "Eh, that's not going to meet the
| market need", "OK, but we can't afford it" -- it's a
| negotiation. But in my experience you really have to have
| someone who isn't a developer at all standing up for the user
| need, for anyone who will be involved in the implementation,
| no matter how smart and user-centered, it's just too tempting
| to decide that the thing that can be met with the incremental
| change to existing stack is "nearly as good" no matter what,
| when it's _so_ much easier and more elegant under the hood.
| pmccarren wrote:
| This is awesome! Firefox Send[0] is another very similar product
| in the space. Excited for more file transfer players!
|
| - [0] https://send.firefox.com
| newscracker wrote:
| And Firefox Send has much higher limits too, for free. Without
| a Firefox Account, the user can upload up to 1GB. And with a
| Firefox Account, the limit is 2.5GB. Dropbox Transfer, in
| comparison, provides (just) 100MB in the free tier.
| knjoy wrote:
| Yea but it doesn't seem like Firefox Send lets you right
| click on any file anywhere on my computer and just right
| click to Transfer. The uploading with Dropbox Transfer
| (through the desktop client) is super reliable too -- I have
| Time Warner internet and when it invariably craps out, my
| upload auto-resumes. Looks like this here:
| https://www.dropbox.com/t/jJNe5IBPrezaw7aE
| knjoy wrote:
| As an existing Dropbox user, Firefox Send or WeTransfer is not
| as good a product for me. Because I already have the Dropbox
| client installed for file storage, I don't need to visit any
| websites to upload to Transfer. I can literally just right
| click on any file anywhere on my laptop and click "Transfer" --
| then it automatically pops up, I can add more files into it,
| rename it, etc. and then just send an email or copy link. I
| screenshot how it looks here
| https://www.dropbox.com/t/jJNe5IBPrezaw7aE (and it's so easy I
| find myself just automatically use it). What I really like
| about it is that it sends a copy of the file and my grandma
| won't just delete my photos like the good ol' Dropbox links.
| kyawzazaw wrote:
| Is visiting send.firefox.com that much of a friction?
| dhosek wrote:
| Compared to the dropbox option, yes.
| tananaev wrote:
| Please make Dropbox Transfer email notifications configurable. I
| use it a lot and I don't want to be spammed with emails.
| nicksundin wrote:
| This is something we started providing recently (based on user
| feedback we heard, like yours). There's now a "let me know when
| someone downloads" checkbox when you're making a Transfer (both
| on Desktop & web)
| tananaev wrote:
| I see it now. Thanks for adding this feature. It would also
| be nice to control other notifications. For example, an email
| notification for transfer creation doesn't really make sense
| in 99% of the cases. And I think ideally all those controls
| should be in the main Dropbox settings (notifications tab).
| Or at least defaults should be there.
| toomuchtodo wrote:
| Can you also support "Let me know when the download completes
| successfully?" I assume this is easy if the recipient is
| adding to their own Dropbox account, or downloading to their
| client using a Dropbox client, but may be slightly more
| tricky with a browser download.
|
| EDIT: In the event my comment was ambiguous, I meant these
| notifications should be configurable by the sender.
| Essentially "delivery confirmation" for bits.
| nicksundin wrote:
| We're definitely looking into ways to make the recipient
| side smoother. Stay tuned ;)
| faitswulff wrote:
| Does anyone know if there's a CLI tool to transfer files? Ideally
| I'd want to give it a file as an input and then have it return a
| shareable URL that transfers the file directly from my computer
| to whoever clicks the URL.
| dguido wrote:
| yep, https://github.com/warner/magic-wormhole
| faitswulff wrote:
| I did look at magic-wormhole, but it's difficult for non-tech
| people to use since, IIUC, you need to use the command line.
| [deleted]
| ColanR wrote:
| So you do or you don't want a CLI tool?
| pornel wrote:
| Firefox Send has a CLI: https://github.com/timvisee/ffsend
| timvisee wrote:
| Thanks for sharing, dev here. It's an unofficial tool, but
| fully featured for sure. Supports link shortening or QR code
| generation as well.
| dguido wrote:
| Firefox Send, SendSafely, and Magic Wormhole are all end-to-end
| encrypted.
|
| https://send.firefox.com/
|
| https://www.sendsafely.com/
|
| https://github.com/warner/magic-wormhole
|
| https://webwormhole.io/ (magic wormhole over WebRTC!)
|
| This seems like table-stakes for a modern file transfer system.
| Accepting unencrypted files and storing them temporarily in the
| clear on your own servers seems like it only introduces tons of
| additional risks without much gain.
| cryo wrote:
| cryo a new file manager also supports end-to-end encrypted file
| transfers.
|
| https://cryonet.io
|
| For confidential content OnionShare is another good option.
|
| https://onionshare.org/
| zaroth wrote:
| If I might ask a lazy question - do any of these work with
| basic (e.g easily available on a lightweight container) command
| line tools on the terminal? I assume not since they must all be
| Javascript-based.
| ColanR wrote:
| magic wormhole is a command line tool.
| pezdeath wrote:
| Found this: https://github.com/timvisee/ffsend
| timvisee wrote:
| Thanks for linking, dev here. Yes, perfectly usable for
| automation!
| yahyaheee wrote:
| Not sure why it still stakes hours to days to upload a video from
| my iPhone.
| nicksundin wrote:
| Author here. Happy to answer questions!
| rkagerer wrote:
| User here. Just want to say:
|
| 1) Thanks for your hard work advocating for us and for that
| interesting writeup, and
|
| 2) This was never a problem I needed solved until Dropbox
| killed the Public folder four years ago. It took exactly two
| clicks to create a direct link I could send to anyone, and it
| just worked.
| godzillabrennus wrote:
| Great work on this. I had to send 100GB files last year (1TB in
| total) and it was extremely difficult to send them as-is. Most
| clients that upload data choked while attempting it. Had to
| break the files up into smaller ones using a compression tool.
| It actually increased the file size overall but the transfer
| could then be facilitated.
|
| Hope to see you guys add a feature like DocSend next. It's like
| a missing puzzle piece from a utility perspective. I have no
| idea who downloads a shared file once a url is generated. Would
| be nice to request emails and get alerts when it's downloaded.
| aroch wrote:
| If you use 7zip on Windows or just split/cat on macOS/Linux,
| you can split without compressing or archiving, saving you
| unnecessary CPU cycles (and space).
| jdxcode wrote:
| You gotta wonder how much easier it would be to put a stamp
| on a letter with an SD card in it sometimes
| [deleted]
| raun1 wrote:
| No Syncthing, rsync, or multi-connection FTPS/SFTP with split
| archives?
| nicksundin wrote:
| Hi! Sending groups of files was definitely one of the focus
| areas here so hope Transfer can be helpful in this role.
|
| Re: Tracking content, we actually do provide this in
| Transfer. If you specify a set of email recipients, rather
| than copying a shared link, we provide a few tracking
| features for you.
|
| 1) Record who has viewed or downloaded your content. 2)
| Remind your recipient if the content you sent is about to
| expire. 3) (Optionally) notify you to let you know when your
| recipient has downloaded your content.
| pkamb wrote:
| "Send with Transfer..." pollutes the context menu of every
| folder on my Mac, not just in the Dropbox folder. Is there a
| way to remove that menu item?
|
| https://apple.stackexchange.com/questions/389415/remove-drop...
| butz wrote:
| Same issue on Windows. If you are adding new context items
| automatically, please have decency to add option to remove
| them, without hacking the registry. Also, please do not add
| it back in later update, which happen quite often.
| psadri wrote:
| What was the most technically challenging part of the project
| for you? And do you do any tricks for handling failures during
| very large downloads or uploads (Eg ranges)
| nicksundin wrote:
| The most technically challenging part was probably wiring the
| storage & sharing side up to not behave like a traditional
| Dropbox shared folder or shared link (e.g. not take up
| storage quota or sync down to your machine). There were
| certainly a long tail of internal reliability work to get
| that operating smoothly (there are many internal services to
| work with here).
|
| In terms of tricks at handling failures:
|
| The first (and probably most important) thing is finding the
| failures. Bug bashing sessions and also just reading failure
| logs can help catch the trickier transient problems our
| integration or unit tests do not catch.
|
| In terms of mitigating them, retries and correct error
| handlers in code are essential (e.g. Promise.catch case
| should not be ignored!). Retries on the client side are made
| a bit easier as we chunk uploads into 4mb blocks so any
| upload errors of actual content can be retried piecemeal.
| WJW wrote:
| Why is there so much talk about hypotheses and market
| validation in the article when the product is a straight up
| clone of WeTransfer almost down to the name?
|
| For those who don't know: compare the visual design of
| https://www.dropbox.com/transfer with the design of
| https://wetransfer.com/. Design and location of key elements
| are super similar.
| txcwpalpha wrote:
| Even if the internal discussions were "other services do this
| and we want a piece of that pie", no company is going to come
| out in a product launch announcement and directly say
| "another service is doing this and we want a piece of that
| pie".
|
| And in this case, I would still hope they did do market
| validation internally, even if it is just a copy of
| wetransfer. Just because some other company is doing
| something doesn't automatically mean it's a good fit for
| _your_ company to do it, too.
|
| As for the design, it's pretty hard to say that "a generic
| file selection panel in the middle of a page and a Sign Up
| button at the top right of the page where it typically is on
| most sites" is a "straight up clone". The color schemes are
| similar, but that's just because the main color branding of
| both companies happens to be light blue.
| DandyDev wrote:
| The author is happy to answer questions unless they're
| somewhat embarrassing
| yjftsjthsd-h wrote:
| The author answered the question, and did so before you
| remarked on it.
| nicksundin wrote:
| Right, this is a really good question. Dropbox Transfer
| achieves the same purpose as WeTransfer with a few
| differentiators. Unpacking this a bit, there are two parts
| here:
|
| 1) For a company with an established sharing model (shared
| folder, live-updating links), a departure or addition to this
| needs to be thoughtfully orchestrated. Moving to snapshotting
| content, gathering content inside and outside your Dropbox,
| and enforcing expiration in this tool is a very different
| paradigm.
|
| 2) However, specifically because we are also a storage
| provider, we can provide a much faster sending experience for
| them as we can remove the upload step, which is especially
| important for folks sending really large files. We have a
| large user base storing their content in Dropbox, this is a
| net new added value/functionality for them.
| [deleted]
| errantmind wrote:
| I uninstalled Dropbox and moved my files elsewhere because there
| was NO WAY to disable the annoying notifications asking me to
| 'upgrade' my account (when my account was almost full). My whole
| family used Dropbox with paid accounts and I'm actively moving
| them elsewhere because of this.
| inopinatus wrote:
| The client is now full of all kinds of irritating nagware to
| enable this and that unnecessary feature, even though I'm a
| paying customer already. Doing so whilst ignoring the design
| language of the OS is a hallmark of applications arrogantly
| deluded of their own importance. The Dropbox client UX now puts
| me in mind of 90s-era RealPlayer, which surely ranks as one of
| the worst software products of all time.
|
| I do not appreciate intrusive micromanagement and passive-
| aggressive interdepartmental memos from humans, let alone from
| my file sync utility, and when this occurs separately on every
| god-damn device and even after a time machine recovery, I find
| myself verbally abusing it, and the product managers that
| spawned it, aloud in the most vulgar language.
|
| Not good, Dropbox.
| new2628 wrote:
| The web interface also went downhill from its initial
| simplicity. Hitting the download button for a pdf file in my
| own dropbox folder (99.99% of my use case) has become a
| whack-a-mole, with a constant danger of accidentally
| connecting some app with the document, entering some kind of
| crazy collaborative editing/commenting mode, updating my
| "profile", installing a dropbox app, getting notified of
| something, or activating some kind of sharing feature, none
| of which I ever used or wanted.
| ykevinator wrote:
| Firefox send (send.firefox.com) is an excellent unknown product
| and I highly recommend it
| leetrout wrote:
| Related I've been wearing out webwormhole.io to send large files
| to coworkers.
|
| Not a 1:1 here but very useful tool in the same space.
___________________________________________________________________
(page generated 2020-06-26 23:01 UTC)