[HN Gopher] Building Firefox's Picture-in-Picture Support
___________________________________________________________________
Building Firefox's Picture-in-Picture Support
Author : primogen
Score : 167 points
Date : 2020-02-09 18:07 UTC (4 hours ago)
(HTM) web link (hacks.mozilla.org)
(TXT) w3m dump (hacks.mozilla.org)
| GeneticGenesis wrote:
| Mozilla's choice to add their own pip button is a massive pain
| for anyone developing video applications - anywhere over the
| player can't be considered a safe space to add our own buttons
| anymore, it really feels like Mozilla didn't talk to any video
| service developers when deciding how to approach this problem.
|
| Pip needs to be common API driven (as Chrome is) to make building
| portable video applications possible.
|
| A terrible misstep from Mozilla. (And yes, this has been
| repeatedly raised on the bug tracker with no answer)
| jancsika wrote:
| I wonder if an attacker could leverage this to play twenty-
| questions with a user.
|
| Video is typically so-and-so dimensions in the bottom right-hand
| corner. Clicking the video (I assume) pauses it. Pausing a video
| (I assume) can change the request pattern for what is being
| streamed. Most sites use a white background (let's say).
|
| So an attacker who controls video content can stream a still
| image of some site or browser security "statement" with a picture
| of a button to click if the statement is untrue. If the user
| clicks the fake button they stop the video and signal to the
| attacker their response to the statement.
|
| Not super useful given an arbitrary page behind the video. But
| what if the video's content is about, say, how to change settings
| on your bank site to get a better APR? I can imagine someone
| assuming that the video has turned into a small "quiz" from the
| bank. Maybe even a "click here to proceed" button to get them to
| unpause the video.
| chrismarlow9 wrote:
| the entire concept reminds me too much of iframes, embedded
| flash and Java applets. i can see many security problems coming
| with this in the future. especially if they try to make it a
| real window that can jump tabs and stay pinned, which seems to
| be what most people here would really want and an obvious next
| feature.
|
| think about click jacking, address bar spoofing, and similar
| mosaic style hijacking. that's not even getting into potential
| CORS and origin spoofing potential.
| dingaling wrote:
| Well it'd be handy if I could PiP another non-video tab.
| Jach wrote:
| I thought this might have been some Steam tie-in or otherwise not
| a Firefox built-in feature, never clicked it.. and now I can't
| even try it as the button's not appearing, it seems pretty
| unstable. Not that I really want to... This solves a problem I've
| never had. What's so hard about moving the video to a new window,
| or if your screen is too small moving the window to a new
| monitor? In Linux you can also make the window smaller to just
| have the video visible and then make it "always on top" of
| whatever else you're doing.
| LEARAX wrote:
| To be perfectly honest, I didn't realize this was a Firefox
| feature. I thought it was just a new part of YouTube.
| [deleted]
| drannex wrote:
| This is honestly one of my favorite features in firefox.
| rasz wrote:
| I much rather they would provide video stream hook so I could
| execute my own player as the external one (VLC, Mplayer, etc). As
| it is I have a bunch of custom userscripts injecting my own hook
| into selected websites.
| frankharv wrote:
| For the life of me I can't see why anyone would need picture in
| picture for browsers. More features most people don't need.
| Please come down to earth Mozilla. You have lost your way. Please
| revisit NCSA Mosaic and witness your roots.
| hhhhhhh4hh wrote:
| I do love that feature.
|
| I wish it was a little bit easier to maximize on other monitors
|
| instead of having to try to resize it perfectly
| usrusr wrote:
| PiP has greatly improved my Zwift+Netflix experience. Zwift is
| a "fitness game" that is best enjoyed with a TV stream
| alongside (sorry Zwift, you're not _that_ engaging), that like
| most games doesn 't really support arbitrary aspect ratio and
| that is best enjoyed on the big TV screen. It can easily
| tolerated _some_ of its UI getting obscured but you do want to
| keep it focused for possible key events. Before PiP arranging
| those windows was a permanent struggle with focus and tricks to
| hide as much of the browser UI as possible (not much). It 's
| still not perfect because PiP is reset between episodes, but I
| don't expect this to remain a problem forever.
| jbrooksuk wrote:
| I use PiP for various videos, whilst I'm working and not
| splitting the screen.
| micael_dias wrote:
| I find it quite handy when browsing YouTube. I even thought it
| was a YouTube feature before I realised it didn't work in
| chrome.
| shrewduser wrote:
| You are mad I use this feature all the time.
| Scarbutt wrote:
| Its helpful for increasing your ADHD.
| faitswulff wrote:
| A comment on reddit said they use it all the time when
| following coding tutorials. Seems like a handy use case.
| gruez wrote:
| That brings up a second question: who watches coding
| tutorials? I'd imagine it's probably people with less than a
| few weeks of coding experience, because it's horribly
| inefficient compared to text tutorials.
| luckylion wrote:
| I find them valuable if you don't need to write code to
| experiment with but can just watch the video at the regular
| speed and get the concepts that you can then use in your
| project. Also allows for watching while you do other stuff
| on the side.
|
| SymfonyCasts is an example where I like the videos. They
| aren't the typical YouTube style of "let me talk about
| random things for 5 minutes so I get this video over the 10
| minute mark", they get pretty much straight to the point.
| They have a text version below, which also makes search
| easy, And you can click on any sentence in the text version
| to jump to the right position in the video. I found them
| very helpful when I started with Symfony, but haven't
| really watched any once I had a solid enough understanding
| to get work done and incrementally learn more things as I
| need them.
| kube-system wrote:
| I've occasionally found them helpful for less mainstream
| stuff where there's a lack of quality resources.
|
| The 'failure mode' for a lazy text tutorial is that
| important details are left out. The 'failure mode' for a
| lazy video tutorial is that they didn't edit anything out.
| yoloClin wrote:
| One of my few very good teachers in school highlighted that
| people learn differently - some people by reading, others
| doing and others again watching. I'm most definitely
| somewhere between the last two. I find that for me watching
| high quality tutorials (pluralsight, CBTNuggets, etc) at
| 1.5-2x speed is a very good way to get introduced into a
| language particularly when I seed a solid foundational
| understanding rather than knowing exactly how to add an
| additional command into npm run, etc.
| hhhhhhh4hh wrote:
| Which approach has better efficiency is dependent on
| person.
| verletx64 wrote:
| > I'd imagine it's probably people with less than a few
| weeks of coding experience
|
| In the end everybody eventually needs to learn from written
| content; not only for the viewers efficiency, but also
| because it's more efficient for content creators to produce
| written content over video content.
|
| You can even guess your 'reckon' from looking at the kind
| of video content that is out there - again, this seems
| mostly to be a consequence of creator efficiency.
|
| With that said, who cares?
|
| Hell, I sometimes hit up streams - for instance, Antirez
| has been streaming himself building out parts of Redis 6,
| because it may introduce me to new techniques or minutae I
| didn't know. It's way, way, way more chill than reading
| source code, and it's a lot better than the blog post on it
| he probably won't write.
| faitswulff wrote:
| Anything with an unfamiliar IDE can be very confusing
| without a video tutorial.
| LandR wrote:
| I use picture in picture all the time! I have 3 monitors, but
| still not enough spave to take a full one up with a browser. So
| if I've got a YouTube video playing I'll pop it out and stick
| it in a corner of one.
| mdw wrote:
| Is there a way to disable PIP support? The blue floating icon
| is quite distracting for me, I wouldn't mind PIP but I can't
| stand the floating icon.
| butterthebuddha wrote:
| https://support.mozilla.org/en-US/kb/about-picture-
| picture-f...
| Barrin92 wrote:
| I use it a lot because I have only one monitor.
| Mirioron wrote:
| Couldn't you just open it in a new window and resize/tile
| your existing windows?
| db48x wrote:
| Yea, I don't get it either. On the other hand, I almost never
| use a window that isn't maximized.
| Zarel wrote:
| I use PIP a lot in Chrome while browsing Reddit. If I see a
| video link, I can play the video while continuing to scroll
| through Reddit comments.
|
| (YouTube PIP is kind of difficult to discover; you have to
| right-click, move the mouse off the menu, and right-click again
| to get the native right-click menu, which is where the PIP
| button is.)
| Mirioron wrote:
| Why not just open it in another tab?
| detaro wrote:
| You mean open in another window, arrange and resize that
| window to not get in the way while making the video
| visible, hoping that the website layout doesn't make that
| close to impossible, and still not easily being able to
| have it stay on top of the window you interact with...
| instead of a single click that does the right thing?
| thekelvinliu wrote:
| browsers let users access content from the internet and some of
| that content is video. for the life of me i can't see why it
| took so long ;)
| serf wrote:
| it allows me to size a video frame arbitrarily from YouTube
| across multiple monitors, rather than relying on YouTubes
| stepped resize and wasting screen real estate on YT comments.
|
| it allows me to size video frames arbitrarily across streaming
| sites like CyTube that don't support frame-out natively.
|
| As someone who had to use mosaic and gopher and all that shit,
| i'd really rather not revisit it. It's nice in the way that
| it's sort of quaint to boot up an old workstation that has
| nothing but QBASIC, but it's not actually pleasant or useful.
| alexis_fr wrote:
| > Given that it would be available on any video content, the
| feature needed to be discoverable and straight-forward for as
| many people as possible.
|
| This is the designer's nightmare: Every feature needs to be
| available to everyone, therefore be visible on the screen. Yet
| they only cite rare usecases: << Watching sports event while
| continuing to browse the web >>. In the meantime, the blue tone
| isn't generally consistent with the websites' colors.
|
| I would have supposed moderatelt-used features belongs in the
| right-click menu like the rest, or they need to find a way to
| merge it with the video controls (I guess an Apple designer would
| imagine a new kind of special press to detach the video, which is
| fine too).
| Wowfunhappy wrote:
| I feel like this is the type of feature that should be a
| browser extension. It could be an officially supported one from
| Mozilla.
| AlphaSite wrote:
| Safari implements this natively and it's super handy.
|
| I imagine if you intently watch videos it's not useful, but
| for multitasking it's great. Or note taking etc.
| prook wrote:
| At first I didn't even realize this was FF feature (I suspected
| something like gstreamer, but was lazy to investigate). Now that
| I know it is indeed FF, I just cannot wrap my head around it...
|
| 1) I can pause the video, but cannot navigate it. I guess the
| navigation would be there if it was technically possible, but
| from practical standpoint this makes it useless even if it
| appealed to me in other ways. Popping a video into PiP mode and
| then go diving for the original tab to seek 10 seconds back to
| rewatch an interesting moment seems utterly bizarre.
|
| 2) I've grown to truly hate this kind-of-PiP thing many websites
| (news outlets mostly) do: Autoplay a video and then shove it to
| lower right corner when I try to scroll away. Along with timed
| "subscribe to newsletter" popup, this is an insta-close for me;
| Now why would I want to replicate this behavior elsewhere?
|
| 3) I'm literally unable to read through even the first words of
| that blogpost because of that gif. Now, thanks to this PiP
| feature, I can make any text unreadable by placing a distracting
| video on my screen.
|
| While I'd welcome features that fight distraction (and not
| support attention fragmentation), there obviously is (significant
| part of?) FF userbase that will make use of this feature. Good
| for them.
|
| Luckily for us grumpy farts, that blue PiP button can be turned
| off in preferences, we can mutter something about bloat, but
| generally go on with our lives.
| cpeterso wrote:
| I am told that Firefox Nightly now allows you to use keyboard
| shortcuts to seek the video. Part of the problem here is that
| there are some sites that have special logic around seeking, so
| having playback controls on the PiP popup can cause the
| original page's custom controls to get confused or out of sync.
| Fnoord wrote:
| My main complaint is that subtitles don't work with it (on
| Netflix, at least).
|
| You don't _have_ to use it. For me, it works when I 'm trying
| to dual task on a single monitor. For example, I use it
| regularly on my smartphone (with a native video player).
|
| As for the example you mentioned: the issue is autoplay.
| Netflix just rolled out an update when they allow you to
| disable that non-feature.
| [deleted]
| fiddlerwoaroof wrote:
| I use this all the time to watch Netflix while programming: put
| Netflix in PiP mode, then put the resulting window in the
| corner and switch to other applications to work.
| tetris11 wrote:
| Same, also for YouTube, KimCartoon and many others.
|
| The reason I mention them is because the window ID of the PiP
| is fixed, meaning that I can easily define a window rule for
| it in my desktop environment independent from Firefox
|
| I'm loving this.
| maxfrai wrote:
| On mac os its mostly unusable, because this PIP window stays on
| same workspace. My browser is always fullscreen and in safari I
| can open small youtube window and move to other workspace (code,
| for example) and window still exists on each of them.
| cpeterso wrote:
| I am told one problem with the native macOS PiP window is that
| it does not allow programs to configure which playback controls
| to show. There is a Firefox bug in file for the workspace
| issue. Chromium works around this and Firefox is likely to use
| the same approach.
| dhritzkiv wrote:
| This would be one advantage of the system's built-in PiP mode
| employed by Safari. I don't find the limitations (maximum size,
| single video) to be hinderances.
| jorvi wrote:
| Can't you trigger that in Firefox also by right clicking on a
| video (or twice on YouTube) and then choosing PiP?
| cheez wrote:
| Main problem I have is captions from Netflix don't work which is
| important for me. Would have been nicer if we could have popped
| out a div instead.
| dthul wrote:
| I have to say I love this feature and have so since the first
| time I encountered it in my Firefox Nightly installation a few
| months ago (on Windows).
|
| Glad to also have it available on Linux and Mac now. It's perfect
| for on-the-side Youtube videos.
| lioeters wrote:
| Yeah, I've been using the picture-in-picture feature regularly
| since I learned about it recently. Love it for running lectures
| and presentations on the side.
| Merg wrote:
| Everything old is new again it seems. Anyone used Maxthon Browser
| before? It was multiengine browser, which used both Webkit and
| Trident under the hood. It has that feature for ages, just for
| flash content back than.
|
| If only someone would built good email client into Browser like
| old Opera, that would be godsend. Opera and Maxthon were ages
| ahead in terms of certain features.
|
| It is little weird for me, they ended up putting video within
| shadow DOM container.
| sdan wrote:
| Picture in Picture is great for lectures or follow-along
| tutorials and I'm glad people are taking it more seriously
| (initially was using Youtube on Safari purely because of this
| feature, but later on Chrome had it... albeit a bit wonky, I
| started using it on Chrome normally).
| jpalomaki wrote:
| I would love to have an external controller for start/stop/fast
| forward/rewind when watching tutorials on Youtube. For example
| a controller [1] people are using to edit videos. Quite often I
| would like to pause the video for a while, repeat last 30 sec
| or by-pass some boring segment.
|
| [1] https://www.amazon.com/Contour-Design-Shuttle-Xpress-
| Multime...
| alexis_fr wrote:
| That is an awesome idea. Would be extremely useful for guitar
| and transcription too, while writing the text in Word.
|
| It typically would be a usecase for the Macbook touch bar,
| does anyone has one and can tell us whether it has an
| integration with web videoplayers?
| daxelrod wrote:
| Videos and audio in Safari use the touchbar as a scrubber
| with controls.
| tjoff wrote:
| You might be aware but youtube at least have j,k,l as
| shortcut keys for reversing 10s, paus/play, and forwarding
| 10s.
|
| Doesn't work within the PIP window though.
| Wowfunhappy wrote:
| > To work around these problems, we rely on the fact that the web
| page is being sent events from the browser engine. At Firefox, we
| control the browser engine! Before sending out a mouse event, we
| can check to see what sort of DOM nodes are directly underneath
| the cursor.
|
| This triggered some mental alarm bells when I read it.
|
| I understand what Mozilla is trying to accomplish, but this feels
| like it could become a serious WTF moment for a developer. "There
| is absolutely no way in hell this element should be able to
| receive mouse events right now, what is Firefox doing?"
|
| Is this a problem? Perhaps the web is so far down the
| hacks/exceptions route that adding one more legitimately won't
| make a difference?
| stingraycharles wrote:
| Yeah I must agree with you. Reading that part, I could only
| think: "Someone will figure out how to turn this into a major
| security issue."
|
| It sounds like the kind of stuff that clickjackers use, but
| this sounds like it could be used for even more nefarious
| purposes.
| reubenmorais wrote:
| A Web developer is never going to see this. The events
| mentioned are being sent to the Shadow DOM which implements the
| PiP UI (as well as the default video controls UI), not the
| webpage.
| detaro wrote:
| I don't think they are talking about mouse events the website
| can observe. Firefox needs to figure out where the video is and
| make it's UI on top of that work, there is no observable
| interaction with the site DOM or its event handlers.
| hoten wrote:
| The post says that the video element is told it's being
| hovered, although it is unclear if that means the rendering
| context has that element set to hover (thus CSS changes even
| if pointer events is set to none), or that some internal-to-
| the-browser state is set to hover for the video.
| dndmmdmdmd wrote:
| rnd
| dndmmdmdmd wrote:
| rnen
| dndmmdmdmd wrote:
| dnfnfnn
| [deleted]
| Jestar342 wrote:
| People using tiling window managers like i3 or sway should get
| friendly with this feature. It allows me to "full window" (note:
| not screen) netflix/amazon/YouTube videos in a tile which was
| impossible to do without PIP.
| explodingcamera wrote:
| At least in i3 and bspwm this is possible by fullscreening in
| firefox and then hitting your windowmanagers fullscreen
| shortcut.
| mxmilkb wrote:
| Hmm, that worked for me in i3 before but not in bspwm.
|
| I tried FF PiP recently, but I have "bspwm config
| honor_size_hints true" and that makes the window resize
| itself until it's the size of a postage stamp.
|
| The above is a known bug in FF, though apparently fixed in
| FF73: https://bugzilla.mozilla.org/show_bug.cgi?id=1600414
| idoubtit wrote:
| > impossible to do without PIP.
|
| This is not my experience. I see no added value in this PIP.
|
| I use XMonad, which is a tiling window manager. I've always
| been able to put a video into applicative full screen, then
| switch the layout to whatever I want. E.g. 2 browsers windows
| each playing a borderless video.
|
| Before XMonad, I remember that when I switched to another
| workspace/window and back, my browser window would have stopped
| being fullscreen.
|
| I'd rather have Firefox not adding hacks to work around the
| flaws of mainstream window managers.
| Jestar342 wrote:
| There's a reason I mentioned only i3 and sway, they stick to
| managing _windows_ and thus only inform the application of
| their window size. Screen size is unabated by them, thus when
| applications that have a full screen mode size-up they get
| the real screen size. I3 author(s) specify purity as their
| reasoning for this and I actually agree with them. There are
| some other nuances with full screen that tie into this, too,
| such as signalling to the app to enter full screen instead of
| simply resizing the applications window means in most cases
| the user gets a more intuitive result.
| darau1 wrote:
| Either awesome or i3 used to allow me to fullscreen a video,
| then pop that fullscreen video out into it's own floating
| window, that I could then stick on top of all other windows,
| and make it stay on every workspace. Really cool way to get a
| hacky picture in picture mode way before firefox did this.
| tetris11 wrote:
| Yep, but now you have a video window with a fixed window
| title, regardless of Tab, which makes it infinitely easier to
| programming window rules for.
| Fnoord wrote:
| VLC has a feature called "float on top". Together with
| resizing, it allows basically something akin to picture in
| picture mode. You can use VLC to play YouTube videos by
| downloading them with youtube-dl. That is what I used before
| Firefox PiP mode was available in stable.
| Jestar342 wrote:
| Yep, sticky containers. Allows you to stick a container to an
| output, floating above all workspaces on that output.
| tomxor wrote:
| I3 can already do floating sticky windows which is essentially
| the same thing. It used to be possible to pop Chrome into a
| floating window from full screen but now both browsers fall out
| of full screen when floating - maybe this will bring it back -
| or is that what you meant?
| sverhagen wrote:
| With (some) television sets in mind that popularized (for, um, me
| at least) the picture-in-picture term as one channel of video on
| top of another channel of video, it feels weird that the term is
| here used when only a minimum of 1 videos is involved...
| dylan604 wrote:
| I feel the same way. This is just a new way of windowing the
| content, not displaying 2 different sources in one screen.
|
| I've seen a couple of this breaking the video out of the main
| layout, but not sure if it was this Firefox feature or not. I
| was pretty sure that once the video broke out of the normal
| layout, the original location of the video was replaced with
| ads playing without audio. I was immediately turned of by it.
___________________________________________________________________
(page generated 2020-02-09 23:00 UTC)