[HN Gopher] Show HN: Unscreen - Automatically remove video backg... ___________________________________________________________________ Show HN: Unscreen - Automatically remove video backgrounds with ML Author : groe Score : 212 points Date : 2020-08-05 08:04 UTC (14 hours ago) (HTM) web link (www.unscreen.com) (TXT) w3m dump (www.unscreen.com) | victornomad wrote: | Looks very impressive, but please do not autoplay all the videos | at once on the website. My computer fans started to spin like | crazy! | mmwelt wrote: | It killed my Android tablet running Firefox! First everything | slowed down, then the status bar disappeared, then... black | screen. Had to restart. | cjmcqueen wrote: | I'm genuinely surprised my Pixel 3 phone had no problems with | it in Chrome. That was a lot of video. | [deleted] | Asuchug4 wrote: | Somehow I parsed the title as next-gen adblock. Well, it is great | to see the required technology is already here. | CyberDildonics wrote: | This is an area of research called natural image matting that has | been going on for over 20 years. | | Some influential papers include: | | https://www.microsoft.com/en-us/research/wp-content/uploads/... | | https://webee.technion.ac.il/people/anat.levin/papers/Mattin... | mobilio wrote: | From same authors that make tool for removing backgrounds from | images - remove.bg | | It was discussed here: | https://news.ycombinator.com/item?id=18697601 | anchpop wrote: | I'm curious if this is just repackaging existing OSS algorithms | for isolating foregrounds, or if they have an actually new | implementation that outperforms them | Jaruzel wrote: | BEWARE. remove.bg seems to have a malware/fake Chrome update | popup. | groe wrote: | It's a hint for outdated browsers, it's not an ad and there's | nothing malicious behind it (unless there's an issue in your | local network or machine perhaps?). | | http://browser-update.org/ | https://www.npmjs.com/package/browser-update | | It's a fine line though to not get into the way with things | like this, and we could certainly improve it's appearance a | bit - sorry for that. | [deleted] | TonyTrapp wrote: | Fresh Chromium installation, no adblocking or other content | blocking plugins. Zero advertisements and popups on that | site. I keep reading these kind of comments on HN and | everytime it seems to be only that one person reporting the | problem that sees that behavior. Are you sure your network is | not compromised? | [deleted] | Jaruzel wrote: | > Are you sure your network is not compromised? | | It's definitely not. Also used 3 different browsers - each | one showed the same dialog but with the name and version of | that browser. It doesn't happen on my mobile over WiFi | (rules out network injection). | | This is what I'm seeing: https://imgur.com/a/o9PNmaU | | It's being inserted via a .js file on static.remove.bg: | | https://static.remove.bg/remove-bg- | web/38c6be57b031c26a2186b... | | (search for 'out of date' ) | | So either the site owner is complicit, or they've been | hacked. | | Edit: Fired up a VM of Windows 7 - Same message - so unless | my routers been hacked to inject that script somehow, i'm | 99% certain it's not me. | ninjaranter wrote: | Are you sure it's malware? Looking at the code it seems | to redirect to "browser-update.org" and is genuinely | redirecting only older browsers based on unsupported | features. | aembleton wrote: | Looking at the code, it is using a script from Browser | Update[1] to determine whether to show that message. | | Here's the line from the js you linked to that checks the | browser version: | window.checkBrowserVersion = function() { i()({ required: | { i: 12, e: -4, f: -3, o: -3, s: -1, c: -3 }, insecure: | !0, unsupported: !0, reminder: 0, reminderClosed: 168, | style: "corner", api: 2019.06, test: !1, onshow: | function() { window.track && | window.track("BrowserUpdate", "outdated_version_dialog", | "Browser version outdated dialog") } }) }, $((function() | { window.checkBrowserVersion() })); | | The c: -3 is the crucial bit that should cause it to | trigger if your version is at least 3 versions out of | date. | | Yours isn't though. Can you access http://browser- | update.org ? If not, then there might be something with | your DNS settings. Have you tried tethering through your | phone? Have you changed your user agent? | | 1. http://browser-update.org/#install | aembleton wrote: | Just checked, and my Chrome is on version 84. Version 80 | is out of date. | | You need to update Chrome. | Jaruzel wrote: | To be frank, I don't _need_ to do anything. I 'll update | Chrome when I'm ready, thanks. | | What I object to, is a dodgy popup appearing on an | unrelated website. | sreekotay wrote: | That's interesting. Number of versions is probably a weak | signal for this --- but how many browser versions should | the dev a small site be looking for? Things DO break... | noja wrote: | You don't _need_ to call an ambulance when you break your | leg, but your browser has at least five high | vulnerabilities (and one critical vulnerability). | noja wrote: | Perhaps the GP is receiving malware through a targeted or | non-targeted advertising campaign. | sytelus wrote: | If you want to get traction, you need to be far more creative in | your business model. The main application of background removal | is online meetings and its already well implemented but it leaves | a lot to be desired. One thing you can do is to make service | completely free. Offer a desktop app that creates a virtual | camera that you can use in Zoom, Teams etc. Differentiate | yourself by adding features such as much more cool and fun | background wall papers or even live videos. Focus on adoption, | not revenue. Once you get traction, offer premium background | images or videos. Create a marketplace where people can buy/sell. | Focus on buyout by big players because ultimately they have the | platforms with millions of users where you can provide far more | value. | | Removal of background in images is also big (your site only seem | to offer video/gifs). I've personally tried half dozen solutions | and all fell short in low lighting scenario. If you are actually | doing good here, you can beat competition. There are probably | hundred websites out there for background removal for images, | none works well. | isochronous wrote: | The very first thing I thought when I saw this wasn't "ooh, | cool tech for voice chat", it was "man, the special effects | industry will KILL for this - no more manually going through | every frame of an effects shot and painstakingly removing | background elements!" | | So basically, I think the main assumption on which you built | the rest of your advice is flawed. | bergstromm466 wrote: | > Once you get traction, offer premium background images or | videos. Create a marketplace where people can buy/sell. | | A lot of this seems like guesswork, and honestly also like | terrible advice. Maybe if the person had added their experience | or a story about their own experience. Like the other person | wrote, some advice might be irrelevant because this person | hasn't understood the product to know it's it's for existing | footage only, not live footage. | | Sometimes advice can be useful, but be wary when it sounds | useful at first glance but includes no examples, and no | credentials. | | Then often it's someone who considers themselves a success | guru, without much sincerity. They just think that their advice | is obvious, when its probably outside their domain. It's not | bad faith, it seems to me they're just a wannabe startup | guru/investor/founder/whisperer (what I write here might also | be crap). | | The best advice I've found comes with intimate stories of | failure. Stories that help you understand the underlying | friction, yet aren't specific to the exact challenges you're | facing, but leave you with solid hypotheses to explore. | sytelus wrote: | There is a lot of negativity in your response without even | deluding to what's really wrong with the advice. There are | tons of very successful founders recommending focusing on | adoption instead of revenue in initial period. Freemium is | very well adopted all over. Literally entire app industry | worth billions of dollars use this model. Again, I'm not | claiming to be expert in this domain. You are entitled to | your opinion and frankly we are all armchair pundits here, | however, it's bad form to spew out so much negativity without | any reasoning or even alternative. | Pyramus wrote: | >There are tons of very successful founders recommending | focusing on adoption instead of revenue in initial period. | | As of 2020, this is the exception rather than the rule. The | problem is that focussing on adoption before focussing on | value is very risky. You are constantly flirting with | premature scaling before having found product/market fit | (see e.g. Startup Genome project). | | However, successful businesses that have been built on | adoption are _by-design_ much more likely to be well known. | Again, it 's entirely possible to build a business that | relies on network effects or economies of scale, but it's | not the default advice I would give to founder. | | >Freemium is very well adopted all over. Literally entire | app industry worth billions of dollars use this model. | | No, you are confusing freemium with advertising here, which | is the predominant business model. The challenge with | freemium is the conversion into paid, which you cannot | predict unless you have proven the value hypothesis | already. | bergstromm466 wrote: | Yeah I understand where you're coming from, and maybe my | reply broke my own rule of giving advice without a personal | story. | | Where i'm coming from is having listened to advice from | people who I looked up to, but who did it to get rid of me | (because I was ignorant of protocols/etiquette), rather | than being genuinely interested in my project. I've learned | that the SV world has a high rate of failure and many are | hungry, and they try to also carve out a role for | themselves, or try to get involved somehow, even if that | means they'll claim competence in an area they're not | competent in. I was more naive then, compared to today, but | believing these 'gurus' was a painful learning experience | for me, because their advice often left me stranded in the | end. | | I'm not negative, it's more just sharing my disappointments | and struggles. Thanks again for your comment, which made me | see I kinda broke my own rule | abraae wrote: | > Again, I'm not claiming to be expert in this domain. | | Actually nowhere in your original post do you include this | "not claiming to be an expert" disclaimer. | | As a result, when I read your original post, I assumed that | perhaps you were an expert as there was no self-doubt at | all evident. | | I think perhaps that's what the allegedly negative post | that you are responding to is getting at. | bitL wrote: | This looks like it's targeting pros like me who need to remove | background from high-quality videos. The price is completely | OK. | Hydraulix989 wrote: | Right, this seems like a feature, not a product -- and it's not | difficult to replicate. ML is becoming more and more of a | commodity these days. | onion2k wrote: | _The main application of background removal is online meetings | and its already well implemented but it leaves a lot to be | desired._ | | That's the most obvious case people see right now because we're | doing more video calls in lockdown and people see it as a funny | gimmick to play with on Google Meet, but background removal has | been around for years in the professional video and film | industry. Video chat is absolutely not the main application of | the technology. | abraae wrote: | The video chat market though would be approximately a | gazillion times larger than the professional video and film | industry wouldn't it? | cvhashim wrote: | What about marketing to live streamers and youtubers | nl wrote: | Pretty hard to compete with the free implementations in | most video chat tools... | onion2k wrote: | I have no idea. There's a gazillion times more people doing | video calls than make professional videos, but not all of | them would be willing to pay for the software and those who | are probably wouldn't pay very much. It's the age old | question of whether or not it's better to sell something | cheaply at high volumes or more expensively at lower | volumes. | addandsubtract wrote: | >the age old question of whether or not it's better to | sell something cheaply at high volumes or more | expensively at lower volumes. | | Factor in the support you need to accommodate your | customers and you have your answer :P | onion2k wrote: | You can just ignore support entirely and then it costs | nothing. It's what Google does. | epanchin wrote: | Good webcams are hard to find at the moment because | people are very willing to spend money on their video | calls. | buzzerbetrayed wrote: | > people are very willing to spend money on their video | calls. | | MORE people are willing to spend money on webcams than in | the recent past. That is why there is a supply issue. | This doesn't necessarily mean a huge number of people are | willing to spend money in it in general. Doubling a | relatively small number doesn't automatically make it a | big number. | mlthoughts2018 wrote: | Surveillance video processing and intelligence services are | probably by far the largest use cases for video background | removal and similar techniques. Far smaller in terms of | number of customers, but far far larger in terms of revenue | specifically for this capability. | | Millions of people paying peanuts (attributable to this | feature) is way less than a few hundred or thousand research | / defense labs, contractors, etc., paying tens of thousands | up to even millions on licensed software for this type of | thing. | rasz wrote: | In both teleconferencing and surveillance you are dealing | with static background, no need for fancy ML algorithms. | mlthoughts2018 wrote: | No, not true. Surveillance cameras mounted on police | cars, helicopters, etc. Even in stationary cameras, scene | understanding is a hard problem. Suppose someone wheels a | dolly of boxes into a warehouse and leaves them. Are they | now part of the background? Are they suspicious boxes? | The coarse level state transition of objects in the scene | is really hard to solve even with heavy ML. | CathedralBorrow wrote: | My rule of thumb for advice on the Internet is that if it is | mostly in the form of statements like "You need to do X, Y and | Z" and has very few questions intended to dig deeper into the | nuances of the product and business model... then it's not | generally worth much at face value. | | Essentially: The less information a person needs to dish out | advice, the less value I am going to place on their advice. | bergstromm466 wrote: | haha you said what i wanted to say in a way more concise way. | nice one | rbinv wrote: | Sounds like that would evolve to become Google AdSense for your | home office. | lawlorino wrote: | According to the FAQ this only works with prerecorded video, | not live video. | codethief wrote: | > I've personally tried half dozen solutions and all fell short | in low lighting scenario. | | Have you tried https://www.remove.bg ? I came across that site | a while ago and was surprised how good the results were that I | got. YMMV, though. | | [EDIT]: Looks like Unscreen is actually from the same people as | remove.bg. [0] | | [0] https://news.ycombinator.com/item?id=24058701 | groe wrote: | We've built Unscreen on top of the tech of remove.bg, at | least partly, and the #1 design goal for both products is a | high result quality. We've summarized a few of the | improvements recently here, if you're curious: | https://www.remove.bg/b/introducing-remove-bg-x2 | htrp wrote: | remove bg/unscreen are pretty much cool ML applications | looking for a market | bioipbiop wrote: | There is a reason background removal sucks on chat apps. I | fiddle around with the state of the art approaches, it takes a | very expensive machine to get to an acceptable frame rate. It | wouldn't scale. | mattigames wrote: | Would this be a good candidate for P2P? As what if your | computer GPU could be used by other app's users to do a small | part of such heavy operation? (when your GPU is not doing | anything else) | theon144 wrote: | Latency. | Roritharr wrote: | Can you elaborate more? What are the approaches? | aspenmayer wrote: | Not OP, and not in this field, so do some more research, I | am just sharing what I found: | | https://paperswithcode.com/task/video-background- | subtraction | maktouch wrote: | I'm pretty happy with XSplit VCam | (https://www.xsplit.com/vcam) | atombender wrote: | Anything like this for macOS? | dividuum wrote: | > Offer a desktop app that creates a virtual camera that you | can use in Zoom, Teams etc. Differentiate yourself by adding | features such as much more cool and fun background wall papers | or even live videos. | | https://www.xsplit.com/vcam does exactly that already and is | available on demand or for a lifetime license of $40. It works | mostly ok. Unscreen's quality seems a lot better though. | NicoJuicy wrote: | Pre-existing example: Snapchat has an app to apply filters in | webcams | ponker wrote: | The quality level here is way beyond what Zoom has or needs. | This is a professional tool for creative pros -- knocking out | an image background is too easy but video is a real challenge. | And this kind of work at this resolution would be prohibitively | expensive to do for live video ... you'd need a Titan RTX or | similar. | manmal wrote: | I don't think it's feasible to do this level of processing in | real-time _for free_. | jchook wrote: | Reading "The Lean Startup" by Eric Ries. | | He argues that this kind of "build it; they will come" aka | "Just Do It" business process is a recipe for failure because | you make a lot of untested assumptions about your market, and | front-load a ton of (often wasteful) work. | | He suggests the "Build, measure, learn" feedback loop, which | you actually plan backwards: | | 1. Figure out what you need to learn by outlining the key | assumptions, including your value hypothesis and growth | hypothesis. | | 2. Decide how you will measure your success in validating (or | invalidating!) those assumptions with science (eg he calls his | method "innovation accounting"). | | 3. Build the smallest experiments (or MVPs) that will allow you | to learn the things you need to learn. | | 4. Make adjustments to your hypothesis, rinse, and repeat. | | Another common theme, "get out of the building" and actually | speak with all kinds of potential customers. Since a start-up | operates in an environment of high uncertainty they cannot rely | on traditional planning. | | Anyway I love reading the book and learning about the early | days of Intuit, Dropbox, etc and how sometimes the way to make | the most progress feels very counter-intuitive. | | It almost feels like a real world version of Stochastic | Gradient Descent for businesses to mechanically gravitate to | various local minima on an invisible failure landscape. | cvhashim wrote: | Also recommend "The Mom Test". | bitL wrote: | The price and business model is perfectly fine: | | 1) most pros use green screens for background removal | | 2) for (rare) shots where background needs to be removed and | green screen cannot be applied, a high quality removal like | the one here is needed | | 3) for such rare occurrences, charging higher amount of money | is completely appropriate and no reasonable pro would | complain as it would save them additional costs of re- | shooting the scene completely or doing the removal manually | | 4) nobody has anything similar, so they can charge premium | while it's possible | | Given what I mentioned I'd say their price is too low anyway. | pmarreck wrote: | the autoplaying movies are massacring my crappy cell internet | connection | | (my home is currently one of the hundreds of thousands without | power on the US east coast) | aembleton wrote: | Set uBlock Origin to block media elements larger than 50KB or | whatever threshold you want. More info: | https://github.com/gorhill/uBlock/wiki/Per-site-switches#no-... | 23d wrote: | Video background removal, when hair is involved, is painful. | Clearly this is not targeted at webcam footage. Anyone who has | had to manually remove a background using a tool like Premiere or | After Effects from pre-recorded video would see the benefit of | this tool/api. It is pricey but i can see motion graphics studios | using this regularly. | cateye wrote: | It's actually quite funny that the Zoom auto remove background | does a pretty good job. I can't get the same results easily in | Adobe Premiere actually. | | Snapchat filters are even more impressive and it is magnitudes | more difficult and time consuming to get similar results in After | Effects. | | I really wondered if I was missing something so watched a lot of | YouTube tutorials on background removal and object tracking and | all I see is that it is very laborious and even than not always a | good enough results with edge/transparency artifacts etc. | | Adobe really needs to implement new features for this because the | expectations are changed by these apps and the bar is much | higher. | manmal wrote: | IMO Zoom's background removal is not good at all. It's surely | way lighter CPU-wise. | bonoboTP wrote: | In my experience zoom does a horrible job. It's a flickering | mess, half the head is missing, etc. Perhaps in great lighting | conditions it's better. | [deleted] | KingOfCoders wrote: | EUR 1.98 / minute - wow. | eps wrote: | I'm guessing the OP aims at professional, high-quality video | editing rather than at a hobby use. Which is probably the right | thing to do as there are low-quality free options already. Hard | to compete with that. | crooked-v wrote: | Wouldn't somebody doing professional video rather use a green | screen instead, though? | slezyr wrote: | It's for 30 FPS. And only up to 1080p. | | > Up to 30 fps: Base price | | > 30-60 fps: Base price x 2 | | > 60-90 fps: Base price x 3 | londons_explore wrote: | That pricing looks very much "what it costs to run the | service, plus some profit margin". | | A service run by anyone with any business acumen will charge | what customers will pay, modified for benefits that customer | brings the company (like referrals, bigger revenue leading to | a higher company valuation, etc.). | | None of that depends on the frame rate or resolution. | wodenokoto wrote: | Very impressive results and great presentation. | | I imagine this can be useful for business that want to record | small 360 of their products, but not invest in a big Physical | setup. | groe wrote: | Thanks! Yes, professional 360deg product shots are way too hard | to produce right now and we're aiming make them much more | accessible, particularly for smaller vendors | bryanrasmussen wrote: | Since it doesn't do live video so no chat background virtual | camera tools - but what if you did a video in the area you wanted | to remove could you then use that for some sort of baseline for | removing the background? So you do a video in the area you want | to use your virtual camera in, and then when you're live it knows | what the actual 'background' looks like so it can be used to | remove live. | manmal wrote: | If you can keep the camera at a fixed position, that's surely a | great option. I think that's not the use case this tool is for. | A green screen would also work obv. | LyalinDotCom wrote: | How long before this is used to help fake political videos from | existing footage. Sigh | pabe wrote: | Looks great and is exactly what one needs for professional | screencasts if she doesn't want to fiddle around with a | greenscreen. Thumbs up! | Roritharr wrote: | Looking really cool, I just hope a Premiere / After Effects | Update won't kill your business. | dirtyid wrote: | Need longer demos to show temporal contiguity. | manmal wrote: | I think processing is done frame per frame. | tsherr wrote: | Looks very impressive, but it seems cripplingly expensive. | | I don't play with this much, but when I use my green screen, I'm | often doing 15 minute videos, so it is well out of my hobby | budget. | rplnt wrote: | The application of this would probably be different from what | your use case is (since you can use green screen). | hadeson wrote: | Most kinds of professional applications I could think of | would better off with a green screen. On the other hand, | using this for editing videos in the wild with messy | background still not good enough. | billconan wrote: | I want to integrate your service to remove background for product | images. However, not everything works well, especially when | background and foreground color are close. | | that's fine. The thing is that on the website, there is a tool to | fix imperfect results, but with the api, there is no such tool. | groe wrote: | Since you mention images I assume you are referring to | remove.bg, not unscreen. | | We're working on an SDK that will allow for editing on external | sites. You can sign up to our mailing list if you want to get | notified when it's ready: https://www.remove.bg/blog#subscribe | md5person wrote: | I think some of the artifacts on the video at 0:12 (handbag | thing) are very noticeable. Same thing at 0:19 (cat)and at 0:28 | (cat again). Otherwise, looks nice :) | groe wrote: | Some scenes, foregrounds, backgrounds, lighting conditions, | movements, etc work better than others - there's a lot of | potential for errors in videos and the videos from the examples | page are straight from Unscreen Pro. We're continuously working | on improving the AI though (NB: We don't use video uploads for | that - see unscreen.com/privacy) | raobit wrote: | Looks Great! How tool is built?using computer vision,image | processing? | groe wrote: | It's a combination of deep learning and traditional computer | vision algorithms. We do a lot of basic research, evaluate many | different approaches, train them in dozens of variants and | ultimately combine what performs best, then try to make it as | easy to use as possible | nixy wrote: | This has been submitted multiple times as a "Show HN" but as it | is a paid service, I guess this is a marketing strategy. | | https://news.ycombinator.com/from?site=unscreen.com | abalaji wrote: | Doesn't this violate the spirit of Show HN? I thought only the | authors could submit them unless they are all authors of the | project. | thekevan wrote: | I feel like this could be presented in a much more engaging | manner. | | There's one quick video on the main page which shows replacement | backgrounds, but the "Examples" page shows about 15 or 16 | examples of the same thing, removing a background. It quickly | gets to the point of "Ok yeah, so what? Show me something cool to | do once the background is gone." | | I get that it may be difficult to remove a background the way | this service does, but the customer is going to be less impressed | with the fact that it works correctly than they are going to be | about it having an actual engaging or useful purpose. | manmal wrote: | You can try it with an uploaded video or GIF here: | https://www.unscreen.com/upload | groe wrote: | That's a great idea, thank you | SEJeff wrote: | This is sort of the inverse of gimp's "foreground select tool". | It was a PHD thesis on the algorithm to select foregrounds that | was contributed to the gimp photo editor as open source. | | https://docs.gimp.org/en/gimp-tool-foreground-select.html | | And the paper with an overview of the algorithm is here: "Image | segementation by uniform color clustering" | | https://www.inf.fu-berlin.de/inst/pubs/tr-b-05-07.pdf | [deleted] | daertommy wrote: | Product hunt is full of this kind of stuff, is there anything | different on this project? | dandigangi wrote: | Whats the tech stack this was built on? | jacknews wrote: | $1.98 per MINUTE? | | I thought I'd woken up in 1980. | | But I guess this is aimed at people making professional video | presentations etc rather than for webcam meetings. | | It seems to work pretty well, but none of the examples have any | sudden movement, which is where there's often tearing etc. | Insanity wrote: | It works on pre-recorded video so webcam meetings are not their | target market. | jacknews wrote: | Exactly as I concluded, and said above. | groe wrote: | Thanks a lot for your comments and feedback - some great great | advice and ideas in here! | | This is the MVP release of Unscreen Pro, so there's many more | things to come in the future, from quality improvements, to an | API, integrations, editing tools and more. Appreciate all the | inputs :) | godmode2019 wrote: | Looks cool, this model is start of the art I believe | https://app.wandb.ai/stacey/greenscreen/reports/Two-Shots-to... | motoboi wrote: | I believe this is homegrow from their photo background-removal | tool remove.bg | | Probably frame by frame, not real-time. But not less | impressive. ___________________________________________________________________ (page generated 2020-08-05 23:01 UTC)