[HN Gopher] OpenCV in the browser using WebAssembly and web workers
___________________________________________________________________
OpenCV in the browser using WebAssembly and web workers
Author : aralroca
Score : 228 points
Date : 2020-05-08 12:39 UTC (10 hours ago)
(HTM) web link (aralroca.com)
(TXT) w3m dump (aralroca.com)
| jonnydubowsky wrote:
| Here's a link to a demo you can try in your browser. (not my
| demo). https://huningxin.github.io/opencv.js/samples/video-
| processi...
| sax wrote:
| This app lets you play around with OpenCV, like jsfiddle or
| codepen: https://cloudvision.app/
| ehsankia wrote:
| That's a... very misleading name. It's called "Cloud" vision,
| but then says:
|
| > Cloud Vision does not track, record or send any images,
| videos, or information provided by the user to any server.
| All image processing is performed within the application.
| ehsankia wrote:
| That's a much better demo, but depending on your webcam size,
| it'll overlap on the controls and block you from changing the
| filters.
| Endlessly wrote:
| How would one benchmark or compare running OpenCV from native
| code and non-native embedded code?
|
| Ask since the intent of the code is not about embedding OpenCV in
| a browser, but offloading the computation workload from +1 users
| from a server to the endpoint user's computer.
|
| Might be wrong, but for a single user, this setup would likely
| not be optimal.
| fest wrote:
| This is probably the first time I am excited to hear about "X
| in webassembly".
|
| While it'll likely be a lot slower than native implementation,
| the benefits of an image not leaving your computer could
| unearth some interesting applications (for example, I am very
| hesitant to use online OCR services and use them only for data
| which is public anyway).
| jjice wrote:
| Does anyone have a good recommendation for resources to learn a
| bit about computer vision? I don't want to go too in depth, but
| I'd like to learn the basics.
| canada_dry wrote:
| If you know python, this is an excellent resource:
|
| https://www.pyimagesearch.com/
| dr_zoidberg wrote:
| In general, I like Adrian's site but over the years he has
| been locking more and more content (even blog posts that used
| to be available in the site) behind his courses. Not that
| there's anything wrong with charging for knowledge (you know,
| teaching and getting paid for it), but it's a shame that
| things that used to be available, and of great quality, have
| been "pay-walled" somehow.
|
| Another good resource is https://learnopencv.com/ . Again,
| good quality stuff, if you know your ways it tends to be
| enough[0] but it's also a big funnel to get you to but one of
| their courses.
|
| [0] though Satya Mallick does hide a lot of complexity from
| the readers, and that bites you if you try to implement
| things on your own
| eastendguy wrote:
| OpenCV (demos) are a good way to get started with "old-school"
| computer vision. For machine-learning kind of computer vision
| see for example
| https://experiments.withgoogle.com/collection/ai
| eastendguy wrote:
| Interesting, this is powerful. UIVision is using
| OpenCV/Webassembly for their image recognition features: "...run
| automated visual UI tests inside the web browser and on the
| desktop, powered by WebAssembly."
| (https://ui.vision/rpa/docs/visual-ui-testing)
|
| You can install the free Chrome/Firefox extension to test it. In
| general, I continue to be amazed how powerful the web assembly
| concept is.
| pjmlp wrote:
| Be amazed, https://www.youtube.com/watch?v=UQiUP2Hd60Y
|
| https://opencv.org/opencv-ported-to-google-chrome-nacl-and-p...
| Chris2048 wrote:
| I wonder what the filesize of opencv.js is..
| ThePadawan wrote:
| It is linked at the end of the article:
|
| https://github.com/vinissimus/opencv-js-webworker/blob/maste...
|
| 7.75MB.
| ape4 wrote:
| Actually not bad
| suyash wrote:
| Nice tutorial, can you do one with Vanilla JS as I don't use
| React.
| extesy wrote:
| Does anyone know how to implement a virtual webcam in python on
| MacOS? I want to implement something like zoom's background
| replacement but I can't find a way to represent the output as a
| webcam that can be used as an input by various conferencing apps.
| KMnO4 wrote:
| CamTwist lets you make custom effects and has an SDK[0], you
| may be able to interface it with Python using PyObjC[1].
|
| It also has direct script functionality but I think the SDK is
| more powerful.
|
| [0]: http://camtwiststudio.com/developers-objective-c/ [1]:
| https://pyobjc.readthedocs.io/en/latest/
| dr_zoidberg wrote:
| A while ago I used OBS and Python+OpenCV to do a goofy webcam
| where I'd had my face replaced by David Bowie's -- not using
| deep learning though, just "plain old" face
| detection+landmarking+morphing.
|
| OBS: https://obsproject.com/es
|
| Now I know it can be done, but I'm not sure how things have
| changed (this has been some 3 or 4 years ago), so I can't
| really give you too many details other than it's possible.
| zamfi wrote:
| Don't know about python, but Snap Camera lets you develop your
| own filters, and takes care of all the virtual device stuff for
| you.
| ehsankia wrote:
| This is definitely not optimal and would be a overkill setup,
| but OBS Studio + VirtualCam plugin let's you basically
| screencapture anything and turn it into a webcam device. So if
| your python app can display a video feed, you can capture the
| window and show it as webcam (with OBS as overhead).
| hughes wrote:
| VirtualCam is not available on MacOS.
| zwieback wrote:
| Sounds interesting but it's a little unclear what's running at
| what layer - it sounds like the JS code from OpenCV.js is now
| running in webassembly? And how much of OpenCV is still running
| in native code, e.g. in prebuilt OpenCV libraries?
| icebraining wrote:
| From what I can tell, the OpenCV C code is directly compiled to
| wasm bytecode; the only JS part is some helper code to let you
| easily call wasm functions from your own code.
| lukevp wrote:
| This is really cool! WebAssembly is such a great concept and I
| can't wait for a better way to manage and preload all these wasm
| libs.
|
| On my iPhone 11, it requested access to the camera and showed the
| image, says it's running at 60fps and is using the camera, but it
| only captured a single frame.
| auggierose wrote:
| On an iPhone 11 it is such a waste though :-) Ported today some
| image processing code from Swift to Metal. Speedup factor:
| 1000.
| elcomet wrote:
| Where did you find the demo? I can't see any links in the
| article. Edit: others posted links
| pjmlp wrote:
| Yeah, back in 2014
|
| https://opencv.org/opencv-ported-to-google-chrome-nacl-and-p...
| ehsankia wrote:
| It's not just on iPhone, that demo is not very good, I think it
| just takes a single frame and makes it grayscale. Try these:
| https://huningxin.github.io/opencv.js/samples/
| ArtWomb wrote:
| Good stuff! It's nice to see your port of OpenCV to the browser
| succeed, and a lot of people would be very interested in adopting
| this! But you may not see more than 2x speedup over raw pixel
| manipulation of the 2D canvas.
|
| WebGPU holds a lot of promise for fast image processing on the
| client. 10x boost is not uncommon for RTX 2000 devices ;)
| mauflows wrote:
| Is there a webGpu based alternative that has the functionality
| of opencv?
| sp332 wrote:
| Not sure of the status, but there is a project to get WebGPU
| working as a backend for OpenCV.js https://summerofcode.withg
| oogle.com/projects/#53528827207352...
___________________________________________________________________
(page generated 2020-05-08 23:00 UTC)