[HN Gopher] Show HN: A tool for motion-capturing 3D characters u...
___________________________________________________________________
Show HN: A tool for motion-capturing 3D characters using a VR
headset
Hi everyone! I'm one of the authors of this project. The demo you
see here is powered by a tool that I recently helped develop and
open-source at Shopify called handy. You can find the repo here:
https://github.com/Shopify/handy Most people don't realize that VR
headsets have become really capable motion capture platforms, so we
decided to release this tool to bring motion capture into the hands
of everyone who owns a headset. With a cheap Quest 2 you can
capture your hands using the headset's hand-tracking feature and
your head. With an expensive Quest Pro you could capture your
facial expressions using the headset's eye and face-tracking
features. Thanks for checking this project out! I'm here to answer
questions if you have any.
Author : diegomacario
Score : 46 points
Date : 2023-01-17 14:03 UTC (8 hours ago)
(HTM) web link (diegomacario.github.io)
(TXT) w3m dump (diegomacario.github.io)
| ngokevin wrote:
| Plugging in this one we did for reference a long time ago!
| Capture + share dances https://blog.mozvr.com/a-saturday-night/
| diegomacario wrote:
| I love this so much, thanks for sharing it! We have been
| thinking about similar concepts at Shopify. Mocapped
| performances convey so much more emotion than images or videos.
| daniel_iversen wrote:
| That's cool! What would the typical use case for this be (and why
| did Shopify build it)?
| diegomacario wrote:
| We actually built this to easily add hands to this concept
| video:
| https://twitter.com/StrangeNative/status/1613218237969494017...
|
| Animating hands by hand (no pun intended) is horrendously
| difficult, so our 3D artist asked if we could help.
|
| We built this within Shopify's Spatial Commerce team, which is
| a team that explores the intersection between AR/VR and
| commerce. Here are some of our other projects:
|
| https://twitter.com/StrangeNative/status/1544666221181734912...
| https://twitter.com/StrangeNative/status/1562450155080597505...
| madlag wrote:
| Thanks for this! I have been working on something similar for an
| upcoming education app, record a course and play in the same app
| with a compact file format (1MB per minute, could be much less
| with some tricks). You can see a demo here :
| https://youtu.be/zcHAzQXm3Hg and more at https://explayn.me I
| will definitely check your lib, and will be happy to switch if
| it's better !
| diegomacario wrote:
| Wow that's super cool! That's definitely the biggest challenge
| - how to keep the size of the mocap data small.
|
| Our library is not the best at that, since it simply records
| everything to Alembic files, which are quite heavy.
|
| Best of luck with Explayn. I really like the idea of learning
| from mocapped lessons.
| madlag wrote:
| Thanks for the kind words! Yes, file size for live action or
| shipping directly in an app is always an issue. I may
| actually contribute some code, my file format is no rocket
| science, just a bare set of floats with some metas, not even
| some diff encoding between frames, so quite easy to interpret
| between languages and platforms.
| endergen wrote:
| This is great, well done, and thanks for open sourcing it.
|
| I started to build something like this early last year, but got
| too busy with another pet project and having a baby. Here's some
| demos and notes if you're interested.
| https://twitter.com/convey_it/status/1433163282597171200
| diegomacario wrote:
| Holy smoke that's so cool, thanks for sharing it! I love that
| your work helped increase the number of morphTargets in
| Three.js. That's real impact!
| endergen wrote:
| Thanks! Yeah, that felt good.
| Keyframe wrote:
| _With an expensive Quest Pro you could capture your facial
| expressions using the headset 's eye and face-tracking features._
|
| Any more on this?
|
| As for hands - is it able to track hands and fingers only or also
| an object within a hand? I can see this being useful to
| 'puppeteer' a 3D object by hand used as a proxy of it... Or a
| simple object within a hand.
|
| Outside of maybe a cliff-climbing game I struggled with finding a
| reason for buying a headset at all. This seems like a plausible
| use for animation.
| klaussilveira wrote:
| How well does it do with occlusion? Holding, or manipulating an
| object, and capturing those interactions.
| karim79 wrote:
| Was wondering the same thing - if it does well there, it could
| be a really cheap way to, for example, motion-capture realistic
| weapon holding in an FPS, provided it can be extended to
| capture the arms as well using CV.
| klaussilveira wrote:
| It would need some cleanup, but definitely usable.
| diegomacario wrote:
| It does amazingly well. Last May Meta released their Hand
| Tracking 2.0 update, and the quality of the hand tracking
| improved enormously. You can see a comparison of 1.0 and 2.0
| here: https://www.youtube.com/watch?v=K-l-
| RXxYnzs&t=4s&ab_channel=...
| klaussilveira wrote:
| What about arms? Or would it have to rely on some realtime IK
| solution to sort them out?
___________________________________________________________________
(page generated 2023-01-17 23:00 UTC)