[HN Gopher] Show HN: I made a Computer Vision addon for Blender
___________________________________________________________________
Show HN: I made a Computer Vision addon for Blender
Author : morroida
Score : 198 points
Date : 2020-10-03 11:24 UTC (11 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| dr_zoidberg wrote:
| Ok, this turned out to be far more interesting than the title
| here reads like. The little abstract at the top is far more
| informative:
|
| > A Blender user-interface to generate synthetic ground truth
| data (benchmarks) for Computer Vision applications.
|
| And it lets you make stereo images, depth maps, segmentation
| masks, surface normals and optical flow data from the rendered
| animation, and export it all in .npz numpy format. Quite
| interesting project.
| santaclaus wrote:
| Very cool! Just the other day I was trying to set Blender's
| camera based on a standard 3x4 computer vision KRT matrix, and it
| is surprisingly a pain in the ass --- I wish more of these
| graphics CAD packages (Blender, Houdini, Maya) made it easier to
| deal with vision data.
| morroida wrote:
| I agree, these tools should have an official computer vision
| module since so many people are using synthetic data these
| days.
| doctoboggan wrote:
| This is awesome! I've been working with blender scripts a lot
| lately for my side project where I generate jewelry for 3D
| printing (https://lulimjewelry.com)
|
| It's an incredibly powerful tool, IMO one of the best large open
| source applications. I've learned some good ideas by reading the
| plug-in here, thank you!
| bluejellybean wrote:
| Really cool! I just sent you an email (I love your contact info
| on HN btw!)
| tim44 wrote:
| Nerb here. I can't say what this does but build a frame for ai
| comparison? Blender doesn't need so much eyes anyways? Buuuuut it
| doesnt have them either in this way?
| morroida wrote:
| So many likes, was not expecting that! I will be presenting this
| work tomorrow at MICCAI and then I will post my presentation link
| in the README of the repository!
| punnerud wrote:
| Thank you for your work! Looking forward to the presentation.
|
| I had to look up MICCAI. To others: 23rd INTERNATIONAL
| CONFERENCE ON MEDICAL IMAGE COMPUTING & COMPUTER ASSISTED
| INTERVENTION (4-8 OCTOBER 2020) https://www.miccai2020.org/en
| blensor wrote:
| Wouldn't a Show HN post make more sense here? I noticed that they
| tend to fare better (keep the interest active for longer)
| btparker wrote:
| Excellent! For a while my job entailed this very thing: creating
| synthetic data for computer vision, and I used Blender as well!
| You've done a great job.
| daenz wrote:
| Very cool! I've done something similar for improving an OCR
| system on crinkled paper[0]. Blender is a powerful and totally
| underutilized tool for this kind of work
|
| 0. https://www.arwmoffat.com/work/synthetic-training-data
| technicolorwhat wrote:
| What the heck. This is beyond awesome I totally want to try it
| out
| modeless wrote:
| I've thought about doing this myself! Did it end up improving
| the OCR system for real world images?
| daenz wrote:
| The startup ran out of money before we could find out :) It
| was sort of a skunkworks project.
| morroida wrote:
| Uou this is awesome! And it's very nicely presented in the
| website. I'm wondering how you mapped from the UV to the 3D
| model. I would like to add that feature to the addon.
| daenz wrote:
| It's been awhile since I've looked at the code, but take a
| look at the code around this
| https://github.com/amoffat/metabrite-receipt-
| tests/blob/mast... for mapping from UV space to image space
|
| TLDR: using a KD-tree, I find the face containing the UV
| coordinate. Then I transform the UV coordinate to barycentric
| coordinates within that containing face, then put that
| barycentric coordinate through the local -> world -> view ->
| perspective transform matrices
| emcq wrote:
| A common approach in rendering engines to convert screen
| space coordinates to objects is to render a second image
| with light and shadow disabled where the color uniquely
| maps to an id. You then can uniquely identify 24 bits worth
| of objects without needing to maintain a KD tree.
| bluejellybean wrote:
| Very, very cool work, I fully expect to use this exact project in
| the future!
|
| To those of you who always thought this stuff looked neat but
| never tried it out, and to those who may have used blender in the
| past and gave up, I would HIGHLY encourage you to try again with
| the latest version of the software. Although there is still a bit
| of a learning curve, there have been massive improvements have to
| the software suite.
|
| The ability to code audio/visual in blender is just incredible. I
| describe it like this: Imagine yourself as someone trying to code
| an image that looks like a tree in machine code. Then imagine
| your partner comes over, sees what you are working on, and hands
| you Python and a fully setup IDE, it's like being given literal
| magic.
|
| I downloaded the latest version earlier this year to write some
| basic AI simulations (cube wars!) and to create models for my
| 3D-printer. Like when I first learned to code, the fun of the
| machine totally sucked me and I got completely off-task from my
| original goal. Lately I've been working on two things with the
| same lines of code, music videos and simulated walks through
| forests(Think the movie Avatar). With only a few hundred lines of
| code I am able to generate infinite forest trails in which you
| can walk (or fly a drone-style camera) through, synced to music
| that is generated by the AI-mushrooms WITHIN in the scene itself!
| Literally was able to go from 0 to highly visually engaging
| trippy music videos in the last year with minimial musical
| production experience and with no music-video production
| background. The ease in which you are able to generate things via
| code is stunning and the limits feel completely boundless.
___________________________________________________________________
(page generated 2020-10-03 23:00 UTC)