[HN Gopher] DAIN: Depth-Aware Video Frame Interpolation
       ___________________________________________________________________
        
       DAIN: Depth-Aware Video Frame Interpolation
        
       Author : ArtWomb
       Score  : 48 points
       Date   : 2020-01-19 13:49 UTC (9 hours ago)
        
 (HTM) web link (grisk.itch.io)
 (TXT) w3m dump (grisk.itch.io)
        
       | Animats wrote:
       | Oh, like Framefree, from 2006.[1][2] Framefree worked on harder
       | cases, too; not just video game characters with nice sharp edges.
       | 
       | That was a really good idea. It came from Kerner Optical, which
       | was spun off from Lucasfilm and went bust. They had their own
       | frameless video compression format, and there was a Firefox plug-
       | in for playing it. Some of that technology went into systems for
       | making 3D movies from 2D movies, and ultra slow motion for
       | sports. I saw it up at Kerner, in Marin north of SF, about 15
       | years ago, and I had the plug-in and some files it could play.
       | 
       | It's basically an automatic delaminator (the hard part) that
       | separates an image into multiple planes. Then each plane from two
       | images in a sequence has morph points found. The interpolation is
       | a standard 2D morph. You can morph as slowly as you want.
       | 
       | The business end of the operation was totally botched - there
       | were holding companies, an offshore shell company in Jersey,
       | something called Monolith, and some company called Quin Land in
       | Japan. All of which seem to have sunk without a trace. Sad.
       | 
       | I wonder if anybody paid the maintenance fees on the patents.
       | 
       | [1]
       | https://web.archive.org/web/20081216024454if_/http://www.fra...
       | [2] https://www.youtube.com/watch?v=VBfss0AaNaU
        
         | jedimastert wrote:
         | > Framefree worked on harder cases, too; not just video game
         | characters with nice sharp edges.
         | 
         | Check out the claymation example:
         | https://www.youtube.com/watch?v=eAUn7Nvx73s
        
           | foota wrote:
           | What's with the artifacts down the middle? Is one side 60 fps
           | and the other not?
        
             | afrodc_ wrote:
             | Yes. It seems like the left is without the effect and the
             | right shows the improvement with the technique.
        
       | sabalaba wrote:
       | Very impressive work. I would love to see what this looks like
       | when you start to drop more and more frames. I'd be curious to
       | see how this technique starts to degrade as the input FPS goes
       | down.
       | 
       | Seems like it already can dramatically reduce the amount of work
       | involved in making an animation. Very cool to see pixel art
       | animated at 60 FPS!
        
       | Sesse__ wrote:
       | There's a huge amount of research within this field. For a
       | ranking list on a standardized test set, you can see e.g.
       | http://vision.middlebury.edu/flow/eval/results/results-i1.ph...
       | (there are many others). DAIN is roughly in top 10.
       | 
       | Sadly, there's next to no focus on the realtime use case, which
       | is useful for e.g. sports. For my own project Futatabi (see
       | https://nageru.sesse.net/), I chose to reimplement one that's
       | fairly obscure, DIS (Dense Inverse Search), on the GPU, so that I
       | could get it realtime. IIRC, it's nowhere as good as DAIN is, but
       | there's that 60 fps demand... :-)
        
         | dharma1 wrote:
         | good info! How much better are the new deep learning based
         | methods compared to older optical flow approaches like Twixtor?
        
       | ArtWomb wrote:
       | Original research:
       | 
       | https://sites.google.com/view/wenbobao/dain
        
       ___________________________________________________________________
       (page generated 2020-01-19 23:00 UTC)