[HN Gopher] Generative Image Dynamics
___________________________________________________________________
Generative Image Dynamics
Author : hughes
Score : 135 points
Date : 2023-09-16 16:08 UTC (6 hours ago)
(HTM) web link (generative-dynamics.github.io)
(TXT) w3m dump (generative-dynamics.github.io)
| soultrees wrote:
| This is super cool. Cinemagraphs have always been a bit of a
| passion of mine, and I try to bring that feeling of subtle-
| stillness in a lot of the work I do, whether it's marketing or
| shooting, so i can see this becoming a regular tool.
|
| The trick to a 10/10 cinemagraph is the more subtle, the bigger
| the impact. You almost want the viewer to think it's a still
| photo before their brain clicks in thinks "wait, something isn't
| normal here, this isn't a photo, it's a video"
| albert_e wrote:
| any good examples you can share please?
| nico wrote:
| Not op, and not sure if these qualify as cinemagraphs, but
| they kinda fit the "subtle image movement" description
|
| https://twitter.com/nicobrenner/status/1685754265393041408?s.
| ..
|
| https://twitter.com/nicobrenner/status/1686074457159274496?s.
| ..
|
| https://twitter.com/nicobrenner/status/1685875401351217152?s.
| ..
|
| https://twitter.com/nicobrenner/status/1686060465783443457?s.
| ..
|
| https://twitter.com/nicobrenner/status/1685170395086045185?s.
| ..
| CSSer wrote:
| They used webGL for the demo. Nice!
| sva_ wrote:
| This would be crazy in a video game. Walking through a bush and
| dragging the plant with you
| behnamoh wrote:
| I'm still waiting for video games to adopt stable diffusion,
| GPT, and other GenAI models. the tech is there, but I guess
| the inertia in the industry doesn't allow us to have nice
| things yet.
| a_wild_dandan wrote:
| You'll see those things! We already have DLSS, for
| instance. But unfortunately we can't simply glue expensive
| black boxes onto games and ship them. Wrangling performant,
| richly interactive media is difficult enough _without_
| these models. This modern ML + gaming fusion space is
| barely in its infancy. We need to explore what 's
| practical, and discover patterns to do it.
|
| Even without further breakthroughs, the next 5 to 10 years
| will be incredible. I'm so excited.
| wayfinder wrote:
| Wouldn't say tech is there yet. It still needs a lot of
| human input and direction so slapping it into a video game
| would just be immersion breaking when it generates
| something out of character randomly.
|
| There's less impactful ways to implement it like generating
| art paintings in a museum dynamically but that is in the "a
| little gimmicky" territory.
| CSSer wrote:
| Wow, that's a neat idea! That could potentially be pretty
| cool. It'd almost be like a form of photogrammetry but for
| physics. Kinegrammetry, maybe? I wonder what the storage
| efficiency and performance would look like. Perhaps something
| like ths could be adapted into a framework for object
| modeling.
| pelorat wrote:
| What do you mean, lots of games have physics that interreact
| with flora?
| [deleted]
| waffletower wrote:
| Nice to see Google researchers continuing to publish open papers
| with bonus demos. Won't beat a dead horse about Google failing to
| productize or open source their AI research.
| divyajg wrote:
| I wonder why in the first picture (red rose) the flower in the bg
| also moves, but we don't see the same affect in the third picture
| (tree). I also find it impressive that the amount of motion
| differs in the first and the second picture, could it be because
| the density around the pointer is considered?
|
| The slo-mo ones are super relaxing to watch!
| Hard_Space wrote:
| This suffers from the same low-vector movement requirements as
| EbSynth.
| GaggiX wrote:
| I think the achievement here is mostly about generating the
| image dynamics, so for example there is a cat in an image, the
| model understand that cats need to breathe so the dynamics show
| the lungs contracting, then the paper covers how to traslate
| the image dynamics and the image itself into a seamless video.
| I could be wrong tho
| [deleted]
| juunpp wrote:
| The tree has severe distortion when dragged from the edge. Still
| an interesting idea.
| Timon3 wrote:
| You'd probably have to combine this with segmenting and
| generative infill for the background layers, but luckily
| there's been a lot of progress there!
| crazygringo wrote:
| This is so cool. Not earth-shattering or productivity-enhancing,
| but still really cool.
|
| I could definitely see this becoming a standard feature on
| desktop and phone wallpapers.
|
| Could also see it being applied selectively to photos in things
| like historical documentaries -- especially if it can handle the
| gentle movement of water and clouds as well.
___________________________________________________________________
(page generated 2023-09-16 23:00 UTC)