[HN Gopher] Show HN: Create your own video clips with Stable Dif...
___________________________________________________________________
Show HN: Create your own video clips with Stable Diffusion
A few months ago, I started making video clips with stable
diffusion and noticed that the tools to do this were too
complicated for everyday people. That's why I built neural frames.
Enjoy.
Author : nicollegah
Score : 160 points
Date : 2023-01-15 12:55 UTC (10 hours ago)
(HTM) web link (www.neuralframes.com)
(TXT) w3m dump (www.neuralframes.com)
| seydor wrote:
| Yannic kilcher had created a videoclip for lyrics made of
| imagenet labels last year:
| https://www.youtube.com/watch?v=rR5_emVeyBk
| tekni5 wrote:
| Very cool, your demo video looks great, would love to try it when
| it's working again. You can also do this type of thing on colab
| with Deforum Stable Diffusion:
| https://colab.research.google.com/github/deforum-art/deforum...
|
| I've been messing with it myself, here is an example:
| https://youtu.be/FsVskNtNazk
| nicollegah wrote:
| It should absolutely work now. Doesn't it?
| tekni5 wrote:
| Seems to be overloaded.
|
| "There's a lot of requests currently and our servers are
| overloaded - sorry. I am trying to increase the capacity.
| Please try again later."
| francis_lewis wrote:
| This is very cool! What sort of values are you using for the
| denoising strength/guidance scale? Each frame is a nice level of
| similar/different to the last to create flowing video.
| RichardGao112 wrote:
| Nice! This is much simpler than using Deforum
|
| Is everything set up on the cloud yourself, or do you use an API?
| nicollegah wrote:
| I've set it up myself. The APIs I saw didn't seem suitable for
| this real-time inference thing.
| jsjohnst wrote:
| I'd much prefer to run this locally on my own. Any chance
| you're willing to share code (or suggestions / links since it
| looks like you might be trying to monetize this per your
| Twitter posts)?
| nicollegah wrote:
| I understand and have been thinking about it. Would somehow
| like to monetize this but if it doesn't work out I might
| just open-source it.
| metadat wrote:
| You might be able to do both. Lots of people don't have
| high-end video cards or an M1, and will pay for the
| convenience you offer.
| jsjohnst wrote:
| Completely agree
| kruuuder wrote:
| Is there an example video somewhere?
|
| Edit: Nevermind - I don't have autoplay active and there was no
| way to see that the "image" on the landing page is actually a
| video (no UI like a play button).
| nicollegah wrote:
| it's on the landing page directly. You can also check
| https://twitter.com/neuralframes
| lxe wrote:
| Lol did you mean to past that last one on your twitter? I
| think you might need some content filtering.
| nicollegah wrote:
| hahaha, i think you're right.
| lukeplato wrote:
| I suggest that people simply run Stable Diffusion Deforum
| themselves (there's an extension for automatic1111's web UI). You
| can run it in a google colab notebook and the cost will likely be
| cheaper or the same, though I haven't bothered to compare.
| ecliptik wrote:
| Reminds me a lot of the "Ghost" music video for by Gunship[1]
| made by aiplague[2].
|
| 1. https://www.youtube.com/watch?v=aUJuwNxNUWQ
|
| 2. https://aiplague.com/
| jsjohnst wrote:
| Couple suggestions after playing with this:
|
| 1) the experience mostly works on mobile (where I first tried
| it). With only minimal changes in the fixed sizing I think you
| could make this mobile friendly.
|
| 2) for posts shared via your Twitter, would be interesting to see
| details about the prompt(s) used vs "new post"
|
| 3) I'd like to have a bit more customization in the options
|
| Overall really nice and good luck with monetizing it. I'd love to
| see a blog post write up on the technical implementation. That's
| something I'd more be willing to pay to see personally.
| nicollegah wrote:
| Those are very good suggestions. I built it with mobile in mind
| but somehow I failed to get it right, yet. Will continue to
| work on it. Would also be nice to make a native app at some
| point.
| tehsauce wrote:
| Very cool! I just released a tutorial on how to do this using the
| computerender api!
|
| https://github.com/computerender/tutorials/tree/main/python/...
|
| If you're interested in saving money on expensive cloud gpus, our
| api is much cheaper than this. (only $0.001-0.0025 per frame)
| sebzim4500 wrote:
| How is it so cheap? I think that's a factor of 3 cheaper than
| others.
| tehsauce wrote:
| The GPUs are rented from vast.ai The individual machines
| aren't as reliable or well-integrated with other cloud
| services as typical cloud machines, but multiple can be put
| behind a queue to create a highly reliable service.
| nicollegah wrote:
| looks cool. somehow the website looks screwed up in my browser
| (chrome) and I cannot get an API key after signing up.
| tehsauce wrote:
| Hm, what device/os are you browsing on? The site should be
| mobile-friendly except for the account page. Also please feel
| free to reach out by email or discord.
| conidig wrote:
| do you plan on adding dreambooth? I would give it a try.
| tehsauce wrote:
| I would like to support dreambooth, if there is a way to
| store the fine-tuned models more efficiently. The challenge
| is that each trained model is quite large and a bunch of
| models can't fit into one gpus memory at once.
| conidig wrote:
| textual inversion maybe? lightweight embeddings and I
| haven't seen any API offering it at the moment.
| pcrh wrote:
| The more AI-generated art I see the more convinced I am that it
| will generate entire new modes of art creation, rather than make
| creative work redundant.
|
| I'm now waiting for a creation that could not have been done
| without AI, the labour that would be involved to create these
| works manually not being considered.
| [deleted]
| nuclearsugar wrote:
| I've been experimenting with creating image datasets via Stable
| Diffusion, then training StyleGAN2, and compositing in After
| Effects. It's a new mode of art creation for me that I cannot
| recreate otherwise. https://www.jasonfletcher.info/vjloops/
| imhoguy wrote:
| One video brought me memory of Autechre - Gantz Graf music
| video (warning, hard IDM style ;) [0]. It was made manually
| in 2002, per Wikipedia: "Rutterford also stated that there
| was no generative element to the imagery; every three-
| dimensional object in the agglomeration was painstakingly and
| manually synchronised with a specific element or frequency
| range within the track" [1].
|
| [0] https://www.youtube.com/watch?v=ev3vENli7wQ
|
| [1] https://en.wikipedia.org/wiki/Gantz_Graf
| jcims wrote:
| Very cool!
|
| I wonder if this could be merged with the work done to build
| music from inverted FFT's to do Aphex Twin type
| visualization, or alternatively visualizations through
| oscilloscope music like Jarobeam Fenderson.
| adzm wrote:
| This is some truly amazing art. I kept reading one post after
| another of yours.
| pcrh wrote:
| Awesome! It reminds me of time-lapse movies in developmental
| biology, showing, for example, the various shapes an organism
| has as it develops.
| mustacheemperor wrote:
| This takes me back to the demo scene days. Seeing something
| that's both a technical and artistic achievement that could
| only have been borne out of the cutting edge. Thanks, awesome
| stuff.
| Magi604 wrote:
| Reminds me of something you would see in a newer modern remake of
| Decasia.
|
| https://en.wikipedia.org/wiki/Decasia
| https://www.youtube.com/watch?v=hDa-mmSldDg
| will5421 wrote:
| What prompts did you use to make the video?
| NicoleJO wrote:
| The company that makes stablediffusion is being sued for
| copyright infringement. Don't use this.
| sebzim4500 wrote:
| I would be shocked if there were any significant tech companies
| that are not currently being sued. Frivolous (or maybe even not
| so frivolous) lawsuits are a fact of nature, best not to waste
| time worrying about them until you are the target.
| wellthisisgreat wrote:
| That's a strange statement.
|
| Who cares about frivolous lawsuits. Stable Diffision and the
| like tools will prevail. The main hope is that it will be their
| OSS versions and not corporate (Dall-E).
|
| The works and ways of lives that are threatened by SD and the
| like are not worth preserving
| ghaff wrote:
| I wouldn't call them frivolous. Quite a few people believe
| training generative AI models on their copyrighted work and
| providing others with access to that model violates their
| copyright. IANAL but it seems a weak argument to me. And it
| seems weak to some IP lawyers I know as well.
|
| It also seems like technology that can't be realistically
| bottled back up. However, I wouldn't call lawsuits frivolous
| and it might actually be useful to get some legal clarity
| julianeon wrote:
| It would be like banning Bittorrent at this point.
|
| Stable Diffusion - or something functionally identical - is
| here to stay.
| jtsiskin wrote:
| Everyone's being sued for something somewhere. You can use
| this.
| nicollegah wrote:
| Hi all, thanks for all that attention to the site. I am super
| happy about that. A word of caution: I can only rent 4 GPU
| instances currently on AWS due to service quota limits.
| "Unfortunately" the traffic due to hackernews is too high for
| that. Sorry for any inconveniences. If you are having troubles at
| the moment, come back later or so. I asked for service quote
| increase but those take usually 24h or so.
|
| Also, very little people actually pay so I can only afford so
| much.
| atylerrice wrote:
| I don't usually comment, but aws is actually rather expensive
| and i've hit this annoying quota problem as well. Been back and
| forth and still hasn't been raised. I can also recommend
| coreweave banana.dev and pipeline.ai as great alternatives.
| This service looks awesome good luck with the launch!
| snissn wrote:
| hey! FYI i think this limit is usally per geograhic site, so if
| you can only rent 4 in us-east-1 try us-west-1 etc
| nicollegah wrote:
| ah sick, i didn't know that. thanks!
| gmuslera wrote:
| The description of what you have to do reminds me the
| instructions on how to draw an owl
| (https://knowyourmeme.com/photos/572078-how-to-draw-an-owl)
| [deleted]
| nerdponx wrote:
| What will be interesting is whether we will soon have a model
| that can actually follow these steps.
| [deleted]
| O__________O wrote:
| Might want to put notice:
|
| "There's a lot of requests currently and our servers are
| overloaded - sorry. I am trying to increase the capacity. Please
| try again later."
|
| ...prior to making users do multiple clicks, opt to not provide
| an email, etc.
| [deleted]
| O__________O wrote:
| Then, if I finally try to render a 5 second click I get:
|
| "There's high demand on the servers currently. Sorry for any
| inconvenience. I am trying to scale up the servers."
|
| RIP
| O__________O wrote:
| Might want to put notice:
|
| "There's a lot of requests currently and our servers are
| overloaded - sorry. I am trying to increase the capacity. Please
| try again later."
|
| ...prior to making users do multiple clicks, opt to not provide
| an email, etc.
|
| Even a prerecorded demo posted to Youtube would be a better
| experience.
| [deleted]
| throwmeup123 wrote:
| ...no noise overlay on the input images to at least get some sort
| of frame by frame consistency? -.-
| nicollegah wrote:
| Do you have some resource to learn how this works? I'd love to
| implement it.
| refulgentis wrote:
| It's baked into Deforum, the bit you'd want to look into here
| is the recent changes for Perlin noise.
___________________________________________________________________
(page generated 2023-01-15 23:00 UTC)