[HN Gopher] Show HN: Open Prompts - dataset of 10M Stable Diffus...
___________________________________________________________________
Show HN: Open Prompts - dataset of 10M Stable Diffusion generations
Open Prompts is the dataset used to build krea.ai. The data comes
from the Stability AI Discord and includes around 10M images from
2M prompts. You can use it for creating semantic search engines of
prompts, training LLMs, fine-tuning image-to-text models like BLIP,
or extracting insights from the data--like the most common
combinations of modifiers.
Author : vipermu
Score : 112 points
Date : 2022-09-22 19:03 UTC (3 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| Philomath wrote:
| That's amazing, thanks for sharing. For how long have you been
| gathering this data?
| vipermu wrote:
| We do not have a continuous system, the data is a mix between
| our own crawled generations and the dataset published by Dave
| Caruso (https://github.com/paperdave). With our crawler, we
| were able to get about 100k generations per day.
| masterspy7 wrote:
| I'm curious, I've seen a few sites like this which grab from the
| Stability Discord. Is there a way to quickly scrape this amount
| of data from a Discord server?
| rhacker wrote:
| someone should do a thing that lets strangers critique AI by
| allowing them to select:
|
| A person has a glitchy face in this photo. A person has a glitchy
| body in this photo. etc..
|
| and then train the AI to have a fixup pass.
| mgraczyk wrote:
| One of the main things were learning from the current
| trajectory of large models is that this kind of supervision
| isn't necessary. It's better to focus on bigger models, more
| data, and better datasets. Models will improve faster than we
| can come up with clever ways to add this supervision.
| dvrp wrote:
| Interesting. Kind of like Scale AI for generative AI.
| vipermu wrote:
| I wonder how CLIP search would work for finding errors in the
| dataset.
|
| For now, a workaround is to create your own "glitch" collection
| in krea.ai and store there images with artifacts.
|
| If you end up doing it we will add a "download all" button
| right away :)
|
| And all the prompts from each collection could also be added to
| Open Prompts for sure.
| jononor wrote:
| Nice. Another meta thing I would like to do, is to generate a
| bunch of prompts around a topic, mashed up with related or
| unrelated other topics. So that I can get a bunch of images and
| just be able to review/curate them all in one go. Does anyone
| know of tooling in that direction?
| vipermu wrote:
| You can use the code in https://github.com/krea-ai/prompt-
| search to do so. You first want to compute the CLIP embeddings
| of each prompt, index them using something like K-Nearest
| Neighbors (so you can get search for similarities fast), and
| then, given an input prompt, you will be able to find other
| indexed prompts that share their semantics.
| nextaccountic wrote:
| Oh my god. This means that we can use stable diffusion to do
| pure text processing, like, given two textual descriptions,
| assess how similar they are
|
| I now expect that the next model like GPT-3 will be multi-
| modal like Stable Diffusion, to better account for those
| semantic connections
| alecdibble wrote:
| This is a very interesting point. It's like using Fourier
| Transforms/Laplace Transforms to do operations or
| comparisons that are way easier in those spaces.
| davidkunz wrote:
| Hi, just for you to know: The "krea.ai" link in the readme gives
| a 404: https://github.com/krea-ai/open-prompts/blob/main/krea.ai
| vipermu wrote:
| fixed!
|
| thanks for spotting the issue :)
| cercatrova wrote:
| How does it compare with https://lexica.art?
| vipermu wrote:
| The source (Stability AI Discord) is the same, but I don't know
| how Sharif gathered his data.
| dvrp wrote:
| Krea dev here.
|
| Lexica is a search engine (like krea.ai), but it doesn't allow
| you to create collections or like generations.
|
| Regarding the API, both have public APIs although I'm not sure
| if you can paginate through several search results using the
| public Lexica API. In the Krea Prompts API, you can do cursor-
| based pagination.
|
| Finally, Lexica API allows you to do CLIP-based search, but
| with Krea we are using PostgreSQL full-text search (for now).
| However, the code to do CLIP search with the dataset (including
| reverse image search) is in the repository.
|
| (edit: also, nor Lexica nor other search engines or similar
| products are offering the dataset afaik.)
| Oras wrote:
| This is fantastic. A few days ago I was checking PromptBase [0]
| and thought it was a really good idea. Yours just took it to the
| next level being free with massive amount of data.
|
| Great work.
|
| [0] https://promptbase.com/
| dvrp wrote:
| Thanks!
|
| I'd love to integrate our crawler with GitHub Actions and make
| it a self-updating dataset...
|
| There's so much stuff to do!
| vipermu wrote:
| Thanks! Prompts can be hard to create, we hope that having
| access to these kinds of datasets we will be able to create
| tools and conduct studies that help us create better images and
| understand better the possibilities of AI models like stable
| diffusion.
| Samin100 wrote:
| Great work! If anyone's planning to use AI generated artwork in
| their projects, I've added an image search API to Lexica, similar
| to Unsplash. All the images are licensed CC0 and millions more
| are being added every few weeks.
|
| Docs here: https://lexica.art/docs
| Lerc wrote:
| Been using Lexica quite a bit for prompt analysis, thanks for
| the work.
|
| General browsing is heavily dominated by portraits though. I
| was wondering if it would be worth having a face detected flag
| on images so you could filter portraits.
| tgtweak wrote:
| Great work on Lexica.art, it's been indispensable for finding
| good prompts and combinations.
|
| How much does it cost to host it? Feels like hosting 500GB of
| images and serving them can't be cheap.
| Samin100 wrote:
| Last month Lexica served a little over 1 billion images and
| the Cloudflare bill (I'm using R2 + workers) was a little
| over $5k. I've since gotten it down to a more reasonable
| amount after spending some time to re-encode the images to
| reduce our bandwidth usage significantly. If Lexica were
| running on AWS/S3 I imagine our first month's bill would be
| closer to $100k rather than $5k. This is only image serving,
| so not including costs to run the beefy CPU servers to run
| CLIP search, frontend, DB, backend, etc.
| vipermu wrote:
| Thanks! Great work with Lexica.
|
| We also released a free API https://devapi.krea.ai/ if anyone
| wants to check it out.
|
| It will soon have endpoints with custom image generation
| features.
| dr_dshiv wrote:
| Radical. I'm imagining randomly sampling images and identifying
| the text attributes associated with human ratings of image
| beauty.
| vipermu wrote:
| in krea.ai you can create collections of images with their
| prompts by pressing the "+" button in an image.
|
| you also have access to all the different components that
| create each prompt, and you can search similar ones by clicking
| them.
| nextaccountic wrote:
| Hey, was the specific Stable Diffusion version used to generate
| each image recorded anywhere in the dataset?
|
| In krea.ai, it doesn't say which version of the model was used to
| generate each image
|
| It appears that later versions are better in generating faces or
| something. Like, Stable Diffusion 1.5 vs 1.4 (I'm not sure but
| there's a great variability nonetheless and I wanted to know if
| the version of the model accounted for this)
| jaimex2 wrote:
| There's also https://lexica.art/
| whalesalad wrote:
| This is wild to me. Now we have meta-ai that is surrounding other
| forms of ai, analyzing the user submitted input as well as the
| image output, using ai to infer intent, identify nouns etc... and
| yet all of this is stipulated on the initial datasets that these
| initial text->img robots were trained on which may or may not be
| a true representation of our actual culture. So we are lava/magma
| layering all of these approximations on top of each other and
| gluing them with scrambled eggs. I think this is all really cool,
| for the record, it's just something I have been thinking about.
| For art, I love it, for a self-driving vehicle, lmao.
| vipermu wrote:
| We're living some crazy times! A truly AI summer.
|
| If you enjoy thinking about how the future of this field might
| look like, I highly recommend watching the interview between
| Yannic Kilcher and Sebastian Risi
| (https://www.youtube.com/watch?v=_7xpGve9QEE).
|
| I was mind-blown after hearing it. It was a long time since I
| didn't hear such an interesting conversation. It's crazy how
| Risi's ideas correlate so well with the way how complex systems
| emerge in nature (optimizing locally), and the idea of self-
| organizing systems is just amazing.
| dr_dshiv wrote:
| > AI Summer
|
| Thanks!
| nextaccountic wrote:
| This is fantastic, thanks for publishing it. I'm glad many
| players in the Stable Diffusion ecosystem is striving for
| openness (not only of the model itself but there are also open
| source frontends and related tooling)
| vipermu wrote:
| Stable Diffusion was released just a month ago and look at the
| amount of applications and improvements that have been
| developed, it feels like a year!
|
| Open-source is the way to get the most out of this tech. We
| plan to keep building all the features that are to come at
| krea.ai in this way.
| hwers wrote:
| What's interesting about datasets like this is that you can
| likely use it to distil an even more compressed SD generator from
| it.
| vipermu wrote:
| yup! btw, if you want compressed generators you'll enjoy this
| discussion https://discuss.huggingface.co/t/decoding-latents-
| to-rgb-wit...
___________________________________________________________________
(page generated 2022-09-22 23:00 UTC)