[HN Gopher] How a Stable Diffusion prompt changes its output for...
___________________________________________________________________
How a Stable Diffusion prompt changes its output for the style of
1500 artists
Author : politelemon
Score : 239 points
Date : 2022-10-02 12:30 UTC (10 hours ago)
(HTM) web link (gorgeous.adityashankar.xyz)
(TXT) w3m dump (gorgeous.adityashankar.xyz)
| jerpint wrote:
| I wonder how constant everything else is kept, e.g. The seed.
| It's interesting that all the poses seem to align
| mjamesaustin wrote:
| I noticed that too! I'm guessing they used the same seeds,
| which does a great job of providing a comparison in style.
| wccrawford wrote:
| Yeah, they use the same seed, which is used to generate random
| color pixels. Then they algorithm takes it from there.
| EVa5I7bHFq9mnYK wrote:
| Interesting, how with billions of nodes and supposed
| "intelligence", the network hasn't been able to deduce a simple
| concept of symmetry in human faces. All of the eyes and all of
| the lips in all of the pictures are asymmetrical, which easily
| gives AI generated images away.
| pmoriarty wrote:
| Midjourney is way better than this, though it too can produces
| some weird results... but at its best it's absolutely
| indistinguishable from photographs or art made by humans.
| chaxor wrote:
| Many of the artists in this list specifically _try_ not* to
| have this symmetry present in the faces. It's what makes many
| styles of art separate from just taking a photo or simply going
| outside.
|
| The system used here is actually astoundingly good at producing
| many artists styles _because_ it 's not going for symmetry.
| MengerSponge wrote:
| It feels like they really only track local continuity without
| any meaningful knowledge. Hands are wonky and wrong in ways
| that don't look like any drawing I've ever seen. Horses with
| five or six legs.
|
| Humans also have a hilariously hard time drawing bicycles, but
| at least we pretty much always nail the number of appendages.
| capableweb wrote:
| "Facial symmetry === beauty" is not that old of a scientific
| concept, relatively new if you compared to how long humanity
| has existed before someone started to really study it.
|
| And even so, too symmetrical faces will look just as un-human
| as a face that is too asymmetric. You need a face that is just
| the right amount of symmetrical in order for it to actually
| look good.
|
| I think you make it sounds simpler than what it is.
|
| It's also not a model that is trained to make as realistic
| people as possible, it's trained on a lot of different things,
| so obviously it won't excel at making realistic people. But one
| can easily imagine that some future models will be heavily
| trained on making realistic people rather than semi-realistic
| _everything_ , like Stable Diffusion is trained to do today.
| pessimizer wrote:
| From an artist to a brand identity.
| pain2022 wrote:
| ShamelessC wrote:
| > What is this list?! No Picasso, Renoir or Vrubel present
|
| Comments like this are why people think HN is nothing but a
| place to receive a shallow dismissal of your efforts.
|
| It's a list. Make your own.
| jazzyjackson wrote:
| > Comments like this are why people think HN is nothing but a
| place to receive a shallow dismissal of your efforts.
|
| whatever
| ShamelessC wrote:
| What is this?! One word comments?!
| the_only_law wrote:
| Yes, it's been like this for a while.
| kwertyoowiyop wrote:
| "Be the change you want to see in the world" has kept me from
| posting many unhelpful comments.
| EGreg wrote:
| How can we set up Stable Diffusion on our own Linux servers, so
| we can generate NFTs or whatever?
| layer8 wrote:
| The result for H. R. Giger didn't quite live up to my
| expectations.
| krumbie wrote:
| What's kind of crazy is how the images tend to have similarities
| in small features that become very apparent when flipping back
| and forth between images, but which are not obvious per se.
|
| For example, I flipped back and forth between Beatrix Potter and
| Paulus Potter. A rounded white bonnet in one picture becomes a
| couple of blossoms in the other. The roof of a house becomes some
| shadowy wall with plants in the other. Two flower pots are very
| similar, just with slightly different coloring.
|
| It makes it more apparent that the algorithm etches the images
| out of noise, and if the seed is the same for two images with
| different prompts, you're likely to see traces of that noise
| represented differently but recognizable in both images.
| djmips wrote:
| So interesting, and really seems somehow related to how we
| dream, hallucinate.. or even experience reality?
| kache_ wrote:
| from "is your blue my blue" to "is the AI's blue our blue" :)
| deltasevennine wrote:
| Arguably it could be different from our experience. It could
| even be a superior and more efficient methodology then the
| things our brain uses to imagine things.
| daveguy wrote:
| The positioning seems very consistent, almost to the point
| where I wonder if that was part of the selection process to
| demonstrate the differences in style. There are only four per
| style, where the position of a subject could be a selection
| factor. Hard to tell if the position similarities are driven by
| the Stable Diffusion model or by the selection of
| representative images.
| olejorgenb wrote:
| Maybe it's based on img2img? Some are different enough that
| it's not obvious that's the way it has been done though.
| adityashankar wrote:
| Hey, I'm the developer of this project, no this is not
| based on img2img all the images just have the same seed
| michael-ax wrote:
| incredible job! i was just showing my own experiments
| with sd to a friend and this just take the cake. thank
| you so much for mixing in that artist list!!
| tough wrote:
| The composition and positioning come from the original seed.
| If the same seed is used, the same background image noise is
| applied for the initial image which is transformed into all
| the styles.
|
| Thus the similarities you see would make sense if using also
| the same seed for the tests.
| [deleted]
| norwalkbear wrote:
| Truly impressive, though it gives me some great fear for all the
| people whose career is art. This will likely take the pareto to a
| new height instead of 20/80 maybe even 1/99
| SergeAx wrote:
| Just read the "Profession" by Isaak Azimov :)
| yaddaor wrote:
| Art is rarely about the product. Some color on a canvas has
| been traded for millions, while anyone could just have dropped
| a bucket of paint to create a similar result.
| otabdeveloper4 wrote:
| After checking some 19th century artists I see it failed hard.
| All of the responses look the same, there isn't enough data in
| the training set to differentiate actual styles beyond "vaguely
| realistic".
| yieldcrv wrote:
| Their career isn't art anymore.
|
| Just in case anyone needed to see this spelled out.
|
| The people making fliers have been replaced by AI prompting
| overnight.
|
| The people doing contemporary fine art with their audience are
| unaffected.
| krisoft wrote:
| > The people making fliers have been replaced by AI prompting
| overnight.
|
| Hmm, no? Do go and try to make a flyer with any of the AI
| generators. I'm not saying it can't happen one day, but the
| current tech is not there.
| pessimizer wrote:
| Flyers are an extremely low bar that AI sails over.
| [deleted]
| theelous3 wrote:
| show an example?
| pessimizer wrote:
| I used to draw the flyers when my band played shows.
| People still went to the shows. If had iterated a few
| times with one of these models to draw the thing that I
| wanted to draw rather than deluding myself, and spent 5
| minutes polishing it in photoshop, the product would have
| been orders of magnitude better imo.
| theelous3 wrote:
| So when you say flyers, you're talking about the kind of
| flyer zero people are employed to make. We're talking
| about paid artists here.
| yieldcrv wrote:
| Its a post about how artists monetize.
|
| That group will never be selling their own work as
| contemporary fine art but want that prestige, and the one
| way they had to make table scraps with that skillset is now
| gone.
|
| A different person is doing their own flyer art with AI and
| adding words around it themselves, as evidence by my last
| months worth of fliers that have reached me. Promotion
| companies have always been up on trendy tools for
| differentiation.
| fassssst wrote:
| A lot of designers are artists that needed an income. Design
| jobs will likely be dramatically impacted by this.
| [deleted]
| seanwilson wrote:
| > The people making fliers have been replaced by AI prompting
| overnight.
|
| I haven't seen good examples of that yet but I'm curious how
| push-button you can make this. Flyers, web design and UI
| design require the copy, layout, information hierarchy,
| colours, illustrations, branding etc. to be cohesive so it's
| a different problem space with way more constraints compared
| to generating a single image.
|
| If getting the final design requires a lot of rounds of
| prompting and tweaks, busy people are going to outsource this
| still (in the hope the prompting and feedback needed to the
| person doing the work will be less).
| theelous3 wrote:
| this seems exactly backwards
|
| SD isn't outputting clean graphic designs with sensible
| content
|
| And haven't there already been people winning contemporary
| fine art contests with SD? lol
|
| https://www.reddit.com/r/StableDiffusion/comments/x2n0r1/aig.
| ..
| marginalia_nu wrote:
| > The people doing contemporary fine art with their audience
| are unaffected.
|
| It's not a particularly well hidden secret that the
| contemporary fine art is really not about the art or the
| artist.
|
| https://www.youtube.com/watch?v=ZZ3F3zWiEmc
| sva_ wrote:
| The Jackson Pollock one amuses me. He didn't really draw such
| symbolic pictures afaik, so applying the style makes little
| sense.
| antman wrote:
| Nice, list could be sorted though
| j5155 wrote:
| It's sorted by last name
| biztos wrote:
| I've been idly working on a similar list but with a much more
| basic prompt.
|
| The results vary wildly, even run by run, but I might put them in
| a few buckets:
|
| 1. Similar enough that someone who doesn't know much about art
| could be fooled
|
| 2. Amateur knockoff but recognizable style
|
| 3. Influence is there if you know what to look for
|
| 4. Artist probably not in the training data at all
|
| The last one kinda surprised me, for artists whose work is online
| and who have unusual names. I would have thought those cases
| would be really good. Maybe they ran out of disk space with all
| the porn?
|
| Also interesting that it gets much closer for figurative painters
| than for abstract painters.
| akie wrote:
| It would be _so_ much better to just have all of these 1500
| pictures next to each other in a list with the artist 's name
| under it.
| [deleted]
| wilg wrote:
| I think the idea is for the comparison to remain spatially
| consistent because that's what it's trying to highlight!
| amelius wrote:
| Including a few real sample pictures from the artist, or a link
| to a google image search of the artist name, so you can check
| if the generated style matches.
| ralfd wrote:
| YES! I got mildly irritated using that clunky list.
| gvv wrote:
| Indeed, what I ended up doing was Inspect Element > Search the
| name I want > Add the "selected" attribute to the option.
| FiReaNG3L wrote:
| Or at the very least with the list of artists alphabetically
| sorted!
| kjeetgill wrote:
| They're sorted by last name, it took me a second too! I just
| wish I knew more of them.
| ido wrote:
| Sorted by last name but listed by first name means you can
| start typing the artist's name and get to the right place
| in the list...
| Kerbonut wrote:
| You can do it anyway assuming they used a moment js
| framework and took 2 seconds to configure the list order
| and search order
| klyrs wrote:
| Most of the names are sorted! Scroll to the very end...
| theiz wrote:
| Wish there was some AI tool that could do some proper ligne
| Claire. It seems to be quite impossible. Oh, as is Escher, but I
| do understand that one.
| bee_rider wrote:
| It is sort of funny/interesting -- I only tried a few, but famous
| anime or manga artists (try "Junji Ito" or "Hayao Miyazaki") seem
| to have at least one picture that is clearly the result of the
| algorithm picking up on their fans' art.
| amelius wrote:
| This website should include a google-image-search link to the
| artist, so you can compare the styles more easily.
| logisticpeach wrote:
| Hate to say it but when i see stuff like this it only reminds me
| of what we could have achieved if this ingenuity had been applied
| in another domain.
|
| Can't help feeling that this accidentally harms creative types
| and risks swamping us with visual junk.
|
| The technical achievment is astounding but no-one would seriously
| claim that crafting an image via a short prompt is creative
| except in the most cursory way.
|
| I'm probably missing some life changing use-case, but apeing art
| in random styles can't be it.
| educaysean wrote:
| I'm of the opposite opinion. AI assisted art is simply the
| natural next chapter for "art" as a whole. It will finally
| kickstart the public discourse about what being an artist means
| in the perspective of artistic vision vs execution.
|
| Most artists spend their lives not refining their brush stroke,
| but rather their eyes. The way I see it, the impact of curation
| and artistic direction will matter more and more in the future.
| intelVISA wrote:
| As with programming, an AI model cannot replace the key parts
| but can help automate the monotony.
|
| For me it's exciting to use as placeholder art and then have
| a 'real' artist review it.
| victor9000 wrote:
| Well, robots have been better than humans at playing chess for
| decades now, and we still have chess players.
| capitalsigma wrote:
| I don't know why everyone assumes that ML researchers have some
| big map of the future where they can make decisions like "yeah,
| let's choose this branch over here, where AI gets good at
| generating art first, rather than that other one where it cures
| cancer a decade earlier." The breakthroughs come where they
| come, and no one knows where some model architecture will have
| an application in the future.
| bawolff wrote:
| > only reminds me of what we could have achieved if this
| ingenuity had been applied in another domain.
|
| I hate arguments like this. Even ignoring how dismisive it is
| of the achievement at hand, why would you assume ingenuity is
| transferable like that? Someone who makes a breakthrough in
| physics is by no means likely to have made an equivalently
| ground breaking advance in biology if they had decided to study
| that field instead.
| logisticpeach wrote:
| Aside from the fact that I explicitly praised the achievment,
| my point actually relies on said appreciation.
|
| I guess my musing was hypothetical but I was careless in
| communicating that. I get that we can't centrally plan
| innovation or human effort - and I certainly wouldn't want to
| live in a society where this was the case.
| nopenopenopeno wrote:
| I am a filmmaker and most films are essentially crafted this
| way. Beyond hiring and securing resources, directors
| essentially create by communicating ideas in short prompts
| because there isn't enough time to do anything more.
|
| I could absolutely see an AI model doing the job of an entire
| film crew. I have issues with this, but only with respect to
| the longer term aggregate affects on culture in the broad
| sense. I cannot honestly believe that much would be lost from
| the perspective of one project or another.
| kchod wrote:
| I would look at it as more of "allowing normal people access to
| unbefore-dreamed-of levels of draftsmanship" vs some comment on
| capital-A Art. It allows non-artists to express themselves
| visually.
|
| I'm biased: I've been working on an image generation app. But
| the beta users I've had so far will generate fifty or a hundred
| images in a day. That isn't a use case traditional artists
| support.
| soulofmischief wrote:
| People said this about cameras. About digital cameras. About
| digital photo editing software. The next generation will
| normalize these tools and find incredible ways to be creative
| within their new cutting edge medium.
|
| The post-art world is here! Just think about how history books
| will remember this period! The styles that will be borne of
| necessity, of the need to break down art and find what makes it
| tick.
| LeoPanthera wrote:
| > People said this about cameras. About digital cameras.
| About digital photo editing software.
|
| Did they? Because I don't think they did. I think most people
| were amazed by all these technologies.
| wpietri wrote:
| Both reactions always happen. With basically anything new,
| people will select some points via happenstance or bias,
| draw one of a few basic trend lines [1], and give a hot
| take. Because they generally think only about first-order
| effects and don't imagine other things that could happen,
| the hot takes are often of the utopia/dystopia variety.
|
| These hot takes generally tell you more about the opiner
| (or the audience they're playing to) than the reality to
| come. It turns out it's hard to model en entire universe
| using 3 pounds of meat.
|
| [1] Heinlein listed some of them way back in 1952: https://
| archive.org/details/galaxymagazine-1952-02/page/n19/...
| derac wrote:
| https://daily.jstor.org/when-photography-was-not-art/
|
| Research beats idle speculation.
|
| Google "photography skeptics early history"
| p1esk wrote:
| Most people haven't heard about recent advancements in
| image generation. When they do, I expect they will be
| amazed.
| logisticpeach wrote:
| True, but then we've essentialy had limitless image
| generation capabilities since we've had the tools to make
| marks. I guess this is faster, and in other ways it
| offers promising new opportinities for people who can't /
| don't want to learn to create stuff directly.
|
| Others are interpreting my original comment as "this is
| not art", but I'm not really trying to make that
| argument. Art is entirely subjective and i don't presume
| to define what is or isn't art.
|
| I guess my point is more specifically "what itch does
| this scratch"?
|
| It's really cool, and that may well be the answer tbh.
| pmoriarty wrote:
| _" People said this about cameras. About digital cameras.
| About digital photo editing software."_
|
| Also about desktop publishing.
|
| Remember all the printers (ie. people working in the printing
| industry operating printing machines) that were put out of a
| job when you could just buy a (electronic) printer for your
| home computer and just print whatever you wanted yourself?
|
| People were wringing their hands about that too back then...
| now we take it for granted that we can instantly print
| whatever we want whenever we want, without having to pay an
| expensive professional to do it for us (something most people
| couldn't afford).
|
| Has it resulted in more junk being printed? Absolutely. But
| it also let people print all sorts of fantastic not to
| mention useful things that would almost never have seen the
| light of day without cheap and easy access to home printers.
|
| The xerox copier was similarly revolutionary... as was the
| printing press itself, which put a lot of scribes out of
| business.
|
| Photoshop put a lot of airbrush artists out of business, and
| who does copy and paste with physical glue and paper anymore?
|
| As with photography, printers, copiers and photoshop, artists
| who embrace this technology will be able to use it to enhance
| their creativity and speed up their creative process.
|
| There'll be a lot more competition, a lot more junk but also
| a lot more fantastic art that we can't even dream of yet.
| etrautmann wrote:
| Yep, I see this as a start, and very curious to see the ways
| in which it'll get used with a human in the loop, and also
| the ways human artists will be pushed to creat art that's out
| of distribution for these models.
| pizza wrote:
| > reminds me of what we could have achieved if this ingenuity
| had been applied in another domain
|
| It will; I think the reason we're seeing diffusion models
| applied to image generation first, is that it's a task that
| meshes well with the models. But also in general I think people
| will still be guided by the principle "use the right tool for
| the job" - this is just another tool. I doubt that the set of
| paths toward realization for any given needed creative imagery
| collapses to just "use a model"
| _zoltan_ wrote:
| what other domain would you suggest which requires roughly the
| same skill set?
| Wistar wrote:
| Music composition?
| whywhywhywhy wrote:
| UI should allow you to choose from the images and get the artist
| name.
| boplicity wrote:
| The concept of "derivative work" is pretty important in copyright
| law. I wonder if anyone has thoughts on this, in terms of this
| type of project. Should there be legal implications to this?
|
| I know someone -- a completely unknown artist -- who used to make
| a fair portion of their living by drawing D&D characters for
| people. Unfortunately, orders slowed down, because someone can
| input one of his images into software like this, and generate
| endless variations in the same style. Should this be allowed?
|
| Are images created "in the style of" a certain artist completely
| dependent on images created by that artist? If so, should that
| artist be compensated? Why or why not?
| pmoriarty wrote:
| _" Should there be legal implications to this? ... Should this
| be allowed?"_
|
| Even if there are laws against it, the cat's out of the bag.
|
| There's no stopping billions of people all over the world
| making derivative works at the push of a button.
| amelius wrote:
| > the cat's out of the bag.
|
| Yeah, just wait until Disney characters get copied and mixed.
|
| I wouldn't be surprised if their lawyers are preparing to
| change copyright law ... again ...
| robbles wrote:
| Interesting that it seems to have no concept of the filmmakers
| they tried to include here - Tim Burton and Walt Disney didn't
| produce anything recognizable and look to me like the default
| stuff you get without providing a style.
| yieldcrv wrote:
| We need to solve more captchas
| sexangel wrote:
| egypturnash wrote:
| How a Stable Diffusion prompt makes working artists sad and
| grumpy about huge abuses of "fair use" in 1500 ways.
| llanowarelves wrote:
| Fix the list: Hunter Biden is missing
| dr_dshiv wrote:
| I sure would love to see this systematically improved.
|
| Angel Adams, for instance, clearly wasn't too present in the
| dataset.
| ttoinou wrote:
| How come the faces features and body parts are almost always at
| the same place on the image ?
| Anunayj wrote:
| I suspect they're used the same seed for every iteration, which
| is great cause it shows the same context in different styles.
| tener wrote:
| Likely generated from the same random seed.
| nopenopenopeno wrote:
| David Hockney was a big disappointment.
|
| Edward Hopper was pretty impressive.
| alexmolas wrote:
| Nice experiment! I would only change two things: (1) sort the
| artists in alphabetical order, and (2) allow users to write the
| name of the artist and show if it's in the list.
|
| I'm saying that because it's a little bit tedious to search for
| the artist you're looking for.
|
| A part from that, I find the idea super-interesting :)
| MattPalmer1086 wrote:
| They are in alphabetical order, but by last name. Took me a
| while to see that though.
| layer8 wrote:
| Only up to a point, it consists of at least three
| concatenated separately sorted lists (the first one being the
| largest by far).
| ghusbands wrote:
| Though with names like "de Hooch" under "d" and names like
| "van Gogh" under "v".
| sva_ wrote:
| You can also just focus the select element and enter the full
| name to make it jump to the right name. But a real input would
| be better.
| LegitShady wrote:
| Honestly I don't think its very good at emulating style. It picks
| up some things but often times misses the heart of the matter.
|
| Did some spot checking with some of my favorite artists.
| Rockwell's paintings are all about storytelling, clearly not
| present in the work. Their emulation for frazetta doesn't look
| like frazetta's work at all. HR Giger emulation is a joke. David
| finch at least gets a penciler's style but misses the use of
| solid blacks and dynamic posing. Frank Miller doesn't look like
| miller's work at all. etc etc etc
|
| This list goes on and on. Personally while I understand using the
| 'in the style of' as a way to change the image results, I think
| in many many cases the results just don't look like the art of
| that artist.
| wcedmisten wrote:
| Very cool! I recently made a game kind of like AI "pictionary"
| where the user has to guess the "artist", subject, and
| description of a piece of art generated by stable diffusion:
|
| https://wcedmisten.fyi/project/paintingGuesser/
|
| I tried to make something more general, but stable diffusion is
| fairly inconsistent in how well the output matches the semantics
| of the input.
| shawnz wrote:
| This could be a good candidate for a wordle-style scoring
| system
| MarcoZavala wrote:
| ForRealsies wrote:
| matthewfcarlson wrote:
| This is really awesome
| [deleted]
| seydor wrote:
| I wonder who is analyzing the weights of the model. At what level
| is the dimension of "artist name" represented, what is above it
| and what is below ?
| hecanjog wrote:
| Funny that the Bob Ross version just makes them look like Bob
| Ross. Maybe there are more pictures of Bob Ross in the training
| set than his actual paintings.
| chris_overseas wrote:
| The same thing happens for Vincent van Gogh, and presumably
| others too.
| pessimizer wrote:
| A good measure for whether you're more of a celebrity or an
| artist is how much of your face a google-image trained AI
| thinks belongs in your work.
| pmoriarty wrote:
| _> A good measure for whether you 're more of a celebrity
| or an artist is how much of your face a google-image
| trained AI thinks belongs in your work._
|
| That this happens at all is evidence that the training data
| hasn't been curated, cleaned, or labeled well enough.
| malikNF wrote:
| Well what if the artist has a popular self portrait?
| pessimizer wrote:
| Smoke comes out of the computer, and Captain Kirk notches
| another victory for humanity.
| kjeetgill wrote:
| Frida Kahlo comes to mind.
| pmoriarty wrote:
| She's a rare exception in that she's mostly known for her
| self-portraits. Most other famous artists are mostly
| known for other things.
|
| Again, if the training data was labeled well enough,
| confusion about this sort of thing shouldn't happen.
| joosters wrote:
| Not a problem with the Banksy images!
| SergeAx wrote:
| Bosch is marvelous. Mucha and Monet are good. Michelangelo tries
| to be more like his sculptures. Not sure about Walt Disney and
| Roy Lichtenstein, and Rembrandt version is especially ugly. Maybe
| it will worth to get rid of "van Rijn" in that sample.
|
| Overall: the impression is better with author's popularity. I
| think if we train the model only with well-tagged filtered
| dataset - results may be much better, but we effectively will get
| a 5-year old Prizm app.
| smilespray wrote:
| Features John Wayne Gacy...
| jhbadger wrote:
| What frustrates me about Stable Diffusion is there doesn't seem
| to be any documentation as to what artists or vocabulary it
| understands. Generally people say "look at existing prompts or
| use various prompt generators" but that doesn't really solve the
| problem. I don't want to just look at what other people have
| randomly discovered; I want to know what the program really
| knows.
| novok wrote:
| You can by searching the training image set and their text tags
| essentially.
|
| https://laion-aesthetic.datasette.io/laion-aesthetic-6pls/im...
| dopidopHN wrote:
| Honest question, would a solid understanding of the open
| training data help?
|
| Having the art vocabulary down as well.
|
| In effect, knowing what is present and how it's tagged so you
| can << invoke >> it more readily in the prompt-result.
|
| Maybe I'm out of my depth. I know the corpus of tagged image
| used for training is enormous ... but I still think that would
| help the user ( a prompt-crafter )
| [deleted]
| pwinnski wrote:
| It's the nature of ML models--nobody is 100% sure what it
| understands until they try something and get results.
|
| It was given a _lot_ of tagged data: 600 million captioned
| images from LAION-5B. So if you want to know what it _might_
| support, you could try any one of the captions from those 600
| million images.
| jhbadger wrote:
| But why isn't the list of words from those captions available
| anywhere (at least as far as I can tell)? There may be 600
| million captions, but the number of unique words would
| probably be 10 or 20 thousand at most, completely feasible to
| browse or grep.
| spijdar wrote:
| I don't think the underlying model is word based, but
| character based. You could download the caption data for
| LAION and grep that, but it's not strictly 1:1 with what SD
| was trained against.
| thedorkknight wrote:
| I haven't downloaded the database myself, but I imagine if
| you did it wouldn't be too hard to get that data. Looks
| like you can get the torrent here
| https://laion.ai/blog/laion-400-open-dataset/
| bawolff wrote:
| Its kind of a weird complaint. If i am having a
| conversation with someone, i wouldn't be concerned about
| knowing the set of all possible nouns.
| spijdar wrote:
| The simple answer is that there is no clean cut list of artists
| that it "understands". The model has no explicitly programmed
| concept of artist or style -- just the CLIP based text encoding
| used to train the conditional autoencoding part of the denoiser
| network, trained on (AFAIK) caption data recorded with the
| image.
|
| So in practice asking for art "in the style of <x>" is sort of
| limiting the denoiser to statistical pathways resembling other
| images captioned "in the style of <x>". At least, that's my
| understanding. Still trying to grok ML and diffusion models.
| astrange wrote:
| You can create (or discover) explicit vocabulary in the model
| using "textual inversion", or train more into it using fine
| tuning.
| smoldesu wrote:
| You can't really debug an AI. The dimensions of it's
| understanding are quite literally beyond human interpretation,
| which makes it both smarter/more efficient than humans while
| also extremely dumb and context-unaware. Most of our attempts
| at adding a 'memory' to AI has been a hack thus far, which is
| why all of these prompts consist of people force-feeding word
| salad down the AI's throat for generally reproducible results.
| the8472 wrote:
| They're debuggable, with effort. Here's the finance neuron in
| CLIP: https://microscope.openai.com/models/contrastive_4x/ima
| ge_bl...
| sdenton4 wrote:
| For memory, check out:
|
| 1) Retro, which is essentially attention over large
| databases, and fast as hell.
|
| 2) S4 Layers, explicitly designed for handling long
| dependencies.
|
| These are orthogonal approaches to memory, and both very
| effective at what they do.
| otabdeveloper4 wrote:
| "It" doesn't understand anything.
|
| It's just a very advanced madlibs engine based on a database of
| a billion of already known images.
| bawolff wrote:
| Some people would probably argue that humans are essentially
| the same.
| krapp wrote:
| Whatever else this emergent "creative AI" phenomenon may or
| may not do, it's definitely touching nerves in people who
| still believe there's something ineffable and transcendent
| about the creative experience.
| kvetching wrote:
| We are going to need a new patent system.
| jenny91 wrote:
| Patents have nothing to do with protecting art: they are
| supposed to protect inventions.
| [deleted]
| vtuulos wrote:
| If you want to experiment with something similar by yourself and
| you don't have the patience to wait for Stable Diffusion to
| crunch through thousands of images on your laptop or in a Colab
| notebook, here's how you can parallelize processing relatively
| easily on AWS Batch or Kubernetes:
| https://outerbounds.com/blog/parallelizing-stable-diffusion-...
___________________________________________________________________
(page generated 2022-10-02 23:00 UTC)