[HN Gopher] Show HN: I made a new AI colorizer
___________________________________________________________________
Show HN: I made a new AI colorizer
Hi HN, I'm Emil, the maker behind Palette. I've been tinkering with
AI and colorization for about five years. This is my latest
colorization model. It's a text-based AI colorizer, so you can edit
the colorizations with natural language. To make it easier to use,
I also automatically create captions and generate filters. Let me
know what you think. You can see some of my results on my reddit
page: https://www.reddit.com/user/emilwallner/?sort=top
Author : emilwallner
Score : 262 points
Date : 2022-10-19 13:00 UTC (9 hours ago)
(HTM) web link (palette.fm)
(TXT) w3m dump (palette.fm)
| nextaccountic wrote:
| Can I paint some colors as a starting point (like img2img) and
| let it finish the rest?
|
| What about a conversation, like, "the dress isn't blue, it should
| be orange", doing on top of previous prompt?
| emilwallner wrote:
| not yet, these are great suggestions. it's always a dilemma to
| add features to mitigate the performance of a weak model,
| instead of making a better model. most of the problems go away
| with a better language and colorization model, and many model-
| specific features are made in vain
| nextaccountic wrote:
| I think there are two uses for an AI colorizer. One is to
| generate a color image that looks great, another is to
| generate an image that accurately reflects the true color of
| things.
|
| A better AI model helps a lot with the first goal, but help
| only so far with the second one. Truth to be told, there is a
| lot of contextual color information in black and white photos
| that an AI model can exploit; but nothing beats someone that
| knows, for sure, the color of the dress of someone in the
| photo.
|
| I mean, take a look at
| https://www.reddit.com/r/ColorizedHistory/ - some of those
| color artists do a lot of research to know the exact shade of
| green of the military uniform of some country in the 19th
| century, and things like that, just to have an accurate
| reference.
|
| So I think that the ability of directing the color output
| (either by rejecting a color textually, or by painting over
| the figure with a starting point - even if maybe I'm not
| painting with the exact tone or texture but a rough color
| that should help the AI to figure out the details) is
| essential for a colorization product, even if the model is
| flawless!
| nextaccountic wrote:
| Actually, thinking of /r/colorizedhistory, here's a
| comparison of a professional colorization and palette.fm ht
| tps://www.reddit.com/r/ColorizedHistory/comments/y5mfqu/pa.
| .. vs https://www.reddit.com/r/ColorizedHistory/comments/y5
| mfqu/pa...
|
| My concern here isn't that the professional photo has
| higher quality (in this case it has, but give it some time
| - months or years - and maybe technology will catch up).
| It's that sometimes we already know the right color, while
| the AI must always guess
| emilwallner wrote:
| true, thanks for adding context!
| poniko wrote:
| Any chance you would license the model for us to use with
| pixlr.com?
| emilwallner wrote:
| i'm planning on doing some sort of API, feel free to ping me at
| emil@palette[dot]fm with the constraints you have
| Wistar wrote:
| This is quite amazing, and useful. Great work.
| krisdol wrote:
| I had some black and white photos of my passed-away tuxedo cat,
| fully expecting this to produce a terrible result, but instead I
| was blown away by accurate and subtle colors added. Pinkish nose,
| ears, green eyes all came out of the photo.
| djleni wrote:
| Heh, a lot of the results are more visually appealing than the
| real landscape I shot in B&W.
| jetrink wrote:
| I've tried a few of my own black and white photos and the
| description of each starts with "a stock photo of a..." Is this a
| comment on my style of photography or is this always added for
| the base palette prompt? (The colorized images have been very
| impressive.)
| waffletower wrote:
| This is a really impressive colorizer. Don't feel guilty
| monetizing.
| meerita wrote:
| This blew my mind. Amazing.
| jbellis wrote:
| I was actually testing Photoshop's new colorizing filter this
| week, so this is timely. My first impression of Palette in
| comparison is that it does a comparably good job of colorizing.
| Sometimes Photoshop does better, sometimes Palette -- usually the
| skin tones is the hardest part, and one does a noticeably better
| job than the other.
|
| The natural language modifications available in Palette ("his
| shirt is light blue") are super useful. Well done.
| dirtyid wrote:
| Incredible service. Just annoyed once I show it to my folks I'll
| be dragged into multi hour tutorial session on how to scan /
| upload old albums. Also curious which "filter" they feel will
| most reflect the subjective reality of the past.
| writeslowly wrote:
| I tried it on the photo linked below and it makes the string
| instruments look like they're made of brass. Not trying to bash
| it since I thought it was pretty impressive overall, but I'm
| curious about what leads to this type of failure.
|
| https://commons.wikimedia.org/wiki/File:Jazzing_orchestra_19...
| emilwallner wrote:
| thanks for the feedback! it's made out of two models, one model
| creates a caption and the second model takes the caption and
| the black and white image and colorizes it. if you click on the
| edit button you can see the text that generated that
| colorization. if the text is incorrect, you can edit the text
| and recolorize it. this often leads to a better result,
| however, some cases are still hard, especially damaged photos.
| cercatrova wrote:
| I assume you're using CLIP or BLIP for the text generation of
| the model, and then img2img or something like that for the
| colorization. What model are you using for the latter
| colorization?
| learndeeply wrote:
| > Vivid Natural -- A slide of jam, jazzman, hook, band, pipe,
| and coil. Small contrasting details in natural colors. vivid
| natural.
|
| This one generates the string instruments with the correct
| colors.
| logifail wrote:
| (Genuine) question, and no snark intended: why do so many people
| want to change history?
|
| Black and white photos are - for many of us - part of our shared
| historical record. Is there really a need to improve (=change)
| them? Can't we appreciate them exactly as they are, without
| modifications?
| croes wrote:
| Some use picture to imagine how life was back then. Life wasn't
| black and white.
|
| Same with the facial expressions in old photos. People look
| pretty serious but that's just because of the technology pf
| photography back then. People were as silly and joyful as
| nowadays.
| logifail wrote:
| > Some use picture to imagine how life was back then. Life
| wasn't black and white.
|
| Life (time) doesn't tend to stand still, either, yet we are
| able to appreciate photographs.
|
| Would we really benefit from old photographs being AI-
| animated into "videos"?
|
| Perhaps I'm showing my age, but the older I get the more I
| feel at one with life's imperfections. I'm fine without
| filters and HDR ... or colour ... or motion.
| croes wrote:
| >Would we really benefit from old photographs being AI-
| animated into "videos"?
|
| Some people seem to think that
|
| https://news.ycombinator.com/item?id=26279372
| kevmo314 wrote:
| > Would we really benefit from old photographs being AI-
| animated into "videos"?
|
| Yeah that would be pretty cool.
| Kuinox wrote:
| Most people see the life in color and are not used to black and
| white images. Color give way more depth and understanding.
| roywiggins wrote:
| The impulse to colorize monochrome photos goes back almost as
| long as photography itself.
|
| https://en.wikipedia.org/wiki/Hand-colouring_of_photographs#...
|
| https://en.wikipedia.org/wiki/Hand-colouring_of_photographs#...
|
| It depends on the photo, obviously, but colorization can also
| give a new dimension of contrast to a photo, even if the colors
| aren't strictly accurate, by separating out the foreground from
| the background better.
| logifail wrote:
| > The impulse to colorize monochrome photos goes back almost
| as long as photography itself
|
| Impulse? A strange word to choose. Impulsive behaviour isn't
| what I'd aspire to, or want others to aspire to. Be
| thoughtful.
| wilg wrote:
| Who are you scolding and why?
| roywiggins wrote:
| I was using it in the sense of definition 1b, _a propensity
| or natural tendency usually other than rational_ :
| https://www.merriam-webster.com/dictionary/impulse
|
| People have liked colorizing photographs for a long time,
| it's a natural tendency even though it may or may not be
| rational.
| fortysixdegrees wrote:
| Nothing is being changed. After using the tool you still have
| the original, plus a new version which can be used alongside
| default-user wrote:
| I've never posted here before but this demands a sincere "thank
| you".
|
| I just colorised some family photos from the 1930s to the 80s and
| the results are heartwarming.
|
| Thank you for making this and making it free and, most
| importantly, easy to use.
| aix1 wrote:
| Quite an incredible compilation of photos on your reddit page,
| thanks for sharing.
| emilwallner wrote:
| thanks, that means a lot!
| akouris wrote:
| Amazing work congratulations!
| draugadrotten wrote:
| I laughed out loud reading your ABOUT page. Thank you!
| knarf180 wrote:
| Wow.
| nakedgremlin wrote:
| Thank you for providing pre-set examples on your site. That's one
| of the major barriers on these types of tools to "test"
| especially when you're not sure where/how/if the images are
| public.
| shubhamjain wrote:
| Awesome job! This makes me a bit envious. I am hobbyist colorizer
| [1] who did it the old school way (with tools like Affinity Photo
| and lot of manual work). I tried de-oldify and other tools but I
| justified my work thinking how horrible they were but pallette.fm
| is way too good. Not sure if I would find the motivation to
| restore old photos anymore. Glad that this was just a small hobby
| for me and I had just started learning the ropes.
|
| But I would be dead scared if I was a professional[2] who did
| this full-time. Is this what AI taking your job feels like?
|
| [1]: https://shubhamjain.co/experiments/
|
| [2]: https://www.youtube.com/watch?v=vubuBrcAwtY
| emilwallner wrote:
| ty! i'm glad you enjoy the tool. As for whether or not this
| tool could replace a professional colorist, I think it depends
| on the specific project. For some projects, Palette could do a
| great job of automatically colorizing photos, and for others, a
| professional colorist would still be necessary to get the best
| results. Especially when the projects require historical
| accuracy or a high aesthetic standard. it also makes
| colorization more accessible, which leads to more opportunities
| to refine results manually or say print the results.
| Valgrim wrote:
| Even if the results are not 100% historically perfect, 99% of
| the hard job is done. Skin tones, plants, skies, water...
| they all look incredibly good, and the object segmentation is
| almost perfect too.
|
| The tool seems to struggle with fabrics, but that part is by
| far the easiest to fix with a traditional photo editor.
|
| Congrats man. You made my mom happy this evening. Please keep
| a free tier on your tool.
| FL33TW00D wrote:
| Excellent work - enjoyed the rick roll. Would you mind sharing
| how you've done the GPU deployment?
| emilwallner wrote:
| lol, thanks! onnx, docker, and fastAPI on CPUs with AWS
| fargate. although i'm switching to GPUs in a month or two, so
| if you have any suggestions let me know.
| punkspider wrote:
| I've also been searching for GPU solutions for potential small
| projects like this, and so far banana.dev and runpod.io seem
| promising.
| wigster wrote:
| very cool. just coloured some old family tree pics and they look
| great. thanks.
| MitPitt wrote:
| This is great. I'd also love to have the option to manually draw
| guiding colors over parts of the image, for even more control.
| smortaz wrote:
| tested on a few pics. excellent colorization. pls provide patreon
| etc link so the site stays up! thxxx
| kurtreed wrote:
| I haven't tried any other colorizer tools but this one works very
| well for me. Thank you.
| theden wrote:
| Wow I put a few b+w photos, initially shot in colour so I can
| compare how it did, and it more or less nailed it--great work.
| They were shot on a iphone, so I wonder if newer digitally-shot
| photos have more data for the app to parse, or if it's generally
| easier compared to film (and whether it uses any EXIF data)
| poulpy123 wrote:
| I expect the training to be done mostly on modern digital
| cameras
| learndeeply wrote:
| Can you summarize the approach you took on creating the model?
| Really curious!
| w_for_wumbo wrote:
| This is absolutely mind-blowing to me, is this using an API of
| any sort? I'd love to be able to take something like my City's
| archive of Black & White images and colorize them.
| waltbosz wrote:
| Very cool.
|
| Your results look much better than the washed out AI colorization
| that I've seen in the past.
|
| I think you could charge money for this service.
|
| A suggestion for something fun to do / marketing tool: recolorize
| this video frame-by-frame
| https://www.youtube.com/watch?v=hZ1OgQL9_Cw
|
| Feature request: colorize B&W comic books. I really want to
| create a full color book of Calvin & Hobbes comics. (Not for
| publication)
| emilwallner wrote:
| Thanks! yeah, a few people have used the tool to colorize
| videos, frame by frame. For example Lord of the flies (1963):
| https://www.dailymotion.com/video/x8eiho4
|
| Although, I'd recommend colorizing a few key frames and then
| use https://github.com/zhangmozhe/Deep-Exemplar-based-Video-
| Colo...
|
| Cool, yeah, my next model will be better for comic books. You
| can also use the 'Surprise Me' button in the editor and you'll
| get some decent results.
| zimpenfish wrote:
| Tried a few old family photos and it's done a good job - mostly
| better than the other ones I've tried. Excellent work.
| callumprentice wrote:
| Thank you so much Emil - I have some old photos B/W of my dad who
| is no longer with us and have wanted to colorize them somehow for
| years. The results from just dropping the photos on your page -
| no tweaking or whatnot - are incredible and have me in tears.
|
| Amazing work.
| kmnc wrote:
| This works amazingly well, just putting in some of my favorite
| black and white scenes from movies... wow! Now I need some kind
| of tool that can do this on every frame of a black and white
| movie. I cannot wait to be able to watch some old classics fully
| colorized.
| BizarroLand wrote:
| Just a quick heads up, when I ran the file and then opened the
| result in irfanview, I get a warning that the file is a JPG file
| with an incorrect extension.
|
| You may want to check your encoding settings to make sure
| everything gels together.
|
| Otherwise, great job! This is pretty nice stuff and way better
| than I could do on my own!
| 3xa wrote:
| Amazing. Colorized a rare picture of my grandfather. Thanks!!!
| JoeAltmaier wrote:
| I love the way it finds features and labels the picture. It
| thinks I am a meteorologist!
| littlestymaar wrote:
| Unfortunately, this has the common bias[1] with IA coloring of
| making old stuff look dull, reinforcing the Hollywood cliche[2].
| The past was actually often very colorful.
|
| [1]:
| https://nitter.lacontrevoie.fr/gwenckatz/status/138165207169...
|
| [2]: see this video by a movie props maker about why Hollywood
| movies make old things look the way they do in movie
| https://youtu.be/mF1VFlCnLQ4?t=434
| napier wrote:
| This is a top tier side project.
| hilyen wrote:
| I ran a test with a photo I took if people are curious. It does a
| great job for guessing, but of course there is room for
| improvement.
|
| https://i.postimg.cc/Y9dH4GxW/palette-fm-test.jpg
| zxcvbn4038 wrote:
| Wow! I am impressed. I uploaded a WWII photo of airmen posing in
| front of their plane and got an amazing result. I'm not certain
| if WWII era life vests were bright yellow, that seems out of
| place, but it's a good guess if not.
| karamanolev wrote:
| Found some that are yellow, not sure how bright we're talking -
| https://www.ima-usa.com/products/original-u-s-wwii-usaaf-mae...
| shisisms wrote:
| This is incredibly well done! Congrats. It's such a solid layer
| on top of recent developments and offers instant value.
|
| Wish you much success!!
| [deleted]
| 3pt14159 wrote:
| I thought from the title it was some JavaScript to auto-colorize
| essentially any programming language and was excited, because
| that stuff is annoying to implement for any new programming
| language out there.
|
| Still cool though.
| speedgoose wrote:
| How does it work? Do you write scientific papers?
| emilwallner wrote:
| I haven't made a write-up of this yet, I still need to figure
| out how to self-fund it. I normally do more layman scientific
| writing like this: https://artsandculture.google.com/story/the-
| klimt-color-enig...
| fritztastic wrote:
| Amazing work! Thanks for sharing!
| brontosaurusrex wrote:
| All I can say is: very very nice (Testing on some 3d renders and
| some bw photos). p.s. Is output image width limited to 1920px?
| emilwallner wrote:
| ty!! yes, 1920x1920. i'm working on an unlimited resolution
| option, but it will take a few weeks.
| holoduke wrote:
| It's funny to let it operate on a pile of random Lego blocks of
| different colours. Funny how it sometimes uses a completely
| different colour. In some other cases it guesses it completely
| right. Sometimes you also see a from one color to another color
| gradient, while the real thing has one color with a shadow
| gradient. How large is your trainingsset?
| vanillax wrote:
| Open Source? Would love to see how this works and also the
| website.
| cercatrova wrote:
| Nice, maybe I can turn the movie The Lighthouse from black and
| white into a color motion picture. I hear color "movies" as they
| call 'em are what's hot these days.
|
| Original: https://i.imgur.com/9CZ7vk1.png
|
| Base Palette: https://i.imgur.com/iTq5H9W.jpeg
| bwoodward wrote:
| It looks pretty great, actually --
| https://i.imgur.com/fxZz6f7.jpg (Base),
| https://i.imgur.com/2JAHnbe.jpg (Vivid Natural)
| cercatrova wrote:
| I tried it as well (edited my comment above), I like the Base
| Palette option better, it looks more realistic compared to
| Vivid Natural which looks like a WW2 movie filter with the
| brown atmosphere.
| surfsvammel wrote:
| This is magic... How the heck does it know that the walls of my
| home gym is light green?!?
|
| Impressive!
| tanvach wrote:
| Nice UI, and the approach works quite well (occasional wrong
| choice of palette compared to ground truth). It's kind of fun to
| turn my photos into B/W and run through the model to try and
| guess the output.
|
| Do you have a privacy policy for the uploaded photos? I'm not
| keen on uploading anything important without knowing how it's
| stored or will be used in the future.
| emilwallner wrote:
| cheers! i'm working on a proper privacy policy. I don't store
| any images that are uploaded. i use google analytics and
| mixpanel to store user interactions.
| atum47 wrote:
| that's incredible. congratulations on the great work and thanks
| for sharing with us.
| WrtCdEvrydy wrote:
| Wow, any way to run this locally?
|
| I love this technology but it would feel kinda terrible to upload
| a whole bunch of stuff to your site and exploit your generosity.
| emilwallner wrote:
| not for now. i might make a local version in the future, but
| for now, enjoy!
| DerWOK wrote:
| Thanks, this is awesome. Colorized a few last images of my WWII
| grandfather who died in this war. One observation: on download
| the MIME type seems to by PNG format, but the binary file
| arriving is actually a (lossy compressed) JPEG. But anyhow! Great
| work!!
___________________________________________________________________
(page generated 2022-10-19 23:00 UTC)