[HN Gopher] Modern smartphone lenses are crazy
___________________________________________________________________
Modern smartphone lenses are crazy
Author : luu
Score : 181 points
Date : 2022-03-04 17:22 UTC (1 days ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| amelius wrote:
| The comment of Daniel Darabos points to an interesting resource,
| explaining how these lenses are designed:
|
| https://www.pencilofrays.com/lens-design-forms/#mobile
| roughly wrote:
| Boy, that page is Comprehensive on lens design - that's an
| awesome resource.
| IshKebab wrote:
| Interesting. He doesn't really explain how they're designed
| though. Must be just automatic global optimisation at this
| point though. No way a human manually optimised all the ray
| paths.
| fsh wrote:
| Usually some form of gradient descent is combined with a
| global optimization routine that tries to explore the
| parameter space. This is nothing new, optics has been
| designed this way since the early 60s.
| GoToRO wrote:
| And yet you can't take a picture of the stars or street lights on
| an iphone...
| mmaunder wrote:
| Physics is still the law. As long as your aperture is small, so
| is the usable amount of light you're collecting at the top of
| that lens funnel.
| GoToRO wrote:
| The problem is that the tiny light from a star or from a
| street lamp post is multiplied (in software) multiple times
| to a comic effect...
| mattlondon wrote:
| Is this just a software problem? Android has had various night
| sight modes for several years that make dark views work pretty
| well?
| GoToRO wrote:
| There is something with the hardware as well, there is some
| green glare even on easy shots, but the fake lights problem I
| think is software, the software tries to "pretty" the image
| and introduces these fake lights.
| TooKool4This wrote:
| Well thats definitely not true
|
| Stars: https://www.macrumors.com/2021/10/10/amazing-night-sky-
| photo...
|
| Street light: https://cdn.vox-
| cdn.com/thumbor/R51UlZi7g1UPq7b9Be05nWfOP_o=...
| GoToRO wrote:
| https://twitter.com/zoneoftech/status/1236746998805680128
| GoToRO wrote:
| C'mon. From your link: "The images were shot using Apple's
| ProRAW format and then edited using the mobile version of
| Lightroom on the iPhone itself"
|
| I have the 12 and it multiplies any tiny source of light in
| dozens of fake lights.
|
| The street light I;m referring too is much darker than this:
| a dark street with a string of lights. All the lights will
| get multiplied as fake lights.
| CharlesW wrote:
| Of course you can.
|
| Stars example: https://www.macrumors.com/2021/10/10/amazing-
| night-sky-photo...
|
| Street light example:
| https://www.cnet.com/tech/mobile/iphone-13-pro-vs-12-pro-max...
| GoToRO wrote:
| C'mon. From your link: "The images were shot using Apple's
| ProRAW format and then edited using the mobile version of
| Lightroom on the iPhone itself"
|
| I have the 12 and it multiplies any tiny source of light in
| dozens of fake lights.
|
| The street light I;m referring too is much darker than this:
| a dark street with a string of lights. All the lights will
| get multiplied as fake lights.
| kazinator wrote:
| It looks way less crazy when you "focus" on just the payload
| areas of the optic pipeline, trimming out the "lunettic fringes".
|
| Here is what I mean:
|
| https://i.imgur.com/ktbWf0X.png
| H8crilA wrote:
| Why are the lenses so complex outside of the area that matters?
| Structural integrity / resistance to dropping?
| fsh wrote:
| The picture in the tweet only shows rays originating from an
| object that is close to the center of the field of view. The
| article mentioned in another comment has better schematics:
| https://www.pencilofrays.com/lens-design-forms/#mobile
|
| Smartphone cameras usually have a pretty wide field of view.
| The complex shapes are required in order to compensate for
| the aberrations caused by the large angles between the
| optical rays and the lens surfaces.
| mrfusion wrote:
| Perhaps this technology could be useful for breakthrough
| starshot: https://en.m.wikipedia.org/wiki/Breakthrough_Starshot
| agluszak wrote:
| https://nitter.net/yiningkarlli/status/1498069538264399872
| tgsovlerkhgsel wrote:
| Thank you!
| ykl wrote:
| Hello! Original author (original tweeter?) here.
|
| This particular design is from the iPhone 7 (or, more precisely,
| my guess is that it's from the iPhone 7 due to both the date of
| the patent [1] and due to the elements matching up with marketing
| images of the iPhone 7), which is 6 years old now, but I think
| it's broadly still representative of modern smartphone lenses. In
| the past 5 years or so, advancements in phone cameras have come
| mostly in better sensors, far better image processing, and adding
| more cameras, but the basic principles of the compact ultra-
| aspherical lens design seem to still be in place.
|
| As an example, here [2] is an exploded view of the iPhone 6's
| lens setup, and here [3] is an exploded view of the more recent
| iPhone 12's lens setup. The iPhone 12 gained an extra element,
| but they both use similarly weird ripply elements and you can see
| the clear lineage between the two phones.
|
| Also, as mentioned in the tweet thread and elsewhere in the
| comments here, Kats Ikeda has an excellent, incredibly detailed
| and thorough explainer on mobile phone lens design [4]. I don't
| actually know a whole lot about the optics field; I'm just a
| dabbling as a hobbyist from a computer graphics perspective. Kats
| Ikeda's site is a much better resource than my Twitter posts for
| learning about this stuff.
|
| [1]
| https://patentimages.storage.googleapis.com/7e/4e/3f/4e88d65...
| [2]
| https://pbs.twimg.com/media/FMyGm6IVkAY77eY?format=jpg&name=...
| [3]
| https://pbs.twimg.com/media/FMyIbszVQAQhMQ_?format=jpg&name=...
| [4] https://www.pencilofrays.com/lens-design-forms/#mobile
| doomlaser wrote:
| Sony makes the lenses sensors on modern iPhones and, if rumors
| are to be believed, also the micro OLED displays on a forthcoming
| new product category.
|
| It's funny. The 1970s Sony Trinitron CRT TVs are what inspired
| Steve Jobs to make the Apple II case, and here they are working
| together, at the highest end of technology, many decades later.
| systemBuilder wrote:
| Sony also made the AppleColor 640x480 monitor in 1987 with the
| release of the Mac II. This was quite likely the most fantastic
| monitor available in this size in this year. It was rumored to
| be a regular Sony Computer monitor without an anti-reflective
| coating so it could be brighter and clearer.
| alphabettsy wrote:
| I think Sony make the sensors only.
| [deleted]
| olliej wrote:
| Sony only make the sensor, it looks like a bunch of companies
| manufacture the lenses (sharp, lg infotech, and another company
| I forgot from my googling 10 seconds ago).
|
| But as the linked thread says, apple _designed_ the lens itself
| (as it has a patent on it, which is apparently common?)
|
| I am curious what the Samsung lens system looks like given it
| has real optical zoom
| turing_complete wrote:
| "You think anyone would be doing this kind of utterly insane
| stuff if it wasn't absolutely necessary?"
|
| - Not a software developer. Obviously.
| olliej wrote:
| The problem is an artifact of size constraints and plastic lens
| having far less variation in optical density, that means a lot
| more work to correctly direct light to avoid various types of
| distortion.
|
| https://www.pencilofrays.com/lens-design-forms/#mobile has a
| much more in depth discussion on the how and why if the lens
| system design (someone referenced this in a reply to the linked
| thread)
| [deleted]
| baybal2 wrote:
| Plastic lenses are actually more expensive to make than glass.
| You need extreme precision, more than with lapped glass lenses.
|
| A Taiwanese factory northwest of Dhaka
| https://www.youngoptics.com/ makes the lion share of molds for
| plastic lenses worldwide.
|
| YoungOptics used to make them for Sony too. I don't know if it's
| still the case today.
| CliffStoll wrote:
| Long gone are my high-school days of grinding and polishing a
| 20cm mirror in the cellar, carefully using Foucault knife tests
| to parabolize it. As an undergrad, using Gaussian formulae when
| matching lenses in eyepieces. In grad school, writing ray-
| tracing codes to design multi-element lenses. Then, as a
| postdoc, using Zernike polynomials to estimate optical errors
| in the hexagonal mirror segments of the Keck 10 meter
| telescope.
|
| Today's iPhone optics astonish and impress me: A lens built
| with over a half-dozen aspherical elements. Coordinated imaging
| with multiple cameras. Wow!
| anfractuosity wrote:
| From reading some of the comments it sounds like the elements for
| these lenses are plastic?
|
| Is there a downside to using plastic lens elements compared to
| glass, in terms of image quality?
|
| If there isn't a downside, could lenses for SLRs/mirrorless
| cameras also use such elements?
|
| Edit: It sounds like aspheric lens elements might be made from
| plastic in SLR lenses. But also I found that plastic might not be
| used for outer lens elements as it scratches easily
| wyager wrote:
| Plastic typically has much worse reflection and losses than AR
| coated glass (at least for narrowband lenses - I'm
| extrapolating to photo lenses).
| post_break wrote:
| My S21 with "scumbag zoom" or creeper zoom. The amount of zoom,
| in something as thin as it is, is mind blowing. To zoom 10x, and
| be able to take great pictures, hand held with almost no shake.
| Then you have the top GoPros shooting 5k with gimbal like
| stabilization. Cameras are getting crazy.
| bufferoverflow wrote:
| This is from iPhone 7, released 6 years ago. I would like to see
| an actually modern lens design. Smartphone cameras got so much
| better in the last 6 years.
| etu wrote:
| Correction: It was released 2016, that's 6 years ago.
|
| The point still stands though. But 6 instead of 8 years.
| tadfisher wrote:
| And mine just got updated to iOS 15.
|
| Meanwhile my Pixel 3 XL is three years old and is no longer
| receiving updates.
| simonh wrote:
| I suspect that's mostly down to sensor design (big pixels,
| reflective layers behind the sensor so the light goes through
| twice, etc) and software.
| londons_explore wrote:
| I don't think there are any really big changes to the lens
| design recently... All the effort has gone into multi-camera
| setups, and different types of lens for macro, wide angle,
| telephoto, etc.
|
| To my understanding, there is still only a single set of moving
| lenses in an iPhone 13 lens.
| ghaff wrote:
| The iPhone 12 Pro is insanely better than the iPhone X--
| especially in low light. I have a bunch of fairly high-end
| camera equipment that I rarely use outside of some fairly
| specific circumstances at this point
| unfocussed_mike wrote:
| Yeah, likewise.
|
| I have high end kit (and some semi-pro lighting) which I love
| to use, but I now use a secondhand iPhone X, an Apexel
| teleconverter lens and a cheap Ulanzi grip, and I really
| enjoy this combination for its immediacy.
|
| The higher end kit is now increasingly used in novel, DIY
| ways, because carrying a phone has taught me that gear should
| be used for its strengths.
| ghaff wrote:
| Cameraphones have gotten to the point where I should
| probably consider getting some external lens gear. It
| didn't really make sense to me when there were so many
| other compromises with respect to image quality but
| offering some additional options when traveling with just a
| phone is probably legit useful at this point.
| BolexNOLA wrote:
| Filmic pro was a total game changer re: compression
| options and image control. Definitely when I started
| using the video a ton.
| unfocussed_mike wrote:
| The Apexel HD 2X lens has been useful for me. Cheap
| enough not to stress about, and just enough extra reach
| to be useful.
|
| I really enjoy the cheap little Ulanzi CapGrip thing,
| too.
|
| It's definitely not 100% like using a real camera (no
| half-press for focus, though I am a back-button-focus guy
| on DSLRs anyway!)
|
| But it adds just enough of a camera-like feel to allow
| relaxed one-handed grips and slightly more immediate
| shooting. And it weighs nothing so it's always in a bag
| or jacket pocket.
| ghaff wrote:
| Thanks. I'll probably wait until my next upgrade which
| will probably be a good couple of years. But I can
| absolutely see pulling the plug on upgrading my other
| cameras at that point. Which would make even going with a
| couple higher-end Moment lenses pretty thinkable.
|
| The always with you is a big thing.
| polskibus wrote:
| Would you mind sharing which Apexels do you recommend?
| unfocussed_mike wrote:
| I have just the one now. It's the Apexel HD 2X telephoto.
|
| I bought it because I wanted a slightly longer lens for
| portrait location scouting, and chose this one not
| because I am certain it is excellent but because I doubt
| there's enough of a quality gap between the Apexels and
| the Moment or Rhinoshield lenses to justify the much
| higher price.
|
| It is really fairly good though, until you get into the
| corners. But then I happily use vintage lenses on my A7II
| that are worse in the corners!
|
| What difference there is, is that the Moment lenses use a
| tiny bayonet, which I think I _would_ prefer to the 17mm
| thread Apexel, Ulanzi and others use. Screwing the lens
| on can be fiddly, especially with the semi-open, three-
| quarter circumference attachment threads you typically
| see on lens adapter phone cases.
|
| Don't use the clamp attachments. They are hard to align
| and I think that is a lot of why people find attachment
| lenses so disappointing.
|
| Get a phone case with a built-in 17mm thread (or bayonet
| if you pick a bayonet lens type). I think I'm using a
| Ulanzi case.
|
| The only frustration that remains is that the iPhone's
| built-in camera app does not let you force the choice of
| either of the lenses. It will choose for you if you pick
| 2x. So you might want an app. I really like the ProCamera
| app.
|
| I'm not sure whether accessory lenses are so useful on
| some of the three and four camera mobiles, but it works
| well enough for my iPhone X.
| hughrr wrote:
| Yeah 13 Pro here. I sold my DSLR last year. I don't need it
| any more and I hated carrying it around everywhere.
| ghaff wrote:
| I have a fairly large FF DSLR Canon system that I rarely
| use at this point--though not sure if it's worth selling.
| (I do use it now and then.) I use my Fujifilm mirrorless
| setup more but even that almost has to be a longer trip
| where I plan to take a lot of pictures. iPhone is fine for
| most purposes.
| mrfusion wrote:
| It's really that comparable??
| unfocussed_mike wrote:
| They aren't _comparable_ , not really. In the same way
| that an 8x10" camera and a full-frame camera aren't.
|
| But they are _excellent_.
|
| Good phone cameras have a value proposition all of their
| own, and they are utterly changing what we expect from
| photography.
|
| My mobile has taught me that a lot of what I relied on or
| worked with in a DSLR or mirrorless is a crutch.
|
| Mobile phone cameras force you, for example, to really
| think about composition, because you can't simply blur
| out the bits you don't like (portrait mode still sucks).
| They force you to think of other ways to isolate
| subjects, other ways to make use of light and contrast.
|
| I've owned a lot of kit over 20 years or so, though I
| still own a lot of it -- I'm using a 14-year-old full
| frame DSLR and a seven year old full-frame mirrorless.
|
| In the last two years, when studio portrait photography
| has been complicated or impossible, I have used my phone
| a lot, when out walking by myself. What I thought was
| just a way to not-totally-give-up photography has turned
| into a work of its own.
|
| It will anger a lot of photographers who like to whine
| about how mobile phone cameras can't do X, Y and Z, but
| here's the truth: if you don't understand what a mobile
| phone camera can teach you as a photographer, you're
| probably not really trying.
| tuyiown wrote:
| Besides telephoto and real macro, the results of the
| iPhone 12 made the idea of gear renewal almost absurd for
| me, it's just incredible and I'm not talking about the 12
| pro.
|
| There is trade offs, none of them justifies carrying
| heavy gear around most of the time.
|
| edit: it's a hobby and I just spent a lot of time trying
| to take pretty pictures, and searching the proper cost
| tradeoff to do them
| unfocussed_mike wrote:
| Right. Gear churn just seems peculiar.
|
| I think this is also why we see lots of people who own
| high end full-frame etc., are now adapting old lenses,
| building extraordinary DIY rigs like the "digital camera
| obscura" trend of the moment, making entirely custom
| lenses with surplus optics and 3D printers, etc.
|
| It's the same reason as for why there's so much interest
| in film.
|
| These things have different strengths. I think mobile
| phones have liberated photography in a way unparalleled
| since the time when 35mm film liberated photography.
| ghaff wrote:
| >Gear churn just seems peculiar.
|
| There was a period when DSLRs were improving pretty
| rapidly and it made sense to move up on a pretty regular
| cadence because the new stuff was so much better. (And
| we've seen this with phones more recently.)
|
| There's definitely a retro aspect to film and vinyl.
| Personally that's not for me having lived with the
| limitations of both. But I spent years working with
| custom film developing chemistry and the like.
| unfocussed_mike wrote:
| I still think of myself as someone who shoots a little
| bit of film, though like other things I still think of
| myself as doing, not since the pandemic.
|
| But I think I view film photography, darkroom work, and
| mobile phones in basically the same way.
|
| They are just _some photography tools among many_ , and
| my photography education would be incomplete without
| them.
| ghaff wrote:
| I was photo editor of my undergrad newspaper, editor of
| high school yearbook, and made beer money with
| photography in various ways. (I was also de facto photo
| editor of a newspaper in grad school among other things.)
|
| I spent a lot of time on it and got a lot of enjoyment
| out of it. I also probably got a bit burned out and
| wasn't really interested any longer once I didn't have
| easy access to a good darkroom any longer. But yeah,
| happy I did it, zero interest in doing any longer.
| Digital pretty much rekindled that interest.
|
| (Also could really not get into video in those days
| because overhead was just too much. Don't have obvious
| entry point into creative video these days but at least
| casual is easier.)
| ghaff wrote:
| It depends on what you're doing and what the photos are
| for. If you're shooting events, sports, wildlife, or want
| to do shots with specialty--like ultra-wideangle--lenses,
| precise aperture control, large blowups, no. But phones
| can now handle a huge amount of what most people--even
| relatively serious photographers, need day to day or on a
| random hike/city visit/etc. need. If you're shooting with
| a DSLR and whatever kit lens it came with on A you're
| probably fine using a phone.
| bagels wrote:
| So are these lens assemblies designed by some optimizer? There
| are a lot of radical shapes in there.
| R0b0t1 wrote:
| It's possible, but they are sufficiently regular I think some
| obvious hand optimization math could get you there. I've seen
| similar in other fields.
| Buttons840 wrote:
| How much does this lens cost?
| trevortheblack wrote:
| Yining Karl Li has mentioned in previous posts their intention to
| write a blog post about modern lens optics, which should outline
| multiple things.
|
| Hopefully, a general overview of the 100s of lenses they've
| found/made the models for, alongside how modern optics fail to be
| accurately modeled by the polynomial model of optics...
| mrfusion wrote:
| I don't really get it. Also could this be used to improve
| telescopes?
| sega_sai wrote:
| (astronomer here). Some comments below partially answered the
| question, as modern optical/ir telescopes are based on mirror
| designs (i.e. they have primary, secondary, and sometimes the
| tertiary mirror like the Vera Rubin telescope). But the lenses
| are still heavily used in the camera part of the instrument or
| in spectrographs. And there is a lot of know-how there, as well
| as optimization in terms of minimising light losses,
| aberrations etc. I think the constraints and goals are clearly
| different from the cameras in iphones, but I'd think the
| techniques must be pretty similar (although Apple clearly has a
| ton of money that astronomers don't so it's possible that there
| are some big innovations there, that I don't know about).
| leeoniya wrote:
| SCT reflector telescopes use an exotic looking "corrector
| plate" (lens) to compensate for the easier-to-produce
| spherical mirrors vs parabolic.
|
| https://en.m.wikipedia.org/wiki/Schmidt_corrector_plate
| bagels wrote:
| These designs are required due to the package size constraints
| and the relatively generous lighting conditions.
| ElephantsMyAnus wrote:
| Isn't the primary reason that the picture has to be projected
| on a flat plane?
| aliher1911 wrote:
| Not an astronomer, but from my limited knowledge telescopes
| usually use reflection e.g. mirrors so they don't need to
| correct for chromatic aberrations introduced by lenses and they
| also don't use things like optical stabilisation. Refractors
| could possibly be corrected, but AFAIK they aren't widely used
| in astronomy and hobby telescopes is such a niche market.
| amelius wrote:
| They are still rotationally symmetric, at least.
| teekert wrote:
| I recently switched from a OnePlus3 to an iPhone 12 mini and
| although the pictures are really much better, one thing is worse,
| the lens glare. There is often a glare and sometimes a bright
| green dot. Is that a consequence of these many elements? Less
| spherical and chromatic aberrations but more glare and other
| internal reflections?
| GoToRO wrote:
| From what I've learn from 3d software tutorials (trying to
| simulate a glare), this glare comes from the lenses and each
| lens will add another ring in the glare? not sure.
| red0point wrote:
| Maybe dumb to suggest, but when this happens usually the lens
| is dirty / a bit greasy, at least from my experience with
| iPhone cameras.
| teekert wrote:
| I'll degrease the lenses now, perhaps this was quite a
| brilliant suggestion :)
| ChuckNorris89 wrote:
| The OnePlus 3 had a sapphire camera lens, Apple does not. Maybe
| that's why the flare.
___________________________________________________________________
(page generated 2022-03-05 23:00 UTC)