[HN Gopher] How computer vision is changing manufacturing in 2023
___________________________________________________________________
How computer vision is changing manufacturing in 2023
Author : sickeythecat
Score : 137 points
Date : 2023-03-09 16:36 UTC (6 hours ago)
(HTM) web link (voxel51.com)
(TXT) w3m dump (voxel51.com)
| djfobbz wrote:
| Here's Fanuc M-1iA series robot organizing pills by color back in
| 2018 @ https://youtube.com/shorts/bdosfVWhhlQ ...I can only
| imagine what they have now!
| snerbles wrote:
| More of what they had then.
|
| That demo of real-time blob detection and sorting by color
| filtering was doable in 1998. Earlier than that, even. I've
| found about 90% of the work in vision applications in
| industrial packaging is in the product handling and scene setup
| - focal length, lens selection, exposure time, etc. - all
| things familiar to a photographer. The last 10% is almost
| always handled by bog simple algorithms that can be more or
| less cobbled together from OpenCV's examples and boilerplate,
| the most complicated usually being OCR.
|
| The value-add of these dedicated industrial vision systems is
| in integration. Fanuc's iRVision is good at sending spatial
| data back to the robot controller, but the interface itself is
| a horrid kludge that specifically requires Internet Explorer
| and in-person training at their own (admittedly very nice)
| facilities and promises of litigation if you so much as _think_
| about sharing documentation with co-workers.
|
| Recording images during trial runs with their native tooling
| was impossible, as their under-powered processor couldn't
| handle saving 640x480 images at 10fps while also running the
| vision application. So we resorted to recording test runs by
| feeding the live view OBS, and everyone thought I was some kind
| of wizard for even considering that.
|
| At least Cognex's In-Sight has the ability to simulate their
| weird spreadsheet-based vision programs without a camera. With
| Fanuc you need the whole $30,000+ robot+controller+camera setup
| _and_ with real-time applications the only way to debug it is
| to run it in situ.
|
| Now my most recent industrial vision experience is from 2019,
| so maybe some things have changed. But these are folks that
| often don't even know what source control is and will run
| screaming for the hills at the first sign of anything that's
| not Excel or ladder logic, and balk at the idea of paying an
| experienced engineer more than $100k all the while wondering
| why they aren't finding any talent.
| tomp wrote:
| Sounds like there's a gap in the market.
|
| I'm hugely enthusiastic hobbyist that would love to chat more
| about robotics, in particular how a hobbyist could get
| started with it (a robot arm + camera maybe?). I'd love to
| buy you virtual coffee, get in touch if you're up to it!
| lnsru wrote:
| There are many gaps in the robotics market. But there are 2
| main show stoppers: 1. Starting robotics venture is very
| expensive. You need at least 3
| engineers(hardware+software+electronics) for at least 3
| years with tons of expensive hardware to reach MVP. 2. The
| clients will not buy from a company that might be gone in 2
| years when the whole installation is planned for a decade.
| The client is mostly integrator choosing familiar system
| components. The 3rd show stopper is that the product must
| work 100% or the time. 99,5% is not enough. Automation is
| here to replace people instead of having a robot with
| maintenance crew nearby.
| sbierwagen wrote:
| I think it's structural. Salaries between EE and CS sharply
| diverged decades ago, and I don't think they'll ever meet
| again.
|
| The finances on pure software are just so much better.
| Better margins, better scale, better return on equity.
| Since ROE is always going to be better (because you're not
| touching atoms) you'll always have better valuation on the
| stock market, and be able to pay programmers better.
|
| It's less a "gap" in the market and more "the market
| functioning correctly". There's no law of the universe that
| says programming a robot has to pay as well as programming
| a SAAS webapp.
|
| Think about scale. If you teach programming at a middle
| school, you have maybe 100 customers at a time. If you work
| for a hardware company, you have 1,000,000 customers. If
| you work for facebook, you have 3,000,000,000 customers.
| Which one of these will pay the most?
| PeterisP wrote:
| If the jobs require similar skills/aptitude/talent and a
| similar level of "investment" in job-specific training
| and experience, then the market force expectation is that
| as the information about the pay gap becomes clear,
| potential engineers would avoid EE jobs in favor on CS
| jobs, and EE training in favor of web development
| training, until the shortage of employees forces hardware
| companies to pay robot programmers as good as SAAS
| programmers or be unable to hire robot programmers.
| vidanay wrote:
| Computer vision has been deeply integrated in manufacturing for
| 20+ years. if you've brushed your teeth or drank a sports drink
| in the last 10 years, your toothbrush or bottle has probably gone
| through a vision system that I write the software for.
|
| (Not Cognex)
| ben_w wrote:
| Aye. Back at university, 19-18 years ago now, I had a
| "mandatory"[0] year in "industry"[1], and some of the job
| advertisements were for adding computer vision to some process
| or other. Given how... out of date the teacher introductory
| module in the final year course was, there was no way this was
| students being asked to actually create those vision systems.
|
| [0] scare quotes because I had the completely free choice
| between two otherwise identical degrees, one of which had a
| mandatory year in industry and the other did not, because the
| UK council tax system demands a different rate if you're a
| student on a course with a mandatory year in industry
|
| [1] also a cheat, I worked for an academic research lab
| codetrotter wrote:
| FANUC?
| vidanay wrote:
| Nope.
| snarf21 wrote:
| Agreed, I worked on manufacturing quality assurance software
| that controlled vision to detect particles in vials of medicine
| 20 years ago. The main thing that has changed is the quality of
| the camera has greatly increased and the price of the camera
| has greatly decreased.
| vidanay wrote:
| Yeah, when I started, we were using RS-170 cameras connected
| to $30k Cognex acquisition boards (all analog). The switch
| over to USB and then GigE has been fantastic.
| snarf21 wrote:
| I miss it in a way. USB cameras were just coming online and
| we're very good yet when I switched to a different
| industry. It looks like the company was acquired into an
| automation machinery parent company.
| vidanay wrote:
| God, the first generations of both USB and Ethernet
| cameras positively SUCKED. Flaky, buggy, and expensive.
| Our first foray into Ethernet cameras was from a company
| called Opteon. They had taken a stock Intel ethernet card
| and flashed custom firmware onto it to support their
| custom framing. If the cards and the cameras didn't match
| exact versions, you could end up bricking one or both of
| them. They had to be sent back to the vendor to be fixed.
|
| edit: Oh, hey! Opteon still exists! I'm sure their
| products are much better than those first generations.
|
| https://www.opteontech.com/products/cameras
| snarf21 wrote:
| oops, meant to say "were _NOT_ very good yet when I
| switched to a different industry ". Apparently I failed
| at typing and paying attention to a meeting at the same
| time. :)
| zwieback wrote:
| Same here, RS-170 into Cognex MVS8100 ca. 20 years ago.
| Their pattern matching was gold standard at the time. We
| also used Matrox but I was a Cognex man at that time. Some
| of those systems are still running today but I do not miss
| the days of analog video one bit.
| goblinux wrote:
| Gotta be Keyence?
|
| My favorite machine vision use case is the tomato sorter. This
| is one I found from YouTube, not affiliated
|
| https://youtu.be/j4RWJTs0QCk
| vidanay wrote:
| No. my company makes entire PC based vision systems including
| material handling and value added services. We are not a
| component level vendor.
| jcynix wrote:
| Ah, a Tomra machine. They too produce the machines which
| collect returnable bottles. Those are installed in each
| Supermarkte in Germany, for example.
|
| And here's documentation on machines which sort grapes
| intended for wine production:
|
| https://www.food.fraunhofer.de/en/beispiele12/Produktschutz/.
| ..
| bilsbie wrote:
| Is anyone using transformers in this field yet?
| snerbles wrote:
| Smaller neural nets are commonly used in character recognition,
| but typical smart cameras or embedded robot controllers don't
| have anywhere near enough compute to run deep networks in real
| time. The Fanuc I was ranting about in this thread had
| something like 64MB of RAM in 2018. Maybe some systems are out
| there using Coral TPUs with custom TF Lite models.
|
| As for PC-based systems, I would be very surprised if deep
| learning models weren't being used in production _somewhere_.
| But in a factory environment you can go a very long way with
| primitive feature recognition and good control over the scene
| and lighting, and the customer just cares that whatever you 're
| doing just works and any new method will have to be enough of
| an improvement to be worth the cost of development time.
| fest wrote:
| > As for PC-based systems, I would be very surprised if deep
| learning models weren't being used in production somewhere.
|
| They definitely are. ~5 years ago I built a PC-based system
| that detected grain direction of wooden boards (looking at
| the end of the board).
|
| Initially I resisted the ML approaches and my first attempt
| was basically hand-crafted image analysis pipeline- split the
| image in segments, apply Gabor filter with kernels of various
| angles and try to fit a curve to results. It kind-of-worked
| but I wasn't entirely happy with it's performance on the test
| data.
|
| Even the simple classifier models that could execute on a
| fanless PC without a GPU outperformed my solution, and after
| a few more training runs the handcrafted code was replaced by
| #include <tensorflow.h>.
|
| This year I'll have to extend the system with on-site
| training mode, where an operator has a pushbutton to label
| the images and re-train the model.
| fest wrote:
| Also, I'm pretty sure all the major smart camera vendors have
| projects underway which utilize NVidia Jetson.
| vidanay wrote:
| The 900lb gorilla in the deep learning room that everyone
| likes to ignore is that machine learning is horrible at
| providing corrective action data. Traditional machine vision
| is well adapted to providing statistical data such as "the
| diameter of the pizza is out of tolerance by 8mm" or "there
| are supposed to be 22 pepperonis on the pizza, but only 19
| were found". Machine learning leans towards "it's not a good
| pizza" and doesn't provide a lot of additional data.
| jeffreyrogers wrote:
| There's a lot of progress on this recently with things like
| conformal prediction.
| krisoft wrote:
| > machine learning is horrible at providing corrective
| action data
|
| I don't recognise the truth in what you are writing.
|
| > there are supposed to be 22 pepperonis on the pizza, but
| only 19 were found
|
| Instance segmentation is a solved problem. A properly
| constructed and trained neural network can tell you exactly
| how many pepperonies it sees and exactly where. Telling if
| that is the right number is a trivial problem from there.
|
| > the diameter of the pizza is out of tolerance by 8mm
|
| Here too, the neural network can recognise the edges of the
| pizza and then you can fit a shape to it. You can do this
| second step either with classical algorithms or with a
| machine learning one. (I would use a classical algorithm if
| the pizza is meant to be circular or rectangular shaped,
| and a machine learning algorithm if they are aiming for
| something weird, like an Italy shaped pizza or something.)
|
| > Machine learning leans towards "it's not a good pizza"
|
| Sounds like you have only heard of simple classifier
| models.
| vidanay wrote:
| I will accept your opinion as I have never implemented a
| complete ML based solution. All of my opinion is based on
| promises and demonstrations for ML products such as
| Cognex VIDI. If those systems have capabilities like you
| describe, they have not been well presented during their
| sales pitches.
| JohnFen wrote:
| Yes, the company I work for is.
| PicassoCTs wrote:
| ? Does not mention Keyence and others.
|
| Honestly though, the suits i worked with, were all very dated and
| used hand constructed feature filters etc. to detect flaws.
| Usually, it was easier to adapt the environment (exclude external
| light etc.) instead of lengthy tuning sessions for the installer.
|
| Usually the industrial cameras were also designed, so that local
| maintainers could readjust them, which excluded complex
| programming and happened in simple wizards or excel like
| programming surfaces. There was no time planned in to "retrain"
| further once the line was running. And it was cheap and good
| enough that way.
|
| Thus the "cutting" edge tech seemed to be eternally 20 years
| behind the cutting edge in other sectors relying on machine
| vision.
| zwieback wrote:
| We use "smart" cameras from Keyence and Cognex but the really
| interesting work tends to still be in PC-based, hand-coded
| vision systems. Usually hand-crafted C++ or C# code but
| increasingly using neural networks for some, usually non-
| quantitative (e.g. locating but not measuring), solutions.
| vidanay wrote:
| As a developer and maintainer of a PC based vision solution,
| I don't like smart cameras. :)
| snerbles wrote:
| As an integrator of vision solutions, smart cameras firmly
| occupy the space of "Nobody Ever Got Fired For Buying IBM"
|
| Though with recent developments in machine learning, the
| case for PC-based solutions is a lot easier now than
| before. Behind all the fluff and shiny marketing, the
| incumbents are _very_ stagnant.
| vidanay wrote:
| 15 years ago, I used to claim that there were more smart
| cameras sitting in engineers desk drawers than there were
| running in production. I think that was true until about
| 7-8 years ago.
| zwieback wrote:
| I would agree, many of those cameras are mouldering in
| our reclaim area now. But the newer generations are
| powerful enough that we can turn normal manufacturing
| engineers loos on simpler vision tasks and leave the
| challenging applications to more traditional systems.
| snerbles wrote:
| Definitely not the case at my old job - we deployed a bit
| over 200 Cognex In-Sight cameras over my five-year stint
| there, almost all for bespoke inspection applications for
| customers.
|
| They gave me plenty of swag, but if I wanted to play with
| one of their cameras I'd have to go out on the production
| floor.
| [deleted]
| vidanay wrote:
| Sounds like you worked for an integrator, so it stands to
| reason that you had a high success rate. Cognex, Keyance,
| DVT, et al sold a lot of smart cameras in batches of 1
| and 2 to non-vision experienced engineers based on the
| lie that they could bolt it up to a conveyor and in an
| afternoon of programming on their game controller they
| could be up and running and miraculously improving their
| quality by 30% by next Monday. I think the vast majority
| of these cameras never saw production.
| snerbles wrote:
| As an OEM the ones we deployed definitely saw (and are
| probably still seeing) production. I can personally
| confirm off the top of my head that Proctor & Gamble,
| Revlon, Duracell, Mars Candy, Bausch & Lomb, Pfizer,
| Gilead Sciences, Boehringer Ingelheim (and more) use
| Cognex cameras in their product packaging lines.
|
| The promise is - as you say yourself - in systems that
| are easily maintained by non-vision experienced
| engineers. As I noted in another post, these are usually
| controls engineers that overwhelmingly prefer ladder
| logic on their PLCs and have little exposure to modern
| software engineering practices such as _source control_.
| Obviously it 's not "up in an afternoon" - any sales rep
| that promised that got sent away ( _Keyence, I 'm looking
| at you_) - debugging consists of a lot of product test
| runs and more mechanical/controls work and definitely
| takes more than a day.
|
| I tried on more than one occasion to put forward PC-based
| systems, but the customers wanted the smart cameras.
| Though I did frequently use OpenCV for batch image
| analysis in-house, I ought to write an article or two
| about that bit.
| vidanay wrote:
| >I can personally confirm off the top of my head that
| Proctor & Gamble, Revlon, Duracell, Mars Candy, Bausch &
| Lomb, Pfizer, Gilead Sciences, Boehringer Ingelheim (and
| more) use Cognex cameras in their product packaging
| lines.
|
| >I tried on more than one occasion to put forward PC-
| based systems, but the customers wanted the smart
| cameras.
|
| Oh, all those that you listed also use PC-based systems.
| I know because all of them are also customers of ours.
| snerbles wrote:
| What can I say, I wish I had your customer PMs.
| zwieback wrote:
| Your end-users, do they mess around with the spreadsheet
| programming interface, via some kind of remote PC
| interface or not at all? We deployed quite a few Cognex
| InSights in the early days but version control and
| distribution of updates was a major headache.
|
| Sidenote, I feel partly responsible because we bought a
| ton of systems from McGarry's previous company Acumen,
| then I guess he took off and created the InSight.
| Colorful guy, I remember him showing up with his fancy
| Porsche around that time...
| snerbles wrote:
| > version control and distribution of updates was a major
| headache
|
| I wound up writing an in-house tool that pulls the
| program files from each camera on the machine LAN over
| FTP and commit them to a local Git repo. There was also
| some futzing around with XML to get the backup metadata
| to work seamlessly, but it's not too hard to figure out.
|
| Now getting co-workers to use Git and not various
| combinations of "Copy of (1) Copy of visionproject
| (FINAL) 3-2-16 2a.zip" was a different challenge.
| kevin_thibedeau wrote:
| I interviewed at Cognex 15 years ago. They eliminated their
| EE department down to one H-1B who broke down in tears as I
| tried to figure out why I was being interviewed by people
| with no knowledge about the job. They were solely interested
| in the ability to reverse engineer something without any
| documentation. It was clear they were just repackaging cheap
| SZ camera modules in overpriced yellow boxes. Everyone was
| glowing about their "legacy" product line from before they
| canned their engineers. Nothing but crickets when asked about
| the new stuff. The founders had constructed this weird
| personality cult around themselves. Glad I dodged that
| bullet.
| zwieback wrote:
| East or West Coast? I had a strange interview experience in
| Oregon in the late 90s but not as bad as yours from the
| sound of it.
| jeffbee wrote:
| Doesn't even mention National Instruments. This article is
| clearly cheerleading for a bunch of startups, and wants us to
| be ignorant of the larger picture. Robots have been picking up
| and stacking junk on assembly lines since the 1990s at the
| latest.
| skeletal88 wrote:
| Useds few Basler GigE and USB3 camerasfor a robotics competition
| at the university, was fun, cameras were easy to use.. only later
| I saw how they are used in the industry.
| syntaxing wrote:
| Unfortunately, until we really need to push high scale
| manufacturing back to the states, it's not changing anything for
| US manufacturing. I worked for a company that took almost a
| decade just to change from devicenet to ethercat, predictive
| analytics took 5. Any sort of "smart" system just doesn't have a
| huge momentum unless we're producing items at China rate and need
| to maintain cost low.
| hummus_bae wrote:
| [dead]
| vidanay wrote:
| I've worked for an industrial vision equipment company for 21
| years and North America has ALWAYS been our strongest market
| segment.
| WheatMillington wrote:
| Is this actually true, or is it just hysteria from someone
| unfamiliar with manufacturing? Because I live in New Zealand,
| which is on a similar deindustrialization path to America, but
| with obviously much less manufacturing in the first place. But
| we nonetheless have a burgeoning manufacturing robotics
| industry for what we still have - wood processing, pulp and
| paper, agriculture, etc.
| jeffbee wrote:
| In what sense do you mean deindustrialization? It appears
| from various reports that NZ manufacturing output it at a
| record high. Do you mean as a % of GDP?
|
| In the U.S. the number of people employed in manufacturing is
| lower than ever but the value of manufacturing has been
| steady at 12% of GDP since World War 2 ended.
| JohnFen wrote:
| > it's not changing anything for US manufacturing.
|
| I don't think this is true. I work for a US company producing
| industrial equipment based heavily on machine vision. Our
| products (along with those of our competitors) have changed the
| entire industry we support, for the better.
|
| Ours is only one specific part of the manufacturing space, but
| I fully expect the impact to spread to other parts as well.
| waldarbeiter wrote:
| Does the industrial equipment enable manufacturers to
| eliminate human labor in the production process or is it more
| of a way to replace existing machines with a more reliable,
| performant etc. solution? If you want to share this info.
| jamarks13 wrote:
| From what I understand, there are increases in efficiency
| due (in part) to the ability of machines to run at all
| hours, helping manufacturers to work with delayed and
| unpredictable supply chains. Also due to reducing tiresome,
| back-breaking manual labor.
| JohnFen wrote:
| It increases the efficiency (that is, reduces waste) of the
| use of raw materials in production. In some circumstances,
| it probably does replace a small number of human workers,
| but that's not its main effect and isn't the source of the
| savings our customers get from it. It's mostly doing a job
| that wasn't being done before.
| ROTMetro wrote:
| High valued assemblies tend to be manufactured in the USA.
| Defect detection early and often prevents waiting until
| completion to scrap them (much better to scrap when you only
| have $10,000 in subassemblies and labor into it than then when
| you have $50,000). If that assembly is a bottleneck you are
| also reducing the impact to your larger production schedule
| (scraping it one week into assembly versus 4). Computer vision
| is hugely important for this. Computer vision capturing each
| stage also greatly helps to quickly isolates what is
| introducing the situation resulting in scrap. Instead of having
| scrap meetings trying to determine why a completed assembly is
| failing you identify the failure point as close to when it
| occurs as possible.
|
| Being able to identify molds reaching end of life prior to
| parts failing QA for being out of tolerance is also huge for
| American manufacturing.
|
| Where it's way less important is when you are spitting out
| eraser tips or other 'high scale' manufacturing.
| zwieback wrote:
| Sort of agree but a couple counterexamples from my machine
| vision career:
|
| - Agriculture and food processing, which cannot be offshored as
| easily, requires very challenging machine vision solutions.
| Dirty environment, unpredictable lighting, unpredictable object
| appearance.
|
| - Proto and small scale high tech manufacturing, pre-offshoring
| or sensitive IP, requires machine vision solutions that are
| both sophisticated and quickly adaptable
| narrator wrote:
| Once robotics and computer vision gets there, there could be
| a lot of money in robotized regenerative agriculture.
| Loughla wrote:
| How is this wishlist related to anything that is being
| talked about in this thread? I'm very confused by what
| you're adding to the conversation.
|
| I also wish robots would do the menial labor that I do not
| enjoy and would take care of all of my basic needs.
|
| But this is an article about basic computer vision
| beginning to impact basic manufacturing. What you're
| talking about is decades in the future if ever. I'm very
| confused.
|
| Edit: The OP originally talked about an agricultural robot
| that could charge itself, do all the home chores, and fix
| things around the house. Now it's just one sentence.
| ghaff wrote:
| A general-purpose robotic handyman for consumers is many
| many decades away (at least). And if such a thing _did_
| exist it would have massive massive implications for the
| labor market--both on its own and because of the
| implications for all the other things that AI could do
| were such a robot possible.
|
| Computer vision in a very constrained environment is much
| much different and often isn't even suitable for many
| "simple" tasks that aren't constrained quite enough.
| narrator wrote:
| What if there was some sort of LLM like breakthrough? I
| could see someone doing a technique where they track all
| movements of a person going throug their day with a fine
| grained bodysuit. They could then use that as tokens for
| generative input that could give a humanoid robot
| intuition about how to move around and perform tasks that
| would match an LLM's ability to respond to arbitrary
| questions.
| akiselev wrote:
| You'd walk face first right into Moravec's Paradox [1]
| which observes that higher order intelligence is far
| easier than basic locomotion & cognition. Probably
| because the former has only been evolving for millions of
| years in humans while the latter has been evolving in all
| animals for hundreds of millions of years.
|
| We can't produce an electromechanical device that is
| capable of the kind of fine motor control 99% of animals
| are capable of, let alone doing it on an industrial
| scale. We're not even at the "promising proof of concept"
| stage and what use is more advanced software when we're
| not even close with the hardware.
|
| [1] https://en.wikipedia.org/wiki/Moravec%27s_paradox
| kilgnad wrote:
| I guess it's cheaper to hire workers in China, but also cheaper
| to have automated machines running in China and have the
| Chinese build those machines.
| jamarks13 wrote:
| definitely safer to have industrial automation systems
| running as well!
| LeanderK wrote:
| I can imagine that China also has massive infrastructure and
| a manufacturing environment built up over the last years that
| may become increasingly hard to replicate in the US. I bet
| there some "critical mass" for high-volume manufacturing
| that's needed, if you don't count subsidies. Even if it's all
| robots, you still need suppliers etc.
___________________________________________________________________
(page generated 2023-03-09 23:00 UTC)