[HN Gopher] The age of machine learning as code has arrived
___________________________________________________________________
The age of machine learning as code has arrived
Author : gk1
Score : 54 points
Date : 2021-10-22 18:43 UTC (4 hours ago)
(HTM) web link (huggingface.co)
(TXT) w3m dump (huggingface.co)
| ragnot wrote:
| I've heard that software engineers are the automobile workers of
| the 50s-70s: Extremely well-compensated professionals who were
| ultimately thrown to the wayside when technology came. I wonder
| if data scientists and AI will do a similar thing to software
| engineers.
| jokoon wrote:
| It depends, because you still need people even in car
| factories.
|
| Also, automation really changes the nature of a product, and
| the quality is often poor.
| snek_case wrote:
| In my opinion, unless we develop AGI, not at all... And if we
| did develop AGI, then data scientists would find themselves out
| of a job too.
|
| I think you'd be very hard pressed to automate systems
| programming, and most forms of programming for that matter. So,
| as long as we develop new hardware, new products, new
| platforms, new programming languages, then there will be lots
| of programming work to be done.
|
| The jobs that machine learning is replacing right now (if any?)
| are fairly repetitive jobs that don't really require any
| reasoning. Maybe machine learning is also helping automate
| phishing schemes, but, is there any programming job out there
| that's really repetitive and thoughtless, where the work is
| predictable and akin to factory work? Do you think a
| transformer model is going to be able to handle debugging?
| ragnot wrote:
| You obviously have more knowledge than me on this subject, so
| take what I say with that in mind. But I look at the march of
| progress and can't help but draw parallels.
|
| Websites used to be made by hand. Then came frameworks and
| then came code-free e-commerce sites like Shopify. Github is
| testing the Copilot AI. How much longer before some architect
| tells an AI to stitch together code that fits some sort of
| functional spec. REST API's are already defined by their
| model and have code generated to match it. Is it that much a
| stretch to say that in the near future (15-25 years), AI
| might take over a significant part of software engineering?
| webmaven wrote:
| _> Is it that much a stretch to say that in the near future
| (15-25 years), AI might take over a significant part of
| software engineering?_
|
| Not much of a stretch at all. But we've seen this sort of
| transition before: lower-level languages (microcode,
| assembly) are now largely generated by other software yet
| the amount of software development work needing to be done
| by humans has only expanded, and I don't expect there to be
| much more than a bump in the road in terms of demand for
| software developers this time around either.
|
| Though I will say that the bump will be larger than the
| non-event that 4GLs turned out to be:
| https://en.wikipedia.org/wiki/Fourth-
| generation_programming_...
|
| The most disruptive scenario I can imagine will result in a
| slew of businesses hit with absolutely massive cloud
| computing bills (due to naive generated code that works,
| but very inefficiently, or perhaps just code running for
| naively specified tasks that HAVE no efficient solution as
| stated), leading to a standard requirement of having an
| expensive human software developer sign off on it before
| deployment like architects signing off on blueprints (which
| might be the thin edge of the wedge for professional
| licensing, but that's a different conversation).
|
| As a secondary effect, there might also be a proliferation
| of new (or newly popular) languages (for which insufficient
| public code exists for ML systems to learn from), as
| developers focus on domain+skill+technology combinations
| that haven't been automated YET.
| pasquinelli wrote:
| i think you're focusing too much on how many programmers
| will be needed in the future without considering what
| kind of pay they'll be able to get.
| thrower123 wrote:
| These are the same arguments that people have made about
| outsourcing for thirty years, and we've only got more
| programmers who are paid vastly more money because of it.
| pasquinelli wrote:
| _because_ of it?
| thrower123 wrote:
| To some extent, yes. There's an awful lot of work that
| comes back to be done correctly.
|
| It's how I got my job originally.
| snek_case wrote:
| However, despite the existence of products like Shopify and
| Wordpress, web developers have no trouble finding jobs, as
| far as I know.
|
| > How much longer before some architect tells an AI to
| stitch together code that fits some sort of functional
| spec.
|
| Would that person not be, essentially, a programmer? And
| what would they do if they needed to interface with
| external hardware that's not yet supported, or some network
| protocol or piece of software that's not built into AI-
| code-generator-3000?
|
| > Is it that much a stretch to say that in the near future
| (15-25 years), AI might take over a significant part of
| software engineering?
|
| In my opinion, people who probably think little of software
| developers, keep imagining that software engineering is
| this primitive, simple, reducible thing, and they are
| simply wrong.
|
| Can there be a use for AI-generated code, and can that take
| over things like generating user-facing forms and website?
| Sure. However, what I think is likely to happen is that in
| 15-25 years, the world will be changing just as fast as it
| is now, and the software engineering discipline will just
| expand. Companies that hire "architects" who only know how
| to use AI-code-generator-3000 will be at a disadvantage
| compared to companies that hire more fully fledged software
| engineers.
|
| Imagine you run a business and you only have architects who
| know how to use some code generator code that runs on the
| Google cloud. You pay some monthly bill to Google and hefty
| fees to use that tool. You're locked-in to that ecosystem.
| If you need a new feature or access to a new API, you may
| need to wait for Google to add support. In contrast, your
| competitors who hired pesky software engineers are paying
| higher salaries but save on Google cloud fees, and they
| don't have to wait for Google to fix problems to implement
| new features.
| SrslyJosh wrote:
| > Can there be a use for AI-generated code, and can that
| take over things like generating user-facing forms and
| website?
|
| Sure, but...I'm not convinced there's any advantage over
| writing deterministic generators for things like that.
|
| On one hand, you have something that produces a
| predictable output for a given input (modulo bugs in the
| generator, which can be corrected over time).
|
| On the other hand, you have a black box that could
| misinterpret some non-obvious aspect of its input and
| produce completely unexpected output. Its training data
| will need to be carefully curated and maintained over
| time, similar to how a deterministic generator would need
| to be maintained, but, again, without the same degree of
| certainty as to what it's going to produce.
|
| Honestly, the black box approach just seems like a
| nightmare to me.
| yuy910616 wrote:
| Progress != destruction of jobs.
|
| Farming, auto, journalism, and marketing fits your model.
| But medicine, finance, and tech would suggest otherwise.
| I've been getting advice about how there is no money in
| software and I should study hardware since 90s.
|
| Yes, maybe this is a freefall and we haven't made impact
| yet. But drawing parallel without further evidence seems to
| be too simple of a analysis.
|
| What cause the jobs in Detroit to be outsourced? What
| skills was replaced? How easy was it to train those skills?
| What was the potential pool of workers? What was the demand
| of workers?
|
| All I'm saying is that this question is a lot more nuanced.
| I'm a software engineer, so I'm biased. But I'm guessing
| that you might be biased in some ways too. The real answer,
| perhaps boring, is that we really don't know. Even if
| you're right - you could easily be off by 200 years.
| pasquinelli wrote:
| yeah, a lot of people are grasping at patterns they see
| in their understanding of the past and treating them like
| rules that will determine the future.
| wongarsu wrote:
| We have spent the last 50 years trying to automate our job
| away. Many projects that 20 years ago took an entire team a
| year or more are now weekend projects for a single person,
| because of how much better our programming languages,
| libraries, APIs, IDEs, linters, etc have become. But demand
| turns out to be incredibly elastic. A 50% increase in
| productivity just means that there's 400% more work, because so
| many new things have now become cost effective to try.
|
| Of course there is bound to be an end to this, a peak-software
| if you so will, where productivity increases will stop being
| offset by demand increases. I might live to see that day, but
| this AI wave is unlikely to be it. Yes, it brings some new
| tools that make programming easier, but it stimulates more than
| enough demand to offset that.
| didibus wrote:
| ML and data-science relies on software to work. Nobody is going
| to do data-science and ML by hand, so already you have a demand
| for software engineers and computer scientists to build tools
| and implement algorithms and information pipelines for the
| data-scientists and businesses who want to leverage ML.
|
| I think there might be a bit of a misconception of what
| existing software engineers did. Very rarely were they focused
| on designing inference models or performing data analysis. Most
| of the time the business side of things, like a business
| analysts, or people working on the business problem would
| perform analysis manually, and come up with business rules and
| logic. Software engineers could help them leverage more
| powerful techniques by building them tools that help for data
| analysis, like BI platforms, or data stores that can do bigger
| and more complex queries faster. Or they could help them with
| letting them know of techniques, generally known as old-school
| AI, such as edit distance algorithms, graph algorithms, logic
| rule engines, expert systems, etc.
|
| So nothing has really changed, except that there is even more
| demand from businesses who want software engineers to help them
| setup tools and pipelines for them to experiment with ML and
| have data scientists analyze their data.
|
| What it would take to get rid of the demand for software
| engineers is to be able to automate their work, and ML is far
| from able to do that. It could one day, but we are talking
| about a computer that can reprogram itself successfully, and
| that can also understand the requests of the user in how it
| should reprogram itself. And we're talking not just
| reprogramming, but it should also be able to replicate itself
| to other machines and all that, since most use case today
| involve distributed system, as no one computer has the compute
| needed for the scale we operate at. Things like GPT-3 show some
| promise, but as it stands today, you'd need a software engineer
| to inspect and review everything it generates, which would take
| almost as much time if not more.
|
| I think what is far more likely is that the offer increases,
| that is, that there eventually is just a lot more people with
| the ability to perform the work of software engineers. This
| will probably happen as a combination of more people joining
| the field, and the barrier to entry becoming smaller. The
| latter could be helped a little bit by ML, if it can deliver
| better auto-complete for example, but I think it largely is
| just driven by frameworks, PAAS, SAAS and IAAS, library, and
| all other type of code reuse we've been building for years now.
| Aerroon wrote:
| > _I wonder if data scientists and AI will do a similar thing
| to software engineers._
|
| I don't really see how. Software development is about
| translating arbitrary requirements into a rigorous form without
| ambiguity (ie the developer has to fill in the blanks). Unless
| we invent AGI we would need the 'clients' of software
| development to be able to unambiguously list their requirements
| (without there being any blanks that need to be filled in).
| Effectively, they'd have to become devs themselves. It might
| not look like the software development of today of writing
| text, but they'd still be doing a similar type of job.
| didibus wrote:
| > A few years ago, Forbes wrote that "Software ate the world, now
| AI is eating Software"
|
| As far as I know, AI is Software, so this type of reasoning
| really confuses me.
|
| I think it is possible that "developers" might have some of their
| work find automation using AI and ML, but Computer Science will
| obviously stick around as long as computers are around.
|
| For example, someone has to code the algorithms which are used to
| teach the computer how to learn. Someone has to code the
| pipelines for capturing data from the physical world into the
| digital world. Someone has to code the storage and exchange of
| that data to the ML models, and someone has to plug the inference
| into the production processes and systems. Someone has to put
| alarms and monitors around all these things to assert their
| operation. Someone has to code the UIs and interfaces that users
| will use to interact with all those systems. Finally, someone
| needs to code compatibility and optimizations of all these models
| and their implementation to continue to work and leverage new
| hardware capabilities.
|
| If people think that developers were spending 90% of their time
| hand tuning business rules and coming up with manually
| implemented inference and decision models they are highly
| mistaken. That could have been 5% of the job in some circles, but
| 95% of the job has always been all these peripheral tasks.
|
| If anything, there will be even more work now, since ML models
| require a lot more periphery to develop, train and ship.
| munk-a wrote:
| Once upon a time you could make a good living building business
| websites (i.e. the local florist) but even without AI and ML
| frameworks have already killed that dead - your florist now
| probably uses instagram as their website, they might use
| Squarespace - and if they're hardcore they use Joomla,
| Wordpress or Drupal. I think a lot of the media thinks that
| developer hours are spent proportional to the stuff they see
| and this is the crux of the matter. Ford isn't retaining a team
| of two hundred devs to maintain their sales pages - it might be
| like ten or so? While an insurer managing a claims system is
| probably retaining a boatload of FT devs and also shipping out
| a bunch of hours to consulting firms. Software programming
| labour is very much an iceberg problem - the things that look
| expensive and difficult are trivial and those things are easy
| to automate away - but we're all working on the hard shit.
| indrax wrote:
| Software is also part of the world, just a very different part.
| didibus wrote:
| True, though I'm assuming by "the world", they mean that
| traditional processes which would have involved pen and paper
| and other non computer based tools and systems are all being
| redesigned to make use of computers, thus Software is eating
| them.
|
| But AI is not replacing processes that rely on software with
| something that no longer relies on software. What AI is doing
| is it lets you replace even more processes that were not yet
| able to be handled by a computer with one that a computer can
| do, and thus it just helps Software eat even more of the
| world. Decision making and judgement tasks were not able to
| be successfully moved to computers prior to the recent
| advancements in AI, now some of those can be.
| snek_case wrote:
| My experience working in a deep learning outfit for a few years
| has been that the software stack is incredibly immature and
| fragile.
|
| You're generally tied to proprietary nvidia drivers that are a
| pain to install, and when they change version, it breaks software
| such as PyTorch, you're forced to update. PyTorch itself often
| has breaking changes. Then, most of the code is written in
| Python, often by academics who know and care very little about
| good software engineering practices. That also leads to fragile
| and breakable code. You'll often find yourself trying to install
| code that's just DOA because of multiple broken dependencies.
|
| The quality of the code can be terrible, because academics are in
| a constant rush to publish the next thing, and tend to view code
| as throwaway. Worse yet, it's often written by students, and
| these students are taught by professors who view software
| engineering as an inferior, unscientific discipline. I've seen
| many github repos with no open source license, and no README,
| just a random code dump with no comments.
|
| I'm personally not super enthusiastic by the idea of replacing a
| lot of code by machine learning just yet, because not only is the
| software ecosystem fragile, but the models can be as well. You're
| likely to introduce weird, unpredictable modes of failure. I can
| see that there's clearly a lot of people who dream of being able
| to basically create software without coding, and not have to deal
| with pesky software developers, but IMO, we're not there yet.
|
| This guy talks about cloud-based machine learning solutions. You
| won't have to deal with drivers and general "IT" concerns there.
| The problem is that, besides being very expensive, these cloud-
| based solution will introduce extra layers of vendor lock-in. If
| you're training on the google cloud and using TPUs that you can't
| even physically buy/own, you might have a hard time porting your
| code to run without a TPU. If Google revokes your access for some
| reason, because they claim you've violated their TOS, you are
| fucked.
| cjv wrote:
| Can you name some breaking changes PyTorch made?
| snek_case wrote:
| Not off-hand because it's been a while, but IIRC the names of
| some methods were changed, arguments were added/removed. I
| will say that was before version 1.0, so it may be more
| stable now.
|
| Last I saw the nvidia driver situation did not improve
| though. I wanted to come back to some old code, installed a
| new version of PyTorch. It complained that my GPU drivers
| were not recent enough, had to do this long tedious process
| that took over an hour to get newer drivers working.
___________________________________________________________________
(page generated 2021-10-22 23:00 UTC)