[HN Gopher] An Interview with Nvidia CEO Jensen Huang About AI's...
___________________________________________________________________
An Interview with Nvidia CEO Jensen Huang About AI's iPhone Moment
Author : pps
Score : 84 points
Date : 2023-03-25 15:26 UTC (7 hours ago)
(HTM) web link (stratechery.com)
(TXT) w3m dump (stratechery.com)
| crop_rotation wrote:
| Nvidia seems in a really great position. They are to AI what
| Intel was to the PC (and unlike the PC era there is not a single
| Microsoft here who controls the entire ecosystem). CUDA still has
| no alternative. Yes Google has TPUs but outside of Google NVIDIA
| still dominates and enjoys the network effects (support in all
| kinds of libs and framework). They face the same problems that
| Intel faced, as in the market just wants the software and if the
| software works the same, the hardware is replaceable. It will be
| interesting to see how they adapt.
| TechnicolorByte wrote:
| Interesting comparison with Intel in the PC market:
|
| > the market just wants the software and if the software works
| the same, the hardware is replaceable
|
| And likely to be true given how much competition is heating up
| in the AI hardware space. Granted, many of these competitors
| and startups especially have existed for years and haven't made
| much of a dent. Even Google's TPU doesn't seem _that_ much
| better than Nvidia's stuff based on their limited MLPerf score
| releases. Maybe this "iPhone moment" for AI will change that
| and force competitors to finally put some real effort in it.
|
| As for Nvidia, looks like they are trying to adapt by selling
| their own enterprise software solutions such as Omniverse and
| their AI model customization stuff. Will be interesting to see
| if they can transform into more of a software solutions
| provider going forward.
| Giorgi wrote:
| Why is this text though? Is there video?
| jaflo wrote:
| What's wrong with text?
| TechnicolorByte wrote:
| Looks like there's a paid podcast available. I'm sure there's
| some TTS apps that could take in the text and make it
| consumable that way, though.
| visarga wrote:
| NaturalReaders is a good one
| tmsh wrote:
| Lots of great quotes like: "Inference will be the
| way software is operated in the future. Inference is simply a
| piece of software that was written by a computer instead of a
| piece of software that was written by a human and every computer
| will just run inference someday."
| ww520 wrote:
| You can argue compilers have been doing that. And JIT was the
| next step to enable runtime rewriting of software. AI directed
| JIT or compilers probably will be next.
| bottlepalm wrote:
| Man how lucky is Nvidia? crypto winds down just as AI is ramps
| up.
| PartiallyTyped wrote:
| Nvidia is part of the reason why it happened and why it
| happened _now_ , they didn't get lucky by any stretch of the
| imagination wrt to AI.
| greatpostman wrote:
| I wouldn't call it luck, just a great product that's enabling
| societal change
| ralphc wrote:
| NVidia got lucky, at least in the long term. They spent years
| perfecting SIMD to make spaceships blow up and, by
| coincidence, that's the same technology that enabled coin
| mining then deep learning.
| orangepanda wrote:
| Is Nvidia taking advantage of the crypto/AI trends, or enabling
| them?
| rchaud wrote:
| they're selling shovels in a gold rush, that's the best
| business to be in.
| danpalmer wrote:
| They have been investing in this for a long time, CUDA is 15
| years old. That said, they have plenty of competition now and
| that's only going to increase.
|
| - Apple have got their own chips now for mobile AI applications
| for iPhones.
|
| - Google have got their own chips for Android and servers.
|
| - AMD are gaining on Intel in the server space, and have their
| own GPUs, being able to sell CPUs and GPUs that complement each
| other may be a good strategy, plus AMD have plenty of their own
| experience with OpenCL.
| noogle wrote:
| How much of a moat is CUDA?
|
| It's indeed ages beyond any of their competitors. However,
| most ML/DS people interact with CUDA via a higher-level
| framework. In recent years this community has consolidated
| around a few (and even only one platform, PyTorch) framework.
| For some reason AMD had not invested in platform backends,
| but there is no network effect or a vendor lock-in to hinder
| a shift from CUDA to ROCm if it is supported equally well.
| floatngupstream wrote:
| There is an enormous investment beside the training side.
| Once you have your model, you still need to run it. This is
| where Triton, TensorRT, and handcrafted CUDA kernels as
| plugins come in. There is no equivalent on ROCm for this
| (MIGraphX is not close).
| A4ET8a8uTh0 wrote:
| Yeah... but have you tried using AMD GPUs for any LLMs for
| example? All the interesting stuff that is publicly released
| is for Nvidia. I would love to be able to focus on AMD since
| Nvidia has been adding some anti-user features lately.
| pier25 wrote:
| > _Google have got their own chips for Android_
|
| What chips for Android?
| mastax wrote:
| I'm not convinced AMD has a good AI play. (Disclaimer: I hold
| a long position in AMD).
|
| AI hardware seems like it can be much simpler than GPGPUs,
| given the successful implementations by many companies
| including small startups.
|
| AI hardware _software_ seems like it is extremely difficult.
| Making a simple programmer and ops interface over a massively
| parallel, distributed, memory bandwidth constrained system
| that needs to be able to compile and run high performance
| custom code out of customers ' shifting piles of random
| python packages.
|
| AMD has continuously struggled at (2) and hasn't seemed to
| recommit to doing it properly. AMD certainly has silicon
| design expertise, but given (1) I don't think that is enough.
|
| Xilinix is in interesting alternative path for products or
| improving AMDs software/devex. I'm not sure what to expect
| from that, yet.
| jacooper wrote:
| AMD GPUs are a joke for anything but gaming
| sofixa wrote:
| Yet. Their GPUs were not great for years, but have managed
| to catch up (and even overperform for their price point),
| so other workloads are yet to come.
| Gigachad wrote:
| Still waiting for rocm support on the 5700xt which they
| kept promising was ready any day now.
| jacooper wrote:
| Unfortunately with the way ROCm is developed, and how its
| only intended for specific GPUs, I doubt it.
| danpalmer wrote:
| At the moment maybe, but owning the CPU may allow for
| better integration.
|
| Imagine if AMD launch a new bus implementation from CPU to
| GPU. That's not something Nvidia can do by themselves.
| Maybe Nvidia buys Intel and does it though!
| imwithstoopid wrote:
| Nvidia is in an excellent position - they have CUDA as you
| point out, and they are moving that into server room
| dominance in this application space
|
| Google has TPUs but have these even made a tiny dent in
| Nvidia's position?
|
| I assume anything Apple is cooking is using Nvidia in the
| server room already
|
| Intel seems completely absent from this market
|
| AMD seems content to limit its ambitions to punching Intel
|
| its Nvidia's game to lose at this point...I wonder when they
| start moving in the other direction and realize they have the
| power to introduce their own client platform (I secretly wish
| they would try to mainstream a linux laptop running on Nvidia
| ARM but obviously this is just a fantasy)
|
| if anything, I think Huang may not be ambitious enough!
| losteric wrote:
| > I assume anything Apple is cooking is using Nvidia in the
| server room already
|
| For training, sure. For inference, Apple has been in a
| solid competitive position since M1. LLaMa, Stable
| Diffusion, etc, can all run on consumer devices that my
| tech-illiterate parents might own.
| smoldesu wrote:
| LLaMa and Stable Diffusion will run on almost any device
| with 4gb of free memory.
| rolenthedeep wrote:
| > AMD seems content to limit its ambitions to punching
| Intel
|
| What's the deal with that anyway? A lot of people want a
| real alternative to Nvidia, and AMD just... Doesn't care?
|
| I guess we'll have to wait for intel to release something
| like CUDA and _then_ AMD will finally do something about
| the GPGPU demand.
| roenxi wrote:
| I was wondering the same thing and thinking about it.
|
| When AMD bought ATI they viewed the GPU as a potential
| differentiator on CPUs. They've invested a lot of effort
| into CPU-GPU fusion with their APU products. That has the
| potential to start paying off in a big way sometime -
| especially if they figure our how to fuse high end GPU
| and CPU and just offer a GPGPU chip to everyone. I can
| see why AMD might put their bets here.
|
| But the trade off was that Nvidia put a lot of effort in
| doing linear algebra quickly and easily on their GPUs and
| AMD doesn't have a response to that. Especially since
| they probably strategised on BLAS on an APU. But it turns
| out there were a lot of benefits to fast BLAS and Nvidia
| is making all the money from that.
|
| In short, Nvidia solved a simpler problem that turned out
| to be really valuable, it would take AMD a long time to
| organise to do the same thing and it may be a misfit in
| their strategy. Hence ROCm sucks and I'm not part of the
| machine learning revolution. :(
| potatolicious wrote:
| > _" Google has TPUs but have these even made a tiny dent
| in Nvidia's position?"_
|
| This seems unknowable without Google's internal data. The
| salient question is: "how many Nvidia GPUs would Google
| have bought if they didn't have TPUs?"
|
| The answer is probably "a lot", but realistically we don't
| know how many TPUs are deployed internally and how many
| Nvidia GPUs it displaced.
| paulmd wrote:
| Tesla and the Dojo architecture is another interesting
| one - that's another Jim Keller project and frankly Dojo
| may be a little underappreciated given how everything
| Keller touches tends to turn into gold.
|
| https://www.nextplatform.com/2022/08/23/inside-teslas-
| innova...
|
| Much like Google, I think Tesla realized this is a
| capability they need, and at the scales they operate,
| it's cheaper than buying a whole bunch of NVIDIA product.
| jitl wrote:
| Tegra & later Shield were attempts to get closer to full
| end user platform. The Nintendo Switch is their most
| successful such device -- with a 2-year old Tegra SKU at
| launch. But going full force into consumer tech is a
| distraction for them right now. Even the enthusiast
| graphics market, which should be high margin, is losing
| their interest. They make much more selling to the big
| enterprise customer CEO Jensen mentions in the open
| paragraph.
| echelon wrote:
| Gamers are going to be so pissed. They subsided the
| advance in GPU compute and will now be ignored for the
| much more lucrative enterprise AI customers.
|
| Nvidia is making the right call, of course.
| anonylizard wrote:
| Have they ever considered that the subsidy goes the other
| way? The margins on an A100 card is probably 100% higher
| than a RTX4090. Gaming industry is also like THE first
| industry to be revolutionized by AI. Current stuff like
| DLSS and AI-accelerated path tracing are mere toys
| compared to what will come.
|
| Nvidia will not give up gaming. When every gamer has a
| Nvidia card, every potential AI developer to spring up
| from those gamers, will use Nvidia by default. It also
| helps gaming GPUs are still lucrative.
| dleslie wrote:
| It's OK, gaming is also having its AI moment.
|
| I fully expect future rendering techniques to lean
| heavily on AI for the final scene. NeRF, diffusion
| models, et cetera are the thin end of the wedge.
| smoldesu wrote:
| Gamers are in heaven right now. Used 30-series cards are
| cheap as dirt, keeping the pressure on Intel/AMD/Apple to
| price their GPUs competitively. The 40-series cards are a
| hedged bet against anything their competitors can develop
| - manufactured at great cost on TSMC's 4nm node and
| priced out-of-reach for most users. Still, it's clear
| that Nvidia isn't holding out their best stuff, just
| charging exorbitant amounts for it.
| layoric wrote:
| Where are these cheap as dirt 30 series? A 10gb 3080 is
| still over $500 usd used ($750 aud) when I've looked.
| When did secondhand GPUs that still cost the same as a
| brand new PS5 start to be considered cheap?
| my123 wrote:
| A PS5 is _significantly_ slower than a 3080, it's more
| RTX 2070 tier.
| capableweb wrote:
| > I assume anything Apple is cooking is using Nvidia in the
| server room already
|
| I wouldn't be so quick at assuming this. Apple already ship
| ML-capable chips in consumer products, and they've designed
| and built revolutionary CPUs in modern time. I'm of course
| not sure about it, but I have a feeling they are gonna
| introduce something that kicks up the notch on the ML side
| sooner or later, the foundation for doing something like
| that is already in place.
| imwithstoopid wrote:
| Apple has no present experience in building big servers
| (they had experience at one point, but all those people
| surely moved on)
|
| Mac Minis don't count
|
| Sure, they are super rich and could just buy their way
| into the space...but so far they are really far behind in
| all things AI with Siri being a punchline at this point
|
| if anything, Apple proves that money alone isn't enough
| capableweb wrote:
| I'm no Apple fan-boy at all (closer to the opposite) so
| it pains me a bit to say, but they have a proven track-
| record of having zero experience in something, then
| releasing something really good in that industry.
|
| The iPhone was their first phone, and it really kicked in
| the smartphone race into high gear. Same for the Apple
| Silicon processor. And those are just two relatively
| recent examples.
| danieldk wrote:
| The iPhone had a lot of prehistory in Apple, from Newton
| to iPod. Apple Silicon alo has a long history, starting
| with the humble beginnings as the Apple A4 in 2010, which
| relied on Samsung's Hummingbird for the CPU and PowerVR
| for the GPU (plus they acquired PA Semi in 2008).
|
| So both are not very good examples, because they build up
| experience over long periods.
| capableweb wrote:
| > So both are not very good examples, because they build
| up experience over long periods.
|
| They are examples of something they could similarly do
| for the Apple Neural Engine but in a bigger scale in the
| future. They have experience deploying it in a smaller
| scale/different versions, they would just have to apply
| it in bigger scale in order to be able to compete with
| NVIDIA.
| rchiang wrote:
| To be fair, Apple released their iPhone after building
| iPods for 6 years. So, it's not like they had zero
| experience with handheld devices at the time.
|
| Also, while Apple did create their first chip (at least
| of their current families) in 2007, they did acquire 150
| or so engineers when they bought PA Semi in 2008. So,
| that gave them a leg up compared to building a chip team
| completely from scratch.
| newsclues wrote:
| I assumed their server experience is still working in the
| iCloud division.
| smoldesu wrote:
| > Apple already ship ML-capable chips in consumer
| products, and they've designed and built revolutionary
| CPUs in modern time.
|
| Has Nvidia not done that too? They shipped ML-capable
| consumer hardware before Apple, and have revolutionary
| SOCs of their own. On top of that, they have a working
| relationship with the server/datacenter market (something
| Apple burned) and a team of researchers that basically
| wrote the rulebook on modern text and image generation.
| Then you factor in CUDA's ubiquity - it runs in cars,
| your desktop, your server, your Nintendo Switch - Nvidia
| is terrifying right now.
|
| If the rest of your argument is a feeling that Apple will
| turn the tables, I'm not sure I can entertain that
| polemic. Apple straight-up doesn't compete in the same
| market segment as Nvidia anymore. They cannot release
| something that seriously threatens their bottom line.
| dividedbyzero wrote:
| > They cannot release something that seriously threatens
| their bottom line.
|
| If they manage to move a significant part of ML compute
| from datacenter to on-device, and if others follow, that
| might hurt Nvidia's bottom line. Big if at this point,
| but not unthinkable.
| smoldesu wrote:
| There are a lot of problems here though. First of all
| being that inferencing isn't hard to do - iPhones were
| capable of running LLMs before LLaMa and even before it
| was accelerated. _Anyone_ can inference a model if they
| have enough memory, I think Nvidia is banking on that
| part.
|
| Then there's the issue of model size. You can fit some
| pruned models on an iPhone, but it's safe to say the
| majority of research and development is going to happen
| on easily provisionable hardware running something
| standard like Linux or FreeBSD.
|
| And all this is ignoring the little things, too; training
| will still happen in-server, and the CDN required to
| distribute these models to a hundred million iPhone users
| is not priced attractively. I stand by what I said -
| Apple forced themselves into a different lane, and now
| Nvidia is taking advantage of it. Unless they intend to
| reverse their stance on FOSS and patch up their burned
| bridges with the community, Apple will get booted out of
| the datacenter like they did with Xserve.
|
| I'm not against a decent Nvidia competitor (AMD is
| amazing) but the game is on lock right now. It would take
| a fundamental shift in computing to unseat them, and AI
| is the shift Nvidia's prepared for.
| KeplerBoy wrote:
| why wouldn't they build a relatively small cluster for
| training tasks using Nvidia hardware? It's simply the
| industry standard, every researcher is familiar with it
| and writing a custom back-end for pytorch that scales to
| hundreds of nodes is no small task.
|
| I doubt Apple cares about spending a few hundred million
| dollars on A100s as long as they make sure the resulting
| models run on billions of apple silicone chips.
| danpalmer wrote:
| > I assume anything Apple is cooking is using Nvidia in the
| server room already
|
| I don't think Apple's server side is big or interesting.
| Far more interesting is the client side, because it's 1bn
| devices, and they all run custom Apple silicon for this.
| Similarly Google has Tensor chips in end user devices.
|
| Nvidia doesn't have a story for edge devices like that, and
| that could be the biggest issue here for them.
| bottlepalm wrote:
| They've been investing for a long time, but it's only blown
| up in the past year due to recent breakthroughs. Good timing
| for Nvidia.
| jonas21 wrote:
| It's been blowing up since at least 2012-13 when deep
| convolutional neural nets started seeing massive success.
| bottlepalm wrote:
| That's not blowing up. What's happening right now is
| blowing up.
| jonas21 wrote:
| What's going on now is the continuation of a growth trend
| that started a decade ago.
| jitl wrote:
| They nerfed hash rate on their cards multiple times
| selectodude wrote:
| And drug manufacturers spend a lot of money to nerf their
| medications to make them harder to inject. Not all customers
| are good customers.
| ww520 wrote:
| It's not just luck but good strategy. In the past 10 to 15
| years, Nvidia has been leveraging its core GPU to go beyond
| gaming video card to expand into different peripheral areas
| where massive parallel computing is needed, such as super
| computing, cloud computing, animation farm, CAD, visualization,
| simulation, car, VR, AI, and Crypto. They have been able to
| catch/enable one wave or the other because it's part of their
| roadmap.
| tpmx wrote:
| They have been going pretty much everywhere with their CUDA
| runtime. LLMs was a random hit.
|
| At the same time, it doesn't seem like a great moat - I think
| AMD should be able compete pretty soon.
___________________________________________________________________
(page generated 2023-03-25 23:00 UTC)