[HN Gopher] Nvidia Grace CPU
___________________________________________________________________
Nvidia Grace CPU
Author : intull
Score : 316 points
Date : 2022-03-22 17:08 UTC (5 hours ago)
(HTM) web link (www.nvidia.com)
(TXT) w3m dump (www.nvidia.com)
| [deleted]
| [deleted]
| kcb wrote:
| Given how larger non-mobile chips are jumping to the LPDDR
| standard what is the point of having a separate DDR standard? Is
| there something about LPDDR5 that makes upgradable dimms not
| possible?
| [deleted]
| monocasa wrote:
| > Is there something about LPDDR5 that makes upgradable dimms
| not possible?
|
| It's theoretically possible, but there's no standard for it.
| wmf wrote:
| AFAIK the higher speed of LPDDR is directly because it avoids
| signal degradation caused by DIMM connectors.
| andrewstuart wrote:
| This leads me to wonder about the microprocessor shortage.
|
| So many computing devices such as Nvidia Jetson and Raspberry Pi
| are simply not available anywhere. I wonder what's he point of
| bringing out new products when existing products can't be
| purchased? Won't the new products also simply not be available?
| frozenport wrote:
| What? They are sold out, not "can't be purchased".
| aftbit wrote:
| What's the difference? If they are perpetually sold out, then
| they cannot be purchased.
| singlow wrote:
| There is constant production and deliveries being made,
| just no standing inventory.
| Gigachad wrote:
| The products don't get produced in order. The high value
| products get priority and continuously bump out low value chips
| like those on the RPI. Not sure what the cost of this Grace
| chip is but it looks to be targeting high value users so it
| gets priority. Notice how there is no shortage of chips for
| iPhones, because Apple just buys the capacity at whatever cost
| it takes.
| arebop wrote:
| Though, there is a shortage of m1 MacBooks. Is it really
| because they are low value (margin?) products relative to
| iPhone? I'm not sure.
| Gigachad wrote:
| Not much of a shortage. I just checked and they are all
| available for pickup right now at my local small city
| store. Compared to other products they are still extremely
| available.
| arebop wrote:
| Interesting, I see nothing available from store.apple.com
| until 6 April earliest, and 29 April for m1 max and even
| later depending on options.
| p1necone wrote:
| I was pretty surprised by the low prices of m1 macbooks
| when even the lowest end models perform so much better than
| the high end of previous models. I'm sure Apple is spending
| less money on manufacturing them now that they're not going
| through Intel, but I would have expected them to just keep
| charging the same and eaten the profit margin themselves.
| eftychis wrote:
| They are trying to establish the new architecture. Also
| you still need to shell out $2-3k to get something decent
| and practically start at $1.5k. I wouldn't call that
| cheap or even cheaper. What is the past difference you
| see?
| p1necone wrote:
| > still need to shell out $2-3k to get something decent
| and practically start at $1.5k
|
| They're all using the exact same CPU, in fact you can
| make the air perform (almost) just as well as the
| pro/mini by opening it up and adding a thermal pad:
| https://www.cultofmac.com/759693/thermal-mod-m1-macbook-
| air/
| rsynnott wrote:
| Starting price for the air is $999, which gets you a very
| fast computer (albeit one a bit anemic in memory). A
| couple of years ago, the starting price for the air was
| still $999, but you got a... much less fast computer.
| 20220322-beans wrote:
| What are people's experience of developing with NVIDIA? I know
| what Linus thinks: https://www.youtube.com/watch?v=iYWzMvlj2RQ
| jlokier wrote:
| I had a laptop with NVIDIA GPU that crashed Xorg and had to be
| rebooted whenever Firefox opened WebGL. Just to complement the
| positive sibling comments :-)
| nl wrote:
| Nvidia's AI APIs are well documented and supported. That's why
| everyone uses them.
| dekhn wrote:
| over the past two decades that I've used nvidia products for
| opengl and other related things, my experince has been largely
| positive although I find installing both the dev packages and
| the runtimes I need to be cumbersome.
| pjmlp wrote:
| Linus might know his way around UNIX clones and SCM systems,
| however he doesn't do graphics.
|
| NVidia tooling is the best among all GPU vendors.
|
| CUDA has been polyglot since version 3.0, you get proper IDE
| and GPGPU debugging tools, and a plethora of libraries for most
| uses cases one could think of using a GPGPU for.
|
| OpenCL did not fail only because of NVidia not caring, Intel
| and AMD have hardly done anything with it that could compete on
| the same tooling level.
| dsign wrote:
| I like CUDA, that stuff works and is rewarding to use. The only
| problem is the tons and tons of hoops one must jump to use it
| in servers. Because a server with a GPU is so expensive, you
| can't just rent one and have it running 24x7 if you don't have
| work for it to do, so you need a serverless or auto-scaling
| deployment. That increases your development workload. Then
| there is the matter of renting a server with GPU; that's still
| a bit of a specialty offering. Until the other day, even major
| cloud providers (i.e. AWS and Google) offered GPUs only in
| certain datacenters.
| valine wrote:
| Anyone have a sense for how much these will cost? Is this more
| akin to the Mac Studio that costs 4k or an A100 gpu that costs
| upward of 30k? Looking for an order of magnitude.
| IshKebab wrote:
| Probably on the order of $100k.
| valine wrote:
| That would be a real shame. I really want someone to make a
| high core count ARM processor in the price range of an AMD
| threadripper that can work with Nvidia gpus.
| freemint wrote:
| Look into Ampere they have 256 core and 160 core dual
| socket systems for decent prices
| https://store.avantek.co.uk/ampere.html .
| wmf wrote:
| Ampere Altra?
| wmf wrote:
| Compare a 72C Grace against an 80C Ampere Altra which is priced
| at $4K (without RAM).
| naikrovek wrote:
| This is definitely not a consumer-grade device, like a Mac
| Studio.
| Hamuko wrote:
| Considering that the URL is "/data-center/grace-cpu/", assume
| much more than a Mac Studio.
| oofbey wrote:
| The top-end datacenter GPUs have been slowly creeping up from
| $5k a few generations back to about $15k for the A100's now. So
| this one will probably continue the trend, probably to $20k or
| maybe $30k but probably not beyond that.
| rsynnott wrote:
| > NVIDIA Grace Hopper Superchip
|
| Finally, a computer optimised for COBOL.
| ksec wrote:
| This is interesting. So without actually targeting a specific
| Cloud / server market for their CPU, which often ends with a
| chicken and egg problem with HyperScaler making their own Design
| or Chip. Nvidia manage to enter the Server CPU market leveraging
| their GPU and AI workload.
|
| All of a sudden there is real choice of ARM CPU on Server. ( What
| will happen to Ampere ? ) The LPDDR5X used here will also be the
| first to come with ECC. And they can cross sell those with
| Nvidia's ConnectX-7 SmartNICs.
|
| Hopefully it will be price competitive.
|
| Edit: Rather than downvoting may be explain why or what you
| disagree with ?
| messe wrote:
| I wonder if Apple also intends to introduce ECC LPDDR5 on the
| Mac Pro. Other than additional expansion, I'm struggling to see
| what else they can add to distinguish it from the Mac Studio.
| MBCook wrote:
| More cores and more RAM is really kind of it. I guess PCIe
| but I'm kind of wondering if they'll do that.
| lostlogin wrote:
| And more worryingly, will a GPU function in the slots.
|
| The questions everyone has, Ram and GPU.
| belval wrote:
| AWS Graviton aren't toys, they work pretty well for a wide
| range of workloads
| [deleted]
| didip wrote:
| heh, does Intel have any chance to catch up? They fell so far
| behind.
| wmf wrote:
| There are some hints that they are redesigning some server
| processors to double core count but that may not be visible for
| 2-3 years. Also keep in mind that Intel has 75% server market
| share and is only losing ~5 points per year.
| hughrr wrote:
| No. Intel worked out it needs to open its production capacity
| to other vendors. They will end up another ARM fab with a
| legacy x86-64 business strapped on the side. That's probably
| not a bad place to be really. I think x86-64 will fizzle out in
| about a decade.
| qbasic_forever wrote:
| I really don't see what they can do. It seems like in the last
| year they pivoted hard into "ok we'll build chips in the US
| again!", but it's going to be years and years before any of
| that pays off or even materializes. The only announcements I've
| heard from them are just regular "Here's the CEO of Intel
| telling us how he's going to fix Intel" PR blurbs and nothing
| else. Best case maybe they just position themselves to be
| bought by Nvidia...
| bullen wrote:
| I think we're all missing the forest because all the cores are in
| the way:
|
| The contention on that memory means that only segregated non-
| cooporative as in not "joint parallel on the same memory atomic"
| will scale on this hardware better than on a 4-core vanilla Xeon
| from 2018 per watt.
|
| So you might aswell buy 20 Jetson Nanos and connect them over the
| network.
|
| Let that sink in... NOTHING is improving at all... there is ZERO
| point to any hardware that CAN be released for eternity at this
| point.
|
| Time to learn JavaSE and roll up those sleves... electricity
| prices are never coming down (in real terms) no matter how high
| the interest rate.
|
| As for GPUs, I'm calling it now: nothing will dethrone the 1030
| in Gflops/W in general and below 30W in particular; DDR4 or DDR5,
| doesn't matter.
|
| Memory is the latency bottleneck since DDR3.
|
| Please respect the comment on downvote principle. Otherwise you
| don't really exist; in a quantum physical way anyway.
| ribit wrote:
| 1030 has been dethroned a while ago. Apple G13 delivers
| 260GFLOPS/W in a general-purpose GPU. I mean, their phone has
| more GPU FLOPS than a 1030.
| [deleted]
| simulate-me wrote:
| Performance per watt isn't so useful for a GPU. People training
| ML algorithms would gladly increase power consumption if they
| could train larger models or train models faster.
| bullen wrote:
| And that's exactly my point: they can't. Power does not solve
| contention and latency! It's over, permanently... (or atleast
| until some photon/quantum alternative, which honestly we
| don't have the energy to imagine, let alone manufacture,
| anymore)
| cma wrote:
| Aren't you are ignoring use cases where all cores read shared
| data, but rarely contentiously write to it. You should get much
| more read bandwidth and latency than over a network.
| bullen wrote:
| Sure, but my point is: why cram more and more cores into the
| same SoC if they can't talk to each other more efficiently
| than separate computers over ethernet?
| luxuryballs wrote:
| Reading this makes a veteran software developer want to become a
| scientific researcher.
| bitwize wrote:
| IKR? Imagine a Beowulf cluster of these...
| nlh wrote:
| Slashdot flashbacks from 2001! Well played. Well played.
| wmf wrote:
| We call it a "SuperPOD" now apparently.
| fennecfoxen wrote:
| https://www.nvidia.com/en-us/data-center/dgx-superpod/
| stonogo wrote:
| I don't think you'll have to imagine. It says on the box it's
| designed for HPC. and every supercomputer in the Top 500 has
| been a Beowulf cluster for years now.
| melling wrote:
| Way too late for me. I think adding machine learning to my
| toolbox at least gets me knowledgeable.
|
| https://www.kaggle.com/
|
| When Jensen talks about Transformers, I know what he's talking
| about because I follow a lot of talented people.
|
| https://www.kaggle.com/code/odins0n/jax-flax-tf-data-vision-...
| donatj wrote:
| Maybe it's just me, but it's just cool to see the CPU market
| competitive again for the first time since the late 90s.
| sedatk wrote:
| You're not alone.
| andrewstuart wrote:
| I wonder why Intel never had a really good go at GPU's? It
| seems strange, given the demand.
| _delirium wrote:
| Besides integrated GPUs for actual graphics usage that other
| comments mentioned, Intel did make some attempts at the GPGPU
| market. They had a design for a GPU aimed primarily at GPGPU
| workloads, Larrabee, that was never released [1], and adapted
| some of the ideas into Xeon Phi, a more CPU-like chip that
| was intended to be a competitor to GPUs, which was released
| but didn't gain a lot of market share [2].
|
| [1]
| https://en.wikipedia.org/wiki/Larrabee_(microarchitecture)
|
| [2] https://en.wikipedia.org/wiki/Xeon_Phi
| nine_k wrote:
| Intel produced good, as in "cheap and always working",
| integrated GPUs. For great many tasks, they are adequate. I'm
| not a gamer, and if I needed to run some ML stuff, my
| laptop's potential discrete GPU won't be much help anyway.
| liotier wrote:
| Also, Intel has a history of producing or commissioning
| open-source drivers for its GPU. I like the peace of mind I
| get from knowing I'm not going to have to fight dirty for
| the privilege of getting my own GPU to do the work I bought
| it to perform.
| bduerst wrote:
| Intel also announced a new GPU offering, supposed to drop in
| 8 days:
|
| https://www.intel.com/content/www/us/en/architecture-and-
| tec...
|
| https://en.wikipedia.org/wiki/Intel_Arc
| tyrfing wrote:
| Discrete GPUs have historically been a relatively small and
| volatile niche compared to CPUs, it's only in the last few
| years that the market has seen extreme growth.
|
| edit: the market pretty much went from gaming as the primary
| pillar to gaming + HPC, which makes it far more attractive
| since you'd expect it to be much less cyclical and less price
| sensitive. Raja Koduri was hired in late 2017 to work on GPU
| related stuff, and it seems like the first major products
| from that effort will be coming out this year. That said,
| they've obviously had a lot of failures in the acelerator and
| graphics area (consider Altera) and Koduri has stated on
| Twitter that Gelsinger is the first CEO to actually treat
| graphics/HPC as a priority.
| marcodiego wrote:
| Time to sell intel shares?
| bloodyplonker22 wrote:
| That time was years and years ago. If you're just thinking
| about it now, you're already in a world of pain.
| namlem wrote:
| Intel stock is up 37% from 5 years ago. Though this past year
| they took quite a beating.
| aabceh wrote:
| This is really not that much considering how much every
| stock has gone up the last couple of years. Nvidia and AMD
| is up 887% and 737% respectively from 5 years ago.
| t0mas88 wrote:
| How likely is it that one of AWS / GCP / Azure will deploy these?
| Nvidia has some relationships there for the A100 chips.
| qbasic_forever wrote:
| Amazon has at least two generations of their own homebrew ARM
| chip, the Graviton. They offer it for people to rent and use in
| AWS, and publicly stated they are rapidly transitioning their
| internal services to use it too. In my experience Graviton 2 is
| much cheaper than x86 for typical web workloads--I've seen
| costs cut by 20-40% with it.
| devmunchies wrote:
| > their own homebrew ARM chip
|
| are they going through TSMC like NVIDIA or are they using
| Samsung?
| lmeyerov wrote:
| AWS+Azure (and I believe GCP) installed prev advances, and are
| having huge GPU shortages in general... so probably!
|
| An interesting angle here is these support partitioning even
| better than in the A100's. AFAICT, the cloud vendors are not
| yet providing partitioned access, so everyone just exhausts
| worldwide g4dn capacity for smaller jobs / devs / etc. But
| partitioning can solve that...
| ksec wrote:
| AWS has their own CPU. Microsoft is an investor in Ampere, but
| I am not sure if they will make one themselves or simply buy
| from Ampere. Google has responded with faster x86 instances,
| still no hint of their own ARM CPU. But judging from the past I
| dont think they are going to go with Nvidia.
|
| That is only the CPU though, they might deploy it as Grace +
| Hopper config.
| ciphol wrote:
| With names like that, I assume that was the intention
| KaoruAoiShiho wrote:
| Pretty sure they all will, they all already have the past gens
| of these things and it's a simple upgrade.
| donkeydoug wrote:
| soooo... would something like this be a viable option for a non-
| mac desktop similar to the 'mac studio' ? def seems targeted at
| the cloud vendors and large labs... but it'd be great to have a
| box like that which could run linux.
| opencl wrote:
| It's viable in the sense that you can just stick a server
| motherboard inside of a desktop case. It certainly won't be
| cheap though.
|
| This has been done as a commercial product with the Ampere ARM
| server chips. The base model is about $8k.
|
| https://store.avantek.co.uk/arm-desktops.html
| wmf wrote:
| Nvidia Orin would be a better fit for an ARM desktop/laptop but
| Nvidia seemingly isn't interested in that market.
| my123 wrote:
| It's a server CPU that runs any OS really (Arm SystemReady with
| UEFI and ACPI).
|
| However, the price tag will be too high for a lot of desktop
| buyers.
|
| (There are smaller Tegras around though)
| oneplane wrote:
| It probably won't run Windows. But other operating systems,
| probably yes. Maybe Microsoft comes up with some sort of
| Windows Server DC Arm edition in the future so they can join
| in as well.
| my123 wrote:
| Modern Tegras can boot arm64 Windows. But yeah without a
| licensable Windows Server arm64 SKU, practical uses are
| quite limited.
| GIFtheory wrote:
| Interesting that this has 7x the cores of a M1 Ultra, but only
| 25% more memory bandwidth. Those will be some thirsty cores!
| wmf wrote:
| The M1 memory bandwidth is mostly for the GPU but Grace does
| not include an (on-chip) GPU.
| my123 wrote:
| https://twitter.com/benbajarin/status/1506296302971334664?s=...
|
| 396MB of on-chip cache... (198MB per die)
|
| That's a significant part of it too.
| Teknoman117 wrote:
| The CPU complex on the M1 series doesn't have anything close to
| the full bandwidth to memory that the SoC has (like, half). The
| only thing that can drive the full bandwidth is the GPU.
| ZetaZero wrote:
| M1 Ultra bandwidth is for CPU and GPU (800GB/s). Grace is just
| the CPU. Hopper, the GPU, has it's own memory and bandwidth (3
| TB/sec).
| oofbey wrote:
| NVIDIA continues to vertically integrate their datacenter
| offerings. They bought mellanox to get infiniband. They tried to
| buy ARM - that didn't work. But they're building & bundling CPUs
| anyway. I guess when you're so far ahead on the compute side,
| it's all the peripherals that hold you back, so they're putting
| together a complete solution.
| DeepYogurt wrote:
| Nvidia's been making their own CPUs for a long time now. IIRC
| the first tegra was used in the Zune HD back in 2009. Hell
| they've even tried their hand at their own cpu core designs
| too.
|
| https://www.anandtech.com/show/7621/nvidia-reveals-first-det...
|
| https://www.anandtech.com/show/7622/nvidia-tegra-k1/2
| my123 wrote:
| Tegra X2 and Xavier are still sold today and contain NVIDIA-
| designed CPU cores. The team behind those is building new
| designs too, I wonder when they're going to announce
| something.
| azzentys wrote:
| Orin
| my123 wrote:
| Orin uses the Cortex-A78AE core for the CPU complex
| instead of NVIDIA-designed cores.
| azzentys wrote:
| Ah, you meant like that. I assumed if they're bringing a
| new module architecture.
| 015a wrote:
| Maybe even more importantly: Tegra powers the Nintendo
| Switch.
| TazeTSchnitzel wrote:
| Note the CPU cores in that design aren't designed by
| NVidia.
| ggreg84 wrote:
| Which is (EDIT: NOT) the most widely sold console ever.
| fazzyfoo wrote:
| Not by a long shot.
|
| PS2 and DS outsell by about 50 million units.
| nickelcitymario wrote:
| "PS2? That can't possibly be right..."
|
| https://www.vgchartz.com/analysis/platform_totals/
|
| Holay molay.
| overtonwhy wrote:
| It was the most affordable DVD player. I think Sony owned
| patents on some DVD player tech? Same with PS4/5 and Blu
| Ray if I'm remembering correctly
| 015a wrote:
| This was also kind of the case with the PS3. Its sales
| weren't fantastic at release, partially because of its...
| $600 (?) price tag. But even at that price, at its
| release, it was one of the cheapest ways to get a Blu-ray
| player, and many people bought it for that.
| BolexNOLA wrote:
| If memory serves, there was less than 1 game per PS3 sold
| at launch.
| genewitch wrote:
| Not just a Blu-ray player, but one that is guaranteed to
| be able to play practically all blu-ray discs as long as
| Blu ray discs are made or the console hardware fails.
|
| Sony pushed updates to the firmware. Most commodity Blu
| ray players don't have an (easy) way to update.
| ggreg84 wrote:
| Indeed, wow.
| shawnthye wrote:
| cjensen wrote:
| "Grace?"
|
| After 13 microarchitectures given the last names of historical
| figures, it's really weird to use someone's first name.
| Interesting that Anandtech and Wikipedia are both calling it
| Hopper. What on Earth are the marketing bros thinking?
| fay59 wrote:
| They also made the "Hopper" architecture to complement it.
| thereddaikon wrote:
| The GPU is Hopper, which is in line with their naming scheme up
| till now. The CPU is call Grace. Clearly they are planning to
| continue the tradition of naming their architectures after
| famous scientists and the CPUs will take on the first name
| while the GPU will continue to use last.
|
| So expect a future Einstein GPU to come with a matching Albert
| CPU.
| [deleted]
___________________________________________________________________
(page generated 2022-03-22 23:00 UTC)