[HN Gopher] Nvidia H100 and A100 GPUs - comparing available capa...
       ___________________________________________________________________
        
       Nvidia H100 and A100 GPUs - comparing available capacity at GPU
       cloud providers
        
       Author : tikkun
       Score  : 20 points
       Date   : 2023-06-14 21:56 UTC (1 hours ago)
        
 (HTM) web link (llm-utils.org)
 (TXT) w3m dump (llm-utils.org)
        
       | nkingsy wrote:
       | People talk about cards not being worth the electricity vs cloud.
       | Seems like an a100 pulls 300w, costs $1.50/hr ish to rent, and
       | costs $12,000 to buy, meaning it pays for itself with 1 year of
       | constant use.
        
       | jpgvm wrote:
       | The only think I am happy about all this AI hype is Infiniband is
       | getting some love again. A lot of people using RoCE on Connect-X
       | HBAs but still a lot of folk doing native IB. If HPC becomes more
       | commonplace maybe we get better subnet managers, IB routing, i.e
       | all the stuff we were promised ~10+ years ago that never had a
       | chance to materialise because HPC became so niche and the
       | machines had different availability etc requirements than OLTP
       | systems that didn't demand that stuff getting built out.
       | Especially the subnet managers as most HPC cluster just compute a
       | static torus or clos-tree topology.
       | 
       | There was a time I was running QDR Infiniband (40G) at home while
       | everyone else was still dreaming of 10G at home because the
       | adapters and switches were so expensive.
        
       ___________________________________________________________________
       (page generated 2023-06-14 23:00 UTC)