[HN Gopher] Show HN: Software for Remote GPU-over-IP ___________________________________________________________________ Show HN: Software for Remote GPU-over-IP We built installable software for Windows & Linux that makes any remote Nvidia GPU accessible to, and shareable across, any number of remote clients running local applications, all over standard networking. Author : stevegolik Score : 44 points Date : 2022-12-14 16:10 UTC (6 hours ago) (HTM) web link (github.com) (TXT) w3m dump (github.com) | allanrbo wrote: | It surprises me that this works well enough to be useful. I would | have thought that network latency, being orders of magnitude | higher than memory latency, would be a huge problem. Latency | Numbers Everyone Should Know: | https://static.googleusercontent.com/media/sre.google/en//st... | capableweb wrote: | For gaming, this is obviously a no-go. But for bunch of AI/ML | related workloads, it might make perfect sense. | bob1029 wrote: | Not so sure about no-go. The amount of GPU latency in modern | AAA titles already approaches 20+ms in the most egregious | cases. | | Unless there is a need to evict all gpu memory on every | frame, I think it is feasible to game on GPUs that live | across a very fast LAN. | Melatonic wrote: | Fast ethernet is getting cheaper than ever - you can easily | get 10gb on consumer gear or even 20 and used hardware on I | believe 40 or maybe 100 is getting pretty affordable. | delijati wrote: | PCI-Express 16x 4.0 has 31,5 GByte/s. The fastest fiber ETH has | 50 GB/s. So it "could" be useful if you have datacenter grade | equipment ;) | AnIdiotOnTheNet wrote: | Those aren't latency numbers though, they're throughput. | [deleted] | cobertos wrote: | I'd be surprised if this works for anything latency sensitive | over anything more than a LAN. | | Even just the time it takes speed of light between NY and LA (4 | _10^6m /3_10^8m/s=1/75s) is roughly how long a 60 fps frame is | (1/60s). Add OS serializing the frame from the GPU onto the | network card, network switching of those packets, and you're | starting to really feel that latency. | denkmoon wrote: | There are people out there gaming at 30fps with their TV set | to Super Duper Image Processing Mode 500ms Latency Edition. | Though I suppose these are realistically already served by | the cloud gaming offerings. | johanvts wrote: | The datacenter is probably not thousands, but hundreds of | kilometers away so there is room to deliver 60fps. I was | surprised how well GeForce Now works. | xrd wrote: | I see lots of comments in various ML repositores about trouble | running on multiple GPUs. This seems like a great way to run | across multiple low VRAM GPUs instead of buying a huge expensive | single card. It feels reminiscent of how Google built their | clusters on commodity hardware where they would just throw away a | failed device rather than trying to fix it. This is really cool. | fock wrote: | Didn't we have those things already? Virtual-GL and Co. say hi. | | Also for most real GPU applications, you need to get the data in | and out. I don't think splitting compute across a (insert any | non-Infiniband-link) solves this | Melatonic wrote: | 100gbe is pretty similar to infiniband no? Or does infiniband | still kill it on latency? | zamadatix wrote: | That's really awesome. I'm not sure what I'd use it for but just | being able to makes me want to find an excuse! What's impressive | is this seems to have more capabilities than most "local" | software vGPU solutions for e.g. VMs. | nimitt wrote: | Do you have any numbers on the viability of using this for ML/AI | workloads? seems like once a model is ingested into a gpu vram | theoretically the transactional new inputs / outputs would be | trivial. | Mo3 wrote: | Damn, this is cool. Nice work. | dezmou wrote: | does it really feel like the GPU I use is one on my machine ? or | do I have lot of boilerplate to make it work client side ? | neuronexmachina wrote: | I haven't tried it yet, but based on their doc it seems like | after setting the host in the juice.cfg, you basically just | need to run `juicify [application path]`: | https://github.com/Juice-Labs/Juice-Labs/wiki/Juice-for-Wind... | yangikan wrote: | Very nice. ___________________________________________________________________ (page generated 2022-12-14 23:01 UTC)