[HN Gopher] DiLoCo: Distributed Low-Communication Training of La... ___________________________________________________________________ DiLoCo: Distributed Low-Communication Training of Language Models Author : Anon84 Score : 14 points Date : 2023-12-06 20:55 UTC (2 hours ago) (HTM) web link (arxiv.org) (TXT) w3m dump (arxiv.org) | lucubratory wrote: | So we can SETI@Home or Folding@Home for large language models, | now? Not sure how small the minimum size of the compute cluster | can be. If it's still out of consumer reach then this would | either be just an intermediate research step, or a way for small- | er (but still professional/well-resourced) labs to collaborate | together. I'm not sure the latter would be helpful, as if they | wanted to collaborate together they could probably already do | that by pooling resources for a large cloud compute run. ___________________________________________________________________ (page generated 2023-12-06 23:00 UTC)