Subj : Re: AI and BBSing
To   : Tracker1
From : claw
Date : Mon Oct 02 2023 07:41 am

On 01 Oct 2023, Tracker1 said the following...
 Tr> Depending on the LLM, yes... Many/most LLMS run on GPU hardware, and the
 Tr> good ones need a card with 16gb or more ram on that GPU in order to run.
 Tr> Even a "low" memory, CPU based LLM will generally want at least a couple
 Tr> cores and a few gigs of ram, so if you're running multiple conversations
 Tr> it's entirely possible to hit those kinds of limits.
 Tr>   
 Tr> --    
 Tr> Michael J. Ryan    
 Tr> +o roughneckbbs.com   
 Tr> tracker1@roughneckbbs.com

Interesting. That makes sense so you need something like a 6950 with loads of ram?

That explains why no one is doing this.  Dedicate a peice of hardward like that to a BBS.  nope.

|23|04Dr|16|12Claw
|16|14Sysop |12Noverdu |14BBS |20|15Radio|10@|14HTTP://Noverdu.com:88
|16|10 Standard ports for SSH/Telnet |04 WEB|14@|12HTTP://noverdu.com:808
|20|15Global Chat, Global Messaging and Games! |16|10Ditch the Unsocial Media

--- Mystic BBS v1.12 A47 2021/12/24 (Linux/64)
 * Origin: Noverdu BBS (21:1/210)

.