he/him

Alts (mostly for modding)

@sga013@lemmy.world

(Earlier also had @sga@lemmy.world for a year before I switched to @sga@lemmings.world, now trying piefed)

  • 0 Posts
  • 1 Comment
Joined 7 months ago
cake
Cake day: March 14th, 2025

help-circle
  • llms can be run. yes you can run stuff in parallel (models reach trillion params. no gpu has that much vram)(most software for llms (for example, llama-cpp, vllm) support parallel). but the only question would be are they any useful? i do not know there benchmarks, but you can try to lookup online what is memory speed for these (vram bandwidth), as that can give a good enough idea of what tps (essentially speed) they can give. but another thing is, power costs. i do ot know where you live, but if power is expensive, running many old cards is going to be very expensive. you may have a better time buying something in 100$ range (like a second hand 3070/80 or something).

    yes you can encode (assuming they have encoding parts in the silicon). That may be most efficient use, as transcoding can be a bit more compute heavy than memory (llms are memory as well as compute heavy, image/video gen is ridiculuously compute heavy, but not much in memory).

    you could potentially use them for scientific compute (stuff like boinc). again, i am not sure on specs, but they are generally compute heavy things (protien folding simulations for example). but that will also require a large amount of regular ram and storage (most scientific compute stuff generates tonnes of data).