bigscience

/

bloom

like 5k

Follow

BigScience Workshop 1.3k

No

It's too big model for Google Colab usage limits. Even for premium or professional plans.

If inference time is not a constraint, you can run BLOOM 176-B on a desktop computer, even if you don’t have a GPU, you just need at least 16GB of RAM. The following blog post contains instructions to do this:

https://towardsdatascience.com/run-bloom-the-largest-open-access-ai-model-on-your-desktop-computer-f48e1e2a9a32

In my computer (i5 11gen, 16GB RAM, 1TB SSD Samsung 980 pro), the generation takes 3 minutes per token using only the CPU, which is a little slow but manageable.

I was wondering if there's any way to use Bloom-176B on Google Colab?

Found this:
https://harishgarg.com/writing/generate-content-using-bloom-open-source-ai-model/
Not sure if this helps or not.

You can run it on Runpod using KoboldAI, here (I believe you need 335GB+ of VRAM though, which isn't cheap):

https://runpod.io?ref=ev1llooa