Llama 3.3 (70B) Finetuning - now with 90K context length and fits on <41GB VRAM.

Hey guys! You can now fine-tune Llama 3.3 (70B) up to 90,000 context lengths with Unsloth, which is 13x longer than what Hugging Face + FA2 supports at 6,900 on a 80GB GPU.

  1. The new ultra long context support is 1.85x longer than previous versions of Unsloth. It utilizes our gradient checkpointing and we worked with Apple to incorporate their new Cut Cross Entropy (CCE) algorithm.
  2. For Llama 3.1 (8B), Unsloth can now do a whopping 342,000 context length, which exceeds the 128K context lengths Llama 3.1 natively supported. HF + FA2 can only do 28,000 on a 80GB GPU, so Unsloth supports 12x context lengths.
  3. You can try the new Llama 3.1 (8B) ultra long context support with our Google Colab notebook.
  4. HF+FA2 goes out of memory for 8GB GPUs, whilst Unsloth supports up to 2,900 context lengths, up from 1,500.
  5. 70B models can now fit on 41GB of VRAM - nearly 40GB which is amazing!
  6. In case you didn't know, we uploaded Llama 3.3 versions including GGUFs, 4bit, 16bit versions in our collection on Hugging Face.
  7. You can read our in depth blog post about the new changes here: https://unsloth.ai/blog/llama3-3

https://preview.redd.it/sgfqictcq26e1.png?width=2500&format=png&auto=webp&s=a78ffb6409ec8adaf1184704ee2b27831b7ae410

Table for all Llama 3.3 versions:

Original HF weights 4bit BnB quants GGUF quants (16,8,6,5,4,3,2 bits)
Llama 3.3 (70B) Instruct Llama 3.3 (70B) Instruct 4bit Llama 3.3 (70B) Instruct GGUF

Let me know if you have any questions and hope you all have a lovely week ahead! :)