README

Gemma in PyTorch

Gemma is a family of lightweight, state-of-the art open models built from research and technology used to create Google Gemini models. They include both text-only and multimodal decoder-only large language models, with open weights, pre-trained variants, and instruction-tuned variants. For more details, please check out the following links:

  Gemma on Google AI
  Gemma on Kaggle
  Gemma on Vertex AI Model Garden

This is the official PyTorch implementation of Gemma models. We provide model and inference implementations using both PyTorch and PyTorch/XLA, and support running inference on CPU, GPU and TPU.

 Updates

  [March 12th, 2025 🔥] Support Gemma v3. You can find the checkpoints on Kaggle and Hugging Face

  [June 26th, 2024] Support Gemma v2. You can find the checkpoints on Kaggle and Hugging Face

  [April 9th, 2024] Support CodeGemma. You can find the checkpoints on Kaggle and Hugging Face

  [April 5, 2024] Support Gemma v1.1. You can find the v1.1 checkpoints on Kaggle and Hugging Face.

 Download Gemma model checkpoint

You can find the model checkpoints on Kaggle:

- Gemma 3
- Gemma 2
- Gemma

Alternatively, you can find the model checkpoints on the Hugging Face Hub here. To download the models, go the the model repository of the model of interest and click the Files and versions tab, and download the model and tokenizer files. For  programmatic downloading, if you have huggingface_hub installed, you can also run:


huggingface-cli download google/gemma-3-4b-it-pytorch


The following model sizes are available:

- Gemma 3: 
  - Text only: 1b
  - Multimodal: 4b, 12b, 27b_v3
- Gemma 2: 
  - Text only: 2b-v2, 9b, 27b
- Gemma: 
  - Text only: 2b, 7b


Note that you can choose between the 1B, 4B, 12B, and 27B variants.


VARIANT=
CKPT_PATH=


 Try it free on Colab

Follow the steps at
https://ai.google.dev/gemma/docs/pytorch_gemma.

 Try it out with PyTorch

Prerequisite: make sure you have setup docker permission properly as a non-root user.

bash
sudo usermod -aG docker $USER
newgrp docker


 Build the docker image.

bash
DOCKER_URI=gemma:${USER}

docker build -f docker/Dockerfile ./ -t ${DOCKER_URI}


 Run Gemma inference on CPU.

 NOTE: This is a multimodal example. Use a multimodal variant.

bash
docker run -t --rm \
    -v ${CKPT_PATH}:/tmp/ckpt \
    ${DOCKER_URI} \
    python scripts/run_multimodal.py \
    --ckpt=/tmp/ckpt \
    --variant="${VARIANT}" \
     add --quant for the int8 quantized model.


 Run Gemma inference on GPU.

 NOTE: This is a multimodal example. Use a multimodal variant.

bash
docker run -t --rm \
    --gpus all \
    -v ${CKPT_PATH}:/tmp/ckpt \
    ${DOCKER_URI} \
    python scripts/run_multimodal.py \
    --device=cuda \
    --ckpt=/tmp/ckpt \
    --variant="${VARIANT}"
     add --quant for the int8 quantized model.


 Try It out with PyTorch/XLA

 Build the docker image (CPU, TPU).

bash
DOCKER_URI=gemma_xla:${USER}

docker build -f docker/xla.Dockerfile ./ -t ${DOCKER_URI}


 Build the