README

OpenChatKit

OpenChatKit provides a powerful, open-source base to create both specialized and general purpose models for various applications. The kit includes an instruction-tuned language models, a moderation model, and an extensible retrieval system for including up-to-date responses from custom repositories. OpenChatKit models were trained on the OIG-43M training dataset, which was a collaboration between Together, LAION, and Ontocord.ai. 

In this repo, you'll find code for:
- Training GPT-NeoXT-Chat-Base-20B, a 20B parameter chat model (see docs/GPT-NeoXT-Chat-Base-20B.md)
- Fine-tuning Llama-2-7B-32K-beta, a 7B parameter long context model
- Training Pythia-Chat-Base-7B, a 7B parameter chat model
- Testing inference using either of the chat models
- Augmenting the model with additional context from a retrieval index

 Contents

- Getting Started
   Requirements
   Chatting with Pythia-Chat-Base-7B
- Fine-tuning Llama-2-7B-32K-beta
   Downloading and converting the base model
   Fine-tuning the model
   Converting trained weights to Hugging Face format
- Reproducing Pythia-Chat-Base-7B
   Downloading training data and the base model
   (Optional) 8bit Adam
   Training the model
   Converting weights to Hugging Face format
   Testing the new model
- Monitoring
   Loguru
   Weights & Biases
- Experimental: Retrieval-Augmented Models
- See Also
- License
- Citing OpenChatKit
- Acknowledgements

 Getting Started

In this tutorial, you will download Pythia-Chat-Base-7B, an instruction-tuned language model, and run some some inference requests against it using a command-line tool.

Pythia-Chat-Base-7B is a 7B-parameter fine-tuned variant of Pythia-6.9B-deduped from Eleuther AI. Pre-trained weights for this model are available on Hugging Face as togethercomputer/Pythia-Chat-Base-7B under an Apache 2.0 license.

More details can be found on the model card for Pythia-Chat-Base-7B on Hugging Face.

 Requirements

Before you begin, you need to install PyTorch and other dependencies.

1. Install Miniconda from their website.

2. Install Git LFS from their website.

3. Install the git lfs hooks.

shell
git lfs install


4. Install mamba in the base environment so it's available in all environments.

shell
conda install mamba -n base -c conda-forge


5. Create an environment called OpenChatKit using the environment.yml file at the root of this repo.

 Note
 Use mamba to create the environment. It's much faster than using conda.

shell
mamba env create -f environment.yml 


6. Activate the new conda environment.

shell
conda activate OpenChatKit


 Chatting with Pythia-Chat-Base-7B

To help you try the model, inference/bot.py is a simple command-line test harness that provides a shell inferface enabling you to chat with the model. Simply enter text at the prompt and the model replies. The test harness also maintains conversation history to provide the model with context.


Start the bot by calling bot.py from the root for the repo.

shell
python inference