README

General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model



 
 
 

Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu,  Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, Chunrui Han, Xiangyu Zhang






 Release
- [2025/2/1] 🚀🚀🚀 GOT-OCR2.0 is merged to Huggingface-transformers/space. It supports inference batched. Thanks to the MLE of Huggingface Yoni.
- [2024/12/24] 🔥🔥🔥 My new work on system-2 perception is released slow-perception.
- [2024/12/18] 🚀🚀🚀 GOT-OCR2.0 is supported in PaddleMIX by Paddle Team. Thanks for the Paddle team!
- [2024/12/8] 🔥🔥🔥 The model download has exceeded 1M on Huggingface.
- [2024/12/5] The seven wechat group.
- [2024/11/4] The six wechat group.
- [2024/10/24] The previous four wechat groups are full, so we created a fifth group.
- [2024/10/11] Too many friends want to join the wechat group, so we created a fourth group.
- [2024/10/2] onnx and mnn versions of GOT-OCR2.0.
- [2024/9/29]🔥🔥🔥 The community has implemented the first version of llama_cpp_inference.
- [2024/9/24]🔥🔥🔥 Support ms-swift quick Fine-tune for your own data. 
- [2024/9/23]🔥🔥🔥 We release the official Modelscope demo. Thanks very much for Modelscope providing the GPU resource.
- [2024/9/19]🔥🔥🔥 GOT-OCR2.0 achieves Huggingface trending 1.
- [2024/9/14]🔥🔥🔥 We release the official demo. Thanks very much for Huggingface providing the GPU resource. 
- [2024/9/13]🔥🔥🔥 We release the Huggingface deployment. 
- [2024/9/03]🔥🔥🔥 We open-source the codes, weights, and benchmarks. The paper can be found in this repo. We also have submitted it to Arxiv. 
- [2024/9/03]🔥🔥🔥 We release the OCR-2.0 model GOT! 








 Community contributions
We encourage everyone to develop GOT applications based on this repo. Thanks for the following contributions :

OpenVINO~ contributor: @can-gaa-hou

GGUF and Llama.cpp inference~ contributor: @MosRat

vllm reference ~ contributor: @Jay

onnx and mnn supports ~ contributor: @BaofengZan

llama_cpp inference ~ contributor: @1694439208

Colab of GOT   ~      contributor: @Zizhe Wang

CPU version of GOT ~ contributor: @ElvisClaros

Online demo ~ contributor: @Joseph Pollack

Dokcer & client demo ~ contributor: @QIN2DIM 

GUI of GOT ~ contributor: @XJF2332 

 Contents
- Install
- GOT Weights
- Benchmarks
- Demo
- Train
- Fine-tune
- Eval






Towards OCR-2.0 via a Unified End-to-end Model       





 Install
0. Our environment is cuda11.8+torch2.0.1
1. Clone this repository and navigate to the GOT folder
bash
git clone https://github.com/Ucas-HaoranWei/GOT-OCR2.0.git
cd 'the GOT folder'

2. Install Package
Shell
conda create -n got python=3.10 -y
conda activate got
pip install -e .


3. Install Flash-Attention

pip install ninja
pip install flash-attn --no-build-isolation

 GOT Weights
- Huggingface
- Google Drive
- BaiduYun code: OCR2

 Benchmarks
- Google Drive
- BaiduYun code: OCR2

 Demo
1. plain texts OCR:
Shell
python3 GOT/demo/run_ocr_2.0.py  --model-name  /GOT_weights/  --image-file  /an/image/file.png