README

UI-TARS: Pioneering Automated GUI Interaction with Native Agents --
!Local Image


        🌐 Website&nbsp&nbsp | 🤗 Hugging Face Models&nbsp&nbsp 
        | &nbsp&nbsp 🔧 Deployment &nbsp&nbsp  | &nbsp&nbsp 📑 Paper &nbsp&nbsp  |&nbsp&nbsp
🖥️ UI-TARS-desktop&nbsp&nbsp  🏄 Midscene (Browser Automation) &nbsp&nbsp | &nbsp&nbsp🫨 Discord&nbsp&nbsp





We also offer a UI-TARS-desktop version, which can operate on your local personal device. To use it, please visit https://github.com/bytedance/UI-TARS-desktop. To use UI-TARS in web automation, you may refer to the open-source project Midscene.js.
❗Notes: Since Qwen 2.5vl based models ultilizes absolute coordinates to ground objects, please kindly refer to our illustration about how to process coordinates in this guide.

 Updates
- 🌟 2025.09.04: We’re excited to announce the release the UI-TARS-2, which is a major upgrade from UI-TARS-1.5, featuring with enhanced capabilities in GUI, Game, Code and Tool Use. It is an "All In One" Agent model, enabling seamless integration of multiple abilities for complex tasks. Please check our new technical report for more details. Refer to more fantastic showcases at our website.
- 🌟 2025.04.16: We shared the latest progress of the UI-TARS-1.5 model in our blog, which excels in playing games and performing GUI tasks, and we open-sourced the UI-TARS-1.5-7B.
- ✨ 2025.03.23: We updated the OSWorld inference scripts from the original official OSWorld repository. Now, you can use the OSWorld official inference scripts to reproduce our results.

 Introduction

UI-TARS-1.5, an open-source multimodal agent built upon a powerful vision-language model. It is capable of effectively performing diverse tasks within virtual worlds.

Leveraging the foundational architecture introduced in our recent paper, UI-TARS-1.5 integrates advanced reasoning enabled by reinforcement learning. This allows the model to reason through its thoughts before taking action, significantly enhancing its performance and adaptability, particularly in inference-time scaling. Our new 1.5 version achieves state-of-the-art results across a variety of standard benchmarks, demonstrating strong reasoning capabilities and notable improvements over prior models.


    
      
    



    
      
    


 🚀 Quick Start Guide: Deploying and Using Our Model

To help you get started quickly with our model, we recommend following the steps below in order. These steps will guide you through deployment, prediction post-processing to make the model take actions in your environment.


 ✅ Step 1: Deployment & Inference

👉 Deployment and Inference.
This includes instructions for model deployment using huggingface endpoint, and running your first prediction.


 ✅ Step 2: Post Processing

 Installation
bash
pip install ui-tars
 or
uv pip install ui-tars

 Usage
python
from ui_tars.action_parser import parse_action_to_structure_output, parsing_response_to_pyautogui_code

response = "Thought: Click the button\nAction: click(start_bo