README

data load tool (dlt) — the open-source Python library that automates all your tedious data loading tasks


Be it a Google Colab notebook, AWS Lambda function, an Airflow DAG, your local laptop,or an AI coding agent—dlt can be dropped in anywhere.





🚀 Join our thriving community of likeminded developers and build the future together!




  
    
  


  
    
  
  
    
  
  
    
  


 We launched dltHub in March 2026. You can now run your dlt pipelines on a managed, agent-native platform with transformations, data quality and observability built in. Learn more or deploy your first pipeline.

 Installation

dlt supports Python 3.10 through Python 3.14. Note that some optional extras are not yet available for Python 3.14, so support for this version is considered experimental.

sh
pip install dlt


Add the extras you need for your sources and destinations, for example:

sh
pip install "dlt[duckdb]"         local DuckDB destination
pip install "dlt[bigquery]"       or snowflake, postgres, redshift, databricks, athena, ...
pip install "dlt[s3]"             or gs, az for cloud filesystems
pip install "dlt[sql_database]"   read from any SQL database
pip install "dlt[hub]"            data quality, transformations, and AI (see below)


Prefer uv? uv add "dlt[duckdb]".

 Quick Start

Describe an API declaratively and load it into DuckDB — dlt handles requests, pagination, schema inference, and typing for you:

python
import dlt
from dlt.sources.rest_api import rest_api_source

 1. Describe the API declaratively
source = rest_api_source({
    "client": {"base_url": "https://api.spotify.com/v1"},
    "resources": [
        {
          "name": "playlist_tracks",
          "endpoint": {"path": "playlists/{playlist_id}/tracks"},
        },
    ],
})

 2. Point a pipeline at any destination
pipeline = dlt.pipeline(
    pipeline_name="spotify",
    destination="duckdb",
    dataset_name="spotify_data",
)

 3. Extract, normalize, and load
pipeline.run(source)

 4. ...and read it straight back as a DataFrame
pipeline.dataset().playlist_tracks.df()


...or load any Python iterable — a resource is just a generator, and dlt infers the schema, types the columns, and writes the table:

python
import dlt

@dlt.resource(table_name="tracks", primary_key="id", write_disposition="merge")
def tracks():
    yield {"id": 1, "title": "Yellow",       "artist": "Coldplay",   "streams": 4_200_000_000}
    yield {"id": 2, "title": "Shape of You", "artist": "Ed Sheeran", "streams": 3_900_000_000}

dlt.pipeline(
    destination="duckdb",
    dataset_name="spotify_data",
).run(
  source=tracks(),
)


Check out a basic example in Colab or a more advanced Hugging Face demo with Marimo notebooks.

Ready to run it in production? Deploy it with dltHub.

 Why dlt

dlt loads data from messy, often unstructured sources into well-structured, typed datasets. It's a library, not a platform — you pip install it into your existing code and keep your workflow and the other tools you already use.