+5
Daily
9,083
stars
2,221
forks
234
watchers
6,116
prs
2,002
issues
39
releases
README
Delta Lake is an open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs for Scala, Java, Rust, Ruby, and Python. See the Delta Lake Documentation for details. See the Quick Start Guide to get started with Scala, Java and Python. Note, this repo is one of many Delta Lake repositories in the delta.io organizations including delta, delta-rs, delta-sharing, kafka-delta-ingest, and website. The following are some of the more popular Delta Lake integrations, refer to delta.io/integrations for the complete list: Apache Spark™: This connector allows Apache Spark™ to read from and write to Delta Lake. Apache Flink (Preview): This connector allows Apache Flink to write to Delta Lake. PrestoDB: This connector allows PrestoDB to read from Delta Lake. Trino: This connector allows Trino to read from and write to Delta Lake. Delta Standalone: This library allows Scala and Java-based projects (including Apache Flink, Apache Hive, Apache Beam, and PrestoDB) to read from and write to Delta Lake. Apache Hive: This connector allows Apache Hive to read from Delta Lake. Delta Rust API: This library allows Rust (with Python and Ruby bindings) low level access to Delta tables and is intended to be used with data processing frameworks like datafusion, ballista, rust-dataframe, vega, etc. Table of Contents Latest binaries API Documentation Compatibility API Compatibility Data Storage Compatibility Roadmap Building Transaction Protocol Requirements for Underlying Storage Systems Concurrency Control Reporting issues Contributing License Community Latest Binaries See the online documentation for the latest release. API Documentation Scala API docs Java API docs Python API docs Compatibility Delta Standalone library is a single-node Java library that can be used to read from and write to Delta tables. Specifically, this library provides APIs to interact with a table’s metadata in the transaction log, implementing the Delta Transaction Log Protocol to achieve the transactional guarantees of the Delta Lake format. API Compatibility There are two types of APIs provided by the Delta Lake project. - Direct Java/Scala/Python APIs - The classes and methods documented in the API docs are considered as stable public APIs. All other classes, interfaces, methods that may be directly accessible in code are considered internal, and they are subject to change across releases. - Spark-based APIs - You can read Delta tables through the DataFrameReader/Writer (i.e. spark.read, df.write, spark.readStream and df.writeStream). Options to these APIs will remain stable within a major release of Delta Lake (e.g., 1.x.x). - See the online documentation for the releases and their compatibility with Apache Spark versions. Data Storage Compatibility Delta Lake guarantees backward compatibility for all Delta Lake tables (i.e., newer versions of
Top issues
#2434[BUG][Spark] Exception while using 3-layer-namespace on Spark49 comments · 2024-01-04#860Not null invariant incorrectly fails on non-nullable field inside a nullable struct28 comments · 2021-12-08#1260[Feature Request] Make OPTIMIZE ZORDER BY skip partitions selectively if the most recent version was already optimized20 comments · 2022-07-09#524Why bucket table is not supported in Delta20 comments · 2020-09-24#581Feature Request - Auto Analyze Table19 comments · 2021-01-13#1775[BUG] Delta chronically behind Databricks18 comments · 2023-05-18#826Delete and Update subquery support18 comments · 2021-11-10#1045[Feature Request] Hive Metastore compatibility between different systems16 comments · 2022-04-01
Discussions
#8095Will there be addition of uuidv7 in delta0 comments · 2026-11-11#7794Liquid-clustering incremental clustering and autoCompact.1 comments · 2026-10-02#7442Metadata.Partitions physical / logical column names?2 comments · 2026-08-12#7348Is there anything in place for streaming tables to have numeric ids created like identity cols1 comments · 2026-08-02#7046AI0 comments · 2026-06-17#6705Delta Lake Community Meetup - May 19th, 20261 comments · 2026-05-01#6619Delta Lake Community Meetup - April 21st, 20262 comments · 2026-04-20#6385Restrictions in GitHub Actions and policy changes2 comments · 2026-03-25
Latest release
Delta Lake 4.4.1 v4.4.1 2026-10-05We are pleased to announce the release of Delta Lake 4.4.1, a patch release on top of Delta Lake 4.4.0. This release improves maintenance safety for catalog-managed tables and fixes snapshot construction for tables that use coordinated commits. Bug Fixes \[Spark\] Safer maintenance for catalog-managed tables: Delta extends existing safeguards on catalog-managed tables to prevent dangerous data and metadata cleanup operations. Automatic maintenance operations are skipped so writes can complete successfully without failure. \[Spark\] S3A Fast Listing: Fixed an issue that could produ