+2
Daily
9,597
stars
2,649
forks
113
watchers
17,237
prs
9,836
issues
0
releases
README
Apache DataFusion [![Crates.io][crates-badge]][crates-url] [![Apache licensed][license-badge]][license-url] [![Build Status][actions-badge]][actions-url] ![Commit Activity][commit-activity-badge] [![Open Issues][open-issues-badge]][open-issues-url] [![Pending PRs][pending-pr-badge]][pending-pr-url] [![Discord chat][discord-badge]][discord-url] [![Linkedin][linkedin-badge]][linkedin-url] ![Crates.io MSRV][msrv-badge] [![Codecov][codecov-badge]][codecov-url] [crates-badge]: https://img.shields.io/crates/v/datafusion.svg [crates-url]: https://crates.io/crates/datafusion [license-badge]: https://img.shields.io/badge/license-Apache%20v2-blue.svg [license-url]: https://github.com/apache/datafusion/blob/main/LICENSE.txt [actions-badge]: https://github.com/apache/datafusion/actions/workflows/rust.yml/badge.svg [actions-url]: https://github.com/apache/datafusion/actions?query=branch%3Amain [discord-badge]: https://img.shields.io/badge/Chat-Discord-purple [discord-url]: https://discord.com/invite/Qw5gKqHxUM [commit-activity-badge]: https://img.shields.io/github/commit-activity/m/apache/datafusion [open-issues-badge]: https://img.shields.io/github/issues-raw/apache/datafusion [open-issues-url]: https://github.com/apache/datafusion/issues [pending-pr-badge]: https://img.shields.io/github/issues-search/apache/datafusion?query=is%3Apr+is%3Aopen+draft%3Afalse+review%3Arequired&label=Pending%20PRs&logo=github [pending-pr-url]: https://github.com/apache/datafusion/pulls?q=is%3Apr+is%3Aopen+draft%3Afalse+review%3Arequired+sort%3Aupdated-desc [linkedin-badge]: https://img.shields.io/badge/Follow-Linkedin-blue [linkedin-url]: https://www.linkedin.com/company/apache-datafusion/ [msrv-badge]: https://img.shields.io/crates/msrv/datafusion?label=Min%20Rust%20Version [codecov-badge]: https://codecov.io/github/apache/datafusion/graph/badge.svg [codecov-url]: https://app.codecov.io/github/apache/datafusion/tree/main Website | API Docs | Chat DataFusion is an extensible query engine written in [Rust] that uses [Apache Arrow] as its in-memory format. This crate provides libraries and binaries for developers building fast and feature-rich database and analytic systems, customized for particular workloads. See [use cases] for examples. The following related subprojects target end users: - DataFusion Python offers a Python interface for SQL and DataFrame queries. - DataFusion Java offers a Java interface for SQL and DataFrame queries. - DataFusion Comet is an accelerator for Apache Spark based on DataFusion. - DataFusion Ballista is a distributed query execution engine that scales DataFusion across a cluster of nodes. "Out of the box," DataFusion offers SQL and DataFrame APIs, excellent [performance], built-in support for CSV, Parquet, JSON, and Avro, extensive customization, and a great community. DataFusion features a full query planner, a columnar, streaming, multi-threaded, vectorized execution engine, and partitioned data sources. You can customize Da
Top issues
#21207[DISCUSSION] Future of Dynamic Filters Sync85 comments · 2026-03-27#24461Release DataFusion `56.0.0` (Oct/Nov 2026)80 comments · 2026-08-18#20324[EPIC] Fix performance regressions when enabling parquet filter pushdown (late materialization)76 comments · 2026-02-12#25088CI takes hours to start50 comments · 2026-09-08#18411TPCH q1 with no predicates is 2x slower than duckdb50 comments · 2025-10-31#5492[dbolympics] General framework to decorrelate the subqueries50 comments · 2023-03-07#24704[EPIC] Use blocked / chunked memory management in hash aggregation49 comments · 2026-08-27#5504Run DataFusion benchmarks regularly and track performance history over time46 comments · 2023-03-07
Discussions
#27385DISCUSSION: London DataFusion Meetup March 20270 comments · 2026-11-12#26425[RFC/Discussion] Partition-at-a-Time Streaming Evaluation in WindowAggExec2 comments · 2026-10-16#25665DISCUSSION: San Francisco DataFusion Meetup November 20262 comments · 2026-09-23#24594datafusion-bloom 0.2.0: Adaptive Predicate Transfer for DataFusion 550 comments · 2026-08-23#24512DISCUSSION: Sep 2026 DataFusion Meetup in London6 comments · 2026-08-20#24413Policy on uptick of LLM PRs from new contributors11 comments · 2026-08-16#24131Relify: Parquet-native vector indexes with DataFusion0 comments · 2026-08-06#24041How practical is to use this for a OLTP DB engine?1 comments · 2026-08-01