apache /spark

Apache Spark

Spark is a unified analytics engine for large-scale data processing. It provides
high-level APIs in Scala, Java, Python, and R (Deprecated), and an optimized engine that
supports general computation graphs for data analysis. It also supports a
rich set of higher-level tools including Spark SQL for SQL and DataFrames,
pandas API on Spark for pandas workloads, MLlib for machine learning, GraphX for graph processing,
and Structured Streaming for stream processing.

- Official version: 
- Development version: 









 Online Documentation

You can find the latest Spark documentation, including a programming
guide, on the project web page.
This README file only contains basic setup instructions.

 Build Pipeline Status

| Branch     | Status                                                                                                                                                                                                          |
|------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| master     |                                                |
|            |                              |
|            |              |
|            |                                      |
|            |                                      |
|            |                                  |
|            |                            |
|            |                                            |
|            |          |
|            |                                        |
|            |                          |
|            |          |
|            |                  |
|            |                                  |
|            |                            |
|            |  |
|            |                    |
|            |            |
|            |          |
|            |                            |
|            |                            |
|            |                |
|            |                            |
|            |                      |
|            |                  |
|            |                      |
|            |                        |
| branch-4.x |                                  |
|            |                    |
|            |                    |
|            |                    |
|            |                |
|            |                      |
|            |        |
|            |                    |
|            |            |
| branch-4.3 |                                  |
|            |                    |
|            |                    |
|            |                    |
|            |                |
|            |                      |
|            |        |
|            |                    |
|            |            |
| branch-4.2 |                                  |
|            |

Top issues

#54603[FEATURE REQUEST] Add ADBC (Arrow Database Connectivity) Data Source19 comments · 2026-03-03 #59787[SPARK] Split wide join and aggregate outputs into methods under whole-stage codegen8 comments · 2026-10-25 #58475Credential Rotation6 comments · 2026-09-02 #59014Shuffle tracking prevents dynamic allocation from removing executors after `collect()` / `toPandas()`3 comments · 2026-09-24 #56738OracleDialect: Hardcoded scale of 10 in Decimal(38,10) causes silent precision loss for bare NUMBER columns3 comments · 2026-06-24 #59842[SQL] SortMergeJoin codegen compilation fails when maxFields > 100 due to inner class variable scoping2 comments · 2026-10-27 #59438Wrong rows from a subquery expression in the select list over a sorted LIMIT or OFFSET (PushProjectionThroughLimitAndOffset)2 comments · 2026-10-13 #59434Wrong result: LIMIT/OFFSET over LIMIT/OFFSET over a LEFT JOIN returns more rows than its LIMIT when adaptive query execution is on2 comments · 2026-10-13

Snapshot: 2026-11-10T09:35:25Z · Data license Apache-2.0 · JSON