spotify /luigi
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
★ 18,789⑂ 2,474👁 446⑃ 2,436● 1,037▤ 67
More from this author
+1
Daily
18,789
stars
2,474
forks
446
watchers
2,436
prs
1,037
issues
67
releases
Top issues
#2915Cannot set pdb breakpoint in `run()` method11 comments · 2020-03-04#2864MissingParameterException with Config parent class9 comments · 2019-12-10#2965luigi.contrib.azureblob needs updating8 comments · 2020-06-24#2905Byte writing with localTarget8 comments · 2020-02-23#2087luigi.target.AtomicLocalFile not so atomic after all7 comments · 2017-04-07#3326Improvements to Testing6 comments · 2024-12-04#3249SVG visualisation breaks on dependencies between tasks with the same name6 comments · 2023-08-26#2935Completed tasks should not be removed from the cache6 comments · 2020-04-24
Latest release
3.8.1 v3.8.1 2026-05-07What's Changed feat: introduce mypy type checking with gradual adoption by @kitagry in https://github.com/spotify/luigi/pull/3403 feat: type check parameter classes by @kitagry in https://github.com/spotify/luigi/pull/3406 Proposal: set default value of OptionalParameter to None by @kitagry in https://github.com/spotify/luigi/pull/3411 Fix deprecated datetime.utcfromtimestamp() calls by @apoorvdarshan in https://github.com/spotify/luigi/pull/3410 Fix BoolParameter to consider global parsing default. by @riga in https://github.com/spotify/luigi/pull/3416 fix(server): resolve /histor