Ballista 0.9.0 Release (October 2022)

[andygrove]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

I plan on cutting the release candidate on October 14th. It would be nice to get as many of these features in as possible, but none should block the release.

• Features
  • Add REST endpoint to get DOT graph of a job #242
  • Clean up job data on both Scheduler and Executor #188
  • Add support for S3 data sources #290
  • Integration test script should test the Python bindings #363
  • Integration test script should test REST API #368
• Bug Fixes
  • Schema inference does not work in Ballista-cli with a remote context #287
  • Scheduler silently replaces ParquetExec with EmptyExec if data path is not correctly mounted in container #353
  • Integration tests fail #360
• Improve User Guide
  • Fix references to DataFusion
  • Incorrect documentation for building Ballista on Linux when using docker-compose #362
  • Add REST API documentation to User Guide #272
  • Add scheduler UI screen shot and documentation
  • Explain how to use query graphs in tuning guide
• Update README
  • Update project status information
  • Add scheduler UI screenshot
• Release Process
  • Upgrade to DataFusion 13.0.0
  • Write documentation on how to verify a release #359
  • Prepare the release candidate
  • Publish the release to crates.io once the vote passes
  • Publish the Python bindings, if possible.
  • Publish the blog post

Describe the solution you'd like
☝️

Describe alternatives you've considered
None

Additional context
None

[andygrove]

@yahoNanJing @thinkharderdev @mingmwang @avantgardnerio I created a Google doc for us to collaborate on for a blog post for this release since there are so many significant improvements.

https://docs.google.com/document/d/1R6xHK30I8C6KZ_4hkcqYdF9m-sxnMf_AoTQJjEyV4CA/edit?usp=sharing