If you need help, please ask your question with tag kiba-etl on StackOverflow so that other can benefit from your contribution! I monitor this specific tag and will reply to you.
Writing reliable, concise, well-tested & maintainable data-processing code is tricky.
Kiba lets you define and run such high-quality ETL (Extract-Transform-Load) jobs using Ruby.
Learn more on the Wiki, on my blog and on StackOverflow.
Kiba 2.0.0 has been recently released with one major improvement for ETL components developers.
Please check out the release notes.
Subscribe to my newsletter to get the upcoming detailed article on Kiba 2 benefits.
Add the gem to your Gemfile and run the bundle command:
gem 'kiba'- How do you define ETL jobs with Kiba?
- Running Kiba jobs from the command line
- Considerations for running Kiba jobs programmatically (from Sidekiq, Faktory, Rake, ...)
- Implementing ETL sources.
- Implementing ETL transforms.
- Implementing ETL destinations.
- Implementing pre and post-processors.
- Ruby Kaigi 2018 conference - Kiba 2 - Past, present & future of data processing with Ruby (slides)
- Live Coding Session - Processing data with Kiba ETL
- Rubyists - are you doing ETL unknowningly?
- How to write solid data processing code
- How to reformat CSV files with Kiba (in-depth, hands-on tutorial)
- How to explode multivalued attributes with Kiba ETL?
- Common techniques to compute aggregates with Kiba
- How to run Kiba in a Rails environment?
- How to pass parameters to the Kiba command line?
Kiba currently supports Ruby 2.3+, JRuby 9.1+ and TruffleRuby. See test matrix.
I'm starting to add commonly used reusable helpers in a separate gem called kiba-common, check it out (work-in-progress).
Consulting services: if your organization needs help to implement a data pipeline or to build a data-intensive application, I provide consulting services. More information.
Kiba Pro: for more features & goodies, check out Kiba Pro.
Copyright (c) LoGeek SARL. Kiba is an Open Source project licensed under the terms of the LGPLv3 license. Please see http://www.gnu.org/licenses/lgpl-3.0.html for license text.
(agreement below borrowed from Sidekiq Legal)
By submitting a Pull Request, you disavow any rights or claims to any changes submitted to the Kiba project and assign the copyright of those changes to LoGeek SARL.
If you cannot or do not want to reassign those rights (your employment contract for your employer may not allow this), you should not submit a PR. Open an issue and someone else can do the work.
This is a legal way of saying "If you submit a PR to us, that code becomes ours". 99.9% of the time that's what you intend anyways; we hope it doesn't scare you away from contributing.