Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The current state of this module #52

Open
lrq3000 opened this issue Dec 8, 2022 · 1 comment
Open

The current state of this module #52

lrq3000 opened this issue Dec 8, 2022 · 1 comment
Labels
ANNOUNCEMENT ANNOUNCEMENT

Comments

@lrq3000
Copy link
Collaborator

lrq3000 commented Dec 8, 2022

Thank you for your interest in or for using this module. Before @tomerfiliba made it 10 years ago and I took the mantle 7 years ago, error correction was something that was available only in proprietary softwares, or hidden in the entrails of academic works as side functions often mixed in with the core routines of these academic projects that were aimed at other purposes than pure error correction. This module was an experiment to provide an out-of-the-box readily useable yet multifunctional tool for general error correction, that could then be used for numerous practical applications, from QR code generators and readers, to hard drive data protection and recovery and of course data stream denoising.

While I am glad of the positive impact this project had, it was never intended to be used so intensively in production. It was made as an educational tool and as robust as possible proof of concept of a universal Reed-Solomon encoder/decoder, to show the ropes with most of the maths worked out and implemented in a working Python package to hopefully inspire others from making a better suite of tools, using this as a reference to check the maths.

But now this Python module is used as-is in numerous other projects (thousands from what GitHub shown me, and likely more given there are >300K monthly downloads!), that are relying on it for the very sensitive task of ensuring data preservation and reliability. This is a huge responsibility, and unfortunately there is no one maintaining this project but me, someone who is not even in the field of signal processing (I worked on some such projects, but this is not my specialty, this project was just a "fun" side project).

In the last 2 years, I hoped someone would jump in to replace this project with another more robust one, but it seems no one did. I will hence continue to try to maintain this project once in a while, very infrequently and with no guarantee, just to ensure the project still continues to work. But no new features will be implemented. PRs are very welcome to bug fixes, otherwise they may not ever be corrected.

I aim to maintain the software to run as-is, and it's already a lot of work given the fast pace of the Python ecosystem evolution. If anyone else wants to step in to maintain or even develop further the project (under open-source of course), please reply in this thread and I will consider your application.

@lrq3000 lrq3000 added the ANNOUNCEMENT ANNOUNCEMENT label Dec 8, 2022
@lrq3000
Copy link
Collaborator Author

lrq3000 commented Apr 6, 2023

Update: Cython v3 is coming out, and this offers new possibilities to optimize, especially because math operators in Cython v3 are now much more like Python v3. I hence decided to do a significant rework of the module to squeeze more speed optimizations in creedsolo. Several optimizations were also translated to the pure python reedsolo module, which can improve speed when run inside a JIT compiler such as PyPy. Along the way, I also repackaged the module with modern standards and precompiled creedsolo binaries are provided for most desktop OSes and Python3 versions thanks to cibuildwheel. A few bugfixing were also resolved.

I plan on continuing to work on polishing the module, by bugfixing and refining speed optimizations. I will not develop new features nor new potentially faster algorithms to compute Reed-Solomon, but if someone want to work on them, I would be excited to collaborate and help make these new features a reality.

All that is to say: if you are currently using reedsolo or creedsolo, you can rest assured it will continue to work for the foreseeable future, with active maintenance and regular (although infrequent) polishing updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ANNOUNCEMENT ANNOUNCEMENT
Projects
None yet
Development

No branches or pull requests

1 participant