GSoC_2018
This is an example of the guided filter in opencv_contrib
>>> Skip directly to the ideas list below <<<
- Program Site for GSoC 2018
- Mailing list for OpenCV GSOC 2018:
opencv-gsoc-2018@googlegroups.com - IRC Channel:
#opencvon freenode - Timelines
| Date | Description | Comment |
|---|---|---|
| January 4 17:00 UTC | Mentoring organizations begin submitting applications to Google | 🆗 |
| January 23 17:00 UTC | Mentoring organization application deadline | ✔️ |
| January 23 - February 11 | Google program administrators review organization applications | 💀 |
| ~~February 12 - March 12 | ||
| ~~March 12 16:00 UTC | ||
| ~~March 27 16:00 UTC | ||
| ~~April 23 16:00 UTC | ||
| ~~Community Bonding Period | ||
UTC to PDT (California uses PST in the winter (from Nov 1st) and PDT in the summer (from March 8)).
- OpenCV official Site
- OpenCV wiki
- How to do a pull request/How to Contribute Code
- Source Code can be found at GitHub/opencv and GitHub/opencv_contrib
- Developer meeting notes
- Student projects to be paid only if:
-
Phase 1:
- You must generate a pull request
- That builds
- Has at least stubbed out functionality
- With OpenCV appropriate Doxygen documentation
- Includes What the function or net is, what the function or net is used for
- Has at least stubbed out unit test
- Has a stubbed out example/tutorial of use that builds
- You must generate a pull request
-
Phase 2:
- You must generate a pull request
- That builds
- Has basic functionality
- With OpenCV appropriate Doxygen documentation
- Includes What the function or net is, what the function or net is used for
- Has basic unit test
- Has a tutorial of how to use the function or net and why you'd want to use it.
- You must generate a pull request
-
End of summer:
- A full pull request
- Full Doxygen documentation
- A good unit test
- Example of use/tutorial of the code or net
- Create a (short!) Movie (preferably on Youtube, but any movie) that demonstrates your code
- We use this to create an overall summary. Past years:
- A full pull request
-
Phase 1:
- You must already know how to program fluently in C++
- Some projects may instead specifically require Python, javascript or Matlab skills
- Ask to join the OpenCV GSoC Forum List
- Discuss projects ideas below or your own ideas with OpenCV mentors on the list now and March 27.
- Always title your proposal with what you want to do (example: Implement Patch Match Stereo Algorithm )
- NOTE: The above is to discuss proposals with mentors. BUT, when the application period starts, you must still sign up with Google Summmer of Code and submit your proposal to the OpenCV organization. If not, you will not show up on the database where we can select you as a student.
- In March 12, Go to the GSoC site and sign up to be a student with OpenCV
- Post the project from below or your own agreed on project on the GSoC to the mailing list.
- Include Name, google email, age
- Include how you think you are qualified to accomplish this project (skills, courses, relevant background)
- Include Country of origin, school you are enrolled in, Professor you work with (if any)
- Include a projected timeline and milestones for the project
- Once (and if!) OpenCV gets accepted as GSoC org this year, and we are told how many slots we will get and you've signed up for a project with us in March before the March 27 deadline: Then:
- We will weigh the students and projects against the mentors we gather and the mentor's interests and choose which students/project to pursue.
- Accepted students will be posted on the GSoC site in May (and we will notify the accepted students ourselves).
- Students are paid over the summer by Google IF the mentor accepts the student's work. There are several milestone based go-nogo points.
- We've had a lot of experience with this: Please only propose projects that you already know how to do.
- It is impossible for a mentor to train you in how to do the task while helping you do it. This always results in failure.
- Mentors exist to keep you on track, and ensure good code, but not to teach you a new area of vision or AI.
- This is also not a research period. If you are a top student with publications in top conferences from a top professor in a top University known to us, we may let you code your research area. This is very rare.
- Documentation, tests and tutorial are as vital as the code. We'd rather have rougher code then miss these items because if these are not in place, your code will be useless to everyone.
- Contact us on the opencv-gsoc googlegroups mailing list above and ask to be a mentor (or we will ask you in some known cases)
- If we accept you, we will post a request from the Google Summer of Code OpenCV project site asking you to join.
- You must accept the request and you are a mentor!
- You then:
- Go to the opencv-gsoc googlegroups mailing list above and look through student project proposals, find a student and project you like and work with them to refine a realistic proposal that they can implement in a summer (you have to judge whether the student is capable - absolutely no non-coders in the language you need, typically C++, accepted! Summer is too short to learn to code and get something done).
- you may also find (good) students and get them to apply, perhaps to your pet project idea
- Once you find or create a project proposal that you want to mentor
- several students might try for the same project
- alternatively, you might have to convince a student to change projects to one you like or recruit an external student to join Google Summer of Code and apply to your project
- But, always encourage students to officially apply through the Google Summer of Code site - it helps us and them.
- Go to the opencv-gsoc googlegroups mailing list above and look through student project proposals, find a student and project you like and work with them to refine a realistic proposal that they can implement in a summer (you have to judge whether the student is capable - absolutely no non-coders in the language you need, typically C++, accepted! Summer is too short to learn to code and get something done).
- We later get a slot allocation from Google, the administrators then "spend" the slots in order of priority influenced by whether there's a capable mentor or not for each topic.
- Students must finally actually accept to do that project (some sign up for multiple organizations and then choose)
- Sheesh!
If you are accepted as a mentor and you find a suitable student and we give you a slot and the student signs up for it, then you are an actual mentor! Otherwise you are not a mentor and have no other obligations.
- Thank you for trying.
- You may contact other mentors and co-mentor a project.
It sounds harder than it is.
You get paid a modest stipend over the summer to mentor, typically $500 minus an org fee of 6%.
Several mentors donate their salary, earning ever better positions in heaven when that comes.
Alphabetic by project title
| Student | Title | Mentor(s) |
|---|
Have to make compliant with Defining a Project Ideas list from the Mentor Guide
Students may propose their own projects
- You must give us a clear summary of what you want to do and
- Proof that you already have the experience to do this project However, below are some of our priorities for this year Contact us and/or discuss ideas on the mailing list.
These are not in order of priority
We have so much good code in opencv_contrib as well as OpenCV that needs better documentation, tutorials, examples of use etc. Particularly:
- Our deep net module
- Curating useful deepnet vision models from the net
- Documenting the whole process of getting it to run in the [DNN module (https://github.com/garybradski/opencv_contrib/tree/master/modules/dnn)
- Show how to use OpenCV with TensorFlow, Keras and/or Pytorch!
- Expand the python documentation and/or tutorials, see the Python idea below
- Make tutorials on more difficult topics such as
- Calibration, stereo calibration, calibration of fisheye lens etc
- Random Forests
- Any topics from opencv_contrib
- Particularly computational vision, new detector/descriptors, new filters, text recognition, depth sensing
OpenCV has a deep neural network module (DNN module). You are welcome to base your proposal on various options with it. We want better performance, more tests, tutorials, python/java wrappers etc.
Improving DNN module ideas:
- Advanced visualizing of deep learning models (with input-output blobs dimensions, layer types, connections and so on). It can be either based on third-party libraries solution or built on OpenCV's drawing functions own implementation.
- Implementing various network compression algorithms, such as quantization, factorization-based compression etc. That will likely involve both scripts for TF/Caffe and the corresponding modifications in DNN to support such compressed models.
- Enable supporting of the most popular deep learning architectures and classes of architectures that DNN does not support yet: DenseNet, GAN's etc. Implement missing layers, check output accuracy, write samples for online-available models.
2b. Curating and optimizing valuable published deep neural networks and turning them into compact models.
We'd like to learn compact (that can be fit on mobile, i.e. models that take a few megabytes of disk or memory space) models, see SqueezeNet for inspiration. Particularly for tasks such as:
- People recognition, instance finding, segmentation
- Background segmentation
- Object tracking
- Car instance, segmentation
- Bike instance, segmentation
- Animals, particularly dogs and cats outside and inside.
- Face tracking and/or finding face features
- Text finding, reading
- Floors, Walls, Ceilings, Windows, Doors, Tables, Chairs
- April tag detectors
- Corner point detectors
- Others....
-
Implement ONNX format parser. We already support Caffe 1 (with various extensions), TF, Torch and part of Darknet. Adding yet another parser is time-consuming but quite straight-forward thing.
-
We have found a good reference starting point for this project here.
- Basically, we need to parse this network structure and fill out the DNN data structures that come with a download of OpenCV. There are several people working on DNN right now, so one of them will mentor this through to finish.
- The code for detecting Aruco and Charuco is here
We have tried to get in reliable building blocks for SLAM (Simultaneous Localization and Mapping) for years. We've made some progress and have good pieces, but it's not there yet.
- Please don't even think about doing this project unless you are already doing SLAM in your advanced PhD work. - This is a hard algorithm to get right and it needs lots of well thought out engineering and you just won't unless you are already working under a leading advisor in the field and/or are a published author in a top conference.
- We have two great mentors in this area, but the student has to have mature knowledge of the area and have mature programming skills in C++
- Write a script and/or use clang to help people move their OpenCV 1.0 or 2.0 code to OpenCV 3.
The python wrapper to OpenCV is absolutely great thanks to several years of continual improvement in GSoC and continued efforts thereafter. This is one of the great success stories of OpenCV. But we want even more.
- Wrap more functions
- Add more documentation
- Add more tutorials and fairly complete applications
- Calibration, face ID, feature detection and identification
- One area is to show how to use OpenCV python with TensorFlow, Keras and/or Pytorch
- Add python wrappers to opencv_contrib functions
Last year, a concerted University + student effort enabled us to create OpenCV.js, OpenCV in javascript. This allows running computer vision natively inside a browser! This year, we want to improve the great effort that started last year. OpenCV.js
- Needs more documentation and tutorials
- Needs more working example code.
- See:
HighGUI is OpenCV's simple graphical user interface including event triggering. It's cool, but it's old. It needs expanding.
- We, the admins, actually place a high priority of getting the simple basic stuff to work well, so we very much encourage application to this project
- HighGUI stands relatively alone and so can be easily changed without a deep knowledge of computer vision.
- Some of it's needs are:
- Add simple buttons and sliders
- It has an optional QT interface, but it involves some messy loading and linking into a very large external code base for what is relatively simple additional functionality.
- Add the QT functionality (mainly zoom to pixel and snap shot) w/o the QT dependency;
- one option is to use OpenGL and draw various controls manually.
- Add the QT functionality (mainly zoom to pixel and snap shot) w/o the QT dependency;
Write better primitives for marking, segmenting, storing image (and possibly 3D!) data
But maybe just work with and improve the VATIC code (contact authors?)
- Eye tracking
- Hand or finger detection
- gesture recognition
- human skeleton based on 2d or 3d
- As above, eye tracking, hand tracking, gesture recognition or human tracking
- Finding/segmenting planar surfaces
- Tracking different kind of markers
- Work better with the ARToolkit or Apple's tool kit ARKit
The vis module is very useful, but it needs:
- Easy way to directly enter cv::Mat containing depth or disparity into a point cloud for vis
- Point cloud to ply converter
- Once in vis, automatic color coded surface normal visualization see here
- And also here
- Tutorials showing how to use its features
- Add onto here
- Or, maybe we just need to document/improve and bring out functionality as in docs here
- Test codes for functionality are here
This is very general and always relevant topic. OpenCV is known for speed, but there is always room for improvement, especially in the experimental part (opencv_contrib). You are welcome to come up with concrete proposals on optimizing a particular functionality.
- Optimisation can be done using OpenCL, parallelisation, SSE/AVX or universal intrinsics.
- It can also be "algorithmic" optimization, which, despite its name, means that the algorithm basically stays the same, but you find some ways to eliminate some unnecessary computations, unnecessary memory allocations etc.
- For example, we would be interested in optimizing
- image processing (imgproc, opencv_contrib/ximgproc),
- TLD tracker (opencv_contrib/tracking),
- computational photography (photo and opencv_contrib/xphoto),
- dnn module (opencv_contrib/dnn),
- feature detectors (features2d and opencv_contrib/xfeatures2d),
- non-dnn-based object detection (opencv_contrib/dpm and opencv_contrib/xobjdetect).
- SGBM stereo correspondence algorithm
- You are welcome to propose for optimization some other functionality, though.
OpenCV 4.0 is going to be written in Halide so that the compiler can be used to quickly optimize OpenCV on all the different hardware platforms that are proliferating.
- See the Halide project here
- We need:
- Optimizing exists algorithms or some parts using Halide.
- Extend implementations for GPU targets.
- Deep learning layers and topologies.
- Automatic domain-specific scheduling.
- Bringing significant improvements into gstreamer backend of videoio
- Hardware-accelerated video codecs via libyami (https://github.com/01org/libyami)
- Replacing jasper with openjpeg to encode and decode Jpeg 2000 images
- Any other good suggestions/projects to reduce OpenCV dependency of ffmpeg
Gary Bradski
CTO Arraiy, CEO/Co-founder OpenCV
Founder of OpenCV
Organized the Vision Team for Stanley the Robot that
won the 2005 DARPA Grand Challenge and started
the autonomous driving industry. Now in the
Smithsonian Museum
Startups (leading role):
Video Surf -- Founding shares, sold to Microsoft 2010
Industrial Perception -- Founder, CTO sold to Google 2013
Magic Leap -- Founded Silicon Valley office, built to 110 employees
Arraiy -- Founder CTO
garybradski(+)gmail.com
Vadim Pisarevsky
OpenCV team lead at Intel
Co-founder of OpenCV
Former principal engineer at Itseez.inc (sold to Intel, 2016)
vadim.pisarevsky(+)gmail.com
- Alexander Mordvintsev zzznah(+)gmail.com
- Sid Bao baoyingze(+)gmail.com
- Vincent Rabaud vincent.rabaud(+)gmail.com
- Vadim Pisarevsky vadim.pisarevsky(+)gmail.com
- Maksim Shabunin maksim.shabunin(+)gmail.com
- Adrian Kaehler therealadrian(+)gmail.com
- Terry Boult tboult(+)vast.uccs.edu
- spmallick spmallick(+)taaz.com
- Serge Belongie sjb344(+)cornell.edu
- Stefano s.fabri10(+)gmail.com
- Prasanna pras.bits(+)gmail.com
- Pablo Alcantarilla pablofdezalc(+)gmail.com
- Bence Magyar mw.mzperx(+)gmail.com
- Manuele manuele.tamburrano(+)gmail.com
- Grace Vesom grace.vesom(+)gmail.com
- Open Source Computer Vision Library (OpenCV) garybradski(+)gmail.com
- Douglas Lee dougabug(+)gmail.com
- Claudia Rapuano c.rapuano(+)gmail.com
- Antonella Cascitelli antonellacascitelli(+)gmail.com
- Anatoly Baksheev anatoly.baksheev(+)intel.com
- Alexander alexander.shishkov(+)intel.com
- Alexander Smorkalov alexander.smorkalov(+)intel.com
- Alexander Bovyrin alexander.bovyrin(+)intel.com