Skip to content

GSoC_2016

Jump to bottom
Maksim Shabunin edited this page Dec 18, 2017 · 25 revisions

Ideas Page for OpenCV Google Summer of Code 2016 (GSoC 2016)

OpenCV Augmented Reality from ArUco Markers last year. (click image for video showing all 2015 projects)

General Information:

  • Next important dates:
    • Feb 19 19:00 UTC: Mentoring organization application deadline. Submitted!
    • Feb 29 19:00 UTC: List of accepted Organizations. We are in!!
    • Mar 14 16:00 UTC: Student Application period begins! Submit proposals!
    • Mar 25 19:00 UTC: Student Application period ends! 95 proposals!
    • Deadline for OpenCV to request # of slots.
    • Google publishes how many slots we get. We have 13 slots!
    • Try to eliminate duplication of students. None
    • IRC for OpenCV to resolve all duplications.
    • Apr 22 19:00 UTC: Accepted students announced. Accepted Proposals, Or here:
    • May 23 Let Coding begin!
    • Jun 20 19:00 UTC: Mid term evaluation begins. Pull requests must be in
    • Jun 27 19:00 UTC: Mid term evaluation ends. Pull requests must be in
    • Aug 23 19:00 UTC: Coding stop; Pull requests must be in; Mentors: final evaluation begins…
    • Aug 29 19:00 UTC: Final evaluation deadline.
    • Late October: Mentor Summit!

Times:

  • UTC to PDT (California uses PST in the winter (from Nov 1st) and PDT in the summer (from March 8)).

Resources:

How you will be evaluated if you are an accepted student

  • Student projects to be paid only if:
    • Midterm:
      • You must generate a pull request
        • That builds
        • Has at least stubbed out functionality
        • With appropriate Doxygen documentation
        • Has at least stubbed out unit test
        • Has a stubbed out example of use that builds
    • End of summer:
      • A full pull request
        • Full Doxygen documentation
        • A good unit test
        • Example of use code
      • Create a (short!) Movie (preferably on Youtube, but any movie) that demonstrates your code

For students interested in applying

  1. You must already know how to program fluently in C++
    • Some projects may require Python or Matlab skills
  2. Ask to join the OpenCV GSoC Forum List
    • Discuss projects below or other ideas with us there (Feb 29 – March 13)
  3. On March 14th – 25th, Go to the GSoC site and sign up to be a student with OpenCV
  4. Post the project from below or your own agreed on project on the GSoC to opencv-gsoc-2016@googlegroups.com
    • Include Name, google email, age
    • Include how you think you are qualified to accomplish this project (skills, courses, relevant background)
    • Include Country of origin, school you are enrolled in, Professor you work with (if any)
    • Include a projected timeline and milestones for the project
  5. Once (and if!) OpenCV gets accepted as GSoC 2016 org (Feb 29th), and we are told how many slots we will get and you’ve signed up for a project with us (March 14-25th)
    • We will weight the students and projects against mentors and their interests and choose which students/project to pursue.
    • Accepted students will be posted on the GSoC site on April 22nd (and we will notify the accepted students ourselves).

For computer vision professionals interested in mentoring

  1. Go to the GSoC site and sign up.
  2. Request to be a mentor for OpenCV there.
  3. We accept or reject you
  4. Students submit projects and you
    • find a project you want to mentor (several students might try for the same project)
    • contact us about it through opencv-gsoc-2016@googlegroups.com so we can be sure there is not someone already handling that project
    • contact the students to assess which one if any are capable of coding that up in a summer.
  5. We get a slot allocation from Google, the administrators then “spend” the slots in order of priority
  6. Students must finally actually accept to do that project (some sign up for multiple organizations and then choose)

If you are accepted as a mentor and you find a suitable student and we give you a slot and the student signs up for it, then you are a mentor.

  • It sounds harder than it is.

2016 Project Ideas:

Students may propose their own projects (give us a clear summary and why you can do this project). However, below are some of our priorities for this year Contact us and/or discuss ideas at https://groups.google.com/forum/?fromgroups#!forum/opencv-gsoc-2016

These are not in order of priority

1. Google Cardboard

  • Do visual odometry, SLAM etc for Google Cardboard

2. Deep learning

  • It would be cool if OpenCV could load and run deep networks trained with popular DNN packages like Caffe, Theano or Torch.
  • GPU/OpenCL optimization and/or other optimizations (that can partly be contributed back to the upstream projects) are also encouraged.

3. Projection mapping (Projected Augmented Reality)

  • Calibrate a camera to a projector
  • Put out structured light to learn the 3D map of what’s in front of the camera
  • Project the camera image, suitably rotated and shifted onto the scene
  • Unity or some other program to start transforming the scene for projection mapping

4. Update OpenCV file storage so that it’s readable by Python in OpenCV (move to YAML 1.2 Edition 3)

5. openh264 support

  • We are currently use ffmpeg, which is great library, but: 1) its API changes often and 2) it does not have its own h264 encoder and instead uses GPL-ed x264.
  • There is BSD-licensed openh264 library that includes H264 encoder and decoder. It may be interesting to use this library directly (although ffmpeg can also use it) as light-weight method to get full support for H264.

6. Better interactive camera calibration

  • Modern algorithms would allow us to recompute camera parameters after almost each frame, compute reprojection error, compute confidence intervals for the parameters.
  • the partially occluded calibration patterns could be handled somehow to, especially if we want to estimate intrinsic parameters only.
  • the application may show us the areas where the corner borders have been shown so that user can see how good are the data

7. Color Calibration and white balance

  • With McBeth chart and with gray world algorithm (improve exisiting color constancy algorithms)
  • Possibly collect deep learning way of doing this (needs large training set)

8. Tracking improvements

  • TLD algorithm is implemented in OpenCV, but it needs improvements and optimizations.
  • ASM/AAM can also be implemented, as well as the particle filter.

9. Improving ICF/ACF + Boost training and detection

  • In opencv_contrib there is initial version of the training application and detector based on integral channel features and boosting. Further improvements of this part would be really useful.

10. lua+OpenCV

  • the other language that is now used more often for ML and CV (thanks to Torch) is Lua; there are old bindings for a tiny part of OpenCV; could be useful to extend our autogenerated bindings to Lua.

11. human interaction

  • gesture recognition
  • human skeleton based on 2d or 3d

12. Quad Rotor Contrib

  • Apply much of the above into a quad rotor drone automatic tracking, mapping navigation etc module

13. Efficient dense optical flow algorithm

  • OpenCV will benefit greatly from having some real alternative for the Farneback’s dense optical flow algorithm. A few other algorithms that we’ve implemented (e.g. SimpleFlow, TVL1) are very, very slow. It would be nice to have an algorithm that is at least as fast as Farneback and gives a better quality at the same time.

14. Speeding up various OpenCV parts (it can be multiple projects).
OpenCV is great library, but some of its algorithms may be made more efficient. Here is a list of things that can be optimized:

  • text detection module
  • shape module
  • bioinspired module
  • hdr algorithms from ximgproc and xphoto.
  • non-local-means denoising algorithm from photo module
  • kaze & akaze features
  • you name it – proposals are welcome

15. Parameter optimization and quality estimation of machine learning methods.

Now in OpenCV there is a bunch of ML algorithms (SVM, Boosting, Random Trees, etc.). Except for SVM that has trainAuto() method, all other algorithms need all the parameters to be explicitly specified by user. However, it would be convenient to implement some automatic parameter selections for other models as well:
1) cross-validation for better quality estimation of ML models
2) GridSearch or/and RandomSearch for parameter optimization
3) … possibly some more complicated parameters search techniques (like usage of genetic algorithms, direct optimization methoids or Bayesian optimization, see links in http://blog.dato.com/how-to-evaluate-machine-learning-models-part-4-hyperparameter-tuning)

16. Improve text recognition accuracy in opencv_contrib/text module.

  • Currently we have our own implementation of a shallow CNN-based character recognizer but it’s slow (CPU only) and based in a paper of 2012. It would be interesting to adapt the module to use one of the several Deep CNN models have been made public during the last year (both for character and word recognition) e.g.[http://www.robots.ox.ac.uk/~vgg/research/text/]. Any other further improvement of the module is also welcome.

New 2016 ideas:

  • Architecture
    • Exceptions in OpenCV need to be easily disabled ? (need to check if that’s the case or not)
    • Minimal OpenCV: give the functions you want, it gives you a minimal version of the library.
    • reduce the size of OpenCV on Android
    • create a more C++ type (to be able to deal with constness and separate the pointer from the type)
    • create an image type (that would also have color information, maybe calibration too) and implement a proper resize algorithm for that type

Accepted Projects for OpenCV GSoC 2016:

OpenCV GSoC 2016 students, projects, mentors are below. WHEN THEY ARE KNOWN

GSoC Progress. Need Pull requests (don’t have to be complete) to pass by July 3rd

Mailing Lists (only for use of Mentors and Students, please to not request to join these lists otherwise):

  • Mentors
    • opencv-gsoc2016-mentors
  • Students
    • opencv-gsoc2016-students

Pre-Midterm status:

Project notes will be compiled here .

Mentors

Actual signed up mentors

  • Alexander Mordvintsev zzznah(+)gmail.com
  • Sid Bao ybao(+)magicleap.com
  • Vincent Rabaud vincent.rabaud(+)gmail.com
  • Vadim Pisarevsky vadim.pisarevsky(+)gmail.com
  • Adrian Kaehler therealadrian(+)gmail.com
  • Terry Boult tboult(+)vast.uccs.edu
  • spmallick spmallick(+)taaz.com
  • Serge Belongie sjb344(+)cornell.edu
  • Stefano s.fabri10(+)gmail.com
  • Prasanna pras.bits(+)gmail.com
  • Pablo Alcantarilla pablofdezalc(+)gmail.com
  • Bence Magyar mw.mzperx(+)gmail.com
  • Manuele manuele.tamburrano(+)gmail.com
  • Grace Vesom grace.vesom(+)gmail.com
  • Open Source Computer Vision Library (OpenCV) garybradski(+)gmail.com
  • Douglas Lee dougabug(+)gmail.com
  • Claudia Rapuano c.rapuano(+)gmail.com
  • Antonella Cascitelli antonellacascitelli(+)gmail.com
  • Anatoly Baksheev anatoly.baksheev(+)itseez.com
  • Alexander alexander.shishkov(+)itseez.com
  • Alexander Smorkalov alexander.smorkalov(+)itseez.com
  • Alexander Bovyrin alexander.bovyrin(+)itseez.com

In the below, get rid of the ‘’-delete-’’ to make the emails work.

Anatoly Baksheev
Researcher, Vision Algorithms on GPU
Argus/Itseez
-delete-Anatoly.Baksheev@-delete-itseez.com

Alexander Bovyrin
PhD, Senior Researcher
Argus/Itseez founder
NNU Lecturer
-delete-alexander.bovyrin@-delete-itseez.com

Gary Bradski
Founder, Industrial Perception Inc.
Consulting Prof. Stanford U.
OpenCV Founder, Technical Content Owner, GSoC Admin
Co-author of Learning OpenCV Book
http://www.willowgarage.com/pages/people/gary-bradski-senior-researcher
-delete-garybradski@-delete-gmail.com

Antonella Cascitelli
Grad student, University of Rome
-delete-antonellacascitelli@gmail.com

Eric Christiansen
PhD, UCSD
-delete-echristiansen@-delete-cs.ucsd.edu

Stefano Fabri
CRR Team leader, University of Rome
-delete-s.fabri10@gmail.com

Victor Eruhimov
OpenCV founding team/Senior Researcher
Argus/Itseez founder
NNU Lecturer
-delete-relrotciv@-delete-googlemail.com

Adrian Kaehler
Principle Engineer, Applied Minds
Co-author of Learning OpenCV Book.
-delete-therealadrian@-delete-gmail.com

Peter Karasev
PhD Student, MINERVA Research Group
Georgia Tech
-delete-karasevpa@-delete-gmail.com

Vadim Pisarevsky
OpenCV founding team/Czar
-delete-Vadim.-delete-Pisarevsky@-delete-gmail.com

Manuele Tamburrano
Grad Student, University of Rome
-delete-manuele.tamburrano@gmail.com

Vincent Rabaud
Principle Engineer, Aldebaran Robotics
-delete-vincent.rabaud@-delete-gmail.com

Claudia Rapuano
Grad student, University of Rome
-delete-c.rapuano@gmail.com

Grace Vesom
Senior Software Engineer, Magic Leap
-delete-grace.vesom@-delete_gmail.com

Bence Magyar
Pal Robotics
-delete-bence.magyar@-delete-pal-robotics.com

Pablo Alcantarilla
Toshiba Research Europe Ltd.
-delete-pablofdezalc@-delete-gmail.com

Back up Mentors

Mark Asbach
Fraunhofer IAIS
Schloss Birlinghoven
Sankt Augustin, Germany
http://mmprec.iais.fraunhofer.de/asbach.html
-delete-mark.asbach@-delete-iais.fraunhofer.de


Nicolas Saunier, Ph.D.
Assistant Professor
Civil, Geological and Mining Department (CGM)
École Polytechnique de Montréal
http://nicolas.saunier.confins.net
-delete-nicolas.saunier@-delete-polymtl.ca


Alexander Mordvintsev
Software Engineer
http://znah.net
-delete-zzznah@-delete-gmail.com

Andrey Morozov
Software Engineer
Argus/Itseez
-delete-andrey.morozov@-delete-itseez.com

OpenCV Org. Application to GSoC 2016

OpenCV_GSoC_Application

Previous Years

© Copyright 2024, OpenCV team

Clone this wiki locally