Introduction
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter, and stream audio and video. It is used by a number of media players (including VLC and MPlayer), video websites (including YouTube and Vimeo), web browsers (including Google Chrome/Chromium and Firefox), social media (including Facebook and Twitter) and countless other software projects in the multimedia field and beyond.
This is our ideas page for Google Summer of Code 2020. See the GSoC Timeline for important dates.
At the end of the program you can find all the results on the results page.
Information for Students
Getting Started
- Get to know FFmpeg. If you are a student interested in contributing to FFmpeg, it is recommended to start by subscribing to the ffmpeg-devel mailing-list, visiting our FreeNode IRC channels (#ffmpeg-devel and #ffmpeg although you will find most developers on the first one), and exploring both the codebase and the development workflow. Feel free to contact us if you have any questions. Also do not hesitate to answer questions from other students on our IRC channel if you know the answer to something.
- Find a project. Listed on this page are mentored and un-mentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Un-mentored projects are additional ideas you may want to consider, but you will have to contact us to find a mentor. You can also propose your own project, if you can think of one that better fits your interest and skill level. If a project description is unclear or you have any questions, please get in touch with its mentor and/or join our IRC channel at #ffmpeg-devel.
- Contact us. If you decide on a project, get in touch with the community and let us know. If you want to work on a qualification task, let the respective mentor know so we can avoid duplicated efforts.
- Apply. Students should apply definitely before deadline on March 31st. The "work" period begins on May 18th and ends in August. Take a look at GSoC timeline for additional information. Note, make sure you apply to Google before the 9th even if you have not yet finished your qualification task. And please apply as soon as possible: Applications can be improved until the 9th but not afterwards!
Note: A friendly reminder that while the application to GSoC is important for you and GSoC, FFmpeg mentors will not base their decision solely on the GSoC application. We will judge applicants based on their qualification tasks to understand their abilities in coding, learning the tools, communication skills etc. So please do not worry about your application being perfect for us. Although it is very important to follow GSoC's application rules so they can pay you.
Qualification Tasks
In order to get accepted you have to complete a small qualification task which in all cases include sending a patch to the development mailing list. FFmpeg development can be quite challenging and the qualification task helps us figure out whether you are motivated enough and have the potential to deliver successfully.
The qualification tasks are usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the FFmpeg Bug Tracker for qualification task ideas. In general qualification tasks should include submitting a patch to the ffmpeg-devel mailing list which passes review and is accepted into the FFmpeg codebase. It will be common for such patches to need multiple iterations of submissions and reviews, so don't wait too long with the first submission! Note, please avoid picking a qualification task which another student is already working on, each student should work on a different qualification task.
Development
If you are selected for a particular project then you are not only expected to present a working implementation but you should also submit your work for inclusion for the ffmpeg codebase. This should be done at least 2-3 weeks before the end of the second work period by sending patches to the ffmpeg-devel mailing list where the ffmpeg community and your mentor will review your work. You will likely be asked to make some changes and resend improved versions. If you feel that no consensus is reached about how something should be done then follow the advice of your mentor.
In order to create good quality patches make sure to read the Developer Documentation.
Contacting FFmpeg
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:
- Mailing-list: ffmpeg-devel
- IRC: #ffmpeg-devel on Freenode
- FFmpeg GSoC Admins: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, michaelni@gmx.at), Reynaldo Verdejo (reynaldo in #ffmpeg-devel on Freenode IRC, R Verdejo on g mail), Carl Eugen Hoyos (cehoyos in #ffmpeg-devel on Freenode IRC, ceffmpeg at gmail dot com), Thilo Borgmann (thilo.borgmann at mail dot de)
You may also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.
Mentored Projects
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.
Audio Tones Source Filter
Description: Sound generation, waveform synthesis, with custom seed control, musical scale, tempo and pitch.
Expected results: Fully-Working and Fully-Featured Audio Tones Source Filter
Prerequisites: Good C coding skills, basic familiarity with Git, advanced knowledge of math.
Qualification Task: Write basic audio source filter which would use fluidsynth and will play random tones, please contact mentor before starting qualification task.
Mentor: Paul B Mahol (onemda [at] gmail [dot] com)
Backup Mentor: Thilo Borgmann (thilo.borgmann [at] mail [dot] de)
FLIF Decoder and Encoder
Description: FLIF is a modern lossless image format with very good compression ratio compared to other formats. Understand the FLIF16 spec and reference codec, then implement a decoder and encoder in FFmpeg.
Expected results: Fully featured FLIF16 Decoder and Encoder.
Prerequisites: Good C coding skills, basic familiarity with Git, ability to read and understand FLIF specification & reference implementation (might contain some unstable parts)
Qualification Task: Write a working decoder under libavcodec for a small subset of FLIF features, and get your patch reviewed by the community on ffmpeg-devel@ mailing list. Inform the mentor(s) before starting to prevent clashes among potential applicants.
Mentor: Jai Luthra (darkapex in #ffmpeg-devel on Freenode IRC, me [at] jailuthra [dot] in)
Backup Mentor: Thilo Borgmann (thilo.borgmann [at] mail [dot] de)
Fuzzing FATE tests in libavformat, libavutil and libswscale
Description: FATE is FFmpegs self / regression test system. Fuzzing with oss-fuzz is used to find corner cases which trigger anomalies like undefined behavior. FATE tests are sub optimal to be part of fuzzing as they use hardcoded parameters as input for each test. For example libavutil/tests/des.c runs some random and fixed tests but no tests controlled by some data from a fuzzer. So even if a anomaly occurs during a random test, this would not be reproducible nor could a fuzzer direct execution towards more anomalous cases.
Expected results: Connect most libavformat/tests/*.c libavutil/tests/*.c libswscale/tests/*.c tests to oss-fuzz so that the fuzzer can fully affect the data in each test.
Prerequisites: Good C and Makefile coding skills, basic familiarity with Git, knowledge of oss-fuzz.
Qualification Task: Connect 2 of the 60 FATE tests from libavutil / libavformat / libswscale to work under oss-fuzz, one from libavutil and one from libavformat. Passing qualification will be judged by having working code pushed into FFmpeg git and oss-fuzz git before the deadline given to us for choosing projects.
Mentor: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, michaelni@gmx.at)
Backup Mentor: Carl Eugen Hoyos (ceffmpeg [at] gmail [dot] com)
Fuzzing FATE tests in libavcodec, libavfilter and libswresample
Description: FATE is FFmpegs self / regression test system. Fuzzing with oss-fuzz is used to find corner cases which trigger anomalies like undefined behavior. FATE tests are sub optimal to be part of fuzzing as they use hardcoded parameters as input for each test. For example libavcodec/tests/snowenc.c runs some random and fixed tests but no tests controlled by some data from a fuzzer. So even if a anomaly occurs during a random test, this would not be reproducible nor could a fuzzer direct execution towards more anomalous cases.
Expected results: Connect most libavcodec/tests/*.c libavfilter/tests/*.c libswresample/tests/*.c src/tests/*.c tools/*.c tests/tools to oss-fuzz so that the fuzzer can fully affect the data in each test/tool.
Prerequisites: Good C and Makefile coding skills, basic familiarity with Git, knowledge of oss-fuzz.
Qualification Task: Connect 2 of the 59 FATE tests and tools from libavcodec / libavfilter / libswresample / src / tools to work under oss-fuzz, one from libavcodec and one from libavfilter. Passing qualification will be judged by having working code pushed into FFmpeg git and oss-fuzz git before the deadline given to us for choosing projects.
Mentor: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, michaelni@gmx.at)
Backup Mentor: Carl Eugen Hoyos (ceffmpeg [at] gmail [dot] com)
ABR meets FFmpeg
Description: Adaptive bitrate~(ABR), aiming to pick next chunks' bitrate to overcome the fluctuation of network conditions, is a non-trivial method to ensure good quality-of-experience (QoE) for Internet video (such as DASH and HLS). Exploring the aforementioned FFmpeg projects, however, lacks the essential ABR module. To that end, support ABR methods in FFmpeg.
Expected results: Add the ABR module into HLS or DASH.
Prerequisites: Good C coding skills, basic familiarity with Git, expert of ABR algorithms.
Qualification Task: Fix any open bug(s) on trac, get your patch reviewed on ffmpeg-devel@ mailing list. Inform the mentor(s) before starting to prevent clashes among potential applicants.
Mentor: Steven Liu (lq [at] chinaffmpeg [dot] org)
Backup Mentor: Jun Zhao (mypopy [at] gmail [dot] com)
JPEG XL Decoder
Description: JPEG XL, inspired by FLUF and PIK, is designed as a web friendly successor to JPEG. The specification is under active development and should be stable within a couple of months. A reference implementation is available with a limited set of features.
Expected results: Understand the spec and implement a fully featured JPEG XL decoder.
Prerequisites: Good C coding skills, basic familiarity with Git, ability to read and understand specification & reference implementation (might contain some unstable parts)
Qualification Task: Fix any open bug(s) on trac OR implement a small component of the decoder like parser, header reader etc. Get your patch reviewed on ffmpeg-devel@ mailing list. Inform the mentor(s) before starting to prevent clashes among potential applicants.
Mentor: Jai Luthra (darkapex in #ffmpeg-devel on Freenode IRC, me [at] jailuthra [dot] in)
Backup Mentor: Thilo Borgmann (thilo.borgmann [at] mail [dot] de)
Improving Color Constancy Video Filter
Description: Color Constancy is the ability to perceive colors of objects, invariant to the color of the light source. A Color Constancy video filter typically tries to remedy the effect of the scene illumination either by estimating it and then correcting the scene colors accordingly or directly mapping the scene colors to their most probable origin. Currently, FFmpeg has a basic variation that was developed in a previous GSoC project; but color constancy is an active field of research and a lot of new variations have been introduced that uses advanced image processing algorithms and machine learning.
Expected results: Adding new variations to the current Color Constancy video filter.
Prerequisites: Good C coding skills, basic familiarity with Git, knowledge of machine learning and image processing, familiarity with at least one deep learning framework.
Qualification Task: implementing http://refbase.cvc.uab.es/files/GGW2012.pdf.
Mentor: Mina Sami (minas [dot] gorgy [at] gmail [dot] com)
Backup Mentor: Thilo Borgmann (thilo.borgmann [at] mail [dot] de)
Unmentored Projects
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.
JPEG 2000 Decoder improvements
Description: FFmpeg contains a native JPEG 2000 decoder. It is missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult
Expected results: A JPEG 2000 decoder which can decode more samples.
Qualification Task: Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.
Prerequisites: C coding skills, basic familiarity with Git, good knowledge of Jpeg-2000
JPEG 2000 Encoder improvements
Description: FFmpeg contains a native JPEG 2000 encoder. It is missing many features, see also the FFmpeg bug tracker. This task is difficult
Expected results: A JPEG 2000 encoder that achieves higher compression.
Qualification Task: Fix a bug in the encoder or make a change that shows that you are capable to improve the encoders compression.
Prerequisites: C coding skills, basic familiarity with Git, good knowledge of Jpeg-2000 and concepts like rate distortion.
Your Own Project Idea
A student can also propose a self-defined project! The amount of work defined by such a project should last for the majority of the GSoC work period duration and the task must be approved by the community. Also, a capable mentor from the community must be assigned. Ideas and inspiration for such project ideas could be found by browsing bugs and feature requests on our bug tracker but are of course not limited to such.
Students can discuss an idea with the community via the ffmpeg-devel mailing-list, the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins directly for more information. It is obviously of importance to start such a discussion well before the application deadline.
Improving and implementing advanced Video Frame Interpolation
Description: FFmpeg already has a vf_miterpolate to generate high frame videos. I would like to work on this using the default dnn code provided by FFmpeg and implement new algorithms which improve over previous methods.
Expected results: A fast and efficient filter for video interpolation with different models for different speeds and accuracy by July end and various other useful modules such as flow estimation, etc, required by the model by Phase 2 evaluation.
Qualification Task: Use the DNN module to implement and train a flow estimation network for learning optical flow.
Prerequisites: Good C coding skills, basic familiarity with Git, knowledge of machine learning and image processing, familiarity with deep learning frameworks, familiarity with motion, flow and depth estimation.
Proposed by: Yatendra Singh | email : yatendras@iitbhilai.ac.in | irc : yatendra1999
