Introduction
FFmpeg is the universal multimedia toolkit: a complete, cross-platform solution to record, convert, filter, and stream audio and video. It is used by a number of media players (including VLC and MPlayer), video websites (including YouTube and Vimeo), web browsers (including Google Chrome/Chromium and Firefox), social media (including Facebook and Twitter) and countless other software projects in the multimedia field and beyond.
This is our ideas page for Google Summer of Code 2019. See the GSoC Timeline for important dates.
At the end of the program you can find all the results on the results page.
Information for Students
Getting Started
- Get to know FFmpeg. If you are a student interested in contributing to FFmpeg, it is recommended to start by subscribing to the ffmpeg-devel mailing-list, visiting our FreeNode IRC channels (#ffmpeg-devel and #ffmpeg although you will find most developers on the first one), and exploring both the codebase and the development workflow. Feel free to contact us if you have any questions. Also do not hesitate to answer questions from other students on our IRC channel if you know the answer to something.
- Find a project. Listed on this page are mentored and un-mentored projects. Mentored projects are well-defined and mentor(s) have already volunteered. Un-mentored projects are additional ideas you may want to consider, but you will have to contact us to find a mentor. You can also propose your own project, if you can think of one that better fits your interest and skill level. If a project description is unclear or you have any questions, please get in touch with its mentor and/or join our IRC channel at #ffmpeg-devel.
- Contact us. If you decide on a project, get in touch with the community and let us know. If you want to work on a qualification task, let the respective mentor know so we can avoid duplicated efforts.
- Apply. Students should apply definitely before deadline on April 9th. The "work" period begins on May 27th and ends in August. Take a look at GSoC timeline for additional information. Note, make sure you apply to Google before the 9th even if you have not yet finished your qualification task. And please apply as soon as possible: Applications can be improved until the 9th but not afterwards!
Note: A friendly reminder that while the application to GSoC is important for you and GSoC, FFmpeg mentors will not base their decision solely on the GSoC application. We will judge applicants based on their qualification tasks to understand their abilities in coding, learning the tools, communication skills etc. So please do not worry about your application being perfect for us. Although it is very important to follow GSoC's application rules so they can pay you.
Qualification Tasks
In order to get accepted you have to complete a small qualification task which in all cases include sending a patch to the development mailing list. FFmpeg development can be quite challenging and the qualification task helps us figure out whether you are motivated enough and have the potential to deliver successfully.
The qualification tasks are usually shown in the project description. Contact the respective mentor(s) for assistance on getting a related qualification task or if you want to propose your own. You can also browse the FFmpeg Bug Tracker for qualification task ideas. In general qualification tasks should include submitting a patch to the ffmpeg-devel mailing list which passes review and is accepted into the FFmpeg codebase. It will be common for such patches to need multiple iterations of submissions and reviews, so don't wait too long with the first submission! Note, please avoid picking a qualification task which another student is already working on, each student should work on a different qualification task.
Development
If you are selected for a particular project then you are not only expected to present a working implementation but you should also submit your work for inclusion for the ffmpeg codebase. This should be done at least 2-3 weeks before the end of the second work period by sending patches to the ffmpeg-devel mailing list where the ffmpeg community and your mentor will review your work. You will likely be asked to make some changes and resend improved versions. If you feel that no consensus is reached about how something should be done then follow the advice of your mentor.
In order to create good quality patches make sure to read the Developer Documentation.
Contacting FFmpeg
If you have questions or comments feel free to contact us via our mailing list, IRC channel, or e-mail one of the FFmpeg GSoC admins:
- Mailing-list: ffmpeg-devel
- IRC: #ffmpeg-devel on Freenode
- FFmpeg GSoC Admins: Michael Niedermayer (michaelni in #ffmpeg-devel on Freenode IRC, michaelni@gmx.at), Reynaldo Verdejo (reynaldo in #ffmpeg-devel on Freenode IRC, R Verdejo on g mail), Carl Eugen Hoyos (cehoyos in #ffmpeg-devel on Freenode IRC, ceffmpeg at gmail dot com), Thilo Borgmann (thilo.borgmann at mail dot de)
You may also contact a mentor directly if you have questions specifically related to one of the projects listed on this page.
Mentored Projects
This section lists well-defined projects that have one or more available mentors. If you are new to FFmpeg, and have relatively little experience with multimedia, you should favor a mentored project rather than propose your own. Contact the respective mentor(s) to get more information about the project and the requested qualification task.
Derain Filter
Description: Use deep learning derain techniques to implement a filter which can remove the rain of the input image or video. The student should evaluate the best suited method based on state of the art methods
Expected results: A working drain filter
Prerequisites: Good C and C++ coding skills, knowledge of machine learning and deep learning, basic familiarity with git, familiarity with at least one deep learning framework.
Qualification Task: Implement a ffmpeg derain filter which shows good performance in deraining process.
Mentor: Steven Liu (lq [at] chinaffmpeg [dot] org)
Backup Mentor: Pedro Souza (bygrandao [at] gmail [dot] com)
DICOM support
Description: FFmpeg currently does not support DICOM files although this is a widely used image format that is often passed to patients who have no access to DICOM equipment.
Expected results: Support reading as many features of the format as possible.
Prerequisites: Good C coding skills, basic familiarity with git
Qualification Task: Write a simple demuxer for a format currently unsupported by FFmpeg. The following tickets contain possible tasks, not all of them are equally simple / difficult, some may turn out to be less useful: #5154, #4748, #3986 (this may be too difficult or impossible), #3720, #2956, #2662 and #2176. Not all possible features of a demuxer have to be implemented (the more the better) but at least auto-detection and video decoding (and remuxing) should be possible to implement for all of them. Tell us if you want to work on one of the tasks to avoid two students working on the same qualification task!
Mentor: Carl Eugen Hoyos (cehoyos in #ffmpeg-devel on Freenode IRC, ceffmpeg at gmail dot com)
Backup Mentor: Thilo Borgmann (thilo.borgmann AT mail de)
HEIF support
Description: FFmpeg currently does not support reading HEIF/HEIC files although this is a format with increasing usage in mobile devices.
Expected results: Support reading the sample files that are attached to ticket #6521.
Prerequisites: Good C coding skills, basic familiarity with git
Qualification Task: Write a trivial (!) heif muxer or see the list of tickets for the DICOM project above.
Mentor: Carl Eugen Hoyos (cehoyos in #ffmpeg-devel on Freenode IRC, ceffmpeg at gmail dot com)
Backup Mentor:
Complete DNG support
Description: FFmpeg currently does not support completely DNG raw image format.
Expected results: Fully support decoding all variants of DNG raw image formats.
Prerequisites: Good C coding skills, basic familiarity with git. Able to read and understand DNG specification.
Qualification Task: Improve current DNG support in some minor way.
Mentor: Paul B Mahol (onemda [at] gmail [dot] com)
Backup Mentor:
360° video filter
Description: Write filter which will be supporting all projections (equirectangular, cubemap, etc), as output and input, found on Internet.
Expected results: A working, full featured 360° video transformation filter which will also allow to extract FoV and do roll/yaw/pitch.
Prerequisites: Good C coding skills, basic familiarity with git.
Qualification Task: Find filter posted on FFmpeg development mailing list which deals with two kind of projections and extend it.
Mentor: Paul B Mahol (onemda [at] gmail [dot] com)
Backup Mentor: Thilo Borgmann (thilo.borgmann AT mail de)
Video filtering with OpenCL
Description: Write video filters to perform useful common operations on the GPU.
Currently FFmpeg has good support for offloading decoding and encoding from the CPU (with support for DXVA2, D3D11, VAAPI, VDPAU, V4L2-M2M, and then many manufacturer-specific APIs), but not such good support for doing anything else with that video without using the CPU. There is interop support for mapping between APIs so that generic methods like OpenCL which can be run on any GPU can be used together with specific video APIs for decoding and encoding, but currently there are only a few operations which can actually use this so many use-cases require expensive additional steps to download frames from the GPU and then upload them again later after performing some processing on the CPU. Therefore, we would like to add more OpenCL filter support to libavfilter so that more operations can be offloaded.
In particular:
- Scaler, supporting a choice of scaling algorithms.
- Deinterlacer, probably based on an existing filter such as yadif or bwdif.
- Colour conversion, for both RGB and YUV, supporting different ranges and colour spaces.
- Any other video filters you would find useful!
There are also some related tasks which may be helpful:
- Adding more interop modes if you have a device which isn't currently supported.
- Adding support for testing GPU filters. Currently the regression test suite only runs in software, extending it to be able to test the functionality of GPU filters would allow better testing - this is especially helpful given the wide variety of different devices it can run on.
Expected results: More filters which can run on the GPU, increasing the scope of operations which can be offloaded to the GPU and the pipelines which can run on the GPU only.
Prerequisites:
- C coding ability and familiarity with git.
- Some knowledge of GPU concepts and programming (need not be OpenCL, much is transferrable so easy to learn).
- OpenCL-compatible GPU - easiest with Intel or AMD devices running on Linux or Windows; likely possible with many others. More than one from different manufacturers would be helpful for testing. CPU implementations can also be tested, but should be avoided for the main implementation work because they have very different performance properties.
Qualification task: Implement a new video filter for libavfilter which uses OpenCL. This can be a copy of any existing CPU-only filter or a new filter of your choice (it need not be efficient or do anything complex).
Mentor: Mark Thompson (jkqxz in #ffmpeg-devel, <sw@jkqxz.net>)
Backup mentor:
AVIF support
Description: FFmpeg currently does not support reading avif files.
Expected results: Support demuxing and remuxing the sample files that are attached to ticket #7621.
Prerequisites: Good C coding skills, basic familiarity with git
Qualification Task: Write a trivial (!) avif muxer or see the list of tickets for the DICOM project above.
Mentor: Tomas Härdin (tjoppen at acc dot umu dot se)
Backup Mentor: Carl Eugen Hoyos (cehoyos in #ffmpeg-devel on Freenode IRC, ceffmpeg at gmail dot com)
Unmentored Projects
This is a list of projects that students are encouraged to consider if a mentored project is unavailable or not within the students skill or interests. The student will have to find a mentor for the project. A student can also propose their own project.
JPEG 2000 Decoder improvements
Description: FFmpeg contains a native JPEG 2000 decoder. It is missing many features, see also the FFmpeg bug tracker for some unsupported samples. This task is difficult
Expected results: A JPEG 2000 decoder which can decode more samples.
Qualification Task: Fix an issue (for example from the bug tracker) to show that you are capable of improving the codec implementation.
Prerequisites: C coding skills, basic familiarity with git, good knowledge of Jpeg-2000
JPEG 2000 Encoder improvements
Description: FFmpeg contains a native JPEG 2000 encoder. It is missing many features, see also the FFmpeg bug tracker. This task is difficult
Expected results: A JPEG 2000 encoder that achieves higher compression.
Qualification Task: Fix a bug in the encoder or make a change that shows that you are capable to improve the encoders compression.
Prerequisites: C coding skills, basic familiarity with git, good knowledge of Jpeg-2000 and concepts like rate distortion.
Your Own Project Idea
A student can propose a project. Ideas can also be found by browsing bugs and feature requests on our bug tracker. The work should last the majority of the GSoC internship duration, the task must be approved by the developers, and a mentor must be assigned.
Students can discuss an idea in the ffmpeg-devel mailing-list, the #ffmpeg-devel IRC channel, or contact the FFmpeg GSoC admins for more information.
