MIT’S Pensieve is a system intent on reducing video buffering

MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) has come up with a solution to reduce the video buffering problem that plagues online streaming despite the advances in internet bandwidth. Titled Pensieve, the system takes an AI-based approach towards delivering video by compressing it on the fly using a complex system of neural networks.

The process can be glimpsed at in the below video uploaded on the lab’s official YouTube channel. Unlike compression methods employed by, say YouTube and now other streaming sites which downgrade or upgrade the quality of the video after examining the internet to ensure the video keeps playing, Pensieve’s neural network uses past performance and tries to predict what quality could be required in the video frames ahead. It employs a neural network for the task that keeps feeding in data on the basis of which it decides the algorithm to be used for compressing and delivering the video on the fly, altering the video quality over time to achieve a smoother playback. MIT explains that Pensieve can be targeted to focus on speed, quality or a mix of both.

The research is available on a paper co-authored by Hongzi Mao, Ravi Netravali and Mohammad Alizadeh. In the paper, the trio explain that where Pensieve differs from traditional state-of-the-art methods is that the latter employ static algorithms that change the video quality over fixed points. For instance, in the case of YouTube, the video is generally encoded in certain pre-defined quality points: 144p, 240p, 360p, 480p, 720p and 1080p (with support for up to 2160p in case of 4K videos) and the algorithm keeps switching between these qualities depending upon the internet connection. With Pensieve though, the algorithm is subject to change based on factors like the amount of activity in the video and how prior frames have played out. The result, the paper quotes, is an impressive 12 to 25 percent improvement in Quality of Experience (QoE) over current algorithms.

The team has open-sourced their findings with the entire source-code written in python being available on GitHub. The team hopes that current streaming companies will adapt the concepts behind Pensieve and implement them in their streaming services. Looking at the current state of the streaming market, Google seems most likely to give it a shot with YouTube seeing how heavily invested the company is in AI, with CEO Sundar Pichai even pivoting and boldly declaring Google’s focus from a Mobile-First to an AI-First company back during Google IO. Also, with that level of compression, the change could certainly benefit developing markets where bandwidth is often found to be in acute shortage, which is again a market where Google is paying a lot of attention to.