From your favorite Netflix show, to a viral YouTube clip that your friend sent you, to Zoom calls with your colleagues, video is everywhere. While video has never been more ubiquitous for consumers, for developers it remains one of the most challenging use cases to integrate.

“There’s a lot of complexity to delivering video that works across all possible contexts for as many users as possible.”

Said Kwindla Kramer, co-founder of Daily, a company that designs video APIs that make it easier to build and embed video into websites and applications.

On our latest episode of the Decoded podcast, Kramer shared his insight into the challenges of incorporating video and what developers need to consider when exploring use cases.

 

The Challenges of Video

Although you can watch almost any video you want no matter where you are, keep in mind that online video has only reached maturity relatively recently. Netflix didn’t start experimenting with streaming until 2007, YouTube started in 2005, and Zoom was founded in 2011. Not only that, but it's only really been in the last few years that we’ve seen the explosion of new streaming services, video collaboration applications, and the integration of video into social media.

Kramer said that the same transition that happened with the internet and mobile — from interesting technology to ubiquitous experience — is now happening to video, with video becoming layered on top of everything that everybody does. As he explained:

“A bunch of pieces had to come together for it to become widely adopted. The first piece that had to come together was enough CPU and enough bandwidth available all the time, wherever you happen to be, on any device. Even more challenging from an engineering perspective is that a live video experience can't tolerate any latency, buffering, or packet loss. Live video has to work no matter what device you’re on, no matter where in the world you are, and no matter what kind of network connection you have.”

When building for video, Kramer explained that there are three things that make video a particularly difficult challenge to manage as companies scale:

  1. Video usually consumes more CPU and bandwidth than any other function in your app.
  2. Video is still rapidly developing as a technology, with best practices changing by the day forcing developers to relearn the technology as they go. The rapid evolution of mobile means that new capabilities are constantly being introduced, which requires developers to spend extra time on maintenance and updates.
  3. The challenge that every developer faces: making it work at scale.
“One of the things we see over and over is that as you scale, you have these problems with people in different geographies, which is a challenge because video is so bit-heavy.”

Using Frameworks to Get Ahead

To overcome those challenges, Kramer says that developers are now leveraging frameworks that allow developers to quickly integrate video without starting from scratch.

“Our whole world is moving toward the way the payments world moved, where you don't try to build it yourself, but instead build on top of whatever the best fit is for your use case. That lets you as an application developer focus on what you really want to do with the video instead of getting the plumbing to work.”

Of course, different video use cases require different approaches. Video use cases like Netflix and YouTube are considered on-demand, which allows the app to dynamically access chunks of video at a time so that it can ensure a consistent video experience.

If the network struggles to accommodate it, the app can downshift to a lower bit rate and preload video onto the device so that the app can access it locally instead of from a server. Live broadcasts can build in a delay of 20 seconds to allow traffic to buffer, ensuring a quality experience.

However, real-time video like a Zoom call requires 200 milliseconds of latency, not 20 seconds. For these use cases, developers need to access a completely different set of protocols and standards. While on-demand video is able to leverage 30 years of internet architecture built on TCP, Kramer says that real-time video leverages architectures that have been in existence only for ten years, placing them further behind the architecture cycle.

“Trying to deliver video at scale for lots of different use cases cost-effectively is really new. It’s all still very bespoke; there aren’t as many building blocks yet that are commoditized.”

Check out this week’s Decoded podcast to hear Kwindla Kramer dive deeper into the technical challenges of video and learn where he sees the development of video evolving from here. Listen now, and subscribe to future episodes.