By Doug Petkanics

Livepeer’s mission has always been to build the world’s open video infrastructure. By pairing the best free to use open source video software, with an optimally cost effective, maximally reliable, and infinitely scalable infrastructure network, Livepeer has the opportunity to be the backbone for the future of video over the internet. The path to achieving this ambitious mission is a worthwhile challenge, and being that the project is 7 years in the making, it is worth checking in on the progress and next steps to move closer to this vision.

The Livepeer ecosystem has evolved tremendously since the last time the project’s technical roadmap was revisited two years ago. At the time, the priorities could be categorized as:

1. Driving Demand through wrapping Livepeer within robust products that the market accepted to meet their video streaming needs.

2. Expanding the Core Technology through adding additional capabilities to the Livepeer Network.

3. Ushering in Decentralization, through layer 2 scalability, governance, and more.

Focus areas one and three saw incredible progress through the evolution and growth of the Livepeer Studio hosted gateway and the many applications built on top, as well as the Confluence upgrade which migrated Livepeer to L2, onchain decentralized protocol governance, and the Delta upgrade introducing a community governed onchain treasury. Capability expansion was achieved through the Livepeer integrated software like MistServer, and hosted products like Livepeer Studio and MuseMe, which added these types of capabilities - however attempts to add this natively into the Livepeer network remained in R&D mode.

Now that it’s been a couple of years, and the project has reached its next level of maturity through a growing Studio gateway product, and sustainable community funding available through Delta, it’s time to revisit the key technical opportunities that the ecosystem can consider taking on with urgency in the coming months. There are three key themes that are critical for Livepeer to achieve its vision.

1. Increase the pace of innovation by making permissionless development on top of the Livepeer software and network easier for developers.

2. Add additional video-focused capabilities to the network itself, enabling node operators to earn economically and play a key role in the future of AI-based video compute, content ingest, delivery, and additional job types.

3. Continue strengthening the network’s resilience through decentralization, achieving full trustless protocol status.

Accelerate the pace of innovation

Catalyst - A modern media server as Livepeer Broadcast Node

Perhaps the most important item on the project roadmap is shipping a well documented, easy to use, updated Livepeer Broadcast Node. This should double as a full featured media server allowing video developers to build their own custom video workflows and experiences, build and scale permissionlessly against the Livepeer network infrastructure taking advantage of its superpower capabilities, run in a self-hosted environment, and have no reliance on any trusted third party or hosted service.

The original Livepeer Broadcast node, go-livepeer started in broadcast mode, is very limited in its interface and video capabilities. It interacts with the Livepeer network for transcoding video segments, but doesn’t do much else. More mature pieces of software built on top, either bundled many video capabilities behind hosted services (like the very useful Livepeer Studio), or integrated with Studio itself for transcoding (requiring an account and trust in a service provider). A better path would be for a great open source media server to integrate direct with the Livepeer network, enabling a full video development suite, in an open and trustless environment.

There has been an early attempt at this project via the Livepeer Catalyst project, which combines MistServer with go-livepeer. This actually forms the building blocks for the Livepeer Studio hosted gateway, however as a released node replacement, it remains undocumented with the rapid development moving the API targets quickly. One consumer app, Xenon, even hosted and built directly against this software despite the challenges of it being a moving target. It’s time to get a version of Catalyst packaged up, documented, and shipped to the community and user base in an easy-to-use way. This should lower the barrier to innovation in any self-sovereign use case, and lead to far more stakeholders building direct on the network.

There is a working group that meets bi-weekly called the Livepeer Catalyst Hackers Meetup, focused on this goal, and the next session is on Friday, October 27th. Join us!

Continue supporting onchain media innovation

There are undoubtedly new primitives that are unlocked in video when you combine media with blockchains. These touch on the areas of video verifiability, provable content ownership, and collaborative media creation, especially when paired with the impacts of generative AI. Because of Livepeer’s web3 and blockchain integrated roots, open innovation environment, and aligned community, this next wave of disruptive onchain media should certainly be built on top of Livepeer rather than centralized video infrastructures.

The Livepeer ecosystem should try to accelerate innovation in these new areas in two main ways.

1. Participate collaboratively in industry working groups to ensure that we’re at the forefront of the collaborative experiments and standards emerging in this new world. An example here is on the verifiable video track. See Livepeer’s existing onchain verifiable video provenance proof of concept, and collaborative work with the Content Authenticity Initiative.

   The Livepeer Grants program is seeking a community project lead to pick up the existing work done by Livepeer focused engineers on this track, and organize a roadmap to push this forward with the broader community.

2. Accelerate and fund innovative and showcaseable experiments that push the boundaries of what’s now possible, and inspire other builders to incorporate these themes into full fledged products. Innovations in this space emerge from simple examples that inspire a wave of new developers and entrepreneurs to re-use the breakthrough concepts. See Cryptokitties inspiring the NFT wave, Synthetix inspiring DeFi incentives through farming, or X-to-earn concepts yielding from one game’s inspiration. The Livepeer ecosystem should be a thriving hub of these types of innovative experiments as it relates to the future of media, and as such there are a number of builder microgrants, streaming credits programs, and a whole video builder community eager to take part in enabling these and other concepts.

• A 24/7 onchain generative video “channel”, co-owned, with content derived from those coordinating onchain inputs.

• Onchain coordinated post-production and distribution incentives. Video creators should be leveraging and giving co-ownership to their community members who do the work of clipping highlights and sharing them out on social, driving awareness back to the main content and growing the communities.

• Community growth video CMS tools. Countless decentralized communities want to use daily live streams and archived video content libraries to engage with and grow their communities. The traditional CMS’ aren’t cutting it in a world of wallet-based identities and user owned content.

These are just a few of the concepts that the Livepeer ecosystem should be at the forefront of experimenting around, and grants are available to those committed teams and individuals looking to showcase inspiring experiments here. The best way to get involved in these innovations is to join the Livepeer Discord and discuss your ideas in the #developer-lounge channel.

Expanding the Capabilities of the Network - The Road to AI Video Compute

Since its origin, the Livepeer network’s bread and butter has been performing various flavors of video transcoding. Despite all the capabilities of the surrounding software for video development, the node operators are primarily performing one function - receiving a video segment in, and transcoding it into different outputs. It’s long been dreamed that the network can perform additional job types, and in R&D it has even been used to do various forms of AI based video compute - scene classification, content type detection, closed caption generation, object detection in video, and more. Recent capabilities in AI, and the explosion of demand for GPU based AI compute, mean that Livepeer could be the leading video compute network for things like generative video, deep fake detection/generation, realistic avatar generation, and more.

In addition to those compute capabilities, the Livepeer software is used in other elements of the video pipeline, from ingest, to origin services, to content delivery networks, and in interfacing with storage providers and indexing services.

It’s time to take the concrete steps to bring some of these capabilities to the node operators and network itself. This will open up more revenue opportunities for Orchestrators, more products built on top of the network, and more independence from third party services. The two areas required to push forward on the roadmap to get there are job type abstraction, and security+verification research.

Job Type Abstraction

As mentioned above, currently the Livepeer protocol and client software supports transcoding. Node operators register on chain, advertise a price they’ll charge for transcoding offchain, and understand the security + verification process for confirming a transcode job was done correctly.

Job Type Abstraction is about removing the assumption of transcoding into the Livepeer protocol and software, and instead abstracting it out so that nodes can register on chain for, and perform, multiple job types. Imagine that onchain nodes are advertising a [job type, verification function] pair that defines the type of job and verification function that they’ll subject themselves to for that job to enforce slashing in the case that they cheat. The verification function is necessary to give confidence and security to the user in a trustless environment. Offchain, they can give an independent price for the different job types that a Broadcaster requests. And the client software, for both Orchestrators and Broadcasters knows how to pass job inputs/outputs and micropayment tickets around for these additional job types.

The protocol goes from being just a transcoding resource allocation protocol to an abstract compute and services protocol, where users can coordinate and leverage any service provided by O’s on the network, as long as they are both using clients that support that capability. It is strongly suggested that the ecosystem retain focus on video use cases initially, because the productization chain of going from capability on the network all the way up to usable product and go to market is long. A video-focus is mission aligned, and will attract a lot more help from existing developers and design partners. In the long term these job types can even expand beyond video compute and services.

A recommended first step here is identifying and building a spike of one end to end additional job type that fits nicely with existing transcoding requests, such as closed caption generation for videos. It would be easy to embed the open source Whisper model into a client spike to support this, and one could quickly hack the protocol and payments changes to demonstrate this capability. We will learn a lot by building, seeing, and using the spike, and we can move towards actual protocol LIPs and production client support from there.

There will be an open grant RFP for a small group or individual to take on this spike, and please join the #video-builders channel to introduce yourself and discuss if you’re interested.

Security + Verification Research

Full verification on video transcoding so that it can be trusted in a decentralized and permissionless environment has been an ongoing area of research since project inception. A Truebit style verification game works for deterministic CPU transcoding, but is not directly possible in a non-deterministic GPU environment. As such the fast+full verification framework is in place to give users a high degree of confidence that their segments of video were transcoded correctly, but the “full” half of the equation still needs more research to be deployed in a slashing-enabled environment.

Going from transcoding to additional job types will require even more research and solutions when it comes to verification. And the bounds of what “verification” means likely needs to move beyond predictable verified compute, to instead take into account things like slashable stake deposits, reputation, and governance based mechanisms like judiciary panels to weigh in on disputes in the case of clearly presented evidence of misdeeds.

The existing body of research and potential paths gets a bit technical for this post, but needless to say, that this track pairs hand in hand with job type abstraction, and working groups should form and be expanded to push this forward in the near term.

Trustless Protocol Status Through Decentralization

Livepeer has long been one of the most decentralized DePIN and resource allocation protocols, and has made significant progress on its Path To Decentralization since launch. Given its decentralized governance, widely distributed stake, fully decentralized node operator network, and community governed treasury, it has all the building blocks in place to take the final steps towards reducing dependence and single points of failure on any small groups or entities.

1. Binding on chain execution of protocol upgrades and parameter updates via governance.

   These types of upgrades are governed by the community through the LIP process, but require the security committee to actually execute the upgrades. With the introduction of the Governor framework for the Delta treasury management, the similarities in governance mechanism over the treasury and LIP process, and the capability of that framework to execute any onchain action it has permission to, there exists a possible straightforward path to this framework being used to execute binding protocol upgrades on chain, removing the role of the security committee in this process.

   Of course the actual governance mechanisms should be debated and potentially modified should the community wish to evolve governance itself as well, but the scope of governance mechanisms is a bit beyond the technical scope of automating execution here.

2. Introducing direct community governed control over the security committee process itself. Once again, the committee’s power is ratified via LIP-19, and it operates according to a social contract commitment. This includes a commitment to modify its own size or processes via the LIP process. However it would be more trustless and decentralized if the governance process itself was binding on the security committee - including adding or removing keyholders, modifying signing thresholds, or removing the committee altogether. The community could also consider a formal election process, like recently seen in the Arbitrum ecosystem, should it determine that to be a valuable step in decentralized control over the committee and protocol.

Engaging in the processes towards decentralization is open to all through Livepeer’s LIP process, and the #governance channel in Discord is a great place to get involved in the discussion.

A Flourishing Hotbed of Video Innovation

Some items on this list of themes are high priority and instantly actionable, such as shipping Catalyst as an open media server Broadcast node. Others will be ongoing and neverending aspirational objectives, like perfect verification on infinite job types on the network. Ultimately, they each play a key role in helping the ecosystem surrounding the world’s open video infrastructure become the hotbed of video innovation that it has the potential to be.

There’s absolutely no reason that developers shouldn’t choose to use the best free and open source video software, run on the cost effective, scalable, and reliable video network that is Livepeer, should it meet their product and infrastructure needs. If you’re interested in getting involved and helping to bring this vision to reality join us in our community Discord today and introduce yourself!