100y

Posted on Jul 05, 2024Read on Mirror.xyz

Professors in the House: SAKSHI, Decentralizing AI Inference

Compute will become the future currency and we need AI inference open marketplace.

What is the purpose of blockchain technology? While it can be utilized across various fields such as social networks, gaming, and commerce, I believe, as discussed in my previous article “Will it be the same as ever?: Money, AI, and Blockchain,” that the ultimate endgame for blockchain is to serve as a hard currency and an anti-thesis to AGI.

Building on this idea, numerous projects have emerged aiming to address the issues within the centralized AI industry through the decentralized nature of blockchain. These projects are typically referred to as decentralized AI (dAI) initiatives. A major development in the dAI sector occurred recently when Sentient, a project with Sandeep Nailwal, co-founder of Polygon, as a core contributor, raised $85 million in seed funding.

As the AI industry advances, leading language models like GPT-4 and Claude 3.5 Sonnet are mostly operated as closed-source. Sentient aims to counter this by building community-driven open-source AI models. By leveraging blockchain protocols, developers can 1) monetize their models, 2) collaborate to collectively build AI models, and 3) become stakeholders in an Open AGI economy. According to Sandeep, Sentient will be built on the Polygon AggLayer, suggesting it will be a zk L2 based on Polygon CDK.

(Source: SAKSHI: Decentralized AI Platforms)

During my research on Sentient, I came across an intriguing project called SAKSHI. Clicking the research tab on Sentient’s website automatically redirects to the Open AGI Research forum, where the post “SAKSHI: Decentralized AI Platforms” is featured. This post caught my attention primarily due to its authors, which include Sreeram Kannan (founder of EigenLayer), co-founders of Sentient, and professors from prestigious universities.

1. SAKSHI: Decentralized AI Platforms

1.1 EigenLayer, Babylon, … SAKSHI?

Recently, there has been an increase in prominent university professors and students from the United States building projects in the crypto scene. Historically, examples include Silvio Micali from MIT, UPenn, UToronto, and Tsinghua University with Algorand, and Emin Gun Sirer from Cornell University with Avalanche. More recently, notable examples are Sreeram Kannan (University of Washington) with EigenLayer and David Tse (Stanford) with Babylon.

In my opinion, compared to the AI industry, the blockchain industry lacks significant academic involvement, which implies that examples above are very promising. So, what are the professors involved in Sentient and SAKSHI aiming to build? Before diving into the projects, let’s take a look at the backgrounds and research areas of the professors involved.

Sentient

  • Pramod Viswanath: Forrest G. Hamrick Professor in Engineering in the Department of Electrical and Computer Engineering, Princeton University (Blockchain, Deep Learning, Wireless Communication)

  • Himanshu Tyagi: Associate Professor in the Department of Electrical Communication Engineering, Indian Institute of Science (Blockchain, Privacy, Federated Learning, Statistics, …)

  • Sewoong Oh: Professor, Allen School of Computer Science & Engineering, University of Washington (Machine Learning, Federated Learning)

SAKSHI

  • Suma Bhat: Assistant Professor in Electrical and Computer Engineering at the University of Illinois at Urbana-Champaign (Machine Learning, Natural Language Processing)

  • Zhixuan Fang: Assistant Professor in the Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University (Blockchain, Collaborative Learning, Network Economics)

  • Sreeram Kannan: Affiliate Associate Professor in the Department of Electrical & Computer Engineering, University of Washington (Blockchain)

  • Xuechao Wang: Assistant Professor in Thrust of Fintech at HKUST (Blockchain, DeFi)

  • Pramod Viswanath: Forrest G. Hamrick Professor in Engineering in the Department of Electrical and Computer Engineering, Princeton University (Blockchain, Deep Learning, Wireless Communication)

  • Himanshu Tyagi: Associate Professor in the Department of Electrical Communication Engineering, Indian Institute of Science (Blockchain, Privacy, Federated Learning, Statistics, …)

All are renowned university professors, and experts in either blockchain or AI. Given the recent success of notable professor-led projects like EigenLayer and Babylon, I couldn't help but be intrigued by SAKSHI.

1.2 Motivation & Aim of SAKSHI

The issue the professors aim to solve is straightforward: the centralization of AI. In the future, most computing is expected to be used for inference rather than training, and currently, inference is highly centralized. Companies like OpenAI and Anthropic offer closed-source LLMs through web interfaces or APIs, raising potential issues of privacy, transparency, and rent-seeking.

(Source: SAKSHI: Decentralized AI Platforms)

SAKSHI aims to create a decentralized marketplace for inference using blockchain. Clients needing inference can request tasks on SAKSHI, and numerous AI suppliers can provide AI models and computing power on the platform, earning appropriate rewards. SAKSHI addresses several potential issues with this open marketplace approach:

  1. Insufficient Clients for Individual Suppliers: SAKSHI introduces aggregators to collectively provide services on behalf of suppliers.

  2. Poor Quality or Irrelevant AI Models: SAKSHI uses proof of inference to verify that tasks are completed correctly.

  3. Non-Payment by Clients: SAKSHI enforces payments through SLA contracts and proof of service delivery via smart contracts.

In essence, SAKSHI’s core goal is to establish an open aggregate marketplace for inference, where clients and suppliers can freely transact, enhancing decentralization and transparency by removing the need for third-party trust through blockchain, smart contracts, and proof systems.

1.3 The Six Layer Architecture

(Source: SAKSHI: Decentralized AI Platforms)

SAKSHI consists of six layers: Service Layer, Control Layer, Transaction Layer, Proof Layer, Economic Layer, and Marketplace. The first two layers are web2 components, while the remaining four are related to blockchain. A brief explanation of each layer is provided below, with detailed descriptions covered in “2. Architecture of SAKSHI”:

  1. Service Layer: Facilitates the exchange of inference services between clients and servers.

  2. Control Layer: Matches clients with servers based on server network state and client requests.

  3. Transaction Layer: Handles payments for services.

  4. Proof Layer: Resolves disputes arising from inaccurate inferences or AI model duplications.

  5. Economic Layer: Ensures the economic security of the SAKSHI platform.

  6. Marketplace: A decentralized platform for buying and selling inference services.

1.4 Process in a Nutshell

(Source: SAKSHI: Decentralized AI Platforms)

Before delving into detailed explanations of each layer, let's take a high-level look at how the protocol operates. Given that SAKSHI includes an Aggregator to mediate between Clients and Servers, the first step involves signing SLAs (Service Level Agreements) through smart contracts between Clients and Aggregators, and Aggregators and Servers (Transaction Layer).

When a Client requires AI inference, they send a request via the API (Service Layer). The Aggregator then matches this request to an appropriate Server based on the AI models, computing power, and other relevant factors available (Control Layer). The Server provides the AI model and inference service, receiving appropriate compensation. If the inference is inaccurate, challengers can raise disputes, which are handled within the Proof Layer.

2. Architecture of SAKSHI

2.1 Service Layer

(Source: SAKSHI: Decentralized AI Platforms)

The Service Layer facilitates the actual exchange of inference services between clients and servers (AI suppliers), similar to a traditional web2 server-client architecture. When a client sends an inference query, the Control Layer matches it with the appropriate server. Once the server provides the AI model and inference service, payments are transferred from the client to the server via the Transaction Layer. To prevent malicious activities, signed inference requests, output data posted on the decentralized application (DA) layer, and previously exchanged micropayments can be used for dispute resolution.

2.2 Control Layer

(Source: SAKSHI: Decentralized AI Platforms)

The Control Layer tracks the network state, including model capacity, hardware capacity, request load, and location of servers. It also monitors SLA contract information from the Transaction Layer between client-aggregator and aggregator-server, ensuring appropriate client-server matching based on this data.

2.3 Transaction Layer

(Source: SAKSHI: Decentralized AI Platforms)

The Transaction Layer handles payments for inference services. Since SAKSHI is a decentralized system, it ensures seamless payments through smart contract-coded Service Level Agreements (SLAs), facilitating payments upon service delivery. This layer utilizes decentralized middleware provided by Witness Chain.

2.4 Proof Layer

(Source: SAKSHI: Decentralized AI Platforms)

The Proof Layer resolves disputes related to malicious activities within SAKSHI. It includes Proof of Inference, which verifies the accuracy and validity of AI model computations, and Proof of Model-ownership, which addresses intellectual property disputes. Witness Chain AVS operators act as challengers in these disputes.

Proof of Inference

There are two methods for verifying the accuracy of inference: zkML (zero-knowledge machine learning) and opML (optimistic machine learning). zkML is impractical due to its time and cost inefficiency for complex AI computations.

(bisection scheme | Source: Offchain Labs)

SAKSHI adopts opML, where challengers contest the validity of computations by re-running models. To manage large computations, SAKSHI employs Arbitrum's bisection scheme, which breaks down large computations into smaller parts.

(Source: SAKSHI: Decentralized AI Platforms)

AI models, often non-sequential, are structured as Directed Acyclic Graphs (DAGs) in SAKSHI to identify faulty entries efficiently.

Proof of Model-ownership

To address the issue of malicious actors copying and monetizing open-source AI models, SAKSHI requires AI model suppliers to embed watermarks during training. These watermarks, committed to the blockchain, help trusted judges verify if an AI model has been stolen. This system, however, is limited to the SAKSHI platform and cannot prevent external misuse of open-source models.

2.5 Economic Layer

In SAKSHI, incentivized challengers report malicious activities, ensuring fair behavior among entities in a decentralized system. Instead of issuing its governance token, SAKSHI leverages EigenLayer, which utilizes the crypto-economic security of staked ETH, providing substantial security. Protocols using EigenLayer are known as AVS, with Witness Chain serving as an AVS for the Transaction Layer. The security of SAKSHI is expected to depend on the re-staked ETH delegated to Witness Chain.

3. Final Thoughts: Computing as the Future Currency

(Source: Flourish)

Kojo Osei, a partner at Matrix VC, recently predicted that the cost of inference will approach zero due to increasing competition among AI models and the enhanced performance of consumer device hardware. I agree, anticipating that in a few years, decent AI model inference will be feasible on user devices.

If this prediction holds true, computing could become a future currency, as suggested by Sam Altman. Similar to torrent services, individuals could earn rewards by contributing idle computing power from their devices to the network. For this to happen, a decentralized open marketplace must exist, allowing transparent participation and rewards. Will SAKSHI play a pivotal role in this future?

Recommended Reading