Author: PSE Trading Analyst @Minta

Key Insights

• AI Agent, based on the LLM model, facilitates the creation of user-friendly interactive applications and services.

• The AI industry's future primarily revolves around "General LLM Models + Niche Applications." AI Agents serve as the middleware linking these general models and Dapps, with relatively low competitive barriers. To maintain long-term competitiveness, they rely on fostering network effects and enhancing user engagement.

• This article outlines developments in the Web3 gaming sector, covering "General LLM Models, Niche Application Agents, and Generative AI Applications." The incorporation of Generative AI technology holds considerable potential for rapidly launching hit games.

01 Reintroducing LLM

In the realm of the rapidly evolving Artificial General Intelligence (AGI) technology, Large Language Models (LLMs) have taken center stage. Core technologists at OpenAI, Andrej Karpathy and Lilian Weng, have asserted that AI Agents based on LLMs represent a pivotal direction for future developments in the AGI field. Numerous teams are also engaged in the development of AI Agents driven by LLMs. In essence, an AI Agent is a computer program that utilizes extensive data and complex algorithms to simulate human thinking and decision-making processes, enabling it to perform a wide array of tasks and interactions, such as autonomous driving, speech recognition, and game strategy, etc. Abacus.ai provides a clear depiction of the working flow of AI Agents, which can be outlined as follows:

• Data Collection: AI Agents gather information from various sources, including sensors, cameras, microphones, etc, covering aspects such as game states, images, and sounds.

• Data Transformation: The collected data is processed and converted into a format that the Agent can understand, often as vectors or tensors suitable for neural networks.

• Neural Network Models: The decision-making and learning processes predominantly rely on deep neural network models. Like convolutional neural networks (CNNs) for image processing, recurrent neural networks (RNNs) for handling sequential data, etc.

• Reinforcement Learning: Agents acquire optimal action strategies through dynamic interactions with the environment.

In a nutshell, AI-Agents are like clever companions: they understand, make decisions, and take actions on their own. They can be real game-changers in various fields, especially in the gaming world. A tech whiz from OpenAI, Lilian Weng, wrote a comprehensive article titled " LLM Powered Autonomous Agents ," which delves into the principles behind these AI-Agents. In it, there's a fascinating experiment known as Generative Agents.

Generative Agents (GA) drew inspiration from the "Simulated Life" game. They use Large Language Models (LLM) to create 25 virtual characters, each controlled by an Agent powered by LLM, living and interacting in a sandbox environment. GA's design is pretty clever, and it combines LLM with memory, planning, and reflective functions, allowing Agent programs to make decisions based on past experiences and interact with other Agents.

The article goes into great detail about how Agents continually train and optimize their decision-making paths using strategic networks, value networks, and interactions with their environment.

Here's how it works: The Memory Stream is like a long-term memory module that stores all of an Agent's past experiences. The Retrieval model provides memories based on relevance, freshness, and importance, helping the Agent make decisions, which is Planning. The Reflective mechanism summarizes past events, guiding the Agent's future actions. Planning and Reflection work together to help the Agent turn reflection and environmental information into final actions.

This intriguing experiment shows us what AI Agents can do. All in all, AI-Agents are a really fascinating tool, and their applications in gaming are definitely worth delving into.

02 Navigating AI Trends

2.1 AI Landscape

ABCDE's partners have outlined the prevailing consensus in Silicon Valley's VCs regarding the future trends in AI Agent development:

1. Future industry landscape is leaning towards "large universal LLM models alongside specialized applications."

2. AI based on edge devices presents an opportunity because data collection from edge devices has a competitive advantage.

3. With the increase in Context length, we can expect more amazing use cases.

Looking at the typical industry trends, it's clear that there won't be multiple large universal models on the market due to their substantial size and high development costs. The real opportunities lie in how to leverage these large universal models to create applications tailored to specific niche domains.

Simultaneously, when we talk about "edge devices," we refer to devices that usually operate independently of cloud computing centers or remote servers, performing data processing and decision-making locally. Given the diversity of edge devices, deploying AI Agents on them to run smoothly and effectively gather device data is both a challenge and a fresh opportunity.

Lastly, the matter of "Context" is under the spotlight. To put it simply, in the context of LLM (Large Language Models), "Context" can be understood as the volume of information, and "Context length" relates to how many dimensions the data has. For instance, consider a big data model for an e-commerce website used to predict the likelihood of a user purchasing a particular product. In such a scenario, Context might encompass the user's browsing history, purchase history, search records, and user attributes, among other data points. Context length, on the other hand, refers to the stacking of feature information, like the purchase history of a 30-year-old male user from Shanghai, combined with recent purchase frequency and recent browsing history, and so on. Expanding Context length can assist the model in gaining a more comprehensive understanding of the factors influencing a user's purchase decisions.

Currently, the consensus is that while using vector databases as AI memory limits Context length, the future holds the promise of a qualitative shift in Context length. Subsequently, LLM models can explore more advanced methods for processing and comprehending longer and more intricate Context information, leading to the emergence of unexpected and imaginative application scenarios.

2.2 AI-Agent Landscape

Folius Ventures has summarized the application patterns of AI Agents in the gaming track as shown in the following diagram:

In the diagram, symbol 1 represents the LLM model, primarily responsible for translating user intentions from traditional keyboard/click inputs into natural language inputs, thereby reducing the user's entry barrier.

Symbol 2 is the frontend Dapp integrated with AI Agents. It not only provides functional services to users but also collects user habits and data from the terminal.

Symbol 3 encompasses various AI Agents, which can exist directly in the form of in-app functionalities, bots, etc.

Overall, AI Agents, as code-based tools, can serve as the underlying programs for extending Dapp functionalities and act as catalysts for platform growth. They act as intermediaries linking LLM models with specialized applications. In terms of user scenarios, Dapps most likely to integrate AI Agents are generally open Social apps, Chatbots, games, etc. Alternatively, they can be used to transform existing Web2 traffic gateways into more user-friendly AI+Web3 gateways, addressing the ongoing industry discussion about lowering the user barrier to Web3.

Following industry trends, the middleware layer where AI Agents operate often becomes a highly competitive arena with very few moats. Therefore, apart from continually enhancing the user experience to meet B2C demands, AI Agents can enhance their competitiveness by creating network effects or fostering user stickiness.

03 Market Landscape

AI's applications in the world of Web3 gaming have seen a variety of different approaches, which can be grouped into the following categories:

1. General LLM Models: Some projects focus on creating versatile AI models specifically tailored to meet the demands of Web3 projects. They search for the right neural network and general models that fit the bill.

2. Niche Applications: Niche applications are designed to address specific issues within games or provide specialized services. They often come in the form of Agents, Bots, BotKits, etc.

3. Generative AI Applications: The most straightforward application of large models is content generation. In the gaming domain, which thrives on content, Generative AI applications are particularly noteworthy. They enable automatic generation of elements, characters, missions, and storylines within virtual worlds. Moreover, they can autonomously evolve game strategies, decisions, and even the in-game ecosystem, injecting more diversity and depth into the gaming experience.

4. AI Games: Presently, many games have integrated AI technology into their gameplay, each with its own unique application scenarios. Examples of these will be discussed in the following sections.

3.1 General LLM

Currently, in the realm of Web3, there already have simulation models designed for economic model creation and the development of economic ecosystems. One such example is the Quantitative Token Model (QTM).

During his presentation at ETHCC, Dr. Achim Struve from Outlier Venture shared insights into economic model design. For instance, in consideration of the robustness of economic systems, project teams can employ Large Language Models(LLM) to create a digital twin, essentially a 1:1 simulation of the entire ecosystem.

The QTM (Quantitative Token Model) depicted in the diagram is an AI-driven model. QTM operates with a fixed simulation period of ten years, with each time step representing one month. At the beginning of each time step, tokens are minted into the ecosystem, leading to the inclusion of incentive modules, token ownership modules, airdrop modules, and more within the model. These tokens are then allocated into several meta-buckets for further fine-grained generalized utility redistribution. Subsequently, reward payments and other aspects are defined based on these utility tools. Additionally, considerations extend to off-chain business aspects, such as general financial conditions, allowing for actions like burning or buybacks. It can also measure user adoption rates or define user adoption scenarios.

Of course, the quality of the model's output depends on the quality of its input. Therefore, before using the QTM, thorough market research is essential to gather more accurate input information. However, the QTM model is already a well-established application of AI-driven models in Web3 economic models. Many project teams have leveraged the QTM model to create user-friendly 2C/2B applications, thereby reducing the entry barriers for project participants.

3.2 Niche Applications

Niche Applications primarily exist in the form of Agents, and agents can take various forms, such as Bots, BotKits, virtual assistants, intelligent decision support systems, etc.

Typically, Agents leverage OpenAI's general models as their foundation, complemented by other open-source or in-house technologies like Text-to-Speech (TTS). They also incorporate specific data for FineTuning, a machine learning and deep learning training technique aimed at optimizing a pre-trained model that has been trained on a large-scale dataset, enabling the creation of AI Agents that outperform ChatGPT in a specific user scenario.

Currently, the most mature application in the Web3 gaming domain is the NFT Agent. There's a consensus within the gaming field that NFTs are an integral part of Web3 games. With the development of metadata management technologies in the Ethereum ecosystem, programmable dynamic NFTs have emerged. For NFT creators, these NFTs can be made more versatile through algorithms. For users, there's greater interactivity with NFTs, generating valuable interaction data. AI Agents optimize the interaction process and expand the application scenarios for interaction data, injecting innovation and value into the NFT ecosystem.

Case one: Gelato's development framework allows developers to customize logic for updating NFT metadata based on off-chain events or specific time intervals. Gelato nodes trigger metadata changes when certain conditions are met, enabling automatic updates of on-chain NFTs. For instance, this technology can be used to fetch real-time sports data from a sports API and automatically upgrade an NFT's skill attributes under specific conditions, such as when an athlete wins a game.

Case Two: Paima also offers application-specific Agents for Dynamic NFTs. Paima's NFT compression protocol mints a set of minimal NFTs on L1 and then evolves them based on the game state on L2, providing players with a more immersive and interactive gaming experience. For instance, NFTs can change based on factors like a character's experience points, task completion status, equipment, and more.

Case Three: Mudulas Labs Labs is a well-known project in the ZKML space and has made strides in the NFT domain as well. Mudulas has introduced the zkMon NFT series, allowing AI-generated NFTs to be minted on-chain while generating zero-knowledge proof (zkp). Users can use this zkp to verify that their NFT was indeed generated by the corresponding AI model. For more detailed information, please refer to Chapter 7.2: The World’s 1st zkGAN NFTs.

3.3 GA's Application

As mentioned earlier, since gaming itself falls within the realm of content creation, AI Agents are capable of generating a large volume of content quickly and cost-effectively. Hence, Generative AI finds excellent applications in the gaming domain. Currently, Generative AI's applications in the gaming field can be summarized into the following main categories:

1. AI-generated Game Characters: This involves scenarios such as battling AI opponents, having AI simulate and control in-game NPCs, or even directly generating characters using AI.

2. AI-generated Game Content: Here, AI directly generates various in-game elements like missions, storylines, items, maps, and more.

3. AI-generated Game Environments: This type of application supports the use of AI to automatically generate, optimize, or expand the game world's terrain, landscapes, ambiance, and more.

3.3.1 AGI - Characters

Case One: MyShell

MyShell is a platform for creating Bots where users can tailor their own Bots to chat, practice conversational skills, play games, or even seek psychological counseling, among other things. Additionally, MyShell incorporates Text-to-Speech (TTS) technology, enabling the automatic creation of Bots that mimic anyone's voice with just a few seconds of voice samples. Beyond this, MyShell employs AutoPrompt, allowing users to give instructions to the Large Language Model (LLM) solely by describing their ideas, laying the foundation for private large-scale language models (LLMs).

Users of MyShell have expressed that its voice chat feature is exceptionally smooth, with response times even faster than GPT's voice chat, and it also features Live2D.

Case Two: AI Arena

AI Arena is an AI battle game where users can continuously train their battling sprites (NFTs) using LLM models and then send their well-trained sprites into PvP/PvE battles. The battle mode is similar to Nintendo's Super Smash Bros, but AI training adds an extra layer of competitive fun. Paradigm has led to investment in AI Arena, and it is currently in the public testing phase. Players can enter the game for free and also have the option to purchase NFTs to enhance their training strength.

Case Three: Leela vs the World

Leela vs the World is a chess game developed by Mudulas Labs. In this game, the opponents are an AI and a human player, with the game state stored in a smart contract. Players interact with the game using their wallets, and the AI reads the updated game state, makes decisions, and generates a Zero-Knowledge Proof (zkp) for the entire computation process. Both of these steps are executed on AWS cloud infrastructure, while the zkp is handed over to the on-chain smart contract for verification. Once the verification is successful, the smart contract triggers the "move" in the chess game.

3.3.2 AGI - Content

Case One：AI Town

AI Town, a collaborative creation between a16z and their portfolio company Convex Dev, draws inspiration from Stanford University's "Generative Agent" paper.

Think of AI Town as a virtual town where every AI resident can craft its own unique story through interactions and experiences. The magic behind AI Town involves a technology stack featuring Convex's serverless backend framework, Pinecone's vector storage, Clerk for authentication, OpenAI's natural language text generation, and Fly for deployment.

What makes AI Town even more exciting is that it's entirely open-source, allowing in-game developers to customize various components, such as feature data, sprite sheets, Tilemap visuals, text generation prompts, game rules, and logic. This flexibility ensures that not only regular players but also developers can explore and create a wide range of applications, both within and beyond the game.

In essence, AI Town isn't just a game that generates AI-driven content. It's a thriving development ecosystem and a versatile tool for creators of all kinds.

Case Two：Paul

Paul, the AI Story Generator tailor-made for the world of blockchain gaming! Paul has a knack for crafting AI-generated stories and seamlessly integrating them into blockchain-based games. The secret sauce behind Paul's operation involves feeding a plethora of prior rules into an LLM and allowing players to effortlessly generate secondary content following those rules.

One prime example of Paul in action is the game Straylight Protocol. Paul Seidler, in collaboration with Straylight Protocol, has brought this game to life. Straylight Protocol is a multiplayer NFT game, and at its core, it's a blockchain gaming rendition of "Minecraft." In this exciting world, players can autonomously mint NFTs and, guided by the foundational rules set by the model, embark on the creative journey of constructing their own unique virtual realms.

3.3.3 AGI - Environment

Case One：Pahdo Labs

Pahdo Labs is a game development studio currently hard at work on Halcyon Zero, an extraordinary venture in the realm of anime-inspired fantasy role-playing games and online game creation platforms, all built on the formidable Godot engine. The game unfolds within an ethereal realm, with a bustling town at its heart, serving as a social hub.

What sets this game apart is its exceptional feature that allows players to rapidly craft more 3D immersive backgrounds and even introduce their beloved characters into the game, using AI creation tools provided by the game developers. This game truly empowers the gaming community, equipping them with the tools and gaming environments needed to dive into the world of user-generated content on a grand scale.

Case Two：Kaedim

Kaedim has developed a Generative AI-based 3D model generation tool specifically for game studios. This tool enables game studios to rapidly generate 3D in-game scenes/assets that meet their requirements in large quantities. Currently, Kaedim's universal product is still in development, with an expected release for game studios in 2024.

The core logic of Kaedim's product is identical to that of an AI Agent. It is built on a foundation of a universal LLM model. The team's artists continuously input high-quality data, providing feedback on Agent outputs. Through iterative machine learning training, the model is refined, allowing the AI Agent to produce 3D scenes that meet the specified criteria.

04 Discussion

In this article, we've delved into a comprehensive analysis and summary of AI applications within the gaming industry. It's evident that general models and Generative AI are poised to birth standout unicorn projects in the gaming sector. While niche applications may possess a narrower competitive advantage, their pioneering status offers substantial potential. If they can harness this lead to foster network effects and enhance user engagement, the growth prospects are immense. Moreover, Generative AI naturally aligns with the content-centric gaming landscape, and numerous teams are already venturing into its applications, making it highly likely that we'll witness breakout games driven by GA in the near future.

Beyond the directions highlighted in the article, there are other avenues for exploration in the coming years. For instance:

1. the Data Track + Application Layer: The AI data track has already birthed several multi-billion-dollar unicorn projects, and the synergy between data and the application layer holds considerable promise.

2. Integration with Socialfi: This involves introducing innovative social interaction methods, optimizing community identity verification and governance using AI Agents, and refining personalized recommendations.

3. As Agents become increasingly automated and mature, the question arises: In the future's Autonomous World, will the primary actors be humans or bots? Could on-chain autonomous worlds have a user base predominantly composed of bots, akin to Uniswap, where over 80% of Daily Active Users are bots? If so, exploring governance Agents in conjunction with Web3 governance concepts holds significant merit.

05 Reference

https://messari.io/report/application-scaling-botkits?referrer=research-reports

https://mirror.xyz/1kx.eth/9lMkZYQgrO2G6ei2dQFU6RmulsPHuVxQETK3fATtd9o

https://mirror.xyz/1kx.eth/oBuaEp5jgGbe2gCsa6Z-_mLAeMRUhsIdZsaScHQNXS0

https://docsend.com/view/4rm9mp56ypr5ae6p