Web3dAppDevCamp

Posted on Feb 26, 2022Read on Mirror.xyz

RSS in Crypto@NonceGeekLab

Supported by NonceGeekLab <> PlayerLink

Attention: All articles by NonceGeekLab do not constitute any investment advice.

PlayerLink:

A Trustless Decentralized Web3-Oriented Service Aggregation Protocol. Through reliable, secure, and scalable technology, PlayerLink empowers billions to create and get a new service experience.

NonceGeekLab:

Focus on Crypto, the most interesting track in the world!

https://github.com/WeLightProject/NonceGeekLab

Authors: msfew、leeduckgo、Erqi

0x00 Key questions

Good questions are the key for research

  • Does Web3 need RSS?
  • What kind of economic model will Web3's RSS have?
  • What benefits can be given to users of Web3's RSS?

0x01 Area introduction

These are some objective analysis. The research is completed at January 2022, so some of the contents may not be updated.

1.1 Introduction

In the Web3 era, user data is open source, and developers can go to a public database like the blockchain for free and quick retrieval of data such as NFTs, footprints, DAO members, etc. owned by users.

Web2 social media, such as Twitter, are also actively embracing Web3 NFT artwork as a new way to interact and socialize. And in the context of high blockchain transaction fees and a more limited environment for smart contracts to operate, it is difficult for us to have a complete decentralized social application on Ethereum or other chains at this stage.

OpenSea only provides a personal space for NFT display, Twitter only provides a personal space for Web2 tweets, Mirror only provides a display of articles, and even no native subscription function. Because of this, developers started with feed streams and created aggregators for Web3 content, bringing each platform together in one place, bringing secondary interactions to the blockchain, and gradually exploring the possibilities of decentralized social.

In our article, we will focus on the following Web3 feed projects: RSS3, CyberConnect, Project Galaxy, and Showme.

1.2 Questions

For each project we can ask the following questions:

  1. is the project sufficiently Web3 (decentralized, open source, open data)?
  2. the existing ecosystem and progress of the project?

1.3 RSS3

"The feed of Web3"

Whitepaper

Let's know about RSS first before introducing RSS3.

1.3.1 RSS

RSS is an information aggregating protocol that unifies the format of information distribution and content synchronization. Users can subscribe to a website's RSS feed to receive content updates sorted by timeline, thus aggregating content from all websites.

RSS is characterized by openness and neutrality. In the Web2 era, RSS lost its appeal to the average user with the creation of centralized platforms. A centralized platform began to do evil in the "rodeo" of user data, leading to privacy violations, algorithmic abuses, and data monopolies.

1.3.2 RSS3 Introduction

RSS3 is an underlying indexing and syndication protocol that aggregates both Web2 and Web3 content on top of RSS, enabling efficient and centralized content distribution. It is also redundance-tolerant and fault-tolerant, under the control of the user.

The difference between RSS3 and traditional social media, such as Facebook, is that it gives users control of their feeds back to them. Users have the autonomy to decide what content they receive, rather than being "brainwashed" by a recommendation system.

The RSS3 team members have a strong belief in decentralization, and many of them have blogged about it. In addition, DIYgod, the developer of the well-known open source project RSSHub, is also on the RSS3 team, providing a strong endorsement of the indexing and syndication technology used by RSS3.

1.3.3 RSS3 Products

  • rss3.bio(renamed to cheers.io in 2022/02)

rss3.bio is the first application of RSS3. rss3.bio is similar to QQ space in the Web3 era, showing a user's entire on-chain footprint, including NFT records, Gitcoin donation records, and Twitter and Mirror posting records. Users can get their own RNS with $PASS tokens to create personalized domains like leeduckgo.rss3.bio.

rss3.bio is a very small and compact Web3 personal business card that shows users' engagement with Web3 under one page. rss3.bio is relatively simple and currently only allows sharing business cards and following by domain name.

revery.so is a more complete application based on rss3.bio. revery.so provides full RSS3 feed aggregation, and each user can bring social connections from rss3.bio to revery.so to get their own feed stream.

1.3.4 RSS3 technical architecture

The technical implementation of RSS3 includes the implementation of the protocol and the implementation of the applications implemented using the protocol.

  • RSS3 protocol

RSS3 protocol has interfaces and data structures for each address, and on-chain data such as personal information and NFT assets owned by the address are indexed to facilitate fast data acquisition and interface adaptation in the network.

Each of these addresses will have attributes such as id, profile avatar, personal link, etc.

  • RSS3 network

Based on separate RSS3 files for each address, the entire data network parses the RSS3 files for each address and organizes the network of relationships, and finally indexes accounts that are said to be encrypted or non-encrypted, storing this data on a blockchain network. the blockchain network for RSS3 is overseen by a DAO as well as distributed nodes.

Users interact with the application and receive feed streams from RSS3's network, while the application interacts directly with the RSS3 network to submit user data that needs to be updated to the network.

The source code of rss3.bio and revery.so are both in the RSS3 development group's GitHub account, and both are front-end applications that use the TypeScript language and the Vue or NextJS frameworks. The structure of the application is not complicated, in short, it consists of two main parts: taking data and styling it.

Since the RSS3 protocol has already organized the data very neatly, the front-end application can quickly build a Web3 social application with this same set of data. We can get the user data from the RSS3 network very easily and quickly by using a function like this.

async function getRecommendGroupMembers(type: string) {
  try {
    const res: RecommendationUsersResponse = (
      await axios.get('/recommendation/list', {
        baseURL: config.recommendations.endpoint,
        params: {
          type,
          limit: config.recommendations.userLimit,
        },
      })
    ).data;
    return <RSS3Index[]>res.response.filter((member) => member !== null); // What's wrong with you dude?
  } catch (e) {
    console.log(e);
  }
  return [];
}

Then, once you have the data, just render it in the page component:

    const getRecommendationGroups = async (type: string) => {
        setIsLoadingRecommendGroupMembers(true);
        const recommendGroupMemberIndexes = await RSS3.getRecommendGroupMembers(type);
...

It is very easy to develop social applications under the provisions of the open and neutral RSS3 protocol.

1.3.5 Answers

  1. Is the project sufficiently Web3 (decentralized, open source, open data)?

RSS itself is a very neutral and open protocol, but in the current web culture, the survival space of RSS is getting smaller and smaller. RSS3 has been strengthened and reborn in the Web3 era, and as an open and unified protocol, RSS3 can help applications to get unified data very well, and will promote the development of the whole ecosystem as well as the ERC standard of Ethereumw.

For the decentralized property of RSS3, I think the team's concept and the architecture of the blockchain network are very good. However, I did not have access to specific information about the DAO architecture and nodes of the blockchain network. If this information can also be published, then it will reach the true decentralized network properties.

For open source, RSS3 members are active contributors on GitHub, and almost all projects are open source for developers to learn and view, which is something that is very much in the DNA of Web3.

For open data, I don't currently have access to developer documentation or an open interface to use. I think a better approach is to open up the data for developers to use and build applications first, rather than constructing applications by using the data first. Of course, in the future, when RSS3 develops its own projects that use its protocols to gain a foothold in Web3, RSS3 will certainly make the data public, which is a very good thing.

  1. The current ecosystem and progress of the project?

Web3Pass, which went live in September 2021, currently(January 2022) has 35,000 individual pages. In January 2022, the official RSS3 white paper was released.

1.4 CyberConnect

"Connect everyone on Web3"

1.4.1 CyberConnect introduction

CyberConnect is also a content aggregation and indexing protocol. CyberConnect's mission is to build a social relationship graph infrastructure for Web3, returning ownership of social data to users while providing an infrastructure for all Web3 developers to integrate and build.

CyberConnect's emphasis is not only on aggregating data on the chain together, but more importantly, on empowering users with the Web3 social relationships through decentralized storage. It's as much about the on-chain interactions that the user has at their address as it is about the data they have that they can take with them through the Web3 world. For example, Twitter followers and followers can be brought directly into the CyberConnect application, and interactions in CyberConnect can be brought into other social applications.

CyberConnect believes that Web3 will be developer-driven, so what they are doing is aggregating and indexing decentralized social relationships, and then handing them off to different projects to be presented on the front end.

1.4.2 Cyberconnect projects

Cyberconnect.me is similar to RSS3's two products in that it aggregates data and presents social connections on the chain.

Cyberconnect.me uses a recommendation engine to better recommend potential user interactions, which is more in line with most users' habits. It also provides the ability to comment on users' homepages, allowing secondary interactions to be truly reflected in the Web3 Feed product.

  • CyberChat

CyberChat is a real-time communication application based on CyberConnect's social graphs and is still in development.

1.4.3 CyberConnect architecture

  • CyberConnect Indexer

The open source code of this project is available at this repository. Indexer open-sources the method of aggregating and indexing data from various social networks. We can access the data we need through the contract addresses and APIs of each different platform:.

const (
	SuperrareContractAddress  = "0x41a322b28d0ff354040e2cbc676f0320d8c8850d"
	OpenSeaContractAddress    = "0x495f947276749ce646f68ac8c248420045cb7b5e"
	RaribleContractAddress    = "0xd07dc4262bcdbf85190c01c996b4c06a461d2430"
	FoundationContractAddress = "0x3b3ee1931dc30c1957379fac9aba94d1c48a5405"
	ZoraContractAddress       = "0xabefbc9fd2f806065b4f3c237d4b59d9a97bcac7"
	ContextContractAddress    = "ctx"
)

const (
	ContextUrl          = "https://context.app/api/profile/%s"
	SuperrareUrl        = "https://superrare.com/api/v2/user?address=%s"
	RaribleFollowingUrl = "https://api-mainnet.rarible.com/marketplace/api/v4/followings?owner=%s"
	RaribleFollowerUrl  = "https://api-mainnet.rarible.com/marketplace/api/v4/followers?user=%s"
)
func sendRequest(client *http.Client, args RequestArgs) ([]byte, error) {
	var req *http.Request
	var err error

	switch args.method {
	case "GET":
		req, err = http.NewRequest(args.method, args.url, nil)
		if err != nil {
			return nil, err
		}
		for k, v := range args.header {
			req.Header.Add(k, v)
		}
		query := req.URL.Query()
		for k, v := range args.params {
			query.Add(k, v)
		}
		req.URL.RawQuery = query.Encode()
...

Such an approach is more customizable and flexible than RSS3, but is not as out-of-the-box as RSS3 and may require manual deployment and debugging by the developer.

  • CyberConnect packages

The source code for the CyberConnect npm library is available at this repository. This library provides CyberConnect classes that include functions for DID and interactive social graph data, encapsulating complex authentication logic into easy-to-use functions. Developers can use this library for social software development.

1.4.4 Answers

  1. Is the project sufficiently Web3 (decentralized, open source, open data)?

CyberConnect is more of a developer-oriented project, and it opens up the interface to the ceramic network for developers to use faster than RSS3.

For decentralization, the Ceramic used by CyberConnect is a decentralized network that uses IPFS for storage.

For open source, the third-party libraries and indexer for the CyberConnect project are open source, but the rest of the client code is not.

For open data, CyberConnect has opened up its own data for developers to use.

  1. What is the current ecosystem and progress of the project?

Currently Cyberconnect's client has 38w users, and the account with the most followers has more than 1w followers.

1.5 Project Galaxy

"Build better communities in Web3"

Project Galaxy is an NFT-as-a-service infrastructure platform that allows anyone to create, distribute, or gamify NFTs at will using custom on-chain data, developers to create custom loyalty programs, and individuals to distribute and subscribe to NFTs.

Project Galaxy is similar to OpenSea in that it aggregates various families of NFTs, features NFT showcases and follows users, and more. It also includes an aggregation page for activities such as AMA, which makes it more intuitive and easy to follow the trends and progress of each eco-community.

1.6 Showme

"A subscription NFT social network built on Web3"

Showme provides subscriptions to communities such as NFT to follow, allowing users to more closely engage and stay up to date with community trends. Whenever a user subscribes to a new community, a new NFT is Minted and used as an on-chain credential. At the same time, community credentials on different chains are separate from the communities themselves.

Showme's on-chain footprint index uses the protocol provided by RSS3 for crawling.

1.7 RSS3, CyberConnect, Project Galaxy, Showme

Both RSS3 and CyberConnect are protocols for capturing on-chain and off-chain data to do so, bridging the silos of data across Web3's blockchain networks. They form a social data network that belongs entirely to users and uses decentralized storage that can be used by different projects.

CyberConnect is more advanced than RSS3 and is more Web2 social, with personal page likes and soon a live chat feature.

The other difference is that RSS3 has RSS as its main component, which means that RSS3 has a natural contradiction with the recommendation system and the list of recommended followers on revery.so is randomly generated; Cyberconnect is more flexible and uses a recommendation system to better capture users' possible preferences and anticipate their next actions.

NFT is an important part of social in the Web3 era, and Project Galaxy and Showme focus on cultivating NFT communities, with Project Galaxy working closely with Cyberconnect and Showme working closely with RSS3. Project Galaxy focuses on community activities such as AMA of service projects, while Showme focuses on spontaneous communication and announcement of service communities.

0x02 NonceGeek views

This chapter describes the subjective view of NonceGeek.

RSS is a web feed that allows users and applications to access updates to websites in a standardized, computer-readable format. Subscribing to RSS feeds can allow a user to keep track of many different websites in a single news aggregator, which constantly monitor sites for new content, removing the need for the user to manually check them. News aggregators (or "RSS readers") can be built into a browser, installed on a desktop computer, or installed on a mobile device.

-- Wikipedia

RSS originated in the early days of the Internet era, before the existence of giant whales of apps like today (Facebook, Twitter, Google, Tencent, TikTok......). People used RSS to parse their favorite feeds.

With the development of the Internet, various App whales have emerged, and we just need to register an account, and the App will feed us information. We no longer need to customize our own feeds. App will "continuously push" based on our browsing preferences...... So, RSS is gradually being eliminated by history.

This is certainly an inevitable process.

However, there are some interesting questions to ponder. Is this inevitable process the "right" one, and is RSS likely to be revived? And if so, what will the new generation of RSS look like?

2.1 The "humane" and "right"

Too often, we believe that "survival of the fittest" means that those who survive are naturally right.

However, this crude judgment is problematic. Very often, what survives is "in line with human nature", not "right".

For example, if we think "healthy food = right, unhealthy food = wrong". The reason why fast food has survived so well in the world is not because it is right, but because it responds to human nature.

216,501 Hamburger Stock Photos, Pictures & Royalty-Free Images - iStock

Similarly, why is it that "recommendation algorithm-based apps" are alive and well, but RSS is dying out?

I think it's because...

The App based on recommendation algorithm is in line with human nature, but it doesn't make it right.

As the old saying goes, "good advice often jars on the ear," and saying nice things is human nature and will be liked naturally.

Likewise, recommending content based on your existing preferences is human nature and will be naturally liked.

However, just as being addicted to fast food can make us less and less healthy, being caught up in recommendation algorithms can affect our "mental health" as well.

推荐系统下的喜好加强模型(EN)

We can draw a simple model to describe it.

Suppose initially I show "Preference A" and "Preference B" in the App (e.g. TikTok), then the App will keep recommending me content of "Preference A" and "Preference B", thus reinforcing my preference for "Preference A" and "Preference B", and then the cycle continues......

It's naturally in the App's interest for users to get caught up in their existing preferences - because they stick to the App.

But is it really in the user's interest?

I was addicted to soap operas for a while, so my timeline was full of soap operas. Then suddenly one day I broke away. I wanted to cut ties with soap operas, but the app still pushed soap operas based on my preferences.

This is the essence of the recommendation algorithm - "to keep strengthening your existing preferences, creating stickiness to your existing preferences, which in turn becomes stickiness to the App".

We can see that its essence is closed and not open at the spiritual level. After a long period of time under the system of "recommendation algorithm", we will eventually come to a state - we no longer make changes to our preferences, but the existing preferences keep increasing.

Even if we link the Internet, we will still become one closed system after another.

The opposite is RSS, which is absolutely anti-human, requiring you to get your own subscription feeds and then manage them manually on your own - similar to how we go about making complicated dishes at home.

RSS 下的模型(EN)

However, this sacrifice is rewarded with an open and comfortable information system!

As shown above, in the RSS-based system, we have Preference A and Preference B. We are free to control the "traffic" of each preference, let's say 100 messages per day for "Preference A" and 50 messages per day for "Preference B". I used to like "fishing" and subscribed to a lot of "fishing" related feeds, but now I don't like it anymore, Ok, cut all the related "feeds"!

TimeLine is instantly clean.

Therefore, this system is "open", and people who use this system will also have "high openness".

Openness is the essential spirit of the Internet, isn't it?

2.2 Is it possible for RSS to be revived?

The road to RSS revival may be a tortuous one, but in the long run RSS will regain a larger market.

But one key factor is that there are "incentives" to offset the "costs of hassle" associated with RSS use, depending on the design. We can divide the incentives into "economic incentives" and "non-economic incentives".

  • Economic Incentives

    There is a financial incentive for providing a good RSS feed, which may be created by the provider or integrated by the provider.

  • Non-financial Incentives

    For example, reputation in the Crypto world, etc.

2.3 What does a Base on Blockchain/Web3 RSS look like?

The following is purely a personal opinion.

2.3.1 The redefined personal blog

A personal blog is a blog with its own domain name that is not dependent on any blogging platform.

Although personal blogs have declined in China (only some of the tech leads are still running and maintaining blogs), there are still many personal bloggers from a global perspective.

In the Crypto world, we can redefine blogging, which is supposed to be an important source of information for RSS. So it seems we should talk about a "blogging renaissance" before a RSS renaissance? But that's another topic.

Here's a brief list of possible ways Crypto blogs could play out.

  • The Crypto identity of the blogger can be seen
  • The ability to aggregate the blogger's NFTs
  • Can aggregate the blogger's multiple Web3 dApp content, such as Mirror
  • If the blogger is a technical blogger, it can show the Web3 projects the blogger is involved in
  • There can be a gateway to the Crypto World/Metaverse where the blogger resides
  • Finally, of course, the blogger can provide RSS feeds

2.3.2 Penetrating revenue and traceable re-creation

This is different from the Web2 RSS system. Because it's based on blockchain, the new generation of RSS is technically "revenue-penetrating" and "re-creation-traceable".

In other words, through smart contracts, the creators and distributors of RSS feeds, original works and re-creations can both receive revenue, so that the new generation of RSS can really carry "tangible value" and thus offset the "cost of trouble" brought about by the use of RSS.

However, the proposition of blockchain empowering creators has been mentioned for years, but we have not seen any particularly successful product. Is this because the previous products were not designed from the perspective of "RSS revival"? This requires further thought and discussion.

0x03 Secondary development practice

Performing secondary development practices is a mean to gain a deeper understanding of the product.

3.1 RSS3 architecture

Pics Got From: https://twitter.com/diygod/status/1492462952322109444?s=21

Image

Image

Image

3.2 Development ideas

Because the RSS3 data are publicly available:

https://github.com/NaturalSelectionLabs/RSS3-Network-Data

So it is easy to implement RSS3-based projects through the SDK or directly through the API interface. Here, we implement an example of an on-chain resume incorporating Markdown documents in rss3.bio (now renamed cheers.bio);

https://erqi.github.io/Web3Pass/#/

image-20220226224246371

The source code is available at;

https://github.com/ErQi/Web3Pass/tree/resume

3.3 Practices

The Footprint area of the original project was removed and filled with Content. Then a custom Markdown component was introduced to complement the Content area.

3.3.1 Add Markdown.vue

This component group is to be used to load the content display of Markdown.

Loading mechanism

    Get markdown content -> Process into markdown html format -> Render by introducing markdown style file

3.3.2 Get markdown content

Here we have two forms of fetching. We get it from local files and github api respectively.

Local loading method

To load locally, you need webpack to recognize the md file, and you need to configure the corresponding loader. Add the html-loader package dependency via npm, and then configure the corresponding loader in the rules section of the webpack.config.js configuration file.

            {
                test: /\.md$/,
                use: [
                    {
                        loader: 'html-loader',
                    },
                ],
            },

Once configured, you can import the local md file via import to get the corresponding content. To load locally, you need to modify the v-html attribute of the article tag in Markdown.vue.

import md from '@/assets/RSS3源码分析.md';

...

    <article className="markdown-body" v-html="compiledMarkdown"></article>

Create your own GitHub repository quickly

Network loading method

The network load uses the github api to get the content of the file in the corresponding path. The use of the api is not described in detail. The normal content is base64 encoded. So we add a base64 file to decode it.

        axios.get(config.mdUrl).then((response) => {
            this.mdText = response.data.content;
            this.mdText = BASE64Util.base64_decode(this.mdText);
            this.mdText = marked(this.mdText || '', {
                sanitize: true,
            });
            console.log(this.mdText);
        });

Also, the v-html attribute of the article tag in Markdown.vue uses the parsed mdText attribute directly.

    <article className="markdown-body" v-html="mdText"></article>

The default load file path is configured in the /mdUrl property of the /src/config.ts file.

// The standard format is as follows.
https://api.github.com/repos/WeLightProject/WeLightBlockchainOS/contents/README.md?ref=main
// https://api.github.com/repos/{organization/username}/{projectname}/contents/{folderpath,name}?ref={branchname}

Processing to html format

The markdown content into html format requires the marked library, which can be imported and used after adding the dependencies via npm. Baidu's import is without brackets, but here without brackets it will report an error, the reason is unknown, I also saw this import method in other files, so I tried to add brackets, only to solve the problem that has been reported as unable to import.

After importing the original md content directly as parameters passed in, the return is the content of the html format

import { marked } from 'marked';

            .....

            marked(md || '', {
                sanitize: true,
            })

Introduce css style rendering

Introduce the markdown-it-vue dependency via npm, and then import the following style.

import 'markdown-it-vue/dist/markdown-it-vue.css';

I wanted to use a third-party component to display vue directly in one step, but after I introduced it, I got all kinds of errors. The reason is unknown. I suspect it's because the current vue version is 5.x. Because in the markdown-it-vue issue, I saw that vue 5.x version was reporting errors when importing. The error message is not the same though. I've tried all the other markdown rendering and editing components, but it's just a mess of errors. It doesn't work properly.

So finally I had to go for introduce basic content->convert html format->css style rendering basic processing to handle it.

3.3.3 Fixed home page's access address

In the original project, the wallet address is used to capture user data and then visit the home page of the wallet user. In the case of on-chain resume, this step is not needed and the page of the fixed user is displayed directly.

Tried to locate the home page directly through the route and splice the user to get the data, and found that it did not work well.

The final solution was a bit of a detour, redirecting directly to the home page inside the home page.

Need to display a user data directly modify address in src/config.ts can be.

3.3.4 Deployment

After downloading the project, modify the wallet address in src/config.ts and the corresponding md file url address to suit your needs.

export default {
    address: '0xC994B5384C0d0611De2ecE7d6fF1aD16C34A812F',
    mdUrl: 'https://api.github.com/repos/komomoo/vuepress-theme-resume/contents/example/README.md?ref=master',
    ...
}

Then use the webpack command to package, the package can be accessed normally (packaging time is long, no progress bar, you need to wait patiently, about 1 minute or so).

msfew

leeduckgo