Equilibrium develops a RPC node for Starknet and provides a Docker image to run it.

However, when it comes to running a container in production, some extra settings (like a load balancer, automatic reboot, CPU scaling, etc.) are required.

In this tutorial, I will explain how to deploy such production-grade nodes in AWS, Azure and GCP. All the files used in this tutorial are available in the corresponding repo.

Note: some of the snippets use jq for easily parsing and manipulating json data from the console. If you don't want to install it, just break the command and copy/paste manually the selected entry.

AWS

AWS CLI setup

To deploy programmatically to AWS, you first need to install and configure the AWS CLI. If you are okay with this step, just make sure that your configured credentials have the appropriate policy and jump to the next section.

It is recommended not to use your own root user credentials with the CLI. Instead, you should create an specific policy for the task and then an IAM User granted only these accesses. Eventually, you can generate credentials for this user and create a profile in the CLI for using them. Ouch. But we've got you covered!

TL;DR copy/paste

curl https://raw.githubusercontent.com/ClementWalter/starknet-nodes/main/docs/aws/policy.json -o policy-document.json
aws iam create-policy --policy-name docker-ecs-context --policy-document 'file://policy-document.json' > policy.json
rm policy-document.json
aws iam create-group --group-name docker-ecs-users
aws iam attach-group-policy --group-name docker-ecs-users --policy-arn `cat policy.json | jq ".Policy.Arn" -r`
aws iam create-user --user-name docker-ecs-user
aws iam add-user-to-group --group-name docker-ecs-users --user-name docker-ecs-user
aws iam create-access-key --user-name docker-ecs-user > access_key.json
aws configure --profile docker-ecs

Details

• create an AWS policy using this policy file

  • either using the AWS console

  • or with the cli:

  aws iam create-policy --policy-name docker-ecs-context --policy-document 'file://docs/aws/policy.json' > policy.json
  

• create an AWS user group and add it the above created policy

  • either with the AWS console

  • or with the cli:

  aws iam create-group --group-name docker-ecs-users
  aws iam attach-group-policy --group-name docker-ecs-users --policy-arn `cat policy.json | jq ".Policy.Arn" -r`
  

• create an AWS User, select only "Access key - Programmatic access", and add it to the above created group

  • either using the AWS console

  • or with the cli:

  aws iam create-user --user-name docker-ecs-user
  aws iam add-user-to-group --group-name docker-ecs-users --user-name docker-ecs-user
  aws iam create-access-key --user-name docker-ecs-user > access_key.json
  

• create an AWS profile for deploying the node using the generated credentials

  aws configure --profile docker-ecs
  

CloudFormation deployment

Now that the hard part is done, let us focus on the easy one: deploying the eqlabs/pathfinder docker image to AWS.

Indeed, we use the Docker<>ECS integration to deploy a whole CloudFormation stack right from a simple docker-compose.yml file. This deployment only requires to change the docker context to use ECS.

So basically a simple docker compose up is enough. Which means that you need to have docker compose installed on your machine. Refer to their doc for your own configuration.

TL;DR copy/paste

curl https://raw.githubusercontent.com/ClementWalter/starknet-nodes/main/docker-compose.yml -o docker-compose.yml
curl https://raw.githubusercontent.com/ClementWalter/starknet-nodes/main/docs/aws/docker-compose.aws.yml -o docker-compose.aws.yml
docker context create ecs starknet-ecs
docker context use starknet-ecs
docker compose -f docker-compose.yml -f docker-compose.aws.yml up

Detailed story

The creation of the docker context lets "update" all the docker commands to use AWS services in a pre-defined manner. For instance, it creates a AWS Cloudformation stack with the following main components:

• Elastic File Systems for the volumes

• an ECS cluster with goerli and mainnet services and tasks.

• AWS Fargate to run the containers

The deployed stack can be monitored on the AWS CloudFormation home page. All the aws cloudformation commands are available for managing the stack. Indeed, you can also use the docker compose convert tool to generate the corresponding Stack configuration and eventually use the aws cli instead:

docker compose convert > stack.yaml
aws cloudformation deploy --template-file stack.yml --stack-name starknet-nodes --capabilities CAPABILITY_IAM --profile docker-ecs

Eventually, deploying the nodes requires:

• to create a docker ecs context: docker context create ecs <chose a context name>

  • for example, docker context create ecs starknet-ecs

  • use the above created profile

• and use it: docker context use <context name>

  • or pass the --context <context name> to every following commands

• to execute docker compose --project-name <chose a name visible in aws console> -f docker-compose.yml -f docs/aws/docker-compose.aws.yml up

  • for example, docker compose --project-name starknet-nodes -f docker-compose.yml -f docs/aws/docker-compose.aws.yml up

  • ignore the WARNING services.scale: unsupported attribute

  • the --project-name option lets define the name of the stack but creates bugs for later use of docker compose logs/down/etc. so I don't recommend to use it for now.

The created endpoints can be found in the ECS Cluster page:

• Cluster > Services > Networking > DNS names

or directly using docker compose:

docker compose ps

You can export those URLs easily for later user with the --format json option:

docker compose ps --format json > nodes.json

You can then check that the nodes are running using for example curl:

curl `docker compose ps --format json | jq ".[0].Publishers[0].URL" -r` \
  -H 'content-type: application/json' \
  --data-raw '{"method":"starknet_chainId","jsonrpc":"2.0","params":[],"id":0}' \
  --compressed | jq .result | xxd -rp
# SN_GOERLI
curl `docker compose ps --format json | jq ".[1].Publishers[0].URL" -r` \
  -H 'content-type: application/json' \
  --data-raw '{"method":"starknet_chainId","jsonrpc":"2.0","params":[],"id":0}' \
  --compressed | jq .result | xxd -rp
# SN_MAIN

Troubleshooting

The backup download time before the node actually starts can be quite long. If the service keeps restarting because of health check, you can extend the health check grace period.

In the following snippet, we put 3000s because it should be enough time to download ~100Gb at relatively low rate of ~33Mb/s:

aws ecs list-services --cluster starknet-nodes | \
  jq '.serviceArns[] | select (. | contains ("goerli") )' -r | cut -d '/' -f 3 | xargs -L1 \
aws ecs update-service \
  --cluster starknet-nodes \
  --health-check-grace-period-seconds 3000 \
  --service $1

Monitoring

The deployed stack can be monitored on the AWS CloudFormation home page. All the aws cloudformation commands are available for managing the stack. The docker compose logs command will output the logs otherwise found in CloudWatch.

Cleaning

To delete your nodes and clean everything, just run:

docker context use starknet-ecs
docker compose down
docker context use default
docker context rm starknet-ecs
aws efs describe-file-systems | jq ".FileSystems[].FileSystemId" | xargs -L1 aws efs delete-file-system --file-system-id $1
export POLICY_ARN=$(aws iam list-attached-group-policies --group-name docker-ecs-users | jq ".AttachedPolicies[0].PolicyArn" -r)
aws iam detach-group-policy --group-nam docker-ecs-users --policy-arn $POLICY_ARN
aws iam delete-policy --policy-arn $POLICY_ARN
aws iam remove-user-from-group --group-name docker-ecs-users --user docker-ecs-user
aws iam delete-group --group-name docker-ecs-users
aws iam delete-access-key --user-name docker-ecs-user --access-key-id $(cat access_key.json | jq ".AccessKey.AccessKeyId" -r)
aws iam delete-user --user-name docker-ecs-user

Azure

Azure setup

There is also a Docker<>Azure ACI integration using docker context and the strategy is consequently similar to the one above. However, Azure made it easier by removing the need for using their CLI nor generating credentials. In short, the whole AWS CLI Setup section boils down to:

docker login azure

However, it may be convenient to still install the Azure CLI for later manipulations.

We are almost done with Azure config. You just need to create a Resource group for the nodes and chose an appropriate region for your application.

If you installed the cli, just run:

az configure --defaults location=francecentral
az group create --name starknet-nodes

Cloud deployment

TL;DR copy/paste

docker context create aci starknet-aci
docker context use starknet-aci
docker volume create goerli-data --storage-account starknetnodes
docker volume create mainnet-data --storage-account starknetnodes
curl https://raw.githubusercontent.com/ClementWalter/starknet-nodes/main/docker-compose.yml -o docker-compose.yml
curl https://raw.githubusercontent.com/ClementWalter/starknet-nodes/main/docs/azure/docker-compose.azure.yml -o docker-compose.azure.yml
docker compose -f docker-compose.yml -f docs/azure/docker-compose.azure.yml up

Detailed story

The docker volume command creates File shares. The storage account (starknetnodes) is created if it does not exist. These volumes are then used in the docker-compose.azure.yml azure overriding configuration.

When running docker compose up, the aci context creates Azure Container Instances. Eventually, the commands are:

• create a docker aci context: docker context create aci <chose a context name>

  • use the above created resource group

• docker context use <context name>

  • or pass the --context <context name> to every following commands

• create volumes:

  • docker volume create goerli-data --storage-account <storage account name>

  • docker volume create mainnet-data --storage-account <storage account name>

• execute docker compose --project-name <chose a name project name> -f docker-compose.yml -f docs/azure/docker-compose.azure.yml up

You can then retrieve the node urls using :

docker ps

You can then check that the node are running using curl:

curl $(docker ps --format json | jq '.[] | select( .ID | contains("goerli") ) | .Ports[0]' -r | cut -d "-" -f 1) \
  -H 'content-type: application/json' \
  --data-raw '{"method":"starknet_chainId","jsonrpc":"2.0","params":[],"id":0}' \
  --compressed | jq .result | xxd -rp
# SN_GOERLI
curl $(docker ps --format json | jq '.[] | select( .ID | contains("mainnet") ) | .Ports[0]' -r | cut -d "-" -f 1) \
  -H 'content-type: application/json' \
  --data-raw '{"method":"starknet_chainId","jsonrpc":"2.0","params":[],"id":0}' \
  --compressed | jq .result | xxd -rp
# SN_MAIN

Monitoring

You can find info about your deployment and containers in the Resource group's page.

Cleaning

To delete your nodes and clean everything, just run:

docker context use starknet-aci
docker compose down
docker volume rm starknetnodes/goerli-data
docker volume rm starknetnodes/mainnet-data
docker context use default
docker context rm starknet-aci

GCP

GCP setup

There is no native Docker<>GCP integration and the usual way to deploy a containerized stack to GCP is to use the Google Kubernetes Engine.

You first need to install the Google Cloud CLI.

If you have already a gcloud project, you can simply use it. Otherwise — or if you prefer — you need to create one:

gcloud projects create <unique project id>
gcloud config set project <unique project id>

Then, we set for the project the region and zone we want to deploy in (see regions/zones). For example:

gcloud config set compute/region europe-west1
gcloud config set compute/zone europe-west1-b

Cloud deployment

TL;DR copy/paste

gcloud container clusters create starknet-nodes
gcloud container clusters get-credentials starknet-nodes
export $(xargs <.env)
kubectl run starknet-mainnet \
 --image=clementwalter/pathfinder-curl \
 --port=9545 \
 --env="PATHFINDER_ETHEREUM_API_URL=${PATHFINDER_ETHEREUM_API_URL_MAINNET}" \
 --command -- /bin/bash -c 'curl https://pathfinder-starknet-node-backup.s3.eu-west-3.amazonaws.com/mainnet/mainnet.sqlite --output /usr/share/pathfinder/data/mainnet.sqlite && pathfinder'
kubectl expose pod starknet-mainnet --port=9545 --target-port=9545 --type=LoadBalancer
kubectl run starknet-goerli \
 --image=clementwalter/pathfinder-curl \
 --port=9545 \
 --env="PATHFINDER_ETHEREUM_API_URL=${PATHFINDER_ETHEREUM_API_URL_GOERLI}" \
 --command -- /bin/bash -c 'curl https://pathfinder-starknet-node-backup.s3.eu-west-3.amazonaws.com/goerli/goerli.sqlite --output /usr/share/pathfinder/data/goerli.sqlite && pathfinder'
kubectl expose pod starknet-goerli --port=9545 --target-port=9545 --type=LoadBalancer

Detailed story

We first create a Kubernetes cluster on GCP. You can check all the options using gcloud container clusters create --help. Especially, by default, the cluster uses 3 nodes (param --num-nodes=NUM_NODES; default=3).

Then we log into this cluster using the gcloud cli to have the kubectl command working directly on GCP. Depending on the versions of gcloud used, you may need to follow this doc to be able to login correctly. We also export all the env variables defined in a .env file (where we put the PATHFINDER_ETHEREUM_API_URL_MAINNET and PATHFINDER_ETHEREUM_API_URL_GOERLI definition).

Eventually, kubectl run lets define the pods to run, and kubectl expose expose the pods (the starknet nodes).

You can then retrieve the node urls using :

kubectl get services

and eventually check that the nodes are running using curl:

curl $(kubectl get service starknet-goerli --output=json | jq ".status.loadBalancer.ingress[0].ip" -r):9545 \
  -H 'content-type: application/json' \
  --data-raw '{"method":"starknet_chainId","jsonrpc":"2.0","params":[],"id":0}' \
  --compressed | jq .result | xxd -rp
# SN_GOERLI
curl $(kubectl get service starknet-mainnet --output=json | jq ".status.loadBalancer.ingress[0].ip" -r):9545 \
  -H 'content-type: application/json' \
  --data-raw '{"method":"starknet_chainId","jsonrpc":"2.0","params":[],"id":0}' \
  --compressed | jq .result | xxd -rp
# SN_MAIN

Troubleshooting

You can connect to your cluster using Cloud Shell, from browser or using the gcloud cli:

gcloud cloud-shell ssh

It may happen that the initial backup download fails. This does not prevent the node to start but can make its warmup time several days long. So if it happened, just ssh into the container and re-run the download command:

kubectl exec -it starknet-goerli -c starknet-goerli -- sh
curl https://pathfinder-starknet-node-backup.s3.eu-west-3.amazonaws.com/goerli/goerli.sqlite --output /usr/share/pathfinder/data/goerli.sqlite

Monitoring

You can find info about your deployment and containers in the Kubernetes cluster page

Cleaning

To delete your nodes and clean everything, just run:

kubectl delete pods --all
kubectl delete services --all
gcloud container clusters delete starknet-nodes
gcloud projects delete starknet-nodes

Conclusion

While running a container is rather easy, running a prod-grade micro-service requires some more configurations (load balancing, auto restart, auto scaling, vpc, etc.). In this tutorial, I wanted to show how to deploy prod-grade Starknet nodes using managed services of the three main cloud providers.

These guidelines may be taken "as is" to start and eventually updated depending on your needs. Please fell free to reach out to me directly on Twitter or Discord should you have any comment or feedbacks!