On AWS Aurora and ECS​

In this guide we show how to deploy EdgeDB on AWS using Amazon Aurora and Elastic Container Service.

Prerequisites​

• AWS account with billing enabled (or a free trial)
• (optional) aws CLI (install)

Quick Install with CloudFormation​

We maintain a CloudFormation template for easy automated deployment of EdgeDB in your AWS account. The template deploys EdgeDB to a new ECS service and connects it to a newly provisioned Aurora PostgreSQL cluster. The EdgeDB instance created by the template is exposed to the Internet and is protected by TLS and a password you provide.

CloudFormation Web Portal​

Click here to start the deployment process using CloudFormation portal and follow the prompts.

CloudFormation CLI​

Alternatively, if you prefer to use AWS CLI, run the following command in your terminal:

1. aws cloudformation create-stack \
2. --stack-name EdgeDB \
3. --template-url \
4. https://edgedb-deploy.s3.us-east-2.amazonaws.com/edgedb-aurora.yml \
5. --capabilities CAPABILITY_NAMED_IAM \
6. --parameters ParameterKey=SuperUserPassword,ParameterValue=<password>

Manual Install with CLI​

The following instructions produce a deployment that is very similar to the CloudFormation option above.

Create a VPC​

1. read -p "Deployment Name: " NAME

1. REGION=us-west-2

1. VPC_ID=$( \
2. aws ec2 create-vpc \
3. --region $REGION \
4. --output text \
5. --query "Vpc.VpcId" \
6. --cidr-block "10.0.0.0/16" \
7. --instance-tenancy default \
8. --tag-specifications \
9. "ResourceType=vpc,Tags=[{Key=Name,Value=${NAME}-vpc}]" \
10. )

1. aws ec2 modify-vpc-attribute \
2. --region $REGION \
3. --vpc-id $VPC_ID \
4. --enable-dns-support

1. aws ec2 modify-vpc-attribute \
2. --region $REGION \
3. --vpc-id $VPC_ID \
4. --enable-dns-hostnames

Create a Gateway​

Allow communication between the VPC and the internet by creating an Internet Gateway.

1. GATEWAY_ID=$( \
2. aws ec2 create-internet-gateway \
3. --region $REGION \
4. --output text \
5. --query "InternetGateway.InternetGatewayId" \
6. --tag-specifications \
7. "ResourceType=internet-gateway, \
8. Tags=[{Key=Name,Value=${NAME}-internet-gateway}]" \
9. )

1. aws ec2 attach-internet-gateway \
2. --region $REGION \
3. --internet-gateway-id $GATEWAY_ID \
4. --vpc-id $VPC_ID

Create a Public Network ACL​

A Network Access Control List will act as a firewall for a publicly accessible subnet.

1. PUBLIC_ACL_ID=$( \
2. aws ec2 create-network-acl \
3. --region $REGION \
4. --output text \
5. --query "NetworkAcl.NetworkAclId" \
6. --vpc-id $VPC_ID \
7. --tag-specifications \
8. "ResourceType=network-acl, \
9. Tags=[{Key=Name,Value=${NAME}-public-network-acl}]" \
10. )

1. aws ec2 create-network-acl-entry \
2. --region $REGION \
3. --network-acl-id $PUBLIC_ACL_ID \
4. --rule-number 99 \
5. --protocol 6 \
6. --port-range From=0,To=65535 \
7. --rule-action allow \
8. --ingress \
9. --cidr-block 0.0.0.0/0

1. aws ec2 create-network-acl-entry \
2. --region $REGION \
3. --network-acl-id $PUBLIC_ACL_ID \
4. --rule-number 99 \
5. --protocol 6 \
6. --port-range From=0,To=65535 \
7. --rule-action allow \
8. --egress \
9. --cidr-block 0.0.0.0/0

Create a Private Network ACL​

A second ACL will be the firewall for a private subnet to provide an extra boundary around the PostgreSQL cluster.

1. PRIVATE_ACL_ID="$( \
2. aws ec2 create-network-acl \
3. --region $REGION \
4. --output text \
5. --query "NetworkAcl.NetworkAclId" \
6. --vpc-id $VPC_ID \
7. --tag-specifications \
8. "ResourceType=network-acl, \
9. Tags=[{Key=Name,Value=${NAME}-private-network-acl}]" \
10. )"

1. aws ec2 create-network-acl-entry \
2. --region $REGION \
3. --network-acl-id $PRIVATE_ACL_ID \
4. --rule-number 99 \
5. --protocol -1 \
6. --rule-action allow \
7. --ingress \
8. --cidr-block 0.0.0.0/0

1. aws ec2 create-network-acl-entry \
2. --region $REGION \
3. --network-acl-id $PRIVATE_ACL_ID \
4. --rule-number 99 \
5. --protocol -1 \
6. --rule-action allow \
7. --egress \
8. --cidr-block 0.0.0.0/0

Create a Public Subnet in Availability Zone “A”​

1. AVAILABILITY_ZONE_A="$( \
2. aws ec2 describe-availability-zones \
3. --region $REGION \
4. --output text \
5. --query "AvailabilityZones[0].ZoneName" \
6. )"

1. SUBNET_A_PUBLIC_ID=$( \
2. aws ec2 create-subnet \
3. --region $REGION \
4. --output text \
5. --query "Subnet.SubnetId" \
6. --availability-zone $AVAILABILITY_ZONE_A \
7. --cidr-block 10.0.0.0/20 \
8. --vpc-id $VPC_ID \
9. --tag-specifications \
10. "ResourceType=subnet, \
11. Tags=[{Key=Name,Value=${NAME}-subnet-a-public}, \
12. {Key=Reach,Value=public}]" \
13. )

1. aws ec2 replace-network-acl-association \
2. --region $REGION \
3. --network-acl-id $PUBLIC_ACL_ID \
4. --association-id $( \
5. aws ec2 describe-network-acls \
6. --region $REGION \
7. --output text \
8. --query " \
9. NetworkAcls[*].Associations[?SubnetId=='${SUBNET_A_PUBLIC_ID}'][] \
10. | [0].NetworkAclAssociationId" \
11. )

1. ROUTE_TABLE_A_PUBLIC_ID=$( \
2. aws ec2 create-route-table \
3. --region $REGION \
4. --output text \
5. --query "RouteTable.RouteTableId" \
6. --vpc-id $VPC_ID \
7. --tag-specifications \
8. "ResourceType=route-table, \
9. Tags=[{Key=Name,Value=${NAME}-route-table-a-public}]" \
10. )

1. aws ec2 create-route \
2. --region $REGION \
3. --route-table-id $ROUTE_TABLE_A_PUBLIC_ID \
4. --destination-cidr-block 0.0.0.0/0 \
5. --gateway-id $GATEWAY_ID

1. aws ec2 associate-route-table \
2. --region $REGION \
3. --route-table-id $ROUTE_TABLE_A_PUBLIC_ID \
4. --subnet-id $SUBNET_A_PUBLIC_ID

Create a Private Subnet in Availability Zone “A”​

1. SUBNET_A_PRIVATE_ID=$( \
2. aws ec2 create-subnet \
3. --region $REGION \
4. --output text \
5. --query "Subnet.SubnetId" \
6. --availability-zone $AVAILABILITY_ZONE_A \
7. --cidr-block 10.0.16.0/20 \
8. --vpc-id $VPC_ID \
9. --tag-specifications \
10. "ResourceType=subnet, \
11. Tags=[{Key=Name,Value=${NAME}-subnet-a-private}, \
12. {Key=Reach,Value=private}]" \
13. )

1. aws ec2 replace-network-acl-association \
2. --region $REGION \
3. --network-acl-id $PRIVATE_ACL_ID \
4. --association-id $( \
5. aws ec2 describe-network-acls \
6. --region $REGION \
7. --output text \
8. --query " \
9. NetworkAcls[*].Associations[?SubnetId == '${SUBNET_A_PRIVATE_ID}' \
10. ][] | [0].NetworkAclAssociationId" \
11. )

1. ROUTE_TABLE_A_PRIVATE_ID=$( \
2. aws ec2 create-route-table \
3. --region $REGION \
4. --output text \
5. --query "RouteTable.RouteTableId" \
6. --vpc-id $VPC_ID \
7. --tag-specifications \
8. "ResourceType=route-table, \
9. Tags=[{Key=Name,Value=${NAME}-route-table-a-private}]" \
10. )

1. aws ec2 associate-route-table \
2. --region $REGION \
3. --route-table-id $ROUTE_TABLE_A_PRIVATE_ID \
4. --subnet-id $SUBNET_A_PRIVATE_ID

Create a Public Subnet in Availability Zone “B”​

1. AVAILABILITY_ZONE_B="$( \
2. aws ec2 describe-availability-zones \
3. --region $REGION \
4. --output text \
5. --query "AvailabilityZones[1].ZoneName" \
6. )"

1. SUBNET_B_PUBLIC_ID=$( \
2. aws ec2 create-subnet \
3. --region $REGION \
4. --output text \
5. --query "Subnet.SubnetId" \
6. --availability-zone $AVAILABILITY_ZONE_B \
7. --cidr-block 10.0.32.0/20 \
8. --vpc-id $VPC_ID \
9. --tag-specifications \
10. "ResourceType=subnet, \
11. Tags=[{Key=Name,Value=${NAME}-subnet-b-public}, \
12. {Key=Reach,Value=public}]" \
13. )

1. aws ec2 replace-network-acl-association \
2. --region $REGION \
3. --network-acl-id $PUBLIC_ACL_ID \
4. --association-id $( \
5. aws ec2 describe-network-acls \
6. --region $REGION \
7. --output text \
8. --query " \
9. NetworkAcls[*].Associations[?SubnetId == '${SUBNET_B_PUBLIC_ID}'\
10. ][] | [0].NetworkAclAssociationId" \
11. )

1. ROUTE_TABLE_B_PUBLIC_ID=$( \
2. aws ec2 create-route-table \
3. --region $REGION \
4. --output text \
5. --query "RouteTable.RouteTableId" \
6. --vpc-id $VPC_ID \
7. --tag-specifications \
8. "ResourceType=route-table, \
9. Tags=[{Key=Name,Value=${NAME}-route-table-b-public}]" \
10. )

1. aws ec2 create-route \
2. --region $REGION \
3. --route-table-id $ROUTE_TABLE_B_PUBLIC_ID \
4. --destination-cidr-block 0.0.0.0/0 \
5. --gateway-id $GATEWAY_ID

1. aws ec2 associate-route-table \
2. --region $REGION \
3. --route-table-id $ROUTE_TABLE_B_PUBLIC_ID \
4. --subnet-id $SUBNET_B_PUBLIC_ID

Create a Private Subnet in Availability Zone “B”​

1. SUBNET_B_PRIVATE_ID=$( \
2. aws ec2 create-subnet \
3. --region $REGION \
4. --output text \
5. --query "Subnet.SubnetId" \
6. --availability-zone $AVAILABILITY_ZONE_B \
7. --cidr-block 10.0.48.0/20 \
8. --vpc-id $VPC_ID \
9. --tag-specifications \
10. "ResourceType=subnet, \
11. Tags=[{Key=Name,Value=${NAME}-subnet-b-private}, \
12. {Key=Reach,Value=private}]" \
13. )

1. aws ec2 replace-network-acl-association \
2. --region $REGION \
3. --network-acl-id $PRIVATE_ACL_ID \
4. --association-id $( \
5. aws ec2 describe-network-acls \
6. --region $REGION \
7. --output text \
8. --query " \
9. NetworkAcls[*].Associations[?SubnetId=='${SUBNET_B_PRIVATE_ID}'][] \
10. | [0].NetworkAclAssociationId" \
11. )

1. ROUTE_TABLE_B_PRIVATE_ID=$( \
2. aws ec2 create-route-table \
3. --region $REGION \
4. --output text \
5. --query "RouteTable.RouteTableId" \
6. --vpc-id $VPC_ID \
7. --tag-specifications \
8. "ResourceType=route-table, \
9. Tags=[{Key=Name,Value=${NAME}-route-table-b-private}]" \
10. )

1. aws ec2 associate-route-table \
2. --region $REGION \
3. --route-table-id $ROUTE_TABLE_B_PRIVATE_ID \
4. --subnet-id $SUBNET_B_PRIVATE_ID

Create an EC2 security group​

1. EC2_SECURITY_GROUP_ID=$( \
2. aws ec2 create-security-group \
3. --region $REGION \
4. --output text \
5. --query "GroupId" \
6. --group-name "${NAME}-ec2-security-group" \
7. --description "Controls access to ${NAME} stack EC2 instances." \
8. --vpc-id $VPC_ID \
9. --tag-specifications \
10. "ResourceType=security-group, \
11. Tags=[{Key=Name,Value=${NAME}-ec2-security-group}]" \
12. )

1. aws ec2 authorize-security-group-ingress \
2. --region $REGION \
3. --group-id $EC2_SECURITY_GROUP_ID \
4. --protocol tcp \
5. --cidr 0.0.0.0/0 \
6. --port 5656 \
7. --tag-specifications \
8. "ResourceType=security-group-rule, \
9. Tags=[{Key=Name,Value=${NAME}-ec2-security-group-ingress}]"

Create an RDS Security Group​

1. RDS_SECURITY_GROUP_ID=$( \
2. aws ec2 create-security-group \
3. --region $REGION \
4. --output text \
5. --query "GroupId" \
6. --group-name "${NAME}-rds-security-group" \
7. --description "Controls access to ${NAME} stack RDS instances." \
8. --vpc-id $VPC_ID \
9. --tag-specifications \
10. "ResourceType=security-group, \
11. Tags=[{Key=Name,Value=${NAME}-rds-security-group}]" \
12. )

1. aws ec2 authorize-security-group-ingress \
2. --region $REGION \
3. --group-id $RDS_SECURITY_GROUP_ID \
4. --protocol tcp \
5. --source-group $EC2_SECURITY_GROUP_ID \
6. --port 5432 \
7. --tag-specifications \
8. "ResourceType=security-group-rule, \
9. Tags=[{Key=Name,Value=${NAME}-rds-security-group-ingress}]"

1. RDS_SUBNET_GROUP_NAME="${NAME}-rds-subnet-group"

1. aws rds create-db-subnet-group \
2. --region $REGION \
3. --db-subnet-group-name "$RDS_SUBNET_GROUP_NAME" \
4. --db-subnet-group-description "EdgeDB RDS subnet group for ${NAME}" \
5. --subnet-ids $SUBNET_A_PRIVATE_ID $SUBNET_B_PRIVATE_ID

Create an RDS Cluster​

1. read -rsp "Password: " PASSWORD

1. PASSWORD_ARN="$( \
2. aws secretsmanager create-secret \
3. --region $REGION \
4. --output text \
5. --query "ARN" \
6. --name "${NAME}-password" \
7. --secret-string "$PASSWORD" \
8. )"

1. DB_CLUSTER_IDENTIFIER="${NAME}-postgres-cluster"

1. DB_CLUSTER_ADDRESS="$( \
2. aws rds create-db-cluster \
3. --region $REGION \
4. --output text \
5. --query "DBCluster.Endpoint" \
6. --engine aurora-postgresql \
7. --engine-version 13.4 \
8. --db-cluster-identifier "$DB_CLUSTER_IDENTIFIER" \
9. --db-subnet-group-name "$RDS_SUBNET_GROUP_NAME" \
10. --master-username postgres \
11. --master-user-password "$PASSWORD" \
12. --port 5432 \
13. --vpc-security-group-ids "$RDS_SECURITY_GROUP_ID" \
14. )"

1. aws rds create-db-instance \
2. --region $REGION \
3. --availability-zone "$AVAILABILITY_ZONE_A" \
4. --engine "aurora-postgresql" \
5. --db-cluster-identifier "$DB_CLUSTER_IDENTIFIER" \
6. --db-instance-identifier "${NAME}-postgres-instance-a" \
7. --db-instance-class "db.t3.medium" \
8. --db-subnet-group-name "$RDS_SUBNET_GROUP_NAME"

1. aws rds create-db-instance \
2. --region $REGION \
3. --availability-zone "$AVAILABILITY_ZONE_B" \
4. --engine "aurora-postgresql" \
5. --db-cluster-identifier "$DB_CLUSTER_IDENTIFIER" \
6. --db-instance-identifier "${NAME}-postgres-instance-b" \
7. --db-instance-class "db.t3.medium" \
8. --db-subnet-group-name "$RDS_SUBNET_GROUP_NAME"

1. DSN_ARN="$( \
2. aws secretsmanager create-secret \
3. --region $REGION \
4. --output text \
5. --query "ARN" \
6. --name "${NAME}-backend-dsn" \
7. --secret-string \
8. "postgres://postgres:${PASSWORD}@${DB_CLUSTER_ADDRESS}:5432/postgres" \
9. )"

Create a Load Balancer​

Adding a load balancer will facilitate scaling the EdgeDB cluster.

1. TARGET_GROUP_ARN="$( \
2. aws elbv2 create-target-group \
3. --region $REGION \
4. --output text \
5. --query "TargetGroups[0].TargetGroupArn" \
6. --health-check-interval-seconds 10 \
7. --health-check-path "/server/status/ready" \
8. --health-check-protocol HTTPS \
9. --unhealthy-threshold-count 2 \
10. --healthy-threshold-count 2 \
11. --name "${NAME}-target-group" \
12. --port 5656 \
13. --protocol TCP \
14. --target-type ip \
15. --vpc-id $VPC_ID \
16. )"

1. LOAD_BALANCER_NAME="${NAME}-load-balancer"

1. LOAD_BALANCER_ARN="$( \
2. aws elbv2 create-load-balancer \
3. --region $REGION \
4. --output text \
5. --query "LoadBalancers[0].LoadBalancerArn" \
6. --type network \
7. --name "$LOAD_BALANCER_NAME" \
8. --scheme internet-facing \
9. --subnets "$SUBNET_A_PUBLIC_ID" "$SUBNET_B_PUBLIC_ID" \
10. )"

1. aws elbv2 create-listener \
2. --region $REGION \
3. --default-actions \
4. '[{"TargetGroupArn": "'"$TARGET_GROUP_ARN"'","Type": "forward"}]' \
5. --load-balancer-arn "$LOAD_BALANCER_ARN" \
6. --port 5656 \
7. --protocol TCP

Create an ECS Cluster​

The only thing left to do is create and ECS cluster and deploy the EdgeDB container in it.

1. EXECUTION_ROLE_NAME="${NAME}-execution-role"

1. EXECUTION_ROLE_ARN="$( \
2. aws iam create-role \
3. --region $REGION \
4. --output text \
5. --query "Role.Arn" \
6. --role-name "$EXECUTION_ROLE_NAME" \
7. --assume-role-policy-document \
8. "{ \
9. \"Version\": \"2012-10-17\", \
10. \"Statement\": [{ \
11. \"Effect\": \"Allow\", \
12. \"Principal\": {\"Service\": \"ecs-tasks.amazonaws.com\"}, \
13. \"Action\": \"sts:AssumeRole\" \
14. }] \
15. }" \
16. )"

1. SECRETS_ACCESS_POLICY_ARN="$( \
2. aws iam create-policy \
3. --region $REGION \
4. --output text \
5. --query "Policy.Arn" \
6. --policy-name "${NAME}-secrets-access-policy" \
7. --policy-document \
8. "{ \
9. \"Version\": \"2012-10-17\", \
10. \"Statement\": [{ \
11. \"Effect\": \"Allow\", \
12. \"Action\": \"secretsmanager:GetSecretValue\", \
13. \"Resource\": [ \
14. \"$PASSWORD_ARN\", \
15. \"$DSN_ARN\" \
16. ] \
17. }] \
18. }" \
19. )"

1. aws iam attach-role-policy \
2. --region $REGION \
3. --role-name "$EXECUTION_ROLE_NAME" \
4. --policy-arn \
5. "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"

1. aws iam attach-role-policy \
2. --region $REGION \
3. --role-name "$EXECUTION_ROLE_NAME" \
4. --policy-arn "$SECRETS_ACCESS_POLICY_ARN"

1. TASK_ROLE_ARN="$( \
2. aws iam create-role \
3. --region $REGION \
4. --output text \
5. --query "Role.Arn" \
6. --role-name "${NAME}-task-role" \
7. --assume-role-policy-document \
8. "{ \
9. \"Version\": \"2012-10-17\", \
10. \"Statement\": [{ \
11. \"Effect\": \"Allow\",  \
12. \"Principal\": {\"Service\": \"ecs-tasks.amazonaws.com\"}, \
13. \"Action\": \"sts:AssumeRole\" \
14. }] \
15. }" \
16. )"

1. LOG_GROUP_NAME="/ecs/edgedb/$NAME"

1. aws logs create-log-group \
2. --region $REGION \
3. --log-group-name "$LOG_GROUP_NAME"

1. CLUSTER_NAME="${NAME}-server-cluster"

1. aws ecs create-cluster \
2. --region $REGION \
3. --cluster-name "$CLUSTER_NAME"

1. LOG_GROUP_ARN="$( \
2. aws logs describe-log-groups \
3. --region $REGION \
4. --output text \
5. --query "logGroups[0].arn" \
6. --log-group-name-prefix "$LOG_GROUP_NAME" \
7. )"

1. TASK_DEFINITION_ARN="$( \
2. aws ecs register-task-definition \
3. --region $REGION \
4. --output text \
5. --query "taskDefinition.taskDefinitionArn" \
6. --requires-compatibilities "FARGATE" \
7. --network-mode "awsvpc" \
8. --execution-role-arn "$EXECUTION_ROLE_ARN" \
9. --task-role-arn "$TASK_ROLE_ARN" \
10. --family "${NAME}-task-definition" \
11. --cpu 1024 \
12. --memory 2GB \
13. --container-definitions \
14. "[{ \
15. \"name\": \"$NAME\", \
16. \"image\": \"edgedb/edgedb\", \
17. \"portMappings\": [{\"containerPort\": 5656}], \
18. \"command\": [\"edgedb-server\"], \
19. \"environment\": [{ \
20. \"name\": \"EDGEDB_SERVER_GENERATE_SELF_SIGNED_CERT\", \
21. \"value\": \"1\" \
22. }], \
23. \"secrets\": [ \
24. { \
25. \"name\": \"EDGEDB_SERVER_PASSWORD\", \
26. \"valueFrom\": \"$PASSWORD_ARN\" \
27. }, \
28. { \
29. \"name\": \"EDGEDB_SERVER_BACKEND_DSN\", \
30. \"valueFrom\": \"$DSN_ARN\" \
31. } \
32. ], \
33. \"logConfiguration\": { \
34. \"logDriver\": \"awslogs\", \
35. \"options\": { \
36. \"awslogs-region\": \"$REGION\", \
37. \"awslogs-group\": \"$LOG_GROUP_NAME\", \
38. \"awslogs-stream-prefix\": \"ecs\" \
39. } \
40. } \
41. }]" \
42. )"

1. aws ecs create-service \
2. --region $REGION \
3. --service-name "$NAME" \
4. --cluster "$CLUSTER_NAME" \
5. --task-definition "$TASK_DEFINITION_ARN" \
6. --deployment-configuration \
7. "minimumHealthyPercent=100,maximumPercent=200" \
8. --desired-count 2 \
9. --health-check-grace-period-seconds 120 \
10. --launch-type FARGATE \
11. --network-configuration \
12. "awsvpcConfiguration={ \
13. assignPublicIp=ENABLED, \
14. subnets=[$SUBNET_A_PUBLIC_ID,$SUBNET_B_PUBLIC_ID], \
15. securityGroups=[$EC2_SECURITY_GROUP_ID] \
16. }" \
17. --load-balancers \
18. "containerName=$NAME, \
19. containerPort=5656, \
20. targetGroupArn=$TARGET_GROUP_ARN"

Create a local link to the new EdgeDB instance​

To access the EdgeDB instance you’ve just provisioned on AWS from your local machine run edgedb instance link:

1. printf $PASSWORD | edgedb instance link \
2. --password-from-stdin \
3. --trust-tls-cert \
4. --non-interactive \
5. --host "$( \
6. aws ec2 describe-network-interfaces \
7. --output text \
8. --region $REGION \
9. --query \
10. "NetworkInterfaces[?contains(Description, '$LOAD_BALANCER_NAME')] \
11. | [0].Association.PublicIp" \
12. )" \
13. aws