Cloud Alert v2.0.2 – Installation Guide

Cloud alert architecture

1. Initial & account administration

1.1 Account administration

1.1.1 AWS account
  • Step 1: Client provides username, password of AWS account
  • Step 2 (optional): Setup MFA
  • For console: log out and log in again to enable MFA
  • For CLI: use aws sts get-session-token to generate a set of Access Key ID, Secret Access Key and Session Token. Create a new account credential, for example account_mfa credential, by adding these 3 keys and token. 
  • Step 3: Add user to Group and attach role, policy to be fully authorized

2. Infrastructure

2.1 Create VPC

  • Use AWS CloudFormation as a Infrastructure as code service to create infrastructure including VPC, public Subnet (2), private subnet (2), default security group (2), route table, internet gateway, DHCP, NAT gateways …
  • Image of a stack created by CloudFormation
  • Image of a VPC created by CloudFormation

2.2 Create security groups

- The system need 3 more security groups

SG nameDescribeInboundOutbound
akaclaud-bastion-securitygroupSet rule for traffic going through bastion host instanceSSH
IP range external of client
Mongo DB
Kafka TLS
Kafka Plain Text
akaclaud-eks-securitygroupSet rule for traffic going through clusterSSH
Mongo DB
Kafka TLS
Kafka Plain Text
akaclaud-service-securitygroupSet rule for traffic going through service clusterSSH
Kafka TLS
Kafka Plain Text

- After creating a cluster, 2 more security groups will be created automatically

SG nameDescribeInboundOutbound
eks-cluster-sg-akaclaudEKSClusterEKS created security group applied to ENI that is attached to EKS Control Plane master nodes, as well as any managed workloadsAll2 target groups (using 2 ports are generated by creating Network load balancer)All
eks-remoteAccessSecurity group for all nodes in the nodeGroup to allow SSH accessSSHAll

3. Create AWS EKS cluster

3.1 Create cluster

  • Please contact our team to get more information.

3.2 Create node group

  • Generate private key
  • Setup scaling-config: minSize, maxSize, desiredSize
  • Choose instance type, instance OS
  • Capacity type: spot or on demand (highly recommended. But can change in AWS EC2/Auto scaling)

3.3 Create Autoscaler

  • Please contact our team to get more information.

4. Ingress NGINX

4.1 Create internal Load balancer

  • Create load balancer according these requirements
  • Type: Internal
  • Availability zones: attach to 2 private subnets
  • Proxy: TLS termination
  • After setting up all configuration of the load balancer, <client> need to attach a record (check in AWS Route53 service console) to DNS name of the load balancer

4.2 Setup listener

  • Setup protocol and port, SSL certificate, target group
ProtocolPortSSL certificateTarget group
TCP80Noneautomatically generated
TLS443Create using ACMautomatically generated

4.3 Create ACM

  • Use AWS certificate manager to create SSL certificates for https for all these domains (need to create in 5.Route53 firstly)
  • cloudalert.<client name>.com
  • workflow-cloudalert.<client name>.com
  • k8s-dashboard-cloudalert.<client name>.com
  • sso-cloudalert.<client name>.com

5. Route 53

  • Here are 4 public domains which used to created records.
  • cloudalert.<client name>.com
  • workflow-cloudalert.<client name>.com
  • k8s-dashboard-cloudalert.<client name>.com
  • sso-cloudalert.<client name>.com
  • Step 1: Create host zone (public or private)
  • Step 2: Create a domain which is attached to Network Load Balancer
  • Step 3: Create records which use public domains (4 above domains) and private domains (used by backend and frontend services)
  • In case using 2 VPCs and using Bitbucket, one is managed by client and one is managed by our team, there are several steps:
  • Step 1: Create Bitbucket repository
  • Step 2: Create Alert VPC and client VPC
  • Step 3: Create VPC association authorization
  • Step 4: Associate VPC with hosted zone

6. Airflow

6.1 Generate key pair

  • Create Git repo with sub folder “dags”
  • Generate key pair and add public key to repo
  • Create secret git-credentials for k8s using account provided by client
  • User name
  • Password

6.2 Setup SMTP

  • Use AWS SES to create identity (<client name>.com)
  • Setup using configuration below:
  • smtp_host: check in SNS in SMTP endpoint
  • smtp_user: created by SMTP credential
  • smtp_password: created by SMTP credential
  • smtp_port: check in SNS
  • smtp_mail_from: check in SNS
  • SMTP declares region in smtp_host, so that only email accounts in this region are able to received email messages

7. Database

  • Use AWS RDS
  • Type: MySQL
  • Access: by creating a tunnel from local machine through bastion host to AWS RDS instance
  • Setup networking
  • VPC
  • Subnet group: 2 private subnet group created in Infrastructure step

8. Container registry

  • Use ECR
  • Login to ecr using aws ecr get-login-password
  • Login to docker using docker login
  • Pull image from DXG repository to local machine in release branch
  • Make a tag of image
  • Push image to AWS ECR

9. Cert manager

  • Cert-manager helps to automatically provision certificates for Ingress resources
  • Please contact our team to get more information.

10. Keycloak

  • Please contact our team to get more information.

11. Helm

  • Start services using helm
  • Update repository, tag, host, config

12. Event stream

  • Create lambda function
  • Upload code
  • Declare webhook
  • Attach VPC, subnet, SGs
  • Create rules using AWS EventBridge
  • Declare event pattern: catch EC instance status change