Cloud alert architecture
1. Initial & account administration
1.1 Account administration
1.1.1 AWS account
- Step 1: Client provides username, password of AWS account
- Step 2 (optional): Setup MFA
- For console: log out and log in again to enable MFA
- For CLI: use aws sts get-session-token to generate a set of Access Key ID, Secret Access Key and Session Token. Create a new account credential, for example account_mfa credential, by adding these 3 keys and token.
- Step 3: Add user to Group and attach role, policy to be fully authorized
2. Infrastructure
2.1 Create VPC
- Use AWS CloudFormation as a Infrastructure as code service to create infrastructure including VPC, public Subnet (2), private subnet (2), default security group (2), route table, internet gateway, DHCP, NAT gateways …
- Image of a stack created by CloudFormation
- Image of a VPC created by CloudFormation
2.2 Create security groups
- The system need 3 more security groups
| SG name | Describe | Inbound | Outbound |
| akaclaud-bastion-securitygroup | Set rule for traffic going through bastion host instance | SSH IP range external of client HTTP HTTPS Mongo DB MySQL Kafka TLS Kafka Plain Text | All |
| akaclaud-eks-securitygroup | Set rule for traffic going through cluster | SSH HTTP HTTPS Mongo DB MySQL Kafka TLS Kafka Plain Text | All |
| akaclaud-service-securitygroup | Set rule for traffic going through service cluster | SSH HTTP HTTPS MySQL Kafka TLS Kafka Plain Text | All |
- After creating a cluster, 2 more security groups will be created automatically
| SG name | Describe | Inbound | Outbound |
| eks-cluster-sg-akaclaudEKSCluster | EKS created security group applied to ENI that is attached to EKS Control Plane master nodes, as well as any managed workloads | All2 target groups (using 2 ports are generated by creating Network load balancer) | All |
| eks-remoteAccess | Security group for all nodes in the nodeGroup to allow SSH access | SSH | All |
3. Create AWS EKS cluster
3.1 Create cluster
- Please contact our team to get more information.
3.2 Create node group
- Generate private key
- Setup scaling-config: minSize, maxSize, desiredSize
- Choose instance type, instance OS
- Capacity type: spot or on demand (highly recommended. But can change in AWS EC2/Auto scaling)
3.3 Create Autoscaler
- Please contact our team to get more information.
4. Ingress NGINX
4.1 Create internal Load balancer
- Create load balancer according these requirements
- Type: Internal
- Availability zones: attach to 2 private subnets
- Proxy: TLS termination
- After setting up all configuration of the load balancer, <client> need to attach a record (check in AWS Route53 service console) to DNS name of the load balancer
4.2 Setup listener
- Setup protocol and port, SSL certificate, target group
| Protocol | Port | SSL certificate | Target group |
| TCP | 80 | None | automatically generated |
| TLS | 443 | Create using ACM | automatically generated |
4.3 Create ACM
- Use AWS certificate manager to create SSL certificates for https for all these domains (need to create in 5.Route53 firstly)
- cloudalert.<client name>.com
- workflow-cloudalert.<client name>.com
- k8s-dashboard-cloudalert.<client name>.com
- sso-cloudalert.<client name>.com
5. Route 53
- Here are 4 public domains which used to created records.
- cloudalert.<client name>.com
- workflow-cloudalert.<client name>.com
- k8s-dashboard-cloudalert.<client name>.com
- sso-cloudalert.<client name>.com
- Step 1: Create host zone (public or private)
- Step 2: Create a domain which is attached to Network Load Balancer
- Step 3: Create records which use public domains (4 above domains) and private domains (used by backend and frontend services)
- In case using 2 VPCs and using Bitbucket, one is managed by client and one is managed by our team, there are several steps:
- Step 1: Create Bitbucket repository
- Step 2: Create Alert VPC and client VPC
- Step 3: Create VPC association authorization
- Step 4: Associate VPC with hosted zone
6. Airflow
6.1 Generate key pair
- Create Git repo with sub folder “dags”
- Generate key pair and add public key to repo
- Create secret git-credentials for k8s using account provided by client
- User name
- Password
6.2 Setup SMTP
- Use AWS SES to create identity (<client name>.com)
- Setup using configuration below:
- smtp_host: check in SNS in SMTP endpoint
- smtp_user: created by SMTP credential
- smtp_password: created by SMTP credential
- smtp_port: check in SNS
- smtp_mail_from: check in SNS
- SMTP declares region in smtp_host, so that only email accounts in this region are able to received email messages
7. Database
- Use AWS RDS
- Type: MySQL
- Access: by creating a tunnel from local machine through bastion host to AWS RDS instance
- Setup networking
- VPC
- Subnet group: 2 private subnet group created in Infrastructure step
8. Container registry
- Use ECR
- Login to ecr using aws ecr get-login-password
- Login to docker using docker login
- Pull image from DXG repository to local machine in release branch
- Make a tag of image
- Push image to AWS ECR
9. Cert manager
- Cert-manager helps to automatically provision certificates for Ingress resources
- Please contact our team to get more information.
10. Keycloak
- Please contact our team to get more information.
11. Helm
- Start services using helm
- Update repository, tag, host, config
12. Event stream
- Create lambda function
- Upload code
- Declare webhook
- Attach VPC, subnet, SGs
- Create rules using AWS EventBridge
- Declare event pattern: catch EC instance status change
