aws(学习笔记第三十四课) dockerized-app with asg-alb

• 使用cdk生成dockerized-app并使用AutoScalingGroup和ApplicationLoaderBalancer

学习内容：

• 使用cdk生成dockerized-app并使用AutoScalingGroup和ApplicationLoaderBalancer
• 在AutoScalingGroup中使用efs以及RDS

---

1. 整体架构

1.1 代码链接

代码链接（docker-app-with-asg-alb）

1.2 代码手动修改部分

这里的代码没有完全实现是理想的部署就能运行，需要修改几个地方。

1.2.1 rds_stack.py

修改instance_type=ec2.InstanceType.of这里的参数。不修改的话创建数据库运行失败。
修改后的代码：

from aws_cdk import (
    aws_rds as rds,
    aws_ec2 as ec2,
    RemovalPolicy, Stack
    )
from constructs import Construct


class RDSStack(Stack):

    def __init__(self, scope: Construct, id: str, props, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        # Creates a security group for AWS RDS
        sg_rds = ec2.SecurityGroup(
                self,
                id="sg_rds",
                vpc=props['vpc'],
                security_group_name="sg_rds"
        )

        # Adds an ingress rule which allows resources in the VPC's CIDR
        # to access the database.
        sg_rds.add_ingress_rule(
            peer=ec2.Peer.ipv4("10.0.0.0/16"),
            connection=ec2.Port.tcp(3306)
        )

        # Master username is 'admin' and database password is automatically
        # generated and stored in AWS Secrets Manager
        my_sql = rds.DatabaseInstance(
                self, "RDS",
                engine=rds.DatabaseInstanceEngine.mysql(
                    version=rds.MysqlEngineVersion.VER_8_0_39
                ),
                vpc=props['vpc'],
                port=3306,
                instance_type=ec2.InstanceType.of(
                    ec2.InstanceClass.R7G,
                    ec2.InstanceSize.LARGE,
                    ),
                removal_policy=RemovalPolicy.DESTROY,
                security_groups=[sg_rds]
                )

1.2.2 efs_stack.py

修改了efs创建和security group的创建顺序

修改后的代码：

from aws_cdk import (
    aws_efs as efs,
    aws_ec2 as ec2,
    Stack
    )
from constructs import Construct

class StorageStack(Stack):

    def __init__(self, scope: Construct, id: str, props, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        sg_efs = ec2.SecurityGroup(
                self,
                id="sg_efs",
                vpc=props['vpc'],
                security_group_name="sg_efs"
                )

        sg_efs.add_ingress_rule(
                peer=ec2.Peer.ipv4("10.0.0.0/16"),
                connection=ec2.Port.tcp(2049)
                )
        elasticfilestore = efs.CfnFileSystem(
                self, "efs-storage",
                encrypted=False,
                lifecycle_policies=None
                # security_group_ids=[sg_efs.security_group_id]
        )

1.2.3 asg_stack.py

这里主要是因为AutoScalingGroup默认使用的是config方式来创建，但是最新版aws已经不支持config方式，这里主要改成launchTemplate方式。而且AutoScalingGroup启动的ec2是在private subnet，因此，需要public ec2进行访问，这里Allows only the IP of "123.123.123.123"换成自己的公网的ec2。

from aws_cdk.aws_ec2 import SubnetType
from aws_cdk import (
    aws_ec2 as ec2,
    aws_autoscaling as autoscaling,
    aws_elasticloadbalancingv2 as elbv2,
    Stack
    )
from constructs import Construct

class ASGStack(Stack):

    def __init__(self, scope: Construct, id: str, props, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        userdata_file = open("./userdata.sh", "rb").read()

        # Creates a userdata object for Linux hosts
        userdata = ec2.UserData.for_linux()
        # Adds one or more commands to the userdata object.
        userdata.add_commands(str(userdata_file, 'utf-8'))

        # Creates a security group for our application
        sg_nextcloud = ec2.SecurityGroup(
                self,
                id="sg_nextcloud",
                vpc=props['vpc'],
                security_group_name="sg_nextcloud"
        )

        # Allows only the IP of "123.123.123.123"
        # to access this security group for SSH
        sg_nextcloud.add_ingress_rule(
            peer=ec2.Peer.ipv4("18.183.236.249/32"),
            connection=ec2.Port.tcp(22)
        )
        # create launch template
        launch_template = ec2.LaunchTemplate(
            self, "MyLaunchTemplate",
            machine_image=ec2.AmazonLinuxImage(
                generation=ec2.AmazonLinuxGeneration.AMAZON_LINUX_2
            ),
            instance_type=ec2.InstanceType.of(
                ec2.InstanceClass.MEMORY5, ec2.InstanceSize.XLARGE
            ),
            security_group=sg_nextcloud,
            user_data=userdata,
            key_name="***", # 
        )

        asg = autoscaling.AutoScalingGroup(
            self,
            "app-asg",
            vpc=props['vpc'],
            vpc_subnets=ec2.SubnetSelection(subnet_type=SubnetType.PRIVATE_WITH_NAT),
            launch_template=launch_template,
            min_capacity=1,
            max_capacity=3,
        )

        # Creates a security group for the application load balancer
        sg_alb = ec2.SecurityGroup(
                self,
                id="sg_alb",
                vpc=props['vpc'],
                security_group_name="sg_alb"
        )

        # Allows connections from security group "sg_alb"
        # inside the "sg_nextcloud" security group to access port 8080
        # where our app listens
        sg_nextcloud.connections.allow_from(
                sg_alb, ec2.Port.tcp(8080), "Ingress")

        # Creates an application load balance
        lb = elbv2.ApplicationLoadBalancer(
                self,
                "ALB",
                vpc=props['vpc'],
                security_group=sg_alb,
                internet_facing=True)

        listener = lb.add_listener("Listener", port=80)
        # Adds the autoscaling group's (asg) instance to be registered
        # as targets on port 8080
        listener.add_targets("Target", port=8080, targets=[asg])
        # This creates a "0.0.0.0/0" rule to allow every one to access the
        # application
        listener.connections.allow_default_port_from_any_ipv4(
                "Open to the world"
                )

1.2.4 userdata.sh

这里主要修改docker-compose的版本，因为太低的版本已经不支持了。
另外DB的设置，要根据实际的RDSStack创建的DB进行重新设置。后面会详细说明。

#!/bin/sh

yum install docker -y
yum install -y amazon-efs-utils

# makes a directory
mkdir /nextclouddata
mount -t efs fs-d48c7f8c:/ /nextclouddata


# enable and start docker
systemctl enable docker
systemctl start docker

# bootstraps "docker compose"
yum install libxcrypt-compat
curl -L "https://github.com/docker/compose/releases/download/v2.34.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
usermod -aG docker ec2-user

# Gets local-persist
curl -fsSL https://raw.githubusercontent.com/CWSpear/local-persist/master/scripts/install.sh | bash
docker volume create -d local-persist -o mountpoint=/nextclouddata/nextcloud-data --name=nextcloud-data

# Heredoc for a docker-compose.yaml file
cat << 'EOF' > /home/ec2-user/docker-compose.yaml
version: '2'

volumes:
  nextcloud-data:
    external: true

services:
  app:
    image: nextcloud
    ports:
      - 8080:80
    volumes:
      - nextcloud-data:/var/www/html
    restart: always
    environment:
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=admin
      - MYSQL_PASSWORD=bflbfl1980314
      - MYSQL_HOST=nextcloud.csetrrtzbxti.ap-northeast-1.rds.amazonaws.com
EOF

docker-compose -f /home/ec2-user/docker-compose.yaml up

1.2 整体架构

这里，整体的架构已经被分为四个stack，并且其他三个stack都依赖NetworkStack。

• 首先构建网络，VPC和public subnet以及private subnet，之后构筑MySQL数据库，以及EFS存储，最后构建AutoScalingGroup以及ApplicationLoaderBalancer。AutoScalingGroup需要每个主机共享存储，所以需要构建EFS。

2.代码解析

2.1 全体app.py

props = {'namespace': 'NetworkStack '}
app = App()
ns = NetworkStack(app, "NetworkStack", props)

rds = RDSStack(app, "RDSStack", ns.outputs)
rds.add_dependency(ns)

asg = ASGStack(app, "ASGStack", ns.outputs)
asg.add_dependency(ns)

efs = StorageStack(app, "StorageStack", ns.outputs)
app.synth()

每个stack独立定义，都可以独立执行，也可以-all进行执行。

cdk --require-approval never deploy -all
cdk --require-approval never deploy NetworkStack
cdk --require-approval never deploy RDSStack
cdk --require-approval never deploy StorageStack
cdk --require-approval never deploy ASGStack

注意，这里后三个stack依赖NetowokStack。

2.2 NetworkStatck网络

class NetworkStack(Stack):

    def __init__(self, scope: Construct, id: str, props, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        # Subnet configurations for a public and private tier
        subnet1 = SubnetConfiguration(
                name="Public",
                subnet_type=SubnetType.PUBLIC,
                cidr_mask=24)
        subnet2 = SubnetConfiguration(
                name="Private",
                subnet_type=SubnetType.PRIVATE_WITH_NAT,
                cidr_mask=24)

        vpc = Vpc(self,
                  "TheVPC",
                  cidr="10.0.0.0/16",
                  enable_dns_hostnames=True,
                  enable_dns_support=True,
                  max_azs=2,
                  nat_gateway_provider=NatProvider.gateway(),
                  nat_gateways=1,
                  subnet_configuration=[subnet1, subnet2]
                  )

        # This will export the VPC's ID in CloudFormation under the key
        # 'vpcid'
        CfnOutput(self, "vpcid", value=vpc.vpc_id)

        # Prepares output attributes to be passed into other stacks
        # In this case, it is our VPC and subnets.
        self.output_props = props.copy()
        self.output_props['vpc'] = vpc
        self.output_props['subnets'] = vpc.public_subnets

    @property
    def outputs(self):
        return self.output_props

这里构建如下：

• 构建整个vpc
• 构建一个public subnet
• 构建一个private subnet
• 将vpc和subnet作为props参数保存起来。

2.3 MySQL数据库

class RDSStack(Stack):

    def __init__(self, scope: Construct, id: str, props, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        # Creates a security group for AWS RDS
        sg_rds = ec2.SecurityGroup(
                self,
                id="sg_rds",
                vpc=props['vpc'],
                security_group_name="sg_rds"
        )

        # Adds an ingress rule which allows resources in the VPC's CIDR
        # to access the database.
        sg_rds.add_ingress_rule(
            peer=ec2.Peer.ipv4("10.0.0.0/16"),
            connection=ec2.Port.tcp(3306)
        )

        # Master username is 'admin' and database password needs to be set after creation
        my_sql = rds.DatabaseInstance(
                self, "RDS",
                engine=rds.DatabaseInstanceEngine.mysql(
                    version=rds.MysqlEngineVersion.VER_8_0_39
                ),
                vpc=props['vpc'],
                port=3306,
                instance_type=ec2.InstanceType.of(
                    ec2.InstanceClass.R7G,
                    ec2.InstanceSize.LARGE,
                    ),
                removal_policy=RemovalPolicy.DESTROY,
                security_groups=[sg_rds]
                )

构建的为绿框的部分。

主要构建如下：

• 为RDS构建一个SecurityGroup
• 设定ingress_rule
• 之后构建RDS数据库

2.4 创建efs

因为AutoScalingGroup启动的多个ec2 instance都需要将/var/www/html使用相同的磁盘进行存储，所以要创建efs实现各个AutoScalingGroup启动的多个ec2 instance共享磁盘。

from aws_cdk import (
    aws_efs as efs,
    aws_ec2 as ec2,
    Stack
    )
from constructs import Construct


class StorageStack(Stack):

    def __init__(self, scope: Construct, id: str, props, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        sg_efs = ec2.SecurityGroup(
                self,
                id="sg_efs",
                vpc=props['vpc'],
                security_group_name="sg_efs"
                )

        sg_efs.add_ingress_rule(
                peer=ec2.Peer.ipv4("10.0.0.0/16"),
                connection=ec2.Port.tcp(2049)
                )
        elasticfilestore = efs.CfnFileSystem(
                self, "efs-storage",
                encrypted=False,
                lifecycle_policies=None
                # security_group_ids=[sg_efs.security_group_id]
        )

从ec2对efs进行mount如下图所示：

2.4 对efs创建mount target

aws(学习笔记第十一课) 使用AWS的EFS，以及AWS Storage Gateway，这里也对efs进行了详细的介绍。在ec2对efs进行mount之前，需要对其创建挂载目标mount target。可以认为，efs就是创建了一块磁盘，而创建挂载目标mount target就是对这块磁盘，又绑定了一个nfs server。

3.部署执行

3.1.部署NetworkStack

cdk --require-approval never deploy NetworkStack

3.2.部署RDSStack

cdk --require-approval never deploy RDSStack

3.3.部署StorageStack

cdk --require-approval never deploy StorageStack

3.4 修改userdata.sh文件

根据生成的DB和efs的具体值，重新设置userdata.sh文件。

3.5 最后部署ASGStack

cdk --require-approval never deploy ASGStack