뚠뚜니의 블로그

절차
1. EC2 생성
2. IAM Role 생성
3. EC2에 IAM Role 연결
4. Cloudwatch-agent 설치
5. Cloudwatch-agent config.json 생성/설정
6. Cloudwatch-agent 시작
7. 로그그룹 생성
8. 로그스트림 생성
9. 로그 확인
10. SSM을 통한 Run command 수행

1. EC2 생성 후 IAM Role 연결

4. Cloudwatch-agent 설치

$ wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm

$ sudo rpm -U ./amazon-cloudwatch-agent.rpm

5. Cloudwatch-agent config.json 생성/설정

amazon-cloudwatch-agent-config-wizard를 실행하여 기본설정을 시작한다.

$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

$ sudo vi /opt/aws/amazon-cloudwatch-agent/bin/config.json

file_path, log_group_name, log_stream_name을 셋팅해준다.

{
        "agent": {
                "metrics_collection_interval": 10,
                "run_as_user": "root",
                "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log"
        },
        "logs": {
                "logs_collected": {
                        "files": {
                                "collect_list": [
                                        {
                                                "file_path": "/home/ubuntu/logs/spring-boot-logging.log",
                                                "log_group_name": "spring-boot-logging",
                                                "log_stream_name": "{instance_id}"
                                        }
                                ]
                        }
                }
        },
        "metrics": {
                "append_dimensions": {
                        "AutoScalingGroupName": "${aws:AutoScalingGroupName}",
                        "ImageId": "${aws:ImageId}",
                        "InstanceId": "${aws:InstanceId}",
                        "InstanceType": "${aws:InstanceType}"
                },
                "metrics_collected": {
                        "collectd": {
                                "metrics_aggregation_interval": 60
                        },
                        "disk": {
                                "measurement": [
                                        "used_percent"
                                ],
                                "metrics_collection_interval": 60,
                                "resources": [
                                        "*"
                                ]
                        },
                        "mem": {
                                "measurement": [
                                        "mem_used_percent"
                                ],
                                "metrics_collection_interval": 60
                        },
                        "statsd": {
                                "metrics_aggregation_interval": 60,
                                "metrics_collection_interval": 10,
                                "service_address": ":8125"
                        }
                }
        }
}

6. Cloudwatch-agent 시작

Cloudwatch-agent 중지

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -m ec2 -a stop

위에서 수정한 config.json을 사용해 설정 파일 업데이트 후 시작 

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s

에러발생 시 /usr/share/collectd/types.db 파일이 없다는 에러발생시 types.db 파일생성

$ mkdir /usr/share/collectd
$ cd /usr/share/collectd
$ touch types.db

/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log 로그 확인하여 정상기동되었음을 확인

2020/03/18 13:30:23 I! I! Detected the instance is EC2
2020/03/18 13:30:23 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json ...
/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json does not exist or cannot read. Skipping it.
2020/03/18 13:30:23 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_config.json ...
Valid Json input schema.
I! Detecting runasuser...
No csm configuration found.
Configuration validation first phase succeeded
 
2020/03/18 13:30:23 I! Config has been translated into TOML /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml 
2020/03/18 13:30:23 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json ...
2020/03/18 13:30:23 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/file_config.json ...
2020/03/18 13:30:23 I! Detected runAsUser: root
2020/03/18 13:30:23 I! Change ownership to root:root
2020-03-18T13:30:23Z I! cloudwatch: get unique roll up list []
2020-03-18T13:30:23Z I! Starting AmazonCloudWatchAgent (version 1.237768.0)
2020-03-18T13:30:23Z I! Loaded outputs: cloudwatch cloudwatchlogs
2020-03-18T13:30:23Z I! Loaded inputs: disk mem socket_listener statsd tail
2020-03-18T13:30:23Z I! Tags enabled: host=ip-172-31-41-190
2020-03-18T13:30:23Z I! Agent Config: Interval:10s, Quiet:false, Hostname:"ip-172-31-41-190", Flush Interval:1s 
2020-03-18T13:30:23Z I! Started the statsd service on :8125
2020-03-18T13:30:23Z I! cloudwatch: publish with ForceFlushInterval: 1m0s, Publish Jitter: 37s
2020-03-18T13:30:23Z I! Statsd listener listening on:  [::]:8125
2020-03-18T13:30:24Z I! Reading from offset 9083 in /home/ubuntu/logs/spring-boot-logging.log

7. 로그그룹 생성

8. 로그스트림 생성

9. 로그 확인

10. SSM을 통한 Run command 수행

1.Tunneling 구성

1) Jumphost_Server의 Public Key, Public IP 확인 후 설정

2)확인된 IP와 Port 설정

3)포트:12022로 수신대기 설정 (이후 대상으로 전송)

4)연결 이후 Listening 확인

2.Client(sftp) 설정

ELB 에 접근한 Public IP 주소는 해당 어플리케이션에서 x-forwarded-for 값을 이용하여 조회가 가능.

코드 예제 :

AWS 에서도 제공하는 가이드 문서 링크 :

https://aws.amazon.com/ko/premiumsupport/knowledge-center/elb-capture-client-ip-addresses/

https://docs.aws.amazon.com/ko_kr/elasticloadbalancing/latest/userguide/how-elastic-load-balancing-works.html

1.EC2 생성 후 보안그룹 적용

2.EC2 -> AWS Redis로 접근 가능 하도록 보안그룹 생성 (EC2의 보안그룹과 맵핑)

3.AWS Redis 생성

4.XShell에 터널링 설정시 redis의 기본 엔드포인트 사설 주소를 입력

5.로컬에서 jedis 테스트 완료

Dynamo DB Retry 횟수 설정

- AS-IS : default 10회
- TO-BE : 8회 설정 예정 (사유 : default 설정인 경우 최대 51~60초에 delay 발생으로 response time이 길어지는 악순환이 되기 때문에 재시도 횟수를 줄여서 빠른 response 처리로 서버 부하 감소하기 위함)

[Dynamo DB Retry 정책 참고]

[총 재시도 횟수 별 시간 참고]

[설정 방법]

/**
 * 처음 설정할 때 대기 시간 (밀리 초)
 * */
private static final int CONNECTION_TIMEOUT = 60 * 1000;

/**
 * 클라이언트가 실행을 완료 할 수있는 시간 (밀리 초)
 * */
private static final int CLIENT_EXECUTION_TIMEOUT = 60 * 1000;

/**
 * 요청이 완료되기까지 대기하는 시간 (밀리 초)
 * */
private static final int REQUEST_TIMEOUT = 60 * 1000;

/**
 * 데이터가 전송 될 때까지 기다리는 시간 (밀리 초)
 * */
private static final int SOCKET_TIMEOUT = 60 * 1000;

/**
 * 요청 실패시 재시도 카운트
 * */
private static final int RETRY_COUNT = 8;

private AWSCredentialsProvider awsCredentialsProvider() {
   final BasicAWSCredentials awsCredentials = new BasicAWSCredentials(accessKey, secretKey);
   final AWSCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(awsCredentials);
   return credentialsProvider;
}

private ClientConfiguration createDynamoDBClientConfiguration() {
ClientConfiguration clientConfiguration = new ClientConfiguration()
   .withConnectionTimeout(CONNECTION_TIMEOUT)
   .withClientExecutionTimeout(CLIENT_EXECUTION_TIMEOUT)
   .withRequestTimeout(REQUEST_TIMEOUT)
   .withSocketTimeout(SOCKET_TIMEOUT)
   .withRetryPolicy(PredefinedRetryPolicies.getDynamoDBDefaultRetryPolicyWithCustomMaxRetries(RETRY_COUNT));

return clientConfiguration;
}

@Bean
public DynamoDBMapper dynamoDBMapper() {
Regions region = Regions.fromName(regionName);
AmazonDynamoDBClientBuilder builder = AmazonDynamoDBClientBuilder.standard()
   .withCredentials(awsCredentialsProvider())
   .withClientConfiguration(createDynamoDBClientConfiguration())
   .withRegion(region);
if (!StringUtils.isEmpty(serviceEndpoint)) {
   System.out.println(String.format("serviceEndpoint: '%s'", serviceEndpoint));
   EndpointConfiguration configuration = new EndpointConfiguration(serviceEndpoint, regionName);
   builder.withEndpointConfiguration(configuration);
}
AmazonDynamoDB dynamoDB = builder.build();

DynamoDBMapper mapper = new DynamoDBMapper(dynamoDB);
   return mapper;
}