limit range/resource-quota 시나리오

시나리오1

rq.yaml 명세서 작성

apiVersion: v1
kind: ResourceQuota
metadata:
  name: rq
  namespace: space1
spec:
  hard:
    requests.memory: 1Gi
    requests.cpu: "1000m" #1000밀리 코어 = 1코어
    limits.memory: 1Gi
    limits.cpu: "1500m"

namespace 생성/namespace 기반의 ResourceQuota 생성

root@ip-172-31-4-27:~/resource# kubectl create ns space1
namespace/space1 created

#@ namespace에 ResourceQuota를 적용한다.
root@ip-172-31-4-27:~/resource# kubectl create -n space1 -f rq.yaml
resourcequota/rq created

#@ResourceQuota 적용 확인
root@ip-172-31-4-27:~/resource# kubectl describe resourcequotas --namespace=space1 
Name:            rq
Namespace:       space1
Resource         Used  Hard
--------         ----  ----
limits.cpu       0     1500m
limits.memory    0     1Gi
requests.cpu     0     1
requests.memory  0     1Gi

#@alias로 간편하게 설정
$ alias ksp1="kubectl -n space1 "

#@ namespace가 space1의 ResourceQuota 정보 확인
root@ip-172-31-4-27:~/resource# ksp1 get resourcequotas
NAME   AGE    REQUEST                                     LIMIT
rq     2m7s   requests.cpu: 0/1, requests.memory: 0/1Gi   limits.cpu: 0/1500m, limits.memory: 0/1Gi

 

pod.yaml 명세서 작성

apiVersion: v1 #request limit 을 적용하지 않은 경우
kind: Pod
metadata:
  name: pod-rq
spec:
  containers:
  - name: nasamjang02
    image: nginx

 

pod 생성시 reject 되는지 확인한다.

#@ reject 사유 : must specify limits.cpu,limits.memory,requests.cpu,requests.memory
root@ip-172-31-4-27:~/resource# ksp1 create -f pod.yaml
Error from server (Forbidden): error when creating "pod.yaml": pods "pod-rq" is forbidden: failed quota: rq: must specify limits.cpu,limits.memory,requests.cpu,requests.memory

 

pod-rq1.yaml 생성

apiVersion: v1
kind: Pod
metadata:
  name: pod-rq
spec:
  containers:
  - name: nasamjang02
    image: nginx
    resources:          #<--------메모리 관련 설정 이하 추가
      requests:
        memory: 0.1Gi
      limits:
        memory: 0.1Gi

 

pod 생성시 forbidden 이유를 확인한다.

#@ 쿼터에 CPU 값도 설정을 했기 떄문에 또 reject이 발생한다.
root@ip-172-31-4-27:~/resource# ksp1 create -f pod-rq1.yaml 
Error from server (Forbidden): error when creating "pod-rq1.yaml": pods "pod-rq" is forbidden: failed quota: rq: must specify limits.cpu,requests.cpu

 

pod-rq2.yaml 명세서 생성

apiVersion: v1
kind: Pod
metadata:
  name: pod-rq
spec:
  containers:
  - name: nasamjang02
    image: nginx
    resources:
      requests:
        memory: 0.1Gi
        cpu: "100m"  #<--------CPU까지 추가
      limits:
        memory: 0.1Gi
        cpu: "100m" #<--------추가

ResourceQuota에 모두 만족하므로 생성 가능

root@ip-172-31-4-27:~/resource# ksp1 create -f pod-rq2.yaml 
pod/pod-rq created

 

rq.yaml 명세서 생성

apiVersion: v1
kind: ResourceQuota
metadata:
  name: rq
  namespace: space1
spec:
  hard:
    requests.memory: 1Gi
    limits.memory: 1Gi
    pods: 2     #<------추가

pod: 2를 추가 하여 ResourceQuota 수정/적용

root@ip-172-31-4-27:~/resource# ksp1 apply -f rq.yaml 
Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply
resourcequota/rq configured

 

pods : 2를 정상 적용된 것을 확인한다.

root@ip-172-31-4-27:~/resource# ksp1 describe resourcequotas
Name:            rq
Namespace:       space1
Resource         Used           Hard
--------         ----           ----
limits.cpu       100m           1500m
limits.memory    107374182400m  1Gi
pods             1              2
requests.cpu     100m           1
requests.memory  107374182400m  1Gi

pod-rq2.yaml 명세서에서 pod name을 변경

 

pod-rq2 생성 완료 확인

root@ip-172-31-4-27:~/resource# ksp1 create -f pod-rq2.yaml
pod/pod-rq2 created

pod-rq2.yaml 명세서에서 pod name을 변경

 

rq-3에 대해서 forbidden 사유 : pod 갯수가 2개로 제한되어 있기때문에 3은 forbidden 된것을 확인

root@ip-172-31-4-27:~/resource# ksp1 create -f pod-rq2.yaml
Error from server (Forbidden): error when creating "pod-rq2.yaml": pods "pod-rq3" is forbidden: exceeded quota: rq, requested: pods=1, used: pods=2, limited: pods=2

 

 

시나리오2 (이미 pod가 배포된 상태, ResourceQuota를 변경하면 어떻게 되는지 확인하는 시나리오)

신규 space2 namespace 생성

root@ip-172-31-4-27:~/resource# kubectl create ns space2
namespace/space2 created

pod-rq1.yaml 명세서 생성

apiVersion: v1
kind: Pod
metadata:
  name: pod-rq1
spec:
  containers:
  - name: nasamjang02
    image: nginx
    resources:
      requests:
        memory: 1Gi
      limits:
        memory: 1Gi

$alias ksp2='kubectl -n space2 '
root@ip-172-31-4-27:~/resource# ksp2 create -f pod-rq1.yaml
pod/pod-rq1 created

namespace : space2 에는 아직 ResourceQuota를 설정해 주지 않았다.

root@ip-172-31-4-27:~/resource# ksp2 describe resourcequotas
No resources found in space2 namespace.

 

namespace : space2 에 ResourceQuota 적용

root@ip-172-31-4-27:~/resource# ksp2 create -f rq.yaml
resourcequota/rq created

 

namespace : space2 에 적용된 ResourceQuota 확인

root@ip-172-31-4-27:~/resource# ksp2 describe resourcequotas
Name:            rq
Namespace:       space2
Resource         Used  Hard
--------         ----  ----
limits.memory    1Gi   1Gi
pods             1     2
requests.memory  1Gi   1Gi

pod-rq1.yaml 명세서 작성 (이름 변경해서 적용을 했을때 어떻게 되는가?)

 

이미 설정값을 만족했기 때문에 forbidden이 발생한다.

root@ip-172-31-4-27:~/resource# ksp2 create -f pod-rq1.yaml
Error from server (Forbidden): error when creating "pod-rq1.yaml": pods "pod-rq2" is forbidden: exceeded quota: rq, requested: limits.memory=1Gi,requests.memory=1Gi, used: limits.memory=1Gi,requests.memory=1Gi, limited: limits.memory=1Gi,requests.memory=1Gi

 

어떤 오류메시지인지 확인필요

root@ip-172-31-4-27:~/resource# ksp2 create -f pod.yaml 
Error from server (Forbidden): error when creating "pod.yaml": pods "pod-rq" is forbidden: failed quota: rq: must specify limits.memory,requests.memory

 

object-counts.yaml 명세서 작성

apiVersion: v1 
kind: ResourceQuota 
metadata: 
  name: object-counts 
spec: 
  hard: 
    configmaps: "10" 
    persistentvolumeclaims: "4"
    secrets: "10" 
    services: "10" 
    services.loadbalancers: "2"

 

root@ip-172-31-4-27:~/resource# kubectl create -f object-counts.yaml --namespace=space2
resourcequota/object-counts created

 

root@ip-172-31-4-27:~/resource# ksp2 describe resourcequotas
Name:                   object-counts
Namespace:              space2
Resource                Used  Hard
--------                ----  ----
configmaps              0     10
persistentvolumeclaims  0     4
secrets                 1     10
services                0     10
services.loadbalancers  0     2


Name:            rq
Namespace:       space2
Resource         Used  Hard
--------         ----  ----
limits.memory    1Gi   1Gi
pods             1     2
requests.memory  1Gi   1Gi

 

시나리오3 : resource Limit test

limit.yaml 명세서 작성

apiVersion: v1
kind: LimitRange # LimitRange를 설정한다.
metadata:
  name: lr-base
spec:
  limits:
  - type: Pod #Pod 레벨의 limit 설정
    min:
      cpu: 50m
      memory: 5Mi
    max:
      cpu: 1
      memory: 1Gi
  - type: Container
    defaultRequest: # 컨테이너의 기본값 설정
      cpu: 100m
      memory: 10Mi
    default:
       cpu: 200m
       memory: 100Mi
    min:
       cpu: 50m
       memory: 5Mi
    max:
      cpu: 1
      memory: 1Gi
    maxLimitRequestRatio:
      cpu: 4
      memory: 10

 

root@ip-172-31-4-27:~/resource# kubectl create -f limit.yaml --namespace=space2
limitrange/lr-base created

#@ 리소스를 확인한다.
root@ip-172-31-4-27:~/resource# kubectl get -f limit.yaml --namespace=space2
NAME      CREATED AT
lr-base   2020-12-04T01:28:06Z

#@ 파드레벨, 컨테이너레벨의 정보 조회 확인
root@ip-172-31-4-27:~/resource# kubectl describe -f limit.yaml --namespace=space2
Name:       lr-base
Namespace:  space2
Type        Resource  Min  Max  Default Request  Default Limit  Max Limit/Request Ratio
----        --------  ---  ---  ---------------  -------------  -----------------------
Pod         cpu       50m  1    -                -              -
Pod         memory    5Mi  1Gi  -                -              -
Container   cpu       50m  1    100m             200m           4
Container   memory    5Mi  1Gi  10Mi             100Mi          10

#@ 제한된 리소스 초과(메모리, CPU)로 인해 forbidden 발생 확인
root@ip-172-31-4-27:~/resource# kubectl create -f pod.yaml --namespace=space2
Error from server (Forbidden): error when creating "pod.yaml": pods "pod-rq" is forbidden: exceeded quota: rq, requested: limits.memory=100Mi,requests.memory=10Mi, used: limits.memory=1Gi,requests.memory=1Gi, limited: limits.memory=1Gi,requests.memory=1Gi

#@ namespace=space2로 생성된 모든 po를 삭제
root@ip-172-31-4-27:~/resource# kubectl delete po --all --namespace=space2
pod "pod-rq1" deleted

#@ 제한된 리소스를 초과하지 않으므로 pod 생성 완료
root@ip-172-31-4-27:~/resource# kubectl create -f pod.yaml --namespace=space2
pod/pod-rq created

#@ pod.yaml은 limit 을 설정하지 않았었다. 기본 limit_range에 의해서 적용되어 있는지 확인한다.
root@ip-172-31-4-27:~/resource# kubectl get -f pod.yaml --namespace=space2 -o yaml
	생략
	:
	spec:
	  containers:
	  - image: nginx
		imagePullPolicy: Always
		name: nasamjang02
		resources:
		  limits:
			cpu: 200m
			memory: 100Mi
		  requests:
			cpu: 100m
			memory: 10Mi
            
root@ip-172-31-4-27:~/resource# ksp2 describe resourcequotas 
Name:                   object-counts
Namespace:              space2
Resource                Used  Hard
--------                ----  ----
configmaps              0     10
persistentvolumeclaims  0     4
secrets                 1     10
services                0     10
services.loadbalancers  0     2

Name:            rq
Namespace:       space2
Resource         Used   Hard
--------         ----   ----
limits.memory    100Mi  1Gi
pods             1      2
requests.memory  10Mi   1Gi

root@ip-172-31-4-27:~/resource# ksp2 describe limitranges
Name:       lr-base
Namespace:  space2
Type        Resource  Min  Max  Default Request  Default Limit  Max Limit/Request Ratio
----        --------  ---  ---  ---------------  -------------  -----------------------
Pod         cpu       50m  1    -                -              -
Pod         memory    5Mi  1Gi  -                -              -
Container   cpu       50m  1    100m             200m           4
Container   memory    5Mi  1Gi  10Mi             100Mi          10

 

pod-rq2.yaml 명세서 작성 및 실행/확인

apiVersion: v1
kind: Pod
metadata:
  name: pod-rq-3
spec:
  containers:
  - name: nasamjang02
    image: nginx
    resources:
      requests:
        memory: 2Mi  #<--------- min 5Mi 이하 test. limit_range 테스트
        cpu: "100m"
#      limits:
#        memory: 0.1Gi
#        cpu: "100m"

 

#@ ratio에 의해서 forbidden
root@ip-172-31-4-27:~/resource# kubectl create -f pod-rq2.yaml --namespace=space2
Error from server (Forbidden): error when creating "pod-rq2.yaml": pods "pod-rq-3" is forbidden: [minimum memory usage per Pod is 5Mi, but request is 2097152, minimum memory usage per Container is 5Mi, but request is 2Mi, memory max limit to request ratio per Container is 10, but provided ratio is 50.000000]

$ vi pod-rq2.yaml <------- memory: 5Mi  로 수정 (ratio 위배)

root@ip-172-31-4-27:~/resource# kubectl create -f pod-rq2.yaml --namespace=space2
Error from server (Forbidden): error when creating "pod-rq2.yaml": pods "pod-rq-3" is forbidden: memory max limit to request ratio per Container is 10, but provided ratio is 20.000000

$ vi pod-rq2.yaml  <------- memory: 10Mi (ratio 10)

root@ip-172-31-4-27:~/resource# kubectl create -f pod-rq2.yaml --namespace=space2
pod/pod-rq-3 created

root@ip-172-31-4-27:~/resource# kubectl get -f pod-rq2.yaml --namespace=space2
NAME       READY   STATUS    RESTARTS   AGE
pod-rq-3   1/1     Running   0          16s

$ 
spec:
  containers:
  - image: nginx
    imagePullPolicy: Always
    name: nasamjang02
    resources:
      limits:
        cpu: 200m
        memory: 100Mi
      requests:
        cpu: 100m
        memory: 10Mi

root@ip-172-31-4-27:~/resource# ksp2 describe resourcequotas 
Name:                   object-counts
Namespace:              space2
Resource                Used  Hard
--------                ----  ----
configmaps              0     10
persistentvolumeclaims  0     4
secrets                 1     10
services                0     10
services.loadbalancers  0     2


Name:            rq
Namespace:       space2
Resource         Used   Hard
--------         ----   ----
limits.memory    200Mi  1Gi
pods             2      2
requests.memory  20Mi   1Gi

root@ip-172-31-4-27:~/resource# ksp2 describe limitranges
Name:       lr-base
Namespace:  space2
Type        Resource  Min  Max  Default Request  Default Limit  Max Limit/Request Ratio
----        --------  ---  ---  ---------------  -------------  -----------------------
Pod         cpu       50m  1    -                -              -
Pod         memory    5Mi  1Gi  -                -              -
Container   memory    5Mi  1Gi  10Mi             100Mi          10
Container   cpu       50m  1    100m             200m           4