kubernetes install - kubespray

Posted by 크롱_k
2018.07.06 17:41 서버/리눅스 이야기

1. kubespray?


 - kubespray 는 ansible 기반의 kubernetes 설치 프로젝트.

 - 다중 마스터 클러스터를 손쉽게 구축 가능.

- 아래는 kubespray의 github의 설명. ( https://github.com/kubernetes-incubator/kubespray )

  • Can be deployed on AWS, GCE, Azure, OpenStack, vSphere or Baremetal
  • Highly available cluster
  • Composable (Choice of the network plugin for instance)
  • Supports most popular Linux distributions
  • Continuous integration tests

 - 해당 글은 CentOS 7.5( 2018,07,06 최신 ) 에서 설치 되었음.



2. kubespray 사전 준비


2.1. SELinux Disable


# setenforce 0
# sed ---follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
cs


2.2 Master 서버의 firewall 설정


# firewall-cmd --permanent --add-port=6443/tcp
# firewall-cmd --permanent --add-port=2379-2380/tcp
# firewall-cmd --permanent --add-port=10250/tcp
# firewall-cmd --permanent --add-port=10251/tcp
# firewall-cmd --permanent --add-port=10252/tcp
# firewall-cmd --permanent --add-port=10255/tcp
# firewall-cmd --reload
# modprobe br_netfilter
# echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
# sysctl -w net.ipv4.ip_forward=1
cs


2.3 node ( minion ) 서버의 firewall 설정


# firewall-cmd --permanent --add-port=10250/tcp
# firewall-cmd --permanent --add-port=10255/tcp
# firewall-cmd --permanent --add-port=30000-32767/tcp
# firewall-cmd --permanent --add-port=6783/tcp
# firewall-cmd --permanent --add-port=2380/tcp
# firewall-cmd --reload
# modprobe br_netfilter
# echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables
# sysctl -w net.ipv4.ip_forward=1
cs

2.4 Master 서버에 필수 패키지 설치


2.4.0 pip 설치


# yum install epel-release
# yum install python-pip
# pip --version
pip 8.1.2 from /usr/lib/python2.7/site-packages (python 2.7)

# pip install --upgrade pip
# pip --version
pip 10.0.1 from /usr/lib/python2.7/site-packages/pip (python 2.7)
cs

2.4.1 Ansible 설치 


# pip install ansible

Collecting ansible
.
.
. [ 중략 ]
Successfully installed MarkupSafe-1.0 PyYAML-3.13 ansible-2.6.1 asn1crypto-0.24.0 bcrypt-3.1.4 cffi-1.11.5 cryptography-2.2.2 enum34-1.1.6 idna-2.7 jinja2-2.10 paramiko-2.4.1 pyasn1-0.4.3 pycparser-2.18 pynacl-1.2.1
You are using pip version 8.1.2, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
cs


2.4.2 Jinja2 설치


# pip install jinja2
Collecting jinja2
.
.
. [ 중략 ]
Successfully installed jinja-1.2
You are using pip version 8.1.2, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
 
cs


2.4.3 Python 설치 


# yum install python36 –y
 
Installed:
  python36.x86_64 0:3.6.3-7.el7 
cs


2.4.4 ansible-modules-hashivault 설치


# pip install ansible-modules-hashivault==3.8.0
Collecting ansible-modules-hashivault==3.8.0
 
 
# pip show ansible-modules-hashivault
Name: ansible-modules-hashivault
Version: 3.8.0
 
cs


2.4.5 hvac 설치

1
2
3
4
5
6
# pip install hvac==0.3.0
Collecting hvac==0.3.0
 
# pip show hvac
Name: hvac
Version: 0.3.0
cs


2.4.6 requests  설치 


# cd /usr/local/src/
# wget https://files.pythonhosted.org/packages/54/1f/782a5734931ddf2e1494e4cd615a51ff98e1879cbe9eecbdfeaf09aa75e9/requests-2.19.1.tar.gz
# tar zxvf requests-2.19.1.tar.gz 
# cd requests-2.19.1/
# python setup.py install
 
-------------------------------------------
Installed /usr/lib/python2.7/site-packages/requests-2.19.1-py2.7.egg
Processing dependencies for requests==2.19.1
Searching for certifi==2018.4.16
Best match: certifi 2018.4.16
Adding certifi 2018.4.16 to easy-install.pth file
 
Using /usr/lib/python2.7/site-packages
Searching for urllib3==1.23
Best match: urllib3 1.23
Adding urllib3 1.23 to easy-install.pth file
 
Using /usr/lib/python2.7/site-packages
Searching for idna==2.7
Best match: idna 2.7
Adding idna 2.7 to easy-install.pth file
 
Using /usr/lib/python2.7/site-packages
Searching for chardet==3.0.4
Best match: chardet 3.0.4
Adding chardet 3.0.4 to easy-install.pth file
Installing chardetect script to /usr/bin
 
Using /usr/lib/python2.7/site-packages
Finished processing dependencies for requests==2.19.1
 
cs


2.5 Master 서버의 ssh keygen 생성 및 배포 


 # ssh-keygen -t rsa -4096 -"root@192.168.0.11"
 # ssh-copy-id root@192.168.0.11
 # ssh-copy-id root@192.168.0.12
 # ssh-copy-id root@192.168.0.13
 # ssh-copy-id root@192.168.0.14
cs


3. kubespray 소스 다운로드


# cd /usr/local/src
# git clone https://github.com/kubernetes-incubator/kubespray.git
cs


4. kubespray 의존성 페키지 설치  ( requirements.txt 이용 )


# cd kubespray/
# pip install -r requirements.txt 
 
Installing collected packages: netaddr, pbr, ansible-modules-hashivault
  Found existing installation: ansible-modules-hashivault 3.8.0
    Uninstalling ansible-modules-hashivault-3.8.0:
      Successfully uninstalled ansible-modules-hashivault-3.8.0
  Running setup.py install for ansible-modules-hashivault ... done
Successfully installed ansible-modules-hashivault-3.9.6 netaddr-0.7.19 pbr-4.1.0
cs


5. inventory에 클러스터 구성 Ansible 업데이트


# pwd
/usr/local/src/kubespray
 
# cp -arp inventory/sample inventory/kubecluster
# declare -a IPS=(192.168.0.11 192.168.0.12 192.168.0.13 192.168.0.14)
# CONFIG_FILE=inventory/kubecluster/hosts.ini python36 contrib/inventory_builder/inventory.py ${IPS[@]}
 
DEBUG: Adding group all
DEBUG: Adding group kube-master
DEBUG: Adding group kube-node
DEBUG: Adding group etcd
DEBUG: Adding group k8s-cluster:children
DEBUG: Adding group calico-rr
DEBUG: Adding group vault
DEBUG: adding host node1 to group all
DEBUG: adding host node2 to group all
DEBUG: adding host node3 to group all
DEBUG: adding host node4 to group all
DEBUG: adding host kube-node to group k8s-cluster:children
DEBUG: adding host kube-master to group k8s-cluster:children
DEBUG: adding host node1 to group etcd
DEBUG: adding host node1 to group vault
DEBUG: adding host node2 to group etcd
DEBUG: adding host node2 to group vault
DEBUG: adding host node3 to group etcd
DEBUG: adding host node3 to group vault
DEBUG: adding host node1 to group kube-master
DEBUG: adding host node2 to group kube-master
DEBUG: adding host node1 to group kube-node
DEBUG: adding host node2 to group kube-node
DEBUG: adding host node3 to group kube-node
DEBUG: adding host node4 to group kube-node
cs


6. 클러스터 구성에 대한 inventory/kubecluster/hosts.ini 설정 ( 위의 5번 단계 진행시 자동 입력됨, 그러나 다른구성을 원한다면 수정 진행 )


# vi inventory/kubecluster/hosts.ini
 
[all]
node1    ansible_host=192.168.0.11 ip=192.168.0.11
node2    ansible_host=192.168.0.12 ip=192.168.0.12
node3    ansible_host=192.168.0.13 ip=192.168.0.13
node4    ansible_host=192.168.0.14 ip=192.168.0.14
 
[kube-master]
node1
node2
 
[kube-node]
node1
node2
node3
node4
 
[etcd]
node1
node2
node3
 
[k8s-cluster:children]
kube-node
kube-master
 
[calico-rr]
 
[vault]
node1
node2
node3
cs


7. 네트워크 플러그인 선택 ( 규모에 맞게 설정 진행 ) 


vi inventory/kubecluster/group_vars/k8s-cluster.yml 
 
67 # Can also be set to 'cloud', which lets the cloud provider setup appropriate routing
68 kube_network_plugin: calico
cs


8. 메트릭스가 클러스터 리소스 사용 데이터 ( nodes, pods ) 를 가저올 수 있도록 주석 제거 


# vi inventory/kubecluster/group_vars/all.yml
 
132 # The read-only port for the Kubelet to serve on with no authentication/authorization. Uncomment to enable.
133 kube_read_only_port: 10255
cs


9.  Ansible Playbook를 이용하여 Kubespray 배포


# pwd
/usr/local/src/kubespray
 
 
# ansible-playbook -i inventory/kubecluster/hosts.ini cluster.yml
.
.
.[중략]
 
PLAY RECAP *************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0   
node1                      : ok=343  changed=111  unreachable=0    failed=0   
node2                      : ok=314  changed=107  unreachable=0    failed=0   
node3                      : ok=281  changed=88   unreachable=0    failed=0   
node4                      : ok=230  changed=68   unreachable=0    failed=0   
 
# kubectl get nodes
NAME      STATUS    ROLES         AGE       VERSION
node1     Ready     master,node   3m        v1.10.4
node2     Ready     master,node   3m        v1.10.4
node3     Ready     node          3m        v1.10.4
node4     Ready     node          3m        v1.10.4
cs


'서버 > 리눅스 이야기' 카테고리의 다른 글

kubernetes Monitoring - prometheus  (0) 2018.07.18
kubernetes - dashboard 사용자 생성  (0) 2018.07.12
kubernetes install - kubespray  (1) 2018.07.06
vi 공백 지우기  (0) 2017.01.06
linux 메모리 슬롯확인  (0) 2016.11.16
vpopmail dovecot qmail 연동  (0) 2016.10.28
이 댓글을 비밀 댓글로
  1. 위처럼 설치시 문제는 없으나 /var/log/messages 로그상에 노드 포트 관련 커넥션 문제 확인됨
    해당 포트를 노드포트쪽에 방화벽 열어주면됨..