附016.Kubernetes_v1.17.4高可用部署

• Etcd混布方式

1. 所需机器资源少
2. 部署简单，利于管理
3. 容易进行横向扩展
4. 风险大，一台宿主机挂了，master和etcd就都少了一套，集群冗余度受到的影响比较大。

• Etcd独立部署方式：

1. 所需机器资源多（按照Etcd集群的奇数原则，这种拓扑的集群关控制平面最少需要6台宿主机了）
2. 部署相对复杂，要独立管理etcd集群和和master集群
3. 解耦了控制平面和Etcd，集群风险小健壮性强，单独挂了一台master或etcd对集群的影响很小

  1 #!/bin/sh
  2 #****************************************************************#
  3 # ScriptName: k8sinit.sh
  4 # Author: xhy
  5 # Create Date: 2019-06-23 22:19
  6 # Modify Author: xhy
  7 # Modify Date: 2020-03-16 20:44
  8 # Version: 
  9 #***************************************************************#
 10 # Initialize the machine. This needs to be executed on every machine.
 11 
 12 # Add host domain name.
 13 cat >> /etc/hosts  14 172.24.8.71 master01
 15 172.24.8.72 master02
 16 172.24.8.73 master03
 17 172.24.8.74 worker01
 18 172.24.8.75 worker02
 19 172.24.8.76 worker03
 20 EOF
 21 
 22 # Add docker user
 23 useradd -m docker
 24 
 25 # Disable the SELinux.
 26 sed -i ‘s/^SELINUX=.*/SELINUX=disabled/‘ /etc/selinux/config
 27 
 28 # Turn off and disable the firewalld.
 29 systemctl stop firewalld
 30 systemctl disable firewalld
 31 
 32 # Modify related kernel parameters & Disable the swap.
 33 cat > /etc/sysctl.d/k8s.conf  34 net.ipv4.ip_forward = 1
 35 net.bridge.bridge-nf-call-ip6tables = 1
 36 net.bridge.bridge-nf-call-iptables = 1
 37 net.ipv4.tcp_tw_recycle = 0
 38 vm.swappiness = 0
 39 vm.overcommit_memory = 1
 40 vm.panic_on_oom = 0
 41 net.ipv6.conf.all.disable_ipv6 = 1
 42 EOF
 43 sysctl -p /etc/sysctl.d/k8s.conf >&/dev/null
 44 swapoff -a
 45 sed -i ‘/ swap / s/^\(.*\)$/#\1/g‘ /etc/fstab
 46 modprobe br_netfilter
 47 
 48 # Add ipvs modules
 49 cat > /etc/sysconfig/modules/ipvs.modules  50 #!/bin/bash
 51 modprobe -- ip_vs
 52 modprobe -- ip_vs_rr
 53 modprobe -- ip_vs_wrr
 54 modprobe -- ip_vs_sh
 55 modprobe -- nf_conntrack_ipv4
 56 EOF
 57 chmod 755 /etc/sysconfig/modules/ipvs.modules
 58 bash /etc/sysconfig/modules/ipvs.modules
 59 
 60 # Install rpm
 61 yum install -y conntrack ntpdate ntp ipvsadm ipset jq iptables curl sysstat libseccomp wget
 62 
 63 # Install Docker Compose
 64 sudo curl -L "http://down.linuxsb.com:8888/docker/compose/releases/download/1.25.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
 65 sudo chmod +x /usr/local/bin/docker-compose
 66 
 67 # Update kernel
 68 rpm --import http://down.linuxsb.com:8888/RPM-GPG-KEY-elrepo.org
 69 rpm -Uvh http://down.linuxsb.com:8888/elrepo-release-7.0-4.el7.elrepo.noarch.rpm
 70 yum --disablerepo="*" --enablerepo="elrepo-kernel" install -y kernel-ml
 71 sed -i ‘s/^GRUB_DEFAULT=.*/GRUB_DEFAULT=0/‘ /etc/default/grub
 72 grub2-mkconfig -o /boot/grub2/grub.cfg
 73 yum update -y
 74 
 75 # Reboot the machine.
 76 # reboot
 77 

  1 #!/bin/sh
  2 #****************************************************************#
  3 # ScriptName: environment.sh
  4 # Author: xhy
  5 # Create Date: 2020-03-16 23:11
  6 # Modify Author: xhy
  7 # Modify Date: 2020-03-16 23:11
  8 # Version: 
  9 #***************************************************************#
 10 # 集群 MASTER 机器 IP 数组
 11 export MASTER_IPS=(172.24.8.71 172.24.8.72 172.24.8.73)
 12 
 13 # 集群 MASTER IP 对应的主机名数组
 14 export MASTER_NAMES=(master01 master02 master03)
 15 
 16 # 集群 NODE 机器 IP 数组
 17 export NODE_IPS=(172.24.8.74 172.24.8.75 172.24.8.76)
 18 
 19 # 集群 NODE IP 对应的主机名数组
 20 export NODE_NAMES=(worker01 worker02 worker03)
 21 
 22 # 集群所有机器 IP 数组
 23 export ALL_IPS=(172.24.8.71 172.24.8.72 172.24.8.73 172.24.8.74 172.24.8.75 172.24.8.76)
 24 
 25 # 集群所有IP 对应的主机名数组
 26 export ALL_NAMES=(master01 master02 master03 worker01 worker02 worker03)
 27 

  1 #!/bin/bash
  2 
  3 #######################################
  4 # set variables below to create the config files, all files will create at ./config directory
  5 #######################################
  6 
  7 # master keepalived virtual ip address
  8 export K8SHA_VIP=172.24.8.71
  9 
 10 # master01 ip address
 11 export K8SHA_IP1=172.24.8.72
 12 
 13 # master02 ip address
 14 export K8SHA_IP2=172.24.8.73
 15 
 16 # master03 ip address
 17 export K8SHA_IP3=172.24.8.74
 18 
 19 # master keepalived virtual ip hostname
 20 export K8SHA_VHOST=master
 21 
 22 # master01 hostname
 23 export K8SHA_HOST1=master01
 24 
 25 # master02 hostname
 26 export K8SHA_HOST2=master02
 27 
 28 # master03 hostname
 29 export K8SHA_HOST3=master03
 30 
 31 # master01 network interface name
 32 export K8SHA_NETINF1=eth0
 33 
 34 # master02 network interface name
 35 export K8SHA_NETINF2=eth0
 36 
 37 # master03 network interface name
 38 export K8SHA_NETINF3=eth0
 39 
 40 # keepalived auth_pass config
 41 export K8SHA_KEEPALIVED_AUTH=412f7dc3bfed32194d1600c483e10ad1d
 42 
 43 # calico reachable ip address
 44 export K8SHA_CALICO_REACHABLE_IP=172.24.8.2
 45 
 46 # kubernetes CIDR pod subnet
 47 export K8SHA_CIDR=10.10.0.0
 48 

• kubeadm-config.yaml：kubeadm初始化配置文件，位于kubeadm-ha代码的./根目录
• keepalived：keepalived配置文件，位于各个master节点的/etc/keepalived目录
• nginx-lb：nginx-lb负载均衡配置文件，位于各个master节点的/root/nginx-lb目录
• calico.yaml：calico网络组件部署文件，位于kubeadm-ha代码的./calico目录

  1 #!/bin/bash
  2 # 使用如下脚本下载国内镜像，并修改tag为google的tag
  3 KUBE_VERSION=v1.17.4
  4 KUBE_PAUSE_VERSION=3.1
  5 ETCD_VERSION=3.4.3-0
  6 CORE_DNS_VERSION=1.6.5
  7 GCR_URL=k8s.gcr.io
  8 ALIYUN_URL=registry.cn-hangzhou.aliyuncs.com/google_containers
  9 
 10 images=(kube-proxy:${KUBE_VERSION}
 11 kube-scheduler:${KUBE_VERSION}
 12 kube-controller-manager:${KUBE_VERSION}
 13 kube-apiserver:${KUBE_VERSION}
 14 pause:${KUBE_PAUSE_VERSION}
 15 etcd:${ETCD_VERSION}
 16 coredns:${CORE_DNS_VERSION})
 17 
 18 for imageName in ${images[@]} ; do
 19   docker pull $ALIYUN_URL/$imageName
 20   docker tag $ALIYUN_URL/$imageName $GCR_URL/$imageName
 21   docker rmi $ALIYUN_URL/$imageName
 22 done
 23 

  1 You can now join any number of the control-plane node running the following command on each as root:
  2 
  3   kubeadm join 172.24.8.254:16443 --token r0awcu.lqd10ud79onpnti7   4     --discovery-token-ca-cert-hash sha256:b010761631964b9ff381eb2dfcdd0708acd50b544782c3ed241e303f67cbe4ab   5     --control-plane --certificate-key b238ee75df1efb88880f2453e1e6e1afb7e774b57817eb3fb15a4a9e578e92fd
  6 
  7 Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
  8 As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
  9 "kubeadm init phase upload-certs --upload-certs" to reload certs afterward.
 10 
 11 Then you can join any number of worker nodes by running the following on each as root:
 12 
 13 kubeadm join 172.24.8.254:16443 --token r0awcu.lqd10ud79onpnti7  14     --discovery-token-ca-cert-hash sha256:b010761631964b9ff381eb2dfcdd0708acd50b544782c3ed241e303f67cbe4ab
 15 

• [kubelet-start] 生成kubelet的配置文件”/var/lib/kubelet/config.yaml”
• [certificates]生成相关的各种证书
• [kubeconfig]生成相关的kubeconfig文件
• [bootstraptoken]生成token记录下来，后边使用kubeadm join往集群中添加节点时会用到