kubernetes（k8s）集群部署（超详细）_kubernetes集群部署详细教程

k8s部署

• kubernetes
• • 集群图例
  • kubernetes 安装
  • • 仓库初始化
    • • 1、创建云主机
      • 2、初始化私有仓库
    • kube-master安装
    • • 1、防火墙相关配置
      • 2、配置yum仓库(跳板机)
      • 3、安装软件包(master)
      • 4、镜像导入私有仓库
      • 5、Tab键设置
      • 6、安装代理软件包
      • 7、配置内核参数
      • 8、使用kubeadm部署
      • 9、验证安装结果
    • 计算节点安装
    • • 1、获取token
      • 2、node安装
      • 3、验证安装
    • 网络插件安装配置
    • • 1、上传镜像到私有仓库
      • 2、修改配置文件并安装
      • 3、验证结果

kubernetes

集群图例

#mermaid-svg-fEJzgwgf3Hns34rd {font-family:\"trebuchet ms\",verdana,arial,sans-serif;font-size:16px;fill:#333;}#mermaid-svg-fEJzgwgf3Hns34rd .error-icon{fill:#552222;}#mermaid-svg-fEJzgwgf3Hns34rd .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-fEJzgwgf3Hns34rd .edge-thickness-normal{stroke-width:2px;}#mermaid-svg-fEJzgwgf3Hns34rd .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-fEJzgwgf3Hns34rd .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-fEJzgwgf3Hns34rd .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-fEJzgwgf3Hns34rd .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-fEJzgwgf3Hns34rd .marker{fill:#333333;stroke:#333333;}#mermaid-svg-fEJzgwgf3Hns34rd .marker.cross{stroke:#333333;}#mermaid-svg-fEJzgwgf3Hns34rd svg{font-family:\"trebuchet ms\",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-fEJzgwgf3Hns34rd .label{font-family:\"trebuchet ms\",verdana,arial,sans-serif;color:#333;}#mermaid-svg-fEJzgwgf3Hns34rd .cluster-label text{fill:#333;}#mermaid-svg-fEJzgwgf3Hns34rd .cluster-label span{color:#333;}#mermaid-svg-fEJzgwgf3Hns34rd .label text,#mermaid-svg-fEJzgwgf3Hns34rd span{fill:#333;color:#333;}#mermaid-svg-fEJzgwgf3Hns34rd .node rect,#mermaid-svg-fEJzgwgf3Hns34rd .node circle,#mermaid-svg-fEJzgwgf3Hns34rd .node ellipse,#mermaid-svg-fEJzgwgf3Hns34rd .node polygon,#mermaid-svg-fEJzgwgf3Hns34rd .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-fEJzgwgf3Hns34rd .node .label{text-align:center;}#mermaid-svg-fEJzgwgf3Hns34rd .node.clickable{cursor:pointer;}#mermaid-svg-fEJzgwgf3Hns34rd .arrowheadPath{fill:#333333;}#mermaid-svg-fEJzgwgf3Hns34rd .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-fEJzgwgf3Hns34rd .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-fEJzgwgf3Hns34rd .edgeLabel{background-color:#e8e8e8;text-align:center;}#mermaid-svg-fEJzgwgf3Hns34rd .edgeLabel rect{opacity:0.5;background-color:#e8e8e8;fill:#e8e8e8;}#mermaid-svg-fEJzgwgf3Hns34rd .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-fEJzgwgf3Hns34rd .cluster text{fill:#333;}#mermaid-svg-fEJzgwgf3Hns34rd .cluster span{color:#333;}#mermaid-svg-fEJzgwgf3Hns34rd div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\"trebuchet ms\",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-fEJzgwgf3Hns34rd :root{--mermaid-font-family:\"trebuchet ms\",verdana,arial,sans-serif;}#mermaid-svg-fEJzgwgf3Hns34rd .Cluster>*{fill:#ffffc0!important;color:#ff00ff!important;stroke-width:4px!important;}#mermaid-svg-fEJzgwgf3Hns34rd .Cluster span{fill:#ffffc0!important;color:#ff00ff!important;stroke-width:4px!important;}#mermaid-svg-fEJzgwgf3Hns34rd .Node>*{fill:#ccffbb!important;color:#000000!important;stroke-width:3px!important;}#mermaid-svg-fEJzgwgf3Hns34rd .Node span{fill:#ccffbb!important;color:#000000!important;stroke-width:3px!important;}#mermaid-svg-fEJzgwgf3Hns34rd .IMG>*{fill:#40aff0!important;color:#f0ff00!important;stroke:#f06080!important;stroke-width:3px!important;}#mermaid-svg-fEJzgwgf3Hns34rd .IMG span{fill:#40aff0!important;color:#f0ff00!important;stroke:#f06080!important;stroke-width:3px!important;}#mermaid-svg-fEJzgwgf3Hns34rd .User>*{fill:#ccddee!important;color:#000000!important;stroke:#555555!important;stroke-width:3px!important;}#mermaid-svg-fEJzgwgf3Hns34rd .User span{fill:#ccddee!important;color:#000000!important;stroke:#555555!important;stroke-width:3px!important;} k8s cluster node
计算节点 node
计算节点 node
计算节点 控制节点
Control Plane 私有镜像仓库 Client

kubernetes 安装

仓库初始化

1、创建云主机

按照如下配置准备云主机

主机名 IP地址 最低配置 master 192.168.1.50 2CPU,4G内存 node-0001 192.168.1.51 2CPU,4G内存 node-0002 192.168.1.52 2CPU,4G内存 node-0003 192.168.1.53 2CPU,4G内存 registry 192.168.1.30 2CPU,4G内存

2、初始化私有仓库

[root@registry ~]# vim /etc/hosts192.168.1.30registry[root@registry ~]# yum install -y docker-distribution[root@registry ~]# systemctl enable --now docker-distribution[root@registry ~]# curl -s http://registry:5000/v2/_catalog{\"repositories\":[]}

kube-master安装

1、防火墙相关配置

参考前面知识点完成禁用 selinux，禁用 swap，卸载 firewalld-*

2、配置yum仓库(跳板机)

[root@ecs-proxy ~]# rsync -av docker/ /var/ftp/localrepo/docker/[root@ecs-proxy ~]# rsync -av kubernetes/packages/ /var/ftp/localrepo/packages/[root@ecs-proxy ~]# createrepo --update /var/ftp/localrepo/

3、安装软件包(master)

安装kubeadm、kubectl、kubelet、docker-ce

[root@master ~]# yum makecache[root@master ~]# yum install -y kubeadm kubelet kubectl docker-ce[root@master ~]# mkdir -p /etc/docker[root@master ~]# vim /etc/docker/daemon.json { \"exec-opts\":[\"native.cgroupdriver=systemd\"], \"registry-mirrors\":[\"http://registry:5000\"], \"insecure-registries\":[\"registry:5000\",\"192.168.1.30:5000\"]}[root@master ~]# vim /etc/hosts192.168.1.30registry192.168.1.50master192.168.1.51node-0001192.168.1.52node-0002192.168.1.53node-0003[root@master ~]# systemctl enable --now docker kubelet[root@master ~]# docker info |grep Cgroup Cgroup Driver: systemd Cgroup Version: 1

4、镜像导入私有仓库

点击下载v1.22.5.tar.xz镜像

# 拷贝 第五阶段 kubernetes/v1.22.5.tar.xz 镜像到 master[root@ecs-proxy ~]# rsync -av kubernetes/v1.22.5.tar.xz 192.168.1.50:./init/

[root@master ~]# docker load -i init/v1.22.5.tar.xz[root@master ~]# docker images|while read i t _;do [[ \"${t}\" == \"TAG\" ]] && continue docker tag ${i}:${t} registry:5000/k8s/${i##*/}:${t} docker push registry:5000/k8s/${i##*/}:${t} docker rmi ${i}:${t} registry:5000/k8s/${i##*/}:${t}done# 查看验证[root@master ~]# curl -s http://registry:5000/v2/_catalog|python -m json.tool{ \"repositories\": [ \"k8s/coredns\", \"k8s/etcd\", \"k8s/kube-apiserver\", \"k8s/kube-controller-manager\", \"k8s/kube-proxy\", \"k8s/kube-scheduler\", \"k8s/pause\" ]}

5、Tab键设置

[root@master ~]# source <(kubeadm completion bash|tee /etc/bash_completion.d/kubeadm)[root@master ~]# source <(kubectl completion bash|tee /etc/bash_completion.d/kubectl)

6、安装代理软件包

[root@master ~]# yum install -y ipvsadm ipset

7、配置内核参数

[root@master ~]# for i in overlay br_netfilter;do modprobe ${i} echo \"${i}\" >>/etc/modules-load.d/containerd.confdone[root@master ~]# cat >/etc/sysctl.d/99-kubernetes-cri.conf<<EOFnet.ipv4.ip_forward = 1net.bridge.bridge-nf-call-iptables = 1net.bridge.bridge-nf-call-ip6tables = 1EOF[root@master ~]# sysctl --system

8、使用kubeadm部署

拷贝应答文件 kubernetes/config/kubeadm-init.yaml 到 master 主机

# 拷贝 kubeadm-init.yaml 到 master 云主机 init 目录下[root@ecs-proxy ~]# rsync -av kubernetes/config/kubeadm-init.yaml 192.168.1.50:./init/#---------------------------------------------------------------------[root@master ~]# kubeadm init --config=init/kubeadm-init.yaml --dry-run # 输出信息若干，没有 Error 和 Warning 就是正常[root@master ~]# rm -rf /etc/kubernetes/tmp[root@master ~]# kubeadm init --config=init/kubeadm-init.yaml |tee init/init.log# 根据提示执行命令[root@master ~]# mkdir -p $HOME/.kube[root@master ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config[root@master ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

9、验证安装结果

[root@master init]# kubectl cluster-info Kubernetes control plane is running at https://192.168.1.50:6443CoreDNS is running at https://192.168.1.50:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxyTo further debug and diagnose cluster problems, use \'kubectl cluster-info dump\'.[root@master init]# kubectl get nodesNAME STATUS ROLES  AGE VERSIONmaster NotReady control-plane,master 90s v1.22.5

计算节点安装

1、获取token

# 查看 token[root@master ~]# kubeadm token listTOKEN  TTL EXPIRES abcdef.0123456789abcdef 23h 2022-04-12T14:04:34Z# 删除 token[root@master ~]# kubeadm token delete abcdef.0123456789abcdefbootstrap token \"abcdef\" deleted# 创建 token[root@master ~]# kubeadm token create --ttl=0 --print-join-commandkubeadm join 192.168.1.50:6443 --token fhf6gk.bhhvsofvd672yd41 --discovery-token-ca-cert-hash sha256:ea07de5929dab8701c1bddc347155fe51c3fb6efd2ce8a4177f6dc03d5793467# 获取token_hash# 1、查看安装日志 2、在创建token时候显示 3、使用 openssl 计算得到[root@master ~]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt |openssl rsa -pubin -outform der |openssl dgst -sha256 -hex

2、node安装

拷贝 kubernetes/nodejoin 到跳板机

[root@ecs-proxy 5]# cp -a kubernetes/nodejoin /root/[root@ecs-proxy 5]# cd ~root/nodejoin/[root@ecs-proxy nodejoin]# vim nodeinit.yaml... ... vars: master: \'192.168.1.50:6443\' token: \'这里改成你自己的token\' token_hash: \'sha256:这里改成你自己的token ca hash\'... ...[root@ecs-proxy node-install]# ansible-playbook nodeinit.yaml

3、验证安装

[root@master ~]# kubectl get nodesNAME STATUS ROLES AGE VERSIONmaster NotReady master 130m v1.22.5node-0001 NotReady <none> 2m14s v1.22.5node-0002 NotReady <none> 2m15s v1.22.5node-0003 NotReady <none> 2m9s v1.22.5

网络插件安装配置

1、上传镜像到私有仓库

# 拷贝 kubernetes/plugins 目录到 master 云主机上[root@ecs-proxy ~]# rsync -av kubernetes/plugins 192.168.1.50:./#---------------------------------------------------------------------[root@master ~]# cd plugins/flannel[root@master flannel]# docker load -i flannel.tar.xz[root@master flannel]# docker images|while read i t _;do [[ \"${t}\" == \"TAG\" ]] && continue [[ \"${i}\" =~ ^\"registry:5000/\".+ ]] && continue docker tag ${i}:${t} registry:5000/plugins/${i##*/}:${t} docker push registry:5000/plugins/${i##*/}:${t} docker rmi ${i}:${t} registry:5000/plugins/${i##*/}:${t}done

2、修改配置文件并安装

[root@master flannel]# sed -ri \'s,^(\\s+image: ).+/(.+),\\1registry:5000/plugins/\\2,\' kube-flannel.yml128: \"Network\": \"10.244.0.0/16\",169: image: registry:5000/plugins/mirrored-flannelcni-flannel-cni-plugin:v1.0.0180: image: registry:5000/plugins/mirrored-flannelcni-flannel:v0.16.1194: image: registry:5000/plugins/mirrored-flannelcni-flannel:v0.16.1[root@master flannel]# kubectl apply -f kube-flannel.yml

3、验证结果

# 验证节点工作状态[root@master flannel]# kubectl get nodesNAME  STATUS ROLES AGE VERSIONmaster Ready master 26h v1.22.5node-0001 Ready <none> 151m v1.22.5node-0002 Ready <none> 152m v1.22.5node-0003 Ready <none> 153m v1.22.5# 验证容器工作状态[root@master ~]# kubectl -n kube-system get podsNAME READY STATUS RESTARTS AGEcoredns-54b6487f4d-t7f9m 1/1 Running 0 64mcoredns-54b6487f4d-v2zbg 1/1 Running 0 64metcd-master1/1 Running 0 64mkube-apiserver-master 1/1 Running 0 64mkube-controller-manager-master 1/1 Running 0 64mkube-flannel-ds-8x4hq 1/1 Running 0 55mkube-flannel-ds-c5rkv 1/1 Running 0 55mkube-flannel-ds-sk2gj 1/1 Running 0 55mkube-flannel-ds-v26sx 1/1 Running 0 55mkube-proxy-6gprw  1/1 Running 0 58mkube-proxy-6tfn8  1/1 Running 0 58mkube-proxy-9t5ln  1/1 Running 0 58mkube-proxy-f956k  1/1 Running 0 64mkube-scheduler-master 1/1 Running 0 64m