阿里云服务器

三台阿里云ECS：

• 4核8G（k8s-master)，随机分配的私有IP 172.24.155.189
• 8核16G（k8s-node01)，随机分配的私有IP 172.24.155.190
• 8核16G（k8s-node02), 随机分配的私有IP 172.24.155.191

付费模式：按量付费
按量付费带宽可以拉满，进入ECS的流量才收费
操作系统：CentOS7.9
三台服务器都需要公网IP
三台服务器需要在同一个VPC下
三台服务器需要在同一个安全组下（分开创建3台服务器将位于不同安全组）

注意

1. 三台服务器需要在同一个安全组下（分开创建3台服务器将位于不同安全组）。ping不同可能不在同一个安全组下
2. 实验完毕请“停机不收费”，只收取磁盘费用
3. k8s结点之间通信请使用私有IP
4. dashboard使用https协议，火狐浏览器才能进一步访问
5. 云主机“停机不收费”再启动后，需要在k8s-master执行export KUBECONFIG=/etc/kubernetes/admin.conf

k8s集群安装

【公共操作】安装docker

所有服务器

# 准备工作
yum remove docker*
yum install -y yum-utils

# 配置docker yum地址
yum-config-manager 
--add-repo 
http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

# 安装指定版本，docker-ce-1.20.9 将产生报警信息 Docker版本Update the latest validated version of Docker to 19.03，可以忽略
# yum install -y docker-ce-1.20.9 docker-ce-cli-1.20.9 containerd.io-1.4.6
yum install -y docker-ce-19.03.05 docker-ce-cli-19.03.05 containerd.io-1.4.6

# 设置开机启动，并立即启动
systemctl enable docker --now

# docker加速配置
mkdir -p /etc/docker
tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://ung2thfc.mirror.aliyuncs.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF
systemctl daemon-reload
systemctl restart docker

【公共操作】配置服务器

所有服务器

# 【私有操作】设置每台服务器各自的主机名
## k8s-master
hostnamectl set-hostname k8s-master
## k8s-node01
hostnamectl set-hostname k8s-node01
## k8s-node02
hostnamectl set-hostname k8s-node02

# 禁用selinux
sudo setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

# 关闭swap, 影响k8s速度
swapoff -a  
sed -ri 's/.*swap.*/#&/' /etc/fstab

# 设置iptables，允许检查桥接流量
cat <<EOF | tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system

【公共操作】预拉取镜像

tee ./images.sh <<-'EOF'
#!/bin/bash
images=(
kube-apiserver:v1.20.9
kube-proxy:v1.20.9
kube-controller-manager:v1.20.9
kube-scheduler:v1.20.9
coredns:1.7.0
etcd:3.4.13-0
pause:3.2
)
for imageName in ${images[@]} ; do
docker pull registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/$imageName
done
EOF

chmod +x ./images.sh && ./images.sh

【公共操作】安装kubelet, kubectl, kubeadm

# 配置k8s yum源地址
cat <<EOF | tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
   http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

# 安装 kubelet，kubectl, kubeadm
yum install -y kubelet-1.20.9 kubectl-1.20.9 kubeadm-1.20.9

# 设置kubectl开机启动，并立即启动
systemctl enable --now kubelet

# 【私有操作】所有机器配置master域名
## k8s-master
echo "172.24.155.189  k8s-master" >> /etc/hosts
## k8s-node01
echo "172.24.155.189  k8s-master
172.24.155.190  k8s-node01" >> /etc/hosts
## k8s-node02
echo "172.24.155.189  k8s-master
172.24.155.191  k8s-node01" >> /etc/hosts

【master私有操作】k8s-master初始化

# apiserver-advertise-address是你的k8s-master结点的私有IP
# control-plane-endpoint是你的k8s-master结点的hostname
# pod-network-cidr pod的虚拟IP，全k8s集群唯一
# service-cidr svc的虚拟IP，全k8s集群唯一
kubeadm init 
--apiserver-advertise-address=172.24.155.189 
--control-plane-endpoint=k8s-master 
--image-repository registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images 
--kubernetes-version v1.20.9 
--service-cidr=10.96.0.0/16 
--pod-network-cidr=192.168.0.0/16

记录安装结果

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:
	!!! just master
  export KUBECONFIG=/etc/kubernetes/admin.conf

 !!! just master
 curl https://docs.projectcalico.org/manifests/calico.yaml -O

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:

  kubeadm join k8s-master:6443 --token 56btfl.sacevww9mh3dypag 
    --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxx61b8b11afad06f31e31a1dc19a704ba3ae0 
    --control-plane

Then you can join any number of worker nodes by running the following on each as root:
!!! just node
kubeadm join k8s-master:6443 --token 56btfl.sacevww9mh3dypag 
    --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxfad06f31e31a1dc19a704ba3ae0

根据上面的内容在k8s-master执行

# root用户执行
export KUBECONFIG=/etc/kubernetes/admin.conf

【master私有操作】k8s-master扁平化网络

# Calico网络插件，apply安装失败可以使用提供的yaml文件
curl https://docs.projectcalico.org/manifests/calico.yaml -O
kubectl apply -f calico.yaml

本文不执行【master私有操作】control-plane nodes (多master)

kubeadm join k8s-master:6443 --token 56btfl.sacevww9mh3dypag 
    --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxx61b8b11afad06f31e31a1dc19a704ba3ae0 
    --control-plane

【node私有操作】k8s-nodexxx加入k8s-master

kubeadm join k8s-master:6443 --token 56btfl.sacevww9mh3dypag 
    --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxfad06f31e31a1dc19a704ba3ae0

【master私有操作】k8s-master检验集群结点状态

# ready 状态即可
kubectl get nodes

安装dashboard

【master私有操作】

# 可以下载之后在apply
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.3.1/aio/deploy/recommended.yaml

# svc 修改 type: ClusterIP 为 type: NodePort。修改之后自动立即生效
kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard
# nodeport端口，在阿里云安全组 入方向放行
kubectl get svc -A |grep kubernetes-dashboard

# 火狐访问 https://k8s集群任意公网IP:端口
# 现在还不能使用token进入，需要创建serviceAccount
# 创建ServiceAccount
vim dashboard_ServiceAccount.yaml
kubectl apply -f dash.yaml
# 获取访问令牌
kubectl -n kubernetes-dashboard get secret $(kubectl -n kubernetes-dashboard get sa/admin-user -o jsonpath="{.secrets[0].name}") -o go-template="{{.data.token | base64decode}}"
# 火狐界面输入token即可进入dashboard

dashboard_ServiceAccount.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

参考

• 参考网址