您现在的位置是:首页 >其他 >k8s 集群搭建详细教程网站首页其他
k8s 集群搭建详细教程
参考: Kubernetes 文档 / 入门 / 生产环境 / 使用部署工具安装 Kubernetes / 使用 kubeadm 引导集群 / 安装 kubeadm
B. 准备开始
- 一台兼容的 Linux 主机。Kubernetes 项目为基于 Debian 和 Red Hat 的 Linux 发行版以及一些不提供包管理器的发行版提供通用的指令
- 每台机器
2 GB
或更多的 RAM (如果少于这个数字将会影响你应用的运行内存) 2 CPU 核
或更多- 集群中的所有机器的网络彼此均能相互连接(公网和内网都可以)
- 节点之中
不可以有重复的
主机名、MAC 地址或 product_uuid。请参见这里了解更多详细信息。 - 开启机器上的某些端口。请参见这里 了解更多详细信息。
- 禁用交换分区。为了保证 kubelet 正常工作,你 必须
禁用交换分区
。
U. 确保每个节点上 MAC 地址和 product_uuid 的唯一性
- 你可以使用命令
ip link
或ifconfig -a
来获取网络接口的 MAC 地址 - 可以使用
sudo cat /sys/class/dmi/id/product_uuid
命令对 product_uuid 校验
一般来讲,硬件设备会拥有唯一的地址,但是有些虚拟机的地址可能会重复。 Kubernetes 使用这些值来唯一确定集群中的节点。 如果这些值在每个节点上不唯一,可能会导致安装 失败。
V. 虚拟机
新建… / 创建自定虚拟机 /
Linux /Ubuntu 64位
- 设置过程
ID | 『虚拟机』设置 | 建议配置 | 默认值 | 说明 |
---|---|---|---|---|
1 | 处理器 | - | 2 | 最低要求 |
2 | 内存 | - | 4096 MB | 节约内存 |
3 | 显示器 | 取消复选加速 3D 图形 | 复选 | 节约内存 |
4 | 网络适配器 | - | nat | 需上网 |
5 | 硬盘 | 40 GB | 20 GB | 保证练习容量 |
6 | 选择固件类型 | UEFI | 传统 BIOS | VMware Fusion 支持嵌套虚拟化 |
- 设置结果
ID | Your computer’s name | CPU 核 | RAM | DISK | NIC |
---|---|---|---|---|---|
1 | k8s-master | 4 或更多 | 8 GB或更多 | 40 GB | nat |
2 | k8s-worker1 | 同上 | 2 GB或更多 | 同上 | 同上 |
3 | k8s-worker2 | 同上 | 同上 | 同上 | 同上 |
I. 安装 Ubuntu 22.04 LTS
-
Willkommen! Bienvenue! Welcome! Welkom!
[
English
]
-
Installer update available
[
Continue without updating
]
-
Keyboard configuration
[
Done
]
-
Choose type of install
(
X
) Ubuntu Server (minimized)
/ [Done
]
-
Network connections
[
Done
]
-
Configure proxy
[
Done
]
-
Configure Ubuntu archive mirror
Mirror address: http://mirror.nju.edu.cn/ubuntu
/ [Done
]
-
Guided storage configuration
[
Done
] -
Storage configuration
[
Done
]
-
Profile setup
Your name:
kiosk
Your server 's name:k8s-master
Pick a username:kiosk
Choose a password:ubuntu
Confirm your password:ubuntu
/ [Done
] -
SSH Setup
[
X
] Install OpenSSH server
/ [Done
]
-
Featured Server Snaps
[
Done
]
-
Install complete!
🅰️ [
Cancel update and reboot
]🅱️ [
Reboot Now
] -
建议(可选)
关机后,做个快照
P. 准备工作
[kiosk@k8s-master|k8s-worker1|k8s-worker2]$
-
设置当前用户sudo免密
sudo tee /etc/sudoers.d/$USER >/dev/null <<EOF $USER ALL=(ALL) NOPASSWD:ALL EOF
-
使用国内镜像仓库
# 国内镜像仓库 MIRROR_URL=http://mirror.nju.edu.cn/ubuntu # 生成软件仓库源 sudo tee /etc/apt/sources.list >/dev/null <<EOF deb $MIRROR_URL jammy main restricted universe multiverse deb $MIRROR_URL jammy-updates main restricted universe multiverse deb $MIRROR_URL jammy-backports main restricted universe multiverse deb $MIRROR_URL jammy-security main restricted universe multiverse EOF
-
安装相关软件
# 更新 sudo apt -y update # 安装 sudo apt install -y openssh-server vim sshpass nfs-common bash-completion netcat-openbsd open-vm-tools
[kiosk@k8s-master]$
-
设置静态IP
# 配置IP sudo tee /etc/netplan/00-installer-config.yaml >/dev/null <<EOF network: ethernets: ens33: dhcp4: false addresses: [192.168.147.128/24] gateway4: 192.168.147.2 nameservers: addresses: [8.8.8.8] version: 2 EOF # dns sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf # active sudo netplan apply
[kiosk@k8s-worker1]$
-
设置静态IP
# 配置IP sudo tee /etc/netplan/00-installer-config.yaml >/dev/null <<EOF network: ethernets: ens33: dhcp4: false addresses: [192.168.147.129/24] gateway4: 192.168.147.2 nameservers: addresses: [8.8.8.8] version: 2 EOF # dns sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf # active sudo netplan apply
[kiosk@k8s-worker2]$
-
设置静态IP
# 配置IP sudo tee /etc/netplan/00-installer-config.yaml >/dev/null <<EOF network: ethernets: ens33: dhcp4: false addresses: [192.168.147.130/24] gateway4: 192.168.147.2 nameservers: addresses: [8.8.8.8] version: 2 EOF # dns sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf # active sudo netplan apply
[kiosk@k8s-master|k8s-worker1|k8s-worker2]$
-
编辑 hosts
sudo tee -a /etc/hosts >/dev/null <<EOF 192.168.147.128 k8s-master 192.168.147.129 k8s-worker1 192.168.147.130 k8s-worker2 EOF # 设置 root 密码 (echo ubuntu; echo ubuntu) | sudo passwd root echo PermitRootLogin yes | sudo tee -a /etc/ssh/sshd_config sudo systemctl restart sshd
[kiosk@k8s-master]$
-
ssh免密
# 生成 keypair ssh-keygen -f ~/.ssh/id_rsa -N '' # 拷贝公钥 for i in k8s-master k8s-worker1 k8s-worker2; do sshpass -pubuntu ssh-copy-id -o StrictHostKeyChecking=no kiosk@$i sshpass -pubuntu ssh-copy-id -o StrictHostKeyChecking=no root@$i done
[kiosk@k8s-master|k8s-worker1|k8s-worker2]$
-
禁用swap
# 交换文件 SWAPF=$(awk '/swap/ {print $1}' /etc/fstab) # 立即禁用 sudo swapoff $SWAPF # 永久禁用 sudo sed -i '/swap/d' /etc/fstab # 删除交换文件 sudo rm $SWAPF
-
扩容
# 逻辑卷名 export LVN=$(sudo lvdisplay | awk '/Path/ {print $3}') # 扩容 sudo lvextend -l 100%PVS $LVN # 立即生效 sudo resize2fs $LVN # 验证 df -h /
-
模块支持
# 安装 sudo apt -y install bridge-utils # 立即生效 sudo modprobe br_netfilter # 内核支持 sudo tee /etc/sysctl.d/k8s.conf >/dev/null <<EOF net.ipv4.ip_forward=1 vm.swappiness=0 vm.overcommit_memory=1 vm.panic_on_oom=0 EOF # 立即生效 sudo sysctl -p /etc/sysctl.d/k8s.conf
- docker: k8s-master, k8s-worker1
- containerd: k8s-worker2
[kiosk@k8s-master|k8s-worker1]$
-
安装运行时
# 创建文件夹 sudo mkdir -p /etc/docker # 生成配置文件 sudo tee /etc/docker/daemon.json >/dev/null <<EOF { "exec-opts": ["native.cgroupdriver=systemd"], "log-driver": "json-file", "log-opts": { "max-size": "100m", "max-file": "10" }, "registry-mirrors": ["https://docker.nju.edu.cn/"] } EOF # 安装 runtime sudo apt -y install docker.io # 开机自启 sudo systemctl enable docker # 立即重启 sudo systemctl restart docker
# 安装命令 cri-dockerd curl -# https://vmcc.xyz:8443/k8s/cri-docker/cri-dockerd-0.2.5.amd64.tgz -o cri-dockerd-0.2.5.amd64.tgz tar -xf cri-dockerd-0.2.5.amd64.tgz sudo cp cri-dockerd/cri-dockerd /usr/bin/ # 安装服务 cri-docker.service sudo curl -s https://vmcc.xyz:8443/k8s/cri-docker/cri-docker.service -o /usr/lib/systemd/system/cri-docker.service sudo sed -i '/ExecStart/s+$+ --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.8+' /usr/lib/systemd/system/cri-docker.service # 安装 cri-docker.socket sudo curl -s https://vmcc.xyz:8443/k8s/cri-docker/cri-docker.socket -o /usr/lib/systemd/system/cri-docker.socket # 启动服务 cri-dockerd sudo systemctl daemon-reload sudo systemctl enable cri-docker sudo systemctl restart cri-docker # 安装 crictl 命令 curl -# https://vmcc.xyz:8443/k8s/crictl-v1.24.2-linux-amd64.tar.gz -o crictl-v1.24.2-linux-amd64.tar.gz tar -xf crictl-v1.24.2-linux-amd64.tar.gz sudo cp crictl /usr/bin/ # crictl 配置文件 sudo tee /etc/crictl.yaml >/dev/null <<EOF runtime-endpoint: unix:///var/run/cri-dockerd.sock image-endpoint: unix:///var/run/cri-dockerd.sock timeout: 10 debug: false pull-image-on-create: true EOF
[kiosk@k8s-worker2]$
-
安装运行时
# 安装 containerd sudo apt install -y containerd # 创建目录 sudo mkdir /etc/containerd # 生成默认配置文件 containerd config default | sudo tee /etc/containerd/config.toml >/dev/null # 修改配置文件 sudo sed -i -e '/sandbox_image/s?k8s.gcr.io?registry.aliyuncs.com/google_containers?' -e '/SystemdCgroup/s?false?true?' -e '/registry.mirrors/a [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]' -e '/registry.mirrors/a endpoint = ["https://docker.nju.edu.cn/"]' /etc/containerd/config.toml # 服务重启 sudo systemctl restart containerd # 安装 crictl 命令 curl -# https://vmcc.xyz:8443/k8s/crictl-v1.24.2-linux-amd64.tar.gz -o crictl-v1.24.2-linux-amd64.tar.gz tar -xf crictl-v1.24.2-linux-amd64.tar.gz sudo cp crictl /usr/bin/ # crictl 配置文件 sudo tee /etc/crictl.yaml >/dev/null <<EOF runtime-endpoint: unix:///run/containerd/containerd.sock image-endpoint: unix:///run/containerd/containerd.sock timeout: 10 debug: false pull-image-on-create: true EOF
K. 安装 k8s
[kiosk@k8s-master|k8s-worker1|k8s-worker2]$
-
安装 kubeadm、kubelet 和 kubectl
# 更新 apt 包索引并安装使用 Kubernetes apt 仓库所需要的包 sudo apt -y install apt-transport-https ca-certificates curl # 下载 Google Cloud 公开签名秘钥 curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add - # 添加 Kubernetes apt 仓库 MIRROR_URL=https://mirror.nju.edu.cn/kubernetes/apt/ sudo tee /etc/apt/sources.list.d/kubernetes.list >/dev/null <<EOF deb $MIRROR_URL kubernetes-xenial main EOF # 更新 apt 包索引 sudo cp /etc/apt/trusted.gpg /etc/apt/trusted.gpg.d sudo apt update -y sudo apt-cache madison kubelet | grep 1.24 # 安装 kubelet、kubeadm 和 kubectl 考试版本 sudo apt install -y kubelet=1.24.1-00 kubeadm=1.24.1-00 kubectl=1.24.1-00 # 锁定版本 sudo apt-mark hold kubelet kubeadm kubectl
[kiosk@k8s-worker2]$
# 增加 k8s 支持
sudo sed -i '/ExecStart=//s|$| --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --cgroup-driver=systemd|' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
# 重启 kubelet 服务
sudo systemctl daemon-reload
sudo systemctl restart kubelet
[kiosk@k8s-master]$
-
初始化
# 生成初始文件 sudo kubeadm config print init-defaults > kubeadm-config.yaml # 修改文件 sudo sed -i -e "/advertiseAddress/s?:.*?: 192.168.147.128?" -e "/name/s?:.*?: k8s-master?" -e "/clusterName/s?:.*?: ck8s?" -e "/imageRepository/s?:.*?: registry.aliyuncs.com/google_containers?" -e "/criSocket/s+containerd/containerd+cri-dockerd+" kubeadm-config.yaml # 使用初始文件,初始化集群 sudo kubeadm init --config kubeadm-config.yaml
…输出省略…
Your Kubernetes control-plane has initializedsuccessfully
!To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run “kubectl apply -f [podnetwork].yaml
” with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.147.128:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:c4781194de65ebb47984fc5e7e64d4897875410825ce4d18df81da1a298afa1f
-
配置文件
# 创建目录 mkdir -p $HOME/.kube # user 复制配置文件 sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config # root 变量 sudo tee -a ~root/.bashrc >/dev/null <<EOF export KUBECONFIG=/etc/kubernetes/admin.conf EOF
-
创建网络
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
-
命令补全
# 立即生效 source <(kubectl completion bash) # 永久生效 mkdir ~/.kube kubectl completion bash > ~/.kube/completion.bash.inc printf " # Kubectl shell completion source '$HOME/.kube/completion.bash.inc' " >> $HOME/.bash_profile source $HOME/.bash_profile
-
命令别名
# 永久生效 tee -a $HOME/.bashrc >/dev/null <<EOF alias k='kubectl' complete -F __start_kubectl k EOF # 立即生效 source $HOME/.bashrc
[kiosk@k8s-worker1]$
-
加入集群
sudo kubeadm join 192.168.147.128:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:c4781194de65ebb47984fc5e7e64d4897875410825ce4d18df81da1a298afa1f --cri-socket unix://var/run/cri-dockerd.sock
[kiosk@k8s-worker2]$
-
加入集群
sudo kubeadm join 192.168.147.128:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:c4781194de65ebb47984fc5e7e64d4897875410825ce4d18df81da1a298afa1f
C. 确认环境正常
[kiosk@k8s-master]
$ kubectl get componentstatuses
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME STATUS MESSAGE ERROR
scheduler `Healthy` ok
controller-manager `Healthy` ok
etcd-0 `Healthy` {"health":"true","reason":""}
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-worker1 `Ready` <none> 4m4s `v1.24.1`
k8s-worker2 `Ready` <none> 4m44s `v1.24.1`
k8s-master `Ready` control-plane,master 13m `v1.24.1`
$ kubectl -n kube-system get pod -w
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-555bc4b957-8ccgh 1/1 Running 0 27m
calico-node-5qqcq 1/1 Running 0 9m29s
calico-node-7qclz 1/1 Running 0 27m
calico-node-kcvt5 1/1 Running 0 9m29s
coredns-74586cf9b6-69fn7 1/1 Running 0 156m
coredns-74586cf9b6-8mgl9 1/1 Running 0 156m
etcd-k8s-master 1/1 Running 0 156m
kube-apiserver-k8s-master 1/1 Running 0 156m
kube-controller-manager-k8s-master 1/1 Running 0 156m
kube-proxy-8j248 1/1 Running 0 9m29s
kube-proxy-g7r55 1/1 Running 0 9m29s
kube-proxy-rbdcp 1/1 Running 0 156m
kube-scheduler-k8s-master 1/1 Running 0 156m
<Ctrl-C>