您现在的位置是:首页 >其他 >k8s 集群搭建详细教程网站首页其他

k8s 集群搭建详细教程

焱宣 2023-06-26 20:00:07
简介k8s 集群搭建详细教程

参考: Kubernetes 文档 / 入门 / 生产环境 / 使用部署工具安装 Kubernetes / 使用 kubeadm 引导集群 / 安装 kubeadm


B. 准备开始

  • 一台兼容的 Linux 主机。Kubernetes 项目为基于 Debian 和 Red Hat 的 Linux 发行版以及一些不提供包管理器的发行版提供通用的指令
  • 每台机器 2 GB 或更多的 RAM (如果少于这个数字将会影响你应用的运行内存)
  • 2 CPU 核或更多
  • 集群中的所有机器的网络彼此均能相互连接(公网和内网都可以)
  • 节点之中不可以有重复的主机名、MAC 地址或 product_uuid。请参见这里了解更多详细信息。
  • 开启机器上的某些端口。请参见这里 了解更多详细信息。
  • 禁用交换分区。为了保证 kubelet 正常工作,你 必须 禁用交换分区

U. 确保每个节点上 MAC 地址和 product_uuid 的唯一性

  • 你可以使用命令 ip linkifconfig -a 来获取网络接口的 MAC 地址
  • 可以使用 sudo cat /sys/class/dmi/id/product_uuid 命令对 product_uuid 校验

一般来讲,硬件设备会拥有唯一的地址,但是有些虚拟机的地址可能会重复。 Kubernetes 使用这些值来唯一确定集群中的节点。 如果这些值在每个节点上不唯一,可能会导致安装 失败


V. 虚拟机

新建… / 创建自定虚拟机 /
Linux / Ubuntu 64位

  • 设置过程
ID『虚拟机』设置建议配置默认值说明
1处理器-2最低要求
2内存-4096 MB节约内存
3显示器取消复选加速 3D 图形复选节约内存
4网络适配器-nat需上网
5硬盘40GB20 GB保证练习容量
6选择固件类型UEFI传统 BIOSVMware Fusion 支持嵌套虚拟化
  • 设置结果
IDYour computer’s nameCPU 核RAMDISKNIC
1k8s-master4 或更多8 GB或更多40 GBnat
2k8s-worker1同上2 GB或更多同上同上
3k8s-worker2同上同上同上同上

I. 安装 Ubuntu 22.04 LTS

  1. Willkommen! Bienvenue! Welcome! Welkom!

        [ English ]

  2. Installer update available

     [ Continue without updating ]

  3. Keyboard configuration

    [ Done ]

  4. Choose type of install

    (X) Ubuntu Server (minimized)
    / [ Done ]

  5. Network connections

    [ Done ]

  6. Configure proxy

    [ Done ]

  7. Configure Ubuntu archive mirror

    Mirror address: http://mirror.nju.edu.cn/ubuntu
    / [ Done ]

  8. Guided storage configuration

    [ Done ]

  9. Storage configuration

    [ Done ]

  10. Profile setup

                             Your name: kiosk
             Your server 's name: k8s-master
                   Pick a username: kiosk
              Choose a password: ubuntu
        Confirm your password: ubuntu
        / [ Done ]

  11. SSH Setup

    ​ [X] Install OpenSSH server
    ​ / [ Done ]

  12. Featured Server Snaps

    [ Done ]

  13. Install complete!

    🅰️ [ Cancel update and reboot ]

    🅱️ [ Reboot Now ]

  14. 建议(可选)

    关机后,做个快照


P. 准备工作

[kiosk@k8s-master|k8s-worker1|k8s-worker2]$

  1. 设置当前用户sudo免密

    sudo tee /etc/sudoers.d/$USER >/dev/null <<EOF
    $USER ALL=(ALL) NOPASSWD:ALL
    EOF
    
  2. 使用国内镜像仓库

    # 国内镜像仓库
    MIRROR_URL=http://mirror.nju.edu.cn/ubuntu
    
    # 生成软件仓库源
    sudo tee /etc/apt/sources.list >/dev/null <<EOF
    deb $MIRROR_URL jammy main restricted universe multiverse
    deb $MIRROR_URL jammy-updates main restricted universe  multiverse
    deb $MIRROR_URL jammy-backports main restricted universe multiverse
    deb $MIRROR_URL jammy-security main restricted universe multiverse
    EOF
    
  3. 安装相关软件

    # 更新
    sudo apt -y update
    
    # 安装
    sudo apt install -y openssh-server 
    	vim sshpass nfs-common 
    	bash-completion netcat-openbsd 
    	open-vm-tools
    

[kiosk@k8s-master]$

  1. 设置静态IP

    # 配置IP
    sudo tee /etc/netplan/00-installer-config.yaml >/dev/null <<EOF
    network:
      ethernets:
        ens33:
          dhcp4: false
          addresses: [192.168.147.128/24]
          gateway4: 192.168.147.2
          nameservers:
            addresses: [8.8.8.8]
      version: 2
    EOF
    
    # dns
    sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf
    
    # active
    sudo netplan apply
    

[kiosk@k8s-worker1]$

  1. 设置静态IP

    # 配置IP
    sudo tee /etc/netplan/00-installer-config.yaml >/dev/null <<EOF
    network:
      ethernets:
        ens33:
          dhcp4: false
          addresses: [192.168.147.129/24]
          gateway4: 192.168.147.2
          nameservers:
            addresses: [8.8.8.8]
      version: 2
    EOF
    
    # dns
    sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf
    
    # active
    sudo netplan apply
    

[kiosk@k8s-worker2]$

  1. 设置静态IP

    # 配置IP
    sudo tee /etc/netplan/00-installer-config.yaml >/dev/null <<EOF
    network:
      ethernets:
        ens33:
          dhcp4: false
          addresses: [192.168.147.130/24]
          gateway4: 192.168.147.2
          nameservers:
            addresses: [8.8.8.8]
      version: 2
    EOF
    
    # dns
    sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf
    
    # active
    sudo netplan apply
    

[kiosk@k8s-master|k8s-worker1|k8s-worker2]$

  1. 编辑 hosts

    sudo tee -a /etc/hosts >/dev/null <<EOF
    192.168.147.128	k8s-master
    192.168.147.129 k8s-worker1
    192.168.147.130 k8s-worker2
    EOF
    
    # 设置 root 密码
    (echo ubuntu; echo ubuntu) | sudo passwd root
    echo PermitRootLogin yes | sudo tee -a /etc/ssh/sshd_config
    sudo systemctl restart sshd
    

[kiosk@k8s-master]$

  1. ssh免密

    # 生成 keypair
    ssh-keygen -f ~/.ssh/id_rsa -N ''
    
    # 拷贝公钥
    for i in k8s-master k8s-worker1 k8s-worker2; do
       sshpass -pubuntu ssh-copy-id -o StrictHostKeyChecking=no kiosk@$i
       sshpass -pubuntu ssh-copy-id -o StrictHostKeyChecking=no root@$i
    done
    

[kiosk@k8s-master|k8s-worker1|k8s-worker2]$

  1. 禁用swap

    # 交换文件
    SWAPF=$(awk '/swap/ {print $1}' /etc/fstab)
    
    # 立即禁用
    sudo swapoff $SWAPF
    
    # 永久禁用
    sudo sed -i '/swap/d' /etc/fstab
    
    # 删除交换文件
    sudo rm $SWAPF
    
  2. 扩容

    # 逻辑卷名
    export LVN=$(sudo lvdisplay | awk '/Path/ {print $3}')
    
    # 扩容
    sudo lvextend -l 100%PVS $LVN
    
    # 立即生效
    sudo resize2fs $LVN
    
    # 验证
    df -h /
    
  3. 模块支持

    # 安装
    sudo apt -y install bridge-utils
    
    # 立即生效
    sudo modprobe br_netfilter
    
    # 内核支持
    sudo tee /etc/sysctl.d/k8s.conf >/dev/null <<EOF
    net.ipv4.ip_forward=1
    vm.swappiness=0
    vm.overcommit_memory=1
    vm.panic_on_oom=0
    EOF
    
    # 立即生效
    sudo sysctl -p /etc/sysctl.d/k8s.conf
    
  • docker: k8s-master, k8s-worker1
  • containerd: k8s-worker2

[kiosk@k8s-master|k8s-worker1]$

  1. 安装运行时

    # 创建文件夹
    sudo mkdir -p /etc/docker
    
    # 生成配置文件
    sudo tee /etc/docker/daemon.json >/dev/null <<EOF
    {
      "exec-opts": ["native.cgroupdriver=systemd"],
      "log-driver": "json-file",
      "log-opts": {
        "max-size": "100m",
        "max-file": "10"
      },
      "registry-mirrors": ["https://docker.nju.edu.cn/"]
    }
    EOF
    
    # 安装 runtime
    sudo apt -y install docker.io
    
    # 开机自启
    sudo systemctl enable docker
    
    # 立即重启
    sudo systemctl restart docker
    
    # 安装命令 cri-dockerd
    curl -# https://vmcc.xyz:8443/k8s/cri-docker/cri-dockerd-0.2.5.amd64.tgz 
    	-o cri-dockerd-0.2.5.amd64.tgz
    tar -xf cri-dockerd-0.2.5.amd64.tgz
    sudo cp cri-dockerd/cri-dockerd /usr/bin/
    
    # 安装服务 cri-docker.service
    sudo curl -s https://vmcc.xyz:8443/k8s/cri-docker/cri-docker.service 
    	-o /usr/lib/systemd/system/cri-docker.service
    sudo sed -i '/ExecStart/s+$+ --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.8+' /usr/lib/systemd/system/cri-docker.service
    
    # 安装 cri-docker.socket
    sudo curl -s https://vmcc.xyz:8443/k8s/cri-docker/cri-docker.socket 
    	-o /usr/lib/systemd/system/cri-docker.socket
    
    # 启动服务 cri-dockerd
    sudo systemctl daemon-reload
    sudo systemctl enable cri-docker
    sudo systemctl restart cri-docker
    
    # 安装 crictl 命令
    curl -# https://vmcc.xyz:8443/k8s/crictl-v1.24.2-linux-amd64.tar.gz 
    	-o crictl-v1.24.2-linux-amd64.tar.gz
    tar -xf crictl-v1.24.2-linux-amd64.tar.gz
    sudo cp crictl /usr/bin/
    # crictl 配置文件
    sudo tee /etc/crictl.yaml >/dev/null <<EOF
    runtime-endpoint: unix:///var/run/cri-dockerd.sock
    image-endpoint: unix:///var/run/cri-dockerd.sock
    timeout: 10
    debug: false
    pull-image-on-create: true
    EOF
    

[kiosk@k8s-worker2]$

  1. 安装运行时

    # 安装 containerd
    sudo apt install -y containerd
    
    # 创建目录
    sudo mkdir /etc/containerd
    
    # 生成默认配置文件
    containerd config default | 
    sudo tee /etc/containerd/config.toml >/dev/null
    
    # 修改配置文件
    sudo sed -i 
    -e '/sandbox_image/s?k8s.gcr.io?registry.aliyuncs.com/google_containers?' 
    -e '/SystemdCgroup/s?false?true?' 
    -e '/registry.mirrors/a        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]' 
    -e '/registry.mirrors/a          endpoint = ["https://docker.nju.edu.cn/"]' /etc/containerd/config.toml
    
    # 服务重启
    sudo systemctl restart containerd
    
    # 安装 crictl 命令
    curl -# https://vmcc.xyz:8443/k8s/crictl-v1.24.2-linux-amd64.tar.gz 
    	-o crictl-v1.24.2-linux-amd64.tar.gz
    tar -xf crictl-v1.24.2-linux-amd64.tar.gz
    sudo cp crictl /usr/bin/
    # crictl 配置文件
    sudo tee /etc/crictl.yaml >/dev/null <<EOF
    runtime-endpoint: unix:///run/containerd/containerd.sock
    image-endpoint: unix:///run/containerd/containerd.sock
    timeout: 10
    debug: false
    pull-image-on-create: true
    EOF
    
    

K. 安装 k8s

[kiosk@k8s-master|k8s-worker1|k8s-worker2]$

  1. 安装 kubeadm、kubelet 和 kubectl

    # 更新 apt 包索引并安装使用 Kubernetes apt 仓库所需要的包
    sudo apt -y install apt-transport-https ca-certificates curl
    
    # 下载 Google Cloud 公开签名秘钥
    curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -
    
    # 添加 Kubernetes apt 仓库
    MIRROR_URL=https://mirror.nju.edu.cn/kubernetes/apt/
    sudo tee /etc/apt/sources.list.d/kubernetes.list >/dev/null <<EOF
    deb $MIRROR_URL kubernetes-xenial main
    EOF
    
    # 更新 apt 包索引
    sudo cp /etc/apt/trusted.gpg /etc/apt/trusted.gpg.d
    sudo apt update -y
    sudo apt-cache madison kubelet | grep 1.24
    
    # 安装 kubelet、kubeadm 和 kubectl 考试版本
    sudo apt install -y kubelet=1.24.1-00 kubeadm=1.24.1-00 kubectl=1.24.1-00
    
    # 锁定版本
    sudo apt-mark hold kubelet kubeadm kubectl
    
    

[kiosk@k8s-worker2]$

# 增加 k8s 支持
 sudo sed -i '/ExecStart=//s|$| --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --cgroup-driver=systemd|' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
# 重启 kubelet 服务
sudo systemctl daemon-reload
sudo systemctl restart kubelet

[kiosk@k8s-master]$

  1. 初始化

    # 生成初始文件
    sudo kubeadm config print init-defaults > kubeadm-config.yaml
    
    # 修改文件
    sudo sed -i 
      -e "/advertiseAddress/s?:.*?: 192.168.147.128?" 
      -e "/name/s?:.*?: k8s-master?" 
      -e "/clusterName/s?:.*?: ck8s?" 
      -e "/imageRepository/s?:.*?: registry.aliyuncs.com/google_containers?" 
      -e "/criSocket/s+containerd/containerd+cri-dockerd+" kubeadm-config.yaml
    
    # 使用初始文件,初始化集群
    sudo kubeadm init --config kubeadm-config.yaml
    

    …输出省略…
    Your Kubernetes control-plane has initialized successfully!

    To start using your cluster, you need to run the following as a regular user:

    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config
    

    Alternatively, if you are the root user, you can run:

    export KUBECONFIG=/etc/kubernetes/admin.conf
    

    You should now deploy a pod network to the cluster.
    Run “kubectl apply -f [podnetwork].yaml” with one of the options listed at:
    https://kubernetes.io/docs/concepts/cluster-administration/addons/

    Then you can join any number of worker nodes by running the following on each as root:

    kubeadm join 192.168.147.128:6443 --token abcdef.0123456789abcdef 
    --discovery-token-ca-cert-hash sha256:c4781194de65ebb47984fc5e7e64d4897875410825ce4d18df81da1a298afa1f
    
  2. 配置文件

    # 创建目录
    mkdir -p $HOME/.kube
    
    # user 复制配置文件
    sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config
    
    # root 变量
    sudo tee -a ~root/.bashrc >/dev/null <<EOF
    export KUBECONFIG=/etc/kubernetes/admin.conf
    EOF
    
  3. 创建网络

    kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
    
  4. 命令补全

    # 立即生效
    source <(kubectl completion bash)
        
    # 永久生效
    mkdir ~/.kube
    kubectl completion bash > ~/.kube/completion.bash.inc
    printf "
    # Kubectl shell completion
    source '$HOME/.kube/completion.bash.inc'
    " >> $HOME/.bash_profile
    source $HOME/.bash_profile
    
  5. 命令别名

    # 永久生效
    tee -a $HOME/.bashrc >/dev/null <<EOF
    alias k='kubectl'
    complete -F __start_kubectl k
    EOF
    
    # 立即生效
    source $HOME/.bashrc
    

[kiosk@k8s-worker1]$

  1. 加入集群

    sudo kubeadm join 192.168.147.128:6443 
      --token abcdef.0123456789abcdef 
      --discovery-token-ca-cert-hash sha256:c4781194de65ebb47984fc5e7e64d4897875410825ce4d18df81da1a298afa1f 
      --cri-socket unix://var/run/cri-dockerd.sock
    

[kiosk@k8s-worker2]$

  1. 加入集群

    sudo kubeadm join 192.168.147.128:6443 
      --token abcdef.0123456789abcdef 
      --discovery-token-ca-cert-hash sha256:c4781194de65ebb47984fc5e7e64d4897875410825ce4d18df81da1a298afa1f
    

C. 确认环境正常

[kiosk@k8s-master]

$ kubectl get componentstatuses
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                 STATUS    MESSAGE                         ERROR
scheduler           `Healthy`  ok
controller-manager  `Healthy`  ok
etcd-0              `Healthy`  {"health":"true","reason":""}

$ kubectl get nodes
NAME          STATUS   ROLES                  AGE     VERSION
k8s-worker1  `Ready`  <none>                 4m4s    `v1.24.1`
k8s-worker2  `Ready`  <none>                 4m44s   `v1.24.1`
k8s-master   `Ready`  control-plane,master   13m     `v1.24.1`

$ kubectl -n kube-system get pod -w
NAME                                       READY   STATUS    RESTARTS   AGE
calico-kube-controllers-555bc4b957-8ccgh   1/1     Running   0          27m
calico-node-5qqcq                          1/1     Running   0          9m29s
calico-node-7qclz                          1/1     Running   0          27m
calico-node-kcvt5                          1/1     Running   0          9m29s
coredns-74586cf9b6-69fn7                   1/1     Running   0          156m
coredns-74586cf9b6-8mgl9                   1/1     Running   0          156m
etcd-k8s-master                            1/1     Running   0          156m
kube-apiserver-k8s-master                  1/1     Running   0          156m
kube-controller-manager-k8s-master         1/1     Running   0          156m
kube-proxy-8j248                           1/1     Running   0          9m29s
kube-proxy-g7r55                           1/1     Running   0          9m29s
kube-proxy-rbdcp                           1/1     Running   0          156m
kube-scheduler-k8s-master                  1/1     Running   0          156m
<Ctrl-C>
风语者!平时喜欢研究各种技术,目前在从事后端开发工作,热爱生活、热爱工作。