OLAP系列：三、clickhouse Docker集群部署指南(3分片1副本模式)网站首页 技术杂谈

OLAP系列：三、clickhouse Docker集群部署指南(3分片1副本模式)

幽幽之心 2024-07-02 00:01:02

简介OLAP系列：三、clickhouse Docker集群部署指南(3分片1副本模式)

一、背景

一是为了学习容器，另外也是帮助一些同学解决机器资源紧缺，能够在一台是宿主机部署一套clickhouse集群服务。

本章内容适合开发测试环境使用，生产环境还需要更多细节的处理工作，只能作为参考。

二、部署准备

1、机器准备

172.25.16.108

2、创建容器网络

docker network create ckcluster

3、镜像准备

# 查看仓库镜像列表
curl -XGET https://nexus.***.com:8445/v2/_catalog | json_reformat
# 查看clickhouse镜像信息
curl -XGET https://nexus.***.com:8445/v2/yandex/clickhouse-server/tags/list | json_reformat
# 查看zookeeper镜像信息
curl -XGET https://nexus.***.com:8445/v2/zookeeper/tags/list | json_reformat

三、zookeeper单机安装(1节点)

1、下载镜像

docker pull nexus.***.com:8445/zookeeper:3.8.0

2、容器启动

docker run --restart always -d 
--name zookeeper 
--ulimit nofile=262144:262144 
--hostname zookeeper 
--network ckcluster 
-p 2181:2181 
nexus.***.com:8445/zookeeper:3.8.0

四、clickhouse集群安装（3分片1副本）

1、创建目录

cd /home
mkdir ckcluster
cd cluster
mkdir ck1
mkdir ck2
mkdir ck3

2、创建节点配置

# 启动ck1
docker run -d --name clickhouse-server-1 --ulimit nofile=262144:262144 -v /home/ckcluster/ck1/:/var/lib/clickhouse -p 18123:8123 nexus.***.com:8445/yandex/clickhouse-server:latest
# 拷贝容器内容的配置到/data/clickhouse/目录下
docker cp clickhouse-server-1:/etc/clickhouse-server/ /home/ckcluster/ck1/

# 启动ck2
docker run -d --name clickhouse-server-2 --ulimit nofile=262144:262144 -v /home/ckcluster/ck2/:/var/lib/clickhouse -p 28123:8123 nexus.***.com:8445/yandex/clickhouse-server:latest
# 拷贝容器内容的配置到/data/clickhouse/目录下
docker cp clickhouse-server-2:/etc/clickhouse-server/ /home/ckcluster/ck2/

# 启动ck3
docker run -d --name clickhouse-server-3 --ulimit nofile=262144:262144 -v /home/ckcluster/ck3/:/var/lib/clickhouse -p 38123:8123 nexus.***.com:8445/yandex/clickhouse-server:latest
# 拷贝容器内容的配置到/data/clickhouse/目录下
docker cp clickhouse-server-3:/etc/clickhouse-server/ /home/ckcluster/ck3/

3、清除临时容器

docker stop **

docker rm **

4、修改config.xml配置

vim /etc/clickhouse-server/config.xml
<include_from>/etc/clickhouse-server/config.d/metrika.xml</include_from> <!--添加该参数-->

5、增加服务节点配置文件【metrika.xml】

cd /home/ckcluster/ck1/
cd clickhouse-server
vim config.d/metrika.xml

### 在metrika.xml中我们配置后期使用的clickhouse集群中创建分布式表时使用3个分片，每个分片有1个副本，配置如下：
<yandex>
    <remote_servers>
        <clickhouse_cluster_3shards_1replicas>
            <shard>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>clickhouse-server01</host>
                    <port>19000</port>
                </replica>
            </shard>
            <shard>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>clickhouse-server02</host>
                    <port>29000</port>
                </replica>
            </shard>
            <shard>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>clickhouse-server03</host>
                    <port>39000</port>
                </replica>
            </shard>
        </clickhouse_cluster_3shards_1replicas>
    </remote_servers>
   
    <zookeeper>
        <node index="1">
            <host>zookeeper</host>
            <port>2181</port>
        </node>
    </zookeeper>
    <macros>
        <shard>01</shard> 
        <replica>node1</replica>
    </macros>
    <networks>
        <ip>0.0.0.0/0</ip>
    </networks>
    <clickhouse_compression>
        <case>
            <min_part_size>10000000000</min_part_size>
            <min_part_size_ratio>0.01</min_part_size_ratio>
            <method>lz4</method>
        </case>
    </clickhouse_compression>
</yandex>

注意：需要在每台clickhouse节点上配置metrika.xml文件，并且修改每个节点的 macros配置名称。

<!-- node2节点修改metrika.xml中的宏变量如下：-->
    <macros>
        <shard>02</shard> 
        <replica>node2</replica>
    </macros>

<!-- node3节点修改metrika.xml中的宏变量如下: -->
    <macros>
        <shard>03</shard> 
        <replica>node3</replica>
    </macros>

6、启动服务

节点1

# clickhouse-server01
docker run --restart always -d 
--name clickhouse-server01 
--ulimit nofile=262144:262144 
--hostname clickhouse-server01 
--network ckcluster 
-v /home/ckcluster/ck1/data/:/var/lib/clickhouse/ 
-v /home/ckcluster/ck1/clickhouse-server/:/etc/clickhouse-server/ 
-v /home/ckcluster/ck1/logs/:/var/log/clickhouse-server/ 
-p 19000:9000 -p 18123:8123 -p 19009:9009 
nexus.***.com:8445/yandex/clickhouse-server:latest

节点2

# clickhouse-server02
docker run --restart always -d 
--name clickhouse-server02 
--ulimit nofile=262144:262144 
--hostname clickhouse-server02 
--network ckcluster 
-v /home/ckcluster/ck2/data/:/var/lib/clickhouse/ 
-v /home/ckcluster/ck2/clickhouse-server/:/etc/clickhouse-server/ 
-v /home/ckcluster/ck2/logs/:/var/log/clickhouse-server/ 
-p 29000:9000 -p 28123:8123 -p 29009:9009 
nexus.***.com:8445/yandex/clickhouse-server:latest

节点3

# clickhouse-server03
docker run --restart always -d 
--name clickhouse-server03 
--ulimit nofile=262144:262144 
--hostname clickhouse-server03 
--network ckcluster 
-v /home/ckcluster/ck3/data/:/var/lib/clickhouse/ 
-v /home/ckcluster/ck3/clickhouse-server/:/etc/clickhouse-server/ 
-v /home/ckcluster/ck3/logs/:/var/log/clickhouse-server/ 
-p 39000:9000 -p 38123:8123 -p 39009:9009 
nexus.***.com:8445/yandex/clickhouse-server:latest