Deploy the Docker cluster

Background description

This article will briefly describe how to quickly build a complete Doris test cluster through docker run or docker-compose up commands.

Applicable scene

It is recommended to use Doris Docker in SIT or DEV environment to simplify the deployment process.

If you want to test a certain function point in the new version, you can use Doris Docker to deploy a Playground environment. Or when you want to reproduce a certain problem during debugging, you can also use the docker environment to simulate.

In the production environment, currently try to avoid using containerized solutions for Doris deployment.

Software Environment

Software Version
Docker 20.0 and above
docker-compose 2.10 and above

Hardware environment

Configuration Type Hardware Information Maximum Running Cluster Size
Minimum configuration 2C 4G 1FE 1BE
Recommended configuration 4C 16G 3FE 3BE

Pre-environment preparation

The following command needs to be executed on the host machine

1. sysctl -w vm.max_map_count=2000000

Docker Compose

Different platforms need to use different Image images. This article takes the X86_64 platform as an example.

Network Mode Description

There are two network modes applicable to Doris Docker.

  1. HOST mode suitable for deployment across multiple nodes, this mode is suitable for deploying 1FE 1BE on each node.
  2. The subnet bridge mode is suitable for deploying multiple Doris processes on a single node. This mode is suitable for single-node deployment (recommended). If you want to deploy multiple nodes, you need to deploy more components (not recommended).

For the sake of presentation, this chapter only demonstrates scripts written in subnet bridge mode.

Interface Description

From the version of Apache Doris 1.2.1 Docker Image, the interface list of each process image is as follows:

process name interface name interface definition interface example
FE\BE\BROKER FE_SERVERS FE node main information fe1:172.20.80.2:9010,fe2:172.20.80.3:9010,fe3:172.20.80.4:9010
FE FE_ID FE node ID 1
BE BE_ADDR BE node main information 172.20.80.5:9050
BE NODE_ROLE BE node type computation
BROKER BROKER_ADDR Main information of BROKER node 172.20.80.6:8000

Note that the above interface must fill in the information, otherwise the process cannot be started.

FE_SERVERS interface rules are: FE_NAME:FE_HOST:FE_EDIT_LOG_PORT[,FE_NAME:FE_HOST:FE_EDIT_LOG_PORT]

The FE_ID interface rule is: an integer of 1-9, where the FE number 1 is the Master node.

BE_ADDR interface rule is: BE_HOST:BE_HEARTBEAT_SERVICE_PORT

The NODE_ROLE interface rule is: computation or empty, where empty or other values indicate that the node type is mix type

BROKER_ADDR interface rule is: BROKER_HOST:BROKER_IPC_PORT

Script Template

Docker Run command

Create a subnet bridge

1. docker network create --driver bridge --subnet=172.20.80.0/24 doris-network

1FE & 1BE Command Templates

1. docker run -itd \
2. --name=fe \
3. --env FE_SERVERS="fe1:172.20.80.2:9010" \
4. --env FE_ID=1 \
5. -p 8030:8030\
6. -p 9030:9030 \
7. -v /data/fe/doris-meta:/opt/apache-doris/fe/doris-meta \
8. -v /data/fe/conf:/opt/apache-doris/fe/conf \
9. -v /data/fe/log:/opt/apache-doris/fe/log \
10. --network=doris-network \
11. --ip=172.20.80.2\
12. apache/doris:1.2.1-fe-x86_64

14. docker run -itd \
15. --name=be\
16. --env FE_SERVERS="fe1:172.20.80.2:9010" \
17. --env BE_ADDR="172.20.80.3:9050" \
18. -p 8040:8040 \
19. -v /data/be/storage:/opt/apache-doris/be/storage \
20. -v /data/be/conf:/opt/apache-doris/be/conf \
21. -v /data/be/log:/opt/apache-doris/be/log \
22. --network=doris-network \
23. --ip=172.20.80.3\
24. apache/doris:1.2.1-be-x86_64

3FE & 3BE Run command template if needed click here to access Downloads.

Docker Compose script

1FE & 1BE template

1. version: '3'
2. services:
3. docker-fe:
4. image: "apache/doris:1.2.1-fe-x86_64"
5. container_name: "doris-fe"
6. hostname: "fe"
7. environment:
8. - FE_SERVERS=fe1:172.20.80.2:9010
9. - FE_ID=1
10. ports:
11. - 8030:8030
12. - 9030:9030
13. volumes:
14. - /data/fe/doris-meta:/opt/apache-doris/fe/doris-meta
15. - /data/fe/conf:/opt/apache-doris/fe/conf
16. - /data/fe/log:/opt/apache-doris/fe/log
17. networks:
18. doris_net:
19. ipv4_address: 172.20.80.2
20. docker-be:
21. image: "apache/doris:1.2.1-be-x86_64"
22. container_name: "doris-be"
23. hostname: "be"
24. depends_on:
25. - docker-fe
26. environment:
27. - FE_SERVERS=fe1:172.20.80.2:9010
28. - BE_ADDR=172.20.80.3:9050
29. ports:
30. - 8040:8040
31. volumes:
32. - /data/be/storage:/opt/apache-doris/be/storage
33. - /data/be/conf:/opt/apache-doris/be/conf
34. - /data/be/script:/docker-entrypoint-initdb.d
35. - /data/be/log:/opt/apache-doris/be/log
36. networks:
37. doris_net:
38. ipv4_address: 172.20.80.3
39. networks:
40. doris_net:
41. ipam:
42. config:
43. - subnet: 172.20.80.0/16

3FE & 3BE Docker Compose script template if needed click here access to download.

Deploy Doris Docker

You can choose one of the two deployment methods:

  1. Execute the docker run command to create a cluster
  2. Save the docker-compose.yaml script and execute the docker-compose up -d command in the same directory to create a cluster

Special case description

Due to the different ways of implementing containers internally on MacOS, it may not be possible to directly modify the value of max_map_count on the host during deployment. You need to create the following containers first:

1. docker run -it --privileged --pid=host --name=change_count debian nsenter -t 1 -m -u -n -i sh

The container was created successfully executing the following command:

1. sysctl -w vm.max_map_count=2000000

Then exit exits and creates the Doris Docker cluster.

Unfinished business

  1. Compose Demo List