Docker Deployment

Docker Deployment

Environmental Preparation

Docker Installation

1. #Taking Ubuntu as an example, other operating systems can search for installation methods themselves
2. #step1: Install some necessary system tools
3. sudo apt-get update
4. sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common
5. #step2: Install GPG certificate
6. curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | sudo apt-key add -
7. #step3: Write software source information
8. sudo add-apt-repository "deb [arch=amd64] https://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable"
9. #step4: Update and install Docker CE
10. sudo apt-get -y update
11. sudo apt-get -y install docker-ce
12. #step5: Set Docker to start automatically upon startup
13. sudo systemctl enable docker
14. #step6： Verify if Docker installation is successful
15. docker --version  #Display version information, indicating successful installation

Docker-compose Installation

1. #Installation command
2. curl -L "https://github.com/docker/compose/releases/download/v2.20.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
3. chmod +x /usr/local/bin/docker-compose
4. ln -s  /usr/local/bin/docker-compose  /usr/bin/docker-compose
5. #Verify if the installation was successful
6. docker-compose --version  #Displaying version information indicates successful installation

Stand-Alone Deployment

This section demonstrates how to deploy a standalone Docker version of 1C1D.

Pull Image File

The Docker image of Apache IoTDB has been uploaded to <https://hub.docker.com/r/apache/iotdb>.

Taking obtaining version 1.3.2 as an example, pull the image command:

1. docker pull apache/iotdb:1.3.2-standalone

View image:

1. docker images

Create Docker Bridge Network

1. docker network create --driver=bridge --subnet=172.18.0.0/16 --gateway=172.18.0.1  iotdb

Write The Yml File For Docker-Compose

Here we take the example of consolidating the IoTDB installation directory and yml files in the/docker iotdb folder:

The file directory structure is:/docker-iotdb/iotdb, /docker-iotdb/docker-compose-standalone.yml

1. docker-iotdb：
2. ├── iotdb  #Iotdb installation directory
3. │── docker-compose-standalone.yml #YML file for standalone Docker Composer

The complete docker-compose-standalone.yml content is as follows:

1. version: "3"
2. services:
3. iotdb-service:
4. image: apache/iotdb:1.3.2-standalone #The image used
5. hostname: iotdb
6. container_name: iotdb
7. restart: always
8. ports:
9. - "6667:6667"
10. environment:
11. - cn_internal_address=iotdb
12. - cn_internal_port=10710
13. - cn_consensus_port=10720
14. - cn_seed_config_node=iotdb:10710
15. - dn_rpc_address=iotdb
16. - dn_internal_address=iotdb
17. - dn_rpc_port=6667
18. - dn_internal_port=10730
19. - dn_mpp_data_exchange_port=10740
20. - dn_schema_region_consensus_port=10750
21. - dn_data_region_consensus_port=10760
22. - dn_seed_config_node=iotdb:10710
23. privileged: true
24. volumes:
25. - ./iotdb/data:/iotdb/data
26. - ./iotdb/logs:/iotdb/logs
27. networks:
28. iotdb:
29. ipv4_address: 172.18.0.6
30. # Note: Some environments set an extremely high container nofile limit (~2^30 = 1073741824).
31. # This can make the startup step "Checking whether the ports are already occupied..." appear to hang (lsof slow).
32. # If you see that line for a long time, lower the nofile limit by uncommenting below:
33. # ulimits:
34. #   nofile:
35. #     soft: 1048576
36. #     hard: 1048576
37. networks:
38. iotdb:
39. external: true

Start IoTDB

Use the following command to start:

1. cd　/docker-iotdb
2. docker-compose -f docker-compose-standalone.yml up  -d  #Background startup

Validate Deployment

Viewing the log, the following words indicate successful startup

1. docker logs -f iotdb-datanode #View log command
2. 2024-07-21 08:22:38,457 [main] INFO  o.a.i.db.service.DataNode:227 - Congratulations, IoTDB DataNode is set up successfully. Now, enjoy yourself!

Enter the container to view the service running status and activation information

View the launched container

1. docker ps

Enter the container, log in to the database through CLI, and use the show cluster command to view the service status and activation status

1. docker exec -it iotdb   /bin/bash        #Entering the container
2. ./start-cli.sh -h iotdb                  #Log in to the database
3. IoTDB> show cluster                      #View status

You can see that all services are running and the activation status shows as activated.

Map/conf Directory (optional)

If you want to directly modify the configuration file in the physical machine in the future, you can map the/conf folder in the container in three steps:

Step 1: Copy the /conf directory from the container to /docker-iotdb/iotdb/conf

1. docker cp iotdb:/iotdb/conf /docker-iotdb/iotdb/conf

Step 2: Add mappings in docker-compose-standalone.yml

1. volumes:
2. - ./iotdb/conf:/iotdb/conf   #Add mapping for this/conf folder
3. - ./iotdb/data:/iotdb/data
4. - ./iotdb/logs:/iotdb/logs

Step 3: Restart IoTDB

1. docker-compose  -f docker-compose-standalone.yml  up  -d

Cluster Deployment

This section describes how to manually deploy an instance that includes 3 Config Nodes and 3 Data Nodes, commonly known as a 3C3D cluster.

Note: The cluster version currently only supports host and overlay networks, and does not support bridge networks.

Taking the host network as an example, we will demonstrate how to deploy a 3C3D cluster.

Set Host Name

Assuming there are currently three Linux servers, the IP addresses and service role assignments are as follows:

Node IP	Host Name	Service
192.168.1.3	iotdb-1	ConfigNode、DataNode
192.168.1.4	iotdb-2	ConfigNode、DataNode
192.168.1.5	iotdb-3	ConfigNode、DataNode

Configure the host names on three machines separately. To set the host names, configure /etc/hosts on the target server using the following command:

1. echo "192.168.1.3  iotdb-1"  >> /etc/hosts
2. echo "192.168.1.4  iotdb-2"  >> /etc/hosts
3. echo "192.168.1.5  iotdb-3"  >> /etc/hosts

Pull Image File

The Docker image of Apache IoTDB has been uploaded to<https://hub.docker.com/r/apache/iotdb。>

Pull IoTDB images from three servers separately, taking version 1.3.2 as an example. The pull image command is:

1. docker pull apache/iotdb:1.3.2-standalone

View image:

1. docker images

Write The Yml File For Docker Compose

Here we take the example of consolidating the IoTDB installation directory and yml files in the /docker-iotdb folder:

The file directory structure is :/docker-iotdb/iotdb, /docker-iotdb/confignode.yml，/docker-iotdb/datanode.yml

1. docker-iotdb：
2. ├── confignode.yml #Yml file of confignode
3. ├── datanode.yml   #Yml file of datanode
4. └── iotdb          #IoTDB installation directory

On each server, two yml files need to be written, namely confignnode. yml and datanode. yml. The example of yml is as follows:

confignode.yml：

1. #confignode.yml
2. version: "3"
3. services:
4. iotdb-confignode:
5. image: iotdb-enterprise:1.3.2.3-standalone #The image used
6. hostname: iotdb-1|iotdb-2|iotdb-3 #Choose from three options based on the actual situation
7. container_name: iotdb-confignode
8. command: ["bash", "-c", "entrypoint.sh confignode"]
9. restart: always
10. environment:
11. - cn_internal_address=iotdb-1|iotdb-2|iotdb-3 #Choose from three options based on the actual situation
12. - cn_internal_port=10710
13. - cn_consensus_port=10720
14. - cn_seed_config_node=iotdb-1:10710   #The default first node is the seed node
15. - schema_replication_factor=3         #Number of metadata copies
16. - data_replication_factor=2           #Number of data replicas
17. privileged: true
18. volumes:
19. - ./iotdb/activation:/iotdb/activation
20. - ./iotdb/data:/iotdb/data
21. - ./iotdb/logs:/iotdb/logs
22. - /usr/sbin/dmidecode:/usr/sbin/dmidecode:ro
23. network_mode: "host"    #Using the host network
24. # Note: Some environments set an extremely high container nofile limit (~2^30 = 1073741824).
25. # This can make the startup step "Checking whether the ports are already occupied..." appear to hang (lsof slow).
26. # If you see that line for a long time, lower the nofile limit by uncommenting below:
27. # ulimits:
28. #   nofile:
29. #     soft: 1048576
30. #     hard: 1048576

datanode.yml：

1. #datanode.yml
2. version: "3"
3. services:
4. iotdb-datanode:
5. image: iotdb-enterprise:1.3.2.3-standalone #The image used
6. hostname: iotdb-1|iotdb-2|iotdb-3 #Choose from three options based on the actual situation
7. container_name: iotdb-datanode
8. command: ["bash", "-c", "entrypoint.sh datanode"]
9. restart: always
10. ports:
11. - "6667:6667"
12. privileged: true
13. environment:
14. - dn_rpc_address=iotdb-1|iotdb-2|iotdb-3 #Choose from three options based on the actual situation
15. - dn_internal_address=iotdb-1|iotdb-2|iotdb-3 #Choose from three options based on the actual situation
16. - dn_seed_config_node=iotdb-1:10710      #The default first node is the seed node
17. - dn_rpc_port=6667
18. - dn_internal_port=10730
19. - dn_mpp_data_exchange_port=10740
20. - dn_schema_region_consensus_port=10750
21. - dn_data_region_consensus_port=10760
22. - schema_replication_factor=3         #Number of metadata copies
23. - data_replication_factor=2           #Number of data replicas
24. volumes:
25. - ./iotdb/activation:/iotdb/activation
26. - ./iotdb/data:/iotdb/data
27. - ./iotdb/logs:/iotdb/logs
28. - /usr/sbin/dmidecode:/usr/sbin/dmidecode:ro
29. network_mode: "host"   #Using the host network
30. # Note: Some environments set an extremely high container nofile limit (~2^30 = 1073741824).
31. # This can make the startup step "Checking whether the ports are already occupied..." appear to hang (lsof slow).
32. # If you see that line for a long time, lower the nofile limit by uncommenting below:
33. # ulimits:
34. #   nofile:
35. #     soft: 1048576
36. #     hard: 1048576

Starting Confignode For The First Time

First, start configNodes on each of the three servers to obtain the machine code. Pay attention to the startup order, start the first iotdb-1 first, then start iotdb-2 and iotdb-3.

1. cd　/docker-iotdb
2. docker-compose -f confignode.yml up  -d #Background startup

Start Datanode

Start datanodes on 3 servers separately

1. cd /docker-iotdb
2. docker-compose  -f  datanode.yml  up -d #Background startup

Validate Deployment

Viewing the logs, the following words indicate that the datanode has successfully started

1. docker logs -f iotdb-datanode #View log command 2. 2024-07-21 09:40:58,120 [main] INFO o.a.i.db.service.DataNode:227 - Congratulations, IoTDB DataNode is set up successfully. Now, enjoy yourself!

- Enter any container to view the service running status and activation information

View the launched container

1. docker ps

Enter the container, log in to the database through CLI, and use the show cluster command to view the service status and activation status

1. docker exec -it iotdb-datanode /bin/bash #Entering the container 2. ./start-cli.sh -h iotdb-1 #Log in to the database 3. IoTDB> show cluster #View status

You can see that all services are running and the activation status shows as activated.

Map/conf Directory (optional)

If you want to directly modify the configuration file in the physical machine in the future, you can map the/conf folder in the container in three steps:

Step 1: Copy the /conf directory from the container to /docker-iotdb/iotdb/conf on each of the three servers

1. docker cp iotdb-confignode:/iotdb/conf /docker-iotdb/iotdb/conf
2. or
3. docker cp iotdb-datanode:/iotdb/conf   /docker-iotdb/iotdb/conf

Step 2: Add /conf directory mapping in confignode.yml and datanode. yml on 3 servers

1. #confignode.yml
2. volumes:
3. - ./iotdb/conf:/iotdb/conf  #Add mapping for this /conf folder
4. - ./iotdb/data:/iotdb/data
5. - ./iotdb/logs:/iotdb/logs

7. #datanode.yml
8. volumes:
9. - ./iotdb/conf:/iotdb/conf   #Add mapping for this /conf folder
10. - ./iotdb/data:/iotdb/data
11. - ./iotdb/logs:/iotdb/logs

Step 3: Restart IoTDB on 3 servers

1. cd /docker-iotdb
2. docker-compose  -f confignode.yml  up  -d
3. docker-compose  -f datanode.yml    up  -d