Kubernetes

1. Environment Preparation

1.1 Prepare a Kubernetes Cluster

Ensure that you have an available Kubernetes cluster (minimum recommended version: Kubernetes 1.24) as the foundation for deploying the IoTDB cluster.

Kubernetes Version Requirement: The recommended version is Kubernetes 1.24 or above.

IoTDB Version Requirement: The version of IoTDB must not be lower than v1.3.3.

2. Create Namespace

2.1 Create Namespace

Note: Before executing the namespace creation operation, verify that the specified namespace name has not been used in the Kubernetes cluster. If the namespace already exists, the creation command will fail, which may lead to errors during the deployment process.

1. kubectl create ns iotdb-ns

2.2 View Namespace

1. kubectl get ns

3. Create PersistentVolume (PV)

3.1 Create PV Configuration File

PV is used for persistent storage of IoTDB’s ConfigNode and DataNode data. You need to create one PV for each node.

Note: One ConfigNode and one DataNode count as two nodes, requiring two PVs.

For example, with 3 ConfigNodes and 3 DataNodes:

  1. Create a pv.yaml file and make six copies, renaming them to pv01.yaml through pv06.yaml.
1. # Create a directory to store YAML files
2. # Create pv.yaml file
3. touch pv.yaml
  1. Modify the name and path in each file to ensure consistency.

pv.yaml Example：

1. # pv.yaml
2. apiVersion: v1
3. kind: PersistentVolume
4. metadata:
5. name: iotdb-pv-01
6. spec:
7. capacity:
8. storage: 10Gi # Storage capacity
9. accessModes: # Access modes
10. - ReadWriteOnce
11. persistentVolumeReclaimPolicy: Retain # Reclaim policy
12. # Storage class name, if using local static storage, do not configure; if using dynamic storage, this must be set
13. storageClassName: local-storage
14. # Add the corresponding configuration based on your storage type
15. hostPath: # If using a local path
16. path: /data/k8s-data/iotdb-pv-01
17. type: DirectoryOrCreate  # If this line is not configured, you need to manually create the directory

3.2 Apply PV Configuration

1. kubectl apply -f pv01.yaml
2. kubectl apply -f pv-02.yaml
3. ...

3.3 View PV

1. kubectl get pv

Kubernetes Deployment - 图1

3.4 Manually Create Directories

Note: If the type in the hostPath of the YAML file is not configured, you need to manually create the corresponding directories.

Create the corresponding directories on all Kubernetes nodes:

1. mkdir -p /data/k8s-data/iotdb-pv-01
2. mkdir -p /data/k8s-data/iotdb-pv-02
3. ...

4. Install Helm

For installation steps, please refer to theHelm Official Website.

5. Configure IoTDB Helm Chart

5.1 Clone IoTDB Kubernetes Deployment Code

Clone Helm : Source Code

If encountering proxy issues, cancel proxy settings:

The git clone error is as follows, indicating that the proxy has been configured and needs to be turned off fatal: unable to access ‘<https://github.com/apache/iotdb-extras.git>‘: gnutls_handshake() failed: The TLS connection was non-properly terminated.

1. unset HTTPS_PROXY

5.2 Modify YAML Files

Ensure that the version used is supported (>=1.3.3.2):

values.yaml Example：

1. nameOverride: "iotdb"
2. fullnameOverride: "iotdb"   # Name after installation

4. image:
5. repository: apache/iotdb
6. pullPolicy: IfNotPresent
7. tag: latest    # Repository and version used

9. storage:
10. # Storage class name, if using local static storage, do not configure; if using dynamic storage, this must be set
11. className: local-storage

13. datanode:
14. name: datanode
15. nodeCount: 3        # Number of DataNode nodes
16. enableRestService: true
17. storageCapacity: 10Gi       # Available space for DataNode
18. resources:
19. requests:
20. memory: 2Gi    # Initial memory size for DataNode
21. cpu: 1000m     # Initial CPU size for DataNode
22. limits:
23. memory: 4Gi    # Maximum memory size for DataNode
24. cpu: 1000m     # Maximum CPU size for DataNode

26. confignode:
27. name: confignode
28. nodeCount: 3      # Number of ConfigNode nodes
29. storageCapacity: 10Gi      # Available space for ConfigNode
30. resources:
31. requests:
32. memory: 512Mi    # Initial memory size for ConfigNode
33. cpu: 1000m      # Initial CPU size for ConfigNode
34. limits:
35. memory: 1024Mi   # Maximum memory size for ConfigNode
36. cpu: 2000m     # Maximum CPU size for ConfigNode
37. configNodeConsensusProtocolClass: org.apache.iotdb.consensus.ratis.RatisConsensus
38. schemaReplicationFactor: 3
39. schemaRegionConsensusProtocolClass: org.apache.iotdb.consensus.ratis.RatisConsensus
40. dataReplicationFactor: 2
41. dataRegionConsensusProtocolClass: org.apache.iotdb.consensus.iot.IoTConsensus

6. Install IoTDB

6.1 Install IoTDB

1. # Enter the directory
2. cd iotdb-cluster-k8s/helm

4. # Install IoTDB
5. helm install iotdb ./ -n iotdb-ns

6.2 View Helm Installation List

1. # helm list
2. helm list -n iotdb-ns

6.3 View Pods

1. # View IoTDB pods
2. kubectl get pods -n iotdb-ns -o wide

After executing the command, if the output shows 6 Pods with confignode and datanode labels (3 each), it indicates a successful installation. Note that not all Pods may be in the Running state initially; inactive datanode Pods may keep restarting but will normalize after activation.

6.4 Troubleshooting

1. # View k8s creation logs
2. kubectl get events -n iotdb-ns
3. watch kubectl get events -n iotdb-ns

5. # Get detailed information
6. kubectl describe pod confignode-0 -n iotdb-ns
7. kubectl describe pod datanode-0 -n iotdb-ns

9. # View ConfigNode logs
10. kubectl logs -n iotdb-ns confignode-0 -f

7. Verify IoTDB

7.1 Check the Status of Pods within the Namespace

View the IP, status, and other information of the pods in the iotdb-ns namespace to ensure they are all running normally.

1. kubectl get pods -n iotdb-ns -o wide

3. # Example output:
4. # NAME           READY   STATUS    RESTARTS         AGE   IP             NODE   NOMINATED NODE   READINESS GATES
5. # confignode-0   1/1     Running   0                75m   10.20.187.14   a87    <none>           <none>
6. # confignode-1   1/1     Running   0                75m   10.20.191.75   a88    <none>           <none>
7. # confignode-2   1/1     Running   0                75m   10.20.187.16   a87    <none>           <none>
8. # datanode-0     1/1     Running   10 (5m54s ago)   75m   10.20.191.74   a88    <none>           <none>
9. # datanode-1     1/1     Running   10 (5m42s ago)   75m   10.20.187.15   a87    <none>           <none>
10. # datanode-2     1/1     Running   10 (5m55s ago)   75m   10.20.191.76   a88    <none>           <none>

7.2 Check the Port Mapping within the Namespace

1. kubectl get svc -n iotdb-ns

3. # Example output:
4. # NAME             TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
5. # confignode-svc   NodePort       10.10.226.151   <none>        80:31026/TCP     7d8h
6. # datanode-svc     NodePort       10.10.194.225   <none>        6667:31563/TCP   7d8h
7. # jdbc-balancer    LoadBalancer   10.10.191.209   <pending>     6667:31895/TCP   7d8h

7.3 Start the CLI Script on Any Server to Verify the IoTDB Cluster Status

Use the port of jdbc-balancer and the IP of any k8s node.

1. start-cli.sh -h 172.20.31.86 -p 31895
2. start-cli.sh -h 172.20.31.87 -p 31895
3. start-cli.sh -h 172.20.31.88 -p 31895

Kubernetes Deployment - 图2

8. Scaling

8.1 Add New PV

Add a new PV; scaling is only possible with available PVs.

Kubernetes Deployment - 图3

Note: DataNode cannot join the cluster after restart

Reason：The static storage hostPath mode is configured, and the script modifies the iotdb-system.properties file to set dn_data_dirs to /iotdb6/iotdb_data,/iotdb7/iotdb_data. However, the default storage path /iotdb/data is not mounted, leading to data loss upon restart. Solution：Mount the /iotdb/data directory as well, and ensure this setting is applied to both ConfigNode and DataNode to maintain data integrity and cluster stability.

8.2 Scale ConfigNode

Example: Scale from 3 ConfigNodes to 4 ConfigNodes

Modify the values.yaml file in iotdb-cluster-k8s/helm to change the number of ConfigNodes from 3 to 4.

1. helm upgrade iotdb . -n iotdb-ns

Kubernetes Deployment - 图4

8.3 Scale DataNode

Example: Scale from 3 DataNodes to 4 DataNodes

Modify the values.yaml file in iotdb-cluster-k8s/helm to change the number of DataNodes from 3 to 4.

1. helm upgrade iotdb . -n iotdb-ns

8.4 Verify IoTDB Status

1. kubectl get pods -n iotdb-ns -o wide

3. # NAME           READY   STATUS    RESTARTS         AGE   IP             NODE   NOMINATED NODE   READINESS GATES
4. # confignode-0   1/1     Running   0                75m   10.20.187.14   a87    <none>           <none>
5. # confignode-1   1/1     Running   0                75m   10.20.191.75   a88    <none>           <none>
6. # confignode-2   1/1     Running   0                75m   10.20.187.16   a87    <none>           <none>
7. # datanode-0     1/1     Running   10 (5m54s ago)   75m   10.20.191.74   a88    <none>           <none>
8. # datanode-1     1/1     Running   10 (5m42s ago)   75m   10.20.187.15   a87    <none>           <none>
9. # datanode-2     1/1     Running   10 (5m55s ago)   75m   10.20.191.76   a88    <none>           <none>
10. # datanode-3     1/1     Running   10 (5m55s ago)   75m   10.20.191.76   a88    <none>           <none>