我的书签
添加书签
移除书签- TiFlash 部署拓扑
- # Global variables are applied to all deployments and used as the default value of
- # the deployments if a specific deployment value is missing.
- # Monitored variables are applied to all the machines.
- deploy_dir: “/tidb-deploy/monitored-9100”
- data_dir: “/tidb-data/monitored-9100”
- log_dir: “/tidb-deploy/monitored-9100/log”
- # Server configs are used to specify the runtime configuration of TiDB components.
- # All configuration items can be found in TiDB docs:
- # - TiDB: https://docs.pingcap.com/zh/tidb/stable/tidb-configuration-file
- # - TiKV: https://docs.pingcap.com/zh/tidb/stable/tikv-configuration-file
- # - PD: https://docs.pingcap.com/zh/tidb/stable/pd-configuration-file
- # All configuration items use points to represent the hierarchy, e.g:
- # readpool.storage.use-unified-pool
- # You can overwrite this configuration via the instance-level config field.
- # Monitored variables are applied to all the machines.
- deploy_dir: “/tidb-deploy/monitored-9100”
- data_dir: “/tidb-data/monitored-9100”
- log_dir: “/tidb-deploy/monitored-9100/log”
- # Server configs are used to specify the runtime configuration of TiDB components.
- # All configuration items can be found in TiDB docs:
- # - TiDB: https://docs.pingcap.com/zh/tidb/stable/tidb-configuration-file
- # - TiKV: https://docs.pingcap.com/zh/tidb/stable/tikv-configuration-file
- # - PD: https://docs.pingcap.com/zh/tidb/stable/pd-configuration-file
- # All configuration items use points to represent the hierarchy, e.g:
- # readpool.storage.use-unified-pool
- # You can overwrite this configuration via the instance-level config field.
TiFlash 部署拓扑
本文介绍在部署最小拓扑集群的基础上，部署 TiFlash 的拓扑结构。TiFlash 是列式的存储引擎，已经成为集群拓扑的标配，适合 Real-Time HTAP 业务。
拓扑信息
| 实例 | 个数 | 物理机配置 | IP | 配置 |
|---|---|---|---|---|
| TiDB | 3 | 16 VCore 32GB 1 | 10.0.1.7 10.0.1.8 10.0.1.9 | 默认端口 全局目录配置 |
| PD | 3 | 4 VCore 8GB 1 | 10.0.1.4 10.0.1.5 10.0.1.6 | 默认端口 全局目录配置 |
| TiKV | 3 | 16 VCore 32GB 2TB (nvme ssd) 1 | 10.0.1.1 10.0.1.2 10.0.1.3 | 默认端口 全局目录配置 |
| TiFlash | 1 | 32 VCore 64 GB 2TB (nvme ssd) 1 | 10.0.1.11 | 默认端口 全局目录配置 |
| Monitoring & Grafana | 1 | 4 VCore 8GB * 1 500GB (ssd) | 10.0.1.10 | 默认端口 全局目录配置 |
拓扑模版
简单 TiFlash 配置模版
1. # # Global variables are applied to all deployments and used as the default value of
2. # # the deployments if a specific deployment value is missing.
3. global:
4. user: "tidb"
5. ssh_port: 22
6. deploy_dir: "/tidb-deploy"
7. data_dir: "/tidb-data"
9. server_configs:
10. pd:
11. replication.enable-placement-rules: true
13. pd_servers:
14. - host: 10.0.1.4
15. - host: 10.0.1.5
16. - host: 10.0.1.6
18. tidb_servers:
19. - host: 10.0.1.7
20. - host: 10.0.1.8
21. - host: 10.0.1.9
23. tikv_servers:
24. - host: 10.0.1.1
25. - host: 10.0.1.2
26. - host: 10.0.1.3
28. tiflash_servers:
29. - host: 10.0.1.11
30. data_dir: /tidb-data/tiflash-9000
31. deploy_dir: /tidb-deploy/tiflash-9000
33. monitoring_servers:
34. - host: 10.0.1.10
36. grafana_servers:
37. - host: 10.0.1.10
39. alertmanager_servers:
40. - host: 10.0.1.10
41. ``` 详细 TiFlash 配置模版
# Global variables are applied to all deployments and used as the default value of
# the deployments if a specific deployment value is missing.
global: user: “tidb” ssh_port: 22 deploy_dir: “/tidb-deploy” data_dir: “/tidb-data”
# Monitored variables are applied to all the machines.
monitored: node_exporter_port: 9100 blackbox_exporter_port: 9115
deploy_dir: “/tidb-deploy/monitored-9100”
data_dir: “/tidb-data/monitored-9100”
log_dir: “/tidb-deploy/monitored-9100/log”
# Server configs are used to specify the runtime configuration of TiDB components.
# All configuration items can be found in TiDB docs:
# - TiDB: https://docs.pingcap.com/zh/tidb/stable/tidb-configuration-file
# - TiKV: https://docs.pingcap.com/zh/tidb/stable/tikv-configuration-file
# - PD: https://docs.pingcap.com/zh/tidb/stable/pd-configuration-file
# All configuration items use points to represent the hierarchy, e.g:
# readpool.storage.use-unified-pool
# You can overwrite this configuration via the instance-level config field.
server_configs: tidb: log.slow-threshold: 300 tikv:
1. # server.grpc-concurrency: 4
2. # raftstore.apply-pool-size: 2
3. # raftstore.store-pool-size: 2
4. # rocksdb.max-sub-compactions: 1
5. # storage.block-cache.capacity: "16GB"
6. # readpool.unified.max-thread-count: 12
7. readpool.storage.use-unified-pool: false
8. readpool.coprocessor.use-unified-pool: true
pd: schedule.leader-schedule-limit: 4 schedule.region-schedule-limit: 2048 schedule.replica-schedule-limit: 64 replication.enable-placement-rules: true tiflash:
1. # Maximum memory usage for processing a single query. Zero means unlimited.
2. profiles.default.max_memory_usage: 0
3. # Maximum memory usage for processing all concurrently running queries on the server. Zero means unlimited.
4. profiles.default.max_memory_usage_for_all_queries: 0
tiflash-learner:
1. # The allowable number of threads in the pool that flushes Raft data to storage.
2. raftstore.apply-pool-size: 4
3. # The allowable number of threads that process Raft, which is the size of the Raftstore thread pool.
4. raftstore.store-pool-size: 4
pd_servers:
- host: 10.0.1.4
# ssh_port: 22
# name: “pd-1”
# client_port: 2379
# peer_port: 2380
# deploy_dir: “/tidb-deploy/pd-2379”
# data_dir: “/tidb-data/pd-2379”
# log_dir: “/tidb-deploy/pd-2379/log”
# numa_node: “0,1”
# # The following configs are used to overwrite the server_configs.pd values.
# config:
# schedule.max-merge-region-size: 20
# schedule.max-merge-region-keys: 200000 - host: 10.0.1.5 - host: 10.0.1.6 tidb_servers: - host: 10.0.1.7
# ssh_port: 22
# port: 4000
# status_port: 10080
# deploy_dir: “/tidb-deploy/tidb-4000”
# log_dir: “/tidb-deploy/tidb-4000/log”
# numa_node: “0,1”
# # The following configs are used to overwrite the server_configs.tidb values.
# config:
# log.slow-query-file: tidb-slow-overwrited.log - host: 10.0.1.8 - host: 10.0.1.9 tikv_servers: - host: 10.0.1.1
# ssh_port: 22
# port: 20160
# status_port: 20180
# deploy_dir: “/tidb-deploy/tikv-20160”
# data_dir: “/tidb-data/tikv-20160”
# log_dir: “/tidb-deploy/tikv-20160/log”
# numa_node: “0,1”
# # The following configs are used to overwrite the server_configs.tikv values.
# config:
# server.grpc-concurrency: 4
# server.labels:
# zone: “zone1”
# dc: “dc1”
# host: “host1” - host: 10.0.1.2 - host: 10.0.1.3
tiflash_servers:
- host: 10.0.1.11
# ssh_port: 22
# tcp_port: 9000
# flash_service_port: 3930
# flash_proxy_port: 20170
# flash_proxy_status_port: 20292
# metrics_port: 8234
# deploy_dir: “/tidb-deploy/tiflash-9000”
## The data_dir will be overwritten if you define storage.main.dir configurations in the config section.
# data_dir: “/tidb-data/tiflash-9000”
# log_dir: “/tidb-deploy/tiflash-9000/log”
# numa_node: “0,1”
# # The following configs are used to overwrite the server_configs.tiflash values.
# config:
# logger.level: “info”
# ## Multi-disk deployment introduced in v4.0.9
# ## Check https://docs.pingcap.com/tidb/stable/tiflash-configuration#multi-disk-deployment for more details.
# ## Example1:
# # storage.main.dir: [ “/nvme_ssd0_512/tiflash”, “/nvme_ssd1_512/tiflash” ]
# # storage.main.capacity: [ 536870912000, 536870912000 ]
# ## Example2:
# # storage.main.dir: [ “/sata_ssd0_512/tiflash”, “/sata_ssd1_512/tiflash”, “/sata_ssd2_512/tiflash” ]
# # storage.latest.dir: [ “/nvme_ssd0_150/tiflash” ]
# # storage.main.capacity: [ 536870912000, 536870912000, 536870912000 ]
# # storage.latest.capacity: [ 161061273600 ]
# learner_config:
# log-level: “info”
# server.labels:
# zone: “zone2”
# dc: “dc2”
# host: “host2”
# - host: 10.0.1.12
# - host: 10.0.1.13
monitoring_servers:
- host: 10.0.1.10
# ssh_port: 22
# port: 9090
# deploy_dir: “/tidb-deploy/prometheus-8249”
# data_dir: “/tidb-data/prometheus-8249”
# log_dir: “/tidb-deploy/prometheus-8249/log”
grafana_servers:
- host: 10.0.1.10
# port: 3000
# deploy_dir: /tidb-deploy/grafana-3000
alertmanager_servers:
- host: 10.0.1.10
# ssh_port: 22
# web_port: 9093
# cluster_port: 9094
# deploy_dir: “/tidb-deploy/alertmanager-9093”
# data_dir: “/tidb-data/alertmanager-9093”
# log_dir: “/tidb-deploy/alertmanager-9093/log”# # Global variables are applied to all deployments and used as the default value of
# # the deployments if a specific deployment value is missing.
global: user: “tidb” ssh_port: 22 deploy_dir: “/tidb-deploy” data_dir: “/tidb-data”
# Monitored variables are applied to all the machines.
monitored: node_exporter_port: 9100 blackbox_exporter_port: 9115
deploy_dir: “/tidb-deploy/monitored-9100”
data_dir: “/tidb-data/monitored-9100”
log_dir: “/tidb-deploy/monitored-9100/log”
# Server configs are used to specify the runtime configuration of TiDB components.
# All configuration items can be found in TiDB docs:
# - TiDB: https://docs.pingcap.com/zh/tidb/stable/tidb-configuration-file
# - TiKV: https://docs.pingcap.com/zh/tidb/stable/tikv-configuration-file
# - PD: https://docs.pingcap.com/zh/tidb/stable/pd-configuration-file
# All configuration items use points to represent the hierarchy, e.g:
# readpool.storage.use-unified-pool
# You can overwrite this configuration via the instance-level config field.
server_configs: tidb: log.slow-threshold: 300 tikv:
1. # server.grpc-concurrency: 4
2. # raftstore.apply-pool-size: 2
3. # raftstore.store-pool-size: 2
4. # rocksdb.max-sub-compactions: 1
5. # storage.block-cache.capacity: "16GB"
6. # readpool.unified.max-thread-count: 12
7. readpool.storage.use-unified-pool: false
8. readpool.coprocessor.use-unified-pool: true
pd: schedule.leader-schedule-limit: 4 schedule.region-schedule-limit: 2048 schedule.replica-schedule-limit: 64 replication.enable-placement-rules: true tiflash:
1. # Maximum memory usage for processing a single query. Zero means unlimited.
2. profiles.default.max_memory_usage: 0
3. # Maximum memory usage for processing all concurrently running queries on the server. Zero means unlimited.
4. profiles.default.max_memory_usage_for_all_queries: 0
tiflash-learner:
1. # The allowable number of threads in the pool that flushes Raft data to storage.
2. raftstore.apply-pool-size: 4
3. # The allowable number of threads that process Raft, which is the size of the Raftstore thread pool.
4. raftstore.store-pool-size: 4
pd_servers:
- host: 10.0.1.4
# ssh_port: 22
# name: “pd-1”
# client_port: 2379
# peer_port: 2380
# deploy_dir: “/tidb-deploy/pd-2379”
# data_dir: “/tidb-data/pd-2379”
# log_dir: “/tidb-deploy/pd-2379/log”
# numa_node: “0,1”
# # The following configs are used to overwrite the server_configs.pd values.
# config:
# schedule.max-merge-region-size: 20
# schedule.max-merge-region-keys: 200000 - host: 10.0.1.5 - host: 10.0.1.6 tidb_servers: - host: 10.0.1.7
# ssh_port: 22
# port: 4000
# status_port: 10080
# deploy_dir: “/tidb-deploy/tidb-4000”
# log_dir: “/tidb-deploy/tidb-4000/log”
# numa_node: “0,1”
# # The following configs are used to overwrite the server_configs.tidb values.
# config:
# log.slow-query-file: tidb-slow-overwrited.log - host: 10.0.1.8 - host: 10.0.1.9 tikv_servers: - host: 10.0.1.1
# ssh_port: 22
# port: 20160
# status_port: 20180
# deploy_dir: “/tidb-deploy/tikv-20160”
# data_dir: “/tidb-data/tikv-20160”
# log_dir: “/tidb-deploy/tikv-20160/log”
# numa_node: “0,1”
# # The following configs are used to overwrite the server_configs.tikv values.
# config:
# server.grpc-concurrency: 4
# server.labels:
# zone: “zone1”
# dc: “dc1”
# host: “host1” - host: 10.0.1.2 - host: 10.0.1.3
tiflash_servers:
- host: 10.0.1.11
# ssh_port: 22
# tcp_port: 9000
# flash_service_port: 3930
# flash_proxy_port: 20170
# flash_proxy_status_port: 20292
# metrics_port: 8234
# deploy_dir: “/tidb-deploy/tiflash-9000”
## The data_dir will be overwritten if you define storage.main.dir configurations in the config section.
# data_dir: “/tidb-data/tiflash-9000”
# log_dir: “/tidb-deploy/tiflash-9000/log”
# numa_node: “0,1”
# # The following configs are used to overwrite the server_configs.tiflash values.
# config:
# logger.level: “info”
# ## Multi-disk deployment introduced in v4.0.9
# ## Check https://docs.pingcap.com/tidb/stable/tiflash-configuration#multi-disk-deployment for more details.
# ## Example1:
# # storage.main.dir: [ “/nvme_ssd0_512/tiflash”, “/nvme_ssd1_512/tiflash” ]
# # storage.main.capacity: [ 536870912000, 536870912000 ]
# ## Example2:
# # storage.main.dir: [ “/sata_ssd0_512/tiflash”, “/sata_ssd1_512/tiflash”, “/sata_ssd2_512/tiflash” ]
# # storage.latest.dir: [ “/nvme_ssd0_150/tiflash” ]
# # storage.main.capacity: [ 536870912000, 536870912000, 536870912000 ]
# # storage.latest.capacity: [ 161061273600 ]
# learner_config:
# log-level: “info”
# server.labels:
# zone: “zone2”
# dc: “dc2”
# host: “host2”
# - host: 10.0.1.12
# - host: 10.0.1.13
monitoring_servers:
- host: 10.0.1.10
# ssh_port: 22
# port: 9090
# deploy_dir: “/tidb-deploy/prometheus-8249”
# data_dir: “/tidb-data/prometheus-8249”
# log_dir: “/tidb-deploy/prometheus-8249/log”
grafana_servers:
- host: 10.0.1.10
# port: 3000
# deploy_dir: /tidb-deploy/grafana-3000
alertmanager_servers:
- host: 10.0.1.10
# ssh_port: 22
# web_port: 9093
# cluster_port: 9094
# deploy_dir: “/tidb-deploy/alertmanager-9093”
# data_dir: “/tidb-data/alertmanager-9093”
# log_dir: “/tidb-deploy/alertmanager-9093/log”
```
以上 TiDB 集群拓扑文件中，详细的配置项说明见通过 TiUP 部署 TiDB 集群的拓扑文件配置。
关键参数介绍
- 需要将配置模板中
replication.enable-placement-rules设置为true，以开启 PD 的 Placement Rules 功能。 tiflash_servers实例级别配置"-host"目前只支持 IP，不支持域名。- TiFlash 具体的参数配置介绍可参考 TiFlash 参数配置。
注意
- 无需手动创建配置文件中的
tidb用户，TiUP cluster 组件会在目标主机上自动创建该用户。可以自定义用户，也可以和中控机的用户保持一致。 - 如果部署目录配置为相对路径，会部署在用户的 Home 目录下。
