使用 OpenAPI 运维 TiDB Data Migration 集群

使用 OpenAPI 运维 TiDB Data Migration 集群

TiDB Data Migration (DM) 提供 OpenAPI 功能，你可以通过 OpenAPI 方便地对 DM 集群进行查询和运维操作。OpenAPI 的功能范围和 dmctl 工具相当。

如需开启 OpenAPI，可通过以下方法：

如果你的 DM 集群是通过二进制直接部署的，则在 DM-master 的配置文件中添加如下配置：

1. openapi = true - 如果你的 DM 集群是通过 TiUP 部署的，则在拓扑文件中添加如下配置：

1. server_configs: 2. master: 3. openapi: true

注意

DM 提供符合 OpenAPI 3.0.0 标准的 Spec 文档，其中包含了所有 API 的请求参数和返回体，你可自行复制到如 Swagger Editor 等工具中在线预览文档。
部署 DM-master 后，你可访问 http://{master-addr}/api/v1/docs 在线预览文档。
配置文件中支持的某些功能在 OpenAPI 中是不支持的，二者的功能没有完全对齐。在生产环境中，建议使用配置文件。

你可以通过 OpenAPI 完成 DM 集群的如下运维操作：

数据源相关 API

创建数据源
获取数据源
删除数据源
更新数据源
启用数据源
停用数据源
获取数据源状态
获取数据源列表
对数据源开启 relay-log 功能
对数据源停止 relay-log 功能
清除数据源不需要的 relay-log 文件
更改数据源和 DM-worker 的绑定关系
获取数据源的数据库名列表
获取数据源的指定数据库的表名列表

同步任务相关 API

创建同步任务
获取同步任务
删除同步任务
更新同步任务
开始同步任务
停止同步任务
获取同步任务状态
获取同步任务列表
获取同步任务的同步规则列表
获取同步任务关联数据源的数据库名列表
获取同步任务关联数据源的数据表名列表
获取同步任务关联数据源的数据表的创建语句
更新同步任务关联数据源的数据表的创建语句
删除同步任务关联数据源的数据表

本文档以下部分描述当前提供的 API 的具体使用方法。

API 统一错误格式

对 API 发起的请求后，如发生错误，返回错误信息的格式如下所示：

1. {
2. "error_msg": "",
3. "error_code": ""
4. }

如上所示，error_msg 描述错误信息，error_code 则是对应的错误码。

获取 DM-master 节点信息

该接口是一个同步接口，请求成功会返回对应节点的状态信息。

请求 URI

GET /api/v1/cluster/masters

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/cluster/masters' \
3. -H 'accept: application/json'

1. {
2. "total": 1,
3. "data": [
4. {
5. "name": "master1",
6. "alive": true,
7. "leader": true,
8. "addr": "127.0.0.1:8261"
9. }
10. ]
11. }

下线 DM-master 节点

该接口是一个同步接口，请求成功后返回体的 Status Code 是 204。

请求 URI

DELETE /api/v1/cluster/masters/{master-name}

使用样例

1. curl -X 'DELETE' \
2. 'http://127.0.0.1:8261/api/v1/cluster/masters/master1' \
3. -H 'accept: */*'

获取 DM-worker 节点信息

该接口是一个同步接口，请求成功会返回对应节点的状态信息。

请求 URI

GET /api/v1/cluster/workers

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/cluster/workers' \
3. -H 'accept: application/json'

1. {
2. "total": 1,
3. "data": [
4. {
5. "name": "worker1",
6. "addr": "127.0.0.1:8261",
7. "bound_stage": "bound",
8. "bound_source_name": "mysql-01"
9. }
10. ]
11. }

下线 DM-worker 节点

该接口是一个同步接口，请求成功后返回体的 Status Code 是 204。

请求 URI

DELETE /api/v1/cluster/workers/{worker-name}

使用样例

1. curl -X 'DELETE' \
2. 'http://127.0.0.1:8261/api/v1/cluster/workers/worker1' \
3. -H 'accept: */*'

创建数据源

该接口是一个同步接口，请求成功会返回对应数据源信息。

请求 URI

POST /api/v1/sources

使用样例

1. curl -X 'POST' \
2. 'http://127.0.0.1:8261/api/v1/sources' \
3. -H 'accept: application/json' \
4. -H 'Content-Type: application/json' \
5. -d '{
6. "source_name": "mysql-01",
7. "host": "127.0.0.1",
8. "port": 3306,
9. "user": "root",
10. "password": "123456",
11. "enable": true,
12. "enable_gtid": false,
13. "security": {
14. "ssl_ca_content": "",
15. "ssl_cert_content": "",
16. "ssl_key_content": "",
17. "cert_allowed_cn": [
18. "string"
19. ]
20. },
21. "purge": {
22. "interval": 3600,
23. "expires": 0,
24. "remain_space": 15
25. }
26. }'

1. {
2. "source_name": "mysql-01",
3. "host": "127.0.0.1",
4. "port": 3306,
5. "user": "root",
6. "password": "123456",
7. "enable": true,
8. "enable_gtid": false,
9. "security": {
10. "ssl_ca_content": "",
11. "ssl_cert_content": "",
12. "ssl_key_content": "",
13. "cert_allowed_cn": [
14. "string"
15. ]
16. },
17. "purge": {
18. "interval": 3600,
19. "expires": 0,
20. "remain_space": 15
21. },
22. "status_list": [
23. {
24. "source_name": "mysql-replica-01",
25. "worker_name": "worker-1",
26. "relay_status": {
27. "master_binlog": "(mysql-bin.000001, 1979)",
28. "master_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
29. "relay_dir": "./sub_dir",
30. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
31. "relay_catch_up_master": true,
32. "stage": "Running"
33. },
34. "error_msg": "string"
35. }
36. ]
37. }

获取数据源

该接口是一个同步接口，请求成功会返回数据源列表信息。

请求 URI

GET /api/v1/sources/{source-name}

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/sources/source-1?with_status=true' \
3. -H 'accept: application/json'

1. {
2. "source_name": "mysql-01",
3. "host": "127.0.0.1",
4. "port": 3306,
5. "user": "root",
6. "password": "123456",
7. "enable_gtid": false,
8. "enable": false,
9. "flavor": "mysql",
10. "task_name_list": [
11. "task1"
12. ],
13. "security": {
14. "ssl_ca_content": "",
15. "ssl_cert_content": "",
16. "ssl_key_content": "",
17. "cert_allowed_cn": [
18. "string"
19. ]
20. },
21. "purge": {
22. "interval": 3600,
23. "expires": 0,
24. "remain_space": 15
25. },
26. "status_list": [
27. {
28. "source_name": "mysql-replica-01",
29. "worker_name": "worker-1",
30. "relay_status": {
31. "master_binlog": "(mysql-bin.000001, 1979)",
32. "master_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
33. "relay_dir": "./sub_dir",
34. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
35. "relay_catch_up_master": true,
36. "stage": "Running"
37. },
38. "error_msg": "string"
39. }
40. ],
41. "relay_config": {
42. "enable_relay": true,
43. "relay_binlog_name": "mysql-bin.000002",
44. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
45. "relay_dir": "./relay_log"
46. }
47. }

删除数据源

该接口是一个同步接口，请求成功后返回的 Status Code 是 204。

请求 URI

DELETE /api/v1/sources/{source-name}

使用样例

1. curl -X 'DELETE' \
2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01?force=true' \
3. -H 'accept: application/json'

更新数据源

该接口是一个同步接口，请求成功会返回对应的数据源信息。

注意

更新数据源配置时，须确保当前数据源下没有任何正在运行的同步任务。

请求 URI

PUT /api/v1/sources/{source-name}

使用样例

1. curl -X 'PUT' \
2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01' \
3. -H 'accept: application/json' \
4. -H 'Content-Type: application/json' \
5. -d '{
6. "source": {
7. "source_name": "mysql-01",
8. "host": "127.0.0.1",
9. "port": 3306,
10. "user": "root",
11. "password": "123456",
12. "enable_gtid": false,
13. "enable": false,
14. "flavor": "mysql",
15. "task_name_list": [
16. "task1"
17. ],
18. "security": {
19. "ssl_ca_content": "",
20. "ssl_cert_content": "",
21. "ssl_key_content": "",
22. "cert_allowed_cn": [
23. "string"
24. ]
25. },
26. "purge": {
27. "interval": 3600,
28. "expires": 0,
29. "remain_space": 15
30. },
31. "relay_config": {
32. "enable_relay": true,
33. "relay_binlog_name": "mysql-bin.000002",
34. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
35. "relay_dir": "./relay_log"
36. }
37. }
38. }'

1. {
2. "source_name": "mysql-01",
3. "host": "127.0.0.1",
4. "port": 3306,
5. "user": "root",
6. "password": "123456",
7. "enable": true,
8. "enable_gtid": false,
9. "security": {
10. "ssl_ca_content": "",
11. "ssl_cert_content": "",
12. "ssl_key_content": "",
13. "cert_allowed_cn": [
14. "string"
15. ]
16. },
17. "purge": {
18. "interval": 3600,
19. "expires": 0,
20. "remain_space": 15
21. }
22. }

启用数据源

这是一个同步接口，请求成功后会启用此数据源，并批量开始数据迁移任务中依赖该数据源的所有子任务。

请求 URI

POST /api/v1/sources/{source-name}/enable

使用样例

1. curl -X 'POST' \
2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/enable' \
3. -H 'accept: */*' \
4. -H 'Content-Type: application/json'

停用数据源

这是一个同步接口，请求成功后会停用此数据源，并批量停止数据迁移任务中依赖该数据源的所有子任务。

请求 URI

POST /api/v1/sources/{source-name}/disable

使用样例

1. curl -X 'POST' \
2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/disable' \
3. -H 'accept: */*' \
4. -H 'Content-Type: application/json'

获取数据源列表

该接口是一个同步接口，请求成功会返回数据源列表信息。

请求 URI

GET /api/v1/sources

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/sources?with_status=true' \
3. -H 'accept: application/json'

1. {
2. "data": [
3. {
4. "enable_gtid": false,
5. "host": "127.0.0.1",
6. "password": "******",
7. "port": 3306,
8. "purge": {
9. "expires": 0,
10. "interval": 3600,
11. "remain_space": 15
12. },
13. "security": null,
14. "source_name": "mysql-01",
15. "user": "root"
16. },
17. {
18. "enable_gtid": false,
19. "host": "127.0.0.1",
20. "password": "******",
21. "port": 3307,
22. "purge": {
23. "expires": 0,
24. "interval": 3600,
25. "remain_space": 15
26. },
27. "security": null,
28. "source_name": "mysql-02",
29. "user": "root"
30. }
31. ],
32. "total": 2
33. }

获取数据源状态

该接口是一个同步接口，请求成功会返回对应节点的状态信息。

请求 URI

GET /api/v1/sources/{source-name}/status

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/sources/mysql-replica-01/status' \
3. -H 'accept: application/json'

1. {
2. "total": 1,
3. "data": [
4. {
5. "source_name": "mysql-replica-01",
6. "worker_name": "worker-1",
7. "relay_status": {
8. "master_binlog": "(mysql-bin.000001, 1979)",
9. "master_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
10. "relay_dir": "./sub_dir",
11. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
12. "relay_catch_up_master": true,
13. "stage": "Running"
14. },
15. "error_msg": "string"
16. }
17. ]
18. }

对数据源开启 relay-log 功能

这是一个异步接口，请求成功的 Status Code 是 200，可通过获取数据源状态接口获取最新的状态。

请求 URI

POST /api/v1/sources/{source-name}/relay/enable

使用样例

1. curl -X 'POST' \
2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/relay/enable' \
3. -H 'accept: */*' \
4. -H 'Content-Type: application/json' \
5. -d '{
6. "worker_name_list": [
7. "worker-1"
8. ],
9. "relay_binlog_name": "mysql-bin.000002",
10. "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
11. "relay_dir": "./relay_log"
12. }'

对数据源停止 relay-log 功能

这是一个异步接口，请求成功的 Status Code 是 200，可通过获取数据源状态接口获取最新的状态。

请求 URI

POST /api/v1/sources/{source-name}/relay/disable

使用样例

1. curl -X 'POST' \
2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/relay/disable' \
3. -H 'accept: */*' \
4. -H 'Content-Type: application/json' \
5. -d '{
6. "worker_name_list": [
7. "worker-1"
8. ]
9. }'

清除数据源不需要的 relay-log 文件

这是一个异步接口，请求成功的 Status Code 是 200，可通过获取数据源状态接口获取最新的状态。

请求 URI

POST /api/v1/sources/{source-name}/relay/purge

使用样例

1. curl -X 'POST' \
2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/relay/purge' \
3. -H 'accept: */*' \
4. -H 'Content-Type: application/json' \
5. -d '{
6. "relay_binlog_name": "mysql-bin.000002",
7. "relay_dir": "string"
8. }'

更改数据源和 DM-worker 的绑定关系

这是一个异步接口，请求成功的 Status Code 是 200，可通过获取 DM-worker 节点信息接口获取最新的状态。

请求 URI

POST /api/v1/sources/{source-name}/transfer

使用样例

1. curl -X 'POST' \
2. 'http://127.0.0.1:8261/api/v1/sources/mysql-01/transfer' \
3. -H 'accept: */*' \
4. -H 'Content-Type: application/json' \
5. -d '{
6. "worker_name": "worker-1"
7. }'

获取数据源的数据库名列表

该接口是一个同步接口，请求成功会返回对应的列表。

请求 URI

GET /api/v1/sources/{source-name}/schemas

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/sources/source-1/schemas' \
3. -H 'accept: application/json'

1. [
2. "db1"
3. ]

获取数据源的指定数据库的表名列表

该接口是一个同步接口，请求成功会返回对应的列表。

请求 URI

GET /api/v1/sources/{source-name}/schemas/{schema-name}

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/sources/source-1/schemas/db1' \
3. -H 'accept: application/json'

1. [
2. "table1"
3. ]

创建同步任务

这是一个同步接口，请求成功的 Status Code 是 200，请求成功会返回对应的同步任务信息。

请求 URI

POST /api/v1/tasks

使用样例

1. curl -X 'POST' \
2. 'http://127.0.0.1:8261/api/v1/tasks' \
3. -H 'accept: application/json' \
4. -H 'Content-Type: application/json' \
5. -d '{
6. "task": {
7. "name": "task-1",
8. "task_mode": "all",
9. "shard_mode": "pessimistic",
10. "meta_schema": "dm-meta",
11. "enhance_online_schema_change": true,
12. "on_duplicate": "overwrite",
13. "target_config": {
14. "host": "127.0.0.1",
15. "port": 3306,
16. "user": "root",
17. "password": "123456",
18. "security": {
19. "ssl_ca_content": "",
20. "ssl_cert_content": "",
21. "ssl_key_content": "",
22. "cert_allowed_cn": [
23. "string"
24. ]
25. }
26. },
27. "binlog_filter_rule": {
28. "rule-1": {
29. "ignore_event": [
30. "all dml"
31. ],
32. "ignore_sql": [
33. "^Drop"
34. ]
35. },
36. "rule-2": {
37. "ignore_event": [
38. "all dml"
39. ],
40. "ignore_sql": [
41. "^Drop"
42. ]
43. },
44. "rule-3": {
45. "ignore_event": [
46. "all dml"
47. ],
48. "ignore_sql": [
49. "^Drop"
50. ]
51. }
52. },
53. "table_migrate_rule": [
54. {
55. "source": {
56. "source_name": "source-name",
57. "schema": "db-*",
58. "table": "tb-*"
59. },
60. "target": {
61. "schema": "db1",
62. "table": "tb1"
63. },
64. "binlog_filter_rule": [
65. "rule-1",
66. "rule-2",
67. "rule-3",
68. ]
69. }
70. ],
71. "source_config": {
72. "full_migrate_conf": {
73. "export_threads": 4,
74. "import_threads": 16,
75. "data_dir": "./exported_data",
76. "consistency": "auto",
77. "import_mode": "physical",
78. "sorting_dir": "./sort_dir",
79. "disk_quota": "80G",
80. "checksum": "required",
81. "analyze": "optional",
82. "range_concurrency": 0,
83. "compress-kv-pairs": "",
84. "pd_addr": "",
85. "on_duplicate_logical": "error",
86. "on_duplicate_physical": "none"
87. },
88. "incr_migrate_conf": {
89. "repl_threads": 16,
90. "repl_batch": 100
91. },
92. "source_conf": [
93. {
94. "source_name": "mysql-replica-01",
95. "binlog_name": "binlog.000001",
96. "binlog_pos": 4,
97. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
98. }
99. ]
100. }
101. }
102. }'

1. {
2. "name": "task-1",
3. "task_mode": "all",
4. "shard_mode": "pessimistic",
5. "meta_schema": "dm-meta",
6. "enhance_online_schema_change": true,
7. "on_duplicate": "overwrite",
8. "target_config": {
9. "host": "127.0.0.1",
10. "port": 3306,
11. "user": "root",
12. "password": "123456",
13. "security": {
14. "ssl_ca_content": "",
15. "ssl_cert_content": "",
16. "ssl_key_content": "",
17. "cert_allowed_cn": [
18. "string"
19. ]
20. }
21. },
22. "binlog_filter_rule": {
23. "rule-1": {
24. "ignore_event": [
25. "all dml"
26. ],
27. "ignore_sql": [
28. "^Drop"
29. ]
30. },
31. "rule-2": {
32. "ignore_event": [
33. "all dml"
34. ],
35. "ignore_sql": [
36. "^Drop"
37. ]
38. },
39. "rule-3": {
40. "ignore_event": [
41. "all dml"
42. ],
43. "ignore_sql": [
44. "^Drop"
45. ]
46. }
47. },
48. "table_migrate_rule": [
49. {
50. "source": {
51. "source_name": "source-name",
52. "schema": "db-*",
53. "table": "tb-*"
54. },
55. "target": {
56. "schema": "db1",
57. "table": "tb1"
58. },
59. "binlog_filter_rule": [
60. "rule-1",
61. "rule-2",
62. "rule-3",
63. ]
64. }
65. ],
66. "source_config": {
67. "full_migrate_conf": {
68. "export_threads": 4,
69. "import_threads": 16,
70. "data_dir": "./exported_data",
71. "consistency": "auto",
72. "import_mode": "physical",
73. "sorting_dir": "./sort_dir",
74. "disk_quota": "80G",
75. "checksum": "required",
76. "analyze": "optional",
77. "range_concurrency": 0,
78. "compress-kv-pairs": "",
79. "pd_addr": "",
80. "on_duplicate_logical": "error",
81. "on_duplicate_physical": "none"
82. },
83. "incr_migrate_conf": {
84. "repl_threads": 16,
85. "repl_batch": 100
86. },
87. "source_conf": [
88. {
89. "source_name": "mysql-replica-01",
90. "binlog_name": "binlog.000001",
91. "binlog_pos": 4,
92. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
93. }
94. ]
95. }
96. }

获取同步任务

这是一个同步接口，请求成功的 Status Code 是 200。

请求 URI

GET /api/v1/tasks/{task-name}?with_status=true

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1?with_status=true' \
3. -H 'accept: application/json'

1. {
2. "name": "task-1",
3. "task_mode": "all",
4. "shard_mode": "pessimistic",
5. "meta_schema": "dm-meta",
6. "enhance_online_schema_change": true,
7. "on_duplicate": "overwrite",
8. "target_config": {
9. "host": "127.0.0.1",
10. "port": 3306,
11. "user": "root",
12. "password": "123456",
13. "security": {
14. "ssl_ca_content": "",
15. "ssl_cert_content": "",
16. "ssl_key_content": "",
17. "cert_allowed_cn": [
18. "string"
19. ]
20. }
21. },
22. "binlog_filter_rule": {
23. "rule-1": {
24. "ignore_event": [
25. "all dml"
26. ],
27. "ignore_sql": [
28. "^Drop"
29. ]
30. },
31. "rule-2": {
32. "ignore_event": [
33. "all dml"
34. ],
35. "ignore_sql": [
36. "^Drop"
37. ]
38. },
39. "rule-3": {
40. "ignore_event": [
41. "all dml"
42. ],
43. "ignore_sql": [
44. "^Drop"
45. ]
46. }
47. },
48. "table_migrate_rule": [
49. {
50. "source": {
51. "source_name": "source-name",
52. "schema": "db-*",
53. "table": "tb-*"
54. },
55. "target": {
56. "schema": "db1",
57. "table": "tb1"
58. },
59. "binlog_filter_rule": [
60. "rule-1",
61. "rule-2",
62. "rule-3",
63. ]
64. }
65. ],
66. "source_config": {
67. "full_migrate_conf": {
68. "export_threads": 4,
69. "import_threads": 16,
70. "data_dir": "./exported_data",
71. "consistency": "auto",
72. "import_mode": "physical",
73. "sorting_dir": "./sort_dir",
74. "disk_quota": "80G",
75. "checksum": "required",
76. "analyze": "optional",
77. "range_concurrency": 0,
78. "compress-kv-pairs": "",
79. "pd_addr": "",
80. "on_duplicate_logical": "error",
81. "on_duplicate_physical": "none"
82. },
83. "incr_migrate_conf": {
84. "repl_threads": 16,
85. "repl_batch": 100
86. },
87. "source_conf": [
88. {
89. "source_name": "mysql-replica-01",
90. "binlog_name": "binlog.000001",
91. "binlog_pos": 4,
92. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
93. }
94. ]
95. }
96. }

删除同步任务

该接口是一个同步接口，请求成功后返回的 Status Code 是 204。

请求 URI

DELETE /api/v1/tasks/{task-name}

使用样例

1. curl -X 'DELETE' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1' \
3. -H 'accept: application/json'

更新同步任务

该接口是一个同步接口，请求成功会返回对应同步任务的信息。

注意

更新同步任务配置时，须确保该任务处于暂停状态，并已经运行到增量同步的阶段，且仅有部分字段可以更新。

请求 URI

PUT /api/v1/tasks/{task-name}

使用样例

1. curl -X 'PUT' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1' \
3. -H 'accept: application/json' \
4. -H 'Content-Type: application/json' \
5. -d '{
6. "task": {
7. "name": "task-1",
8. "task_mode": "all",
9. "shard_mode": "pessimistic",
10. "meta_schema": "dm-meta",
11. "enhance_online_schema_change": true,
12. "on_duplicate": "overwrite",
13. "target_config": {
14. "host": "127.0.0.1",
15. "port": 3306,
16. "user": "root",
17. "password": "123456",
18. "security": {
19. "ssl_ca_content": "",
20. "ssl_cert_content": "",
21. "ssl_key_content": "",
22. "cert_allowed_cn": [
23. "string"
24. ]
25. }
26. },
27. "binlog_filter_rule": {
28. "rule-1": {
29. "ignore_event": [
30. "all dml"
31. ],
32. "ignore_sql": [
33. "^Drop"
34. ]
35. },
36. "rule-2": {
37. "ignore_event": [
38. "all dml"
39. ],
40. "ignore_sql": [
41. "^Drop"
42. ]
43. },
44. "rule-3": {
45. "ignore_event": [
46. "all dml"
47. ],
48. "ignore_sql": [
49. "^Drop"
50. ]
51. }
52. },
53. "table_migrate_rule": [
54. {
55. "source": {
56. "source_name": "source-name",
57. "schema": "db-*",
58. "table": "tb-*"
59. },
60. "target": {
61. "schema": "db1",
62. "table": "tb1"
63. },
64. "binlog_filter_rule": [
65. "rule-1",
66. "rule-2",
67. "rule-3",
68. ]
69. }
70. ],
71. "source_config": {
72. "full_migrate_conf": {
73. "export_threads": 4,
74. "import_threads": 16,
75. "data_dir": "./exported_data",
76. "consistency": "auto",
77. "import_mode": "physical",
78. "sorting_dir": "./sort_dir",
79. "disk_quota": "80G",
80. "checksum": "required",
81. "analyze": "optional",
82. "range_concurrency": 0,
83. "compress-kv-pairs": "",
84. "pd_addr": "",
85. "on_duplicate_logical": "error",
86. "on_duplicate_physical": "none"
87. },
88. "incr_migrate_conf": {
89. "repl_threads": 16,
90. "repl_batch": 100
91. },
92. "source_conf": [
93. {
94. "source_name": "mysql-replica-01",
95. "binlog_name": "binlog.000001",
96. "binlog_pos": 4,
97. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
98. }
99. ]
100. }
101. }
102. }'

1. {
2. "name": "task-1",
3. "task_mode": "all",
4. "shard_mode": "pessimistic",
5. "meta_schema": "dm-meta",
6. "enhance_online_schema_change": true,
7. "on_duplicate": "overwrite",
8. "target_config": {
9. "host": "127.0.0.1",
10. "port": 3306,
11. "user": "root",
12. "password": "123456",
13. "security": {
14. "ssl_ca_content": "",
15. "ssl_cert_content": "",
16. "ssl_key_content": "",
17. "cert_allowed_cn": [
18. "string"
19. ]
20. }
21. },
22. "binlog_filter_rule": {
23. "rule-1": {
24. "ignore_event": [
25. "all dml"
26. ],
27. "ignore_sql": [
28. "^Drop"
29. ]
30. },
31. "rule-2": {
32. "ignore_event": [
33. "all dml"
34. ],
35. "ignore_sql": [
36. "^Drop"
37. ]
38. },
39. "rule-3": {
40. "ignore_event": [
41. "all dml"
42. ],
43. "ignore_sql": [
44. "^Drop"
45. ]
46. }
47. },
48. "table_migrate_rule": [
49. {
50. "source": {
51. "source_name": "source-name",
52. "schema": "db-*",
53. "table": "tb-*"
54. },
55. "target": {
56. "schema": "db1",
57. "table": "tb1"
58. },
59. "binlog_filter_rule": [
60. "rule-1",
61. "rule-2",
62. "rule-3",
63. ]
64. }
65. ],
66. "source_config": {
67. "full_migrate_conf": {
68. "export_threads": 4,
69. "import_threads": 16,
70. "data_dir": "./exported_data",
71. "consistency": "auto",
72. "import_mode": "physical",
73. "sorting_dir": "./sort_dir",
74. "disk_quota": "80G",
75. "checksum": "required",
76. "analyze": "optional",
77. "range_concurrency": 0,
78. "compress-kv-pairs": "",
79. "pd_addr": "",
80. "on_duplicate_logical": "error",
81. "on_duplicate_physical": "none"
82. },
83. "incr_migrate_conf": {
84. "repl_threads": 16,
85. "repl_batch": 100
86. },
87. "source_conf": [
88. {
89. "source_name": "mysql-replica-01",
90. "binlog_name": "binlog.000001",
91. "binlog_pos": 4,
92. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
93. }
94. ]
95. }
96. }

开始同步任务

这是一个异步接口，请求成功的 Status Code 是 200。可通过获取同步任务状态接口获取最新的任务状态。

请求 URI

POST /api/v1/tasks/{task-name}/start

使用样例

1. curl -X 'POST' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/start' \
3. -H 'accept: */*'

停止同步任务

这是一个异步接口，请求成功的 Status Code 是 200，可通过获取同步任务状态接口获取最新的任务状态。

请求 URI

POST /api/v1/tasks/{task-name}/stop

使用样例

1. curl -X 'POST' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/stop' \
3. -H 'accept: */*'

获取同步任务状态

该接口是一个同步接口，请求成功会返回对应节点的状态信息。

请求 URI

GET /api/v1/tasks/task-1/status

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/status?stage=running' \
3. -H 'accept: application/json'

1. {
2. "total": 1,
3. "data": [
4. {
5. "name": "string",
6. "source_name": "string",
7. "worker_name": "string",
8. "stage": "running",
9. "unit": "sync",
10. "unresolved_ddl_lock_id": "string",
11. "load_status": {
12. "finished_bytes": 0,
13. "total_bytes": 0,
14. "progress": "string",
15. "meta_binlog": "string",
16. "meta_binlog_gtid": "string"
17. },
18. "sync_status": {
19. "total_events": 0,
20. "total_tps": 0,
21. "recent_tps": 0,
22. "master_binlog": "string",
23. "master_binlog_gtid": "string",
24. "syncer_binlog": "string",
25. "syncer_binlog_gtid": "string",
26. "blocking_ddls": [
27. "string"
28. ],
29. "unresolved_groups": [
30. {
31. "target": "string",
32. "ddl_list": [
33. "string"
34. ],
35. "first_location": "string",
36. "synced": [
37. "string"
38. ],
39. "unsynced": [
40. "string"
41. ]
42. }
43. ],
44. "synced": true,
45. "binlog_type": "string",
46. "seconds_behind_master": 0
47. }
48. }
49. ]
50. }

获取同步任务列表

该接口是一个同步接口，请求成功会返回对应的同步任务列表。

请求 URI

GET /api/v1/tasks

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/tasks' \
3. -H 'accept: application/json'

1. {
2. "total": 2,
3. "data": [
4. {
5. "name": "task-1",
6. "task_mode": "all",
7. "shard_mode": "pessimistic",
8. "meta_schema": "dm-meta",
9. "enhance_online_schema_change": true,
10. "on_duplicate": "overwrite",
11. "target_config": {
12. "host": "127.0.0.1",
13. "port": 3306,
14. "user": "root",
15. "password": "123456",
16. "security": {
17. "ssl_ca_content": "",
18. "ssl_cert_content": "",
19. "ssl_key_content": "",
20. "cert_allowed_cn": [
21. "string"
22. ]
23. }
24. },
25. "binlog_filter_rule": {
26. "rule-1": {
27. "ignore_event": [
28. "all dml"
29. ],
30. "ignore_sql": [
31. "^Drop"
32. ]
33. },
34. "rule-2": {
35. "ignore_event": [
36. "all dml"
37. ],
38. "ignore_sql": [
39. "^Drop"
40. ]
41. },
42. "rule-3": {
43. "ignore_event": [
44. "all dml"
45. ],
46. "ignore_sql": [
47. "^Drop"
48. ]
49. }
50. },
51. "table_migrate_rule": [
52. {
53. "source": {
54. "source_name": "source-name",
55. "schema": "db-*",
56. "table": "tb-*"
57. },
58. "target": {
59. "schema": "db1",
60. "table": "tb1"
61. },
62. "binlog_filter_rule": [
63. "rule-1",
64. "rule-2",
65. "rule-3",
66. ]
67. }
68. ],
69. "source_config": {
70. "full_migrate_conf": {
71. "export_threads": 4,
72. "import_threads": 16,
73. "data_dir": "./exported_data",
74. "consistency": "auto",
75. "import_mode": "physical",
76. "sorting_dir": "./sort_dir",
77. "disk_quota": "80G",
78. "checksum": "required",
79. "analyze": "optional",
80. "range_concurrency": 0,
81. "compress-kv-pairs": "",
82. "pd_addr": "",
83. "on_duplicate_logical": "error",
84. "on_duplicate_physical": "none"
85. },
86. "incr_migrate_conf": {
87. "repl_threads": 16,
88. "repl_batch": 100
89. },
90. "source_conf": [
91. {
92. "source_name": "mysql-replica-01",
93. "binlog_name": "binlog.000001",
94. "binlog_pos": 4,
95. "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
96. }
97. ]
98. }
99. }
100. ]
101. }

获取同步任务的同步规则列表

该接口是一个同步接口，请求成功会返回对应同步任务的同步规则列表。

请求 URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/migrate_targets

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/migrate_targets' \
3. -H 'accept: application/json'

1. {
2. "total": 0,
3. "data": [
4. {
5. "source_schema": "db1",
6. "source_table": "tb1",
7. "target_schema": "db1",
8. "target_table": "tb1"
9. }
10. ]
11. }

获取同步任务关联数据源的数据库名列表

该接口是一个同步接口，请求成功会返回对应的列表。

请求 URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/schemas

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas' \
3. -H 'accept: application/json'

1. [
2. "db1"
3. ]

获取同步任务关联数据源的数据表名列表

该接口是一个同步接口，请求成功会返回对应的列表。

请求 URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas/db1' \
3. -H 'accept: application/json'

1. [
2. "table1"
3. ]

获取同步任务关联数据源的数据表的创建语句

该接口是一个同步接口，请求成功会返回对应的创建语句。

请求 URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}/{table-name}

使用样例

1. curl -X 'GET' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas/db1/table1' \
3. -H 'accept: application/json'

1. {
2. "schema_name": "db1",
3. "table_name": "table1",
4. "schema_create_sql": "CREATE TABLE `t1` (`id` int(11) NOT NULL AUTO_INCREMENT,PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin"
5. }

更新同步任务关联数据源的数据表的创建语句

该接口是一个同步接口，返回体的 Status Code 是 200。

请求 URI

POST /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}/{table-name}

使用样例

1. curl -X 'PUT' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/task-1/schemas/db1/table1' \
3. -H 'accept: */*' \
4. -H 'Content-Type: application/json' \
5. -d '{
6. "sql_content": "CREATE TABLE `t1` ( `c1` int(11) DEFAULT NULL, `c2` int(11) DEFAULT NULL, `c3` int(11) DEFAULT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;",
7. "flush": true,
8. "sync": true
9. }'

删除同步任务关联数据源的数据表

该接口是一个同步接口，返回体的 Status Code 是 200。

请求 URI

DELETE /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}/{table-name}

使用样例

1. curl -X 'DELETE' \
2. 'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas/db1/table1' \
3. -H 'accept: */*'