SpaceTL Delete

1. Overview

IoTDB(V1.3.2+) supports configuring the SpaceTL deletion policy through parameters: when the storage space of the last-level storage tier (object storage space is considered unlimited) is insufficient, users can choose whether to trigger the SpaceTL deletion mechanism to free up space.

Note: This mechanism only deletes files in the last storage tier.

2. Parameter Introduction

Parameter Explanation:

• If the last-level storage tier includes object storage, SpaceTL will not be triggered.
• This parameter works in conjunction with the dn_default_space_usage_thresholdsparameter. SpaceTL is triggered when the available space of the last-level storage tier falls below the space_usage_thresholds. By default, deletion begins when disk space usage reaches 85% and continues until usage drops below 85%.

3. Usage Example

In a three-tier storage setup, when the storage space of the last tier reaches 85%, the SpaceTL deletion mechanism is triggered to start deleting files.

1. # data dirs
2. # If this property is unset, system will save the data in the default relative path directory under the IoTDB folder(i.e., %IOTDB_HOME%/data/datanode/data).
3. # If it is absolute, system will save the data in exact location it points to.
4. # If it is relative, system will save the data in the relative path directory it indicates under the IoTDB folder.
5. # If there are more than one directory, please separate them by commas ",".
6. # If tiered storage is enabled, please separate directories of different tiers by semicolons ";".
7. # Use keyword OBJECT_STORAGE to denote object storage media, object storage can only exist on the last tier and must have local dirs in the front tiers.
8. # Note: If data_dirs is assigned an empty string(i.e.,zero-size), it will be handled as a relative path.
9. # effectiveMode: hot_reload
10. # For windows platform
11. # If its prefix is a drive specifier followed by "\\", or if its prefix is "\\\\", then the path is absolute. Otherwise, it is relative.
12. # dn_data_dirs=data\\datanode\\data
13. # For Linux platform
14. # If its prefix is "/", then the path is absolute. Otherwise, it is relative.
15. dn_data_dirs=/data1/data;/data2/data,/data3/data;/data4/data

17. # Default tier TTL. When the survival time of the data exceeds the threshold, it will be migrated to the next tier.
18. # Negative value means the tier TTL is unlimited.
19. # effectiveMode: restart
20. # Datatype: long
21. # Unit: ms
22. tier_ttl_in_ms=86400000;864000000;-1

24. # Disk usage threshold, data will be moved to the next tier when the usage of the tier is higher than this threshold.
25. # If tiered storage is enabled, please separate thresholds of different tiers by semicolons ";".
26. # effectiveMode: hot_reload
27. # Datatype: double(percentage)
28. dn_default_space_usage_thresholds=0.75;0.8;0.85

30. # How to deal with the last tier?s data when its used space has been higher than its dn_default_space_usage_thresholds.
31. # The info of the two policies are as follows:
32. # effectiveMode: hot_reload
33. # 1. NULL: do nothing.
34. # 2. DELETE: delete the oldest data.
35. dn_tier_full_policy=DELETE

In this example, three tiers of storage are configured as follows: