内置时间变量

用户希望在写代码时，对时间的格式要求千变万化,已有的Linkis自定义变量目前还不足以支撑这些需求。另外,已有的时间运算-1有些表示减一个月,有些则是减一天,用户很容易混淆

  • 其他日期内置变量都是相对run_date计算出来
  • 支持Pattern格式时间且用户可以自行随意指定
  • 支持±y/±M/±d/±H等等

Pattern格式对照表:

Letter Date or Time Component Presentation Examples
G Era designator Text AD
y Year Year 1996; 96
Y Week year Year 2009; 09
M Month in year Month July; Jul; 07
w Week in year Number 27
W Week in month Number 2
D Day in year Number 189
d Day in month Number 10
F Day of week in month Number 2
E Day name in week Text Tuesday; Tue
e Day number of week (1 = Monday, …, 7 = Sunday) Number 1
a Am/pm marker Text PM
H Hour in day (0-23) Number 0
k Hour in day (1-24) Number 24
K Hour in am/pm (0-11) Number 0
h Hour in am/pm (1-12) Number 12
m Minute in hour Number 30
s Second in minute Number 55
S Millisecond Number 978
z Time zone General time zone Pacific Standard Time; PST; GMT-08:00
Z Time zone RFC 822 time zone -0800
X Time zone ISO 8601 time zone -08; -0800; -08:00

总体设计和技术架构参照Linkis自定义变量

  • Linkis支持的变量类型分为自定义变量(不做赘述)和系统内置变量，自定变量日期支持+-。
  • 其中+-是先对linkis内置参数run_date进行运算,然后将%之前的pattern字段进行替换,非Pattern字符不支持运算替换。

可以根据自己的喜好/业务实际情况定义需要动态渲染的参数

variable result
&{yyyy-01-01} 2021-01-01
&{yyyy-01-01%-2y} 2019-01-01
&{yyyy-MM-01%-2M} 2021-02-01
&{yyyy-MM-dd%-2d} 2021-03-31
&{yyyy MM ——- HH%-1H} 2021 04 ——- 14
&{yyyyMMdd%-1d} 20210401
&{yyyyMM01%-1M} 20210301
&{HH%-1H} 14
  • 例子1: sql
1. SELECT * FROM hive.tmp.fund_nav_histories
2. WHERE dt <= DATE_FORMAT(DATE_ADD('day', -1, DATE(Date_parse('&{yyyyMMdd%-1d}', '%Y%m%d'))), '%Y%m%d')

渲染后

1. SELECT * FROM hive.tmp.fund_nav_histories
2. WHERE dt <= DATE_FORMAT(DATE_ADD('day', -1, DATE(Date_parse('20220705', '%Y%m%d'))), '%Y%m%d')
  • 例子2: shell
1. aws s3 ls  s3://***/ads/tmp/dws_member_active_detail_d_20210601_20211231/pt=&{yyyyMMdd%-1d}/

渲染后

1. aws s3 ls  s3://***/ads/tmp/dws_member_active_detail_d_20210601_20211231/pt=20220705/
  • 例子3: datax json
1. {
2. "job": {
3. "setting": {
4. "speed": {
5. "channel": 1
6. }
7. },
8. "content": [
9. {
10. "reader": {
11. "name": "s3reader",
12. "parameter": {
13. "bucket": "**************",
14. "path": [
15. "ads/tmp/ccass_tm_announcements/&{yyyyMMdd%-1d}/"
16. ],
17. "stored": "parquet",
18. "compression": "NONE",
19. "column": [
20. {
21. "index": 0,
22. "type": "int"
23. },
24. {
25. "index": 1,
26. "type": "string",
27. "constant": "&{yyyyMMdd%-1d}"
28. }
29. ]
30. }
31. },
32. "writer": {
33. "name": "streamwriter",
34. "parameter": {
35. "print": true
36. }
37. }
38. }
39. ]
40. }
41. }

渲染后

1. {
2. "job": {
3. "setting": {
4. "speed": {
5. "channel": 1
6. }
7. },
8. "content": [
9. {
10. "reader": {
11. "name": "s3reader",
12. "parameter": {
13. "bucket": "**************",
14. "path": [
15. "ads/tmp/ccass_tm_announcements/20220705/"
16. ],
17. "stored": "parquet",
18. "compression": "NONE",
19. "column": [
20. {
21. "index": 0,
22. "type": "int"
23. },
24. {
25. "index": 1,
26. "type": "string",
27. "constant": "20220705"
28. }
29. ]
30. }
31. },
32. "writer": {
33. "name": "streamwriter",
34. "parameter": {
35. "print": true
36. }
37. }
38. }
39. ]
40. }
41. }
  • 例子4:python
1. print(&{yyyyMMdd%-1d})

渲染后

1. 20220705