Release 0.291

Highlights

  • Add ALTER TABLE SET PROPERTIES statement. #21495
  • Add catalog and schema level access checks in USE statement. #24182
  • Add single worker execution. To improve latency of tiny queries running on a large cluster, we introduce single worker execution mode: query will only use one node to execute and plan would be optimized accordingly. This feature can be turned on by the configuration property single-node-execution-enabled or the session property single_node_execution_enabled. #24172
  • Add support for the histogram statistic type. #22365
  • Add support for ALTER VIEW RENAME TO operation, including the necessary infrastructure for connector implementations. #23749
  • Improve scheduling for CTE materialization: Now, only the stages containing CTE table scans that reference CTE table write stages are blocked till the write is complete, instead of the entire query being blocked as was the case previously. This is controlled by the session property enhanced_cte_scheduling_enabled (on by default). #24108
  • Add support for time type partitioning in the ORC file format for Iceberg. #24091
  • Enable scale-writers by default. #24107
  • Improve UUID comparisons so they conform to IETF RFC 4122. #23847

Details

General Changes

  • Fix behavior of width_bucket(x, bins) -> bigint() which previously treated all null elements in bins as 0. The function throws an error if it finds a null or non-finite element in bins. #24103
  • Fix bug in strrpos function for multibyte characters. #24226
  • Improve Request Headers in the Authentication Filter Class. #23380
  • Improve coordinator task management performance. #24369
  • Improve efficiency of coordinator when running a large number of tasks. #24288
  • Improve scheduling for CTE materialization: Now, only the stages containing CTE table scans that reference CTE table write stages are blocked till the write is complete, instead of the entire query being blocked as was the case previously. This is controlled by the session property enhanced_cte_scheduling_enabled (on by default). #24108
  • Improve security of internal HTTP endpoints by sanitizing headers before they are used. #24004
  • Add ALTER TABLE SET PROPERTIES statement. #21495
  • Add google_polyline_decode() function to convert Google polyline to Presto ST_Geometry types. #23999
  • Add google_polyline_encode() function to convert Presto ST_Geometry to Google polyline types. #23999
  • Add ClientRequestFilter.java interface in Presto-spi: Client Request Filter. #23380
  • Add a configuration property plan-checker.config-dir to set the configuration directory for PlanCheckerProvider configurations. #23955
  • Add a session property include_values_node_in_connector_optimizer to enable connector optimizer optimize plans with values node. #24227
  • Add native_enforce_join_build_input_partition session property to not enforce input partition for join build. #24163
  • Add catalog and schema level access checks in USE statement. #24182
  • Add delete node in subfield pruning optimizer. #24206
  • Add single worker execution. To improve latency of tiny queries running on a large cluster, we introduce single worker execution mode: query will only use one node to execute and plan would be optimized accordingly. This feature can be turned on by the configuration property single-node-execution-enabled or the session property single_node_execution_enabled. #24172
  • Add support for ORC metadata cache invalidation based on file modification time. #24346
  • Add support for ALTER VIEW RENAME TO operation, including the necessary infrastructure for connector implementations. #23749
  • Add support for the histogram statistic type. #22365
  • Enable scale-writers by default. #24107
  • Update usage of MD5 to SHA256. #23903
  • Improve UUID comparisons so they conform to IETF RFC 4122. #23847

Prestissimo (native Execution) Changes

  • Improve partitioned remote exchanges for wide data sets (more than 500 columns) to use row wise encoding. #23929
  • Add native plan checker to the native sidecar plugin and native endpoint for Velox plan conversion. #23596
  • Add session properties native_spill_prefixsort_enabled, native_prefixsort_normalized_key_max_bytes, and native_prefixsort_min_rows. #24043
  • Add support for automatic scaling of writer threads for partitioned tables. Can be enabled with the native_execution_scale_partitioned_writer_threads_enabled session property. Native execution only. #24155
  • Remove the experimental.table-writer-merge-operator-enabled configuration property and the table_writer_merge_operator_enabled session property. #24145
  • Remove deprecated native_query_trace_task_reg_exp session property from Prestissimo. #24270
  • Add utilizing environment variables as stand in for configuration values. The environment variable value is retrieved and used for the configuration option. #23880
  • Add session property native_query_trace_fragment_id and native_query_trace_shard_id for easier trace control. #24209
  • Add session property: native_table_scan_scaled_processing_enabled. #24284
  • Add session property: native_table_scan_scale_up_memory_usage_ratio. #24284
  • Add session property native_scaled_writer_rebalance_max_memory_usage_ratio. #24261
  • Add session property native_scaled_writer_max_partitions_per_writer. #24261
  • Add session property native_scaled_writer_min_partition_processed_bytes_rebalance_threshold. #24261
  • Add session property native_scaled_writer_min_processed_bytes_rebalance_threshold. #24261

Security Changes

BigQuery Connector Changes

  • Fix SELECT statements failing with NoClassDefFoundError: io/grpc/CallCredentials2. #23957

Cassandra Connector Changes

Clickhouse Connector Changes

  • Add DateTime64 type support. #24344

Delta Connector Changes

  • Add catalog.system.invalidate_metastore_cache procedure to invalidate all, or portions of, the metastore cache. #23401

Hive Connector Changes

  • Add catalog.system.invalidate_metastore_cache procedure to invalidate all, or portions of, the metastore cache. #23401

Hudi Connector Changes

  • Add catalog.system.invalidate_metastore_cache procedure to invalidate all, or portions of, the metastore cache. #23401

Iceberg Connector Changes

  • Improve performance of scan planning in Iceberg Connector. #24376
  • Add catalog.system.invalidate_metastore_cache procedure to invalidate all, or portions of, the metastore cache. #23401
  • Add configuration property iceberg.rest.nested.namespace.enabled to support nested namespaces in Iceberg’s REST Catalog. Defaults to true. #24083
  • Add support for ALTER VIEW RENAME TO. #23749
  • Add support of view for Iceberg connector when configured with REST and NESSIE. #23793
  • Add support of specifying table location on creation for Iceberg connector when configured with REST and NESSIE. #24218
  • Remove in-memory hive metastore cache in Iceberg connector. #24326
  • Add support for time type partitioning in the ORC file format for Iceberg. #24091
  • Add testing for partitioning using time type in Iceberg. #24091

Memory Connector Changes

  • Add support for ALTER VIEW RENAME TO. #23749

MongoDB Connector Changes

SPI Changes

  • Improve ExpressionOptimizer#optimize method with a variable resolver to return RowExpression. #24287
  • Add WindowNode, JoinNode, SemiJoinNode, MergeJoinNode, and SpatialJoinNode to the SPI. #23976
  • Add Delete, TableWriter, and TableFinish node to SPI. #24088
  • Add SemiJoin, Join, TableWriter, Delete, and TableFinish node to connector optimizer. #24154
  • Add a partition by attribute to specify the scope of sort node. #24095
  • Remove ConnectorJoinNode from the SPI. JoinNode can now be used instead. #23976
  • Remove experimental getPreferredShuffleLayout methods from the connector metadata in favor of existing getNewTableLayout, getInsertLayout methods #24106
  • Modify the signature of PlanCheckerProviderFactory.create to pass in a map of configuration properties and replace SimplePlanFragmentSerde with a PlanCheckerProviderContext. #23955

UI Changes

  • Add support for BigInt data type in the SQL Client on Presto UI on supported browsers. See compatibility for the supported browsers. #24336

Credits

Abe Varghese, Abhisek Saikia, Ajay Kharat, Amit Dutta, Anant Aneja, Andrii Rosa, Ann-Megha-Rajesh1, Arjun Gupta, Bryan Cutler, Christian Zentgraf, Deepa-George, Deepak Majeti, Deepak Mehra, Denodo Research Labs, Dilli-Babu-Godari, Emanuel F., Emily Chan, Facebook Community Bot, Feilong Liu, Ge Gao, Georg Schäfer, Hazmi, Heidi Han, Jacob Khaliqi, Jalpreet Singh Nanda (:imjalpreet), Jalpreet Singh Nanda (:imjalpreet), Jeremy Quirke, Jialiang Tan, Jiaqi Zhang, Joe Abraham, Ke, Kevin Tang, Konjac Huang, Leonid Chistov, Linsong Wang, Luís Fernandes, MariamAlmesfer, Matthew Peveler, Minhan Cao, Natasha Sehgal, Naveen Mahadevuni, Nikhil Collooru, Nishitha-Bhaskaran, NivinCS, Pramod Satya, Pratik Joseph Dabre, Rebecca Schlussel, Reetika Agrawal, Richard Barnes, Serge Druzkin, Sergey Pershin, Shakyan Kushwaha, Shang Ma, Shijin, Steve Burnett, SthuthiGhosh9400, Sumi Mathew, Tim Meehan, Xiao Du, Xiaoxuan Meng, Yihong Wang, Ying, Yuanda (Yenda) Li, Zac Blanco, Zac Wen, aditi-pandit, auden-woolfson, dependabot[bot], jaystarshot, pratyakshsharma, unidevel, wangd, wypb, zuyu