Release 0.293

Highlights

  • Fix ROLLBACK statement to ensure it successfully aborts non-auto commit transactions corrupted by failed statements. #23247
  • Improve coordinator performance by introducing Thrift serialization. #25079 and #25020
  • Improve performance of ORDER BY queries on single node execution. #25022
  • Add authentication capabilities to Presto router. #24407
  • Add coordinator health checks to Presto router. #24449
  • Add support for custom scheduler plugin in the Presto Router. #24439
  • Add DDL SQL support for SHOW CREATE SCHEMA. #24356
  • Add longest_common_prefix(string1, string2) -> varchar() string function. #24891
  • Add support for row filtering and column masking in access control. #24277
  • Add security-related headers to the static resources served from the Presto Router UI, including: Content-Security-Policy, X-Content-Type-Options. See reference docs Content-Security-Policy and X-Content-Type-Options. #25165
  • Add support for SSL/TLS encryption for HMS. #24745
  • Add support for the procedure <catalog-name>.system.invalidate_manifest_file_cache() for ManifestFile cache invalidation in Iceberg. #24831
  • Add support for JSON type in MongoDB. #25089
  • Add support for GEOMETRY type in the MySQL connector. #24996
  • Add a display for number of queued and running queries for each Resource Group subgroup in the UI. #24830
  • Add runtime metrics collection for S3 Filesystem. #24554

Details

General Changes

  • Fix ROLLBACK statement to ensure it successfully aborts non-auto commit transactions corrupted by failed statements. #23247
  • Fix a bug in left join to semi join optimizer which leads to filter source variable not found error. #25111
  • Fix a bug where a mirrored arrays_overlap(x, y) -> boolean() function does not return the correct value. #23845
  • Fix returning incorrect results from the second(x) -> bigint() UDF when a timestamp is in a time zone with an offset that is at the granularity of seconds. #25090
  • Fix issue with loading Redis HBO provider. #24835
  • Improve memory usage of readers of complex type columns. #24912
  • Improve the efficacy of ACL checks by delaying them until after SQL view processing. #24955 and #24927
  • Improve coordinator performance by introducing Thrift serialization. #25079 and #25020
  • Improve performance of operator stats reporting. #24921
  • Improve performance of ORDER BY queries on single node execution. #25022
  • Improve query plans by converting table scans without data to empty values nodes. #25155
  • Improve performance of LOJ + IS NULL queries by adding distinct on right side of semi-join for it. #24884
  • Add DDL SQL support for SHOW CREATE SCHEMA. #24356
  • Add configuration property hive.metastore.catalog.name to pass catalog names to the metastore, enabling catalog-based schema management and filtering. #24235
  • Add cosine_similarity(x, y) -> double() for array arguments. #25056
  • Add type rewrite support for native execution. This feature can be enabled by native-execution-type-rewrite-enabled configuration property and native_execution_type_rewrite_enabled session property. #24916
  • Add session property query_client_timeout to configure how long a query can run without contact from the client application, such as the CLI, before it is abandoned. #25210
  • Add longest_common_prefix(string1, string2) -> varchar() string function. #24891
  • Replace exchange.compression-enabled, fragment-result-cache.block-encoding-compression-enabled, experimental.spill-compression-enabled with exchange.compression-codec, fragment-result-cache.block-encoding-compression-codec to enable compression codec configurations. Supported codecs include GZIP, LZ4, LZO, SNAPPY, ZLIB and ZSTD. #24670
  • Replace dependency from PostgreSQL to redshift-jdbc42 to address CVE-2024-1597, CVE-2022-31197, and CVE-2020-13692. #25106
  • Upgrade netty version to 4.1.119.Final. #24971

Prestissimo (Native Execution) Changes

  • Improve batch shuffle performance by doing sorted serialization. #24953
  • Improve batch shuffle sorted serialization by using appropriate sorting key values for each buffer. #25015
  • Add type rewrite support for native execution. This feature can be enabled by native-execution-type-rewrite-enabled configuration property and native_execution_type_rewrite_enabled session property. #24916
  • Add runtime metrics collection for S3 Filesystem. #24554
  • Add session property native_request_data_sizes_max_wait_sec for the maximum wait time for exchange long poll requests in seconds. #24918
  • Add session property native_streaming_aggregation_eager_flush to control if streaming aggregation should flush its output rows as quickly as it can. #24947
  • Add session property native_debug_memory_pool_name_regex to trace allocations of memory pools matching the regex. #24833
  • Replace using native functions with Java functions for creating failure functions when native execution is enabled. #24792
  • Remove worker configuration property register-test-functions. #24853

Router Changes

  • Add support for custom scheduler plugin in the Presto Router. #24439
  • Fix Round Round robin scheduler candidate cluster index, by adding group specific index. #24580
  • Add authentication capabilities to Presto router. #24407
  • Add coordinator health checks to Presto router. #24449
  • Add counter JMX metrics to Presto router. #24449

Security Changes

Web UI Changes

  • Add a display for number of queued and running queries for each Resource Group subgroup in the UI. #24830

Delta Lake Connector Changes

  • Fix a bug where after an incremental update with null values is made, reads start timing out. #24920

Elasticsearch Connector Changes

Hive Connector Changes

  • Add support for Web Identity authentication in S3 security mapping with the hive.s3.webidentity.enabled property. #24645
  • Add support for SSL/TLS encryption for HMS with configuration properties hive.metastore.thrift.client.tls.enabled, hive.metastore.thrift.client.tls.keystore-path, hive.metastore.thrift.client.tls.keystore-password, and hive.metastore.thrift.client.tls.truststore-password. #24745
  • Replace listObjects with listObjectsV2 in PrestoS3FileSystem listPrefix. #24794

Iceberg Connector Changes

  • Fix to pass full session to avoid Unknown connector errors using the Nessie catalog. #24803
  • Add support for the procedure <catalog-name>.system.invalidate_manifest_file_cache() for ManifestFile cache invalidation in Iceberg. #24831
  • Add support for the procedure <catalog-name>.system.invalidate_statistics_file_cache() for StatisticsFile cache invalidation in Iceberg. #24831
  • Add support for bucket transform for columns of type TimeType in Iceberg table. #24829
  • Replace RowDelta with AppendFiles for insert-only statements such as INSERT and CTAS. #24989

JDBC Connector Changes

  • Add list-schemas-ignored-schemas configuration property for JDBC connectors. #24994

Kafka Connector Changes

  • Add support for optional Apache Kafka SASL. #24798

MongoDB Connector Changes

MySQL Connector Changes

SQL Server Connector Changes

  • Upgrade SQL Server driver to version 12.8.1 to support NTLM authentication. See Authentication. This is a breaking change for existing connections, as the driver sets the encrypt property to true by default. To connect to a non-SSL SQL Server instance, you must set encrypt=false in your connection configuration to avoid connectivity issues. #24686

Documentation Changes

Credits

Akinori Musha, Amit Dutta, Anant Aneja, Andrew Xie, Andrii Rosa, Anurag Dwivedi, Arjun Gupta, Bryan Cutler, Chen Yang, Christian Zentgraf, Deepak Majeti, Deepak Mehra, Denodo Research Labs, Elbin Pallimalil, Emily (Xuetong) Sun, Ethan Zhang, Facebook Community Bot, Feilong Liu, Gary Helmling, Haritha Koloth, Hazmi, HeidiHan0000, Heng Xiao, Jacob Khaliqi, James Petty, Jay Narale, Jim Simon, Jimmy Lu, Joe Abraham, Ke Wang, Ke Wang, Kevin Tang, Kevin Wilfong, Krishna Pai, Li Zhou, Linsong Wang, Mariam Almesfer, Miguel Blanco Godón, Najib Adan, Natasha Sehgal, Nidhin Varghese, Nikhil Collooru, Nivin C S, Pradeep Vaka, Pramod Satya, Prashant Golash, Pratik Joseph Dabre, Rebecca Schlussel, Reetika Agrawal, Samuel Majoros, Sayari Mukherjee, Serge Druzkin, Sergey Pershin, Shahim Sharafudeen, Shang Ma, Shelton Cai, Shijin, Steve Burnett, Tim Meehan, Xiao Du, Xiaoxuan Meng, Xin Zhang, Yihong Wang, Ying, Yuanda (Yenda) Li, Zac Blanco, Zac Wen, aditi-pandit, auden-woolfson, ebonnal, jp-sivaprasad, lukmanulhakkeem, mecit-san, mima0000, mohsaka, namya28, tanjialiang, vhsu14, wangd, wraymo