Release 0.295

Breaking Changes

  • Add all inline SQL invoked functions into a new plugin presto-sql-invoked-functions-plugin. The following functions were moved: replace_first, trail, key_sampling_percent, no_values_match, no_keys_match, any_values_match, any_keys_match, all_keys_match, map_remove_null_values, map_top_n_values, map_top_n_keys, map_top_n, map_key_exists, map_keys_by_top_n_values, map_normalize, array_top_n, remove_nulls, array_sort_desc, array_min_by, array_max_by, array_least_frequent, array_has_duplicates, array_duplicates, array_frequency, array_split_into_chunks, array_average, array_intersect. See #26025 and presto-sql-helpers/README.md. #25818
  • Upgrade Presto to require Java 17. The Presto client and Presto-on-Spark remain Java 8-compatible. Presto now requires a Java 17 VM to run both coordinator and workers. #24866

Highlights

  • Add OAuth2 support for WebUI and JDBC Presto Client. #24443
  • Add a new configuration property query.max-queued-time to specify maximum queued time for a query before killing it. This can be overridden by the query_max_queued_time session property. #25589
  • Add spatial join support for native execution. #25823
  • Add support for mutual TLS (mTLS) authentication in the Arrow Flight connector. #25388
  • Add support for GEOMETRY type in the PostgreSQL connector. #25240
  • Add documentation about the Presto Release Process and Version Support. #25742
  • Add support for configuring http2 server on worker for communication between coordinator and workers. To enable, set the configuration property http-server.http2.enabled to true. #25708
  • Add support for cross-cluster query retry. Failed queries can be automatically retried on a backup cluster by providing the retry URL and expiration time as query parameters. #25625

Details

General Changes

  • Fix localtime and current_time issues in legacy timestamp semantics. #25985
  • Fix a bug where map(varchar, json) does not canonicalize values. See Map Functions and Operators. #24232
  • Fix add exchange and add local exchange optimizers to simplify query plans with unique columns. #25882
  • Fix failure when preparing statements or creating views that contain a quoted reserved word as a table name. #25528
  • Fix weak cipher mode usage during spilling by switching to a stronger algorithm. #25603
  • Improve DELETE on columns with special characters in their names. #25737
  • Improve the protocol efficiency of the C++ worker by supporting thrift codec for connector-specific data. #25595
  • Improve the protocol efficiency of coordinator by supporting thrift codec for connector-specific data. #25242
  • Add Scale and Precision columns to SHOW COLUMNS to get the respective scale of the decimal value and precision of numerical values. A Length column is introduced to get the length of CHAR and VARCHAR fields. #25351
  • Add Cache-Control header with max-age to statement API responses. #25433
  • Add X-Presto-Retry-Query header to identify queries that are being retried on a backup cluster. #25625
  • Add presto-sql-helpers directory for inlined SQL invoked function plugins with plugin loading rules. #26025
  • Add a new plugin presto-native-sql-invoked-functions-plugin that contains all inline SQL functions, except those with overridden native implementations. #25870
  • Add max_serializable_object_size session property to change the maximum serializable object size at the coordinator. #25616
  • Add all inline SQL invoked functions into a new plugin presto-sql-invoked-functions-plugin. The following functions were moved: replace_first, trail, key_sampling_percent, no_values_match, no_keys_match, any_values_match, any_keys_match, all_keys_match, map_remove_null_values, map_top_n_values, map_top_n_keys, map_top_n, map_key_exists, map_keys_by_top_n_values, map_normalize, array_top_n, remove_nulls, array_sort_desc, array_min_by, array_max_by, array_least_frequent, array_has_duplicates, array_duplicates, array_frequency, array_split_into_chunks, array_average, array_intersect. See #26025 and presto-sql-helpers/README.md. #25818
  • Add array_sort(array, function) support for key-based sorting. See Array Functions and Operators. #25851
  • Add array_sort_desc(array, function) support for key-based sorting. See Array Functions and Operators. #25851
  • Add OAuth2 support for WebUI and JDBC Presto Client. #24443
  • Add a new configuration property query.max-queued-time to specify maximum queued time for a query before killing it. This can be overridden by the query_max_queued_time session property. #25589
  • Add support for BuiltInFunctionKind enum parameter in BuiltInFunctionHandle’s JSON constructor creator. #25821
  • Add support for configuring http2 server on worker for communication between coordinator and workers. To enable, set the configuration property http-server.http2.enabled to true. #25708
  • Add support for cross-cluster query retry. Failed queries can be automatically retried on a backup cluster by providing the retry URL and expiration time as query parameters. #25625
  • Add support for using a Netty client to do HTTP communication between coordinator and worker. To enable, set the configuration property reactor.netty-http-client-enabled to true on the coordinator. #25573
  • Add test methods assertStartTransaction and assertEndTransaction to better support non-autocommit transaction testing scenarios. #25053
  • Add a database-based session property manager. See Session Property Managers. #24995
  • Add support to use the MariaDB Java client with a MySQL based function server. #25698
  • Add support and plumbing for DELETE queries to identify modified partitions as outputs in the generated QueryIOMetadata. #26134
  • Add reporting lineage details for columns which are created or inserted to the event listener. #25913
  • Upgrade Presto to require Java 17. The Presto client and Presto-on-Spark remain Java 8-compatible. Presto now requires a Java 17 VM to run both coordinator and workers. #24866
  • Update Provisio packaging to split plugin packaging into plugins and native-plugin directory. #25984
  • Update Provisio plugin to package the memory connector plugin under the native-plugin directory. #26044
  • Update to preserve table name quoting in the output of SHOW CREATE VIEW. #25528

Prestissimo (Native Execution) Changes

  • Fix an issue when processing multiple splits for the same plan node from multiple sources. #26031
  • Fix constant folding to handle deeply nested call statements. #26088
  • Fix constant folding in sidecar enabled clusters. #26125
  • Improve native execution of sidecar query analysis by enabling Presto built-in functions. #25135
  • Add the parameterized VARCHAR type in the list of supported types in NativeTypeManager. #26003
  • Add session property native_index_lookup_join_max_prefetch_batches which controls the max number of input batches to prefetch to do index lookup ahead. If it is set to 0, then process one input batch at a time. #25886
  • Add session property native_index_lookup_join_split_output. If set to true, then the index join operator might split output for each input batch based on the output batch size control. Otherwise, it tries to produce a single output for each input batch. #25886
  • Add session property native_unnest_split_output. If this is set to true, then the unnest operator might split output for each input batch based on the output batch size control. Otherwise, it produces a single output for each input batch. #25886
  • Add session properties native_debug_memory_pool_name_regex and native_debug_memory_pool_warn_threshold_bytes to help debug memory pool usage patterns. 25750
  • Add limited use of the CHAR(N) type with PrestoC++. When CHAR(N) is used in a query it is mapped to the Velox VARCHAR type. As a result CHAR(N) semantics are not preserved in the exectution engine. #25843
  • Add spatial join support for native execution. #25823
  • Rename native_query_trace_node_ids to native_query_trace_node_id to provide a single plan node id for tracing. #25684
  • Update coordinator behavior to validate sidecar function signatures against plugin loaded function signatures at startup. #25919

Security Changes

JDBC Driver Changes

  • Add DECIMAL type support to query builder. #25699

Web UI Changes

  • Fix the query id tooltip being displayed at an incorrect position. #25809

Arrow Flight Connector Changes

BigQuery Connector Changes

  • Fix query failures on SELECT operations by aligning BigQuery v1beta1 with protobuf-java 3.25.8, preventing runtime incompatibility with protobuf 4.x. #25805
  • Add support for case-sensitive identifiers in BigQuery. To enable, set the configuration property case-sensitive-name-matching=true in the catalog file. #25764

Cassandra Connector Changes

  • Add support to read TUPLE type as a Presto VARCHAR. #25516

ClickHouse Connector Changes

  • Add support for case-sensitive identifiers in Clickhouse. To enable, set the configuration property case-sensitive-name-matching=true in the catalog file. #25863

Delta Lake Connector Changes

  • Upgrade to Hadoop 3.4.1. #24799

Hive Connector Changes

  • Fix Hive connector to ignore unsupported table formats when querying system.jdbc.columns to prevent errors. #25779
  • Add session property hive.orc_use_column_names to toggle the accessing of columns based on the names recorded in the ORC file rather than their ordinal position in the file. #25285
  • Upgrade to Hadoop 3.4.1. #24799

Hudi Connector Changes

  • Upgrade to Hadoop 3.4.1. #24799

Iceberg Connector Changes

  • Fix null pointer exception (NPE) error in getViews API call when a schema is not provided. #25695
  • Fix implementation of commit to do one operation as opposed to two. #25615
  • Fix Iceberg connector rename column failed if the column is used as source column of non-identity transform. #25697
  • Improve Iceberg’s apply_changelog function by migrating it from the global namespace to the connector-specific namespace. The function is now available as iceberg.system.apply_changelog() instead of apply_changelog(). #25871
  • Improve the property mechanism to enable a property to accept and process property values of multiple types. #25862
  • Add Iceberg bucket scalar function. #25951
  • Add iceberg.engine.hive.lock-enabled configuration to disable Hive locks. #25615
  • Add support for specifying multiple transforms when adding a column. #25862
  • Upgrade Iceberg version to 1.8.1. #25999
  • Upgrade Nessie to version 0.95.0. #25593
  • Upgrade to Hadoop 3.4.1. #24799
  • Update to implement ConnectorMetadata::finishDeleteWithOutput(). #26134

Kudu Connector Changes

  • Update to implement ConnectorMetadata::finishDeleteWithOutput(). #26134

MongoDB Connector Changes

  • Add support for case-sensitive identifiers in MongoDB. To enable, set the configuration property case-sensitive-name-matching=true in the catalog file. #25853
  • Upgrade MongoDB java driver to 3.12.14. #25436

PostgreSQL Connector Changes

Redis Connector Changes

  • Add changes to enable TLS support. #25373

SPI Changes

  • Add a new getSqlInvokedFunctions SPI in Presto, which only supports SQL invoked functions. #25597
  • Add a new ConnectorMetadata::finishDeleteWithOutput() method, returning Optional<ConnectorOutputMetadata>. This allows connectors implementing DELETE to identify partitions modified in queries, which can be important for tracing lineage. #26134
  • Add AuthenticatorNotApplicableException to prevent irrelevant authenticator errors from being returned to clients. #25606
  • Deprecate the existing ConnectorMetadata::finishDelete() method. By default, the new finishDeleteWithOutput() method delegates to the existing finishDelete() method, and returns Optional.empty(). This allows existing connectors to continue working without changes. #26134

Documentation Changes

Credits

Abhash Jain, Adrian Carpente (Denodo), Amit Dutta, Amritanshu Darbari, Anant Aneja, Andrew Xie, Arjun Gupta, Artem Selishchev, Bryan Cutler, Christian Zentgraf, Dilli-Babu-Godari, Elbin Pallimalil, Facebook Community Bot, Feilong Liu, Gary Helmling, Ge Gao, Hazmi, HeidiHan0000, Jalpreet Singh Nanda (:imjalpreet), James Gill, Jay Narale, Jialiang Tan, Joe Abraham, Joe O’Hallaron, Karthikeyan Natarajan, Ke Wang, Ke Wang, Kevin Tang, Kewen Wang, Krishna Pai, Mahadevuni Naveen Kumar, Maria Basmanova, Mariam Almesfer, Matt Karrmann, Miguel Blanco Godón, Natasha Sehgal, Naveen Nitturu, Nidhin Varghese, Nikhil Collooru, Nishitha-Bhaskaran, PRASHANT GOLASH, Ping Liu, Pradeep Vaka, Pramod Satya, Prashant Sharma, Pratik Joseph Dabre, Raaghav Ravishankar, Rebecca Schlussel, Rebecca Whitworth, Reetika Agrawal, Richard Barnes, Sayari Mukherjee, Sergey Pershin, Shahim Sharafudeen, Shang Ma, Shijin, Shrinidhi Joshi, Steve Burnett, Sumi Mathew, Timothy Meehan, Valery Mironov, Vamsi Karnika, Vivian Hsu, Wei He, Xiaoxuan Meng, Xin Zhang, Yihong Wang, Ying, Zac Blanco, Zac Wen, abhinavmuk04, aditi-pandit, adkharat, aspegren_david, auden-woolfson, beinan, dnskr, ericyuliu, haneel-kumar, j-sund, juwentus1234, lingbin, mehradpk, mohsaka, pratik.pugalia@gmail.com, pratyakshsharma, singcha, unidevel, wangd, yangbin09