Change log

Releases

e5.1.0

Enhancements

  • #11035 Change log - 图1 (opens new window) Upgraded Cassandra driver to avoid username and password leakage in data bridge logs.
  • #10584 Change log - 图2 (opens new window) Added log level configuration to SSL communication
  • #10678 Change log - 图3 (opens new window) Optimized counter increment calls to avoid work if increment is zero.
  • #10690 Change log - 图4 (opens new window) Added a retry mechanism to webhook bridge that attempts to improve throughput. This optimization retries request failures without blocking the buffering layer, which can improve throughput in situations of high messaging rate.
  • #10702 Change log - 图5 (opens new window) Introduced a more straightforward configuration option keepalive_multiplier and deprecate the old keepalive_backoff configuration. After this enhancement, EMQX checks the client’s keepalive timeout status period by multiplying the “Client Requested Keepalive Interval” with keepalive_multiplier.
  • #10698 Change log - 图6 (opens new window) Optimized memory usage when accessing the configuration during runtime.
  • #10778 Change log - 图7 (opens new window) Refactored Pulsar Producer bridge to avoid leaking resources in case bridge crashed during initialization phase.
  • #10813 Change log - 图8 (opens new window) Refactored Kafka Producer and Consumer bridges to avoid leaking resources in case bridge crashed during initialization phase.
  • #10858 Change log - 图9 (opens new window) A new utility function timezone_to_offset_seconds/1 has been added to the rule engine SQL language. This function converts a timezone string (for example, “+02:00”, “Z” and “local”) to the corresponding offset in seconds.
  • #10841 Change log - 图10 (opens new window) Added a schema validation to ensure message key is not empty when “key_dispatch” strategy is selected in Kafka and Pulsar Producer bridges.
  • #10754 Change log - 图11 (opens new window) The MQTT bridge has been enhanced to utilize connection pooling and leverage available parallelism, substantially improving throughput. As a consequence, single MQTT bridge now uses a pool of clientids to connect to the remote broker.
  • #10782 Change log - 图12 (opens new window) Added a new deliver_rate option to the retainer configuration, which can limit the maximum delivery rate per session in the retainer.
  • #10877 Change log - 图13 (opens new window) Upgraded RocketMQ driver to enhance security for sensitive data.
  • #10598 Change log - 图14 (opens new window) Provided a callback method of Unary type in ExProto to avoid possible message disorder issues.
  • #10895 Change log - 图15 (opens new window) Refactored most of the bridges to avoid resource leaks in case bridge crashed during initialization phase.
  • #10790 Change log - 图16 (opens new window) Optimized access to configuration in runtime by reducing overhead of reading configuration per zone.
  • #10892 Change log - 图17 (opens new window) Added the requirement for setting SID or Service Name in Oracle Database bridge creation.
  • #10910 Change log - 图18 (opens new window) The data bridge resource option auto_restart_interval was deprecated in favor of health_check_interval, and request_timeout was renamed to request_ttl. Also, the default request_ttl value went from 15 seconds to 45 seconds. The previous existence of both auto_restart_interval and health_check_interval was a source of confusion, as both parameters influenced the recovery of data bridges under failures. An inconsistent configuration of those two parameters could lead to messages being expired without a chance to retry. Now, health_check_interval is used both to control the interval of health checks that may transition the data bridge into disconnected or connecting states, as well as recovering from disconnected.
  • #10929 Change log - 图19 (opens new window) Upgraded Erlang/OTP to 25.3.2-1.
  • #10909 Change log - 图20 (opens new window) Removed the deprecated HTTP APIs for gateways.
  • #10908 Change log - 图21 (opens new window) Refactored the RocketMQ bridge to avoid resources leaks in case bridge crashed during initialization phase.
  • #10924 Change log - 图22 (opens new window) Refactored Influxdb bridge connector to avoid resource leaks in case bridge crashed during initialization phase.
  • #10944 Change log - 图23 (opens new window) Improved the GCP PubSub bridge to avoid a potential issue that the bridge could fail to send messsages after node restart.
  • #10933 Change log - 图24 (opens new window) Added support for configuring TCP keep-alive in MQTT/TCP and MQTT/SSL listeners.
  • #10948 Change log - 图25 (opens new window) Added live_connections field for some HTTP APIs, i.e:
    • /monitor_current, /monitor_current/nodes/{node}
    • /monitor/nodes/{node}, /monitor
    • /node/{node}, /nodes
  • #10941 Change log - 图26 (opens new window) Improved the collection speed of Prometheus metrics when setting prometheus.vm_dist_collector=disabled and metric erlang_vm_statistics_run_queues_length_total is renamed to erlang_vm_statistics_run_queues_length
  • #10985 Change log - 图27 (opens new window) Renamed emqx ctl command cluster_call to conf cluster_sync. The old command cluster_call is still a valid command, but not included in usage info.
  • #10988 Change log - 图28 (opens new window) Improved log security when data bridge creation fails to ensure sensitive data is always obfuscated.
  • #10926 Change log - 图29 (opens new window) Allowed enable as well as enabled as the state flag for listeners. Prior to this change, listener can be enable/disabled by setting the true or false on the enabled config. This is slightly different naming comparing to other state flags in the system. Now the enable flag is added as an alias in listener config.
  • #10970 Change log - 图30 (opens new window) A query_mode parameter has been added to the Kafka producer bridge. This parameter allows you to specify if the bridge should use the asynchronous or synchronous mode when sending data to Kafka. The default is asynchronous mode.
  • #10676 Change log - 图31 (opens new window) Added CLI commands emqx ctl export and emqx ctl import for importing/exporting configuration and user data. This allows exporting configurations and built-in database data from a running EMQX cluster and importing them into the same or another running EMQX cluster.
  • #11003 Change log - 图32 (opens new window) Added an option to configure TCP keepalive in Kafka bridge.
  • #10961 Change log - 图33 (opens new window) Added support for unlimited max connections for gateway listeners by allowing infinity as a valid value for the max_connections field in the configuration and HTTP API.
  • #11019 Change log - 图34 (opens new window) Improved log security for JWT, now it will be obfuscated before print.
  • #11024 Change log - 图35 (opens new window) Added a small improvement to reduce the chance of seeing the connecting state when creating/updating a Pulsar Producer bridge.
  • #11034 Change log - 图36 (opens new window) Hid the broker config and changed the broker.shared_subscription_strategy to mqtt.shared_subscription_strategy as it belongs to mqtt.
  • #11045 Change log - 图37 (opens new window) The listener’s authentication and zone related apis have been officially removed in version 5.1.0.
  • #11062 Change log - 图38 (opens new window) Renamed config log.file.to to log.file.path.

Bug Fixes

  • #11018 Change log - 图39 (opens new window) Fixed multiple issues with the Stomp gateway, including:

    • Fixed an issue where is_superuser was not working correctly.
    • Fixed an issue where the mountpoint was not being removed in message delivery.
    • After a message or subscription request fails, the Stomp client should be disconnected immediately after replying with an ERROR message.
    • #11051 Change log - 图40 (opens new window) Added validation to ensure that certificate depth (listener SSL option) is a non negative integer.

  • #10563 Change log - 图41 (opens new window) Corrected an issue where the no_local flag was not functioning correctly in subscription.

  • #10653 Change log - 图42 (opens new window) Stored gateway authentication TLS certificates and keys in the data directory to fix the problem of memory leakage.

  • #10682 Change log - 图43 (opens new window) Fixed the timestamp for the will message is incorrectly assigned at the session creation time, now this timestamp is the disconnected time of the session.

  • #10701 Change log - 图44 (opens new window) RPM package for Amazon Linux 2 did not support TLS v1.3 as it was assembled with Erlang/OTP built with openssl 1.0.

  • #10677 Change log - 图45 (opens new window) Fixed an issue in the Rule API where attempting to delete a non-existent rule resulted in a 404 HTTP error code response.

  • #10715 Change log - 图46 (opens new window) Support for getting the client certificate in the client.connected hook. Previously, this data was removed after the connection was established to reduce memory usage.

  • #10737 Change log - 图47 (opens new window) Fixed the issue where the HTTP API interface of Gateway cannot handle ClientIDs with special characters, such as: !@#$%^&*()_+{}:"<>?/.

  • #10809 Change log - 图48 (opens new window) Addressed ** ERROR ** Mnesia post_commit hook failed: error:badarg error messages happening during node shutdown or restart. Mria pull request: https://github.com/emqx/mria/pull/142 Change log - 图49 (opens new window)

  • #10807 Change log - 图50 (opens new window) The debug-level logs related to license checks will no longer be printed. These logs were generated too frequently and could interfere with log recording.

  • #10818 Change log - 图51 (opens new window) Fixed emqx_ctl traces command error where the traces start command in the emqx_mgmt_cli module was not working properly with some filters.

  • #10600 Change log - 图52 (opens new window) Deleted emqx_statsd application.

  • #10820 Change log - 图53 (opens new window) Fixed the issue where newly added nodes in the cluster would not apply the new license after a cluster license update and would continue to use the old license. Sometimes the new node must start with a outdated license. e.g. use emqx-operator deployed and needed to scale up after license expired. At the time the cluster’s license key already updated by API/CLI, but the new node won’t use it.

  • #10851 Change log - 图54 (opens new window) Obfuscated sensitive data in the bad API logging.

  • #10884 Change log - 图55 (opens new window) Fixed an issue where trying to get rule info or metrics could result in a crash when a node is joining a cluster.

  • #10887 Change log - 图56 (opens new window) Fixed a potential issue where requests to bridges might take a long time to be retried. This only affected low throughput scenarios, where the buffering layer could take a long time to detect connectivity and driver problems.

  • #10878 Change log - 图57 (opens new window) Fixed a vulnerability in the RabbitMQ bridge, which could potentially expose passwords to log files.

  • #10871 Change log - 图58 (opens new window) Fixed an issue where the Dashboard shows that the connection still exists after a CoAP connection is disconnected, but deletion and message posting requests do not take effect.

  • #10880 Change log - 图59 (opens new window) Added a new REST API POST /clients/kickout/bulk for kicking out multiple clients in bulk.

  • #10913 Change log - 图60 (opens new window) Fixed an issue where the plugin status REST API of a node would still include the cluster node status after the node left the cluster.

  • #10923 Change log - 图61 (opens new window) Fixed a race-condition in channel info registration. Prior to this fix, when system is under heavy load, it might happen that a client is disconnected (or has its session expired) but still can be found in the clients page in dashboard. One of the possible reasons is a race condition fixed in this PR: the connection is killed in the middle of channel data registration.

  • #10930 Change log - 图62 (opens new window) Added a schema validation for duration data type to avoid invalid values. Before this fix, it was possible to use absurd values in the schema that would exceed the system limit, causing a crash.

  • #10952 Change log - 图63 (opens new window) Disallow enabling fail_if_no_peer_cert in listener SSL options if verify = verify_none is set. Setting fail_if_no_peer_cert = true and verify = verify_none caused connection errors due to incompatible options. This fix validates the options when creating or updating a listener to avoid these errors.

Note: any old listener configuration with fail_if_no_peer_cert = true and verify = verify_none that was previously allowed will fail to load after applying this fix and must be manually fixed.

  • #10951 Change log - 图64 (opens new window) Fixed the issue in MQTT-SN gateway when the mountpoint did not take effect on message publishing.

  • #10943 Change log - 图65 (opens new window) Deprecated UDP mcast mechanism for cluster discovery. This feature has been planed for deprecation since 5.0 mainly due to the lack of actual production use. This feature code is not yet removed in 5.1, but the document interface is demoted.

  • #10902 Change log - 图66 (opens new window) Avoid syncing cluser.hocon file from the nodes running a newer version than the self-node. During cluster rolling upgrade, if an older version node has to restart due to whatever reason, if it copies the cluster.hocon file from a newer version node, it may fail to start. After this fix, the older version node will not copy the cluster.hocon file from a newer, so it will use its own cluster.hocon file to start.

  • #10967 Change log - 图67 (opens new window) Fixed error message formatting in rebalance API: previously they could be displayed as unclear dumps of internal Erlang structures. Added wait_health_check option to node evacuation CLI and API. This is a time interval when the node reports “unhealthy status” without beginning actual evacuation. We need this to allow a Load Balancer (if any) to remove the evacuated node from balancing and not forward (re)connecting clients to the evacuated node.

  • #10911 Change log - 图68 (opens new window) The error message and log entry that appear when one tries to create a bridge with a name the exceeds 255 bytes is now easier to understand.

  • #10983 Change log - 图69 (opens new window) Fixed the issue when mqtt clients could not connect over TLS if the listener was configured to use TLS v1.3 only. The problem was that TLS connection was trying to use options incompatible with TLS v1.3.

  • #10977 Change log - 图70 (opens new window) Fixed the delay in updating subscription count metric and corrected configuration issues in Stomp gateway.

  • #10950 Change log - 图71 (opens new window) Fixed the issue where the enable_qos option does not take effect in the MQTT-SN gateway.

  • #10999 Change log - 图72 (opens new window) Changed schema validation for Kafka fields ‘Partition Count Refresh Interval’ and ‘Offset Commit Interval’ to avoid accepting values larger then maximum allowed.

  • #10997 Change log - 图73 (opens new window) The ClickHouse bridge had a problem that could cause messages to be dropped when the ClickHouse server is closed while sending messages even when the request_ttl is set to infinity. This has been fixed by treating errors due to a closed connection as recoverable errors.

  • #10994 Change log - 图74 (opens new window) Redacted proxy-authorization headers as used by HTTP connector to avoid leaking secrets into log files.

  • #10996 Change log - 图75 (opens new window) For any unknown HTTP/API request, the default response is a 404 error rather than the dashboard’s index.html.

  • #11005 Change log - 图76 (opens new window) Fixed the issue where the method field cannot be correctly printed in the trace logs of AuthN HTTP.

  • #11006 Change log - 图77 (opens new window) Fixed QUIC listeners’s default cert file paths. Prior to this change, the default cert file paths are prefixed with environment variable ${EMQX_ETC_DIR} which were not interpolated before used in QUIC listeners.

  • #10998 Change log - 图78 (opens new window) Do not allow batch_size option for MongoDB bridge resource. MongoDB connector currently does not support batching, the batch_size config value is forced to be 1 if provided.

  • #10955 Change log - 图79 (opens new window) Fixed the issue in MQTT-SN gateway where deleting Predefined Topics configuration does not work.

  • #11025 Change log - 图80 (opens new window) Fixed a case_clause error that could arise in race conditions in Pulsar Producer bridge.

  • #11030 Change log - 图81 (opens new window) Improved error messages when a validation error occurs while using the Listeners HTTP API.

  • #11033 Change log - 图82 (opens new window) Deprecated the mountpoint field in AuthenticateRequest in ExProto gateway. This field was introduced in e4.x, but in fact, in e5.0 we have provided gateway.exproto.mountpoint for configuration, so there is no need to override it through the Authenticate request.

Additionally, updates the default value of subscriptions_max, inflight_max, mqueue_max to infinity.

  • #11040 Change log - 图83 (opens new window) Fixed a health check issue for Kafka Producer that could lead to loss of messages when the connection to Kafka’s brokers were down.

  • #11038 Change log - 图84 (opens new window) Fixed a health check issue for Pulsar Producer that could lead to loss of messages when the connection to Pulsar’s brokers were down.

  • #11042 Change log - 图85 (opens new window) Fixed crash on REST API GET /listeners when listener’s max_connections is set to a string.

  • #11028 Change log - 图86 (opens new window) Disallowed using multiple TLS versions in the listener config that include tlsv1.3 but exclude tlsv1.2. Using TLS configuration with such version gap caused connection errors. Additionally, drop and log TLS options that are incompatible with the selected TLS version(s).

Note: any old listener configuration with the version gap described above will fail to load after applying this fix and must be manually fixed.

Breaking Changes Change log - 图93 (opens new window)

Known Issues Change log - 图94 (opens new window)