=ERROR REPORT==== 11-Oct-2025::10:53:45.224940 === inet_config: syntax error in /etc/rabbitmq/erl_inetrc =ERROR REPORT==== 11-Oct-2025::10:53:45.224940 === inet_config: syntax error in /etc/rabbitmq/erl_inetrc 2025-10-11 10:53:47.883224+00:00 [info] <0.250.0> Feature flags: list of feature flags found: 2025-10-11 10:53:47.888876+00:00 [info] <0.250.0> Feature flags: [ ] implicit_default_bindings 2025-10-11 10:53:47.888897+00:00 [info] <0.250.0> Feature flags: [ ] maintenance_mode_status 2025-10-11 10:53:47.888915+00:00 [info] <0.250.0> Feature flags: [ ] quorum_queue 2025-10-11 10:53:47.888931+00:00 [info] <0.250.0> Feature flags: [ ] stream_queue 2025-10-11 10:53:47.888947+00:00 [info] <0.250.0> Feature flags: [ ] user_limits 2025-10-11 10:53:47.888959+00:00 [info] <0.250.0> Feature flags: [ ] virtual_host_metadata 2025-10-11 10:53:47.888974+00:00 [info] <0.250.0> Feature flags: feature flag states written to disk: yes 2025-10-11 10:53:48.290041+00:00 [notice] <0.44.0> Application syslog exited with reason: stopped 2025-10-11 10:53:48.290129+00:00 [notice] <0.250.0> Logging: switching to configured handler(s); following messages may not be visible in this log output 2025-10-11 10:53:48.304305+00:00 [notice] <0.250.0> Logging: configured log handlers are now ACTIVE 2025-10-11 10:53:48.536362+00:00 [info] <0.250.0> ra: starting system quorum_queues 2025-10-11 10:53:48.536436+00:00 [info] <0.250.0> starting Ra system: quorum_queues in directory: /var/lib/rabbitmq/mnesia/rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack/quorum/rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack 2025-10-11 10:53:48.607500+00:00 [info] <0.286.0> ra system 'quorum_queues' running pre init for 0 registered servers 2025-10-11 10:53:48.617506+00:00 [info] <0.287.0> ra: meta data store initialised for system quorum_queues. 0 record(s) recovered 2025-10-11 10:53:48.630779+00:00 [notice] <0.292.0> WAL: ra_log_wal init, open tbls: ra_log_open_mem_tables, closed tbls: ra_log_closed_mem_tables 2025-10-11 10:53:48.641434+00:00 [info] <0.250.0> ra: starting system coordination 2025-10-11 10:53:48.641494+00:00 [info] <0.250.0> starting Ra system: coordination in directory: /var/lib/rabbitmq/mnesia/rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack/coordination/rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack 2025-10-11 10:53:48.650280+00:00 [info] <0.299.0> ra system 'coordination' running pre init for 0 registered servers 2025-10-11 10:53:48.650932+00:00 [info] <0.300.0> ra: meta data store initialised for system coordination. 0 record(s) recovered 2025-10-11 10:53:48.651063+00:00 [notice] <0.305.0> WAL: ra_coordination_log_wal init, open tbls: ra_coordination_log_open_mem_tables, closed tbls: ra_coordination_log_closed_mem_tables 2025-10-11 10:53:48.657897+00:00 [info] <0.250.0>  2025-10-11 10:53:48.657897+00:00 [info] <0.250.0> Starting RabbitMQ 3.9.21 on Erlang 24.3.4.2 [jit] 2025-10-11 10:53:48.657897+00:00 [info] <0.250.0> Copyright (c) 2007-2022 VMware, Inc. or its affiliates. 2025-10-11 10:53:48.657897+00:00 [info] <0.250.0> Licensed under the MPL 2.0. Website: https://rabbitmq.com ## ## RabbitMQ 3.9.21 ## ## ########## Copyright (c) 2007-2022 VMware, Inc. or its affiliates. ###### ## ########## Licensed under the MPL 2.0. Website: https://rabbitmq.com Erlang: 24.3.4.2 [jit] TLS Library: OpenSSL - OpenSSL 3.5.1 1 Jul 2025 Doc guides: https://rabbitmq.com/documentation.html Support: https://rabbitmq.com/contact.html Tutorials: https://rabbitmq.com/getstarted.html Monitoring: https://rabbitmq.com/monitoring.html Logs: /var/lib/rabbitmq/rabbitmq_upgrade.log Config file(s): /etc/rabbitmq/advanced.config /etc/rabbitmq/conf.d/10-operatorDefaults.conf /etc/rabbitmq/conf.d/11-default_user.conf /etc/rabbitmq/conf.d/90-userDefinedConfiguration.conf Starting broker...2025-10-11 10:53:48.659431+00:00 [info] <0.250.0>  2025-10-11 10:53:48.659431+00:00 [info] <0.250.0> node : rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack 2025-10-11 10:53:48.659431+00:00 [info] <0.250.0> home dir : /var/lib/rabbitmq 2025-10-11 10:53:48.659431+00:00 [info] <0.250.0> config file(s) : /etc/rabbitmq/advanced.config 2025-10-11 10:53:48.659431+00:00 [info] <0.250.0> : /etc/rabbitmq/conf.d/10-operatorDefaults.conf 2025-10-11 10:53:48.659431+00:00 [info] <0.250.0> : /etc/rabbitmq/conf.d/11-default_user.conf 2025-10-11 10:53:48.659431+00:00 [info] <0.250.0> : /etc/rabbitmq/conf.d/90-userDefinedConfiguration.conf 2025-10-11 10:53:48.659431+00:00 [info] <0.250.0> cookie hash : 4ervU2rn7if45GXtXZuXkA== 2025-10-11 10:53:48.659431+00:00 [info] <0.250.0> log(s) : /var/lib/rabbitmq/rabbitmq_upgrade.log 2025-10-11 10:53:48.659431+00:00 [info] <0.250.0> :  2025-10-11 10:53:48.659431+00:00 [info] <0.250.0> database dir : /var/lib/rabbitmq/mnesia/rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack 2025-10-11 10:53:48.868398+00:00 [info] <0.250.0> Feature flags: list of feature flags found: 2025-10-11 10:53:48.868456+00:00 [info] <0.250.0> Feature flags: [ ] drop_unroutable_metric 2025-10-11 10:53:48.868482+00:00 [info] <0.250.0> Feature flags: [ ] empty_basic_get_metric 2025-10-11 10:53:48.868516+00:00 [info] <0.250.0> Feature flags: [ ] implicit_default_bindings 2025-10-11 10:53:48.868535+00:00 [info] <0.250.0> Feature flags: [ ] maintenance_mode_status 2025-10-11 10:53:48.868552+00:00 [info] <0.250.0> Feature flags: [ ] quorum_queue 2025-10-11 10:53:48.868565+00:00 [info] <0.250.0> Feature flags: [ ] stream_queue 2025-10-11 10:53:48.868620+00:00 [info] <0.250.0> Feature flags: [ ] user_limits 2025-10-11 10:53:48.868632+00:00 [info] <0.250.0> Feature flags: [ ] virtual_host_metadata 2025-10-11 10:53:48.868645+00:00 [info] <0.250.0> Feature flags: feature flag states written to disk: yes 2025-10-11 10:53:51.738150+00:00 [info] <0.250.0> Running boot step pre_boot defined by app rabbit 2025-10-11 10:53:51.738245+00:00 [info] <0.250.0> Running boot step rabbit_global_counters defined by app rabbit 2025-10-11 10:53:51.738409+00:00 [info] <0.250.0> Running boot step rabbit_osiris_metrics defined by app rabbit 2025-10-11 10:53:51.738568+00:00 [info] <0.250.0> Running boot step rabbit_core_metrics defined by app rabbit 2025-10-11 10:53:51.739597+00:00 [info] <0.250.0> Running boot step rabbit_alarm defined by app rabbit 2025-10-11 10:53:51.756037+00:00 [info] <0.339.0> Memory high watermark set to 655 MiB (687194767 bytes) of 1638 MiB (1717986919 bytes) total 2025-10-11 10:53:51.768482+00:00 [info] <0.341.0> Enabling free disk space monitoring 2025-10-11 10:53:51.768583+00:00 [info] <0.341.0> Disk free limit set to 2000MB 2025-10-11 10:53:51.779044+00:00 [info] <0.250.0> Running boot step code_server_cache defined by app rabbit 2025-10-11 10:53:51.779154+00:00 [info] <0.250.0> Running boot step file_handle_cache defined by app rabbit 2025-10-11 10:53:51.779771+00:00 [info] <0.346.0> Limiting to approx 1048479 file handles (943629 sockets) 2025-10-11 10:53:51.779927+00:00 [info] <0.347.0> FHC read buffering: OFF 2025-10-11 10:53:51.779960+00:00 [info] <0.347.0> FHC write buffering: ON 2025-10-11 10:53:51.780812+00:00 [info] <0.250.0> Running boot step worker_pool defined by app rabbit 2025-10-11 10:53:51.780981+00:00 [info] <0.307.0> Will use 16 processes for default worker pool 2025-10-11 10:53:51.781024+00:00 [info] <0.307.0> Starting worker pool 'worker_pool' with 16 processes in it 2025-10-11 10:53:51.781872+00:00 [info] <0.250.0> Running boot step database defined by app rabbit 2025-10-11 10:53:51.783472+00:00 [info] <0.250.0> Node database directory at /var/lib/rabbitmq/mnesia/rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack is empty. Assuming we need to join an existing cluster or initialise from scratch... 2025-10-11 10:53:51.783520+00:00 [info] <0.250.0> Configured peer discovery backend: rabbit_peer_discovery_k8s 2025-10-11 10:53:51.783703+00:00 [info] <0.250.0> Will try to lock with peer discovery backend rabbit_peer_discovery_k8s 2025-10-11 10:53:51.786775+00:00 [warning] <0.366.0> Description: "Authenticity is not established by certificate path validation" 2025-10-11 10:53:51.786775+00:00 [warning] <0.366.0> Reason: "Option {verify, verify_peer} and cacertfile/cacerts is missing" 2025-10-11 10:53:51.786775+00:00 [warning] <0.366.0>  2025-10-11 10:53:51.823524+00:00 [info] <0.250.0> Will try to lock connecting to nodes ['rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack', 2025-10-11 10:53:51.823524+00:00 [info] <0.250.0> 'rabbit@rabbitmq-cell1-server-1.rabbitmq-cell1-nodes.openstack', 2025-10-11 10:53:51.823524+00:00 [info] <0.250.0> 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack'] 2025-10-11 10:54:02.569836+00:00 [info] <0.250.0> All discovered existing cluster peers: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack, rabbit@rabbitmq-cell1-server-1.rabbitmq-cell1-nodes.openstack, rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack 2025-10-11 10:54:02.569918+00:00 [info] <0.250.0> Peer nodes we can cluster with: rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack, rabbit@rabbitmq-cell1-server-1.rabbitmq-cell1-nodes.openstack 2025-10-11 10:54:02.577259+00:00 [info] <0.250.0> Node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack' selected for auto-clustering 2025-10-11 10:54:02.610702+00:00 [info] <0.250.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-11 10:54:02.610850+00:00 [info] <0.250.0> Successfully synced tables from a peer 2025-10-11 10:54:03.123899+00:00 [info] <0.250.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-11 10:54:03.124038+00:00 [info] <0.250.0> Successfully synced tables from a peer 2025-10-11 10:54:03.124074+00:00 [warning] <0.250.0> Feature flags: the previous instance of this node must have failed to write the `feature_flags` file at `/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack-feature_flags`: 2025-10-11 10:54:03.124097+00:00 [warning] <0.250.0> Feature flags: - list of previously enabled feature flags now marked as such: [maintenance_mode_status, 2025-10-11 10:54:03.124097+00:00 [warning] <0.250.0> quorum_queue, 2025-10-11 10:54:03.124097+00:00 [warning] <0.250.0> user_limits, 2025-10-11 10:54:03.124097+00:00 [warning] <0.250.0> virtual_host_metadata] 2025-10-11 10:54:03.125712+00:00 [info] <0.250.0> Feature flags: list of feature flags found: 2025-10-11 10:54:03.125757+00:00 [info] <0.250.0> Feature flags: [ ] drop_unroutable_metric 2025-10-11 10:54:03.125773+00:00 [info] <0.250.0> Feature flags: [ ] empty_basic_get_metric 2025-10-11 10:54:03.125821+00:00 [info] <0.250.0> Feature flags: [ ] implicit_default_bindings 2025-10-11 10:54:03.125841+00:00 [info] <0.250.0> Feature flags: [x] maintenance_mode_status 2025-10-11 10:54:03.125859+00:00 [info] <0.250.0> Feature flags: [x] quorum_queue 2025-10-11 10:54:03.125908+00:00 [info] <0.250.0> Feature flags: [ ] stream_queue 2025-10-11 10:54:03.125928+00:00 [info] <0.250.0> Feature flags: [x] user_limits 2025-10-11 10:54:03.125968+00:00 [info] <0.250.0> Feature flags: [x] virtual_host_metadata 2025-10-11 10:54:03.125989+00:00 [info] <0.250.0> Feature flags: feature flag states written to disk: yes 2025-10-11 10:54:03.135290+00:00 [info] <0.250.0> Feature flag `drop_unroutable_metric`: mark as enabled=true 2025-10-11 10:54:03.138744+00:00 [info] <0.250.0> Feature flags: list of feature flags found: 2025-10-11 10:54:03.138800+00:00 [info] <0.250.0> Feature flags: [x] drop_unroutable_metric 2025-10-11 10:54:03.138817+00:00 [info] <0.250.0> Feature flags: [ ] empty_basic_get_metric 2025-10-11 10:54:03.138838+00:00 [info] <0.250.0> Feature flags: [ ] implicit_default_bindings 2025-10-11 10:54:03.138852+00:00 [info] <0.250.0> Feature flags: [x] maintenance_mode_status 2025-10-11 10:54:03.138864+00:00 [info] <0.250.0> Feature flags: [x] quorum_queue 2025-10-11 10:54:03.138886+00:00 [info] <0.250.0> Feature flags: [ ] stream_queue 2025-10-11 10:54:03.138899+00:00 [info] <0.250.0> Feature flags: [x] user_limits 2025-10-11 10:54:03.138916+00:00 [info] <0.250.0> Feature flags: [x] virtual_host_metadata 2025-10-11 10:54:03.138937+00:00 [info] <0.250.0> Feature flags: feature flag states written to disk: yes 2025-10-11 10:54:03.146718+00:00 [info] <0.250.0> Feature flag `empty_basic_get_metric`: mark as enabled=true 2025-10-11 10:54:03.150484+00:00 [info] <0.250.0> Feature flags: list of feature flags found: 2025-10-11 10:54:03.150530+00:00 [info] <0.250.0> Feature flags: [x] drop_unroutable_metric 2025-10-11 10:54:03.150550+00:00 [info] <0.250.0> Feature flags: [x] empty_basic_get_metric 2025-10-11 10:54:03.150566+00:00 [info] <0.250.0> Feature flags: [ ] implicit_default_bindings 2025-10-11 10:54:03.150582+00:00 [info] <0.250.0> Feature flags: [x] maintenance_mode_status 2025-10-11 10:54:03.150694+00:00 [info] <0.250.0> Feature flags: [x] quorum_queue 2025-10-11 10:54:03.150721+00:00 [info] <0.250.0> Feature flags: [ ] stream_queue 2025-10-11 10:54:03.150746+00:00 [info] <0.250.0> Feature flags: [x] user_limits 2025-10-11 10:54:03.150758+00:00 [info] <0.250.0> Feature flags: [x] virtual_host_metadata 2025-10-11 10:54:03.150774+00:00 [info] <0.250.0> Feature flags: feature flag states written to disk: yes 2025-10-11 10:54:03.160042+00:00 [info] <0.250.0> Waiting for Mnesia tables for 30000 ms, 0 retries left 2025-10-11 10:54:03.160227+00:00 [info] <0.250.0> Successfully synced tables from a peer 2025-10-11 10:54:03.160326+00:00 [info] <0.250.0> Feature flag `implicit_default_bindings`: mark as enabled=true 2025-10-11 10:54:03.162573+00:00 [info] <0.250.0> Feature flags: list of feature flags found: 2025-10-11 10:54:03.162620+00:00 [info] <0.250.0> Feature flags: [x] drop_unroutable_metric 2025-10-11 10:54:03.162642+00:00 [info] <0.250.0> Feature flags: [x] empty_basic_get_metric 2025-10-11 10:54:03.162703+00:00 [info] <0.250.0> Feature flags: [x] implicit_default_bindings 2025-10-11 10:54:03.162729+00:00 [info] <0.250.0> Feature flags: [x] maintenance_mode_status 2025-10-11 10:54:03.162745+00:00 [info] <0.250.0> Feature flags: [x] quorum_queue 2025-10-11 10:54:03.162787+00:00 [info] <0.250.0> Feature flags: [ ] stream_queue 2025-10-11 10:54:03.162811+00:00 [info] <0.250.0> Feature flags: [x] user_limits 2025-10-11 10:54:03.162829+00:00 [info] <0.250.0> Feature flags: [x] virtual_host_metadata 2025-10-11 10:54:03.162844+00:00 [info] <0.250.0> Feature flags: feature flag states written to disk: yes 2025-10-11 10:54:03.171478+00:00 [info] <0.250.0> Feature flag `stream_queue`: mark as enabled=true 2025-10-11 10:54:03.173751+00:00 [info] <0.250.0> Feature flags: list of feature flags found: 2025-10-11 10:54:03.173797+00:00 [info] <0.250.0> Feature flags: [x] drop_unroutable_metric 2025-10-11 10:54:03.173826+00:00 [info] <0.250.0> Feature flags: [x] empty_basic_get_metric 2025-10-11 10:54:03.173899+00:00 [info] <0.250.0> Feature flags: [x] implicit_default_bindings 2025-10-11 10:54:03.173921+00:00 [info] <0.250.0> Feature flags: [x] maintenance_mode_status 2025-10-11 10:54:03.173938+00:00 [info] <0.250.0> Feature flags: [x] quorum_queue 2025-10-11 10:54:03.174017+00:00 [info] <0.250.0> Feature flags: [x] stream_queue 2025-10-11 10:54:03.174037+00:00 [info] <0.250.0> Feature flags: [x] user_limits 2025-10-11 10:54:03.174053+00:00 [info] <0.250.0> Feature flags: [x] virtual_host_metadata 2025-10-11 10:54:03.174076+00:00 [info] <0.250.0> Feature flags: feature flag states written to disk: yes 2025-10-11 10:54:03.185796+00:00 [info] <0.250.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-11 10:54:03.185951+00:00 [info] <0.250.0> Successfully synced tables from a peer 2025-10-11 10:54:03.229789+00:00 [info] <0.250.0> Waiting for Mnesia tables for 30000 ms, 9 retries left 2025-10-11 10:54:03.229926+00:00 [info] <0.250.0> Successfully synced tables from a peer 2025-10-11 10:54:03.244384+00:00 [info] <0.250.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack' 2025-10-11 10:54:03.313712+00:00 [info] <0.250.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack' 2025-10-11 10:54:03.328297+00:00 [info] <0.250.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack' 2025-10-11 10:54:03.329453+00:00 [info] <0.250.0> Will register with peer discovery backend rabbit_peer_discovery_k8s 2025-10-11 10:54:03.341776+00:00 [info] <0.250.0> Will try to unlock with peer discovery backend rabbit_peer_discovery_k8s 2025-10-11 10:54:03.342845+00:00 [info] <0.250.0> Running boot step database_sync defined by app rabbit 2025-10-11 10:54:03.343017+00:00 [info] <0.250.0> Running boot step feature_flags defined by app rabbit 2025-10-11 10:54:03.343232+00:00 [info] <0.250.0> Running boot step codec_correctness_check defined by app rabbit 2025-10-11 10:54:03.343291+00:00 [info] <0.250.0> Running boot step external_infrastructure defined by app rabbit 2025-10-11 10:54:03.343322+00:00 [info] <0.250.0> Running boot step rabbit_registry defined by app rabbit 2025-10-11 10:54:03.343410+00:00 [info] <0.250.0> Running boot step rabbit_auth_mechanism_cr_demo defined by app rabbit 2025-10-11 10:54:03.343513+00:00 [info] <0.250.0> Running boot step rabbit_queue_location_random defined by app rabbit 2025-10-11 10:54:03.343585+00:00 [info] <0.250.0> Running boot step rabbit_event defined by app rabbit 2025-10-11 10:54:03.343734+00:00 [info] <0.250.0> Running boot step rabbit_auth_mechanism_amqplain defined by app rabbit 2025-10-11 10:54:03.343802+00:00 [info] <0.250.0> Running boot step rabbit_auth_mechanism_plain defined by app rabbit 2025-10-11 10:54:03.343880+00:00 [info] <0.250.0> Running boot step rabbit_exchange_type_direct defined by app rabbit 2025-10-11 10:54:03.343952+00:00 [info] <0.250.0> Running boot step rabbit_exchange_type_fanout defined by app rabbit 2025-10-11 10:54:03.344021+00:00 [info] <0.250.0> Running boot step rabbit_exchange_type_headers defined by app rabbit 2025-10-11 10:54:03.344078+00:00 [info] <0.250.0> Running boot step rabbit_exchange_type_topic defined by app rabbit 2025-10-11 10:54:03.344130+00:00 [info] <0.250.0> Running boot step rabbit_mirror_queue_mode_all defined by app rabbit 2025-10-11 10:54:03.344179+00:00 [info] <0.250.0> Running boot step rabbit_mirror_queue_mode_exactly defined by app rabbit 2025-10-11 10:54:03.344226+00:00 [info] <0.250.0> Running boot step rabbit_mirror_queue_mode_nodes defined by app rabbit 2025-10-11 10:54:03.344308+00:00 [info] <0.250.0> Running boot step rabbit_priority_queue defined by app rabbit 2025-10-11 10:54:03.344342+00:00 [info] <0.250.0> Priority queues enabled, real BQ is rabbit_variable_queue 2025-10-11 10:54:03.344399+00:00 [info] <0.250.0> Running boot step rabbit_queue_location_client_local defined by app rabbit 2025-10-11 10:54:03.344468+00:00 [info] <0.250.0> Running boot step rabbit_queue_location_min_masters defined by app rabbit 2025-10-11 10:54:03.344534+00:00 [info] <0.250.0> Running boot step kernel_ready defined by app rabbit 2025-10-11 10:54:03.344554+00:00 [info] <0.250.0> Running boot step rabbit_sysmon_minder defined by app rabbit 2025-10-11 10:54:03.344647+00:00 [info] <0.250.0> Running boot step rabbit_epmd_monitor defined by app rabbit 2025-10-11 10:54:03.350572+00:00 [info] <0.556.0> epmd monitor knows us, inter-node communication (distribution) port: 25672 2025-10-11 10:54:03.350704+00:00 [info] <0.250.0> Running boot step guid_generator defined by app rabbit 2025-10-11 10:54:03.356050+00:00 [info] <0.250.0> Running boot step rabbit_node_monitor defined by app rabbit 2025-10-11 10:54:03.357661+00:00 [info] <0.560.0> Starting rabbit_node_monitor 2025-10-11 10:54:03.357834+00:00 [info] <0.250.0> Running boot step delegate_sup defined by app rabbit 2025-10-11 10:54:03.358323+00:00 [info] <0.250.0> Running boot step rabbit_memory_monitor defined by app rabbit 2025-10-11 10:54:03.358527+00:00 [info] <0.250.0> Running boot step core_initialized defined by app rabbit 2025-10-11 10:54:03.358564+00:00 [info] <0.250.0> Running boot step upgrade_queues defined by app rabbit 2025-10-11 10:54:03.366803+00:00 [info] <0.250.0> message_store upgrades: 1 to apply 2025-10-11 10:54:03.366916+00:00 [info] <0.250.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store 2025-10-11 10:54:03.367038+00:00 [info] <0.250.0> message_store upgrades: No durable queues found. Skipping message store migration 2025-10-11 10:54:03.367079+00:00 [info] <0.250.0> message_store upgrades: Removing the old message store data 2025-10-11 10:54:03.367704+00:00 [info] <0.250.0> message_store upgrades: All upgrades applied successfully 2025-10-11 10:54:03.381336+00:00 [info] <0.250.0> Running boot step channel_tracking defined by app rabbit 2025-10-11 10:54:03.394274+00:00 [info] <0.250.0> Setting up a table for channel tracking on this node: 'tracked_channel_on_node_rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack' 2025-10-11 10:54:03.407023+00:00 [info] <0.250.0> Setting up a table for channel tracking on this node: 'tracked_channel_table_per_user_on_node_rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack' 2025-10-11 10:54:03.407354+00:00 [info] <0.250.0> Running boot step rabbit_channel_tracking_handler defined by app rabbit 2025-10-11 10:54:03.407429+00:00 [info] <0.250.0> Running boot step connection_tracking defined by app rabbit 2025-10-11 10:54:03.408936+00:00 [info] <0.250.0> Setting up a table for connection tracking on this node: 'tracked_connection_on_node_rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack' 2025-10-11 10:54:03.410353+00:00 [info] <0.250.0> Setting up a table for per-vhost connection counting on this node: 'tracked_connection_per_vhost_on_node_rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack' 2025-10-11 10:54:03.412971+00:00 [info] <0.250.0> Setting up a table for per-user connection counting on this node: 'tracked_connection_table_per_user_on_node_rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack' 2025-10-11 10:54:03.413226+00:00 [info] <0.250.0> Running boot step rabbit_connection_tracking_handler defined by app rabbit 2025-10-11 10:54:03.413311+00:00 [info] <0.250.0> Running boot step rabbit_exchange_parameters defined by app rabbit 2025-10-11 10:54:03.413392+00:00 [info] <0.250.0> Running boot step rabbit_mirror_queue_misc defined by app rabbit 2025-10-11 10:54:03.413564+00:00 [info] <0.250.0> Running boot step rabbit_policies defined by app rabbit 2025-10-11 10:54:03.413895+00:00 [info] <0.250.0> Running boot step rabbit_policy defined by app rabbit 2025-10-11 10:54:03.413964+00:00 [info] <0.250.0> Running boot step rabbit_queue_location_validator defined by app rabbit 2025-10-11 10:54:03.414036+00:00 [info] <0.250.0> Running boot step rabbit_quorum_memory_manager defined by app rabbit 2025-10-11 10:54:03.414087+00:00 [info] <0.250.0> Running boot step rabbit_stream_coordinator defined by app rabbit 2025-10-11 10:54:03.414296+00:00 [info] <0.250.0> Running boot step rabbit_vhost_limit defined by app rabbit 2025-10-11 10:54:03.414377+00:00 [info] <0.250.0> Running boot step rabbit_mgmt_reset_handler defined by app rabbitmq_management 2025-10-11 10:54:03.414425+00:00 [info] <0.250.0> Running boot step rabbit_mgmt_db_handler defined by app rabbitmq_management_agent 2025-10-11 10:54:03.414485+00:00 [info] <0.250.0> Management plugin: using rates mode 'basic' 2025-10-11 10:54:03.419927+00:00 [info] <0.250.0> Running boot step recovery defined by app rabbit 2025-10-11 10:54:03.422763+00:00 [info] <0.604.0> Making sure data directory '/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists 2025-10-11 10:54:03.434265+00:00 [info] <0.604.0> Setting segment_entry_count for vhost '/' with 0 queues to '2048' 2025-10-11 10:54:03.440728+00:00 [info] <0.604.0> Starting message stores for vhost '/' 2025-10-11 10:54:03.440997+00:00 [info] <0.608.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index 2025-10-11 10:54:03.445245+00:00 [info] <0.604.0> Started message store of type transient for vhost '/' 2025-10-11 10:54:03.445436+00:00 [info] <0.612.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index 2025-10-11 10:54:03.449201+00:00 [warning] <0.612.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch 2025-10-11 10:54:03.450114+00:00 [info] <0.604.0> Started message store of type persistent for vhost '/' 2025-10-11 10:54:03.450340+00:00 [info] <0.604.0> Recovering 0 queues of type rabbit_classic_queue took 15ms 2025-10-11 10:54:03.450379+00:00 [info] <0.604.0> Recovering 0 queues of type rabbit_quorum_queue took 0ms 2025-10-11 10:54:03.450421+00:00 [info] <0.604.0> Recovering 0 queues of type rabbit_stream_queue took 0ms 2025-10-11 10:54:03.451741+00:00 [info] <0.250.0> Running boot step empty_db_check defined by app rabbit 2025-10-11 10:54:03.451810+00:00 [info] <0.250.0> Will not seed default virtual host and user: have definitions to load... 2025-10-11 10:54:03.451839+00:00 [info] <0.250.0> Running boot step rabbit_looking_glass defined by app rabbit 2025-10-11 10:54:03.451864+00:00 [info] <0.250.0> Running boot step rabbit_core_metrics_gc defined by app rabbit 2025-10-11 10:54:03.451985+00:00 [info] <0.250.0> Running boot step background_gc defined by app rabbit 2025-10-11 10:54:03.452110+00:00 [info] <0.250.0> Running boot step routing_ready defined by app rabbit 2025-10-11 10:54:03.452139+00:00 [info] <0.250.0> Running boot step pre_flight defined by app rabbit 2025-10-11 10:54:03.452161+00:00 [info] <0.250.0> Running boot step notify_cluster defined by app rabbit 2025-10-11 10:54:03.452188+00:00 [info] <0.250.0> Running boot step networking defined by app rabbit 2025-10-11 10:54:03.452221+00:00 [info] <0.250.0> Running boot step rabbit_maintenance_mode_state defined by app rabbit 2025-10-11 10:54:03.452254+00:00 [info] <0.250.0> Creating table rabbit_node_maintenance_states for feature flag `maintenance_mode_status` 2025-10-11 10:54:03.452773+00:00 [info] <0.560.0> rabbit on node 'rabbit@rabbitmq-cell1-server-1.rabbitmq-cell1-nodes.openstack' up 2025-10-11 10:54:03.454574+00:00 [info] <0.250.0> Running boot step definition_import_worker_pool defined by app rabbit 2025-10-11 10:54:03.454655+00:00 [info] <0.307.0> Starting worker pool 'definition_import_pool' with 16 processes in it 2025-10-11 10:54:03.455264+00:00 [info] <0.250.0> Running boot step cluster_name defined by app rabbit 2025-10-11 10:54:03.455307+00:00 [info] <0.250.0> Setting cluster name to 'rabbitmq-cell1' as configured 2025-10-11 10:54:03.466039+00:00 [info] <0.560.0> rabbit on node 'rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack' up 2025-10-11 10:54:03.475191+00:00 [info] <0.250.0> Running boot step direct_client defined by app rabbit 2025-10-11 10:54:03.475387+00:00 [info] <0.250.0> Running boot step rabbit_management_load_definitions defined by app rabbitmq_management 2025-10-11 10:54:03.475522+00:00 [info] <0.656.0> Resetting node maintenance status 2025-10-11 10:54:03.502785+00:00 [info] <0.716.0> Management plugin: HTTP (non-TLS) listener started on port 15672 2025-10-11 10:54:03.513414+00:00 [info] <0.716.0> Management plugin: HTTPS listener started on port 15671 2025-10-11 10:54:03.513590+00:00 [info] <0.771.0> Statistics database started. 2025-10-11 10:54:03.513722+00:00 [info] <0.770.0> Starting worker pool 'management_worker_pool' with 3 processes in it 2025-10-11 10:54:03.521859+00:00 [info] <0.787.0> Peer discovery: node cleanup is disabled 2025-10-11 10:54:03.525123+00:00 [info] <0.795.0> Prometheus metrics: HTTP (non-TLS) listener started on port 15692 2025-10-11 10:54:03.528881+00:00 [info] <0.795.0> Prometheus metrics: HTTPS listener started on port 15691 2025-10-11 10:54:03.529021+00:00 [info] <0.656.0> Ready to start client connection listeners 2025-10-11 10:54:03.533428+00:00 [info] <0.868.0> started TLS (SSL) listener on [::]:5671 completed with 6 plugins. 2025-10-11 10:54:03.620910+00:00 [info] <0.656.0> Server startup complete; 6 plugins started. 2025-10-11 10:54:03.620910+00:00 [info] <0.656.0> * rabbitmq_prometheus 2025-10-11 10:54:03.620910+00:00 [info] <0.656.0> * rabbitmq_peer_discovery_k8s 2025-10-11 10:54:03.620910+00:00 [info] <0.656.0> * rabbitmq_peer_discovery_common 2025-10-11 10:54:03.620910+00:00 [info] <0.656.0> * rabbitmq_management 2025-10-11 10:54:03.620910+00:00 [info] <0.656.0> * rabbitmq_web_dispatch 2025-10-11 10:54:03.620910+00:00 [info] <0.656.0> * rabbitmq_management_agent 2025-10-11 10:56:52.719762+00:00 [info] <0.1049.0> queue 'conductor' in vhost '/': granting vote for {'%2F_conductor','rabbit@rabbitmq-cell1-server-1.rabbitmq-cell1-nodes.openstack'} with last indexterm {0,0} for term 1 previous term was 0 2025-10-11 10:56:52.729335+00:00 [info] <0.1049.0> queue 'conductor' in vhost '/': detected a new leader {'%2F_conductor','rabbit@rabbitmq-cell1-server-1.rabbitmq-cell1-nodes.openstack'} in term 1 2025-10-11 10:56:52.887907+00:00 [info] <0.1057.0> queue 'conductor.nova-cell1-conductor-0' in vhost '/': granting vote for {'%2F_conductor.nova-cell1-conductor-0','rabbit@rabbitmq-cell1-server-1.rabbitmq-cell1-nodes.openstack'} with last indexterm {0,0} for term 1 previous term was 0 2025-10-11 10:56:52.899471+00:00 [info] <0.1057.0> queue 'conductor.nova-cell1-conductor-0' in vhost '/': detected a new leader {'%2F_conductor.nova-cell1-conductor-0','rabbit@rabbitmq-cell1-server-1.rabbitmq-cell1-nodes.openstack'} in term 1 2025-10-11 10:56:54.685527+00:00 [info] <0.1068.0> accepting AMQP connection <0.1068.0> (10.129.0.164:53506 -> 10.128.0.122:5671) 2025-10-11 10:56:54.687103+00:00 [info] <0.1068.0> Connection <0.1068.0> (10.129.0.164:53506 -> 10.128.0.122:5671) has a client-provided name: nova-conductor:1:2aaa1495-ca24-4a9e-9ea5-49c911b1a7c7 2025-10-11 10:56:54.689709+00:00 [info] <0.1068.0> connection <0.1068.0> (10.129.0.164:53506 -> 10.128.0.122:5671 - nova-conductor:1:2aaa1495-ca24-4a9e-9ea5-49c911b1a7c7): user 'default_user_Ih8CBxepTS8DcAgFqxu' authenticated and granted access to vhost '/' 2025-10-11 10:56:56.234153+00:00 [info] <0.1082.0> queue 'compute' in vhost '/': granting vote for {'%2F_compute','rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack'} with last indexterm {0,0} for term 1 previous term was 0 2025-10-11 10:56:56.244414+00:00 [info] <0.1082.0> queue 'compute' in vhost '/': detected a new leader {'%2F_compute','rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack'} in term 1 2025-10-11 10:56:56.330184+00:00 [info] <0.1090.0> queue 'compute.nova-cell1-compute-ironic-compute-0' in vhost '/': granting vote for {'%2F_compute.nova-cell1-compute-ironic-compute-0','rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack'} with last indexterm {0,0} for term 1 previous term was 0 2025-10-11 10:56:56.340307+00:00 [info] <0.1090.0> queue 'compute.nova-cell1-compute-ironic-compute-0' in vhost '/': detected a new leader {'%2F_compute.nova-cell1-compute-ironic-compute-0','rabbit@rabbitmq-cell1-server-0.rabbitmq-cell1-nodes.openstack'} in term 1 2025-10-11 11:17:01.572941+00:00 [info] <0.4196.0> accepting AMQP connection <0.4196.0> (10.129.0.156:42806 -> 10.128.0.122:5671) 2025-10-11 11:17:01.574299+00:00 [info] <0.4196.0> Connection <0.4196.0> (10.129.0.156:42806 -> 10.128.0.122:5671) has a client-provided name: nova-compute:1:2371666d-e79e-46c0-b536-ed2e1f42c2eb 2025-10-11 11:17:01.575624+00:00 [info] <0.4196.0> connection <0.4196.0> (10.129.0.156:42806 -> 10.128.0.122:5671 - nova-compute:1:2371666d-e79e-46c0-b536-ed2e1f42c2eb): user 'default_user_Ih8CBxepTS8DcAgFqxu' authenticated and granted access to vhost '/' 2025-10-11 11:28:03.534526+00:00 [info] <0.5907.0> accepting AMQP connection <0.5907.0> (100.64.0.3:48060 -> 10.128.0.122:5671) 2025-10-11 11:28:03.536383+00:00 [info] <0.5907.0> Connection <0.5907.0> (100.64.0.3:48060 -> 10.128.0.122:5671) has a client-provided name: nova-compute:2:a3787a54-0ae8-4300-9256-42e72466cdcd 2025-10-11 11:28:03.538024+00:00 [info] <0.5907.0> connection <0.5907.0> (100.64.0.3:48060 -> 10.128.0.122:5671 - nova-compute:2:a3787a54-0ae8-4300-9256-42e72466cdcd): user 'default_user_Ih8CBxepTS8DcAgFqxu' authenticated and granted access to vhost '/' 2025-10-11 11:28:04.557733+00:00 [info] <0.5924.0> accepting AMQP connection <0.5924.0> (10.129.0.164:56680 -> 10.128.0.122:5671) 2025-10-11 11:28:04.559137+00:00 [info] <0.5924.0> Connection <0.5924.0> (10.129.0.164:56680 -> 10.128.0.122:5671) has a client-provided name: nova-conductor:1:ff700cb4-081b-4580-9956-bcab3f3b08d9 2025-10-11 11:28:04.560403+00:00 [info] <0.5924.0> connection <0.5924.0> (10.129.0.164:56680 -> 10.128.0.122:5671 - nova-conductor:1:ff700cb4-081b-4580-9956-bcab3f3b08d9): user 'default_user_Ih8CBxepTS8DcAgFqxu' authenticated and granted access to vhost '/' 2025-10-11 11:28:06.165341+00:00 [warning] <0.5907.0> closing AMQP connection <0.5907.0> (100.64.0.3:48060 -> 10.128.0.122:5671 - nova-compute:2:a3787a54-0ae8-4300-9256-42e72466cdcd, vhost: '/', user: 'default_user_Ih8CBxepTS8DcAgFqxu'): 2025-10-11 11:28:06.165341+00:00 [warning] <0.5907.0> client unexpectedly closed TCP connection 2025-10-11 11:28:13.255910+00:00 [info] <0.5981.0> queue 'compute.edpm-compute-0.ctlplane.openstack.lab' in vhost '/': granting vote for {'%2F_compute.edpm-compute-0.ctlplane.openstack.lab','rabbit@rabbitmq-cell1-server-1.rabbitmq-cell1-nodes.openstack'} with last indexterm {0,0} for term 1 previous term was 0 2025-10-11 11:28:13.269104+00:00 [info] <0.5981.0> queue 'compute.edpm-compute-0.ctlplane.openstack.lab' in vhost '/': detected a new leader {'%2F_compute.edpm-compute-0.ctlplane.openstack.lab','rabbit@rabbitmq-cell1-server-1.rabbitmq-cell1-nodes.openstack'} in term 1 2025-10-11 11:28:13.409072+00:00 [info] <0.5993.0> accepting AMQP connection <0.5993.0> (100.64.0.3:48012 -> 10.128.0.122:5671) 2025-10-11 11:28:13.410603+00:00 [info] <0.5993.0> Connection <0.5993.0> (100.64.0.3:48012 -> 10.128.0.122:5671) has a client-provided name: nova-compute:2:6d5dcac3-7a8e-4d0c-9c69-80676878edf3 2025-10-11 11:28:13.411885+00:00 [info] <0.5993.0> connection <0.5993.0> (100.64.0.3:48012 -> 10.128.0.122:5671 - nova-compute:2:6d5dcac3-7a8e-4d0c-9c69-80676878edf3): user 'default_user_Ih8CBxepTS8DcAgFqxu' authenticated and granted access to vhost '/' 2025-10-11 11:28:13.473613+00:00 [notice] <0.6011.0> queue 'compute.edpm-compute-1.ctlplane.openstack.lab' in vhost '/': candidate -> leader in term: 1 machine version: 1 2025-10-11 11:28:13.474739+00:00 [info] <0.6002.0> ra: started cluster %2F_compute.edpm-compute-1.ctlplane.openstack.lab with 3 servers 2025-10-11 11:28:13.474739+00:00 [info] <0.6002.0> 0 servers failed to start: [] 2025-10-11 11:28:13.474739+00:00 [info] <0.6002.0> Leader: {'%2F_compute.edpm-compute-1.ctlplane.openstack.lab','rabbit@rabbitmq-cell1-server-2.rabbitmq-cell1-nodes.openstack'}