2025-11-27T08:11:38.929256991+00:00 stderr F + sudo -E kolla_set_configs 2025-11-27T08:11:38.944121702+00:00 stderr F sudo: unable to send audit message: Operation not permitted 2025-11-27T08:11:39.010090765+00:00 stderr F INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json 2025-11-27T08:11:39.010319103+00:00 stderr F INFO:__main__:Validating config file 2025-11-27T08:11:39.010353294+00:00 stderr F INFO:__main__:Kolla config strategy set to: COPY_ALWAYS 2025-11-27T08:11:39.010381055+00:00 stderr F INFO:__main__:Copying service configuration files 2025-11-27T08:11:39.010562020+00:00 stderr F INFO:__main__:Copying /dev/null to /etc/libqb/force-filesystem-sockets 2025-11-27T08:11:39.010990344+00:00 stderr F INFO:__main__:Setting permission for /etc/libqb/force-filesystem-sockets 2025-11-27T08:11:39.011113007+00:00 stderr F INFO:__main__:Copying /dev/null to /var/log/btmp 2025-11-27T08:11:39.011328984+00:00 stderr F INFO:__main__:Setting permission for /var/log/btmp 2025-11-27T08:11:39.011772467+00:00 stderr F INFO:__main__:Deleting /etc/pam.d/system-auth 2025-11-27T08:11:39.011849830+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/pam.d/system-auth to /etc/pam.d/system-auth 2025-11-27T08:11:39.012588682+00:00 stderr F INFO:__main__:Creating directory /etc/rabbitmq/ssl 2025-11-27T08:11:39.012716916+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/rabbitmq/enabled_plugins to /etc/rabbitmq/enabled_plugins 2025-11-27T08:11:39.012901172+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/rabbitmq/inetrc to /etc/rabbitmq/inetrc 2025-11-27T08:11:39.013079439+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/rabbitmq/rabbitmq-env.conf to /etc/rabbitmq/rabbitmq-env.conf 2025-11-27T08:11:39.013270904+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/rabbitmq/rabbitmq.config to /etc/rabbitmq/rabbitmq.config 2025-11-27T08:11:39.013439340+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/rabbitmq/ssl-dist.conf to /etc/rabbitmq/ssl-dist.conf 2025-11-27T08:11:39.013826091+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/security/limits.d/rabbitmq-server.conf to /etc/security/limits.d/rabbitmq-server.conf 2025-11-27T08:11:39.014001207+00:00 stderr F INFO:__main__:Writing out command to execute 2025-11-27T08:11:39.014220013+00:00 stderr F INFO:__main__:Setting permission for /var/lib/rabbitmq 2025-11-27T08:11:39.014281445+00:00 stderr F INFO:__main__:Setting permission for /var/lib/rabbitmq/.erlang.cookie 2025-11-27T08:11:39.014414859+00:00 stderr F INFO:__main__:Setting permission for /var/log/rabbitmq 2025-11-27T08:11:39.022887692+00:00 stderr F ++ cat /run_command 2025-11-27T08:11:39.024346348+00:00 stderr F + CMD=/usr/sbin/pacemaker_remoted 2025-11-27T08:11:39.024365338+00:00 stderr F + ARGS= 2025-11-27T08:11:39.024365338+00:00 stderr F + [[ ! -n '' ]] 2025-11-27T08:11:39.024372728+00:00 stderr F + . kolla_extend_start 2025-11-27T08:11:39.024516903+00:00 stderr F ++ [[ -n '' ]] 2025-11-27T08:11:39.024525563+00:00 stderr F + echo 'Running command: '\''/usr/sbin/pacemaker_remoted'\''' 2025-11-27T08:11:39.024532783+00:00 stdout F Running command: '/usr/sbin/pacemaker_remoted' 2025-11-27T08:11:39.024540964+00:00 stderr F + umask 0022 2025-11-27T08:11:39.024549094+00:00 stderr F + exec /usr/sbin/pacemaker_remoted 2025-11-27T08:11:39.029749954+00:00 stderr F (pcmk__add_logfile) warning: Logging to '/var/log/pacemaker/pacemaker.log' is disabled: No such file or directory | uid=0 gid=0 2025-11-27T08:11:39.030005513+00:00 stderr F (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores 2025-11-27T08:11:39.030114886+00:00 stderr F (main) notice: Starting Pacemaker remote executor 2025-11-27T08:11:39.030219510+00:00 stderr F (qb_ipcs_us_publish) info: server name: lrmd 2025-11-27T08:11:39.030639362+00:00 stderr F (pcmk__init_tls_dh) info: Generating Diffie-Hellman parameters with 2048-bit prime for TLS 2025-11-27T08:11:39.140172506+00:00 stderr F (qb_ipcs_us_publish) info: server name: cib_ro 2025-11-27T08:11:39.140286610+00:00 stderr F (qb_ipcs_us_publish) info: server name: cib_rw 2025-11-27T08:11:39.140317680+00:00 stderr F (qb_ipcs_us_publish) info: server name: cib_shm 2025-11-27T08:11:39.140379312+00:00 stderr F (qb_ipcs_us_publish) info: server name: attrd 2025-11-27T08:11:39.140434454+00:00 stderr F (qb_ipcs_us_publish) info: server name: stonith-ng 2025-11-27T08:11:39.140517287+00:00 stderr F (qb_ipcs_us_publish) info: server name: pacemakerd 2025-11-27T08:11:39.140568828+00:00 stderr F (qb_ipcs_us_publish) info: server name: crmd 2025-11-27T08:11:39.140650481+00:00 stderr F (main) notice: Pacemaker remote executor successfully started and accepting connections 2025-11-27T08:11:39.140650481+00:00 stderr F (main) notice: OCF resource agent search path is /usr/lib/ocf/resource.d 2025-11-27T08:11:50.695080688+00:00 stderr F (pcmk__accept_remote_connection) info: Accepted new remote client connection from ::ffff:172.17.0.103 2025-11-27T08:11:50.696030557+00:00 stderr F (lrmd_remote_listen) info: Remote client pending authentication | 0x558b174d4660 id: b3197f42-7b29-42d5-9d09-0d84b84facb0 2025-11-27T08:11:51.239104544+00:00 stderr F (remoted__read_handshake_data) notice: Remote client connection accepted 2025-11-27T08:11:52.468632271+00:00 stderr F (process_lrmd_get_rsc_info) info: Agent information for 'rabbitmq' not in cache 2025-11-27T08:11:52.509381003+00:00 stderr F (process_lrmd_get_rsc_info) info: Agent information for 'rabbitmq:0' not in cache 2025-11-27T08:11:52.591501268+00:00 stderr F (process_lrmd_rsc_register) info: Cached agent information for 'rabbitmq' 2025-11-27T08:11:54.171999499+00:00 stderr F (log_execute) info: executing - rsc:rabbitmq action:start call_id:13 2025-11-27T08:12:04.285561811+00:00 stderr F (log_op_output) info: rabbitmq_start_0[107] error output [ Call cib_query failed: No such device or address ] 2025-11-27T08:12:04.285561811+00:00 stderr F (log_op_output) info: rabbitmq_start_0[107] error output [ Call cib_query failed: No such device or address ] 2025-11-27T08:12:04.285561811+00:00 stderr F (log_op_output) info: rabbitmq_start_0[107] error output [ Schema validation of configuration is disabled (enabling is encouraged and prevents common misconfigurations) ] 2025-11-27T08:12:04.285561811+00:00 stderr F (log_op_output) info: rabbitmq_start_0[107] error output [ Error: ] 2025-11-27T08:12:04.285561811+00:00 stderr F (log_op_output) info: rabbitmq_start_0[107] error output [ {{:case_clause, {:internal_user, :_, :_, :_, :_, :_}}, [{:erl_eval, :case_clauses, 6, [file: 'erl_eval.erl', line: 973]}, {:erl_eval, :exprs, 5, [file: 'erl_eval.erl', line: 123]}, {:erl_eval, :expr, 5, [file: 'erl_eval.erl', line: 459]}, {:erl_eval, :exprs, 2, []}]} ] 2025-11-27T08:12:04.285600612+00:00 stderr F (log_finished) info: rabbitmq start (call 13, PID 107) exited with status 0 (execution time 10.114s) 2025-11-27T08:12:04.428850071+00:00 stderr F (log_execute) info: executing - rsc:rabbitmq action:notify call_id:38 2025-11-27T08:12:04.446884350+00:00 stderr F (log_finished) info: rabbitmq notify (call 38, PID 580) exited with status 0 (execution time 18ms) 2025-11-27T08:12:19.521047914+00:00 stderr F (log_execute) info: executing - rsc:rabbitmq action:notify call_id:39 2025-11-27T08:12:19.538208906+00:00 stderr F (log_finished) info: rabbitmq notify (call 39, PID 585) exited with status 0 (execution time 18ms) 2025-11-27T08:12:34.664685480+00:00 stderr F (log_execute) info: executing - rsc:rabbitmq action:notify call_id:41 2025-11-27T08:12:34.675639040+00:00 stderr F (log_finished) info: rabbitmq notify (call 41, PID 592) exited with status 0 (execution time 11ms) 2025-11-27T09:04:09.664400753+00:00 stderr F (log_execute) info: executing - rsc:rabbitmq action:notify call_id:3240 2025-11-27T09:04:12.018824367+00:00 stderr F (log_finished) info: rabbitmq notify (call 3240, PID 45444) exited with status 0 (execution time 2.354s) 2025-11-27T09:04:16.315345293+00:00 stderr F (log_execute) info: executing - rsc:rabbitmq action:notify call_id:3254 2025-11-27T09:04:16.328371620+00:00 stderr F (log_finished) info: rabbitmq notify (call 3254, PID 45449) exited with status 0 (execution time 13ms) 2025-11-27T09:04:21.459472673+00:00 stderr F (log_execute) info: executing - rsc:rabbitmq action:notify call_id:3255 2025-11-27T09:04:21.477909069+00:00 stderr F (log_finished) info: rabbitmq notify (call 3255, PID 45452) exited with status 0 (execution time 19ms) 2025-11-27T09:04:21.523624677+00:00 stderr F (cancel_recurring_action) info: Cancelling ocf operation rabbitmq_monitor_10000 2025-11-27T09:04:21.606957481+00:00 stderr F (log_execute) info: executing - rsc:rabbitmq action:stop call_id:3257 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ Error: ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ {{:case_clause, {:internal_user, :_, :_, :_, :_, :_}}, [{:erl_eval, :case_clauses, 6, [file: 'erl_eval.erl', line: 973]}, {:erl_eval, :expr, 5, [file: 'erl_eval.erl', line: 446]}, {:erl_eval, :exprs, 5, [file: 'erl_eval.erl', line: 123]}, {:erl_eval, :expr, 5, [file: 'erl_eval.erl', line: 459]}, {:erl_eval, :exprs, 2, []}]} ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ Error: unable to perform an operation on node 'rabbit@np0005537441.internalapi.localdomain'. Please see diagnostics information and suggestions below. ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ Most common reasons for this are: ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues) ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server) ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * Target node is not running ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ In addition to the diagnostics info below: ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more ] 2025-11-27T09:04:25.178621699+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * Consult server logs on node rabbit@np0005537441.internalapi.localdomain ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * If target node is configured to use long node names, don't forget to use --longnames with CLI tools ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ DIAGNOSTICS ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ =========== ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ attempted to contact: ['rabbit@np0005537441.internalapi.localdomain'] ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ rabbit@np0005537441.internalapi.localdomain: ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * connected to epmd (port 4369) on np0005537441.internalapi.localdomain ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * epmd reports: node 'rabbit' not running at all ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ no other nodes on np0005537441.internalapi.localdomain ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * suggestion: start the node ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ Current node details: ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * node name: 'rabbitmqcli-84-rabbit@np0005537441.internalapi.localdomain' ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * effective user's home directory: /var/lib/rabbitmq ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ * Erlang cookie hash: hZzHGw2NwIA9uX7hAudEag== ] 2025-11-27T09:04:25.178680811+00:00 stderr F (log_op_output) info: rabbitmq_stop_0[45455] error output [ ] 2025-11-27T09:04:25.178735863+00:00 stderr F (log_finished) info: rabbitmq stop (call 3257, PID 45455) exited with status 0 (execution time 3.572s) 2025-11-27T09:04:25.321742661+00:00 stderr F (lrmd_remote_client_msg) info: Remote client disconnected while reading from it 2025-11-27T09:04:25.321779972+00:00 stderr F (lrmd_remote_client_destroy) notice: Cleaning up after remote client pacemaker-remote-np0005537441:3122 disconnected 2025-11-27T09:04:25.604736653+00:00 stderr F (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) 2025-11-27T09:04:25.604736653+00:00 stderr F (lrmd_exit) info: Terminating with 0 clients 2025-11-27T09:04:25.604736653+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-11-27T09:04:25.604895538+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-11-27T09:04:25.604930259+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-11-27T09:04:25.604977061+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-11-27T09:04:25.605009542+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-11-27T09:04:25.605046983+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-11-27T09:04:25.605083014+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-11-27T09:04:25.605135526+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-11-27T09:04:25.605256719+00:00 stderr F (crm_exit) info: Exiting pacemaker-remoted | with status 0