2025-12-01T10:58:30.419894416+00:00 stderr F + sudo -E kolla_set_configs 2025-12-01T10:58:30.431263691+00:00 stderr F sudo: unable to send audit message: Operation not permitted 2025-12-01T10:58:30.500805245+00:00 stderr F INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json 2025-12-01T10:58:30.501133684+00:00 stderr F INFO:__main__:Validating config file 2025-12-01T10:58:30.501203936+00:00 stderr F INFO:__main__:Kolla config strategy set to: COPY_ALWAYS 2025-12-01T10:58:30.501242957+00:00 stderr F INFO:__main__:Copying service configuration files 2025-12-01T10:58:30.501402441+00:00 stderr F INFO:__main__:Copying /dev/null to /etc/libqb/force-filesystem-sockets 2025-12-01T10:58:30.502240863+00:00 stderr F INFO:__main__:Setting permission for /etc/libqb/force-filesystem-sockets 2025-12-01T10:58:30.502996553+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/security/limits.d/redis.conf to /etc/security/limits.d/redis.conf 2025-12-01T10:58:30.503259179+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/redis.conf to /etc/redis.conf 2025-12-01T10:58:30.503961877+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/redis.conf.puppet to /etc/redis.conf.puppet 2025-12-01T10:58:30.504191573+00:00 stderr F INFO:__main__:Writing out command to execute 2025-12-01T10:58:30.504439509+00:00 stderr F INFO:__main__:Setting permission for /run/redis 2025-12-01T10:58:30.504566524+00:00 stderr F INFO:__main__:Setting permission for /var/lib/redis 2025-12-01T10:58:30.504691727+00:00 stderr F INFO:__main__:Setting permission for /var/log/redis 2025-12-01T10:58:30.512961542+00:00 stderr F ++ cat /run_command 2025-12-01T10:58:30.515517817+00:00 stderr F + CMD=/usr/sbin/pacemaker_remoted 2025-12-01T10:58:30.515550938+00:00 stderr F + ARGS= 2025-12-01T10:58:30.515586709+00:00 stderr F + [[ ! -n '' ]] 2025-12-01T10:58:30.515593169+00:00 stderr F + . kolla_extend_start 2025-12-01T10:58:30.515753614+00:00 stderr F + echo 'Running command: '\''/usr/sbin/pacemaker_remoted'\''' 2025-12-01T10:58:30.515777015+00:00 stdout F Running command: '/usr/sbin/pacemaker_remoted' 2025-12-01T10:58:30.515801715+00:00 stderr F + umask 0022 2025-12-01T10:58:30.515823646+00:00 stderr F + exec /usr/sbin/pacemaker_remoted 2025-12-01T10:58:30.525500156+00:00 stderr F (pcmk__add_logfile) notice: Additional logging available in /var/log/pacemaker/pacemaker.log 2025-12-01T10:58:30.525564268+00:00 stderr F (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores 2025-12-01T10:58:30.525660820+00:00 stderr F (main) notice: Starting Pacemaker remote executor 2025-12-01T10:58:30.525759083+00:00 stderr F (qb_ipcs_us_publish) info: server name: lrmd 2025-12-01T10:58:30.526182514+00:00 stderr F (pcmk__init_tls_dh) info: Generating Diffie-Hellman parameters with 2048-bit prime for TLS 2025-12-01T10:58:30.713130484+00:00 stderr F (lrmd__init_remote_key) warning: Could not read Pacemaker Remote key from default location /etc/pacemaker/authkey (or fallback location /etc/corosync/authkey): No such file or directory 2025-12-01T10:58:30.713130484+00:00 stderr F (lrmd_init_remote_tls_server) warning: A cluster connection will not be possible until the key is available 2025-12-01T10:58:30.713558085+00:00 stderr F (qb_ipcs_us_publish) info: server name: cib_ro 2025-12-01T10:58:30.713744310+00:00 stderr F (qb_ipcs_us_publish) info: server name: cib_rw 2025-12-01T10:58:30.713794271+00:00 stderr F (qb_ipcs_us_publish) info: server name: cib_shm 2025-12-01T10:58:30.713841192+00:00 stderr F (qb_ipcs_us_publish) info: server name: attrd 2025-12-01T10:58:30.713886153+00:00 stderr F (qb_ipcs_us_publish) info: server name: stonith-ng 2025-12-01T10:58:30.713930325+00:00 stderr F (qb_ipcs_us_publish) info: server name: pacemakerd 2025-12-01T10:58:30.713975386+00:00 stderr F (qb_ipcs_us_publish) info: server name: crmd 2025-12-01T10:58:30.714030697+00:00 stderr F (main) notice: Pacemaker remote executor successfully started and accepting connections 2025-12-01T10:58:30.714030697+00:00 stderr F (main) notice: OCF resource agent search path is /usr/lib/ocf/resource.d 2025-12-01T10:58:35.277666292+00:00 stderr F (crm_signal_dispatch) notice: Caught 'Terminated' signal | 15 (invoking handler) 2025-12-01T10:58:35.277666292+00:00 stderr F (lrmd_exit) info: Terminating with 0 clients 2025-12-01T10:58:35.277666292+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-12-01T10:58:35.278096724+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-12-01T10:58:35.278155406+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-12-01T10:58:35.278209067+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-12-01T10:58:35.278244738+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-12-01T10:58:35.278287609+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-12-01T10:58:35.278328450+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-12-01T10:58:35.278359581+00:00 stderr F (qb_ipcs_us_withdraw) info: withdrawing server sockets 2025-12-01T10:58:35.278427863+00:00 stderr F (crm_exit) info: Exiting pacemaker-remoted | with status 0 2025-12-01T10:58:35.847900666+00:00 stderr F + sudo -E kolla_set_configs 2025-12-01T10:58:35.861757796+00:00 stderr F sudo: unable to send audit message: Operation not permitted 2025-12-01T10:58:35.917760739+00:00 stderr F INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json 2025-12-01T10:58:35.917760739+00:00 stderr F INFO:__main__:Validating config file 2025-12-01T10:58:35.917760739+00:00 stderr F INFO:__main__:Kolla config strategy set to: COPY_ALWAYS 2025-12-01T10:58:35.917760739+00:00 stderr F INFO:__main__:Copying service configuration files 2025-12-01T10:58:35.917760739+00:00 stderr F INFO:__main__:Copying /dev/null to /etc/libqb/force-filesystem-sockets 2025-12-01T10:58:35.917760739+00:00 stderr F INFO:__main__:Setting permission for /etc/libqb/force-filesystem-sockets 2025-12-01T10:58:35.917760739+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/security/limits.d/redis.conf to /etc/security/limits.d/redis.conf 2025-12-01T10:58:35.917847631+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/redis.conf to /etc/redis.conf 2025-12-01T10:58:35.918139329+00:00 stderr F INFO:__main__:Copying /var/lib/kolla/config_files/src/etc/redis.conf.puppet to /etc/redis.conf.puppet 2025-12-01T10:58:35.918310563+00:00 stderr F INFO:__main__:Writing out command to execute 2025-12-01T10:58:35.918547329+00:00 stderr F INFO:__main__:Setting permission for /run/redis 2025-12-01T10:58:35.918667612+00:00 stderr F INFO:__main__:Setting permission for /var/lib/redis 2025-12-01T10:58:35.918835096+00:00 stderr F INFO:__main__:Setting permission for /var/log/redis 2025-12-01T10:58:35.925365556+00:00 stderr F ++ cat /run_command 2025-12-01T10:58:35.926859165+00:00 stderr F + CMD=/usr/sbin/pacemaker_remoted 2025-12-01T10:58:35.926883005+00:00 stderr F + ARGS= 2025-12-01T10:58:35.926917106+00:00 stderr F + [[ ! -n '' ]] 2025-12-01T10:58:35.926917106+00:00 stderr F + . kolla_extend_start 2025-12-01T10:58:35.927069510+00:00 stderr F + echo 'Running command: '\''/usr/sbin/pacemaker_remoted'\''' 2025-12-01T10:58:35.927075840+00:00 stdout F Running command: '/usr/sbin/pacemaker_remoted' 2025-12-01T10:58:35.927092571+00:00 stderr F + umask 0022 2025-12-01T10:58:35.927110171+00:00 stderr F + exec /usr/sbin/pacemaker_remoted 2025-12-01T10:58:35.933550578+00:00 stderr F (pcmk__add_logfile) warning: Logging to '/var/log/pacemaker/pacemaker.log' is disabled: No such file or directory | uid=0 gid=0 2025-12-01T10:58:35.933878727+00:00 stderr F (crm_log_init) info: Changed active directory to /var/lib/pacemaker/cores 2025-12-01T10:58:35.933941099+00:00 stderr F (main) notice: Starting Pacemaker remote executor 2025-12-01T10:58:35.933977170+00:00 stderr F (qb_ipcs_us_publish) info: server name: lrmd 2025-12-01T10:58:35.934776370+00:00 stderr F (pcmk__init_tls_dh) info: Generating Diffie-Hellman parameters with 2048-bit prime for TLS 2025-12-01T10:58:36.196070399+00:00 stderr F (qb_ipcs_us_publish) info: server name: cib_ro 2025-12-01T10:58:36.196070399+00:00 stderr F (qb_ipcs_us_publish) info: server name: cib_rw 2025-12-01T10:58:36.196070399+00:00 stderr F (qb_ipcs_us_publish) info: server name: cib_shm 2025-12-01T10:58:36.196070399+00:00 stderr F (qb_ipcs_us_publish) info: server name: attrd 2025-12-01T10:58:36.196070399+00:00 stderr F (qb_ipcs_us_publish) info: server name: stonith-ng 2025-12-01T10:58:36.196070399+00:00 stderr F (qb_ipcs_us_publish) info: server name: pacemakerd 2025-12-01T10:58:36.196070399+00:00 stderr F (qb_ipcs_us_publish) info: server name: crmd 2025-12-01T10:58:36.196304965+00:00 stderr F (main) notice: Pacemaker remote executor successfully started and accepting connections 2025-12-01T10:58:36.196304965+00:00 stderr F (main) notice: OCF resource agent search path is /usr/lib/ocf/resource.d 2025-12-01T10:58:36.566980812+00:00 stderr F (pcmk__accept_remote_connection) info: Accepted new remote client connection from ::ffff:172.17.0.100 2025-12-01T10:58:36.567279940+00:00 stderr F (lrmd_remote_listen) info: Remote client pending authentication | 0x559674f3c6d0 id: db7d0f94-76dd-44b1-b5bb-28abafbf4f96 2025-12-01T10:58:37.110183074+00:00 stderr F (remoted__read_handshake_data) notice: Remote client connection accepted 2025-12-01T10:58:37.245019482+00:00 stderr F (process_lrmd_get_rsc_info) info: Agent information for 'redis' not in cache 2025-12-01T10:58:37.286015985+00:00 stderr F (process_lrmd_get_rsc_info) info: Agent information for 'redis:0' not in cache 2025-12-01T10:58:37.369758698+00:00 stderr F (process_lrmd_rsc_register) info: Cached agent information for 'redis' 2025-12-01T10:58:37.764175931+00:00 stderr F (log_execute) info: executing - rsc:redis action:start call_id:8 2025-12-01T10:58:41.684298521+00:00 stderr F (log_op_output) info: redis_start_0[55] error output [ Schema validation of configuration is disabled (enabling is encouraged and prevents common misconfigurations) ] 2025-12-01T10:58:41.684298521+00:00 stderr F (log_finished) info: redis start (call 8, PID 55) exited with status 0 (execution time 3.921s) 2025-12-01T10:58:41.763348592+00:00 stderr F (log_execute) info: executing - rsc:redis action:notify call_id:38 2025-12-01T10:58:41.822113637+00:00 stderr F (log_finished) info: redis notify (call 38, PID 217) exited with status 0 (execution time 59ms) 2025-12-01T10:58:42.062618125+00:00 stderr F (log_execute) info: executing - rsc:redis action:notify call_id:39 2025-12-01T10:58:42.112171371+00:00 stderr F (log_finished) info: redis notify (call 39, PID 263) exited with status 0 (execution time 49ms) 2025-12-01T10:58:42.155221197+00:00 stderr F (log_execute) info: executing - rsc:redis action:promote call_id:40 2025-12-01T10:58:43.026421489+00:00 stderr F (log_finished) info: redis promote (call 40, PID 309) exited with status 0 (execution time 872ms) 2025-12-01T10:58:43.106283392+00:00 stderr F (log_execute) info: executing - rsc:redis action:notify call_id:53 2025-12-01T10:58:43.433872140+00:00 stderr F (log_finished) info: redis notify (call 53, PID 382) exited with status 0 (execution time 327ms) 2025-12-01T11:01:27.783004181+00:00 stderr F (lrmd_remote_client_msg) info: Remote client disconnected while reading from it 2025-12-01T11:01:27.783152995+00:00 stderr F (lrmd_remote_client_destroy) notice: Cleaning up after remote client pacemaker-remote-standalone:3124 disconnected 2025-12-01T11:01:27.783152995+00:00 stderr F (ipc_proxy_remove_provider) info: ipc proxy connection for client 99e4475b-7c36-40fb-a9eb-0b1e8fd7da04 pid 905 destroyed because cluster node disconnected. 2025-12-01T11:01:27.783346440+00:00 stderr F (send_tls) info: TLS connection terminated: The specified session has been invalidated for some reason. | rc=-10 2025-12-01T11:01:27.783457082+00:00 stderr F (pcmk__remote_send_xml) error: Could not send remote message: Software caused connection abort | rc=103 2025-12-01T11:01:27.784364766+00:00 stderr F (cancel_recurring_action) info: Cancelling ocf operation redis_monitor_20000 2025-12-01T11:01:27.784364766+00:00 stderr F (services_action_cancel) info: Terminating in-flight op redis_monitor_20000[848] early because it was cancelled 2025-12-01T11:01:27.785557985+00:00 stderr F (async_action_complete) info: redis_monitor_20000[848] terminated with signal 9 (Killed) 2025-12-01T11:01:27.785557985+00:00 stderr F (cancel_recurring_action) info: Cancelling ocf operation redis_monitor_20000 2025-12-01T11:01:27.785655797+00:00 stderr F (send_tls) info: TLS connection terminated: The specified session has been invalidated for some reason. | rc=-10 2025-12-01T11:01:27.785803381+00:00 stderr F (pcmk__remote_send_xml) error: Could not send remote message: Software caused connection abort | rc=103 2025-12-01T11:01:27.785861032+00:00 stderr F (send_client_notify) warning: Could not notify client pacemaker-remote-standalone:3124: Software caused connection abort | rc=103 2025-12-01T11:02:20.151066899+00:00 stderr F (pcmk__accept_remote_connection) info: Accepted new remote client connection from ::ffff:172.17.0.100 2025-12-01T11:02:20.151264174+00:00 stderr F (lrmd_remote_listen) info: Remote client pending authentication | 0x559674f4d250 id: 4991ee58-b299-4504-a353-152a0529e1f5 2025-12-01T11:02:20.693837958+00:00 stderr F (remoted__read_handshake_data) notice: Remote client connection accepted 2025-12-01T12:09:14.508660450+00:00 stderr F (log_execute) info: executing - rsc:redis action:notify call_id:1025 2025-12-01T12:09:14.602058467+00:00 stderr F (log_finished) info: redis notify (call 1025, PID 12511) exited with status 0 (execution time 95ms) 2025-12-01T12:09:14.646279718+00:00 stderr F (cancel_recurring_action) info: Cancelling ocf operation redis_monitor_20000 2025-12-01T12:09:14.731475579+00:00 stderr F (log_execute) info: executing - rsc:redis action:demote call_id:1027 2025-12-01T12:09:14.858356143+00:00 stderr F (log_finished) info: redis demote (call 1027, PID 12557) exited with status 7 (execution time 128ms) 2025-12-01T12:09:14.904445135+00:00 stderr F (log_execute) info: executing - rsc:redis action:notify call_id:1028 2025-12-01T12:09:14.984884417+00:00 stderr F (log_finished) info: redis notify (call 1028, PID 12619) exited with status 0 (execution time 80ms) 2025-12-01T12:09:15.074240802+00:00 stderr F (log_execute) info: executing - rsc:redis action:notify call_id:1029 2025-12-01T12:09:15.160368871+00:00 stderr F (log_finished) info: redis notify (call 1029, PID 12665) exited with status 0 (execution time 87ms) 2025-12-01T12:09:15.204599462+00:00 stderr F (log_execute) info: executing - rsc:redis action:stop call_id:1030 2025-12-01T12:09:15.545158885+00:00 stderr F (log_finished) info: redis stop (call 1030, PID 12711) exited with status 0 (execution time 340ms) 2025-12-01T12:09:15.757440946+00:00 stderr F (log_execute) info: executing - rsc:redis action:start call_id:1036 2025-12-01T12:12:35.761019246+00:00 stderr F (async_action_complete) info: Resource agent redis_start_0[12764] timed out after 3m20s 2025-12-01T12:12:35.761019246+00:00 stderr F (log_finished) info: redis start (call 1036, PID 12764) could not be executed: Timed Out (Resource agent did not complete within 3m20s) (execution time 3m20.002s) 2025-12-01T12:12:35.813398770+00:00 stderr F (log_execute) info: executing - rsc:redis action:notify call_id:1044 2025-12-01T12:12:35.910004485+00:00 stderr F (log_finished) info: redis notify (call 1044, PID 13032) exited with status 0 (execution time 96ms) 2025-12-01T12:12:36.026995337+00:00 stderr F (log_execute) info: executing - rsc:redis action:notify call_id:1045 2025-12-01T12:12:36.111502331+00:00 stderr F (log_finished) info: redis notify (call 1045, PID 13078) exited with status 0 (execution time 84ms) 2025-12-01T12:12:36.154537118+00:00 stderr F (log_execute) info: executing - rsc:redis action:demote call_id:1046 2025-12-01T12:12:36.262958507+00:00 stderr F (log_finished) info: redis demote (call 1046, PID 13124) exited with status 7 (execution time 108ms) 2025-12-01T12:12:36.308527084+00:00 stderr F (log_execute) info: executing - rsc:redis action:notify call_id:1047 2025-12-01T12:12:36.397980903+00:00 stderr F (log_finished) info: redis notify (call 1047, PID 13186) exited with status 0 (execution time 89ms) 2025-12-01T12:12:36.493459777+00:00 stderr F (log_execute) info: executing - rsc:redis action:notify call_id:1048 2025-12-01T12:12:36.572799959+00:00 stderr F (log_finished) info: redis notify (call 1048, PID 13232) exited with status 0 (execution time 79ms) 2025-12-01T12:12:36.616267019+00:00 stderr F (log_execute) info: executing - rsc:redis action:stop call_id:1049 2025-12-01T12:12:36.943045475+00:00 stderr F (log_finished) info: redis stop (call 1049, PID 13278) exited with status 0 (execution time 327ms)