Mar 07 15:51:43 np0005641307.novalocal systemd[1]: Starting HotsTac(k)os OVN Northd Service...
Mar 07 15:51:43 np0005641307.novalocal podman[84212]: Error: no container with name or ID "hotstack-os-ovn-northd" found: no such container
Mar 07 15:51:43 np0005641307.novalocal hotstack-healthcheck.sh[84319]: Waiting for hotstack-os-ovn-northd to become healthy...
Mar 07 15:51:43 np0005641307.novalocal hotstack-healthcheck.sh[84319]:   Command: sh -c test -f /run/ovn/ovn-northd.pid && kill -0 $(cat /run/ovn/ovn-northd.pid) && ss -tln | grep -q :6641 && ss -tln | grep -q :6642
Mar 07 15:51:43 np0005641307.novalocal hotstack-healthcheck.sh[84319]:   Interval: 10s, Timeout: 5s, Retries: 5, Start Period: 20s
Mar 07 15:51:43 np0005641307.novalocal hotstack-healthcheck.sh[84319]:   Waiting 20s start period...
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07 15:51:43.273706737 -0500 EST m=+0.086226665 container create b4e8c7198d3d8700a6773745d5101c6cc3372253b5052e6cc8396bfff3bed042 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07 15:51:43.211547729 -0500 EST m=+0.024067667 image pull a9f3101b4ce182379971ca81be77dfa6d6bceaa91a11e080f7011c8455ddb33a localhost/hotstack-os-ovn:latest
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07 15:51:43.557554126 -0500 EST m=+0.370074044 container init b4e8c7198d3d8700a6773745d5101c6cc3372253b5052e6cc8396bfff3bed042 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07 15:51:43.566476423 -0500 EST m=+0.378996361 container start b4e8c7198d3d8700a6773745d5101c6cc3372253b5052e6cc8396bfff3bed042 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: Initializing OVN databases...
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: Initializing OVN databases...
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07 15:51:43.571649594 -0500 EST m=+0.384169532 container attach b4e8c7198d3d8700a6773745d5101c6cc3372253b5052e6cc8396bfff3bed042 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: Creating OVN Northbound database...
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: Creating OVN Northbound database...
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: Creating OVN Southbound database...
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: Creating OVN Southbound database...
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: Starting OVN databases...
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: Starting OVN databases...
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: OVN databases started
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: OVN databases started
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: Starting OVN northd...
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: Starting OVN northd...
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:51:43Z|00001|ovn_northd|INFO|OVN internal version is : [23.03.1-20.27.0-70.6]
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07T20:51:43Z|00001|ovn_northd|INFO|OVN internal version is : [23.03.1-20.27.0-70.6]
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07T20:51:43Z|00002|reconnect|INFO|unix:/run/ovn/ovnnb_db.sock: connecting...
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07T20:51:43Z|00003|reconnect|INFO|unix:/run/ovn/ovnnb_db.sock: connected
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07T20:51:43Z|00004|ovn_northd|INFO|OVN NB IDL reconnected, force recompute.
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:51:43Z|00002|reconnect|INFO|unix:/run/ovn/ovnnb_db.sock: connecting...
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07T20:51:43Z|00005|reconnect|INFO|unix:/run/ovn/ovnsb_db.sock: connecting...
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07T20:51:43Z|00006|reconnect|INFO|unix:/run/ovn/ovnsb_db.sock: connected
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:51:43Z|00003|reconnect|INFO|unix:/run/ovn/ovnnb_db.sock: connected
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:51:43Z|00004|ovn_northd|INFO|OVN NB IDL reconnected, force recompute.
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:51:43Z|00005|reconnect|INFO|unix:/run/ovn/ovnsb_db.sock: connecting...
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:51:43Z|00006|reconnect|INFO|unix:/run/ovn/ovnsb_db.sock: connected
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:51:43Z|00007|ovn_northd|INFO|OVN SB IDL reconnected, force recompute.
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07T20:51:43Z|00007|ovn_northd|INFO|OVN SB IDL reconnected, force recompute.
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07T20:51:43Z|00008|ovn_northd|INFO|ovn-northd lock acquired. This ovn-northd instance is now active.
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:51:43Z|00008|ovn_northd|INFO|ovn-northd lock acquired. This ovn-northd instance is now active.
Mar 07 15:51:43 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:51:43Z|00009|memory_trim|INFO|Detected inactivity (last active 0 ms ago): trimming memory
Mar 07 15:51:43 np0005641307.novalocal podman[84318]: 2026-03-07T20:51:43Z|00009|memory_trim|INFO|Detected inactivity (last active 0 ms ago): trimming memory
Mar 07 15:52:03 np0005641307.novalocal hotstack-healthcheck.sh[84319]:   Attempt 1/5...
Mar 07 15:52:03 np0005641307.novalocal podman[85187]: 2026-03-07 15:52:03.28858157 -0500 EST m=+0.091570690 container exec b4e8c7198d3d8700a6773745d5101c6cc3372253b5052e6cc8396bfff3bed042 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:52:03 np0005641307.novalocal podman[85187]: 2026-03-07 15:52:03.305317624 -0500 EST m=+0.108306734 container exec_died b4e8c7198d3d8700a6773745d5101c6cc3372253b5052e6cc8396bfff3bed042 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:52:03 np0005641307.novalocal hotstack-healthcheck.sh[84319]: [OK] hotstack-os-ovn-northd is healthy
Mar 07 15:52:03 np0005641307.novalocal systemd[1]: Started HotsTac(k)os OVN Northd Service.
Mar 07 15:52:03 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:52:03Z|00010|memory|INFO|9728 kB peak resident set size after 20.0 seconds
Mar 07 15:52:03 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:52:03Z|00011|memory|INFO|idl-cells-OVN_Northbound:12 idl-cells-OVN_Southbound:176
Mar 07 15:52:03 np0005641307.novalocal podman[84318]: 2026-03-07T20:52:03Z|00010|memory|INFO|9728 kB peak resident set size after 20.0 seconds
Mar 07 15:52:03 np0005641307.novalocal podman[84318]: 2026-03-07T20:52:03Z|00011|memory|INFO|idl-cells-OVN_Northbound:12 idl-cells-OVN_Southbound:176
Mar 07 15:53:03 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:53:03Z|00012|memory_trim|INFO|Dropped 1 log messages in last 80 seconds (most recently, 80 seconds ago) due to excessive rate
Mar 07 15:53:03 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:53:03Z|00013|memory_trim|INFO|Detected inactivity (last active 30030 ms ago): trimming memory
Mar 07 15:53:03 np0005641307.novalocal podman[84318]: 2026-03-07T20:53:03Z|00012|memory_trim|INFO|Dropped 1 log messages in last 80 seconds (most recently, 80 seconds ago) due to excessive rate
Mar 07 15:53:03 np0005641307.novalocal podman[84318]: 2026-03-07T20:53:03Z|00013|memory_trim|INFO|Detected inactivity (last active 30030 ms ago): trimming memory
Mar 07 15:54:17 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:54:17Z|00014|memory_trim|INFO|Detected inactivity (last active 30028 ms ago): trimming memory
Mar 07 15:54:17 np0005641307.novalocal podman[84318]: 2026-03-07T20:54:17Z|00014|memory_trim|INFO|Detected inactivity (last active 30028 ms ago): trimming memory
Mar 07 15:56:12 np0005641307.novalocal systemd[1]: Stopping HotsTac(k)os OVN Northd Service...
Mar 07 15:56:12 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:56:12Z|00001|fatal_signal(stopwatch0)|WARN|terminating with signal 15 (Terminated)
Mar 07 15:56:12 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:56:12Z|00015|fatal_signal|WARN|terminating with signal 15 (Terminated)
Mar 07 15:56:12 np0005641307.novalocal podman[84318]: 2026-03-07T20:56:12Z|00001|fatal_signal(stopwatch0)|WARN|terminating with signal 15 (Terminated)
Mar 07 15:56:12 np0005641307.novalocal podman[84318]: 2026-03-07T20:56:12Z|00015|fatal_signal|WARN|terminating with signal 15 (Terminated)
Mar 07 15:56:12 np0005641307.novalocal podman[84318]: 2026-03-07T20:56:12Z|00001|fatal_signal(urcu1)|WARN|terminating with signal 15 (Terminated)
Mar 07 15:56:12 np0005641307.novalocal hotstack-os-ovn-northd[84520]: 2026-03-07T20:56:12Z|00001|fatal_signal(urcu1)|WARN|terminating with signal 15 (Terminated)
Mar 07 15:56:13 np0005641307.novalocal podman[84318]: 2026-03-07 15:56:13.030994411 -0500 EST m=+269.843514349 container died b4e8c7198d3d8700a6773745d5101c6cc3372253b5052e6cc8396bfff3bed042 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:56:13 np0005641307.novalocal systemd[1]: hotstack-os-ovn-northd.service: Main process exited, code=exited, status=139/n/a
Mar 07 15:56:13 np0005641307.novalocal podman[87822]: 2026-03-07 15:56:13.097851949 -0500 EST m=+0.319883134 container cleanup b4e8c7198d3d8700a6773745d5101c6cc3372253b5052e6cc8396bfff3bed042 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:56:13 np0005641307.novalocal podman[87822]: hotstack-os-ovn-northd
Mar 07 15:56:13 np0005641307.novalocal podman[87862]: 2026-03-07 15:56:13.167154051 -0500 EST m=+0.041585328 container remove b4e8c7198d3d8700a6773745d5101c6cc3372253b5052e6cc8396bfff3bed042 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:56:13 np0005641307.novalocal podman[87862]: hotstack-os-ovn-northd
Mar 07 15:56:13 np0005641307.novalocal systemd[1]: hotstack-os-ovn-northd.service: Failed with result 'exit-code'.
Mar 07 15:56:13 np0005641307.novalocal systemd[1]: Stopped HotsTac(k)os OVN Northd Service.
Mar 07 15:56:13 np0005641307.novalocal systemd[1]: hotstack-os-ovn-northd.service: Consumed 1.092s CPU time.
Mar 07 15:57:31 np0005641307.novalocal systemd[1]: Starting HotsTac(k)os OVN Northd Service...
Mar 07 15:57:31 np0005641307.novalocal podman[89349]: Error: no container with name or ID "hotstack-os-ovn-northd" found: no such container
Mar 07 15:57:31 np0005641307.novalocal hotstack-healthcheck.sh[89475]: Waiting for hotstack-os-ovn-northd to become healthy...
Mar 07 15:57:31 np0005641307.novalocal hotstack-healthcheck.sh[89475]:   Command: sh -c test -f /run/ovn/ovn-northd.pid && kill -0 $(cat /run/ovn/ovn-northd.pid) && ss -tln | grep -q :6641 && ss -tln | grep -q :6642
Mar 07 15:57:31 np0005641307.novalocal hotstack-healthcheck.sh[89475]:   Interval: 10s, Timeout: 5s, Retries: 5, Start Period: 20s
Mar 07 15:57:31 np0005641307.novalocal hotstack-healthcheck.sh[89475]:   Waiting 20s start period...
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07 15:57:31.296444099 -0500 EST m=+0.110293618 container create ef9fa592682d291ae90339323f9e945ebff48f5d303a6694319bd589fa431a70 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07 15:57:31.217133177 -0500 EST m=+0.030982716 image pull a9f3101b4ce182379971ca81be77dfa6d6bceaa91a11e080f7011c8455ddb33a localhost/hotstack-os-ovn:latest
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07 15:57:31.480336647 -0500 EST m=+0.294186166 container init ef9fa592682d291ae90339323f9e945ebff48f5d303a6694319bd589fa431a70 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: Initializing OVN databases...
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07 15:57:31.486968863 -0500 EST m=+0.300818382 container start ef9fa592682d291ae90339323f9e945ebff48f5d303a6694319bd589fa431a70 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: Starting OVN databases...
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07 15:57:31.494390287 -0500 EST m=+0.308239826 container attach ef9fa592682d291ae90339323f9e945ebff48f5d303a6694319bd589fa431a70 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: Initializing OVN databases...
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: Starting OVN databases...
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: OVN databases started
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: OVN databases started
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: Starting OVN northd...
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: Starting OVN northd...
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:57:31Z|00001|ovn_northd|INFO|OVN internal version is : [23.03.1-20.27.0-70.6]
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07T20:57:31Z|00001|ovn_northd|INFO|OVN internal version is : [23.03.1-20.27.0-70.6]
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:57:31Z|00002|reconnect|INFO|unix:/run/ovn/ovnnb_db.sock: connecting...
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:57:31Z|00003|reconnect|INFO|unix:/run/ovn/ovnnb_db.sock: connected
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:57:31Z|00004|ovn_northd|INFO|OVN NB IDL reconnected, force recompute.
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:57:31Z|00005|reconnect|INFO|unix:/run/ovn/ovnsb_db.sock: connecting...
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:57:31Z|00006|reconnect|INFO|unix:/run/ovn/ovnsb_db.sock: connected
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:57:31Z|00007|ovn_northd|INFO|OVN SB IDL reconnected, force recompute.
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07T20:57:31Z|00002|reconnect|INFO|unix:/run/ovn/ovnnb_db.sock: connecting...
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07T20:57:31Z|00003|reconnect|INFO|unix:/run/ovn/ovnnb_db.sock: connected
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07T20:57:31Z|00004|ovn_northd|INFO|OVN NB IDL reconnected, force recompute.
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07T20:57:31Z|00005|reconnect|INFO|unix:/run/ovn/ovnsb_db.sock: connecting...
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07T20:57:31Z|00006|reconnect|INFO|unix:/run/ovn/ovnsb_db.sock: connected
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07T20:57:31Z|00007|ovn_northd|INFO|OVN SB IDL reconnected, force recompute.
Mar 07 15:57:31 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:57:31Z|00008|ovn_northd|INFO|ovn-northd lock acquired. This ovn-northd instance is now active.
Mar 07 15:57:31 np0005641307.novalocal podman[89470]: 2026-03-07T20:57:31Z|00008|ovn_northd|INFO|ovn-northd lock acquired. This ovn-northd instance is now active.
Mar 07 15:57:51 np0005641307.novalocal hotstack-healthcheck.sh[89475]:   Attempt 1/5...
Mar 07 15:57:51 np0005641307.novalocal podman[90307]: 2026-03-07 15:57:51.251333371 -0500 EST m=+0.055406000 container exec ef9fa592682d291ae90339323f9e945ebff48f5d303a6694319bd589fa431a70 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:57:51 np0005641307.novalocal podman[90307]: 2026-03-07 15:57:51.297070441 -0500 EST m=+0.101143050 container exec_died ef9fa592682d291ae90339323f9e945ebff48f5d303a6694319bd589fa431a70 (image=localhost/hotstack-os-ovn:latest, name=hotstack-os-ovn-northd, io.buildah.version=1.43.0)
Mar 07 15:57:51 np0005641307.novalocal hotstack-healthcheck.sh[89475]: [OK] hotstack-os-ovn-northd is healthy
Mar 07 15:57:51 np0005641307.novalocal systemd[1]: Started HotsTac(k)os OVN Northd Service.
Mar 07 15:58:01 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:58:01Z|00009|memory|INFO|8576 kB peak resident set size after 30.0 seconds
Mar 07 15:58:01 np0005641307.novalocal podman[89470]: 2026-03-07T20:58:01Z|00009|memory|INFO|8576 kB peak resident set size after 30.0 seconds
Mar 07 15:58:01 np0005641307.novalocal podman[89470]: 2026-03-07T20:58:01Z|00010|memory|INFO|idl-cells-OVN_Northbound:340 idl-cells-OVN_Southbound:2688
Mar 07 15:58:01 np0005641307.novalocal podman[89470]: 2026-03-07T20:58:01Z|00011|memory_trim|INFO|Detected inactivity (last active 30013 ms ago): trimming memory
Mar 07 15:58:01 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:58:01Z|00010|memory|INFO|idl-cells-OVN_Northbound:340 idl-cells-OVN_Southbound:2688
Mar 07 15:58:01 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:58:01Z|00011|memory_trim|INFO|Detected inactivity (last active 30013 ms ago): trimming memory
Mar 07 15:58:35 np0005641307.novalocal hotstack-os-ovn-northd[89672]: 2026-03-07T20:58:35Z|00012|memory_trim|INFO|Detected inactivity (last active 30024 ms ago): trimming memory
Mar 07 15:58:35 np0005641307.novalocal podman[89470]: 2026-03-07T20:58:35Z|00012|memory_trim|INFO|Detected inactivity (last active 30024 ms ago): trimming memory