Nov 25 13:00:48.013692 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-582c1f1e-8b98-4fc6-9356-0be2fc54da55', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-298f4d65-a755-4b13-bea3-495f0cc14548', 'neutron:subnet_ids': '50767475-6d85-4ae4-99ed-c90ebc1b6f68'}, mac=fa:16:3e:e0:44:19, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-57b46412-60a4-41bf-b294-2936df5b0aaa, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:00:48.044203 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-582c1f1e-8b98-4fc6-9356-0be2fc54da55 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:00:48.045234 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-582c1f1e-8b98-4fc6-9356-0be2fc54da55, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '582c1f1e-8b98-4fc6-9356-0be2fc54da55', 'neutron:router_id': '298f4d65-a755-4b13-bea3-495f0cc14548'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:48.045234 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-582c1f1e-8b98-4fc6-9356-0be2fc54da55, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:50.579210 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=98974) _process_scheduled /usr/local/lib/python3.12/site-packages/futurist/periodics.py:638}} Nov 25 13:00:50.590284 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}} Nov 25 13:00:50.590502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Number of inconsistencies found at create/update: networks=1, security_groups=1, routers=1, router_ports=1 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:00:50.590740 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Number of inconsistencies found at delete: router_ports=15 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:00:50.590920 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource b35bfd86-f1c6-464b-917a-de86934fd7e8 (type: networks) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:00:50.638724 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-b35bfd86-f1c6-464b-917a-de86934fd7e8, resource={'id': 'b35bfd86-f1c6-464b-917a-de86934fd7e8', 'name': 'tempest-ServerActionsTestJSON-627147231-network', 'tenant_id': '32b689fc56464babbf2b70be40e620f8', 'project_id': '32b689fc56464babbf2b70be40e620f8', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['0586b32e-06b5-4106-a48e-ad8816ba6384'], 'standard_attr_id': 476, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2025-11-25T17:59:34Z', 'updated_at': '2025-11-25T17:59:35Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 43159}, resource_type=networks, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:50.639178 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-b35bfd86-f1c6-464b-917a-de86934fd7e8, col_values=(('external_ids', {'neutron:network_name': 'tempest-ServerActionsTestJSON-627147231-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve', 'neutron:provnet-physical-network': ''}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:50.639701 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d92c314e-c96e-46d3-8052-e3d336de2855, external_ids_update=None, columns={'tag': 43159}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:50.687369 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Successfully bumped revision number for resource b35bfd86-f1c6-464b-917a-de86934fd7e8 (type: networks) to 2 Nov 25 13:00:50.687517 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 59d18c08-1773-4843-9d6b-0fc5c912e018 (type: security_groups) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:00:50.713408 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Successfully bumped revision number for resource 59d18c08-1773-4843-9d6b-0fc5c912e018 (type: security_groups) to 3 Nov 25 13:00:50.713584 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource ead643f8-be77-4e46-ab58-587aa84f4dac (type: routers) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:00:50.772323 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-ead643f8-be77-4e46-ab58-587aa84f4dac, resource={'id': 'ead643f8-be77-4e46-ab58-587aa84f4dac', 'name': 'tempest-ServerActionsTestJSON-627147231-router', 'tenant_id': '32b689fc56464babbf2b70be40e620f8', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.130'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::3a8'}], 'enable_snat': True}, 'gw_port_id': '7c580102-8302-4f95-bd9c-d4ca0907d53c', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'ip_address': '172.24.5.130', 'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59'}, {'ip_address': '2001:db8::3a8', 'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2025-11-25T17:59:35Z', 'updated_at': '2025-11-25T17:59:37Z', 'revision_number': 3, 'project_id': '32b689fc56464babbf2b70be40e620f8'}, resource_type=routers, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:50.772596 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-ead643f8-be77-4e46-ab58-587aa84f4dac, columns={'external_ids': {'neutron:router_name': 'tempest-ServerActionsTestJSON-627147231-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:50.772670 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "7c580102-8302-4f95-bd9c-d4ca0907d53c"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:50.772732 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "7c580102-8302-4f95-bd9c-d4ca0907d53c"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:50.850917 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Successfully bumped revision number for resource ead643f8-be77-4e46-ab58-587aa84f4dac (type: routers) to 3 Nov 25 13:00:50.850999 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 7c580102-8302-4f95-bd9c-d4ca0907d53c (type: router_ports) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:00:50.947130 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-7c580102-8302-4f95-bd9c-d4ca0907d53c, resource={'id': '7c580102-8302-4f95-bd9c-d4ca0907d53c', 'name': '', 'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'tenant_id': '', 'mac_address': 'fa:16:3e:d5:98:c6', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ead643f8-be77-4e46-ab58-587aa84f4dac', 'device_owner': 'network:router_gateway', 'standard_attr_id': 481, 'fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.130'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::3a8'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0005535648.novalocal', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2025-11-25T17:59:35Z', 'updated_at': '2025-11-25T17:59:38Z', 'revision_number': 5, 'project_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:50.947505 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-7c580102-8302-4f95-bd9c-d4ca0907d53c, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-ead643f8-be77-4e46-ab58-587aa84f4dac'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::3a8/64', '172.24.5.130/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:50.948009 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=7c580102-8302-4f95-bd9c-d4ca0907d53c, lrouter_port=lrp-7c580102-8302-4f95-bd9c-d4ca0907d53c, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:00:50.973872 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Successfully bumped revision number for resource 7c580102-8302-4f95-bd9c-d4ca0907d53c (type: router_ports) to 5 Nov 25 13:00:50.974020 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 08d08798-e14a-4246-9c50-29973dc55258 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:50.974284 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(08d08798-e14a-4246-9c50-29973dc55258) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:50.977204 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource d13227fb-8aa8-4a0c-a8e5-9d2b1a10f94f (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:50.977423 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(d13227fb-8aa8-4a0c-a8e5-9d2b1a10f94f) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:50.980467 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource cee5575c-dd53-4b3d-ad81-0580c5c939d9 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:50.980707 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(cee5575c-dd53-4b3d-ad81-0580c5c939d9) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:50.984168 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource bd989c52-0c89-4ece-8b2f-0252d32c82f2 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:50.984386 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(bd989c52-0c89-4ece-8b2f-0252d32c82f2) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:50.987631 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource a3db1a0e-c983-4e94-b845-6a8e12988b6d (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:50.987848 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(a3db1a0e-c983-4e94-b845-6a8e12988b6d) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:50.991154 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 8fae90f5-3c45-4165-9775-b5df549fb292 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:50.991387 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(8fae90f5-3c45-4165-9775-b5df549fb292) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:50.994007 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 88f3a49e-9afa-4592-94cd-17599f958aa6 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:50.994220 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(88f3a49e-9afa-4592-94cd-17599f958aa6) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:50.996950 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 749731d6-0009-45f5-87bc-8571ae29c089 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:50.997168 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(749731d6-0009-45f5-87bc-8571ae29c089) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:51.000384 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 506d9448-f9e2-40ab-a13e-88055047d507 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:51.000692 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(506d9448-f9e2-40ab-a13e-88055047d507) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:51.005309 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 4e0fdb7f-1a4a-4c11-92fb-9a72f1d5030e (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:51.005580 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(4e0fdb7f-1a4a-4c11-92fb-9a72f1d5030e) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:51.008948 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 407731dd-352c-4e2a-a4cb-b171319c7c4d (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:51.009275 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(407731dd-352c-4e2a-a4cb-b171319c7c4d) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:51.011770 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 2b9a6d1b-cfa5-4d1b-a156-53ed690f9418 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:51.011950 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(2b9a6d1b-cfa5-4d1b-a156-53ed690f9418) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:51.014717 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 255b9ebf-5c2d-4af9-9fd9-555e3544fe98 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:51.014948 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(255b9ebf-5c2d-4af9-9fd9-555e3544fe98) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:51.018153 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 0b493dca-c6a9-4ca5-85ca-bc2a49f8ddb4 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:51.018366 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(0b493dca-c6a9-4ca5-85ca-bc2a49f8ddb4) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:51.021926 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource f31cd819-368e-4d05-ac6a-ed7a59ab4754 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:00:51.022133 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] delete_revision(f31cd819-368e-4d05-ac6a-ed7a59ab4754) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:00:51.025278 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Synchronization completed (took 0.43 seconds) Nov 25 13:01:01.023908 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=231,neutron:UPDATE=4,neutron:DELETE=15 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:01:12.059424 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-b843d9e8-6aa9-4e69-a20a-e91fcff2d4c3', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-a5ea25c9-8fe4-4407-8591-c285affc79aa', 'neutron:subnet_ids': '377d180a-777c-4d2f-b5c7-f7631bf97894'}, mac=fa:16:3e:42:d4:42, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-a45c3aa7-c7c1-4846-85d6-89a54784c5ee, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:01:12.063960 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-b843d9e8-6aa9-4e69-a20a-e91fcff2d4c3, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:01:12.263434 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-582c1f1e-8b98-4fc6-9356-0be2fc54da55', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-298f4d65-a755-4b13-bea3-495f0cc14548', 'neutron:subnet_ids': '50767475-6d85-4ae4-99ed-c90ebc1b6f68'}, mac=fa:16:3e:e0:44:19, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-57b46412-60a4-41bf-b294-2936df5b0aaa, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:01:12.265879 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-582c1f1e-8b98-4fc6-9356-0be2fc54da55, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-a5ea25c9-8fe4-4407-8591-c285affc79aa Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:12.629298 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:01:12.629839 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:01:12.629839 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-a5ea25c9-8fe4-4407-8591-c285affc79aa Nov 25 13:01:12.629839 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-298f4d65-a755-4b13-bea3-495f0cc14548 Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:12.995949 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:01:12.996289 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:01:12.996289 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-298f4d65-a755-4b13-bea3-495f0cc14548 Nov 25 13:01:12.996289 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:01:16.735038 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-c4a74776-f18b-4025-b770-e625dea17fd2', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:6a:fd:ef, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.142/24', '2001:db8::1b5/64'], name=lrp-b50a7bd3-aa28-46c9-8f74-e3fe3655818f, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:01:19.035603 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-e9c2eedd-e21e-4bc3-b41c-2d82b166b39c', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-c4a74776-f18b-4025-b770-e625dea17fd2', 'neutron:subnet_ids': '93630e32-39eb-480a-bb9a-efae3ef1db93'}, mac=fa:16:3e:f8:03:11, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-e6268043-0539-4a74-aa13-2271ad0b8d7b, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:01:19.079365 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-e9c2eedd-e21e-4bc3-b41c-2d82b166b39c synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:01:19.079812 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-e9c2eedd-e21e-4bc3-b41c-2d82b166b39c, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': 'e9c2eedd-e21e-4bc3-b41c-2d82b166b39c', 'neutron:router_id': 'c4a74776-f18b-4025-b770-e625dea17fd2'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:01:19.080092 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-e9c2eedd-e21e-4bc3-b41c-2d82b166b39c, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:01:22.281284 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-32832586-1400-48d6-b7b4-26f966057f1d', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:d2:c5:4c, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.178/24', '2001:db8::205/64'], name=lrp-ef4e1b01-07e8-41d8-868c-f0c4fd858c1a, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:01:23.605503 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-d6f5f373-12dc-4ce4-a4d8-7a948bb7d742', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-32832586-1400-48d6-b7b4-26f966057f1d', 'neutron:subnet_ids': '0c139ec3-8e37-4dd3-9c62-617558323606'}, mac=fa:16:3e:da:db:83, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-cbc33c6a-bdbe-4a62-8df0-038be7a77604, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:01:23.651641 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-d6f5f373-12dc-4ce4-a4d8-7a948bb7d742 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:01:23.652404 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-d6f5f373-12dc-4ce4-a4d8-7a948bb7d742, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': 'd6f5f373-12dc-4ce4-a4d8-7a948bb7d742', 'neutron:router_id': '32832586-1400-48d6-b7b4-26f966057f1d'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:01:23.653317 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-d6f5f373-12dc-4ce4-a4d8-7a948bb7d742, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:01:33.650099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=34 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:01:36.341639 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-3d874be2-37f2-4f60-990b-a3259e8f9985', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-8b37ba08-76c8-4f67-9a92-be88737264c6', 'neutron:subnet_ids': 'ae1c39be-03f7-4ff7-8dd3-244fbc23a2e6'}, mac=fa:16:3e:7f:3b:e2, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-24bed290-4c25-4866-be15-b94bea8700e6, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:01:36.342192 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-3d874be2-37f2-4f60-990b-a3259e8f9985, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-8b37ba08-76c8-4f67-9a92-be88737264c6 Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:01:36.889392 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:01:36.890151 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:01:36.890151 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-8b37ba08-76c8-4f67-9a92-be88737264c6 Nov 25 13:01:36.890151 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:01:40.806739 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-f033da69-f81d-47ee-9a9a-f421eb95bfd6', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:04:68:8d, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.38/24', '2001:db8::2f3/64'], name=lrp-f1fec9e8-7099-43d7-a3a3-68cddd8b0630, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:01:41.969445 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-da47964d-cb94-4b30-8e4c-e7346efd158a', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-f033da69-f81d-47ee-9a9a-f421eb95bfd6', 'neutron:subnet_ids': '314a76dc-288c-4a97-8860-40ca25406c50'}, mac=fa:16:3e:d6:25:27, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-bade92d8-466f-4f31-80ef-78b49acf4ce3, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:01:42.005222 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-da47964d-cb94-4b30-8e4c-e7346efd158a synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:01:42.005914 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-da47964d-cb94-4b30-8e4c-e7346efd158a, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': 'da47964d-cb94-4b30-8e4c-e7346efd158a', 'neutron:router_id': 'f033da69-f81d-47ee-9a9a-f421eb95bfd6'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:01:42.006107 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-da47964d-cb94-4b30-8e4c-e7346efd158a, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:01:52.003725 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:02:03.232602 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-950fae5f-40ea-4668-af0a-383569d6040d', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:94:bd:82, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.225/24', '2001:db8::2ce/64'], name=lrp-3df4f1ac-98a2-492d-81a6-7c95aeff6a92, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:02:05.112940 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-8437b51a-aaa7-4859-9d9b-3e07ced7e456', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-950fae5f-40ea-4668-af0a-383569d6040d', 'neutron:subnet_ids': '2d238a05-4434-4988-8d99-26217f6745fd'}, mac=fa:16:3e:60:ba:24, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-7968267e-8cf9-4347-b772-161c6b2cfe9f, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:02:05.137744 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-8437b51a-aaa7-4859-9d9b-3e07ced7e456 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:02:05.137744 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-8437b51a-aaa7-4859-9d9b-3e07ced7e456, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '8437b51a-aaa7-4859-9d9b-3e07ced7e456', 'neutron:router_id': '950fae5f-40ea-4668-af0a-383569d6040d'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:02:05.137744 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-8437b51a-aaa7-4859-9d9b-3e07ced7e456, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:02:15.135809 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:02:32.701623 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-da47964d-cb94-4b30-8e4c-e7346efd158a', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-f033da69-f81d-47ee-9a9a-f421eb95bfd6', 'neutron:subnet_ids': '314a76dc-288c-4a97-8860-40ca25406c50'}, mac=fa:16:3e:d6:25:27, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-bade92d8-466f-4f31-80ef-78b49acf4ce3, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:02:32.702477 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-da47964d-cb94-4b30-8e4c-e7346efd158a, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-f033da69-f81d-47ee-9a9a-f421eb95bfd6 Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:33.223118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:02:33.223462 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:02:33.223462 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-f033da69-f81d-47ee-9a9a-f421eb95bfd6 Nov 25 13:02:33.223462 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:02:56.385248 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-e9c2eedd-e21e-4bc3-b41c-2d82b166b39c', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-c4a74776-f18b-4025-b770-e625dea17fd2', 'neutron:subnet_ids': '93630e32-39eb-480a-bb9a-efae3ef1db93'}, mac=fa:16:3e:f8:03:11, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-e6268043-0539-4a74-aa13-2271ad0b8d7b, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:02:56.386976 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-e9c2eedd-e21e-4bc3-b41c-2d82b166b39c, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-c4a74776-f18b-4025-b770-e625dea17fd2 Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:57.019200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:02:57.019718 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:02:57.019718 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-c4a74776-f18b-4025-b770-e625dea17fd2 Nov 25 13:02:57.019718 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:02:58.130807 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-d6f5f373-12dc-4ce4-a4d8-7a948bb7d742', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-32832586-1400-48d6-b7b4-26f966057f1d', 'neutron:subnet_ids': '0c139ec3-8e37-4dd3-9c62-617558323606'}, mac=fa:16:3e:da:db:83, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-cbc33c6a-bdbe-4a62-8df0-038be7a77604, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:02:58.133812 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-d6f5f373-12dc-4ce4-a4d8-7a948bb7d742, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-32832586-1400-48d6-b7b4-26f966057f1d Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:02:58.734099 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:02:58.734341 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:02:58.734341 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-32832586-1400-48d6-b7b4-26f966057f1d Nov 25 13:02:58.734341 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:03:02.526197 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:0c:64:78, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.49/24', '2001:db8::22/64'], name=lrp-ce36154c-dccd-4b9b-84a9-341523a6eaea, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:03:03.892119 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-2f562a57-7707-4e23-98a9-cf331afdeb9b', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93', 'neutron:subnet_ids': '8720ad12-350d-4fcd-a8b2-b68bbbea605a'}, mac=fa:16:3e:80:34:df, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-7a880589-d2c0-4fb0-af72-99c4b7a2c57d, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:03:03.931767 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-2f562a57-7707-4e23-98a9-cf331afdeb9b synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:03:03.932399 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-2f562a57-7707-4e23-98a9-cf331afdeb9b, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '2f562a57-7707-4e23-98a9-cf331afdeb9b', 'neutron:router_id': 'e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:03:03.932902 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-2f562a57-7707-4e23-98a9-cf331afdeb9b, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:03:13.930067 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:03:39.744094 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-8437b51a-aaa7-4859-9d9b-3e07ced7e456', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-950fae5f-40ea-4668-af0a-383569d6040d', 'neutron:subnet_ids': '2d238a05-4434-4988-8d99-26217f6745fd'}, mac=fa:16:3e:60:ba:24, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-7968267e-8cf9-4347-b772-161c6b2cfe9f, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:03:39.746736 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-8437b51a-aaa7-4859-9d9b-3e07ced7e456, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-950fae5f-40ea-4668-af0a-383569d6040d Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:03:40.428863 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:03:40.429136 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:03:40.429136 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-950fae5f-40ea-4668-af0a-383569d6040d Nov 25 13:03:40.429136 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:03:46.705953 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-2a92c525-b4c7-40c6-bd7a-efa45e6486b9', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:4d:8a:2f, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.227/24', '2001:db8::385/64'], name=lrp-42265ba2-3540-41d8-84cb-e1f5274e4e9d, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:03:48.042885 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-84f7390f-e870-4d99-881f-c0057e9dff9a', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-2a92c525-b4c7-40c6-bd7a-efa45e6486b9', 'neutron:subnet_ids': '52b6c849-6a86-49f5-bba4-c9d0501c4347'}, mac=fa:16:3e:a5:7d:c7, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-1524b7ef-cb12-400d-a2f5-2cfc6a3725c8, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:03:48.080268 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-84f7390f-e870-4d99-881f-c0057e9dff9a synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:03:48.081186 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-84f7390f-e870-4d99-881f-c0057e9dff9a, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '84f7390f-e870-4d99-881f-c0057e9dff9a', 'neutron:router_id': '2a92c525-b4c7-40c6-bd7a-efa45e6486b9'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:03:48.081669 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-84f7390f-e870-4d99-881f-c0057e9dff9a, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:03:58.078972 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:05:10.591790 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-9a2e5bab-f45c-4905-90c5-f2f4864d9789', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-ea8ee881-4ad4-4775-9a72-78fbe5c360d6', 'neutron:subnet_ids': '3a2211b3-6181-4062-8763-2fce274216c9'}, mac=fa:16:3e:14:d2:16, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-ef3bcdee-759a-473a-831d-fcc518cc366c, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:11.934651 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-3d3a5a6c-7936-4cdd-922a-8afc0435f665', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-ea8ee881-4ad4-4775-9a72-78fbe5c360d6', 'neutron:subnet_ids': '194e7f21-6d81-4446-86fc-b1906ae5a60f'}, mac=fa:16:3e:84:03:01, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.16.1/28'], name=lrp-16e4ee2f-bd64-469b-b0fd-c75d2e4893b0, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:12.831617 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-3d3a5a6c-7936-4cdd-922a-8afc0435f665', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-ea8ee881-4ad4-4775-9a72-78fbe5c360d6', 'neutron:subnet_ids': '194e7f21-6d81-4446-86fc-b1906ae5a60f'}, mac=fa:16:3e:84:03:01, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.16.1/28'], name=lrp-16e4ee2f-bd64-469b-b0fd-c75d2e4893b0, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:13.267416 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-9a2e5bab-f45c-4905-90c5-f2f4864d9789', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-ea8ee881-4ad4-4775-9a72-78fbe5c360d6', 'neutron:subnet_ids': '3a2211b3-6181-4062-8763-2fce274216c9'}, mac=fa:16:3e:14:d2:16, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-ef3bcdee-759a-473a-831d-fcc518cc366c, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:16.853787 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-0eacdbae-a91f-4f10-8b25-7acd017f1a7c', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-dfa79807-0183-4490-b235-926ce0360e31', 'neutron:subnet_ids': '8d732186-9691-4718-8624-0d158976bc35'}, mac=fa:16:3e:cc:69:03, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.3/28'], name=lrp-1c549b27-a78f-4d4d-8a41-4758fb913327, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:17.909499 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-0eacdbae-a91f-4f10-8b25-7acd017f1a7c', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-dfa79807-0183-4490-b235-926ce0360e31', 'neutron:subnet_ids': '8d732186-9691-4718-8624-0d158976bc35'}, mac=fa:16:3e:cc:69:03, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.3/28'], name=lrp-1c549b27-a78f-4d4d-8a41-4758fb913327, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:20.800682 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-32013954-07f7-4437-a23f-aa0a84053fbb', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-438125b3-9e6b-4e83-86d0-e3dad56979b6', 'neutron:subnet_ids': 'fe42ad61-bdb7-4277-9f2c-b7f750fa54a0'}, mac=fa:16:3e:be:d0:a9, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-64823ef0-efc2-4813-a9d1-c31209cd3d14, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:21.551857 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-32013954-07f7-4437-a23f-aa0a84053fbb', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-438125b3-9e6b-4e83-86d0-e3dad56979b6', 'neutron:subnet_ids': 'fe42ad61-bdb7-4277-9f2c-b7f750fa54a0'}, mac=fa:16:3e:be:d0:a9, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-64823ef0-efc2-4813-a9d1-c31209cd3d14, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:23.369150 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-0750c910-b189-4ec0-8878-ee7de426d034', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:7b:41:35, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.248/24', '2001:db8::66/64'], name=lrp-49d2fd6a-6775-4ea4-8fd9-8624aa10dfb8, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-0750c910-b189-4ec0-8878-ee7de426d034 Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:24.709299 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:05:24.710016 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:05:24.710016 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-0750c910-b189-4ec0-8878-ee7de426d034 Nov 25 13:05:24.710016 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:05:26.936599 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-c8d08245-80a9-41bf-ab01-5c17f38456d9', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-d81ca101-72a0-47b5-84c1-b60ba8db1472', 'neutron:subnet_ids': '18794558-d68b-4c39-bff5-5fc17422802a'}, mac=fa:16:3e:e3:94:fe, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-856bf27e-149d-40ed-8916-7aafef6bd1a5, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:29.998175 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-c8d08245-80a9-41bf-ab01-5c17f38456d9', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-d81ca101-72a0-47b5-84c1-b60ba8db1472', 'neutron:subnet_ids': '18794558-d68b-4c39-bff5-5fc17422802a'}, mac=fa:16:3e:e3:94:fe, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-856bf27e-149d-40ed-8916-7aafef6bd1a5, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:33.039748 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-f1ff371a-2f9d-4502-ae2a-01b7927e9ecf', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-4acfcec6-cf63-4b0a-953b-d63b724ea9fa', 'neutron:subnet_ids': 'dce89945-a6f8-493c-b5d7-00ef125d2af9'}, mac=fa:16:3e:eb:03:34, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-d92ccb57-9c64-43be-8e01-f1ab3307955e, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:35.814157 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-db3ec9e1-48bd-45c6-aef5-75dc57248395', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-4acfcec6-cf63-4b0a-953b-d63b724ea9fa', 'neutron:subnet_ids': '74b2e8c7-d68c-495c-a46d-7722646a2ecd'}, mac=fa:16:3e:57:26:73, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.16.1/28'], name=lrp-ba80b60b-c672-4d2b-ace2-71e737a1728b, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:37.948253 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-690f4747-695c-44f2-903c-d63ac0ccebee', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-4acfcec6-cf63-4b0a-953b-d63b724ea9fa', 'neutron:subnet_ids': '1fcd0d40-6cca-43b2-a3b5-b392e63fd8f1'}, mac=fa:16:3e:36:8f:0b, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.32.1/28'], name=lrp-17b58e38-40aa-45e7-bf4c-bd1d13ab1e48, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:40.416742 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-251af013-987a-41e5-82e9-71fd4a56249d', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-4acfcec6-cf63-4b0a-953b-d63b724ea9fa', 'neutron:subnet_ids': 'abc12cf6-3288-454f-9bf5-d537f9bc5e6e'}, mac=fa:16:3e:af:56:e3, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.48.1/28'], name=lrp-acc7bc82-5684-44c6-9b8b-8d8c66d4d86e, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:46.425160 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-251af013-987a-41e5-82e9-71fd4a56249d', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-4acfcec6-cf63-4b0a-953b-d63b724ea9fa', 'neutron:subnet_ids': 'abc12cf6-3288-454f-9bf5-d537f9bc5e6e'}, mac=fa:16:3e:af:56:e3, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.48.1/28'], name=lrp-acc7bc82-5684-44c6-9b8b-8d8c66d4d86e, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:47.900892 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-690f4747-695c-44f2-903c-d63ac0ccebee', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-4acfcec6-cf63-4b0a-953b-d63b724ea9fa', 'neutron:subnet_ids': '1fcd0d40-6cca-43b2-a3b5-b392e63fd8f1'}, mac=fa:16:3e:36:8f:0b, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.32.1/28'], name=lrp-17b58e38-40aa-45e7-bf4c-bd1d13ab1e48, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:49.020862 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-db3ec9e1-48bd-45c6-aef5-75dc57248395', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-4acfcec6-cf63-4b0a-953b-d63b724ea9fa', 'neutron:subnet_ids': '74b2e8c7-d68c-495c-a46d-7722646a2ecd'}, mac=fa:16:3e:57:26:73, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.16.1/28'], name=lrp-ba80b60b-c672-4d2b-ace2-71e737a1728b, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:50.332578 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-f1ff371a-2f9d-4502-ae2a-01b7927e9ecf', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-4acfcec6-cf63-4b0a-953b-d63b724ea9fa', 'neutron:subnet_ids': 'dce89945-a6f8-493c-b5d7-00ef125d2af9'}, mac=fa:16:3e:eb:03:34, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-d92ccb57-9c64-43be-8e01-f1ab3307955e, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:50.580220 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=98974) _process_scheduled /usr/local/lib/python3.12/site-packages/futurist/periodics.py:638}} Nov 25 13:05:50.595531 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}} Nov 25 13:05:50.595531 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Number of inconsistencies found at create/update: networks=2, security_groups=1, routers=2, router_ports=2 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:05:50.597592 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Number of inconsistencies found at delete: router_ports=8 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:05:50.597592 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource 2f562a57-7707-4e23-98a9-cf331afdeb9b (type: networks) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:05:50.630825 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-2f562a57-7707-4e23-98a9-cf331afdeb9b, resource={'id': '2f562a57-7707-4e23-98a9-cf331afdeb9b', 'name': 'tempest-ServerDiskConfigTestJSON-108258855-network', 'tenant_id': 'b8e2aff376ff41ccbe3d45dddc4acf55', 'project_id': 'b8e2aff376ff41ccbe3d45dddc4acf55', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['8720ad12-350d-4fcd-a8b2-b68bbbea605a'], 'standard_attr_id': 691, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2025-11-25T18:03:00Z', 'updated_at': '2025-11-25T18:03:01Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 52412}, resource_type=networks, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.631103 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-2f562a57-7707-4e23-98a9-cf331afdeb9b, col_values=(('external_ids', {'neutron:network_name': 'tempest-ServerDiskConfigTestJSON-108258855-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve', 'neutron:provnet-physical-network': ''}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.631472 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-184e27a4-6572-40b0-9590-d840ffc6a513, external_ids_update=None, columns={'tag': 52412}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.649819 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Successfully bumped revision number for resource 2f562a57-7707-4e23-98a9-cf331afdeb9b (type: networks) to 2 Nov 25 13:05:50.649958 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource 84f7390f-e870-4d99-881f-c0057e9dff9a (type: networks) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:05:50.683311 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-84f7390f-e870-4d99-881f-c0057e9dff9a, resource={'id': '84f7390f-e870-4d99-881f-c0057e9dff9a', 'name': 'tempest-AttachVolumeMultiAttachTest-2060305290-network', 'tenant_id': 'aca63b6b13c245909c931b0dc84e6764', 'project_id': 'aca63b6b13c245909c931b0dc84e6764', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['52b6c849-6a86-49f5-bba4-c9d0501c4347'], 'standard_attr_id': 731, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2025-11-25T18:03:44Z', 'updated_at': '2025-11-25T18:03:45Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 43776}, resource_type=networks, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.683511 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-84f7390f-e870-4d99-881f-c0057e9dff9a, col_values=(('external_ids', {'neutron:network_name': 'tempest-AttachVolumeMultiAttachTest-2060305290-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve', 'neutron:provnet-physical-network': ''}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.683766 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-eeb6395b-655a-49bb-a88f-f74a86dfbe94, external_ids_update=None, columns={'tag': 43776}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.706108 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Successfully bumped revision number for resource 84f7390f-e870-4d99-881f-c0057e9dff9a (type: networks) to 2 Nov 25 13:05:50.706239 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource e04df9b6-b9ca-4336-8643-e9e4ef82aaa2 (type: security_groups) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:05:50.740904 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Successfully bumped revision number for resource e04df9b6-b9ca-4336-8643-e9e4ef82aaa2 (type: security_groups) to 3 Nov 25 13:05:50.741354 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource 2a92c525-b4c7-40c6-bd7a-efa45e6486b9 (type: routers) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:05:50.874106 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-2a92c525-b4c7-40c6-bd7a-efa45e6486b9, resource={'id': '2a92c525-b4c7-40c6-bd7a-efa45e6486b9', 'name': 'tempest-AttachVolumeMultiAttachTest-2060305290-router', 'tenant_id': 'aca63b6b13c245909c931b0dc84e6764', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.227'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::385'}], 'enable_snat': True}, 'gw_port_id': '42265ba2-3540-41d8-84cb-e1f5274e4e9d', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'ip_address': '172.24.5.227', 'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59'}, {'ip_address': '2001:db8::385', 'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2025-11-25T18:03:45Z', 'updated_at': '2025-11-25T18:03:47Z', 'revision_number': 3, 'project_id': 'aca63b6b13c245909c931b0dc84e6764'}, resource_type=routers, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.874280 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-2a92c525-b4c7-40c6-bd7a-efa45e6486b9, columns={'external_ids': {'neutron:router_name': 'tempest-AttachVolumeMultiAttachTest-2060305290-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.874460 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "42265ba2-3540-41d8-84cb-e1f5274e4e9d"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.874583 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "42265ba2-3540-41d8-84cb-e1f5274e4e9d"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.926896 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Successfully bumped revision number for resource 2a92c525-b4c7-40c6-bd7a-efa45e6486b9 (type: routers) to 3 Nov 25 13:05:50.927056 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93 (type: routers) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:05:50.986830 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93, resource={'id': 'e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93', 'name': 'tempest-ServerDiskConfigTestJSON-108258855-router', 'tenant_id': 'b8e2aff376ff41ccbe3d45dddc4acf55', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.49'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::22'}], 'enable_snat': True}, 'gw_port_id': 'ce36154c-dccd-4b9b-84a9-341523a6eaea', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'ip_address': '172.24.5.49', 'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59'}, {'ip_address': '2001:db8::22', 'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2025-11-25T18:03:01Z', 'updated_at': '2025-11-25T18:03:03Z', 'revision_number': 3, 'project_id': 'b8e2aff376ff41ccbe3d45dddc4acf55'}, resource_type=routers, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.987047 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93, columns={'external_ids': {'neutron:router_name': 'tempest-ServerDiskConfigTestJSON-108258855-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.987206 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "ce36154c-dccd-4b9b-84a9-341523a6eaea"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:50.987321 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "ce36154c-dccd-4b9b-84a9-341523a6eaea"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:51.034425 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Successfully bumped revision number for resource e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93 (type: routers) to 3 Nov 25 13:05:51.034595 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource ce36154c-dccd-4b9b-84a9-341523a6eaea (type: router_ports) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:05:51.116579 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-ce36154c-dccd-4b9b-84a9-341523a6eaea, resource={'id': 'ce36154c-dccd-4b9b-84a9-341523a6eaea', 'name': '', 'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0c:64:78', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93', 'device_owner': 'network:router_gateway', 'standard_attr_id': 696, 'fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.49'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::22'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0005535648.novalocal', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2025-11-25T18:03:01Z', 'updated_at': '2025-11-25T18:03:03Z', 'revision_number': 5, 'project_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:51.116736 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-ce36154c-dccd-4b9b-84a9-341523a6eaea, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.49/24', '2001:db8::22/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:51.116946 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=ce36154c-dccd-4b9b-84a9-341523a6eaea, lrouter_port=lrp-ce36154c-dccd-4b9b-84a9-341523a6eaea, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:51.137575 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Successfully bumped revision number for resource ce36154c-dccd-4b9b-84a9-341523a6eaea (type: router_ports) to 5 Nov 25 13:05:51.137575 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource 42265ba2-3540-41d8-84cb-e1f5274e4e9d (type: router_ports) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:05:51.228479 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-42265ba2-3540-41d8-84cb-e1f5274e4e9d, resource={'id': '42265ba2-3540-41d8-84cb-e1f5274e4e9d', 'name': '', 'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'tenant_id': '', 'mac_address': 'fa:16:3e:4d:8a:2f', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2a92c525-b4c7-40c6-bd7a-efa45e6486b9', 'device_owner': 'network:router_gateway', 'standard_attr_id': 736, 'fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.227'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::385'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0005535648.novalocal', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2025-11-25T18:03:45Z', 'updated_at': '2025-11-25T18:03:47Z', 'revision_number': 5, 'project_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:51.228622 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-42265ba2-3540-41d8-84cb-e1f5274e4e9d, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-2a92c525-b4c7-40c6-bd7a-efa45e6486b9'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::385/64', '172.24.5.227/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:51.228839 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=42265ba2-3540-41d8-84cb-e1f5274e4e9d, lrouter_port=lrp-42265ba2-3540-41d8-84cb-e1f5274e4e9d, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:51.254929 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Successfully bumped revision number for resource 42265ba2-3540-41d8-84cb-e1f5274e4e9d (type: router_ports) to 5 Nov 25 13:05:51.254978 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource 0170d6d6-2093-4da5-9849-81830278d765 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:05:51.255172 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] delete_revision(0170d6d6-2093-4da5-9849-81830278d765) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:05:51.261869 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource 3df4f1ac-98a2-492d-81a6-7c95aeff6a92 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:05:51.262031 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] delete_revision(3df4f1ac-98a2-492d-81a6-7c95aeff6a92) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:05:51.264847 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource 49d2fd6a-6775-4ea4-8fd9-8624aa10dfb8 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:05:51.265009 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] delete_revision(49d2fd6a-6775-4ea4-8fd9-8624aa10dfb8) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:05:51.268075 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource 868a6a0d-d971-490c-8c36-568f9cb257da (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:05:51.268242 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] delete_revision(868a6a0d-d971-490c-8c36-568f9cb257da) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:05:51.271585 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource 9dab27a4-e9ca-4bf1-ba60-8e0395a50f62 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:05:51.271754 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] delete_revision(9dab27a4-e9ca-4bf1-ba60-8e0395a50f62) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:05:51.274635 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource b50a7bd3-aa28-46c9-8f74-e3fe3655818f (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:05:51.274819 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] delete_revision(b50a7bd3-aa28-46c9-8f74-e3fe3655818f) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:05:51.277936 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource ef4e1b01-07e8-41d8-868c-f0c4fd858c1a (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:05:51.278100 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] delete_revision(ef4e1b01-07e8-41d8-868c-f0c4fd858c1a) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:05:51.281103 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Fixing resource f1fec9e8-7099-43d7-a3a3-68cddd8b0630 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:05:51.281267 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] delete_revision(f1fec9e8-7099-43d7-a3a3-68cddd8b0630) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:05:51.283897 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-8a73b35c-e80d-45cc-afb6-c43899ebd6bf None None] Maintenance task: Synchronization completed (took 0.69 seconds) Nov 25 13:05:51.925984 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-2f562a57-7707-4e23-98a9-cf331afdeb9b', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93', 'neutron:subnet_ids': '8720ad12-350d-4fcd-a8b2-b68bbbea605a'}, mac=fa:16:3e:80:34:df, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-7a880589-d2c0-4fb0-af72-99c4b7a2c57d, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:51.926679 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-2f562a57-7707-4e23-98a9-cf331afdeb9b, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93 Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:52.460241 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:05:52.460813 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:05:52.460813 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-e52e7ca9-ce4b-4b73-b096-c3bad3ff0b93 Nov 25 13:05:52.460813 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:05:55.716895 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-ac523eb9-e960-4ac9-879e-8c90e32399ff', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:b8:42:6b, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.107/24', '2001:db8::275/64'], name=lrp-4c20ac5b-bb17-49dc-9ec4-6ec46058fbc3, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:57.776691 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-88f8eb40-ad0c-4221-acfb-7f5619e517da', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-ac523eb9-e960-4ac9-879e-8c90e32399ff', 'neutron:subnet_ids': '5a9a69a6-aba4-44d5-aa21-7a468e17c9ae'}, mac=fa:16:3e:db:b8:a8, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-74ce5da7-03ee-4f7f-b378-e5fc32b89d52, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:57.817396 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-88f8eb40-ad0c-4221-acfb-7f5619e517da synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:05:57.818041 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-88f8eb40-ad0c-4221-acfb-7f5619e517da, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '88f8eb40-ad0c-4221-acfb-7f5619e517da', 'neutron:router_id': 'ac523eb9-e960-4ac9-879e-8c90e32399ff'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:57.818657 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-88f8eb40-ad0c-4221-acfb-7f5619e517da, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:58.593058 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-b35bfd86-f1c6-464b-917a-de86934fd7e8', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-ead643f8-be77-4e46-ab58-587aa84f4dac', 'neutron:subnet_ids': '0586b32e-06b5-4106-a48e-ad8816ba6384'}, mac=fa:16:3e:96:80:ef, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-f2aa6c54-6785-4c44-9142-4de0e8124835, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:05:58.593598 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-b35bfd86-f1c6-464b-917a-de86934fd7e8, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-ead643f8-be77-4e46-ab58-587aa84f4dac Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:05:59.160295 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:05:59.161343 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:05:59.161343 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-ead643f8-be77-4e46-ab58-587aa84f4dac Nov 25 13:05:59.161343 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:06:02.743473 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-d7e6d528-ffe1-4c0f-87cd-b09bbb52faf1', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:8b:9e:b7, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.66/24', '2001:db8::b4/64'], name=lrp-e05f3775-137e-476b-955f-08e7c383aec1, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:06:04.591013 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-6bab6fca-b660-4576-bca6-f4b266bc8380', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-d7e6d528-ffe1-4c0f-87cd-b09bbb52faf1', 'neutron:subnet_ids': 'aeeffbc1-917c-4dfa-903f-2702e2be9e0b'}, mac=fa:16:3e:42:ea:2f, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-c5733e7f-426b-49ea-b4cd-04ffe978ed77, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:06:04.627760 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-6bab6fca-b660-4576-bca6-f4b266bc8380 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:06:04.628235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-6bab6fca-b660-4576-bca6-f4b266bc8380, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '6bab6fca-b660-4576-bca6-f4b266bc8380', 'neutron:router_id': 'd7e6d528-ffe1-4c0f-87cd-b09bbb52faf1'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:06:04.628408 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-6bab6fca-b660-4576-bca6-f4b266bc8380, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:06:14.623007 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=425,neutron:UPDATE=7,neutron:DELETE=8 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:06:31.163448 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-88f8eb40-ad0c-4221-acfb-7f5619e517da', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-ac523eb9-e960-4ac9-879e-8c90e32399ff', 'neutron:subnet_ids': '5a9a69a6-aba4-44d5-aa21-7a468e17c9ae'}, mac=fa:16:3e:db:b8:a8, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-74ce5da7-03ee-4f7f-b378-e5fc32b89d52, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:06:31.163448 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-88f8eb40-ad0c-4221-acfb-7f5619e517da, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-ac523eb9-e960-4ac9-879e-8c90e32399ff Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:06:31.819609 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:06:31.820291 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:06:31.820291 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-ac523eb9-e960-4ac9-879e-8c90e32399ff Nov 25 13:06:31.820291 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:06:35.283126 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-881c0842-f0c8-4550-b5fd-0f775a8da033', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:e9:e9:aa, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.196/24', '2001:db8::10c/64'], name=lrp-8cd1df32-a07e-4593-9ebc-546b6e50a6f5, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:06:36.760984 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-7a93ec5d-02f0-42ac-a233-ea5867e7b066', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-881c0842-f0c8-4550-b5fd-0f775a8da033', 'neutron:subnet_ids': '7aa7365c-e3ad-46df-9cec-2c59237917a7'}, mac=fa:16:3e:58:12:77, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-47004804-a031-4745-b06e-24d898610ab4, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:06:36.785130 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-7a93ec5d-02f0-42ac-a233-ea5867e7b066 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:06:36.786127 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-7a93ec5d-02f0-42ac-a233-ea5867e7b066, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '7a93ec5d-02f0-42ac-a233-ea5867e7b066', 'neutron:router_id': '881c0842-f0c8-4550-b5fd-0f775a8da033'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:06:36.786127 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-7a93ec5d-02f0-42ac-a233-ea5867e7b066, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:06:46.840644 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:07:00.158328 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-7a93ec5d-02f0-42ac-a233-ea5867e7b066', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-881c0842-f0c8-4550-b5fd-0f775a8da033', 'neutron:subnet_ids': '7aa7365c-e3ad-46df-9cec-2c59237917a7'}, mac=fa:16:3e:58:12:77, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-47004804-a031-4745-b06e-24d898610ab4, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:07:00.161041 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-7a93ec5d-02f0-42ac-a233-ea5867e7b066, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-881c0842-f0c8-4550-b5fd-0f775a8da033 Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:07:00.730676 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:07:00.731384 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:07:00.731384 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-881c0842-f0c8-4550-b5fd-0f775a8da033 Nov 25 13:07:00.731384 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:07:03.055934 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-ff3dc5e4-5f7b-4f82-a0b6-2ce51801e21e', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:87:fd:92, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.31/24', '2001:db8::b9/64'], name=lrp-7e459cc0-81f3-4948-8a8a-48901ef77dce, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:07:05.668073 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-945eaf99-0725-446b-8169-96595adaa2e3', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-ff3dc5e4-5f7b-4f82-a0b6-2ce51801e21e', 'neutron:subnet_ids': '82b3d91e-ae72-4d06-8dc4-7476a0259c24'}, mac=fa:16:3e:f3:09:fc, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-761eeba8-ab0d-4397-ae6b-949e4edde18d, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:07:05.717938 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-945eaf99-0725-446b-8169-96595adaa2e3 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:07:05.718499 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-945eaf99-0725-446b-8169-96595adaa2e3, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '945eaf99-0725-446b-8169-96595adaa2e3', 'neutron:router_id': 'ff3dc5e4-5f7b-4f82-a0b6-2ce51801e21e'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:07:05.718802 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-945eaf99-0725-446b-8169-96595adaa2e3, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:07:15.716610 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:08:56.231061 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-a247837e-dc11-4f72-8022-cdfe7c896687', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:37:54:50, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.190/24', '2001:db8::7f/64'], name=lrp-57876aca-9199-4f43-a53e-fb75dedb62d6, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:08:58.620141 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-14ae165a-532b-4104-a09d-24e89579ccfe', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-a247837e-dc11-4f72-8022-cdfe7c896687', 'neutron:subnet_ids': '41c0e100-adf5-4201-9fb0-85a701b7f37f'}, mac=fa:16:3e:31:3f:e7, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-5ca1bf30-2cc2-4311-89a5-22da71854213, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:08:58.668482 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-14ae165a-532b-4104-a09d-24e89579ccfe synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:08:58.668787 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-14ae165a-532b-4104-a09d-24e89579ccfe, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '14ae165a-532b-4104-a09d-24e89579ccfe', 'neutron:router_id': 'a247837e-dc11-4f72-8022-cdfe7c896687'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:08:58.669069 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-14ae165a-532b-4104-a09d-24e89579ccfe, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:09:08.670000 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:09:23.312920 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-945eaf99-0725-446b-8169-96595adaa2e3', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-ff3dc5e4-5f7b-4f82-a0b6-2ce51801e21e', 'neutron:subnet_ids': '82b3d91e-ae72-4d06-8dc4-7476a0259c24'}, mac=fa:16:3e:f3:09:fc, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-761eeba8-ab0d-4397-ae6b-949e4edde18d, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:09:23.313740 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-945eaf99-0725-446b-8169-96595adaa2e3, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-ff3dc5e4-5f7b-4f82-a0b6-2ce51801e21e Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:09:23.890726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:09:23.890969 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:09:23.890969 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-ff3dc5e4-5f7b-4f82-a0b6-2ce51801e21e Nov 25 13:09:23.890969 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:10:50.581277 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=98974) _process_scheduled /usr/local/lib/python3.12/site-packages/futurist/periodics.py:638}} Nov 25 13:10:50.589138 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}} Nov 25 13:10:50.589312 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Number of inconsistencies found at create/update: networks=2, security_groups=2, routers=2, router_ports=2 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:10:50.589462 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Number of inconsistencies found at delete: router_ports=5 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:10:50.589587 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource 14ae165a-532b-4104-a09d-24e89579ccfe (type: networks) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:10:50.643249 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-14ae165a-532b-4104-a09d-24e89579ccfe, resource={'id': '14ae165a-532b-4104-a09d-24e89579ccfe', 'name': 'tempest-ServerActionsV293TestJSON-2145363590-network', 'tenant_id': 'ff335d4de5614da5883b1794458cb6c1', 'project_id': 'ff335d4de5614da5883b1794458cb6c1', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['41c0e100-adf5-4201-9fb0-85a701b7f37f'], 'standard_attr_id': 1190, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2025-11-25T18:08:53Z', 'updated_at': '2025-11-25T18:08:54Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 58490}, resource_type=networks, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.645233 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-14ae165a-532b-4104-a09d-24e89579ccfe, col_values=(('external_ids', {'neutron:network_name': 'tempest-ServerActionsV293TestJSON-2145363590-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve', 'neutron:provnet-physical-network': ''}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.645233 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-6b417b2a-5bbf-4a3e-91c4-90bcda888cf9, external_ids_update=None, columns={'tag': 58490}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.685418 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Successfully bumped revision number for resource 14ae165a-532b-4104-a09d-24e89579ccfe (type: networks) to 2 Nov 25 13:10:50.685604 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource 6bab6fca-b660-4576-bca6-f4b266bc8380 (type: networks) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:10:50.710287 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-6bab6fca-b660-4576-bca6-f4b266bc8380, resource={'id': '6bab6fca-b660-4576-bca6-f4b266bc8380', 'name': 'tempest-ServerActionsTestOtherA-157732704-network', 'tenant_id': '7bf7426aaea34462a24d0bf089464c7a', 'project_id': '7bf7426aaea34462a24d0bf089464c7a', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['aeeffbc1-917c-4dfa-903f-2702e2be9e0b'], 'standard_attr_id': 1068, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2025-11-25T18:06:00Z', 'updated_at': '2025-11-25T18:06:01Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 46071}, resource_type=networks, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.710695 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-6bab6fca-b660-4576-bca6-f4b266bc8380, col_values=(('external_ids', {'neutron:network_name': 'tempest-ServerActionsTestOtherA-157732704-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve', 'neutron:provnet-physical-network': ''}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.711204 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-86a50253-f512-4d8f-9d87-efab1da29e86, external_ids_update=None, columns={'tag': 46071}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.740887 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Successfully bumped revision number for resource 6bab6fca-b660-4576-bca6-f4b266bc8380 (type: networks) to 2 Nov 25 13:10:50.741045 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource 24e079b0-ea7f-4608-93b2-c80228dc15fc (type: security_groups) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:10:50.761156 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Successfully bumped revision number for resource 24e079b0-ea7f-4608-93b2-c80228dc15fc (type: security_groups) to 3 Nov 25 13:10:50.761252 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource 92a75aa3-bd03-47c3-b375-a90373bba0c5 (type: security_groups) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:10:50.778448 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Successfully bumped revision number for resource 92a75aa3-bd03-47c3-b375-a90373bba0c5 (type: security_groups) to 3 Nov 25 13:10:50.778557 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource a247837e-dc11-4f72-8022-cdfe7c896687 (type: routers) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:10:50.875709 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-a247837e-dc11-4f72-8022-cdfe7c896687, resource={'id': 'a247837e-dc11-4f72-8022-cdfe7c896687', 'name': 'tempest-ServerActionsV293TestJSON-2145363590-router', 'tenant_id': 'ff335d4de5614da5883b1794458cb6c1', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.190'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::7f'}], 'enable_snat': True}, 'gw_port_id': '57876aca-9199-4f43-a53e-fb75dedb62d6', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'ip_address': '172.24.5.190', 'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59'}, {'ip_address': '2001:db8::7f', 'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2025-11-25T18:08:55Z', 'updated_at': '2025-11-25T18:08:57Z', 'revision_number': 3, 'project_id': 'ff335d4de5614da5883b1794458cb6c1'}, resource_type=routers, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.875927 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-a247837e-dc11-4f72-8022-cdfe7c896687, columns={'external_ids': {'neutron:router_name': 'tempest-ServerActionsV293TestJSON-2145363590-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.876143 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "57876aca-9199-4f43-a53e-fb75dedb62d6"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.876303 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "57876aca-9199-4f43-a53e-fb75dedb62d6"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.920456 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Successfully bumped revision number for resource a247837e-dc11-4f72-8022-cdfe7c896687 (type: routers) to 3 Nov 25 13:10:50.920571 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource d7e6d528-ffe1-4c0f-87cd-b09bbb52faf1 (type: routers) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:10:50.968519 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-d7e6d528-ffe1-4c0f-87cd-b09bbb52faf1, resource={'id': 'd7e6d528-ffe1-4c0f-87cd-b09bbb52faf1', 'name': 'tempest-ServerActionsTestOtherA-157732704-router', 'tenant_id': '7bf7426aaea34462a24d0bf089464c7a', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.66'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::b4'}], 'enable_snat': True}, 'gw_port_id': 'e05f3775-137e-476b-955f-08e7c383aec1', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'ip_address': '172.24.5.66', 'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59'}, {'ip_address': '2001:db8::b4', 'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2025-11-25T18:06:01Z', 'updated_at': '2025-11-25T18:06:03Z', 'revision_number': 3, 'project_id': '7bf7426aaea34462a24d0bf089464c7a'}, resource_type=routers, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.968704 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-d7e6d528-ffe1-4c0f-87cd-b09bbb52faf1, columns={'external_ids': {'neutron:router_name': 'tempest-ServerActionsTestOtherA-157732704-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.968847 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "e05f3775-137e-476b-955f-08e7c383aec1"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:50.968951 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "e05f3775-137e-476b-955f-08e7c383aec1"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:51.079373 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Successfully bumped revision number for resource d7e6d528-ffe1-4c0f-87cd-b09bbb52faf1 (type: routers) to 3 Nov 25 13:10:51.079553 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource e05f3775-137e-476b-955f-08e7c383aec1 (type: router_ports) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:10:51.181952 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-e05f3775-137e-476b-955f-08e7c383aec1, resource={'id': 'e05f3775-137e-476b-955f-08e7c383aec1', 'name': '', 'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'tenant_id': '', 'mac_address': 'fa:16:3e:8b:9e:b7', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd7e6d528-ffe1-4c0f-87cd-b09bbb52faf1', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1079, 'fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.66'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::b4'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0005535648.novalocal', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2025-11-25T18:06:01Z', 'updated_at': '2025-11-25T18:06:03Z', 'revision_number': 5, 'project_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:51.182118 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-e05f3775-137e-476b-955f-08e7c383aec1, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-d7e6d528-ffe1-4c0f-87cd-b09bbb52faf1'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::b4/64', '172.24.5.66/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:51.182573 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=e05f3775-137e-476b-955f-08e7c383aec1, lrouter_port=lrp-e05f3775-137e-476b-955f-08e7c383aec1, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:51.212641 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Successfully bumped revision number for resource e05f3775-137e-476b-955f-08e7c383aec1 (type: router_ports) to 5 Nov 25 13:10:51.212641 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource 57876aca-9199-4f43-a53e-fb75dedb62d6 (type: router_ports) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:10:51.361247 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-57876aca-9199-4f43-a53e-fb75dedb62d6, resource={'id': '57876aca-9199-4f43-a53e-fb75dedb62d6', 'name': '', 'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'tenant_id': '', 'mac_address': 'fa:16:3e:37:54:50', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'a247837e-dc11-4f72-8022-cdfe7c896687', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1195, 'fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.190'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::7f'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0005535648.novalocal', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2025-11-25T18:08:55Z', 'updated_at': '2025-11-25T18:08:57Z', 'revision_number': 5, 'project_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:51.361587 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-57876aca-9199-4f43-a53e-fb75dedb62d6, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-a247837e-dc11-4f72-8022-cdfe7c896687'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::7f/64', '172.24.5.190/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:51.361990 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=57876aca-9199-4f43-a53e-fb75dedb62d6, lrouter_port=lrp-57876aca-9199-4f43-a53e-fb75dedb62d6, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:51.401294 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Successfully bumped revision number for resource 57876aca-9199-4f43-a53e-fb75dedb62d6 (type: router_ports) to 5 Nov 25 13:10:51.401360 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource 4c20ac5b-bb17-49dc-9ec4-6ec46058fbc3 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:10:51.401815 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] delete_revision(4c20ac5b-bb17-49dc-9ec4-6ec46058fbc3) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:10:51.415561 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource 7c580102-8302-4f95-bd9c-d4ca0907d53c (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:10:51.415828 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] delete_revision(7c580102-8302-4f95-bd9c-d4ca0907d53c) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:10:51.425656 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource 7e459cc0-81f3-4948-8a8a-48901ef77dce (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:10:51.426168 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] delete_revision(7e459cc0-81f3-4948-8a8a-48901ef77dce) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:10:51.431876 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource 8cd1df32-a07e-4593-9ebc-546b6e50a6f5 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:10:51.432282 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] delete_revision(8cd1df32-a07e-4593-9ebc-546b6e50a6f5) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:10:51.441015 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Fixing resource ce36154c-dccd-4b9b-84a9-341523a6eaea (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:10:51.441499 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] delete_revision(ce36154c-dccd-4b9b-84a9-341523a6eaea) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:10:51.446931 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-72280554-5116-49f2-b333-ac21b269ae86 None None] Maintenance task: Synchronization completed (took 0.86 seconds) Nov 25 13:10:59.114478 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-84f7390f-e870-4d99-881f-c0057e9dff9a', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-2a92c525-b4c7-40c6-bd7a-efa45e6486b9', 'neutron:subnet_ids': '52b6c849-6a86-49f5-bba4-c9d0501c4347'}, mac=fa:16:3e:a5:7d:c7, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-1524b7ef-cb12-400d-a2f5-2cfc6a3725c8, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:10:59.117371 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-84f7390f-e870-4d99-881f-c0057e9dff9a, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-2a92c525-b4c7-40c6-bd7a-efa45e6486b9 Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:10:59.634459 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:10:59.634776 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:10:59.634776 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-2a92c525-b4c7-40c6-bd7a-efa45e6486b9 Nov 25 13:10:59.634776 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:11:01.444790 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=401,neutron:UPDATE=8,neutron:DELETE=5 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:11:33.470857 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-14ae165a-532b-4104-a09d-24e89579ccfe', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-a247837e-dc11-4f72-8022-cdfe7c896687', 'neutron:subnet_ids': '41c0e100-adf5-4201-9fb0-85a701b7f37f'}, mac=fa:16:3e:31:3f:e7, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-5ca1bf30-2cc2-4311-89a5-22da71854213, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:11:33.474370 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-14ae165a-532b-4104-a09d-24e89579ccfe, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-a247837e-dc11-4f72-8022-cdfe7c896687 Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:11:34.233659 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:11:34.234208 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:11:34.234208 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-a247837e-dc11-4f72-8022-cdfe7c896687 Nov 25 13:11:34.234208 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:11:38.154969 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-2305f34e-a644-4650-a236-23934e81b6aa', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:f9:77:c7, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.240/24', '2001:db8::191/64'], name=lrp-963b8808-794c-4b6a-8076-7339e47e9a30, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:11:39.849839 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-75baa738-baf1-4d34-b893-9c989e0f1b63', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-2305f34e-a644-4650-a236-23934e81b6aa', 'neutron:subnet_ids': '5b7e5ebe-4b16-4b08-87f6-902839d20cc8'}, mac=fa:16:3e:e8:d1:07, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-8623edfe-e731-4339-a5d8-b2a4476b0870, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:11:39.896481 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-75baa738-baf1-4d34-b893-9c989e0f1b63 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:11:39.897036 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-75baa738-baf1-4d34-b893-9c989e0f1b63, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '75baa738-baf1-4d34-b893-9c989e0f1b63', 'neutron:router_id': '2305f34e-a644-4650-a236-23934e81b6aa'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:11:39.897668 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-75baa738-baf1-4d34-b893-9c989e0f1b63, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:11:49.894657 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:12:02.861082 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-75baa738-baf1-4d34-b893-9c989e0f1b63', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-2305f34e-a644-4650-a236-23934e81b6aa', 'neutron:subnet_ids': '5b7e5ebe-4b16-4b08-87f6-902839d20cc8'}, mac=fa:16:3e:e8:d1:07, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-8623edfe-e731-4339-a5d8-b2a4476b0870, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:12:02.867999 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-75baa738-baf1-4d34-b893-9c989e0f1b63, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-2305f34e-a644-4650-a236-23934e81b6aa Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:12:03.425264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:12:03.425595 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:12:03.425595 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-2305f34e-a644-4650-a236-23934e81b6aa Nov 25 13:12:03.425595 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:12:55.207056 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-7e06bcc1-1006-4ece-9c14-d05e51cb0704', 'neutron:revision_number': '3', 'neutron:router_name': 'neutron-ea9b733c-4cd5-452b-9414-798700c23113', 'neutron:subnet_ids': 'c4745665-efb4-46fc-ba5a-2103984a2fbc'}, mac=fa:16:3e:04:55:b0, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::393/64'], name=lrp-af9f98bc-f8d6-41f8-9741-9a9b434f1956, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:12:55.697857 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-7e06bcc1-1006-4ece-9c14-d05e51cb0704', 'neutron:revision_number': '3', 'neutron:router_name': 'neutron-ea9b733c-4cd5-452b-9414-798700c23113', 'neutron:subnet_ids': 'c4745665-efb4-46fc-ba5a-2103984a2fbc'}, mac=fa:16:3e:04:55:b0, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::393/64'], name=lrp-af9f98bc-f8d6-41f8-9741-9a9b434f1956, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:13:23.012658 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-e6ff2b5e-86c6-4263-8dcc-055af32064fb', 'neutron:revision_number': '3', 'neutron:router_name': 'neutron-b441c872-4c03-418b-a0cb-e8e43f10ecce', 'neutron:subnet_ids': 'f79de4b9-5444-433d-9ce7-f888876f4444'}, mac=fa:16:3e:ea:4f:d8, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.9/28'], name=lrp-4e57c7c8-e53c-4c1b-8ac5-dfc25f8e18c2, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:13:23.572462 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-e6ff2b5e-86c6-4263-8dcc-055af32064fb', 'neutron:revision_number': '3', 'neutron:router_name': 'neutron-b441c872-4c03-418b-a0cb-e8e43f10ecce', 'neutron:subnet_ids': 'f79de4b9-5444-433d-9ce7-f888876f4444'}, mac=fa:16:3e:ea:4f:d8, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.9/28'], name=lrp-4e57c7c8-e53c-4c1b-8ac5-dfc25f8e18c2, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:14:25.034716 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-6bab6fca-b660-4576-bca6-f4b266bc8380', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-d7e6d528-ffe1-4c0f-87cd-b09bbb52faf1', 'neutron:subnet_ids': 'aeeffbc1-917c-4dfa-903f-2702e2be9e0b'}, mac=fa:16:3e:42:ea:2f, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-c5733e7f-426b-49ea-b4cd-04ffe978ed77, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:14:25.036110 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-6bab6fca-b660-4576-bca6-f4b266bc8380, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-d7e6d528-ffe1-4c0f-87cd-b09bbb52faf1 Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:14:25.582594 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:14:25.583359 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:14:25.583359 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-d7e6d528-ffe1-4c0f-87cd-b09bbb52faf1 Nov 25 13:14:25.583359 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:14:29.342883 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-eca35f71-24e3-4800-94ed-9d6241527898', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:19:ef:73, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.148/24', '2001:db8::297/64'], name=lrp-b74c2e4e-dab9-4309-93ae-d8882a3f5ef3, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:14:31.770515 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-bedad589-7da2-43cb-a5d2-0377af836711', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-eca35f71-24e3-4800-94ed-9d6241527898', 'neutron:subnet_ids': 'b302b5f5-ba28-41d2-b43a-b6427121e67b'}, mac=fa:16:3e:f7:c7:11, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-b9ee85eb-aeda-410d-8df4-210489cef4f7, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:14:31.805557 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-bedad589-7da2-43cb-a5d2-0377af836711 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:14:31.806055 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-bedad589-7da2-43cb-a5d2-0377af836711, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': 'bedad589-7da2-43cb-a5d2-0377af836711', 'neutron:router_id': 'eca35f71-24e3-4800-94ed-9d6241527898'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:14:31.806336 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-bedad589-7da2-43cb-a5d2-0377af836711, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:14:41.804414 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:15:01.973389 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-76c35247-ddc7-4532-b843-c6a3afd3c40a', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:d9:21:fc, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.14/24', '2001:db8::163/64'], name=lrp-56e8f3ff-22fc-4822-a7e1-fb5539d47513, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:15:03.707332 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-98921d26-4f55-4dac-8177-6b6b1a45032d', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-76c35247-ddc7-4532-b843-c6a3afd3c40a', 'neutron:subnet_ids': 'ce261f80-a82e-4bd1-becd-f356a5870024'}, mac=fa:16:3e:92:94:04, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-a6e702b3-1664-4cf8-b8ca-43e86ce6a6b5, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:15:03.741944 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-98921d26-4f55-4dac-8177-6b6b1a45032d synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:15:03.742552 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-98921d26-4f55-4dac-8177-6b6b1a45032d, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '98921d26-4f55-4dac-8177-6b6b1a45032d', 'neutron:router_id': '76c35247-ddc7-4532-b843-c6a3afd3c40a'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:03.742918 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-98921d26-4f55-4dac-8177-6b6b1a45032d, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:13.741526 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:15:40.865705 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-32e116d4-6eae-49aa-8de3-1278dc2a3954', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-48a46402-d725-4b6b-9a5b-b601a57921f2', 'neutron:subnet_ids': '14210296-a2aa-49d1-849b-be353ed2107b'}, mac=fa:16:3e:49:06:92, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-a4008d60-020d-4d48-8cf9-f6a8bd74c280, gateway_chassis=[], enabled=[], ipv6_ra_configs={'address_mode': 'dhcpv6_stateful', 'mtu': '1372', 'send_periodic': 'true'}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:15:41.644979 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-32e116d4-6eae-49aa-8de3-1278dc2a3954', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-48a46402-d725-4b6b-9a5b-b601a57921f2', 'neutron:subnet_ids': '14210296-a2aa-49d1-849b-be353ed2107b'}, mac=fa:16:3e:49:06:92, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-a4008d60-020d-4d48-8cf9-f6a8bd74c280, gateway_chassis=[], enabled=[], ipv6_ra_configs={'address_mode': 'dhcpv6_stateful', 'mtu': '1372', 'send_periodic': 'true'}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:15:43.591774 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-32e116d4-6eae-49aa-8de3-1278dc2a3954', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-a4d5a54d-806a-4572-b7d1-995101b17ef1', 'neutron:subnet_ids': 'bfdc78ef-e804-4bb6-9756-74b4bc4ae465'}, mac=fa:16:3e:01:54:93, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-172ce468-d732-4f4b-8f43-67ba21cbaaa9, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:15:44.217198 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-32e116d4-6eae-49aa-8de3-1278dc2a3954', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-a4d5a54d-806a-4572-b7d1-995101b17ef1', 'neutron:subnet_ids': 'bfdc78ef-e804-4bb6-9756-74b4bc4ae465'}, mac=fa:16:3e:01:54:93, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-172ce468-d732-4f4b-8f43-67ba21cbaaa9, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:15:50.583010 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=98974) _process_scheduled /usr/local/lib/python3.12/site-packages/futurist/periodics.py:638}} Nov 25 13:15:50.592382 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}} Nov 25 13:15:50.592486 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Number of inconsistencies found at create/update: networks=1, security_groups=1, routers=1, router_ports=1 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:15:50.593249 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Number of inconsistencies found at delete: router_ports=4 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:15:50.593381 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Fixing resource bedad589-7da2-43cb-a5d2-0377af836711 (type: networks) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:15:50.625951 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-bedad589-7da2-43cb-a5d2-0377af836711, resource={'id': 'bedad589-7da2-43cb-a5d2-0377af836711', 'name': 'tempest-ServerActionsTestOtherB-504004282-network', 'tenant_id': '684cecb5362d45ff85e90096019cc322', 'project_id': '684cecb5362d45ff85e90096019cc322', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['b302b5f5-ba28-41d2-b43a-b6427121e67b'], 'standard_attr_id': 1831, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2025-11-25T18:14:27Z', 'updated_at': '2025-11-25T18:14:27Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 1309}, resource_type=networks, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:50.625951 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-bedad589-7da2-43cb-a5d2-0377af836711, col_values=(('external_ids', {'neutron:network_name': 'tempest-ServerActionsTestOtherB-504004282-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve', 'neutron:provnet-physical-network': ''}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:50.626210 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-d6af5ce8-a435-43c7-a5f7-7abfa0cde9a1, external_ids_update=None, columns={'tag': 1309}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:50.658136 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Successfully bumped revision number for resource bedad589-7da2-43cb-a5d2-0377af836711 (type: networks) to 2 Nov 25 13:15:50.658136 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Fixing resource 5ee00da0-a86f-443d-8ccc-dab71fd41c8f (type: security_groups) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:15:50.680788 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Successfully bumped revision number for resource 5ee00da0-a86f-443d-8ccc-dab71fd41c8f (type: security_groups) to 3 Nov 25 13:15:50.680902 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Fixing resource eca35f71-24e3-4800-94ed-9d6241527898 (type: routers) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:15:50.736620 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-eca35f71-24e3-4800-94ed-9d6241527898, resource={'id': 'eca35f71-24e3-4800-94ed-9d6241527898', 'name': 'tempest-ServerActionsTestOtherB-504004282-router', 'tenant_id': '684cecb5362d45ff85e90096019cc322', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.148'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::297'}], 'enable_snat': True}, 'gw_port_id': 'b74c2e4e-dab9-4309-93ae-d8882a3f5ef3', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'ip_address': '172.24.5.148', 'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59'}, {'ip_address': '2001:db8::297', 'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2025-11-25T18:14:28Z', 'updated_at': '2025-11-25T18:14:30Z', 'revision_number': 3, 'project_id': '684cecb5362d45ff85e90096019cc322'}, resource_type=routers, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:50.736832 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-eca35f71-24e3-4800-94ed-9d6241527898, columns={'external_ids': {'neutron:router_name': 'tempest-ServerActionsTestOtherB-504004282-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:50.737044 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "b74c2e4e-dab9-4309-93ae-d8882a3f5ef3"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:50.737192 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "b74c2e4e-dab9-4309-93ae-d8882a3f5ef3"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:50.782530 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Successfully bumped revision number for resource eca35f71-24e3-4800-94ed-9d6241527898 (type: routers) to 3 Nov 25 13:15:50.782707 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Fixing resource b74c2e4e-dab9-4309-93ae-d8882a3f5ef3 (type: router_ports) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:15:50.905030 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-b74c2e4e-dab9-4309-93ae-d8882a3f5ef3, resource={'id': 'b74c2e4e-dab9-4309-93ae-d8882a3f5ef3', 'name': '', 'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'tenant_id': '', 'mac_address': 'fa:16:3e:19:ef:73', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'eca35f71-24e3-4800-94ed-9d6241527898', 'device_owner': 'network:router_gateway', 'standard_attr_id': 1836, 'fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.148'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::297'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0005535648.novalocal', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2025-11-25T18:14:28Z', 'updated_at': '2025-11-25T18:14:30Z', 'revision_number': 5, 'project_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:50.905272 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-b74c2e4e-dab9-4309-93ae-d8882a3f5ef3, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-eca35f71-24e3-4800-94ed-9d6241527898'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['2001:db8::297/64', '172.24.5.148/24'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:50.905574 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=b74c2e4e-dab9-4309-93ae-d8882a3f5ef3, lrouter_port=lrp-b74c2e4e-dab9-4309-93ae-d8882a3f5ef3, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:15:50.936150 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Successfully bumped revision number for resource b74c2e4e-dab9-4309-93ae-d8882a3f5ef3 (type: router_ports) to 5 Nov 25 13:15:50.936239 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Fixing resource 42265ba2-3540-41d8-84cb-e1f5274e4e9d (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:15:50.936457 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] delete_revision(42265ba2-3540-41d8-84cb-e1f5274e4e9d) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:15:50.939997 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Fixing resource 57876aca-9199-4f43-a53e-fb75dedb62d6 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:15:50.940194 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] delete_revision(57876aca-9199-4f43-a53e-fb75dedb62d6) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:15:50.943425 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Fixing resource 963b8808-794c-4b6a-8076-7339e47e9a30 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:15:50.943645 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] delete_revision(963b8808-794c-4b6a-8076-7339e47e9a30) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:15:50.946697 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Fixing resource e05f3775-137e-476b-955f-08e7c383aec1 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:15:50.946861 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] delete_revision(e05f3775-137e-476b-955f-08e7c383aec1) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:15:50.950067 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-ffca59dd-7ca1-4853-8373-cd784b18b74b None None] Maintenance task: Synchronization completed (took 0.36 seconds) Nov 25 13:16:00.949073 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=203,neutron:UPDATE=4,neutron:DELETE=4 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:16:13.088572 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-8f16fbd0-bf13-4fe6-ada3-9c57d225f2f5', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:c7:44:bc, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.137/24', '2001:db8::97/64'], name=lrp-82a05621-d50c-4ac6-a9b5-bafc9ddc1a5d, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:16:14.664286 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-4733e6cb-2dfe-4981-94a2-829b4f2b6981', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-8f16fbd0-bf13-4fe6-ada3-9c57d225f2f5', 'neutron:subnet_ids': 'ddf4b612-450c-4f72-a2dd-84d88938918c'}, mac=fa:16:3e:72:c5:21, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-afd8b308-45f0-47cb-8bdd-9f1dc05ccce3, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:16:14.694059 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-4733e6cb-2dfe-4981-94a2-829b4f2b6981 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:16:14.694737 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-4733e6cb-2dfe-4981-94a2-829b4f2b6981, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '4733e6cb-2dfe-4981-94a2-829b4f2b6981', 'neutron:router_id': '8f16fbd0-bf13-4fe6-ada3-9c57d225f2f5'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:16:14.695207 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-4733e6cb-2dfe-4981-94a2-829b4f2b6981, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:16:24.692919 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:16:27.318196 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-d2da96cd-48c4-4083-878f-5114b61508b2', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:86:2d:fe, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.184/24', '2001:db8::379/64'], name=lrp-c4ffe309-32ba-4701-aab8-e90bc7f9d5b6, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:16:28.662979 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-05c54ca6-e0f9-410b-a2a0-0e69d3c3ef00', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-d2da96cd-48c4-4083-878f-5114b61508b2', 'neutron:subnet_ids': '9097df4a-c0c5-4a10-b547-7128bc2e759b'}, mac=fa:16:3e:2a:02:61, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-7ab2e55e-9215-4a59-a7a3-298eec198fd7, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:16:28.697708 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-05c54ca6-e0f9-410b-a2a0-0e69d3c3ef00 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:16:28.698274 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-05c54ca6-e0f9-410b-a2a0-0e69d3c3ef00, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '05c54ca6-e0f9-410b-a2a0-0e69d3c3ef00', 'neutron:router_id': 'd2da96cd-48c4-4083-878f-5114b61508b2'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:16:28.698714 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-05c54ca6-e0f9-410b-a2a0-0e69d3c3ef00, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:16:30.704138 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-05c54ca6-e0f9-410b-a2a0-0e69d3c3ef00', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-d2da96cd-48c4-4083-878f-5114b61508b2', 'neutron:subnet_ids': '9097df4a-c0c5-4a10-b547-7128bc2e759b'}, mac=fa:16:3e:2a:02:61, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-7ab2e55e-9215-4a59-a7a3-298eec198fd7, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:16:30.705200 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-05c54ca6-e0f9-410b-a2a0-0e69d3c3ef00, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-d2da96cd-48c4-4083-878f-5114b61508b2 Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:16:31.514523 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:16:31.514875 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:16:31.514875 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-d2da96cd-48c4-4083-878f-5114b61508b2 Nov 25 13:16:31.514875 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:16:34.222688 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-4733e6cb-2dfe-4981-94a2-829b4f2b6981', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-8f16fbd0-bf13-4fe6-ada3-9c57d225f2f5', 'neutron:subnet_ids': 'ddf4b612-450c-4f72-a2dd-84d88938918c'}, mac=fa:16:3e:72:c5:21, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-afd8b308-45f0-47cb-8bdd-9f1dc05ccce3, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:16:34.226921 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-4733e6cb-2dfe-4981-94a2-829b4f2b6981, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-8f16fbd0-bf13-4fe6-ada3-9c57d225f2f5 Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:16:34.724089 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:16:34.724689 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:16:34.724689 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-8f16fbd0-bf13-4fe6-ada3-9c57d225f2f5 Nov 25 13:16:34.724689 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:16:38.695814 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:18:35.759062 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-98921d26-4f55-4dac-8177-6b6b1a45032d', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-76c35247-ddc7-4532-b843-c6a3afd3c40a', 'neutron:subnet_ids': 'ce261f80-a82e-4bd1-becd-f356a5870024'}, mac=fa:16:3e:92:94:04, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-a6e702b3-1664-4cf8-b8ca-43e86ce6a6b5, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:18:35.761798 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-98921d26-4f55-4dac-8177-6b6b1a45032d, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-76c35247-ddc7-4532-b843-c6a3afd3c40a Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:18:36.522235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:18:36.523106 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:18:36.523106 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-76c35247-ddc7-4532-b843-c6a3afd3c40a Nov 25 13:18:36.523106 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:20:28.972159 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-bedad589-7da2-43cb-a5d2-0377af836711', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-eca35f71-24e3-4800-94ed-9d6241527898', 'neutron:subnet_ids': 'b302b5f5-ba28-41d2-b43a-b6427121e67b'}, mac=fa:16:3e:f7:c7:11, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-b9ee85eb-aeda-410d-8df4-210489cef4f7, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:20:28.972159 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-bedad589-7da2-43cb-a5d2-0377af836711, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-eca35f71-24e3-4800-94ed-9d6241527898 Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:20:29.478244 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:20:29.478799 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:20:29.478799 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-eca35f71-24e3-4800-94ed-9d6241527898 Nov 25 13:20:29.478799 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:20:33.086723 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-1fb7e9f8-3503-4df9-9bb6-af269c976001', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:d0:4a:13, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.19/24', '2001:db8::36d/64'], name=lrp-9d94eeda-2a78-4317-a8d3-f225a6ae3c61, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:20:34.942231 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-78ba2d0f-d43a-4b32-8808-3857e444fcae', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-1fb7e9f8-3503-4df9-9bb6-af269c976001', 'neutron:subnet_ids': '4cd3d5b2-2939-435a-a5ea-c3ba74dce656'}, mac=fa:16:3e:35:e2:b0, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-41e63c3d-9408-4037-aae4-4851a2bd4970, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:20:34.975764 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-78ba2d0f-d43a-4b32-8808-3857e444fcae synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:20:34.976655 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-78ba2d0f-d43a-4b32-8808-3857e444fcae, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '78ba2d0f-d43a-4b32-8808-3857e444fcae', 'neutron:router_id': '1fb7e9f8-3503-4df9-9bb6-af269c976001'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:20:34.977123 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-78ba2d0f-d43a-4b32-8808-3857e444fcae, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:20:44.975032 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:20:50.583869 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=98974) _process_scheduled /usr/local/lib/python3.12/site-packages/futurist/periodics.py:638}} Nov 25 13:20:50.592693 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e6fa335a-ab05-47e8-9a65-88078b32a1a1 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}} Nov 25 13:20:50.592992 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e6fa335a-ab05-47e8-9a65-88078b32a1a1 None None] Maintenance task: Number of inconsistencies found at delete: router_ports=4 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:20:50.593264 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e6fa335a-ab05-47e8-9a65-88078b32a1a1 None None] Maintenance task: Fixing resource 56e8f3ff-22fc-4822-a7e1-fb5539d47513 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:20:50.593790 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-e6fa335a-ab05-47e8-9a65-88078b32a1a1 None None] delete_revision(56e8f3ff-22fc-4822-a7e1-fb5539d47513) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:20:50.598904 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e6fa335a-ab05-47e8-9a65-88078b32a1a1 None None] Maintenance task: Fixing resource 82a05621-d50c-4ac6-a9b5-bafc9ddc1a5d (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:20:50.599247 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-e6fa335a-ab05-47e8-9a65-88078b32a1a1 None None] delete_revision(82a05621-d50c-4ac6-a9b5-bafc9ddc1a5d) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:20:50.604876 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e6fa335a-ab05-47e8-9a65-88078b32a1a1 None None] Maintenance task: Fixing resource b74c2e4e-dab9-4309-93ae-d8882a3f5ef3 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:20:50.605209 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-e6fa335a-ab05-47e8-9a65-88078b32a1a1 None None] delete_revision(b74c2e4e-dab9-4309-93ae-d8882a3f5ef3) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:20:50.610032 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e6fa335a-ab05-47e8-9a65-88078b32a1a1 None None] Maintenance task: Fixing resource c4ffe309-32ba-4701-aab8-e90bc7f9d5b6 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:20:50.610367 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-e6fa335a-ab05-47e8-9a65-88078b32a1a1 None None] delete_revision(c4ffe309-32ba-4701-aab8-e90bc7f9d5b6) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:20:50.614732 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-e6fa335a-ab05-47e8-9a65-88078b32a1a1 None None] Maintenance task: Synchronization completed (took 0.02 seconds) Nov 25 13:20:56.889253 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-78ba2d0f-d43a-4b32-8808-3857e444fcae', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-1fb7e9f8-3503-4df9-9bb6-af269c976001', 'neutron:subnet_ids': '4cd3d5b2-2939-435a-a5ea-c3ba74dce656'}, mac=fa:16:3e:35:e2:b0, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-41e63c3d-9408-4037-aae4-4851a2bd4970, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:20:56.891929 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-78ba2d0f-d43a-4b32-8808-3857e444fcae, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-1fb7e9f8-3503-4df9-9bb6-af269c976001 Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:20:57.528098 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:20:57.528628 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:20:57.528628 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-1fb7e9f8-3503-4df9-9bb6-af269c976001 Nov 25 13:20:57.528628 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:21:00.612979 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=6,neutron:DELETE=4 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:21:01.095058 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-b3675bd7-b808-4955-bc87-144c26d657d8', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:8e:6a:67, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.245/24', '2001:db8::31a/64'], name=lrp-66ee4b77-7750-478b-acdc-36c9bb95f311, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:21:02.737122 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-136d1b16-2f55-4bd8-b09f-28159c831aee', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-b3675bd7-b808-4955-bc87-144c26d657d8', 'neutron:subnet_ids': 'b7b5342d-b307-46ba-b4e6-ed426d6c7a04'}, mac=fa:16:3e:6f:d1:cd, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-7ad880f6-aa00-476e-bf53-713436339997, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:21:02.773629 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-136d1b16-2f55-4bd8-b09f-28159c831aee synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:21:02.774383 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-136d1b16-2f55-4bd8-b09f-28159c831aee, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '136d1b16-2f55-4bd8-b09f-28159c831aee', 'neutron:router_id': 'b3675bd7-b808-4955-bc87-144c26d657d8'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:21:02.774891 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-136d1b16-2f55-4bd8-b09f-28159c831aee, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:21:12.771992 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:21:24.564720 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-136d1b16-2f55-4bd8-b09f-28159c831aee', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-b3675bd7-b808-4955-bc87-144c26d657d8', 'neutron:subnet_ids': 'b7b5342d-b307-46ba-b4e6-ed426d6c7a04'}, mac=fa:16:3e:6f:d1:cd, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-7ad880f6-aa00-476e-bf53-713436339997, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:21:24.566770 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-136d1b16-2f55-4bd8-b09f-28159c831aee, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-b3675bd7-b808-4955-bc87-144c26d657d8 Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:21:25.230318 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:21:25.230685 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:21:25.230685 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-b3675bd7-b808-4955-bc87-144c26d657d8 Nov 25 13:21:25.230685 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:21:28.797804 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-e370fc5d-6e77-4b24-97cc-be17f1ecc85d', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:34:df:9c, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.209/24', '2001:db8::1d9/64'], name=lrp-d0c41dcb-0e22-4ae1-b5b1-569701954d99, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:21:30.446159 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-322cd348-7df1-4719-b044-3aff36e3d74b', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-e370fc5d-6e77-4b24-97cc-be17f1ecc85d', 'neutron:subnet_ids': '0f6824b4-9bb3-4640-a0d4-e3dcf562cb5d'}, mac=fa:16:3e:29:99:c8, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-7f2fe91a-ceaf-4a7e-b43a-ff7e168f15b3, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:21:30.483251 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-322cd348-7df1-4719-b044-3aff36e3d74b synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:21:30.484457 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-322cd348-7df1-4719-b044-3aff36e3d74b, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '322cd348-7df1-4719-b044-3aff36e3d74b', 'neutron:router_id': 'e370fc5d-6e77-4b24-97cc-be17f1ecc85d'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:21:30.485066 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-322cd348-7df1-4719-b044-3aff36e3d74b, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:21:40.482087 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:25:50.585032 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=98974) _process_scheduled /usr/local/lib/python3.12/site-packages/futurist/periodics.py:638}} Nov 25 13:25:50.594674 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}} Nov 25 13:25:50.594674 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Maintenance task: Number of inconsistencies found at create/update: networks=1, routers=1, router_ports=1 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:25:50.594674 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Maintenance task: Number of inconsistencies found at delete: router_ports=2 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:25:50.594674 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Maintenance task: Fixing resource 322cd348-7df1-4719-b044-3aff36e3d74b (type: networks) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:25:50.646060 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-322cd348-7df1-4719-b044-3aff36e3d74b, resource={'id': '322cd348-7df1-4719-b044-3aff36e3d74b', 'name': 'tempest-ServerStableDeviceRescueTest-1912946509-network', 'tenant_id': '075a76e8a1574afc863bcb379be1dc26', 'project_id': '075a76e8a1574afc863bcb379be1dc26', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['0f6824b4-9bb3-4640-a0d4-e3dcf562cb5d'], 'standard_attr_id': 2101, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2025-11-25T18:21:26Z', 'updated_at': '2025-11-25T18:21:27Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 34190}, resource_type=networks, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:25:50.646573 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-322cd348-7df1-4719-b044-3aff36e3d74b, col_values=(('external_ids', {'neutron:network_name': 'tempest-ServerStableDeviceRescueTest-1912946509-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve', 'neutron:provnet-physical-network': ''}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:25:50.647044 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-4d4b16be-ca46-4103-ad92-7e668d18c84d, external_ids_update=None, columns={'tag': 34190}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:25:50.683419 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Successfully bumped revision number for resource 322cd348-7df1-4719-b044-3aff36e3d74b (type: networks) to 2 Nov 25 13:25:50.683726 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Maintenance task: Fixing resource e370fc5d-6e77-4b24-97cc-be17f1ecc85d (type: routers) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:25:50.793080 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-e370fc5d-6e77-4b24-97cc-be17f1ecc85d, resource={'id': 'e370fc5d-6e77-4b24-97cc-be17f1ecc85d', 'name': 'tempest-ServerStableDeviceRescueTest-1912946509-router', 'tenant_id': '075a76e8a1574afc863bcb379be1dc26', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.209'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::1d9'}], 'enable_snat': True}, 'gw_port_id': 'd0c41dcb-0e22-4ae1-b5b1-569701954d99', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'ip_address': '172.24.5.209', 'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59'}, {'ip_address': '2001:db8::1d9', 'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2025-11-25T18:21:27Z', 'updated_at': '2025-11-25T18:21:29Z', 'revision_number': 3, 'project_id': '075a76e8a1574afc863bcb379be1dc26'}, resource_type=routers, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:25:50.793340 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-e370fc5d-6e77-4b24-97cc-be17f1ecc85d, columns={'external_ids': {'neutron:router_name': 'tempest-ServerStableDeviceRescueTest-1912946509-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:25:50.793621 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "d0c41dcb-0e22-4ae1-b5b1-569701954d99"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:25:50.793792 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "d0c41dcb-0e22-4ae1-b5b1-569701954d99"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:25:50.847088 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Successfully bumped revision number for resource e370fc5d-6e77-4b24-97cc-be17f1ecc85d (type: routers) to 3 Nov 25 13:25:50.847337 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Maintenance task: Fixing resource d0c41dcb-0e22-4ae1-b5b1-569701954d99 (type: router_ports) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:25:50.959813 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-d0c41dcb-0e22-4ae1-b5b1-569701954d99, resource={'id': 'd0c41dcb-0e22-4ae1-b5b1-569701954d99', 'name': '', 'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'tenant_id': '', 'mac_address': 'fa:16:3e:34:df:9c', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e370fc5d-6e77-4b24-97cc-be17f1ecc85d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 2106, 'fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.209'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::1d9'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0005535648.novalocal', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2025-11-25T18:21:27Z', 'updated_at': '2025-11-25T18:21:29Z', 'revision_number': 5, 'project_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:25:50.959967 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-d0c41dcb-0e22-4ae1-b5b1-569701954d99, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-e370fc5d-6e77-4b24-97cc-be17f1ecc85d'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.209/24', '2001:db8::1d9/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:25:50.959967 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=d0c41dcb-0e22-4ae1-b5b1-569701954d99, lrouter_port=lrp-d0c41dcb-0e22-4ae1-b5b1-569701954d99, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:25:50.985438 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Successfully bumped revision number for resource d0c41dcb-0e22-4ae1-b5b1-569701954d99 (type: router_ports) to 5 Nov 25 13:25:50.985569 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Maintenance task: Fixing resource 66ee4b77-7750-478b-acdc-36c9bb95f311 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:25:50.985781 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] delete_revision(66ee4b77-7750-478b-acdc-36c9bb95f311) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:25:50.990528 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Maintenance task: Fixing resource 9d94eeda-2a78-4317-a8d3-f225a6ae3c61 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:25:50.990798 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] delete_revision(9d94eeda-2a78-4317-a8d3-f225a6ae3c61) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:25:50.994843 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-460e346e-c103-4589-be38-04b3bbf33798 None None] Maintenance task: Synchronization completed (took 0.40 seconds) Nov 25 13:25:53.550742 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-322cd348-7df1-4719-b044-3aff36e3d74b', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-e370fc5d-6e77-4b24-97cc-be17f1ecc85d', 'neutron:subnet_ids': '0f6824b4-9bb3-4640-a0d4-e3dcf562cb5d'}, mac=fa:16:3e:29:99:c8, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-7f2fe91a-ceaf-4a7e-b43a-ff7e168f15b3, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:25:53.551850 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-322cd348-7df1-4719-b044-3aff36e3d74b, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-e370fc5d-6e77-4b24-97cc-be17f1ecc85d Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:25:54.208502 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:25:54.209343 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:25:54.209343 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-e370fc5d-6e77-4b24-97cc-be17f1ecc85d Nov 25 13:25:54.209343 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:25:59.765877 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-139d33df-6e9d-4362-b79a-79b0231a4023', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:54:7f:6d, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.195/24', '2001:db8::314/64'], name=lrp-e3cf76e3-32c6-4c5f-a92c-ca2c5ffb2ed5, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:26:00.992930 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=188,neutron:UPDATE=3,neutron:DELETE=2 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:26:01.923656 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-ae22e108-7494-468e-a2a1-03adb2447f6f', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-139d33df-6e9d-4362-b79a-79b0231a4023', 'neutron:subnet_ids': 'bca8bd08-869d-40c5-aefd-fb7aa82be78a'}, mac=fa:16:3e:ae:12:38, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-c62249a2-ebee-485d-bd7d-8d703b7a2d88, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:26:01.973611 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-ae22e108-7494-468e-a2a1-03adb2447f6f synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:26:01.974668 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-ae22e108-7494-468e-a2a1-03adb2447f6f, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': 'ae22e108-7494-468e-a2a1-03adb2447f6f', 'neutron:router_id': '139d33df-6e9d-4362-b79a-79b0231a4023'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:26:01.975235 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-ae22e108-7494-468e-a2a1-03adb2447f6f, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:26:11.971861 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:28:32.581172 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-ae22e108-7494-468e-a2a1-03adb2447f6f', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-139d33df-6e9d-4362-b79a-79b0231a4023', 'neutron:subnet_ids': 'bca8bd08-869d-40c5-aefd-fb7aa82be78a'}, mac=fa:16:3e:ae:12:38, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-c62249a2-ebee-485d-bd7d-8d703b7a2d88, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:28:32.584565 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-ae22e108-7494-468e-a2a1-03adb2447f6f, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-139d33df-6e9d-4362-b79a-79b0231a4023 Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:28:33.232735 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:28:33.233579 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:28:33.233579 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-139d33df-6e9d-4362-b79a-79b0231a4023 Nov 25 13:28:33.233579 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:30:11.059933 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-371bbb10-4708-4da4-9a81-ce503b69bf6b', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-cd2510d8-64df-43c1-a040-83c61b9ce559', 'neutron:subnet_ids': 'ad4bd48f-ff0f-4d98-a84d-17c8dcf34376'}, mac=fa:16:3e:08:65:ae, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-fc6fc5b5-48a0-4fe4-82ec-73c009fe6566, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:12.333758 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-827b6a0c-5abc-4e27-9709-7877146a546e', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-cd2510d8-64df-43c1-a040-83c61b9ce559', 'neutron:subnet_ids': '6fa90791-bf9e-44cc-9097-6a37b9ae084c'}, mac=fa:16:3e:96:c6:da, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8:1::1/64'], name=lrp-113503ab-c0a4-4533-96b0-f512af8443d8, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:18.391970 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-827b6a0c-5abc-4e27-9709-7877146a546e', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-cd2510d8-64df-43c1-a040-83c61b9ce559', 'neutron:subnet_ids': '6fa90791-bf9e-44cc-9097-6a37b9ae084c'}, mac=fa:16:3e:96:c6:da, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8:1::1/64'], name=lrp-113503ab-c0a4-4533-96b0-f512af8443d8, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:18.902830 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-371bbb10-4708-4da4-9a81-ce503b69bf6b', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-cd2510d8-64df-43c1-a040-83c61b9ce559', 'neutron:subnet_ids': 'ad4bd48f-ff0f-4d98-a84d-17c8dcf34376'}, mac=fa:16:3e:08:65:ae, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-fc6fc5b5-48a0-4fe4-82ec-73c009fe6566, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:22.486630 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-cda5a41e-89ed-44ef-bf25-79b4f667999f', 'neutron:revision_number': '3', 'neutron:router_name': 'neutron-23cf3465-ed3a-41d6-9d68-d27f2c95ba7f', 'neutron:subnet_ids': '5ec56fe9-b091-4781-9197-c3584ce80477'}, mac=fa:16:3e:0c:d1:24, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::320/64'], name=lrp-93265fc3-7c2c-420a-9e0a-e891f25a9d86, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:23.145321 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-cda5a41e-89ed-44ef-bf25-79b4f667999f', 'neutron:revision_number': '3', 'neutron:router_name': 'neutron-23cf3465-ed3a-41d6-9d68-d27f2c95ba7f', 'neutron:subnet_ids': '5ec56fe9-b091-4781-9197-c3584ce80477'}, mac=fa:16:3e:0c:d1:24, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::320/64'], name=lrp-93265fc3-7c2c-420a-9e0a-e891f25a9d86, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:25.874202 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-06051ce3-86e6-4f1d-9202-308f3d1b9446', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-06c99413-bd4f-4426-9288-c587a7930902', 'neutron:subnet_ids': '77ff149b-f4ef-4bd8-addb-83be4c7f90a9'}, mac=fa:16:3e:65:11:e6, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-b7770bbc-25f7-47dc-8efb-cc3c6b0a1d29, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:26.998289 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-06051ce3-86e6-4f1d-9202-308f3d1b9446', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-06c99413-bd4f-4426-9288-c587a7930902', 'neutron:subnet_ids': '77ff149b-f4ef-4bd8-addb-83be4c7f90a9'}, mac=fa:16:3e:65:11:e6, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-b7770bbc-25f7-47dc-8efb-cc3c6b0a1d29, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:28.677865 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-69dc3258-0f98-49bf-9fe2-121121f44de0', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:92:49:cb, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.143/24', '2001:db8::271/64'], name=lrp-391d2c73-0ddf-4101-984c-5326b3a3eff0, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-69dc3258-0f98-49bf-9fe2-121121f44de0 Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:30:30.017188 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:30:30.018174 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:30:30.018174 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-69dc3258-0f98-49bf-9fe2-121121f44de0 Nov 25 13:30:30.018174 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:30:31.891835 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-c510c4a4-5e33-43f1-99ff-4c9b74d60fc8', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-b38bab69-d72e-4793-97be-f74405642dc7', 'neutron:subnet_ids': '919d8ea0-ec6a-4b34-b43b-54ee4d7421e1'}, mac=fa:16:3e:4d:16:15, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-4f431bcb-a52a-4ded-9655-502ed9ab1603, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:35.023596 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-c510c4a4-5e33-43f1-99ff-4c9b74d60fc8', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-b38bab69-d72e-4793-97be-f74405642dc7', 'neutron:subnet_ids': '919d8ea0-ec6a-4b34-b43b-54ee4d7421e1'}, mac=fa:16:3e:4d:16:15, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-4f431bcb-a52a-4ded-9655-502ed9ab1603, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:37.454661 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-9fd5305e-b343-4cdb-a65e-0a6c27dae86d', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-4da39c43-5980-4eea-b725-7317061e2b4a', 'neutron:subnet_ids': '18baec78-a374-44f0-8588-aab49e47d848'}, mac=fa:16:3e:1f:65:97, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-cd2126e8-5a99-4c71-b1d3-e5abae2a6760, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:39.227525 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-cd5b5a34-44b4-4cd9-b3be-4c1964464c27', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-4da39c43-5980-4eea-b725-7317061e2b4a', 'neutron:subnet_ids': 'fbf86eb6-d428-4b6e-b9a3-28686b39efe0'}, mac=fa:16:3e:a0:2c:d1, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8:1::1/64'], name=lrp-186a78e6-1c17-4a3e-9d01-b48910646e50, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:41.263484 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-6e8910c6-4ec9-466a-a921-3fa2464f27d7', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-4da39c43-5980-4eea-b725-7317061e2b4a', 'neutron:subnet_ids': 'dc7e1049-7ce4-42ff-94c9-4e7cb98e9b94'}, mac=fa:16:3e:b1:ff:e1, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8:2::1/64'], name=lrp-d472037e-49b3-4096-a826-0e72342dc80e, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:43.665259 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-e3308791-1f5f-434b-b4ed-683c540feef8', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-4da39c43-5980-4eea-b725-7317061e2b4a', 'neutron:subnet_ids': '1a577f6f-1493-40fa-80e8-d33437927d4d'}, mac=fa:16:3e:cc:d2:0d, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8:3::1/64'], name=lrp-48e88f8b-50dc-4904-8630-13dc6dcfe7f0, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:50.358530 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-e3308791-1f5f-434b-b4ed-683c540feef8', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-4da39c43-5980-4eea-b725-7317061e2b4a', 'neutron:subnet_ids': '1a577f6f-1493-40fa-80e8-d33437927d4d'}, mac=fa:16:3e:cc:d2:0d, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8:3::1/64'], name=lrp-48e88f8b-50dc-4904-8630-13dc6dcfe7f0, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:50.586019 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=98974) _process_scheduled /usr/local/lib/python3.12/site-packages/futurist/periodics.py:638}} Nov 25 13:30:50.597022 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1db3b3c4-d296-4cf0-988e-9e118fd6cf22 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}} Nov 25 13:30:50.597254 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1db3b3c4-d296-4cf0-988e-9e118fd6cf22 None None] Maintenance task: Number of inconsistencies found at delete: router_ports=3 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:30:50.597464 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1db3b3c4-d296-4cf0-988e-9e118fd6cf22 None None] Maintenance task: Fixing resource 391d2c73-0ddf-4101-984c-5326b3a3eff0 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:30:50.597925 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-1db3b3c4-d296-4cf0-988e-9e118fd6cf22 None None] delete_revision(391d2c73-0ddf-4101-984c-5326b3a3eff0) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:30:50.622378 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1db3b3c4-d296-4cf0-988e-9e118fd6cf22 None None] Maintenance task: Fixing resource d0c41dcb-0e22-4ae1-b5b1-569701954d99 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:30:50.622975 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-1db3b3c4-d296-4cf0-988e-9e118fd6cf22 None None] delete_revision(d0c41dcb-0e22-4ae1-b5b1-569701954d99) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:30:50.655971 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1db3b3c4-d296-4cf0-988e-9e118fd6cf22 None None] Maintenance task: Fixing resource e3cf76e3-32c6-4c5f-a92c-ca2c5ffb2ed5 (type: router_ports) at delete {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:418}} Nov 25 13:30:50.656246 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.db.ovn_revision_numbers_db [None req-1db3b3c4-d296-4cf0-988e-9e118fd6cf22 None None] delete_revision(e3cf76e3-32c6-4c5f-a92c-ca2c5ffb2ed5) {{(pid=98974) delete_revision /opt/stack/neutron/neutron/db/ovn_revision_numbers_db.py:112}} Nov 25 13:30:50.665796 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-1db3b3c4-d296-4cf0-988e-9e118fd6cf22 None None] Maintenance task: Synchronization completed (took 0.07 seconds) Nov 25 13:30:51.568363 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-6e8910c6-4ec9-466a-a921-3fa2464f27d7', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-4da39c43-5980-4eea-b725-7317061e2b4a', 'neutron:subnet_ids': 'dc7e1049-7ce4-42ff-94c9-4e7cb98e9b94'}, mac=fa:16:3e:b1:ff:e1, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8:2::1/64'], name=lrp-d472037e-49b3-4096-a826-0e72342dc80e, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:52.554673 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-cd5b5a34-44b4-4cd9-b3be-4c1964464c27', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-4da39c43-5980-4eea-b725-7317061e2b4a', 'neutron:subnet_ids': 'fbf86eb6-d428-4b6e-b9a3-28686b39efe0'}, mac=fa:16:3e:a0:2c:d1, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8:1::1/64'], name=lrp-186a78e6-1c17-4a3e-9d01-b48910646e50, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:30:53.469108 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-9fd5305e-b343-4cdb-a65e-0a6c27dae86d', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-4da39c43-5980-4eea-b725-7317061e2b4a', 'neutron:subnet_ids': '18baec78-a374-44f0-8588-aab49e47d848'}, mac=fa:16:3e:1f:65:97, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['2001:db8::1/64'], name=lrp-cd2126e8-5a99-4c71-b1d3-e5abae2a6760, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:31:00.658998 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=5,neutron:DELETE=3 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:31:42.201143 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-15dd356d-ffa4-4616-a4bf-afe56969ff51', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:9f:09:40, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.85/24', '2001:db8::11/64'], name=lrp-5eb04c57-4964-4fc1-931a-666f73adc2a9, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:31:43.703616 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-62bb3376-8539-4df4-a6ff-a891d9c053e2', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-15dd356d-ffa4-4616-a4bf-afe56969ff51', 'neutron:subnet_ids': '91d4630d-0957-4db0-9f69-c34715ad433d'}, mac=fa:16:3e:77:96:ff, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-1ae61c1a-baaf-4946-922f-5670f01d711c, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:31:43.746824 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-62bb3376-8539-4df4-a6ff-a891d9c053e2 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:31:43.747968 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-62bb3376-8539-4df4-a6ff-a891d9c053e2, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '62bb3376-8539-4df4-a6ff-a891d9c053e2', 'neutron:router_id': '15dd356d-ffa4-4616-a4bf-afe56969ff51'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:31:43.748440 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-62bb3376-8539-4df4-a6ff-a891d9c053e2, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:31:53.745105 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=17 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:35:50.587913 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG futurist.periodics [-] Submitting periodic callback 'neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance.DBInconsistenciesPeriodics.check_for_inconsistencies' {{(pid=98974) _process_scheduled /usr/local/lib/python3.12/site-packages/futurist/periodics.py:638}} Nov 25 13:35:50.600730 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Synchronizing Neutron and OVN databases started {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:386}} Nov 25 13:35:50.600993 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Number of inconsistencies found at create/update: networks=1, routers=1, router_ports=1 {{(pid=98974) _log /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:367}} Nov 25 13:35:50.601185 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 62bb3376-8539-4df4-a6ff-a891d9c053e2 (type: networks) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:35:50.657920 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-62bb3376-8539-4df4-a6ff-a891d9c053e2, resource={'id': '62bb3376-8539-4df4-a6ff-a891d9c053e2', 'name': 'tempest-VolumesBackupsTest-785415460-network', 'tenant_id': '8ad212ee97684fec9bf2f2dc25f47549', 'project_id': '8ad212ee97684fec9bf2f2dc25f47549', 'admin_state_up': True, 'mtu': 1372, 'status': 'ACTIVE', 'subnets': ['91d4630d-0957-4db0-9f69-c34715ad433d'], 'standard_attr_id': 2387, 'shared': False, 'availability_zone_hints': [], 'availability_zones': [], 'ipv4_address_scope': None, 'ipv6_address_scope': None, 'router:external': False, 'qinq': False, 'vlan_transparent': False, 'description': '', 'port_security_enabled': True, 'tags': [], 'created_at': '2025-11-25T18:31:40Z', 'updated_at': '2025-11-25T18:31:40Z', 'revision_number': 2, 'provider:network_type': 'geneve', 'provider:physical_network': None, 'provider:segmentation_id': 21292}, resource_type=networks, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:35:50.658342 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Logical_Switch, record=neutron-62bb3376-8539-4df4-a6ff-a891d9c053e2, col_values=(('external_ids', {'neutron:network_name': 'tempest-VolumesBackupsTest-785415460-network', 'neutron:mtu': '1372', 'neutron:revision_number': '2', 'neutron:availability_zone_hints': '', 'neutron:provnet-network-type': 'geneve', 'neutron:provnet-physical-network': ''}), ('other_config', {'mcast_snoop': 'false', 'mcast_flood_unregistered': 'false', 'vlan-passthru': 'false'})), if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:35:50.658888 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLSwitchPortCommand(_result=None, lport=provnet-a6761537-f207-49a7-b731-7b42729f3d74, external_ids_update=None, columns={'tag': 21292}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:35:50.692847 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Successfully bumped revision number for resource 62bb3376-8539-4df4-a6ff-a891d9c053e2 (type: networks) to 2 Nov 25 13:35:50.693003 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 15dd356d-ffa4-4616-a4bf-afe56969ff51 (type: routers) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:35:50.969009 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=neutron-15dd356d-ffa4-4616-a4bf-afe56969ff51, resource={'id': '15dd356d-ffa4-4616-a4bf-afe56969ff51', 'name': 'tempest-VolumesBackupsTest-785415460-router', 'tenant_id': '8ad212ee97684fec9bf2f2dc25f47549', 'admin_state_up': True, 'status': 'ACTIVE', 'enable_snat': True, 'external_gateway_info': {'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.85'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::11'}], 'enable_snat': True}, 'gw_port_id': '5eb04c57-4964-4fc1-931a-666f73adc2a9', 'description': '', 'availability_zones': [], 'distributed': False, 'ha': True, 'ha_vr_id': 0, 'availability_zone_hints': [], 'enable_default_route_ecmp': False, 'enable_default_route_bfd': False, 'external_gateways': [{'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'external_fixed_ips': [{'ip_address': '172.24.5.85', 'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59'}, {'ip_address': '2001:db8::11', 'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f'}]}], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2025-11-25T18:31:41Z', 'updated_at': '2025-11-25T18:31:43Z', 'revision_number': 3, 'project_id': '8ad212ee97684fec9bf2f2dc25f47549'}, resource_type=routers, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:35:50.969268 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterCommand(_result=None, name=neutron-15dd356d-ffa4-4616-a4bf-afe56969ff51, columns={'external_ids': {'neutron:router_name': 'tempest-VolumesBackupsTest-785415460-router', 'neutron:revision_number': '3', 'neutron:availability_zone_hints': ''}, 'enabled': True}, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:35:50.969600 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'from-lport'), ('priority', '=', 2002), ('match', '=', 'inport == "5eb04c57-4964-4fc1-931a-666f73adc2a9"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:35:50.969821 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): QoSDelCommand(_result=None, switch=neutron-b78b8a49-a328-4f90-8362-b211cafafafc, conditions=[('direction', '=', 'to-lport'), ('priority', '=', 2002), ('match', '=', 'outport == "5eb04c57-4964-4fc1-931a-666f73adc2a9"')], if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:35:51.081436 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Successfully bumped revision number for resource 15dd356d-ffa4-4616-a4bf-afe56969ff51 (type: routers) to 3 Nov 25 13:35:51.081690 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Fixing resource 5eb04c57-4964-4fc1-931a-666f73adc2a9 (type: router_ports) at create/update {{(pid=98974) check_for_inconsistencies /opt/stack/neutron/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/maintenance.py:396}} Nov 25 13:35:51.183509 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): CheckRevisionNumberCommand(_result=None, name=lrp-5eb04c57-4964-4fc1-931a-666f73adc2a9, resource={'id': '5eb04c57-4964-4fc1-931a-666f73adc2a9', 'name': '', 'network_id': 'b78b8a49-a328-4f90-8362-b211cafafafc', 'tenant_id': '', 'mac_address': 'fa:16:3e:9f:09:40', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '15dd356d-ffa4-4616-a4bf-afe56969ff51', 'device_owner': 'network:router_gateway', 'standard_attr_id': 2392, 'fixed_ips': [{'subnet_id': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59', 'ip_address': '172.24.5.85'}, {'subnet_id': '6a9ec510-9d38-4017-abd6-7fc4a311593f', 'ip_address': '2001:db8::11'}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'np0005535648.novalocal', 'binding:vif_type': 'ovs', 'binding:vif_details': {'port_filter': True, 'connectivity': 'l2', 'bridge_name': 'br-int', 'datapath_type': 'system', 'bound_drivers': {'0': 'ovn'}}, 'port_security_enabled': False, 'tags': [], 'created_at': '2025-11-25T18:31:41Z', 'updated_at': '2025-11-25T18:31:43Z', 'revision_number': 5, 'project_id': ''}, resource_type=router_ports, if_exists=True) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:35:51.183765 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): UpdateLRouterPortCommand(_result=None, name=lrp-5eb04c57-4964-4fc1-931a-666f73adc2a9, columns={'external_ids': {'neutron:revision_number': '5', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:is_ext_gw': 'True', 'neutron:router_name': 'neutron-15dd356d-ffa4-4616-a4bf-afe56969ff51'}, 'options': {'reside-on-redirect-chassis': 'true', 'gateway_mtu': '1372'}, 'networks': ['172.24.5.85/24', '2001:db8::11/64'], 'ipv6_ra_configs': {}}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:35:51.183961 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): SetLRouterPortInLSwitchPortCommand(_result=None, lswitch_port=5eb04c57-4964-4fc1-931a-666f73adc2a9, lrouter_port=lrp-5eb04c57-4964-4fc1-931a-666f73adc2a9, is_gw_port=True, if_exists=True, lsp_address=router) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:35:51.213496 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.db.ovn_revision_numbers_db [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Successfully bumped revision number for resource 5eb04c57-4964-4fc1-931a-666f73adc2a9 (type: router_ports) to 5 Nov 25 13:35:51.213496 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.maintenance [None req-a3a96104-4cdf-4d00-99c1-8cf5ca21f8a1 None None] Maintenance task: Synchronization completed (took 0.61 seconds) Nov 25 13:36:01.210522 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=186,neutron:UPDATE=3 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:37:06.464758 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-62bb3376-8539-4df4-a6ff-a891d9c053e2', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-15dd356d-ffa4-4616-a4bf-afe56969ff51', 'neutron:subnet_ids': '91d4630d-0957-4db0-9f69-c34715ad433d'}, mac=fa:16:3e:77:96:ff, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-1ae61c1a-baaf-4946-922f-5670f01d711c, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:37:06.469718 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-62bb3376-8539-4df4-a6ff-a891d9c053e2, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-15dd356d-ffa4-4616-a4bf-afe56969ff51 Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:37:06.959833 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:37:06.961917 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:37:06.961917 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-15dd356d-ffa4-4616-a4bf-afe56969ff51 Nov 25 13:37:06.961917 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:37:40.890088 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-c4f4c384-6696-42f8-949c-124971aa0f67', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:22:62:09, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.236/24', '2001:db8::e5/64'], name=lrp-fc91bb0b-71aa-475a-a204-1062f209a097, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:37:44.244151 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-73bc0dd7-b90d-494b-83c9-010374dbbfe2', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-c4f4c384-6696-42f8-949c-124971aa0f67', 'neutron:subnet_ids': '37eb5e62-de25-4a5e-81a1-ec3de70a6eb0'}, mac=fa:16:3e:06:1f:1e, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-690d60ee-d3d6-4fcd-a954-d7ab3858df67, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:37:44.279268 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-73bc0dd7-b90d-494b-83c9-010374dbbfe2 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:37:44.280027 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-73bc0dd7-b90d-494b-83c9-010374dbbfe2, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '73bc0dd7-b90d-494b-83c9-010374dbbfe2', 'neutron:router_id': 'c4f4c384-6696-42f8-949c-124971aa0f67'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:37:44.280560 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-73bc0dd7-b90d-494b-83c9-010374dbbfe2, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:37:46.986641 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'True', 'neutron:network_name': 'neutron-b78b8a49-a328-4f90-8362-b211cafafafc', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-1a214688-2bb6-4560-b80c-64a60cd68efa', 'neutron:subnet_ids': 'd8246a81-26f7-4b7a-b48c-fd5e24c9ba59 6a9ec510-9d38-4017-abd6-7fc4a311593f'}, mac=fa:16:3e:7c:6a:2d, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['172.24.5.53/24', '2001:db8::102/64'], name=lrp-5751dec1-2f9a-430e-a987-a30677a8a4f9, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={'gateway_mtu': '1430', 'reside-on-redirect-chassis': 'true'}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:37:48.546165 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched CREATE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-317531d0-5776-4925-bb27-3dcd5ebedab4', 'neutron:revision_number': '1', 'neutron:router_name': 'neutron-1a214688-2bb6-4560-b80c-64a60cd68efa', 'neutron:subnet_ids': 'edbb7eeb-9a27-48c6-855b-ecfe98301352'}, mac=fa:16:3e:e0:b6:88, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-180d5909-f827-477a-90ff-6f05685401e5, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:37:48.585760 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: INFO neutron.common.ovn.utils [-] HA Chassis Group neutron-317531d0-5776-4925-bb27-3dcd5ebedab4 synchronized; highest priority chassis 6a9bebab-14ce-4e1d-bd80-e72c835c4b0e Nov 25 13:37:48.586622 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): HAChassisGroupAddCommand(_result=None, name=neutron-317531d0-5776-4925-bb27-3dcd5ebedab4, may_exist=True, columns={'external_ids': {'neutron:availability_zone_hints': '', 'neutron:network_id': '317531d0-5776-4925-bb27-3dcd5ebedab4', 'neutron:router_id': '1a214688-2bb6-4560-b80c-64a60cd68efa'}}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:37:48.587050 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): HAChassisGroupAddChassisCommand(_result=None, hcg=neutron-317531d0-5776-4925-bb27-3dcd5ebedab4, chassis=6a9bebab-14ce-4e1d-bd80-e72c835c4b0e, priority=1, columns={}) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:37:58.584472 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG dbcounter [-] [98974] Writing DB stats neutron:SELECT=34 {{(pid=98974) stat_writer /usr/local/lib/python3.12/site-packages/dbcounter.py:115}} Nov 25 13:39:03.271708 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-73bc0dd7-b90d-494b-83c9-010374dbbfe2', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-c4f4c384-6696-42f8-949c-124971aa0f67', 'neutron:subnet_ids': '37eb5e62-de25-4a5e-81a1-ec3de70a6eb0'}, mac=fa:16:3e:06:1f:1e, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-690d60ee-d3d6-4fcd-a954-d7ab3858df67, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:39:03.273474 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-73bc0dd7-b90d-494b-83c9-010374dbbfe2, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-c4f4c384-6696-42f8-949c-124971aa0f67 Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:39:03.808786 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:39:03.809928 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:39:03.809928 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-c4f4c384-6696-42f8-949c-124971aa0f67 Nov 25 13:39:03.809928 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Nov 25 13:39:05.015182 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.event [-] Matched DELETE: LogicalRouterPortEvent(events=('create', 'delete'), table='Logical_Router_Port', conditions=None, old_conditions=None), priority=20 to row=Logical_Router_Port(ha_chassis_group=[], external_ids={'neutron:is_ext_gw': 'False', 'neutron:network_name': 'neutron-317531d0-5776-4925-bb27-3dcd5ebedab4', 'neutron:revision_number': '2', 'neutron:router_name': 'neutron-1a214688-2bb6-4560-b80c-64a60cd68efa', 'neutron:subnet_ids': 'edbb7eeb-9a27-48c6-855b-ecfe98301352'}, mac=fa:16:3e:e0:b6:88, peer=[], dhcp_relay=[], ipv6_prefix=[], networks=['10.1.0.1/28'], name=lrp-180d5909-f827-477a-90ff-6f05685401e5, gateway_chassis=[], enabled=[], ipv6_ra_configs={}, status={}, options={}) old= {{(pid=98974) matches /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py:55}} Nov 25 13:39:05.015864 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbRemoveCommand(_result=None, table=HA_Chassis_Group, record=neutron-317531d0-5776-4925-bb27-3dcd5ebedab4, column=external_ids, values=('neutron:router_id',), keyvalues={}, if_exists=False) {{(pid=98974) do_commit /usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}} Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event [-] Event not matched due to this exception: Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: : ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-1a214688-2bb6-4560-b80c-64a60cd68efa Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event Traceback (most recent call last): Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/event.py", line 147, in match Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return candidate.matches(event, row, updates) Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/event.py", line 53, in matches Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event if not self.match_fn(event, row, old): Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/opt/stack/neutron/neutron/services/ovn_l3/ovsdb_monitor.py", line 47, in match_fn Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event lr = self.driver._nb_ovn.lookup('Logical_Router', lr_name) Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 184, in lookup Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event return self._lookup(table, record) Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 235, in _lookup Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event row = idlutils.row_by_value(self, rl.table, rl.column, record) Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Nov 25 13:39:05.509327 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event File "/usr/local/lib/python3.12/site-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 114, in row_by_value Nov 25 13:39:05.509760 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event raise RowNotFound(table=table, col=column, match=match) Nov 25 13:39:05.509760 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Router with name=neutron-1a214688-2bb6-4560-b80c-64a60cd68efa Nov 25 13:39:05.509760 np0005535648.novalocal neutron-ovn-maintenance-worker[98974]: ERROR ovsdbapp.event