:_mod-docs-content-type: PROCEDURE
[id="adopting-networker-services-to-the-data-plane_{context}"]

= Adopting Networker services to the {rhos_acro} data plane

[role="_abstract"]
Adopt the Networker services in your existing {rhos_prev_long} deployment to the {rhos_long} data plane. The `Networker` services could be running on `Conroller` nodes or dedicated `Networker` nodes. You decide which services you want to run on the Networker nodes, and create a separate `OpenStackDataPlaneNodeSet` custom resource (CR) for the Networker nodes. You might also decide to implement the following options if they apply to your environment:

* Depending on your topology, you might need to run the `neutron-metadata` service on the nodes, specifically when you want to serve metadata to SR-IOV ports that are hosted on Compute nodes.

* If you want to continue running OVN gateway services on Networker nodes, keep `ovn` service in the list to deploy.

* Optional: You can run the `neutron-dhcp` service on your Networker nodes instead of your Compute nodes. You might not need to use `neutron-dhcp` with OVN, unless your deployment uses DHCP relays, or advanced DHCP options that are supported by dnsmasq but not by the OVN DHCP implementation.

Adopt each Controller or Networker node in your existing {rhos_prev_long} deployment to the {rhos_long} when your node is set as an OVN chassis gateway. Any node with
parameter set to `enable-chassis-as-gw` is considered OVN gateway chassis. In this case, such nodes will become edpm networker nodes after adoption.

. Check for the nodes where `OVN Controller Gateway agent` agents are running. The list of agents varies depending on the services you enabled:
+
----
$ oc exec openstackclient -- openstack network agent list
+--------------------------------------+------------------------------+--------------------------+-------------------+-------+-------+----------------------------+
| ID                                   | Agent Type                   | Host                     | Availability Zone | Alive | State | Binary                     |
+--------------------------------------+------------------------------+--------------------------+-------------------+-------+-------+----------------------------+
| e5075ee0-9dd9-4f0a-a42a-6bbdf1a6111c | OVN Controller Gateway agent | controller-0.localdomain |                   | XXX   | UP    | ovn-controller             |
| f3112349-054c-403a-b00a-e219238192b8 | OVN Controller agent         | compute-0.localdomain    |                   | XXX   | UP    | ovn-controller             |
| af9dae2d-1c1c-55a8-a743-f84719f6406d | OVN Metadata agent           | compute-0.localdomain    |                   | XXX   | UP    | neutron-ovn-metadata-agent |
| 51a11df8-a66e-47a2-aec0-52eb8589626c | OVN Controller Gateway agent | controller-1.localdomain |                   | XXX   | UP    | ovn-controller             |
| bb817e5e-7832-410a-9e67-934dac8c602f | OVN Controller Gateway agent | controller-2.localdomain |                   | XXX   | UP    | ovn-controller             |
+--------------------------------------+------------------------------+--------------------------+-------------------+-------+-------+----------------------------+
----

.Prerequisites

* Define the shell variable. Based on above agent list output,
controller-0, controller-1, controller-2 are our target
hosts. If you have both `Controller` and `Networker` nodes running
networker services then add all those hosts below.
+
[subs=+quotes]
----
declare -A networkers
networkers+=(
  ["controller-0.localdomain"]="192.168.122.100"
  ["controller-1.localdomain"]="192.168.122.101"
  ["controller-2.localdomain"]="192.168.122.102"
  # ...
)
----
+
** Replace `["<node-name>"]="192.168.122.100"` with the name and IP address of the corresponding Networker or Controller node as per your environment.

.Procedure

. Deploy the `OpenStackDataPlaneNodeSet` CR for your nodes:
+
[NOTE]
You can reuse most of the `nodeTemplate` section from the `OpenStackDataPlaneNodeSet` CR that is designated for your Compute nodes. You can omit some of the variables because of the limited set of services that are running on the Networker nodes.
+
[subs="+quotes"]
----
$ oc apply -f - <<EOF
apiVersion: dataplane.openstack.org/v1beta1
kind: OpenStackDataPlaneNodeSet
metadata:
  name: openstack-networker
spec:
  tlsEnabled: false
  networkAttachments:
      - ctlplane
  preProvisioned: true
  services:
ifeval::["{build}" == "downstream"]
    - redhat
endif::[]
    - bootstrap
    - download-cache
    - configure-network
    - validate-network
    - install-os
    - configure-os
    - ssh-known-hosts
    - run-os
    - install-certs
    - ovn
  env:
    - name: ANSIBLE_CALLBACKS_ENABLED
      value: "profile_tasks"
    - name: ANSIBLE_FORCE_COLOR
      value: "True"
  nodes:
    controller-0:
      hostName: controller-0
      ansible:
        ansibleHost: ${networkers[controller-0.localdomain]}
      networks:
      - defaultRoute: true
        fixedIP: ${networkers[controller-0.localdomain]}
        name: ctlplane
        subnetName: subnet1
      - name: internalapi
        subnetName: subnet1
      - name: storage
        subnetName: subnet1
      - name: tenant
        subnetName: subnet1
    controller-1:
      hostName: controller-1
      ansible:
        ansibleHost: ${networkers[controller-1.localdomain]}
      networks:
      - defaultRoute: true
        fixedIP: ${networkers[controller-1.localdomain]}
        name: ctlplane
        subnetName: subnet1
      - name: internalapi
        subnetName: subnet1
      - name: storage
        subnetName: subnet1
      - name: tenant
        subnetName: subnet1
    controller-2:
      hostName: controller-2
      ansible:
        ansibleHost: ${networkers[controller-2.localdomain]}
      networks:
      - defaultRoute: true
        fixedIP: ${networkers[controller-2.localdomain]}
        name: ctlplane
        subnetName: subnet1
      - name: internalapi
        subnetName: subnet1
      - name: storage
        subnetName: subnet1
      - name: tenant
        subnetName: subnet1
  nodeTemplate:
    ansibleSSHPrivateKeySecret: dataplane-adoption-secret
    ansible:
      ansibleUser: root
ifeval::["{build}" == "downstream"]
      ansibleVarsFrom:
      - secretRef:
          name: subscription-manager
      - secretRef:
          name: redhat-registry
endif::[]
      ansibleVars:
ifeval::["{build}" == "downstream"]
        rhc_release: 9.2
        rhc_repositories:
            - {name: "*", state: disabled}
            - {name: "rhel-9-for-x86_64-baseos-eus-rpms", state: enabled}
            - {name: "rhel-9-for-x86_64-appstream-eus-rpms", state: enabled}
            - {name: "rhel-9-for-x86_64-highavailability-eus-rpms", state: enabled}
            - {name: "rhoso-18.0-for-rhel-9-x86_64-rpms", state: enabled}
            - {name: "fast-datapath-for-rhel-9-x86_64-rpms", state: enabled}
            - {name: "rhceph-7-tools-for-rhel-9-x86_64-rpms", state: enabled}
endif::[]
        edpm_bootstrap_release_version_package: []
        # edpm_network_config
        # Default nic config template for a EDPM node
        # These vars are edpm_network_config role vars
        edpm_network_config_template: |
           ---
           {% set mtu_list = [ctlplane_mtu] %}
           {% for network in nodeset_networks %}
           {% set _ = mtu_list.append(lookup('vars', networks_lower[network] ~ '_mtu')) %}
           {%- endfor %}
           {% set min_viable_mtu = mtu_list | max %}
           network_config:
           - type: ovs_bridge
             name: {{ neutron_physical_bridge_name }}
             mtu: {{ min_viable_mtu }}
             use_dhcp: false
             dns_servers: {{ ctlplane_dns_nameservers }}
             domain: {{ dns_search_domains }}
             addresses:
             - ip_netmask: {{ ctlplane_ip }}/{{ ctlplane_cidr }}
             routes: {{ ctlplane_host_routes }}
             members:
             - type: interface
               name: nic1
               mtu: {{ min_viable_mtu }}
               # force the MAC address of the bridge to this interface
               primary: true
           {% for network in nodeset_networks %}
             - type: vlan
               mtu: {{ lookup('vars', networks_lower[network] ~ '_mtu') }}
               vlan_id: {{ lookup('vars', networks_lower[network] ~ '_vlan_id') }}
               addresses:
               - ip_netmask:
                   {{ lookup('vars', networks_lower[network] ~ '_ip') }}/{{ lookup('vars', networks_lower[network] ~ '_cidr') }}
               routes: {{ lookup('vars', networks_lower[network] ~ '_host_routes') }}
           {% endfor %}
        edpm_network_config_nmstate: false
        edpm_network_config_hide_sensitive_logs: false
        #
        # These vars are for the network config templates themselves and are
        # considered EDPM network defaults.
        neutron_physical_bridge_name: br-ctlplane
        neutron_public_interface_name: eth0

        # edpm_nodes_validation
        edpm_nodes_validation_validate_controllers_icmp: false
        edpm_nodes_validation_validate_gateway_icmp: false

        # edpm ovn-controller configuration
        edpm_ovn_bridge_mappings: <bridge_mappings>
        edpm_ovn_bridge: br-int
        edpm_ovn_encap_type: geneve
        ovn_monitor_all: true
        edpm_ovn_remote_probe_interval: 60000
        edpm_ovn_ofctrl_wait_before_clear: 8000

        # serve as a OVN gateway
        edpm_enable_chassis_gw: true

        timesync_ntp_servers:
ifeval::["{build}" != "downstream"]
        - hostname: pool.ntp.org
endif::[]
ifeval::["{build}" == "downstream"]
        - hostname: clock.redhat.com
        - hostname: clock2.redhat.com
endif::[]

ifeval::["{build}" != "downstream"]
        edpm_bootstrap_command: |
          # This is a hack to deploy RDO Delorean repos to RHEL as if it were Centos 9 Stream
          set -euxo pipefail
          curl -sL https://github.com/openstack-k8s-operators/repo-setup/archive/refs/heads/main.tar.gz | tar -xz
          python3 -m venv ./venv
          PBR_VERSION=0.0.0 ./venv/bin/pip install ./repo-setup-main
          # This is required for FIPS enabled until trunk.rdoproject.org
          # is not being served from a centos7 host, tracked by
          # https://issues.redhat.com/browse/RHOSZUUL-1517
          dnf -y install crypto-policies
          update-crypto-policies --set FIPS:NO-ENFORCE-EMS
          ./venv/bin/repo-setup current-podified -b antelope -d centos9 --stream
          rm -rf repo-setup-main
endif::[]

        gather_facts: false
        enable_debug: false
        # edpm firewall, change the allowed CIDR if needed
        edpm_sshd_configure_firewall: true
        edpm_sshd_allowed_ranges: ['192.168.122.0/24']
        # SELinux module
        edpm_selinux_mode: enforcing

        # Do not attempt OVS major upgrades here
        edpm_ovs_packages:
        - openvswitch3.3
EOF
----
+
* `spec.tlsEnabled` specifies whether TLS Everywhere is enabled. If TLS is enabled, change `spec:tlsEnabled` to `true`.
* `edpm_ovn_bridge_mappings: <bridge_mappings>` specifies the bridge mapping values that you used in your {rhos_prev_long} {rhos_prev_ver} deployment.
* `edpm_enable_chassis_gw` specifies whether to run `ovn-controller` in gateway mode.
ifeval::["{build}" != "downstream"]
+
[IMPORTANT]
====
For environments that are enabled with border gateway protocol (BGP), preserve the default routes on the data plane nodes.

When adopting {rhos_prev_long} {rhos_prev_ver} environments with BGP, default routes can be lost when the data plane adoption procedure stops the {rhos_prev_long} services, specifically when FRRouting (FRR) is stopped. This causes connectivity issues during the {rhos_acro} data plane deployment.

To prevent this, configure the required routes by using `os-net-config` on the data plane nodes (Compute nodes and Networker nodes) affected by this issue.

Modify your `os-net-config` configuration file by adding the required routes, and then apply it:

----
$ sudo os-net-config -c /etc/os-net-config/modified_config_with_routes.yaml --provider ifcfg
----

This temporary default route is needed during the installation of the first services (such as `download-cache`) and is removed when the `configure-network` service applies the new network configuration.
====
+
[NOTE]
====
For environments that are enabled with border gateway protocol (BGP), you must add the following services to the `services` list in the order shown:

* After `configure-network` and before `validate-network`: Add `frr` service for FRRouting BGP support
* After `ovn` and `neutron-metadata` services: Add `ovn-bgp-agent` service

You must also configure the following additional Ansible variables in the `nodeTemplate.ansible.ansibleVars` section:

* `edpm_frr_image`: The FRRouting container image
* `edpm_ovn_bgp_agent_image`: The OVN BGP agent container image
* `edpm_frr_bgp_ipv4_src_network`: The network name for BGP IPv4 source (for example, `bgpmainnet`)
* `edpm_frr_bgp_ipv6_src_network`: The network name for BGP IPv6 source (for example, `bgpmainnetv6`)
* `edpm_frr_bgp_neighbor_password`: The BGP neighbor password
* `edpm_ovn_encap_ip`: Set to the BGP main network IP (for example, `{{ lookup("vars", "bgpmainnet_ip") }}`)
====
endif::[]

. Ensure that you use the same `ovn-controller` settings in the `OpenStackDataPlaneNodeSet` CR that you used in the Networker nodes before adoption. This configuration is stored in the `external_ids` column in the `Open_vSwitch` table in the Open vSwitch database:
+
----
ovs-vsctl list Open .
...
external_ids        : {hostname=controller-0.localdomain, ovn-bridge=br-int, ovn-bridge-mappings=<bridge_mappings>, ovn-chassis-mac-mappings="datacentre:1e:0a:bb:e6:7c:ad", ovn-cms-options=enable-chassis-as-gw, ovn-encap-ip="172.19.0.100", ovn-encap-tos="0", ovn-encap-type=geneve, ovn-match-northd-version=False, ovn-monitor-all=True, ovn-ofctrl-wait-before-clear="8000", ovn-openflow-probe-interval="60", ovn-remote="tcp:ovsdbserver-sb.openstack.svc:6642", ovn-remote-probe-interval="60000", rundir="/var/run/openvswitch", system-id="2eec68e6-aa21-4c95-a868-31aeafc11736"}
...
----
+
* Replace `<bridge_mappings>` with the value of the bridge mappings in your configuration, for example, `"datacentre:br-ctlplane"`.

. Optional: Enable `neutron-metadata` in the `OpenStackDataPlaneNodeSet` CR:
+
----
$ oc patch openstackdataplanenodeset <networker_CR_name> --type='json' --patch='[
  {
    "op": "add",
    "path": "/spec/services/-",
    "value": "neutron-metadata"
  }]'
----
+
* Replace `<networker_CR_name>` with the name of the CR that you deployed for your Networker nodes, for example, `openstack-networker`.

. Optional: Enable `neutron-dhcp` in the `OpenStackDataPlaneNodeSet` CR:
+
----
$ oc patch openstackdataplanenodeset <networker_CR_name> --type='json' --patch='[
  {
    "op": "add",
    "path": "/spec/services/-",
    "value": "neutron-dhcp"
  }]'
----

. Run the `pre-adoption-validation` service for Networker nodes:

.. Create a `OpenStackDataPlaneDeployment` CR that runs only the validation:
+
----
$ oc apply -f - <<EOF
apiVersion: dataplane.openstack.org/v1beta1
kind: OpenStackDataPlaneDeployment
metadata:
  name: openstack-pre-adoption-networker
spec:
  nodeSets:
  - openstack-networker
  servicesOverride:
  - pre-adoption-validation
EOF
----

.. When the validation is finished, confirm that the status of the Ansible EE pods is `Completed`:
+
----
$ watch oc get pod -l app=openstackansibleee
----
+
----
$ oc logs -l app=openstackansibleee -f --max-log-requests 20
----

.. Wait for the deployment to reach the `Ready` status:
+
----
$ oc wait --for condition=Ready openstackdataplanedeployment/openstack-pre-adoption-networker --timeout=10m
----

. Deploy the `OpenStackDataPlaneDeployment` CR for Networker nodes:
+
----
$ oc apply -f - <<EOF
apiVersion: dataplane.openstack.org/v1beta1
kind: OpenStackDataPlaneDeployment
metadata:
  name: openstack-networker
spec:
  nodeSets:
  - openstack-networker
EOF
----
+
[NOTE]
Alternatively, you can include the Networker node set in the `nodeSets` list before you deploy the main `OpenStackDataPlaneDeployment` CR. You cannot add new node sets to the `OpenStackDataPlaneDeployment` CR after deployment.

. Clean up any {networking_first_ref} agents that are no longer running.
+
[NOTE]
In some cases, agents from the old data plane that are replaced or retired remain in {rhos_acro}. The function these agents provided might be provided by a new agent that is running in {rhos_acro}, or the function might be replaced by other components. For example, DHCP agents might no longer be needed, since OVN DHCP in {rhos_acro} can provide this function.

.. List the agents:
+
----
$ oc exec openstackclient -- openstack network agent list
+--------------------------------------+------------------------------+--------------------------+-------------------+-------+-------+----------------------------+
| ID                                   | Agent Type                   | Host                     | Availability Zone | Alive | State | Binary                     |
+--------------------------------------+------------------------------+--------------------------+-------------------+-------+-------+----------------------------+
| e5075ee0-9dd9-4f0a-a42a-6bbdf1a6111c | OVN Controller Gateway agent | controller-0.localdomain |                   | :-)   | UP    | ovn-controller             |
| 856960f0-5530-46c7-a331-6eadcba362da | DHCP agent                   | controller-1.localdomain | nova              | XXX   | UP    | neutron-dhcp-agent         |
| 8bd22720-789f-45b8-8d7d-006dee862bf9 | DHCP agent                   | controller-2.localdomain | nova              | XXX   | UP    | neutron-dhcp-agent         |
| e584e00d-be4c-4e98-a11a-4ecd87d21be7 | DHCP agent                   | controller-0.localdomain | nova              | XXX   | UP    | neutron-dhcp-agent         |
+--------------------------------------+------------------------------+--------------------------+-------------------+-------+-------+----------------------------+
----

.. If any agent in the list shows `XXX` in the `Alive` field, verify the Host and Agent Type, if the functions of this agent is no longer required, and the agent has been permanently stopped on the {rhos_prev_long} host. Then, delete the agent:
+
----
$ oc exec openstackclient -- openstack network agent <agent_id>
----
* Replace `<agent_id>` with the ID of the agent to delete, for example, `856960f0-5530-46c7-a331-6eadcba362da`.


.Verification

. Confirm that all the Ansible EE pods reach a `Completed` status:
+
----
$ watch oc get pod -l app=openstackansibleee
----
+
----
$ oc logs -l app=openstackansibleee -f --max-log-requests 20
----

. Wait for the data plane node set to reach the `Ready` status:
+
----
$ oc wait --for condition=Ready osdpns/<networker_CR_name> --timeout=30m
----
+
* Replace `<networker_CR_name>` with the name of the CR that you deployed for your Networker nodes, for example, `openstack-networker`.

. Verify that the {networking_first_ref} agents are running. The list of agents varies depending on the services you enabled:
+
----
$ oc exec openstackclient -- openstack network agent list
+--------------------------------------+------------------------------+--------------------------+-------------------+-------+-------+----------------------------+
| ID                                   | Agent Type                   | Host                     | Availability Zone | Alive | State | Binary                     |
+--------------------------------------+------------------------------+--------------------------+-------------------+-------+-------+----------------------------+
| e5075ee0-9dd9-4f0a-a42a-6bbdf1a6111c | OVN Controller Gateway agent | controller-0.localdomain |                   | :-)   | UP    | ovn-controller             |
| f3112349-054c-403a-b00a-e219238192b8 | OVN Controller agent         | compute-0.localdomain    |                   | :-)   | UP    | ovn-controller             |
| af9dae2d-1c1c-55a8-a743-f84719f6406d | OVN Metadata agent           | compute-0.localdomain    |                   | :-)   | UP    | neutron-ovn-metadata-agent |
| 51a11df8-a66e-47a2-aec0-52eb8589626c | OVN Controller Gateway agent | controller-1.localdomain |                   | :-)   | UP    | ovn-controller             |
| bb817e5e-7832-410a-9e67-934dac8c602f | OVN Controller Gateway agent | controller-2.localdomain |                   | :-)   | UP    | ovn-controller             |
+--------------------------------------+------------------------------+--------------------------+-------------------+-------+-------+----------------------------+
----