:_mod-docs-content-type: PROCEDURE [id="configuring-control-plane-networking-for-spine-leaf_{context}"] = Configuring control plane networking for spine-leaf topologies [role="_abstract"] If you are adopting a spine-leaf or Distributed Compute Node (DCN) deployment, update the control plane networking for communication across sites. Add subnets for remote sites to your existing `NetConfig` custom resource (CR) and update `NetworkAttachmentDefinition` CRs with routes to enable connectivity between the central control plane and remote sites. .Prerequisites * You have deployed the {rhos_long} control plane. * You have configured a `NetConfig` CR for the central site. For more information, see xref:configuring-isolated-networks_configuring-network[Configuring isolated networks]. * You have the network topology information for all remote sites, including: ** IP address ranges for each service network at each site ** VLAN IDs for each service network at each site ** Gateway addresses for inter-site routing .Procedure . Update your existing `NetConfig` CR to add subnets for each remote site. Each service network must include a subnet for the central site and each remote site. Use unique VLAN IDs for each site. For example: + * Central site: VLANs 20-23 * Edge site 1: VLANs 30-33 * Edge site 2: VLANs 40-43 + [source,yaml] ---- apiVersion: network.openstack.org/v1beta1 kind: NetConfig metadata: name: netconfig spec: networks: - name: ctlplane dnsDomain: ctlplane.example.com subnets: - name: allocationRanges: - end: 192.168.122.120 start: 192.168.122.100 cidr: 192.168.122.0/24 gateway: 192.168.122.1 - name: allocationRanges: - end: 192.168.133.120 start: 192.168.133.100 cidr: 192.168.133.0/24 gateway: 192.168.133.1 - name: allocationRanges: - end: 192.168.144.120 start: 192.168.144.100 cidr: 192.168.144.0/24 gateway: 192.168.144.1 - name: internalapi dnsDomain: internalapi.example.com subnets: - name: subnet1 allocationRanges: - end: 172.17.0.250 start: 172.17.0.100 cidr: 172.17.0.0/24 vlan: 20 - name: internalapisite1 allocationRanges: - end: 172.17.10.250 start: 172.17.10.100 cidr: 172.17.10.0/24 vlan: 30 - name: internalapisite2 allocationRanges: - end: 172.17.20.250 start: 172.17.20.100 cidr: 172.17.20.0/24 vlan: 40 - name: storage dnsDomain: storage.example.com subnets: - name: subnet1 allocationRanges: - end: 172.18.0.250 start: 172.18.0.100 cidr: 172.18.0.0/24 vlan: 21 - name: storagesite1 allocationRanges: - end: 172.18.10.250 start: 172.18.10.100 cidr: 172.18.10.0/24 vlan: 31 - name: storagesite2 allocationRanges: - end: 172.18.20.250 start: 172.18.20.100 cidr: 172.18.20.0/24 vlan: 41 - name: tenant dnsDomain: tenant.example.com subnets: - name: subnet1 allocationRanges: - end: 172.19.0.250 start: 172.19.0.100 cidr: 172.19.0.0/24 vlan: 22 - name: tenantsite1 allocationRanges: - end: 172.19.10.250 start: 172.19.10.100 cidr: 172.19.10.0/24 vlan: 32 - name: tenantsite2 allocationRanges: - end: 172.19.20.250 start: 172.19.20.100 cidr: 172.19.20.0/24 vlan: 42 ---- + where: ``:: Specifies a user-defined subnet name for the central site subnet. ``:: Specifies a user-defined subnet for the first DCN edge site. ``:: Specifies a user-defined subnet for the second DCN edge site. + [NOTE] ==== You must have the `storagemgmt` network on OpenShift nodes when using DCN with Swift storage. It is not necessary when using Red Hat Ceph Storage. ==== . Update the `NetworkAttachmentDefinition` CR for the `internalapi` network to include routes to remote site subnets. These `routes` fields enable control plane pods attached to the `internalapi` network, such as OVN Southbound database, to communicate with Compute nodes at remote sites through the central site gateway, and are required for DCN: + [source,yaml] ---- apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: name: internalapi spec: config: | { "cniVersion": "0.3.1", "name": "internalapi", "type": "macvlan", "master": "internalapi", "ipam": { "type": "whereabouts", "range": "172.17.0.0/24", "range_start": "172.17.0.30", "range_end": "172.17.0.70", "routes": [ { "dst": "172.17.10.0/24", "gw": "172.17.0.1" }, { "dst": "172.17.20.0/24", "gw": "172.17.0.1" } ] } } ---- . Update the `NetworkAttachmentDefinition` CR for the `ctlplane` network to include routes to remote site subnets: + [source,yaml] ---- apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: name: ctlplane spec: config: | { "cniVersion": "0.3.1", "name": "ctlplane", "type": "macvlan", "master": "ospbr", "ipam": { "type": "whereabouts", "range": "192.168.122.0/24", "range_start": "192.168.122.30", "range_end": "192.168.122.70", "routes": [ { "dst": "192.168.133.0/24", "gw": "192.168.122.1" }, { "dst": "192.168.144.0/24", "gw": "192.168.122.1" } ] } } ---- . Update the `NetworkAttachmentDefinition` CR for the `storage` network to include routes to remote site subnets: + [source,yaml] ---- apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: name: storage spec: config: | { "cniVersion": "0.3.1", "name": "storage", "type": "macvlan", "master": "storage", "ipam": { "type": "whereabouts", "range": "172.18.0.0/24", "range_start": "172.18.0.30", "range_end": "172.18.0.70", "routes": [ { "dst": "172.18.10.0/24", "gw": "172.18.0.1" }, { "dst": "172.18.20.0/24", "gw": "172.18.0.1" } ] } } ---- . Update the `NetworkAttachmentDefinition` CR for the `tenant` network to include routes to remote site subnets: + [source,yaml] ---- apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: name: tenant spec: config: | { "cniVersion": "0.3.1", "name": "tenant", "type": "macvlan", "master": "tenant", "ipam": { "type": "whereabouts", "range": "172.19.0.0/24", "range_start": "172.19.0.30", "range_end": "172.19.0.70", "routes": [ { "dst": "172.19.10.0/24", "gw": "172.19.0.1" }, { "dst": "172.19.20.0/24", "gw": "172.19.0.1" } ] } } ---- + [NOTE] ==== Adjust the IP ranges, subnets, and gateway addresses in all NAD configurations to match your network topology. The `master` interface name must match the interface on the OpenShift nodes where the VLAN is configured. ==== . If you have already deployed OVN services, restart the OVN Southbound database pods to pick up the new routes: + ---- $ oc delete pod -l service=ovsdbserver-sb ---- + The pods are automatically recreated with the updated network configuration. . Configure the {networking_first_ref} to recognize all site physnets. In the `OpenStackControlPlane` CR, ensure the Networking service configuration includes all physnets: + [source,yaml] ---- apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane metadata: name: openstack spec: neutron: template: customServiceConfig: | [ml2_type_vlan] network_vlan_ranges = leaf0:1:1000,leaf1:1:1000,leaf2:1:1000 [ovn] ovn_emit_need_to_frag = false ---- + where: leaf0:: Represents the physnet for the central site. leaf1:: Represents the physnet for the first remote site. leaf2:: Represents the physnet for the second remote site. + [NOTE] ==== Adjust the physnet names to match your {rhos_prev_long} deployment. Common conventions include `leaf0/leaf1/leaf2` or `datacentre/dcn1/dcn2`. ==== .Verification . Verify that the `NetConfig` CR is created with all subnets: + ---- $ oc get netconfig netconfig -o yaml | grep -A2 "name: subnet1\|name: .*site" ---- . Verify that each `NetworkAttachmentDefinition` includes routes to remote site subnets: + ---- for nad in ctlplane internalapi storage tenant; do echo "=== $nad ===" oc get net-attach-def $nad -o jsonpath='{.spec.config}' | jq '.ipam.routes done ---- . After restarting OVN SB pods, verify they have routes to remote site subnets: + ---- $ oc exec $(oc get pod -l service=ovsdbserver-sb -o name | head -1) -- ip route show | grep 172.17 ---- + Sample output: + ---- 172.17.10.0/24 via 172.17.0.1 dev internalapi 172.17.20.0/24 via 172.17.0.1 dev internalapi ----