:_mod-docs-content-type: PROCEDURE
[id="migrating-object-storage-data-to-rhoso-nodes_{context}"]

= Migrating the {object_storage} data from {OpenStackShort} to {rhos_acro} nodes

[role="_abstract"]
The {object_storage_first_ref} migration involves the following steps:

. Add new nodes to the {object_storage} rings.
. Set weights of existing nodes to 0.
. Rebalance rings by moving one replica.
. Copy rings to old nodes and restart services.
. Check replication status and repeat the previous two steps until the old nodes are drained.
. Remove the old nodes from the rings.

.Prerequisites

* Adopt the {object_storage}. For more information, see xref:adopting-the-object-storage-service_adopt-control-plane[Adopting the {object_storage}].
* For DNS servers, ensure that all existing nodes are able to resolve the hostnames of the {rhocp_long} pods, for example, by using the external IP of the DNSMasq service as the nameserver in `/etc/resolv.conf`:
+
----
$ oc get service dnsmasq-dns -o jsonpath="{.status.loadBalancer.ingress[0].ip}" | $CONTROLLER1_SSH sudo tee /etc/resolv.conf
----
* Track the current status of the replication by using the `swift-dispersion` tool:
+
----
$ oc debug --keep-labels=true job/swift-ring-rebalance -- /bin/sh -c 'swift-ring-tool get && swift-dispersion-populate'
----
+
The command might need a few minutes to complete. It creates 0-byte objects that are distributed across the {object_storage} deployment, and you can use the `swift-dispersion-report` afterward to show the current replication status:
+
----
$ oc debug --keep-labels=true job/swift-ring-rebalance -- /bin/sh -c 'swift-ring-tool get && swift-dispersion-report'
----
+
The output of the `swift-dispersion-report` command looks similar to the following:
+
----
Queried 1024 containers for dispersion reporting, 5s, 0 retries
100.00% of container copies found (3072 of 3072)
Sample represents 100.00% of the container partition space
Queried 1024 objects for dispersion reporting, 4s, 0 retries
There were 1024 partitions missing 0 copies.
100.00% of object copies found (3072 of 3072)
Sample represents 100.00% of the object partition space
----

.Procedure

. Add new nodes by scaling up the SwiftStorage resource from 0 to 3:
// TODO add paragraph / link on EDPM node usage for Swift
+
----
$ oc patch openstackcontrolplane openstack --type=merge -p='{"spec":{"swift":{"template":{"swiftStorage":{"replicas": 3}}}}}'
----
+
This command creates three storage instances on the {rhocp_long} cluster that use Persistent Volume Claims.

. Wait until all three pods are running and the rings include the new devices:
+
----
$ oc wait pods --for condition=Ready -l component=swift-storage
$ oc debug --keep-labels=true job/swift-ring-rebalance -- /bin/sh -c 'swift-ring-tool get && swift-ring-builder object.builder search --device pv'
----

. From the current rings, get the storage management IP addresses of the nodes to drain:
+
----
$ oc debug --keep-labels=true job/swift-ring-rebalance -- /bin/sh -c 'swift-ring-tool get && swift-ring-builder object.builder search _' | tail -n +2 | awk '{print $4}' | sort -u

----
+
The output looks similar to the following:
+
----
172.20.0.100
swift-storage-0.swift-storage.openstack.svc
swift-storage-1.swift-storage.openstack.svc
swift-storage-2.swift-storage.openstack.svc
----

. Drain the old nodes. In the following example, the old node `172.20.0.100` is drained:
+
----
$ oc debug --keep-labels=true job/swift-ring-rebalance -- /bin/sh -c '
swift-ring-tool get
swift-ring-tool drain 172.20.0.100
swift-ring-tool rebalance
swift-ring-tool push'
----
+
Depending on your deployment, you might have more nodes to include in the command.

. Copy and apply the updated rings to the original nodes. Run the
ssh commands for your existing nodes that store the {object_storage} data:
+
----
$ oc extract --confirm cm/swift-ring-files
$CONTROLLER1_SSH "tar -C /var/lib/config-data/puppet-generated/swift/etc/swift/ -xzf -" < swiftrings.tar.gz
$CONTROLLER1_SSH "systemctl restart tripleo_swift_*"
----

. Track the replication progress by using the `swift-dispersion-report` tool:
+
----
$ oc debug --keep-labels=true job/swift-ring-rebalance -- /bin/sh -c "swift-ring-tool get && swift-dispersion-report"
----
+
The output shows less than 100% of copies found. Repeat the command until all container and object copies are found:
+
----
Queried 1024 containers for dispersion reporting, 6s, 0 retries
There were 5 partitions missing 1 copy.
99.84% of container copies found (3067 of 3072)
Sample represents 100.00% of the container partition space
Queried 1024 objects for dispersion reporting, 7s, 0 retries
There were 739 partitions missing 1 copy.
There were 285 partitions missing 0 copies.
75.94% of object copies found (2333 of 3072)
Sample represents 100.00% of the object partition space
----

. Move the next replica to the new nodes by rebalancing and distributing the rings:
+
----
$ oc debug --keep-labels=true job/swift-ring-rebalance -- /bin/sh -c '
swift-ring-tool get
swift-ring-tool rebalance
swift-ring-tool push'

$ oc extract --confirm cm/swift-ring-files
$CONTROLLER1_SSH "tar -C /var/lib/config-data/puppet-generated/swift/etc/swift/ -xzf -" < swiftrings.tar.gz
$CONTROLLER1_SSH "systemctl restart tripleo_swift_*"
----
+
Monitor the `swift-dispersion-report` output again, wait until all copies are found, and then repeat this step until all your replicas are moved to the new nodes.

. Remove the nodes from the rings:
+
----
$ oc debug --keep-labels=true job/swift-ring-rebalance -- /bin/sh -c '
swift-ring-tool get
swift-ring-tool remove 172.20.0.100
swift-ring-tool rebalance
swift-ring-tool push'
----
+
[NOTE]
Even if all replicas are on the new nodes and the `swift-dispersion-report` command reports 100% of the copies found, there might still be data on the old nodes. The replicators remove this data, but it might take more time.

.Verification

. Check the disk usage of all disks in the cluster:
+
----
$ oc debug --keep-labels=true job/swift-ring-rebalance -- /bin/sh -c 'swift-ring-tool get && swift-recon -d'
----

. Confirm that there are no more `\*.db` or `*.data` files in the `/srv/node` directory on the nodes:
+
----
$CONTROLLER1_SSH "find /srv/node/ -type f -name '*.db' -o -name '*.data' | wc -l"
----