2025-06-02 10:32:50,418 p=33732 u=zuul n=ansible | [WARNING]: Found variable using reserved name: namespace 2025-06-02 10:32:50,419 p=33732 u=zuul n=ansible | PLAY [Execute the default-telemetry scenario] ********************************** 2025-06-02 10:32:50,452 p=33732 u=zuul n=ansible | TASK [Wait until telemetry is finished reconciling and its conditions are all True _raw_params=oc get telemetry telemetry --output=jsonpath --template={.status.conditions[*].status}] *** 2025-06-02 10:32:50,452 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:32:50 +0000 (0:00:00.048) 0:00:00.048 *********** 2025-06-02 10:32:50,922 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:32:50,934 p=33732 u=zuul n=ansible | TASK [Get current time operator_reconciled_timestamp={{ now(utc=true,fmt='%Y-%m-%dT%H:%M:%SZ') }}] *** 2025-06-02 10:32:50,934 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:32:50 +0000 (0:00:00.481) 0:00:00.529 *********** 2025-06-02 10:32:50,957 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:32:50,966 p=33732 u=zuul n=ansible | TASK [Wait for new logs to be generated timeout=120] *************************** 2025-06-02 10:32:50,966 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:32:50 +0000 (0:00:00.031) 0:00:00.561 *********** 2025-06-02 10:34:51,347 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:34:51,360 p=33732 u=zuul n=ansible | TASK [Get new telemetry-operator logs _raw_params=oc logs -n openstack-operators -l "openstack.org/operator-name=telemetry" --tail=-1 --since-time "{{ operator_reconciled_timestamp }}"] *** 2025-06-02 10:34:51,360 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:34:51 +0000 (0:02:00.394) 0:02:00.956 *********** 2025-06-02 10:34:51,710 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:34:51,721 p=33732 u=zuul n=ansible | TASK [TEST Check telemetry-operator logs for errors after it's started RHOSO-123456 error_list={{ operator_logs.stdout | ansible.builtin.regex_findall('ERROR.*') }}] *** 2025-06-02 10:34:51,721 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:34:51 +0000 (0:00:00.360) 0:02:01.316 *********** 2025-06-02 10:34:51,742 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:34:51,752 p=33732 u=zuul n=ansible | TASK [Output found errors for debugging purposes var=error_list] *************** 2025-06-02 10:34:51,753 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:34:51 +0000 (0:00:00.031) 0:02:01.348 *********** 2025-06-02 10:34:51,770 p=33732 u=zuul n=ansible | skipping: [localhost] 2025-06-02 10:34:51,781 p=33732 u=zuul n=ansible | TASK [Get telemetry-operator pod data _raw_params=oc get pods -n openstack-operators -l "openstack.org/operator-name=telemetry" --output=jsonpath --template='{.items[0]}'] *** 2025-06-02 10:34:51,781 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:34:51 +0000 (0:00:00.028) 0:02:01.376 *********** 2025-06-02 10:34:52,128 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:34:52,138 p=33732 u=zuul n=ansible | TASK [Get telemetry-operator container statuses container_statuses={{ (output.stdout | from_json).status.containerStatuses }}] *** 2025-06-02 10:34:52,138 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:34:52 +0000 (0:00:00.357) 0:02:01.733 *********** 2025-06-02 10:34:52,159 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:34:52,172 p=33732 u=zuul n=ansible | TASK [Get telemetry-operator restart counts restart_counts={{ range(container_statuses | length) | map('extract', container_statuses) | map(attribute='restartCount') }}] *** 2025-06-02 10:34:52,172 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:34:52 +0000 (0:00:00.033) 0:02:01.767 *********** 2025-06-02 10:34:52,194 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:34:52,240 p=33732 u=zuul n=ansible | TASK [Get telemetry-operator failed container logs _raw_params=oc logs -n openstack-operators -p -l "openstack.org/operator-name=telemetry" --tail=-1] *** 2025-06-02 10:34:52,241 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:34:52 +0000 (0:00:00.068) 0:02:01.836 *********** 2025-06-02 10:34:52,254 p=33732 u=zuul n=ansible | skipping: [localhost] 2025-06-02 10:34:52,264 p=33732 u=zuul n=ansible | TASK [Output logs of failed container for debugging purposes var=operator_logs_previous.stdout_lines] *** 2025-06-02 10:34:52,264 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:34:52 +0000 (0:00:00.023) 0:02:01.860 *********** 2025-06-02 10:34:52,276 p=33732 u=zuul n=ansible | skipping: [localhost] 2025-06-02 10:34:52,285 p=33732 u=zuul n=ansible | TASK [Enable MetricStorage _raw_params=oc patch oscp/{{ control_plane_name }} --type='json' -p '[{"op": "replace", "path": "/spec/telemetry/template/metricStorage/enabled", "value":true}]' ] *** 2025-06-02 10:34:52,285 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:34:52 +0000 (0:00:00.021) 0:02:01.881 *********** 2025-06-02 10:34:52,728 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:34:52,737 p=33732 u=zuul n=ansible | TASK [Enable Autoscaling _raw_params=oc patch oscp/{{ control_plane_name }} --type='json' -p '[{"op": "replace", "path": "/spec/telemetry/template/autoscaling/enabled", "value":true}]' ] *** 2025-06-02 10:34:52,737 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:34:52 +0000 (0:00:00.451) 0:02:02.333 *********** 2025-06-02 10:34:53,252 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:34:53,265 p=33732 u=zuul n=ansible | TASK [Wait until reconciliation finishes timeout=120] ************************** 2025-06-02 10:34:53,265 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:34:53 +0000 (0:00:00.527) 0:02:02.860 *********** 2025-06-02 10:36:53,451 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:36:53,462 p=33732 u=zuul n=ansible | TASK [Get current time operator_reconciled_timestamp={{ now(utc=true,fmt='%Y-%m-%dT%H:%M:%SZ') }}] *** 2025-06-02 10:36:53,462 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:36:53 +0000 (0:02:00.197) 0:04:03.058 *********** 2025-06-02 10:36:53,481 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:36:53,493 p=33732 u=zuul n=ansible | TASK [Wait for new logs to be generated timeout=120] *************************** 2025-06-02 10:36:53,493 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:36:53 +0000 (0:00:00.030) 0:04:03.089 *********** 2025-06-02 10:38:53,790 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:38:53,798 p=33732 u=zuul n=ansible | TASK [Get new telemetry-operator logs _raw_params=oc logs -n openstack-operators -l "openstack.org/operator-name=telemetry" --tail=-1 --since-time "{{ operator_reconciled_timestamp }}"] *** 2025-06-02 10:38:53,799 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:53 +0000 (0:02:00.305) 0:06:03.394 *********** 2025-06-02 10:38:54,204 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:38:54,214 p=33732 u=zuul n=ansible | TASK [TEST Check that telemetry-operator logs include expected error logs when MetricStorage is enabled, but COO isn't installed RHOSO-123459 that=['(operator_logs.stdout | ansible.builtin.regex_search( ".*" + item + ".*" ))'], fail_msg=telemetry-operator logs don't include expected log: {{ item }}] *** 2025-06-02 10:38:54,214 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:54 +0000 (0:00:00.415) 0:06:03.810 *********** 2025-06-02 10:38:54,235 p=33732 u=zuul n=ansible | ok: [localhost] => (item=INFO Controllers.MetricStorage Can't own MonitoringStack resource) => ansible_loop_var: item changed: false item: "INFO\tControllers.MetricStorage\tCan't own MonitoringStack resource" msg: All assertions passed 2025-06-02 10:38:54,246 p=33732 u=zuul n=ansible | TASK [Get telemetry-operator error logs error_list={{ operator_logs.stdout | ansible.builtin.regex_findall('ERROR.*') }}] *** 2025-06-02 10:38:54,246 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:54 +0000 (0:00:00.032) 0:06:03.842 *********** 2025-06-02 10:38:54,264 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:38:54,279 p=33732 u=zuul n=ansible | TASK [TEST Check that telemetry-operator logs don't include unexpected errors when MetricStorage is enabled, but COO isn't installed RHOSO-123460 that=['item in expected_logs'], fail_msg=telemetry-operator logs include an unexpected error: {{ item }}] *** 2025-06-02 10:38:54,279 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:54 +0000 (0:00:00.032) 0:06:03.874 *********** 2025-06-02 10:38:54,289 p=33732 u=zuul n=ansible | skipping: [localhost] 2025-06-02 10:38:54,298 p=33732 u=zuul n=ansible | TASK [Get telemetry-operator pod data _raw_params=oc get pods -n openstack-operators -l "openstack.org/operator-name=telemetry" --output=jsonpath --template='{.items[0]}'] *** 2025-06-02 10:38:54,299 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:54 +0000 (0:00:00.019) 0:06:03.894 *********** 2025-06-02 10:38:54,621 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:38:54,631 p=33732 u=zuul n=ansible | TASK [Get telemetry-operator container statuses container_statuses={{ (output.stdout | from_json).status.containerStatuses }}] *** 2025-06-02 10:38:54,631 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:54 +0000 (0:00:00.332) 0:06:04.227 *********** 2025-06-02 10:38:54,651 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:38:54,663 p=33732 u=zuul n=ansible | TASK [Get telemetry-operator restart counts restart_counts={{ range(container_statuses | length) | map('extract', container_statuses) | map(attribute='restartCount') }}] *** 2025-06-02 10:38:54,663 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:54 +0000 (0:00:00.032) 0:06:04.259 *********** 2025-06-02 10:38:54,685 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:38:54,696 p=33732 u=zuul n=ansible | TASK [Get telemetry-operator failed container logs _raw_params=oc logs -n openstack-operators -p -l "openstack.org/operator-name=telemetry" --tail=-1] *** 2025-06-02 10:38:54,696 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:54 +0000 (0:00:00.032) 0:06:04.291 *********** 2025-06-02 10:38:54,710 p=33732 u=zuul n=ansible | skipping: [localhost] 2025-06-02 10:38:54,722 p=33732 u=zuul n=ansible | TASK [Output logs of failed container for debugging purposes var=operator_logs_previous.stdout_lines] *** 2025-06-02 10:38:54,722 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:54 +0000 (0:00:00.026) 0:06:04.317 *********** 2025-06-02 10:38:54,737 p=33732 u=zuul n=ansible | skipping: [localhost] 2025-06-02 10:38:54,747 p=33732 u=zuul n=ansible | TASK [Get MetricStorage condition types _raw_params=oc get metricstorage metric-storage -o jsonpath='{.status.conditions[*].type}'] *** 2025-06-02 10:38:54,747 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:54 +0000 (0:00:00.024) 0:06:04.342 *********** 2025-06-02 10:38:55,116 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:38:55,125 p=33732 u=zuul n=ansible | TASK [Get MetricStorage condition values _raw_params=oc get metricstorage metric-storage -o jsonpath='{.status.conditions[?(@.type == "{{ item }}")].message}'] *** 2025-06-02 10:38:55,125 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:55 +0000 (0:00:00.378) 0:06:04.720 *********** 2025-06-02 10:38:55,469 p=33732 u=zuul n=ansible | changed: [localhost] => (item=Ready) 2025-06-02 10:38:55,814 p=33732 u=zuul n=ansible | changed: [localhost] => (item=DashboardDatasourceReady) 2025-06-02 10:38:56,164 p=33732 u=zuul n=ansible | changed: [localhost] => (item=DashboardDefinitionReady) 2025-06-02 10:38:56,510 p=33732 u=zuul n=ansible | changed: [localhost] => (item=DashboardPluginReady) 2025-06-02 10:38:56,846 p=33732 u=zuul n=ansible | changed: [localhost] => (item=DashboardPrometheusRuleReady) 2025-06-02 10:38:57,180 p=33732 u=zuul n=ansible | changed: [localhost] => (item=MonitoringStackReady) 2025-06-02 10:38:57,520 p=33732 u=zuul n=ansible | changed: [localhost] => (item=NetworkAttachmentsReady) 2025-06-02 10:38:57,872 p=33732 u=zuul n=ansible | changed: [localhost] => (item=PrometheusReady) 2025-06-02 10:38:58,201 p=33732 u=zuul n=ansible | changed: [localhost] => (item=ScrapeConfigReady) 2025-06-02 10:38:58,507 p=33732 u=zuul n=ansible | changed: [localhost] => (item=TLSInputReady) 2025-06-02 10:38:58,521 p=33732 u=zuul n=ansible | TASK [Construct MetricStorage condition dictionary conditions={{ conditions | default({}) | combine({item.item: item.stdout}) }}] *** 2025-06-02 10:38:58,522 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:58 +0000 (0:00:03.396) 0:06:08.117 *********** 2025-06-02 10:38:58,545 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Error occured when trying to own: customresourcedefinitions.apiextensions.k8s.io "monitoringstacks.monitoring.rhobs" not found', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'metricstorage', 'metric-storage', '-o', 'jsonpath={.status.conditions[?(@.type == "Ready")].message}'], 'start': '2025-06-02 10:38:55.298834', 'end': '2025-06-02 10:38:55.444501', 'delta': '0:00:00.145667', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get metricstorage metric-storage -o jsonpath=\'{.status.conditions[?(@.type == "Ready")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Error occured when trying to own: customresourcedefinitions.apiextensions.k8s.io "monitoringstacks.monitoring.rhobs" not found'], 'stderr_lines': [], 'failed': False, 'item': 'Ready', 'ansible_loop_var': 'item'}) 2025-06-02 10:38:58,551 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Dashboard Datasource not started', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'metricstorage', 'metric-storage', '-o', 'jsonpath={.status.conditions[?(@.type == "DashboardDatasourceReady")].message}'], 'start': '2025-06-02 10:38:55.621588', 'end': '2025-06-02 10:38:55.788667', 'delta': '0:00:00.167079', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get metricstorage metric-storage -o jsonpath=\'{.status.conditions[?(@.type == "DashboardDatasourceReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Dashboard Datasource not started'], 'stderr_lines': [], 'failed': False, 'item': 'DashboardDatasourceReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:38:58,558 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Dashboard Definition not started', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'metricstorage', 'metric-storage', '-o', 'jsonpath={.status.conditions[?(@.type == "DashboardDefinitionReady")].message}'], 'start': '2025-06-02 10:38:55.976017', 'end': '2025-06-02 10:38:56.142533', 'delta': '0:00:00.166516', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get metricstorage metric-storage -o jsonpath=\'{.status.conditions[?(@.type == "DashboardDefinitionReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Dashboard Definition not started'], 'stderr_lines': [], 'failed': False, 'item': 'DashboardDefinitionReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:38:58,564 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Dashboard Plugin not started', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'metricstorage', 'metric-storage', '-o', 'jsonpath={.status.conditions[?(@.type == "DashboardPluginReady")].message}'], 'start': '2025-06-02 10:38:56.323872', 'end': '2025-06-02 10:38:56.487332', 'delta': '0:00:00.163460', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get metricstorage metric-storage -o jsonpath=\'{.status.conditions[?(@.type == "DashboardPluginReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Dashboard Plugin not started'], 'stderr_lines': [], 'failed': False, 'item': 'DashboardPluginReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:38:58,571 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Dashboard PrometheusRule not started', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'metricstorage', 'metric-storage', '-o', 'jsonpath={.status.conditions[?(@.type == "DashboardPrometheusRuleReady")].message}'], 'start': '2025-06-02 10:38:56.647557', 'end': '2025-06-02 10:38:56.821436', 'delta': '0:00:00.173879', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get metricstorage metric-storage -o jsonpath=\'{.status.conditions[?(@.type == "DashboardPrometheusRuleReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Dashboard PrometheusRule not started'], 'stderr_lines': [], 'failed': False, 'item': 'DashboardPrometheusRuleReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:38:58,577 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Error occured when trying to own: customresourcedefinitions.apiextensions.k8s.io "monitoringstacks.monitoring.rhobs" not found', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'metricstorage', 'metric-storage', '-o', 'jsonpath={.status.conditions[?(@.type == "MonitoringStackReady")].message}'], 'start': '2025-06-02 10:38:57.011430', 'end': '2025-06-02 10:38:57.158221', 'delta': '0:00:00.146791', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get metricstorage metric-storage -o jsonpath=\'{.status.conditions[?(@.type == "MonitoringStackReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Error occured when trying to own: customresourcedefinitions.apiextensions.k8s.io "monitoringstacks.monitoring.rhobs" not found'], 'stderr_lines': [], 'failed': False, 'item': 'MonitoringStackReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:38:58,584 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'NetworkAttachments not started', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'metricstorage', 'metric-storage', '-o', 'jsonpath={.status.conditions[?(@.type == "NetworkAttachmentsReady")].message}'], 'start': '2025-06-02 10:38:57.333366', 'end': '2025-06-02 10:38:57.495585', 'delta': '0:00:00.162219', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get metricstorage metric-storage -o jsonpath=\'{.status.conditions[?(@.type == "NetworkAttachmentsReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['NetworkAttachments not started'], 'stderr_lines': [], 'failed': False, 'item': 'NetworkAttachmentsReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:38:58,591 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Prometheus not ready', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'metricstorage', 'metric-storage', '-o', 'jsonpath={.status.conditions[?(@.type == "PrometheusReady")].message}'], 'start': '2025-06-02 10:38:57.697613', 'end': '2025-06-02 10:38:57.845428', 'delta': '0:00:00.147815', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get metricstorage metric-storage -o jsonpath=\'{.status.conditions[?(@.type == "PrometheusReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Prometheus not ready'], 'stderr_lines': [], 'failed': False, 'item': 'PrometheusReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:38:58,599 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'ScrapeConfig not started', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'metricstorage', 'metric-storage', '-o', 'jsonpath={.status.conditions[?(@.type == "ScrapeConfigReady")].message}'], 'start': '2025-06-02 10:38:58.019644', 'end': '2025-06-02 10:38:58.176093', 'delta': '0:00:00.156449', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get metricstorage metric-storage -o jsonpath=\'{.status.conditions[?(@.type == "ScrapeConfigReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['ScrapeConfig not started'], 'stderr_lines': [], 'failed': False, 'item': 'ScrapeConfigReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:38:58,604 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Input data not checked', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'metricstorage', 'metric-storage', '-o', 'jsonpath={.status.conditions[?(@.type == "TLSInputReady")].message}'], 'start': '2025-06-02 10:38:58.344428', 'end': '2025-06-02 10:38:58.484803', 'delta': '0:00:00.140375', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get metricstorage metric-storage -o jsonpath=\'{.status.conditions[?(@.type == "TLSInputReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Input data not checked'], 'stderr_lines': [], 'failed': False, 'item': 'TLSInputReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:38:58,616 p=33732 u=zuul n=ansible | TASK [TEST Check that all MetricStorage conditions are as expected when COO isn't installed RHOSO-123461 that=['conditions[item.key] == item.value'], fail_msg=Expected {{ item.key }} condition field to be {{ item.value }}, not {{ output }}] *** 2025-06-02 10:38:58,617 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:58 +0000 (0:00:00.094) 0:06:08.212 *********** 2025-06-02 10:38:58,644 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'key': 'MonitoringStackReady', 'value': 'Error occured when trying to own: customresourcedefinitions.apiextensions.k8s.io "monitoringstacks.monitoring.rhobs" not found'}) => ansible_loop_var: item changed: false item: key: MonitoringStackReady value: 'Error occured when trying to own: customresourcedefinitions.apiextensions.k8s.io "monitoringstacks.monitoring.rhobs" not found' msg: All assertions passed 2025-06-02 10:38:58,647 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'key': 'Ready', 'value': 'Error occured when trying to own: customresourcedefinitions.apiextensions.k8s.io "monitoringstacks.monitoring.rhobs" not found'}) => ansible_loop_var: item changed: false item: key: Ready value: 'Error occured when trying to own: customresourcedefinitions.apiextensions.k8s.io "monitoringstacks.monitoring.rhobs" not found' msg: All assertions passed 2025-06-02 10:38:58,657 p=33732 u=zuul n=ansible | TASK [Get Autoscaling condition types _raw_params=oc get autoscaling autoscaling -o jsonpath='{.status.conditions[*].type}'] *** 2025-06-02 10:38:58,658 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:58 +0000 (0:00:00.041) 0:06:08.253 *********** 2025-06-02 10:38:58,973 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:38:58,982 p=33732 u=zuul n=ansible | TASK [Get Autoscaling condition values _raw_params=oc get autoscaling autoscaling -o jsonpath='{.status.conditions[?(@.type == "{{ item }}")].message}'] *** 2025-06-02 10:38:58,982 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:38:58 +0000 (0:00:00.324) 0:06:08.578 *********** 2025-06-02 10:38:59,319 p=33732 u=zuul n=ansible | changed: [localhost] => (item=Ready) 2025-06-02 10:38:59,639 p=33732 u=zuul n=ansible | changed: [localhost] => (item=CreateServiceReady) 2025-06-02 10:38:59,962 p=33732 u=zuul n=ansible | changed: [localhost] => (item=DBReady) 2025-06-02 10:39:00,291 p=33732 u=zuul n=ansible | changed: [localhost] => (item=DBSyncReady) 2025-06-02 10:39:00,629 p=33732 u=zuul n=ansible | changed: [localhost] => (item=DeploymentReady) 2025-06-02 10:39:00,963 p=33732 u=zuul n=ansible | changed: [localhost] => (item=HeatReady) 2025-06-02 10:39:01,344 p=33732 u=zuul n=ansible | changed: [localhost] => (item=KeystoneEndpointReady) 2025-06-02 10:39:01,737 p=33732 u=zuul n=ansible | changed: [localhost] => (item=KeystoneServiceReady) 2025-06-02 10:39:02,077 p=33732 u=zuul n=ansible | changed: [localhost] => (item=MariaDBAccountReady) 2025-06-02 10:39:02,390 p=33732 u=zuul n=ansible | changed: [localhost] => (item=MemcachedReady) 2025-06-02 10:39:02,712 p=33732 u=zuul n=ansible | changed: [localhost] => (item=RabbitMqTransportURLReady) 2025-06-02 10:39:03,059 p=33732 u=zuul n=ansible | changed: [localhost] => (item=RoleBindingReady) 2025-06-02 10:39:03,396 p=33732 u=zuul n=ansible | changed: [localhost] => (item=RoleReady) 2025-06-02 10:39:03,718 p=33732 u=zuul n=ansible | changed: [localhost] => (item=ServiceAccountReady) 2025-06-02 10:39:04,033 p=33732 u=zuul n=ansible | changed: [localhost] => (item=ServiceConfigReady) 2025-06-02 10:39:04,371 p=33732 u=zuul n=ansible | changed: [localhost] => (item=TLSInputReady) 2025-06-02 10:39:04,384 p=33732 u=zuul n=ansible | TASK [Construct Autoscaling condition dictionary conditions={{ conditions | default({}) | combine({item.item: item.stdout}) }}] *** 2025-06-02 10:39:04,385 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:39:04 +0000 (0:00:05.402) 0:06:13.980 *********** 2025-06-02 10:39:04,410 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Setup complete', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "Ready")].message}'], 'start': '2025-06-02 10:38:59.149245', 'end': '2025-06-02 10:38:59.293546', 'delta': '0:00:00.144301', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "Ready")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Setup complete'], 'stderr_lines': [], 'failed': False, 'item': 'Ready', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,417 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Create service completed', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "CreateServiceReady")].message}'], 'start': '2025-06-02 10:38:59.474267', 'end': '2025-06-02 10:38:59.617490', 'delta': '0:00:00.143223', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "CreateServiceReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Create service completed'], 'stderr_lines': [], 'failed': False, 'item': 'CreateServiceReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,423 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'DB create completed', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "DBReady")].message}'], 'start': '2025-06-02 10:38:59.796867', 'end': '2025-06-02 10:38:59.939774', 'delta': '0:00:00.142907', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "DBReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['DB create completed'], 'stderr_lines': [], 'failed': False, 'item': 'DBReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,429 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'DBsync completed', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "DBSyncReady")].message}'], 'start': '2025-06-02 10:39:00.110495', 'end': '2025-06-02 10:39:00.269003', 'delta': '0:00:00.158508', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "DBSyncReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['DBsync completed'], 'stderr_lines': [], 'failed': False, 'item': 'DBSyncReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,435 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Deployment completed', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "DeploymentReady")].message}'], 'start': '2025-06-02 10:39:00.447323', 'end': '2025-06-02 10:39:00.606891', 'delta': '0:00:00.159568', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "DeploymentReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Deployment completed'], 'stderr_lines': [], 'failed': False, 'item': 'DeploymentReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,441 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Setup complete', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "HeatReady")].message}'], 'start': '2025-06-02 10:39:00.786633', 'end': '2025-06-02 10:39:00.942857', 'delta': '0:00:00.156224', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "HeatReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Setup complete'], 'stderr_lines': [], 'failed': False, 'item': 'HeatReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,481 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Setup complete', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "KeystoneEndpointReady")].message}'], 'start': '2025-06-02 10:39:01.124475', 'end': '2025-06-02 10:39:01.314701', 'delta': '0:00:00.190226', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "KeystoneEndpointReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Setup complete'], 'stderr_lines': [], 'failed': False, 'item': 'KeystoneEndpointReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,489 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Setup complete', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "KeystoneServiceReady")].message}'], 'start': '2025-06-02 10:39:01.573324', 'end': '2025-06-02 10:39:01.713499', 'delta': '0:00:00.140175', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "KeystoneServiceReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Setup complete'], 'stderr_lines': [], 'failed': False, 'item': 'KeystoneServiceReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,495 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'MariaDBAccount creation complete', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "MariaDBAccountReady")].message}'], 'start': '2025-06-02 10:39:01.902217', 'end': '2025-06-02 10:39:02.053818', 'delta': '0:00:00.151601', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "MariaDBAccountReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['MariaDBAccount creation complete'], 'stderr_lines': [], 'failed': False, 'item': 'MariaDBAccountReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,501 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': ' Memcached instance has been provisioned', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "MemcachedReady")].message}'], 'start': '2025-06-02 10:39:02.224231', 'end': '2025-06-02 10:39:02.369559', 'delta': '0:00:00.145328', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "MemcachedReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': [' Memcached instance has been provisioned'], 'stderr_lines': [], 'failed': False, 'item': 'MemcachedReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,511 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'RabbitMqTransportURL successfully created', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "RabbitMqTransportURLReady")].message}'], 'start': '2025-06-02 10:39:02.530633', 'end': '2025-06-02 10:39:02.684652', 'delta': '0:00:00.154019', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "RabbitMqTransportURLReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['RabbitMqTransportURL successfully created'], 'stderr_lines': [], 'failed': False, 'item': 'RabbitMqTransportURLReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,517 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'RoleBinding created', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "RoleBindingReady")].message}'], 'start': '2025-06-02 10:39:02.879172', 'end': '2025-06-02 10:39:03.029493', 'delta': '0:00:00.150321', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "RoleBindingReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['RoleBinding created'], 'stderr_lines': [], 'failed': False, 'item': 'RoleBindingReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,523 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Role created', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "RoleReady")].message}'], 'start': '2025-06-02 10:39:03.211869', 'end': '2025-06-02 10:39:03.367485', 'delta': '0:00:00.155616', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "RoleReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Role created'], 'stderr_lines': [], 'failed': False, 'item': 'RoleReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,531 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'ServiceAccount created', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "ServiceAccountReady")].message}'], 'start': '2025-06-02 10:39:03.544494', 'end': '2025-06-02 10:39:03.692795', 'delta': '0:00:00.148301', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "ServiceAccountReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['ServiceAccount created'], 'stderr_lines': [], 'failed': False, 'item': 'ServiceAccountReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,537 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Service config create completed', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "ServiceConfigReady")].message}'], 'start': '2025-06-02 10:39:03.866380', 'end': '2025-06-02 10:39:04.010875', 'delta': '0:00:00.144495', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "ServiceConfigReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Service config create completed'], 'stderr_lines': [], 'failed': False, 'item': 'ServiceConfigReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,542 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'changed': True, 'stdout': 'Input data complete', 'stderr': '', 'rc': 0, 'cmd': ['oc', 'get', 'autoscaling', 'autoscaling', '-o', 'jsonpath={.status.conditions[?(@.type == "TLSInputReady")].message}'], 'start': '2025-06-02 10:39:04.175577', 'end': '2025-06-02 10:39:04.350612', 'delta': '0:00:00.175035', 'msg': '', 'invocation': {'module_args': {'_raw_params': 'oc get autoscaling autoscaling -o jsonpath=\'{.status.conditions[?(@.type == "TLSInputReady")].message}\'', '_uses_shell': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['Input data complete'], 'stderr_lines': [], 'failed': False, 'item': 'TLSInputReady', 'ansible_loop_var': 'item'}) 2025-06-02 10:39:04,561 p=33732 u=zuul n=ansible | TASK [TEST Check that all Autoscaling conditions are as expected when COO isn't installed RHOSO-123462 that=['conditions[item.key] == item.value'], fail_msg=Expected {{ item.key }} condition field to be {{ item.value }}, not {{ output }}] *** 2025-06-02 10:39:04,561 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:39:04 +0000 (0:00:00.176) 0:06:14.157 *********** 2025-06-02 10:39:04,588 p=33732 u=zuul n=ansible | ok: [localhost] => (item={'key': 'Ready', 'value': 'Setup complete'}) => ansible_loop_var: item changed: false item: key: Ready value: Setup complete msg: All assertions passed 2025-06-02 10:39:04,624 p=33732 u=zuul n=ansible | TASK [Install COO _raw_params=create-coo-subscription.yaml] ******************** 2025-06-02 10:39:04,625 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:39:04 +0000 (0:00:00.063) 0:06:14.220 *********** 2025-06-02 10:39:04,646 p=33732 u=zuul n=ansible | included: /home/zuul/src/github.com/openstack-k8s-operators/telemetry-operator/ci/create-coo-subscription.yaml for localhost 2025-06-02 10:39:04,659 p=33732 u=zuul n=ansible | TASK [Create the COO subscription _raw_params=oc create -f - < changed: false msg: All assertions passed 2025-06-02 10:42:54,584 p=33732 u=zuul n=ansible | TASK [Patch MetricStorage to use CustomMonitoringStack field _raw_params=oc patch oscp/{{ control_plane_name }} --type merge -p '{"spec":{"telemetry":{"template":{"metricStorage":{"monitoringStack": null, "customMonitoringStack":{"prometheusConfig":{"replicas": 1}, "resourceSelector":{"matchLabels":{"service":"metricStorage"}}}}}}}}' ] *** 2025-06-02 10:42:54,584 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:42:54 +0000 (0:00:00.040) 0:10:04.180 *********** 2025-06-02 10:42:55,130 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:42:55,144 p=33732 u=zuul n=ansible | TASK [Wait until MetricStorage is ready _raw_params=oc wait telemetry telemetry --for=condition=Ready --timeout=2m] *** 2025-06-02 10:42:55,145 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:42:55 +0000 (0:00:00.560) 0:10:04.740 *********** 2025-06-02 10:42:55,575 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:42:55,585 p=33732 u=zuul n=ansible | TASK [Get current time operator_reconciled_timestamp={{ now(utc=true,fmt='%Y-%m-%dT%H:%M:%SZ') }}] *** 2025-06-02 10:42:55,585 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:42:55 +0000 (0:00:00.440) 0:10:05.180 *********** 2025-06-02 10:42:55,600 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:42:55,611 p=33732 u=zuul n=ansible | TASK [Wait for new logs to be generated timeout=120] *************************** 2025-06-02 10:42:55,611 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:42:55 +0000 (0:00:00.025) 0:10:05.206 *********** 2025-06-02 10:44:55,904 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:44:55,912 p=33732 u=zuul n=ansible | TASK [Get new telemetry-operator logs _raw_params=oc logs -n openstack-operators -l "openstack.org/operator-name=telemetry" --tail=-1 --since-time "{{ operator_reconciled_timestamp }}"] *** 2025-06-02 10:44:55,913 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:44:55 +0000 (0:02:00.301) 0:12:05.508 *********** 2025-06-02 10:44:56,321 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:44:56,334 p=33732 u=zuul n=ansible | TASK [TEST Check that telemetry-operator logs don't include any errors when using the CustomMonitoringStack RHOSO-123464 error_list={{ operator_logs.stdout | ansible.builtin.regex_findall('ERROR.*') }}] *** 2025-06-02 10:44:56,334 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:44:56 +0000 (0:00:00.421) 0:12:05.929 *********** 2025-06-02 10:44:56,354 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:44:56,364 p=33732 u=zuul n=ansible | TASK [Patch Autoscaling to use a custom Prometheus instance _raw_params=oc patch oscp/{{ control_plane_name }} --type merge -p '{"spec":{"telemetry":{"template":{"autoscaling":{"prometheusHost":"someprometheus.openstack.svc", "prometheusPort":1234}}}}}' ] *** 2025-06-02 10:44:56,364 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:44:56 +0000 (0:00:00.030) 0:12:05.960 *********** 2025-06-02 10:44:56,897 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:44:56,906 p=33732 u=zuul n=ansible | TASK [Wait until Autoscaling is ready _raw_params=oc wait telemetry telemetry --for=condition=Ready --timeout=2m] *** 2025-06-02 10:44:56,907 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:44:56 +0000 (0:00:00.542) 0:12:06.502 *********** 2025-06-02 10:44:57,365 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:44:57,374 p=33732 u=zuul n=ansible | TASK [Get current time operator_reconciled_timestamp={{ now(utc=true,fmt='%Y-%m-%dT%H:%M:%SZ') }}] *** 2025-06-02 10:44:57,375 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:44:57 +0000 (0:00:00.468) 0:12:06.970 *********** 2025-06-02 10:44:57,394 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:44:57,404 p=33732 u=zuul n=ansible | TASK [Wait for new logs to be generated timeout=120] *************************** 2025-06-02 10:44:57,404 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:44:57 +0000 (0:00:00.029) 0:12:07.000 *********** 2025-06-02 10:46:57,699 p=33732 u=zuul n=ansible | ok: [localhost] 2025-06-02 10:46:57,708 p=33732 u=zuul n=ansible | TASK [Get new telemetry-operator logs _raw_params=oc logs -n openstack-operators -l "openstack.org/operator-name=telemetry" --tail=-1 --since-time "{{ operator_reconciled_timestamp }}"] *** 2025-06-02 10:46:57,708 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:46:57 +0000 (0:02:00.304) 0:14:07.304 *********** 2025-06-02 10:46:58,110 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:46:58,121 p=33732 u=zuul n=ansible | TASK [TEST Check that telemetry-operator logs don't include any errors when using a custom Prometheus instance for Autoscaling RHOSO-123465 error_list={{ operator_logs.stdout | ansible.builtin.regex_findall('ERROR.*') }}] *** 2025-06-02 10:46:58,121 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:46:58 +0000 (0:00:00.412) 0:14:07.716 *********** 2025-06-02 10:46:58,136 p=33732 u=zuul n=ansible | skipping: [localhost] 2025-06-02 10:46:58,146 p=33732 u=zuul n=ansible | TASK [TEST Check Prometheus host is set correctly in the aodh-evaluator container when using a custom Prometheus RHOSO-123466 _raw_params=oc rsh -c aodh-evaluator aodh-0 cat /etc/openstack/prometheus.yaml | grep host | cut -d " " -f 2] *** 2025-06-02 10:46:58,146 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:46:58 +0000 (0:00:00.025) 0:14:07.742 *********** 2025-06-02 10:46:58,638 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:46:58,648 p=33732 u=zuul n=ansible | TASK [TEST Check Prometheus port is set correctly in the aodh-evaluator container when using a custom Prometheus RHOSO-123467 _raw_params=oc rsh -c aodh-evaluator aodh-0 cat /etc/openstack/prometheus.yaml | grep port | cut -d " " -f 2] *** 2025-06-02 10:46:58,648 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:46:58 +0000 (0:00:00.501) 0:14:08.243 *********** 2025-06-02 10:46:59,128 p=33732 u=zuul n=ansible | changed: [localhost] 2025-06-02 10:46:59,167 p=33732 u=zuul n=ansible | PLAY RECAP ********************************************************************* 2025-06-02 10:46:59,167 p=33732 u=zuul n=ansible | localhost : ok=49 changed=24 unreachable=0 failed=0 skipped=7 rescued=0 ignored=0 2025-06-02 10:46:59,167 p=33732 u=zuul n=ansible | Monday 02 June 2025 10:46:59 +0000 (0:00:00.519) 0:14:08.763 *********** 2025-06-02 10:46:59,167 p=33732 u=zuul n=ansible | =============================================================================== 2025-06-02 10:46:59,167 p=33732 u=zuul n=ansible | Wait for new logs to be generated ------------------------------------- 120.39s 2025-06-02 10:46:59,167 p=33732 u=zuul n=ansible | Wait for new logs to be generated ------------------------------------- 120.31s 2025-06-02 10:46:59,167 p=33732 u=zuul n=ansible | Wait for new logs to be generated ------------------------------------- 120.30s 2025-06-02 10:46:59,167 p=33732 u=zuul n=ansible | Wait for new logs to be generated ------------------------------------- 120.30s 2025-06-02 10:46:59,167 p=33732 u=zuul n=ansible | Wait for new logs to be generated ------------------------------------- 120.28s 2025-06-02 10:46:59,167 p=33732 u=zuul n=ansible | Wait until reconciliation finishes ------------------------------------ 120.20s 2025-06-02 10:46:59,167 p=33732 u=zuul n=ansible | Wait until Autoscaling and MetricStorage are ready --------------------- 62.49s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | Wait for the resources to be available --------------------------------- 25.27s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | Wait for the required resource to be created --------------------------- 21.04s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | Get Autoscaling condition values ---------------------------------------- 5.40s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | Get MetricStorage condition values -------------------------------------- 3.40s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | Patch MetricStorage to use CustomMonitoringStack field ------------------ 0.56s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | Patch Autoscaling to use a custom Prometheus instance ------------------- 0.54s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | Enable Autoscaling ------------------------------------------------------ 0.53s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | TEST Check Prometheus port is set correctly in the aodh-evaluator container when using a custom Prometheus RHOSO-123467 --- 0.52s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | TEST Check Prometheus host is set correctly in the aodh-evaluator container when using a custom Prometheus RHOSO-123466 --- 0.50s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | Wait until telemetry is finished reconciling and its conditions are all True --- 0.48s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | Wait until Autoscaling is ready ----------------------------------------- 0.47s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | Enable MetricStorage ---------------------------------------------------- 0.45s 2025-06-02 10:46:59,168 p=33732 u=zuul n=ansible | Wait until MetricStorage is ready --------------------------------------- 0.44s