2026-03-19 19:40:50,183 p=37311 u=zuul n=ansible | [WARNING]: Found variable using reserved name: namespace 2026-03-19 19:40:50,183 p=37311 u=zuul n=ansible | PLAY [Verify all the applicable projects, endpoints, pods & services for cloudkitty] *** 2026-03-19 19:40:50,193 p=37311 u=zuul n=ansible | TASK [Gathering Facts] ********************************************************* 2026-03-19 19:40:50,193 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:50 +0000 (0:00:00.026) 0:00:00.026 ******** 2026-03-19 19:40:51,148 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:51,166 p=37311 u=zuul n=ansible | TASK [common : Run pod tests] ************************************************** 2026-03-19 19:40:51,166 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:51 +0000 (0:00:00.973) 0:00:00.999 ******** 2026-03-19 19:40:51,234 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-api) 2026-03-19 19:40:51,236 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-compactor) 2026-03-19 19:40:51,238 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-distributor) 2026-03-19 19:40:51,241 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway) 2026-03-19 19:40:51,243 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-ingester) 2026-03-19 19:40:51,246 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-querier) 2026-03-19 19:40:51,248 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend) 2026-03-19 19:40:51,251 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-proc) 2026-03-19 19:40:51,259 p=37311 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-03-19 19:40:51,259 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:51 +0000 (0:00:00.093) 0:00:01.092 ******** 2026-03-19 19:40:51,673 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:51,681 p=37311 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-api pod is Running in openstack namespace] *** 2026-03-19 19:40:51,681 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:51 +0000 (0:00:00.421) 0:00:01.514 ******** 2026-03-19 19:40:52,000 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:52,008 p=37311 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-03-19 19:40:52,008 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:52 +0000 (0:00:00.326) 0:00:01.841 ******** 2026-03-19 19:40:52,383 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:52,390 p=37311 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-compactor pod is Running in openstack namespace] *** 2026-03-19 19:40:52,390 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:52 +0000 (0:00:00.382) 0:00:02.223 ******** 2026-03-19 19:40:52,710 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:52,717 p=37311 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-03-19 19:40:52,718 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:52 +0000 (0:00:00.327) 0:00:02.550 ******** 2026-03-19 19:40:53,083 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:53,092 p=37311 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-distributor pod is Running in openstack namespace] *** 2026-03-19 19:40:53,092 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:53 +0000 (0:00:00.374) 0:00:02.925 ******** 2026-03-19 19:40:53,416 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:53,424 p=37311 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-03-19 19:40:53,425 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:53 +0000 (0:00:00.332) 0:00:03.257 ******** 2026-03-19 19:40:53,776 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:53,784 p=37311 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-index-gateway pod is Running in openstack namespace] *** 2026-03-19 19:40:53,785 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:53 +0000 (0:00:00.359) 0:00:03.617 ******** 2026-03-19 19:40:54,139 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:54,149 p=37311 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-03-19 19:40:54,149 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:54 +0000 (0:00:00.364) 0:00:03.982 ******** 2026-03-19 19:40:54,531 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:54,539 p=37311 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-ingester pod is Running in openstack namespace] *** 2026-03-19 19:40:54,539 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:54 +0000 (0:00:00.389) 0:00:04.372 ******** 2026-03-19 19:40:54,844 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:54,852 p=37311 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-03-19 19:40:54,852 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:54 +0000 (0:00:00.312) 0:00:04.684 ******** 2026-03-19 19:40:55,216 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:55,223 p=37311 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-querier pod is Running in openstack namespace] *** 2026-03-19 19:40:55,224 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:55 +0000 (0:00:00.371) 0:00:05.056 ******** 2026-03-19 19:40:55,512 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:55,519 p=37311 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-03-19 19:40:55,520 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:55 +0000 (0:00:00.295) 0:00:05.352 ******** 2026-03-19 19:40:55,881 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:55,889 p=37311 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-query-frontend pod is Running in openstack namespace] *** 2026-03-19 19:40:55,889 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:55 +0000 (0:00:00.369) 0:00:05.722 ******** 2026-03-19 19:40:56,190 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:56,199 p=37311 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-03-19 19:40:56,199 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:56 +0000 (0:00:00.309) 0:00:06.032 ******** 2026-03-19 19:40:56,570 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:56,578 p=37311 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-proc pod is Running in openstack namespace] *** 2026-03-19 19:40:56,578 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:56 +0000 (0:00:00.378) 0:00:06.410 ******** 2026-03-19 19:40:56,880 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:56,886 p=37311 u=zuul n=ansible | TASK [common : Run subscription tests] ***************************************** 2026-03-19 19:40:56,887 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:56 +0000 (0:00:00.308) 0:00:06.719 ******** 2026-03-19 19:40:56,902 p=37311 u=zuul n=ansible | skipping: [localhost] 2026-03-19 19:40:56,908 p=37311 u=zuul n=ansible | TASK [common : Run endpoint tests] ********************************************* 2026-03-19 19:40:56,908 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:56 +0000 (0:00:00.021) 0:00:06.741 ******** 2026-03-19 19:40:56,934 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/endpoint_tests.yml for localhost => (item=['cloudkitty', 'rating', 'public']) 2026-03-19 19:40:56,936 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/endpoint_tests.yml for localhost => (item=['cloudkitty', 'rating', 'internal']) 2026-03-19 19:40:56,956 p=37311 u=zuul n=ansible | TASK [common : Verify endpoint "['cloudkitty', 'rating', 'public']"] *********** 2026-03-19 19:40:56,956 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:56 +0000 (0:00:00.047) 0:00:06.789 ******** 2026-03-19 19:40:59,084 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:40:59,092 p=37311 u=zuul n=ansible | TASK [common : TEST Check that cloudkitty public endpoint exists] ************** 2026-03-19 19:40:59,092 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:59 +0000 (0:00:02.136) 0:00:08.925 ******** 2026-03-19 19:40:59,124 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "The endpoint cloudkitty is found." } 2026-03-19 19:40:59,132 p=37311 u=zuul n=ansible | TASK [common : Verify endpoint "['cloudkitty', 'rating', 'internal']"] ********* 2026-03-19 19:40:59,132 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:40:59 +0000 (0:00:00.039) 0:00:08.964 ******** 2026-03-19 19:41:01,430 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:01,437 p=37311 u=zuul n=ansible | TASK [common : TEST Check that cloudkitty internal endpoint exists] ************ 2026-03-19 19:41:01,437 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:01 +0000 (0:00:02.305) 0:00:11.270 ******** 2026-03-19 19:41:01,456 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "The endpoint cloudkitty is found." } 2026-03-19 19:41:01,463 p=37311 u=zuul n=ansible | TASK [common : Run project tests] ********************************************** 2026-03-19 19:41:01,463 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:01 +0000 (0:00:00.026) 0:00:11.296 ******** 2026-03-19 19:41:01,498 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/project_tests.yml for localhost => (item=openstack) 2026-03-19 19:41:01,501 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/project_tests.yml for localhost => (item=openstack-operators) 2026-03-19 19:41:01,529 p=37311 u=zuul n=ansible | TASK [common : Get the openstack project] ************************************** 2026-03-19 19:41:01,529 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:01 +0000 (0:00:00.066) 0:00:11.362 ******** 2026-03-19 19:41:01,868 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:01,875 p=37311 u=zuul n=ansible | TASK [common : TEST Verify openstack project exists] *************************** 2026-03-19 19:41:01,875 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:01 +0000 (0:00:00.345) 0:00:11.708 ******** 2026-03-19 19:41:01,893 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "project openstack is found." } 2026-03-19 19:41:01,902 p=37311 u=zuul n=ansible | TASK [common : Get the openstack-operators project] **************************** 2026-03-19 19:41:01,902 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:01 +0000 (0:00:00.027) 0:00:11.735 ******** 2026-03-19 19:41:02,257 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:02,269 p=37311 u=zuul n=ansible | TASK [common : TEST Verify openstack-operators project exists] ***************** 2026-03-19 19:41:02,270 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:02 +0000 (0:00:00.367) 0:00:12.102 ******** 2026-03-19 19:41:02,286 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "project openstack-operators is found." } 2026-03-19 19:41:02,298 p=37311 u=zuul n=ansible | TASK [common : Run service tests] ********************************************** 2026-03-19 19:41:02,298 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:02 +0000 (0:00:00.028) 0:00:12.131 ******** 2026-03-19 19:41:02,396 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-internal) 2026-03-19 19:41:02,398 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-compactor-grpc) 2026-03-19 19:41:02,400 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-compactor-http) 2026-03-19 19:41:02,402 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-distributor-grpc) 2026-03-19 19:41:02,404 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-distributor-http) 2026-03-19 19:41:02,406 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-gateway-http) 2026-03-19 19:41:02,408 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-gossip-ring) 2026-03-19 19:41:02,410 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway-grpc) 2026-03-19 19:41:02,412 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway-http) 2026-03-19 19:41:02,414 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-ingester-grpc) 2026-03-19 19:41:02,416 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-ingester-http) 2026-03-19 19:41:02,418 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-querier-grpc) 2026-03-19 19:41:02,420 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-querier-http) 2026-03-19 19:41:02,422 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend-grpc) 2026-03-19 19:41:02,424 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend-http) 2026-03-19 19:41:02,426 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-public) 2026-03-19 19:41:02,450 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-internal"] ************************* 2026-03-19 19:41:02,451 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:02 +0000 (0:00:00.152) 0:00:12.283 ******** 2026-03-19 19:41:02,778 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:02,785 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-internal service is running in openstack namespace] *** 2026-03-19 19:41:02,785 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:02 +0000 (0:00:00.334) 0:00:12.618 ******** 2026-03-19 19:41:02,800 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-internal is running." } 2026-03-19 19:41:02,807 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-compactor-grpc"] ********* 2026-03-19 19:41:02,807 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:02 +0000 (0:00:00.022) 0:00:12.640 ******** 2026-03-19 19:41:03,145 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:03,157 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-compactor-grpc service is running in openstack namespace] *** 2026-03-19 19:41:03,158 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:03 +0000 (0:00:00.350) 0:00:12.990 ******** 2026-03-19 19:41:03,172 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-compactor-grpc is running." } 2026-03-19 19:41:03,179 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-compactor-http"] ********* 2026-03-19 19:41:03,179 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:03 +0000 (0:00:00.021) 0:00:13.012 ******** 2026-03-19 19:41:03,493 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:03,500 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-compactor-http service is running in openstack namespace] *** 2026-03-19 19:41:03,500 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:03 +0000 (0:00:00.320) 0:00:13.333 ******** 2026-03-19 19:41:03,517 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-compactor-http is running." } 2026-03-19 19:41:03,525 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-distributor-grpc"] ******* 2026-03-19 19:41:03,525 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:03 +0000 (0:00:00.025) 0:00:13.358 ******** 2026-03-19 19:41:03,832 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:03,840 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-distributor-grpc service is running in openstack namespace] *** 2026-03-19 19:41:03,840 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:03 +0000 (0:00:00.314) 0:00:13.673 ******** 2026-03-19 19:41:03,856 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-distributor-grpc is running." } 2026-03-19 19:41:03,863 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-distributor-http"] ******* 2026-03-19 19:41:03,863 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:03 +0000 (0:00:00.022) 0:00:13.696 ******** 2026-03-19 19:41:04,200 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:04,208 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-distributor-http service is running in openstack namespace] *** 2026-03-19 19:41:04,208 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:04 +0000 (0:00:00.345) 0:00:14.041 ******** 2026-03-19 19:41:04,225 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-distributor-http is running." } 2026-03-19 19:41:04,234 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-gateway-http"] *********** 2026-03-19 19:41:04,234 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:04 +0000 (0:00:00.025) 0:00:14.066 ******** 2026-03-19 19:41:04,568 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:04,576 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-gateway-http service is running in openstack namespace] *** 2026-03-19 19:41:04,576 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:04 +0000 (0:00:00.342) 0:00:14.409 ******** 2026-03-19 19:41:04,602 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-gateway-http is running." } 2026-03-19 19:41:04,612 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-gossip-ring"] ************ 2026-03-19 19:41:04,612 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:04 +0000 (0:00:00.036) 0:00:14.445 ******** 2026-03-19 19:41:04,973 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:04,983 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-gossip-ring service is running in openstack namespace] *** 2026-03-19 19:41:04,983 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:04 +0000 (0:00:00.370) 0:00:14.816 ******** 2026-03-19 19:41:05,000 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-gossip-ring is running." } 2026-03-19 19:41:05,008 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-index-gateway-grpc"] ***** 2026-03-19 19:41:05,008 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:05 +0000 (0:00:00.025) 0:00:14.841 ******** 2026-03-19 19:41:05,346 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:05,354 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-index-gateway-grpc service is running in openstack namespace] *** 2026-03-19 19:41:05,354 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:05 +0000 (0:00:00.345) 0:00:15.187 ******** 2026-03-19 19:41:05,373 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-index-gateway-grpc is running." } 2026-03-19 19:41:05,380 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-index-gateway-http"] ***** 2026-03-19 19:41:05,380 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:05 +0000 (0:00:00.026) 0:00:15.213 ******** 2026-03-19 19:41:05,718 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:05,729 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-index-gateway-http service is running in openstack namespace] *** 2026-03-19 19:41:05,729 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:05 +0000 (0:00:00.348) 0:00:15.562 ******** 2026-03-19 19:41:05,747 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-index-gateway-http is running." } 2026-03-19 19:41:05,754 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-ingester-grpc"] ********** 2026-03-19 19:41:05,754 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:05 +0000 (0:00:00.025) 0:00:15.587 ******** 2026-03-19 19:41:06,081 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:06,089 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-ingester-grpc service is running in openstack namespace] *** 2026-03-19 19:41:06,089 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:06 +0000 (0:00:00.334) 0:00:15.922 ******** 2026-03-19 19:41:06,106 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-ingester-grpc is running." } 2026-03-19 19:41:06,116 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-ingester-http"] ********** 2026-03-19 19:41:06,116 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:06 +0000 (0:00:00.026) 0:00:15.948 ******** 2026-03-19 19:41:06,449 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:06,457 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-ingester-http service is running in openstack namespace] *** 2026-03-19 19:41:06,457 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:06 +0000 (0:00:00.341) 0:00:16.290 ******** 2026-03-19 19:41:06,478 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-ingester-http is running." } 2026-03-19 19:41:06,486 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-querier-grpc"] *********** 2026-03-19 19:41:06,486 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:06 +0000 (0:00:00.028) 0:00:16.319 ******** 2026-03-19 19:41:06,810 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:06,819 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-querier-grpc service is running in openstack namespace] *** 2026-03-19 19:41:06,819 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:06 +0000 (0:00:00.332) 0:00:16.651 ******** 2026-03-19 19:41:06,841 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-querier-grpc is running." } 2026-03-19 19:41:06,849 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-querier-http"] *********** 2026-03-19 19:41:06,849 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:06 +0000 (0:00:00.030) 0:00:16.682 ******** 2026-03-19 19:41:07,222 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:07,230 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-querier-http service is running in openstack namespace] *** 2026-03-19 19:41:07,230 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:07 +0000 (0:00:00.381) 0:00:17.063 ******** 2026-03-19 19:41:07,253 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-querier-http is running." } 2026-03-19 19:41:07,261 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-query-frontend-grpc"] **** 2026-03-19 19:41:07,261 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:07 +0000 (0:00:00.030) 0:00:17.093 ******** 2026-03-19 19:41:07,598 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:07,605 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-query-frontend-grpc service is running in openstack namespace] *** 2026-03-19 19:41:07,605 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:07 +0000 (0:00:00.344) 0:00:17.438 ******** 2026-03-19 19:41:07,621 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-query-frontend-grpc is running." } 2026-03-19 19:41:07,629 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-query-frontend-http"] **** 2026-03-19 19:41:07,630 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:07 +0000 (0:00:00.024) 0:00:17.462 ******** 2026-03-19 19:41:07,949 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:07,957 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-query-frontend-http service is running in openstack namespace] *** 2026-03-19 19:41:07,957 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:07 +0000 (0:00:00.327) 0:00:17.790 ******** 2026-03-19 19:41:07,974 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-query-frontend-http is running." } 2026-03-19 19:41:07,981 p=37311 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-public"] *************************** 2026-03-19 19:41:07,981 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:07 +0000 (0:00:00.023) 0:00:17.814 ******** 2026-03-19 19:41:08,311 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:08,319 p=37311 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-public service is running in openstack namespace] *** 2026-03-19 19:41:08,319 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:08 +0000 (0:00:00.337) 0:00:18.152 ******** 2026-03-19 19:41:08,337 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-public is running." } 2026-03-19 19:41:08,344 p=37311 u=zuul n=ansible | TASK [common : Run file tests] ************************************************* 2026-03-19 19:41:08,344 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:08 +0000 (0:00:00.024) 0:00:18.177 ******** 2026-03-19 19:41:08,360 p=37311 u=zuul n=ansible | skipping: [localhost] 2026-03-19 19:41:08,367 p=37311 u=zuul n=ansible | TASK [common : Get the manifest list once] ************************************* 2026-03-19 19:41:08,367 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:08 +0000 (0:00:00.022) 0:00:18.200 ******** 2026-03-19 19:41:08,384 p=37311 u=zuul n=ansible | skipping: [localhost] 2026-03-19 19:41:08,391 p=37311 u=zuul n=ansible | TASK [common : Check that each package manifest is present] ******************** 2026-03-19 19:41:08,391 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:08 +0000 (0:00:00.024) 0:00:18.224 ******** 2026-03-19 19:41:08,411 p=37311 u=zuul n=ansible | skipping: [localhost] 2026-03-19 19:41:08,419 p=37311 u=zuul n=ansible | TASK [common : Run crd tests] ************************************************** 2026-03-19 19:41:08,420 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:08 +0000 (0:00:00.028) 0:00:18.253 ******** 2026-03-19 19:41:08,437 p=37311 u=zuul n=ansible | skipping: [localhost] 2026-03-19 19:41:08,444 p=37311 u=zuul n=ansible | TASK [common : Run CR tests] *************************************************** 2026-03-19 19:41:08,444 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:08 +0000 (0:00:00.024) 0:00:18.277 ******** 2026-03-19 19:41:08,462 p=37311 u=zuul n=ansible | skipping: [localhost] 2026-03-19 19:41:08,471 p=37311 u=zuul n=ansible | TASK [common : Verify container tests] ***************************************** 2026-03-19 19:41:08,472 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:08 +0000 (0:00:00.027) 0:00:18.304 ******** 2026-03-19 19:41:08,492 p=37311 u=zuul n=ansible | skipping: [localhost] 2026-03-19 19:41:08,506 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Validate Chargeback Feature deployed correctly] *** 2026-03-19 19:41:08,507 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:08 +0000 (0:00:00.034) 0:00:18.339 ******** 2026-03-19 19:41:08,526 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/chargeback_tests.yml for localhost 2026-03-19 19:41:08,536 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Enable Cloudkitty Module (hashmap)] *************** 2026-03-19 19:41:08,537 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:08 +0000 (0:00:00.029) 0:00:18.369 ******** 2026-03-19 19:41:12,427 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 19:41:12,435 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Find the current value of hashmap] **************** 2026-03-19 19:41:12,435 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:12 +0000 (0:00:03.898) 0:00:22.268 ******** 2026-03-19 19:41:14,957 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:14,971 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Change priority for CloudKitty hashmap module] **** 2026-03-19 19:41:14,971 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:14 +0000 (0:00:02.536) 0:00:24.804 ******** 2026-03-19 19:41:18,157 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 19:41:18,168 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Get status of all CloudKitty rating modules] ****** 2026-03-19 19:41:18,168 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:18 +0000 (0:00:03.196) 0:00:28.001 ******** 2026-03-19 19:41:20,377 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:20,386 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Validate CloudKitty module states] *********** 2026-03-19 19:41:20,386 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:20 +0000 (0:00:02.218) 0:00:30.219 ******** 2026-03-19 19:41:20,409 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "SUCCESS: CloudKitty modules (hashmap=True) are configured correctly." } 2026-03-19 19:41:20,417 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Set priority for CloudKitty hashmap module] *** 2026-03-19 19:41:20,417 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:20 +0000 (0:00:00.030) 0:00:30.249 ******** 2026-03-19 19:41:20,436 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "SUCCESS: The hashmap priority is set to 100" } 2026-03-19 19:41:20,445 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Setup Loki Environment] *************************** 2026-03-19 19:41:20,445 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:20 +0000 (0:00:00.028) 0:00:30.278 ******** 2026-03-19 19:41:20,468 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/setup_loki_env.yml for localhost 2026-03-19 19:41:20,483 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Get Loki Public Route Host] *********************** 2026-03-19 19:41:20,483 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:20 +0000 (0:00:00.037) 0:00:30.316 ******** 2026-03-19 19:41:20,867 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:20,877 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Set Loki URLs] ************************************ 2026-03-19 19:41:20,877 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:20 +0000 (0:00:00.393) 0:00:30.710 ******** 2026-03-19 19:41:20,891 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:20,898 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Set Derived Loki URLs] **************************** 2026-03-19 19:41:20,898 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:20 +0000 (0:00:00.021) 0:00:30.731 ******** 2026-03-19 19:41:20,912 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:20,919 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Debug URLs] *************************************** 2026-03-19 19:41:20,920 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:20 +0000 (0:00:00.021) 0:00:30.752 ******** 2026-03-19 19:41:20,935 p=37311 u=zuul n=ansible | ok: [localhost] => { "msg": [ "Loki Route: https://cloudkitty-lokistack-openstack.apps-crc.testing", "Push URL: https://cloudkitty-lokistack-openstack.apps-crc.testing/api/logs/v1/cloudkitty/loki/api/v1/push", "Flush URL: https://cloudkitty-lokistack-ingester-http.openstack.svc:3100/flush", "Query URL: https://cloudkitty-lokistack-openstack.apps-crc.testing/api/logs/v1/cloudkitty/loki/api/v1/query_range" ] } 2026-03-19 19:41:20,944 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Ensure Local Certificate Directory Exists] ******** 2026-03-19 19:41:20,944 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:20 +0000 (0:00:00.024) 0:00:30.777 ******** 2026-03-19 19:41:21,264 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 19:41:21,270 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Extract Certificates from Openshift Secret] ******* 2026-03-19 19:41:21,271 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:21 +0000 (0:00:00.326) 0:00:31.103 ******** 2026-03-19 19:41:21,592 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 19:41:21,599 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Extract Client Certificates] ********************** 2026-03-19 19:41:21,599 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:21 +0000 (0:00:00.328) 0:00:31.432 ******** 2026-03-19 19:41:21,916 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 19:41:21,923 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Extract CA Bundle] ******************************** 2026-03-19 19:41:21,923 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:21 +0000 (0:00:00.324) 0:00:31.756 ******** 2026-03-19 19:41:22,108 p=37311 u=zuul n=ansible | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ValueError: No closing quotation 2026-03-19 19:41:22,108 p=37311 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": true, "module_stderr": "Traceback (most recent call last):\n File \"/home/zuul/.ansible/tmp/ansible-tmp-1773949281.9443507-38454-17852292548554/AnsiballZ_command.py\", line 107, in \n _ansiballz_main()\n File \"/home/zuul/.ansible/tmp/ansible-tmp-1773949281.9443507-38454-17852292548554/AnsiballZ_command.py\", line 99, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/home/zuul/.ansible/tmp/ansible-tmp-1773949281.9443507-38454-17852292548554/AnsiballZ_command.py\", line 47, in invoke_module\n runpy.run_module(mod_name='ansible.modules.command', init_globals=dict(_module_fqn='ansible.modules.command', _modlib_path=modlib_path),\n File \"/usr/lib64/python3.9/runpy.py\", line 225, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/lib64/python3.9/runpy.py\", line 97, in _run_module_code\n _run_code(code, mod_globals, init_globals,\n File \"/usr/lib64/python3.9/runpy.py\", line 87, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_ansible.legacy.command_payload_jiruallv/ansible_ansible.legacy.command_payload.zip/ansible/modules/command.py\", line 354, in \n File \"/tmp/ansible_ansible.legacy.command_payload_jiruallv/ansible_ansible.legacy.command_payload.zip/ansible/modules/command.py\", line 276, in main\n File \"/usr/lib64/python3.9/shlex.py\", line 315, in split\n return list(lex)\n File \"/usr/lib64/python3.9/shlex.py\", line 300, in __next__\n token = self.get_token()\n File \"/usr/lib64/python3.9/shlex.py\", line 109, in get_token\n raw = self.read_token()\n File \"/usr/lib64/python3.9/shlex.py\", line 191, in read_token\n raise ValueError(\"No closing quotation\")\nValueError: No closing quotation\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1} 2026-03-19 19:41:22,108 p=37311 u=zuul n=ansible | ...ignoring 2026-03-19 19:41:22,115 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Cloudkitty debug ON] ****************************** 2026-03-19 19:41:22,115 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:22 +0000 (0:00:00.191) 0:00:31.948 ******** 2026-03-19 19:41:22,129 p=37311 u=zuul n=ansible | skipping: [localhost] 2026-03-19 19:41:22,137 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Cloudkitty debug OFF] ***************************** 2026-03-19 19:41:22,137 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:22 +0000 (0:00:00.021) 0:00:31.969 ******** 2026-03-19 19:41:22,154 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:22,161 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Get admin project ID for CI] ********************** 2026-03-19 19:41:22,161 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:22 +0000 (0:00:00.024) 0:00:31.994 ******** 2026-03-19 19:41:24,356 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:24,369 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Set admin project ID for CI] ********************** 2026-03-19 19:41:24,369 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:24 +0000 (0:00:02.207) 0:00:34.202 ******** 2026-03-19 19:41:24,386 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:24,393 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Get admin user ID for CI] ************************* 2026-03-19 19:41:24,394 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:24 +0000 (0:00:00.024) 0:00:34.226 ******** 2026-03-19 19:41:26,629 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:26,643 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Set admin user ID for CI] ************************* 2026-03-19 19:41:26,644 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:26 +0000 (0:00:02.250) 0:00:36.476 ******** 2026-03-19 19:41:26,663 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:26,671 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Find test files] ********************************** 2026-03-19 19:41:26,671 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:26 +0000 (0:00:00.027) 0:00:36.504 ******** 2026-03-19 19:41:26,986 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:26,993 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Extract only the filenames into a clean list] ***** 2026-03-19 19:41:26,994 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:26 +0000 (0:00:00.322) 0:00:36.826 ******** 2026-03-19 19:41:27,012 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:27,018 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Process and Loop if files exist] ****************** 2026-03-19 19:41:27,018 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:27 +0000 (0:00:00.024) 0:00:36.851 ******** 2026-03-19 19:41:27,048 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_dyn_basic) 2026-03-19 19:41:27,056 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_dyn_basic] *** 2026-03-19 19:41:27,056 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:27 +0000 (0:00:00.037) 0:00:36.889 ******** 2026-03-19 19:41:27,074 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost 2026-03-19 19:41:27,085 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically "test_dyn_basic"] ******* 2026-03-19 19:41:27,085 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:27 +0000 (0:00:00.028) 0:00:36.917 ******** 2026-03-19 19:41:27,107 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:27,113 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] **************** 2026-03-19 19:41:27,113 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:27 +0000 (0:00:00.028) 0:00:36.946 ******** 2026-03-19 19:41:27,411 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:27,419 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data "test_dyn_basic"] ********* 2026-03-19 19:41:27,419 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:27 +0000 (0:00:00.305) 0:00:37.252 ******** 2026-03-19 19:41:27,747 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 19:41:27,755 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file "test_dyn_basic"] *** 2026-03-19 19:41:27,755 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:27 +0000 (0:00:00.335) 0:00:37.588 ******** 2026-03-19 19:41:28,014 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 19:41:28,021 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] ********************** 2026-03-19 19:41:28,021 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:28 +0000 (0:00:00.266) 0:00:37.854 ******** 2026-03-19 19:41:28,037 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:28,045 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_dyn_basic] **************** 2026-03-19 19:41:28,046 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:28 +0000 (0:00:00.024) 0:00:37.878 ******** 2026-03-19 19:41:28,064 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost 2026-03-19 19:41:28,078 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log "test_dyn_basic"] ***** 2026-03-19 19:41:28,078 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:28 +0000 (0:00:00.032) 0:00:37.910 ******** 2026-03-19 19:41:28,094 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost 2026-03-19 19:41:28,103 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] **************************** 2026-03-19 19:41:28,103 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:28 +0000 (0:00:00.024) 0:00:37.935 ******** 2026-03-19 19:41:28,397 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:28,406 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ******************************** 2026-03-19 19:41:28,406 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:28 +0000 (0:00:00.302) 0:00:38.238 ******** 2026-03-19 19:41:29,113 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:29,124 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] ********************************** 2026-03-19 19:41:29,124 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:29 +0000 (0:00:00.718) 0:00:38.957 ******** 2026-03-19 19:41:29,137 p=37311 u=zuul n=ansible | ok: [localhost] => { "msg": "Ingestion Successful!" } 2026-03-19 19:41:29,145 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage " test_dyn_basic"] ***** 2026-03-19 19:41:29,145 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:29 +0000 (0:00:00.021) 0:00:38.978 ******** 2026-03-19 19:41:29,170 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost 2026-03-19 19:41:29,187 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************ 2026-03-19 19:41:29,187 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:29 +0000 (0:00:00.041) 0:00:39.020 ******** 2026-03-19 19:41:29,690 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:29,699 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] ******* 2026-03-19 19:41:29,699 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:29 +0000 (0:00:00.511) 0:00:39.532 ******** 2026-03-19 19:41:29,919 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 19:41:29,927 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] *************** 2026-03-19 19:41:29,927 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:29 +0000 (0:00:00.227) 0:00:39.759 ******** 2026-03-19 19:41:30,400 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 19:41:30,406 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************ 2026-03-19 19:41:30,407 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:30 +0000 (0:00:00.479) 0:00:40.239 ******** 2026-03-19 19:41:30,836 p=37311 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["oc", "exec", "-n", "openstack", "openstackclient", "--", "curl", "-v", "-X", "POST", "https://cloudkitty-lokistack-ingester-http.openstack.svc:3100/flush", "--cert", "osp-certs/tls.crt", "--key", "osp-certs/tls.key", "--cacert", "osp-certs/service-ca.crt"], "delta": "0:00:00.227148", "end": "2026-03-19 19:41:30.798689", "failed_when_result": true, "msg": "non-zero return code", "rc": 77, "start": "2026-03-19 19:41:30.571541", "stderr": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 10.217.5.225:3100...\n* Connected to cloudkitty-lokistack-ingester-http.openstack.svc (10.217.5.225) port 3100 (#0)\n* ALPN, offering h2\n* ALPN, offering http/1.1\n* error setting certificate file: osp-certs/service-ca.crt\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\n* Closing connection 0\ncurl: (77) error setting certificate file: osp-certs/service-ca.crt\ncommand terminated with exit code 77", "stderr_lines": [" % Total % Received % Xferd Average Speed Time Time Time Current", " Dload Upload Total Spent Left Speed", "", " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0", " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 10.217.5.225:3100...", "* Connected to cloudkitty-lokistack-ingester-http.openstack.svc (10.217.5.225) port 3100 (#0)", "* ALPN, offering h2", "* ALPN, offering http/1.1", "* error setting certificate file: osp-certs/service-ca.crt", "", " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0", "* Closing connection 0", "curl: (77) error setting certificate file: osp-certs/service-ca.crt", "command terminated with exit code 77"], "stdout": "", "stdout_lines": []} 2026-03-19 19:41:30,836 p=37311 u=zuul n=ansible | ...ignoring 2026-03-19 19:41:30,843 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ****************************** 2026-03-19 19:41:30,843 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:30 +0000 (0:00:00.436) 0:00:40.675 ******** 2026-03-19 19:41:30,866 p=37311 u=zuul n=ansible | fatal: [localhost]: FAILED! => { "assertion": "'204' in flush_response.stderr or '200' in flush_response.stderr", "changed": false, "evaluated_to": false, "msg": "Flush failed" } 2026-03-19 19:41:30,866 p=37311 u=zuul n=ansible | ...ignoring 2026-03-19 19:41:30,877 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki "test_dyn_basic"] ***** 2026-03-19 19:41:30,878 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:30 +0000 (0:00:00.034) 0:00:40.710 ******** 2026-03-19 19:41:30,898 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost 2026-03-19 19:41:30,922 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_dyn_basic] ******************** 2026-03-19 19:41:30,923 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:30 +0000 (0:00:00.044) 0:00:40.755 ******** 2026-03-19 19:41:30,942 p=37311 u=zuul n=ansible | ok: [localhost] => { "msg": "Input file has 42 data entries that Loki has to return" } 2026-03-19 19:41:30,949 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] *********************************** 2026-03-19 19:41:30,949 p=37311 u=zuul n=ansible | Thursday 19 March 2026 19:41:30 +0000 (0:00:00.026) 0:00:40.781 ******** 2026-03-19 20:06:43,306 p=37311 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"attempts": 25, "changed": false, "connection": "close", "content": "{\"status\":\"success\",\"data\":{\"resultType\":\"streams\",\"result\":[],\"stats\":{\"summary\":{\"bytesProcessedPerSecond\":0,\"linesProcessedPerSecond\":0,\"totalBytesProcessed\":0,\"totalLinesProcessed\":0,\"execTime\":0.110445,\"queueTime\":0,\"subqueries\":0,\"totalEntriesReturned\":0,\"splits\":50,\"shards\":50,\"totalPostFilterLines\":0,\"totalStructuredMetadataBytesProcessed\":0},\"querier\":{\"store\":{\"totalChunksRef\":0,\"totalChunksDownloaded\":0,\"chunksDownloadTime\":0,\"queryReferencedStructuredMetadata\":false,\"chunk\":{\"headChunkBytes\":0,\"headChunkLines\":0,\"decompressedBytes\":0,\"decompressedLines\":0,\"compressedBytes\":0,\"totalDuplicates\":0,\"postFilterLines\":0,\"headChunkStructuredMetadataBytes\":0,\"decompressedStructuredMetadataBytes\":0},\"chunkRefsFetchTime\":46839320,\"congestionControlLatency\":0,\"pipelineWrapperFilteredLines\":0}},\"ingester\":{\"totalReached\":7,\"totalChunksMatched\":0,\"totalBatches\":7,\"totalLinesSent\":0,\"store\":{\"totalChunksRef\":0,\"totalChunksDownloaded\":0,\"chunksDownloadTime\":0,\"queryReferencedStructuredMetadata\":false,\"chunk\":{\"headChunkBytes\":0,\"headChunkLines\":0,\"decompressedBytes\":0,\"decompressedLines\":0,\"compressedBytes\":0,\"totalDuplicates\":0,\"postFilterLines\":0,\"headChunkStructuredMetadataBytes\":0,\"decompressedStructuredMetadataBytes\":0},\"chunkRefsFetchTime\":0,\"congestionControlLatency\":0,\"pipelineWrapperFilteredLines\":0}},\"cache\":{\"chunk\":{\"entriesFound\":0,\"entriesRequested\":0,\"entriesStored\":0,\"bytesReceived\":0,\"bytesSent\":0,\"requests\":0,\"downloadTime\":0,\"queryLengthServed\":0},\"index\":{\"entriesFound\":0,\"entriesRequested\":0,\"entriesStored\":0,\"bytesReceived\":0,\"bytesSent\":0,\"requests\":0,\"downloadTime\":0,\"queryLengthServed\":0},\"result\":{\"entriesFound\":0,\"entriesRequested\":0,\"entriesStored\":0,\"bytesReceived\":0,\"bytesSent\":0,\"requests\":0,\"downloadTime\":0,\"queryLengthServed\":0},\"statsResult\":{\"entriesFound\":49,\"entriesRequested\":49,\"entriesStored\":0,\"bytesReceived\":10927,\"bytesSent\":0,\"requests\":49,\"downloadTime\":294246,\"queryLengthServed\":87152000000000},\"volumeResult\":{\"entriesFound\":0,\"entriesRequested\":0,\"entriesStored\":0,\"bytesReceived\":0,\"bytesSent\":0,\"requests\":0,\"downloadTime\":0,\"queryLengthServed\":0},\"seriesResult\":{\"entriesFound\":0,\"entriesRequested\":0,\"entriesStored\":0,\"bytesReceived\":0,\"bytesSent\":0,\"requests\":0,\"downloadTime\":0,\"queryLengthServed\":0},\"labelResult\":{\"entriesFound\":0,\"entriesRequested\":0,\"entriesStored\":0,\"bytesReceived\":0,\"bytesSent\":0,\"requests\":0,\"downloadTime\":0,\"queryLengthServed\":0},\"instantMetricResult\":{\"entriesFound\":0,\"entriesRequested\":0,\"entriesStored\":0,\"bytesReceived\":0,\"bytesSent\":0,\"requests\":0,\"downloadTime\":0,\"queryLengthServed\":0}},\"index\":{\"totalChunks\":0,\"postFilterChunks\":0,\"shardsDuration\":0,\"usedBloomFilters\":false}}}}\n", "content_type": "application/json; charset=UTF-8", "cookies": {}, "cookies_string": "", "date": "Thu, 19 Mar 2026 20:06:43 GMT", "elapsed": 0, "json": {"data": {"result": [], "resultType": "streams", "stats": {"cache": {"chunk": {"bytesReceived": 0, "bytesSent": 0, "downloadTime": 0, "entriesFound": 0, "entriesRequested": 0, "entriesStored": 0, "queryLengthServed": 0, "requests": 0}, "index": {"bytesReceived": 0, "bytesSent": 0, "downloadTime": 0, "entriesFound": 0, "entriesRequested": 0, "entriesStored": 0, "queryLengthServed": 0, "requests": 0}, "instantMetricResult": {"bytesReceived": 0, "bytesSent": 0, "downloadTime": 0, "entriesFound": 0, "entriesRequested": 0, "entriesStored": 0, "queryLengthServed": 0, "requests": 0}, "labelResult": {"bytesReceived": 0, "bytesSent": 0, "downloadTime": 0, "entriesFound": 0, "entriesRequested": 0, "entriesStored": 0, "queryLengthServed": 0, "requests": 0}, "result": {"bytesReceived": 0, "bytesSent": 0, "downloadTime": 0, "entriesFound": 0, "entriesRequested": 0, "entriesStored": 0, "queryLengthServed": 0, "requests": 0}, "seriesResult": {"bytesReceived": 0, "bytesSent": 0, "downloadTime": 0, "entriesFound": 0, "entriesRequested": 0, "entriesStored": 0, "queryLengthServed": 0, "requests": 0}, "statsResult": {"bytesReceived": 10927, "bytesSent": 0, "downloadTime": 294246, "entriesFound": 49, "entriesRequested": 49, "entriesStored": 0, "queryLengthServed": 87152000000000, "requests": 49}, "volumeResult": {"bytesReceived": 0, "bytesSent": 0, "downloadTime": 0, "entriesFound": 0, "entriesRequested": 0, "entriesStored": 0, "queryLengthServed": 0, "requests": 0}}, "index": {"postFilterChunks": 0, "shardsDuration": 0, "totalChunks": 0, "usedBloomFilters": false}, "ingester": {"store": {"chunk": {"compressedBytes": 0, "decompressedBytes": 0, "decompressedLines": 0, "decompressedStructuredMetadataBytes": 0, "headChunkBytes": 0, "headChunkLines": 0, "headChunkStructuredMetadataBytes": 0, "postFilterLines": 0, "totalDuplicates": 0}, "chunkRefsFetchTime": 0, "chunksDownloadTime": 0, "congestionControlLatency": 0, "pipelineWrapperFilteredLines": 0, "queryReferencedStructuredMetadata": false, "totalChunksDownloaded": 0, "totalChunksRef": 0}, "totalBatches": 7, "totalChunksMatched": 0, "totalLinesSent": 0, "totalReached": 7}, "querier": {"store": {"chunk": {"compressedBytes": 0, "decompressedBytes": 0, "decompressedLines": 0, "decompressedStructuredMetadataBytes": 0, "headChunkBytes": 0, "headChunkLines": 0, "headChunkStructuredMetadataBytes": 0, "postFilterLines": 0, "totalDuplicates": 0}, "chunkRefsFetchTime": 46839320, "chunksDownloadTime": 0, "congestionControlLatency": 0, "pipelineWrapperFilteredLines": 0, "queryReferencedStructuredMetadata": false, "totalChunksDownloaded": 0, "totalChunksRef": 0}}, "summary": {"bytesProcessedPerSecond": 0, "execTime": 0.110445, "linesProcessedPerSecond": 0, "queueTime": 0, "shards": 50, "splits": 50, "subqueries": 0, "totalBytesProcessed": 0, "totalEntriesReturned": 0, "totalLinesProcessed": 0, "totalPostFilterLines": 0, "totalStructuredMetadataBytesProcessed": 0}}}, "status": "success"}, "msg": "OK (unknown bytes)", "redirected": false, "status": 200, "transfer_encoding": "chunked", "url": "https://cloudkitty-lokistack-openstack.apps-crc.testing/api/logs/v1/cloudkitty/loki/api/v1/query_range?query=%7Bservice%3D%22cloudkitty%22%7D&start=1773862887000000000&limit=50", "vary": "Accept-Encoding"} 2026-03-19 20:06:43,307 p=37311 u=zuul n=ansible | ...ignoring 2026-03-19 20:06:43,315 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file] ********************** 2026-03-19 20:06:43,315 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:43 +0000 (0:25:12.366) 0:25:53.148 ******** 2026-03-19 20:06:43,782 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 20:06:43,790 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_dyn_basic] ************* 2026-03-19 20:06:43,791 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:43 +0000 (0:00:00.475) 0:25:53.623 ******** 2026-03-19 20:06:43,819 p=37311 u=zuul n=ansible | fatal: [localhost]: FAILED! => { "assertion": "loki_response.json.data.result | length > 0", "changed": false, "evaluated_to": false, "msg": "Query did not return all data entries. Expected 42 log entries, but Loki only returned 0" } 2026-03-19 20:06:43,819 p=37311 u=zuul n=ansible | ...ignoring 2026-03-19 20:06:43,826 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_dyn_basic] *** 2026-03-19 20:06:43,826 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:43 +0000 (0:00:00.035) 0:25:53.659 ******** 2026-03-19 20:06:44,063 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 20:06:44,072 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_dyn_basic] ********* 2026-03-19 20:06:44,072 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:44 +0000 (0:00:00.245) 0:25:53.905 ******** 2026-03-19 20:06:44,091 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost 2026-03-19 20:06:44,109 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_dyn_basic] *** 2026-03-19 20:06:44,109 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:44 +0000 (0:00:00.036) 0:25:53.942 ******** 2026-03-19 20:06:50,914 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 20:06:50,925 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_dyn_basic] *** 2026-03-19 20:06:50,925 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:50 +0000 (0:00:06.815) 0:26:00.757 ******** 2026-03-19 20:06:50,946 p=37311 u=zuul n=ansible | ok: [localhost] => { "cost_totals_by_type.stdout": "" } 2026-03-19 20:06:50,957 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_dyn_basic] *** 2026-03-19 20:06:50,958 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:50 +0000 (0:00:00.032) 0:26:00.790 ******** 2026-03-19 20:06:51,336 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 20:06:51,348 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_dyn_basic] *** 2026-03-19 20:06:51,348 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:51 +0000 (0:00:00.390) 0:26:01.181 ******** 2026-03-19 20:06:57,698 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 20:06:57,714 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_dyn_basic] *** 2026-03-19 20:06:57,715 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:57 +0000 (0:00:06.366) 0:26:07.547 ******** 2026-03-19 20:06:57,744 p=37311 u=zuul n=ansible | ok: [localhost] => { "cost_totals_summary.stdout": "+---------------------------+---------------------------+-----+------+\n| Begin | End | Qty | Rate |\n+---------------------------+---------------------------+-----+------+\n| 2026-03-01T00:00:00+00:00 | 2026-04-01T00:00:00+00:00 | 0.0 | 0.0 |\n+---------------------------+---------------------------+-----+------+" } 2026-03-19 20:06:57,753 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Check synthetic totals file exists] *************** 2026-03-19 20:06:57,753 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:57 +0000 (0:00:00.038) 0:26:07.586 ******** 2026-03-19 20:06:58,017 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 20:06:58,026 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Check loki totals file exists] ******************** 2026-03-19 20:06:58,027 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:58 +0000 (0:00:00.273) 0:26:07.859 ******** 2026-03-19 20:06:58,244 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 20:06:58,255 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Totals files exist test_dyn_basic] *********** 2026-03-19 20:06:58,255 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:58 +0000 (0:00:00.228) 0:26:08.088 ******** 2026-03-19 20:06:58,279 p=37311 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "PASSED! Required file(s) exist test_dyn_basic: \n- /home/zuul/ci-framework-data/artifacts/test_dyn_basic-synth_metrics_summary.yml\n- /home/zuul/ci-framework-data/artifacts/test_dyn_basic-loki_metrics_summary.yml \n" } 2026-03-19 20:06:58,289 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic totals file] ******************* 2026-03-19 20:06:58,289 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:58 +0000 (0:00:00.034) 0:26:08.122 ******** 2026-03-19 20:06:58,495 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 20:06:58,502 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] *********************** 2026-03-19 20:06:58,502 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:58 +0000 (0:00:00.212) 0:26:08.335 ******** 2026-03-19 20:06:58,719 p=37311 u=zuul n=ansible | ok: [localhost] 2026-03-19 20:06:58,727 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data results test_dyn_basic] *** 2026-03-19 20:06:58,727 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:58 +0000 (0:00:00.224) 0:26:08.560 ******** 2026-03-19 20:06:58,759 p=37311 u=zuul n=ansible | fatal: [localhost]: FAILED! => { "assertion": "(synth_data.content | b64decode | from_yaml).data_log == (loki_data.content | b64decode | from_yaml).data_log", "changed": false, "evaluated_to": false, "msg": "FAILED! test_dyn_basic" } 2026-03-19 20:06:58,759 p=37311 u=zuul n=ansible | ...ignoring 2026-03-19 20:06:58,765 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Cleanup after job run] **************************** 2026-03-19 20:06:58,765 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:58 +0000 (0:00:00.038) 0:26:08.598 ******** 2026-03-19 20:06:58,784 p=37311 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/cleanup_ck.yml for localhost 2026-03-19 20:06:58,794 p=37311 u=zuul n=ansible | TASK [telemetry_chargeback : Cleanup local certificates] *********************** 2026-03-19 20:06:58,795 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:58 +0000 (0:00:00.029) 0:26:08.627 ******** 2026-03-19 20:06:59,003 p=37311 u=zuul n=ansible | changed: [localhost] 2026-03-19 20:06:59,040 p=37311 u=zuul n=ansible | PLAY RECAP ********************************************************************* 2026-03-19 20:06:59,040 p=37311 u=zuul n=ansible | localhost : ok=146 changed=16 unreachable=0 failed=0 skipped=8 rescued=0 ignored=6 2026-03-19 20:06:59,094 p=37311 u=zuul n=ansible | Thursday 19 March 2026 20:06:59 +0000 (0:00:00.299) 0:26:08.927 ******** 2026-03-19 20:06:59,094 p=37311 u=zuul n=ansible | =============================================================================== 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Query Loki API -------------------------------- 1512.37s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_dyn_basic --- 6.82s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_dyn_basic --- 6.37s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Enable Cloudkitty Module (hashmap) --------------- 3.90s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Change priority for CloudKitty hashmap module ---- 3.20s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Find the current value of hashmap ---------------- 2.54s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Verify endpoint "['cloudkitty', 'rating', 'internal']" --------- 2.31s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Get admin user ID for CI ------------------------- 2.25s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Get status of all CloudKitty rating modules ------ 2.22s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Get admin project ID for CI ---------------------- 2.21s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Verify endpoint "['cloudkitty', 'rating', 'public']" ----------- 2.14s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | Gathering Facts --------------------------------------------------------- 0.97s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.72s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.51s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 0.48s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file ---------------------- 0.48s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.44s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.42s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Get Loki Public Route Host ----------------------- 0.39s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_dyn_basic --- 0.39s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.39s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.38s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-querier-http" ----------- 0.38s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.38s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.37s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.37s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-gossip-ring" ------------ 0.37s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.37s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : Get the openstack-operators project ---------------------------- 0.37s 2026-03-19 20:06:59,095 p=37311 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-index-gateway pod is Running in openstack namespace --- 0.36s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.36s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-compactor-grpc" --------- 0.35s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-index-gateway-http" ----- 0.35s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Get the openstack project -------------------------------------- 0.35s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-index-gateway-grpc" ----- 0.35s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-distributor-http" ------- 0.35s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-query-frontend-grpc" ---- 0.34s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-gateway-http" ----------- 0.34s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-ingester-http" ---------- 0.34s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-public" --------------------------- 0.34s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data "test_dyn_basic" --------- 0.34s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-ingester-grpc" ---------- 0.33s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-internal" ------------------------- 0.33s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-querier-grpc" ----------- 0.33s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-distributor pod is Running in openstack namespace --- 0.33s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | telemetry_chargeback : Extract Certificates from Openshift Secret ------- 0.33s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-query-frontend-http" ---- 0.33s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-compactor pod is Running in openstack namespace --- 0.33s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : TEST Check cloudkitty-api pod is Running in openstack namespace --- 0.33s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | telemetry_chargeback : Ensure Local Certificate Directory Exists -------- 0.33s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | telemetry_chargeback : Extract Client Certificates ---------------------- 0.32s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | telemetry_chargeback : Find test files ---------------------------------- 0.32s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-compactor-http" --------- 0.32s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-distributor-grpc" ------- 0.31s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-ingester pod is Running in openstack namespace --- 0.31s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-query-frontend pod is Running in openstack namespace --- 0.31s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | common : TEST Check cloudkitty-proc pod is Running in openstack namespace --- 0.31s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.31s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.30s 2026-03-19 20:06:59,096 p=37311 u=zuul n=ansible | telemetry_chargeback : Cleanup local certificates ----------------------- 0.30s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-querier pod is Running in openstack namespace --- 0.30s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Check synthetic totals file exists --------------- 0.27s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file "test_dyn_basic" --- 0.27s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_dyn_basic --- 0.25s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Check loki totals file exists -------------------- 0.23s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.23s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.22s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Read the synthetic totals file ------------------- 0.21s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Extract CA Bundle -------------------------------- 0.19s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | common : Run service tests ---------------------------------------------- 0.15s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | common : Run pod tests -------------------------------------------------- 0.09s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | common : Run project tests ---------------------------------------------- 0.07s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | common : Run endpoint tests --------------------------------------------- 0.05s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki "test_dyn_basic" ----- 0.05s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage " test_dyn_basic" ----- 0.04s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | common : TEST Check that cloudkitty public endpoint exists ------------- 0.04s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data results test_dyn_basic --- 0.04s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_dyn_basic --- 0.04s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Setup Loki Environment --------------------------- 0.04s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Process and Loop if files exist ------------------ 0.04s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_dyn_basic --------- 0.04s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-gateway-http service is running in openstack namespace --- 0.04s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_dyn_basic ------------- 0.04s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | common : Verify container tests ----------------------------------------- 0.03s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.03s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : TEST Totals files exist test_dyn_basic ----------- 0.03s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_dyn_basic --- 0.03s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_dyn_basic ---------------- 0.03s 2026-03-19 20:06:59,097 p=37311 u=zuul n=ansible | telemetry_chargeback : TEST Validate CloudKitty module states ----------- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-querier-http service is running in openstack namespace --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-querier-grpc service is running in openstack namespace --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : Validate Chargeback Feature deployed correctly --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : Cleanup after job run ---------------------------- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : Check that each package manifest is present -------------------- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_dyn_basic --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically "test_dyn_basic" ------- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify openstack-operators project exists ---------------- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-ingester-http service is running in openstack namespace --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : TEST Set priority for CloudKitty hashmap module --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : Run CR tests --------------------------------------------------- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : Set admin user ID for CI ------------------------- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify openstack project exists -------------------------- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-ingester-grpc service is running in openstack namespace --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-index-gateway-grpc service is running in openstack namespace --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : Expected Count test_dyn_basic -------------------- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Check that cloudkitty internal endpoint exists ----------- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-distributor-http service is running in openstack namespace --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-index-gateway-http service is running in openstack namespace --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-compactor-http service is running in openstack namespace --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-gossip-ring service is running in openstack namespace --- 0.03s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-public service is running in openstack namespace --- 0.02s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log "test_dyn_basic" ----- 0.02s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : Extract only the filenames into a clean list ----- 0.02s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : Cloudkitty debug OFF ----------------------------- 0.02s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : Debug URLs --------------------------------------- 0.02s 2026-03-19 20:06:59,098 p=37311 u=zuul n=ansible | telemetry_chargeback : Set admin project ID for CI ---------------------- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | common : Get the manifest list once ------------------------------------- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | common : Run crd tests -------------------------------------------------- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-query-frontend-grpc service is running in openstack namespace --- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-query-frontend-http service is running in openstack namespace --- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | common : Run file tests ------------------------------------------------- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-distributor-grpc service is running in openstack namespace --- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-internal service is running in openstack namespace --- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | telemetry_chargeback : Cloudkitty debug ON ------------------------------ 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-compactor-grpc service is running in openstack namespace --- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | common : Run subscription tests ----------------------------------------- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | telemetry_chargeback : Confirm Success ---------------------------------- 0.02s 2026-03-19 20:06:59,099 p=37311 u=zuul n=ansible | telemetry_chargeback : Set Derived Loki URLs ---------------------------- 0.02s