2026-02-26 18:03:40,081 p=37214 u=zuul n=ansible | [WARNING]: Found variable using reserved name: namespace 2026-02-26 18:03:40,081 p=37214 u=zuul n=ansible | PLAY [Verify all the applicable projects, endpoints, pods & services for cloudkitty] *** 2026-02-26 18:03:40,092 p=37214 u=zuul n=ansible | TASK [Gathering Facts] ********************************************************* 2026-02-26 18:03:40,092 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:40 +0000 (0:00:00.033) 0:00:00.033 ***** 2026-02-26 18:03:41,193 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:41,213 p=37214 u=zuul n=ansible | TASK [common : Run pod tests] ************************************************** 2026-02-26 18:03:41,213 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:41 +0000 (0:00:01.120) 0:00:01.153 ***** 2026-02-26 18:03:41,298 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-api) 2026-02-26 18:03:41,301 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-compactor) 2026-02-26 18:03:41,304 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-distributor) 2026-02-26 18:03:41,306 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway) 2026-02-26 18:03:41,309 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-ingester) 2026-02-26 18:03:41,311 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-querier) 2026-02-26 18:03:41,314 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend) 2026-02-26 18:03:41,316 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-proc) 2026-02-26 18:03:41,326 p=37214 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 18:03:41,326 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:41 +0000 (0:00:00.113) 0:00:01.267 ***** 2026-02-26 18:03:42,191 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:42,200 p=37214 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-api pod is Running in openstack namespace] *** 2026-02-26 18:03:42,200 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:42 +0000 (0:00:00.873) 0:00:02.140 ***** 2026-02-26 18:03:42,520 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:42,530 p=37214 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 18:03:42,530 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:42 +0000 (0:00:00.329) 0:00:02.470 ***** 2026-02-26 18:03:42,882 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:42,891 p=37214 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-compactor pod is Running in openstack namespace] *** 2026-02-26 18:03:42,892 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:42 +0000 (0:00:00.361) 0:00:02.832 ***** 2026-02-26 18:03:43,266 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:43,276 p=37214 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 18:03:43,277 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:43 +0000 (0:00:00.384) 0:00:03.217 ***** 2026-02-26 18:03:43,666 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:43,675 p=37214 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-distributor pod is Running in openstack namespace] *** 2026-02-26 18:03:43,675 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:43 +0000 (0:00:00.398) 0:00:03.615 ***** 2026-02-26 18:03:44,049 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:44,055 p=37214 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 18:03:44,055 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:44 +0000 (0:00:00.380) 0:00:03.996 ***** 2026-02-26 18:03:44,475 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:44,483 p=37214 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-index-gateway pod is Running in openstack namespace] *** 2026-02-26 18:03:44,483 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:44 +0000 (0:00:00.427) 0:00:04.424 ***** 2026-02-26 18:03:44,874 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:44,882 p=37214 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 18:03:44,882 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:44 +0000 (0:00:00.399) 0:00:04.823 ***** 2026-02-26 18:03:45,317 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:45,326 p=37214 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-ingester pod is Running in openstack namespace] *** 2026-02-26 18:03:45,326 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:45 +0000 (0:00:00.443) 0:00:05.266 ***** 2026-02-26 18:03:45,664 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:45,672 p=37214 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 18:03:45,672 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:45 +0000 (0:00:00.345) 0:00:05.612 ***** 2026-02-26 18:03:46,076 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:46,085 p=37214 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-querier pod is Running in openstack namespace] *** 2026-02-26 18:03:46,086 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:46 +0000 (0:00:00.413) 0:00:06.026 ***** 2026-02-26 18:03:46,419 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:46,427 p=37214 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 18:03:46,427 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:46 +0000 (0:00:00.341) 0:00:06.368 ***** 2026-02-26 18:03:46,812 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:46,826 p=37214 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-query-frontend pod is Running in openstack namespace] *** 2026-02-26 18:03:46,827 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:46 +0000 (0:00:00.399) 0:00:06.767 ***** 2026-02-26 18:03:47,167 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:47,174 p=37214 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 18:03:47,174 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:47 +0000 (0:00:00.347) 0:00:07.114 ***** 2026-02-26 18:03:47,583 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:47,590 p=37214 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-proc pod is Running in openstack namespace] *** 2026-02-26 18:03:47,590 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:47 +0000 (0:00:00.416) 0:00:07.530 ***** 2026-02-26 18:03:47,959 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:47,970 p=37214 u=zuul n=ansible | TASK [common : Run subscription tests] ***************************************** 2026-02-26 18:03:47,970 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:47 +0000 (0:00:00.379) 0:00:07.910 ***** 2026-02-26 18:03:47,996 p=37214 u=zuul n=ansible | skipping: [localhost] 2026-02-26 18:03:48,006 p=37214 u=zuul n=ansible | TASK [common : Run endpoint tests] ********************************************* 2026-02-26 18:03:48,007 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:48 +0000 (0:00:00.036) 0:00:07.947 ***** 2026-02-26 18:03:48,053 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/endpoint_tests.yml for localhost => (item=['cloudkitty', 'rating', 'public']) 2026-02-26 18:03:48,057 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/endpoint_tests.yml for localhost => (item=['cloudkitty', 'rating', 'internal']) 2026-02-26 18:03:48,089 p=37214 u=zuul n=ansible | TASK [common : Verify endpoint "['cloudkitty', 'rating', 'public']"] *********** 2026-02-26 18:03:48,089 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:48 +0000 (0:00:00.082) 0:00:08.029 ***** 2026-02-26 18:03:50,414 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:50,426 p=37214 u=zuul n=ansible | TASK [common : TEST Check that cloudkitty public endpoint exists] ************** 2026-02-26 18:03:50,426 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:50 +0000 (0:00:02.337) 0:00:10.367 ***** 2026-02-26 18:03:50,445 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "The endpoint cloudkitty is found." } 2026-02-26 18:03:50,456 p=37214 u=zuul n=ansible | TASK [common : Verify endpoint "['cloudkitty', 'rating', 'internal']"] ********* 2026-02-26 18:03:50,457 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:50 +0000 (0:00:00.030) 0:00:10.397 ***** 2026-02-26 18:03:52,708 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:52,716 p=37214 u=zuul n=ansible | TASK [common : TEST Check that cloudkitty internal endpoint exists] ************ 2026-02-26 18:03:52,717 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:52 +0000 (0:00:02.259) 0:00:12.657 ***** 2026-02-26 18:03:52,736 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "The endpoint cloudkitty is found." } 2026-02-26 18:03:52,744 p=37214 u=zuul n=ansible | TASK [common : Run project tests] ********************************************** 2026-02-26 18:03:52,744 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:52 +0000 (0:00:00.027) 0:00:12.684 ***** 2026-02-26 18:03:52,777 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/project_tests.yml for localhost => (item=openstack) 2026-02-26 18:03:52,779 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/project_tests.yml for localhost => (item=openstack-operators) 2026-02-26 18:03:52,804 p=37214 u=zuul n=ansible | TASK [common : Get the openstack project] ************************************** 2026-02-26 18:03:52,804 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:52 +0000 (0:00:00.059) 0:00:12.744 ***** 2026-02-26 18:03:53,164 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:53,172 p=37214 u=zuul n=ansible | TASK [common : TEST Verify openstack project exists] *************************** 2026-02-26 18:03:53,172 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:53 +0000 (0:00:00.368) 0:00:13.113 ***** 2026-02-26 18:03:53,191 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "project openstack is found." } 2026-02-26 18:03:53,199 p=37214 u=zuul n=ansible | TASK [common : Get the openstack-operators project] **************************** 2026-02-26 18:03:53,199 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:53 +0000 (0:00:00.027) 0:00:13.140 ***** 2026-02-26 18:03:53,527 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:53,534 p=37214 u=zuul n=ansible | TASK [common : TEST Verify openstack-operators project exists] ***************** 2026-02-26 18:03:53,534 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:53 +0000 (0:00:00.334) 0:00:13.474 ***** 2026-02-26 18:03:53,551 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "project openstack-operators is found." } 2026-02-26 18:03:53,558 p=37214 u=zuul n=ansible | TASK [common : Run service tests] ********************************************** 2026-02-26 18:03:53,559 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:53 +0000 (0:00:00.024) 0:00:13.499 ***** 2026-02-26 18:03:53,663 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-internal) 2026-02-26 18:03:53,665 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-compactor-grpc) 2026-02-26 18:03:53,667 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-compactor-http) 2026-02-26 18:03:53,669 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-distributor-grpc) 2026-02-26 18:03:53,672 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-distributor-http) 2026-02-26 18:03:53,674 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-gateway-http) 2026-02-26 18:03:53,676 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-gossip-ring) 2026-02-26 18:03:53,678 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway-grpc) 2026-02-26 18:03:53,680 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway-http) 2026-02-26 18:03:53,682 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-ingester-grpc) 2026-02-26 18:03:53,684 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-ingester-http) 2026-02-26 18:03:53,686 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-querier-grpc) 2026-02-26 18:03:53,689 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-querier-http) 2026-02-26 18:03:53,691 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend-grpc) 2026-02-26 18:03:53,694 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend-http) 2026-02-26 18:03:53,696 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-public) 2026-02-26 18:03:53,725 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-internal"] ************************* 2026-02-26 18:03:53,725 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:53 +0000 (0:00:00.166) 0:00:13.666 ***** 2026-02-26 18:03:54,085 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:54,095 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-internal service is running in openstack namespace] *** 2026-02-26 18:03:54,095 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:54 +0000 (0:00:00.369) 0:00:14.035 ***** 2026-02-26 18:03:54,115 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-internal is running." } 2026-02-26 18:03:54,123 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-compactor-grpc"] ********* 2026-02-26 18:03:54,123 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:54 +0000 (0:00:00.028) 0:00:14.064 ***** 2026-02-26 18:03:54,436 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:54,443 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-compactor-grpc service is running in openstack namespace] *** 2026-02-26 18:03:54,443 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:54 +0000 (0:00:00.319) 0:00:14.383 ***** 2026-02-26 18:03:54,458 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-compactor-grpc is running." } 2026-02-26 18:03:54,465 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-compactor-http"] ********* 2026-02-26 18:03:54,465 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:54 +0000 (0:00:00.022) 0:00:14.405 ***** 2026-02-26 18:03:54,787 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:54,795 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-compactor-http service is running in openstack namespace] *** 2026-02-26 18:03:54,796 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:54 +0000 (0:00:00.330) 0:00:14.736 ***** 2026-02-26 18:03:54,813 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-compactor-http is running." } 2026-02-26 18:03:54,822 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-distributor-grpc"] ******* 2026-02-26 18:03:54,822 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:54 +0000 (0:00:00.026) 0:00:14.762 ***** 2026-02-26 18:03:55,154 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:55,162 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-distributor-grpc service is running in openstack namespace] *** 2026-02-26 18:03:55,162 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:55 +0000 (0:00:00.339) 0:00:15.102 ***** 2026-02-26 18:03:55,179 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-distributor-grpc is running." } 2026-02-26 18:03:55,187 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-distributor-http"] ******* 2026-02-26 18:03:55,187 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:55 +0000 (0:00:00.025) 0:00:15.128 ***** 2026-02-26 18:03:55,537 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:55,544 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-distributor-http service is running in openstack namespace] *** 2026-02-26 18:03:55,544 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:55 +0000 (0:00:00.357) 0:00:15.485 ***** 2026-02-26 18:03:55,563 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-distributor-http is running." } 2026-02-26 18:03:55,570 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-gateway-http"] *********** 2026-02-26 18:03:55,570 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:55 +0000 (0:00:00.025) 0:00:15.511 ***** 2026-02-26 18:03:55,922 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:55,930 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-gateway-http service is running in openstack namespace] *** 2026-02-26 18:03:55,930 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:55 +0000 (0:00:00.359) 0:00:15.871 ***** 2026-02-26 18:03:55,949 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-gateway-http is running." } 2026-02-26 18:03:55,961 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-gossip-ring"] ************ 2026-02-26 18:03:55,962 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:55 +0000 (0:00:00.031) 0:00:15.902 ***** 2026-02-26 18:03:56,330 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:56,338 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-gossip-ring service is running in openstack namespace] *** 2026-02-26 18:03:56,339 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:56 +0000 (0:00:00.376) 0:00:16.279 ***** 2026-02-26 18:03:56,355 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-gossip-ring is running." } 2026-02-26 18:03:56,363 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-index-gateway-grpc"] ***** 2026-02-26 18:03:56,363 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:56 +0000 (0:00:00.024) 0:00:16.304 ***** 2026-02-26 18:03:56,706 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:56,713 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-index-gateway-grpc service is running in openstack namespace] *** 2026-02-26 18:03:56,713 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:56 +0000 (0:00:00.349) 0:00:16.653 ***** 2026-02-26 18:03:56,732 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-index-gateway-grpc is running." } 2026-02-26 18:03:56,739 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-index-gateway-http"] ***** 2026-02-26 18:03:56,740 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:56 +0000 (0:00:00.026) 0:00:16.680 ***** 2026-02-26 18:03:57,067 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:57,075 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-index-gateway-http service is running in openstack namespace] *** 2026-02-26 18:03:57,075 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:57 +0000 (0:00:00.335) 0:00:17.016 ***** 2026-02-26 18:03:57,092 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-index-gateway-http is running." } 2026-02-26 18:03:57,101 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-ingester-grpc"] ********** 2026-02-26 18:03:57,102 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:57 +0000 (0:00:00.026) 0:00:17.042 ***** 2026-02-26 18:03:57,422 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:57,429 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-ingester-grpc service is running in openstack namespace] *** 2026-02-26 18:03:57,429 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:57 +0000 (0:00:00.327) 0:00:17.370 ***** 2026-02-26 18:03:57,446 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-ingester-grpc is running." } 2026-02-26 18:03:57,454 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-ingester-http"] ********** 2026-02-26 18:03:57,454 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:57 +0000 (0:00:00.024) 0:00:17.395 ***** 2026-02-26 18:03:57,756 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:57,763 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-ingester-http service is running in openstack namespace] *** 2026-02-26 18:03:57,763 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:57 +0000 (0:00:00.309) 0:00:17.704 ***** 2026-02-26 18:03:57,779 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-ingester-http is running." } 2026-02-26 18:03:57,787 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-querier-grpc"] *********** 2026-02-26 18:03:57,787 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:57 +0000 (0:00:00.024) 0:00:17.728 ***** 2026-02-26 18:03:58,106 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:58,115 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-querier-grpc service is running in openstack namespace] *** 2026-02-26 18:03:58,115 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:58 +0000 (0:00:00.327) 0:00:18.056 ***** 2026-02-26 18:03:58,135 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-querier-grpc is running." } 2026-02-26 18:03:58,143 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-querier-http"] *********** 2026-02-26 18:03:58,143 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:58 +0000 (0:00:00.027) 0:00:18.083 ***** 2026-02-26 18:03:58,482 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:58,490 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-querier-http service is running in openstack namespace] *** 2026-02-26 18:03:58,490 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:58 +0000 (0:00:00.347) 0:00:18.430 ***** 2026-02-26 18:03:58,509 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-querier-http is running." } 2026-02-26 18:03:58,517 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-query-frontend-grpc"] **** 2026-02-26 18:03:58,517 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:58 +0000 (0:00:00.027) 0:00:18.458 ***** 2026-02-26 18:03:58,855 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:58,863 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-query-frontend-grpc service is running in openstack namespace] *** 2026-02-26 18:03:58,863 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:58 +0000 (0:00:00.345) 0:00:18.804 ***** 2026-02-26 18:03:58,880 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-query-frontend-grpc is running." } 2026-02-26 18:03:58,888 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-query-frontend-http"] **** 2026-02-26 18:03:58,888 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:58 +0000 (0:00:00.024) 0:00:18.828 ***** 2026-02-26 18:03:59,219 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:59,227 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-query-frontend-http service is running in openstack namespace] *** 2026-02-26 18:03:59,227 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:59 +0000 (0:00:00.339) 0:00:19.167 ***** 2026-02-26 18:03:59,247 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-query-frontend-http is running." } 2026-02-26 18:03:59,255 p=37214 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-public"] *************************** 2026-02-26 18:03:59,255 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:59 +0000 (0:00:00.028) 0:00:19.195 ***** 2026-02-26 18:03:59,585 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:03:59,593 p=37214 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-public service is running in openstack namespace] *** 2026-02-26 18:03:59,593 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:59 +0000 (0:00:00.338) 0:00:19.534 ***** 2026-02-26 18:03:59,610 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-public is running." } 2026-02-26 18:03:59,617 p=37214 u=zuul n=ansible | TASK [common : Run file tests] ************************************************* 2026-02-26 18:03:59,617 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:59 +0000 (0:00:00.023) 0:00:19.558 ***** 2026-02-26 18:03:59,632 p=37214 u=zuul n=ansible | skipping: [localhost] 2026-02-26 18:03:59,640 p=37214 u=zuul n=ansible | TASK [common : Get the manifest list once] ************************************* 2026-02-26 18:03:59,640 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:59 +0000 (0:00:00.022) 0:00:19.580 ***** 2026-02-26 18:03:59,656 p=37214 u=zuul n=ansible | skipping: [localhost] 2026-02-26 18:03:59,662 p=37214 u=zuul n=ansible | TASK [common : Check that each package manifest is present] ******************** 2026-02-26 18:03:59,662 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:59 +0000 (0:00:00.022) 0:00:19.603 ***** 2026-02-26 18:03:59,677 p=37214 u=zuul n=ansible | skipping: [localhost] 2026-02-26 18:03:59,684 p=37214 u=zuul n=ansible | TASK [common : Run crd tests] ************************************************** 2026-02-26 18:03:59,684 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:59 +0000 (0:00:00.021) 0:00:19.625 ***** 2026-02-26 18:03:59,699 p=37214 u=zuul n=ansible | skipping: [localhost] 2026-02-26 18:03:59,706 p=37214 u=zuul n=ansible | TASK [common : Run CR tests] *************************************************** 2026-02-26 18:03:59,706 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:59 +0000 (0:00:00.022) 0:00:19.647 ***** 2026-02-26 18:03:59,718 p=37214 u=zuul n=ansible | skipping: [localhost] 2026-02-26 18:03:59,725 p=37214 u=zuul n=ansible | TASK [common : Verify container tests] ***************************************** 2026-02-26 18:03:59,725 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:59 +0000 (0:00:00.018) 0:00:19.665 ***** 2026-02-26 18:03:59,738 p=37214 u=zuul n=ansible | skipping: [localhost] 2026-02-26 18:03:59,749 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Validate Chargeback Feature deployed correctly] *** 2026-02-26 18:03:59,749 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:59 +0000 (0:00:00.024) 0:00:19.690 ***** 2026-02-26 18:03:59,765 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/chargeback_tests.yml for localhost 2026-02-26 18:03:59,774 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Enable Cloudkitty Module (hashmap)] *************** 2026-02-26 18:03:59,774 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:03:59 +0000 (0:00:00.024) 0:00:19.714 ***** 2026-02-26 18:04:03,437 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:04:03,444 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Find the current value of hashmap] **************** 2026-02-26 18:04:03,444 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:03 +0000 (0:00:03.670) 0:00:23.385 ***** 2026-02-26 18:04:06,172 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:06,183 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Change priority for CloudKitty hashmap module] **** 2026-02-26 18:04:06,184 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:06 +0000 (0:00:02.739) 0:00:26.124 ***** 2026-02-26 18:04:09,471 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:04:09,481 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Get status of all CloudKitty rating modules] ****** 2026-02-26 18:04:09,481 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:09 +0000 (0:00:03.297) 0:00:29.421 ***** 2026-02-26 18:04:11,729 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:11,737 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Validate CloudKitty module states] *********** 2026-02-26 18:04:11,738 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:11 +0000 (0:00:02.256) 0:00:31.678 ***** 2026-02-26 18:04:11,765 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "SUCCESS: CloudKitty modules (hashmap=True) are configured correctly." } 2026-02-26 18:04:11,774 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Set priority for CloudKitty hashmap module] *** 2026-02-26 18:04:11,774 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:11 +0000 (0:00:00.036) 0:00:31.715 ***** 2026-02-26 18:04:11,806 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "SUCCESS: The hashmap priority is set to 100" } 2026-02-26 18:04:11,812 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Setup Loki Environment] *************************** 2026-02-26 18:04:11,812 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:11 +0000 (0:00:00.037) 0:00:31.753 ***** 2026-02-26 18:04:11,832 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/setup_loki_env.yml for localhost 2026-02-26 18:04:11,842 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Get Loki Public Route Host] *********************** 2026-02-26 18:04:11,842 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:11 +0000 (0:00:00.029) 0:00:31.783 ***** 2026-02-26 18:04:12,251 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:12,260 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Set Loki URLs] ************************************ 2026-02-26 18:04:12,260 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:12 +0000 (0:00:00.417) 0:00:32.200 ***** 2026-02-26 18:04:12,277 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:12,284 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Set Derived Loki URLs] **************************** 2026-02-26 18:04:12,284 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:12 +0000 (0:00:00.023) 0:00:32.224 ***** 2026-02-26 18:04:12,299 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:12,305 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Debug URLs] *************************************** 2026-02-26 18:04:12,305 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:12 +0000 (0:00:00.021) 0:00:32.246 ***** 2026-02-26 18:04:12,323 p=37214 u=zuul n=ansible | ok: [localhost] => { "msg": [ "Loki Route: https://cloudkitty-lokistack-openstack.apps-crc.testing", "Push URL: https://cloudkitty-lokistack-openstack.apps-crc.testing/api/logs/v1/cloudkitty/loki/api/v1/push", "Flush URL: https://cloudkitty-lokistack-ingester-http.openstack.svc:3100/flush", "Query URL: https://cloudkitty-lokistack-openstack.apps-crc.testing/api/logs/v1/cloudkitty/loki/api/v1/query_range" ] } 2026-02-26 18:04:12,330 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Ensure Local Certificate Directory Exists] ******** 2026-02-26 18:04:12,330 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:12 +0000 (0:00:00.025) 0:00:32.271 ***** 2026-02-26 18:04:12,664 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:04:12,673 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Extract Certificates from Openshift Secret] ******* 2026-02-26 18:04:12,673 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:12 +0000 (0:00:00.342) 0:00:32.613 ***** 2026-02-26 18:04:13,032 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:04:13,039 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Extract Client Certificates] ********************** 2026-02-26 18:04:13,039 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:13 +0000 (0:00:00.365) 0:00:32.979 ***** 2026-02-26 18:04:13,441 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:04:13,448 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Extract CA Bundle] ******************************** 2026-02-26 18:04:13,448 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:13 +0000 (0:00:00.408) 0:00:33.388 ***** 2026-02-26 18:04:13,793 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:04:13,801 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Get admin project ID for CI] ********************** 2026-02-26 18:04:13,801 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:13 +0000 (0:00:00.353) 0:00:33.742 ***** 2026-02-26 18:04:16,002 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:16,014 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Set admin project ID for CI] ********************** 2026-02-26 18:04:16,014 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:16 +0000 (0:00:02.212) 0:00:35.954 ***** 2026-02-26 18:04:16,041 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:16,051 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Get admin user ID for CI] ************************* 2026-02-26 18:04:16,051 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:16 +0000 (0:00:00.037) 0:00:35.991 ***** 2026-02-26 18:04:18,372 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:18,382 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Set admin user ID for CI] ************************* 2026-02-26 18:04:18,382 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:18 +0000 (0:00:02.330) 0:00:38.322 ***** 2026-02-26 18:04:18,405 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:18,415 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Find test files] ********************************** 2026-02-26 18:04:18,416 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:18 +0000 (0:00:00.033) 0:00:38.356 ***** 2026-02-26 18:04:18,770 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:18,781 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Extract only the filenames into a clean list] ***** 2026-02-26 18:04:18,782 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:18 +0000 (0:00:00.366) 0:00:38.722 ***** 2026-02-26 18:04:18,807 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:18,818 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Process and Loop if files exist] ****************** 2026-02-26 18:04:18,818 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:18 +0000 (0:00:00.035) 0:00:38.758 ***** 2026-02-26 18:04:18,865 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_static_basic) 2026-02-26 18:04:18,875 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_static_basic_gid) 2026-02-26 18:04:18,890 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic] *** 2026-02-26 18:04:18,890 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:18 +0000 (0:00:00.072) 0:00:38.830 ***** 2026-02-26 18:04:18,915 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost 2026-02-26 18:04:18,925 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically test_static_basic] ****** 2026-02-26 18:04:18,925 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:18 +0000 (0:00:00.034) 0:00:38.865 ***** 2026-02-26 18:04:18,947 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:18,959 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] **************** 2026-02-26 18:04:18,959 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:18 +0000 (0:00:00.034) 0:00:38.900 ***** 2026-02-26 18:04:19,297 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:19,306 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data test_static_basic] ******** 2026-02-26 18:04:19,307 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:19 +0000 (0:00:00.347) 0:00:39.247 ***** 2026-02-26 18:04:19,683 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:04:19,692 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic] *** 2026-02-26 18:04:19,692 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:19 +0000 (0:00:00.385) 0:00:39.633 ***** 2026-02-26 18:04:19,994 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:04:20,002 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] ********************** 2026-02-26 18:04:20,002 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:20 +0000 (0:00:00.309) 0:00:39.943 ***** 2026-02-26 18:04:20,021 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:20,030 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_static_basic] ************* 2026-02-26 18:04:20,030 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:20 +0000 (0:00:00.028) 0:00:39.971 ***** 2026-02-26 18:04:20,055 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost 2026-02-26 18:04:20,068 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic] *** 2026-02-26 18:04:20,069 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:20 +0000 (0:00:00.038) 0:00:40.009 ***** 2026-02-26 18:04:20,092 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost 2026-02-26 18:04:20,101 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] **************************** 2026-02-26 18:04:20,101 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:20 +0000 (0:00:00.032) 0:00:40.041 ***** 2026-02-26 18:04:20,535 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:20,545 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ******************************** 2026-02-26 18:04:20,545 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:20 +0000 (0:00:00.444) 0:00:40.486 ***** 2026-02-26 18:04:21,187 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:21,200 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] ********************************** 2026-02-26 18:04:21,200 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:21 +0000 (0:00:00.655) 0:00:41.141 ***** 2026-02-26 18:04:21,223 p=37214 u=zuul n=ansible | ok: [localhost] => { "msg": "Ingestion Successful!" } 2026-02-26 18:04:21,233 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage: test_static_basic] **** 2026-02-26 18:04:21,233 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:21 +0000 (0:00:00.032) 0:00:41.173 ***** 2026-02-26 18:04:21,259 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost 2026-02-26 18:04:21,274 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************ 2026-02-26 18:04:21,274 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:21 +0000 (0:00:00.041) 0:00:41.215 ***** 2026-02-26 18:04:21,733 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:21,742 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] ******* 2026-02-26 18:04:21,743 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:21 +0000 (0:00:00.468) 0:00:41.683 ***** 2026-02-26 18:04:21,976 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:04:21,984 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] *************** 2026-02-26 18:04:21,984 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:21 +0000 (0:00:00.241) 0:00:41.924 ***** 2026-02-26 18:04:22,559 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:04:22,566 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************ 2026-02-26 18:04:22,566 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:22 +0000 (0:00:00.582) 0:00:42.506 ***** 2026-02-26 18:04:23,049 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:04:23,056 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ****************************** 2026-02-26 18:04:23,056 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:23 +0000 (0:00:00.489) 0:00:42.996 ***** 2026-02-26 18:04:23,077 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "Ingester Memory Flushed successfully" } 2026-02-26 18:04:23,085 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki: test_static_basic] *** 2026-02-26 18:04:23,085 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:23 +0000 (0:00:00.029) 0:00:43.026 ***** 2026-02-26 18:04:23,105 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost 2026-02-26 18:04:23,122 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_static_basic] ***************** 2026-02-26 18:04:23,122 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:23 +0000 (0:00:00.037) 0:00:43.063 ***** 2026-02-26 18:04:23,142 p=37214 u=zuul n=ansible | ok: [localhost] => { "msg": "Input file has 24 data entries that Loki has to return" } 2026-02-26 18:04:23,148 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] *********************************** 2026-02-26 18:04:23,148 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:04:23 +0000 (0:00:00.025) 0:00:43.088 ***** 2026-02-26 18:22:34,198 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:22:34,210 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file test_static_basic] **** 2026-02-26 18:22:34,210 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:34 +0000 (0:18:11.061) 0:18:54.150 ***** 2026-02-26 18:22:34,231 p=37214 u=zuul n=ansible | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: re.error: unterminated character set at position 3 2026-02-26 18:22:34,231 p=37214 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": false} 2026-02-26 18:22:34,231 p=37214 u=zuul n=ansible | ...ignoring 2026-02-26 18:22:34,240 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_static_basic] ********** 2026-02-26 18:22:34,240 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:34 +0000 (0:00:00.030) 0:18:54.181 ***** 2026-02-26 18:22:34,276 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "Query returned all data entries. Input file had 24 entries and Loki returned 24" } 2026-02-26 18:22:34,284 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic] *** 2026-02-26 18:22:34,285 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:34 +0000 (0:00:00.044) 0:18:54.225 ***** 2026-02-26 18:22:34,573 p=37214 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": false, "cmd": ["python3", "/home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/files/gen_synth_loki_metrics_totals.py", "-j", "/home/zuul/ci-framework-data/artifacts/test_static_basic-loki_data.json", "-o", "/home/zuul/ci-framework-data/artifacts/test_static_basic-loki_metrics_totals.yml"], "delta": "0:00:00.050971", "end": "2026-02-26 18:22:34.533131", "msg": "non-zero return code", "rc": 1, "start": "2026-02-26 18:22:34.482160", "stderr": "", "stderr_lines": [], "stdout": "Error reading JSON file /home/zuul/ci-framework-data/artifacts/test_static_basic-loki_data.json: [Errno 2] No such file or directory: '/home/zuul/ci-framework-data/artifacts/test_static_basic-loki_data.json'", "stdout_lines": ["Error reading JSON file /home/zuul/ci-framework-data/artifacts/test_static_basic-loki_data.json: [Errno 2] No such file or directory: '/home/zuul/ci-framework-data/artifacts/test_static_basic-loki_data.json'"]} 2026-02-26 18:22:34,574 p=37214 u=zuul n=ansible | ...ignoring 2026-02-26 18:22:34,586 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_static_basic] ****** 2026-02-26 18:22:34,586 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:34 +0000 (0:00:00.301) 0:18:54.526 ***** 2026-02-26 18:22:34,611 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost 2026-02-26 18:22:34,627 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic] *** 2026-02-26 18:22:34,627 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:34 +0000 (0:00:00.041) 0:18:54.567 ***** 2026-02-26 18:22:40,999 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:22:41,012 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_static_basic] *** 2026-02-26 18:22:41,012 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:41 +0000 (0:00:06.384) 0:19:00.952 ***** 2026-02-26 18:22:41,034 p=37214 u=zuul n=ansible | ok: [localhost] => { "cost_totals_by_type": { "changed": true, "cmd": [ "oc", "rsh", "openstackclient", "openstack", "rating", "summary", "get", "-g", "type" ], "delta": "0:00:06.174982", "end": "2026-02-26 18:22:40.964805", "failed": false, "failed_when_result": false, "msg": "", "rc": 0, "start": "2026-02-26 18:22:34.789823", "stderr": "", "stderr_lines": [], "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time | End Time |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+", "stdout_lines": [ "+-----------+---------------+------+---------------------+---------------------+", "| Tenant ID | Resource Type | Rate | Begin Time | End Time |", "+-----------+---------------+------+---------------------+---------------------+", "| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |", "+-----------+---------------+------+---------------------+---------------------+" ] } } 2026-02-26 18:22:41,047 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_static_basic] *** 2026-02-26 18:22:41,047 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:41 +0000 (0:00:00.035) 0:19:00.988 ***** 2026-02-26 18:22:41,555 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:22:41,562 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic] *** 2026-02-26 18:22:41,562 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:41 +0000 (0:00:00.515) 0:19:01.503 ***** 2026-02-26 18:22:44,608 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:22:44,616 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_static_basic] *** 2026-02-26 18:22:44,616 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:44 +0000 (0:00:03.053) 0:19:04.557 ***** 2026-02-26 18:22:44,631 p=37214 u=zuul n=ansible | ok: [localhost] => { "cost_totals_summary": { "changed": true, "cmd": [ "oc", "rsh", "openstackclient", "openstack", "rating", "summary", "get" ], "delta": "0:00:02.842684", "end": "2026-02-26 18:22:44.585523", "failed": false, "failed_when_result": false, "msg": "", "rc": 0, "start": "2026-02-26 18:22:41.742839", "stderr": "", "stderr_lines": [], "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time | End Time |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+", "stdout_lines": [ "+-----------+---------------+------+---------------------+---------------------+", "| Tenant ID | Resource Type | Rate | Begin Time | End Time |", "+-----------+---------------+------+---------------------+---------------------+", "| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |", "+-----------+---------------+------+---------------------+---------------------+" ] } } 2026-02-26 18:22:44,638 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** help for openstack rating summary cmd] *** 2026-02-26 18:22:44,638 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:44 +0000 (0:00:00.021) 0:19:04.578 ***** 2026-02-26 18:22:46,535 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:22:46,549 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary get help output] *** 2026-02-26 18:22:46,550 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:46 +0000 (0:00:01.911) 0:19:06.490 ***** 2026-02-26 18:22:46,571 p=37214 u=zuul n=ansible | ok: [localhost] => { "cost_totals_h.stdout_lines": [ "usage: openstack rating summary get [-h]", " [-f {csv,df-to-csv,json,table,value,yaml}]", " [-c COLUMN]", " [--quote {all,minimal,none,nonnumeric}]", " [--noindent] [--max-width ]", " [--fit-width] [--print-empty]", " [--format-config-file FORMAT_CONFIG]", " [--sort-column SORT_COLUMN]", " [--sort-ascending | --sort-descending]", " [-t TENANT_ID] [-b BEGIN] [-e END]", " [-s SERVICE] [-g GROUPBY [GROUPBY ...]]", " [-a]", "", "Get a summary report.", "", "optional arguments:", " -h, --help show this help message and exit", " -t TENANT_ID, --tenant-id TENANT_ID", " Tenant id.", " -b BEGIN, --begin BEGIN", " Begin timestamp.", " -e END, --end END End timestamp.", " -s SERVICE, --service SERVICE", " Service Type.", " -g GROUPBY [GROUPBY ...], --groupby GROUPBY [GROUPBY ...]", " Fields to group by, space-separated. (res_type and", " tenant_id are supported for now)", " -a, --all-tenants Allows to get summary from all tenants (admin only).", " Defaults to False.", "", "output formatters:", " output formatter options", "", " -f {csv,df-to-csv,json,table,value,yaml}, --format {csv,df-to-csv,json,table,value,yaml}", " the output format, defaults to table", " -c COLUMN, --column COLUMN", " specify the column(s) to include, can be repeated to", " show multiple columns", " --sort-column SORT_COLUMN", " specify the column(s) to sort the data (columns", " specified first have a priority, non-existing columns", " are ignored), can be repeated", " --sort-ascending sort the column(s) in ascending order", " --sort-descending sort the column(s) in descending order", "", "CSV Formatter:", " --quote {all,minimal,none,nonnumeric}", " when to include quotes, defaults to nonnumeric", "", "json formatter:", " --noindent whether to disable indenting the JSON", "", "table formatter:", " --max-width ", " Maximum display width, <1 to disable. You can also use", " the CLIFF_MAX_TERM_WIDTH environment variable, but the", " parameter takes precedence.", " --fit-width Fit the table to the display width. Implied if --max-", " width greater than 0. Set the environment variable", " CLIFF_FIT_WIDTH=1 to always enable", " --print-empty Print empty table if there is no data to show.", "", "dataframe-to-csv formatter:", " --format-config-file FORMAT_CONFIG", " Config file for the dict-to-csv formatter", "", "This command is provided by the python-cloudkittyclient plugin." ] } 2026-02-26 18:22:46,578 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic data file] ********************* 2026-02-26 18:22:46,579 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:46 +0000 (0:00:00.028) 0:19:06.519 ***** 2026-02-26 18:22:46,773 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:22:46,780 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] *********************** 2026-02-26 18:22:46,780 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:46 +0000 (0:00:00.201) 0:19:06.721 ***** 2026-02-26 18:22:46,972 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:22:46,979 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic] *** 2026-02-26 18:22:46,979 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:46 +0000 (0:00:00.199) 0:19:06.920 ***** 2026-02-26 18:22:47,017 p=37214 u=zuul n=ansible | fatal: [localhost]: FAILED! => { "assertion": "(synth_data.content | b64decode | from_yaml) == (loki_data.content | b64decode | from_yaml)", "changed": false, "evaluated_to": false, "msg": "THE FILES ARE DIFFERENT!\nPath A: \"/home/zuul/ci-framework-data/artifacts/test_static_basic-synth_data.json\"\nPath B: \"/home/zuul/ci-framework-data/artifacts/test_static_basic-loki_totals.yml\"\n" } 2026-02-26 18:22:47,017 p=37214 u=zuul n=ansible | ...ignoring 2026-02-26 18:22:47,029 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic_gid] *** 2026-02-26 18:22:47,029 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:47 +0000 (0:00:00.050) 0:19:06.970 ***** 2026-02-26 18:22:47,049 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost 2026-02-26 18:22:47,059 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically test_static_basic_gid] *** 2026-02-26 18:22:47,059 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:47 +0000 (0:00:00.029) 0:19:06.999 ***** 2026-02-26 18:22:47,076 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:22:47,083 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] **************** 2026-02-26 18:22:47,083 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:47 +0000 (0:00:00.023) 0:19:07.023 ***** 2026-02-26 18:22:47,273 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:22:47,289 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data test_static_basic_gid] **** 2026-02-26 18:22:47,289 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:47 +0000 (0:00:00.206) 0:19:07.230 ***** 2026-02-26 18:22:47,597 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:22:47,605 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic_gid] *** 2026-02-26 18:22:47,605 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:47 +0000 (0:00:00.315) 0:19:07.545 ***** 2026-02-26 18:22:47,906 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:22:47,913 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] ********************** 2026-02-26 18:22:47,913 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:47 +0000 (0:00:00.308) 0:19:07.854 ***** 2026-02-26 18:22:47,928 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:22:47,936 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_static_basic_gid] ********* 2026-02-26 18:22:47,936 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:47 +0000 (0:00:00.022) 0:19:07.876 ***** 2026-02-26 18:22:47,955 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost 2026-02-26 18:22:47,968 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic_gid] *** 2026-02-26 18:22:47,968 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:47 +0000 (0:00:00.032) 0:19:07.909 ***** 2026-02-26 18:22:47,987 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost 2026-02-26 18:22:47,995 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] **************************** 2026-02-26 18:22:47,995 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:47 +0000 (0:00:00.026) 0:19:07.935 ***** 2026-02-26 18:22:48,215 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:22:48,222 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ******************************** 2026-02-26 18:22:48,222 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:48 +0000 (0:00:00.226) 0:19:08.162 ***** 2026-02-26 18:22:48,580 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:22:48,588 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] ********************************** 2026-02-26 18:22:48,589 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:48 +0000 (0:00:00.366) 0:19:08.529 ***** 2026-02-26 18:22:48,602 p=37214 u=zuul n=ansible | ok: [localhost] => { "msg": "Ingestion Successful!" } 2026-02-26 18:22:48,609 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage: test_static_basic_gid] *** 2026-02-26 18:22:48,609 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:48 +0000 (0:00:00.020) 0:19:08.549 ***** 2026-02-26 18:22:48,626 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost 2026-02-26 18:22:48,640 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************ 2026-02-26 18:22:48,641 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:48 +0000 (0:00:00.031) 0:19:08.581 ***** 2026-02-26 18:22:49,074 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:22:49,088 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] ******* 2026-02-26 18:22:49,088 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:49 +0000 (0:00:00.447) 0:19:09.028 ***** 2026-02-26 18:22:49,327 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:22:49,334 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] *************** 2026-02-26 18:22:49,334 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:49 +0000 (0:00:00.246) 0:19:09.274 ***** 2026-02-26 18:22:50,832 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:22:50,848 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************ 2026-02-26 18:22:50,848 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:50 +0000 (0:00:01.513) 0:19:10.788 ***** 2026-02-26 18:22:51,384 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:22:51,396 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ****************************** 2026-02-26 18:22:51,396 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:51 +0000 (0:00:00.547) 0:19:11.336 ***** 2026-02-26 18:22:51,415 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "Ingester Memory Flushed successfully" } 2026-02-26 18:22:51,423 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki: test_static_basic_gid] *** 2026-02-26 18:22:51,423 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:51 +0000 (0:00:00.027) 0:19:11.364 ***** 2026-02-26 18:22:51,442 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost 2026-02-26 18:22:51,459 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_static_basic_gid] ************* 2026-02-26 18:22:51,459 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:51 +0000 (0:00:00.035) 0:19:11.399 ***** 2026-02-26 18:22:51,472 p=37214 u=zuul n=ansible | ok: [localhost] => { "msg": "Input file has 24 data entries that Loki has to return" } 2026-02-26 18:22:51,478 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] *********************************** 2026-02-26 18:22:51,478 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:51 +0000 (0:00:00.019) 0:19:11.419 ***** 2026-02-26 18:22:51,896 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:22:51,909 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file test_static_basic_gid] *** 2026-02-26 18:22:51,909 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:51 +0000 (0:00:00.430) 0:19:11.849 ***** 2026-02-26 18:22:51,925 p=37214 u=zuul n=ansible | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: re.error: unterminated character set at position 3 2026-02-26 18:22:51,925 p=37214 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": false} 2026-02-26 18:22:51,925 p=37214 u=zuul n=ansible | ...ignoring 2026-02-26 18:22:51,935 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_static_basic_gid] ****** 2026-02-26 18:22:51,935 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:51 +0000 (0:00:00.026) 0:19:11.876 ***** 2026-02-26 18:22:51,967 p=37214 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "Query returned all data entries. Input file had 24 entries and Loki returned 24" } 2026-02-26 18:22:51,979 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic_gid] *** 2026-02-26 18:22:51,979 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:51 +0000 (0:00:00.043) 0:19:11.919 ***** 2026-02-26 18:22:52,251 p=37214 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": false, "cmd": ["python3", "/home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/files/gen_synth_loki_metrics_totals.py", "-j", "/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_data.json", "-o", "/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_metrics_totals.yml"], "delta": "0:00:00.067879", "end": "2026-02-26 18:22:52.226907", "msg": "non-zero return code", "rc": 1, "start": "2026-02-26 18:22:52.159028", "stderr": "", "stderr_lines": [], "stdout": "Error reading JSON file /home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_data.json: [Errno 2] No such file or directory: '/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_data.json'", "stdout_lines": ["Error reading JSON file /home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_data.json: [Errno 2] No such file or directory: '/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_data.json'"]} 2026-02-26 18:22:52,251 p=37214 u=zuul n=ansible | ...ignoring 2026-02-26 18:22:52,259 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_static_basic_gid] *** 2026-02-26 18:22:52,259 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:52 +0000 (0:00:00.280) 0:19:12.200 ***** 2026-02-26 18:22:52,280 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost 2026-02-26 18:22:52,296 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic_gid] *** 2026-02-26 18:22:52,297 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:52 +0000 (0:00:00.037) 0:19:12.237 ***** 2026-02-26 18:22:55,298 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:22:55,310 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_static_basic_gid] *** 2026-02-26 18:22:55,310 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:55 +0000 (0:00:03.013) 0:19:15.250 ***** 2026-02-26 18:22:55,327 p=37214 u=zuul n=ansible | ok: [localhost] => { "cost_totals_by_type": { "changed": true, "cmd": [ "oc", "rsh", "openstackclient", "openstack", "rating", "summary", "get", "-g", "type" ], "delta": "0:00:02.793677", "end": "2026-02-26 18:22:55.268740", "failed": false, "failed_when_result": false, "msg": "", "rc": 0, "start": "2026-02-26 18:22:52.475063", "stderr": "", "stderr_lines": [], "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time | End Time |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+", "stdout_lines": [ "+-----------+---------------+------+---------------------+---------------------+", "| Tenant ID | Resource Type | Rate | Begin Time | End Time |", "+-----------+---------------+------+---------------------+---------------------+", "| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |", "+-----------+---------------+------+---------------------+---------------------+" ] } } 2026-02-26 18:22:55,338 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_static_basic_gid] *** 2026-02-26 18:22:55,338 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:55 +0000 (0:00:00.027) 0:19:15.278 ***** 2026-02-26 18:22:55,679 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:22:55,687 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic_gid] *** 2026-02-26 18:22:55,687 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:55 +0000 (0:00:00.348) 0:19:15.627 ***** 2026-02-26 18:22:58,449 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:22:58,461 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_static_basic_gid] *** 2026-02-26 18:22:58,461 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:58 +0000 (0:00:02.774) 0:19:18.401 ***** 2026-02-26 18:22:58,482 p=37214 u=zuul n=ansible | ok: [localhost] => { "cost_totals_summary": { "changed": true, "cmd": [ "oc", "rsh", "openstackclient", "openstack", "rating", "summary", "get" ], "delta": "0:00:02.575987", "end": "2026-02-26 18:22:58.416877", "failed": false, "failed_when_result": false, "msg": "", "rc": 0, "start": "2026-02-26 18:22:55.840890", "stderr": "", "stderr_lines": [], "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time | End Time |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+", "stdout_lines": [ "+-----------+---------------+------+---------------------+---------------------+", "| Tenant ID | Resource Type | Rate | Begin Time | End Time |", "+-----------+---------------+------+---------------------+---------------------+", "| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |", "+-----------+---------------+------+---------------------+---------------------+" ] } } 2026-02-26 18:22:58,492 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** help for openstack rating summary cmd] *** 2026-02-26 18:22:58,493 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:22:58 +0000 (0:00:00.031) 0:19:18.433 ***** 2026-02-26 18:23:00,389 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:23:00,397 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary get help output] *** 2026-02-26 18:23:00,397 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:23:00 +0000 (0:00:01.904) 0:19:20.337 ***** 2026-02-26 18:23:00,412 p=37214 u=zuul n=ansible | ok: [localhost] => { "cost_totals_h.stdout_lines": [ "usage: openstack rating summary get [-h]", " [-f {csv,df-to-csv,json,table,value,yaml}]", " [-c COLUMN]", " [--quote {all,minimal,none,nonnumeric}]", " [--noindent] [--max-width ]", " [--fit-width] [--print-empty]", " [--format-config-file FORMAT_CONFIG]", " [--sort-column SORT_COLUMN]", " [--sort-ascending | --sort-descending]", " [-t TENANT_ID] [-b BEGIN] [-e END]", " [-s SERVICE] [-g GROUPBY [GROUPBY ...]]", " [-a]", "", "Get a summary report.", "", "optional arguments:", " -h, --help show this help message and exit", " -t TENANT_ID, --tenant-id TENANT_ID", " Tenant id.", " -b BEGIN, --begin BEGIN", " Begin timestamp.", " -e END, --end END End timestamp.", " -s SERVICE, --service SERVICE", " Service Type.", " -g GROUPBY [GROUPBY ...], --groupby GROUPBY [GROUPBY ...]", " Fields to group by, space-separated. (res_type and", " tenant_id are supported for now)", " -a, --all-tenants Allows to get summary from all tenants (admin only).", " Defaults to False.", "", "output formatters:", " output formatter options", "", " -f {csv,df-to-csv,json,table,value,yaml}, --format {csv,df-to-csv,json,table,value,yaml}", " the output format, defaults to table", " -c COLUMN, --column COLUMN", " specify the column(s) to include, can be repeated to", " show multiple columns", " --sort-column SORT_COLUMN", " specify the column(s) to sort the data (columns", " specified first have a priority, non-existing columns", " are ignored), can be repeated", " --sort-ascending sort the column(s) in ascending order", " --sort-descending sort the column(s) in descending order", "", "CSV Formatter:", " --quote {all,minimal,none,nonnumeric}", " when to include quotes, defaults to nonnumeric", "", "json formatter:", " --noindent whether to disable indenting the JSON", "", "table formatter:", " --max-width ", " Maximum display width, <1 to disable. You can also use", " the CLIFF_MAX_TERM_WIDTH environment variable, but the", " parameter takes precedence.", " --fit-width Fit the table to the display width. Implied if --max-", " width greater than 0. Set the environment variable", " CLIFF_FIT_WIDTH=1 to always enable", " --print-empty Print empty table if there is no data to show.", "", "dataframe-to-csv formatter:", " --format-config-file FORMAT_CONFIG", " Config file for the dict-to-csv formatter", "", "This command is provided by the python-cloudkittyclient plugin." ] } 2026-02-26 18:23:00,420 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic data file] ********************* 2026-02-26 18:23:00,420 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:23:00 +0000 (0:00:00.022) 0:19:20.360 ***** 2026-02-26 18:23:00,597 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:23:00,604 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] *********************** 2026-02-26 18:23:00,604 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:23:00 +0000 (0:00:00.184) 0:19:20.544 ***** 2026-02-26 18:23:00,787 p=37214 u=zuul n=ansible | ok: [localhost] 2026-02-26 18:23:00,795 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic_gid] *** 2026-02-26 18:23:00,795 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:23:00 +0000 (0:00:00.190) 0:19:20.735 ***** 2026-02-26 18:23:00,813 p=37214 u=zuul n=ansible | fatal: [localhost]: FAILED! => { "assertion": "(synth_data.content | b64decode | from_yaml) == (loki_data.content | b64decode | from_yaml)", "changed": false, "evaluated_to": false, "msg": "THE FILES ARE DIFFERENT!\nPath A: \"/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-synth_data.json\"\nPath B: \"/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_totals.yml\"\n" } 2026-02-26 18:23:00,813 p=37214 u=zuul n=ansible | ...ignoring 2026-02-26 18:23:00,821 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Cleanup after job run] **************************** 2026-02-26 18:23:00,821 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:23:00 +0000 (0:00:00.026) 0:19:20.762 ***** 2026-02-26 18:23:00,837 p=37214 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/cleanup_ck.yml for localhost 2026-02-26 18:23:00,850 p=37214 u=zuul n=ansible | TASK [telemetry_chargeback : Cleanup local certificates] *********************** 2026-02-26 18:23:00,850 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:23:00 +0000 (0:00:00.028) 0:19:20.791 ***** 2026-02-26 18:23:01,039 p=37214 u=zuul n=ansible | changed: [localhost] 2026-02-26 18:23:01,065 p=37214 u=zuul n=ansible | PLAY RECAP ********************************************************************* 2026-02-26 18:23:01,065 p=37214 u=zuul n=ansible | localhost : ok=179 changed=23 unreachable=0 failed=0 skipped=7 rescued=0 ignored=6 2026-02-26 18:23:01,072 p=37214 u=zuul n=ansible | Thursday 26 February 2026 18:23:01 +0000 (0:00:00.221) 0:19:21.012 ***** 2026-02-26 18:23:01,072 p=37214 u=zuul n=ansible | =============================================================================== 2026-02-26 18:23:01,072 p=37214 u=zuul n=ansible | telemetry_chargeback : Query Loki API -------------------------------- 1091.06s 2026-02-26 18:23:01,072 p=37214 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic --- 6.38s 2026-02-26 18:23:01,072 p=37214 u=zuul n=ansible | telemetry_chargeback : Enable Cloudkitty Module (hashmap) --------------- 3.67s 2026-02-26 18:23:01,072 p=37214 u=zuul n=ansible | telemetry_chargeback : Change priority for CloudKitty hashmap module ---- 3.30s 2026-02-26 18:23:01,072 p=37214 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic --- 3.05s 2026-02-26 18:23:01,072 p=37214 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic_gid --- 3.01s 2026-02-26 18:23:01,072 p=37214 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic_gid --- 2.77s 2026-02-26 18:23:01,072 p=37214 u=zuul n=ansible | telemetry_chargeback : Find the current value of hashmap ---------------- 2.74s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Verify endpoint "['cloudkitty', 'rating', 'public']" ----------- 2.34s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Get admin user ID for CI ------------------------- 2.33s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Verify endpoint "['cloudkitty', 'rating', 'internal']" --------- 2.26s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Get status of all CloudKitty rating modules ------ 2.26s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Get admin project ID for CI ---------------------- 2.21s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : **INFO** help for openstack rating summary cmd --- 1.91s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : **INFO** help for openstack rating summary cmd --- 1.90s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 1.51s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | Gathering Facts --------------------------------------------------------- 1.12s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.87s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.66s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 0.58s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.55s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_static_basic --- 0.52s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.49s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.47s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.45s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.44s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.44s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Query Loki API ----------------------------------- 0.43s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.43s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Get Loki Public Route Host ----------------------- 0.42s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.42s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.41s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Extract Client Certificates ---------------------- 0.41s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.40s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-index-gateway pod is Running in openstack namespace --- 0.40s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.40s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data test_static_basic -------- 0.39s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-compactor pod is Running in openstack namespace --- 0.38s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-distributor pod is Running in openstack namespace --- 0.38s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : TEST Check cloudkitty-proc pod is Running in openstack namespace --- 0.38s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-gossip-ring" ------------ 0.38s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-internal" ------------------------- 0.37s 2026-02-26 18:23:01,073 p=37214 u=zuul n=ansible | common : Get the openstack project -------------------------------------- 0.37s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.37s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Find test files ---------------------------------- 0.37s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Extract Certificates from Openshift Secret ------- 0.37s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.36s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-gateway-http" ----------- 0.36s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-distributor-http" ------- 0.36s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Extract CA Bundle -------------------------------- 0.35s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-index-gateway-grpc" ----- 0.35s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_static_basic_gid --- 0.35s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-querier-http" ----------- 0.35s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.35s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-query-frontend pod is Running in openstack namespace --- 0.35s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-query-frontend-grpc" ---- 0.35s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-ingester pod is Running in openstack namespace --- 0.35s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Ensure Local Certificate Directory Exists -------- 0.34s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-querier pod is Running in openstack namespace --- 0.34s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-distributor-grpc" ------- 0.34s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-query-frontend-http" ---- 0.34s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-public" --------------------------- 0.34s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-index-gateway-http" ----- 0.34s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Get the openstack-operators project ---------------------------- 0.33s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-compactor-http" --------- 0.33s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : TEST Check cloudkitty-api pod is Running in openstack namespace --- 0.33s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-querier-grpc" ----------- 0.33s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-ingester-grpc" ---------- 0.33s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-compactor-grpc" --------- 0.32s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data test_static_basic_gid ---- 0.32s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic --- 0.31s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-ingester-http" ---------- 0.31s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic_gid --- 0.31s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic --- 0.30s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic_gid --- 0.28s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.25s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.24s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.23s 2026-02-26 18:23:01,074 p=37214 u=zuul n=ansible | telemetry_chargeback : Cleanup local certificates ----------------------- 0.22s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.21s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Read the synthetic data file --------------------- 0.20s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.20s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.19s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Read the synthetic data file --------------------- 0.18s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | common : Run service tests ---------------------------------------------- 0.17s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | common : Run pod tests -------------------------------------------------- 0.11s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | common : Run endpoint tests --------------------------------------------- 0.08s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Process and Loop if files exist ------------------ 0.07s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | common : Run project tests ---------------------------------------------- 0.06s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic --- 0.05s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_static_basic ---------- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_static_basic_gid ------ 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage: test_static_basic ---- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_static_basic ------ 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_static_basic ------------- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : TEST Set priority for CloudKitty hashmap module --- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_static_basic_gid --- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Set admin project ID for CI ---------------------- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki: test_static_basic --- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | common : Run subscription tests ----------------------------------------- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : TEST Validate CloudKitty module states ----------- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Extract only the filenames into a clean list ----- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki: test_static_basic_gid --- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_static_basic --- 0.04s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic --- 0.03s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically test_static_basic ------ 0.03s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Set admin user ID for CI ------------------------- 0.03s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Confirm Success ---------------------------------- 0.03s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_static_basic_gid --------- 0.03s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic --- 0.03s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_static_basic_gid --- 0.03s 2026-02-26 18:23:01,075 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-gateway-http service is running in openstack namespace --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage: test_static_basic_gid --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file test_static_basic ---- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Check that cloudkitty public endpoint exists ------------- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Setup Loki Environment --------------------------- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic_gid --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Cleanup after job run ---------------------------- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary get help output --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-internal service is running in openstack namespace --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-query-frontend-http service is running in openstack namespace --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_static_basic_gid --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Check that cloudkitty internal endpoint exists ----------- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-querier-grpc service is running in openstack namespace --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-querier-http service is running in openstack namespace --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify openstack project exists -------------------------- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic_gid --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-compactor-http service is running in openstack namespace --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-index-gateway-grpc service is running in openstack namespace --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic_gid --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-index-gateway-http service is running in openstack namespace --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file test_static_basic_gid --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-distributor-http service is running in openstack namespace --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Expected Count test_static_basic ----------------- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-distributor-grpc service is running in openstack namespace --- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Debug URLs --------------------------------------- 0.03s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-ingester-grpc service is running in openstack namespace --- 0.02s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-gossip-ring service is running in openstack namespace --- 0.02s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify openstack-operators project exists ---------------- 0.02s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-query-frontend-grpc service is running in openstack namespace --- 0.02s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | telemetry_chargeback : Validate Chargeback Feature deployed correctly --- 0.02s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : Verify container tests ----------------------------------------- 0.02s 2026-02-26 18:23:01,076 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-ingester-http service is running in openstack namespace --- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically test_static_basic_gid --- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-public service is running in openstack namespace --- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | telemetry_chargeback : Set Loki URLs ------------------------------------ 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | common : Run file tests ------------------------------------------------- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary get help output --- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | common : Get the manifest list once ------------------------------------- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-compactor-grpc service is running in openstack namespace --- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | common : Run crd tests -------------------------------------------------- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | common : Check that each package manifest is present -------------------- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_static_basic --- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | telemetry_chargeback : Set Derived Loki URLs ---------------------------- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | telemetry_chargeback : Confirm Success ---------------------------------- 0.02s 2026-02-26 18:23:01,077 p=37214 u=zuul n=ansible | telemetry_chargeback : Expected Count test_static_basic_gid ------------- 0.02s