2026-02-26 20:39:09,296 p=37229 u=zuul n=ansible | [WARNING]: Found variable using reserved name: namespace 2026-02-26 20:39:09,296 p=37229 u=zuul n=ansible | PLAY [Verify all the applicable projects, endpoints, pods & services for cloudkitty] *** 2026-02-26 20:39:09,305 p=37229 u=zuul n=ansible | TASK [Gathering Facts] ********************************************************* 2026-02-26 20:39:09,305 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:09 +0000 (0:00:00.024) 0:00:00.024 ***** 2026-02-26 20:39:10,246 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:10,271 p=37229 u=zuul n=ansible | TASK [common : Run pod tests] ************************************************** 2026-02-26 20:39:10,272 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:10 +0000 (0:00:00.966) 0:00:00.990 ***** 2026-02-26 20:39:10,344 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-api) 2026-02-26 20:39:10,347 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-compactor) 2026-02-26 20:39:10,350 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-distributor) 2026-02-26 20:39:10,352 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway) 2026-02-26 20:39:10,354 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-ingester) 2026-02-26 20:39:10,357 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-querier) 2026-02-26 20:39:10,359 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend) 2026-02-26 20:39:10,362 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-proc) 2026-02-26 20:39:10,371 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 20:39:10,371 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:10 +0000 (0:00:00.099) 0:00:01.090 ***** 2026-02-26 20:39:10,814 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:10,826 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-api pod is Running in openstack namespace] *** 2026-02-26 20:39:10,826 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:10 +0000 (0:00:00.454) 0:00:01.545 ***** 2026-02-26 20:39:11,203 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:11,211 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 20:39:11,211 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:11 +0000 (0:00:00.385) 0:00:01.930 ***** 2026-02-26 20:39:11,617 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:11,628 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-compactor pod is Running in openstack namespace] *** 2026-02-26 20:39:11,628 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:11 +0000 (0:00:00.416) 0:00:02.347 ***** 2026-02-26 20:39:12,028 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:12,038 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 20:39:12,038 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:12 +0000 (0:00:00.410) 0:00:02.757 ***** 2026-02-26 20:39:12,397 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:12,406 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-distributor pod is Running in openstack namespace] *** 2026-02-26 20:39:12,406 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:12 +0000 (0:00:00.367) 0:00:03.125 ***** 2026-02-26 20:39:12,744 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:12,752 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 20:39:12,752 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:12 +0000 (0:00:00.345) 0:00:03.471 ***** 2026-02-26 20:39:13,140 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:13,148 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-index-gateway pod is Running in openstack namespace] *** 2026-02-26 20:39:13,148 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:13 +0000 (0:00:00.395) 0:00:03.867 ***** 2026-02-26 20:39:13,496 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:13,503 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 20:39:13,504 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:13 +0000 (0:00:00.355) 0:00:04.222 ***** 2026-02-26 20:39:13,880 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:13,891 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-ingester pod is Running in openstack namespace] *** 2026-02-26 20:39:13,891 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:13 +0000 (0:00:00.387) 0:00:04.610 ***** 2026-02-26 20:39:14,233 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:14,241 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 20:39:14,241 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:14 +0000 (0:00:00.349) 0:00:04.960 ***** 2026-02-26 20:39:14,612 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:14,619 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-querier pod is Running in openstack namespace] *** 2026-02-26 20:39:14,619 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:14 +0000 (0:00:00.378) 0:00:05.338 ***** 2026-02-26 20:39:14,932 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:14,942 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 20:39:14,942 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:14 +0000 (0:00:00.322) 0:00:05.661 ***** 2026-02-26 20:39:15,310 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:15,317 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-query-frontend pod is Running in openstack namespace] *** 2026-02-26 20:39:15,318 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:15 +0000 (0:00:00.375) 0:00:06.036 ***** 2026-02-26 20:39:15,670 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:15,677 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ******************************** 2026-02-26 20:39:15,678 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:15 +0000 (0:00:00.359) 0:00:06.396 ***** 2026-02-26 20:39:16,047 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:16,055 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-proc pod is Running in openstack namespace] *** 2026-02-26 20:39:16,055 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:16 +0000 (0:00:00.377) 0:00:06.774 ***** 2026-02-26 20:39:16,337 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:16,344 p=37229 u=zuul n=ansible | TASK [common : Run subscription tests] ***************************************** 2026-02-26 20:39:16,344 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:16 +0000 (0:00:00.289) 0:00:07.063 ***** 2026-02-26 20:39:16,359 p=37229 u=zuul n=ansible | skipping: [localhost] 2026-02-26 20:39:16,365 p=37229 u=zuul n=ansible | TASK [common : Run endpoint tests] ********************************************* 2026-02-26 20:39:16,366 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:16 +0000 (0:00:00.021) 0:00:07.084 ***** 2026-02-26 20:39:16,394 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/endpoint_tests.yml for localhost => (item=['cloudkitty', 'rating', 'public']) 2026-02-26 20:39:16,397 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/endpoint_tests.yml for localhost => (item=['cloudkitty', 'rating', 'internal']) 2026-02-26 20:39:16,417 p=37229 u=zuul n=ansible | TASK [common : Verify endpoint "['cloudkitty', 'rating', 'public']"] *********** 2026-02-26 20:39:16,418 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:16 +0000 (0:00:00.051) 0:00:07.136 ***** 2026-02-26 20:39:18,886 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:18,922 p=37229 u=zuul n=ansible | TASK [common : TEST Check that cloudkitty public endpoint exists] ************** 2026-02-26 20:39:18,922 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:18 +0000 (0:00:02.504) 0:00:09.641 ***** 2026-02-26 20:39:18,941 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "The endpoint cloudkitty is found." } 2026-02-26 20:39:18,949 p=37229 u=zuul n=ansible | TASK [common : Verify endpoint "['cloudkitty', 'rating', 'internal']"] ********* 2026-02-26 20:39:18,949 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:18 +0000 (0:00:00.026) 0:00:09.668 ***** 2026-02-26 20:39:21,064 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:21,075 p=37229 u=zuul n=ansible | TASK [common : TEST Check that cloudkitty internal endpoint exists] ************ 2026-02-26 20:39:21,075 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:21 +0000 (0:00:02.125) 0:00:11.794 ***** 2026-02-26 20:39:21,094 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "The endpoint cloudkitty is found." } 2026-02-26 20:39:21,103 p=37229 u=zuul n=ansible | TASK [common : Run project tests] ********************************************** 2026-02-26 20:39:21,103 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:21 +0000 (0:00:00.028) 0:00:11.822 ***** 2026-02-26 20:39:21,142 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/project_tests.yml for localhost => (item=openstack) 2026-02-26 20:39:21,145 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/project_tests.yml for localhost => (item=openstack-operators) 2026-02-26 20:39:21,181 p=37229 u=zuul n=ansible | TASK [common : Get the openstack project] ************************************** 2026-02-26 20:39:21,181 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:21 +0000 (0:00:00.077) 0:00:11.900 ***** 2026-02-26 20:39:21,518 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:21,524 p=37229 u=zuul n=ansible | TASK [common : TEST Verify openstack project exists] *************************** 2026-02-26 20:39:21,525 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:21 +0000 (0:00:00.343) 0:00:12.243 ***** 2026-02-26 20:39:21,543 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "project openstack is found." } 2026-02-26 20:39:21,551 p=37229 u=zuul n=ansible | TASK [common : Get the openstack-operators project] **************************** 2026-02-26 20:39:21,551 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:21 +0000 (0:00:00.026) 0:00:12.270 ***** 2026-02-26 20:39:21,977 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:21,988 p=37229 u=zuul n=ansible | TASK [common : TEST Verify openstack-operators project exists] ***************** 2026-02-26 20:39:21,989 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:21 +0000 (0:00:00.437) 0:00:12.707 ***** 2026-02-26 20:39:22,015 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "project openstack-operators is found." } 2026-02-26 20:39:22,023 p=37229 u=zuul n=ansible | TASK [common : Run service tests] ********************************************** 2026-02-26 20:39:22,024 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:22 +0000 (0:00:00.035) 0:00:12.742 ***** 2026-02-26 20:39:22,149 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-internal) 2026-02-26 20:39:22,152 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-compactor-grpc) 2026-02-26 20:39:22,156 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-compactor-http) 2026-02-26 20:39:22,160 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-distributor-grpc) 2026-02-26 20:39:22,163 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-distributor-http) 2026-02-26 20:39:22,167 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-gateway-http) 2026-02-26 20:39:22,171 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-gossip-ring) 2026-02-26 20:39:22,175 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway-grpc) 2026-02-26 20:39:22,178 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway-http) 2026-02-26 20:39:22,181 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-ingester-grpc) 2026-02-26 20:39:22,183 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-ingester-http) 2026-02-26 20:39:22,185 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-querier-grpc) 2026-02-26 20:39:22,187 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-querier-http) 2026-02-26 20:39:22,189 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend-grpc) 2026-02-26 20:39:22,191 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend-http) 2026-02-26 20:39:22,193 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-public) 2026-02-26 20:39:22,218 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-internal"] ************************* 2026-02-26 20:39:22,218 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:22 +0000 (0:00:00.194) 0:00:12.937 ***** 2026-02-26 20:39:22,645 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:22,653 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-internal service is running in openstack namespace] *** 2026-02-26 20:39:22,654 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:22 +0000 (0:00:00.435) 0:00:13.372 ***** 2026-02-26 20:39:22,682 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-internal is running." } 2026-02-26 20:39:22,690 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-compactor-grpc"] ********* 2026-02-26 20:39:22,690 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:22 +0000 (0:00:00.036) 0:00:13.409 ***** 2026-02-26 20:39:23,092 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:23,102 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-compactor-grpc service is running in openstack namespace] *** 2026-02-26 20:39:23,102 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:23 +0000 (0:00:00.412) 0:00:13.821 ***** 2026-02-26 20:39:23,124 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-compactor-grpc is running." } 2026-02-26 20:39:23,132 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-compactor-http"] ********* 2026-02-26 20:39:23,132 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:23 +0000 (0:00:00.029) 0:00:13.851 ***** 2026-02-26 20:39:23,522 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:23,531 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-compactor-http service is running in openstack namespace] *** 2026-02-26 20:39:23,531 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:23 +0000 (0:00:00.398) 0:00:14.250 ***** 2026-02-26 20:39:23,549 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-compactor-http is running." } 2026-02-26 20:39:23,557 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-distributor-grpc"] ******* 2026-02-26 20:39:23,557 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:23 +0000 (0:00:00.026) 0:00:14.276 ***** 2026-02-26 20:39:23,916 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:23,924 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-distributor-grpc service is running in openstack namespace] *** 2026-02-26 20:39:23,924 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:23 +0000 (0:00:00.366) 0:00:14.643 ***** 2026-02-26 20:39:23,948 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-distributor-grpc is running." } 2026-02-26 20:39:23,956 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-distributor-http"] ******* 2026-02-26 20:39:23,956 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:23 +0000 (0:00:00.032) 0:00:14.675 ***** 2026-02-26 20:39:24,284 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:24,292 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-distributor-http service is running in openstack namespace] *** 2026-02-26 20:39:24,292 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:24 +0000 (0:00:00.335) 0:00:15.011 ***** 2026-02-26 20:39:24,310 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-distributor-http is running." } 2026-02-26 20:39:24,317 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-gateway-http"] *********** 2026-02-26 20:39:24,317 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:24 +0000 (0:00:00.024) 0:00:15.036 ***** 2026-02-26 20:39:24,682 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:24,693 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-gateway-http service is running in openstack namespace] *** 2026-02-26 20:39:24,694 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:24 +0000 (0:00:00.376) 0:00:15.412 ***** 2026-02-26 20:39:24,715 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-gateway-http is running." } 2026-02-26 20:39:24,724 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-gossip-ring"] ************ 2026-02-26 20:39:24,725 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:24 +0000 (0:00:00.030) 0:00:15.443 ***** 2026-02-26 20:39:25,119 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:25,128 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-gossip-ring service is running in openstack namespace] *** 2026-02-26 20:39:25,128 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:25 +0000 (0:00:00.403) 0:00:15.847 ***** 2026-02-26 20:39:25,152 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-gossip-ring is running." } 2026-02-26 20:39:25,161 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-index-gateway-grpc"] ***** 2026-02-26 20:39:25,161 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:25 +0000 (0:00:00.033) 0:00:15.880 ***** 2026-02-26 20:39:25,557 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:25,568 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-index-gateway-grpc service is running in openstack namespace] *** 2026-02-26 20:39:25,568 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:25 +0000 (0:00:00.406) 0:00:16.287 ***** 2026-02-26 20:39:25,595 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-index-gateway-grpc is running." } 2026-02-26 20:39:25,604 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-index-gateway-http"] ***** 2026-02-26 20:39:25,604 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:25 +0000 (0:00:00.035) 0:00:16.323 ***** 2026-02-26 20:39:26,016 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:26,024 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-index-gateway-http service is running in openstack namespace] *** 2026-02-26 20:39:26,024 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:26 +0000 (0:00:00.420) 0:00:16.743 ***** 2026-02-26 20:39:26,056 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-index-gateway-http is running." } 2026-02-26 20:39:26,065 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-ingester-grpc"] ********** 2026-02-26 20:39:26,065 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:26 +0000 (0:00:00.040) 0:00:16.783 ***** 2026-02-26 20:39:26,449 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:26,462 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-ingester-grpc service is running in openstack namespace] *** 2026-02-26 20:39:26,462 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:26 +0000 (0:00:00.397) 0:00:17.181 ***** 2026-02-26 20:39:26,483 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-ingester-grpc is running." } 2026-02-26 20:39:26,492 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-ingester-http"] ********** 2026-02-26 20:39:26,493 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:26 +0000 (0:00:00.030) 0:00:17.211 ***** 2026-02-26 20:39:26,891 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:26,901 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-ingester-http service is running in openstack namespace] *** 2026-02-26 20:39:26,901 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:26 +0000 (0:00:00.408) 0:00:17.620 ***** 2026-02-26 20:39:26,926 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-ingester-http is running." } 2026-02-26 20:39:26,935 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-querier-grpc"] *********** 2026-02-26 20:39:26,935 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:26 +0000 (0:00:00.033) 0:00:17.654 ***** 2026-02-26 20:39:27,318 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:27,329 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-querier-grpc service is running in openstack namespace] *** 2026-02-26 20:39:27,329 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:27 +0000 (0:00:00.394) 0:00:18.048 ***** 2026-02-26 20:39:27,355 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-querier-grpc is running." } 2026-02-26 20:39:27,367 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-querier-http"] *********** 2026-02-26 20:39:27,367 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:27 +0000 (0:00:00.037) 0:00:18.086 ***** 2026-02-26 20:39:27,745 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:27,754 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-querier-http service is running in openstack namespace] *** 2026-02-26 20:39:27,754 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:27 +0000 (0:00:00.386) 0:00:18.473 ***** 2026-02-26 20:39:27,776 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-querier-http is running." } 2026-02-26 20:39:27,785 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-query-frontend-grpc"] **** 2026-02-26 20:39:27,785 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:27 +0000 (0:00:00.031) 0:00:18.504 ***** 2026-02-26 20:39:28,176 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:28,186 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-query-frontend-grpc service is running in openstack namespace] *** 2026-02-26 20:39:28,187 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:28 +0000 (0:00:00.401) 0:00:18.905 ***** 2026-02-26 20:39:28,208 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-query-frontend-grpc is running." } 2026-02-26 20:39:28,217 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-query-frontend-http"] **** 2026-02-26 20:39:28,217 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:28 +0000 (0:00:00.030) 0:00:18.935 ***** 2026-02-26 20:39:28,614 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:28,627 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-query-frontend-http service is running in openstack namespace] *** 2026-02-26 20:39:28,628 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:28 +0000 (0:00:00.411) 0:00:19.346 ***** 2026-02-26 20:39:28,649 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-lokistack-query-frontend-http is running." } 2026-02-26 20:39:28,662 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-public"] *************************** 2026-02-26 20:39:28,662 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:28 +0000 (0:00:00.034) 0:00:19.381 ***** 2026-02-26 20:39:29,064 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:29,073 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-public service is running in openstack namespace] *** 2026-02-26 20:39:29,073 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:29 +0000 (0:00:00.410) 0:00:19.792 ***** 2026-02-26 20:39:29,095 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "service cloudkitty-public is running." } 2026-02-26 20:39:29,102 p=37229 u=zuul n=ansible | TASK [common : Run file tests] ************************************************* 2026-02-26 20:39:29,103 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:29 +0000 (0:00:00.029) 0:00:19.821 ***** 2026-02-26 20:39:29,128 p=37229 u=zuul n=ansible | skipping: [localhost] 2026-02-26 20:39:29,139 p=37229 u=zuul n=ansible | TASK [common : Get the manifest list once] ************************************* 2026-02-26 20:39:29,139 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:29 +0000 (0:00:00.036) 0:00:19.857 ***** 2026-02-26 20:39:29,158 p=37229 u=zuul n=ansible | skipping: [localhost] 2026-02-26 20:39:29,167 p=37229 u=zuul n=ansible | TASK [common : Check that each package manifest is present] ******************** 2026-02-26 20:39:29,167 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:29 +0000 (0:00:00.028) 0:00:19.886 ***** 2026-02-26 20:39:29,187 p=37229 u=zuul n=ansible | skipping: [localhost] 2026-02-26 20:39:29,194 p=37229 u=zuul n=ansible | TASK [common : Run crd tests] ************************************************** 2026-02-26 20:39:29,194 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:29 +0000 (0:00:00.027) 0:00:19.913 ***** 2026-02-26 20:39:29,215 p=37229 u=zuul n=ansible | skipping: [localhost] 2026-02-26 20:39:29,221 p=37229 u=zuul n=ansible | TASK [common : Run CR tests] *************************************************** 2026-02-26 20:39:29,222 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:29 +0000 (0:00:00.027) 0:00:19.940 ***** 2026-02-26 20:39:29,241 p=37229 u=zuul n=ansible | skipping: [localhost] 2026-02-26 20:39:29,248 p=37229 u=zuul n=ansible | TASK [common : Verify container tests] ***************************************** 2026-02-26 20:39:29,248 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:29 +0000 (0:00:00.026) 0:00:19.967 ***** 2026-02-26 20:39:29,266 p=37229 u=zuul n=ansible | skipping: [localhost] 2026-02-26 20:39:29,279 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Validate Chargeback Feature deployed correctly] *** 2026-02-26 20:39:29,279 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:29 +0000 (0:00:00.030) 0:00:19.997 ***** 2026-02-26 20:39:29,303 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/chargeback_tests.yml for localhost 2026-02-26 20:39:29,312 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Enable Cloudkitty Module (hashmap)] *************** 2026-02-26 20:39:29,312 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:29 +0000 (0:00:00.033) 0:00:20.031 ***** 2026-02-26 20:39:32,857 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:39:32,866 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Find the current value of hashmap] **************** 2026-02-26 20:39:32,866 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:32 +0000 (0:00:03.554) 0:00:23.585 ***** 2026-02-26 20:39:35,622 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:35,629 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Change priority for CloudKitty hashmap module] **** 2026-02-26 20:39:35,629 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:35 +0000 (0:00:02.763) 0:00:26.348 ***** 2026-02-26 20:39:38,976 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:39:38,983 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get status of all CloudKitty rating modules] ****** 2026-02-26 20:39:38,984 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:38 +0000 (0:00:03.354) 0:00:29.702 ***** 2026-02-26 20:39:41,152 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:41,160 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Validate CloudKitty module states] *********** 2026-02-26 20:39:41,161 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:41 +0000 (0:00:02.176) 0:00:31.879 ***** 2026-02-26 20:39:41,183 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "SUCCESS: CloudKitty modules (hashmap=True) are configured correctly." } 2026-02-26 20:39:41,189 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Set priority for CloudKitty hashmap module] *** 2026-02-26 20:39:41,189 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:41 +0000 (0:00:00.028) 0:00:31.908 ***** 2026-02-26 20:39:41,211 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "SUCCESS: The hashmap priority is set to 100" } 2026-02-26 20:39:41,217 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Setup Loki Environment] *************************** 2026-02-26 20:39:41,217 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:41 +0000 (0:00:00.027) 0:00:31.936 ***** 2026-02-26 20:39:41,236 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/setup_loki_env.yml for localhost 2026-02-26 20:39:41,246 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get Loki Public Route Host] *********************** 2026-02-26 20:39:41,246 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:41 +0000 (0:00:00.028) 0:00:31.965 ***** 2026-02-26 20:39:41,585 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:41,592 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set Loki URLs] ************************************ 2026-02-26 20:39:41,593 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:41 +0000 (0:00:00.346) 0:00:32.311 ***** 2026-02-26 20:39:41,608 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:41,615 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set Derived Loki URLs] **************************** 2026-02-26 20:39:41,615 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:41 +0000 (0:00:00.022) 0:00:32.334 ***** 2026-02-26 20:39:41,630 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:41,637 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Debug URLs] *************************************** 2026-02-26 20:39:41,637 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:41 +0000 (0:00:00.021) 0:00:32.355 ***** 2026-02-26 20:39:41,654 p=37229 u=zuul n=ansible | ok: [localhost] => { "msg": [ "Loki Route: https://cloudkitty-lokistack-openstack.apps-crc.testing", "Push URL: https://cloudkitty-lokistack-openstack.apps-crc.testing/api/logs/v1/cloudkitty/loki/api/v1/push", "Flush URL: https://cloudkitty-lokistack-ingester-http.openstack.svc:3100/flush", "Query URL: https://cloudkitty-lokistack-openstack.apps-crc.testing/api/logs/v1/cloudkitty/loki/api/v1/query_range" ] } 2026-02-26 20:39:41,660 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Ensure Local Certificate Directory Exists] ******** 2026-02-26 20:39:41,660 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:41 +0000 (0:00:00.023) 0:00:32.379 ***** 2026-02-26 20:39:42,050 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:39:42,059 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Extract Certificates from Openshift Secret] ******* 2026-02-26 20:39:42,059 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:42 +0000 (0:00:00.399) 0:00:32.778 ***** 2026-02-26 20:39:42,452 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:39:42,459 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Extract Client Certificates] ********************** 2026-02-26 20:39:42,460 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:42 +0000 (0:00:00.400) 0:00:33.178 ***** 2026-02-26 20:39:42,860 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:39:42,867 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Extract CA Bundle] ******************************** 2026-02-26 20:39:42,867 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:42 +0000 (0:00:00.407) 0:00:33.586 ***** 2026-02-26 20:39:43,254 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:39:43,261 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get admin project ID for CI] ********************** 2026-02-26 20:39:43,262 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:43 +0000 (0:00:00.394) 0:00:33.980 ***** 2026-02-26 20:39:45,492 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:45,499 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set admin project ID for CI] ********************** 2026-02-26 20:39:45,499 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:45 +0000 (0:00:02.237) 0:00:36.218 ***** 2026-02-26 20:39:45,533 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:45,540 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get admin user ID for CI] ************************* 2026-02-26 20:39:45,540 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:45 +0000 (0:00:00.040) 0:00:36.259 ***** 2026-02-26 20:39:47,730 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:47,744 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set admin user ID for CI] ************************* 2026-02-26 20:39:47,745 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:47 +0000 (0:00:02.204) 0:00:38.463 ***** 2026-02-26 20:39:47,773 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:47,787 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Find test files] ********************************** 2026-02-26 20:39:47,788 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:47 +0000 (0:00:00.043) 0:00:38.506 ***** 2026-02-26 20:39:48,170 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:48,182 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Extract only the filenames into a clean list] ***** 2026-02-26 20:39:48,182 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:48 +0000 (0:00:00.394) 0:00:38.901 ***** 2026-02-26 20:39:48,207 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:48,217 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Process and Loop if files exist] ****************** 2026-02-26 20:39:48,217 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:48 +0000 (0:00:00.035) 0:00:38.936 ***** 2026-02-26 20:39:48,264 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_static_basic) 2026-02-26 20:39:48,274 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_static_basic_gid) 2026-02-26 20:39:48,283 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic] *** 2026-02-26 20:39:48,283 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:48 +0000 (0:00:00.066) 0:00:39.002 ***** 2026-02-26 20:39:48,313 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost 2026-02-26 20:39:48,324 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically test_static_basic] ****** 2026-02-26 20:39:48,324 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:48 +0000 (0:00:00.040) 0:00:39.042 ***** 2026-02-26 20:39:48,347 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:48,355 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] **************** 2026-02-26 20:39:48,355 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:48 +0000 (0:00:00.031) 0:00:39.074 ***** 2026-02-26 20:39:48,682 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:48,693 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data test_static_basic] ******** 2026-02-26 20:39:48,693 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:48 +0000 (0:00:00.338) 0:00:39.412 ***** 2026-02-26 20:39:49,041 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:39:49,055 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic] *** 2026-02-26 20:39:49,055 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:49 +0000 (0:00:00.361) 0:00:39.774 ***** 2026-02-26 20:39:49,330 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:39:49,337 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] ********************** 2026-02-26 20:39:49,337 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:49 +0000 (0:00:00.282) 0:00:40.056 ***** 2026-02-26 20:39:49,354 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:49,366 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_static_basic] ************* 2026-02-26 20:39:49,366 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:49 +0000 (0:00:00.028) 0:00:40.085 ***** 2026-02-26 20:39:49,389 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost 2026-02-26 20:39:49,402 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic] *** 2026-02-26 20:39:49,403 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:49 +0000 (0:00:00.036) 0:00:40.121 ***** 2026-02-26 20:39:49,423 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost 2026-02-26 20:39:49,432 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] **************************** 2026-02-26 20:39:49,432 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:49 +0000 (0:00:00.029) 0:00:40.151 ***** 2026-02-26 20:39:49,756 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:49,763 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ******************************** 2026-02-26 20:39:49,763 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:49 +0000 (0:00:00.330) 0:00:40.482 ***** 2026-02-26 20:39:50,517 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:50,535 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] ********************************** 2026-02-26 20:39:50,536 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:50 +0000 (0:00:00.772) 0:00:41.254 ***** 2026-02-26 20:39:50,557 p=37229 u=zuul n=ansible | ok: [localhost] => { "msg": "Ingestion Successful!" } 2026-02-26 20:39:50,566 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage: test_static_basic] **** 2026-02-26 20:39:50,567 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:50 +0000 (0:00:00.031) 0:00:41.285 ***** 2026-02-26 20:39:50,590 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost 2026-02-26 20:39:50,605 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************ 2026-02-26 20:39:50,605 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:50 +0000 (0:00:00.038) 0:00:41.324 ***** 2026-02-26 20:39:51,057 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:51,063 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] ******* 2026-02-26 20:39:51,063 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:51 +0000 (0:00:00.458) 0:00:41.782 ***** 2026-02-26 20:39:51,272 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:39:51,279 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] *************** 2026-02-26 20:39:51,279 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:51 +0000 (0:00:00.215) 0:00:41.998 ***** 2026-02-26 20:39:51,782 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:39:51,791 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************ 2026-02-26 20:39:51,791 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:51 +0000 (0:00:00.511) 0:00:42.510 ***** 2026-02-26 20:39:52,311 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:39:52,322 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ****************************** 2026-02-26 20:39:52,322 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:52 +0000 (0:00:00.531) 0:00:43.041 ***** 2026-02-26 20:39:52,340 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "Ingester Memory Flushed successfully" } 2026-02-26 20:39:52,351 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki: test_static_basic] *** 2026-02-26 20:39:52,351 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:52 +0000 (0:00:00.029) 0:00:43.070 ***** 2026-02-26 20:39:52,377 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost 2026-02-26 20:39:52,404 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_static_basic] ***************** 2026-02-26 20:39:52,404 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:52 +0000 (0:00:00.052) 0:00:43.123 ***** 2026-02-26 20:39:52,423 p=37229 u=zuul n=ansible | ok: [localhost] => { "msg": "Input file has 24 data entries that Loki has to return" } 2026-02-26 20:39:52,432 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] *********************************** 2026-02-26 20:39:52,432 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:39:52 +0000 (0:00:00.028) 0:00:43.151 ***** 2026-02-26 20:47:56,656 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:47:56,671 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file test_static_basic] **** 2026-02-26 20:47:56,671 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:47:56 +0000 (0:08:04.238) 0:08:47.390 ***** 2026-02-26 20:47:56,701 p=37229 u=zuul n=ansible | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: re.error: unterminated character set at position 3 2026-02-26 20:47:56,701 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": false} 2026-02-26 20:47:56,701 p=37229 u=zuul n=ansible | ...ignoring 2026-02-26 20:47:56,709 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_static_basic] ********** 2026-02-26 20:47:56,709 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:47:56 +0000 (0:00:00.038) 0:08:47.428 ***** 2026-02-26 20:47:56,762 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "Query returned all data entries. Input file had 24 entries and Loki returned 24" } 2026-02-26 20:47:56,769 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic] *** 2026-02-26 20:47:56,770 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:47:56 +0000 (0:00:00.060) 0:08:47.488 ***** 2026-02-26 20:47:57,047 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": false, "cmd": ["python3", "/home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/files/gen_synth_loki_metrics_totals.py", "-j", "/home/zuul/ci-framework-data/artifacts/test_static_basic-loki_data.json", "-o", "/home/zuul/ci-framework-data/artifacts/test_static_basic-loki_metrics_totals.yml"], "delta": "0:00:00.064406", "end": "2026-02-26 20:47:57.018610", "msg": "non-zero return code", "rc": 1, "start": "2026-02-26 20:47:56.954204", "stderr": "", "stderr_lines": [], "stdout": "Error reading JSON file /home/zuul/ci-framework-data/artifacts/test_static_basic-loki_data.json: [Errno 2] No such file or directory: '/home/zuul/ci-framework-data/artifacts/test_static_basic-loki_data.json'", "stdout_lines": ["Error reading JSON file /home/zuul/ci-framework-data/artifacts/test_static_basic-loki_data.json: [Errno 2] No such file or directory: '/home/zuul/ci-framework-data/artifacts/test_static_basic-loki_data.json'"]} 2026-02-26 20:47:57,048 p=37229 u=zuul n=ansible | ...ignoring 2026-02-26 20:47:57,055 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_static_basic] ****** 2026-02-26 20:47:57,055 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:47:57 +0000 (0:00:00.285) 0:08:47.774 ***** 2026-02-26 20:47:57,077 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost 2026-02-26 20:47:57,094 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic] *** 2026-02-26 20:47:57,094 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:47:57 +0000 (0:00:00.038) 0:08:47.813 ***** 2026-02-26 20:48:03,308 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:03,320 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_static_basic] *** 2026-02-26 20:48:03,320 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:03 +0000 (0:00:06.226) 0:08:54.039 ***** 2026-02-26 20:48:03,346 p=37229 u=zuul n=ansible | ok: [localhost] => { "cost_totals_by_type": { "changed": true, "cmd": [ "oc", "rsh", "openstackclient", "openstack", "rating", "summary", "get", "-g", "type" ], "delta": "0:00:05.971770", "end": "2026-02-26 20:48:03.276546", "failed": false, "failed_when_result": false, "msg": "", "rc": 0, "start": "2026-02-26 20:47:57.304776", "stderr": "", "stderr_lines": [], "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time | End Time |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+", "stdout_lines": [ "+-----------+---------------+------+---------------------+---------------------+", "| Tenant ID | Resource Type | Rate | Begin Time | End Time |", "+-----------+---------------+------+---------------------+---------------------+", "| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |", "+-----------+---------------+------+---------------------+---------------------+" ] } } 2026-02-26 20:48:03,356 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_static_basic] *** 2026-02-26 20:48:03,356 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:03 +0000 (0:00:00.036) 0:08:54.075 ***** 2026-02-26 20:48:03,869 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:03,881 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic] *** 2026-02-26 20:48:03,881 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:03 +0000 (0:00:00.524) 0:08:54.600 ***** 2026-02-26 20:48:06,473 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:06,481 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_static_basic] *** 2026-02-26 20:48:06,481 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:06 +0000 (0:00:02.600) 0:08:57.200 ***** 2026-02-26 20:48:06,498 p=37229 u=zuul n=ansible | ok: [localhost] => { "cost_totals_summary": { "changed": true, "cmd": [ "oc", "rsh", "openstackclient", "openstack", "rating", "summary", "get" ], "delta": "0:00:02.378371", "end": "2026-02-26 20:48:06.448073", "failed": false, "failed_when_result": false, "msg": "", "rc": 0, "start": "2026-02-26 20:48:04.069702", "stderr": "", "stderr_lines": [], "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time | End Time |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+", "stdout_lines": [ "+-----------+---------------+------+---------------------+---------------------+", "| Tenant ID | Resource Type | Rate | Begin Time | End Time |", "+-----------+---------------+------+---------------------+---------------------+", "| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |", "+-----------+---------------+------+---------------------+---------------------+" ] } } 2026-02-26 20:48:06,504 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** help for openstack rating summary cmd] *** 2026-02-26 20:48:06,504 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:06 +0000 (0:00:00.022) 0:08:57.223 ***** 2026-02-26 20:48:08,260 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:08,275 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary get help output] *** 2026-02-26 20:48:08,275 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:08 +0000 (0:00:01.770) 0:08:58.994 ***** 2026-02-26 20:48:08,297 p=37229 u=zuul n=ansible | ok: [localhost] => { "cost_totals_h.stdout_lines": [ "usage: openstack rating summary get [-h]", " [-f {csv,df-to-csv,json,table,value,yaml}]", " [-c COLUMN]", " [--quote {all,minimal,none,nonnumeric}]", " [--noindent] [--max-width ]", " [--fit-width] [--print-empty]", " [--format-config-file FORMAT_CONFIG]", " [--sort-column SORT_COLUMN]", " [--sort-ascending | --sort-descending]", " [-t TENANT_ID] [-b BEGIN] [-e END]", " [-s SERVICE] [-g GROUPBY [GROUPBY ...]]", " [-a]", "", "Get a summary report.", "", "optional arguments:", " -h, --help show this help message and exit", " -t TENANT_ID, --tenant-id TENANT_ID", " Tenant id.", " -b BEGIN, --begin BEGIN", " Begin timestamp.", " -e END, --end END End timestamp.", " -s SERVICE, --service SERVICE", " Service Type.", " -g GROUPBY [GROUPBY ...], --groupby GROUPBY [GROUPBY ...]", " Fields to group by, space-separated. (res_type and", " tenant_id are supported for now)", " -a, --all-tenants Allows to get summary from all tenants (admin only).", " Defaults to False.", "", "output formatters:", " output formatter options", "", " -f {csv,df-to-csv,json,table,value,yaml}, --format {csv,df-to-csv,json,table,value,yaml}", " the output format, defaults to table", " -c COLUMN, --column COLUMN", " specify the column(s) to include, can be repeated to", " show multiple columns", " --sort-column SORT_COLUMN", " specify the column(s) to sort the data (columns", " specified first have a priority, non-existing columns", " are ignored), can be repeated", " --sort-ascending sort the column(s) in ascending order", " --sort-descending sort the column(s) in descending order", "", "CSV Formatter:", " --quote {all,minimal,none,nonnumeric}", " when to include quotes, defaults to nonnumeric", "", "json formatter:", " --noindent whether to disable indenting the JSON", "", "table formatter:", " --max-width ", " Maximum display width, <1 to disable. You can also use", " the CLIFF_MAX_TERM_WIDTH environment variable, but the", " parameter takes precedence.", " --fit-width Fit the table to the display width. Implied if --max-", " width greater than 0. Set the environment variable", " CLIFF_FIT_WIDTH=1 to always enable", " --print-empty Print empty table if there is no data to show.", "", "dataframe-to-csv formatter:", " --format-config-file FORMAT_CONFIG", " Config file for the dict-to-csv formatter", "", "This command is provided by the python-cloudkittyclient plugin." ] } 2026-02-26 20:48:08,308 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic data file] ********************* 2026-02-26 20:48:08,308 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:08 +0000 (0:00:00.032) 0:08:59.026 ***** 2026-02-26 20:48:08,526 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:48:08,533 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] *********************** 2026-02-26 20:48:08,533 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:08 +0000 (0:00:00.225) 0:08:59.252 ***** 2026-02-26 20:48:08,715 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": false, "msg": "file not found: /home/zuul/ci-framework-data/artifacts/test_static_basic-loki_metrics_totals.yml"} 2026-02-26 20:48:08,715 p=37229 u=zuul n=ansible | ...ignoring 2026-02-26 20:48:08,729 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic] *** 2026-02-26 20:48:08,729 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:08 +0000 (0:00:00.195) 0:08:59.448 ***** 2026-02-26 20:48:08,750 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"msg": "The conditional check '(synth_data.content | b64decode | from_yaml) == (loki_data.content | b64decode | from_yaml)' failed. The error was: error while evaluating conditional ((synth_data.content | b64decode | from_yaml) == (loki_data.content | b64decode | from_yaml)): 'dict object' has no attribute 'content'. 'dict object' has no attribute 'content'"} 2026-02-26 20:48:08,750 p=37229 u=zuul n=ansible | ...ignoring 2026-02-26 20:48:08,758 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic_gid] *** 2026-02-26 20:48:08,758 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:08 +0000 (0:00:00.029) 0:08:59.477 ***** 2026-02-26 20:48:08,779 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost 2026-02-26 20:48:08,788 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically test_static_basic_gid] *** 2026-02-26 20:48:08,789 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:08 +0000 (0:00:00.030) 0:08:59.507 ***** 2026-02-26 20:48:08,807 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:48:08,813 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] **************** 2026-02-26 20:48:08,813 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:08 +0000 (0:00:00.024) 0:08:59.532 ***** 2026-02-26 20:48:09,016 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:48:09,023 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data test_static_basic_gid] **** 2026-02-26 20:48:09,024 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:09 +0000 (0:00:00.210) 0:08:59.742 ***** 2026-02-26 20:48:09,331 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:09,347 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic_gid] *** 2026-02-26 20:48:09,347 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:09 +0000 (0:00:00.323) 0:09:00.066 ***** 2026-02-26 20:48:09,661 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:09,669 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] ********************** 2026-02-26 20:48:09,669 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:09 +0000 (0:00:00.321) 0:09:00.388 ***** 2026-02-26 20:48:09,688 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:48:09,696 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_static_basic_gid] ********* 2026-02-26 20:48:09,696 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:09 +0000 (0:00:00.027) 0:09:00.415 ***** 2026-02-26 20:48:09,717 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost 2026-02-26 20:48:09,731 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic_gid] *** 2026-02-26 20:48:09,731 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:09 +0000 (0:00:00.034) 0:09:00.450 ***** 2026-02-26 20:48:09,752 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost 2026-02-26 20:48:09,760 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] **************************** 2026-02-26 20:48:09,760 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:09 +0000 (0:00:00.028) 0:09:00.479 ***** 2026-02-26 20:48:09,991 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:48:09,998 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ******************************** 2026-02-26 20:48:09,998 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:09 +0000 (0:00:00.237) 0:09:00.716 ***** 2026-02-26 20:48:10,362 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:48:10,371 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] ********************************** 2026-02-26 20:48:10,371 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:10 +0000 (0:00:00.373) 0:09:01.090 ***** 2026-02-26 20:48:10,386 p=37229 u=zuul n=ansible | ok: [localhost] => { "msg": "Ingestion Successful!" } 2026-02-26 20:48:10,393 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage: test_static_basic_gid] *** 2026-02-26 20:48:10,393 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:10 +0000 (0:00:00.022) 0:09:01.112 ***** 2026-02-26 20:48:10,412 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost 2026-02-26 20:48:10,424 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************ 2026-02-26 20:48:10,424 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:10 +0000 (0:00:00.031) 0:09:01.143 ***** 2026-02-26 20:48:10,808 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:48:10,815 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] ******* 2026-02-26 20:48:10,815 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:10 +0000 (0:00:00.390) 0:09:01.533 ***** 2026-02-26 20:48:11,012 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:48:11,019 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] *************** 2026-02-26 20:48:11,019 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:11 +0000 (0:00:00.204) 0:09:01.738 ***** 2026-02-26 20:48:11,467 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:11,474 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************ 2026-02-26 20:48:11,475 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:11 +0000 (0:00:00.455) 0:09:02.193 ***** 2026-02-26 20:48:12,049 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:12,057 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ****************************** 2026-02-26 20:48:12,057 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:12 +0000 (0:00:00.582) 0:09:02.776 ***** 2026-02-26 20:48:12,076 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "Ingester Memory Flushed successfully" } 2026-02-26 20:48:12,084 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki: test_static_basic_gid] *** 2026-02-26 20:48:12,084 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:12 +0000 (0:00:00.027) 0:09:02.803 ***** 2026-02-26 20:48:12,104 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost 2026-02-26 20:48:12,124 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_static_basic_gid] ************* 2026-02-26 20:48:12,124 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:12 +0000 (0:00:00.039) 0:09:02.842 ***** 2026-02-26 20:48:12,141 p=37229 u=zuul n=ansible | ok: [localhost] => { "msg": "Input file has 24 data entries that Loki has to return" } 2026-02-26 20:48:12,147 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] *********************************** 2026-02-26 20:48:12,147 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:12 +0000 (0:00:00.023) 0:09:02.866 ***** 2026-02-26 20:48:12,552 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:48:12,562 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file test_static_basic_gid] *** 2026-02-26 20:48:12,562 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:12 +0000 (0:00:00.414) 0:09:03.281 ***** 2026-02-26 20:48:12,578 p=37229 u=zuul n=ansible | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: re.error: unterminated character set at position 3 2026-02-26 20:48:12,578 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": false} 2026-02-26 20:48:12,578 p=37229 u=zuul n=ansible | ...ignoring 2026-02-26 20:48:12,597 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_static_basic_gid] ****** 2026-02-26 20:48:12,597 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:12 +0000 (0:00:00.034) 0:09:03.316 ***** 2026-02-26 20:48:12,632 p=37229 u=zuul n=ansible | ok: [localhost] => { "changed": false, "msg": "Query returned all data entries. Input file had 24 entries and Loki returned 24" } 2026-02-26 20:48:12,640 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic_gid] *** 2026-02-26 20:48:12,640 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:12 +0000 (0:00:00.043) 0:09:03.359 ***** 2026-02-26 20:48:12,887 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": false, "cmd": ["python3", "/home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/files/gen_synth_loki_metrics_totals.py", "-j", "/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_data.json", "-o", "/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_metrics_totals.yml"], "delta": "0:00:00.053409", "end": "2026-02-26 20:48:12.864910", "msg": "non-zero return code", "rc": 1, "start": "2026-02-26 20:48:12.811501", "stderr": "", "stderr_lines": [], "stdout": "Error reading JSON file /home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_data.json: [Errno 2] No such file or directory: '/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_data.json'", "stdout_lines": ["Error reading JSON file /home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_data.json: [Errno 2] No such file or directory: '/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_data.json'"]} 2026-02-26 20:48:12,887 p=37229 u=zuul n=ansible | ...ignoring 2026-02-26 20:48:12,894 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_static_basic_gid] *** 2026-02-26 20:48:12,895 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:12 +0000 (0:00:00.254) 0:09:03.613 ***** 2026-02-26 20:48:12,912 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost 2026-02-26 20:48:12,927 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic_gid] *** 2026-02-26 20:48:12,927 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:12 +0000 (0:00:00.032) 0:09:03.645 ***** 2026-02-26 20:48:15,772 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:15,789 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_static_basic_gid] *** 2026-02-26 20:48:15,789 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:15 +0000 (0:00:02.862) 0:09:06.508 ***** 2026-02-26 20:48:15,808 p=37229 u=zuul n=ansible | ok: [localhost] => { "cost_totals_by_type": { "changed": true, "cmd": [ "oc", "rsh", "openstackclient", "openstack", "rating", "summary", "get", "-g", "type" ], "delta": "0:00:02.639796", "end": "2026-02-26 20:48:15.729755", "failed": false, "failed_when_result": false, "msg": "", "rc": 0, "start": "2026-02-26 20:48:13.089959", "stderr": "", "stderr_lines": [], "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time | End Time |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+", "stdout_lines": [ "+-----------+---------------+------+---------------------+---------------------+", "| Tenant ID | Resource Type | Rate | Begin Time | End Time |", "+-----------+---------------+------+---------------------+---------------------+", "| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |", "+-----------+---------------+------+---------------------+---------------------+" ] } } 2026-02-26 20:48:15,824 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_static_basic_gid] *** 2026-02-26 20:48:15,824 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:15 +0000 (0:00:00.034) 0:09:06.543 ***** 2026-02-26 20:48:16,199 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:16,208 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic_gid] *** 2026-02-26 20:48:16,209 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:16 +0000 (0:00:00.384) 0:09:06.927 ***** 2026-02-26 20:48:18,586 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:18,594 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_static_basic_gid] *** 2026-02-26 20:48:18,595 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:18 +0000 (0:00:02.385) 0:09:09.313 ***** 2026-02-26 20:48:18,609 p=37229 u=zuul n=ansible | ok: [localhost] => { "cost_totals_summary": { "changed": true, "cmd": [ "oc", "rsh", "openstackclient", "openstack", "rating", "summary", "get" ], "delta": "0:00:02.186285", "end": "2026-02-26 20:48:18.562149", "failed": false, "failed_when_result": false, "msg": "", "rc": 0, "start": "2026-02-26 20:48:16.375864", "stderr": "", "stderr_lines": [], "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time | End Time |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+", "stdout_lines": [ "+-----------+---------------+------+---------------------+---------------------+", "| Tenant ID | Resource Type | Rate | Begin Time | End Time |", "+-----------+---------------+------+---------------------+---------------------+", "| ALL | ALL | 0 | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |", "+-----------+---------------+------+---------------------+---------------------+" ] } } 2026-02-26 20:48:18,616 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** help for openstack rating summary cmd] *** 2026-02-26 20:48:18,616 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:18 +0000 (0:00:00.021) 0:09:09.335 ***** 2026-02-26 20:48:20,381 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:20,389 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary get help output] *** 2026-02-26 20:48:20,389 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:20 +0000 (0:00:01.772) 0:09:11.108 ***** 2026-02-26 20:48:20,416 p=37229 u=zuul n=ansible | ok: [localhost] => { "cost_totals_h.stdout_lines": [ "usage: openstack rating summary get [-h]", " [-f {csv,df-to-csv,json,table,value,yaml}]", " [-c COLUMN]", " [--quote {all,minimal,none,nonnumeric}]", " [--noindent] [--max-width ]", " [--fit-width] [--print-empty]", " [--format-config-file FORMAT_CONFIG]", " [--sort-column SORT_COLUMN]", " [--sort-ascending | --sort-descending]", " [-t TENANT_ID] [-b BEGIN] [-e END]", " [-s SERVICE] [-g GROUPBY [GROUPBY ...]]", " [-a]", "", "Get a summary report.", "", "optional arguments:", " -h, --help show this help message and exit", " -t TENANT_ID, --tenant-id TENANT_ID", " Tenant id.", " -b BEGIN, --begin BEGIN", " Begin timestamp.", " -e END, --end END End timestamp.", " -s SERVICE, --service SERVICE", " Service Type.", " -g GROUPBY [GROUPBY ...], --groupby GROUPBY [GROUPBY ...]", " Fields to group by, space-separated. (res_type and", " tenant_id are supported for now)", " -a, --all-tenants Allows to get summary from all tenants (admin only).", " Defaults to False.", "", "output formatters:", " output formatter options", "", " -f {csv,df-to-csv,json,table,value,yaml}, --format {csv,df-to-csv,json,table,value,yaml}", " the output format, defaults to table", " -c COLUMN, --column COLUMN", " specify the column(s) to include, can be repeated to", " show multiple columns", " --sort-column SORT_COLUMN", " specify the column(s) to sort the data (columns", " specified first have a priority, non-existing columns", " are ignored), can be repeated", " --sort-ascending sort the column(s) in ascending order", " --sort-descending sort the column(s) in descending order", "", "CSV Formatter:", " --quote {all,minimal,none,nonnumeric}", " when to include quotes, defaults to nonnumeric", "", "json formatter:", " --noindent whether to disable indenting the JSON", "", "table formatter:", " --max-width ", " Maximum display width, <1 to disable. You can also use", " the CLIFF_MAX_TERM_WIDTH environment variable, but the", " parameter takes precedence.", " --fit-width Fit the table to the display width. Implied if --max-", " width greater than 0. Set the environment variable", " CLIFF_FIT_WIDTH=1 to always enable", " --print-empty Print empty table if there is no data to show.", "", "dataframe-to-csv formatter:", " --format-config-file FORMAT_CONFIG", " Config file for the dict-to-csv formatter", "", "This command is provided by the python-cloudkittyclient plugin." ] } 2026-02-26 20:48:20,422 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic data file] ********************* 2026-02-26 20:48:20,423 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:20 +0000 (0:00:00.033) 0:09:11.141 ***** 2026-02-26 20:48:20,650 p=37229 u=zuul n=ansible | ok: [localhost] 2026-02-26 20:48:20,659 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] *********************** 2026-02-26 20:48:20,659 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:20 +0000 (0:00:00.236) 0:09:11.377 ***** 2026-02-26 20:48:20,851 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"changed": false, "msg": "file not found: /home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_metrics_totals.yml"} 2026-02-26 20:48:20,851 p=37229 u=zuul n=ansible | ...ignoring 2026-02-26 20:48:20,859 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic_gid] *** 2026-02-26 20:48:20,859 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:20 +0000 (0:00:00.200) 0:09:11.578 ***** 2026-02-26 20:48:20,878 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {"msg": "The conditional check '(synth_data.content | b64decode | from_yaml) == (loki_data.content | b64decode | from_yaml)' failed. The error was: error while evaluating conditional ((synth_data.content | b64decode | from_yaml) == (loki_data.content | b64decode | from_yaml)): 'dict object' has no attribute 'content'. 'dict object' has no attribute 'content'"} 2026-02-26 20:48:20,878 p=37229 u=zuul n=ansible | ...ignoring 2026-02-26 20:48:20,885 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Cleanup after job run] **************************** 2026-02-26 20:48:20,885 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:20 +0000 (0:00:00.025) 0:09:11.603 ***** 2026-02-26 20:48:20,902 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/cleanup_ck.yml for localhost 2026-02-26 20:48:20,916 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Cleanup local certificates] *********************** 2026-02-26 20:48:20,916 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:20 +0000 (0:00:00.031) 0:09:11.635 ***** 2026-02-26 20:48:21,117 p=37229 u=zuul n=ansible | changed: [localhost] 2026-02-26 20:48:21,160 p=37229 u=zuul n=ansible | PLAY RECAP ********************************************************************* 2026-02-26 20:48:21,161 p=37229 u=zuul n=ansible | localhost : ok=179 changed=23 unreachable=0 failed=0 skipped=7 rescued=0 ignored=8 2026-02-26 20:48:21,171 p=37229 u=zuul n=ansible | Thursday 26 February 2026 20:48:21 +0000 (0:00:00.254) 0:09:11.890 ***** 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | =============================================================================== 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : Query Loki API --------------------------------- 484.24s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic --- 6.23s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : Enable Cloudkitty Module (hashmap) --------------- 3.55s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : Change priority for CloudKitty hashmap module ---- 3.35s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic_gid --- 2.86s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : Find the current value of hashmap ---------------- 2.76s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic --- 2.60s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | common : Verify endpoint "['cloudkitty', 'rating', 'public']" ----------- 2.50s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic_gid --- 2.39s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : Get admin project ID for CI ---------------------- 2.24s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : Get admin user ID for CI ------------------------- 2.20s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : Get status of all CloudKitty rating modules ------ 2.18s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | common : Verify endpoint "['cloudkitty', 'rating', 'internal']" --------- 2.13s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** help for openstack rating summary cmd --- 1.77s 2026-02-26 20:48:21,172 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** help for openstack rating summary cmd --- 1.77s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | Gathering Facts --------------------------------------------------------- 0.97s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.77s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.58s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.53s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_static_basic --- 0.52s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 0.51s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.46s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 0.46s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.45s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | common : Get the openstack-operators project ---------------------------- 0.44s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-internal" ------------------------- 0.44s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-index-gateway-http" ----- 0.42s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.42s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | telemetry_chargeback : Query Loki API ----------------------------------- 0.41s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-compactor-grpc" --------- 0.41s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-query-frontend-http" ---- 0.41s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-public" --------------------------- 0.41s 2026-02-26 20:48:21,173 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-compactor pod is Running in openstack namespace --- 0.41s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-ingester-http" ---------- 0.41s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | telemetry_chargeback : Extract Client Certificates ---------------------- 0.41s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-index-gateway-grpc" ----- 0.41s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-gossip-ring" ------------ 0.40s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-query-frontend-grpc" ---- 0.40s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | telemetry_chargeback : Extract Certificates from Openshift Secret ------- 0.40s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | telemetry_chargeback : Ensure Local Certificate Directory Exists -------- 0.40s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-compactor-http" --------- 0.40s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-ingester-grpc" ---------- 0.40s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.40s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-querier-grpc" ----------- 0.39s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | telemetry_chargeback : Extract CA Bundle -------------------------------- 0.39s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | telemetry_chargeback : Find test files ---------------------------------- 0.39s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.39s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.39s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-querier-http" ----------- 0.39s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-api pod is Running in openstack namespace --- 0.39s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_static_basic_gid --- 0.38s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.38s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.38s 2026-02-26 20:48:21,174 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-gateway-http" ----------- 0.38s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.38s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.37s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.37s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-distributor-grpc" ------- 0.37s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data test_static_basic -------- 0.36s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-query-frontend pod is Running in openstack namespace --- 0.36s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-index-gateway pod is Running in openstack namespace --- 0.36s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-ingester pod is Running in openstack namespace --- 0.35s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | telemetry_chargeback : Get Loki Public Route Host ----------------------- 0.35s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-distributor pod is Running in openstack namespace --- 0.35s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | common : Get the openstack project -------------------------------------- 0.34s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.34s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-distributor-http" ------- 0.34s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.33s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data test_static_basic_gid ---- 0.32s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-querier pod is Running in openstack namespace --- 0.32s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic_gid --- 0.32s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-proc pod is Running in openstack namespace --- 0.29s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic --- 0.29s 2026-02-26 20:48:21,175 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic --- 0.28s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Cleanup local certificates ----------------------- 0.26s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic_gid --- 0.25s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.24s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Read the synthetic data file --------------------- 0.24s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Read the synthetic data file --------------------- 0.23s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.22s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.21s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.20s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.20s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.20s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | common : Run service tests ---------------------------------------------- 0.19s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | common : Run pod tests -------------------------------------------------- 0.10s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | common : Run project tests ---------------------------------------------- 0.08s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Process and Loop if files exist ------------------ 0.07s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_static_basic ---------- 0.06s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki: test_static_basic --- 0.05s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | common : Run endpoint tests --------------------------------------------- 0.05s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Set admin user ID for CI ------------------------- 0.04s 2026-02-26 20:48:21,176 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_static_basic_gid ------ 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : Set admin project ID for CI ---------------------- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-index-gateway-http service is running in openstack namespace --- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic --- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki: test_static_basic_gid --- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage: test_static_basic ---- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_static_basic ------ 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file test_static_basic ---- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-querier-grpc service is running in openstack namespace --- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-internal service is running in openstack namespace --- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_static_basic ------------- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | common : Run file tests ------------------------------------------------- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_static_basic --- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-index-gateway-grpc service is running in openstack namespace --- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : Extract only the filenames into a clean list ----- 0.04s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_static_basic_gid --- 0.03s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_static_basic_gid --------- 0.03s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file test_static_basic_gid --- 0.03s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | common : TEST Verify openstack-operators project exists ---------------- 0.03s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-query-frontend-http service is running in openstack namespace --- 0.03s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-ingester-http service is running in openstack namespace --- 0.03s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary get help output --- 0.03s 2026-02-26 20:48:21,177 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-gossip-ring service is running in openstack namespace --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : Validate Chargeback Feature deployed correctly --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary get help output --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-distributor-grpc service is running in openstack namespace --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_static_basic_gid --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : Cleanup after job run ---------------------------- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically test_static_basic ------ 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage: test_static_basic_gid --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-querier-http service is running in openstack namespace --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : Confirm Success ---------------------------------- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-gateway-http service is running in openstack namespace --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | common : Verify container tests ----------------------------------------- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic_gid --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-ingester-grpc service is running in openstack namespace --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-query-frontend-grpc service is running in openstack namespace --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-compactor-grpc service is running in openstack namespace --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-public service is running in openstack namespace --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic_gid --- 0.03s 2026-02-26 20:48:21,178 p=37229 u=zuul n=ansible | telemetry_chargeback : Setup Loki Environment --------------------------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Validate CloudKitty module states ----------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | common : TEST Check that cloudkitty internal endpoint exists ----------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : Expected Count test_static_basic ----------------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | common : Get the manifest list once ------------------------------------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Set priority for CloudKitty hashmap module --- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | common : Check that each package manifest is present -------------------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | common : Run crd tests -------------------------------------------------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | common : TEST Check that cloudkitty public endpoint exists ------------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | common : Run CR tests --------------------------------------------------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | common : TEST Verify openstack project exists -------------------------- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-compactor-http service is running in openstack namespace --- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic_gid --- 0.03s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-distributor-http service is running in openstack namespace --- 0.02s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically test_static_basic_gid --- 0.02s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : Expected Count test_static_basic_gid ------------- 0.02s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : Debug URLs --------------------------------------- 0.02s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_static_basic --- 0.02s 2026-02-26 20:48:21,179 p=37229 u=zuul n=ansible | telemetry_chargeback : Confirm Success ---------------------------------- 0.02s 2026-02-26 20:48:21,180 p=37229 u=zuul n=ansible | telemetry_chargeback : Set Loki URLs ------------------------------------ 0.02s 2026-02-26 20:48:21,180 p=37229 u=zuul n=ansible | telemetry_chargeback : Set Derived Loki URLs ---------------------------- 0.02s 2026-02-26 20:48:21,180 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_static_basic_gid --- 0.02s