2026-02-27 00:51:38,307 p=37229 u=zuul n=ansible | [WARNING]: Found variable using reserved name: namespace

2026-02-27 00:51:38,307 p=37229 u=zuul n=ansible | PLAY [Verify all the applicable projects, endpoints, pods & services for cloudkitty] ***
2026-02-27 00:51:38,319 p=37229 u=zuul n=ansible | TASK [Gathering Facts] *********************************************************
2026-02-27 00:51:38,319 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:38 +0000 (0:00:00.027)       0:00:00.027 ******* 
2026-02-27 00:51:39,250 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:39,270 p=37229 u=zuul n=ansible | TASK [common : Run pod tests] **************************************************
2026-02-27 00:51:39,270 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:39 +0000 (0:00:00.951)       0:00:00.979 ******* 
2026-02-27 00:51:39,345 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-api)
2026-02-27 00:51:39,348 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-compactor)
2026-02-27 00:51:39,351 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-distributor)
2026-02-27 00:51:39,353 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway)
2026-02-27 00:51:39,356 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-ingester)
2026-02-27 00:51:39,358 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-querier)
2026-02-27 00:51:39,361 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend)
2026-02-27 00:51:39,363 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-proc)
2026-02-27 00:51:39,373 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-27 00:51:39,373 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:39 +0000 (0:00:00.102)       0:00:01.082 ******* 
2026-02-27 00:51:39,884 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:39,892 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-api pod is Running in openstack namespace] ***
2026-02-27 00:51:39,892 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:39 +0000 (0:00:00.518)       0:00:01.600 ******* 
2026-02-27 00:51:40,199 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:40,206 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-27 00:51:40,206 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:40 +0000 (0:00:00.314)       0:00:01.915 ******* 
2026-02-27 00:51:40,593 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:40,601 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-compactor pod is Running in openstack namespace] ***
2026-02-27 00:51:40,602 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:40 +0000 (0:00:00.395)       0:00:02.310 ******* 
2026-02-27 00:51:40,925 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:40,934 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-27 00:51:40,934 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:40 +0000 (0:00:00.332)       0:00:02.643 ******* 
2026-02-27 00:51:41,310 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:41,318 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-distributor pod is Running in openstack namespace] ***
2026-02-27 00:51:41,318 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:41 +0000 (0:00:00.383)       0:00:03.026 ******* 
2026-02-27 00:51:41,643 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:41,653 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-27 00:51:41,653 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:41 +0000 (0:00:00.335)       0:00:03.362 ******* 
2026-02-27 00:51:42,101 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:42,111 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-index-gateway pod is Running in openstack namespace] ***
2026-02-27 00:51:42,111 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:42 +0000 (0:00:00.457)       0:00:03.820 ******* 
2026-02-27 00:51:42,462 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:42,470 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-27 00:51:42,470 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:42 +0000 (0:00:00.359)       0:00:04.179 ******* 
2026-02-27 00:51:42,859 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:42,867 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-ingester pod is Running in openstack namespace] ***
2026-02-27 00:51:42,867 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:42 +0000 (0:00:00.396)       0:00:04.576 ******* 
2026-02-27 00:51:43,230 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:43,238 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-27 00:51:43,238 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:43 +0000 (0:00:00.370)       0:00:04.946 ******* 
2026-02-27 00:51:43,643 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:43,653 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-querier pod is Running in openstack namespace] ***
2026-02-27 00:51:43,653 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:43 +0000 (0:00:00.415)       0:00:05.362 ******* 
2026-02-27 00:51:44,003 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:44,011 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-27 00:51:44,011 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:44 +0000 (0:00:00.357)       0:00:05.720 ******* 
2026-02-27 00:51:44,423 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:44,431 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-query-frontend pod is Running in openstack namespace] ***
2026-02-27 00:51:44,431 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:44 +0000 (0:00:00.420)       0:00:06.140 ******* 
2026-02-27 00:51:44,772 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:44,780 p=37229 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-27 00:51:44,781 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:44 +0000 (0:00:00.349)       0:00:06.489 ******* 
2026-02-27 00:51:45,174 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:45,183 p=37229 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-proc pod is Running in openstack namespace] ***
2026-02-27 00:51:45,183 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:45 +0000 (0:00:00.402)       0:00:06.891 ******* 
2026-02-27 00:51:45,508 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:45,515 p=37229 u=zuul n=ansible | TASK [common : Run subscription tests] *****************************************
2026-02-27 00:51:45,516 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:45 +0000 (0:00:00.332)       0:00:07.224 ******* 
2026-02-27 00:51:45,532 p=37229 u=zuul n=ansible | skipping: [localhost]
2026-02-27 00:51:45,540 p=37229 u=zuul n=ansible | TASK [common : Run endpoint tests] *********************************************
2026-02-27 00:51:45,540 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:45 +0000 (0:00:00.024)       0:00:07.249 ******* 
2026-02-27 00:51:45,572 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/endpoint_tests.yml for localhost => (item=['cloudkitty', 'rating', 'public'])
2026-02-27 00:51:45,574 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/endpoint_tests.yml for localhost => (item=['cloudkitty', 'rating', 'internal'])
2026-02-27 00:51:45,595 p=37229 u=zuul n=ansible | TASK [common : Verify endpoint "['cloudkitty', 'rating', 'public']"] ***********
2026-02-27 00:51:45,595 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:45 +0000 (0:00:00.055)       0:00:07.304 ******* 
2026-02-27 00:51:47,855 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:47,864 p=37229 u=zuul n=ansible | TASK [common : TEST Check that cloudkitty public endpoint exists] **************
2026-02-27 00:51:47,864 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:47 +0000 (0:00:02.269)       0:00:09.573 ******* 
2026-02-27 00:51:47,886 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "The endpoint cloudkitty is found."
}
2026-02-27 00:51:47,894 p=37229 u=zuul n=ansible | TASK [common : Verify endpoint "['cloudkitty', 'rating', 'internal']"] *********
2026-02-27 00:51:47,894 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:47 +0000 (0:00:00.029)       0:00:09.603 ******* 
2026-02-27 00:51:50,123 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:50,134 p=37229 u=zuul n=ansible | TASK [common : TEST Check that cloudkitty internal endpoint exists] ************
2026-02-27 00:51:50,135 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:50 +0000 (0:00:02.240)       0:00:11.843 ******* 
2026-02-27 00:51:50,152 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "The endpoint cloudkitty is found."
}
2026-02-27 00:51:50,162 p=37229 u=zuul n=ansible | TASK [common : Run project tests] **********************************************
2026-02-27 00:51:50,163 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:50 +0000 (0:00:00.028)       0:00:11.871 ******* 
2026-02-27 00:51:50,193 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/project_tests.yml for localhost => (item=openstack)
2026-02-27 00:51:50,195 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/project_tests.yml for localhost => (item=openstack-operators)
2026-02-27 00:51:50,218 p=37229 u=zuul n=ansible | TASK [common : Get the openstack project] **************************************
2026-02-27 00:51:50,218 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:50 +0000 (0:00:00.055)       0:00:11.926 ******* 
2026-02-27 00:51:50,558 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:50,567 p=37229 u=zuul n=ansible | TASK [common : TEST Verify openstack project exists] ***************************
2026-02-27 00:51:50,567 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:50 +0000 (0:00:00.348)       0:00:12.275 ******* 
2026-02-27 00:51:50,585 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "project openstack is found."
}
2026-02-27 00:51:50,592 p=37229 u=zuul n=ansible | TASK [common : Get the openstack-operators project] ****************************
2026-02-27 00:51:50,593 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:50 +0000 (0:00:00.025)       0:00:12.301 ******* 
2026-02-27 00:51:50,929 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:50,938 p=37229 u=zuul n=ansible | TASK [common : TEST Verify openstack-operators project exists] *****************
2026-02-27 00:51:50,938 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:50 +0000 (0:00:00.345)       0:00:12.647 ******* 
2026-02-27 00:51:50,955 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "project openstack-operators is found."
}
2026-02-27 00:51:50,965 p=37229 u=zuul n=ansible | TASK [common : Run service tests] **********************************************
2026-02-27 00:51:50,965 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:50 +0000 (0:00:00.026)       0:00:12.674 ******* 
2026-02-27 00:51:51,073 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-internal)
2026-02-27 00:51:51,076 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-compactor-grpc)
2026-02-27 00:51:51,079 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-compactor-http)
2026-02-27 00:51:51,081 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-distributor-grpc)
2026-02-27 00:51:51,084 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-distributor-http)
2026-02-27 00:51:51,087 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-gateway-http)
2026-02-27 00:51:51,089 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-gossip-ring)
2026-02-27 00:51:51,092 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway-grpc)
2026-02-27 00:51:51,095 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway-http)
2026-02-27 00:51:51,097 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-ingester-grpc)
2026-02-27 00:51:51,100 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-ingester-http)
2026-02-27 00:51:51,102 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-querier-grpc)
2026-02-27 00:51:51,105 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-querier-http)
2026-02-27 00:51:51,108 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend-grpc)
2026-02-27 00:51:51,110 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend-http)
2026-02-27 00:51:51,112 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-public)
2026-02-27 00:51:51,136 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-internal"] *************************
2026-02-27 00:51:51,136 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:51 +0000 (0:00:00.170)       0:00:12.845 ******* 
2026-02-27 00:51:51,466 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:51,473 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-internal service is running in openstack namespace] ***
2026-02-27 00:51:51,473 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:51 +0000 (0:00:00.337)       0:00:13.182 ******* 
2026-02-27 00:51:51,490 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-internal is running."
}
2026-02-27 00:51:51,498 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-compactor-grpc"] *********
2026-02-27 00:51:51,498 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:51 +0000 (0:00:00.024)       0:00:13.207 ******* 
2026-02-27 00:51:51,858 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:51,865 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-compactor-grpc service is running in openstack namespace] ***
2026-02-27 00:51:51,866 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:51 +0000 (0:00:00.367)       0:00:13.574 ******* 
2026-02-27 00:51:51,882 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-compactor-grpc is running."
}
2026-02-27 00:51:51,890 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-compactor-http"] *********
2026-02-27 00:51:51,890 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:51 +0000 (0:00:00.024)       0:00:13.598 ******* 
2026-02-27 00:51:52,206 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:52,215 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-compactor-http service is running in openstack namespace] ***
2026-02-27 00:51:52,215 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:52 +0000 (0:00:00.325)       0:00:13.923 ******* 
2026-02-27 00:51:52,231 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-compactor-http is running."
}
2026-02-27 00:51:52,239 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-distributor-grpc"] *******
2026-02-27 00:51:52,239 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:52 +0000 (0:00:00.024)       0:00:13.948 ******* 
2026-02-27 00:51:52,621 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:52,629 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-distributor-grpc service is running in openstack namespace] ***
2026-02-27 00:51:52,629 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:52 +0000 (0:00:00.390)       0:00:14.338 ******* 
2026-02-27 00:51:52,647 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-distributor-grpc is running."
}
2026-02-27 00:51:52,656 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-distributor-http"] *******
2026-02-27 00:51:52,656 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:52 +0000 (0:00:00.026)       0:00:14.365 ******* 
2026-02-27 00:51:53,038 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:53,047 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-distributor-http service is running in openstack namespace] ***
2026-02-27 00:51:53,047 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:53 +0000 (0:00:00.391)       0:00:14.756 ******* 
2026-02-27 00:51:53,067 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-distributor-http is running."
}
2026-02-27 00:51:53,076 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-gateway-http"] ***********
2026-02-27 00:51:53,077 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:53 +0000 (0:00:00.029)       0:00:14.785 ******* 
2026-02-27 00:51:53,432 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:53,439 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-gateway-http service is running in openstack namespace] ***
2026-02-27 00:51:53,440 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:53 +0000 (0:00:00.362)       0:00:15.148 ******* 
2026-02-27 00:51:53,456 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-gateway-http is running."
}
2026-02-27 00:51:53,465 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-gossip-ring"] ************
2026-02-27 00:51:53,465 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:53 +0000 (0:00:00.025)       0:00:15.173 ******* 
2026-02-27 00:51:53,818 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:53,829 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-gossip-ring service is running in openstack namespace] ***
2026-02-27 00:51:53,829 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:53 +0000 (0:00:00.364)       0:00:15.538 ******* 
2026-02-27 00:51:53,848 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-gossip-ring is running."
}
2026-02-27 00:51:53,856 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-index-gateway-grpc"] *****
2026-02-27 00:51:53,857 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:53 +0000 (0:00:00.027)       0:00:15.565 ******* 
2026-02-27 00:51:54,229 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:54,237 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-index-gateway-grpc service is running in openstack namespace] ***
2026-02-27 00:51:54,237 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:54 +0000 (0:00:00.380)       0:00:15.945 ******* 
2026-02-27 00:51:54,256 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-index-gateway-grpc is running."
}
2026-02-27 00:51:54,263 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-index-gateway-http"] *****
2026-02-27 00:51:54,263 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:54 +0000 (0:00:00.026)       0:00:15.972 ******* 
2026-02-27 00:51:54,580 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:54,588 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-index-gateway-http service is running in openstack namespace] ***
2026-02-27 00:51:54,588 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:54 +0000 (0:00:00.324)       0:00:16.297 ******* 
2026-02-27 00:51:54,605 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-index-gateway-http is running."
}
2026-02-27 00:51:54,614 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-ingester-grpc"] **********
2026-02-27 00:51:54,614 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:54 +0000 (0:00:00.025)       0:00:16.322 ******* 
2026-02-27 00:51:54,944 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:54,953 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-ingester-grpc service is running in openstack namespace] ***
2026-02-27 00:51:54,953 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:54 +0000 (0:00:00.338)       0:00:16.661 ******* 
2026-02-27 00:51:54,970 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-ingester-grpc is running."
}
2026-02-27 00:51:54,978 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-ingester-http"] **********
2026-02-27 00:51:54,978 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:54 +0000 (0:00:00.025)       0:00:16.687 ******* 
2026-02-27 00:51:55,326 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:55,336 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-ingester-http service is running in openstack namespace] ***
2026-02-27 00:51:55,337 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:55 +0000 (0:00:00.358)       0:00:17.045 ******* 
2026-02-27 00:51:55,357 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-ingester-http is running."
}
2026-02-27 00:51:55,366 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-querier-grpc"] ***********
2026-02-27 00:51:55,367 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:55 +0000 (0:00:00.029)       0:00:17.075 ******* 
2026-02-27 00:51:55,720 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:55,729 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-querier-grpc service is running in openstack namespace] ***
2026-02-27 00:51:55,730 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:55 +0000 (0:00:00.363)       0:00:17.438 ******* 
2026-02-27 00:51:55,746 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-querier-grpc is running."
}
2026-02-27 00:51:55,754 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-querier-http"] ***********
2026-02-27 00:51:55,754 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:55 +0000 (0:00:00.023)       0:00:17.462 ******* 
2026-02-27 00:51:56,067 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:56,074 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-querier-http service is running in openstack namespace] ***
2026-02-27 00:51:56,075 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:56 +0000 (0:00:00.320)       0:00:17.783 ******* 
2026-02-27 00:51:56,091 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-querier-http is running."
}
2026-02-27 00:51:56,098 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-query-frontend-grpc"] ****
2026-02-27 00:51:56,099 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:56 +0000 (0:00:00.023)       0:00:17.807 ******* 
2026-02-27 00:51:56,434 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:56,442 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-query-frontend-grpc service is running in openstack namespace] ***
2026-02-27 00:51:56,442 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:56 +0000 (0:00:00.343)       0:00:18.151 ******* 
2026-02-27 00:51:56,459 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-query-frontend-grpc is running."
}
2026-02-27 00:51:56,466 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-query-frontend-http"] ****
2026-02-27 00:51:56,466 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:56 +0000 (0:00:00.024)       0:00:18.175 ******* 
2026-02-27 00:51:56,791 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:56,800 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-query-frontend-http service is running in openstack namespace] ***
2026-02-27 00:51:56,800 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:56 +0000 (0:00:00.333)       0:00:18.508 ******* 
2026-02-27 00:51:56,819 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-query-frontend-http is running."
}
2026-02-27 00:51:56,827 p=37229 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-public"] ***************************
2026-02-27 00:51:56,827 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:56 +0000 (0:00:00.027)       0:00:18.536 ******* 
2026-02-27 00:51:57,193 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:51:57,203 p=37229 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-public service is running in openstack namespace] ***
2026-02-27 00:51:57,203 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:57 +0000 (0:00:00.375)       0:00:18.911 ******* 
2026-02-27 00:51:57,219 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-public is running."
}
2026-02-27 00:51:57,227 p=37229 u=zuul n=ansible | TASK [common : Run file tests] *************************************************
2026-02-27 00:51:57,228 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:57 +0000 (0:00:00.024)       0:00:18.936 ******* 
2026-02-27 00:51:57,243 p=37229 u=zuul n=ansible | skipping: [localhost]
2026-02-27 00:51:57,251 p=37229 u=zuul n=ansible | TASK [common : Get the manifest list once] *************************************
2026-02-27 00:51:57,251 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:57 +0000 (0:00:00.023)       0:00:18.959 ******* 
2026-02-27 00:51:57,264 p=37229 u=zuul n=ansible | skipping: [localhost]
2026-02-27 00:51:57,272 p=37229 u=zuul n=ansible | TASK [common : Check that each package manifest is present] ********************
2026-02-27 00:51:57,272 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:57 +0000 (0:00:00.020)       0:00:18.980 ******* 
2026-02-27 00:51:57,287 p=37229 u=zuul n=ansible | skipping: [localhost]
2026-02-27 00:51:57,296 p=37229 u=zuul n=ansible | TASK [common : Run crd tests] **************************************************
2026-02-27 00:51:57,296 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:57 +0000 (0:00:00.024)       0:00:19.005 ******* 
2026-02-27 00:51:57,312 p=37229 u=zuul n=ansible | skipping: [localhost]
2026-02-27 00:51:57,319 p=37229 u=zuul n=ansible | TASK [common : Run CR tests] ***************************************************
2026-02-27 00:51:57,319 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:57 +0000 (0:00:00.023)       0:00:19.028 ******* 
2026-02-27 00:51:57,335 p=37229 u=zuul n=ansible | skipping: [localhost]
2026-02-27 00:51:57,342 p=37229 u=zuul n=ansible | TASK [common : Verify container tests] *****************************************
2026-02-27 00:51:57,342 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:57 +0000 (0:00:00.022)       0:00:19.050 ******* 
2026-02-27 00:51:57,357 p=37229 u=zuul n=ansible | skipping: [localhost]
2026-02-27 00:51:57,369 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Validate Chargeback Feature deployed correctly] ***
2026-02-27 00:51:57,369 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:57 +0000 (0:00:00.027)       0:00:19.078 ******* 
2026-02-27 00:51:57,387 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/chargeback_tests.yml for localhost
2026-02-27 00:51:57,396 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Enable Cloudkitty Module (hashmap)] ***************
2026-02-27 00:51:57,396 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:51:57 +0000 (0:00:00.026)       0:00:19.104 ******* 
2026-02-27 00:52:01,169 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 00:52:01,176 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Find the current value of hashmap] ****************
2026-02-27 00:52:01,176 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:01 +0000 (0:00:03.780)       0:00:22.885 ******* 
2026-02-27 00:52:03,835 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:03,843 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Change priority for CloudKitty hashmap module] ****
2026-02-27 00:52:03,843 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:03 +0000 (0:00:02.667)       0:00:25.552 ******* 
2026-02-27 00:52:06,964 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 00:52:06,973 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get status of all CloudKitty rating modules] ******
2026-02-27 00:52:06,974 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:06 +0000 (0:00:03.130)       0:00:28.682 ******* 
2026-02-27 00:52:09,239 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:09,249 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Validate CloudKitty module states] ***********
2026-02-27 00:52:09,249 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:09 +0000 (0:00:02.275)       0:00:30.958 ******* 
2026-02-27 00:52:09,282 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "SUCCESS: CloudKitty modules (hashmap=True) are configured correctly."
}
2026-02-27 00:52:09,290 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Set priority for CloudKitty hashmap module] ***
2026-02-27 00:52:09,290 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:09 +0000 (0:00:00.040)       0:00:30.999 ******* 
2026-02-27 00:52:09,316 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "SUCCESS: The hashmap priority is set to 100"
}
2026-02-27 00:52:09,322 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Setup Loki Environment] ***************************
2026-02-27 00:52:09,322 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:09 +0000 (0:00:00.032)       0:00:31.031 ******* 
2026-02-27 00:52:09,349 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/setup_loki_env.yml for localhost
2026-02-27 00:52:09,362 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get Loki Public Route Host] ***********************
2026-02-27 00:52:09,362 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:09 +0000 (0:00:00.039)       0:00:31.071 ******* 
2026-02-27 00:52:09,719 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:09,727 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set Loki URLs] ************************************
2026-02-27 00:52:09,727 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:09 +0000 (0:00:00.365)       0:00:31.436 ******* 
2026-02-27 00:52:09,742 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:09,749 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set Derived Loki URLs] ****************************
2026-02-27 00:52:09,749 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:09 +0000 (0:00:00.021)       0:00:31.457 ******* 
2026-02-27 00:52:09,762 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:09,769 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Debug URLs] ***************************************
2026-02-27 00:52:09,769 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:09 +0000 (0:00:00.020)       0:00:31.478 ******* 
2026-02-27 00:52:09,785 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "msg": [
        "Loki Route: https://cloudkitty-lokistack-openstack.apps-crc.testing",
        "Push URL:   https://cloudkitty-lokistack-openstack.apps-crc.testing/api/logs/v1/cloudkitty/loki/api/v1/push",
        "Flush URL:  https://cloudkitty-lokistack-ingester-http.openstack.svc:3100/flush",
        "Query URL:  https://cloudkitty-lokistack-openstack.apps-crc.testing/api/logs/v1/cloudkitty/loki/api/v1/query_range"
    ]
}
2026-02-27 00:52:09,791 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Ensure Local Certificate Directory Exists] ********
2026-02-27 00:52:09,792 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:09 +0000 (0:00:00.022)       0:00:31.500 ******* 
2026-02-27 00:52:10,129 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 00:52:10,137 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Extract Certificates from Openshift Secret] *******
2026-02-27 00:52:10,137 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:10 +0000 (0:00:00.345)       0:00:31.846 ******* 
2026-02-27 00:52:10,477 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 00:52:10,484 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Extract Client Certificates] **********************
2026-02-27 00:52:10,484 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:10 +0000 (0:00:00.347)       0:00:32.193 ******* 
2026-02-27 00:52:10,842 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 00:52:10,850 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Extract CA Bundle] ********************************
2026-02-27 00:52:10,850 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:10 +0000 (0:00:00.365)       0:00:32.559 ******* 
2026-02-27 00:52:11,203 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 00:52:11,212 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get admin project ID for CI] **********************
2026-02-27 00:52:11,212 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:11 +0000 (0:00:00.361)       0:00:32.920 ******* 
2026-02-27 00:52:13,451 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:13,459 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set admin project ID for CI] **********************
2026-02-27 00:52:13,459 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:13 +0000 (0:00:02.247)       0:00:35.168 ******* 
2026-02-27 00:52:13,479 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:13,487 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get admin user ID for CI] *************************
2026-02-27 00:52:13,487 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:13 +0000 (0:00:00.028)       0:00:35.196 ******* 
2026-02-27 00:52:15,804 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:15,816 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set admin user ID for CI] *************************
2026-02-27 00:52:15,817 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:15 +0000 (0:00:02.329)       0:00:37.525 ******* 
2026-02-27 00:52:15,838 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:15,847 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Find test files] **********************************
2026-02-27 00:52:15,848 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:15 +0000 (0:00:00.030)       0:00:37.556 ******* 
2026-02-27 00:52:16,247 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:16,260 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Extract only the filenames into a clean list] *****
2026-02-27 00:52:16,260 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:16 +0000 (0:00:00.412)       0:00:37.969 ******* 
2026-02-27 00:52:16,279 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:16,288 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Process and Loop if files exist] ******************
2026-02-27 00:52:16,288 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:16 +0000 (0:00:00.027)       0:00:37.997 ******* 
2026-02-27 00:52:16,344 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_all_qty_zero)
2026-02-27 00:52:16,355 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_dyn_basic)
2026-02-27 00:52:16,364 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_static_basic)
2026-02-27 00:52:16,371 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_static_basic_10x)
2026-02-27 00:52:16,378 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_static_basic_gid)
2026-02-27 00:52:16,388 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_all_qty_zero] ***
2026-02-27 00:52:16,388 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:16 +0000 (0:00:00.099)       0:00:38.097 ******* 
2026-02-27 00:52:16,410 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost
2026-02-27 00:52:16,421 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically test_all_qty_zero] ******
2026-02-27 00:52:16,421 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:16 +0000 (0:00:00.033)       0:00:38.130 ******* 
2026-02-27 00:52:16,440 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:16,447 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] ****************
2026-02-27 00:52:16,447 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:16 +0000 (0:00:00.025)       0:00:38.155 ******* 
2026-02-27 00:52:16,806 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:16,814 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data test_all_qty_zero] ********
2026-02-27 00:52:16,814 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:16 +0000 (0:00:00.367)       0:00:38.522 ******* 
2026-02-27 00:52:17,143 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 00:52:17,151 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file test_all_qty_zero] ***
2026-02-27 00:52:17,151 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:17 +0000 (0:00:00.336)       0:00:38.859 ******* 
2026-02-27 00:52:17,450 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 00:52:17,458 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] **********************
2026-02-27 00:52:17,458 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:17 +0000 (0:00:00.307)       0:00:39.166 ******* 
2026-02-27 00:52:17,474 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:17,483 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_all_qty_zero] *************
2026-02-27 00:52:17,483 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:17 +0000 (0:00:00.025)       0:00:39.192 ******* 
2026-02-27 00:52:17,505 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost
2026-02-27 00:52:17,518 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log: test_all_qty_zero] ***
2026-02-27 00:52:17,518 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:17 +0000 (0:00:00.034)       0:00:39.227 ******* 
2026-02-27 00:52:17,539 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost
2026-02-27 00:52:17,549 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] ****************************
2026-02-27 00:52:17,549 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:17 +0000 (0:00:00.030)       0:00:39.257 ******* 
2026-02-27 00:52:17,855 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:17,865 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ********************************
2026-02-27 00:52:17,865 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:17 +0000 (0:00:00.316)       0:00:39.574 ******* 
2026-02-27 00:52:18,635 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:18,643 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] **********************************
2026-02-27 00:52:18,643 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:18 +0000 (0:00:00.778)       0:00:40.352 ******* 
2026-02-27 00:52:18,655 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Ingestion Successful!"
}
2026-02-27 00:52:18,662 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage: test_all_qty_zero] ****
2026-02-27 00:52:18,662 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:18 +0000 (0:00:00.018)       0:00:40.371 ******* 
2026-02-27 00:52:18,714 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost
2026-02-27 00:52:18,726 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************
2026-02-27 00:52:18,726 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:18 +0000 (0:00:00.063)       0:00:40.435 ******* 
2026-02-27 00:52:19,153 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:19,159 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] *******
2026-02-27 00:52:19,159 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:19 +0000 (0:00:00.433)       0:00:40.868 ******* 
2026-02-27 00:52:19,369 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 00:52:19,379 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] ***************
2026-02-27 00:52:19,379 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:19 +0000 (0:00:00.219)       0:00:41.087 ******* 
2026-02-27 00:52:20,007 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 00:52:20,020 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************
2026-02-27 00:52:20,020 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:20 +0000 (0:00:00.641)       0:00:41.728 ******* 
2026-02-27 00:52:20,496 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 00:52:20,505 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ******************************
2026-02-27 00:52:20,505 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:20 +0000 (0:00:00.484)       0:00:42.213 ******* 
2026-02-27 00:52:20,523 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "Ingester Memory Flushed successfully"
}
2026-02-27 00:52:20,532 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki: test_all_qty_zero] ***
2026-02-27 00:52:20,532 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:20 +0000 (0:00:00.026)       0:00:42.240 ******* 
2026-02-27 00:52:20,556 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost
2026-02-27 00:52:20,581 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_all_qty_zero] *****************
2026-02-27 00:52:20,581 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:20 +0000 (0:00:00.049)       0:00:42.289 ******* 
2026-02-27 00:52:20,598 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Input file has 42 data entries that Loki has to return"
}
2026-02-27 00:52:20,606 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] ***********************************
2026-02-27 00:52:20,606 p=37229 u=zuul n=ansible | Friday 27 February 2026  00:52:20 +0000 (0:00:00.025)       0:00:42.315 ******* 
2026-02-27 01:06:28,865 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:28,880 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file] **********************
2026-02-27 01:06:28,881 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:28 +0000 (0:14:08.274)       0:14:50.589 ******* 
2026-02-27 01:06:29,391 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:29,400 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_all_qty_zero] **********
2026-02-27 01:06:29,401 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:29 +0000 (0:00:00.519)       0:14:51.109 ******* 
2026-02-27 01:06:29,437 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "Query returned all data entries. Input file had 42 entries and Loki returned 42"
}
2026-02-27 01:06:29,446 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_all_qty_zero] ***
2026-02-27 01:06:29,446 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:29 +0000 (0:00:00.045)       0:14:51.155 ******* 
2026-02-27 01:06:29,714 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:29,722 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_all_qty_zero] ******
2026-02-27 01:06:29,722 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:29 +0000 (0:00:00.275)       0:14:51.431 ******* 
2026-02-27 01:06:29,740 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost
2026-02-27 01:06:29,757 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_all_qty_zero] ***
2026-02-27 01:06:29,757 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:29 +0000 (0:00:00.035)       0:14:51.466 ******* 
2026-02-27 01:06:36,172 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:36,182 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_all_qty_zero] ***
2026-02-27 01:06:36,182 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:36 +0000 (0:00:06.424)       0:14:57.891 ******* 
2026-02-27 01:06:36,204 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_by_type": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get",
            "-g",
            "type"
        ],
        "delta": "0:00:06.183086",
        "end": "2026-02-27 01:06:36.141352",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-27 01:06:29.958266",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-27 01:06:36,220 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_all_qty_zero] ***
2026-02-27 01:06:36,220 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:36 +0000 (0:00:00.037)       0:14:57.928 ******* 
2026-02-27 01:06:36,627 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:36,637 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_all_qty_zero] ***
2026-02-27 01:06:36,637 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:36 +0000 (0:00:00.416)       0:14:58.345 ******* 
2026-02-27 01:06:39,683 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:39,692 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_all_qty_zero] ***
2026-02-27 01:06:39,693 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:39 +0000 (0:00:03.055)       0:15:01.401 ******* 
2026-02-27 01:06:39,711 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_summary": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get"
        ],
        "delta": "0:00:02.801543",
        "end": "2026-02-27 01:06:39.648346",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-27 01:06:36.846803",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-27 01:06:39,719 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** help for openstack rating summary cmd] ***
2026-02-27 01:06:39,719 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:39 +0000 (0:00:00.026)       0:15:01.427 ******* 
2026-02-27 01:06:41,735 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:41,744 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary get help output] ***
2026-02-27 01:06:41,744 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:41 +0000 (0:00:02.025)       0:15:03.453 ******* 
2026-02-27 01:06:41,769 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_h.stdout_lines": [
        "usage: openstack rating summary get [-h]",
        "                                    [-f {csv,df-to-csv,json,table,value,yaml}]",
        "                                    [-c COLUMN]",
        "                                    [--quote {all,minimal,none,nonnumeric}]",
        "                                    [--noindent] [--max-width <integer>]",
        "                                    [--fit-width] [--print-empty]",
        "                                    [--format-config-file FORMAT_CONFIG]",
        "                                    [--sort-column SORT_COLUMN]",
        "                                    [--sort-ascending | --sort-descending]",
        "                                    [-t TENANT_ID] [-b BEGIN] [-e END]",
        "                                    [-s SERVICE] [-g GROUPBY [GROUPBY ...]]",
        "                                    [-a]",
        "",
        "Get a summary report.",
        "",
        "optional arguments:",
        "  -h, --help            show this help message and exit",
        "  -t TENANT_ID, --tenant-id TENANT_ID",
        "                        Tenant id.",
        "  -b BEGIN, --begin BEGIN",
        "                        Begin timestamp.",
        "  -e END, --end END     End timestamp.",
        "  -s SERVICE, --service SERVICE",
        "                        Service Type.",
        "  -g GROUPBY [GROUPBY ...], --groupby GROUPBY [GROUPBY ...]",
        "                        Fields to group by, space-separated. (res_type and",
        "                        tenant_id are supported for now)",
        "  -a, --all-tenants     Allows to get summary from all tenants (admin only).",
        "                        Defaults to False.",
        "",
        "output formatters:",
        "  output formatter options",
        "",
        "  -f {csv,df-to-csv,json,table,value,yaml}, --format {csv,df-to-csv,json,table,value,yaml}",
        "                        the output format, defaults to table",
        "  -c COLUMN, --column COLUMN",
        "                        specify the column(s) to include, can be repeated to",
        "                        show multiple columns",
        "  --sort-column SORT_COLUMN",
        "                        specify the column(s) to sort the data (columns",
        "                        specified first have a priority, non-existing columns",
        "                        are ignored), can be repeated",
        "  --sort-ascending      sort the column(s) in ascending order",
        "  --sort-descending     sort the column(s) in descending order",
        "",
        "CSV Formatter:",
        "  --quote {all,minimal,none,nonnumeric}",
        "                        when to include quotes, defaults to nonnumeric",
        "",
        "json formatter:",
        "  --noindent            whether to disable indenting the JSON",
        "",
        "table formatter:",
        "  --max-width <integer>",
        "                        Maximum display width, <1 to disable. You can also use",
        "                        the CLIFF_MAX_TERM_WIDTH environment variable, but the",
        "                        parameter takes precedence.",
        "  --fit-width           Fit the table to the display width. Implied if --max-",
        "                        width greater than 0. Set the environment variable",
        "                        CLIFF_FIT_WIDTH=1 to always enable",
        "  --print-empty         Print empty table if there is no data to show.",
        "",
        "dataframe-to-csv formatter:",
        "  --format-config-file FORMAT_CONFIG",
        "                        Config file for the dict-to-csv formatter",
        "",
        "This command is provided by the python-cloudkittyclient plugin."
    ]
}
2026-02-27 01:06:41,778 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic totals file] *******************
2026-02-27 01:06:41,778 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:41 +0000 (0:00:00.033)       0:15:03.487 ******* 
2026-02-27 01:06:42,027 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:42,035 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] ***********************
2026-02-27 01:06:42,036 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:42 +0000 (0:00:00.257)       0:15:03.744 ******* 
2026-02-27 01:06:42,279 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:42,290 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data results test_all_qty_zero] ***
2026-02-27 01:06:42,291 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:42 +0000 (0:00:00.254)       0:15:03.999 ******* 
2026-02-27 01:06:42,323 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "PASSED - Data totals are identical."
}
2026-02-27 01:06:42,334 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_dyn_basic] ***
2026-02-27 01:06:42,335 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:42 +0000 (0:00:00.044)       0:15:04.043 ******* 
2026-02-27 01:06:42,364 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost
2026-02-27 01:06:42,380 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically test_dyn_basic] *********
2026-02-27 01:06:42,380 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:42 +0000 (0:00:00.045)       0:15:04.089 ******* 
2026-02-27 01:06:42,405 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:42,415 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] ****************
2026-02-27 01:06:42,415 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:42 +0000 (0:00:00.034)       0:15:04.123 ******* 
2026-02-27 01:06:42,662 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:42,672 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data test_dyn_basic] ***********
2026-02-27 01:06:42,672 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:42 +0000 (0:00:00.257)       0:15:04.381 ******* 
2026-02-27 01:06:43,064 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:43,075 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file test_dyn_basic] ***
2026-02-27 01:06:43,075 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:43 +0000 (0:00:00.403)       0:15:04.784 ******* 
2026-02-27 01:06:43,376 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:43,384 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] **********************
2026-02-27 01:06:43,384 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:43 +0000 (0:00:00.308)       0:15:05.093 ******* 
2026-02-27 01:06:43,408 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:43,420 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_dyn_basic] ****************
2026-02-27 01:06:43,420 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:43 +0000 (0:00:00.035)       0:15:05.129 ******* 
2026-02-27 01:06:43,449 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost
2026-02-27 01:06:43,463 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log: test_dyn_basic] ******
2026-02-27 01:06:43,464 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:43 +0000 (0:00:00.043)       0:15:05.172 ******* 
2026-02-27 01:06:43,486 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost
2026-02-27 01:06:43,495 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] ****************************
2026-02-27 01:06:43,495 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:43 +0000 (0:00:00.031)       0:15:05.204 ******* 
2026-02-27 01:06:43,776 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:43,786 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ********************************
2026-02-27 01:06:43,786 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:43 +0000 (0:00:00.291)       0:15:05.495 ******* 
2026-02-27 01:06:44,245 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:44,252 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] **********************************
2026-02-27 01:06:44,252 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:44 +0000 (0:00:00.466)       0:15:05.961 ******* 
2026-02-27 01:06:44,268 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Ingestion Successful!"
}
2026-02-27 01:06:44,277 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage: test_dyn_basic] *******
2026-02-27 01:06:44,277 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:44 +0000 (0:00:00.025)       0:15:05.986 ******* 
2026-02-27 01:06:44,297 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost
2026-02-27 01:06:44,312 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************
2026-02-27 01:06:44,313 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:44 +0000 (0:00:00.035)       0:15:06.021 ******* 
2026-02-27 01:06:44,793 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:44,803 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] *******
2026-02-27 01:06:44,803 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:44 +0000 (0:00:00.490)       0:15:06.511 ******* 
2026-02-27 01:06:45,092 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:45,100 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] ***************
2026-02-27 01:06:45,101 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:45 +0000 (0:00:00.297)       0:15:06.809 ******* 
2026-02-27 01:06:45,669 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:45,679 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************
2026-02-27 01:06:45,680 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:45 +0000 (0:00:00.578)       0:15:07.388 ******* 
2026-02-27 01:06:46,180 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:46,189 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ******************************
2026-02-27 01:06:46,189 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:46 +0000 (0:00:00.509)       0:15:07.898 ******* 
2026-02-27 01:06:46,216 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "Ingester Memory Flushed successfully"
}
2026-02-27 01:06:46,226 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki: test_dyn_basic] ******
2026-02-27 01:06:46,227 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:46 +0000 (0:00:00.037)       0:15:07.935 ******* 
2026-02-27 01:06:46,253 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost
2026-02-27 01:06:46,276 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_dyn_basic] ********************
2026-02-27 01:06:46,277 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:46 +0000 (0:00:00.050)       0:15:07.985 ******* 
2026-02-27 01:06:46,302 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Input file has 18 data entries that Loki has to return"
}
2026-02-27 01:06:46,310 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] ***********************************
2026-02-27 01:06:46,311 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:46 +0000 (0:00:00.033)       0:15:08.019 ******* 
2026-02-27 01:06:46,860 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:46,871 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file] **********************
2026-02-27 01:06:46,872 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:46 +0000 (0:00:00.561)       0:15:08.580 ******* 
2026-02-27 01:06:47,329 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:47,340 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_dyn_basic] *************
2026-02-27 01:06:47,341 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:47 +0000 (0:00:00.468)       0:15:09.049 ******* 
2026-02-27 01:06:47,386 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {
    "assertion": "actual_count | int == (synth_data_rates.data_log.log_count | int)",
    "changed": false,
    "evaluated_to": false,
    "msg": "Query did not return all data entries. Expected 18 log entries, but Loki only returned 35"
}
2026-02-27 01:06:47,386 p=37229 u=zuul n=ansible | ...ignoring
2026-02-27 01:06:47,398 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_dyn_basic] ***
2026-02-27 01:06:47,398 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:47 +0000 (0:00:00.057)       0:15:09.106 ******* 
2026-02-27 01:06:47,759 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:47,769 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_dyn_basic] *********
2026-02-27 01:06:47,770 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:47 +0000 (0:00:00.371)       0:15:09.478 ******* 
2026-02-27 01:06:47,796 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost
2026-02-27 01:06:47,814 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_dyn_basic] ***
2026-02-27 01:06:47,815 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:47 +0000 (0:00:00.045)       0:15:09.523 ******* 
2026-02-27 01:06:50,533 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:50,544 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_dyn_basic] ***
2026-02-27 01:06:50,544 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:50 +0000 (0:00:02.729)       0:15:12.253 ******* 
2026-02-27 01:06:50,566 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_by_type": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get",
            "-g",
            "type"
        ],
        "delta": "0:00:02.455566",
        "end": "2026-02-27 01:06:50.494377",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-27 01:06:48.038811",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-27 01:06:50,576 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_dyn_basic] ***
2026-02-27 01:06:50,577 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:50 +0000 (0:00:00.032)       0:15:12.286 ******* 
2026-02-27 01:06:51,074 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:51,083 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_dyn_basic] ***
2026-02-27 01:06:51,083 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:51 +0000 (0:00:00.506)       0:15:12.792 ******* 
2026-02-27 01:06:54,085 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:54,094 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_dyn_basic] ***
2026-02-27 01:06:54,094 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:54 +0000 (0:00:03.010)       0:15:15.802 ******* 
2026-02-27 01:06:54,115 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_summary": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get"
        ],
        "delta": "0:00:02.757318",
        "end": "2026-02-27 01:06:54.050579",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-27 01:06:51.293261",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-27 01:06:54,124 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** help for openstack rating summary cmd] ***
2026-02-27 01:06:54,124 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:54 +0000 (0:00:00.030)       0:15:15.833 ******* 
2026-02-27 01:06:56,061 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:56,069 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary get help output] ***
2026-02-27 01:06:56,069 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:56 +0000 (0:00:01.944)       0:15:17.777 ******* 
2026-02-27 01:06:56,087 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_h.stdout_lines": [
        "usage: openstack rating summary get [-h]",
        "                                    [-f {csv,df-to-csv,json,table,value,yaml}]",
        "                                    [-c COLUMN]",
        "                                    [--quote {all,minimal,none,nonnumeric}]",
        "                                    [--noindent] [--max-width <integer>]",
        "                                    [--fit-width] [--print-empty]",
        "                                    [--format-config-file FORMAT_CONFIG]",
        "                                    [--sort-column SORT_COLUMN]",
        "                                    [--sort-ascending | --sort-descending]",
        "                                    [-t TENANT_ID] [-b BEGIN] [-e END]",
        "                                    [-s SERVICE] [-g GROUPBY [GROUPBY ...]]",
        "                                    [-a]",
        "",
        "Get a summary report.",
        "",
        "optional arguments:",
        "  -h, --help            show this help message and exit",
        "  -t TENANT_ID, --tenant-id TENANT_ID",
        "                        Tenant id.",
        "  -b BEGIN, --begin BEGIN",
        "                        Begin timestamp.",
        "  -e END, --end END     End timestamp.",
        "  -s SERVICE, --service SERVICE",
        "                        Service Type.",
        "  -g GROUPBY [GROUPBY ...], --groupby GROUPBY [GROUPBY ...]",
        "                        Fields to group by, space-separated. (res_type and",
        "                        tenant_id are supported for now)",
        "  -a, --all-tenants     Allows to get summary from all tenants (admin only).",
        "                        Defaults to False.",
        "",
        "output formatters:",
        "  output formatter options",
        "",
        "  -f {csv,df-to-csv,json,table,value,yaml}, --format {csv,df-to-csv,json,table,value,yaml}",
        "                        the output format, defaults to table",
        "  -c COLUMN, --column COLUMN",
        "                        specify the column(s) to include, can be repeated to",
        "                        show multiple columns",
        "  --sort-column SORT_COLUMN",
        "                        specify the column(s) to sort the data (columns",
        "                        specified first have a priority, non-existing columns",
        "                        are ignored), can be repeated",
        "  --sort-ascending      sort the column(s) in ascending order",
        "  --sort-descending     sort the column(s) in descending order",
        "",
        "CSV Formatter:",
        "  --quote {all,minimal,none,nonnumeric}",
        "                        when to include quotes, defaults to nonnumeric",
        "",
        "json formatter:",
        "  --noindent            whether to disable indenting the JSON",
        "",
        "table formatter:",
        "  --max-width <integer>",
        "                        Maximum display width, <1 to disable. You can also use",
        "                        the CLIFF_MAX_TERM_WIDTH environment variable, but the",
        "                        parameter takes precedence.",
        "  --fit-width           Fit the table to the display width. Implied if --max-",
        "                        width greater than 0. Set the environment variable",
        "                        CLIFF_FIT_WIDTH=1 to always enable",
        "  --print-empty         Print empty table if there is no data to show.",
        "",
        "dataframe-to-csv formatter:",
        "  --format-config-file FORMAT_CONFIG",
        "                        Config file for the dict-to-csv formatter",
        "",
        "This command is provided by the python-cloudkittyclient plugin."
    ]
}
2026-02-27 01:06:56,095 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic totals file] *******************
2026-02-27 01:06:56,095 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:56 +0000 (0:00:00.026)       0:15:17.803 ******* 
2026-02-27 01:06:56,293 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:56,300 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] ***********************
2026-02-27 01:06:56,301 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:56 +0000 (0:00:00.205)       0:15:18.009 ******* 
2026-02-27 01:06:56,508 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:56,515 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data results test_dyn_basic] ***
2026-02-27 01:06:56,516 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:56 +0000 (0:00:00.215)       0:15:18.224 ******* 
2026-02-27 01:06:56,535 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {
    "assertion": "(synth_data.content | b64decode | from_yaml).synth_rate == (loki_data.content | b64decode | from_yaml).synth_rate",
    "changed": false,
    "evaluated_to": false,
    "msg": "METRICS MISMATCH!\nThe synth_rate or data_log totals do not match between:\ntest_dyn_basic-synth_metrics_totals.yml\ntest_dyn_basic-loki_metrics_totals.yml\n"
}
2026-02-27 01:06:56,535 p=37229 u=zuul n=ansible | ...ignoring
2026-02-27 01:06:56,545 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic] ***
2026-02-27 01:06:56,545 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:56 +0000 (0:00:00.029)       0:15:18.253 ******* 
2026-02-27 01:06:56,563 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost
2026-02-27 01:06:56,574 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically test_static_basic] ******
2026-02-27 01:06:56,574 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:56 +0000 (0:00:00.029)       0:15:18.283 ******* 
2026-02-27 01:06:56,594 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:56,601 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] ****************
2026-02-27 01:06:56,601 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:56 +0000 (0:00:00.026)       0:15:18.310 ******* 
2026-02-27 01:06:56,808 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:56,822 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data test_static_basic] ********
2026-02-27 01:06:56,822 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:56 +0000 (0:00:00.221)       0:15:18.531 ******* 
2026-02-27 01:06:57,213 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:57,222 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic] ***
2026-02-27 01:06:57,222 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:57 +0000 (0:00:00.399)       0:15:18.931 ******* 
2026-02-27 01:06:57,564 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:57,573 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] **********************
2026-02-27 01:06:57,573 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:57 +0000 (0:00:00.350)       0:15:19.281 ******* 
2026-02-27 01:06:57,594 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:57,604 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_static_basic] *************
2026-02-27 01:06:57,604 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:57 +0000 (0:00:00.031)       0:15:19.313 ******* 
2026-02-27 01:06:57,630 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost
2026-02-27 01:06:57,645 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic] ***
2026-02-27 01:06:57,645 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:57 +0000 (0:00:00.040)       0:15:19.354 ******* 
2026-02-27 01:06:57,671 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost
2026-02-27 01:06:57,683 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] ****************************
2026-02-27 01:06:57,684 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:57 +0000 (0:00:00.038)       0:15:19.392 ******* 
2026-02-27 01:06:57,904 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:57,917 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ********************************
2026-02-27 01:06:57,918 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:57 +0000 (0:00:00.234)       0:15:19.626 ******* 
2026-02-27 01:06:58,355 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:58,365 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] **********************************
2026-02-27 01:06:58,365 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:58 +0000 (0:00:00.447)       0:15:20.074 ******* 
2026-02-27 01:06:58,386 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Ingestion Successful!"
}
2026-02-27 01:06:58,397 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage: test_static_basic] ****
2026-02-27 01:06:58,397 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:58 +0000 (0:00:00.031)       0:15:20.106 ******* 
2026-02-27 01:06:58,421 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost
2026-02-27 01:06:58,437 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************
2026-02-27 01:06:58,437 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:58 +0000 (0:00:00.039)       0:15:20.145 ******* 
2026-02-27 01:06:58,886 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:58,896 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] *******
2026-02-27 01:06:58,896 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:58 +0000 (0:00:00.459)       0:15:20.605 ******* 
2026-02-27 01:06:59,141 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:06:59,150 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] ***************
2026-02-27 01:06:59,151 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:59 +0000 (0:00:00.254)       0:15:20.859 ******* 
2026-02-27 01:06:59,722 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:06:59,730 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************
2026-02-27 01:06:59,730 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:06:59 +0000 (0:00:00.579)       0:15:21.439 ******* 
2026-02-27 01:07:00,239 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:00,250 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ******************************
2026-02-27 01:07:00,250 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:00 +0000 (0:00:00.520)       0:15:21.959 ******* 
2026-02-27 01:07:00,274 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "Ingester Memory Flushed successfully"
}
2026-02-27 01:07:00,283 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki: test_static_basic] ***
2026-02-27 01:07:00,283 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:00 +0000 (0:00:00.032)       0:15:21.992 ******* 
2026-02-27 01:07:00,305 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost
2026-02-27 01:07:00,332 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_static_basic] *****************
2026-02-27 01:07:00,332 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:00 +0000 (0:00:00.048)       0:15:22.041 ******* 
2026-02-27 01:07:00,349 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Input file has 24 data entries that Loki has to return"
}
2026-02-27 01:07:00,357 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] ***********************************
2026-02-27 01:07:00,357 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:00 +0000 (0:00:00.025)       0:15:22.066 ******* 
2026-02-27 01:07:00,849 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:00,859 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file] **********************
2026-02-27 01:07:00,859 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:00 +0000 (0:00:00.501)       0:15:22.567 ******* 
2026-02-27 01:07:01,350 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:01,359 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_static_basic] **********
2026-02-27 01:07:01,359 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:01 +0000 (0:00:00.500)       0:15:23.068 ******* 
2026-02-27 01:07:01,411 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {
    "assertion": "actual_count | int == (synth_data_rates.data_log.log_count | int)",
    "changed": false,
    "evaluated_to": false,
    "msg": "Query did not return all data entries. Expected 24 log entries, but Loki only returned 37"
}
2026-02-27 01:07:01,411 p=37229 u=zuul n=ansible | ...ignoring
2026-02-27 01:07:01,421 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic] ***
2026-02-27 01:07:01,421 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:01 +0000 (0:00:00.061)       0:15:23.130 ******* 
2026-02-27 01:07:01,720 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:01,731 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_static_basic] ******
2026-02-27 01:07:01,732 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:01 +0000 (0:00:00.310)       0:15:23.440 ******* 
2026-02-27 01:07:01,758 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost
2026-02-27 01:07:01,777 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic] ***
2026-02-27 01:07:01,777 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:01 +0000 (0:00:00.045)       0:15:23.486 ******* 
2026-02-27 01:07:04,899 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:04,909 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_static_basic] ***
2026-02-27 01:07:04,910 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:04 +0000 (0:00:03.132)       0:15:26.618 ******* 
2026-02-27 01:07:04,934 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_by_type": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get",
            "-g",
            "type"
        ],
        "delta": "0:00:02.855701",
        "end": "2026-02-27 01:07:04.865257",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-27 01:07:02.009556",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-27 01:07:04,944 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_static_basic] ***
2026-02-27 01:07:04,944 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:04 +0000 (0:00:00.034)       0:15:26.652 ******* 
2026-02-27 01:07:05,345 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:05,354 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic] ***
2026-02-27 01:07:05,354 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:05 +0000 (0:00:00.410)       0:15:27.063 ******* 
2026-02-27 01:07:07,930 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:07,944 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_static_basic] ***
2026-02-27 01:07:07,944 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:07 +0000 (0:00:02.589)       0:15:29.652 ******* 
2026-02-27 01:07:07,963 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_summary": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get"
        ],
        "delta": "0:00:02.334306",
        "end": "2026-02-27 01:07:07.897907",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-27 01:07:05.563601",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-27 01:07:07,979 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** help for openstack rating summary cmd] ***
2026-02-27 01:07:07,980 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:07 +0000 (0:00:00.035)       0:15:29.688 ******* 
2026-02-27 01:07:09,890 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:09,898 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary get help output] ***
2026-02-27 01:07:09,898 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:09 +0000 (0:00:01.918)       0:15:31.606 ******* 
2026-02-27 01:07:09,917 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_h.stdout_lines": [
        "usage: openstack rating summary get [-h]",
        "                                    [-f {csv,df-to-csv,json,table,value,yaml}]",
        "                                    [-c COLUMN]",
        "                                    [--quote {all,minimal,none,nonnumeric}]",
        "                                    [--noindent] [--max-width <integer>]",
        "                                    [--fit-width] [--print-empty]",
        "                                    [--format-config-file FORMAT_CONFIG]",
        "                                    [--sort-column SORT_COLUMN]",
        "                                    [--sort-ascending | --sort-descending]",
        "                                    [-t TENANT_ID] [-b BEGIN] [-e END]",
        "                                    [-s SERVICE] [-g GROUPBY [GROUPBY ...]]",
        "                                    [-a]",
        "",
        "Get a summary report.",
        "",
        "optional arguments:",
        "  -h, --help            show this help message and exit",
        "  -t TENANT_ID, --tenant-id TENANT_ID",
        "                        Tenant id.",
        "  -b BEGIN, --begin BEGIN",
        "                        Begin timestamp.",
        "  -e END, --end END     End timestamp.",
        "  -s SERVICE, --service SERVICE",
        "                        Service Type.",
        "  -g GROUPBY [GROUPBY ...], --groupby GROUPBY [GROUPBY ...]",
        "                        Fields to group by, space-separated. (res_type and",
        "                        tenant_id are supported for now)",
        "  -a, --all-tenants     Allows to get summary from all tenants (admin only).",
        "                        Defaults to False.",
        "",
        "output formatters:",
        "  output formatter options",
        "",
        "  -f {csv,df-to-csv,json,table,value,yaml}, --format {csv,df-to-csv,json,table,value,yaml}",
        "                        the output format, defaults to table",
        "  -c COLUMN, --column COLUMN",
        "                        specify the column(s) to include, can be repeated to",
        "                        show multiple columns",
        "  --sort-column SORT_COLUMN",
        "                        specify the column(s) to sort the data (columns",
        "                        specified first have a priority, non-existing columns",
        "                        are ignored), can be repeated",
        "  --sort-ascending      sort the column(s) in ascending order",
        "  --sort-descending     sort the column(s) in descending order",
        "",
        "CSV Formatter:",
        "  --quote {all,minimal,none,nonnumeric}",
        "                        when to include quotes, defaults to nonnumeric",
        "",
        "json formatter:",
        "  --noindent            whether to disable indenting the JSON",
        "",
        "table formatter:",
        "  --max-width <integer>",
        "                        Maximum display width, <1 to disable. You can also use",
        "                        the CLIFF_MAX_TERM_WIDTH environment variable, but the",
        "                        parameter takes precedence.",
        "  --fit-width           Fit the table to the display width. Implied if --max-",
        "                        width greater than 0. Set the environment variable",
        "                        CLIFF_FIT_WIDTH=1 to always enable",
        "  --print-empty         Print empty table if there is no data to show.",
        "",
        "dataframe-to-csv formatter:",
        "  --format-config-file FORMAT_CONFIG",
        "                        Config file for the dict-to-csv formatter",
        "",
        "This command is provided by the python-cloudkittyclient plugin."
    ]
}
2026-02-27 01:07:09,925 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic totals file] *******************
2026-02-27 01:07:09,925 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:09 +0000 (0:00:00.026)       0:15:31.633 ******* 
2026-02-27 01:07:10,138 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:10,147 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] ***********************
2026-02-27 01:07:10,147 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:10 +0000 (0:00:00.222)       0:15:31.855 ******* 
2026-02-27 01:07:10,344 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:10,353 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data results test_static_basic] ***
2026-02-27 01:07:10,353 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:10 +0000 (0:00:00.206)       0:15:32.061 ******* 
2026-02-27 01:07:10,377 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {
    "assertion": "(synth_data.content | b64decode | from_yaml).synth_rate == (loki_data.content | b64decode | from_yaml).synth_rate",
    "changed": false,
    "evaluated_to": false,
    "msg": "METRICS MISMATCH!\nThe synth_rate or data_log totals do not match between:\ntest_static_basic-synth_metrics_totals.yml\ntest_static_basic-loki_metrics_totals.yml\n"
}
2026-02-27 01:07:10,377 p=37229 u=zuul n=ansible | ...ignoring
2026-02-27 01:07:10,388 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic_10x] ***
2026-02-27 01:07:10,388 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:10 +0000 (0:00:00.035)       0:15:32.097 ******* 
2026-02-27 01:07:10,407 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost
2026-02-27 01:07:10,417 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically test_static_basic_10x] ***
2026-02-27 01:07:10,417 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:10 +0000 (0:00:00.028)       0:15:32.126 ******* 
2026-02-27 01:07:10,435 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:10,443 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] ****************
2026-02-27 01:07:10,444 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:10 +0000 (0:00:00.026)       0:15:32.152 ******* 
2026-02-27 01:07:10,741 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:10,750 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data test_static_basic_10x] ****
2026-02-27 01:07:10,750 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:10 +0000 (0:00:00.306)       0:15:32.459 ******* 
2026-02-27 01:07:11,135 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:11,147 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic_10x] ***
2026-02-27 01:07:11,148 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:11 +0000 (0:00:00.397)       0:15:32.856 ******* 
2026-02-27 01:07:11,483 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:11,494 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] **********************
2026-02-27 01:07:11,494 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:11 +0000 (0:00:00.346)       0:15:33.202 ******* 
2026-02-27 01:07:11,517 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:11,526 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_static_basic_10x] *********
2026-02-27 01:07:11,527 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:11 +0000 (0:00:00.032)       0:15:33.235 ******* 
2026-02-27 01:07:11,554 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost
2026-02-27 01:07:11,569 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic_10x] ***
2026-02-27 01:07:11,569 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:11 +0000 (0:00:00.042)       0:15:33.278 ******* 
2026-02-27 01:07:11,594 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost
2026-02-27 01:07:11,606 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] ****************************
2026-02-27 01:07:11,607 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:11 +0000 (0:00:00.037)       0:15:33.315 ******* 
2026-02-27 01:07:11,845 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:11,854 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ********************************
2026-02-27 01:07:11,855 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:11 +0000 (0:00:00.248)       0:15:33.563 ******* 
2026-02-27 01:07:12,267 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:12,276 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] **********************************
2026-02-27 01:07:12,277 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:12 +0000 (0:00:00.422)       0:15:33.985 ******* 
2026-02-27 01:07:12,298 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Ingestion Successful!"
}
2026-02-27 01:07:12,308 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage: test_static_basic_10x] ***
2026-02-27 01:07:12,309 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:12 +0000 (0:00:00.031)       0:15:34.017 ******* 
2026-02-27 01:07:12,335 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost
2026-02-27 01:07:12,354 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************
2026-02-27 01:07:12,355 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:12 +0000 (0:00:00.045)       0:15:34.063 ******* 
2026-02-27 01:07:12,865 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:12,874 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] *******
2026-02-27 01:07:12,875 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:12 +0000 (0:00:00.520)       0:15:34.583 ******* 
2026-02-27 01:07:13,150 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:13,159 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] ***************
2026-02-27 01:07:13,160 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:13 +0000 (0:00:00.284)       0:15:34.868 ******* 
2026-02-27 01:07:13,744 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:13,754 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************
2026-02-27 01:07:13,754 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:13 +0000 (0:00:00.594)       0:15:35.462 ******* 
2026-02-27 01:07:14,269 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:14,278 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ******************************
2026-02-27 01:07:14,278 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:14 +0000 (0:00:00.524)       0:15:35.987 ******* 
2026-02-27 01:07:14,298 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "Ingester Memory Flushed successfully"
}
2026-02-27 01:07:14,308 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki: test_static_basic_10x] ***
2026-02-27 01:07:14,308 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:14 +0000 (0:00:00.029)       0:15:36.017 ******* 
2026-02-27 01:07:14,331 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost
2026-02-27 01:07:14,355 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_static_basic_10x] *************
2026-02-27 01:07:14,356 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:14 +0000 (0:00:00.047)       0:15:36.064 ******* 
2026-02-27 01:07:14,371 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Input file has 24 data entries that Loki has to return"
}
2026-02-27 01:07:14,378 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] ***********************************
2026-02-27 01:07:14,378 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:14 +0000 (0:00:00.022)       0:15:36.087 ******* 
2026-02-27 01:07:14,848 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:14,857 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file] **********************
2026-02-27 01:07:14,857 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:14 +0000 (0:00:00.478)       0:15:36.565 ******* 
2026-02-27 01:07:15,287 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:15,296 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_static_basic_10x] ******
2026-02-27 01:07:15,296 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:15 +0000 (0:00:00.439)       0:15:37.005 ******* 
2026-02-27 01:07:15,329 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {
    "assertion": "actual_count | int == (synth_data_rates.data_log.log_count | int)",
    "changed": false,
    "evaluated_to": false,
    "msg": "Query did not return all data entries. Expected 24 log entries, but Loki only returned 39"
}
2026-02-27 01:07:15,329 p=37229 u=zuul n=ansible | ...ignoring
2026-02-27 01:07:15,338 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic_10x] ***
2026-02-27 01:07:15,338 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:15 +0000 (0:00:00.041)       0:15:37.046 ******* 
2026-02-27 01:07:15,603 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:15,613 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_static_basic_10x] ***
2026-02-27 01:07:15,614 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:15 +0000 (0:00:00.275)       0:15:37.322 ******* 
2026-02-27 01:07:15,634 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost
2026-02-27 01:07:15,652 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic_10x] ***
2026-02-27 01:07:15,653 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:15 +0000 (0:00:00.038)       0:15:37.361 ******* 
2026-02-27 01:07:18,189 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:18,202 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_static_basic_10x] ***
2026-02-27 01:07:18,202 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:18 +0000 (0:00:02.549)       0:15:39.911 ******* 
2026-02-27 01:07:18,224 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_by_type": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get",
            "-g",
            "type"
        ],
        "delta": "0:00:02.320414",
        "end": "2026-02-27 01:07:18.147942",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-27 01:07:15.827528",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-27 01:07:18,234 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_static_basic_10x] ***
2026-02-27 01:07:18,234 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:18 +0000 (0:00:00.031)       0:15:39.943 ******* 
2026-02-27 01:07:18,709 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:18,722 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic_10x] ***
2026-02-27 01:07:18,722 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:18 +0000 (0:00:00.487)       0:15:40.430 ******* 
2026-02-27 01:07:21,449 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:21,459 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_static_basic_10x] ***
2026-02-27 01:07:21,459 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:21 +0000 (0:00:02.737)       0:15:43.168 ******* 
2026-02-27 01:07:21,481 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_summary": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get"
        ],
        "delta": "0:00:02.480700",
        "end": "2026-02-27 01:07:21.404254",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-27 01:07:18.923554",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-27 01:07:21,491 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** help for openstack rating summary cmd] ***
2026-02-27 01:07:21,492 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:21 +0000 (0:00:00.032)       0:15:43.200 ******* 
2026-02-27 01:07:23,457 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:23,469 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary get help output] ***
2026-02-27 01:07:23,469 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:23 +0000 (0:00:01.977)       0:15:45.177 ******* 
2026-02-27 01:07:23,493 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_h.stdout_lines": [
        "usage: openstack rating summary get [-h]",
        "                                    [-f {csv,df-to-csv,json,table,value,yaml}]",
        "                                    [-c COLUMN]",
        "                                    [--quote {all,minimal,none,nonnumeric}]",
        "                                    [--noindent] [--max-width <integer>]",
        "                                    [--fit-width] [--print-empty]",
        "                                    [--format-config-file FORMAT_CONFIG]",
        "                                    [--sort-column SORT_COLUMN]",
        "                                    [--sort-ascending | --sort-descending]",
        "                                    [-t TENANT_ID] [-b BEGIN] [-e END]",
        "                                    [-s SERVICE] [-g GROUPBY [GROUPBY ...]]",
        "                                    [-a]",
        "",
        "Get a summary report.",
        "",
        "optional arguments:",
        "  -h, --help            show this help message and exit",
        "  -t TENANT_ID, --tenant-id TENANT_ID",
        "                        Tenant id.",
        "  -b BEGIN, --begin BEGIN",
        "                        Begin timestamp.",
        "  -e END, --end END     End timestamp.",
        "  -s SERVICE, --service SERVICE",
        "                        Service Type.",
        "  -g GROUPBY [GROUPBY ...], --groupby GROUPBY [GROUPBY ...]",
        "                        Fields to group by, space-separated. (res_type and",
        "                        tenant_id are supported for now)",
        "  -a, --all-tenants     Allows to get summary from all tenants (admin only).",
        "                        Defaults to False.",
        "",
        "output formatters:",
        "  output formatter options",
        "",
        "  -f {csv,df-to-csv,json,table,value,yaml}, --format {csv,df-to-csv,json,table,value,yaml}",
        "                        the output format, defaults to table",
        "  -c COLUMN, --column COLUMN",
        "                        specify the column(s) to include, can be repeated to",
        "                        show multiple columns",
        "  --sort-column SORT_COLUMN",
        "                        specify the column(s) to sort the data (columns",
        "                        specified first have a priority, non-existing columns",
        "                        are ignored), can be repeated",
        "  --sort-ascending      sort the column(s) in ascending order",
        "  --sort-descending     sort the column(s) in descending order",
        "",
        "CSV Formatter:",
        "  --quote {all,minimal,none,nonnumeric}",
        "                        when to include quotes, defaults to nonnumeric",
        "",
        "json formatter:",
        "  --noindent            whether to disable indenting the JSON",
        "",
        "table formatter:",
        "  --max-width <integer>",
        "                        Maximum display width, <1 to disable. You can also use",
        "                        the CLIFF_MAX_TERM_WIDTH environment variable, but the",
        "                        parameter takes precedence.",
        "  --fit-width           Fit the table to the display width. Implied if --max-",
        "                        width greater than 0. Set the environment variable",
        "                        CLIFF_FIT_WIDTH=1 to always enable",
        "  --print-empty         Print empty table if there is no data to show.",
        "",
        "dataframe-to-csv formatter:",
        "  --format-config-file FORMAT_CONFIG",
        "                        Config file for the dict-to-csv formatter",
        "",
        "This command is provided by the python-cloudkittyclient plugin."
    ]
}
2026-02-27 01:07:23,504 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic totals file] *******************
2026-02-27 01:07:23,504 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:23 +0000 (0:00:00.034)       0:15:45.212 ******* 
2026-02-27 01:07:23,760 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:23,771 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] ***********************
2026-02-27 01:07:23,772 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:23 +0000 (0:00:00.267)       0:15:45.480 ******* 
2026-02-27 01:07:24,025 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:24,038 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data results test_static_basic_10x] ***
2026-02-27 01:07:24,038 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:24 +0000 (0:00:00.266)       0:15:45.747 ******* 
2026-02-27 01:07:24,065 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {
    "assertion": "(synth_data.content | b64decode | from_yaml).synth_rate == (loki_data.content | b64decode | from_yaml).synth_rate",
    "changed": false,
    "evaluated_to": false,
    "msg": "METRICS MISMATCH!\nThe synth_rate or data_log totals do not match between:\ntest_static_basic_10x-synth_metrics_totals.yml\ntest_static_basic_10x-loki_metrics_totals.yml\n"
}
2026-02-27 01:07:24,065 p=37229 u=zuul n=ansible | ...ignoring
2026-02-27 01:07:24,075 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic_gid] ***
2026-02-27 01:07:24,075 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:24 +0000 (0:00:00.036)       0:15:45.784 ******* 
2026-02-27 01:07:24,100 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost
2026-02-27 01:07:24,111 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically test_static_basic_gid] ***
2026-02-27 01:07:24,112 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:24 +0000 (0:00:00.036)       0:15:45.820 ******* 
2026-02-27 01:07:24,137 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:24,146 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] ****************
2026-02-27 01:07:24,147 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:24 +0000 (0:00:00.034)       0:15:45.855 ******* 
2026-02-27 01:07:24,408 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:24,417 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data test_static_basic_gid] ****
2026-02-27 01:07:24,417 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:24 +0000 (0:00:00.270)       0:15:46.126 ******* 
2026-02-27 01:07:24,829 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:24,838 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic_gid] ***
2026-02-27 01:07:24,839 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:24 +0000 (0:00:00.421)       0:15:46.547 ******* 
2026-02-27 01:07:25,152 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:25,160 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] **********************
2026-02-27 01:07:25,161 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:25 +0000 (0:00:00.321)       0:15:46.869 ******* 
2026-02-27 01:07:25,182 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:25,192 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_static_basic_gid] *********
2026-02-27 01:07:25,193 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:25 +0000 (0:00:00.032)       0:15:46.901 ******* 
2026-02-27 01:07:25,218 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost
2026-02-27 01:07:25,233 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic_gid] ***
2026-02-27 01:07:25,233 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:25 +0000 (0:00:00.040)       0:15:46.942 ******* 
2026-02-27 01:07:25,259 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost
2026-02-27 01:07:25,270 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] ****************************
2026-02-27 01:07:25,270 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:25 +0000 (0:00:00.037)       0:15:46.979 ******* 
2026-02-27 01:07:25,508 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:25,516 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ********************************
2026-02-27 01:07:25,516 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:25 +0000 (0:00:00.245)       0:15:47.225 ******* 
2026-02-27 01:07:25,921 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:25,929 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] **********************************
2026-02-27 01:07:25,930 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:25 +0000 (0:00:00.413)       0:15:47.638 ******* 
2026-02-27 01:07:25,949 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Ingestion Successful!"
}
2026-02-27 01:07:25,957 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage: test_static_basic_gid] ***
2026-02-27 01:07:25,958 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:25 +0000 (0:00:00.027)       0:15:47.666 ******* 
2026-02-27 01:07:25,985 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost
2026-02-27 01:07:26,002 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************
2026-02-27 01:07:26,002 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:26 +0000 (0:00:00.044)       0:15:47.711 ******* 
2026-02-27 01:07:26,526 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:26,533 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] *******
2026-02-27 01:07:26,534 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:26 +0000 (0:00:00.531)       0:15:48.242 ******* 
2026-02-27 01:07:26,787 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:26,796 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] ***************
2026-02-27 01:07:26,796 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:26 +0000 (0:00:00.262)       0:15:48.504 ******* 
2026-02-27 01:07:27,335 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:27,345 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************
2026-02-27 01:07:27,345 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:27 +0000 (0:00:00.549)       0:15:49.054 ******* 
2026-02-27 01:07:27,884 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:27,893 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ******************************
2026-02-27 01:07:27,893 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:27 +0000 (0:00:00.547)       0:15:49.602 ******* 
2026-02-27 01:07:27,916 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "Ingester Memory Flushed successfully"
}
2026-02-27 01:07:27,926 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki: test_static_basic_gid] ***
2026-02-27 01:07:27,926 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:27 +0000 (0:00:00.033)       0:15:49.635 ******* 
2026-02-27 01:07:27,949 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost
2026-02-27 01:07:27,974 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_static_basic_gid] *************
2026-02-27 01:07:27,974 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:27 +0000 (0:00:00.047)       0:15:49.683 ******* 
2026-02-27 01:07:27,998 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Input file has 24 data entries that Loki has to return"
}
2026-02-27 01:07:28,011 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] ***********************************
2026-02-27 01:07:28,011 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:28 +0000 (0:00:00.036)       0:15:49.720 ******* 
2026-02-27 01:07:28,515 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:28,524 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file] **********************
2026-02-27 01:07:28,524 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:28 +0000 (0:00:00.513)       0:15:50.233 ******* 
2026-02-27 01:07:28,965 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:28,976 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_static_basic_gid] ******
2026-02-27 01:07:28,976 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:28 +0000 (0:00:00.451)       0:15:50.684 ******* 
2026-02-27 01:07:29,013 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {
    "assertion": "actual_count | int == (synth_data_rates.data_log.log_count | int)",
    "changed": false,
    "evaluated_to": false,
    "msg": "Query did not return all data entries. Expected 24 log entries, but Loki only returned 41"
}
2026-02-27 01:07:29,013 p=37229 u=zuul n=ansible | ...ignoring
2026-02-27 01:07:29,021 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic_gid] ***
2026-02-27 01:07:29,022 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:29 +0000 (0:00:00.045)       0:15:50.730 ******* 
2026-02-27 01:07:29,313 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:29,324 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_static_basic_gid] ***
2026-02-27 01:07:29,324 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:29 +0000 (0:00:00.302)       0:15:51.032 ******* 
2026-02-27 01:07:29,354 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost
2026-02-27 01:07:29,374 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic_gid] ***
2026-02-27 01:07:29,374 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:29 +0000 (0:00:00.050)       0:15:51.083 ******* 
2026-02-27 01:07:32,065 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:32,075 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_static_basic_gid] ***
2026-02-27 01:07:32,075 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:32 +0000 (0:00:02.700)       0:15:53.783 ******* 
2026-02-27 01:07:32,096 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_by_type": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get",
            "-g",
            "type"
        ],
        "delta": "0:00:02.423702",
        "end": "2026-02-27 01:07:32.028562",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-27 01:07:29.604860",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-27 01:07:32,108 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_static_basic_gid] ***
2026-02-27 01:07:32,108 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:32 +0000 (0:00:00.033)       0:15:53.817 ******* 
2026-02-27 01:07:32,518 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:32,528 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic_gid] ***
2026-02-27 01:07:32,529 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:32 +0000 (0:00:00.420)       0:15:54.237 ******* 
2026-02-27 01:07:35,233 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:35,243 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_static_basic_gid] ***
2026-02-27 01:07:35,243 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:35 +0000 (0:00:02.714)       0:15:56.952 ******* 
2026-02-27 01:07:35,264 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_summary": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get"
        ],
        "delta": "0:00:02.456050",
        "end": "2026-02-27 01:07:35.196250",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-27 01:07:32.740200",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-27 01:07:35,272 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** help for openstack rating summary cmd] ***
2026-02-27 01:07:35,272 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:35 +0000 (0:00:00.029)       0:15:56.981 ******* 
2026-02-27 01:07:37,236 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:37,245 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary get help output] ***
2026-02-27 01:07:37,245 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:37 +0000 (0:00:01.972)       0:15:58.954 ******* 
2026-02-27 01:07:37,268 p=37229 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_h.stdout_lines": [
        "usage: openstack rating summary get [-h]",
        "                                    [-f {csv,df-to-csv,json,table,value,yaml}]",
        "                                    [-c COLUMN]",
        "                                    [--quote {all,minimal,none,nonnumeric}]",
        "                                    [--noindent] [--max-width <integer>]",
        "                                    [--fit-width] [--print-empty]",
        "                                    [--format-config-file FORMAT_CONFIG]",
        "                                    [--sort-column SORT_COLUMN]",
        "                                    [--sort-ascending | --sort-descending]",
        "                                    [-t TENANT_ID] [-b BEGIN] [-e END]",
        "                                    [-s SERVICE] [-g GROUPBY [GROUPBY ...]]",
        "                                    [-a]",
        "",
        "Get a summary report.",
        "",
        "optional arguments:",
        "  -h, --help            show this help message and exit",
        "  -t TENANT_ID, --tenant-id TENANT_ID",
        "                        Tenant id.",
        "  -b BEGIN, --begin BEGIN",
        "                        Begin timestamp.",
        "  -e END, --end END     End timestamp.",
        "  -s SERVICE, --service SERVICE",
        "                        Service Type.",
        "  -g GROUPBY [GROUPBY ...], --groupby GROUPBY [GROUPBY ...]",
        "                        Fields to group by, space-separated. (res_type and",
        "                        tenant_id are supported for now)",
        "  -a, --all-tenants     Allows to get summary from all tenants (admin only).",
        "                        Defaults to False.",
        "",
        "output formatters:",
        "  output formatter options",
        "",
        "  -f {csv,df-to-csv,json,table,value,yaml}, --format {csv,df-to-csv,json,table,value,yaml}",
        "                        the output format, defaults to table",
        "  -c COLUMN, --column COLUMN",
        "                        specify the column(s) to include, can be repeated to",
        "                        show multiple columns",
        "  --sort-column SORT_COLUMN",
        "                        specify the column(s) to sort the data (columns",
        "                        specified first have a priority, non-existing columns",
        "                        are ignored), can be repeated",
        "  --sort-ascending      sort the column(s) in ascending order",
        "  --sort-descending     sort the column(s) in descending order",
        "",
        "CSV Formatter:",
        "  --quote {all,minimal,none,nonnumeric}",
        "                        when to include quotes, defaults to nonnumeric",
        "",
        "json formatter:",
        "  --noindent            whether to disable indenting the JSON",
        "",
        "table formatter:",
        "  --max-width <integer>",
        "                        Maximum display width, <1 to disable. You can also use",
        "                        the CLIFF_MAX_TERM_WIDTH environment variable, but the",
        "                        parameter takes precedence.",
        "  --fit-width           Fit the table to the display width. Implied if --max-",
        "                        width greater than 0. Set the environment variable",
        "                        CLIFF_FIT_WIDTH=1 to always enable",
        "  --print-empty         Print empty table if there is no data to show.",
        "",
        "dataframe-to-csv formatter:",
        "  --format-config-file FORMAT_CONFIG",
        "                        Config file for the dict-to-csv formatter",
        "",
        "This command is provided by the python-cloudkittyclient plugin."
    ]
}
2026-02-27 01:07:37,276 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic totals file] *******************
2026-02-27 01:07:37,277 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:37 +0000 (0:00:00.031)       0:15:58.985 ******* 
2026-02-27 01:07:37,519 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:37,527 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] ***********************
2026-02-27 01:07:37,528 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:37 +0000 (0:00:00.251)       0:15:59.236 ******* 
2026-02-27 01:07:37,753 p=37229 u=zuul n=ansible | ok: [localhost]
2026-02-27 01:07:37,764 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data results test_static_basic_gid] ***
2026-02-27 01:07:37,764 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:37 +0000 (0:00:00.236)       0:15:59.473 ******* 
2026-02-27 01:07:37,790 p=37229 u=zuul n=ansible | fatal: [localhost]: FAILED! => {
    "assertion": "(synth_data.content | b64decode | from_yaml).synth_rate == (loki_data.content | b64decode | from_yaml).synth_rate",
    "changed": false,
    "evaluated_to": false,
    "msg": "METRICS MISMATCH!\nThe synth_rate or data_log totals do not match between:\ntest_static_basic_gid-synth_metrics_totals.yml\ntest_static_basic_gid-loki_metrics_totals.yml\n"
}
2026-02-27 01:07:37,790 p=37229 u=zuul n=ansible | ...ignoring
2026-02-27 01:07:37,800 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Cleanup after job run] ****************************
2026-02-27 01:07:37,801 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:37 +0000 (0:00:00.036)       0:15:59.509 ******* 
2026-02-27 01:07:37,824 p=37229 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/cleanup_ck.yml for localhost
2026-02-27 01:07:37,846 p=37229 u=zuul n=ansible | TASK [telemetry_chargeback : Cleanup local certificates] ***********************
2026-02-27 01:07:37,846 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:37 +0000 (0:00:00.045)       0:15:59.555 ******* 
2026-02-27 01:07:38,087 p=37229 u=zuul n=ansible | changed: [localhost]
2026-02-27 01:07:38,127 p=37229 u=zuul n=ansible | PLAY RECAP *********************************************************************
2026-02-27 01:07:38,127 p=37229 u=zuul n=ansible | localhost                  : ok=284  changed=57   unreachable=0    failed=0    skipped=7    rescued=0    ignored=8   
2026-02-27 01:07:38,139 p=37229 u=zuul n=ansible | Friday 27 February 2026  01:07:38 +0000 (0:00:00.292)       0:15:59.847 ******* 
2026-02-27 01:07:38,139 p=37229 u=zuul n=ansible | =============================================================================== 
2026-02-27 01:07:38,139 p=37229 u=zuul n=ansible | telemetry_chargeback : Query Loki API --------------------------------- 848.27s
2026-02-27 01:07:38,139 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_all_qty_zero --- 6.42s
2026-02-27 01:07:38,139 p=37229 u=zuul n=ansible | telemetry_chargeback : Enable Cloudkitty Module (hashmap) --------------- 3.78s
2026-02-27 01:07:38,139 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic --- 3.13s
2026-02-27 01:07:38,139 p=37229 u=zuul n=ansible | telemetry_chargeback : Change priority for CloudKitty hashmap module ---- 3.13s
2026-02-27 01:07:38,139 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_all_qty_zero --- 3.06s
2026-02-27 01:07:38,139 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_dyn_basic --- 3.01s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic_10x --- 2.74s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_dyn_basic --- 2.73s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic_gid --- 2.71s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic_gid --- 2.70s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Find the current value of hashmap ---------------- 2.67s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic --- 2.59s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic_10x --- 2.55s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Get admin user ID for CI ------------------------- 2.33s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Get status of all CloudKitty rating modules ------ 2.28s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | common : Verify endpoint "['cloudkitty', 'rating', 'public']" ----------- 2.27s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Get admin project ID for CI ---------------------- 2.25s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | common : Verify endpoint "['cloudkitty', 'rating', 'internal']" --------- 2.24s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** help for openstack rating summary cmd --- 2.03s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** help for openstack rating summary cmd --- 1.98s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** help for openstack rating summary cmd --- 1.97s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** help for openstack rating summary cmd --- 1.94s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** help for openstack rating summary cmd --- 1.92s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | Gathering Facts --------------------------------------------------------- 0.95s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.78s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 0.64s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 0.59s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 0.58s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 0.58s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Query Loki API ----------------------------------- 0.56s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 0.55s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.55s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.53s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.52s
2026-02-27 01:07:38,140 p=37229 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.52s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.52s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file ---------------------- 0.52s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.52s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Query Loki API ----------------------------------- 0.51s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.51s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_dyn_basic --- 0.51s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Query Loki API ----------------------------------- 0.50s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file ---------------------- 0.50s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.49s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_static_basic_10x --- 0.49s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.48s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Query Loki API ----------------------------------- 0.48s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file ---------------------- 0.47s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.47s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.46s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.46s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file ---------------------- 0.45s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.45s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file ---------------------- 0.44s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.43s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.42s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data test_static_basic_gid ---- 0.42s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_static_basic_gid --- 0.42s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.42s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_all_qty_zero --- 0.42s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.42s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.41s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Find test files ---------------------------------- 0.41s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_static_basic --- 0.41s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data test_dyn_basic ----------- 0.40s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.40s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data test_static_basic -------- 0.40s
2026-02-27 01:07:38,141 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data test_static_basic_10x ---- 0.40s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.40s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.40s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-distributor-http" ------- 0.39s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-distributor-grpc" ------- 0.39s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.38s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-index-gateway-grpc" ----- 0.38s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-public" --------------------------- 0.38s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_dyn_basic --- 0.37s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-ingester pod is Running in openstack namespace
 --- 0.37s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-compactor-grpc" --------- 0.37s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.37s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | telemetry_chargeback : Extract Client Certificates ---------------------- 0.37s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | telemetry_chargeback : Get Loki Public Route Host ----------------------- 0.37s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-gossip-ring" ------------ 0.36s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-querier-grpc" ----------- 0.36s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-gateway-http" ----------- 0.36s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | telemetry_chargeback : Extract CA Bundle -------------------------------- 0.36s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-index-gateway pod is Running in openstack namespace
 --- 0.36s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-ingester-http" ---------- 0.36s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-querier pod is Running in openstack namespace
 --- 0.36s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic --- 0.35s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-query-frontend pod is Running in openstack namespace
 --- 0.35s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Get the openstack project -------------------------------------- 0.35s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | telemetry_chargeback : Extract Certificates from Openshift Secret ------- 0.35s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic_10x --- 0.35s
2026-02-27 01:07:38,142 p=37229 u=zuul n=ansible | common : Get the openstack-operators project ---------------------------- 0.35s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Ensure Local Certificate Directory Exists -------- 0.35s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-query-frontend-grpc" ---- 0.34s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-ingester-grpc" ---------- 0.34s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-internal" ------------------------- 0.34s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data test_all_qty_zero -------- 0.34s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-distributor pod is Running in openstack namespace
 --- 0.34s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-query-frontend-http" ---- 0.33s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-proc pod is Running in openstack namespace
 --- 0.33s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-compactor pod is Running in openstack namespace
 --- 0.33s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-compactor-http" --------- 0.33s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-index-gateway-http" ----- 0.32s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic_gid --- 0.32s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-querier-http" ----------- 0.32s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.32s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | common : TEST Check cloudkitty-api pod is Running in openstack namespace
 --- 0.31s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic --- 0.31s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file test_dyn_basic --- 0.31s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file test_all_qty_zero --- 0.31s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.31s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic_gid --- 0.30s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.30s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Cleanup local certificates ----------------------- 0.29s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.29s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.28s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_all_qty_zero --- 0.28s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic_10x --- 0.28s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.27s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Read the synthetic totals file ------------------- 0.27s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.27s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.26s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Read the synthetic totals file ------------------- 0.26s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.26s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.26s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.25s
2026-02-27 01:07:38,143 p=37229 u=zuul n=ansible | telemetry_chargeback : Read the synthetic totals file ------------------- 0.25s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.25s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.25s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.24s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.23s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Read the synthetic totals file ------------------- 0.22s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.22s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.22s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.22s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.21s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Read the synthetic totals file ------------------- 0.21s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | common : Run service tests ---------------------------------------------- 0.17s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | common : Run pod tests -------------------------------------------------- 0.10s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Process and Loop if files exist ------------------ 0.10s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage: test_all_qty_zero ---- 0.06s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_static_basic ---------- 0.06s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_dyn_basic ------------- 0.06s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | common : Run endpoint tests --------------------------------------------- 0.06s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | common : Run project tests ---------------------------------------------- 0.06s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_static_basic_gid --- 0.05s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki: test_dyn_basic ------ 0.05s
2026-02-27 01:07:38,144 p=37229 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki: test_all_qty_zero --- 0.05s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki: test_static_basic --- 0.05s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki: test_static_basic_gid --- 0.05s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki: test_static_basic_10x --- 0.05s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_static_basic_gid ------ 0.05s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Cleanup after job run ---------------------------- 0.05s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage: test_static_basic_10x --- 0.05s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_all_qty_zero ---------- 0.05s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_dyn_basic --- 0.05s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_static_basic ------ 0.05s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_dyn_basic --------- 0.05s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage: test_static_basic_gid --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data results test_all_qty_zero --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_dyn_basic ---------------- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_static_basic_10x --------- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_static_basic_10x ------ 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Validate CloudKitty module states ----------- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_static_basic ------------- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_static_basic_gid --------- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Setup Loki Environment --------------------------- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage: test_static_basic ---- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_static_basic_10x --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_all_qty_zero --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic_10x --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic_gid --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Expected Count test_static_basic_gid ------------- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data results test_static_basic_10x --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic_gid --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data results test_static_basic_gid --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_static_basic --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_all_qty_zero ------ 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data results test_static_basic --- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage: test_dyn_basic ------- 0.04s
2026-02-27 01:07:38,145 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary get help output --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically test_static_basic_gid --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_all_qty_zero ------------- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically test_dyn_basic --------- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_static_basic --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Expected Count test_dyn_basic -------------------- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary get help output --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_all_qty_zero --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_static_basic_gid --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_dyn_basic --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_static_basic_10x --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Set priority for CloudKitty hashmap module --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_static_basic_10x --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Confirm Success ---------------------------------- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Confirm Success ---------------------------------- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log: test_dyn_basic ------ 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary get help output --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Set admin user ID for CI ------------------------- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log: test_all_qty_zero --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_dyn_basic --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-ingester-http service is running in openstack namespace
 --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | common : TEST Check that cloudkitty public endpoint exists
 ------------- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_static_basic_gid --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data results test_dyn_basic --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-distributor-http service is running in openstack namespace
 --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic_10x --- 0.03s
2026-02-27 01:07:38,146 p=37229 u=zuul n=ansible | common : TEST Check that cloudkitty internal endpoint exists
 ----------- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Set admin project ID for CI ---------------------- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Confirm Success ---------------------------------- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Extract only the filenames into a clean list ----- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-query-frontend-http service is running in openstack namespace
 --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-gossip-ring service is running in openstack namespace
 --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : Verify container tests ----------------------------------------- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically test_static_basic ------ 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify openstack-operators project exists
 ---------------- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary get help output --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-distributor-grpc service is running in openstack namespace
 --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically test_static_basic_10x --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Validate Chargeback Feature deployed correctly --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-index-gateway-grpc service is running in openstack namespace
 --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary get help output --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_all_qty_zero --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify openstack project exists
 -------------------------- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-gateway-http service is running in openstack namespace
 --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-index-gateway-http service is running in openstack namespace
 --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-ingester-grpc service is running in openstack namespace
 --- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically test_all_qty_zero ------ 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Expected Count test_all_qty_zero ----------------- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Expected Count test_static_basic ----------------- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | telemetry_chargeback : Confirm Success ---------------------------------- 0.03s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-public service is running in openstack namespace
 --- 0.02s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-internal service is running in openstack namespace
 --- 0.02s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : Check that each package manifest is present -------------------- 0.02s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : Run subscription tests ----------------------------------------- 0.02s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-compactor-http service is running in openstack namespace
 --- 0.02s
2026-02-27 01:07:38,147 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-compactor-grpc service is running in openstack namespace
 --- 0.02s
2026-02-27 01:07:38,148 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-query-frontend-grpc service is running in openstack namespace
 --- 0.02s
2026-02-27 01:07:38,148 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-querier-grpc service is running in openstack namespace
 --- 0.02s
2026-02-27 01:07:38,148 p=37229 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-querier-http service is running in openstack namespace
 --- 0.02s
2026-02-27 01:07:38,148 p=37229 u=zuul n=ansible | common : Run file tests ------------------------------------------------- 0.02s
2026-02-27 01:07:38,148 p=37229 u=zuul n=ansible | common : Run crd tests -------------------------------------------------- 0.02s
2026-02-27 01:07:38,148 p=37229 u=zuul n=ansible | telemetry_chargeback : Expected Count test_static_basic_10x ------------- 0.02s
2026-02-27 01:07:38,148 p=37229 u=zuul n=ansible | common : Run CR tests --------------------------------------------------- 0.02s
2026-02-27 01:07:38,148 p=37229 u=zuul n=ansible | telemetry_chargeback : Debug URLs --------------------------------------- 0.02s
2026-02-27 01:07:38,148 p=37229 u=zuul n=ansible | telemetry_chargeback : Set Loki URLs ------------------------------------ 0.02s
2026-02-27 01:07:38,148 p=37229 u=zuul n=ansible | common : Get the manifest list once ------------------------------------- 0.02s
2026-02-27 01:07:38,148 p=37229 u=zuul n=ansible | telemetry_chargeback : Set Derived Loki URLs ---------------------------- 0.02s