2026-02-26 22:44:08,032 p=37225 u=zuul n=ansible | [WARNING]: Found variable using reserved name: namespace

2026-02-26 22:44:08,033 p=37225 u=zuul n=ansible | PLAY [Verify all the applicable projects, endpoints, pods & services for cloudkitty] ***
2026-02-26 22:44:08,041 p=37225 u=zuul n=ansible | TASK [Gathering Facts] *********************************************************
2026-02-26 22:44:08,042 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:08 +0000 (0:00:00.024)       0:00:00.024 ***** 
2026-02-26 22:44:08,911 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:08,930 p=37225 u=zuul n=ansible | TASK [common : Run pod tests] **************************************************
2026-02-26 22:44:08,930 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:08 +0000 (0:00:00.888)       0:00:00.913 ***** 
2026-02-26 22:44:09,006 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-api)
2026-02-26 22:44:09,009 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-compactor)
2026-02-26 22:44:09,012 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-distributor)
2026-02-26 22:44:09,014 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway)
2026-02-26 22:44:09,016 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-ingester)
2026-02-26 22:44:09,019 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-querier)
2026-02-26 22:44:09,021 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend)
2026-02-26 22:44:09,023 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/pod_tests.yml for localhost => (item=cloudkitty-proc)
2026-02-26 22:44:09,032 p=37225 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-26 22:44:09,032 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:09 +0000 (0:00:00.101)       0:00:01.014 ***** 
2026-02-26 22:44:09,540 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:09,553 p=37225 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-api pod is Running in openstack namespace] ***
2026-02-26 22:44:09,553 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:09 +0000 (0:00:00.521)       0:00:01.536 ***** 
2026-02-26 22:44:09,948 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:09,958 p=37225 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-26 22:44:09,959 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:09 +0000 (0:00:00.405)       0:00:01.941 ***** 
2026-02-26 22:44:10,369 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:10,377 p=37225 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-compactor pod is Running in openstack namespace] ***
2026-02-26 22:44:10,377 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:10 +0000 (0:00:00.418)       0:00:02.359 ***** 
2026-02-26 22:44:10,724 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:10,732 p=37225 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-26 22:44:10,732 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:10 +0000 (0:00:00.355)       0:00:02.715 ***** 
2026-02-26 22:44:11,145 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:11,155 p=37225 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-distributor pod is Running in openstack namespace] ***
2026-02-26 22:44:11,155 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:11 +0000 (0:00:00.423)       0:00:03.138 ***** 
2026-02-26 22:44:11,515 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:11,522 p=37225 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-26 22:44:11,522 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:11 +0000 (0:00:00.366)       0:00:03.504 ***** 
2026-02-26 22:44:11,927 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:11,934 p=37225 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-index-gateway pod is Running in openstack namespace] ***
2026-02-26 22:44:11,934 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:11 +0000 (0:00:00.412)       0:00:03.916 ***** 
2026-02-26 22:44:12,316 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:12,324 p=37225 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-26 22:44:12,324 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:12 +0000 (0:00:00.390)       0:00:04.307 ***** 
2026-02-26 22:44:12,724 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:12,734 p=37225 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-ingester pod is Running in openstack namespace] ***
2026-02-26 22:44:12,734 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:12 +0000 (0:00:00.409)       0:00:04.717 ***** 
2026-02-26 22:44:13,134 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:13,143 p=37225 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-26 22:44:13,143 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:13 +0000 (0:00:00.408)       0:00:05.125 ***** 
2026-02-26 22:44:13,511 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:13,519 p=37225 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-querier pod is Running in openstack namespace] ***
2026-02-26 22:44:13,519 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:13 +0000 (0:00:00.375)       0:00:05.501 ***** 
2026-02-26 22:44:13,826 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:13,834 p=37225 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-26 22:44:13,834 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:13 +0000 (0:00:00.314)       0:00:05.816 ***** 
2026-02-26 22:44:14,242 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:14,252 p=37225 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-lokistack-query-frontend pod is Running in openstack namespace] ***
2026-02-26 22:44:14,252 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:14 +0000 (0:00:00.418)       0:00:06.235 ***** 
2026-02-26 22:44:14,576 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:14,585 p=37225 u=zuul n=ansible | TASK [common : Get Pod Instance name "Running"] ********************************
2026-02-26 22:44:14,585 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:14 +0000 (0:00:00.332)       0:00:06.567 ***** 
2026-02-26 22:44:14,943 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:14,951 p=37225 u=zuul n=ansible | TASK [common : TEST Check cloudkitty-proc pod is Running in openstack namespace] ***
2026-02-26 22:44:14,951 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:14 +0000 (0:00:00.366)       0:00:06.934 ***** 
2026-02-26 22:44:15,308 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:15,315 p=37225 u=zuul n=ansible | TASK [common : Run subscription tests] *****************************************
2026-02-26 22:44:15,315 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:15 +0000 (0:00:00.363)       0:00:07.297 ***** 
2026-02-26 22:44:15,330 p=37225 u=zuul n=ansible | skipping: [localhost]
2026-02-26 22:44:15,336 p=37225 u=zuul n=ansible | TASK [common : Run endpoint tests] *********************************************
2026-02-26 22:44:15,336 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:15 +0000 (0:00:00.021)       0:00:07.319 ***** 
2026-02-26 22:44:15,363 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/endpoint_tests.yml for localhost => (item=['cloudkitty', 'rating', 'public'])
2026-02-26 22:44:15,365 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/endpoint_tests.yml for localhost => (item=['cloudkitty', 'rating', 'internal'])
2026-02-26 22:44:15,387 p=37225 u=zuul n=ansible | TASK [common : Verify endpoint "['cloudkitty', 'rating', 'public']"] ***********
2026-02-26 22:44:15,387 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:15 +0000 (0:00:00.050)       0:00:07.369 ***** 
2026-02-26 22:44:17,764 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:17,773 p=37225 u=zuul n=ansible | TASK [common : TEST Check that cloudkitty public endpoint exists] **************
2026-02-26 22:44:17,773 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:17 +0000 (0:00:02.385)       0:00:09.755 ***** 
2026-02-26 22:44:17,807 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "The endpoint cloudkitty is found."
}
2026-02-26 22:44:17,815 p=37225 u=zuul n=ansible | TASK [common : Verify endpoint "['cloudkitty', 'rating', 'internal']"] *********
2026-02-26 22:44:17,815 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:17 +0000 (0:00:00.041)       0:00:09.797 ***** 
2026-02-26 22:44:20,262 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:20,284 p=37225 u=zuul n=ansible | TASK [common : TEST Check that cloudkitty internal endpoint exists] ************
2026-02-26 22:44:20,284 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:20 +0000 (0:00:02.469)       0:00:12.267 ***** 
2026-02-26 22:44:20,309 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "The endpoint cloudkitty is found."
}
2026-02-26 22:44:20,315 p=37225 u=zuul n=ansible | TASK [common : Run project tests] **********************************************
2026-02-26 22:44:20,316 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:20 +0000 (0:00:00.031)       0:00:12.298 ***** 
2026-02-26 22:44:20,356 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/project_tests.yml for localhost => (item=openstack)
2026-02-26 22:44:20,359 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/project_tests.yml for localhost => (item=openstack-operators)
2026-02-26 22:44:20,385 p=37225 u=zuul n=ansible | TASK [common : Get the openstack project] **************************************
2026-02-26 22:44:20,386 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:20 +0000 (0:00:00.070)       0:00:12.368 ***** 
2026-02-26 22:44:20,715 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:20,723 p=37225 u=zuul n=ansible | TASK [common : TEST Verify openstack project exists] ***************************
2026-02-26 22:44:20,723 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:20 +0000 (0:00:00.337)       0:00:12.706 ***** 
2026-02-26 22:44:20,739 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "project openstack is found."
}
2026-02-26 22:44:20,747 p=37225 u=zuul n=ansible | TASK [common : Get the openstack-operators project] ****************************
2026-02-26 22:44:20,748 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:20 +0000 (0:00:00.024)       0:00:12.730 ***** 
2026-02-26 22:44:21,125 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:21,137 p=37225 u=zuul n=ansible | TASK [common : TEST Verify openstack-operators project exists] *****************
2026-02-26 22:44:21,137 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:21 +0000 (0:00:00.389)       0:00:13.120 ***** 
2026-02-26 22:44:21,153 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "project openstack-operators is found."
}
2026-02-26 22:44:21,161 p=37225 u=zuul n=ansible | TASK [common : Run service tests] **********************************************
2026-02-26 22:44:21,162 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:21 +0000 (0:00:00.024)       0:00:13.144 ***** 
2026-02-26 22:44:21,280 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-internal)
2026-02-26 22:44:21,283 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-compactor-grpc)
2026-02-26 22:44:21,287 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-compactor-http)
2026-02-26 22:44:21,290 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-distributor-grpc)
2026-02-26 22:44:21,294 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-distributor-http)
2026-02-26 22:44:21,296 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-gateway-http)
2026-02-26 22:44:21,299 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-gossip-ring)
2026-02-26 22:44:21,303 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway-grpc)
2026-02-26 22:44:21,306 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-index-gateway-http)
2026-02-26 22:44:21,309 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-ingester-grpc)
2026-02-26 22:44:21,312 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-ingester-http)
2026-02-26 22:44:21,315 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-querier-grpc)
2026-02-26 22:44:21,317 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-querier-http)
2026-02-26 22:44:21,319 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend-grpc)
2026-02-26 22:44:21,322 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-lokistack-query-frontend-http)
2026-02-26 22:44:21,324 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/common/tasks/service_tests.yml for localhost => (item=cloudkitty-public)
2026-02-26 22:44:21,349 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-internal"] *************************
2026-02-26 22:44:21,349 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:21 +0000 (0:00:00.187)       0:00:13.331 ***** 
2026-02-26 22:44:21,679 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:21,686 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-internal service is running in openstack namespace] ***
2026-02-26 22:44:21,686 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:21 +0000 (0:00:00.337)       0:00:13.669 ***** 
2026-02-26 22:44:21,702 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-internal is running."
}
2026-02-26 22:44:21,710 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-compactor-grpc"] *********
2026-02-26 22:44:21,710 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:21 +0000 (0:00:00.024)       0:00:13.693 ***** 
2026-02-26 22:44:22,023 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:22,030 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-compactor-grpc service is running in openstack namespace] ***
2026-02-26 22:44:22,030 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:22 +0000 (0:00:00.320)       0:00:14.013 ***** 
2026-02-26 22:44:22,044 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-compactor-grpc is running."
}
2026-02-26 22:44:22,051 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-compactor-http"] *********
2026-02-26 22:44:22,051 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:22 +0000 (0:00:00.020)       0:00:14.034 ***** 
2026-02-26 22:44:22,375 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:22,384 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-compactor-http service is running in openstack namespace] ***
2026-02-26 22:44:22,384 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:22 +0000 (0:00:00.332)       0:00:14.366 ***** 
2026-02-26 22:44:22,400 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-compactor-http is running."
}
2026-02-26 22:44:22,407 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-distributor-grpc"] *******
2026-02-26 22:44:22,407 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:22 +0000 (0:00:00.023)       0:00:14.390 ***** 
2026-02-26 22:44:22,710 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:22,720 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-distributor-grpc service is running in openstack namespace] ***
2026-02-26 22:44:22,720 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:22 +0000 (0:00:00.312)       0:00:14.703 ***** 
2026-02-26 22:44:22,737 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-distributor-grpc is running."
}
2026-02-26 22:44:22,745 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-distributor-http"] *******
2026-02-26 22:44:22,745 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:22 +0000 (0:00:00.024)       0:00:14.728 ***** 
2026-02-26 22:44:23,068 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:23,077 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-distributor-http service is running in openstack namespace] ***
2026-02-26 22:44:23,077 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:23 +0000 (0:00:00.331)       0:00:15.059 ***** 
2026-02-26 22:44:23,094 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-distributor-http is running."
}
2026-02-26 22:44:23,106 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-gateway-http"] ***********
2026-02-26 22:44:23,106 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:23 +0000 (0:00:00.028)       0:00:15.088 ***** 
2026-02-26 22:44:23,493 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:23,501 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-gateway-http service is running in openstack namespace] ***
2026-02-26 22:44:23,501 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:23 +0000 (0:00:00.395)       0:00:15.484 ***** 
2026-02-26 22:44:23,519 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-gateway-http is running."
}
2026-02-26 22:44:23,527 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-gossip-ring"] ************
2026-02-26 22:44:23,528 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:23 +0000 (0:00:00.026)       0:00:15.510 ***** 
2026-02-26 22:44:23,859 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:23,868 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-gossip-ring service is running in openstack namespace] ***
2026-02-26 22:44:23,868 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:23 +0000 (0:00:00.340)       0:00:15.850 ***** 
2026-02-26 22:44:23,891 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-gossip-ring is running."
}
2026-02-26 22:44:23,899 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-index-gateway-grpc"] *****
2026-02-26 22:44:23,900 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:23 +0000 (0:00:00.031)       0:00:15.882 ***** 
2026-02-26 22:44:24,298 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:24,306 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-index-gateway-grpc service is running in openstack namespace] ***
2026-02-26 22:44:24,306 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:24 +0000 (0:00:00.406)       0:00:16.288 ***** 
2026-02-26 22:44:24,323 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-index-gateway-grpc is running."
}
2026-02-26 22:44:24,330 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-index-gateway-http"] *****
2026-02-26 22:44:24,330 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:24 +0000 (0:00:00.024)       0:00:16.313 ***** 
2026-02-26 22:44:24,681 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:24,689 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-index-gateway-http service is running in openstack namespace] ***
2026-02-26 22:44:24,690 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:24 +0000 (0:00:00.359)       0:00:16.672 ***** 
2026-02-26 22:44:24,706 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-index-gateway-http is running."
}
2026-02-26 22:44:24,714 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-ingester-grpc"] **********
2026-02-26 22:44:24,714 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:24 +0000 (0:00:00.024)       0:00:16.697 ***** 
2026-02-26 22:44:25,095 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:25,106 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-ingester-grpc service is running in openstack namespace] ***
2026-02-26 22:44:25,106 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:25 +0000 (0:00:00.391)       0:00:17.088 ***** 
2026-02-26 22:44:25,123 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-ingester-grpc is running."
}
2026-02-26 22:44:25,133 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-ingester-http"] **********
2026-02-26 22:44:25,133 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:25 +0000 (0:00:00.027)       0:00:17.116 ***** 
2026-02-26 22:44:25,493 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:25,504 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-ingester-http service is running in openstack namespace] ***
2026-02-26 22:44:25,504 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:25 +0000 (0:00:00.370)       0:00:17.486 ***** 
2026-02-26 22:44:25,526 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-ingester-http is running."
}
2026-02-26 22:44:25,537 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-querier-grpc"] ***********
2026-02-26 22:44:25,537 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:25 +0000 (0:00:00.032)       0:00:17.519 ***** 
2026-02-26 22:44:25,873 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:25,882 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-querier-grpc service is running in openstack namespace] ***
2026-02-26 22:44:25,882 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:25 +0000 (0:00:00.345)       0:00:17.865 ***** 
2026-02-26 22:44:25,899 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-querier-grpc is running."
}
2026-02-26 22:44:25,907 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-querier-http"] ***********
2026-02-26 22:44:25,908 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:25 +0000 (0:00:00.025)       0:00:17.890 ***** 
2026-02-26 22:44:26,228 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:26,236 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-querier-http service is running in openstack namespace] ***
2026-02-26 22:44:26,236 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:26 +0000 (0:00:00.328)       0:00:18.218 ***** 
2026-02-26 22:44:26,252 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-querier-http is running."
}
2026-02-26 22:44:26,259 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-query-frontend-grpc"] ****
2026-02-26 22:44:26,259 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:26 +0000 (0:00:00.023)       0:00:18.242 ***** 
2026-02-26 22:44:26,582 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:26,590 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-query-frontend-grpc service is running in openstack namespace] ***
2026-02-26 22:44:26,590 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:26 +0000 (0:00:00.330)       0:00:18.573 ***** 
2026-02-26 22:44:26,607 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-query-frontend-grpc is running."
}
2026-02-26 22:44:26,615 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-lokistack-query-frontend-http"] ****
2026-02-26 22:44:26,615 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:26 +0000 (0:00:00.024)       0:00:18.598 ***** 
2026-02-26 22:44:26,953 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:26,963 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-lokistack-query-frontend-http service is running in openstack namespace] ***
2026-02-26 22:44:26,963 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:26 +0000 (0:00:00.347)       0:00:18.945 ***** 
2026-02-26 22:44:26,982 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-lokistack-query-frontend-http is running."
}
2026-02-26 22:44:26,990 p=37225 u=zuul n=ansible | TASK [common : Verify Service - "cloudkitty-public"] ***************************
2026-02-26 22:44:26,990 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:26 +0000 (0:00:00.027)       0:00:18.973 ***** 
2026-02-26 22:44:27,336 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:27,343 p=37225 u=zuul n=ansible | TASK [common : TEST Verify cloudkitty-public service is running in openstack namespace] ***
2026-02-26 22:44:27,343 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:27 +0000 (0:00:00.352)       0:00:19.326 ***** 
2026-02-26 22:44:27,370 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "service cloudkitty-public is running."
}
2026-02-26 22:44:27,377 p=37225 u=zuul n=ansible | TASK [common : Run file tests] *************************************************
2026-02-26 22:44:27,377 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:27 +0000 (0:00:00.034)       0:00:19.360 ***** 
2026-02-26 22:44:27,401 p=37225 u=zuul n=ansible | skipping: [localhost]
2026-02-26 22:44:27,411 p=37225 u=zuul n=ansible | TASK [common : Get the manifest list once] *************************************
2026-02-26 22:44:27,411 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:27 +0000 (0:00:00.033)       0:00:19.393 ***** 
2026-02-26 22:44:27,430 p=37225 u=zuul n=ansible | skipping: [localhost]
2026-02-26 22:44:27,439 p=37225 u=zuul n=ansible | TASK [common : Check that each package manifest is present] ********************
2026-02-26 22:44:27,439 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:27 +0000 (0:00:00.028)       0:00:19.422 ***** 
2026-02-26 22:44:27,459 p=37225 u=zuul n=ansible | skipping: [localhost]
2026-02-26 22:44:27,468 p=37225 u=zuul n=ansible | TASK [common : Run crd tests] **************************************************
2026-02-26 22:44:27,468 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:27 +0000 (0:00:00.028)       0:00:19.450 ***** 
2026-02-26 22:44:27,489 p=37225 u=zuul n=ansible | skipping: [localhost]
2026-02-26 22:44:27,497 p=37225 u=zuul n=ansible | TASK [common : Run CR tests] ***************************************************
2026-02-26 22:44:27,498 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:27 +0000 (0:00:00.029)       0:00:19.480 ***** 
2026-02-26 22:44:27,519 p=37225 u=zuul n=ansible | skipping: [localhost]
2026-02-26 22:44:27,528 p=37225 u=zuul n=ansible | TASK [common : Verify container tests] *****************************************
2026-02-26 22:44:27,528 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:27 +0000 (0:00:00.030)       0:00:19.511 ***** 
2026-02-26 22:44:27,550 p=37225 u=zuul n=ansible | skipping: [localhost]
2026-02-26 22:44:27,566 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Validate Chargeback Feature deployed correctly] ***
2026-02-26 22:44:27,567 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:27 +0000 (0:00:00.038)       0:00:19.549 ***** 
2026-02-26 22:44:27,590 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/chargeback_tests.yml for localhost
2026-02-26 22:44:27,601 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Enable Cloudkitty Module (hashmap)] ***************
2026-02-26 22:44:27,601 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:27 +0000 (0:00:00.034)       0:00:19.584 ***** 
2026-02-26 22:44:31,518 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:44:31,531 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Find the current value of hashmap] ****************
2026-02-26 22:44:31,531 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:31 +0000 (0:00:03.929)       0:00:23.514 ***** 
2026-02-26 22:44:34,083 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:34,097 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Change priority for CloudKitty hashmap module] ****
2026-02-26 22:44:34,098 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:34 +0000 (0:00:02.566)       0:00:26.080 ***** 
2026-02-26 22:44:37,418 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:44:37,431 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Get status of all CloudKitty rating modules] ******
2026-02-26 22:44:37,431 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:37 +0000 (0:00:03.333)       0:00:29.414 ***** 
2026-02-26 22:44:40,094 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:40,103 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Validate CloudKitty module states] ***********
2026-02-26 22:44:40,103 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:40 +0000 (0:00:02.672)       0:00:32.086 ***** 
2026-02-26 22:44:40,144 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "SUCCESS: CloudKitty modules (hashmap=True) are configured correctly."
}
2026-02-26 22:44:40,153 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Set priority for CloudKitty hashmap module] ***
2026-02-26 22:44:40,153 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:40 +0000 (0:00:00.049)       0:00:32.136 ***** 
2026-02-26 22:44:40,176 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "SUCCESS: The hashmap priority is set to 100"
}
2026-02-26 22:44:40,185 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Setup Loki Environment] ***************************
2026-02-26 22:44:40,185 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:40 +0000 (0:00:00.031)       0:00:32.167 ***** 
2026-02-26 22:44:40,211 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/setup_loki_env.yml for localhost
2026-02-26 22:44:40,227 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Get Loki Public Route Host] ***********************
2026-02-26 22:44:40,227 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:40 +0000 (0:00:00.041)       0:00:32.209 ***** 
2026-02-26 22:44:40,862 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:40,871 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Set Loki URLs] ************************************
2026-02-26 22:44:40,871 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:40 +0000 (0:00:00.644)       0:00:32.853 ***** 
2026-02-26 22:44:40,887 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:40,894 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Set Derived Loki URLs] ****************************
2026-02-26 22:44:40,894 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:40 +0000 (0:00:00.023)       0:00:32.877 ***** 
2026-02-26 22:44:40,912 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:40,919 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Debug URLs] ***************************************
2026-02-26 22:44:40,919 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:40 +0000 (0:00:00.024)       0:00:32.901 ***** 
2026-02-26 22:44:40,938 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "msg": [
        "Loki Route: https://cloudkitty-lokistack-openstack.apps-crc.testing",
        "Push URL:   https://cloudkitty-lokistack-openstack.apps-crc.testing/api/logs/v1/cloudkitty/loki/api/v1/push",
        "Flush URL:  https://cloudkitty-lokistack-ingester-http.openstack.svc:3100/flush",
        "Query URL:  https://cloudkitty-lokistack-openstack.apps-crc.testing/api/logs/v1/cloudkitty/loki/api/v1/query_range"
    ]
}
2026-02-26 22:44:40,946 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Ensure Local Certificate Directory Exists] ********
2026-02-26 22:44:40,946 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:40 +0000 (0:00:00.026)       0:00:32.928 ***** 
2026-02-26 22:44:41,248 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:44:41,266 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Extract Certificates from Openshift Secret] *******
2026-02-26 22:44:41,266 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:41 +0000 (0:00:00.320)       0:00:33.249 ***** 
2026-02-26 22:44:41,602 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:44:41,609 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Extract Client Certificates] **********************
2026-02-26 22:44:41,609 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:41 +0000 (0:00:00.343)       0:00:33.592 ***** 
2026-02-26 22:44:41,916 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:44:41,923 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Extract CA Bundle] ********************************
2026-02-26 22:44:41,923 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:41 +0000 (0:00:00.313)       0:00:33.906 ***** 
2026-02-26 22:44:42,215 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:44:42,222 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Get admin project ID for CI] **********************
2026-02-26 22:44:42,222 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:42 +0000 (0:00:00.298)       0:00:34.205 ***** 
2026-02-26 22:44:44,388 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:44,403 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Set admin project ID for CI] **********************
2026-02-26 22:44:44,404 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:44 +0000 (0:00:02.181)       0:00:36.386 ***** 
2026-02-26 22:44:44,426 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:44,435 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Get admin user ID for CI] *************************
2026-02-26 22:44:44,435 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:44 +0000 (0:00:00.031)       0:00:36.418 ***** 
2026-02-26 22:44:46,713 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:46,728 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Set admin user ID for CI] *************************
2026-02-26 22:44:46,728 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:46 +0000 (0:00:02.292)       0:00:38.711 ***** 
2026-02-26 22:44:46,747 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:46,757 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Find test files] **********************************
2026-02-26 22:44:46,758 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:46 +0000 (0:00:00.029)       0:00:38.740 ***** 
2026-02-26 22:44:47,064 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:47,071 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Extract only the filenames into a clean list] *****
2026-02-26 22:44:47,071 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:47 +0000 (0:00:00.313)       0:00:39.054 ***** 
2026-02-26 22:44:47,104 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:47,110 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Process and Loop if files exist] ******************
2026-02-26 22:44:47,110 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:47 +0000 (0:00:00.038)       0:00:39.093 ***** 
2026-02-26 22:44:47,147 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_static_basic)
2026-02-26 22:44:47,153 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/run_test_scenarios.yml for localhost => (item=test_static_basic_gid)
2026-02-26 22:44:47,161 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic] ***
2026-02-26 22:44:47,161 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:47 +0000 (0:00:00.051)       0:00:39.144 ***** 
2026-02-26 22:44:47,181 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost
2026-02-26 22:44:47,190 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically test_static_basic] ******
2026-02-26 22:44:47,190 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:47 +0000 (0:00:00.028)       0:00:39.172 ***** 
2026-02-26 22:44:47,213 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:47,220 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] ****************
2026-02-26 22:44:47,220 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:47 +0000 (0:00:00.030)       0:00:39.203 ***** 
2026-02-26 22:44:47,519 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:47,529 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data test_static_basic] ********
2026-02-26 22:44:47,529 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:47 +0000 (0:00:00.308)       0:00:39.511 ***** 
2026-02-26 22:44:47,868 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:44:47,876 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic] ***
2026-02-26 22:44:47,876 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:47 +0000 (0:00:00.347)       0:00:39.859 ***** 
2026-02-26 22:44:48,157 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:44:48,164 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] **********************
2026-02-26 22:44:48,165 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:48 +0000 (0:00:00.288)       0:00:40.147 ***** 
2026-02-26 22:44:48,180 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:48,188 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_static_basic] *************
2026-02-26 22:44:48,188 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:48 +0000 (0:00:00.023)       0:00:40.171 ***** 
2026-02-26 22:44:48,210 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost
2026-02-26 22:44:48,224 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic] ***
2026-02-26 22:44:48,224 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:48 +0000 (0:00:00.035)       0:00:40.206 ***** 
2026-02-26 22:44:48,243 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost
2026-02-26 22:44:48,251 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] ****************************
2026-02-26 22:44:48,251 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:48 +0000 (0:00:00.027)       0:00:40.234 ***** 
2026-02-26 22:44:48,533 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:48,539 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ********************************
2026-02-26 22:44:48,539 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:48 +0000 (0:00:00.287)       0:00:40.522 ***** 
2026-02-26 22:44:49,083 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:49,101 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] **********************************
2026-02-26 22:44:49,102 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:49 +0000 (0:00:00.562)       0:00:41.084 ***** 
2026-02-26 22:44:49,120 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Ingestion Successful!"
}
2026-02-26 22:44:49,139 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage: test_static_basic] ****
2026-02-26 22:44:49,139 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:49 +0000 (0:00:00.037)       0:00:41.121 ***** 
2026-02-26 22:44:49,170 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost
2026-02-26 22:44:49,219 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************
2026-02-26 22:44:49,219 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:49 +0000 (0:00:00.080)       0:00:41.202 ***** 
2026-02-26 22:44:49,697 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:49,706 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] *******
2026-02-26 22:44:49,706 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:49 +0000 (0:00:00.486)       0:00:41.689 ***** 
2026-02-26 22:44:49,956 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:44:49,969 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] ***************
2026-02-26 22:44:49,969 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:49 +0000 (0:00:00.262)       0:00:41.952 ***** 
2026-02-26 22:44:50,574 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:44:50,581 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************
2026-02-26 22:44:50,581 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:50 +0000 (0:00:00.611)       0:00:42.564 ***** 
2026-02-26 22:44:51,135 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:44:51,149 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ******************************
2026-02-26 22:44:51,149 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:51 +0000 (0:00:00.568)       0:00:43.132 ***** 
2026-02-26 22:44:51,167 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "Ingester Memory Flushed successfully"
}
2026-02-26 22:44:51,182 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki: test_static_basic] ***
2026-02-26 22:44:51,182 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:51 +0000 (0:00:00.032)       0:00:43.165 ***** 
2026-02-26 22:44:51,215 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost
2026-02-26 22:44:51,242 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_static_basic] *****************
2026-02-26 22:44:51,242 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:51 +0000 (0:00:00.059)       0:00:43.224 ***** 
2026-02-26 22:44:51,260 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Input file has 24 data entries that Loki has to return"
}
2026-02-26 22:44:51,268 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] ***********************************
2026-02-26 22:44:51,268 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:44:51 +0000 (0:00:00.026)       0:00:43.251 ***** 
2026-02-26 22:50:55,467 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:50:55,481 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file] **********************
2026-02-26 22:50:55,481 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:50:55 +0000 (0:06:04.212)       0:06:47.464 ***** 
2026-02-26 22:50:55,977 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:50:55,985 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_static_basic] **********
2026-02-26 22:50:55,985 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:50:55 +0000 (0:00:00.503)       0:06:47.967 ***** 
2026-02-26 22:50:56,031 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "Query returned all data entries. Input file had 24 entries and Loki returned 24"
}
2026-02-26 22:50:56,038 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic] ***
2026-02-26 22:50:56,038 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:50:56 +0000 (0:00:00.053)       0:06:48.021 ***** 
2026-02-26 22:50:56,329 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:50:56,342 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_static_basic] ******
2026-02-26 22:50:56,342 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:50:56 +0000 (0:00:00.304)       0:06:48.325 ***** 
2026-02-26 22:50:56,369 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost
2026-02-26 22:50:56,389 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic] ***
2026-02-26 22:50:56,389 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:50:56 +0000 (0:00:00.047)       0:06:48.372 ***** 
2026-02-26 22:51:02,866 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:02,882 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_static_basic] ***
2026-02-26 22:51:02,882 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:02 +0000 (0:00:06.492)       0:06:54.865 ***** 
2026-02-26 22:51:02,914 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_by_type": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get",
            "-g",
            "type"
        ],
        "delta": "0:00:06.230832",
        "end": "2026-02-26 22:51:02.820706",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-26 22:50:56.589874",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-26 22:51:02,926 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_static_basic] ***
2026-02-26 22:51:02,926 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:02 +0000 (0:00:00.043)       0:06:54.909 ***** 
2026-02-26 22:51:03,355 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:03,363 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic] ***
2026-02-26 22:51:03,363 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:03 +0000 (0:00:00.436)       0:06:55.345 ***** 
2026-02-26 22:51:06,493 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:06,510 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_static_basic] ***
2026-02-26 22:51:06,511 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:06 +0000 (0:00:03.147)       0:06:58.493 ***** 
2026-02-26 22:51:06,540 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_summary": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get"
        ],
        "delta": "0:00:02.870817",
        "end": "2026-02-26 22:51:06.445378",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-26 22:51:03.574561",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-26 22:51:06,555 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** help for openstack rating summary cmd] ***
2026-02-26 22:51:06,556 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:06 +0000 (0:00:00.045)       0:06:58.538 ***** 
2026-02-26 22:51:08,621 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:08,631 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary get help output] ***
2026-02-26 22:51:08,631 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:08 +0000 (0:00:02.075)       0:07:00.614 ***** 
2026-02-26 22:51:08,652 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_h.stdout_lines": [
        "usage: openstack rating summary get [-h]",
        "                                    [-f {csv,df-to-csv,json,table,value,yaml}]",
        "                                    [-c COLUMN]",
        "                                    [--quote {all,minimal,none,nonnumeric}]",
        "                                    [--noindent] [--max-width <integer>]",
        "                                    [--fit-width] [--print-empty]",
        "                                    [--format-config-file FORMAT_CONFIG]",
        "                                    [--sort-column SORT_COLUMN]",
        "                                    [--sort-ascending | --sort-descending]",
        "                                    [-t TENANT_ID] [-b BEGIN] [-e END]",
        "                                    [-s SERVICE] [-g GROUPBY [GROUPBY ...]]",
        "                                    [-a]",
        "",
        "Get a summary report.",
        "",
        "optional arguments:",
        "  -h, --help            show this help message and exit",
        "  -t TENANT_ID, --tenant-id TENANT_ID",
        "                        Tenant id.",
        "  -b BEGIN, --begin BEGIN",
        "                        Begin timestamp.",
        "  -e END, --end END     End timestamp.",
        "  -s SERVICE, --service SERVICE",
        "                        Service Type.",
        "  -g GROUPBY [GROUPBY ...], --groupby GROUPBY [GROUPBY ...]",
        "                        Fields to group by, space-separated. (res_type and",
        "                        tenant_id are supported for now)",
        "  -a, --all-tenants     Allows to get summary from all tenants (admin only).",
        "                        Defaults to False.",
        "",
        "output formatters:",
        "  output formatter options",
        "",
        "  -f {csv,df-to-csv,json,table,value,yaml}, --format {csv,df-to-csv,json,table,value,yaml}",
        "                        the output format, defaults to table",
        "  -c COLUMN, --column COLUMN",
        "                        specify the column(s) to include, can be repeated to",
        "                        show multiple columns",
        "  --sort-column SORT_COLUMN",
        "                        specify the column(s) to sort the data (columns",
        "                        specified first have a priority, non-existing columns",
        "                        are ignored), can be repeated",
        "  --sort-ascending      sort the column(s) in ascending order",
        "  --sort-descending     sort the column(s) in descending order",
        "",
        "CSV Formatter:",
        "  --quote {all,minimal,none,nonnumeric}",
        "                        when to include quotes, defaults to nonnumeric",
        "",
        "json formatter:",
        "  --noindent            whether to disable indenting the JSON",
        "",
        "table formatter:",
        "  --max-width <integer>",
        "                        Maximum display width, <1 to disable. You can also use",
        "                        the CLIFF_MAX_TERM_WIDTH environment variable, but the",
        "                        parameter takes precedence.",
        "  --fit-width           Fit the table to the display width. Implied if --max-",
        "                        width greater than 0. Set the environment variable",
        "                        CLIFF_FIT_WIDTH=1 to always enable",
        "  --print-empty         Print empty table if there is no data to show.",
        "",
        "dataframe-to-csv formatter:",
        "  --format-config-file FORMAT_CONFIG",
        "                        Config file for the dict-to-csv formatter",
        "",
        "This command is provided by the python-cloudkittyclient plugin."
    ]
}
2026-02-26 22:51:08,661 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic data file] *********************
2026-02-26 22:51:08,661 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:08 +0000 (0:00:00.029)       0:07:00.643 ***** 
2026-02-26 22:51:08,857 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:08,863 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] ***********************
2026-02-26 22:51:08,863 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:08 +0000 (0:00:00.202)       0:07:00.846 ***** 
2026-02-26 22:51:09,072 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:09,080 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic] ***
2026-02-26 22:51:09,080 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:09 +0000 (0:00:00.216)       0:07:01.063 ***** 
2026-02-26 22:51:09,112 p=37225 u=zuul n=ansible | fatal: [localhost]: FAILED! => {
    "assertion": "(synth_data.content | b64decode | from_yaml) == (loki_data.content | b64decode | from_yaml)",
    "changed": false,
    "evaluated_to": false,
    "msg": "THE FILES ARE DIFFERENT!\nPath A: \"/home/zuul/ci-framework-data/artifacts/test_static_basic-synth_data.json\"\nPath B: \"/home/zuul/ci-framework-data/artifacts/test_static_basic-loki_metrics_totals.yml\"\n"
}
2026-02-26 22:51:09,112 p=37225 u=zuul n=ansible | ...ignoring
2026-02-26 22:51:09,120 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic_gid] ***
2026-02-26 22:51:09,120 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:09 +0000 (0:00:00.039)       0:07:01.102 ***** 
2026-02-26 22:51:09,141 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/gen_synth_loki_data.yml for localhost
2026-02-26 22:51:09,151 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Set variables dynamically test_static_basic_gid] ***
2026-02-26 22:51:09,151 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:09 +0000 (0:00:00.031)       0:07:01.133 ***** 
2026-02-26 22:51:09,180 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:09,186 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Check for preexisting output file] ****************
2026-02-26 22:51:09,186 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:09 +0000 (0:00:00.035)       0:07:01.169 ***** 
2026-02-26 22:51:09,389 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:09,397 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Generate Synthetic Data test_static_basic_gid] ****
2026-02-26 22:51:09,397 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:09 +0000 (0:00:00.210)       0:07:01.379 ***** 
2026-02-26 22:51:09,705 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:09,712 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic_gid] ***
2026-02-26 22:51:09,712 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:09 +0000 (0:00:00.315)       0:07:01.695 ***** 
2026-02-26 22:51:09,966 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:09,976 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Load metrics from YAML file] **********************
2026-02-26 22:51:09,976 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:09 +0000 (0:00:00.263)       0:07:01.959 ***** 
2026-02-26 22:51:09,991 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:10,002 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Load data to loki: test_static_basic_gid] *********
2026-02-26 22:51:10,002 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:10 +0000 (0:00:00.026)       0:07:01.985 ***** 
2026-02-26 22:51:10,026 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/load_loki_data.yml for localhost
2026-02-26 22:51:10,040 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic_gid] ***
2026-02-26 22:51:10,040 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:10 +0000 (0:00:00.038)       0:07:02.023 ***** 
2026-02-26 22:51:10,059 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/ingest_loki_data.yml for localhost
2026-02-26 22:51:10,068 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Read log file content] ****************************
2026-02-26 22:51:10,068 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:10 +0000 (0:00:00.027)       0:07:02.051 ***** 
2026-02-26 22:51:10,287 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:10,293 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Push data to Loki] ********************************
2026-02-26 22:51:10,293 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:10 +0000 (0:00:00.224)       0:07:02.276 ***** 
2026-02-26 22:51:10,671 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:10,678 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Confirm Success] **********************************
2026-02-26 22:51:10,678 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:10 +0000 (0:00:00.384)       0:07:02.660 ***** 
2026-02-26 22:51:10,707 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Ingestion Successful!"
}
2026-02-26 22:51:10,716 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Flush Data to loki Storage: test_static_basic_gid] ***
2026-02-26 22:51:10,717 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:10 +0000 (0:00:00.038)       0:07:02.699 ***** 
2026-02-26 22:51:10,753 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/flush_loki_data.yml for localhost
2026-02-26 22:51:10,770 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Create directory inside openstack CLI] ************
2026-02-26 22:51:10,771 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:10 +0000 (0:00:00.053)       0:07:02.753 ***** 
2026-02-26 22:51:11,256 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:11,274 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Create a directory to extract certificates] *******
2026-02-26 22:51:11,274 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:11 +0000 (0:00:00.503)       0:07:03.257 ***** 
2026-02-26 22:51:11,490 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:11,501 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Copy certificates to openstack CLI] ***************
2026-02-26 22:51:11,501 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:11 +0000 (0:00:00.226)       0:07:03.484 ***** 
2026-02-26 22:51:12,053 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:12,066 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Trigger Flush] ************************************
2026-02-26 22:51:12,066 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:12 +0000 (0:00:00.565)       0:07:04.049 ***** 
2026-02-26 22:51:12,652 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:12,662 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Flush Status] ******************************
2026-02-26 22:51:12,662 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:12 +0000 (0:00:00.595)       0:07:04.645 ***** 
2026-02-26 22:51:12,687 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "Ingester Memory Flushed successfully"
}
2026-02-26 22:51:12,699 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Retrieve Data log from loki: test_static_basic_gid] ***
2026-02-26 22:51:12,699 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:12 +0000 (0:00:00.036)       0:07:04.682 ***** 
2026-02-26 22:51:12,726 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/retrieve_loki_data.yml for localhost
2026-02-26 22:51:12,750 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Expected Count test_static_basic_gid] *************
2026-02-26 22:51:12,750 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:12 +0000 (0:00:00.051)       0:07:04.733 ***** 
2026-02-26 22:51:12,773 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "msg": "Input file has 24 data entries that Loki has to return"
}
2026-02-26 22:51:12,781 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Query Loki API] ***********************************
2026-02-26 22:51:12,781 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:12 +0000 (0:00:00.030)       0:07:04.763 ***** 
2026-02-26 22:51:13,411 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:13,428 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Save Loki Data to JSON file] **********************
2026-02-26 22:51:13,429 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:13 +0000 (0:00:00.647)       0:07:05.411 ***** 
2026-02-26 22:51:13,817 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:13,824 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Verify Data Integrity test_static_basic_gid] ******
2026-02-26 22:51:13,824 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:13 +0000 (0:00:00.395)       0:07:05.807 ***** 
2026-02-26 22:51:13,855 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "changed": false,
    "msg": "Query returned all data entries. Input file had 24 entries and Loki returned 24"
}
2026-02-26 22:51:13,863 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic_gid] ***
2026-02-26 22:51:13,863 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:13 +0000 (0:00:00.039)       0:07:05.846 ***** 
2026-02-26 22:51:14,160 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:14,167 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Get total rate from loki: test_static_basic_gid] ***
2026-02-26 22:51:14,167 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:14 +0000 (0:00:00.304)       0:07:06.150 ***** 
2026-02-26 22:51:14,186 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/loki_rate.yml for localhost
2026-02-26 22:51:14,200 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic_gid] ***
2026-02-26 22:51:14,200 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:14 +0000 (0:00:00.032)       0:07:06.182 ***** 
2026-02-26 22:51:16,977 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:16,988 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating by type test_static_basic_gid] ***
2026-02-26 22:51:16,988 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:16 +0000 (0:00:02.788)       0:07:08.971 ***** 
2026-02-26 22:51:17,008 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_by_type": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get",
            "-g",
            "type"
        ],
        "delta": "0:00:02.578682",
        "end": "2026-02-26 22:51:16.946782",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-26 22:51:14.368100",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-26 22:51:17,018 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Save output as a loadable variable file test_static_basic_gid] ***
2026-02-26 22:51:17,018 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:17 +0000 (0:00:00.030)       0:07:09.001 ***** 
2026-02-26 22:51:17,387 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:17,394 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic_gid] ***
2026-02-26 22:51:17,394 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:17 +0000 (0:00:00.375)       0:07:09.376 ***** 
2026-02-26 22:51:20,036 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:20,045 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary test_static_basic_gid] ***
2026-02-26 22:51:20,045 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:20 +0000 (0:00:02.651)       0:07:12.027 ***** 
2026-02-26 22:51:20,061 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_summary": {
        "changed": true,
        "cmd": [
            "oc",
            "rsh",
            "openstackclient",
            "openstack",
            "rating",
            "summary",
            "get"
        ],
        "delta": "0:00:02.448293",
        "end": "2026-02-26 22:51:20.011113",
        "failed": false,
        "failed_when_result": false,
        "msg": "",
        "rc": 0,
        "start": "2026-02-26 22:51:17.562820",
        "stderr": "",
        "stderr_lines": [],
        "stdout": "+-----------+---------------+------+---------------------+---------------------+\n| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |\n+-----------+---------------+------+---------------------+---------------------+\n| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |\n+-----------+---------------+------+---------------------+---------------------+",
        "stdout_lines": [
            "+-----------+---------------+------+---------------------+---------------------+",
            "| Tenant ID | Resource Type | Rate | Begin Time          | End Time            |",
            "+-----------+---------------+------+---------------------+---------------------+",
            "| ALL       | ALL           | 0    | 2026-02-01T00:00:00 | 2026-03-01T00:00:00 |",
            "+-----------+---------------+------+---------------------+---------------------+"
        ]
    }
}
2026-02-26 22:51:20,068 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** help for openstack rating summary cmd] ***
2026-02-26 22:51:20,068 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:20 +0000 (0:00:00.022)       0:07:12.050 ***** 
2026-02-26 22:51:21,937 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:21,945 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : **INFO** Print the rating summary get help output] ***
2026-02-26 22:51:21,946 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:21 +0000 (0:00:01.877)       0:07:13.928 ***** 
2026-02-26 22:51:21,976 p=37225 u=zuul n=ansible | ok: [localhost] => {
    "cost_totals_h.stdout_lines": [
        "usage: openstack rating summary get [-h]",
        "                                    [-f {csv,df-to-csv,json,table,value,yaml}]",
        "                                    [-c COLUMN]",
        "                                    [--quote {all,minimal,none,nonnumeric}]",
        "                                    [--noindent] [--max-width <integer>]",
        "                                    [--fit-width] [--print-empty]",
        "                                    [--format-config-file FORMAT_CONFIG]",
        "                                    [--sort-column SORT_COLUMN]",
        "                                    [--sort-ascending | --sort-descending]",
        "                                    [-t TENANT_ID] [-b BEGIN] [-e END]",
        "                                    [-s SERVICE] [-g GROUPBY [GROUPBY ...]]",
        "                                    [-a]",
        "",
        "Get a summary report.",
        "",
        "optional arguments:",
        "  -h, --help            show this help message and exit",
        "  -t TENANT_ID, --tenant-id TENANT_ID",
        "                        Tenant id.",
        "  -b BEGIN, --begin BEGIN",
        "                        Begin timestamp.",
        "  -e END, --end END     End timestamp.",
        "  -s SERVICE, --service SERVICE",
        "                        Service Type.",
        "  -g GROUPBY [GROUPBY ...], --groupby GROUPBY [GROUPBY ...]",
        "                        Fields to group by, space-separated. (res_type and",
        "                        tenant_id are supported for now)",
        "  -a, --all-tenants     Allows to get summary from all tenants (admin only).",
        "                        Defaults to False.",
        "",
        "output formatters:",
        "  output formatter options",
        "",
        "  -f {csv,df-to-csv,json,table,value,yaml}, --format {csv,df-to-csv,json,table,value,yaml}",
        "                        the output format, defaults to table",
        "  -c COLUMN, --column COLUMN",
        "                        specify the column(s) to include, can be repeated to",
        "                        show multiple columns",
        "  --sort-column SORT_COLUMN",
        "                        specify the column(s) to sort the data (columns",
        "                        specified first have a priority, non-existing columns",
        "                        are ignored), can be repeated",
        "  --sort-ascending      sort the column(s) in ascending order",
        "  --sort-descending     sort the column(s) in descending order",
        "",
        "CSV Formatter:",
        "  --quote {all,minimal,none,nonnumeric}",
        "                        when to include quotes, defaults to nonnumeric",
        "",
        "json formatter:",
        "  --noindent            whether to disable indenting the JSON",
        "",
        "table formatter:",
        "  --max-width <integer>",
        "                        Maximum display width, <1 to disable. You can also use",
        "                        the CLIFF_MAX_TERM_WIDTH environment variable, but the",
        "                        parameter takes precedence.",
        "  --fit-width           Fit the table to the display width. Implied if --max-",
        "                        width greater than 0. Set the environment variable",
        "                        CLIFF_FIT_WIDTH=1 to always enable",
        "  --print-empty         Print empty table if there is no data to show.",
        "",
        "dataframe-to-csv formatter:",
        "  --format-config-file FORMAT_CONFIG",
        "                        Config file for the dict-to-csv formatter",
        "",
        "This command is provided by the python-cloudkittyclient plugin."
    ]
}
2026-02-26 22:51:21,984 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Read the synthetic data file] *********************
2026-02-26 22:51:21,984 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:21 +0000 (0:00:00.038)       0:07:13.967 ***** 
2026-02-26 22:51:22,200 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:22,207 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Read loki totals yaml file] ***********************
2026-02-26 22:51:22,207 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:22 +0000 (0:00:00.222)       0:07:14.189 ***** 
2026-02-26 22:51:22,387 p=37225 u=zuul n=ansible | ok: [localhost]
2026-02-26 22:51:22,395 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic_gid] ***
2026-02-26 22:51:22,395 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:22 +0000 (0:00:00.188)       0:07:14.378 ***** 
2026-02-26 22:51:22,413 p=37225 u=zuul n=ansible | fatal: [localhost]: FAILED! => {
    "assertion": "(synth_data.content | b64decode | from_yaml) == (loki_data.content | b64decode | from_yaml)",
    "changed": false,
    "evaluated_to": false,
    "msg": "THE FILES ARE DIFFERENT!\nPath A: \"/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-synth_data.json\"\nPath B: \"/home/zuul/ci-framework-data/artifacts/test_static_basic_gid-loki_metrics_totals.yml\"\n"
}
2026-02-26 22:51:22,413 p=37225 u=zuul n=ansible | ...ignoring
2026-02-26 22:51:22,419 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Cleanup after job run] ****************************
2026-02-26 22:51:22,420 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:22 +0000 (0:00:00.024)       0:07:14.402 ***** 
2026-02-26 22:51:22,436 p=37225 u=zuul n=ansible | included: /home/zuul/src/github.com/infrawatch/feature-verification-tests/roles/telemetry_chargeback/tasks/cleanup_ck.yml for localhost
2026-02-26 22:51:22,449 p=37225 u=zuul n=ansible | TASK [telemetry_chargeback : Cleanup local certificates] ***********************
2026-02-26 22:51:22,449 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:22 +0000 (0:00:00.029)       0:07:14.431 ***** 
2026-02-26 22:51:22,644 p=37225 u=zuul n=ansible | changed: [localhost]
2026-02-26 22:51:22,682 p=37225 u=zuul n=ansible | PLAY RECAP *********************************************************************
2026-02-26 22:51:22,682 p=37225 u=zuul n=ansible | localhost                  : ok=179  changed=27   unreachable=0    failed=0    skipped=7    rescued=0    ignored=2   
2026-02-26 22:51:22,692 p=37225 u=zuul n=ansible | Thursday 26 February 2026  22:51:22 +0000 (0:00:00.243)       0:07:14.675 ***** 
2026-02-26 22:51:22,692 p=37225 u=zuul n=ansible | =============================================================================== 
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Query Loki API --------------------------------- 364.21s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic --- 6.49s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Enable Cloudkitty Module (hashmap) --------------- 3.93s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Change priority for CloudKitty hashmap module ---- 3.33s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic --- 3.15s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty by type from Cloudkitty test_static_basic_gid --- 2.79s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Get status of all CloudKitty rating modules ------ 2.67s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : TEST Get Rate and Qty Summary from Cloudkitty test_static_basic_gid --- 2.65s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Find the current value of hashmap ---------------- 2.57s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | common : Verify endpoint "['cloudkitty', 'rating', 'internal']" --------- 2.47s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | common : Verify endpoint "['cloudkitty', 'rating', 'public']" ----------- 2.39s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Get admin user ID for CI ------------------------- 2.29s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Get admin project ID for CI ---------------------- 2.18s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : **INFO** help for openstack rating summary cmd --- 2.08s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : **INFO** help for openstack rating summary cmd --- 1.88s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | Gathering Facts --------------------------------------------------------- 0.89s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Query Loki API ----------------------------------- 0.65s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Get Loki Public Route Host ----------------------- 0.64s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 0.61s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.60s
2026-02-26 22:51:22,693 p=37225 u=zuul n=ansible | telemetry_chargeback : Trigger Flush ------------------------------------ 0.57s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | telemetry_chargeback : Copy certificates to openstack CLI --------------- 0.57s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.56s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.52s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.50s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file ---------------------- 0.50s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | telemetry_chargeback : Create directory inside openstack CLI ------------ 0.49s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_static_basic --- 0.44s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.42s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.42s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.42s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.41s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.41s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-ingester pod is Running in openstack namespace
 --- 0.41s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-index-gateway-grpc" ----- 0.41s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : TEST Check cloudkitty-api pod is Running in openstack namespace
 --- 0.41s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-gateway-http" ----------- 0.40s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | telemetry_chargeback : Save Loki Data to JSON file ---------------------- 0.40s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-ingester-grpc" ---------- 0.39s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-index-gateway pod is Running in openstack namespace
 --- 0.39s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Get the openstack-operators project ---------------------------- 0.39s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | telemetry_chargeback : Push data to Loki -------------------------------- 0.38s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.38s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | telemetry_chargeback : Save output as a loadable variable file test_static_basic_gid --- 0.38s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-ingester-http" ---------- 0.37s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-distributor pod is Running in openstack namespace
 --- 0.37s
2026-02-26 22:51:22,694 p=37225 u=zuul n=ansible | common : Get Pod Instance name "Running" -------------------------------- 0.37s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : TEST Check cloudkitty-proc pod is Running in openstack namespace
 --- 0.36s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-index-gateway-http" ----- 0.36s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-compactor pod is Running in openstack namespace
 --- 0.36s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-public" --------------------------- 0.35s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-query-frontend-http" ---- 0.35s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data test_static_basic -------- 0.35s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-querier-grpc" ----------- 0.35s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Extract Certificates from Openshift Secret ------- 0.34s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-gossip-ring" ------------ 0.34s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Get the openstack project -------------------------------------- 0.34s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-internal" ------------------------- 0.34s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-compactor-http" --------- 0.33s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-query-frontend pod is Running in openstack namespace
 --- 0.33s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-distributor-http" ------- 0.33s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-query-frontend-grpc" ---- 0.33s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-querier-http" ----------- 0.33s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Ensure Local Certificate Directory Exists -------- 0.32s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-compactor-grpc" --------- 0.32s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data test_static_basic_gid ---- 0.32s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : TEST Check cloudkitty-lokistack-querier pod is Running in openstack namespace
 --- 0.31s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Extract Client Certificates ---------------------- 0.31s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Find test files ---------------------------------- 0.31s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | common : Verify Service - "cloudkitty-lokistack-distributor-grpc" ------- 0.31s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.31s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic --- 0.30s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Generate chargeback stats from Loki-retrieved data file: test_static_basic_gid --- 0.30s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Extract CA Bundle -------------------------------- 0.30s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic --- 0.29s
2026-02-26 22:51:22,695 p=37225 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.29s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Generate chargeback rating from synthetic data file test_static_basic_gid --- 0.26s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.26s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Cleanup local certificates ----------------------- 0.24s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Create a directory to extract certificates ------- 0.23s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Read log file content ---------------------------- 0.23s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Read the synthetic data file --------------------- 0.22s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.22s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Check for preexisting output file ---------------- 0.21s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Read the synthetic data file --------------------- 0.20s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Read loki totals yaml file ----------------------- 0.19s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | common : Run service tests ---------------------------------------------- 0.19s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | common : Run pod tests -------------------------------------------------- 0.10s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage: test_static_basic ---- 0.08s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | common : Run project tests ---------------------------------------------- 0.07s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki: test_static_basic --- 0.06s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Flush Data to loki Storage: test_static_basic_gid --- 0.05s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_static_basic ---------- 0.05s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Retrieve Data log from loki: test_static_basic_gid --- 0.05s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Process and Loop if files exist ------------------ 0.05s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | common : Run endpoint tests --------------------------------------------- 0.05s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : TEST Validate CloudKitty module states ----------- 0.05s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_static_basic ------ 0.05s
2026-02-26 22:51:22,696 p=37225 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_static_basic --- 0.05s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_static_basic --- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | common : TEST Check that cloudkitty public endpoint exists
 ------------- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Setup Loki Environment --------------------------- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic --- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Verify Data Integrity test_static_basic_gid ------ 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Extract only the filenames into a clean list ----- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Confirm Success ---------------------------------- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary get help output --- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | common : Verify container tests ----------------------------------------- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_static_basic_gid --------- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Confirm Success ---------------------------------- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Load data to loki: test_static_basic ------------- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically test_static_basic_gid --- 0.04s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Validate Chargeback Feature deployed correctly --- 0.03s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-public service is running in openstack namespace
 --- 0.03s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | common : Run file tests ------------------------------------------------- 0.03s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-ingester-http service is running in openstack namespace
 --- 0.03s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Verify Flush Status ------------------------------ 0.03s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Get total rate from loki: test_static_basic_gid --- 0.03s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-gossip-ring service is running in openstack namespace
 --- 0.03s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : TEST Set priority for CloudKitty hashmap module --- 0.03s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Set admin project ID for CI ---------------------- 0.03s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | common : TEST Check that cloudkitty internal endpoint exists
 ----------- 0.03s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic_gid --- 0.03s
2026-02-26 22:51:22,697 p=37225 u=zuul n=ansible | common : Run CR tests --------------------------------------------------- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : Set variables dynamically test_static_basic ------ 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : Expected Count test_static_basic_gid ------------- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating by type test_static_basic_gid --- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : Run crd tests -------------------------------------------------- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary get help output --- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : Set admin user ID for CI ------------------------- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : Cleanup after job run ---------------------------- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-distributor-http service is running in openstack namespace
 --- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : Generate Synthetic Data for each file: test_static_basic --- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : Check that each package manifest is present -------------------- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : Get the manifest list once ------------------------------------- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic_gid --- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-ingester-grpc service is running in openstack namespace
 --- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : Ingests Cloudkitty Data log: test_static_basic --- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-query-frontend-http service is running in openstack namespace
 --- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : Debug URLs --------------------------------------- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : Expected Count test_static_basic ----------------- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-gateway-http service is running in openstack namespace
 --- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-querier-grpc service is running in openstack namespace
 --- 0.03s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-query-frontend-grpc service is running in openstack namespace
 --- 0.02s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-distributor-grpc service is running in openstack namespace
 --- 0.02s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-index-gateway-http service is running in openstack namespace
 --- 0.02s
2026-02-26 22:51:22,698 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-index-gateway-grpc service is running in openstack namespace
 --- 0.02s
2026-02-26 22:51:22,699 p=37225 u=zuul n=ansible | telemetry_chargeback : Set Derived Loki URLs ---------------------------- 0.02s
2026-02-26 22:51:22,699 p=37225 u=zuul n=ansible | telemetry_chargeback : TEST Compare synthetic data vs loki data result files test_static_basic_gid --- 0.02s
2026-02-26 22:51:22,699 p=37225 u=zuul n=ansible | common : TEST Verify openstack-operators project exists
 ---------------- 0.02s
2026-02-26 22:51:22,699 p=37225 u=zuul n=ansible | common : TEST Verify openstack project exists
 -------------------------- 0.02s
2026-02-26 22:51:22,699 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-internal service is running in openstack namespace
 --- 0.02s
2026-02-26 22:51:22,699 p=37225 u=zuul n=ansible | telemetry_chargeback : Load metrics from YAML file ---------------------- 0.02s
2026-02-26 22:51:22,699 p=37225 u=zuul n=ansible | telemetry_chargeback : Set Loki URLs ------------------------------------ 0.02s
2026-02-26 22:51:22,699 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-querier-http service is running in openstack namespace
 --- 0.02s
2026-02-26 22:51:22,699 p=37225 u=zuul n=ansible | common : TEST Verify cloudkitty-lokistack-compactor-http service is running in openstack namespace
 --- 0.02s
2026-02-26 22:51:22,699 p=37225 u=zuul n=ansible | telemetry_chargeback : **INFO** Print the rating summary test_static_basic_gid --- 0.02s
2026-02-26 22:51:22,699 p=37225 u=zuul n=ansible | common : Run subscription tests ----------------------------------------- 0.02s