Login

u/invalidpath

AAP 2.6-8 Containerized, Time Zone breaks get_service_token()
▲ 5 r/ansible

AAP 2.6-8 Containerized, Time Zone breaks get_service_token()

So for the past three weeks I've gone through the guide here, Upgrading and Migrating from AAP 2.5 RPM to 2.6 Container and wanted to document a bug I hit and I resolved it.

I'm going to fast-forward and not touch on every little tid bit during this process, but please ask questions if you want. I also do not claim 100% accuracy on the statements I make below.. this is just what I found, and what worked for me.

So went through the guide and got the the past where it's time to install 2.6 and upgrade the containerized 2.5 hosts. I've gotta mention how the installation process took upwards of 8 hours (!!!), this is on M6i.xlarge EC2's as well!

Anyway, so the installation was completing however the web gui reported an Error connecting to the Controller API. Web dev tools showed an HTTP 401 to /api/controller/v2/me/. So I started digging into the controller and gateway logs. This is a gist of what I found:

  • Envoy auths theuser, add a JWT then forwards to the controller
  • Controller validates the JWT, then needs the user claims from the gateway
  • Controller generates a service token using datetime.now(), which in this case returned CDT or 5 hours behind UTC
  • PyJWT encoded the CDT time as UTC
  • Controller sends the expired token value to the gateway's /api/gateway/v1/jwt_claims/ endpoint
  • Gateway rejects the expired token, returns HTTP 401
  • Controller can't validate the user, returns HTTP 401 to the browser
  • Dashboard shows "Error connecting to the controller api"

Now at the time I was unaware that containers default to UTC time.. I did find a RH KB on resolving the receptor images TZ, but I did not find info on the other containers. Now since I like seeing logs and events in my local timezone, I used those steps and created a mounts.conf under .config/container/ for the AAP user on all 8 hosts to set it to 'America/Chicago'.

Now this did not resolve the token issue.. I figure because Django doesn't care what the OS is set to use. So I changed the time_zone in my inventory file to 'UTC'.. went ahead and tested this by editing the /home/aap_user/aap/controller/etc/settings.py under controller_extra_settings time_zone to UTC and restarted the containers.

Controller API error resolved.

From what I gather, this might not be a problem if get_service_token() at /var/lib/awx/venv/awx/lib64/python3.12/site-packages/ansible_base/resource_registry/resource_server.py line:

payload["exp"] = datetime.now() + timedelta(seconds=expiration)
Was changed to:
from datetime import timezone

payload["exp"] = datetime.now(tz=timezone.utc) + timedelta(seconds=expiration)

But Im no python dev so.. IDK. Anyway thanks for reading.

u/invalidpath — 3 days ago
▲ 0 r/WaterTreatment

Kinetico AIO, high pressure backwash?

Disclaimer, My Kinetico 1054fr AIO is used.

Just wondering if the backwash cycle should be at line pressure? I was greatly surprised when I ran the first couple because theres just so much water, at feeder pressure being ejected from the waste outlet. I run my internal pressures a smidge higher at 70lbs.

The oem elbow adapter has a very small orifice, and Im using 1/2” ID tubing.

reddit.com
u/invalidpath — 6 days ago
▲ 10 r/Locksmith

Larsen Storm Door lock, repin?

Bought this door today, guy lost the key long ago. I’m fairly handy so I’d like to try repinning it, except I’m hung up on step one. How to remove these blanks without damaging the housing?

u/invalidpath — 9 days ago
▲ 6 r/ansible

So I'm running the 2.5-23 installer for the containerized variant and encountering a super weird issue.. the installer fails with:

TASK [ansible.containerized_installer.automationgateway : Render proxy variables] ***
fatal: [gw01.domain.com]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible.vars.hostvars.HostVarsVars object' has no attribute '_controller_hostname'. 'ansible.vars.hostvars.HostVarsVars object' has no attribute '_controller_hostname'. 'ansible.vars.hostvars.HostVarsVars object' has no attribute '_controller_hostname'. 'ansible.vars.hostvars.HostVarsVars object' has no attribute '_controller_hostname'\n\nThe error appears to be in '/home/user/ansible-2.5-23-container/collections/ansible_collections/ansible/containerized_installer/roles/automationgateway/tasks/facts.yml': line 148, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Render proxy variables\n  ^ here\n"}

Splitting the installer log file by `PLAY [*` (and verifying the raw file) the automationcontroller task calling facts.yml just doesn't run. Like.. nothing. here's a log excerpt:

TASK [Install and configure redis tcp socket] **********************************
skipping: [eda01.domain.com] => {"changed": false, "false_condition": "redis_mode | default('cluster') == 'standalone'", "skip_reason": "Conditional result was False"}
skipping: [eda02.domain.com] => {"changed": false, "false_condition": "redis_mode | default('cluster') == 'standalone'", "skip_reason": "Conditional result was False"}
skipping: [gw01.domain.com] => {"changed": false, "false_condition": "redis_mode | default('cluster') == 'standalone'", "skip_reason": "Conditional result was False"}
skipping: [gw02.domain.com] => {"changed": false, "false_condition": "redis_mode | default('cluster') == 'standalone'", "skip_reason": "Conditional result was False"}
skipping: [hub01.domain.com] => {"changed": false, "false_condition": "redis_mode | default('cluster') == 'standalone'", "skip_reason": "Conditional result was False"}
skipping: [hub02.domain.com] => {"changed": false, "false_condition": "redis_mode | default('cluster') == 'standalone'", "skip_reason": "Conditional result was False"}

PLAY [Get Automation Controller configuration for gateway] *********************

PLAY [Get Automation EDA configuration for gateway] ****************************

TASK [ansible.containerized_installer.automationeda : Set eda hostname, port(s) and protocol] ***
ok: [eda01.domain.com] => {"ansible_facts": {"_eda_hostname": "eda01.domain.com", "_eda_port": "8445", "_eda_ports": [8082], "_eda_protocol": "https", "_eda_ws_protocol": "wss"}, "changed": false}
ok: [eda02.domain.com] => {"ansible_facts": {"_eda_hostname": "eda02.domain.com", "_eda_port": "8445", "_eda_ports": [8082], "_eda_protocol": "https", "_eda_ws_protocol": "wss"}, "changed": false}

TASK [ansible.containerized_installer.automationeda : Set eda redis hostname] ***
ok: [eda01.domain.com] => {"ansible_facts": {"_eda_redis_hostname": "eda01.domain.com"}, "changed": false}
ok: [eda02.domain.com] => {"ansible_facts": {"_eda_redis_hostname": "eda02.domain.com"}, "changed": false}

The preflight checks are all good, the installer host can def reach all 8 target hosts.. ansible -m ping returns pongs. The inventory file group for the controllers is correct (or else the preflights would fail).

My inventory files controller section is:

[automationcontroller]
ctl01.domain.com receptor_type=hybrid
ctl02.domain.com receptor_type=hybrid

As a test I added a debug task to the very beginning of the /roles/automationcontroller/tasks/facts.yml and theresulting log was unchanged.. which means for some reason the call is failing or not being ran(?)

- name: Get Automation Controller configuration for gateway
  hosts: automationcontroller
  any_errors_fatal: true
  gather_facts: false
  become: false
  tasks:
    - name: Set automation controller facts
      ansible.builtin.import_role:
        name: automationcontroller
        tasks_from: facts.yml

I did check just now and there is not an updated installer, 23 is the latest I saw for download. Just curious if anyone has seen this before?

reddit.com
u/invalidpath — 28 days ago