Ansible append dict results for all hosts

Question

I am attempting to use Ansible to send a command to several hosts that returns a dict. I then want to append the the dict results of each host to accumulate the results of all the hosts. Finally, I want to print the dict of accumulated results for later processing or to write to a file. It appears I am failing to combine the dicts due to results showing as string. Is there a way to remedy this? Also, is there a more Ansible efficient way to accomplish this?

Example Playbook:

---
  - hosts: myhosts
    gather_facts: False
    vars:
      mydict: {}
    tasks:
    - name: Get dict result
      shell: "cat /path/dict_output"
      register: output
    - set_fact:
       result_dict="{{ output.stdout}}"

    - debug: var=result_dict

Debug Output:

TASK [debug] ****************************************************************************************************************************************************************
ok: [host-a] => {
    "result_dict": {
        "host_b": [
            {
                "ip-1": {
                    "port": "22", 
                    "service": "ssh"
                }
            }, 
            {
                "ip-2": {
                    "port": "21", 
                    "service": "ftp"
                }
            }
        ]
    }
}
ok: [host-b] => {
    "result_dict": {
        "host_a": [
            {
                "ip-1": {
                    "port": "22", 
                    "service": "ssh"
                }
            }, 
            {
                "ip-2": {
                    "port": "21", 
                    "service": "ftp"
                }
            }
        ]
    }
}

My Attempt to combine results of each host:

- set_fact:
   mydict: "{{ mydict | combine(output.stdout) }}"
- debug: var=mydict

Failed Result:

TASK [set_fact] *************************************************************************************************************************************************************
    fatal: [host-b]: FAILED! => {"msg": "|combine expects dictionaries, got u"{'host_b': [{'ip-1': {'service': 'ssh', 'port': '22'}}, {'ip-2': {'service': 'ftp', 'port': '21'}}]}""}
    fatal: [host-a]: FAILED! => {"msg": "|combine expects dictionaries, got u"{'host_a': [{'ip-1': {'service': 'ssh', 'port': '22'}}, {'ip-2': {'service': 'ftp', 'port': '21'}}]}""}

Desired output of accumulated results:

{
    "host_a": [
        {
            "ip-1": {
                "port": "22",
                "service": "ssh"
            }
        },
        {
            "ip-2": {
                "port": "21",
                "service": "ftp"
            }
        }
    ],
    "host_b": [
        {
            "ip-1": {
                "port": "22",
                "service": "ssh"
            }
        },
        {
            "ip-2": {
                "port": "21",
                "service": "ftp"
            }
        }
    ]
}

Asked By: MBasith

||

Source

Answer 1

You can create that hashmap in a single task running on localhost after you gathered all the info on all the hosts.

You can browse facts from any hosts in the hostvars hashmap, and access a list of all machines in a group through groups['name_of_group'].

Knowing those 2 info, the basic idea is:

Extract all hostvars for the machines in your group and make sure we get a list out of that => groups["myhosts"] | map("extract", hostvars)
Filter that result to remove any item where result_dict is not defined (in case the host failed for example). This is done with the selectattr filter
Reduce that result to retain only the result_dict key from each element in the list. We can do this using the map filter again => map(attribute="result_dict"). We are already very close to what your are looking for, it will be a list of hashmaps (one element for each host). But you are looking for a single hashmap, so….
Use the combine filter to merge the dictionnaries in that list into a single one.

The following play ran after your other tasks should meet your requirements:

- name: Consolidate and display my result
  hosts: localhost

  vars:
    my_final_map: >-
        {{
          groups["myhosts"]
          | map("extract", hostvars)
          | selectattr("result_dict", "defined") 
          | map(attribute="result_dict")
          | combine
        }}

  tasks:
    - name: Display consolidated result
      debug:
        var: my_final_map

Answered By: Zeitounator

Answer 2

The expression below gives "the desired output of accumulated results"

  aresult: "{{ ansible_play_hosts|
               map('extract', hostvars, 'result_dict')|
               combine }}"

You don’t have to worry about the existence of the variable result_dict because failed hosts are excluded. The variable ansible_play_hosts is quote:

List of hosts in the current play run, not limited by the serial. Failed/Unreachable hosts are excluded from this list.

In the example below, read the files /tmp/dict_output on the hosts host_a and host_b and store the dictionaries, returned in stdout, to the variable result_dict

    - command: cat /tmp/dict_output
      register: output
    - set_fact:
        result_dict: "{{ output.stdout }}"
    - debug:
        var: result_dict

gives the same result as in your example (displayed in YAML)

TASK [debug] *********************************************************************************
ok: [host_a] => 
  result_dict:
    host_b:
    - ip-1:
        port: '22'
        service: ssh
    - ip-2:
        port: '21'
        service: ftp
ok: [host_b] => 
  result_dict:
    host_a:
    - ip-1:
        port: '22'
        service: ssh
    - ip-2:
        port: '21'
        service: FTP

To get the "accumulated results" use the special variable ansible_play_hosts to extract the dictionaries result_dict from hostvars. Then, simply combine the dictionaries in the list retuned by map

  aresult: "{{ ansible_play_hosts|
               map('extract', hostvars, 'result_dict')|
               combine }}"

gives the "desired output of accumulated results" (displayed in YAML)

  aresult:
    host_a:
    - ip-1:
        port: '22'
        service: ssh
    - ip-2:
        port: '21'
        service: ftp
    host_b:
    - ip-1:
        port: '22'
        service: ssh
    - ip-2:
        port: '21'
        service: FTP

Example of a complete playbook for testing

- hosts: all

  vars:

    aresult: "{{ ansible_play_hosts|
                 map('extract', hostvars, 'result_dict')|
                 combine }}"

  tasks:

    - command: cat /tmp/dict_output
      register: output
    - set_fact:
        result_dict: "{{ output.stdout }}"
    - debug:
        var: result_dict

    - debug:
        var: aresult
      run_once: true

Inventory

shell> cat hosts
host_a
host_b

Files on the remote hosts

shell> ssh admin@host_a cat /tmp/dict_output
{
    "host_b": [
        {
            "ip-1": {
                "port": "22", 
                "service": "ssh"
                    }
        }, 
        {
            "ip-2": {
                "port": "21", 
                "service": "FTP"
                    }
        }
    ]
}

shell> ssh admin@host_b cat /tmp/dict_output
{
    "host_a": [
        {
            "ip-1": {
                "port": "22", 
                "service": "ssh"
                    }
        }, 
        {
            "ip-2": {
                "port": "21", 
                "service": "FTP"
                    }
        }
    ]
}

Answered By: Vladimir Botka

Ansible append dict results for all hosts

Question:

Answers: