Converting YAML file to python dict

Question:

I am having the following problem of mapping documents within a YAML file to a dict and properly mapping them.

I have the following YAML file, which represents a server (db.yml):

instanceId: i-aaaaaaaa
     environment:us-east
     serverId:someServer
     awsHostname:ip-someip
     serverName:somewebsite.com
     ipAddr:192.168.0.1
     roles:[webserver,php]

I load this YAML file, which I can do without any problems, I think I understand that.

instanceId = getInstanceId()
stream = file('db.yml', 'r')
dict = yaml.load_all(stream)

for key in dict:
    if key in dict == "instanceId":
        print key, dict[key]

I’d like the logic to work like the following:

  • load yaml, map to dict
  • look in every dict in the document, if the instanceId matches that which was set by getInstanceId(), then print out all of the keys and values for that document.

If I look at the map data structure from the command line, I get:

{'instanceId': 'i-aaaaaaaa environment:us-east serverId:someServer awsHostname:ip-someip serverName:someserver ipAddr:192.168.0.1 roles:[webserver,php]'}

I think I might be creating the data structure for the YAML file improperly, and on matching the contents on the dict, I am a bit lost.

Side note: I cannot load all of the documents in this file using yaml.load(), I tried yaml.load_all(), which seems to work but my main issue still exists.

Asked By: Zippy Zeppoli

||
Source

Answers:

I think your yaml file should look like (or at least something like, so it’s structured correctly anyway):

instance:
     Id: i-aaaaaaaa
     environment: us-east
     serverId: someServer
     awsHostname: ip-someip
     serverName: somewebsite.com
     ipAddr: 192.168.0.1
     roles: [webserver,php]

Then, yaml.load(...) returns:

{'instance': {'environment': 'us-east', 'roles': ['webserver', 'php'], 'awsHostname': 'ip-someip', 'serverName': 'somewebsite.com', 'ipAddr': '192.168.0.1', 'serverId': 'someServer', 'Id': 'i-aaaaaaaa'}}

And you can go from there…

So used like:

>>> for key, value in yaml.load(open('test.txt'))['instance'].iteritems():
    print key, value


environment us-east
roles ['webserver', 'php']
awsHostname ip-someip
serverName somewebsite.com
ipAddr 192.168.0.1
serverId someServer
Id i-aaaaaaaa
Answered By: Jon Clements

An additional bug in your code, that doesn’t have to do with YAML:

for key in dict:
    if key in dict == "instanceId": # This doesn't do what you want
        print key, dict[key]

in is an operator that works on sequence types, and also on maps. This is why this isn’t a syntax error… but it doesn’t do what you want.

key in dict will always evaluate to True, because all the keys you’re iterating through are in the dict. So your code boils down to True == "instanceId", which will always evaluate to False, because the boolean value True is never equal to that string.

You might have noticed that the print statement doesn’t produce any output; this is because it never gets called.

Answered By: Michael Scheper

Just use python-benedict, it’s a dict subclass that provides I/O support for most common formats, including yaml.

from benedict import benedict

# path can be a yaml string, a filepath or a remote url
path = 'path/to/data.yml'

d = benedict.from_yaml(path)

# do stuff with your dict
# ...

# write it back to disk
d.to_yaml(filepath=path)

It’s well tested and documented, check the README to see all the features:
https://github.com/fabiocaccamo/python-benedict

Install using pip: pip install python-benedict

Note: I am the author of this project

Answered By: Fabio Caccamo

You can use the bios package for Python3 like below:

import bios

my_dict = bios.read('data.yml')

bios is reading the raw data from a file and converting a python dict object. According to the extension of the file, it can figure out the file type.

Answered By: Bilgehan

If you have reached this question trying to figure out how to get a python dictionary from a yaml file using the pyyaml library try the safe_load option as shown below.

import yaml
yaml_dict = yaml.safe_load("data.yml")
Answered By: najeem

I love to use Path, makes a beautiful one-liner

import yaml
from pathlib import Path
conf = yaml.safe_load(Path('data.yml').read_text())
Answered By: Prox