Cannot customize logging behaviour in Hydra (fb-hydra)

Question:

I am new to Hydra and this is my first question in SO.

My purpose is to customize logging according to this example.

I have the following folder structure for configs:

configs/
├── dataset_structurizers
│   ├── config.yaml
│   └── datasets
│       └── brats_2021.yaml
└── hydra
    └── job_logging
        └── custom.yaml

configs/datasets_structurizers/config.yaml has the structure:

defaults:
  - override hydra/job_logging: custom
  - datasets:
    - brats_2021

And hydra/job_logging/custom.yaml:

version: 1
formatters:
  simple:
    format: '[%(levelname)s] : [%(filename)s] : [%(funcName)s] : %(message)s'
handlers:
  console:
    class: logging.StreamHandler
    formatter: simple
    level: DEBUG
    stream: ext://sys.stdout
loggers:
  dataset_structurizers:
    level: DEBUG
    handlers: [console]
    propagate: no
root:
  level: DEBUG
  handlers: [console]
  propagate: no

If I comment - override hydra/job_logging: custom the code works fine. It also works if I move everything in configs/dataset_structurizers/ to configs. However, with my current folder structure if I run the code I get the following error:

Traceback (most recent call last):
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
    return func()
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/utils.py", line 378, in <lambda>
    lambda: hydra.run(
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 88, in run
    cfg = self.compose_config(
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 559, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 141, in load_configuration
    return self._load_configuration_impl(
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 239, in _load_configuration_impl
    defaults_list = create_defaults_list(
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 748, in create_defaults_list
    defaults, tree = _create_defaults_list(
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 718, in _create_defaults_list
    defaults_tree = _create_defaults_tree(
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 356, in _create_defaults_tree
    ret = _create_defaults_tree_impl(
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 455, in _create_defaults_tree_impl
    return _expand_virtual_root(repo, root, overrides, skip_missing)
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 280, in _expand_virtual_root
    subtree = _create_defaults_tree_impl(
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 512, in _create_defaults_tree_impl
    _update_overrides(defaults_list, overrides, parent, interpolated_subtree)
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 396, in _update_overrides
    In {pcp}: Override '{okey} : {oval}' is defined before '{d.get_override_key()}: {d.get_name()}'.
  File "/home/yere/anaconda3/envs/brain_tumor/lib/python3.8/site-packages/hydra/core/default_element.py", line 495, in get_name
    assert self.value is None or isinstance(self.value, str)
AssertionError

What is the proper way to access hydra/job_logging/custom.yaml from configs/datasets_structurizers/config.yaml?

Asked By: Yere

||

Answers:

You need to put the override default at the end of the defaults list:

# config.yaml
defaults:
  - datasets:
    - brats_2021
  - override hydra/job_logging: custom
Answered By: Jasha

Th problem was not related to the position of the override in the defaults. It is solved appending to the hydra.searchpath the necessary path.

Answered By: Yere
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.