Mypy can't find obvious type mismatch when using namespace-packages and subfolders with the same name

Question:

The problem is as simple as can be if put in one file:

class A():
  a = 1

class B():
  b = 2

def test(x: A):
  return x

def testit():
  b = B()
  test(b)

And mypy finds the issue here with mypy test_folder --check-untyped-defs

But if those functions / classes are put in various files mypy doesn’t detect the problem. It doesn’t even warn that it can’t piece information together which IMO is the biggest problem.

I spent some time to create a minimal reproducible version:

The file tree looks like this:

src
  mb2
    base.py
    steps
      init
        step.py
        impl.py
      jmeter
        step.py
  test
    test.py

The content of the 5 involved files is:

base.py

class BaseCfg:
    pass

test.py

from mb2.steps.jmeter.step import JCfg
from mb2.steps.init.impl import get_resource_costs

class TestCollectMetrics:
    def test_get_resource_costs(self):
        cfg = JCfg()
        get_resource_costs(cfg)  # <- Wrong type should be detected

init/impl.py

from .step import Cfg

def get_resource_costs(config: Cfg):
    pass

init/step.py

from mb2.base import BaseCfg

class Cfg(BaseCfg):
    pass

jmeter/step.py

from mb2.base import BaseCfg

class JCfg(BaseCfg):
    pass

The test is passing an object of type JCfg to the method get_resource_costs. While the method specifies it needs type Cfg. This doens’t get detected by mypy.

It only runs with --namespace-packages as otherwise the duplicate step.py files become a problem. So the command to reproduce is:

mypy src --check-untyped-defs --namespace-packages

This returns Success: no issues found in 5 source files

putting reveal_type(cfg) and reveal_type(get_resource_costs) into test.py returns Revealed type is "Any".

Putting local errors in any of the 5 files gets detected, so mypy is checking all files.

I assume it is connected to namespace-packages not being able to resolve paths and silently failing. Any way to fix this?

Asked By: David Schumann

||

Answers:

The reason the errors were not found were due to ignore_missing_imports = True in my setup.cfg.

The underlying problem is related to module discovery:

  • Mypy is detecting modules differently than python. Which will always be a problem in my opinion
  • Mypy can’t deal with "Duplicate modules".

Together this makes it hard to properly discover all modules, but only discover them once.

In the end I needed to use MYPYPATH=src mypy src --strict --namespace-packages --explicit-package-bases.

Setting MYPYPATH=src in addition to using mypy src is important. Unclear to me why.

Answered By: David Schumann
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.