Python docstrings and inline code; meaning of the ">>>" syntax
Question:
I have some experience in Python but only recently came across extensive usage of docstrings. I’m going through the Financial Market Simulator (FMS) source code, and when I open it in PyCharm I see the following syntax highlighting (screenshot of a code snippet of one of the modules in FMS):
Why are the statements after the >>> highlighted as if they are executable? From what I have read of docstrings, both on the official documentation and on SO (for example here) I think that those statements should not execute, but the syntax highlighting is confusing me, leading me to think that the >>> is a marker for code within the docstring that one wants executed. Or is this just a PyCharm ‘bug’? None of the documentation mentions anything related to this and I’m worried if I’m missing something.
PS: For the record, looking at the code in SublimeText does not reproduce the same behaviour.
Answers:
Your intuition is correct, they are to be executed. But don’t worry, they are doctest strings. They won’t interfere with the normal execution of a module, so everything is fine. PyCharm is just being helpful by recognizing them.
The behavior you are seeing is part of the testing support for Python available in Pycharm.
The settings option is called “Analyze Python code in docstrings” and is available under Python Integrated Tools:
If this check box is selected, PyCharm highlights the code example and
performs syntax checks and code inspections. If this check box is not
selected, the code fragments inside docstrings are not analyzed.
You can disable it if you prefer.
The online documentation details how to run tests and view their results.
The statements written with >>> in the docstrings are doctests.
It lets you test your code by running examples embedded in the documentation and verifying that they produce the expected results. It parses the help text to find examples, runs them and then compares the output text against the expected value.
In your case, PyCharm has done the extra task of highlighting the python code in the docstrings. It won’t affect your normal function execution so you don’t need to worry about it.
Example:
Lets say I have a script named doctest_simple_addition in which i have written some doctests for add() function where some test cases gives proper output and some raises an exception. Then i can verify that my function produces the expected results by running those doctests.
doctest_simple_addition.py
def add(a,b):
"""
>>> add(1, 2)
3
>>> add(5, 3)
8
>>> add('a', 1)
Traceback (most recent call last):
...
TypeError: cannot concatenate 'str' and 'int' objects
"""
return a + b
To run the doctests, use doctest as the main program via the -m option to the interpreter. Usually, no output is produced while the tests are running. You can add the -v option and doctest will then print a detailed log of what it’s trying with a summary at the end.
Doctest looks for lines beginning with the interpreter prompt, >>>, to find the beginning of a test case. The test case is ended by a blank line, or by the next interpreter prompt.
$ python -m doctest -v doctest_simple_addition.py
Trying:
add(1, 2)
Expecting:
3
ok
Trying:
add(5, 3)
Expecting:
8
ok
Trying:
add('a', 1)
Expecting:
Traceback (most recent call last):
...
TypeError: cannot concatenate 'str' and 'int' objects
ok
1 items had no tests:
doctest_simple_addition
1 items passed all tests:
3 tests in doctest_simple_addition.add
3 tests in 2 items.
3 passed and 0 failed.
Test passed.
Note: When doctest sees a traceback header line (either Traceback (most recent call last): or Traceback (innermost last):, depending on the version of Python you are running), it skips ahead to find the exception type and message, ignoring the intervening lines entirely.
This is done because paths in a traceback depend on the location where a module is installed on the filesystem on a given system and it would be impossible to write portable tests as the path would change from system to system.
I have some experience in Python but only recently came across extensive usage of docstrings. I’m going through the Financial Market Simulator (FMS) source code, and when I open it in PyCharm I see the following syntax highlighting (screenshot of a code snippet of one of the modules in FMS):
Why are the statements after the >>> highlighted as if they are executable? From what I have read of docstrings, both on the official documentation and on SO (for example here) I think that those statements should not execute, but the syntax highlighting is confusing me, leading me to think that the >>> is a marker for code within the docstring that one wants executed. Or is this just a PyCharm ‘bug’? None of the documentation mentions anything related to this and I’m worried if I’m missing something.
PS: For the record, looking at the code in SublimeText does not reproduce the same behaviour.
Your intuition is correct, they are to be executed. But don’t worry, they are doctest strings. They won’t interfere with the normal execution of a module, so everything is fine. PyCharm is just being helpful by recognizing them.
The behavior you are seeing is part of the testing support for Python available in Pycharm.
The settings option is called “Analyze Python code in docstrings” and is available under Python Integrated Tools:
If this check box is selected, PyCharm highlights the code example and
performs syntax checks and code inspections. If this check box is not
selected, the code fragments inside docstrings are not analyzed.
You can disable it if you prefer.
The online documentation details how to run tests and view their results.
The statements written with >>> in the docstrings are doctests.
It lets you test your code by running examples embedded in the documentation and verifying that they produce the expected results. It parses the help text to find examples, runs them and then compares the output text against the expected value.
In your case, PyCharm has done the extra task of highlighting the python code in the docstrings. It won’t affect your normal function execution so you don’t need to worry about it.
Example:
Lets say I have a script named doctest_simple_addition in which i have written some doctests for add() function where some test cases gives proper output and some raises an exception. Then i can verify that my function produces the expected results by running those doctests.
doctest_simple_addition.py
def add(a,b):
"""
>>> add(1, 2)
3
>>> add(5, 3)
8
>>> add('a', 1)
Traceback (most recent call last):
...
TypeError: cannot concatenate 'str' and 'int' objects
"""
return a + b
To run the doctests, use doctest as the main program via the -m option to the interpreter. Usually, no output is produced while the tests are running. You can add the -v option and doctest will then print a detailed log of what it’s trying with a summary at the end.
Doctest looks for lines beginning with the interpreter prompt, >>>, to find the beginning of a test case. The test case is ended by a blank line, or by the next interpreter prompt.
$ python -m doctest -v doctest_simple_addition.py
Trying:
add(1, 2)
Expecting:
3
ok
Trying:
add(5, 3)
Expecting:
8
ok
Trying:
add('a', 1)
Expecting:
Traceback (most recent call last):
...
TypeError: cannot concatenate 'str' and 'int' objects
ok
1 items had no tests:
doctest_simple_addition
1 items passed all tests:
3 tests in doctest_simple_addition.add
3 tests in 2 items.
3 passed and 0 failed.
Test passed.
Note: When doctest sees a traceback header line (either Traceback (most recent call last): or Traceback (innermost last):, depending on the version of Python you are running), it skips ahead to find the exception type and message, ignoring the intervening lines entirely.
This is done because paths in a traceback depend on the location where a module is installed on the filesystem on a given system and it would be impossible to write portable tests as the path would change from system to system.