What does `python setup.py check` actually do?

First stop, the distutils package documentation:

The check command performs some tests on the meta-data of a package. For example, it verifies that all required meta-data are provided as the arguments passed to the setup() function.

So it tests if you have filled in your metadata correctly; see it as a quality control step when creating a Python package.

Next, we can check if the command line offers any help:

$ python setup.py --help-commands | grep check
  check             perform some checks on the package
$ python setup.py check --help
# ...
Options for 'check' command:
  --metadata (-m)          Verify meta-data
  --restructuredtext (-r)  Checks if long string meta-data syntax are
                           reStructuredText-compliant
  --strict (-s)            Will exit with an error if a check fails

So we can check for metadata and validate the long description as reStructuredText. The latter requires that you have docutils installed:

$ python setup.py check -rs
running check
error: The docutils package is needed.

If you do have it installed and there are no problems, the script just runs and exits with no messages:

$ python setup.py check -r
running check

but if required metadata is missing you get warning messages:

$ python setup.py check -r
running check
warning: check: missing required meta-data: url

warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied

which becomes an error if you have the -s flag given:

$ python setup.py check -rs
running check
warning: check: missing required meta-data: url

warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied

error: Please correct your package.

By default, -m is enabled, -r and -s are disabled.

Also see the command source code.