My rst README is not formatted on pypi.python.org

Question:

When I submit my package to the Python Package Index (https://pypi.python.org/pypi) my README file, which is written with valid reStructuredText and saved as README.rst, is displayed as plain text without any formatting.

I have run it through validators (rstctl and collective.checkdocs) and no errors are returned.

My package is at:
https://pypi.python.org/pypi/lcinvestor

It’s in github at:
https://github.com/jgillick/LendingClubAutoInvestor

Asked By: Jeremy Gillick

||

Answers:

The first thing that pops out at me (after a quick scan) is that in your Advanced Filters section you use two underscores after a link, e.g.,

`Link text <http://example.com>`__

Where it should be

`Link text <http://example.com>`_

It’s odd that the reStructuredText checkers didn’t catch that. If you have docutils installed as well, you can run rst2html.py README.rst and it should print out the HTML. If there are errors it will fail and tell you where the errors were.

Also, fair warning, lists should have no leading spaces, i.e., you have

 - foo
 - bar

Instead of

- foo
- bar

(To make it more visually clear)

- foo # correct
 - one too many for a regular list, it will show up as a quoted list

Also, relative linking doesn’t work like so Text to link <#link>_. If you want to link to a separate section you have to do the following:

Here's my `link <section_name>`_ to the other section.

.. Other stuff here ...

.. _section_name:

Min/Max Investment Opportunities and Other Foo Biz Baz
------------------------------------------------------

It turns out that the answer from @sigmavirus regarding the links was close. I started a discussion on the distutils mailing list and found out that in-page links (i.e. #minimum-cash) are not allowed by the pypi reStructuredText parser and will invalidate the entire document.

It seems that pypi uses a whitelist to filter link protocols (http vs ftp vs gopher), and sees ‘#’ as an invalid protocol. It seems like this would be pretty easy to fix on their end, but until then, I’ll be removing my in-page anchor links.

Answered By: Jeremy Gillick
  • You may use collective.checkdocs package to detect invalid constructs:

    pip install collective.checkdocs
    python setup.py checkdocs

  • You may then use the following python function to filter-out sphinx-only constructs (it might be necessary to add more regexes, to match your content):

#!/usr/bin/python3
"""
Cleans-up Sphinx-only constructs (ie from README.rst),
so that *PyPi* can format it properly.

To check for remaining errors, install ``sphinx`` and run::

        python setup.py --long-description | sed -file 'this_file.sed' | rst2html.py  --halt=warning

"""

import re
import sys, io


def yield_sphinx_only_markup(lines):
    """
    :param file_inp:     a `filename` or ``sys.stdin``?
    :param file_out:     a `filename` or ``sys.stdout`?`

    """
    substs = [
        ## Selected Sphinx-only Roles.
        #
        (r':abbr:`([^`]+)`',        r'1'),
        (r':ref:`([^`]+)`',         r'`1`_'),
        (r':term:`([^`]+)`',        r'**1**'),
        (r':dfn:`([^`]+)`',         r'**1**'),
        (r':(samp|guilabel|menuselection):`([^`]+)`',        r'``2``'),


        ## Sphinx-only roles:
        #        :foo:`bar`   --> foo(``bar``)
        #        :a:foo:`bar` XXX afoo(``bar``)
        #
        #(r'(:(w+))?:(w+):`([^`]*)`', r'23(``4``)'),
        (r':(w+):`([^`]*)`', r'1(``2``)'),


        ## Sphinx-only Directives.
        #
        (r'.. doctest',           r'code-block'),
        (r'.. plot::',            r'.. '),
        (r'.. seealso',           r'info'),
        (r'.. glossary',          r'rubric'),
        (r'.. figure::',          r'.. '),


        ## Other
        #
        (r'|version|',              r'x.x.x'),
    ]

    regex_subs = [ (re.compile(regex, re.IGNORECASE), sub) for (regex, sub) in substs ]

    def clean_line(line):
        try:
            for (regex, sub) in regex_subs:
                line = regex.sub(sub, line)
        except Exception as ex:
            print("ERROR: %s, (line(%s)"%(regex, sub))
            raise ex

        return line

    for line in lines:
        yield clean_line(line)

and/or in your setup.py file, use something like this::

def read_text_lines(fname):
    with io.open(os.path.join(mydir, fname)) as fd:
        return fd.readlines()

readme_lines = read_text_lines('README.rst')
long_desc = ''.join(yield_sphinx_only_markup(readme_lines)),

Alternatively you can use the sed unix-utility with this file:

## Sed-file to clean-up README.rst from Sphinx-only constructs,
##   so that *PyPi* can format it properly.
##   To check for remaining errors, install ``sphinx`` and run:
##
##          sed -f "this_file.txt" README.rst | rst2html.py  --halt=warning
##

## Selected Sphinx-only Roles.
#
s/:abbr:`([^`]*)`/1/gi
s/:ref:`([^`]*)`/`1`_/gi
s/:term:`([^`]*)`/**1**/gi
s/:dfn:`([^`]*)`/**1**/gi
s/:(samp|guilabel|menuselection):`([^`]*)`/``1``/gi


## Sphinx-only roles:
#        :foo:`bar` --> foo(``bar``)
#
s/:([a-z]*):`([^`]*)`/1(``2``)/gi


## Sphinx-only Directives.
#
s/.. +doctest/code-block/i
s/.. +plot/raw/i
s/.. +seealso/info/i
s/.. +glossary/rubric/i
s/.. +figure::/../i


## Other
#
s/|version|/x.x.x/gi
Answered By: ankostis

EDIT: You can use the following to find errors in your RST that will show up on PyPI:

twine check

You’ll need twine version 1.12.0 or higher. If you don’t have it you can install or update it using:

pip install --upgrade twine

Source


Deprecated answer:

python setup.py check --restructuredtext

Source

Answered By: Serp C

I had the same problem when uploading my python module to pypi .

Later I checked the README.rst for errors using rst-lint which showed that my readme file was right.

I found that the problem was not in the README file but in setup.py itself.

Follow the below points while writing Readme and setup.py

  • DO NOT WRITE MULTI LINE python strings for description or summary or anything that goes into the setup( ) arguments .
  • Don’t use relative links in the README file .(like ./path1/path2 ).
  • Make sure the rst syntax is all right using a checking tool like rst-lint.
  • If you have a markdown file , you can convert it to Restructured text using pandoc easily
Answered By: Natesh bhat
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.