Valid characters in a python class name

The thing that makes this interesting is that the first character of an identifier is special. After the first character, numbers ‘0’ through ‘9’ are valid for identifiers, but they must not be the first character.

Here’s a function that will return a valid identifier given any random string of characters. Here’s how it works:

First, we use itr = iter(seq) to get an explicit iterator on the input. Then there is a first loop, which uses the iterator itr to look at characters until it finds a valid first character for an identifier. Then it breaks out of that loop and runs the second loop, using the same iterator (which we named itr) for the second loop. The iterator itr keeps our place for us; the characters the first loop pulled out of the iterator are still gone when the second loop runs.

def gen_valid_identifier(seq):
    # get an iterator
    itr = iter(seq)
    # pull characters until we get a legal one for first in identifer
    for ch in itr:
        if ch == '_' or ch.isalpha():
            yield ch
            break
    # pull remaining characters and yield legal ones for identifier
    for ch in itr:
        if ch == '_' or ch.isalpha() or ch.isdigit():
            yield ch

def sanitize_identifier(name):
    return ''.join(gen_valid_identifier(name))

This is a clean and Pythonic way to handle a sequence two different ways. For a problem this simple, we could just have a Boolean variable that indicates whether we have seen the first character yet or not:

def gen_valid_identifier(seq):
    saw_first_char = False
    for ch in seq:
        if not saw_first_char and (ch == '_' or ch.isalpha()):
            saw_first_char = True 
            yield ch
        elif saw_first_char and (ch == '_' or ch.isalpha() or ch.isdigit()):
            yield ch

I don’t like this version nearly as much as the first version. The special handling for one character is now tangled up in the whole flow of control, and this will be slower than the first version as it has to keep checking the value of saw_first_char constantly. But this is the way you would have to handle the flow of control in most languages! Python’s explicit iterator is a nifty feature, and I think it makes this code a lot better.

Looping on an explicit iterator is just as fast as letting Python implicitly get an iterator for you, and the explicit iterator lets us split up the loops that handle the different rules for different parts of the identifier. So the explicit iterator gives us cleaner code that also runs faster. Win/win.

As per Python Language Reference, §2.3, “Identifiers and keywords”, a valid Python identifier is defined as:

(letter|"_") (letter | digit | "_")*

Or, in regex:

[a-zA-Z_][a-zA-Z0-9_]*

This is an old question by now, but I’d like to add an answer on how to do this in Python 3 as I’ve made an implementation.

The allowed characters are documented here: https://docs.python.org/3/reference/lexical_analysis.html#identifiers . They include quite a lot of special characters, including punctuation, underscore, and a whole slew of foreign characters. Luckily the unicodedata module can help. Here’s my implementation implementing directly what the Python documentation says:

import unicodedata

def is_valid_name(name):
    if not _is_id_start(name[0]):
        return False
    for character in name[1:]:
        if not _is_id_continue(character):
            return False
    return True #All characters are allowed.

_allowed_id_continue_categories = {"Ll", "Lm", "Lo", "Lt", "Lu", "Mc", "Mn", "Nd", "Nl", "Pc"}
_allowed_id_continue_characters = {"_", "u00B7", "u0387", "u1369", "u136A", "u136B", "u136C", "u136D", "u136E", "u136F", "u1370", "u1371", "u19DA", "u2118", "u212E", "u309B", "u309C"}
_allowed_id_start_categories = {"Ll", "Lm", "Lo", "Lt", "Lu", "Nl"}
_allowed_id_start_characters = {"_", "u2118", "u212E", "u309B", "u309C"}

def _is_id_start(character):
    return unicodedata.category(character) in _allowed_id_start_categories or character in _allowed_id_start_categories or unicodedata.category(unicodedata.normalize("NFKC", character)) in _allowed_id_start_categories or unicodedata.normalize("NFKC", character) in _allowed_id_start_characters

def _is_id_continue(character):
    return unicodedata.category(character) in _allowed_id_continue_categories or character in _allowed_id_continue_characters or unicodedata.category(unicodedata.normalize("NFKC", character)) in _allowed_id_continue_categories or unicodedata.normalize("NFKC", character) in _allowed_id_continue_characters

This code is adapted from here under CC0: https://github.com/Ghostkeeper/Luna/blob/d69624cd0dd5648aec2139054fae4d45b634da7e/plugins/data/enumerated/enumerated_type.py#L91 . It has been well tested.