Python regex, remove all punctuation except hyphen for unicode string


I have this code for removing all punctuation from a regex string:

import regex as re    
re.sub(ur"p{P}+", "", txt)

How would I change it to allow hyphens? If you could explain how you did it, that would be great. I understand that here, correct me if I’m wrong, P with anything after it is punctuation.

Asked By: John



You could either specify the punctuation you want to remove manually, as in [._,] or supply a function instead of the replacement string:

re.sub(r"p{P}", lambda m: "-" if == "-" else "", text)
Answered By: Cu3PO42

P is the complementary of p – not punctuation. So this matches anything that is not (not punctuation or a dash) – resulting in all punctuation except dashes.


If you want a non-convoluted way, an alternative is p{P}(?<!-): match all punctuation, and then check it wasn’t a dash (using negative lookbehind).
Working example:

Answered By: Kobi

Here’s how to do it with the re module, in case you have to stick with the standard libraries:

# works in python 2 and 3
import re
import string

remove = string.punctuation
remove = remove.replace("-", "") # don't remove hyphens
pattern = r"[{}]".format(remove) # create the pattern

txt = ")*^%{}[]thi's - is - @@#!a !%%!!%- test."
re.sub(pattern, "", txt) 
# >>> 'this - is - a - test'

If performance matters, you may want to use str.translate, since it’s faster than using a regex. In Python 3, the code is txt.translate({ord(char): None for char in remove}).

Answered By: Galen Long
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.