Do Python regular expressions have an equivalent to Ruby's atomic grouping?
Question:
Ruby’s regular expressions have a feature called atomic grouping (?>regexp), described here, is there any equivalent in Python’s re module?
Answers:
from http://docs.python.org/2/library/re.html#regular-expression-syntax
(?P<name>…)
Similar to regular parentheses, but the substring matched by the group is accessible within the rest of the regular expression via the symbolic group name name. Group names must be valid Python identifiers, and each group name must be defined only once within a regular expression. A symbolic group is also a numbered group, just as if the group were not named. So the group named id in the example below can also be referenced as the numbered group 1.
For example, if the pattern is (?P[a-zA-Z_]w*), the group can be referenced by its name in arguments to methods of match objects, such as m.group(‘id’) or m.end(‘id’), and also by name in the regular expression itself (using (?P=id)) and replacement text given to .sub() (using g).
(?P=name)
Matches whatever text was matched by the earlier group named name.
Python does not directly support this feature, but you can emulate it by using a zero-width lookahead assert ((?=RE)), which matches from the current point with the same semantics you want, putting a named group ((?P<name>RE)) inside the lookahead, and then using a named backreference ((?P=name)) to match exactly whatever the zero-width assertion matched. Combined together, this gives you the same semantics, at the cost of creating an additional matching group, and a lot of syntax.
For example, the link you provided gives the Ruby example of
/"(?>.*)"/.match('"Quote"') #=> nil
We can emulate that in Python as such:
re.search(r'"(?=(?P<tmp>.*))(?P=tmp)"', '"Quote"') # => None
We can show that I’m doing something useful and not just spewing line noise, because if we change it so that the inner group doesn’t eat the final ", it still matches:
re.search(r'"(?=(?P<tmp>[A-Za-z]*))(?P=tmp)"', '"Quote"').groupdict()
# => {'tmp': 'Quote'}
You can also use anonymous groups and numeric backreferences, but this gets awfully full of line-noise:
re.search(r'"(?=(.*))1"', '"Quote"') # => None
(Full disclosure: I learned this trick from perl’s perlre documentation, which mentions it under the documentation for (?>...).)
In addition to having the right semantics, this also has the appropriate performance properties. If we port an example out of perlre:
[nelhage@anarchique:~/tmp]$ cat re.py
import re
import timeit
re_1 = re.compile(r'''(
(
[^()]+ # x+
|
( [^()]* )
)+
)
''', re.X)
re_2 = re.compile(r'''(
(
(?=(?P<tmp>[^()]+ ))(?P=tmp) # Emulate (?> x+)
|
( [^()]* )
)+
)''', re.X)
print timeit.timeit("re_1.search('((()' + 'a' * 25)",
setup = "from __main__ import re_1",
number = 10)
print timeit.timeit("re_2.search('((()' + 'a' * 25)",
setup = "from __main__ import re_2",
number = 10)
We see a dramatic improvement:
[nelhage@anarchique:~/tmp]$ python re.py
96.0800571442
7.41481781006e-05
Which only gets more dramatic as we extend the length of the search string.
According to this table, the answer is no. A RFE was created to add it to Python 3, but was declined in favor of the new regex module, which supports it:
>>> import regex
>>> regex.match('"(?>.*)"', '"Quote"')
>>> regex.match('"(.*)"', '"Quote"')
<_regex.Match object at 0x00C6F058>
Note: regex is also available for Python 2.
It would seem not.
http://www.regular-expressions.info/atomic.html
Atomic grouping is supported by most modern regular expression flavors, including the JGsoft flavor, Java, PCRE, .NET, Perl and Ruby.
You can emulate the non-capturing-ness of them by using non-capturing groups, (?:RE), but if I’m reading it right, that still won’t give you the optimization benefits.
As of March 21, 2022, about 10 years after this thread was created, Python has finally added atomic grouping (and possessive matching) in the standard library re module in Python 3.11.0a7: commit link. So now, all the answers to this thread might be rendered obsolete; before, only the third-party regex module has atomic grouping and possessive matching. Now, it is already built-in to Python’s re module.
Ruby’s regular expressions have a feature called atomic grouping (?>regexp), described here, is there any equivalent in Python’s re module?
from http://docs.python.org/2/library/re.html#regular-expression-syntax
(?P<name>…)
Similar to regular parentheses, but the substring matched by the group is accessible within the rest of the regular expression via the symbolic group name name. Group names must be valid Python identifiers, and each group name must be defined only once within a regular expression. A symbolic group is also a numbered group, just as if the group were not named. So the group named id in the example below can also be referenced as the numbered group 1.
For example, if the pattern is (?P[a-zA-Z_]w*), the group can be referenced by its name in arguments to methods of match objects, such as m.group(‘id’) or m.end(‘id’), and also by name in the regular expression itself (using (?P=id)) and replacement text given to .sub() (using g).
(?P=name)
Matches whatever text was matched by the earlier group named name.
Python does not directly support this feature, but you can emulate it by using a zero-width lookahead assert ((?=RE)), which matches from the current point with the same semantics you want, putting a named group ((?P<name>RE)) inside the lookahead, and then using a named backreference ((?P=name)) to match exactly whatever the zero-width assertion matched. Combined together, this gives you the same semantics, at the cost of creating an additional matching group, and a lot of syntax.
For example, the link you provided gives the Ruby example of
/"(?>.*)"/.match('"Quote"') #=> nil
We can emulate that in Python as such:
re.search(r'"(?=(?P<tmp>.*))(?P=tmp)"', '"Quote"') # => None
We can show that I’m doing something useful and not just spewing line noise, because if we change it so that the inner group doesn’t eat the final ", it still matches:
re.search(r'"(?=(?P<tmp>[A-Za-z]*))(?P=tmp)"', '"Quote"').groupdict()
# => {'tmp': 'Quote'}
You can also use anonymous groups and numeric backreferences, but this gets awfully full of line-noise:
re.search(r'"(?=(.*))1"', '"Quote"') # => None
(Full disclosure: I learned this trick from perl’s perlre documentation, which mentions it under the documentation for (?>...).)
In addition to having the right semantics, this also has the appropriate performance properties. If we port an example out of perlre:
[nelhage@anarchique:~/tmp]$ cat re.py
import re
import timeit
re_1 = re.compile(r'''(
(
[^()]+ # x+
|
( [^()]* )
)+
)
''', re.X)
re_2 = re.compile(r'''(
(
(?=(?P<tmp>[^()]+ ))(?P=tmp) # Emulate (?> x+)
|
( [^()]* )
)+
)''', re.X)
print timeit.timeit("re_1.search('((()' + 'a' * 25)",
setup = "from __main__ import re_1",
number = 10)
print timeit.timeit("re_2.search('((()' + 'a' * 25)",
setup = "from __main__ import re_2",
number = 10)
We see a dramatic improvement:
[nelhage@anarchique:~/tmp]$ python re.py
96.0800571442
7.41481781006e-05
Which only gets more dramatic as we extend the length of the search string.
According to this table, the answer is no. A RFE was created to add it to Python 3, but was declined in favor of the new regex module, which supports it:
>>> import regex
>>> regex.match('"(?>.*)"', '"Quote"')
>>> regex.match('"(.*)"', '"Quote"')
<_regex.Match object at 0x00C6F058>
Note: regex is also available for Python 2.
It would seem not.
http://www.regular-expressions.info/atomic.html
Atomic grouping is supported by most modern regular expression flavors, including the JGsoft flavor, Java, PCRE, .NET, Perl and Ruby.
You can emulate the non-capturing-ness of them by using non-capturing groups, (?:RE), but if I’m reading it right, that still won’t give you the optimization benefits.
As of March 21, 2022, about 10 years after this thread was created, Python has finally added atomic grouping (and possessive matching) in the standard library re module in Python 3.11.0a7: commit link. So now, all the answers to this thread might be rendered obsolete; before, only the third-party regex module has atomic grouping and possessive matching. Now, it is already built-in to Python’s re module.