What is __peg_parser__ in Python?
Question:
I was using the keyword built-in module to get a list of all the keywords of the current Python version. And this is what I did:
>>> import keyword
>>> print(keyword.kwlist)
['False', 'None', 'True', '__peg_parser__', 'and', 'as', 'assert', 'async', 'await', 'break', 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield']
And in the keyword.kwlist list there is __peg_parser__. So to see what it does, I type __peg_parser__ in a Python 3.9 interpreter on Windows (you’ll get the same output on Mac OS and Linux as well), and this is what is get:
>>> __peg_parser__
File "<stdin>", line 1
__peg_parser__
^
SyntaxError: You found it!
So my question is, what is __peg_parser__ and why do I get SyntaxError: You found it!?
Answers:
It was an easter egg related to the rollout of the new PEG parser. The easter egg, along with the old LL(1) parser, will be removed in 3.10.
What is __peg_parser__?
It is an Easter Egg in Python (of the Peg Parser) for when the new Peg Parser was released. As mentioned in this discussion, it will be removed in Python 3.10.
Before the Easter Egg was called __new_parser__, but was changed to __peg_parser__, to make it future proof as mentioned in the message:
new, ex or ng are not really future proof names. Can we rename the keyword to __peg_parser__?
Why do you get SyntaxError: You found it!?
You get SyntaxError: You found it! because it is part of the Easter Egg.
Will it be removed in the future?
Since the LL(1) parser will be replaced be the new Peg Parser, it will be removed in Python 3.10.
__peg_parser__ in Earlier and Later Versions of Python
It didn’t exist in earlier versions of Python.
Python 3.8 and earlier:
>>> __peg_parser__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name '__peg_parser__' is not defined
Python 3.9:
>>> __peg_parser__
File "<stdin>", line 1
__peg_parser__
^
SyntaxError: You found it!
Python 3.10:
>>> __peg_parser__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name '__peg_parser__' is not defined
Guido published on github here for the new PEG parser.
It’s also on Python PEP.
As it mentions:
This PEP proposes replacing the current LL(1)-based parser of CPython with a new PEG-based parser. This new parser would allow the elimination of multiple "hacks" that exist in the current grammar to circumvent the LL(1)-limitation. It would substantially reduce the maintenance costs in some areas related to the compiling pipeline such as the grammar, the parser and the AST generation. The new PEG parser will also lift the LL(1) restriction on the current Python grammar.
Also mentioned in Python 3.9 What’s new page.
In Python 3.10, the LL(1) parser will be removed. Python 3.9 uses a new parser, based on PEG instead of LL(1).
In Python 3.6, it is not defined:
>>> __peg_parser__
Traceback (most recent call last):
File "<pyshell#13>", line 1, in <module>
__peg_parser__
NameError: name '__peg_parser__' is not defined
>>>
This PEP proposes replacing the current LL(1)-based parser of CPython with a new PEG-based parser. This new parser would allow the elimination of multiple "hacks" that exist in the current grammar to circumvent the LL(1)-limitation. It would substantially reduce the maintenance costs in some areas related to the compiling pipeline such as the grammar, the parser and the AST generation. The new PEG parser will also lift the LL(1) restriction on the current Python grammar.
I was using the keyword built-in module to get a list of all the keywords of the current Python version. And this is what I did:
>>> import keyword
>>> print(keyword.kwlist)
['False', 'None', 'True', '__peg_parser__', 'and', 'as', 'assert', 'async', 'await', 'break', 'class', 'continue', 'def', 'del', 'elif', 'else', 'except', 'finally', 'for', 'from', 'global', 'if', 'import', 'in', 'is', 'lambda', 'nonlocal', 'not', 'or', 'pass', 'raise', 'return', 'try', 'while', 'with', 'yield']
And in the keyword.kwlist list there is __peg_parser__. So to see what it does, I type __peg_parser__ in a Python 3.9 interpreter on Windows (you’ll get the same output on Mac OS and Linux as well), and this is what is get:
>>> __peg_parser__
File "<stdin>", line 1
__peg_parser__
^
SyntaxError: You found it!
So my question is, what is __peg_parser__ and why do I get SyntaxError: You found it!?
It was an easter egg related to the rollout of the new PEG parser. The easter egg, along with the old LL(1) parser, will be removed in 3.10.
What is __peg_parser__?
It is an Easter Egg in Python (of the Peg Parser) for when the new Peg Parser was released. As mentioned in this discussion, it will be removed in Python 3.10.
Before the Easter Egg was called __new_parser__, but was changed to __peg_parser__, to make it future proof as mentioned in the message:
new,exorngare not really future proof names. Can we rename the keyword to__peg_parser__?
Why do you get SyntaxError: You found it!?
You get SyntaxError: You found it! because it is part of the Easter Egg.
Will it be removed in the future?
Since the LL(1) parser will be replaced be the new Peg Parser, it will be removed in Python 3.10.
__peg_parser__ in Earlier and Later Versions of Python
It didn’t exist in earlier versions of Python.
Python 3.8 and earlier:
>>> __peg_parser__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name '__peg_parser__' is not defined
Python 3.9:
>>> __peg_parser__
File "<stdin>", line 1
__peg_parser__
^
SyntaxError: You found it!
Python 3.10:
>>> __peg_parser__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name '__peg_parser__' is not defined
Guido published on github here for the new PEG parser.
It’s also on Python PEP.
As it mentions:
This PEP proposes replacing the current LL(1)-based parser of CPython with a new PEG-based parser. This new parser would allow the elimination of multiple "hacks" that exist in the current grammar to circumvent the LL(1)-limitation. It would substantially reduce the maintenance costs in some areas related to the compiling pipeline such as the grammar, the parser and the AST generation. The new PEG parser will also lift the LL(1) restriction on the current Python grammar.
Also mentioned in Python 3.9 What’s new page.
In Python 3.10, the LL(1) parser will be removed. Python 3.9 uses a new parser, based on PEG instead of LL(1).
In Python 3.6, it is not defined:
>>> __peg_parser__
Traceback (most recent call last):
File "<pyshell#13>", line 1, in <module>
__peg_parser__
NameError: name '__peg_parser__' is not defined
>>>
This PEP proposes replacing the current LL(1)-based parser of CPython with a new PEG-based parser. This new parser would allow the elimination of multiple "hacks" that exist in the current grammar to circumvent the LL(1)-limitation. It would substantially reduce the maintenance costs in some areas related to the compiling pipeline such as the grammar, the parser and the AST generation. The new PEG parser will also lift the LL(1) restriction on the current Python grammar.