Adding method to Python's NoneType

Question

I’m using BeautifulSoup to do some crawling, and want to chain find calls, for example:

soup.find('div', class_="class1").find('div', class_="class2").find('div', class_="class3")

Of course, this breaks whenever one of the divs cannot be found, throwing an

AttributeError: 'NoneType' object has no attribute 'find'

Is there a way to modify NoneType to add a find method such as

class NoneType:
    def find(*args):
        return None

so that I can do something like

thing = soup.find('div', class_="class1").find('div', class_="class2").find('div', class_="class3")
if thing:
    do more stuff

instead of

thing1 = soup.find('div', class_="class1")
if thing1:
    thing2 = thing1.find('div', class_="class2")
    if thing2:
        thing3 = thing2.find('div', class_="class3")
        etc.

I think I might be able to do something similar by using a parser with XPath capabilities, but the question is not specific to this use case and is more about modifying/overriding built in classes.

Asked By: colblitz

||

Source

Answer 1

You can’t modify builtin class such as NoneType or str:

>>> nt = type(None)
>>> nt.bla = 23
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't set attributes of built-in/extension type 'NoneType'

For some of them (eg str), you can inherit from:

>>> class bla(str):
...      def toto(self): return 1
>>> bla('2123').toto()
1

It’s not possible with NoneType. And it won’t help you either:

>>> class myNoneType(nt):
...      def find(self): return 1
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Error when calling the metaclass bases
    type 'NoneType' is not an acceptable base type

Answered By: hivert

Answer 2

Why not use a try/except statement instead (since you cannot modify NoneType)?

try:
    thing = soup.find('div', class_="class1").find('div', class_="class2").find('div', class_="class3")
    do more stuff
except AttributeError:
    thing = None  # if you need to do more with thing

Answered By: Justin O Barber

Answer 3

You can’t inherit from None:

>>> class Noneish(type(None)):
...   pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: type 'NoneType' is not an acceptable base type

Answered By: Russia Must Remove Putin

Answer 4

An approach might be to have a

class FindCaller(object):
    def __init__(self, *a, **k):
        self.a = a
        self.k = k
    def __call__(self, obj):
        return obj.find(*self.a, **self.k)

def callchain(root, *fcs):
    for fc in fcs:
        root = fc(root)
        if root is None: return
    return root

and then do

thing = callchain(soup,
    FindCaller('div', class_="class1"),
    FindCaller('div', class_="class2"),
    FindCaller('div', class_="class3"),
)

Answered By: glglgl

Answer 5

You cannot modify the class and the real question is why you would try? NoneType means there is no data there so when you attempt a .find() on that type even if it did exist you would only get null or no values from it. I would reccomend something like this.

try:
    var = soup.find('div', class_="class1").find('div', class_="class2").find('div', class_="class3")
except AttributeError:
    do something else instead or message saying there was no div

Answered By: inuasha

Answer 6

You can’t. For good reasons…
In fact, NoneType is even less accessible than other built-in types:

type(None).foo = lambda x: x
# ---------------------------------------------------------------------------
# TypeError                                 Traceback (most recent call last)
# <ipython-input-12-61bbde54e51b> in <module>()
# ----> 1 type(None).foo = lambda x: x

# TypeError: can't set attributes of built-in/extension type 'NoneType'

NoneType.foo = lambda x: x
# ---------------------------------------------------------------------------
# NameError                                 Traceback (most recent call last)
# <ipython-input-13-22af1ed98023> in <module>()
# ----> 1 NoneType.foo = lambda x: x

# NameError: name 'NoneType' is not defined

int.foo = lambda x: x
# ---------------------------------------------------------------------------
# TypeError                                 Traceback (most recent call last)
# <ipython-input-14-c46c4e33b8cc> in <module>()
# ----> 1 int.foo = lambda x: x

# TypeError: can't set attributes of built-in/extension type 'int'

As suggested above, use try: ... except AttributeError: clause.

Answered By: m.wasowski

Adding method to Python's NoneType

Question:

Answers: