how to tell a variable is iterable but not a string
Question:
I have a function that take an argument which can be either a single item or a double item:
def iterable(arg)
if #arg is an iterable:
print "yes"
else:
print "no"
so that:
>>> iterable( ("f","f") )
yes
>>> iterable( ["f","f"] )
yes
>>> iterable("ff")
no
The problem is that string is technically iterable, so I can’t just catch the ValueError when trying arg[1]
. I don’t want to use isinstance(), because that’s not good practice (or so I’m told).
Answers:
Use isinstance (I don’t see why it’s bad practice)
import types
if not isinstance(arg, types.StringTypes):
Note the use of StringTypes. It ensures that we don’t forget about some obscure type of string.
On the upside, this also works for derived string classes.
class MyString(str):
pass
isinstance(MyString(" "), types.StringTypes) # true
Also, you might want to have a look at this previous question.
Cheers.
NB: behavior changed in Python 3 as StringTypes
and basestring
are no longer defined. Depending on your needs, you can replace them in isinstance
by str
, or a subset tuple of (str, bytes, unicode)
, e.g. for Cython users.
As @Theron Luhn mentionned, you can also use six
.
As you point out correctly, a single string is a character sequence.
So the thing you really want to do is to find out what kind of sequence arg
is by using isinstance or type(a)==str.
If you want to realize a function that takes a variable amount of parameters, you should do it like this:
def function(*args):
# args is a tuple
for arg in args:
do_something(arg)
function(“ff”) and function(“ff”, “ff”) will work.
I can’t see a scenario where an isiterable() function like yours is needed. It isn’t isinstance() that is bad style but situations where you need to use isinstance().
Since Python 2.6, with the introduction of abstract base classes, isinstance
(used on ABCs, not concrete classes) is now considered perfectly acceptable. Specifically:
from abc import ABCMeta, abstractmethod
class NonStringIterable:
__metaclass__ = ABCMeta
@abstractmethod
def __iter__(self):
while False:
yield None
@classmethod
def __subclasshook__(cls, C):
if cls is NonStringIterable:
if any("__iter__" in B.__dict__ for B in C.__mro__):
return True
return NotImplemented
This is an exact copy (changing only the class name) of Iterable
as defined in _abcoll.py
(an implementation detail of collections.py
)… the reason this works as you wish, while collections.Iterable
doesn’t, is that the latter goes the extra mile to ensure strings are considered iterable, by calling Iterable.register(str)
explicitly just after this class
statement.
Of course it’s easy to augment __subclasshook__
by returning False
before the any
call for other classes you want to specifically exclude from your definition.
In any case, after you have imported this new module as myiter
, isinstance('ciao', myiter.NonStringIterable)
will be False
, and isinstance([1,2,3], myiter.NonStringIterable)
will be True
, just as you request — and in Python 2.6 and later this is considered the proper way to embody such checks… define an abstract base class and check isinstance
on it.
I realise this is an old post but thought it was worth adding my approach for Internet posterity. The function below seems to work for me under most circumstances with both Python 2 and 3:
def is_collection(obj):
""" Returns true for any iterable which is not a string or byte sequence.
"""
try:
if isinstance(obj, unicode):
return False
except NameError:
pass
if isinstance(obj, bytes):
return False
try:
iter(obj)
except TypeError:
return False
try:
hasattr(None, obj)
except TypeError:
return True
return False
This checks for a non-string iterable by (mis)using the built-in hasattr
which will raise a TypeError
when its second argument is not a string or unicode string.
By combining previous replies, I’m using:
import types
import collections
#[...]
if isinstance(var, types.StringTypes )
or not isinstance(var, collections.Iterable):
#[Do stuff...]
Not 100% fools proof, but if an object is not an iterable you still can let it pass and fall back to duck typing.
Edit: Python3
types.StringTypes == (str, unicode)
. The Phython3 equivalent is:
if isinstance(var, str )
or not isinstance(var, collections.Iterable):
Edit: Python3.3
types.StringTypes == (str, unicode)
. The Phython3 equivalent is:
if isinstance(var, str )
or not isinstance(var, collections.abc.Iterable):
2.x
I would have suggested:
hasattr(x, '__iter__')
or in view of David Charles’ comment tweaking this for Python3, what about:
hasattr(x, '__iter__') and not isinstance(x, (str, bytes))
3.x
the builtin basestring
abstract type was removed. Use str
instead. The str
and bytes
types don’t have functionality enough in common to warrant a shared base class.
As of 2017, here is a portable solution that works with all versions of Python:
#!/usr/bin/env python
import collections
import six
def iterable(arg):
return (
isinstance(arg, collections.Iterable)
and not isinstance(arg, six.string_types)
)
# non-string iterables
assert iterable(("f", "f")) # tuple
assert iterable(["f", "f"]) # list
assert iterable(iter("ff")) # iterator
assert iterable(range(44)) # generator
assert iterable(b"ff") # bytes (Python 2 calls this a string)
# strings or non-iterables
assert not iterable(u"ff") # string
assert not iterable(44) # integer
assert not iterable(iterable) # function
To explicitly expand on Alex Martelli’s excellent hack of collections.py
and address some of the questions around it: The current working solution in python 3.6+ is
import collections
import _collections_abc as cabc
import abc
class NonStringIterable(metaclass=abc.ABCMeta):
__slots__ = ()
@abc.abstractmethod
def __iter__(self):
while False:
yield None
@classmethod
def __subclasshook__(cls, c):
if cls is NonStringIterable:
if issubclass(c, str):
return False
return cabc._check_methods(c, "__iter__")
return NotImplemented
and demonstrated
>>> typs = ['string', iter(''), list(), dict(), tuple(), set()]
>>> [isinstance(o, NonStringIterable) for o in typs]
[False, True, True, True, True, True]
If you want to add iter('')
into the exclusions, for example, modify the line
if issubclass(c, str):
return False
to be
# `str_iterator` is just a shortcut for `type(iter(''))`*
if issubclass(c, (str, cabc.str_iterator)):
return False
to get
[False, False, True, True, True, True]
Adding another answer here that doesn’t require extra imports and is maybe more "pythonic", relying on duck typing and the fact that str
has had a unicode casefold
method since Python 3.
def iterable_not_string(x):
'''
Check if input has an __iter__ method and then determine if it's a
string by checking for a casefold method.
'''
try:
assert x.__iter__
try:
assert x.casefold
# could do the following instead for python 2.7 because
# str and unicode types both had a splitlines method
# assert x.splitlines
return False
except AttributeError:
return True
except AttributeError:
return False
Python 3.X
Notes:
- You need implement "isListable" method.
In my case dict is not iterable because iter(obj_dict) returns an iterator of just the keys.
- Sequences are iterables, but not all iterables are sequences (immutable, mutable).
- set, dict are iterables but not sequence.
- list is iterable and sequence.
- str is an iterable and immutable sequence.
Sources:
- https://docs.python.org/3/library/stdtypes.html
- https://opensource.com/article/18/3/loop-better-deeper-look-iteration-python
See this example:
from typing import Iterable, Sequence, MutableSequence, Mapping, Text
class Custom():
pass
def isListable(obj):
if(isinstance(obj, type)): return isListable(obj.__new__(obj))
return isinstance(obj, MutableSequence)
try:
# Listable
#o = [Custom()]
#o = ["a","b"]
#o = [{"a":"va"},{"b":"vb"}]
#o = list # class type
# Not listable
#o = {"a" : "Value"}
o = "Only string"
#o = 1
#o = False
#o = 2.4
#o = None
#o = Custom()
#o = {1, 2, 3} #type set
#o = (n**2 for n in {1, 2, 3})
#o = bytes("Only string", 'utf-8')
#o = Custom # class type
if isListable(o):
print("Is Listable[%s]: %s" % (o.__class__, str(o)))
else:
print("Not Listable[%s]: %s" % (o.__class__, str(o)))
except Exception as exc:
raise exc
If you like to test if the variable is a iterable object and not a "string like" object (str, bytes, …) you can use the fact that the __mod__()
function exists in such "string like" objects for formatting proposes. So you can do a check like this:
>>> def is_not_iterable(item):
... return hasattr(item, '__trunc__') or hasattr(item, '__mod__')
>>> is_not_iterable('')
True
>>> is_not_iterable(b'')
True
>>> is_not_iterable(())
False
>>> is_not_iterable([])
False
>>> is_not_iterable(1)
True
>>> is_not_iterable({})
False
>>> is_not_iterable(set())
False
>>> is_not_iterable(range(19)) #considers also Generators or Iterators
False
I have a function that take an argument which can be either a single item or a double item:
def iterable(arg)
if #arg is an iterable:
print "yes"
else:
print "no"
so that:
>>> iterable( ("f","f") ) yes >>> iterable( ["f","f"] ) yes >>> iterable("ff") no
The problem is that string is technically iterable, so I can’t just catch the ValueError when trying arg[1]
. I don’t want to use isinstance(), because that’s not good practice (or so I’m told).
Use isinstance (I don’t see why it’s bad practice)
import types
if not isinstance(arg, types.StringTypes):
Note the use of StringTypes. It ensures that we don’t forget about some obscure type of string.
On the upside, this also works for derived string classes.
class MyString(str):
pass
isinstance(MyString(" "), types.StringTypes) # true
Also, you might want to have a look at this previous question.
Cheers.
NB: behavior changed in Python 3 as StringTypes
and basestring
are no longer defined. Depending on your needs, you can replace them in isinstance
by str
, or a subset tuple of (str, bytes, unicode)
, e.g. for Cython users.
As @Theron Luhn mentionned, you can also use six
.
As you point out correctly, a single string is a character sequence.
So the thing you really want to do is to find out what kind of sequence arg
is by using isinstance or type(a)==str.
If you want to realize a function that takes a variable amount of parameters, you should do it like this:
def function(*args):
# args is a tuple
for arg in args:
do_something(arg)
function(“ff”) and function(“ff”, “ff”) will work.
I can’t see a scenario where an isiterable() function like yours is needed. It isn’t isinstance() that is bad style but situations where you need to use isinstance().
Since Python 2.6, with the introduction of abstract base classes, isinstance
(used on ABCs, not concrete classes) is now considered perfectly acceptable. Specifically:
from abc import ABCMeta, abstractmethod
class NonStringIterable:
__metaclass__ = ABCMeta
@abstractmethod
def __iter__(self):
while False:
yield None
@classmethod
def __subclasshook__(cls, C):
if cls is NonStringIterable:
if any("__iter__" in B.__dict__ for B in C.__mro__):
return True
return NotImplemented
This is an exact copy (changing only the class name) of Iterable
as defined in _abcoll.py
(an implementation detail of collections.py
)… the reason this works as you wish, while collections.Iterable
doesn’t, is that the latter goes the extra mile to ensure strings are considered iterable, by calling Iterable.register(str)
explicitly just after this class
statement.
Of course it’s easy to augment __subclasshook__
by returning False
before the any
call for other classes you want to specifically exclude from your definition.
In any case, after you have imported this new module as myiter
, isinstance('ciao', myiter.NonStringIterable)
will be False
, and isinstance([1,2,3], myiter.NonStringIterable)
will be True
, just as you request — and in Python 2.6 and later this is considered the proper way to embody such checks… define an abstract base class and check isinstance
on it.
I realise this is an old post but thought it was worth adding my approach for Internet posterity. The function below seems to work for me under most circumstances with both Python 2 and 3:
def is_collection(obj):
""" Returns true for any iterable which is not a string or byte sequence.
"""
try:
if isinstance(obj, unicode):
return False
except NameError:
pass
if isinstance(obj, bytes):
return False
try:
iter(obj)
except TypeError:
return False
try:
hasattr(None, obj)
except TypeError:
return True
return False
This checks for a non-string iterable by (mis)using the built-in hasattr
which will raise a TypeError
when its second argument is not a string or unicode string.
By combining previous replies, I’m using:
import types
import collections
#[...]
if isinstance(var, types.StringTypes )
or not isinstance(var, collections.Iterable):
#[Do stuff...]
Not 100% fools proof, but if an object is not an iterable you still can let it pass and fall back to duck typing.
Edit: Python3
types.StringTypes == (str, unicode)
. The Phython3 equivalent is:
if isinstance(var, str )
or not isinstance(var, collections.Iterable):
Edit: Python3.3
types.StringTypes == (str, unicode)
. The Phython3 equivalent is:
if isinstance(var, str )
or not isinstance(var, collections.abc.Iterable):
2.x
I would have suggested:
hasattr(x, '__iter__')
or in view of David Charles’ comment tweaking this for Python3, what about:
hasattr(x, '__iter__') and not isinstance(x, (str, bytes))
3.x
the builtin basestring
abstract type was removed. Use str
instead. The str
and bytes
types don’t have functionality enough in common to warrant a shared base class.
As of 2017, here is a portable solution that works with all versions of Python:
#!/usr/bin/env python
import collections
import six
def iterable(arg):
return (
isinstance(arg, collections.Iterable)
and not isinstance(arg, six.string_types)
)
# non-string iterables
assert iterable(("f", "f")) # tuple
assert iterable(["f", "f"]) # list
assert iterable(iter("ff")) # iterator
assert iterable(range(44)) # generator
assert iterable(b"ff") # bytes (Python 2 calls this a string)
# strings or non-iterables
assert not iterable(u"ff") # string
assert not iterable(44) # integer
assert not iterable(iterable) # function
To explicitly expand on Alex Martelli’s excellent hack of collections.py
and address some of the questions around it: The current working solution in python 3.6+ is
import collections
import _collections_abc as cabc
import abc
class NonStringIterable(metaclass=abc.ABCMeta):
__slots__ = ()
@abc.abstractmethod
def __iter__(self):
while False:
yield None
@classmethod
def __subclasshook__(cls, c):
if cls is NonStringIterable:
if issubclass(c, str):
return False
return cabc._check_methods(c, "__iter__")
return NotImplemented
and demonstrated
>>> typs = ['string', iter(''), list(), dict(), tuple(), set()]
>>> [isinstance(o, NonStringIterable) for o in typs]
[False, True, True, True, True, True]
If you want to add iter('')
into the exclusions, for example, modify the line
if issubclass(c, str):
return False
to be
# `str_iterator` is just a shortcut for `type(iter(''))`*
if issubclass(c, (str, cabc.str_iterator)):
return False
to get
[False, False, True, True, True, True]
Adding another answer here that doesn’t require extra imports and is maybe more "pythonic", relying on duck typing and the fact that str
has had a unicode casefold
method since Python 3.
def iterable_not_string(x):
'''
Check if input has an __iter__ method and then determine if it's a
string by checking for a casefold method.
'''
try:
assert x.__iter__
try:
assert x.casefold
# could do the following instead for python 2.7 because
# str and unicode types both had a splitlines method
# assert x.splitlines
return False
except AttributeError:
return True
except AttributeError:
return False
Python 3.X
Notes:
- You need implement "isListable" method.
In my case dict is not iterable because iter(obj_dict) returns an iterator of just the keys.- Sequences are iterables, but not all iterables are sequences (immutable, mutable).
- set, dict are iterables but not sequence.
- list is iterable and sequence.
- str is an iterable and immutable sequence.
Sources:
- https://docs.python.org/3/library/stdtypes.html
- https://opensource.com/article/18/3/loop-better-deeper-look-iteration-python
See this example:
from typing import Iterable, Sequence, MutableSequence, Mapping, Text
class Custom():
pass
def isListable(obj):
if(isinstance(obj, type)): return isListable(obj.__new__(obj))
return isinstance(obj, MutableSequence)
try:
# Listable
#o = [Custom()]
#o = ["a","b"]
#o = [{"a":"va"},{"b":"vb"}]
#o = list # class type
# Not listable
#o = {"a" : "Value"}
o = "Only string"
#o = 1
#o = False
#o = 2.4
#o = None
#o = Custom()
#o = {1, 2, 3} #type set
#o = (n**2 for n in {1, 2, 3})
#o = bytes("Only string", 'utf-8')
#o = Custom # class type
if isListable(o):
print("Is Listable[%s]: %s" % (o.__class__, str(o)))
else:
print("Not Listable[%s]: %s" % (o.__class__, str(o)))
except Exception as exc:
raise exc
If you like to test if the variable is a iterable object and not a "string like" object (str, bytes, …) you can use the fact that the __mod__()
function exists in such "string like" objects for formatting proposes. So you can do a check like this:
>>> def is_not_iterable(item):
... return hasattr(item, '__trunc__') or hasattr(item, '__mod__')
>>> is_not_iterable('')
True
>>> is_not_iterable(b'')
True
>>> is_not_iterable(())
False
>>> is_not_iterable([])
False
>>> is_not_iterable(1)
True
>>> is_not_iterable({})
False
>>> is_not_iterable(set())
False
>>> is_not_iterable(range(19)) #considers also Generators or Iterators
False