Use slice notation with collections.deque

Question

How would you extract items 3..6 efficiently, elegantly and pythonically from the following deque without altering it:

from collections import deque
q = deque('',maxlen=10)
for i in range(10,20):
    q.append(i)

the slice notation doesn’t seem to work with deque…

Asked By: Jonathan Livni

||

Source

Answer 1

output = [q[i] for i in range(3,6+1)]

Answered By: Jonathan Livni

Answer 2

I would prefer this, it’s shorter so easier to read:

output = list(q)[3:6+1]

Answered By: Roshan Mathews

Answer 3

import itertools
output = list(itertools.islice(q, 3, 7))

For example:

>>> import collections, itertools
>>> q = collections.deque(xrange(10, 20))
>>> q
deque([10, 11, 12, 13, 14, 15, 16, 17, 18, 19])
>>> list(itertools.islice(q, 3, 7))
[13, 14, 15, 16]

This should be more efficient the the other solutions posted so far. Proof?

[me@home]$ SETUP="import itertools,collections; q=collections.deque(xrange(1000000))"

[me@home]$ python -m timeit  "$SETUP" "list(itertools.islice(q, 10000, 20000))"
10 loops, best of 3: 68 msec per loop

[me@home]$ python -m timeit "$SETUP" "[q[i] for i in  xrange(10000, 20000)]"
10 loops, best of 3: 98.4 msec per loop

[me@home]$ python -m timeit "$SETUP" "list(q)[10000:20000]"
10 loops, best of 3: 107 msec per loop

Answered By: Shawn Chin

Answer 4

I’d add this as a new answer, to provide better formatting.

For simplicity, Shawn’s answer is perfect, but if you often need to get a slice from dequeue, you might prefer to subclass it and add a __getslice__ method.

from collections import deque
from itertools import islice
class deque_slice(deque):
    def __new__(cls, *args):
        return deque.__new__(cls, *args)
    def __getslice__(self, start, end):
        return list(islice(self, start, end))

This won’t support setting a new slice, but you can implement your own custom __setslice__ method using the same concept.

NOTE: this is valid for Python <=2.* only. It’s also worth noticing that, while __getslice__ is deprecated since python 2.0, the documentation still reports this for the latest 2.7 release:

(However, built-in types in CPython currently still implement __getslice__(). Therefore, you have to override it in derived classes when implementing slicing.)

Answered By: musicamante

Answer 5

you can override the __getitem__ method and create a SliceableDeque using islice.

There are edge cases, you should consider (for example using negative slices doesn’t work with islice).

here is what I have been using:

import itertools
from collections import deque

class SliceableDeque(deque):
    def __getitem__(self, s):
        try:
            start, stop, step = s.start or 0, s.stop or sys.maxsize, s.step or 1
        except AttributeError:  # not a slice but an int
            return super().__getitem__(s)
        try:  # normal slicing
            return list(itertools.islice(self, start, stop, step))
        except ValueError:  # incase of a negative slice object
            length = len(self)
            start, stop = length + start if start < 0 else start, length + stop if stop < 0 else stop
            return list(itertools.islice(self, start, stop, step))

Answered By: moshevi

Answer 6

This is an old question, but for any future travelers, the Python docs explicitly recommend using rotate for this:

The rotate() method provides a way to implement deque slicing and deletion.

https://docs.python.org/2/library/collections.html

An implementation is relatively simple:

def slice_deque(d, start, stop, step):
    d.rotate(-start)
    slice = list(itertools.islice(d, 0, stop-start, step))
    d.rotate(start)
    return slice

Effectively the same as using islice directly, except that rotate is more efficient for skipping through to the starting point. On the other hand, it also temporarily modifies the deque, which could be a threadsafety concern.

Answered By: MattyV

Use slice notation with collections.deque

Question:

Answers: