How to get the items in Queue without removing the items in Python?
Question:
get() removes and returns an item from Queue in Python.
import queue
q = queue.Queue() # Here
q.put("Apple")
q.put("Orange")
q.put("Banana")
print(q.get())
print(q.get())
print(q.get())
Output:
Apple
Orange
Banana
Now, I want to get the items in Queue without removing the items.
Is it possible to do this?
Answers:
queue_object.queue will return copy of your queue in a deque object which you can then use the slices of. It is of course, not syncronized with the original queue, but will allow you to peek at the queue at the time of the copy.
There’s a good rationalization for why you wouldn’t want to do this explained in detail in this thread comp.lang.python – Queue peek?. But if you’re just trying to understand how Queue works, this is one simple way.
import Queue
q = Queue.Queue()
q.push(1)
q.put('foo')
q.put('bar')
d = q.queue
print(d)
deque(['foo', 'bar'])
print(d[0])
'foo'
The Queue
module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads.
As you can see, the Queue
module was created specifically for use with threads, providing only FIFO, LIFO and priority queues, none of which provide this functionality. However by examining the source code of the Queue
module, you can see that it simply uses a collections.deque
(double ended queue) which can easily accomplish your task. You may index the first item ([0]
) and .popleft()
in constant time.
It is NOT safe to simply access the underlying queue.
The safe way to do it is to extend the Queue class. If you return the underlying dequeue object, you will NOT be getting a copy, you get the live object.
The result of this is that it can change while you are iterating it – which will result in an exception if another thread inserts into the queue during your iteration.
Knowing that python uses the GIL, you can safely use list(q.queue)
, because list() will never cause a context switch.
It’s better to use the same lock the get() function uses, and not make assumptions about the GIL:
import queue
class SnapshotQueue(queue.Queue):
def snapshot(self):
with self.mutex:
return list(self.queue)
That class can be used safely instead of a regular queue, and it will return a snapshot of the queue state… within a mutex and without causing issues in the underlying queue operation.
I found this question, because I needed a way to access top element in a PriorityQueue.
I couldn’t find a way to do that, so I switched to heapq instead.
Although it’s worth mentioning that heapq is not thread safe.
You can get the items in a queue without removing the items as shown below:
import queue
q = queue.Queue()
q.put("Apple")
q.put("Orange")
q.put("Banana")
print(q.queue[0]) # Here
print(q.queue[1]) # Here
print(q.queue[2]) # Here
print(q.queue) # Here
Output:
Apple
Orange
Banana
deque(['Apple', 'Orange', 'Banana'])
And, you can also change the items in a queue as shown below:
import queue
q = queue.Queue()
q.put("Apple")
q.put("Orange")
q.put("Banana")
q.queue[0] = "Strawberry" # Here
q.queue[1] = "Lemon" # Here
q.queue[2] = "kiwi" # Here
print(q.queue[0])
print(q.queue[1])
print(q.queue[2])
print(q.queue)
Output:
Strawberry
Lemon
kiwi
deque(['Strawberry', 'Lemon', 'kiwi'])
But, you cannot add items to a queue without put() as shown below:
import queue
q = queue.Queue()
q.queue[0] = "Apple" # Cannot add
q.queue[1] = "Orange" # Cannot add
q.queue[2] = "Banana" # Cannot add
print(q.queue[0])
print(q.queue[1])
print(q.queue[2])
print(q.queue)
Then, the error below occurs:
IndexError: deque index out of range
get() removes and returns an item from Queue in Python.
import queue
q = queue.Queue() # Here
q.put("Apple")
q.put("Orange")
q.put("Banana")
print(q.get())
print(q.get())
print(q.get())
Output:
Apple
Orange
Banana
Now, I want to get the items in Queue without removing the items.
Is it possible to do this?
queue_object.queue will return copy of your queue in a deque object which you can then use the slices of. It is of course, not syncronized with the original queue, but will allow you to peek at the queue at the time of the copy.
There’s a good rationalization for why you wouldn’t want to do this explained in detail in this thread comp.lang.python – Queue peek?. But if you’re just trying to understand how Queue works, this is one simple way.
import Queue
q = Queue.Queue()
q.push(1)
q.put('foo')
q.put('bar')
d = q.queue
print(d)
deque(['foo', 'bar'])
print(d[0])
'foo'
The
Queue
module implements multi-producer, multi-consumer queues. It is especially useful in threaded programming when information must be exchanged safely between multiple threads.
As you can see, the Queue
module was created specifically for use with threads, providing only FIFO, LIFO and priority queues, none of which provide this functionality. However by examining the source code of the Queue
module, you can see that it simply uses a collections.deque
(double ended queue) which can easily accomplish your task. You may index the first item ([0]
) and .popleft()
in constant time.
It is NOT safe to simply access the underlying queue.
The safe way to do it is to extend the Queue class. If you return the underlying dequeue object, you will NOT be getting a copy, you get the live object.
The result of this is that it can change while you are iterating it – which will result in an exception if another thread inserts into the queue during your iteration.
Knowing that python uses the GIL, you can safely use list(q.queue)
, because list() will never cause a context switch.
It’s better to use the same lock the get() function uses, and not make assumptions about the GIL:
import queue
class SnapshotQueue(queue.Queue):
def snapshot(self):
with self.mutex:
return list(self.queue)
That class can be used safely instead of a regular queue, and it will return a snapshot of the queue state… within a mutex and without causing issues in the underlying queue operation.
I found this question, because I needed a way to access top element in a PriorityQueue.
I couldn’t find a way to do that, so I switched to heapq instead.
Although it’s worth mentioning that heapq is not thread safe.
You can get the items in a queue without removing the items as shown below:
import queue
q = queue.Queue()
q.put("Apple")
q.put("Orange")
q.put("Banana")
print(q.queue[0]) # Here
print(q.queue[1]) # Here
print(q.queue[2]) # Here
print(q.queue) # Here
Output:
Apple
Orange
Banana
deque(['Apple', 'Orange', 'Banana'])
And, you can also change the items in a queue as shown below:
import queue
q = queue.Queue()
q.put("Apple")
q.put("Orange")
q.put("Banana")
q.queue[0] = "Strawberry" # Here
q.queue[1] = "Lemon" # Here
q.queue[2] = "kiwi" # Here
print(q.queue[0])
print(q.queue[1])
print(q.queue[2])
print(q.queue)
Output:
Strawberry
Lemon
kiwi
deque(['Strawberry', 'Lemon', 'kiwi'])
But, you cannot add items to a queue without put() as shown below:
import queue
q = queue.Queue()
q.queue[0] = "Apple" # Cannot add
q.queue[1] = "Orange" # Cannot add
q.queue[2] = "Banana" # Cannot add
print(q.queue[0])
print(q.queue[1])
print(q.queue[2])
print(q.queue)
Then, the error below occurs:
IndexError: deque index out of range