chunks

Python Multiprocessing read and write to large file

Python Multiprocessing read and write to large file Question: I have a large file of 120GB consisting of strings line by line. I would like to loop the file line by line replacing all the German characters ß with characters s. I have a working code, but it is very slow, and in the future, …

Total answers: 1

using a python generator to process large text files

using a python generator to process large text files Question: I’m new to using generators and have read around a bit but need some help processing large text files in chunks. I know this topic has been covered but example code has very limited explanations making it difficult to modify the code if one doesn’t …

Total answers: 4

Splitting a list into N parts of approximately equal length

Splitting a list into N parts of approximately equal length Question: What is the best way to divide a list into roughly equal parts? For example, if the list has 7 elements and is split it into 2 parts, we want to get 3 elements in one part, and the other should have 4 elements. …

Total answers: 36

How to iterate over a list in chunks

How to iterate over a list in chunks Question: I have a Python script which takes as input a list of integers, which I need to work with four integers at a time. Unfortunately, I don’t have control of the input, or I’d have it passed in as a list of four-element tuples. Currently, I’m …

Total answers: 39

How do I split a list into equally-sized chunks?

How do I split a list into equally-sized chunks? Question: How do I split a list of arbitrary length into equal sized chunks? See also: How to iterate over a list in chunks. To chunk strings, see Split string every nth character?. Asked By: jespern || Source Answers: Here’s a generator that yields evenly-sized chunks: …

Total answers: 70