How to generate small lists from continous data list based on a condition such that new lists replicate continuity Peak-Trough-Peak pattern in them

Question

my_list = [{'Filename': '20211004T105041236.jpg',
  'Speed': '0.1',
  'Time': '2021-10-04T10:50:41.236Z'},
 {'Filename': '20211004T105044302.jpg',
  'Speed': '0.1',
  'Time': '2021-10-04T10:50:44.302Z'},
 {'Filename': '20211004T105047374.jpg',
  'Speed': '1.0',
  'Time': '2021-10-04T10:50:47.374Z'},
 {'Filename': '20211004T105050435.jpg',
  'Speed': '2.1',
  'Time': '2021-10-04T10:50:50.435Z'},
 {'Filename': '20211004T105050436.jpg',
  'Speed': '3.1',
  'Time': '2021-10-04T10:50:50.436Z'},
 {'Filename': '20211004T105053500.jpg',
  'Speed': '0.9',
  'Time': '2021-10-04T10:50:53.5Z'},
 {'Filename': '20211004T105056554.jpg',
  'Speed': '1.1',
  'Time': '2021-10-04T10:50:56.554Z'},
 {'Filename': '20211004T105059621.jpg',
  'Speed': '3.8',
  'Time': '2021-10-04T10:50:59.621Z'},
 {'Filename': '20211004T105059621.jpg',
  'Speed': '6.1',
  'Time': '2021-10-04T10:50:59.621Z'},
 {'Filename': '20211004T105102680.jpg',
  'Speed': '8.6',
  'Time': '2021-10-04T10:51:02.68Z'},
 {'Filename': '20211004T105105744.jpg',
  'Speed': '9.4',
  'Time': '2021-10-04T10:51:05.744Z'},
 {'Filename': '20211004T105108776.jpg',
  'Speed': '10.8',
  'Time': '2021-10-04T10:51:08.776Z'},
 {'Filename': '20211004T105108777.jpg',
  'Speed': '8.8',
  'Time': '2021-10-04T10:51:08.777Z'},
 {'Filename': '20211004T105111814.jpg',
  'Speed': '3.3',
  'Time': '2021-10-04T10:51:11.814Z'},
 {'Filename': '20211004T105114858.jpg',
  'Speed': '1.6',
  'Time': '2021-10-04T10:51:14.858Z'},
 {'Filename': '20211004T105117926.jpg',
  'Speed': '0.5',
  'Time': '2021-10-04T10:51:17.926Z'},
 {'Filename': '20211004T105117927.jpg',
  'Speed': '0.0',
  'Time': '2021-10-04T10:51:17.927Z'},
 {'Filename': '20211004T105120993.jpg',
  'Speed': '1.5',
  'Time': '2021-10-04T10:51:20.993Z'},
 {'Filename': '20211004T105124062.jpg',
  'Speed': '2.8',
  'Time': '2021-10-04T10:51:24.062Z'},
 {'Filename': '20211004T105127125.jpg',
  'Speed': '6.7',
  'Time': '2021-10-04T10:51:27.125Z'},
 {'Filename': '20211004T105127125.jpg',
  'Speed': '8.7',
  'Time': '2021-10-04T10:51:27.125Z'},
 {'Filename': '20211004T105130187.jpg',
  'Speed': '9.8',
  'Time': '2021-10-04T10:51:30.187Z'},
 {'Filename': '20211004T105133242.jpg',
  'Speed': '10.0',
  'Time': '2021-10-04T10:51:33.242Z'},
 {'Filename': '20211004T105136300.jpg',
  'Speed': '9.2',
  'Time': '2021-10-04T10:51:36.3Z'},
 {'Filename': '20211004T105136301.jpg',
  'Speed': '8.2',
  'Time': '2021-10-04T10:51:36.301Z'},
 {'Filename': '20211004T105139358.jpg',
  'Speed': '6.1',
  'Time': '2021-10-04T10:51:39.358Z'},
 {'Filename': '20211004T105142436.jpg',
  'Speed': '5.3',
  'Time': '2021-10-04T10:51:42.436Z'},
 {'Filename': '20211004T105145507.jpg',
  'Speed': '5.0',
  'Time': '2021-10-04T10:51:45.507Z'},
 {'Filename': '20211004T105145508.jpg',
  'Speed': '5.0',
  'Time': '2021-10-04T10:51:45.508Z'},
 {'Filename': '20211004T105148566.jpg',
  'Speed': '5.8',
  'Time': '2021-10-04T10:51:48.566Z'},
 {'Filename': '20211004T105151634.jpg',
  'Speed': '3.9',
  'Time': '2021-10-04T10:51:51.634Z'},
 {'Filename': '20211004T105154683.jpg',
  'Speed': '0.1',
  'Time': '2021-10-04T10:51:54.683Z'},
 {'Filename': '20211004T105154685.jpg',
  'Speed': '0.1',
  'Time': '2021-10-04T10:51:54.685Z'},
 {'Filename': '20211004T105157748.jpg',
  'Speed': '0.3',
  'Time': '2021-10-04T10:51:57.748Z'},
 {'Filename': '20211004T105200779.jpg',
  'Speed': '0.1',
  'Time': '2021-10-04T10:52:00.779Z'},
 {'Filename': '20211004T105203826.jpg',
  'Speed': '0.1',
  'Time': '2021-10-04T10:52:03.826Z'},
 {'Filename': '20211004T105203828.jpg',
  'Speed': '0.1',
  'Time': '2021-10-04T10:52:03.828Z'},
 {'Filename': '20211004T105206900.jpg',
  'Speed': '0.1',
  'Time': '2021-10-04T10:52:06.9Z'},
 {'Filename': '20211004T105209974.jpg',
  'Speed': '0.1',
  'Time': '2021-10-04T10:52:09.974Z'},
 {'Filename': '20211004T105213021.jpg',
  'Speed': '0.2',
  'Time': '2021-10-04T10:52:13.021Z'},
 {'Filename': '20211004T105213023.jpg',
  'Speed': '0.2',
  'Time': '2021-10-04T10:52:13.023Z'},
 {'Filename': '20211004T105216095.jpg',
  'Speed': '0.4',
  'Time': '2021-10-04T10:52:16.095Z'},
 {'Filename': '20211004T105219157.jpg',
  'Speed': '1.1',
  'Time': '2021-10-04T10:52:19.157Z'},
 {'Filename': '20211004T105222222.jpg',
  'Speed': '3.0',
  'Time': '2021-10-04T10:52:22.222Z'},
 {'Filename': '20211004T105222223.jpg',
  'Speed': '5.0',
  'Time': '2021-10-04T10:52:22.223Z'},
 {'Filename': '20211004T105225287.jpg',
  'Speed': '6.0',
  'Time': '2021-10-04T10:52:25.287Z'},
 {'Filename': '20211004T105228367.jpg',
  'Speed': '8.1',
  'Time': '2021-10-04T10:52:28.367Z'},
 {'Filename': '20211004T105231447.jpg',
  'Speed': '9.1',
  'Time': '2021-10-04T10:52:31.447Z'},
 {'Filename': '20211004T105231448.jpg',
  'Speed': '10.1',
  'Time': '2021-10-04T10:52:31.448Z'},
 {'Filename': '20211004T105234498.jpg',
  'Speed': '9.0',
  'Time': '2021-10-04T10:52:34.498Z'},
 {'Filename': '20211004T105237565.jpg',
  'Speed': '6.1',
  'Time': '2021-10-04T10:52:37.565Z'},
 {'Filename': '20211004T105240640.jpg',
  'Speed': '3.5',
  'Time': '2021-10-04T10:52:40.64Z'},
 {'Filename': '20211004T105240642.jpg',
  'Speed': '0.8',
  'Time': '2021-10-04T10:52:40.642Z'},
 {'Filename': '20211004T105243719.jpg',
  'Speed': '0.0',
  'Time': '2021-10-04T10:52:43.719Z'},
 {'Filename': '20211004T105246761.jpg',
  'Speed': '1.2',
  'Time': '2021-10-04T10:52:46.761Z'},
 {'Filename': '20211004T105249794.jpg',
  'Speed': '4.8',
  'Time': '2021-10-04T10:52:49.794Z'},
 {'Filename': '20211004T105249795.jpg',
  'Speed': '7.2',
  'Time': '2021-10-04T10:52:49.795Z'},
 {'Filename': '20211004T105252847.jpg',
  'Speed': '10.7',
  'Time': '2021-10-04T10:52:52.847Z'}]

I want to get separate lists for ‘Speed’ data from the above list.
New lists should look like list_1=[0.1,….9.4] ,list_2=[8.8,….9.8],list_3=[9.2,….9.1],list_4=[9.1,….7.2].

Cut off Speed is ‘10.0’
Each new list should replicate continuity of data separated using the ‘Speed’ when reaches to ‘10.0’.
Therefore, it should not pick any random values from the list and store in any lists.

To be more specific it should represent a Peak-Trough-Peak values pattern.

Asked By: Prathap David Carlo

||

Source

Answer 1

This is not a technically complicated problem. You just start gathering records into a sublist, and every time you hit 10.0, you start a new sublist.

from pprint import pprint

my_list = [...]

sublists = [[]]
for row in my_list:
    if float(row['Speed']) >= 10.0:
        if sublists[-1]:
            sublists.append( [] )
    else:
        sublists[-1].append( row )
if not sublists[-1]:
    sublists.pop()
pprint(sublists)

Output:

[[{'Filename': '20211004T105041236.jpg',
   'Speed': '0.1',
   'Time': '2021-10-04T10:50:41.236Z'},
  {'Filename': '20211004T105044302.jpg',
   'Speed': '0.1',
   'Time': '2021-10-04T10:50:44.302Z'},
  {'Filename': '20211004T105047374.jpg',
   'Speed': '1.0',
   'Time': '2021-10-04T10:50:47.374Z'},
  {'Filename': '20211004T105050435.jpg',
   'Speed': '2.1',
   'Time': '2021-10-04T10:50:50.435Z'},
  {'Filename': '20211004T105050436.jpg',
   'Speed': '3.1',
   'Time': '2021-10-04T10:50:50.436Z'},
  {'Filename': '20211004T105053500.jpg',
   'Speed': '0.9',
   'Time': '2021-10-04T10:50:53.5Z'},
  {'Filename': '20211004T105056554.jpg',
   'Speed': '1.1',
   'Time': '2021-10-04T10:50:56.554Z'},
  {'Filename': '20211004T105059621.jpg',
   'Speed': '3.8',
   'Time': '2021-10-04T10:50:59.621Z'},
  {'Filename': '20211004T105059621.jpg',
   'Speed': '6.1',
   'Time': '2021-10-04T10:50:59.621Z'},
  {'Filename': '20211004T105102680.jpg',
   'Speed': '8.6',
   'Time': '2021-10-04T10:51:02.68Z'},
  {'Filename': '20211004T105105744.jpg',
   'Speed': '9.4',
   'Time': '2021-10-04T10:51:05.744Z'}],
 [{'Filename': '20211004T105108777.jpg',
   'Speed': '8.8',
   'Time': '2021-10-04T10:51:08.777Z'},
  {'Filename': '20211004T105111814.jpg',
   'Speed': '3.3',
   'Time': '2021-10-04T10:51:11.814Z'},
  {'Filename': '20211004T105114858.jpg',
   'Speed': '1.6',
   'Time': '2021-10-04T10:51:14.858Z'},
  {'Filename': '20211004T105117926.jpg',
   'Speed': '0.5',
   'Time': '2021-10-04T10:51:17.926Z'},
  {'Filename': '20211004T105117927.jpg',
   'Speed': '0.0',
   'Time': '2021-10-04T10:51:17.927Z'},
  {'Filename': '20211004T105120993.jpg',
   'Speed': '1.5',
   'Time': '2021-10-04T10:51:20.993Z'},
  {'Filename': '20211004T105124062.jpg',
   'Speed': '2.8',
   'Time': '2021-10-04T10:51:24.062Z'},
  {'Filename': '20211004T105127125.jpg',
   'Speed': '6.7',
   'Time': '2021-10-04T10:51:27.125Z'},
  {'Filename': '20211004T105127125.jpg',
   'Speed': '8.7',
   'Time': '2021-10-04T10:51:27.125Z'},
  {'Filename': '20211004T105130187.jpg',
   'Speed': '9.8',
   'Time': '2021-10-04T10:51:30.187Z'}],
 [{'Filename': '20211004T105136300.jpg',
   'Speed': '9.2',
   'Time': '2021-10-04T10:51:36.3Z'},
  {'Filename': '20211004T105136301.jpg',
   'Speed': '8.2',
   'Time': '2021-10-04T10:51:36.301Z'},
  {'Filename': '20211004T105139358.jpg',
   'Speed': '6.1',
   'Time': '2021-10-04T10:51:39.358Z'},
  {'Filename': '20211004T105142436.jpg',
   'Speed': '5.3',
   'Time': '2021-10-04T10:51:42.436Z'},
  {'Filename': '20211004T105145507.jpg',
   'Speed': '5.0',
   'Time': '2021-10-04T10:51:45.507Z'},
  {'Filename': '20211004T105145508.jpg',
   'Speed': '5.0',
   'Time': '2021-10-04T10:51:45.508Z'},
  {'Filename': '20211004T105148566.jpg',
   'Speed': '5.8',
   'Time': '2021-10-04T10:51:48.566Z'},
  {'Filename': '20211004T105151634.jpg',
   'Speed': '3.9',
   'Time': '2021-10-04T10:51:51.634Z'},
  {'Filename': '20211004T105154683.jpg',
   'Speed': '0.1',
   'Time': '2021-10-04T10:51:54.683Z'},
  {'Filename': '20211004T105154685.jpg',
   'Speed': '0.1',
   'Time': '2021-10-04T10:51:54.685Z'},
  {'Filename': '20211004T105157748.jpg',
   'Speed': '0.3',
   'Time': '2021-10-04T10:51:57.748Z'},
  {'Filename': '20211004T105200779.jpg',
   'Speed': '0.1',
   'Time': '2021-10-04T10:52:00.779Z'},
  {'Filename': '20211004T105203826.jpg',
   'Speed': '0.1',
   'Time': '2021-10-04T10:52:03.826Z'},
  {'Filename': '20211004T105203828.jpg',
   'Speed': '0.1',
   'Time': '2021-10-04T10:52:03.828Z'},
  {'Filename': '20211004T105206900.jpg',
   'Speed': '0.1',
   'Time': '2021-10-04T10:52:06.9Z'},
  {'Filename': '20211004T105209974.jpg',
   'Speed': '0.1',
   'Time': '2021-10-04T10:52:09.974Z'},
  {'Filename': '20211004T105213021.jpg',
   'Speed': '0.2',
   'Time': '2021-10-04T10:52:13.021Z'},
  {'Filename': '20211004T105213023.jpg',
   'Speed': '0.2',
   'Time': '2021-10-04T10:52:13.023Z'},
  {'Filename': '20211004T105216095.jpg',
   'Speed': '0.4',
   'Time': '2021-10-04T10:52:16.095Z'},
  {'Filename': '20211004T105219157.jpg',
   'Speed': '1.1',
   'Time': '2021-10-04T10:52:19.157Z'},
  {'Filename': '20211004T105222222.jpg',
   'Speed': '3.0',
   'Time': '2021-10-04T10:52:22.222Z'},
  {'Filename': '20211004T105222223.jpg',
   'Speed': '5.0',
   'Time': '2021-10-04T10:52:22.223Z'},
  {'Filename': '20211004T105225287.jpg',
   'Speed': '6.0',
   'Time': '2021-10-04T10:52:25.287Z'},
  {'Filename': '20211004T105228367.jpg',
   'Speed': '8.1',
   'Time': '2021-10-04T10:52:28.367Z'},
  {'Filename': '20211004T105231447.jpg',
   'Speed': '9.1',
   'Time': '2021-10-04T10:52:31.447Z'}],
 [{'Filename': '20211004T105234498.jpg',
   'Speed': '9.0',
   'Time': '2021-10-04T10:52:34.498Z'},
  {'Filename': '20211004T105237565.jpg',
   'Speed': '6.1',
   'Time': '2021-10-04T10:52:37.565Z'},
  {'Filename': '20211004T105240640.jpg',
   'Speed': '3.5',
   'Time': '2021-10-04T10:52:40.64Z'},
  {'Filename': '20211004T105240642.jpg',
   'Speed': '0.8',
   'Time': '2021-10-04T10:52:40.642Z'},
  {'Filename': '20211004T105243719.jpg',
   'Speed': '0.0',
   'Time': '2021-10-04T10:52:43.719Z'},
  {'Filename': '20211004T105246761.jpg',
   'Speed': '1.2',
   'Time': '2021-10-04T10:52:46.761Z'},
  {'Filename': '20211004T105249794.jpg',
   'Speed': '4.8',
   'Time': '2021-10-04T10:52:49.794Z'},
  {'Filename': '20211004T105249795.jpg',
   'Speed': '7.2',
   'Time': '2021-10-04T10:52:49.795Z'}]]

Answered By: Tim Roberts

Answer 2

As I understand you want the values between peaks, with peak values excluded:

peaks_idx = [my_list.index(elem) for elem in my_list if float(elem["Speed"]) >= 10.]
in_between_values = [my_list[:peaks_idx[0]]]
in_between_values += [my_list[peaks_idx[i]:peaks_idx[i+1]][1:] for i in range(len(peaks_idx) - 1)]

Answered By: Jonathan Ciapetti

How to generate small lists from continous data list based on a condition such that new lists replicate continuity Peak-Trough-Peak pattern in them

Question:

Answers: