How to create a table/dataframe/list from part of a text file in Python?
Question:
I have a text file with the following structure:
I want to create a table of only the ‘StructName’ and the ‘Volume_mm3’ header in Python, but I am not sure how to do this. The information in the rows with # TableCol, # NRows and # NTableCols are not necessary. Does anyone have any suggestions for me? Thanks in advance!
# TableCol 1 ColHeader Index
# TableCol 1 FieldName Index
# TableCol 1 Units NA
# TableCol 2 ColHeader SegId
# TableCol 2 FieldName Segmentation Id
# TableCol 2 Units NA
# TableCol 3 ColHeader NVoxels
# TableCol 3 FieldName Number of Voxels
# TableCol 3 Units unitless
# TableCol 4 ColHeader Volume_mm3
# TableCol 4 FieldName Volume
# TableCol 4 Units mm^3
# TableCol 5 ColHeader StructName
# TableCol 5 FieldName Structure Name
# TableCol 5 Units NA
# TableCol 6 ColHeader normMean
# TableCol 6 FieldName Intensity normMean
# TableCol 6 Units MR
# TableCol 7 ColHeader normStdDev
# TableCol 7 FieldName Intensity normStdDev
# TableCol 7 Units MR
# TableCol 8 ColHeader normMin
# TableCol 8 FieldName Intensity normMin
# TableCol 8 Units MR
# TableCol 9 ColHeader normMax
# TableCol 9 FieldName Intensity normMax
# TableCol 9 Units MR
# TableCol 10 ColHeader normRange
# TableCol 10 FieldName Intensity normRange
# TableCol 10 Units MR
# NRows 45
# NTableCols 10
# ColHeaders Index SegId NVoxels Volume_mm3 StructName normMean normStdDev normMin normMax normRange
1 4 6852 7260.4 Left-Lateral-Ventricle 36.0019 11.9944 16.0000 85.0000 69.0000
2 5 257 328.8 Left-Inf-Lat-Vent 55.5058 9.8354 22.0000 88.0000 66.0000
3 7 15143 15593.6 Left-Cerebellum-White-Matter 88.5000 6.2780 38.0000 104.0000 66.0000
4 8 61301 60538.4 Left-Cerebellum-Cortex 73.6324 10.6985 6.0000 140.0000 134.0000
5 10 8449 8080.4 Left-Thalamus-Proper 87.9671 8.3555 35.0000 110.0000 75.0000
6 11 3142 3040.2 Left-Caudate 78.3740 7.5265 48.0000 102.0000 54.0000
7 12 5425 5274.3 Left-Putamen 86.4428 5.7711 57.0000 106.0000 49.0000
8 13 2017 2016.4 Left-Pallidum 96.4447 3.6342 75.0000 107.0000 32.0000
9 14 1051 1073.5 3rd-Ventricle 43.6946 14.3018 17.0000 78.0000 61.0000
10 15 1525 1673.0 4th-Ventricle 36.8007 11.3505 18.0000 84.0000 66.0000
11 16 23805 23327.6 Brain-Stem 82.9749 8.3897 30.0000 102.0000 72.0000
12 17 4368 4250.0 Left-Hippocampus 75.4716 7.3720 43.0000 104.0000 61.0000
13 18 1590 1546.3 Left-Amygdala 74.8950 5.9385 44.0000 99.0000 55.0000
14 24 993 975.5 CSF 52.2870 12.2449 26.0000 98.0000 72.0000
15 26 566 585.4 Left-Accumbens-area 73.6484 4.3577 50.0000 92.0000 42.0000
16 28 4988 4787.1 Left-VentralDC 88.7536 9.8029 34.0000 107.0000 73.0000
17 30 24 41.1 Left-vessel 59.5000 6.2554 44.0000 74.0000 30.0000
18 31 657 596.5 Left-choroid-plexus 51.8630 11.0620 28.0000 81.0000 53.0000
19 43 6460 6665.8 Right-Lateral-Ventricle 39.4337 12.4180 15.0000 90.0000 75.0000
20 44 193 239.2 Right-Inf-Lat-Vent 48.9223 11.0175 26.0000 82.0000 56.0000
21 46 15472 15474.6 Right-Cerebellum-White-Matter 84.3709 8.0148 26.0000 102.0000 76.0000
22 47 65751 65152.8 Right-Cerebellum-Cortex 72.8820 11.8809 20.0000 138.0000 118.0000
23 49 8771 8361.8 Right-Thalamus-Proper 87.1512 8.7933 30.0000 105.0000 75.0000
24 50 3152 3009.0 Right-Caudate 80.9137 7.7115 47.0000 102.0000 55.0000
25 51 4853 4894.8 Right-Putamen 84.8354 5.7319 51.0000 103.0000 52.0000
26 52 2153 2042.2 Right-Pallidum 93.9145 4.2542 54.0000 105.0000 51.0000
27 53 4503 4414.1 Right-Hippocampus 75.1397 7.3244 39.0000 102.0000 63.0000
28 54 1519 1498.2 Right-Amygdala 75.4924 5.4002 51.0000 93.0000 42.0000
29 58 599 585.9 Right-Accumbens-area 76.0067 4.3593 63.0000 98.0000 35.0000
30 60 4915 4666.9 Right-VentralDC 88.1034 9.8386 33.0000 104.0000 71.0000
31 62 5 12.4 Right-vessel 53.0000 5.8737 46.0000 59.0000 13.0000
32 63 434 426.3 Right-choroid-plexus 56.0392 9.7654 30.0000 83.0000 53.0000
33 72 0 0.0 5th-Ventricle 0.0000 0.0000 0.0000 0.0000 0.0000
34 77 785 720.6 WM-hypointensities 71.3465 11.1410 34.0000 110.0000 76.0000
35 78 0 0.0 Left-WM-hypointensities 0.0000 0.0000 0.0000 0.0000 0.0000
36 79 0 0.0 Right-WM-hypointensities 0.0000 0.0000 0.0000 0.0000 0.0000
37 80 0 0.0 non-WM-hypointensities 0.0000 0.0000 0.0000 0.0000 0.0000
38 81 0 0.0 Left-non-WM-hypointensities 0.0000 0.0000 0.0000 0.0000 0.0000
39 82 0 0.0 Right-non-WM-hypointensities 0.0000 0.0000 0.0000 0.0000 0.0000
40 85 262 269.8 Optic-Chiasm 77.7366 15.2819 33.0000 100.0000 67.0000
41 251 1127 1033.2 CC_Posterior 96.1003 15.1243 40.0000 110.0000 70.0000
42 252 634 563.5 CC_Mid_Posterior 91.9196 14.4353 35.0000 106.0000 71.0000
43 253 970 904.6 CC_Central 93.2072 12.9240 43.0000 107.0000 64.0000
44 254 917 806.9 CC_Mid_Anterior 91.3719 15.3462 45.0000 109.0000 64.0000
45 255 1207 1138.2 CC_Anterior 99.4341 13.0948 30.0000 117.0000 87.0000
I tried converting the text file to a list, but this converts the whole file and I only need the table on the end of the file.
Answers:
Use pd.read_table function with specific parameters to capture the needed columns into a dataframe:
df = pd.read_table(test_txt, delim_whitespace=True, header=None, comment='#',
usecols=[3, 4], names=['Volume_mm3', 'StructName'])
print(df)
Volume_mm3 StructName
0 7260.4 Left-Lateral-Ventricle
1 328.8 Left-Inf-Lat-Vent
2 15593.6 Left-Cerebellum-White-Matter
3 60538.4 Left-Cerebellum-Cortex
4 8080.4 Left-Thalamus-Proper
5 3040.2 Left-Caudate
6 5274.3 Left-Putamen
7 2016.4 Left-Pallidum
8 1073.5 3rd-Ventricle
9 1673.0 4th-Ventricle
10 23327.6 Brain-Stem
11 4250.0 Left-Hippocampus
12 1546.3 Left-Amygdala
13 975.5 CSF
14 585.4 Left-Accumbens-area
15 4787.1 Left-VentralDC
16 41.1 Left-vessel
17 596.5 Left-choroid-plexus
18 6665.8 Right-Lateral-Ventricle
19 239.2 Right-Inf-Lat-Vent
20 15474.6 Right-Cerebellum-White-Matter
21 65152.8 Right-Cerebellum-Cortex
22 8361.8 Right-Thalamus-Proper
23 3009.0 Right-Caudate
24 4894.8 Right-Putamen
25 2042.2 Right-Pallidum
26 4414.1 Right-Hippocampus
27 1498.2 Right-Amygdala
28 585.9 Right-Accumbens-area
29 4666.9 Right-VentralDC
30 12.4 Right-vessel
31 426.3 Right-choroid-plexus
32 0.0 5th-Ventricle
33 720.6 WM-hypointensities
34 0.0 Left-WM-hypointensities
35 0.0 Right-WM-hypointensities
36 0.0 non-WM-hypointensities
37 0.0 Left-non-WM-hypointensities
38 0.0 Right-non-WM-hypointensities
39 269.8 Optic-Chiasm
40 1033.2 CC_Posterior
41 563.5 CC_Mid_Posterior
42 904.6 CC_Central
43 806.9 CC_Mid_Anterior
44 1138.2 CC_Anterior
I have a text file with the following structure:
I want to create a table of only the ‘StructName’ and the ‘Volume_mm3’ header in Python, but I am not sure how to do this. The information in the rows with # TableCol, # NRows and # NTableCols are not necessary. Does anyone have any suggestions for me? Thanks in advance!
# TableCol 1 ColHeader Index
# TableCol 1 FieldName Index
# TableCol 1 Units NA
# TableCol 2 ColHeader SegId
# TableCol 2 FieldName Segmentation Id
# TableCol 2 Units NA
# TableCol 3 ColHeader NVoxels
# TableCol 3 FieldName Number of Voxels
# TableCol 3 Units unitless
# TableCol 4 ColHeader Volume_mm3
# TableCol 4 FieldName Volume
# TableCol 4 Units mm^3
# TableCol 5 ColHeader StructName
# TableCol 5 FieldName Structure Name
# TableCol 5 Units NA
# TableCol 6 ColHeader normMean
# TableCol 6 FieldName Intensity normMean
# TableCol 6 Units MR
# TableCol 7 ColHeader normStdDev
# TableCol 7 FieldName Intensity normStdDev
# TableCol 7 Units MR
# TableCol 8 ColHeader normMin
# TableCol 8 FieldName Intensity normMin
# TableCol 8 Units MR
# TableCol 9 ColHeader normMax
# TableCol 9 FieldName Intensity normMax
# TableCol 9 Units MR
# TableCol 10 ColHeader normRange
# TableCol 10 FieldName Intensity normRange
# TableCol 10 Units MR
# NRows 45
# NTableCols 10
# ColHeaders Index SegId NVoxels Volume_mm3 StructName normMean normStdDev normMin normMax normRange
1 4 6852 7260.4 Left-Lateral-Ventricle 36.0019 11.9944 16.0000 85.0000 69.0000
2 5 257 328.8 Left-Inf-Lat-Vent 55.5058 9.8354 22.0000 88.0000 66.0000
3 7 15143 15593.6 Left-Cerebellum-White-Matter 88.5000 6.2780 38.0000 104.0000 66.0000
4 8 61301 60538.4 Left-Cerebellum-Cortex 73.6324 10.6985 6.0000 140.0000 134.0000
5 10 8449 8080.4 Left-Thalamus-Proper 87.9671 8.3555 35.0000 110.0000 75.0000
6 11 3142 3040.2 Left-Caudate 78.3740 7.5265 48.0000 102.0000 54.0000
7 12 5425 5274.3 Left-Putamen 86.4428 5.7711 57.0000 106.0000 49.0000
8 13 2017 2016.4 Left-Pallidum 96.4447 3.6342 75.0000 107.0000 32.0000
9 14 1051 1073.5 3rd-Ventricle 43.6946 14.3018 17.0000 78.0000 61.0000
10 15 1525 1673.0 4th-Ventricle 36.8007 11.3505 18.0000 84.0000 66.0000
11 16 23805 23327.6 Brain-Stem 82.9749 8.3897 30.0000 102.0000 72.0000
12 17 4368 4250.0 Left-Hippocampus 75.4716 7.3720 43.0000 104.0000 61.0000
13 18 1590 1546.3 Left-Amygdala 74.8950 5.9385 44.0000 99.0000 55.0000
14 24 993 975.5 CSF 52.2870 12.2449 26.0000 98.0000 72.0000
15 26 566 585.4 Left-Accumbens-area 73.6484 4.3577 50.0000 92.0000 42.0000
16 28 4988 4787.1 Left-VentralDC 88.7536 9.8029 34.0000 107.0000 73.0000
17 30 24 41.1 Left-vessel 59.5000 6.2554 44.0000 74.0000 30.0000
18 31 657 596.5 Left-choroid-plexus 51.8630 11.0620 28.0000 81.0000 53.0000
19 43 6460 6665.8 Right-Lateral-Ventricle 39.4337 12.4180 15.0000 90.0000 75.0000
20 44 193 239.2 Right-Inf-Lat-Vent 48.9223 11.0175 26.0000 82.0000 56.0000
21 46 15472 15474.6 Right-Cerebellum-White-Matter 84.3709 8.0148 26.0000 102.0000 76.0000
22 47 65751 65152.8 Right-Cerebellum-Cortex 72.8820 11.8809 20.0000 138.0000 118.0000
23 49 8771 8361.8 Right-Thalamus-Proper 87.1512 8.7933 30.0000 105.0000 75.0000
24 50 3152 3009.0 Right-Caudate 80.9137 7.7115 47.0000 102.0000 55.0000
25 51 4853 4894.8 Right-Putamen 84.8354 5.7319 51.0000 103.0000 52.0000
26 52 2153 2042.2 Right-Pallidum 93.9145 4.2542 54.0000 105.0000 51.0000
27 53 4503 4414.1 Right-Hippocampus 75.1397 7.3244 39.0000 102.0000 63.0000
28 54 1519 1498.2 Right-Amygdala 75.4924 5.4002 51.0000 93.0000 42.0000
29 58 599 585.9 Right-Accumbens-area 76.0067 4.3593 63.0000 98.0000 35.0000
30 60 4915 4666.9 Right-VentralDC 88.1034 9.8386 33.0000 104.0000 71.0000
31 62 5 12.4 Right-vessel 53.0000 5.8737 46.0000 59.0000 13.0000
32 63 434 426.3 Right-choroid-plexus 56.0392 9.7654 30.0000 83.0000 53.0000
33 72 0 0.0 5th-Ventricle 0.0000 0.0000 0.0000 0.0000 0.0000
34 77 785 720.6 WM-hypointensities 71.3465 11.1410 34.0000 110.0000 76.0000
35 78 0 0.0 Left-WM-hypointensities 0.0000 0.0000 0.0000 0.0000 0.0000
36 79 0 0.0 Right-WM-hypointensities 0.0000 0.0000 0.0000 0.0000 0.0000
37 80 0 0.0 non-WM-hypointensities 0.0000 0.0000 0.0000 0.0000 0.0000
38 81 0 0.0 Left-non-WM-hypointensities 0.0000 0.0000 0.0000 0.0000 0.0000
39 82 0 0.0 Right-non-WM-hypointensities 0.0000 0.0000 0.0000 0.0000 0.0000
40 85 262 269.8 Optic-Chiasm 77.7366 15.2819 33.0000 100.0000 67.0000
41 251 1127 1033.2 CC_Posterior 96.1003 15.1243 40.0000 110.0000 70.0000
42 252 634 563.5 CC_Mid_Posterior 91.9196 14.4353 35.0000 106.0000 71.0000
43 253 970 904.6 CC_Central 93.2072 12.9240 43.0000 107.0000 64.0000
44 254 917 806.9 CC_Mid_Anterior 91.3719 15.3462 45.0000 109.0000 64.0000
45 255 1207 1138.2 CC_Anterior 99.4341 13.0948 30.0000 117.0000 87.0000
I tried converting the text file to a list, but this converts the whole file and I only need the table on the end of the file.
Use pd.read_table function with specific parameters to capture the needed columns into a dataframe:
df = pd.read_table(test_txt, delim_whitespace=True, header=None, comment='#',
usecols=[3, 4], names=['Volume_mm3', 'StructName'])
print(df)
Volume_mm3 StructName
0 7260.4 Left-Lateral-Ventricle
1 328.8 Left-Inf-Lat-Vent
2 15593.6 Left-Cerebellum-White-Matter
3 60538.4 Left-Cerebellum-Cortex
4 8080.4 Left-Thalamus-Proper
5 3040.2 Left-Caudate
6 5274.3 Left-Putamen
7 2016.4 Left-Pallidum
8 1073.5 3rd-Ventricle
9 1673.0 4th-Ventricle
10 23327.6 Brain-Stem
11 4250.0 Left-Hippocampus
12 1546.3 Left-Amygdala
13 975.5 CSF
14 585.4 Left-Accumbens-area
15 4787.1 Left-VentralDC
16 41.1 Left-vessel
17 596.5 Left-choroid-plexus
18 6665.8 Right-Lateral-Ventricle
19 239.2 Right-Inf-Lat-Vent
20 15474.6 Right-Cerebellum-White-Matter
21 65152.8 Right-Cerebellum-Cortex
22 8361.8 Right-Thalamus-Proper
23 3009.0 Right-Caudate
24 4894.8 Right-Putamen
25 2042.2 Right-Pallidum
26 4414.1 Right-Hippocampus
27 1498.2 Right-Amygdala
28 585.9 Right-Accumbens-area
29 4666.9 Right-VentralDC
30 12.4 Right-vessel
31 426.3 Right-choroid-plexus
32 0.0 5th-Ventricle
33 720.6 WM-hypointensities
34 0.0 Left-WM-hypointensities
35 0.0 Right-WM-hypointensities
36 0.0 non-WM-hypointensities
37 0.0 Left-non-WM-hypointensities
38 0.0 Right-non-WM-hypointensities
39 269.8 Optic-Chiasm
40 1033.2 CC_Posterior
41 563.5 CC_Mid_Posterior
42 904.6 CC_Central
43 806.9 CC_Mid_Anterior
44 1138.2 CC_Anterior