When using textscan how do I process repeating data?

I have a file that looks like below. It has repeating blocks with a header line followed by a non-constant number of data lines, and then a line containing the characters "&&" denoting the end of each block. I want textscan to know to stop reading at the end of a data line and "reset" when it reaches the line with the "&&".

I've tried different iterations of this code below and it doesn't work. It reads past the && and right into the next section (i.e. it matches the next "K04V" and then stops, I believe because the first %s in the formatSpec = '%s %f %f %f %f' line allows it to match the K04V

while ~feof(fileID)
N = 7
C_header = textscan(fileID,'%s',N,'Delimiter',',')
formatSpec = '%s %f %f %f %f';
C_data1 = textscan(fileID,formatSpec,'Delimiter',',','CollectOutput',1,'EmptyValue', NaN)
end

Data file snippet...

K04V, T_2M_AGL, FH000, model value, bias,  weight, contribution
gfs[0] 20180530 12Z, 290.8087, -0.4281, 0.0336, 9.7993
gfs[1] 20180530 06Z, 289.2202, -0.4313, 0.0000, 0.0000
gefs[0] 20180530 12Z, 292.7851, -0.4179, 0.0000, 0.0000
gefs[1] 20180530 06Z, 293.1193, -0.2692, 0.0000, 0.0000
ecmwf[0] 20180530 12Z, 295.8952, -1.1245, 0.2291, 68.0363
ecmwf[1] 20180530 00Z, 295.2336, -0.9495, 0.0568, 16.8291
ecmwf_ensmean[0] 20180530 12Z, 294.0368, -1.0981, 0.0018, 0.5348
ecmwf_ensmean[1] 20180530 00Z, 293.5686, -0.9417, 0.0000, 0.0000
nam[0] 20180530 18Z, 292.0365, -0.8670, 0.1626, 47.6361
nam[1] 20180530 12Z, 286.7009, -0.8776, 0.0971, 27.9369
mav_mos[0] 20180530 12Z, 294.0023, 0.0071, 0.1256, 36.9327
mav_mos[1] 20180530 06Z, 292.5631, 0.0170, 0.0196, 5.7442
eta_mos[0] 20180530 12Z, 283.1483, 0.2278, 0.0826, 23.3628
eta_mos[1] 20180530 00Z, 289.8123, 0.6199, 0.0733, 21.1937
gridded_mos[0] 20180530 12Z, 292.9989, 0.1147, 0.0527, 15.4304
gridded_mos[1] 20180530 00Z, 292.5909, 0.0562, 0.0000, 0.0000
rpm_conus[0] 20180530 18Z, 288.5560, -1.6245, 0.0651, 18.8827
rpm_conus[1] 20180530 15Z, 292.1360, -1.6092, 0.0000, 0.0000
rpm_global[0] 20180530 12Z, 291.1134, -1.2081, 0.0000, 0.0000
rpm_global[1] 20180530 06Z, 291.1059, -1.3080, 0.0000, 0.0000
integration, 292.2013, , 1.0000, 292.3190
&&
K04V, T_2M_AGL, FH001, model value, bias,  weight, contribution
gfs[0] 20180530 12Z, 291.6395, -0.5546, 0.0003, 0.0851
gfs[1] 20180530 06Z, 289.5999, -0.6698, 0.0000, 0.0000
gefs[0] 20180530 12Z, 292.3162, -0.5535, 0.0000, 0.0000
gefs[1] 20180530 06Z, 292.5030, -0.4473, 0.0000, 0.0000
ecmwf[0] 20180530 12Z, 294.1544, -1.2235, 0.1723, 50.8918
ecmwf[1] 20180530 00Z, 294.0405, -1.1101, 0.0185, 5.4496
ecmwf_ensmean[0] 20180530 12Z, 292.6239, -1.4880, 0.0092, 2.7174
ecmwf_ensmean[1] 20180530 00Z, 292.3820, -1.3889, 0.0000, 0.0000
nam[0] 20180530 18Z, 289.7563, -1.0246, 0.1322, 38.4438
nam[1] 20180530 12Z, 290.5151, -0.9916, 0.1032, 30.0920
mav_mos[0] 20180530 12Z, 293.3039, -0.0930, 0.0937, 27.4987
mav_mos[1] 20180530 06Z, 292.1351, -0.0376, 0.0142, 4.1633
eta_mos[0] 20180530 12Z, 289.9943, 0.1156, 0.0439, 12.7170
eta_mos[1] 20180530 00Z, 291.7776, 0.6356, 0.0928, 27.0132
gridded_mos[0] 20180530 12Z, 292.3479, 0.0019, 0.1417, 41.4112
gridded_mos[1] 20180530 00Z, 291.9238, 0.0423, 0.0356, 10.4014
rpm_conus[0] 20180530 18Z, 290.7409, -1.4553, 0.0932, 27.2218
rpm_conus[1] 20180530 15Z, 290.4744, -1.3876, 0.0492, 14.3585
rpm_global[0] 20180530 12Z, 290.2267, -0.9666, 0.0000, 0.0000
rpm_global[1] 20180530 06Z, 290.5519, -1.0132, 0.0000, 0.0000
integration, 292.3386, , 1.0000, 292.4647
&&

Answer 1

[Shrestha Kumar]

Hi,

As you are scanning the file with only one delimiter ',' the textscan look for that delimiter only to stop reading.

So to solve this problem just provide one more delimiter '\n' in both the textscan function call.

For example -

C_header = textscan(fileID,'%s',N,'Delimiter',',\n')

C_data1 = textscan(fileID,formatSpec,'Delimiter',',\n','CollectOutput',1,'EmptyValue', NaN)

[Joseph Koval]

Thank you Shrestha! Your suggestion solved my issue.