Readtable gives unusual results reading data from a text file.

I'm trying to read a lot of text files using readtable. All the files have 7 columns of data, although the last column is usually blank.

Readtable normally gives this:

39×7 table

Var1 Var2 Var3 Var4 Var5 Var6 Var7

____ ____ ____ ____ ____ ____ __________

2009 10 6 3 33 29 {0×0 char}

2009 10 6 3 36 12.3 {0×0 char}

2009 10 6 3 36 53.3 {0×0 char}

2009 10 6 3 37 6.5 {0×0 char}

...

But one text file gives a very different result:

7×1 table

x20111118015544_7RF

_________________________

{'2011 11 18 01 33 43.7'}

{'2011 11 18 01 33 51.2'}

{'2011 11 18 01 33 60.0'}

{'2011 11 18 01 35 32.3'}

{'2011 11 18 01 36 03.9'}

{'2011 11 18 01 36 25.2'}

{'2011 11 18 01 36 33.4'}

This file actually has 26 lines, but readtable only returns the last seven lines and somehow takes the table column name from the 19th line. This line has a value that should go into Var7, but so do lines in other files and they don't cause this problem.

I suspect the text file has a bad character in it, but I can't find it.

Any suggestions?

[Geoff Hayes]

@dormant - can you attach the data file?

[Jan]

If you post the file, we can check search for the problem also.

[Stephen23]

"Any suggestions?"

The files are different. Click the paperclip button to upload some sample files for us to look at.

[dormant]

oops, sorry.

Attached are two files.

20080505-0553.txt is read succesfully into seven columns.

20111118-0021.txt is the naughty file.

[dormant]

I found some more examples and have spotted a pattern.

If the text file has some lines with values in the seventh column, it will read it succesfully if the last line is one of those with a value, as in the file 20160419-2010.txt.

If the last line doesn't have a value in the seventh column, but other lines do, it is read unsuccessfully.

Answer 1

[Stephen23]

T = readtable('20160419-2010.txt','NumHeaderLines',0)

T = 7×7 table
    Var1    Var2    Var3    Var4    Var5    Var6       Var7   
    ____    ____    ____    ____    ____    ____    __________

    2016     4       19      20      10     51.3    {'RF?'   }
    2016     4       19      20      11      7.6    {0×0 char}
    2016     4       19      20      11     30.4    {0×0 char}
    2016     4       19      20      11     59.1    {0×0 char}
    2016     4       19      20      13     29.4    {0×0 char}
    2016     4       19      20      19     17.4    {0×0 char}
    2016     4       19      20      20       17    {'RF?'   }

T = readtable('20111118-0021.txt','NumHeaderLines',0)

T = 26×7 table
    Var1    Var2    Var3    Var4    Var5    Var6       Var7   
    ____    ____    ____    ____    ____    ____    __________

    2011     11      18      0       21     59.2    {0×0 char}
    2011     11      18      0       29      6.9    {0×0 char}
    2011     11      18      0       31      6.8    {0×0 char}
    2011     11      18      0       31     14.7    {0×0 char}
    2011     11      18      0       32     12.7    {0×0 char}
    2011     11      18      0       33      1.4    {0×0 char}
    2011     11      18      0       34     32.4    {0×0 char}
    2011     11      18      0       34     38.1    {0×0 char}
    2011     11      18      0       34     43.5    {0×0 char}
    2011     11      18      0       35     42.1    {0×0 char}
    2011     11      18      0       37     29.1    {0×0 char}
    2011     11      18      0       41     39.1    {0×0 char}
    2011     11      18      1       33     43.7    {0×0 char}
    2011     11      18      1       33     51.2    {0×0 char}
    2011     11      18      1       33       60    {0×0 char}
    2011     11      18      1       35     32.3    {0×0 char}

T = readtable('20080505-0553.txt','NumHeaderLines',0)

T = 31×7 table
    Var1    Var2    Var3    Var4    Var5    Var6       Var7   
    ____    ____    ____    ____    ____    ____    __________

    2008     5       5       5       53     50.2    {0×0 char}
    2008     5       5       5       53     59.4    {0×0 char}
    2008     5       5       5       54     28.7    {0×0 char}
    2008     5       5       5       54     41.9    {0×0 char}
    2008     5       5       5       55     18.4    {0×0 char}
    2008     5       5       5       55     24.2    {0×0 char}
    2008     5       5       5       55     32.6    {0×0 char}
    2008     5       5       5       55     55.2    {'RF?'   }
    2008     5       5       5       56     16.9    {0×0 char}
    2008     5       5       5       56       28    {0×0 char}
    2008     5       5       5       56     42.8    {0×0 char}
    2008     5       5       5       56     57.3    {0×0 char}
    2008     5       5       5       56     59.7    {0×0 char}
    2008     5       5       5       57      9.7    {0×0 char}
    2008     5       5       5       57     27.2    {0×0 char}
    2008     5       5       5       57     32.1    {0×0 char}

[dormant]

Fantastic, thank you.

I also managed to fix it by setting the format.

T = readtable( fileEventList, 'Format','%f %f %f %f %f %f %s' );