Problem with using readtable:
3 views (last 30 days)
Ilter Onat Korkmaz on 18 Apr 2021
Here is the code:
opts = detectImportOptions("C:\Users\Onat\Desktop\392\vocab.txt");
opts.LineEnding = ["\n"];
vocab = readtable('C:\Users\Onat\Desktop\392\vocab.txt',opts);
I'am working on an NLP application in which I need the vocabulary and frequency of those words in vocabulary. Naturally, the corpus contains tokens such as single apostrophe. It seems that this is a major problem for MATLAB since it detects it as a special char. Notice that in the output given below, after apostrophe the frequencies are seen as comments to MATLAB. Can anyone help with this issue?
vocab.txt is as follows:
..... (i.e. this is not the beginning)
...... (i.e continues)
the output is as follows:
...... (i.e. this is not the beginning)
" 538122↵to 504540↵a 476836↵was 304423↵The 246510↵- 229901↵is 225721↵for 198733↵)
Aghamarsh Varanasi on 21 Apr 2021
The quotes in a text file are used to import strings that contain any charater (even with delimiters). That is why you are seeing this behavior. You can specify the 'Format' of the file you are importing by using the 'Format' Name-Value pair input to the function readtable.
vocab = readtable('data.txt', 'Format' ,'%s %f');
This would import the data as expected.
For more info on how strings are imported from text documents, refer to this Community post.