Easier/Faster way to convert Hex numbers to binary?

I have an input file that has multiple columns of data, I am only interested in two of them. Right now I have both columns importing into MatLab and the first one, timestamps, is fine and I don't have any problems with it. The next one however, is a HEX number in the format 0x ####, I don't like how long it takes to run.

Currently I'm importing my dataArray,

filename = 'D:\MatLab Files\Import Files\InputFile.txt';
delimiter = ',';
startRow = 2;
formatSpec = '%*s%s%*s%s%s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%*s%[^\n\r]';
fileID = fopen(filename,'r');
dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'HeaderLines' ,startRow-1, 'ReturnOnError', false);
fclose(fileID);
dataArray{1} = datenum(dataArray{1}, 'HH:MM:SS.FFF');
Timestamp = dataArray{:, 1};

then having to run:

MsgInfo = dec2bin(hex2dec(cellfun(@(x)(x(4:end)),dataArray{:, 2}, 'UniformOutput', false)),16);

This gets me my variable into a 16 character binary number so that I can then get the bits I need out of the data. I have a total of seven bits that get set depending on if there are any errors detected in the data stream. This is how I'm currently doing this:

% using the sum function to total the number of times the bit is set to 1, giving me the total number of errors for each error type
ManchesterErrors = sum(bind2dec(MSGInfo(:,16))); 
ParityErrors = sum(bind2dec(MSGInfo(:,15)));
OverRunErrors = sum(bind2dec(MSGInfo(:,14)));
TimeOutErrors = sum(bind2dec(MSGInfo(:,13)));
BroadcastMsg = sum(bind2dec(MSGInfo(:,11)));
ModeCodeNoData = sum(bind2dec(MSGInfo(:,10)));
TotalMessageErrors = sum(bind2dec(MSGInfo(:,9)));

However, this adds a lot of time (30+ minutes) to my data analysis. I'm hoping to speed this up. I found the following on the forums,

typecast(uint16(sscanf('MYHEXSTRINGHERE', '%x')), 'int16')

and tried to do this:

MSGInfoHexNumbers = cellfun(@(x)(x(4:end)),dataArray{:, 2},'UniformOutput',false); %outputs a Nx1 cell array (iscellstr = 1)
MSGInfoBinaryNumbes = typecast(uint16(sscanf(MSGInfoHexNumbers,'%x')),'int16');

but I can't get it to work for my data. I keep getting the error "First argument must be a string".

I'm using MatLab R2012b if this helps when providing answers, and I've attached a sample file (Test File.txt) of my input data, and a copy of my code to date (Hex 2 binary help.txt).

Answer 1

[Walter Roberson]

MSGInfoBinaryNumbes = cellfun(@(S) typecast(uint16(sscanf(S(4:end),'%x')),'int16'), dataArray(:,2) );

[dpb]

Ah, see you also had similar idea, Walter. I tried to get an fscanf format written to parse directly but never managed to get it to work correctly (oh, what I'd give for FORMAT strings).

I don't for the life of me understand why TMW doesn't implement '%X' formatting in textscan and friends; I forget whether I've actually submitted it as enhancement request or whether it's also one of the many they've ignored over untold years. :(

It's always seemed TMW thinks dealing with machine words and the like is beneath the dignity of Matlab users or we're supposed to be above such menial fray in the RAD world, I don't know, but it surely can be frustrating.

[joshmartinmont]

Walter, I modified it slightly because as it was written I was getting the error "Error using sscanf, first argument must be a string". Here is what I did:

MSGInfoBinaryNumbes = cellfun(@(S) typecast(uint16(sscanf(S(4:end),'%x')),'int16'), dataArray{:, 2},'UniformOutput',false );

It works wonderfuly to return the HEX number as decimal, but I need it as binary. I may have typed it wrong earlier. I can still perform:

dec2bin(MSGInfoBinaryNumbes)

but that still runs very slowly. Also, any ideas on how to speed up the part where I look at individual bits?

sum(bin2dec('16characterbinarymessage'(:,N)) %where N is the bit I need to look at.

[Walter Roberson]

bitget()

[joshmartinmont]

I timed out

cellfun(@(S) typecast(uint16(sscanf(S(4:end),'%x')),'int16'), dataArray{:, 2},'UniformOutput',false );

versus

hex2dec(cellfun(@(x)(x(4:end)),dataArray{:, 2},'UniformOutput',false))

using sscanf it took ~ 1:50 and using hex2dec it took ~ 0:50 to get the same answer of Hex to Decimal. So I guess that's not where I'm hanging up in time. Let me run some Tic/Toc and see where it's hanging up the most at and see if we can make it faster.

[dpb]

Try the profiler???

I'm not sure you need the typecast nor uint16 (altho they're not likely contributing much), do you? I also don't know if a format string of '0x %X' and eliminating the substring addressing would help much or not, but I'd give it a shot.

Answer 2

[Friedrich Welck]

MsgInfo = dec2bin(hex2dec(cellfun(@(x)(x(4:end)),dataArray{:, 2}, 'UniformOutput', false)),16);

is slow (cellfun is rarely fast). hex2dec, base2dec etc. work fastest on character arrays. Try

dataArray{2} = char(dataArray{2});
MsgInfo = dec2bin(base2dec(dataArray{2}(:,4:7),16),16);

Also there's no need to convert back to decimal for summation of errors. Try

ManchesterErrors = sum(MsgInfo(:,16)=='1');
ParityErrors = sum(MsgInfo(:,15)=='1');
OverRunErrors = sum(MsgInfo(:,14)=='1');
TimeOutErrors = sum(MsgInfo(:,13)=='1');
BroadcastMsg = sum(MsgInfo(:,11)=='1');
ModeCodeNoData = sum(MsgInfo(:,10)=='1');
TotalMessageErrors = sum(MsgInfo(:,9)=='1');