Reading multiple binary formats with fread()

I am working on a project that requires reading in large amounts of data from a binary file. The issue is that I have to be able to read different binary formats depending on the source. All of the data is 16 bit integers stored in a binary file but it could be twos-compliment or unsigned, big endian or little endian, most significant bit first or least significant bit first.

I know that fread will allow you to set the endianess and signed/unsigned, but taking care of the bit order is what is tripping me up.

The following is what I have for signed/unsigned and big/little endian:

%twos compliment-big endian-MSB
sampleChunk = fread(fid, chunkSizeSamples, 'int16=>single','ieee-be');

%twos complement-little endian-MSB
sampleChunk = fread(fid, chunkSizeSamples, 'int16=>single','ieee-le');

%unsinged-big endian-MSB
sampleChunk = fread(fid, chunkSizeSamples, 'uint16=>single','ieee-be');

%unsigned-little endian-MSB
sampleChunk = fread(fid, chunkSizeSamples, 'uint16=>single','ieee-le');

However, there are four more possibilities if the least significant bit is first that I do not know how to handle without costly bit flipping routines. Does anyone have any suggestions to be able to easily read the eight different formats.

Just to be clear since endianess sometimes has different meaning, I am concerned with both byte ordering (endianess) and bit ordering.

Answer 1

[Snehal]

Hi @Brian,

I understand that you are able to cover the cases for signed/unsigned and big/little endian when reading large amounts of data from a binary file, but would like to handle additional cases of least significant bit first and most significant bit first too.

MATLAB’s ‘fread’ function provides the options to handle:

• Signed vs unsigned
• Big vs little endian

But it has no native way to reverse the bit order within each 16-bit sample.

So you will have to manually reverse the bits after reading with ‘fread’.

Here is a sample code for implementing the above-mentioned logic:

% 1. First read normally with ‘fread’ 
% 2. Then, define a helper to reverse the bit order in 16-bit words: 
function reversed = reverse_bits(uint16_vals) 
    % Look-up-table approach (optional for faster), or use bit operations: 
    reversed = zeros(size(uint16_vals), 'uint16'); 
    for i = 1:length(uint16_vals) 
        val = uint16_vals(i); 
        r = 0; 
        for b = 0:15 
            r = bitor(r, bitshift(bitand(val, bitshift(1, b)), 15-2*b));  
        end 
        reversed(i) = r; 
    end 
end 

% 3. Apply bit-reversal: 
sampleChunkFlipped = reverse_bits(sampleChunk); 

Refer to the following documentation links for more details:

• https://www.mathworks.com/help/matlab/ref/fread.html
• https://www.mathworks.com/help/matlab/ref/bitor.html
• https://www.mathworks.com/help/matlab/ref/bitshift.html
• https://www.mathworks.com/help/matlab/ref/bitand.html
• https://www.mathworks.com/help/matlab/ref/zeros.html

Hope this helps!