NN quantification with array calibration data

Hello,

I am currently working on a Convolutional Neural Network (CNN) for image classification on an FPGA. I have already imported a pre-trained neural network into a SeriesNetwork object and successfully performed inference on the FPGA using the Deep Learning Toolbox. However, I would like to perform inference using the int8 data type. According to MATLAB documentation, I need to quantize the network and perform inference using an int8 deep learning processor. To achieve this, I need to convert the neural network into a dlquantizer object, calibrate it, and validate it before obtaining the quantized neural network.

My input data consists of 3D arrays (image format), and the labels are in the form of 2D arrays of binary data (where each 3D input corresponds to a 2D label). My question is how to handle these data using either the ImageDatastore or arrayDatastore object in order to calibrate the neural network or if it is possible to use data arrays as calibration data.

Initially, 3D array were .png files and the 3D arrays are the imported and preprocessed (padding + normalization) images, so I tried to use imageDatastore format but labes, in contrast, are .txt files. Is there any possiplibity to combine imageDatastore nd set the labels to the imported 2D arrays (from .txt files), and then use TransformDatastore to apply the preprocessing functions or any other alternative?

As a summary I need to calibrate a classification image where the label is a binary array. The primitive data are .png and .txt files and I have them procesed into arrays or cell of arrays.

Answer 1

[Adarsh Chitradurga Achutha]

Please let us know the install version of MATLAB.

For calibration in 2023b you could use any of the following ways to provide input to calibrate

1. NumericArray of Images
2. To introduce necessary preprocessing such as padding and normalization to the images, you can create an imageDatastore to import the images and then use the transformedDatastore. The transformedDatastore allows you to apply specific transformations to the images before using them. How to create Transformed Datastore
3. The imageDatastore can be utilized for calibration purposes as well. However, when using the dlquantizer, each image is fitted to the network's inputSize, and no additional transformations are applied.

In order to create and compile a dlhdl.Workflow object for an INT8 bitstream, you would need a calibrated dlquantizer object as the network value.

In the dlquantizer validation step, there is a default validation metric available for semantic segmentation problems. To perform the validation, you need to provide the ground truth data as part of of either a pixelLabelImageDatastore( To be deprecated) or a combinedDatastore.

1. combinedDatastore can have only two datastores with the first datastore being an image or an augmentedImageDatastore and the second one being a pixelLabelDatastore
2. An augmentedImageDatastore can be constructed using processed numeric arrays of images present in the MATLAB workspace. This is particularly useful when you have already preprocessed the images and stored them as numeric arrays.
3. pixelLabelDatastore can be used for the groundtruth. To create this you would need to define a custom read function to read the text files and provide uint8 type output.

function img = customReadFunc(file)

img = uint8(readmatrix(file));

end

pxds = pixelLabelDatastore(loc, ["test1", "test2"], [0 1],"ReadFcn",@customReadFunc,"FileExtensions",['.txt']);

The validate step is not a requirement to create and compile a dlhdl.Workflow object for INT8 bitstream. More information on using validation on FPGA target can be found on validate.

To receive a more accurate and tailored response, please reach out to mathworks support with specific details regarding the problem at hand.

https://www.mathworks.com/support/contact_us.html

Additional Links:

This example demonstrates deploying a semantic segmentation network on an FPGA target using single and INT8 workflows Deploy Semantic Segmentation Network using Dilated Convolutions on FPGA.

Please see Prepare Data For Quantizing Networks and Quantization Workflow Prerequisites