speech signal from Microphone USB

hi , i have question . In matlab , can i take online speech signal from Microphone USB and use it in my code at same time ?? can do that ?????

Answer 1

[Chirag Gupta]

Try this: http://www.mathworks.com/help/techdoc/ref/audiorecorder.html. If you want to record analog data from your soundcard, you should also look at Data Acquisition Toolbox!

[Walter Roberson]

I think it likely that the USB Microphone would have to be connected before you start MATLAB. MATLAB scans the device table once the first time it needs to, and has no mechanisms for re-checking in case the device list has changed.

[hajer]

than you.

but i want to record from multiple Microphones .

can you help me.

[Walter Roberson]

Multiple microphones implies multiple device ID's.

If you need to record from several different microphones at the same time, you would probably need the Data Acquisition Toolbox; audiorecord() is not suitable for that task (because it does not return control until all the data for that device has been read.)

[hajer]

thank you for helping ;

in Data Acquisition Toolbox can i play recorded sound.

[Walter Roberson]

You would be able to output sound to a device, but DAT does not itself provide routines to read recorded sounds.

http://www.mathworks.com/products/daq/

With the toolbox you can configure data acquisition hardware and read data into MATLAB and Simulink® for immediate analysis. You can also send out data over analog and digital output channels provided by data acquisition hardware.

Answer 2

[Daniel Shub]

Depends what you mean by "same time." There is going to be a few milliseconds of latency. I would look at the MATLAB based port audio implementations. There are playrec and pa-wavplay (I am not sure i the latter handles input). My preference is for the audio part of the psychtoolbox http://docs.psychtoolbox.org/PsychPortAudio

[Hamza Ashraf]

hi i have trained a network to detect specific sound now how can i use it. i want to provide audio coming from mike to network. can you tell me how to do that

[Walter Roberson]

Hazma: unfortunately we cannot tell from your postings which MATLAB version you are using, and we cannot tell if you have the Audio System Toolbox. Also, are you using Windows 10 and do you have Data Acquisition Toolbox? (That combination gives you another option)

[Hamza Ashraf]

i am using matlab R2017a and yes i have both Audio System Toolbox and Data Acquisition Toolbox and am using windows 8.1 but i can install windows 10 if that is necessary

[Walter Roberson]

https://www.mathworks.com/matlabcentral/answers/569851-real-time-audio-signal-from-microphone#answer_470941

[Hamza Ashraf]

Oki sir that gives a real time audio signal but how can i provide it to trained network so that it can check either it is the specific sound or not

[Walter Roberson]

You read one buffer-full of sound from the microphone. You then pass that buffer to predict() or classify() depending on exactly which kind of network you are using.

[Hamza Ashraf]

sir i can not understand from the link that you commented above so kindly can you give me an example code to understand better

[Walter Roberson]

fs = 22100;
FrameSize = 512;
adr = audioDeviceReader(fs, FrameSize, 'NumChannels', 2);
while true
    inbuf = adr();
    prediction = predict(YourTrainedNetwork, inbuf);
    if prediction >= 1
        fprintf('You said the secret word at %s\n', char(datetime));
    end
end

[Hamza Ashraf]

load('trained_net_001.mat')

%              audiofilename  %audio_label
[t1, ls] = data('noise.wav',      2);
YTest = classify(net, t1);

[t, ls] = data('noiseambulancewail.wav',1);
YTest = classify(net, t);
amount_wail = sum(YTest == categorical(ls'))/numel(ls)

sir this is how i test my network but i dont understand this line of above code you commented

prediction = predict(YourTrainedNetwork, inbuf);

please tell me how i replace this line according to my case

[Walter Roberson]

[t1, ls] = data('noise.wav',      2);

I do not recognize any function named data that accepts a file name as a handle. Are you loading a function handle as part of trained_net_001.mat ? What is the ls output -- is it the category (label) information associated with the file?

Wail_category = something appropriate that is predicted by your net
fs = 22100;
FrameSize = 512;
adr = audioDeviceReader(fs, FrameSize, 'NumChannels', 2);
while true
    inbuf = adr();
    YTest = classify(net, inbuf);
    was_it_wail == YTest == Wail_category;
    if was_it_wail
        fprintf('Wail detected at time %s\n', char(datetime));
    end
end

[Hamza Ashraf]

function [specs, labels] = data(file_name, label)
    clear specs labels
    [wial_s,wail_fs]=audioread(file_name);
    [wail_freq, ~, ~] = spectrogram(wial_s, 512, 128, 512,wail_fs, 'yaxis');
    wail = 20*log10(abs(wail_freq));

    window = 30;
    overlap = 10;
    size_wail = size(wail);
    copies = 5;
    j = 1;

    %creating positive data samples
    xt = 1: window-overlap: (size_wail(2) - window + 1);

    for xms = 1:copies
        for i = xt
            specs(:,:,1,j) = wail(:,i:window+i-1);
            labels(j) =label;
            j = j+1;
        end
    end
    
    % this is how you draw the spectrogram.
    % surf(t,f,sl, 'EdgeColor', 'none')
    % colormap(jet);
end

this is data function that is used to create data samples for network to train it and yes ls is the label

i tried your above code but i get the following error

Error using DAGNetwork/predict>predictBatch (line 238)

Incorrect input size. The input images must have a size of [257 30 1].

Error in DAGNetwork/predict (line 118)

Y = predictBatch( ...

Error in DAGNetwork/classify (line 115)

scores = this.predict( X, varargin{:} );

Error in SeriesNetwork/classify (line 458)

[labels, scores] = this.UnderlyingDAGNetwork.classify(X, varargin{:});

[Hamza Ashraf]

sir are you there can you please help in this

[Walter Roberson]

You will need to write a new function related to your data() function, that instead of reading the data from a file, accepts wail_s and wail_fs as inputs and does the same kind of calculation of specs (but does not assign labels.)

Then in my loop where I showed

    inbuf = adr();
    YTest = classify(net, inbuf);

You would instead

    inbuf = adr();
    sp = calculate_specs(inbuf, fs);
    YTest = classify(net, sp);
    

Question:

        for i = xt

Are you sure you do not want

for i = 1:xt

?

[Hamza Ashraf]

oki i got it thankyou. it is working now but my tarined network is not giving me correct outcomes. i have trained it on large amount of data about 200 audio files but still am not getting correct outcomes. i dont know where am doing it wrong. can you help me in that case. here is code for my convolution neural network

layers = [imageInputLayer([257 30 1])
          convolution2dLayer(10,3)
          reluLayer
          maxPooling2dLayer(2,'Stride',2)
          convolution2dLayer(5,2)
          reluLayer
          maxPooling2dLayer(2,'Stride',2)
          fullyConnectedLayer(2)
          fullyConnectedLayer(2)
          softmaxLayer
          classificationLayer()];

options = trainingOptions('sgdm','MaxEpochs',60, 'InitialLearnRate',0.0001);

net = trainNetwork(specs, categorical(labels), layers, options);

YTest = classify(net, tests);

[Walter Roberson]

It is not clear to me why you would use image layers to try to process features extracted from audio input.

[Hamza Ashraf]

see in my data function audio files are spectrogram images thats why i used image layers.

the technique am using is explained in following pdf document

[Walter Roberson]

... Then your function that processes the audio from the microphone is going to need to create an image of the spectrogram, instead of just calculating the values for the spectrogram. This is not what your code reading from file did: your code reading from file calculated the values without writing out a spectrogram image.

[Hamza Ashraf]

function specs = dataoutput(wial_s,wail_fs)
    clear specs
    %[wial_s,wail_fs]=audioread(file_name);
    [wail_freq, ~, ~] = spectrogram(wial_s, 512, 128, 512,wail_fs, 'yaxis');
    wail = 20*log10(abs(wail_freq));

    window = 30;
    overlap = 10;
    size_wail = size(wail);
    copies = 1;
    j = 1;

    %creating positive data samples
    xt = 1: window-overlap: (size_wail(2) - window + 1);

    for xms = 1:copies
        for i = xt
            specs(:,:,1,j) = wail(:,i:window+i-1);
           
            j = j+1;
        end
    end
    
    % this is how you draw the spectrogram.
    % surf(t,f,sl, 'EdgeColor', 'none')
    % colormap(jet);
end

this is my function that process the audio from microphone and gives data to network. please look at it and mention if something need to be changed or how can i get correct results

[Walter Roberson]

If you have the signal processing toolbox, then you could consider using buffer() instead of your current for xms / for i loops.

Your dataoutput() routine does appear to be consistent with your earlier data() routine. However, neither of them create the spectrogram images that you indicate that your deep learning code relies on. You would have to imwrite() the images, possibly after capturing the image from the screen.

[Hamza Ashraf]

Oki sir as my data functions are extracting features from audio input which layers should i use in my network. Can you help me modify my network in that scenario

[Walter Roberson]

You just might be able to continue using imageInputLayer, but instead of an image of a spectrogram, you would pass it the array of specs data

[Hamza Ashraf]

From my data fucntion it can be seen that am passing the array of specs and labels but still am not getting acurate results

Answer 3

[Gabriele Bunkheila]

I work at MathWorks and I have just noticed this so I thought I'd drop a note.

I understad from your question that you were looking to process the acquired audio live (aka in "real time", or "in the loop") using a standard sound card. In all such situations we recommend looking first at audioDeviceReader and audioDeviceWriter in Audio Toolbox, which were specifically designed with that use case in mind.

The doc page below should help you get started and it includes a few links to additional resources:https://www.mathworks.com/help/audio/gs/audio-input-and-audio-output.html

Another useful page full with code examples and explanations can be found below:https://www.mathworks.com/help/audio/gs/real-time-audio-in-matlab.html

Thanks and good luck with your project.