writeToLastMemberRead

Write data to member of an ensemble datastore

Syntax

writeToLastMemberRead(ensemble,Name,Value)

writeToLastMemberRead(ensemble,data)

Description

writeToLastMemberRead(ensemble,Name,Value) writes the data specified one or more Name,Value pair arguments to the last-read member of an ensemble datastore. The last-read member is the member most recently accessed using the read command. (See Data Ensembles for Condition Monitoring and Predictive Maintenance.) Each Name must match an entry in the property ensemble.DataVariables. The function writes the corresponding Value to the ensemble datastore.

• If ensemble is a simulationEnsembleDatastore object, then writeToLastMemberRead writes the data to the MAT file corresponding to the last-read ensemble member (ensemble.LastMemberRead).

• If ensemble is a fileEnsembleDatastore object, then writeToLastMemberRead uses the function identified by the property ensemble.WriteToMemberFcn to write the data. If that property is not defined, then writeToLastMemberRead generates an error.

This syntax is not available when the ReadSize property of ensemble is greater than 1. Use writeToLastMemberRead(ensemble,data) instead.

writeToLastMemberRead(ensemble,data) writes the data in a table to the last-read ensemble member. The table variables must match entries in the property ensemble.DataVariables.

Examples

Append Derived Data to Ensemble Members

You can process data in an ensemble datastore and add derived variables to the ensemble members. For this example, process a variable value to compute a label that indicates whether the ensemble member contains data obtained with a fault present. You then add that label to the ensemble.

For this example, use the following code to create a simulationEnsembleDatastore object using data previously generated by running a Simulink® model at a various fault values. (See generateSimulationEnsemble.) The ensemble includes simulation data for five different values of a model parameter, ToothFaultGain. The model was configured to log the simulation data to a variable named logsout in the MAT files that are stored for this example in simEnsData.zip. Because of the volume of data, the unzip operation might take a minute or two.

unzip simEnsData.zip  % extract compressed files
ensemble = simulationEnsembleDatastore(pwd,'logsout')

ensemble = 
  simulationEnsembleDatastore with properties:

           DataVariables: [5x1 string]
    IndependentVariables: [0x0 string]
      ConditionVariables: [0x0 string]
       SelectedVariables: [5x1 string]
                ReadSize: 1
              NumMembers: 5
          LastMemberRead: [0x0 string]
                   Files: [5x1 string]

Read the data from the first member in the ensemble. The software determines which ensemble is the first member, and updates the property ensemble.LastMemberRead to reflect the name of the corresponding file.

data = read(ensemble)

data=1×5 table
    PMSignalLogName           SimulationInput                   SimulationMetadata                   Tacho                Vibration     
    _______________    ______________________________    _________________________________    ___________________    ___________________

      {'logsout'}      {1x1 Simulink.SimulationInput}    {1x1 Simulink.SimulationMetadata}    {20202x1 timetable}    {20202x1 timetable}

By default, all the variables stored in the ensemble data are designated as SelectedVariables. Therefore, the returned table row includes all ensemble variables, including a variable SimulationInput, which contains the Simulink.SimulationInput object that configured the simulation for this ensemble member. That object includes the ToothFaultGain value used for the ensemble member, stored in a data structure in its Variables property. Examine that value. (For more information about how the simulation configuration is stored, see Simulink.SimulationInput (Simulink).)

data.SimulationInput{1}

ans = 
  SimulationInput with properties:

               ModelName: 'TransmissionCasingSimplified'
            InitialState: [0x0 Simulink.op.ModelOperatingPoint]
           ExternalInput: []
         ModelParameters: [0x0 Simulink.Simulation.ModelParameter]
         BlockParameters: [0x0 Simulink.Simulation.BlockParameter]
               Variables: [1x1 Simulink.Simulation.Variable]
               PreSimFcn: []
              PostSimFcn: []
              UserString: ''
    VariantConfiguration: ''

Inputvars = data.SimulationInput{1}.Variables;
Inputvars.Name

ans = 
'ToothFaultGain'

Inputvars.Value

ans = 
-2

Suppose that you want to convert the ToothFaultGain values for each ensemble member into a binary indicator of whether or not a tooth fault is present. Suppose further that you know from your experience with the system that tooth-fault gain values less than 0.1 in magnitude are small enough to be considered healthy operation. Convert the gain value for this ensemble into an indicator that is 0 (no fault) for –0.1 < gain < 0.1, and 1 (fault) otherwise.

sT = abs(Inputvars.Value) < 0.1;

To append the new tooth-fault indicator to the corresponding ensemble data, first expand the list of data variables in the ensemble to include a variable for the indicator.

ensemble.DataVariables = [ensemble.DataVariables; "ToothFault"];
ensemble.DataVariables

ans = 6x1 string
    "PMSignalLogName"
    "SimulationInput"
    "SimulationMetadata"
    "Tacho"
    "Vibration"
    "ToothFault"

This operation is conceptually equivalent to adding a column to the table of ensemble data. Now that DataVariables contains the new variable name, assign the derived value to that column of the member using writeToLastMemberRead.

writeToLastMemberRead(ensemble,'ToothFault',sT);

In practice, you want to append the tooth-fault indicator to every member in the ensemble. To do so, reset the ensemble datastore to its unread state, so that the next read operation starts at the first ensemble member. Then, loop through all the ensemble members, computing ToothFault for each member and appending it. The reset operation does not change ensemble.DataVariables, so "ToothFault" is still present in that list.

reset(ensemble);

sT = false; 
while hasdata(ensemble)
    data = read(ensemble);
    InputVars = data.SimulationInput{1}.Variables;
    TFGain = InputVars.Value;
    sT = abs(TFGain) < 0.1;
    writeToLastMemberRead(ensemble,'ToothFault',sT);
end

Finally, designate the new tooth-fault indicator as a condition variable in the ensemble datastore. You can use this designation to track and refer to variables in the ensemble data that represent conditions under which the member data was generated.

ensemble.ConditionVariables = {"ToothFault"};
ensemble.ConditionVariables

ans = 
"ToothFault"

You can add the new variable to ensemble.SelectedVariables when you want to read it out for further analysis. For an example that shows more ways to manipulate and analyze data stored in a simulationEnsembleDatastore object, see Using Simulink to Generate Fault Data.

Read from and Write to a File Ensemble Datastore

Create a file ensemble datastore for data stored in MATLAB® files, and configure it with functions that tell the software how to read from and write to the datastore. (For more details about configuring file ensemble datastores, see File Ensemble Datastore with Measured Data.)

% Create ensemble datastore that points to datafiles in current folder
unzip fileEnsData.zip  % extract compressed files
location = pwd;
extension = '.mat';
fensemble = fileEnsembleDatastore(location,extension);

% Specify data and condition variables
fensemble.DataVariables = ["gs";"sr";"load";"rate"];
fensemble.ConditionVariables = "label";

% Configure with functions for reading and writing variable data
fensemble.ReadFcn = @readBearingData;
fensemble.WriteToMemberFcn = @writeBearingData; 

The functions tell the read and writeToLastMemberRead commands how to interact with the data files that make up the ensemble. Thus, when you call the read command, it uses readBearingData to read all the variables in fensemble.SelectedVariables. For this example, readBearingData extracts requested variables from a structure, bearing, and other variables stored in the file. It also parses the filename for the fault status of the data.

Specify variables to read, and read them from the first member of the ensemble.

fensemble.SelectedVariables = ["gs";"load";"label"];
data = read(fensemble)

data=1×3 table
     label            gs           load
    ________    _______________    ____

    "Faulty"    {5000x1 double}     0  

You can now process the data from the member as needed. For this example, compute the average value of the signal stored in the variable gs. Extract the data from the table returned by read.

gsdata = data.gs{1};
gsmean = mean(gsdata);

You can write the mean value gsmean back to the data file as a new variable. To do so, first expand the list of data variables in the ensemble to include a variable for the new value. Call the new variable gsMean.

fensemble.DataVariables = [fensemble.DataVariables;"gsMean"]

fensemble = 
  fileEnsembleDatastore with properties:

                 ReadFcn: @readBearingData
        WriteToMemberFcn: @writeBearingData
           DataVariables: [5x1 string]
    IndependentVariables: [0x0 string]
      ConditionVariables: "label"
       SelectedVariables: [3x1 string]
                ReadSize: 1
              NumMembers: 5
          LastMemberRead: "/tmp/Bdoc24b_2679053_60642/tp40ea929e/predmaint-ex34165887/FaultData_01.mat"
                   Files: [5x1 string]

Next, write the derived mean value to the file corresponding to the last-read ensemble member. (See Data Ensembles for Condition Monitoring and Predictive Maintenance.) When you call writeToLastMemberRead, it converts the data to a structure and calls fensemble.WriteToMemberFcn to write the data to the file.

writeToLastMemberRead(fensemble,'gsMean',gsmean);

Calling read again advances the last-read-member indicator to the next file in the ensemble and reads the data from that file.

data = read(fensemble)

data=1×3 table
     label            gs           load
    ________    _______________    ____

    "Faulty"    {5000x1 double}     50 

You can confirm that this data is from a different member by examining the load variable in the table. Here, its value is 50, while in the previously read member, it was 0.

You can repeat the processing steps to compute and append the mean for this ensemble member. In practice, it is more useful to automate the process of reading, processing, and writing data. To do so, reset the ensemble to a state in which no data has been read. Then loop through the ensemble and perform the read, process, and write steps for each member.

reset(fensemble)
while hasdata(fensemble)
    data = read(fensemble);
    gsdata = data.gs{1};
    gsmean = mean(gsdata);
    writeToLastMemberRead(fensemble,'gsMean',gsmean);
end

The hasdata command returns false when every member of the ensemble has been read. Now, each data file in the ensemble includes the gsMean variable derived from the data gs in that file. You can use techniques like this loop to extract and process data from your ensemble files as you develop a predictive-maintenance algorithm. For an example illustrating in more detail the use of a file ensemble datastore in the algorithm-development process, see Rolling Element Bearing Fault Diagnosis. The example also shows how to use Parallel Computing Toolbox™ to speed up the processing of large data ensembles.

To confirm that the derived variable is present in the file ensemble datastore, read it from the first and second ensemble members. To do so, reset the ensemble again, and add the new variable to the selected variables. In practice, after you have computed derived values, it can be useful to read only those values without rereading the unprocessed data, which can take significant space in memory. For this example, read selected variables that include the new variable, gsMean, but do not include the unprocessed data, gs.

reset(fensemble)
fensemble.SelectedVariables = ["label";"load";"gsMean"];
data1 = read(fensemble)

data1=1×3 table
     label      load     gsMean 
    ________    ____    ________

    "Faulty"     0      -0.22648

data2 = read(fensemble)

data2=1×3 table
     label      load     gsMean 
    ________    ____    ________

    "Faulty"     50     -0.22937

Input Arguments

ensemble — Ensemble datastore

simulationEnsembleDatastore object | fileEnsembleDatastore object

Ensemble datastore to add data variables to, specified as a:

• simulationEnsembleDatastore object

• fileEnsembleDatastore object

writeToLastMemberRead writes the data to the last-read member of the specified ensemble, identified by the LastMemberRead property of the ensemble. The last-read ensemble member is the member most recently accessed using the read command. (See Data Ensembles for Condition Monitoring and Predictive Maintenance.)

data — New data

table

New data to write to the current ensemble member, specified as a table. For example, suppose that you have calculated two values that you want to add to the current member: a vector stored as the MATLAB^® workspace variable Afilt, and a scalar stored as Amean. Use the following command to construct data.

data = table(Afilt,Amean,'VariableNames',{'Afilt','Amean'});

The number of rows in the table must match the ReadSize property of ensemble. By default, ReadSize = 1, and you write a single table row to a single ensemble member. When you configure ensemble to read multiple members at once, you must write to the same number of members. For instance, if ReadSize = 3, then data is a three-row table.

Limitations

• When you use a simulationEnsembleDatastore to manage data at a remote location, such as cloud storage using Amazon S3™ (Simple Storage Service), Windows Azure^® Blob Storage, and Hadoop^® Distributed File System (HDFS™), you cannot use writeToLastMemberRead to add data to the ensemble datastore.

Version History

Introduced in R2018a