ClassificationLearner Cross Validation without shuffling

I'm working with time series data with multible reps of stimulus. My research shows that shuffling data between reps unintentionally and virually beifets the accuricies of my modles, and not uniformly between types of modles. Is there anyway to have the ClassificationLearner not randomly shuffling my data? I need it trained on reps 1 and 2, tested on 3, not trained of bits of all the reps and tested on windows temperorly adjacent to my training data.

Answer 1

[Drew]

You can have Classification Learner train on reps 1 and 2 and test on rep 3 by first separating the data at the commandline. The steps are:

(1) Create one workspace variable with the data for reps 1 and 2, and another workspace variable with rep 3.

(2) Start Classification Learner and load the workspace variable for reps 1 and 2 as the training data.

(3) Build models

(4) Load the workspace variable for rep 3 as a test set.

(5) Test models on rep 3