How do you calculate observed F using Scheffe's method?

I'm currently conducting some post-hoc ANOVA analyses for my statistics homework (this is not real data). I first conducted an ANOVA to see if there's a difference between 3 groups (Monetary, Bonus, Feelings).

%data input

ingroups = [5,4,6,7,5,6,7,5,5,6;6,7,7,5,7,7,8,6,7,7;7,8,7,7,8,5,6,7,7,7];

Mon = [5 4 6 7 5 6 7 5 5 6];

Bon = [6 7 7 5 7 7 8 6 7 7];

Feel = [7 8 7 7 8 5 6 7 7 7];

%transpose data rows in to columns for anova1

ingroups2 = transpose(ingroups);

%one-way ANOVA (treats columns as separate groups)

[p,tbl,stats] = anova1(ingroups2);

I was able to reject the null hypothesis, F(2) = 6.18, p = 0.0062 , so now I've been asked (in the homework) to conduct a Scheffe's test, along with some others.

To do so, I ran the command below.

[c,m,h,gnames] = multcompare(stats,'CType','scheffe');

Using, https://www.danielsoper.com/statcalc/calculator.aspx?id=4 I computed my critical F-value to be 3.35.

But how do I compute the observed F in order to compare it against my critical F-value to see if the groups significantly differ? My homework specifically asks for the observed F.

I'm a beginner with MATLAB, so I'd appreciate any help.

[Scott MacKenzie]

You posted this question before. I suggest you ask your instructor for clarification because there is no observed F-statistic for a post hoc comparisons test. Perhaps the question is just asking for the F-statistic computed from the observed data in performing the analysis of variance that precedes the post hoc test. You correctly obtained this value via the anova1 function: F(2,27) = 6.18, p = .0062. (Note that the F-statistic is reported with two df statistics, not one as in your question.)

As for the critical F-value in your question, this is the F-statistic used in the formula to compute the Scheffe critical difference. I've never seen it in a reseach paper where the results of a post hoc comparisons test are reported. And there is no observed F-statistic against which this is compared.

BTW, you used an external web site, but you can also get this value using MATLAB's icdf function:

% F-statistic used in computing Scheffe critical difference
F = icdf('F', 0.95, 2, 27) 

F = 3.3541

[Darla Bonagura]

Yes, I will ask for further clarification. I did find a video on YouTube, where they calculated an "observed F" using Scheffe's method. But I appreicate this critical F function! Thank you very much for your help.

[Scott MacKenzie]

Let me first offer this bare-bones post hoc comparisons test using your test data and the Scheffe method:

% your test data
d = [5,4,6,7,5,6,7,5,5,6;6,7,7,5,7,7,8,6,7,7;7,8,7,7,8,5,6,7,7,7]';

% do the anova
[~, tbl, ~] = anova1(d, [], 'off');

k = size(d,2); % number of conditions (columns)
n = size(d,1); % size of each group (rows)
N = numel(d);  % number of scores (total)

alpha = 0.05;    % desired alpha (probability of Type-I error)
df1 = k-1;       % between-groups degrees of freedom     
df2 = N-k;       % within-groups degrees of freedom
MSWG = tbl{3,4}; % within-groups mean square (from anova1 table)

% critical F used in Scheffe critical difference formula
F = icdf('F', 1-alpha, df1, df2);

% Scheffe critical difference
cd = sqrt((k-1)*F) * sqrt(2*MSWG/n);

% difference in means between pairs of columns
dm12 = abs(mean(d(:,1)) - mean(d(:,2)));
dm13 = abs(mean(d(:,1)) - mean(d(:,3)));
dm23 = abs(mean(d(:,2)) - mean(d(:,3)));

% Voila!
pairwiseComparisons = [dm12, cd; dm13, cd; dm23, cd]

pairwiseComparisons = 3×2

1.1000 1.0312 1.3000 1.0312 0.2000 1.0312

As seen above, the mean difference exceeds the critical difference in the first two rows. These correspond to column 1 vs. 2 and column 1 vs. 3. These pairs of conditions differ significantly according to the Scheffe method.

In my experience, this is the standard way to do post hoc pairwise comparisons. By way of example, here's the output from a commercial stats package from SAS using the same data:

The result is the same, but also provides the p-statistic for the significance of each pairwise comparison.

Now, I watched the YouTube video in your link. My gosh, yes, it seems like a reasonable alternative way to do the same analysis. Instead of calculating the Scheffe critical difference, they compute observed F-statistics and compare them against the computed critcal F-statistic. Yup, seems reasonable. And I see your dilemma: Your homework asks you do the analysis in this manner. Have your tried to implement the steps in the video in MATLAB? They seem fairly straight-forward.

[Darla Bonagura]

This is super helpful. Thank you so much! I'm going to work on following the steps from the video.

[Scott MacKenzie]

You're welcome. BTW, you can compute the observed F-statistics by adding just a few lines to the code in my last comment. All the pieces are there! Good luck.