fitglme results differ from lme4 results in R for mixed effect models

I am doing mixed effect modeling in both fitglme of MatLab and lme4 in R. I found that they give very different answers, specifically for model comparison.

In particular, I am trying to predict a binary variable (correct) using mixed effect model with fitglme. I have 1 group variable (id) and 2 regressors of interest (prew and delay).

Here is the output when I only use prew in fitglme with random intercept:

================================================

Formula:

correct ~ 1 + prew + (1 | id)

Model fit statistics:

AIC BIC LogLikelihood Deviance

1.5737e+05 1.5739e+05 -78681 1.5736e+05

Fixed effects coefficients (95% CIs):

Name Estimate SE tStat DF

{'(Intercept)'} 1.4224 0.051302 27.726 32658

{'prew' } 0.59137 0.016268 36.353 32658

pValue Lower Upper

2.9725e-167 1.3218 1.5229

1.0983e-283 0.55949 0.62326

================================================

Here is the output when I use both prew and delay in fitglme, also with random intercept:

================================================

Formula:

correct ~ 1 + prew + delay + (1 | id)

Model fit statistics:

AIC BIC LogLikelihood Deviance

1.5769e+05 1.5772e+05 -78841 1.5768e+05

Fixed effects coefficients (95% CIs):

Name Estimate SE tStat DF

{'(Intercept)'} 1.428 0.049226 29.009 32657

{'prew' } 0.57954 0.016358 35.428 32657

{'delay' } -0.20041 0.013185 -15.2 32657

pValue Lower Upper

1.0674e-182 1.3315 1.5245

8.3176e-270 0.54747 0.6116

5.3311e-52 -0.22625 -0.17457

================================================

Note that the AIC of the model adding delay is worse, which is possible due to penalization for model complexity. However, what doesn't make sense is the fact that LogLikelihood of the model including delay is also worse. This is very confusing to me. I thought adding one more regressor should always improve the likelihood of the model (one can just set the coefficient of the additional regressor to be 0). Because of this confusion, I decided to do the same modeling using lme4, which is a commonly used R package for mixed effect modeling.

Here is the result when I only use prew in lme4 with random intercept:

================================================

Formula: correct ~ 1 + prew + (1 | id)

AIC BIC logLik deviance df.resid

31360.6 31385.8 -15677.3 31354.6 32657

Fixed effects:

Estimate Std. Error z value Pr(>|z|)

(Intercept) 1.44155 0.05228 27.57 <2e-16 ***

prew 0.59603 0.01627 36.63 <2e-16 ***

================================================

Here is the result when I use both prew and delay in lme4 with random intercept:

================================================

Formula: correct ~ 1 + prew + delay + (1 | id)

AIC BIC logLik deviance df.resid

31128.5 31162.1 -15560.3 31120.5 32656

Fixed effects:

Estimate Std. Error z value Pr(>|z|)

(Intercept) 1.44694 0.05018 28.84 <2e-16 ***

prew 0.58410 0.01636 35.70 <2e-16 ***

delay -0.20204 0.01325 -15.25 <2e-16 ***

================================================

Note that all the fixed effects are pretty close for fitglme and lme4. The main difference lies in model comparison: AIC, BIC, likelihood and deviance. The outcome of lme4 makes more sense to me, since the logLik of the model with delay is better than the one without.

Can anyone shed some light on how fitglme calculates model comparison scores, and why the loglikelihood doesn't make sense in fitglme? Thanks!

[Paul]

Hi,

I don't know anything about R, but without any code it will be impossible to determine why you see a difference. Could you share the code for the production of your table and for the fitglme() and lme4 functions? Specifically we need the input arguments to those functions.

[Aditya Patil]

Can you provide additional information such as a sample code, so that we can further investigate the issue?

Answer 1

[Grace Steward]

I'm having a similar issue when comparing models with fitglme vs lme4. Any clarification would be appreciated. Thank you!

Edit: After digging some more, it seems R uses a Laplace Approximation of Maximum Likelihood by default, which is different than 'fitglme's default settings. However, if the 'FitMethod' property of 'fitglme' is changed to 'Laplace', the discrepency between AIC, BIC, and loglikelihood should be rectified.

[Paul]

We would need to look at your code to answer any questions. If you could share the Matlab and R code, as well as a specific question, we may be able to help.

[Grace Steward]

Hello, thanks for your reponse. After some more digging I believe I found the source of the discrepency. R uses a Laplace Approximation of Maximum Likelihood by default, which is different than 'fitglme's default settings. However, if the 'FitMethod' property of 'fitglme' is changed to 'Laplace', the discrepency no longer exists.