Reliability Assessment

Linear mixed models

As a group-level statistic, reliability refers to the inter-individual or between-subject variability \( V_b \) relative to the intra-individual or within-subject variability \( V_w \). Both the intra- and inter-individual variances can be estimated using linear mixed model (LMM). In network neuroscience, given a functional graph metric \( \phi \), we considered a random sample of \( h \) subjects with \( n \) repeated measurements of a continuous variable. \( \phi_{ij} \) (for \( i=1, ..., n \) and \( j=1, ..., h\)) denotes the metric from the \( j^{th} \) subject's \( i^{th} \) measurement occasions. The two level LMM models \( \phi_{ij} \) as the following equations:

\[ \overbrace{\phi_{ij}}^{\text{Graph metric}} = \underbrace{\gamma_{00}}_{\text{Fixed}\atop\text{intercept}} + \underbrace{s_{0k }}_{\text{Random Intercepts}\atop\text{Level 2, subjects}} + \underbrace{e_{ij}}_{\text{Random}\atop\text{residuals}} \]

Where \( \gamma_{00} \) is a fixed parameter (the group mean) and \( s_{0j} \) and \( e_{ij} \) are independent random effects normally distributed with a mean of 0 and variances \( \sigma_{s0}^{2} \) and \( \sigma_{e}^{2} \). The term \( s_{0j} \) is the subject effect and \( e_{ij} \) is the measurement residual. Age, gender and interval(\( \Delta \mathrm{t} \)) can be covariant.

\[ \mathrm{Reliability}(\phi) =\frac{V_b}{V_b+V_w}= \frac{\sigma^2_{s0}} {\sigma^2_{s0}+\sigma^2_{e}} \tag{ICC} \]

Reliability Map

Reliability Assessment Online toolkit

You can upload the example_data.csv or your own data then select the Response/Random variables, check the model results on the right panels.

How to cite

Please cite this paper or tools as:

"The Test-Retest reliability was estimated using a linear mixed model and calculated using the Reliability Assessment Online toolkit."

Jiang, C., Betzel, R., He, Y., & Zuo, X.-N. (2021). Toward Reliable Network Neuroscience for Mapping Individual Differences. bioRxiv, 2021.05.06.442886. https://doi.org/10.1101/2021.05.06.442886