Note
Go to the end to download the full example code.
Harmonize HBN data using ComBat#
This example loads AFQ data from the Healthy Brain Network (HBN) preprocessed diffusion derivatives [1]. The HBN is a landmark pediatric mental health study. Over the course of the study, it will collect diffusion MRI data from approximately 5,000 children and adolescents. We recently processed the available data from over 2,000 of these subjects, and provide the tract profiles from this dataset, which can be downloaded from AWS thanks to [INDI](http://fcon_1000.projects.nitrc.org/).
We first load the data by using the AFQDataset.from_files()
static method
and supplying AWS S3 URIs instead of local file names. We then impute missing
values and plot the mean bundle profiles by scanning site, noting that there are
substantial site differences. Lastly, we harmonize the site differences using
NeuroComBat [2] and plot the harmonized bundle profiles to verify that the site
differences have been removed.
import numpy as np
from neurocombat_sklearn import CombatModel
from sklearn.impute import SimpleImputer
from sklearn.model_selection import train_test_split
from afqinsight import AFQDataset
from afqinsight.plot import plot_tract_profiles
Fetch the HBN data#
As a shortcut, we have incorporated a few studies into the software. In these
cases, a AFQDataset
class instance can be initialized using the
AFQDataset.from_study()
static method. This expects the name of one of
the studies that are supported (see the method documentation for the list of
these studies). By passing “hbn”, we request that the object download the
HBN dataset from the AWS Open Data program where it has been stored and
initialize the objects with the subjects and nodes information. Subjects’ age
is set as the target variable. After dropping subjects that don’t have their
age recorded, there are 1867 subjects in the dataset.
subjects.tsv: 0%| | 0.00/211k [00:00<?, ?iB/s]
subjects.tsv: 100%|██████████| 211k/211k [00:00<00:00, 2.54MiB/s]
nodes.csv: 0%| | 0.00/543M [00:00<?, ?iB/s]
nodes.csv: 0%| | 388k/543M [00:00<02:39, 3.40MiB/s]
nodes.csv: 1%| | 6.46M/543M [00:00<00:15, 35.3MiB/s]
nodes.csv: 3%|▎ | 17.5M/543M [00:00<00:07, 68.3MiB/s]
nodes.csv: 5%|▌ | 28.4M/543M [00:00<00:06, 84.4MiB/s]
nodes.csv: 7%|▋ | 39.4M/543M [00:00<00:05, 93.2MiB/s]
nodes.csv: 9%|▉ | 50.4M/543M [00:00<00:04, 99.1MiB/s]
nodes.csv: 11%|█▏ | 61.4M/543M [00:00<00:04, 103MiB/s]
nodes.csv: 13%|█▎ | 72.5M/543M [00:00<00:04, 105MiB/s]
nodes.csv: 15%|█▌ | 83.5M/543M [00:00<00:04, 107MiB/s]
nodes.csv: 17%|█▋ | 94.3M/543M [00:01<00:04, 102MiB/s]
nodes.csv: 19%|█▉ | 105M/543M [00:01<00:04, 105MiB/s]
nodes.csv: 21%|██▏ | 116M/543M [00:01<00:04, 106MiB/s]
nodes.csv: 23%|██▎ | 127M/543M [00:01<00:03, 108MiB/s]
nodes.csv: 25%|██▌ | 138M/543M [00:01<00:03, 108MiB/s]
nodes.csv: 27%|██▋ | 149M/543M [00:01<00:03, 106MiB/s]
nodes.csv: 30%|██▉ | 160M/543M [00:01<00:03, 108MiB/s]
nodes.csv: 32%|███▏ | 171M/543M [00:01<00:03, 101MiB/s]
nodes.csv: 34%|███▎ | 182M/543M [00:01<00:03, 104MiB/s]
nodes.csv: 36%|███▌ | 193M/543M [00:01<00:03, 105MiB/s]
nodes.csv: 38%|███▊ | 204M/543M [00:02<00:03, 106MiB/s]
nodes.csv: 40%|███▉ | 215M/543M [00:02<00:03, 107MiB/s]
nodes.csv: 42%|████▏ | 226M/543M [00:02<00:02, 107MiB/s]
nodes.csv: 44%|████▎ | 236M/543M [00:02<00:03, 101MiB/s]
nodes.csv: 45%|████▌ | 247M/543M [00:02<00:02, 102MiB/s]
nodes.csv: 47%|████▋ | 257M/543M [00:02<00:02, 99.6MiB/s]
nodes.csv: 49%|████▉ | 267M/543M [00:02<00:02, 99.2MiB/s]
nodes.csv: 51%|█████ | 277M/543M [00:02<00:02, 99.9MiB/s]
nodes.csv: 53%|█████▎ | 287M/543M [00:02<00:02, 99.7MiB/s]
nodes.csv: 55%|█████▍ | 297M/543M [00:02<00:02, 99.3MiB/s]
nodes.csv: 57%|█████▋ | 307M/543M [00:03<00:02, 98.9MiB/s]
nodes.csv: 58%|█████▊ | 317M/543M [00:03<00:02, 98.7MiB/s]
nodes.csv: 60%|██████ | 327M/543M [00:03<00:02, 97.8MiB/s]
nodes.csv: 62%|██████▏ | 337M/543M [00:03<00:02, 98.1MiB/s]
nodes.csv: 64%|██████▍ | 347M/543M [00:03<00:02, 71.4MiB/s]
nodes.csv: 66%|██████▌ | 356M/543M [00:03<00:02, 76.6MiB/s]
nodes.csv: 67%|██████▋ | 366M/543M [00:03<00:02, 82.8MiB/s]
nodes.csv: 69%|██████▉ | 376M/543M [00:03<00:01, 87.5MiB/s]
nodes.csv: 71%|███████ | 386M/543M [00:04<00:01, 91.1MiB/s]
nodes.csv: 73%|███████▎ | 396M/543M [00:04<00:01, 93.8MiB/s]
nodes.csv: 75%|███████▍ | 406M/543M [00:04<00:01, 95.9MiB/s]
nodes.csv: 77%|███████▋ | 416M/543M [00:04<00:01, 97.5MiB/s]
nodes.csv: 79%|███████▊ | 427M/543M [00:04<00:01, 98.6MiB/s]
nodes.csv: 80%|████████ | 437M/543M [00:04<00:01, 98.4MiB/s]
nodes.csv: 82%|████████▏ | 447M/543M [00:04<00:00, 98.7MiB/s]
nodes.csv: 84%|████████▍ | 456M/543M [00:04<00:00, 98.6MiB/s]
nodes.csv: 86%|████████▌ | 467M/543M [00:04<00:00, 99.6MiB/s]
nodes.csv: 88%|████████▊ | 477M/543M [00:04<00:00, 99.7MiB/s]
nodes.csv: 90%|████████▉ | 487M/543M [00:05<00:00, 101MiB/s]
nodes.csv: 92%|█████████▏| 497M/543M [00:05<00:00, 100MiB/s]
nodes.csv: 93%|█████████▎| 507M/543M [00:05<00:00, 100MiB/s]
nodes.csv: 95%|█████████▌| 517M/543M [00:05<00:00, 100MiB/s]
nodes.csv: 97%|█████████▋| 527M/543M [00:05<00:00, 99.8MiB/s]
nodes.csv: 99%|█████████▉| 537M/543M [00:05<00:00, 97.8MiB/s]
nodes.csv: 100%|██████████| 543M/543M [00:05<00:00, 97.1MiB/s]
AFQDataset(n_samples=1867, n_features=4800, n_targets=3, targets=['age', 'sex', 'scan_site_id'])
Train / test split#
We can pass the AFQDataset
class instance to scikit-learn’s
train_test_split()
function, just as we would with an array.
dataset_train, dataset_test = train_test_split(dataset, test_size=0.5)
Impute missing values#
Next we impute missing values using median imputation. We fit the imputer using the training set and then use it to transform both the training and test sets.
imputer = dataset_train.model_fit(SimpleImputer(strategy="median"))
dataset_train = dataset_train.model_transform(imputer)
dataset_test = dataset_test.model_transform(imputer)
Plot average bundle profiles by scan site#
Next we plot the mean bundle profiles in the test set by scanning site. The
plot_tract_profiles()
function takes as input an AFQDataset
and
returns matplotlib figures displaying the mean bundle profile for each bundle
and metric, optionally grouped by a categorical or continuous variable.
site_figs = plot_tract_profiles(
X=dataset_test,
group_by=dataset_test.classes["scan_site_id"][dataset_test.y[:, 2].astype(int)],
group_by_name="Site",
figsize=(14, 14),
)
0%| | 0/2 [00:00<?, ?it/s]
0%| | 0/24 [00:00<?, ?it/s]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
4%|▍ | 1/24 [00:04<01:32, 4.04s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
8%|▊ | 2/24 [00:08<01:29, 4.09s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
12%|█▎ | 3/24 [00:12<01:25, 4.06s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
17%|█▋ | 4/24 [00:16<01:20, 4.04s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
21%|██ | 5/24 [00:20<01:16, 4.04s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
25%|██▌ | 6/24 [00:24<01:12, 4.04s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
29%|██▉ | 7/24 [00:28<01:08, 4.03s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
33%|███▎ | 8/24 [00:32<01:04, 4.03s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
38%|███▊ | 9/24 [00:36<01:00, 4.03s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
42%|████▏ | 10/24 [00:40<00:56, 4.03s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
46%|████▌ | 11/24 [00:44<00:52, 4.04s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
50%|█████ | 12/24 [00:48<00:48, 4.04s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
54%|█████▍ | 13/24 [00:52<00:44, 4.04s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
58%|█████▊ | 14/24 [00:56<00:40, 4.03s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
62%|██████▎ | 15/24 [01:00<00:36, 4.03s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
67%|██████▋ | 16/24 [01:04<00:32, 4.02s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
71%|███████ | 17/24 [01:08<00:28, 4.01s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
75%|███████▌ | 18/24 [01:12<00:24, 4.08s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
79%|███████▉ | 19/24 [01:16<00:20, 4.06s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
83%|████████▎ | 20/24 [01:20<00:16, 4.02s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
88%|████████▊ | 21/24 [01:24<00:12, 4.00s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
92%|█████████▏| 22/24 [01:28<00:07, 3.99s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
96%|█████████▌| 23/24 [01:32<00:03, 3.98s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
100%|██████████| 24/24 [01:36<00:00, 3.97s/it]
100%|██████████| 24/24 [01:36<00:00, 4.02s/it]
50%|█████ | 1/2 [01:37<01:37, 97.62s/it]
0%| | 0/24 [00:00<?, ?it/s]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
4%|▍ | 1/24 [00:03<01:30, 3.94s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
8%|▊ | 2/24 [00:07<01:26, 3.94s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
12%|█▎ | 3/24 [00:11<01:22, 3.95s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
17%|█▋ | 4/24 [00:15<01:19, 3.96s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
21%|██ | 5/24 [00:19<01:15, 3.97s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
25%|██▌ | 6/24 [00:23<01:11, 3.97s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
29%|██▉ | 7/24 [00:27<01:07, 3.97s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
33%|███▎ | 8/24 [00:31<01:03, 3.97s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
38%|███▊ | 9/24 [00:35<00:59, 3.97s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
42%|████▏ | 10/24 [00:39<00:55, 3.96s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
46%|████▌ | 11/24 [00:43<00:51, 3.97s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
50%|█████ | 12/24 [00:47<00:47, 3.98s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
54%|█████▍ | 13/24 [00:51<00:43, 3.97s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
58%|█████▊ | 14/24 [00:55<00:39, 3.99s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
62%|██████▎ | 15/24 [00:59<00:35, 3.99s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
67%|██████▋ | 16/24 [01:03<00:31, 4.00s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
71%|███████ | 17/24 [01:07<00:28, 4.08s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
75%|███████▌ | 18/24 [01:11<00:24, 4.05s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
79%|███████▉ | 19/24 [01:15<00:20, 4.04s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
83%|████████▎ | 20/24 [01:19<00:15, 3.99s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
88%|████████▊ | 21/24 [01:23<00:11, 3.98s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
92%|█████████▏| 22/24 [01:27<00:07, 3.97s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
96%|█████████▌| 23/24 [01:31<00:03, 3.98s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
100%|██████████| 24/24 [01:35<00:00, 3.97s/it]
100%|██████████| 24/24 [01:35<00:00, 3.98s/it]
100%|██████████| 2/2 [03:14<00:00, 97.05s/it]
100%|██████████| 2/2 [03:14<00:00, 97.13s/it]
Harmonize the sites and replot#
We can see that there are substantial scan site differences in both the FA and MD profiles. Let’s use neuroComBat to harmonize the site differences and then replot the mean bundle profiles.
N.B. We use the excellent neurocombat_sklearn package to apply ComBat to
our data. We love this library, however, it is not fully compliant with the
scikit-learn transformer API, so we cannot use the
AFQDataset.model_fit_transform()
method to apply this transformer to our
dataset. No problem! We can simply copy the unharmonized dataset into a new
variable and then overwrite the features of the new dataset with the ComBat
output.
Lastly, we replot the mean bundle profiles and confirm that ComBat did its job.
# Fit the ComBat transformer to the training set
combat = CombatModel()
combat.fit(
dataset_train.X,
dataset_train.y[:, 2][:, np.newaxis],
dataset_train.y[:, 1][:, np.newaxis],
dataset_train.y[:, 0][:, np.newaxis],
)
# And then transform a copy of the test set
harmonized_test = dataset_test.copy()
harmonized_test.X = combat.transform(
dataset_test.X,
dataset_test.y[:, 2][:, np.newaxis],
dataset_test.y[:, 1][:, np.newaxis],
dataset_test.y[:, 0][:, np.newaxis],
)
site_figs = plot_tract_profiles(
X=harmonized_test,
group_by=harmonized_test.classes["scan_site_id"][
harmonized_test.y[:, 2].astype(int)
],
group_by_name="Site",
figsize=(14, 14),
)
/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/sklearn/preprocessing/_encoders.py:828: FutureWarning: `sparse` was renamed to `sparse_output` in version 1.2 and will be removed in 1.4. `sparse_output` is ignored unless you leave `sparse` to its default value.
warnings.warn(
/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/sklearn/preprocessing/_encoders.py:828: FutureWarning: `sparse` was renamed to `sparse_output` in version 1.2 and will be removed in 1.4. `sparse_output` is ignored unless you leave `sparse` to its default value.
warnings.warn(
0%| | 0/2 [00:00<?, ?it/s]
0%| | 0/24 [00:00<?, ?it/s]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
4%|▍ | 1/24 [00:03<01:30, 3.93s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
8%|▊ | 2/24 [00:07<01:26, 3.94s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
12%|█▎ | 3/24 [00:11<01:22, 3.94s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
17%|█▋ | 4/24 [00:15<01:18, 3.94s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
21%|██ | 5/24 [00:19<01:14, 3.95s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
25%|██▌ | 6/24 [00:23<01:11, 3.96s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
29%|██▉ | 7/24 [00:27<01:07, 3.95s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
33%|███▎ | 8/24 [00:31<01:03, 3.95s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
38%|███▊ | 9/24 [00:35<00:59, 3.95s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
42%|████▏ | 10/24 [00:39<00:55, 3.95s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
46%|████▌ | 11/24 [00:43<00:51, 3.97s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
50%|█████ | 12/24 [00:47<00:47, 3.97s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
54%|█████▍ | 13/24 [00:51<00:43, 3.98s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
58%|█████▊ | 14/24 [00:55<00:39, 3.96s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
62%|██████▎ | 15/24 [00:59<00:35, 3.95s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
67%|██████▋ | 16/24 [01:03<00:31, 3.95s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
71%|███████ | 17/24 [01:07<00:28, 4.05s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
75%|███████▌ | 18/24 [01:11<00:24, 4.03s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
79%|███████▉ | 19/24 [01:15<00:20, 4.01s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
83%|████████▎ | 20/24 [01:19<00:16, 4.01s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
88%|████████▊ | 21/24 [01:23<00:12, 4.02s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
92%|█████████▏| 22/24 [01:27<00:08, 4.00s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
96%|█████████▌| 23/24 [01:31<00:03, 3.99s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
100%|██████████| 24/24 [01:35<00:00, 3.94s/it]
100%|██████████| 24/24 [01:35<00:00, 3.97s/it]
50%|█████ | 1/2 [01:36<01:36, 96.36s/it]
0%| | 0/24 [00:00<?, ?it/s]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
4%|▍ | 1/24 [00:03<01:29, 3.90s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
8%|▊ | 2/24 [00:07<01:26, 3.93s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
12%|█▎ | 3/24 [00:11<01:22, 3.91s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
17%|█▋ | 4/24 [00:15<01:17, 3.87s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
21%|██ | 5/24 [00:19<01:13, 3.87s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
25%|██▌ | 6/24 [00:23<01:09, 3.86s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
29%|██▉ | 7/24 [00:27<01:05, 3.86s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
33%|███▎ | 8/24 [00:30<01:01, 3.86s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
38%|███▊ | 9/24 [00:34<00:57, 3.86s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
42%|████▏ | 10/24 [00:38<00:53, 3.84s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
46%|████▌ | 11/24 [00:42<00:49, 3.84s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
50%|█████ | 12/24 [00:46<00:46, 3.85s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
54%|█████▍ | 13/24 [00:50<00:42, 3.84s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
58%|█████▊ | 14/24 [00:54<00:38, 3.85s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
62%|██████▎ | 15/24 [00:57<00:34, 3.85s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
67%|██████▋ | 16/24 [01:01<00:30, 3.85s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
71%|███████ | 17/24 [01:05<00:26, 3.85s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
75%|███████▌ | 18/24 [01:09<00:23, 3.85s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
79%|███████▉ | 19/24 [01:13<00:19, 3.84s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
83%|████████▎ | 20/24 [01:17<00:15, 3.85s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
88%|████████▊ | 21/24 [01:20<00:11, 3.84s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
92%|█████████▏| 22/24 [01:24<00:07, 3.85s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
96%|█████████▌| 23/24 [01:28<00:03, 3.84s/it]/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/afqinsight/plot.py:269: FutureWarning:
The `ci` parameter is deprecated. Use `errorbar=('ci', 95.0)` for the same effect.
_ = sns.lineplot(
100%|██████████| 24/24 [01:32<00:00, 3.85s/it]
100%|██████████| 24/24 [01:32<00:00, 3.85s/it]
100%|██████████| 2/2 [03:09<00:00, 94.67s/it]
100%|██████████| 2/2 [03:09<00:00, 94.93s/it]
Total running time of the script: (6 minutes 53.149 seconds)