Data quality and reproducibility · three public EEG datasets, read from their files
What do the files show once a public EEG dataset is checked end to end?
Shin2022 is the worked case: its runs, duplicates and windows counted from the files, then a participant-held-out check beside a control that reads only trial duration. eldBETA and ZuCo 2.0 are status cases, with what their files showed and no score.
Shin2022’s files repeat some runs byte for byte, and some feedback windows end early or carry a flat channel; on the windows that remain, a participant-held-out check put alpha and beta band power at 52.59% balanced accuracy, beside 52.33% for a control that reads only how long each trial lasted and 50.00% for the training prior. eldBETA’s raw recordings for one participant pass every check, and with one person the score is held. In one converted ZuCo 2.0 run some fixation times fall outside the recording, so no EEG–text score is computed.
Worked case · Shin2022 · closed-loop cursor control, dry electrodes
What survives a participant-held-out check?
Right-versus-left instructed target in the first second of cursor feedback, for a participant held out entirely: each of the 10 participants is left out once, and a ridge fitted on the other 9 scores their windows. On the signal-valid windows counted below, alpha and beta band power reached 52.59% participant-mean balanced accuracy (51.51%–53.66%). A control that reads only each trial’s full feedback duration reached 52.33% (50.87%–54.02%). The training prior gives 50.00%.
Participant-mean balanced accuracy: the 10 held-out participants’ balanced accuracies averaged with equal weight, with its 95% participant-bootstrap interval. The pooled value counts every signal-valid window together: a secondary figure, without an interval.
Arm
Participant-mean balanced accuracy
Pooled (secondary)
What it reads
Participant-weighted training priorreference
50.00%95% interval 50.00%–50.00%
50.00%
No input: always the larger training class, with each training participant weighted equally.
Normalized spectral ridgeEEG
52.59%95% interval 51.51%–53.66%
52.55%
32 log band powers relative to each channel’s 1–40 Hz power: alpha (8–12 Hz) and beta (13–30 Hz) on each of the 16 stored channels; ridge with alpha 1.0, no tuning.
Retrospective control
Retrospective duration control
52.33%95% interval 50.87%–54.02%
52.15%
One value: the log of the full feedback duration; ridge with alpha 1.0.
The scores cover the signal-valid windows only: feedback lasting at least one second, and a first second that is finite with positive 1–40 Hz power on all 16 channels. The label is the instructed target in closed-loop cursor recordings, which carry visual, controller and behaviour confounds. The duration control reads the whole feedback duration, known only after the trial ends, so it checks a confound; the spectral arm sits close to it, and no test between arms was run. Ten participants, one recording day each; the intervals are descriptive.
From the runs in the release to the scored windows
Each count is read from a hash-pinned record (the label and duplicate audit, the signal census review or the scored export), and each step follows from the one above it. Right and left are the instructed targets.
Step
Count
Right · left
What the files show
Runs in the release
110
—
Runs in the release’s human recordings; the simulated branch is excluded and adds no participant.
Byte-identical pairs
9
—
Pairs of runs that are byte-identical: each pair sits within one participant and joins a BW30 run and an NT0 run.
Distinct runs
101
—
Distinct runs once each byte-identical pair counts once. They hold 2,412 closed feedback windows.
Incomplete runheld
1
—
One run ends inside a feedback window; it is held whole.
Complete runs
100
12 · 12 per run
Complete distinct runs, each with twelve closed feedback windows per target. They cover all 10 participants.
Closed feedback windows
2,400
1,200 · 1,200
Closed feedback windows in the complete runs.
Shorter than one secondexcluded
167
76 · 91
Feedback ended within the first second, so the window holds less than the one second the arms read.
At least one second
2,233
1,124 · 1,109
Windows with at least one second of feedback.
A flat channel in the first secondexcluded
13
7 · 6
Windows whose first second is constant on at least one channel, which leaves that channel no 1–40 Hz power; the rule was frozen after a score-free census and before any fit. Flat on all channels: 8; on some: 5. Windows with non-finite input: 0.
Signal-valid
2,220
1,117 · 1,103
Signal-valid windows: the population all three arms are scored on. That is 92.5% of the closed windows and 99.42% of those lasting at least one second; all 10 participants keep windows of both targets.
MOABB’s dataset summary gives a trial length of 6 s: that is the feedback cap (MaxFeedbackDuration) set in the header of each of the 110 runs read. MOABB’s trial and run counts for Shin2022 are findings on its row in the open register.
How it was checked
Held: The first, strict protocol over all duration-eligible windows met a flat signal interval at its zero-power gate and stopped before any model was fitted; that attempt stays failed and held.
Pass: A census read the signal of every duration-eligible window, with no model and no score.
Pass: A second protocol with the signal-validity rule was frozen after the census and before any fit.
Pass: Predictions were sealed before the audit; metrics were computed only after it.
Pass: An independent audit rebuilt every metric from the sealed predictions.
Pass: An independent release review passed the aggregate values and the coverage for publication.
Held or unestablished
The paper pairs NT0 with BW60, while the byte-identical runs pair BW30 with NT0; which recording NT0 stands for is unresolved.
Comparisons between the BW, CV and NT control-parameter conditions are held: one BW60 header records a 5-second buffer, and the NT and CV headers record 30-second defaults.
The incomplete run is held whole; using its closed windows would need its own frozen rule.
A task over every closed window, whatever its duration, would need its own frozen protocol.
The full scalp montage, the display latency, the absolute hardware calibration and any overlap with pretraining corpora are unestablished.
The physical cause of the flat intervals is unknown.
Exact deduplication leaves trial independence unestablished; every run of a participant stays in that participant’s fold.
Source, licence and consent
Shin2022 · Data from: Closed-loop motor imagery EEG simulation for brain-computer interfaces (Shin, Suma and He)
Closed-loop right/left cursor control with visual feedback; the label is the instructed target. Figshare record 20383716, version 1, the only version the record lists.
Hyonyoung Shin, Daniel Suma and Bin He · Data from: Closed-loop motor imagery EEG simulation for brain-computer interfaces, figshare, doi:10.6084/m9.figshare.20383716.v1, CC BY 4.0. Paper: Hyonyoung Shin, Daniel Suma and Bin He, Closed-loop motor imagery EEG simulation for brain-computer interfaces, Frontiers in Human Neuroscience 16:951591 (2022), doi:10.3389/fnhum.2022.951591. Funding: NIH AT009263. Modifications: canonical deduplication of the byte-identical runs, first-second window selection and signal-validity filtering. Derived analysis by BCI Report; not endorsed by the authors.
The simulator code is a separate work linked from the data record; the data licence covers the data record. Simulator code ↗
Ethics approval: The study was reviewed and approved by the Institutional Review Board of Carnegie Mellon University; the paper gives no approval number.
Informed consent: The participants gave written informed consent to take part in the study.
The data record also states: “All identifying information have been removed.” “Please cite the above paper if you use any data included in this dataset.”
Status case · eldBETA · SSVEP selection by older adults
One participant’s raw recordings, end to end
The raw-recording route runs end to end on one participant’s 7 blocks of 9 targets: all 63 trials pass the reader and the quality checks, none dropped. The continuous recordings hold 248,320 non-finite values outside the analysed windows, left as recorded, and none inside them. With one person every score is that person’s, so the accuracy is held.
Held or unestablished
The derived MATLAB epochs were not used: their amplitude scale, resampling and exact endpoints are unestablished.
The raw recordings were read with the header calibration applied; the recorded unit code has no independent audit.
The non-finite values in the continuous recordings are neither located nor explained.
The record lists 100 participants; the other published participants are neither acquired nor evaluated.
eldBETA: A Large Eldercare-oriented Benchmark Database of SSVEP-BCI for the Aging Population (Liu, Wang, Gao and Chen)
Nine-target SSVEP selection by older adults, recorded in seven blocks per person. Figshare record 18032669, version 3, the latest of the 3 versions the record lists.
Bingchuan Liu, Yijun Wang, Xiaorong Gao and Xiaogang Chen · eldBETA: A Large Eldercare-oriented Benchmark Database of SSVEP-BCI for the Aging Population, figshare, doi:10.6084/m9.figshare.18032669.v3, CC BY 4.0. Data descriptor: Scientific Data, doi:10.1038/s41597-022-01372-9. Funding: National Natural Science Foundation of China (No. 62171473). Modifications: decoding of the original recording format with the header calibration, and fixed channel and window selection; no score from these data is published. Derived analysis by BCI Report; not endorsed by the authors.
Ethics approval: The protocol followed the Declaration of Helsinki and was approved by the institutional review board of Tsinghua University, approval number 20210032.
Informed consent: The participants were familiarised with the protocol and gave full written consent before the experiment.
The data descriptor also states: “The eldBETA database is open-access for research and can be downloaded from the website”
In one converted run of ZuCo 2.0 on NEMAR, some fixation times fall outside the recording. No EEG–text score is computed until the converter’s timing rules and a source-checksummed fixture are available. The hold covers that one converted run; no timing was corrected and no row removed.
The hold clears with
the exact converter revision, configuration and environment, with its rules for casts, time origin and boundaries
a source-checksummed timing fixture with the original start and end timestamps and the recording’s start and end clocks
the canonical mapping of triggers, sentences and runs
Leave one participant out: 10 folds, each fitted on 9 participants. The window is the first second of feedback, read from its recorded start, never extended or padded: 250 samples at 250 Hz from a g.tec g.Nautilus RESEARCH (16 channel) headset with dry electrodes (EEG1–EEG16 in stored order; the headers carry no channel names). The label is the instructed target as each run stores it (TargetCode); the trial’s outcome is never read.
The intervals
Each interval resamples the 10 participants 10,000 times, with one resampling index for all three arms, and reads its percentiles by linear interpolation.
What is not published
Participant rows of any kind: identifiers, run and file names, fold assignments, features, fitted models, decision values and predictions; raw signals from any of the three datasets.
Per-participant figures from the Shin2022 evidence: window counts per participant and their ranges, and which participants had flat windows.
The Shin2022 feedback-duration distribution (quantiles and mean) and the per-channel counts of the signal census.
eldBETA’s one-participant accuracy and its fixed first-frequency comparator.
ZuCo 2.0’s converted-run code, row-level timing values and marker counts; no ZuCo 2.0 data are redistributed.
MOABB’s catalog trial and run counts for Shin2022: register content, in the open register file.
Compute times, storage sizes, hosts and storage locations.
The research exports, audits, reviews, protocols and handoffs themselves: each is pinned by SHA-256 in the release manifest and checked by the export, never published, because several carry private storage paths.
ZuCo 2.0: source and stated terms
NEMAR accession nm000279, version 1.0.0, doi:10.82901/nemar.nm000279.v1.0.0. Licence as stated: CC-BY-4.0, in the dataset_description.json of that version. Not yet verified: the original terms on OSF; terms for the text excerpts the stimuli quote. Dataset record ↗
Hollenstein, N., Troendle, M., Zhang, C., & Langer, N. (2020). ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation. LREC 2020.
BCI Report