# What do the files show once a public EEG dataset is checked end to end?

Data quality and reproducibility · three public EEG datasets, read from their files

Shin2022 is the worked case: its runs, duplicates and windows counted from the files, then a participant-held-out check beside a control that reads only trial duration. eldBETA and ZuCo 2.0 are status cases, with what their files showed and no score.

- [Reviewed aggregate JSON ↓](https://bci.report/data/data-quality-update.json)
- 

Measured on: [Shin2022 · closed-loop motor imagery EEG (Figshare 20383716)](https://bci.report/datasets/shin2022/)

## Short answer

Shin2022’s files repeat some runs byte for byte, and some feedback windows end early or carry a flat channel; on the windows that remain, a participant-held-out check put alpha and beta band power at **52.59%** balanced accuracy, beside **52.33%** for a control that reads only how long each trial lasted and **50.00%** for the training prior. eldBETA’s raw recordings for one participant pass every check, and with one person the score is held. In one converted ZuCo 2.0 run some fixation times fall outside the recording, so no EEG–text score is computed.

Worked case · Shin2022 · closed-loop cursor control, dry electrodes

## What survives a participant-held-out check?

Right-versus-left instructed target in the first second of cursor feedback, for a participant held out entirely: each of the 10 participants is left out once, and a ridge fitted on the other 9 scores their windows. On the signal-valid windows counted below, alpha and beta band power reached 52.59% participant-mean balanced accuracy (51.51%–53.66%). A control that reads only each trial’s full feedback duration reached 52.33% (50.87%–54.02%). The training prior gives 50.00%.

Participant-mean balanced accuracy: the 10 held-out participants’ balanced accuracies averaged with equal weight, with its 95% participant-bootstrap interval. The pooled value counts every signal-valid window together: a secondary figure, without an interval.

| Arm | Participant-mean balanced accuracy | Pooled (secondary) | What it reads |
| --- | --- | --- | --- |
| Participant-weighted training prior — reference | 50.00% (95% interval 50.00%–50.00%) | 50.00% | No input: always the larger training class, with each training participant weighted equally. |
| Normalized spectral ridge — EEG | 52.59% (95% interval 51.51%–53.66%) | 52.55% | 32 log band powers relative to each channel’s 1–40 Hz power: alpha (8–12 Hz) and beta (13–30 Hz) on each of the 16 stored channels; ridge with alpha 1.0, no tuning. |
| Retrospective control |  |  |  |
| Retrospective duration control | 52.33% (95% interval 50.87%–54.02%) | 52.15% | One value: the log of the full feedback duration; ridge with alpha 1.0. |

The scores cover the signal-valid windows only: feedback lasting at least one second, and a first second that is finite with positive 1–40 Hz power on all 16 channels. The label is the instructed target in closed-loop cursor recordings, which carry visual, controller and behaviour confounds. The duration control reads the whole feedback duration, known only after the trial ends, so it checks a confound; the spectral arm sits close to it, and no test between arms was run. Ten participants, one recording day each; the intervals are descriptive.

### From the runs in the release to the scored windows

Each count is read from a hash-pinned record (the label and duplicate audit, the signal census review or the scored export), and each step follows from the one above it. Right and left are the instructed targets.

| Step | Count | Right · left | What the files show |
| --- | --- | --- | --- |
| Runs in the release | 110 | — | Runs in the release’s human recordings; the simulated branch is excluded and adds no participant. |
| Byte-identical pairs | 9 | — | Pairs of runs that are byte-identical: each pair sits within one participant and joins a BW30 run and an NT0 run. |
| Distinct runs | 101 | — | Distinct runs once each byte-identical pair counts once. They hold 2,412 closed feedback windows. |
| Incomplete run — held | 1 | — | One run ends inside a feedback window; it is held whole. |
| Complete runs | 100 | 12 · 12 per run | Complete distinct runs, each with twelve closed feedback windows per target. They cover all 10 participants. |
| Closed feedback windows | 2,400 | 1,200 · 1,200 | Closed feedback windows in the complete runs. |
| Shorter than one second — excluded | 167 | 76 · 91 | Feedback ended within the first second, so the window holds less than the one second the arms read. |
| At least one second | 2,233 | 1,124 · 1,109 | Windows with at least one second of feedback. |
| A flat channel in the first second — excluded | 13 | 7 · 6 | Windows whose first second is constant on at least one channel, which leaves that channel no 1–40 Hz power; the rule was frozen after a score-free census and before any fit. Flat on all channels: 8; on some: 5. Windows with non-finite input: 0. |
| Signal-valid | 2,220 | 1,117 · 1,103 | Signal-valid windows: the population all three arms are scored on. That is 92.5% of the closed windows and 99.42% of those lasting at least one second; all 10 participants keep windows of both targets. |

MOABB’s dataset summary gives a trial length of 6 s: that is the feedback cap (MaxFeedbackDuration) set in the header of each of the 110 runs read. MOABB’s trial and run counts for Shin2022 are findings on [its row in the open register](https://bci.report/datasets/register/#r-site-shin2022).

### How it was checked

- **Held**: The first, strict protocol over all duration-eligible windows met a flat signal interval at its zero-power gate and stopped before any model was fitted; that attempt stays failed and held.
- **Pass**: A census read the signal of every duration-eligible window, with no model and no score.
- **Pass**: A second protocol with the signal-validity rule was frozen after the census and before any fit.
- **Pass**: Predictions were sealed before the audit; metrics were computed only after it.
- **Pass**: An independent audit rebuilt every metric from the sealed predictions.
- **Pass**: An independent release review passed the aggregate values and the coverage for publication.

### Held or unestablished

- The paper pairs NT0 with BW60, while the byte-identical runs pair BW30 with NT0; which recording NT0 stands for is unresolved.
- Comparisons between the BW, CV and NT control-parameter conditions are held: one BW60 header records a 5-second buffer, and the NT and CV headers record 30-second defaults.
- The incomplete run is held whole; using its closed windows would need its own frozen rule.
- A task over every closed window, whatever its duration, would need its own frozen protocol.
- The full scalp montage, the display latency, the absolute hardware calibration and any overlap with pretraining corpora are unestablished.
- The physical cause of the flat intervals is unknown.
- Exact deduplication leaves trial independence unestablished; every run of a participant stays in that participant’s fold.

### Source, licence and consent

### Shin2022 · Data from: Closed-loop motor imagery EEG simulation for brain-computer interfaces (Shin, Suma and He)

Closed-loop right/left cursor control with visual feedback; the label is the instructed target. Figshare record 20383716, version 1, the only version the record lists.

Hyonyoung Shin, Daniel Suma and Bin He · Data from: Closed-loop motor imagery EEG simulation for brain-computer interfaces, figshare, doi:10.6084/m9.figshare.20383716.v1, CC BY 4.0. Paper: Hyonyoung Shin, Daniel Suma and Bin He, Closed-loop motor imagery EEG simulation for brain-computer interfaces, Frontiers in Human Neuroscience 16:951591 (2022), doi:10.3389/fnhum.2022.951591. Funding: NIH AT009263. Modifications: canonical deduplication of the byte-identical runs, first-second window selection and signal-validity filtering. Derived analysis by BCI Report; not endorsed by the authors.

The simulator code is a separate work linked from the data record; the data licence covers the data record. [Simulator code ↗](https://github.com/mcvain/bci-simulator)

[Dataset record ↗](https://doi.org/10.6084/m9.figshare.20383716.v1) · [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) · [Dataset page →](https://bci.report/datasets/shin2022/)

### Ethics and consent, as the source states them

**Ethics approval:** The study was reviewed and approved by the Institutional Review Board of Carnegie Mellon University; the paper gives no approval number.

**Informed consent:** The participants gave written informed consent to take part in the study.

The data record also states: “All identifying information have been removed.” “Please cite the above paper if you use any data included in this dataset.”

[Paper, with its ethics statement ↗](https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2022.951591/full)

Status case · eldBETA · SSVEP selection by older adults

## One participant’s raw recordings, end to end

The raw-recording route runs end to end on one participant’s 7 blocks of 9 targets: all 63 trials pass the reader and the quality checks, none dropped. The continuous recordings hold 248,320 non-finite values outside the analysed windows, left as recorded, and none inside them. With one person every score is that person’s, so the accuracy is held.

### Held or unestablished

- The derived MATLAB epochs were not used: their amplitude scale, resampling and exact endpoints are unestablished.
- The raw recordings were read with the header calibration applied; the recorded unit code has no independent audit.
- The non-finite values in the continuous recordings are neither located nor explained.
- The record lists 100 participants; the other published participants are neither acquired nor evaluated.

eldBETA is in SingLEM’s published pretraining list, Table I of the SingLEM paper; BETA, a separate database, is absent from it, as SingLEM’s exposure row on BETA shows. [SingLEM paper, Table I ↗](https://arxiv.org/abs/2509.17920v3) · [SingLEM’s pretraining-exposure rows →](https://bci.report/methods/singlem/#pretraining-exposure)

**eldBETA · one-person score · Status only.** [Its entry in the holds register →](https://bci.report/releases/#hold-eldbeta-one-participant-score)

### Source, licence and consent

### eldBETA: A Large Eldercare-oriented Benchmark Database of SSVEP-BCI for the Aging Population (Liu, Wang, Gao and Chen)

Nine-target SSVEP selection by older adults, recorded in seven blocks per person. Figshare record 18032669, version 3, the latest of the 3 versions the record lists.

Bingchuan Liu, Yijun Wang, Xiaorong Gao and Xiaogang Chen · eldBETA: A Large Eldercare-oriented Benchmark Database of SSVEP-BCI for the Aging Population, figshare, doi:10.6084/m9.figshare.18032669.v3, CC BY 4.0. Data descriptor: Scientific Data, doi:10.1038/s41597-022-01372-9. Funding: National Natural Science Foundation of China (No. 62171473). Modifications: decoding of the original recording format with the header calibration, and fixed channel and window selection; no score from these data is published. Derived analysis by BCI Report; not endorsed by the authors.

[Dataset record ↗](https://doi.org/10.6084/m9.figshare.18032669.v3) · [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

### Ethics and consent, as the source states them

**Ethics approval:** The protocol followed the Declaration of Helsinki and was approved by the institutional review board of Tsinghua University, approval number 20210032.

**Informed consent:** The participants were familiarised with the protocol and gave full written consent before the experiment.

The data descriptor also states: “The eldBETA database is open-access for research and can be downloaded from the website”

[Data descriptor, with its ethics statement ↗](https://www.nature.com/articles/s41597-022-01372-9)

Status case · ZuCo 2.0 on NEMAR · no score

## A timing hold on one converted run

In one converted run of ZuCo 2.0 on NEMAR, some fixation times fall outside the recording. No EEG–text score is computed until the converter’s timing rules and a source-checksummed fixture are available. The hold covers that one converted run; no timing was corrected and no row removed.

### The hold clears with

- the exact converter revision, configuration and environment, with its rules for casts, time origin and boundaries
- a source-checksummed timing fixture with the original start and end timestamps and the recording’s start and end clocks
- the canonical mapping of triggers, sentences and runs

No consent or ethics statement for this dataset has been read yet. [Source and stated terms ↓](https://bci.report/topics/data-quality/#zuco2-source)

**ZuCo on NEMAR · converted-run timing · No score.** [Its entry in the holds register →](https://bci.report/releases/#hold-zuco2-converted-run-timing)

## Each case in the open register

Each case has a row in the open register, with the furthest step checked and the gaps that remain.

- Shin2022: [Row in the open register →](https://bci.report/datasets/register/#r-site-shin2022)
- eldBETA: [Row in the open register →](https://bci.report/datasets/register/#r-site-eldbeta)
- ZuCo 2.0: [Row in the open register →](https://bci.report/datasets/register/#r-site-zuco2)

Methods & limits

## Methods and limits

### Shin2022: how the check was run

Leave one participant out: 10 folds, each fitted on 9 participants. The window is the first second of feedback, read from its recorded start, never extended or padded: 250 samples at 250 Hz from a g.tec g.Nautilus RESEARCH (16 channel) headset with dry electrodes (EEG1–EEG16 in stored order; the headers carry no channel names). The label is the instructed target as each run stores it (TargetCode); the trial’s outcome is never read.

### The intervals

Each interval resamples the 10 participants 10,000 times, with one resampling index for all three arms, and reads its percentiles by linear interpolation.

### What is not published

- Participant rows of any kind: identifiers, run and file names, fold assignments, features, fitted models, decision values and predictions; raw signals from any of the three datasets.
- Per-participant figures from the Shin2022 evidence: window counts per participant and their ranges, and which participants had flat windows.
- The Shin2022 feedback-duration distribution (quantiles and mean) and the per-channel counts of the signal census.
- eldBETA’s one-participant accuracy and its fixed first-frequency comparator.
- ZuCo 2.0’s converted-run code, row-level timing values and marker counts; no ZuCo 2.0 data are redistributed.
- MOABB’s catalog trial and run counts for Shin2022: register content, in the open register file.
- Compute times, storage sizes, hosts and storage locations.
- The research exports, audits, reviews, protocols and handoffs themselves: each is pinned by SHA-256 in the release manifest and checked by the export, never published, because several carry private storage paths.

### ZuCo 2.0: source and stated terms

NEMAR accession nm000279, version 1.0.0, doi:10.82901/nemar.nm000279.v1.0.0. Licence as stated: CC-BY-4.0, in the dataset_description.json of that version. Not yet verified: the original terms on OSF; terms for the text excerpts the stimuli quote. [Dataset record ↗](https://data.nemar.org/nm000279/v1.0.0/)

Hollenstein, N., Troendle, M., Zhang, C., & Langer, N. (2020). ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation. LREC 2020.

Data source: [data-quality-update.json](https://bci.report/data/data-quality-update.json) · schema bci-report-data-quality-update-v1.

## Related

- [How each source’s licence, consent and ethics statement was read →](https://bci.report/data-use/#sources-data-quality)
- Small pilot studies, each with its own limits: [What can a small pilot show, and what can it not?](https://bci.report/topics/pilot-studies/)
- [The open register: every EEG dataset we know of, and how far we checked it →](https://bci.report/datasets/register/)

Keep exploring

Transfer

- [— Sensor transfer **Dry vs. wet electrodes**](https://bci.report/topics/dry-vs-wet/)
- [— Context transfer **Screen to VR**](https://bci.report/topics/screen-to-vr/)
- [— Montage **Fewer electrodes**](https://bci.report/topics/fewer-electrodes/)
- [— Motion robustness **On the move**](https://bci.report/topics/on-the-move/)
- [— Session transfer · same person **Later sessions**](https://bci.report/topics/later-sessions/)

Adapting models

- [— Calibration budget **How much calibration?**](https://bci.report/topics/calibration-budget/)
- [— Model adaptation **Which part to update?**](https://bci.report/topics/model-adaptation/)
- [— Representation controls **Does pretraining help?**](https://bci.report/topics/does-pretraining-help/)
- [— Shared encoder · route 2 **One model, several questions**](https://bci.report/topics/shared-encoder/)
- [— Questions in language · route 3 **Questions in language**](https://bci.report/topics/questions-in-language/)
- [— Route 3 follow-ups **New SSVEP frequencies**](https://bci.report/topics/new-ssvep-frequencies/)

Reliability & clinical

- [— Abstention · decision research **When not to act**](https://bci.report/topics/when-not-to-act/)
- [— Sleep staging · simple baselines **Sleep-stage balance**](https://bci.report/topics/sleep-staging/)
- [— Clinical research **Clinical groups**](https://bci.report/topics/clinical-groups/)
- [— Data quality and reproducibility **Checked end to end**](https://bci.report/topics/data-quality/)

Other signals & pilots

- [— Other signals · human and macaque **Beyond scalp EEG**](https://bci.report/topics/beyond-scalp-eeg/)
- [— Pilot studies · small cohorts **Pilot studies**](https://bci.report/topics/pilot-studies/)

[All questions and the transfer-coverage map →](https://bci.report/topics/)

## Cite this page

BCI Report (2026). *What do the files show once a public EEG dataset is checked end to end?* https://bci.report/topics/data-quality/

Figures from release [`data-quality-update-20261020`](https://bci.report/releases/#data-quality-update-20261020) (2026-10-20). Cite the upstream datasets as well: their credits are on this page.

Every release is archived on Zenodo: [doi:10.5281/zenodo.23123296](https://doi.org/10.5281/zenodo.23123296). [BibTeX for the site and its releases →](https://bci.report/api/#cite-heading) · [CITATION.cff ↗](https://raw.githubusercontent.com/twu3202/bci-report/main/CITATION.cff)

---
Markdown copy of https://bci.report/topics/data-quality/, generated from the published page. Figures are aggregate results; terms of use: https://bci.report/data-use/