On this site's idle protocol, EEGNet made no false activation in 60 idle trials — and detected 1 of 60 commands. 2 of its 4 people ended up with a threshold that rejects every command. A false-activation rate read on its own rewards a decoder for doing nothing. This page keeps the two numbers together, then sets out a Jev-style research plan: what would have to be measured to say when a decoder should act at all.
Thresholds are chosen on calibration data only, and a threshold may reject every command. For 2 of 4 people it did. Someone who never triggers a command never triggers one by mistake either, so the number of people who always abstained is printed beside every method.
Both numbers, every method
Command detection against idle false activation
4 people; 60 idle and 60 command test trials in total; a command counts if it is detected within 3 seconds of its cue. Sorted by command detection.
Counts over the whole test set, with the percentage beneath. “Always abstained” counts people whose calibrated threshold rejected every command.
Method
Training
Commands detected ≤3 s
Idle false activations
Always abstained
ShallowFBCSPNet
Scratch · 20 epochs
26 / 6043.3%43.3% of commands detected
1 / 601.7%
1of 4 people
CSP+LDA
Supervised fit
14 / 6023.3%23.3% of commands detected
0 / 600.0%
1of 4 people
LaBraM
Frozen encoder + linear head
11 / 6018.3%18.3% of commands detected
2 / 603.3%
0of 4 people
CBraMod
Frozen encoder + linear head
11 / 6018.3%18.3% of commands detected
1 / 601.7%
1of 4 people
Deep4Net
Scratch · 20 epochs
2 / 603.3%3.3% of commands detected
1 / 601.7%
1of 4 people
EEGNet
Scratch · 20 epochs
1 / 601.7%1.7% of commands detected
0 / 600.0%
2of 4 people
Explicit non-control · four-person pilot
Does it accept a window when nobody intends a command?
A second protocol records the states between commands on purpose: looking at a central image with the flicker off, resting on a white wall, and looking at the centre while the targets around it flicker. A fixed CCA decoder with per-person thresholds, fitted on calibration trials only, decides whether to accept each 1.5-second window.
When a command was intended · Accuracy among accepted windows is never shown alone: rejecting hard windows raises it while refusing commands the person meant.
State
Windows
Frequency recognised, before rejection
152 / 19279.2%
Accepted (coverage)
130 / 19267.7%
Accepted and correct (end to end)
121 / 19263.0%
Correct among accepted
121 / 13093.1%
When no command was intended
State
Accepted by mistake
Central image, flicker off
9 / 4818.8%
Looking at a white wall, resting
20 / 4841.7%
Central image while the surrounding targets flicker
11 / 9611.5%
Four people, offline, window by window. These are rates per tested window, not false activations per hour, and the source states no physical amplitude unit.
Jing Zhao, Qian Zhang, Xinrui Wang, Xueshuo Liu, Jiaxin Li, Fengjie Fan, Zhenhu Liang and Xiaoli Li · An EEG dataset for studying asynchronous steady-state visual evoked potential (SSVEP) based brain computer interfaces, Brain-Apparatus Communication 3(1) (2024), doi:10.1080/27706710.2024.2418650; data at doi:10.6084/m9.figshare.24906300.v3.
Before the experiment, every participant signed written informed consent; the study was reviewed and approved by the ethics committee of Qinhuangdao First Hospital. This site uses four of the release’s twenty-four people and publishes pooled counts only.
A small, cue-gated protocol. It shows why both numbers are needed; it cannot say how a decoder would behave in continuous use.
4 people
At this size an average stays close to its parts: a published rate can be turned back into a count, and a count narrowed to one of two people. The arithmetic is written out on the data-use page.
Three minutes of idle
The test holds 180 seconds of idle. That cannot estimate how often a decoder would fire by mistake over an hour of real, unprompted rest.
Cue-gated, not always-on
Detection restarts at every cue, so the decoder always knows a decision window has opened. A system that must notice intent on its own faces a harder problem than this protocol measures.
No probability quality yet
These are thresholded decisions. How well each method’s probabilities are calibrated — the basis for deciding when to defer — has not been measured on this protocol.
ds005342 · Seated motor imagery · cue-gated replay
OpenNeuro ds005342 contributors · version 1.0.3, doi:10.18112/openneuro.ds005342.v1.0.3; associated study doi:10.3389/fninf.2022.961089. Original author credits are retained at the linked source.
The source README reports voluntary signed consent and approval by the ethics committee of Universidad Antonio Nariño. Only cohort aggregates appear here.
A decoder has more choices than a class label. It can act, wait for more evidence, or ask for calibration. How well a model makes those choices can be measured, and nothing on this site measures it yet. This section is a plan: it has no results, and it borrows none.
Status: proposal. No experiment in this section has been run.
“Jev-style” names an interface pattern: encode the signal once, then answer several explicit, typed questions about it. This is not an integration with Jev, not a Jev model that reads EEG, and not a model BCI Report has trained. There are no results here yet.
First · Not run
Reliable decisions
Compare a fixed threshold, a simple statistical policy and a learned reject option on the same data. Report error at matched coverage, acceptance per class, and probability quality — Brier score, log loss and reliability — with calibration data kept apart from test data. Recalibration is a separate experiment, with its label cost counted.
A risk–coverage curve is defined only where something is accepted; at zero coverage there is no error rate, not a zero one.
Then · Not run
One representation, several questions
On the motor-imagery and sleep protocols separately, compare independent task models, a shared encoder with fixed heads, and the same encoder with a question-conditioned head, at matched data and compute.
If fixed heads do as well for less, that is the result, and it will be reported as one.
Later · Not run
Questions in language
Compare task identifiers, label templates and natural-language descriptions. Paraphrases and unseen concepts are tested separately and never merged into one zero-shot number.
Questions about negation, location or waveform shape need signal-level ground truth. Writing more prompts does not supply it.
Two limits of the existing data, before anything runs
The sleep and P300 protocols on this site use balanced subsets. A probability estimated on them describes that balance, not how often each class occurs in real use, so they can compare methods with each other but cannot give a deployment error rate. The idle protocol is cue-gated and three minutes long, so no model run on it can give a false-activation rate per hour.
The first input with explicit non-control states is the four-person pilot above. Further input data sets are named here once their source review is complete.
What published work shows, and what it does not
External evidence, checked against the original records on 27 September 2026. None of it was reproduced here, and its figures are left out so that none can be read as this site’s.
Shows A trained classifier can be given a reject option with a target risk set by the user, trading coverage for error.
Does not show Evidence from image benchmarks. Its guarantee assumes test data that resemble the calibration data, which EEG across people and sessions often does not.
Shows On EEG seizure classification, patient shift left standard conformal prediction short of its nominal coverage, and personalised calibration raised coverage substantially.
Does not show One clinical task. It shows that coverage must be measured under the split that will actually be used, not that conformal methods fail.
Shows In vision, one encoding of an image can serve many forced-choice questions run as a batch — the “Jev-style” interface that prompted this plan.
Does not show Its own typed-head control gave no consistent accuracy advantage, and its training gains stayed within the task families it was trained on. Its speed-ups are vision results, not EEG ones.
BCI Report Research preview