VICENA GYM · HYDROGEN
A benchmark and learning environment built from laboratory records
Hydrogen is the first suite: tasks on materials for fuel cells and electrolysers, each graded against measurements. Models compute figures of merit from raw instrument files, audit records, forecast held-out experiments and run a virtual laboratory within a budget.
The benchmark shown here is built on synthetic placeholder records, which stand in for real laboratory records. Three real records are shown below, and the full real dataset is available under licence.
- 80
- placeholder records
- 4
- research lines
- 334
- benchmark tasks
- 681
- training tasks
- 116
- benchmark episodes
LEADERBOARD
Baselines and model runs, scored on the same tasks
Every row is scored on the short set: 69 tasks and 8 laboratory episodes sampled from the benchmark. Scores run from 0 to 1. A tier's score is the mean of its task types, and the overall score is the mean of the four tiers, so the many single-skill tasks do not outweigh the few predictive and agentic ones.
| Entry | Overall | Measure | Judge | Predict | Act | Type |
|---|---|---|---|---|---|---|
| Reference analysis and best recipe The programmatic reference solutions and the best recipe of each laboratory. It shows that every task is solvable; forecasts have no reference solution. | – | 1.00 | 1.00 | – | 1.00 | Upper bound |
| Gaussian-process regression and Bayesian optimisation Standard statistical tools on the two tiers they can attempt. A model adds value there only where it beats them. | – | – | – | 0.39 | 0.44 | Classical baseline |
| No-skill baseline Empty answers, “insufficient” for every claim, the campaign mean for every forecast and random search in the laboratory. | 0.12 | 0.00 | 0.05 | 0.07 | 0.36 | Baseline |
| Laboratory's standard recipe Submitting the standard recipe without running an experiment. | – | – | – | – | 0.01 | Baseline |
Model runs by task type
Each model ran once, each task in a fresh session that held only that task's files. An entry names the agent software the model ran in. A dash marks a task type an entry did not attempt.
| Entry | Forecast | Case |
|---|---|---|
| Reference analysis and best recipe | – | 1.00 |
| GPT-6.1 Sol OpenAI · Vicena agent | 0.50 | 0.66 |
| Gaussian-process regression and Bayesian optimisation | 0.39 | – |
| No-skill baseline | 0.07 | 0.23 |
REAL RECORDS
Three real laboratory records
The benchmark's tasks are modelled on working laboratory records. These three are iron–nitrogen–carbon oxygen catalysts made by one route, each with its synthesis, its test conditions and its raw instrument files. The values below are computed from the raw files by the benchmark's reference analysis.

| Sample | Ligand : zinc | Oxygen-reduction half-wave potential | Oxygen-evolution overpotential at 10 mA/cm² |
|---|---|---|---|
| FeNC1 | 4 : 1 | 0.767 V | not measured |
| FeNC2 | 4 : 1 | 0.830 V | not measured |
| FeNC3 | 8 : 1 | 0.807 V | 527 mV |
What an audit of the three records found
An impurity nobody named
One sample is labelled “with impurity”. Its diffraction pattern shows quartz and cristobalite: silica from the furnace ware, not the residual zinc or iron a reader would assume.
File names that point to another sample
Every micrograph in one sample's folder carries another sample's name. Which sample a file belongs to has to be established from the record.
A result without its control
On nickel foam the catalyst passes four times the current it passes on glassy carbon, but the foam is a catalyst itself and no bare-foam control was run.
Only results are published here. The records' files are supplied under licence.
These records come from Vicena's experimental data, which supplies records like them for evaluation and training, from laboratory archives or new experiments. See the experimental data
TASKS
Four tiers, from single skills to a whole investigation
The lower tiers are the skills the higher ones use, and every grader gives partial credit per item, so a long task has a reward for each step.
Measure
Compute a figure of merit from a raw instrument export by a stated definition, or read a diffraction pattern, a micrograph or a spectrum. Graded by tolerance against the reference analysis of the same file.
- Quantify
- Characterise
156 benchmark · 313 training
Judge
List what makes a record misleading, decide whether a claim is supported, refuted or undecidable, and propose the controls that would settle a conclusion. Graded against planted defects, verdicts computed from the data and rubrics of decisive controls.
- Audit
- Claims
- Controls
129 benchmark · 291 training
Predict
Forecast the held-out outcomes of a campaign, each with an 80% interval. Graded against the measured values, for ranking and for intervals that cover the outcome without being wide.
- Forecast
24 benchmark · 22 training
Act
Analyse a record's whole folder in a container, or run a virtual laboratory within a budget and submit the best recipe. Graded per figure, phase, finding and verdict, or by how close the recipe is to the best one.
- Case
25 benchmark · 55 training
EXAMPLE TASKS
What a task looks like
Four practice tasks from one synthetic record of a nanofibre oxygen electrode. A model receives the prompt and the record's files, and answers with one JSON object.
Task
From the four symmetric-cell impedance files (attached), report the electrode's ASR at 600 °C in Ω cm² and the activation energy of its polarisation in eV between 550 and 700 °C. The laboratory did not record the electrode area; use 0.5 cm² per electrode. Check that each file is what its name says before you use it. Ohmic resistance: the real impedance where the imaginary part changes sign at high frequency (interpolate linearly between the two neighbouring points), or the real part at the highest frequency if it never changes sign. Total resistance: the real impedance at the lowest measured frequency. Polarisation resistance: total minus ohmic. Area-specific resistance (ASR): polarisation resistance times the electrode area, divided by two for a symmetric cell. Activation energy: minus the slope of ln(1/ASR) against 1/T (T in kelvin), times the Boltzmann constant 8.617 × 10⁻⁵ eV/K.
Task text is in English.
Files
GDC6@PBSCF4 sym EIS 700C.csvGDC6@PBSCF4 sym EIS 650C.csvGDC6@PBSCF4 sym EIS 600C.csvGDC6@PBSCF4 sym EIS 550C.csv
Answer format
{
"asr_600_ohm_cm2": "number",
"ea_ev": "number"
} The reference answer scores 1.00.
LEARNING ENVIRONMENT
A virtual laboratory with a budget
Each research line has a laboratory. An agent chooses what to make and what to measure, pays for each step from a budget, and submits the recipe it believes is best. Every episode draws its own laboratory, and the best recipe moves with it, so a recipe carried over from another episode is only a starting point: the best one has to be found from the episode's own measurements.
- Action
{"action":"synthesise", "recipe":{"gdc_fraction":0.5, "sinter_c":900}}Observation{"batch":"B1", "recipe":{"gdc_fraction":0.5, "coaxial":true, "calcine_c":700, "sinter_c":900, "flow_ml_h":0.6, "voltage_kv":20}, "humidity_pct":33, "mat":"uniform fibres", "budget_left":13} - Action
{"action":"measure", "batch":"B1", "technique":"sym_eis"}Observation{"batch":"B1", "technique":"sym_eis", "asr_ohm_cm2":{"550":0.9638, "600":0.3818, "650":0.1621, "700":0.086}, "budget_left":12.5} - Action
{"action":"synthesise", "recipe":{"gdc_fraction":0.45, "sinter_c":900, "flow_ml_h":0.4, "voltage_kv":25}}Observation{"batch":"B2", "recipe":{"gdc_fraction":0.45, "coaxial":true, "calcine_c":700, "sinter_c":900, "flow_ml_h":0.4, "voltage_kv":25}, "humidity_pct":42, "mat":"uniform fibres", "budget_left":11.5} - Action
{"action":"measure", "batch":"B2", "technique":"sym_eis"}Observation{"batch":"B2", "technique":"sym_eis", "asr_ohm_cm2":{"550":0.7301, "600":0.2787, "650":0.1301, "700":0.068}, "budget_left":11} - Action
{"action":"submit", "recipe":{"gdc_fraction":0.45, "sinter_c":900, "flow_ml_h":0.4, "voltage_kv":25}}Observation{"result":"submitted", "recipe":{"gdc_fraction":0.45, "coaxial":true, "calcine_c":700, "sinter_c":900, "flow_ml_h":0.4, "voltage_kv":25}}Reward 0.59
How far simple strategies get
Mean reward on the benchmark's episodes. Random search and Bayesian optimisation stay well below 1 at these budgets: doing better takes knowledge of the chemistry.
- Standard recipe
- Random search
- Bayesian optimisation
Run it on your own machine
The environment comes with the benchmark package and runs locally. One command starts the laboratories, the tasks and the grader as a service on your machine, with a Gym-style loop: start an episode, send actions, read observations and a final reward.
DATASET
Eighty placeholder records across four research lines
A record is one complete experiment on one material: why it was run, how the material was made, the test conditions, and the instrument files. Historical records are kept as a working group keeps them, with gaps and slips; commissioned records are a designed campaign with the synthesis as run, replicates and controls.
| Research line | Historical | Commissioned | Basis |
|---|---|---|---|
| M–N–C single-atom oxygen catalysts Fe, Co, Fe–Co, Co–Mn and phosphorus-doped catalysts from pyrolysed zinc imidazolate frameworks, tested for oxygen reduction and evolution in alkaline electrolyte. | 14 | 20 | Inventoried records |
| Core–shell nanofibre oxygen electrodes Electrospun perovskite fibres with a doped-ceria shell, tested in protonic ceramic cells from 700 to 550 °C in fuel-cell and electrolysis modes. | 4 | 14 | Inventoried records |
| Catalytic layers for direct-ammonia fuel cells Ni–Fe ammonia-decomposition layers on the anode of protonic ceramic fuel cells, compared in hydrogen and ammonia with outlet-gas conversion and 100 h holds. | 6 | 8 | Planned project |
| Sintering of proton-conducting electrolytes Barium zirconate-cerate electrolytes sintered with NiO, ZnO or CuO, characterised by density, diffraction, microstructure and conductivity from 700 to 400 °C. | 6 | 8 | Planned project |
These records are synthetic placeholders
They stand in for real laboratory records, which are not public. Models of the materials generate them, calibrated to real records where real records exist, to show the structure and difficulty of the real dataset. They are not measurements, they are not what Vicena supplies, and they must not be cited as results.
Three real records are shown here
The results in the section on real records come from three unpublished laboratory records. The full real dataset consists of physical experiments with the scientists' notes and raw files. It is licensed for training and evaluation and is not published.
METHOD
What keeps the scores honest
Keys never travel with tasks
Ground truth is stored apart from everything a model sees, and a test checks that no record or task contains it.
Definitions are stated
Every prompt gives the definition of the quantity it asks for, so a score measures the analysis and not a guess at conventions.
File names cannot be trusted
Records contain mislabelled, duplicated, empty and mis-scaled files, as real ones do. A figure computed from a file taken at its name's word is wrong.
Clean records exist
Most commissioned records have no defects, so a model that reports problems everywhere loses points.
A canary marks the benchmark
Benchmark files carry a unique string so that training corpora can exclude them and leaks can be detected.
VICENA-GYM-HYDROGEN-CANARY 7c1e5a2e-3f0b-4d57-9c44-b1a6d2f8e913
ACCESS
The package, the real dataset and custom environments
Ask for the benchmark package to run the tasks and the environment on your own machine, for a licence to the real dataset, or for an environment built from your own laboratory's records.
CITATION
If you use the benchmark, please cite it as below.
@misc{vicena_gym_hydrogen_2026,
title = {Vicena Gym: Hydrogen. A benchmark and learning environment
from laboratory records of fuel-cell and electrolyser materials},
author = {Vicena},
year = {2026},
note = {Version v1, synthetic records},
url = {https://vicena.ai/rl-environments}
}