Dataset Metadata

Each ICAT dataset can have metadata associated with it, describing experimental and contextual information. The definitions of these metadata elements are documented in the ICAT metadata definitions project. Additionally, metadata specific to the experimental techniques and their ontology is described in the ESRF ontologies documentation.

BLISS

For details, see the BLISS documentation.

In short the SCAN_SAVING object of the BLISS session has attributes proposal, collection and dataset to define metadata uploaded to ICAT for every dataset within the scope of the proposal, collection or individual dataset respectively.

Setting and getting metadata can be done with the namespace attribute metadata or by using keys

DAU [10]: SCAN_SAVING.dataset.metadata.instrument.beam.incident_energy = 7.1
DAU [11]: SCAN_SAVING.dataset["InstrumentBeam_incident_energy"] = 7.1
DAU [12]: SCAN_SAVING.dataset.existing
Out [12]: Namespace contains:
        .startDate                      = '2026-03-14T16:08:56.485562+01:00'
        .InstrumentBeam_incident_energy = 7.1
        .Sample_name                    = 'dautest'

Unknown metadata keys

Using an unknown key raises a KeyError

DAU [15]: SCAN_SAVING.dataset["InstrumentBeam_incident_energ"] = 7.1
!!! === KeyError: "'InstrumentBeam_incident_energ' is not a valid ICAT field" === !!! ( for more details type cmd 'last_error(2)' )

Validation and type/unit coercion

Metadata values are validated, type-coerced and unit-coerced when closing a dataset. Failures will be logged but do not raise exceptions

DAU [11]: newdataset()
Dataset set to '0003'
Data path: /data/id00/inhouse/id002603/id00/20260301/RAW_DATA/dautest/dautest_0003

DAU [5]: SCAN_SAVING.dataset.metadata.instrument.beam.incident_energy = "wrong"
DAU [8]: SCAN_SAVING.dataset["InstrumentBeam_incident_energy"] = "wrong"

DAU [6]: SCAN_SAVING.dataset.existing
Out [6]: Namespace contains:
        .startDate                      = '2026-03-14T16:05:33.333542+01:00'
        .InstrumentBeam_incident_energy = 'wrong'
        .Sample_name                    = 'dautest'

DAU [10]: loopscan(10,0.1,diode1)
Out [24]: Scan(number=1, name=loopscan, path=/data/id00/inhouse/id002603/id00/20260301/RAW_DATA/dautest/dautest_0003/dautest_0003.h5)

DAU [11]: newdataset()
WARNING: ValidationError: 1 validation error for IcatDatasetParameters
instrument.beam.incident_energy
Value error, could not convert string to float: 'wrong' [type=value_error, input_value='wrong', input_type=str]
    For further information visit https://errors.pydantic.dev/2.12/v/value_error

Documentation on ICAT fields: https://icat-esrf-definitions.readthedocs.io/en/v2.5.0/icat.html
Dataset set to '0004'
Data path: /data/id00/inhouse/id002603/id00/20260301/RAW_DATA/dautest/dautest_0004

Metadata units

Values with units can be set like this

DAU [17]: SCAN_SAVING.dataset["InstrumentBeam_incident_energy"] = 7100,"eV"

The value shown in the Data Portal will be 7.1 keV, since the units of this ICAT field are keV.

When the unit does not correspond to the dimensionality, you get a warning but again no exception is raised

DAU [23]: SCAN_SAVING.dataset["InstrumentBeam_incident_energy"] = 7100,"m"

DAU [24]: loopscan(10,0.1,diode1)
Out [24]: Scan(number=1, name=loopscan, path=/data/id00/inhouse/id002603/id00/20260301/RAW_DATA/dautest/dautest_0005/dautest_0005.h5)

DAU [25]: newdataset()
WARNING: DimensionalityError: Cannot convert from 'meter' ([length]) to 'kiloelectron_volt' ([mass] * [length] ** 2 / [time] ** 2)

Documentation on ICAT fields: https://icat-esrf-definitions.readthedocs.io/en/v2.5.0/icat.html
Dataset set to '0006'
Data path: /data/id00/inhouse/id002603/id00/20260301/RAW_DATA/dautest/dautest_0006