From 2c22009d85f074603800bd85877a187f2b6993e3 Mon Sep 17 00:00:00 2001 From: Joshaniel Cooper Date: Tue, 12 Nov 2024 10:09:17 +0100 Subject: [PATCH 1/5] Updated file_io.rst to give a MVP for production --- docs/fileio_usage.rst | 94 +++++++++++++++++++++++++++++++------------ 1 file changed, 69 insertions(+), 25 deletions(-) diff --git a/docs/fileio_usage.rst b/docs/fileio_usage.rst index aa04461..3998c9e 100644 --- a/docs/fileio_usage.rst +++ b/docs/fileio_usage.rst @@ -3,60 +3,104 @@ Usage ===== The easiest way to use :py:mod:`orsopy.fileio` (the module of :py:mod:`orsopy` that includes file reading and writing) to produce **metadata-rich .ort reduced reflectometry files** involves integrating this into your data reduction workflow. -Early in the workflow, the :py:mod:`orsopy.fileio` should be imported and an empty :code:`orsopy.fileio.orso.Orso` header object (here we also import :py:mod:`numpy` which will be used later). +Early in the workflow, the :py:mod:`orsopy.fileio` should be imported and an empty :code:`orsopy.fileio.orso.Orso` header object (here we also import :py:mod:`numpy` which will be used later). -.. code-block:: python +.. code-block:: python - import numpy as np from orsopy import fileio + from orsopy.fileio import (Reduction, Person, Orso, Experiment, + Sample, DataSource, Measurement, InstrumentSettings) + import numpy as np + import datetime header = fileio.orso.Orso.empty() -Having created the empty header object we can start to populate the appropriate components of it. -It is generally a good idea to populate the components as particular steps occur in the reduction process. -For example, if we want to identify the probing radiation as neutrons, we include this as follows. +Having created the empty header object we can start to populate the appropriate components of it. +It is generally a good idea to populate the components as particular steps occur in the reduction process. +In this example we fill out the experiment details and then wrap them in the appropriate class. -.. code-block:: python +.. code-block:: python - header.data_source.experiment.probe = 'neutron' + title = 'my title' + instrument = 'PLATYPUS' + start_date = datetime.date.today() # This needs to be a datetime object + probe = 'neutron' + facility = 'ESS' + proposalID = 999999 + doi = f'10.5286/ISIS.E.RB{99999}' + + experiment = Experiment(title, instrument, start_date, probe, facility, proposalID, doi) Full details of the different components that can be populated can be found in the `documentation`_ here or in the `file format specification`_. Note that this specification includes information regarding the **required** and optional components to be included for a file to be considered a **valid** .ort file. -It is not possible to write a .ort file without defining the columns present in the dataset, in this example we will have four columns of data, namely q, R, dR and dq (the final column is a description of the resolution function). -Columns are defined as follows, using the :code:`orsopy.fileio.base.Column` and :code:`orsopy.fileio.base.ErrorColumn` class objects (note that there are other `base classes`_ that can be used for a variety of objects). +What we show here is not the minimum needed to write an `.ort file, rather the minium to fill out all the major fields with information which is likely to be available at time of writing. + +We now write out the user and sample details, but note that the sample can be alot more descriptive than just a name. + +.. code-block:: python + + user_name = 'Jos Cooper' + user_affil = 'ESS' + + user = Person(user_name, user_affil) + + # Now fill the sample details + sample_name = 'Silicon' + + sample = Sample(sample_name) + + # The measurement details needs some instrument settings assigned first + angle = 0.3 # This can be a float, or a range, though currently not a list + wavelength = (1.8, 8.8) # Again a float or range + + instrument_settings = InstrumentSettings(angle, wavelength) + + # Now add the instrument settings to the names of the files used + files = ['test_data.raw'] # A list of all raw files used reduced + + measurment = Measurement(instrument_settings, files) + +We can now wrap together the user, experiment, sample and measurement information into a single object. + +.. code-block:: python + data_source_info = DataSource(user, experiment, sample, measurment) + +It is not possible to write a .ort file without defining the columns present in the dataset, in this example we will have four columns of data, namely q, R, dR and dq (the final column is a description of the resolution function). +Columns are defined as follows, using the :code:`orsopy.fileio.base.Column` and :code:`orsopy.fileio.base.ErrorColumn` class objects. + +.. code-block:: python -.. code-block:: python - q_column = fileio.base.Column(name='Qz', unit='1/angstrom', physical_quantity='wavevector transfer') + # Note that the unit is not fixed to 1/angstrom by the standard, but currently the data loader interprets all values as 1/angstrom irrespective of what is stated here r_column = fileio.base.Column(name='R', unit=None, physical_quantity='reflectivity') dr_column = fileio.base.ErrorColumn(error_of='R', error_type='uncertainty', value_is='sigma') dq_column = fileio.base.ErrorColumn(error_of='Qz', error_type='resolution', value_is='sigma') header.columns = [q_column, r_column, dr_column, dq_column] + # We can also make some data so that this code example will write something out + q = np.array([0.01,0.02,0.03]) + R = np.array([0.1,0.2,0.3]) + dR = np.array([0.001,0.002,0.003]) + dq = q * 0.02 -Any **required** metadata that is not included in the head will be written in the file as containing :code:`null`. -Having populated the metadata, we can now ensure that the metadata is correct with the following, +Any **required** metadata that is not included in the head will be written in the file as containing :code:`null`. -.. code-block:: python - - correct_header = fileio.orso.Orso(**header.to_dict()) - -This will produce a new object, if the metadata is correct. Now, we then want to assign the data that we want to write (this will be after your data reduction has been performed). -This is achieved by producing a :code:`fileio.orso.OrsoDataset` object, which takes the header and the relevant data columns (below these are :code:`q`, :code:`R`, :code:`dR`, and :code:`dq`) as inputs. +This is achieved by producing a :code:`fileio.orso.OrsoDataset` object, which takes the header and the relevant data columns (below these are :code:`q`, :code:`R`, :code:`dR`, and :code:`dq`) as inputs. -.. code-block:: python +.. code-block:: python - dataset = fileio.orso.OrsoDataset(info=header, data=np.array([q, R, dR, dq]).T) + orso_class = Orso(data_source_info, reduction=Reduction('My own code'), columns=header.columns) # reduction can also be assigned out of this funciton call + dataset = fileio.orso.OrsoDataset(info=orso_class, data=np.array([q, R, dR, dq]).T) -The dataset can then be saved with the following function, where :code:`'my_file.ort'` is the name for the file to be saved under. +The dataset can then be saved with the following function, where :code:`'my_file.ort'` is the name for the file to be saved under. .. code-block:: python - fileio.orso.save_orso(datasets=[dataset], fname='my_file.ort') + fileio.orso.save_orso(datasets=[dataset], fname='my_file.ort') # note that the first input is a list of datasets -Note that if you want to save more than one dataset in a single file, this can be achieved by including these in the list that is passed to this function. +Note that if you want to save more than one dataset in a single file, this can be achieved by including these in the list that is passed to this function. .. _`documentation`: ./modules.html#fileio From 28f65293d95e5f61cc2d560a9803e60a5e26ae83 Mon Sep 17 00:00:00 2001 From: Jos Cooper Date: Tue, 12 Nov 2024 17:07:21 +0100 Subject: [PATCH 2/5] Update docs/fileio_usage.rst Co-authored-by: Andrew McCluskey --- docs/fileio_usage.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/fileio_usage.rst b/docs/fileio_usage.rst index 3998c9e..f5cd77f 100644 --- a/docs/fileio_usage.rst +++ b/docs/fileio_usage.rst @@ -23,7 +23,7 @@ In this example we fill out the experiment details and then wrap them in the app title = 'my title' instrument = 'PLATYPUS' - start_date = datetime.date.today() # This needs to be a datetime object + start_date = datetime.date.today() # This needs to be an object of the type datetime probe = 'neutron' facility = 'ESS' proposalID = 999999 From 0466d5a3721aba90aa295924ee019eee071f032c Mon Sep 17 00:00:00 2001 From: Jos Cooper Date: Tue, 12 Nov 2024 17:07:42 +0100 Subject: [PATCH 3/5] Update docs/fileio_usage.rst Co-authored-by: Andrew McCluskey --- docs/fileio_usage.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/fileio_usage.rst b/docs/fileio_usage.rst index f5cd77f..8730281 100644 --- a/docs/fileio_usage.rst +++ b/docs/fileio_usage.rst @@ -33,7 +33,7 @@ In this example we fill out the experiment details and then wrap them in the app Full details of the different components that can be populated can be found in the `documentation`_ here or in the `file format specification`_. Note that this specification includes information regarding the **required** and optional components to be included for a file to be considered a **valid** .ort file. -What we show here is not the minimum needed to write an `.ort file, rather the minium to fill out all the major fields with information which is likely to be available at time of writing. +What we show here is not the minimum needed to write an `.ort` file, rather the minium to fill out all the major fields with information which is likely to be available at time of writing. We now write out the user and sample details, but note that the sample can be alot more descriptive than just a name. From 8e26d6fa6d95ab9b8f2fb9fa1dc77514804eecf7 Mon Sep 17 00:00:00 2001 From: Jos Cooper Date: Tue, 12 Nov 2024 17:07:57 +0100 Subject: [PATCH 4/5] Update docs/fileio_usage.rst Co-authored-by: Andrew McCluskey --- docs/fileio_usage.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/fileio_usage.rst b/docs/fileio_usage.rst index 8730281..d800a21 100644 --- a/docs/fileio_usage.rst +++ b/docs/fileio_usage.rst @@ -65,7 +65,7 @@ We can now wrap together the user, experiment, sample and measurement informatio .. code-block:: python data_source_info = DataSource(user, experiment, sample, measurment) -It is not possible to write a .ort file without defining the columns present in the dataset, in this example we will have four columns of data, namely q, R, dR and dq (the final column is a description of the resolution function). +It is not possible to write an `.ort` file without defining the columns present in the dataset, in this example we will have four columns of data, namely q, R, dR and dq (the final column is a description of the resolution function). Columns are defined as follows, using the :code:`orsopy.fileio.base.Column` and :code:`orsopy.fileio.base.ErrorColumn` class objects. .. code-block:: python From 5f5ce40f36cc2f400aabb05b19006145d04f233f Mon Sep 17 00:00:00 2001 From: Joshaniel Cooper Date: Tue, 12 Nov 2024 17:19:15 +0100 Subject: [PATCH 5/5] Added unit comment --- docs/fileio_usage.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/fileio_usage.rst b/docs/fileio_usage.rst index d800a21..4bf83b2 100644 --- a/docs/fileio_usage.rst +++ b/docs/fileio_usage.rst @@ -70,8 +70,8 @@ Columns are defined as follows, using the :code:`orsopy.fileio.base.Column` and .. code-block:: python + # Interpreted units are ["1/angstrom", "1/nm", "1", "1/s", None] q_column = fileio.base.Column(name='Qz', unit='1/angstrom', physical_quantity='wavevector transfer') - # Note that the unit is not fixed to 1/angstrom by the standard, but currently the data loader interprets all values as 1/angstrom irrespective of what is stated here r_column = fileio.base.Column(name='R', unit=None, physical_quantity='reflectivity') dr_column = fileio.base.ErrorColumn(error_of='R', error_type='uncertainty', value_is='sigma') dq_column = fileio.base.ErrorColumn(error_of='Qz', error_type='resolution', value_is='sigma')