Updated column mapper and def dicts for release. (#517)

* Add accessors for definitions * Remove attribute columns. Anything that was one now defaults to an Ignore column * Tested the bids-dataset * Updated the definitions returned on assemble * Make ignored columns marked as HedTags show up Handle new schema versioning([prefix:][name_]version) Only process HedTag columns when converting to short Improve error handling of schema prefixes * Rename library_prefix -> schema_prefix * Updated the BIDS dataset to read in new schema format * Make load_schema_version take a list optionally Allow library names to be passed by themselves to load_schema_version * Updated bids dataset * Refactor column mapper function. Add a test for a column named 'HED' issue. Add optional parameters to column mapper to allow blank columns, or not warn on missing columns * fix unit test. Always ignore an unnamed column 0. Later index handling still TBD * Minor updates to docs * Make it a fatal error for columns to be blank. Further cleanups * Make columns return a list again * Having trouble with the unit tests for BidsTabularInput * Fixed tests after environment problem Co-authored-by: IanCa <ianrcallanan@gmail.com>
hed-standard · Jun 20, 2022 · bbf1880 · bbf1880
1 parent 2059d92
commit bbf1880
Show file tree

Hide file tree

Showing 110 changed files with 8,839 additions and 1,560 deletions.
diff --git a/docs/source/api.rst b/docs/source/api.rst
@@ -90,7 +90,6 @@ HED utilities
    :toctree: generated/
 
    hed.util.data_util
-   hed.util.file_util
    hed.util.io_util
 
 .. currentmodule:: hed

diff --git a/docs/source/generated/hed.models.BaseInput.rst b/docs/source/generated/hed.models.BaseInput.rst
@@ -0,0 +1,58 @@
+hed.models.BaseInput
+====================
+
+.. currentmodule:: hed.models
+
+.. autoclass:: hed.models.BaseInput
+
+   .. automethod:: __init__
+
+
+   .. rubric:: Methods
+
+   .. autosummary::
+
+      ~BaseInput.__init__
+      ~BaseInput.convert_to_long
+      ~BaseInput.convert_to_short
+      ~BaseInput.extract_definitions
+      ~BaseInput.get_def_and_mapper_issues
+      ~BaseInput.get_worksheet
+      ~BaseInput.iter_dataframe
+      ~BaseInput.reset_mapper
+      ~BaseInput.set_cell
+      ~BaseInput.to_csv
+      ~BaseInput.to_excel
+      ~BaseInput.update_definition_mapper
+      ~BaseInput.validate_file
+
+
+
+
+
+   .. rubric:: Attributes
+
+   .. autosummary::
+
+      ~BaseInput.COMMA_DELIMITER
+      ~BaseInput.EXCEL_EXTENSION
+      ~BaseInput.FILE_EXTENSION
+      ~BaseInput.FILE_INPUT
+      ~BaseInput.STRING_INPUT
+      ~BaseInput.TAB_DELIMITER
+      ~BaseInput.TEXT_EXTENSION
+      ~BaseInput.columns
+      ~BaseInput.dataframe
+      ~BaseInput.def_dict
+      ~BaseInput.has_column_names
+      ~BaseInput.loaded_workbook
+      ~BaseInput.name
+      ~BaseInput.worksheet_name
+
+
+
+.. autoclass:: hed.models.BaseInput
+   :members:
+   :undoc-members:
+   :show-inheritance:
+   :inherited-members:
diff --git a/docs/source/generated/hed.models.ColumnMapper.rst b/docs/source/generated/hed.models.ColumnMapper.rst
@@ -0,0 +1,37 @@
+hed.models.ColumnMapper
+=======================
+
+.. currentmodule:: hed.models
+
+.. autoclass:: hed.models.ColumnMapper
+
+   .. automethod:: __init__
+
+
+   .. rubric:: Methods
+
+   .. autosummary::
+
+      ~ColumnMapper.__init__
+      ~ColumnMapper.add_columns
+      ~ColumnMapper.expand_row_tags
+      ~ColumnMapper.get_column_mapping_issues
+      ~ColumnMapper.get_def_dicts
+      ~ColumnMapper.get_prefix_remove_func
+      ~ColumnMapper.get_tag_columns
+      ~ColumnMapper.set_column_map
+      ~ColumnMapper.set_column_prefix_dict
+      ~ColumnMapper.set_requested_columns
+      ~ColumnMapper.set_tag_columns
+
+
+
+
+
+
+
+.. autoclass:: hed.models.ColumnMapper
+   :members:
+   :undoc-members:
+   :show-inheritance:
+   :inherited-members:
diff --git a/docs/source/generated/hed.models.ColumnMetadata.rst b/docs/source/generated/hed.models.ColumnMetadata.rst
@@ -0,0 +1,36 @@
+hed.models.ColumnMetadata
+=========================
+
+.. currentmodule:: hed.models
+
+.. autoclass:: hed.models.ColumnMetadata
+
+   .. automethod:: __init__
+
+
+   .. rubric:: Methods
+
+   .. autosummary::
+
+      ~ColumnMetadata.__init__
+      ~ColumnMetadata.expand
+      ~ColumnMetadata.expected_pound_sign_count
+      ~ColumnMetadata.remove_prefix
+
+
+
+
+
+   .. rubric:: Attributes
+
+   .. autosummary::
+
+      ~ColumnMetadata.hed_dict
+
+
+
+.. autoclass:: hed.models.ColumnMetadata
+   :members:
+   :undoc-members:
+   :show-inheritance:
+   :inherited-members:
diff --git a/docs/source/generated/hed.models.DefMapper.rst b/docs/source/generated/hed.models.DefMapper.rst
@@ -0,0 +1,40 @@
+hed.models.DefMapper
+====================
+
+.. currentmodule:: hed.models
+
+.. autoclass:: hed.models.DefMapper
+
+   .. automethod:: __init__
+
+
+   .. rubric:: Methods
+
+   .. autosummary::
+
+      ~DefMapper.__init__
+      ~DefMapper.add_definitions
+      ~DefMapper.add_definitions_from_string_as_temp
+      ~DefMapper.clear_temporary_definitions
+      ~DefMapper.expand_and_remove_definitions
+      ~DefMapper.expand_def_tags
+      ~DefMapper.get_def_entry
+
+
+
+
+
+   .. rubric:: Attributes
+
+   .. autosummary::
+
+      ~DefMapper.gathered_defs
+      ~DefMapper.issues
+
+
+
+.. autoclass:: hed.models.DefMapper
+   :members:
+   :undoc-members:
+   :show-inheritance:
+   :inherited-members:
diff --git a/docs/source/generated/hed.models.DefinitionDict.rst b/docs/source/generated/hed.models.DefinitionDict.rst
@@ -0,0 +1,31 @@
+hed.models.DefinitionDict
+=========================
+
+.. currentmodule:: hed.models
+
+.. autoclass:: hed.models.DefinitionDict
+
+   .. automethod:: __init__
+
+
+   .. rubric:: Methods
+
+   .. autosummary::
+
+      ~DefinitionDict.__init__
+      ~DefinitionDict.check_for_definitions
+      ~DefinitionDict.get
+      ~DefinitionDict.get_as_strings
+      ~DefinitionDict.get_definition_issues
+
+
+
+
+
+
+
+.. autoclass:: hed.models.DefinitionDict
+   :members:
+   :undoc-members:
+   :show-inheritance:
+   :inherited-members:
diff --git a/docs/source/generated/hed.models.DefinitionEntry.rst b/docs/source/generated/hed.models.DefinitionEntry.rst
@@ -0,0 +1,28 @@
+hed.models.DefinitionEntry
+==========================
+
+.. currentmodule:: hed.models
+
+.. autoclass:: hed.models.DefinitionEntry
+
+   .. automethod:: __init__
+
+
+   .. rubric:: Methods
+
+   .. autosummary::
+
+      ~DefinitionEntry.__init__
+      ~DefinitionEntry.get_definition
+
+
+
+
+
+
+
+.. autoclass:: hed.models.DefinitionEntry
+   :members:
+   :undoc-members:
+   :show-inheritance:
+   :inherited-members:
diff --git a/docs/source/generated/hed.models.HedGroup.rst b/docs/source/generated/hed.models.HedGroup.rst
@@ -0,0 +1,55 @@
+hed.models.HedGroup
+===================
+
+.. currentmodule:: hed.models
+
+.. autoclass:: hed.models.HedGroup
+
+   .. automethod:: __init__
+
+
+   .. rubric:: Methods
+
+   .. autosummary::
+
+      ~HedGroup.__init__
+      ~HedGroup.append
+      ~HedGroup.check_if_in_original
+      ~HedGroup.copy
+      ~HedGroup.find_def_tags
+      ~HedGroup.find_exact_tags
+      ~HedGroup.find_placeholder_tag
+      ~HedGroup.find_tags
+      ~HedGroup.find_tags_with_term
+      ~HedGroup.get_all_groups
+      ~HedGroup.get_all_tags
+      ~HedGroup.get_as_form
+      ~HedGroup.get_as_long
+      ~HedGroup.get_as_short
+      ~HedGroup.get_frozen
+      ~HedGroup.get_original_hed_string
+      ~HedGroup.groups
+      ~HedGroup.lower
+      ~HedGroup.remove
+      ~HedGroup.replace
+      ~HedGroup.tags
+
+
+
+
+
+   .. rubric:: Attributes
+
+   .. autosummary::
+
+      ~HedGroup.children
+      ~HedGroup.is_group
+      ~HedGroup.span
+
+
+
+.. autoclass:: hed.models.HedGroup
+   :members:
+   :undoc-members:
+   :show-inheritance:
+   :inherited-members:
diff --git a/docs/source/generated/hed.models.HedGroupBase.rst b/docs/source/generated/hed.models.HedGroupBase.rst
@@ -0,0 +1,50 @@
+hed.models.HedGroupBase
+=======================
+
+.. currentmodule:: hed.models
+
+.. autoclass:: hed.models.HedGroupBase
+
+   .. automethod:: __init__
+
+
+   .. rubric:: Methods
+
+   .. autosummary::
+
+      ~HedGroupBase.__init__
+      ~HedGroupBase.copy
+      ~HedGroupBase.find_def_tags
+      ~HedGroupBase.find_exact_tags
+      ~HedGroupBase.find_placeholder_tag
+      ~HedGroupBase.find_tags
+      ~HedGroupBase.find_tags_with_term
+      ~HedGroupBase.get_all_groups
+      ~HedGroupBase.get_all_tags
+      ~HedGroupBase.get_as_form
+      ~HedGroupBase.get_as_long
+      ~HedGroupBase.get_as_short
+      ~HedGroupBase.get_original_hed_string
+      ~HedGroupBase.groups
+      ~HedGroupBase.lower
+      ~HedGroupBase.tags
+
+
+
+
+
+   .. rubric:: Attributes
+
+   .. autosummary::
+
+      ~HedGroupBase.children
+      ~HedGroupBase.is_group
+      ~HedGroupBase.span
+
+
+
+.. autoclass:: hed.models.HedGroupBase
+   :members:
+   :undoc-members:
+   :show-inheritance:
+   :inherited-members: