release v0.3.0 #273
Merged
release v0.3.0 #273
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…arate consts object
lazy datasets now draw from either properties defined in constructor on available in a meta data file. meta data files are streamlined, both in loading and saving.
…ty database via meta objects
Cellxgene loader
* add plot_npc and plot_active_latent_units (#9) * add plot_npc and plot_active_latent_units * make sure handling of z and z_mean is consistent for VAE embeddings * clean up and documentation * formatting Co-authored-by: Martin König <martin.koenig@ymail.com> Co-authored-by: le-ander <20015434+le-ander@users.noreply.github.com> * added data loader for interactive workflows with unprocessed data * made cell type loading optional in dataset .load() * enabled usage of type estimator on data without labels in prediction mode * recursively search custom model repo for weights files * sort model lookuptable alphabetically before writing it * make sure mode_path is set correctly in model_lookuptable when recursive weights loading is used * fix os.path.join usage in dataloaders * replace path handling through string concatenations with os.paths.join and f-strings * fix bug in lookup table writing * add mdoel file path to lookup table * reset index in model lookuptable before saving * add method to user interface for pushing local model weights to zenodo * fix bug in user interface * fix bux in summaries.py * use absolute model paths when model_lookuptable is used * fix bug in pretrained weights loading * fix bug in pretrained weights loading * automatically create an InteractiveDataset when loading data through the UI * fix bug inUI data loading * Explicitly cast indices and indptr of final backed file to int64. (#17) For the background on this: scverse/anndata#453 * update human lung dataset doi * align mouse organ names with human organ names * fix typo in trachea organ naming in mouse * rename mouse ovary organ to femalegonad * rename mouse ovary organ to femalegonad * sort by model type in classwise f1 heatmap plot * another hacky solution to ensure a summary tab can be created when both vae and other models are loaded at once * allow custom metadata in zenodo submission * do not return doi but deposit url after depositing to zenodo sandbox as dois don't wrk on sandbox * updated model zoo description * recognise all .h5 and .data-0000... files as sfaira weights when constructing lookuptable * Update README.rst * Add selu activation and lecun_normal weight_init scheme for human VAEVAMP. (#19) * update sfaira erpo url and handle .h5 extension in model lookuptable id * add meta_data download information to all human dataloaders * updated docs * updated reference to README in docs * updated index * included reference to svensson et al data base in docs * fixed typo in docs * fixed typos in docs * restructured docs * fixed bug in reference roadmap in docs * updated data and model zoo description * added summary picture into index of docs * fixed typo in docs * updated summary panel * add badges to readme and docs index * updated summary panel (#20) * Doc updates (#21) * updated summary panel * fixed concept figure references * Doc updates (#22) * updated zoo panels * move from `import sfaira.api as sfaira` to `import sfaira` and from `import sfaira_extension.api as sfairae` to `import sfaira_extension` * add custom genomes to sfaira_extension * fix loading of custom topology versions from sfaira_extension * fix circular imports between sfaira_extension and sfaira * fix dataloader * fix celltype versioning through sfaira_extension * fix celltype versioning through sfaira_extension * formatting * Doc updates (#25) * added mention of download scripts into docs Co-authored-by: mk017 <martin.koenig@tum.de> Co-authored-by: Martin König <martin.koenig@ymail.com> Co-authored-by: le-ander <20015434+le-ander@users.noreply.github.com> Co-authored-by: Abdul Moeed <abdulmoeed444@gmail.com>
* add plot_npc and plot_active_latent_units (#9) * add plot_npc and plot_active_latent_units * make sure handling of z and z_mean is consistent for VAE embeddings * clean up and documentation * formatting Co-authored-by: Martin König <martin.koenig@ymail.com> Co-authored-by: le-ander <20015434+le-ander@users.noreply.github.com> * added data loader for interactive workflows with unprocessed data * made cell type loading optional in dataset .load() * enabled usage of type estimator on data without labels in prediction mode * recursively search custom model repo for weights files * sort model lookuptable alphabetically before writing it * make sure mode_path is set correctly in model_lookuptable when recursive weights loading is used * fix os.path.join usage in dataloaders * replace path handling through string concatenations with os.paths.join and f-strings * fix bug in lookup table writing * add mdoel file path to lookup table * reset index in model lookuptable before saving * add method to user interface for pushing local model weights to zenodo * fix bug in user interface * fix bux in summaries.py * use absolute model paths when model_lookuptable is used * fix bug in pretrained weights loading * fix bug in pretrained weights loading * automatically create an InteractiveDataset when loading data through the UI * fix bug inUI data loading * Explicitly cast indices and indptr of final backed file to int64. (#17) For the background on this: scverse/anndata#453 * update human lung dataset doi * align mouse organ names with human organ names * fix typo in trachea organ naming in mouse * rename mouse ovary organ to femalegonad * rename mouse ovary organ to femalegonad * sort by model type in classwise f1 heatmap plot * another hacky solution to ensure a summary tab can be created when both vae and other models are loaded at once * allow custom metadata in zenodo submission * do not return doi but deposit url after depositing to zenodo sandbox as dois don't wrk on sandbox * updated model zoo description * recognise all .h5 and .data-0000... files as sfaira weights when constructing lookuptable * Update README.rst * Add selu activation and lecun_normal weight_init scheme for human VAEVAMP. (#19) * update sfaira erpo url and handle .h5 extension in model lookuptable id * add meta_data download information to all human dataloaders * updated docs * updated reference to README in docs * updated index * included reference to svensson et al data base in docs * fixed typo in docs * fixed typos in docs * restructured docs * fixed bug in reference roadmap in docs * updated data and model zoo description * added summary picture into index of docs * fixed typo in docs * updated summary panel * add badges to readme and docs index * updated summary panel (#20) * Doc updates (#21) * updated summary panel * fixed concept figure references * Doc updates (#22) * updated zoo panels * move from `import sfaira.api as sfaira` to `import sfaira` and from `import sfaira_extension.api as sfairae` to `import sfaira_extension` * add custom genomes to sfaira_extension * fix loading of custom topology versions from sfaira_extension * fix circular imports between sfaira_extension and sfaira * fix dataloader * fix celltype versioning through sfaira_extension * fix celltype versioning through sfaira_extension * formatting * Doc updates (#25) * added mention of download scripts into docs Co-authored-by: mk017 <martin.koenig@tum.de> Co-authored-by: Martin König <martin.koenig@ymail.com> Co-authored-by: le-ander <20015434+le-ander@users.noreply.github.com> Co-authored-by: Abdul Moeed <abdulmoeed444@gmail.com>
* fixed missing import in mouse trachea * fixed meta data accession and added automatic subsetting of datasetgroups during loading subsets to available data sets * depreceated api and added consts into __init__ api
fixed unit tests and fixed adata field related bugs * updated order of data sets into meta loading dict to be alphabetic * fixed bug in setting pandas dataframe index * depreceated kipoi test * fixed vae unit test * many: field related bugs
…ames, ensembl_col=ADATA_IDS_SFAIRA.gene_id_ensembl) uses strings now (#35)
* fixed remaining instance of has_celltypes * fixed fields reference
* Refactor __get_dataset() to always return data generator. Modify functions to handle generator. * Remove unnecessary sparse tensor conversion.
* improved and modified meta data usage * fixed download website to be able to contain multiple entries in meta data saving * added support for variable uns and obs setting of data meta data that can be cell specific * added support to check for equiality of column entries to condition string to define healthy into super method * moved _mapped_features into constants * introduced cell ontology obs keys into base class * updated cellxgene data loader to new constant field standards * coupled .annotated to obs keys that indicate presence of cell types * added dev stage and age meta data * refactored UNS_STRING_META_IN_OBS as constant * updated cellxgene data loader to new data loading format * updated data loaders to new formate * updated interactive data set * grouped data loaders by study * made all data loaders raw data loaders * added new auto chaching * added download method into base class * removed separate caching from data loaders * use "organsism" instead of "species" across sfaira and optimise imports * fixed example code block * exluded data loader tests from git path * took out external.py files from data loaders to make directories leaner * added directory oriented, automated data set groups * added sfaira wide super groups and super groups nesting * enabled parallelised loading * refactored unknown cell type identification * added xlrd dependency * added github workflows * homogenized string style to "" * enabled rapid raw loading of groups saved in one object * remove superflous raw loading docstrings Co-authored-by: le-ander <20015434+le-ander@users.noreply.github.com> Co-authored-by: Lukas Heumos <lukas.heumos@posteo.net>
* FAQ with contributions from @xlancelottx and @lauradmartens * more detailed step-by-step explanation of loader contribution
…ding / writing fails
* fixed bug in config writing of store and built unit test * improved error reportign for Dataset
* adapt meta_writing to new streamlining backend * fix handling of celltype labels in write_meta * fix donload_url_meta property setter * handle combinatorial batch keys correctly in write_meta * handle combinatorial batch keys correctly in write_meta * handle combinatorial batch keys correctly in write_meta
…ger scalar" scipy error (#251)
…es for loading data (#256)
#255) * fixed store to estimator interface and added further unit tests on store and estimators * fixed zoo TopologyContainer interface and added .obs into store * adapated zoo handling and model ontology switched to modelclass_name_provider to be more generalisable modelclass is the only constant beyond provider that is really essential because it maps to a unique estimator class * updated store subsetting * fixed cached_store_writing for testing * fixed store-estimator bugs * stabilised size factor computation, fixed store unit tests and added dense conversion for feature indexing in store generator * improved efficiency of generator queries of store and generator usage by estimator keras * id is retained if uns_to_obs is True in streamlining * removed conitnuous batches option from store generator
* reduced indices in store to non empty indices * fixed unit test * extended store class documentation * added additional edges to UBERON wrapper class * updated DAG break warning in ontology classes * made empty argument error in get_effective_leaves more verbose * warning instead of error if empty sets are passed to target universe writing * removed empty configs * caught non annotated data sets * fix out path in target writing and some syntax * enforced subsetting by selected indices within data set in target universe writing script Co-authored-by: le-ander <20015434+le-ander@users.noreply.github.com>
* add polioudakis data * fix download function * add manual download for polioudakis * fix polioudakis * fix maps writing * update polioudakis * add celltype ids * fix cli print * update manual downlaod workflow * fix manual download * fix download url polioudakis
* renamed sfaira lint to validate Signed-off-by: zethson <lukas.heumos@posteo.net> * remove clean from instructions Signed-off-by: zethson <lukas.heumos@posteo.net>
Signed-off-by: zethson <lukas.heumos@posteo.net>
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Signed-off-by: zethson <lukas.heumos@posteo.net>
* fixes training-related bugs * added zarr-dask distributed acccess store store * fixed indexing bug in h5ad store * fixed store unit test and renamed unknown meta data field in sfaira streamlining * added optional custom cell ontology to estimator constructor (#254) * fix handling of new model_id format * clean up model zoo * fixed grid search summaries * fix celltype handling in UI * include topology files in zenodo deposition Co-authored-by: le-ander <20015434+le-ander@users.noreply.github.com>
* removed previously deleted files reintroduced by merge * removed old rst files
|
Hi @davidsebfischer, It looks like this pull-request is has been made against the theislab/sfaira You do not need to close this PR, you can change the target branch to Thanks again for your contribution! |
|
commit 950abad was a force push to resolve the conflicts with master. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.