Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev #304

Merged
merged 168 commits into from
Jun 9, 2021
Merged

Dev #304

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
168 commits
Select commit Hold shift + click to select a range
b4e12a0
added first version of cellxgene data format loader
davidsebfischer Oct 8, 2020
27fa462
refactored anndata field entries from data loaders to be named in sep…
davidsebfischer Oct 27, 2020
4b082e5
adapted adata field refectoring in data base classs
davidsebfischer Oct 27, 2020
8aa4ef6
updated cellxgene data loader to use refactored constants for adata f…
davidsebfischer Oct 27, 2020
2a145fd
updated missing refactored gene id fields in data loaders
davidsebfischer Oct 27, 2020
d70f524
refactored adata fields constant container classes to reflect core sh…
davidsebfischer Oct 27, 2020
4a280d0
updated old usages of ADATA_IDS to ADATA_IDS_SFAIRA
davidsebfischer Oct 27, 2020
16e9aba
added constants based classses into api to improve interfacing to 3rd…
davidsebfischer Oct 27, 2020
60f5bce
refactored lazy dataset properties and meta data objects
davidsebfischer Oct 27, 2020
1e32103
renamed remaining instances of "animal" into "species"
davidsebfischer Oct 27, 2020
77a6dff
allowed maps of meta data file nomenclature
davidsebfischer Oct 27, 2020
9f844c7
moved meta data code in DatasetBase for readability
davidsebfischer Oct 27, 2020
d3e356a
introduced meta_fn attribute of dataset class and depreceated 3rd par…
davidsebfischer Oct 27, 2020
4f988a9
added datsetgroup subsetting based on meta / lazy properties
davidsebfischer Oct 27, 2020
facab5b
Merge branch 'master' into cellxgene_loader
davidsebfischer Oct 27, 2020
4b85cbb
Merge branch 'master' into cellxgene_loader
davidsebfischer Oct 27, 2020
365739c
Merge pull request #10 from theislab/cellxgene_loader
davidsebfischer Nov 3, 2020
a415e5d
Master merge into dev (#26)
davidsebfischer Dec 10, 2020
ee92ef8
Dev merge fix (#28)
davidsebfischer Dec 10, 2020
af3e575
fixed missing import in mouse trachea (#29)
davidsebfischer Dec 10, 2020
273d7b6
pass paths correctly to extension datasets
le-ander Dec 10, 2020
6a37f0b
Dataloading fix (#33)
davidsebfischer Dec 10, 2020
198350e
self._convert_and_set_var_names(symbol_col=ADATA_IDS_SFAIRA.gene_id_n…
davidsebfischer Dec 10, 2020
ac24ca9
fixed remaining instance of has_celltypes (#36)
davidsebfischer Dec 10, 2020
1356856
Has celltypes bug (#37)
davidsebfischer Dec 10, 2020
582e6c8
Data generator for model evaluation and prediction. (#46)
Abdul-Moeed Dec 30, 2020
91b8a1b
Adjust data loaders to directory-wise set-up (#38)
davidsebfischer Jan 19, 2021
700cd37
extended data loader documentation (#77)
davidsebfischer Jan 19, 2021
f95ea89
added development FAQ section (#79)
davidsebfischer Jan 20, 2021
03d590f
update cached reading to only warn and not throw errors if cached rea…
le-ander Jan 25, 2021
9d0dbe2
fixed bug in d10_1016_j_cmet_2019_01_021 data loader and added except…
davidsebfischer Jan 25, 2021
e532a4e
Fix tutorials (#83)
le-ander Jan 26, 2021
69807fa
Streamline data loader for data set collection (#85)
davidsebfischer Jan 27, 2021
7e11415
fixed bugs in create_meta_and_cache.py (#87)
davidsebfischer Jan 27, 2021
d0a4b1f
fix dataloaders (#88)
le-ander Jan 28, 2021
036f341
fixed meta data loading bug
davidsebfischer Jan 28, 2021
47e8713
cell onotology data base (#12)
davidsebfischer Jan 29, 2021
f35f6bc
Improved co automation (#90)
davidsebfischer Feb 4, 2021
afc694d
File wise celltype maps (#98)
davidsebfischer Feb 4, 2021
3d89f68
File wise celltype maps (#99)
davidsebfischer Feb 6, 2021
409a796
lint
le-ander Feb 8, 2021
2f269bd
added sfaira.data.utils and put cell type search into there, improved…
davidsebfischer Feb 9, 2021
6b74875
Fix bugs encountered in using new data loaders (#104)
davidsebfischer Feb 10, 2021
cfb988f
Fix lenient ontology matching (#110)
davidsebfischer Feb 10, 2021
4a29bbb
fix bugs in data loading (#112)
davidsebfischer Feb 10, 2021
197b8f3
automated dataset_download (#78)
le-ander Feb 11, 2021
94562c7
Improved data path handling (#117)
davidsebfischer Feb 12, 2021
9def399
fixed one file many DS loading (#118)
davidsebfischer Feb 12, 2021
3c56ecd
fixed handling of no doi data sets (#120)
davidsebfischer Feb 12, 2021
e63cd1f
fixed author (#121)
davidsebfischer Feb 12, 2021
4224291
fixed bugs in loading of mouse_brain_2019_10x_hove_001 (#122)
davidsebfischer Feb 12, 2021
92875e6
dataloader updates (#116)
le-ander Feb 12, 2021
8042b50
fixed d10_1038_s41593_019_0393_4 (#124)
davidsebfischer Feb 12, 2021
73f6e4c
translated d10_1038_s41586_020_2157_4 to DatasetBaseGroupLoadingOneFile
davidsebfischer Feb 12, 2021
48d0801
fixed _subset_from_group
davidsebfischer Feb 12, 2021
3b841b1
renaming changes
davidsebfischer Feb 12, 2021
b96bfd3
updated protocol
davidsebfischer Feb 12, 2021
798f307
renamed HCL IDs
davidsebfischer Feb 12, 2021
707d2aa
removed dysfunctional loader 10.1038/s41586-019-1631-3 from auto disc…
davidsebfischer Feb 13, 2021
ed745c8
fixed bugs with handling list input to map_celltype_to_ontology
davidsebfischer Feb 15, 2021
6b95954
fixed d10_1186_s13059_019_1906_x loader
davidsebfischer Feb 15, 2021
4c2ae22
repositioned cache directory and allowed loading without feature spac…
davidsebfischer Feb 15, 2021
646ae6a
fixed bug in d10_1186_s13059_019_1906_x
davidsebfischer Feb 15, 2021
35b64c9
improved raw feature space yielding
davidsebfischer Feb 15, 2021
fea7ea5
set gene version removal in cellmap writing to false
davidsebfischer Feb 15, 2021
d668219
fixed d10_1016_j_cell_2018_02_001
davidsebfischer Feb 15, 2021
31db743
fixed prostate naming
davidsebfischer Feb 16, 2021
1938b4b
fixed MCA
davidsebfischer Feb 16, 2021
930dc08
Update dataloaders (#128)
le-ander Feb 17, 2021
4648a2f
use default cache and meta paths if not user provided (#130)
le-ander Feb 17, 2021
d7f1438
dataloaders' _load() returns adata instead of assigning self.adata at…
le-ander Feb 17, 2021
61547eb
Add sfaira commandline interface to manage dataloaders (#89)
Zethson Feb 17, 2021
dfe252c
fixed DatasetBaseGroupLoadingOneFile given new laod structure
davidsebfischer Feb 17, 2021
1720411
fixed additional bug in DatasetBaseGroupLoadingOneFile
davidsebfischer Feb 17, 2021
8e4e28d
fixed bug in DatasetBaseGroupLoadingOneFile
davidsebfischer Feb 17, 2021
58c808c
miscelaneous fixes (#137)
le-ander Feb 18, 2021
af20e9c
reduce comment duplication (#150)
Zethson Feb 26, 2021
bb68ca5
Cell type mapping csv files (#102)
davidsebfischer Mar 1, 2021
26a00e5
Yaml-alternative to constructor (#151)
davidsebfischer Mar 1, 2021
5b5984a
add model_id validation; closes #75 (#153)
Zethson Mar 1, 2021
23d76f1
fix loading of Rdata dataset. fixes #126 (#157)
le-ander Mar 1, 2021
c864d10
refurbished data split and load interface (#155)
davidsebfischer Mar 3, 2021
ab36d54
Dataset configs (#159)
davidsebfischer Mar 4, 2021
0fd6f8d
depreceated sample_ids (#162)
davidsebfischer Mar 4, 2021
5879b54
fix windows builds (#164)
le-ander Mar 4, 2021
9b4c75a
added entry of unconstrained ontology for doi and id into container (…
davidsebfischer Mar 4, 2021
a713ab0
None ontology matching (#169)
davidsebfischer Mar 4, 2021
f8e3446
fix year ontology (#170)
le-ander Mar 5, 2021
ab78dc6
fixed load call with function handle (#172)
davidsebfischer Mar 5, 2021
dbf4fa8
add fetal data (#171)
le-ander Mar 8, 2021
57e19c7
rewrote collapse_matrix, exported to util and wrapped into unit test …
davidsebfischer Mar 8, 2021
2b15f0b
Support samples other than from primary tissue (#173)
le-ander Mar 9, 2021
945f0a9
Add organoid datasets (#178)
le-ander Mar 11, 2021
09f372c
Moving the CLI to yaml based dataloaders (#182)
Zethson Mar 11, 2021
b6d1b50
fix data loading (#177)
davidsebfischer Mar 12, 2021
492464c
Change hard-coded path to os.path.join (#191)
Abdul-Moeed Mar 12, 2021
63c1b06
add FILE_PATH = __file__ to __init__.py in templates
Zethson Mar 15, 2021
72a25e9
add " strip for questionary answers
Zethson Mar 15, 2021
33f14d3
fix asking for DL urls for multiple ds
Zethson Mar 15, 2021
95df92e
fixed numeric label reading from mapping tsv
davidsebfischer Mar 22, 2021
9c1c4c4
Implementation of test-dataloader (#211)
Zethson Apr 7, 2021
051dbed
Fix loading (#197)
davidsebfischer Apr 12, 2021
ad39079
fixed various bugs (#218)
davidsebfischer Apr 12, 2021
d04360a
updated azimuth dataloder to new structure (#217)
lauradmartens Apr 16, 2021
8336361
Distributed store (#213)
davidsebfischer Apr 16, 2021
1a8cbe8
finished target universe building and caching (#222)
davidsebfischer Apr 16, 2021
e6585a4
Fix GenomeContainer (#223)
davidsebfischer Apr 16, 2021
9679861
fixed EFO 10X labels and moved EBI ontology interface over to network…
davidsebfischer Apr 19, 2021
ef8eece
Generator restructuring (#225)
davidsebfischer Apr 19, 2021
56a2c1e
Ebi fix (#226)
davidsebfischer Apr 19, 2021
5a24c5a
Refactor streamlining (#219)
le-ander Apr 22, 2021
f3a2a3e
updated streamlining in store script
davidsebfischer Apr 26, 2021
cb6b428
added unit test for feature streamlining and fixed store script
davidsebfischer Apr 26, 2021
ec3aca3
updated store writing and added a unit test
davidsebfischer Apr 26, 2021
310a373
updated write_distrbuted_store documentation
davidsebfischer Apr 26, 2021
3ae850f
removed genome annotation defaults
davidsebfischer Apr 26, 2021
9902d6e
updated store defaults
davidsebfischer Apr 26, 2021
3986c33
removed backed object writing scripts
davidsebfischer Apr 26, 2021
e4d3b02
fixed input to make_index_unique
davidsebfischer Apr 26, 2021
95ef15a
fixed column name retaining in obs and var cleaning
davidsebfischer Apr 26, 2021
dd75ae8
Bugfix (#233)
le-ander Apr 26, 2021
507264c
Ontology call for cellxgene (#235)
davidsebfischer Apr 26, 2021
2b36169
Ontology call for cellxgene (#234)
davidsebfischer Apr 26, 2021
1830168
Distributed store update (#236)
davidsebfischer Apr 26, 2021
1f2ce9d
adapted developmental stage label to ontology (#237)
davidsebfischer Apr 26, 2021
a50b45f
D10 1038 s41586 019 1654 9 dev ontology (#239)
davidsebfischer Apr 26, 2021
101ba89
D10 1038 s41586 019 1654 9 dev ontology (#238)
davidsebfischer Apr 26, 2021
be2516d
D10 1038 s41586 020 2157 4 dev stage (#240)
davidsebfischer Apr 26, 2021
2fa3957
added feature matrix collapse and uns clearning (#241)
davidsebfischer Apr 26, 2021
742b593
fixed bug in config writing of store and built unit test (#242)
davidsebfischer Apr 27, 2021
562a668
fixed dev stage assignment (#243)
davidsebfischer Apr 27, 2021
5c872d6
fixed subsetting (#244)
davidsebfischer Apr 27, 2021
b7ddeda
added skipping to store writing and fixed hcl bug (#247)
davidsebfischer Apr 27, 2021
ca82c32
fixed script (#248)
davidsebfischer Apr 27, 2021
25ade54
fix development stage annotation in pisco mouse data. closes #245 (#249)
le-ander Apr 27, 2021
5b0a7b9
Fix meta writing (#246)
le-ander Apr 27, 2021
8a20260
reduce step size in matrix copying to get tid of "cannot convert inte…
le-ander Apr 28, 2021
63f559d
change read from backed=True (r+) to r so that read permission suffic…
Hrovatin Apr 28, 2021
ca70c5b
fixed store to estimator interface and added further unit tests on st…
davidsebfischer Apr 29, 2021
914920c
fixed config scripts (#257)
davidsebfischer Apr 30, 2021
e2c27ff
fix filename in config load (#258)
le-ander Apr 30, 2021
1353efc
Store fix (#259)
davidsebfischer Apr 30, 2021
a40fd0a
fix download function
le-ander May 5, 2021
e07f850
Add polioudakis data and small fixes (#261)
le-ander May 6, 2021
80641f7
renamed sfaira lint to validate (#263)
Zethson May 11, 2021
f89eddb
add sfaira dataloader schema (#265)
Zethson May 11, 2021
4abb78d
Fix error due to undefined variable. (#266)
Abdul-Moeed May 12, 2021
c23b4c8
Fix model_id docstring to suggest correct format. (#267)
Abdul-Moeed May 17, 2021
7372c4f
partially fix test
Zethson May 28, 2021
f03f833
Train fixes (#260)
davidsebfischer May 28, 2021
c3198ed
Dev merge (#278)
davidsebfischer May 28, 2021
950abad
Merge branch 'master' into dev
davidsebfischer May 28, 2021
e615869
split template test into annotate and test (#272)
Zethson May 28, 2021
a5051d4
updated portal link (#281)
davidsebfischer May 31, 2021
9fc08f1
Merge branch 'dev' into release
davidsebfischer May 31, 2021
5b49668
merge release into dev
davidsebfischer May 31, 2021
4764d81
added new line
davidsebfischer May 31, 2021
cfe13cc
Dev merge (#285)
davidsebfischer May 31, 2021
9f35808
Merge branch 'dev_merge' into dev
davidsebfischer May 31, 2021
4305133
Dev release merge (#294)
davidsebfischer Jun 8, 2021
680511c
Release v0.3.0 (#290)
davidsebfischer Jun 8, 2021
6e81f75
Master dev merge (#298)
davidsebfischer Jun 9, 2021
4b9fc64
Dev release merge (#300)
davidsebfischer Jun 9, 2021
7a2d677
Dev release merge (#301)
davidsebfischer Jun 9, 2021
3e2e797
renamed ui test folder (#302)
davidsebfischer Jun 9, 2021
8793f97
Merge branch 'release' into dev
davidsebfischer Jun 9, 2021
2ab8d9b
release v0.3.1 (#297)
davidsebfischer Jun 9, 2021
4bcd88a
removed outdated files
davidsebfischer Jun 9, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 4 additions & 13 deletions sfaira/unit_tests/versions/test_zoo.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,14 +38,6 @@ def simulate(self):
"""
pass

def _test_kipoi_call(self):
"""
Test whether kipoi_experimental model call works.

:return:
"""
self.zoo.call_kipoi()

def _test_basic(self, id: str):
"""
Test all relevant model methods.
Expand All @@ -56,7 +48,6 @@ def _test_basic(self, id: str):
np.random.seed(1)
self.simulate()
self.init_zoo()
# self._test_kipoi_call()
self.zoo_manual.set_model_id(id)


Expand All @@ -68,8 +59,8 @@ def init_zoo(self):
os.path.join(package_dir, '../test_data', 'model_lookuptable.csv'),
header=0, index_col=0
)
self.zoo = ModelZooEmbedding(model_lookuptable=lookup_table)
self.zoo_manual = ModelZooEmbedding(model_lookuptable=None)
self.zoo = ModelZoo(model_lookuptable=lookup_table)
self.zoo_manual = ModelZoo(model_lookuptable=None)

def test_basic(self):
self._test_basic(id="embedding_mouse_lung_vae_theislab_0.1_0.1")
Expand All @@ -86,8 +77,8 @@ def init_zoo(self):
os.path.join(package_dir, '../test_data', 'model_lookuptable.csv'),
header=0, index_col=0
)
self.zoo = ModelZooCelltype(model_lookuptable=lookup_table)
self.zoo_manual = ModelZooCelltype(model_lookuptable=None)
self.zoo = ModelZoo(model_lookuptable=lookup_table)
self.zoo_manual = ModelZoo(model_lookuptable=None)

def test_basic(self):
self._test_basic(id="celltype_mouse_lung_mlp_theislab_0.0.1_0.1")
Expand Down