Exp time kernel #595

michalk8 · 2021-05-27T20:42:46Z

IMPORTANT: Please search among the Pull request before creating one.

Title

Types of changes

Bug fix (non-breaking change which fixes an issue)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
New feature (non-breaking change which adds functionality)

Description

Add Waddington OT kernel, see related issue broadinstitute/wot#91. I've also added option to use existing obs features (PCA, imputed expression) to compute the cost matrices.
Also adds ExpTimeKernel class which can be later used to derive kernels with experimental time.

@Marius1311 I've decided against adding anything to the block diagonal, don't think it's a nice solution - user can always weight with full ConnectivitiesKernel and afaik, the GPCCA stability issues were gone (I've used small weights like 0.05 or so for connectivities). Maybe you add some nice note in .compute_transition_matrix, below Returns as

Notes
-----
...

How has this been tested?

I've added a few.

Closes

closes #551

codecov · 2021-05-27T21:53:14Z

Codecov Report

Merging #595 (6a60dad) into dev (e16e27d) will decrease coverage by 1.77%.
The diff coverage is 26.31%.

❗ Current head 6a60dad differs from pull request most recent head 1870ad7. Consider uploading reports for the commit 1870ad7 to get more accurate results

@@            Coverage Diff             @@
##              dev     #595      +/-   ##
==========================================
- Coverage   87.78%   86.01%   -1.78%     
==========================================
  Files          63       65       +2     
  Lines        7885     8107     +222     
==========================================
+ Hits         6922     6973      +51     
- Misses        963     1134     +171

Flag	Coverage Δ
unittests	`86.01% <26.31%> (-1.78%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
cellrank/external/kernels/_wot_kernel.py	`20.00% <20.00%> (ø)`
cellrank/tl/_utils.py	`82.44% <31.25%> (-1.52%)`	⬇️
cellrank/tl/kernels/_exp_time_kernel.py	`35.18% <35.18%> (ø)`
cellrank/tl/kernels/_base_kernel.py	`86.99% <50.00%> (-0.29%)`	⬇️
cellrank/external/kernels/__init__.py	`100.00% <100.00%> (ø)`
cellrank/external/kernels/_statot_kernel.py	`87.50% <100.00%> (ø)`
cellrank/tl/kernels/__init__.py	`100.00% <100.00%> (ø)`
cellrank/ul/models/_gamr_model.py	`89.79% <0.00%> (-1.37%)`	⬇️
cellrank/tl/estimators/_decomposition.py	`92.51% <0.00%> (-0.54%)`	⬇️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e16e27d...1870ad7. Read the comment docs.

Marius1311 · 2021-05-31T11:06:55Z

Hi Mike, first comment, when trying to import with otk = WOTKernel(adata) (I don't have WOT installed yet), I get

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
~/Projects/cellrank/cellrank/external/kernels/_wot_kernel.py in <module>
     17 try:
---> 18     import wot
     19 

ModuleNotFoundError: No module named 'wot'

The above exception was the direct cause of the following exception:

ImportError                               Traceback (most recent call last)
<ipython-input-20-d38dddb5f66c> in <module>
----> 1 otk = WOTKernel(adata)

~/Projects/cellrank/cellrank/external/kernels/_wot_kernel.py in __init__(self, adata, backward, time_key, compute_cond_num)
     57         compute_cond_num: bool = False,
     58     ):
---> 59         super().__init__(
     60             adata,
     61             backward=backward,

~/Projects/cellrank/cellrank/external/_error_mixin.py in __init__(self, *args, **kwargs)
     24 
     25     def __init__(self, *args: Any, **kwargs: Any):
---> 26         raise ImportError(self.__import_error_message__) from self.__error__
     27 
     28     def __init_subclass__(cls, error: Optional[Exception] = None, **kwargs: Any):

ImportError: Unable to import the kernel.

This is of course expected as I don't have the package installed - however, could we print a message, telling me to install it from pip?

michalk8 · 2021-05-31T11:38:07Z

Hey @Marius1311 , done. I've also exposed .transition_maps + changed the last time pair block not to sparse diagonal, but to normalized all 1s.

Marius1311 · 2021-05-31T12:46:27Z

I installed WOT and initalized the kernel, as above. I get the following warning: WARNING: Time categories are not ordered. Using default order. What is default order in this context?

Marius1311 · 2021-05-31T12:51:12Z

Next point (I'm just going through as I experiment, please excuse the absence of structure here): I print my initialized kernel object, not transition matrix computed yet: print(otk) gives <WOTKernel[cost_matrices=default, solver=duality_gap, growth_rate_key=None, epsilon=0.05, lambda1=1, lamdda2=50, growth_iters=1]>

I'm just wondering why so many of these are object level properties - I suppose they are specified when I compute a transition matrix? Just as a comparison with the PseudotimeKernel:

otk = WOTKernel(adata, time_key='day', )
pk = PseudotimeKernel(adata)

Print otk gives:
<WOTKernel[cost_matrices=default, solver=duality_gap, growth_rate_key=None, epsilon=0.05, lambda1=1, lamdda2=50, growth_iters=1]>

Print pk gives

i.e. the pseudotime kernel is "blank" before I compute the transition matrix, the WOT kernel has lots of attributes already. Why is it different?

michalk8 · 2021-05-31T12:56:11Z

I installed WOT and initalized the kernel, as above. I get the following warning: WARNING: Time categories are not ordered. Using default order. What is default order in this context?

It's in ascending order, changed the message.

I'm just wondering why so many of these are object level properties - I suppose they are specified when I compute a transition matrix?

There's even more if you supply kwargs, see all options here. I don't think this is problematic though.

Marius1311 · 2021-05-31T12:57:36Z

I updated my post from above - I don't think it's problematic either, I just wonder why we handle it differently here than in other kernels.

Marius1311 · 2021-05-31T13:00:34Z

Next point: I think otk.compute_transition_matrix() should not absorb epsilon, lambda1, lamdda2, growth_iters into the kwargs - these are really important arguments and the user should think about them when running the method. Can you please add these to the docstring explicitly, and also emhasize that the user should check out the original documentation (provide a link) before running the function?

michalk8 · 2021-05-31T13:03:50Z

I print my initialized kernel object, not transition matrix computed yet: print(otk) gives <WOTKernel[cost_matrices=default, solver=duality_gap, growth_rate_key=None, epsilon=0.05, lambda1=1, lamdda2=50, growth_iters=1]>

Can't reproduce, only after .compute_transition_matrix you see all the params.

should not absorb epsilon, lambda1, lamdda2, growth_iters into the kwargs

Ok.

and also emhasize that the user should check out the original documentation (provide a link) before running the function?

Can't do, afaik there are no online docs. https://broadinstitute.github.io/wot/ only has tutorials + cmd line interface + additional stuff.

Marius1311 · 2021-05-31T13:50:58Z

I print my initialized kernel object, not transition matrix computed yet: print(otk) gives <WOTKernel[cost_matrices=default, solver=duality_gap, growth_rate_key=None, epsilon=0.05, lambda1=1, lamdda2=50, growth_iters=1]>

Can't reproduce, only after .compute_transition_matrix you see all the params.

You are right, I must have messed it up somehow. All good here.

should not absorb epsilon, lambda1, lamdda2, growth_iters into the kwargs

Ok.

and also emhasize that the user should check out the original documentation (provide a link) before running the function?

Can't do, afaik there are no online docs. https://broadinstitute.github.io/wot/ only has tutorials + cmd line interface + additional stuff.

It's fine to link to that page, it's all explained in the tutorials in length, that's where I got the info from.

Marius1311 · 2021-05-31T13:51:37Z

Is otk.compute_transition_matrix() parallelised across time-point pairs?

Marius1311 · 2021-05-31T13:54:11Z

Some more open points:

We need an example for the usage in the docs. It's currently not clear to me how that data is expected to be pre-processed before feeding it into the kernel
We should brainstorm convenient ways to initialize the growt_rate, essentially folllowing the second code cell in https://nbviewer.jupyter.org/github/broadinstitute/wot/blob/master/notebooks/Notebook-2-compute-transport-maps.ipynb

Marius1311 · 2021-05-31T13:59:15Z

Another point: can we write the final growth rates (last round of iteration) back to adata.obs please? I think that's an important quantity.

michalk8 · 2021-05-31T14:44:57Z

Is otk.compute_transition_matrix() parallelised across time-point pairs?

No, and I don't think it should be - the OT solver is already consuming all cores.

We need an example for the usage in the docs. It's currently not clear to me how that data is expected to be pre-processed before feeding it into the kernel

Ok, can you please add this? I also don't know which pre-processing should be used (would be best), just used the imputed data from scVelo (standard preprocessing).

Another point: can we write the final growth rates (last round of iteration) back to adata.obs please? I think that's an important quantity.

Ok, done.

We should brainstorm convenient ways to initialize the growt_rate, essentially folllowing the second code cell in https://nbviewer.jupyter.org/github/broadinstitute/wot/blob/master/notebooks/Notebook-2-compute-transport-maps.ipynb

How do you propose to do that? We don't have neither prolif./apop. info. 2 options I have in mind:

optionally pass these 2 cols to __init__ and compute the BD process
pass gene sets as in notebook 1, cell 4 to compute the scores and get prolif./apop. info from that
is more general/flexible, 2. is more convenient.

Marius1311 · 2021-05-31T15:55:31Z

Hi Mike!

parallelisation: okay, got it.
workflow: okay, will add. Please add a key like data_key or so, where one can specify which layer/.X to use
actually, for the PCA, do we use the precomputed one now? i.e. not pairwise PCAs? Since this is not default in WOT, should inform/warn the user about this and potentially give a choice.
growth rates: I would go for option 1, i.e. read cell-level proliferation/apoptosis scores from .obs. There is already a scanpy function for computing scores from gene sets, so we shouldn't reinvent the wheel. Can you check whether their gene lists are available somewhere in the github?

Thanks! We're getting there.

michalk8 · 2021-05-31T16:30:09Z

workflow: okay, will add. Please add a key like data_key or so, where one can specify which layer/.X to use

This is already done: cost_matrices can be a key in .layers or even .obsm.

actually, for the PCA, do we use the precomputed one now? i.e. not pairwise PCAs? Since this is not default in WOT, should inform/warn the user about this and potentially give a choice.

We're using WOT's default now (pairwise PCA).

growth rates: I would go for option 1, i.e. read cell-level proliferation/apoptosis scores from .obs. There is already a scanpy function for computing scores from gene sets, so we shouldn't reinvent the wheel. Can you check whether their gene lists are available somewhere in the github?

Ok, agreed, will add this.

michalk8 · 2021-05-31T17:23:25Z

growth rates: I would go for option 1, i.e. read cell-level proliferation/apoptosis scores from .obs. There is already a scanpy function for computing scores from gene sets, so we shouldn't reinvent the wheel. Can you check whether their gene lists are available somewhere in the github?

Done as .compute_initial_growth_rates. Please check the docs and see if you'd improve anything.
Currently envisioned workflow:

# preprocessing

ok = cr.external.kernels.WOT(adata, time_key=...)
ok.compute_initial_growth_rates("prolif_key", "apop_key", ..., key_added="initial_growth_rates")  # optional
ok.compute_transition_matrix(..., growth_rate_key="initial_growth_rates")

Marius1311 · 2021-06-01T08:28:28Z

Also, I think we should add a tutorial for this, maybe in combination with StationaryOT. I can take care of this.

Marius1311 · 2021-06-01T08:37:55Z

Is scanpy used under the hood to compute PCA?

michalk8 · 2021-06-01T08:40:35Z

New question: are time points actually re-ordered in AnnData? Or how do you make sure the final transition matrix has the same row and column ordering as the AnnData object?

The transition matrix has the same order of cells as the original one.

Is scanpy used under the hood to compute PCA?

No. I'm staying 100% true to WOT's implementation.

Also, I think we should add a tutorial for this, maybe in combination with StationaryOT. I can take care of this.

Sounds good, I'd wait for 1.4.

Marius1311 · 2021-06-01T08:41:39Z

New question: are time points actually re-ordered in AnnData? Or how do you make sure the final transition matrix has the same row and column ordering as the AnnData object?

The transition matrix has the same order of cells as the original one.

So will the row/column ordering be the same as in adata.obs?

Is scanpy used under the hood to compute PCA?

No.

What do you use? Is highly variable gene annotation taken into account?

michalk8 · 2021-06-01T08:53:07Z

So will the row/column ordering be the same as in adata.obs?

Yes.

What do you use? Is highly variable gene annotation taken into account?

No. Can add option to subset the adata before to use only HVGs only.

Marius1311 · 2021-06-01T08:57:33Z

Re HVGs, that would be good - I would have a parameter use_highly_variable_genes: bool = True that searches adata.var for a field called highly_variable. If it's present and it contains bool annotations, these are used to select genes, i.e. internal subsetting. If it's not present (despite use_highly_variable_genes=True), a warning should be issued and all genes should be used.

Marius1311 · 2021-06-01T09:11:33Z

We should also rename that OTKernel - That should really be called StationaryOTKernel, might get confusing otherwise.

Marius1311 · 2021-06-01T09:56:22Z

For the record, I used the WOTKernel on the lung data and plotted the transition matrix, once without cell-ordering (Fig. 1) adn once with cell-ordering by time-point (Fig. 2) I see the expected pattern, i.e. radomess in. Fig. 1, off-diagonal blocks in Fig. 2. Why did you decide not to decide self-loops only (diagonal only) for the last time-point?

Also, in Fig. 3, I show estimated growth rates in each iteration for 5 iterations. They seem to converge very slowly.

Fig. 1

Unordered:

Fig. 2

Ordered by time-point:

Fig. 3

Estimated growth rates (not showing the uniform initial growth rates)

Marius1311 · 2021-06-01T10:03:32Z

Note to self: we should add to the docstring how we assign transition probs to the last time-point.

Marius1311 · 2021-06-01T10:09:28Z

Random walks are currently very "jumpy":

Could be a consequence of the way in which we assign transition probs to the last time-point - we will just get random jumps in the last time point, all over the UMAP. Probably not ideal.

michalk8 · 2021-06-01T10:12:25Z

Why did you decide not to decide self-loops only (diagonal only) for the last time-point?

Because I can't justify using only diagonal.

Note to self: we should add to the docstring how we assign transition probs to the last time-point.

Agreed, once we converge on solution, could you please update it?

Marius1311 · 2021-06-01T10:14:50Z

Why did you decide not to decide self-loops only (diagonal only) for the last time-point?

Because I can't justify using only diagonal.

I get that point - in case we know nothing, you say let's use the maximum entropy distribution here which would be uniform. However, taking into account computational considerations, I would go for sparse diagonal because it's

sparse, i.e. takes less memory
terminates random walks. Otherwise, we essentially can't use that plotting function because you will get jumps all over the umap.

Since there is no obvious choice here, how about we leave it to the user? t_max_map: str = 'diagonal' (or uniform). Would be convenient for me to check random walks.

Note to self: we should add to the docstring how we assign transition probs to the last time-point.

Agreed, once we converge on solution, could you please update it?

Will do!

michalk8 · 2021-06-01T10:57:47Z

Since there is no obvious choice here, how about we leave it to the user? t_max_map: str = 'diagonal' (or uniform). Would be convenient for me to check random walks.

Done as uniform: bool = False. I've also changed the cmat messages from debug -> info.

Marius1311 · 2021-06-01T10:58:34Z

Fantastic, thanks!

Marius1311 · 2021-06-01T13:43:29Z

I found the gene sets to compute the growth rates. See Fig. 1 for the proliferation and apoptosis scores and Fig. 2 for the combined computed growth rate. This matches annotations well, see Fig. 3, note the proliferation cluster gets higher growth rate.

Fig. 1

proliferation (left) and apoptosis (right) scores

Fig. 2

Combined growth rate

Fig. 3

clusters. Note proliferation cluster

Marius1311 · 2021-06-01T13:48:30Z

So depending on the initialization, we can end up with growth rates that are quite different. See Fig. 1 below for growth rates computed in 3 iterations, starting from uniform initialization and Fig. 2 for 3 iterations, starting from the score-based initial growth rate computed above.

Fig. 1

Fig. 2

Marius1311 · 2021-06-01T13:53:22Z

Compare also transition matrices from uniform (Fig. 1) or score-based (Fig. 2) initialization.

Fig. 1

uniform

Fig. 2

score-based

Marius1311

This is fine from my side. Please review my suggested workflow and modify if needed, in particular, if you think it can be shortened. Otherwise, feel free to merge.

Marius1311 · 2021-06-01T14:10:55Z

We'll have to look into downstream applications, i.e. inferrint the long-range temporal couplings, but that can be done after this merge, which will be the initial step.

michalk8 · 2021-06-01T14:11:48Z

I found the gene sets to compute the growth rates. See Fig. 1 for the proliferation and apoptosis scores and Fig. 2 for the combined computed growth rate. This matches annotations well, see Fig. 3, note the proliferation cluster gets higher growth rate.

Nice! Should we include these or not?

Compare also transition matrices from uniform (Fig. 1) or score-based (Fig. 2) initialization.

Visually, they seem similar.

This is fine from my side. Please review my suggested workflow and modify if needed, in particular, if you think it can be shortened. Otherwise, feel free to merge.

Ok, thanks, will take a look at this.

Marius1311 · 2021-06-01T14:15:53Z

For the gene sets, they will be specific to the organism (mouse, in this case), so I wouldn't include them. We can add a link to the tutorial again where they can be downloaded.

* Smoothing for Pseudotime kernel (#514) * Rename PalantirKernel -> PseudotimeKernel * Add initial abstraction * Add Palantir scheme * Add soft PT * Remove old test * Fix wrong condition * Update docs and defaults * Fix Pseudotime kernel docstring * Fix more docstrings * Update docs * Add tests, pin down numba Co-authored-by: Marius Lange <marius.lange@t-online.de> * Add compute_projection (#520) * Add compute_projection * Swap order of args * Add tests * Fix params, add plotting test * [ci skip] Update docstring * Pin down numba Co-authored-by: Marius Lange <marius.lange@t-online.de> * Fix bugs, QoL improvements (#535) * Fix bugs, QoL improvements * Add QoL changes v2 * Enable CI * Cytotrace kernel (#527) * Add CytoTRACE kernel * Add new kernel to docs * Fix literal * Work on the docstring * Rename variables * Rename the gcs variable * Update the docstring * Add docstring to kernel class * Move the workflow example * [ci skip] Enable CI * Move to CytoTRACEKernel * Improve logging, remove arbitrary restriction * Do not use short kernel names * Add CytoTRACE test skeletons * Set use_raw=False, add tests * Improve tests * Fix raw and normal adata with different n_vars * Add cytotrace score to adata_200 * Add regression test for ct_score * Add regression test for gene_corr * Fix tests Co-authored-by: Marius Lange <marius.lange@t-online.de> * Add external model (#522) * Add external module, port statOT * Add docs * Add typing_extensions * Add some sanity check * Move external to separate page * Add tests * Temp. pin down numba * Add explicit POT mention in docs, add to external * Use raise from * Fix init subclass * Add rest error mixins * Add tox * Update CONTRIBUTING.rst * Update CONTRIBUTING.rst * Fix sections * Update CONTRIBUTING.rst * Update test requirements * [ci skip] Start addressing comments * [ci skip] Update CONTRIBUTING.rst * Add tutorials to CONTRIBUTING.rst * Different import, enable CI on dev * Update GA issues/discussions * Update OTKernel to reflect new changes * Fix wrong import * [ci skip] Update dev install, docs * [ci skip] Fix link * Remove edit_on_github (#538) * Random walk simulation (#537) * Add random walk class * Parallelize * Random walk plotting * Add docstrings * [ci skip] Fix some imports * [ci skip] Add error messages * Add random walks test skeleton * [ci skip] Add plotting test skeletons * Add RW tests * Add plotting tests * Add legend loc for start/stop indices, update docs * Work on docstrings * Add first/last cell description Co-authored-by: Marius Lange <marius.lange@t-online.de> * Fix dist -> conn, update docstrings, remove copy (#536) * Fix dist -> conn, update docstrings, remove copy * Change default * Work on the docstring * Exchange k for fract_to_keep * Make fract_to_keep a float * Fix tests Co-authored-by: Marius Lange <marius.lange@t-online.de> * Fix plot lin drivers (#543) * Fix lineage drivers plotting * Add tests * Add forgotten GT images * Fix nan in emb when projection tmat (#546) * [ci skip] Add Zebrafish dataset (#547) * [ci skip] Remove fig.show() * Fix some logging messages * Release notes 1.3 (#539) * Fix docs, move latest additions * Change latest to stable * Add most of the release notes * [ci skip] Update release notes - new dataset * Go over release notes * Update release notes: move Zebrafish, add pygpcca * Correct tutorial names * Regenerate some figures to accomodate mpl==1.4.0 Co-authored-by: Marius Lange <marius.lange@t-online.de> * Release 1.3 (#548) * Fix typos, update _static * Add forgotten docstring to abstract method * Fix tutorial links in docs * Update the index * Make it a bit shorter * Add links and contributions seciton * Fix spelling * Fix same plot (#556) * Fix reading from adata * Regenerate figure * Add warning * Compute Schur only for num. states, not +1 * Add test * Update README.rst * Minor update * Remove solver heuristics (#558) * Remove solver/PETSc heuristic * Change tol to 1e-6 * Change test tol, regen image * Add check probs sum to 1 (#566) * Add abs. probs check * Add nicer message, add tests * Increase tolerance to 1e-3 * Fix test * [ci skip] Fix test names * Fix not using percentile (#568) * Fix wrong pyGPCCA attribute in logging * First version of req. pruning (#571) * Fix inconsistent state (#563) * Fix inconsistent cat/cont macrostates/lineages * Add tests, fix 1 minor bug * Add force recompute tmat (#577) * Add force tmat recomputation, fix typo bug * Add test * Change circ. projection TSP default heuristic * Remove -1 in hard scheme, modify test (#582) * [ci skip] Remove sphinx-paramlinks (#584) * Pt kernel changes (#583) * Change PT kernel defaults, remove perc/dnorm * Remove density normalize from tests * Fix docs * [ci skip] Remove sphinx-paramlinks * Fix test * Remove dens correction from docstring * Dont force sparse tmat if already dense (#586) * Do not force tmat to be sparse * Add test * Fix tests expecting sparse matrix * Parallelize pt kernel (#587) * Parallelize PT kernel * [ci skip] Add forgotten kwargs in docs * Fix custom connectivities key (#590) * Fix plot_coarse_T annotation * Exp time kernel (#595) * Resurrect exp. time kernel * Add WOT skeleton class * Add WOT * Add tests, add wot to external requirements * Skip WOT tests on CI, update WOT installation * Better import error message, save tmats, last 1s * Clear order info message, fix test skip * Expose more args * Save estimated growth rates to adata * [ci skip] Add compute_initial_growth_rates * Add to the docstrings * Add HVGs subset, fix docs, rename StatOT kernel * Change debug to info, add uniform * Add example workflow * [ci skip] Polish docs Co-authored-by: Marius Lange <marius.lange@t-online.de> * Wot kernel graph (#600) * Don't require connectivities for external kernels * Fix commented skip * Enable bibtex (#601) * Use bibtex for references * Polish docs * Fix dois * Update CytoTRACE workflow * Update WOT example workflow * Remove old examples, add new skeleton ones (#602) * Remove old examples, add new skeleton ones * Move old release notes * Add 2 new examples - projection and RW * Update latest additions * Update RW docstring * Update docstring to manually set terminal states (#607) * Update docstring to manually set terminal states * Fix bug with doc insertion * Add option to set term states from clusters Co-authored-by: Michal Klein <michal.klein@protonmail.com> * Add option to use connectivities for WOT (#609) * Fix tqdm/ipywidgets (#616) * Vein plot (#615) * Start vein plot * Specialize plot_flow for exp. time kernel * Add first working impl. * Update plotting * Add plotting options * Add flow threshold * Rename argument * Fix cyclic import * Fix plot_flow in ExpTimeKernel * Add tmat kernel * Subset clusters, fix bwd * Allow cluster subset and ordering * Add time point filtering * [ci skip] Fix bugs, start abstraction * [ci skip] Clean code * [ci skip] Remove dicts * [ci skip] Fix indexing error * Remove plot_flow specification for TMatKernel * [ci skip] Control ticks * [ci skip] Order legend * Polish docs * Polish docs * Add option to remove empty clusters * Fix linting * [ci skip] Fix logging and docs * Add skeleton tests, fix typo, name to single flow * Add first batch of tests * Add plotting tests * [ci skip] Fix doi * [ci skip] Fix exp. time kernel copy (#623) * Fix lineage drivers to allow for combinations (#624) * Fix parallelize too many jobs (#633) * Cluster lin clusters (#634) * [ci skip] Add cluster key to cluster_lineage * Fix limits * Use more xticks * Add multiple covariates, expose ratio and cmap * Add tests * Add WOT dataset infra (#631) * Add WOT dataset infra * Add figshare link * Fix indentation in docs, missing dot. Co-authored-by: Marius Lange <marius.lange@t-online.de> * Increate timeout * Prolif apop markers (#630) * Add marker fetching class * Add prolif/apop infra * Remove rat * Add human markers * Add mouse markers * Add references * Add test * Minor dosctring changes Co-authored-by: Marius Lange <marius.lange@t-online.de> * Speed up calculation of absorption probabilities (#638) * Refactor `compute_absorption_probabilities` Refactors the calculation of absorption probabilities to first define a single artificial cell in the macro state by summing all cells identifying the macro state. This is equivalent to solving the linear system for each individual cell and then summing the absorption probabilities. * Update test_cflare.py Updates `TestCFLARE.test_abs_probs_negative` to work correctly with the refactored version of `BaseEstimator.compute_absorption_probabilities`. Co-authored-by: michalk8 <46717574+michalk8@users.noreply.github.com> * Stacked histogram (#641) * Add plot macrostate composition * Set margins * Fix cyclic import * Add test skeletons * Add tests * Remove return ax test * Set yticks * Regenerate figures Co-authored-by: Marius Lange <marius.lange@t-online.de> * Log odds (#642) * Initialize log_odds * Move Marius' function * Implement log odds * Add docs, fix TODOs * Add test skeletons * Add colorbar * Add legend_loc * Fix categorical bug, jitter, seed, add tests * Remove testing artifact * Fix copy * Change threshold * Add show * Add key-specific threshold * Add fontsize * Add xticks step size * Add tests, fix tick step size None * Change verbosity to debug * Add plot gene corr (#640) * Add plot gene corr * Add test skeletons * Add tests, fix raw bug * Fix adjustText * Change defaults, ERROR logging * Fix using wrong params Co-authored-by: Marius Lange <marius.lange@t-online.de> * Update precommit (#645) * Update precommit * Update reqs, setup.py, pyproject.toml * Install Cython for 3.9 * Install Cython v2 * POT v3 * Try latest numba version * Disable 3.9 CI * Try installing umap-learn on CI/tox * Fix macOS, docs umap-learn version * Fix docs build * Add some references * Update release notes (#647) * Update release notes * Pin numba in reqs again * Update test reqs., CI Python hash * Purge CI cache * Remove 3.9 CI * Add links to index.rst * Finalize release notes * Fix reference, add missing doi, download notebook Co-authored-by: Marius Lange <marius.lange@t-online.de> Co-authored-by: Marius Lange <marius.lange@t-online.de> Co-authored-by: Marius Lange <marius.lange@tum.de> Co-authored-by: Philipp Weiler <philipp.weiler@tum.de>

michalk8 added 5 commits April 19, 2021 21:58

Resurrect exp. time kernel

d4e91a5

Add WOT skeleton class

6cc217e

Merge branch 'dev' into exp_time_kernel

845a860

Add WOT

e4cd090

Add tests, add wot to external requirements

f43d902

michalk8 requested a review from Marius1311 May 27, 2021 20:42

Skip WOT tests on CI, update WOT installation

96821a6

michalk8 mentioned this pull request May 28, 2021

WOT integration as an external kernel in CellRank broadinstitute/wot#91

Closed

Better import error message, save tmats, last 1s

ea2ecea

Clear order info message, fix test skip

10a8cce

Expose more args

e03953a

Save estimated growth rates to adata

61db168

[ci skip] Add compute_initial_growth_rates

001565d

Add to the docstrings

00a893d

Add HVGs subset, fix docs, rename StatOT kernel

c34904e

Change debug to info, add uniform

0f103f5

Add example workflow

6a60dad

Marius1311 approved these changes Jun 1, 2021

View reviewed changes

[ci skip] Polish docs

1870ad7

michalk8 merged commit 37e3ce4 into dev Jun 1, 2021

michalk8 deleted the exp_time_kernel branch June 1, 2021 14:51

michalk8 mentioned this pull request Jun 1, 2021

Real-time integration #551

Closed

6 tasks

Exp time kernel #595

Exp time kernel #595

Conversation

michalk8 commented May 27, 2021

Title

Types of changes

Description

How has this been tested?

Closes

codecov bot commented May 27, 2021 • edited Loading

Codecov Report

Marius1311 commented May 31, 2021

michalk8 commented May 31, 2021

Marius1311 commented May 31, 2021

Marius1311 commented May 31, 2021 • edited Loading

michalk8 commented May 31, 2021

Marius1311 commented May 31, 2021

Marius1311 commented May 31, 2021

michalk8 commented May 31, 2021 • edited Loading

Marius1311 commented May 31, 2021

Marius1311 commented May 31, 2021

Marius1311 commented May 31, 2021

Marius1311 commented May 31, 2021

michalk8 commented May 31, 2021

Marius1311 commented May 31, 2021

michalk8 commented May 31, 2021

michalk8 commented May 31, 2021

Marius1311 commented Jun 1, 2021

Marius1311 commented Jun 1, 2021

michalk8 commented Jun 1, 2021 • edited Loading

Marius1311 commented Jun 1, 2021 • edited Loading

michalk8 commented Jun 1, 2021

Marius1311 commented Jun 1, 2021 • edited Loading

Marius1311 commented Jun 1, 2021 • edited Loading

Marius1311 commented Jun 1, 2021 • edited Loading

Fig. 1

Fig. 2

Fig. 3

Marius1311 commented Jun 1, 2021

Marius1311 commented Jun 1, 2021

michalk8 commented Jun 1, 2021

Marius1311 commented Jun 1, 2021 • edited Loading

michalk8 commented Jun 1, 2021

Marius1311 commented Jun 1, 2021

Marius1311 commented Jun 1, 2021

Fig. 1

Fig. 2

Fig. 3

Marius1311 commented Jun 1, 2021

Fig. 1

Fig. 2

Marius1311 commented Jun 1, 2021

Fig. 1

Fig. 2

Marius1311 left a comment

Choose a reason for hiding this comment

Marius1311 commented Jun 1, 2021

michalk8 commented Jun 1, 2021

Marius1311 commented Jun 1, 2021

codecov bot commented May 27, 2021 •

edited

Loading

Marius1311 commented May 31, 2021 •

edited

Loading

michalk8 commented May 31, 2021 •

edited

Loading

michalk8 commented Jun 1, 2021 •

edited

Loading

Marius1311 commented Jun 1, 2021 •

edited

Loading

Marius1311 commented Jun 1, 2021 •

edited

Loading

Marius1311 commented Jun 1, 2021 •

edited

Loading

Marius1311 commented Jun 1, 2021 •

edited

Loading

Marius1311 commented Jun 1, 2021 •

edited

Loading