Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make VS hail independent for the python API #237

Open
rocreguant opened this issue Feb 13, 2024 · 0 comments
Open

Make VS hail independent for the python API #237

rocreguant opened this issue Feb 13, 2024 · 0 comments

Comments

@rocreguant
Copy link
Collaborator

No description provided.

NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
DEV: Created standalone class for Random Forest Models (#237)
5edfbfa 
FEAT: Implemented RF class method for fitting the model

FEAT: Implemented RF class method for obtaining importance analysis
from a fitted RF

FEAT: Implemented RF class method for returning oob error

FEAT: Implemented RF class method for obtaining FDR
from a fitted model

FEAT: Implemented RF class method for exporting forest to JSON

REFACTOR: Make RF model available at package level

CHORE: Added type checking to all methods
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
DEV: Updated varspark python wrapper (#237)
80a9c59 
REFACTOR: Removed FeatureSource and
ImportanceAnalysis classes from core

REFACTOR: Added FeatureSource import so features
can be returned as a class instantiation
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
DEV: Created standalone FeatureSource class in separate file (#237)
23520ec 
REFACTOR: Removed imp analysis and model training

FEAT: Added conversion from feature to RDD (python)

FEAT: Added conversion from feature to RDD (scala)

CHORE: Added type checking
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
REFACTOR: Remove unecessary hail import for hail rf wrapper
4560998 
due to import order warning (#237)
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
DEV: Created standalone ImportanceAnalysis class in
b8b39fd 
separate wrapper file (#237)

REFACTOR: Updated important_variables and variable_importance
methods to convert to pandas DataFrames
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
DEV: Updated ImportanceAnalysis scala class (#237)
4bfaac9 
REFACTOR: Removed model training from object instantation and
updated class to accept a model as a parameter

REFACTOR: Added normalisation as an optional parameter for
variable importance methods

FEAT: Updated variableImportance method to include splitCount in return as it is required for local FDR analysis
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
DEV: Created scala function that trains a forest
0fc736f 
and passes back to python context (#237)
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
REFACTOR: Removed model definition and training
ea069d6 
from importAnalysis method of AnalyticsFunctions (#237)
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
DEV: Create no-hail equivalent of JSON model export (#237)
e08f12a 
FIX: Update export function to process trees in batches,
instead of collecting the whole forest as a map as this
led to OOM errors on large forests
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
REFACTOR: Update importance API test cases to reflect changes (#237)
ddc5912
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
REFACTOR: Update reproducibility test case to reflect changes (#237)
f6d40d4
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
DEV: Update python unit testing (#237)
3356d9a 
REFACTOR: Refactor to mirror changes to python wrapper

FEAT: Include FDR calculation in unit test
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
DEV: Create no hail lfdr class (#237)
59f40bc 
FEAT: Implement function for manhattan plotting negative log p values
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
DEV: Create temp hail notebook for testing JSON export OOM (#237)
3f8066b
NickEdwards7502 added a commit that referenced this issue Sep 11, 2024
@NickEdwards7502
DEV: Create temp notebook for demonstrating VS functionality
de29b45 
without hail (#237)
NickEdwards7502 added a commit that referenced this issue Sep 13, 2024
@NickEdwards7502
DEV: Add covariate import wrapper function (#237)
fe2db4c 
STYLE: Format with black
NickEdwards7502 added a commit that referenced this issue Sep 13, 2024
@NickEdwards7502
DEV: Create python class for covariate imports (#237)
a9b9570
NickEdwards7502 added a commit that referenced this issue Sep 13, 2024
@NickEdwards7502
STYLE: Format with black (#237)
3ea4c8c
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
REFACTOR: Remove covariatesource as not required (#237)
8f11e62
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
STYLE: Format with black (#237)
d671f35
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
DEV: Add wrapper functions for covariate support (#237)
209a463 
FEAT: Add wrapper class for importing covariates

FEAT: Add wrapper class for unioning features and covariates
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
STYLE: Format with black (#237)
b94afcc
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
STYLE: Format with black (#237)
04daae2
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
DEV: Update lfdr to support covariates (#237)
30732ba 
REFACTOR: Include covariate filtering in manhattan plot function

STYLE: Format with black (#237)
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
STYLE: Format with scalamft (#237)
3381e68
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
DEV: Update VSContext to support covariates (#237)
37f4193 
FEAT: Add functions for importing std and transposed CSVs

FEAT: Add function for unioning features and covariates
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
DEV: Update std CSV features to support optional variable type specs (#…
dfae3c2 
…237)
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
DEV: Create class for returning union of features and covariates (#237)
9733844
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
CHORE: Add reproducibility test that includes covariates (#237)
dd32e0f
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
CHORE: Remove print statements from RF reproducibility test (#237)
769ce76
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
CHORE: Update no hail notebook to include covariates (#237)
e9a23cf
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
DEV: Update fdr estimation to return p-value cutoff (#237)
4379b0a
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
FIX: Correct significance line plot to use cutoff p-value (#237)
d2048d0
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
REFACTOR: Update pairwise operation tests based on import changes (#237)
2416b8e 
Reference changed to importTransposedCSV
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
CHORE: Update nohail notebook with significance line changes (#237)
1529bd8
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
FIX: Add covariate type parameter to python wrapper for std csv (#237)
12d6137
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
DEV: Replace to_df functionality with head (#237)
4df6e32 
REFACTOR: Remove python component of converting Feature RDD to pandas

FEAT: Add RDD slice to DF function
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
DEV: Update FeatureSource dataframe conversion (#237)
b1fe760 
REFACTOR: Remove conversion of whole RDD to DataFrame

FEAT: Add function for slicing rows and columns and converting to DF
NickEdwards7502 added a commit that referenced this issue Sep 19, 2024
@NickEdwards7502
CHORE: Update nohail demo to include df slicing (#237)
4506139
NickEdwards7502 added a commit that referenced this issue Sep 20, 2024
@NickEdwards7502
REFACTOR: Make ExportModel compatible with JsonRFAnalyser (#237)
07cd144
NickEdwards7502 added a commit that referenced this issue Sep 23, 2024
@NickEdwards7502
REFACTOR: Make head concrete & inherited by all feature sources (#237)
cef13f4
NickEdwards7502 added a commit that referenced this issue Sep 23, 2024
@NickEdwards7502
FIX: Update regex import for lfdr DataFrame parsing(#237)
b6f4e3b
NickEdwards7502 added a commit that referenced this issue Oct 2, 2024
@NickEdwards7502
CHORE: Remove cutoff pvalue print statement (#237)
9b7f83b
NickEdwards7502 added a commit that referenced this issue Oct 2, 2024
@NickEdwards7502
CHORE: Remove VSHail python wrapper files (#237)
f66f445
NickEdwards7502 added a commit that referenced this issue Oct 2, 2024
@NickEdwards7502
CHORE: Remove VSHail Random Forest scala class (#237)
e6e637c
NickEdwards7502 added a commit that referenced this issue Oct 2, 2024
@NickEdwards7502
CHORE: Update spark context test to use non-hail Kryo registrator (#237)
8f82d29
NickEdwards7502 added a commit that referenced this issue Oct 2, 2024
@NickEdwards7502
CHORE: Remove test cases specifically for hail (#237)
1be8d66
NickEdwards7502 added a commit that referenced this issue Oct 2, 2024
@NickEdwards7502
CHORE: Remove hail from maven build (#237)
bd53f27
NickEdwards7502 added a commit that referenced this issue Oct 2, 2024
@NickEdwards7502
REFACTOR: Update lfdr filename and python wrapper reference (#237)
45ae200
NickEdwards7502 added a commit that referenced this issue Oct 2, 2024
@NickEdwards7502
CHORE: Remove hail unit tests (#237)
fe70285
NickEdwards7502 added a commit that referenced this issue Oct 17, 2024
@NickEdwards7502
FIX: Update LocalFDR import statement in rfmodel.py (#237)
279bd5b
NickEdwards7502 added a commit that referenced this issue Oct 17, 2024
@NickEdwards7502
DEV: Integrate bgzipped file support in VCF import API (#237)
5ad8cc0 
* .bgz loader function implemented by Christina
NickEdwards7502 added a commit that referenced this issue Oct 17, 2024
@NickEdwards7502
DEV: Implement imputation for VCF features (#237)
b686d75 
* Update python wrapper to include imputation strategy parameter

* Update scala API to pass imputation strategy to VCFFeatureSource

* Create functions to handle mode and zero imputation strategies

* Added imputation strategy to test cases

* Added imputation strategy to FeatureSource cli

* Remove sparkPar from test cases due to changes in class signature

* Updated DefVariantToFeatureConverterTest to use zeros imputation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@rocreguant