New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BCF entry point with intensity and contamination checks using BCF for data_catalog usage. #314

Merged

jaamarks merged 36 commits into default from data_catalog

Sep 25, 2024

rajwanir commented Aug 16, 2024

Creates BCF entry point for data catalog.
Makes BPM manifest optional (relevant for aggregated file inputs e.g. BCF, VCF or BED)
Adds scripts to compute median intensity and contamination score from BCF.
Separates the intensity checks from contamination checks.
No observable difference with standard GTC input.

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

jaamarks reviewed

View reviewed changes

src/cgr_gwas_qc/models/config/user_files.py Show resolved Hide resolved

jaamarks reviewed

View reviewed changes

src/cgr_gwas_qc/workflow/sub_workflows/contamination.smk Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

rajwanir force-pushed the data_catalog branch from f7a7396 to 270e105 Compare

September 13, 2024 18:09

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

rajwanir mentioned this pull request

When importing bcf/vcf to plink, alleles can get flipped. #321

Open

This comment was marked as resolved.

Sign in to view

This comment was marked as resolved.

Sign in to view

jaamarks reviewed

View reviewed changes

src/cgr_gwas_qc/workflow/scripts/vcf2adpc.py Outdated Show resolved Hide resolved

jaamarks reviewed

View reviewed changes

src/cgr_gwas_qc/workflow/scripts/vcf2adpc.py Outdated Show resolved Hide resolved

This comment was marked as resolved.

Sign in to view

rajwanir2 added 11 commits

September 25, 2024 16:21


          makes illumina_manifest_file optional in the config

143eb80


          Uses snps_array name instead of bpm to name *.abf.txt

8a28efb

Avoids dependency of bpm to name allele B frequencies (abf) file.


          Adds a new idat_intensity workflow

36cfdab

to separate median idat intensity retrieval from verifyIDintensity bundled into contamination.smk


          Adds vcf2abf.py and vcf2adpc.py as prep scripts for contamination che…

3c8259e

…cks with vcf entry


          Creates a BCF entry point.

a6cb033

Modifis the entry_points.smk to create BCF entry point by simply converting BCF to plink BED. Testing and validation yet to be done.


          Removes IDAT intensity checks from contamination.smk

0146c1b

A previous commit puts them in a separate idat_intensity.smk


          Adds rules contamination.smk to do contamination checks with bcf input.

0dd09f1


          Enables contamiantion check with bcf input in cluster mode.

81697d6

Modifies contamination.smk and grouped_contamination.py to enable contamination check in cluster mode.


          Adds capability to infer median intensity directly from input BCF/VCF.

56dd245

Avoids the dependency on IDAT files for calculating median intensity with VCF/BCF input.
Adds scripts for both in per-sample and grouped/cluster mode.
Modifies the intensity workflow to execute appropriately if VCF/BCF input.


          Adds the distinction of contamination and intensity checks in the sna…

0aaaa9b

…kefile and sample_qc subworkflow

Other than existing 'use_contamination' checks, also adds 'intensity_retreived' and 'contamination_checked'
tests which simply tests specifically if output csv files were created regardless of configs/entry point to feed
them to sample_qc.


          Correcting conditions such with BCF input intensity and contamination…

0dcbaf2

… check is default

rajwanir2 added 21 commits

September 25, 2024 16:21


          Updates the docstring in contamination.smk to indicate when BCF file …

f8ba252

…is used.


          Renames input variable to bcf_file.

20c0a47

Renames from vcf_file to bcf_file explictly indicate that bcf is input.


          Imports snakemake params all at once in grouped_contamination.py

fa2bcf5

snakemake params were imported through named import. Changed to import all params through a loop in unnamed fashion. Allows seemless compatibility when gtc or bcf entry point is used.


          Removes typo duplicated header in entry_points

967ff88

The starting few lines were duplicated in the entry_points in copy/paste. This removes the duplicated lines.


          Shifts to using IGC score instead of gentrain score in vcf2adpc.py fo…

0224aae

…r contamination checks.

Previously GC_SCORE was added to the adpc.bin which had depenency that a cluster egt file had to be used in preparation of vcf/bcf. IGC score is encoded in gtc so doesn't depended on cluster egt file. The conamination scores should also be more similar with the gtc input.


          Adds a config suffix validator for bcf_file.

53b4e33


          Removes some debugging lines from vcf2abf.py

416f266


          Shifts to using IGC as genotype score for preparing abf using vcf2abf.

bbe5006

Consistent with vcf2adpc.py. Now both should IGC and work with vcf/bcf prepared with gtc2vcf workflow.


          Removes commented lines from vcf2adpc.py and vcf2abf.py

76c8304


          fixes a black syntax issue in importing snakemake params in grouped_c…

d840118

…ontamatination.py


          Modifies vcf2abf to not use gentrain score in preparing abf.

2c52669

Earliar gentrain score was used to mark AF as NA if score is negative. This change excludes using it to ensure compatibility with gtc2vcf workflow prepared bcf.


          Includes a None case in the BCF config validator to work with unit te…

4e993bc

…sting.


          Adds the app mode in median_intensity_from_vcf to work directly as st…

605006c

…and-alone.


          Adds the new unit test scripts relevent to BCF entry

b69a3fc


          docs: updates docstring to show bcf as a user_file in config

d1fd658


          docs: Updates the entry_point documentation to show BCF as an entry_p…

5ef948d

…oint.


          docs: Updates contamination subworkflow documentation to show process…

949443e

…ing with BCF input.


          docs: Adds the documentation for new intensity subworkflow and shows …

f28fc9c

…processing for both IDAT and BCF input.


          docs: Updates index to include intensity_check subworkflow.

d42a035


          docs: corrects spellings in documentation

3ea4d4c


          Removes jupyter comment lines from scripts.

4bcf9e6

rajwanir force-pushed the data_catalog branch from 5c6032c to 4bcf9e6 Compare

September 25, 2024 20:25

rajwanir2 added 2 commits

September 25, 2024 16:39


          docs: updates the docstring for user_files in config section to inclu…

37de67c

…de BCF.


          docs: redo updates the user_files docstring to include BCF.

58ed006

jaamarks approved these changes

View reviewed changes

Collaborator

jaamarks left a comment

LGTM

jaamarks approved these changes

View reviewed changes

jaamarks merged commit f2ff2c3 into default

2 checks passed

jaamarks mentioned this pull request

Create VCF entry point #292

Open

carynwillis mentioned this pull request

GTC to VCF creating too large of file for 100K+ samples #318

Closed

rajwanir mentioned this pull request

With convert_gtc2bcf, contamination and intensity checks should be done using BCF. #371

Open

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet