![banner](https://private-user-images.githubusercontent.com/100899073/351260842-682c7b18-fdc6-4353-bf35-28806b296484.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUxMjYwODQyLTY4MmM3YjE4LWZkYzYtNDM1My1iZjM1LTI4ODA2YjI5NjQ4NC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0zYmZiYmQ5MTFhZGM3MTMzZTlmZDEzYTc3ODQzZTMwYmRkMTZjYTExNThkYzBhODUyMGI5OTFkNGFiNTE5MGUyJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.T6aFmIf2VXORLXbzL8Koy-gq93dBjzcfCwZiDNeU9nE)
![banner](https://private-user-images.githubusercontent.com/100899073/351257018-f31ae18f-5846-49d7-b597-3234a7035ab2.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUxMjU3MDE4LWYzMWFlMThmLTU4NDYtNDlkNy1iNTk3LTMyMzRhNzAzNWFiMi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lYjM2NTc3MGMwMWI2ZjczZTVhMzIzOTJkOWQ4ZDQxNTdkMDcwODRlMjg3NGQ3YTkyM2I2ZDdkNDBhMTRhNTVhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.AxNfKPZ0U04X2KgRsr5tt9MXjufHF_3kRqGwstplxuw)
This pipeline generates high-quality Gene Co-expression Networks (TEA-GCN ) that capture tissue/condition-specific co-expression.
For in-depth explanation and evaluation of the TEA-GCN methodology, please refer to our preprint π°
![banner](https://private-user-images.githubusercontent.com/100899073/351325374-1e2d62ab-5a93-484e-b0da-6406fc9d1122.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUxMzI1Mzc0LTFlMmQ2MmFiLTVhOTMtNDg0ZS1iMGRhLTY0MDZmYzlkMTEyMi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1hYzU1NTlhNDMwMzk0ZmNlZTQ5NzBkMDJhMDc5NjJmOGFlNWMzZjI2NWY3NGYzNjc0NzMxNTU4ZWM3MDM1MTZkJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.nZI0ukzHL8wKRLmpRw_mmn3SdJo6ZUEIshnMrgP16_8)
$ git clone https://github.com/pengkenlim/TEA-GCN.git
$ cd TEA-GCN
$ virtualenv -p python3 .venv
$ source ./.venv/bin/activate
$ pip install --upgrade pip
$ pip install -r ./setup/requirements.txt
$ cd TEA-GCN
$ source ./.venv/bin/activate
The TEA-GCN method uses of k-means clustering algorithm to divide gene expression data into partitions before gene co-expression determination. Expression data must be provided in the form of an expression matrix where 1) expression abundances are in the form of Transcript per Million (TPM), 2) rows correspond to different genes, and 3) columns correspond to different samples. The first row of the input expression matrix must consist of column headers (i.e. sample names) and the first column must consist of unique indices (i.e. gene identifiers).
We provide sample data of a gene expression matrices containing 500 and 5,000 Arabidopsis thaliana public RNA-seq samples, respectively. (used in our preprint ).
You can download it from your browser using these links:
$ python main/Generate_partitions.py --output_dir /path/to/output_directory --input_matrix_path /path/to/taxid3702_500n_expression_matrix.tsv
usage: Generate_partitions.py [-h] [-ks] [-ke] [-st] [-t] -o -im [-de] [-pc] [-rs]
Generate_partitions.py Load expression matrix. Perform sample standardization to correct batch effect. Embed samples from gene-space into PC-space. Perform k-means clustering across a
range of K. Outputs principal component embeddings of samples (PCA data), mean silhouette coefficients from each clustering iteration, k-means cluster assignments of samples in pickle
format.
options:
-h, --help show this help message and exit
-ks , --k_start Starting k for K-means clustering. The default is 10.
-ke , --k_end Ending k for K-means clustering. The Default is 50
-st , --step Step to increase between each k. The user is suggested to increase the step if the range is very big to decrease search space.
-t , --threads Number of threads to use for PCA and Kmeans clustering. Handled by Sklearn's parallelization. Try to specify lower than 64 for now.
-o , --output_dir Working directory to output data.
-im , --input_matrix_path
Path of expression matrix to input
-de , --delimiter Delimiter for expression matrix. -de="t" for tab-separated (.tsv). -de="c" for comma separated (.csv). TSV by default.
-pc , --pc_no Number of Principal components to keep. Default = 100. It cannot be more than the number of samples and the number of genes in the expression matrix.
-rs , --randomstate randomstate (seed) for random seeding during k-means clustering. By default, 42 will be used.
After data partitioning, you can start building TEA-GCN by determining co-expression strength between every gene pair. Said co-expression strength is calculated based on measured correlation coefficients between genes from every dataset partition. If you would like more information, please refer to our preprint.
$ python main/Run_TEA-GCN.py --output_dir /path/to/output_directory --input_matrix_path /path/to/taxid3702_500n_expression_matrix.tsv
Intermediate TEA-GCN will be output to /path/to/output_directory/RAW_GCN/<METHOD>/<NETWORK_NAME>
. <NETWORK_NAME>
is generated automatically based on the <METHOD>_<K>k_<AGGREGATION_FUNCTION>
format (e.g. TEA_8k_RAvg)
usage: Run_TEA-GCN.py [-h] [-w] [-t] -o [-de] [-cc] [-am] [-k] -im
Run_TEA-GCN.py. Construct TEA-GCN from expression data (in the form of an expression matrix). Requires prior data partitioning using Generate_partitions.py in order to run.
options:
-h, --help show this help message and exit
-w , --workers Number of workers for parallelization. Affects precalc of all coefficients. But only affects the calc of bicor.
-t , --threads Number of threads for numpy linear algebra operations. Affects PCC and SCC calc.
-o , --output_dir Directory to output. Must be the same as for Generate_partitions.py
-de , --delimiter Delimiter for expression matrix. -de="t" for tab-separated (.tsv). -de="c" for comma-separated (.csv). TSV by default.
-cc , --correlation_coefficient
select 'TEA' to enable Coefficient Aggregation. Select "PCC","SCC","bicor" for constructing Partition Aggregation-only GCN
-am , --aggregation_method
The default is RAvg.
-k , --k_clusters Number of clusters to partition the expression data. For k=0, will use best k as determined in Generate_partitions.py step
-im , --input_matrix_path
Path of expression matrix to input
After building a TEA-GCN that describes the co-expression strengths between every gene pair (edges). We can then load back in all edges for rank transformation (into Mutual Ranks) and generate standardized ranks for meaningful cross-comparison of TEA-GCN across different species.
$ python ./main/Rank_transform.py --output_dir /path/to/output_directory
Completed TEA-GCN will be located in /path/to/output_directory/Completed_GCN/<NETWORK_NAME>
.
usage: Rank_transform.py [-h] -o [-n] [-d] [-full]
Rank_transform.py calculates Mutual Ranks (MR) and Highest reciprocal ranks (HRR) and their standardized forms raw correlation scores. Rank_transform.py outputs the final complete TEA-GCN
as directory separate from the intermediate TEA-GCN generated by Run_TEA_GCN.py
options:
-h, --help show this help message and exit
-o , --output_dir Working directory containing required data of which to output data. Must be same as output_dir for Run_TEA_GCN.py and Generate_partitions.py
-n , --net_name name of the network to calculate MR and HRR for. By default, all networks will be calculated
-d , --delete_net If set to True, Rank_transform.py will delete intermediate TEA-GCN generated by Run_TEA_GCN.py. False by default.
-full , --full_attributes
If set to True, Rank_transform.py will output a full set of attributes for every edge which includes: raw co-exp. strengths, MR, HRR, and their Z score variants.
False by default where only raw co-exp, MR, and Z(MR) are reported.
The resultant network is contained within a directory consisting of files that each describe edges connecting to each gene (source gene). The files are named after genes.
Example of one file:
$ less /path/to/<NETWORK_NAME>/AT5G65360.1
![banner](https://private-user-images.githubusercontent.com/100899073/351607617-bde075ec-7015-4661-9caf-6d477cb50c94.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUxNjA3NjE3LWJkZTA3NWVjLTcwMTUtNDY2MS05Y2FmLTZkNDc3Y2I1MGM5NC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT04OTczNTliNWZiYzI2MmNiMGZlOGM5MmJhNTJlNTQ1NWFiNzRjYzIxYjFmZjczOTIyMWViZGQ0ZmYyNTRhYWVlJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.ET64tpDnRHHHULnyRudcKTR0RF2YN6XMWcPLaCtwVuI)
Description of columns:
Target
- Describes the target gene of the edge
Co-exp_Str
- Co-expression Strength of edge calculated by
Run_TEA-GCN.py
Co-exp_Str_MR
- Co-expression Strength of edge transformed into Mutual Ranks (MR)
zScore(Co-exp_Str_MR)
- z-score standardized MR -- Helpful for cross-comparing GCNs
Back to Navigation
![banner](https://private-user-images.githubusercontent.com/100899073/351321541-af7a133e-bad2-4a72-852f-add13fb173bb.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUxMzIxNTQxLWFmN2ExMzNlLWJhZDItNGE3Mi04NTJmLWFkZDEzZmIxNzNiYi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lY2ExMjllYjBjZjBjY2Y1YWYzNDA2YWM2ZGM1NTAzNTc5YjJkYThjZWU3YzhiMGFhMzI4NGExMWVmOTdmM2IyJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.X_X2z7rzrp_arsyg62Hm49BhQjkBs8eYoxUVo-Bjr5k)
$ python ./main/Prepare_neighbourhoods.py --output_dir /path/to/output_directory --gene_list AT3G02480.1
usage: Prepare_neighbourhoods.py [-h] -o -g [-n]
Prepare_neighbourhoods.py. Prepare co-expression neighbourhoods of Genes-of-intrests from completed TEA-GCNs for gene function prediction using the GSEA algorithm.
options:
-h, --help show this help message and exit
-o , --output_dir Directory to output. Must be the same as for Generate_partitions.py, Run_TEA-GCN.py, and Rank_transform.py.
-g , --gene_list list of comma-separated genes. spaces will be removed. E.g. 'GENE1,GENE2,GENE3'
-n , --net_name name of the TEA-GCN to extract. Set to "auto" if you only generated one TEA-GCN. "auto" by default
Example of one file:
$ less /path/to/output_directory/Co-exp_Neighbourhoods/AT5G65360.1
![banner](https://private-user-images.githubusercontent.com/100899073/351611213-31c0d11a-ccbc-4d8a-8f18-acad1c7e4899.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUxNjExMjEzLTMxYzBkMTFhLWNjYmMtNGQ4YS04ZjE4LWFjYWQxYzdlNDg5OS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lYWQ2ZDZlMDdkMzk3NDViOTY0YzI1ZjRmODM0NWNkZDNlYTQ5ZGQwOGZiYjExNTdmMGZjZmE0NDQ3NDk4YmEwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.KDFP46BBxmDBZA7JmsJRx9Gn-8ZrMiSjhONx48L3-Mc)
Description of columns:
Target
- Describes the target gene of the edge
Score
- z-score standardized MR determined by
Rank_transform.py
Centered_rank
Score
ranked inversely (higher values, have larger ranks) and centered (median rank = 0).
Refer to the Colab notebook (linked below) to predict the biological function of your gene-of-interest based on their co-expression neighbourhood:
TEA-GCN_Function_pred_using_GSEA.ipynb
Back to Navigation
![banner](https://private-user-images.githubusercontent.com/100899073/351324180-bf40485c-768d-4eb7-be0f-4ed39530b884.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUxMzI0MTgwLWJmNDA0ODVjLTc2OGQtNGViNy1iZTBmLTRlZDM5NTMwYjg4NC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lOGEyNjg4N2E0ZjEyNDFmMjJhMzczMTBhMDEyYmU2MWVkMjQxOWYzMDExNzVjYzcyZmFiYjlhNjQxYmZhNjRhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.UMQwhCdQ2G1yAxN3DTQhBfis797baXG9veIoOJwAQbM)
This script downloads the metadata of RNA-seq samples from the European Nucleotide Archive (ENA).
The metadata downloaded includes these search fields*:
sample_description
dev_stage
study_title
sample_title
tissue_lib
tissue_type
*for more information regarding search fields and metadata hosted by ENA, click here
$ python ./main/Download_metadata.py --output_dir /path/to/output_directory --input_matrix_path /path/to/taxid3702_500n_expression_matrix.tsv
usage: Download_metadata.py [-h] -o [-de] -im
Download_metadata.py Download metadata of SRR accessions from European Nucleotide Archive (ENA).
options:
-h, --help show this help message and exit
-o , --output_dir Directory to output. Must be the same as for Generate_partitions.py, Run_TEA-GCN.py and Rank_transform.py.
-de , --delimiter Delimiter for expression matrix. -de="t" for tab-separated (.tsv). -de="c" for comma separated (.csv). TSV by default.
-im , --input_matrix_path
Path of expression matrix to input
![banner](https://private-user-images.githubusercontent.com/100899073/353451229-ed3f63e4-c69a-45ea-90da-9dfc1419180c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUzNDUxMjI5LWVkM2Y2M2U0LWM2OWEtNDVlYS05MGRhLTlkZmMxNDE5MTgwYy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lNmU4NzBiNDRhY2QxOThhMzg1NzE2OWI2OTJhNDgzMDAwYTUwYTgzMjllYjMxOGZiNzc0NjE0Y2EwMzFlNmM4JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.-jnaCd8Y_8Z3-NyskYGibxSHIEzjIJt_6Y8a2VCV55s)
Download and install your spaCy model of choice for Natural Language Processing (NLP). Below we chose the en_core_web_sm
model. This model will be used to lemmatize the metadata of RNA-seq samples downloaded from ENA.
$ python -m spacy download en_core_web_sm
Running this script will find overrepresented lemmas to annotate partitions
$ python ./main/Annotate_partitions.py --output_dir /path/to/output_directory --input_matrix_path /path/to/taxid3702_500n_expression_matrix.tsv
usage: Annotate_partitions.py [-h] -o [-de] -im -k [-m] [-p]
Annotate_partitions.py Lemmatize metadata of RNA-seq samples. Annotate dataset partitions with overrepresented lemmas
options:
-h, --help show this help message and exit
-o , --output_dir Directory to output. Must be the same as for Generate_partitions.py, Run_TEA-GCN.py, and Rank_transform.py.
-de , --delimiter Delimiter for expression matrix. -de="t" for tab-separated (.tsv). -de="c" for comma separated (.csv). TSV by default.
-im , --input_matrix_path
Path of expression matrix to input
-k , --k_clusters Number of clusters to partition the expression data. For k=0, will use best k as determined in Generate_partitions.py step.
-m , --model spaCy model to use for lemmatization. 'en_core_web_sm' will be used by default.
-p , --permutations Number of permutations to shuffle during the calculation of overrepresentation p-values. Default = 10,000
![banner](https://private-user-images.githubusercontent.com/100899073/353454642-9252f188-670b-4d95-8a99-0d86dc19527d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUzNDU0NjQyLTkyNTJmMTg4LTY3MGItNGQ5NS04YTk5LTBkODZkYzE5NTI3ZC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0wNzk4ZWIwYjg5Y2U3YjNlODk1NDZhNGEwMjJjOTJkYjQ1ZDU4M2RjYzQ1MDFkNzM4OWZjOGZkYTA5OWJiYWVhJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.1BnnXOkYkx-qZF8w1id0its6Mk2ztXuexpfnXETAaDM)
Prepare edges-of-interest as input like so.
$ less /path/to/edges_of_interest.csv
![banner](https://private-user-images.githubusercontent.com/100899073/353732686-a4d1fa68-765c-42b1-96a0-4f2c2c553e20.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUzNzMyNjg2LWE0ZDFmYTY4LTc2NWMtNDJiMS05NmEwLTRmMmMyYzU1M2UyMC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT03NmY1NDQ2MDg2MWJmMjcyZjJiZDdkNmMzNzA1YmZkMWRkMzFiZDYxYmRiNGFiNTg0NjM1NTE3NDAxMWY2YmIxJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.d3j30OECmQ0MJxoiyaeOAdcit5jdnsJSHvRrggSDSc4)
Each row describes one edge. Two comma-separated columns for the two genes that connect the edge.
python ./main/Get_Partition_Rankings.py -k 93 --input_matrix_path /path/to/taxid3702_500n_expression_matrix.tsv --edges_path /path/to/edges_of_interest.tsv --output_dir /path/to/output_directory
usage: Get_Partition_Rankings.py [-h] [-w] [-t] -o [-de] [-k] -im -ep [-on]
Get_Partition_Rankings.py.
options:
-h, --help show this help message and exit
-w , --workers Number of workers for parallelization. Affects precalc of all coefficients. But only affects the calc of bicor.
-t , --threads Number of threads for numpy linear algebra operations.
-o , --output_dir Directory to output.
-de , --delimiter Delimiter for expression matrix. -de="t" for tab-separated (.tsv). -de="c" for comma separated (.csv). TSV by default.
-k , --k_clusters Number of clusters to partition the expression data. For k=0, will use best k as determined in Generate_partitions.py step.
-im , --input_matrix_path
Path of expression matrix to input
-ep , --edges_path Path to list of edges to calculate.
-on , --outname_prefix
prefix of to output calculations.
![banner](https://private-user-images.githubusercontent.com/100899073/353452488-1357907f-a701-4585-b522-9223c4c2d14b.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUzNDUyNDg4LTEzNTc5MDdmLWE3MDEtNDU4NS1iNTIyLTkyMjNjNGMyZDE0Yi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0zOTJhZDFhZGZjYTA1MDFkYzkzMTc4NmYzMjE4NjQ1ZjhlZTcwZDRlMTcxNDExYjFiYzI1NzdmY2YyOGUyMzc4JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.mj8iJJaYSA64R6z-hK1DAB00SZiBCPxmE6lmHdnptC0)
To discover experimental contexts underpinning gene co-expression, we use a GSEA-like approach to detect overrepresented lemmas that annotate partitions with high co-expression. However, not all overrepresented lemmas are informative or relevant to experimental conditions. Including them in the downstream analysis will only confound the interpretation of experimental contexts and decrease the sensitivity via harsher multiple-testing correction.
Step 2 generates a list of overrepresented lemmas that are detected in your dataset partitions in /path/to/output_directory/Lemma_annotations/overrepresented_lemmas.tsv
. You can remove irrelevant lemmas from this file. It will be used as input in the Colab notebook below to exclude them from the analysis.
$ less /path/to/output_directory/Lemma_annotations/overrepresented_lemmas.tsv
![banner](https://private-user-images.githubusercontent.com/100899073/353670704-874aa637-1dfd-49af-91cf-f70ab597e004.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzUzNjcwNzA0LTg3NGFhNjM3LTFkZmQtNDlhZi05MWNmLWY3MGFiNTk3ZTAwNC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT00NzBhODNiOGY1YzdlMGM0MDVkMmI5Njg3ODJkNzI5NGEyYTk1ODhmYTdlYTFkN2ZkYzcxYzFjN2ZmM2U5NTdmJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.B1mH5FClG_ZOqPnLBz9Qz_68onqipM8tmmBrz_zYu9s)
Refer to the Colab notebook (linked below) to uncover the Experimental contexts underpinning gene co-expression:
Uncover_the_Experimental_contexts_Underpinning_Gene_Co-expression.ipynb
![banner](https://private-user-images.githubusercontent.com/100899073/354985331-ad84e67f-e8cd-4e37-81c2-bf05b8478fee.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNDExODgsIm5iZiI6MTcyNjE0MDg4OCwicGF0aCI6Ii8xMDA4OTkwNzMvMzU0OTg1MzMxLWFkODRlNjdmLWU4Y2QtNGUzNy04MWMyLWJmMDViODQ3OGZlZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTEyJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkxMlQxMTM0NDhaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT05YWQ4ZjIxNzc2YjFlNTAwZGVmZWY0ZTc5MDE5ZTQ5NjQyMmYwNmUyZTNiY2E4YzA0Nzk2ZmY0ZGE1ZWYwYjA0JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.ZyXyvncsSY_EZuJzWpuFAsXeP1OgFT7ggyF2UMa8v3g)
Back to Navigation
You can cite our preprint for now :)
Lim, P. K., Wang, R., Velankanni, J. P. A., & Mutwil, M. (2024).
Constructing Ensemble Gene Functional Networks Capturing Tissue/condition-specific Co-expression from Unlabled Transcriptomic Data with TEA-GCN
(p. 2024.07.22.604713). bioRxiv.
https://doi.org/10.1101/2024.07.22.604713
- LSTrAP-Kingdom
- Download and process public RNA-seq samples to generate gene expression matrices for your species of interest
- Github page
- Publication
- Cite
Goh W, Mutwil M. LSTrAP-Kingdom: an automated pipeline to generate annotated gene expression atlases for kingdoms of life. Bioinforma Oxf Engl. 2021;37(18):3053-3055. doi:10.1093/bioinformatics/btab168
- LSTrAP-denovo
- Download and process public RNA-seq samples to generate gene expression matrices for your species of interest
- Designed for use for eukaryotic species without sequenced genomes. Employs de novo transcriptome assembly to derive reference transcripts.
- Github page
- Publication
- Cite
Lim, P. K., Wang, R., & Mutwil, M. (2024). LSTrAP-denovo: Automated Generation of Transcriptome Atlases for Eukaryotic Species Without Genomes. Physiologia Plantarum, 176(4), e14407. https://doi.org/10.1111/ppl.14407
Back to Navigation
I am the project supervisor. I advise the students and buy them meals sometimes.
Principal Investigator and Assoc. Prof.,
School of Biological Sciences, Nanyang Technological University
mutwil@ntu.edu.sgπ§
Lab websiteπ
I am the project co-supervisor. I wrote most of the code.
PhD student,
School of Biological Sciences, Nanyang Technological University
pengken001@e.ntu.edu.sgπ§
I wrote some of the code and conducted comparative-GCN analysis for this project.
Undergraduate student,
Shanghai Jiao Tong University
I helped with mining Human and Yeast gene annotations for this project.
Undergraduate student,
School of Biological Sciences, Nanyang Technological University
Back to Navigation