Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add thorney beast support in Beauti #1211

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
a392217
initial look at beauti
jtmccr1 Jul 10, 2024
d368683
git ignore for vscode
jtmccr1 Jul 10, 2024
236324d
unhide thorney options for dev work
jtmccr1 Jul 10, 2024
0637872
removes treedatapartition from sites and substitution model
jtmccr1 Jul 11, 2024
ed3bc28
adds thorney operators
jtmccr1 Jul 11, 2024
b22eb1f
Adds initial tree
jtmccr1 Jul 11, 2024
183d17e
adds likelihood block
jtmccr1 Jul 11, 2024
d2d768f
fixes bug in treemodel idref
jtmccr1 Jul 12, 2024
b952961
operator work
jtmccr1 Jul 12, 2024
3661e10
thorney beast uses bft-intervals nows
jtmccr1 Jul 12, 2024
77fb51d
removes unneeded components
jtmccr1 Jul 12, 2024
0fe7bfb
trying out clock models with tb
jtmccr1 Jul 12, 2024
7de04b3
proxy parameters so we can use usual node height operators
jtmccr1 Jul 12, 2024
b11d856
adds node height proxies and standard operators
jtmccr1 Jul 12, 2024
062fc43
fixes bug in rate covariance that failed if parent was null
jtmccr1 Jul 12, 2024
ad814b2
Merge remote-tracking branch 'origin/master' into every-rose
jtmccr1 Jul 27, 2024
8128df3
adds beauti generated example for 1K SARS2
jtmccr1 Jul 27, 2024
ad4147c
removing comments
jtmccr1 Aug 7, 2024
344a22e
subtree slide works with bft (no trait swaps through)
jtmccr1 Oct 2, 2024
b42c751
forgot treemodel edits in constraints
jtmccr1 Oct 21, 2024
a304f9b
capitalization in rootheightproxy
jtmccr1 Oct 21, 2024
6fed2cb
adds bft to beauti jar
jtmccr1 Oct 21, 2024
f9bb47c
Merge remote-tracking branch 'origin/master' into every-rose
jtmccr1 Nov 19, 2024
e6d89aa
resolving AR conversations
jtmccr1 Nov 19, 2024
8aed018
updates bft and constrained tree to default xml
jtmccr1 Nov 19, 2024
f80a82b
removes duplication of empirical trees from tree panel
jtmccr1 Nov 19, 2024
e8a0c43
adds warning for clock models without TB gradients
jtmccr1 Nov 19, 2024
7d2bd2a
updating constrained tree xml in unit tests
jtmccr1 Nov 19, 2024
8226901
Revert "thorney beast uses bft-intervals nows"
jtmccr1 Dec 18, 2024
d3bd9e6
Merge remote-tracking branch 'origin/master' into beauti-thorne
jtmccr1 Dec 18, 2024
41c40e7
reverting copyright sign
jtmccr1 Dec 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ release/Mac
release/Linux
release/Windows
src/revision.txt
.vscode/launch.json
.vscode/settings.json

*.citations.txt
*.ops
Expand Down
1 change: 1 addition & 0 deletions build.xml
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,7 @@
<include name="dr/evomodel/speciation/**/*.class"/>
<include name="dr/evomodel/substmodel/**/*.class"/>
<include name="dr/evomodel/tree/**/*.class"/>
<include name="dr/evomodel/bigfasttree/**/*.class"/>
<include name="dr/evomodel/treelikelihood/**/*.class"/>
<include name="dr/evomodel/treedatalikelihood/**/*.class"/>
<include name="dr/evomodelxml/**/*.class"/>
Expand Down
3,257 changes: 3,257 additions & 0 deletions examples/1K_SARS-CoV-2-thorney.xml

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions examples/Data/1K_SARS-CoV-2.nexus

Large diffs are not rendered by default.

3,177 changes: 3,177 additions & 0 deletions examples/thorney.xml

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion src/dr/app/beast/development_parsers.properties
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#
# development_parsers.properties
#
# Copyright © 2002-2024 the BEAST Development Team
# Copyright © 2002-2024 the BEAST Development Team
jtmccr1 marked this conversation as resolved.
Show resolved Hide resolved
# http://beast.community/about
#
# This file is part of BEAST.
Expand Down Expand Up @@ -264,6 +264,8 @@ dr.evomodelxml.bigfasttree.thorney.ThorneyTreeGradientParser
dr.evomodelxml.bigfasttree.thorney.UniformSubtreePruneRegraftParser
dr.evomodel.bigfasttree.thorney.ZeroMutationMasker
dr.evomodelxml.bigfasttree.thorney.SubtreeRootHeightStatisticParser
dr.evomodel.bigfasttree.thorney.RootHeightProxyParameter

# GLM
dr.inferencexml.operators.MaskMoveOperatorParser
dr.evomodelxml.continuous.hmc.GlmSubstitutionModelGradientParser
Expand Down
17 changes: 17 additions & 0 deletions src/dr/app/beauti/generator/BeastGenerator.java
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,23 @@ public void checkOptions() throws GeneratorException {
}
}

//+++++++++++++++++ ThorneyBEAST / clock model Verification ++++++++++++++
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warns the user about using a clock model for which gradients for tb are not implemented


for (PartitionClockModel model : options.getPartitionClockModels()) {
for (AbstractPartitionData pd : options.getDataPartitions(model)) {
if(pd.getDataType().getType() ==DataType.TREE){
PartitionTreeModel tm = pd.getPartitionTreeModel();
if(tm.isUsingThorneyBEAST() && !PartitionTreeModel.thorneyCompatibleClocks.contains(model.getClockType())){
throw new GeneratorException("Gradients for "+model.getClockType()+
" are not implemented for Thorney BEAST." +
"Please use a clock model that does not rely on gradients.", BeautiFrame.CLOCK_MODELS);

}
}
}
}


//++++++++++++++++ Prior Bounds ++++++++++++++++++
for (Parameter param : options.selectParameters()) {
if (!Double.isNaN(param.getInitial()) ) {
Expand Down
34 changes: 33 additions & 1 deletion src/dr/app/beauti/generator/InitialTreeGenerator.java
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,39 @@ public void writeStartingTree(PartitionTreeModel model, XMLWriter writer) {
break;

case RANDOM:
// generate a coalescent tree
String simulatorId = prefix + STARTING_TREE;
String constraintID = prefix + "constraintsTree";
if(model.isUsingThorneyBEAST()){
// write tree with dates
writer.writeComment("The user-specified tree with dates used to simualte a compatible bifurcating starting tree.");
writer.writeOpenTag(
NewickParser.NEWICK,
new Attribute[]{
new Attribute.Default<>(XMLParser.ID, constraintID),
new Attribute.Default<Boolean>(SimpleTreeParser.USING_DATES, options.useTipDates)
}
);
writer.writeText(TreeUtils.newick(model.getTreePartitionData().getTrees().getTrees().get(0)));
writer.writeCloseTag(NewickParser.NEWICK);

writer.writeComment("Generate a random starting tree under the coalescent process");
writer.writeOpenTag(
CoalescentSimulatorParser.COALESCENT_SIMULATOR,
new Attribute[]{
new Attribute.Default<String>(XMLParser.ID, simulatorId)
});
writeInitialDemoModelRef(model, writer);
writer.writeTag(
CoalescentSimulatorParser.CONSTRAINTS_TREE,
false
);
writer.writeIDref(NewickParser.NEWICK, constraintID);
writer.writeCloseTag(CoalescentSimulatorParser.CONSTRAINTS_TREE);
writer.writeCloseTag(CoalescentSimulatorParser.COALESCENT_SIMULATOR);

}else{

// generate a coalescent tree

String taxaId = TaxaParser.TAXA;
AbstractPartitionData partition = options.getDataPartitions(model).get(0);
Expand All @@ -132,6 +163,7 @@ public void writeStartingTree(PartitionTreeModel model, XMLWriter writer) {
writeInitialDemoModelRef(model, writer);
writer.writeCloseTag(CoalescentSimulatorParser.COALESCENT_SIMULATOR);
}
}
break;

default:
Expand Down
45 changes: 45 additions & 0 deletions src/dr/app/beauti/generator/OperatorsGenerator.java
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,12 @@
import dr.app.beauti.util.XMLWriter;
import dr.evolution.datatype.DataType;
import dr.evomodel.branchratemodel.BranchSpecificFixedEffects;
import dr.evomodel.bigfasttree.thorney.ConstrainedTreeModel;
import dr.evomodel.operators.BitFlipInSubstitutionModelOperator;
import dr.evomodel.operators.EmpiricalTreeDistributionOperator;
import dr.evomodel.tree.DefaultTreeModel;
import dr.evomodel.tree.EmpiricalTreeDistributionModel;
import dr.evomodelxml.bigfasttree.thorney.UniformSubtreePruneRegraftParser;
import dr.evomodelxml.branchratemodel.AutoCorrelatedBranchRatesDistributionParser;
import dr.evomodelxml.branchratemodel.AutoCorrelatedGradientWrtIncrementsParser;
import dr.evomodelxml.branchratemodel.BranchRateGradientWrtIncrementsParser;
Expand Down Expand Up @@ -255,6 +257,15 @@ private void writeOperator(Operator operator, XMLWriter writer) {
case SHRINKAGE_CLOCK_GIBBS_OPERATOR:
writeShrinkageClockGibbsOperator(operator, prefix, writer);
break;
case NODE_HEIGHT_OPERATOR_UNIFORM :
writeBFTUniformNodeHeightOperator(operator, prefix, writer);
break;
case NODE_HEIGHT_OPERATOR_ROOT:
writeBFTRootScaleOperator(operator, prefix, writer);
break;
case UNIFORM_SUBTREE_PRUNE_REGRAFT:
writeUniformSPROperator(operator, prefix, writer);
break;
default:
throw new IllegalArgumentException("Unknown operator type");
}
Expand Down Expand Up @@ -747,6 +758,40 @@ private void writeEmpiricalTreeSwapOperator(Operator operator, String treeModelP
writer.writeCloseTag(EmpiricalTreeDistributionOperator.EMPIRICAL_TREE_DISTRIBUTION_OPERATOR);
}

private void writeBFTUniformNodeHeightOperator(Operator operator, String treeModelPrefix, XMLWriter writer) {
writer.writeOpenTag(NodeHeightOperatorParser.NODE_HEIGHT_OPERATOR, // should this be in the class and not the parser?
new Attribute[]{
new Attribute.Default<String>("type",NodeHeightOperatorParser.OperatorType.UNIFORM.toString()),
getWeightAttribute(operator.getWeight())
}
);
writer.writeIDref(ConstrainedTreeModel.CONSTRAINED_TREE_MODEL, treeModelPrefix + DefaultTreeModel.TREE_MODEL);
writer.writeCloseTag(NodeHeightOperatorParser.NODE_HEIGHT_OPERATOR);
}

private void writeBFTRootScaleOperator(Operator operator, String treeModelPrefix, XMLWriter writer) {
writer.writeOpenTag(NodeHeightOperatorParser.NODE_HEIGHT_OPERATOR, // should this be in the class and not the parser?
new Attribute[]{
new Attribute.Default<String>("type",NodeHeightOperatorParser.OperatorType.SCALEROOT.toString()),
// todo scale factor here
new Attribute.Default<Double>(NodeHeightOperatorParser.SCALE_FACTOR, operator.getTuning()),
getWeightAttribute(operator.getWeight())
}
);
writer.writeIDref(ConstrainedTreeModel.CONSTRAINED_TREE_MODEL, treeModelPrefix + DefaultTreeModel.TREE_MODEL);
writer.writeCloseTag(NodeHeightOperatorParser.NODE_HEIGHT_OPERATOR);
}

private void writeUniformSPROperator(Operator operator, String treeModelPrefix, XMLWriter writer) {
writer.writeOpenTag(UniformSubtreePruneRegraftParser.UNIFORM_SUBTREE_PRUNE_REGRAFT, // should this be in the class and not the parser?
new Attribute[]{
getWeightAttribute(operator.getWeight())
});
writer.writeIDref(ConstrainedTreeModel.CONSTRAINED_TREE_MODEL, treeModelPrefix + DefaultTreeModel.TREE_MODEL);
writer.writeCloseTag(UniformSubtreePruneRegraftParser.UNIFORM_SUBTREE_PRUNE_REGRAFT);
}


// tuneable version of FHSPR but not currently being used
private void writeSubtreeJumpOperator(Operator operator, String treeModelPrefix, XMLWriter writer) {
writer.writeOpenTag(SubtreeJumpOperatorParser.SUBTREE_JUMP,
Expand Down
92 changes: 90 additions & 2 deletions src/dr/app/beauti/generator/TreeLikelihoodGenerator.java
Original file line number Diff line number Diff line change
Expand Up @@ -29,19 +29,29 @@

import dr.app.beauti.types.ClockType;
import dr.evomodel.tree.DefaultTreeModel;
import dr.evomodelxml.bigfasttree.thorney.ConstrainedBranchLengthProviderParser;
import dr.evomodelxml.bigfasttree.thorney.PoissonBranchLengthLikelihoodParser;
import dr.evomodelxml.bigfasttree.thorney.ThorneyTreeLikelihoodParser;
import dr.evomodelxml.treedatalikelihood.TreeDataLikelihoodParser;
import dr.evomodelxml.treelikelihood.MarkovJumpsTreeLikelihoodParser;
import dr.app.beauti.components.ComponentFactory;
import dr.app.beauti.components.ancestralstates.AncestralStatesComponentOptions;
import dr.app.beauti.options.*;
import dr.app.beauti.util.XMLWriter;
import dr.evolution.datatype.DataType;
import dr.evolution.tree.TreeUtils;
import dr.evomodel.bigfasttree.thorney.ConstrainedTreeBranchLengthProvider;
import dr.evomodel.bigfasttree.thorney.ConstrainedTreeModel;
import dr.evomodel.bigfasttree.thorney.MutationBranchMap;
import dr.evomodel.bigfasttree.thorney.PoissonBranchLengthLikelihoodDelegate;
import dr.evomodel.branchratemodel.BranchRateModel;
import dr.oldevomodel.sitemodel.GammaSiteModel;
import dr.oldevomodel.sitemodel.SiteModel;
import dr.oldevomodelxml.treelikelihood.AncestralStateTreeLikelihoodParser;
import dr.oldevomodelxml.treelikelihood.TreeLikelihoodParser;
import dr.evoxml.AlignmentParser;
import dr.evoxml.NewickParser;
import dr.evoxml.SimpleTreeParser;
import dr.evoxml.SitePatternsParser;
import dr.util.Attribute;
import dr.xml.XMLParser;
Expand Down Expand Up @@ -74,13 +84,81 @@ public void writeAllTreeLikelihoods(XMLWriter writer) throws GeneratorException

// if the partition isn't an instanceof PartitionData then it doesn't
// need a TreeLikelihood (it is probably a Tree Partition)
// } else {
// throw new GeneratorException("Unrecognized partition:\n" + partition.getName());
} else {
if(partition instanceof TreePartitionData) {
PartitionTreeModel model = partition.getPartitionTreeModel();
if(model.isUsingThorneyBEAST()){
writeThorneyTreeLikelihood((TreePartitionData) partition, writer);
writer.writeText("");
}

}
}
}
}
}

public void writeThorneyTreeLikelihood(TreePartitionData partition,XMLWriter writer){
// write mutation branch map

/**
* <simpleMutationBranchMap id="branchMutationCounts" scale="29903.0">
<constrainedTreeModel idref="treeModel"/>
<dataTree>
<tree idref="dataTree"/>
</dataTree>
</simpleMutationBranchMap>
*/

PartitionTreeModel treeModel = partition.getPartitionTreeModel();
PartitionClockModel clockModel = partition.getPartitionClockModel();
String prefix = treeModel.getPrefix() + clockModel.getPrefix(); // use the treemodel prefix

String idString = prefix + "treeLikelihood";
String branchMapIdD = prefix + "branchMutationCounts";


writer.writeComment("A map between the branches in the tree model and the data providing the number of mutations along each branch.");
writer.writeTag(ConstrainedBranchLengthProviderParser.MUTATION_BRANCH_MAP, new Attribute[]{
new Attribute.Default<>(XMLParser.ID,branchMapIdD),
new Attribute.Default<Double>("scale",treeModel.getThorneyScaler()),
},false);
writer.writeIDref(ConstrainedTreeModel.CONSTRAINED_TREE_MODEL,prefix+ DefaultTreeModel.TREE_MODEL);
writer.writeTag(ConstrainedBranchLengthProviderParser.DATA_TREE, new Attribute[]{}, false);
writer.writeOpenTag(
NewickParser.NEWICK,
new Attribute[]{
new Attribute.Default<>(XMLParser.ID, prefix + "dataTree"),
new Attribute.Default<Boolean>(NewickParser.USING_HEIGHTS, true),
new Attribute.Default<Boolean>(NewickParser.USING_DATES, false)
}
);
writer.writeText(TreeUtils.newick(treeModel.getTreePartitionData().getTrees().getTrees().get(0)));
writer.writeCloseTag(NewickParser.NEWICK);
writer.writeCloseTag(ConstrainedBranchLengthProviderParser.DATA_TREE);
writer.writeCloseTag(ConstrainedBranchLengthProviderParser.MUTATION_BRANCH_MAP);



writer.writeOpenTag(ThorneyTreeLikelihoodParser.THORNEY_DATA_LIKELIHOOD_DELEGATE, new Attribute[]{
new Attribute.Default<String>(XMLParser.ID, idString)
});

writer.writeIDref(ConstrainedTreeModel.CONSTRAINED_TREE_MODEL, prefix + DefaultTreeModel.TREE_MODEL);
writer.writeIDref(ConstrainedBranchLengthProviderParser.MUTATION_BRANCH_MAP, branchMapIdD );


writer.writeTag( PoissonBranchLengthLikelihoodParser.POISSON_BRANCH_LENGTH_LIKELIHOOD,
new Attribute[] {
new Attribute.Default<>(XMLParser.ID, prefix + "branchMutationLikelihood"),
new Attribute.Default<Double>("scale", treeModel.getThorneyScaler()),
}, true);

ClockModelGenerator.writeBranchRatesModelRef(clockModel, writer);

writer.writeCloseTag(ThorneyTreeLikelihoodParser.THORNEY_DATA_LIKELIHOOD_DELEGATE);
}

public void writeTreeDataLikelihood(List<PartitionData> partitions, XMLWriter writer) {

PartitionSubstitutionModel substModel = partitions.get(0).getPartitionSubstitutionModel();
Expand Down Expand Up @@ -287,6 +365,16 @@ public void writeTreeLikelihoodReferences(XMLWriter writer) {
AncestralStatesComponentOptions ancestralStatesOptions = (AncestralStatesComponentOptions) options
.getComponentOptions(AncestralStatesComponentOptions.class);

// TODO moving poisson branch length to a substitution model would make this logic simplier

PartitionTreeModel treeModel = partition.getPartitionTreeModel();
if(treeModel.isUsingThorneyBEAST()){
PartitionClockModel clockModel = partition.getPartitionClockModel();
String prefix = treeModel.getPrefix() + clockModel.getPrefix(); // use the treemodel prefix
String idString = prefix + "treeLikelihood"; // Line 117
writer.writeIDref(ThorneyTreeLikelihoodParser.THORNEY_DATA_LIKELIHOOD_DELEGATE, idString);

}
PartitionSubstitutionModel model = partition.getPartitionSubstitutionModel();

if (model == null) {
Expand Down
Loading
Loading