Add SVStratify and GroupedSVCluster tools #8990
Conversation
This is a combination of 2 commits. Implement SVStratify Interchrom events Start unit tests Finish unit tests Multiple contexts per stratum Integration tests; fix empty context bug Test duplicate context name Handle CPX/CTX, add some tests Compiler error Prioritize SR and PE evidence types for representative breakpoint strategy Add SVStratifiedCluster Integration tests for SVStratifiedClustering Unused line Spooled sorting in cluster tools Start fixing JointGCNVSegmentation Fix JointGCNVSegmentation integration test Rename to GroupedSVCluster Fix sorting bug Comment Use RD_CN for CNV sample overlap; improve testing Documentation Add comment about requiresOverlapAndProximity Allow empty EVIDENCE list Add STRAT INFO field Representative breakpoint by variant quality Handle BOTHSIDES_SUPPORT and HIGH_SR_BACKGROUND in variant collapsing Add expected exception
| @@ -77,17 +77,17 @@ protected static boolean hasSampleOverlap(final SVCallRecord a, final SVCallReco | |||
| final Set<String> samples = new HashSet<>(SVUtils.hashMapCapacity(genotypesA.size() + genotypesB.size())); | |||
| samples.addAll(genotypesA.getSampleNames()); | |||
| samples.addAll(genotypesB.getSampleNames()); | |||
| if (samples.isEmpty()) { | |||
| // Empty case considered perfect overlap | |||
There was a problem hiding this comment.
For my own edification, why is that?
| if (matchedSampleGenotype != null) { | ||
| return VariantContextGetters.getAttributeAsInt(matchedSampleGenotype, GATKSVVCFConstants.EXPECTED_COPY_NUMBER_FORMAT, 0); | ||
| } else { | ||
| return 0; |
There was a problem hiding this comment.
I understand we have to choose something as the default, but I feel weird defaulting to homozygous deletion. Why not -1?
| protected GATKPath outputFile; | ||
|
|
||
| /** | ||
| * Expected format is tab-delimited and contains a header with the first column SAMPLE and remaining columns |
There was a problem hiding this comment.
Do we have a tool that outputs this file? This is not familiar to me.
| * clustering, but any associated annotation fields will be set to null in the output. | ||
| */ | ||
| @Argument( | ||
| doc = "Fast mode. Drops hom-ref and no-call genotype fields and emits them as no-calls.", |
There was a problem hiding this comment.
I wouldn't call the attributes no-calls, I would call them missing
| * Default genotypes are assigned when they cannot be inferred from the inputs, such as when VCFs with different | ||
| * variants and samples are provided. | ||
| */ | ||
| @Argument(fullName = DEFAULT_NO_CALL_LONG_NAME, |
There was a problem hiding this comment.
By default you get 0/0 with no attributes?
| runCommandLine(args, SVStratify.class.getSimpleName()); | ||
| } | ||
|
|
||
| @Test(expectedExceptions = {GATKException.class, IllegalArgumentException.class}) |
There was a problem hiding this comment.
I've been burned this way before where another class throws and IllegalArgumentException. I also made a note where the Utils arg validation is.
| public void testBwaMeltCohortRedundant() { | ||
| final File outputDir = createTempDir("stratify"); | ||
| final String inputVcfPath = getToolTestDataDir() + "bwa_melt.chr22.vcf.gz"; | ||
| final String configFile = getToolTestDataDir() + "test_config_redundant.tsv"; |
There was a problem hiding this comment.
What happens if the config conflicts with the cmdline args?
| * @param svType SV type, may be null | ||
| * @param minSize minimum size in bp (inclusive), may be null | ||
| * @param maxSize maximum size in bp (exclusive), may be null | ||
| * @param contextNames reference context names |
There was a problem hiding this comment.
This confused me for a while because in the engine the ReferenceContext is like the actual FASTA sequence. You think this is obvious enough for users who haven't done GATK development?
| expectedOutputSuffixes.put("DUP_lt5kb_RM", 0); | ||
| expectedOutputSuffixes.put("INV_gt1kb", 26); | ||
| expectedOutputSuffixes.put("BND_SD", 77); | ||
| expectedOutputSuffixes.put(SVStratify.DEFAULT_STRATUM, 1196); |
There was a problem hiding this comment.
I'm going to trust that you verified this with an orthogonal python script or somesuch
| public GATKPath configFile; | ||
|
|
||
| @Argument( | ||
| doc = "Context intervals file. Can be specified multiple times.", |
There was a problem hiding this comment.
I would call this something more like "repeat track intervals file". I'm sure you could generalize beyond repeats, but I found this really vague.
Implements two new tools and updates some methods for a revamp of the
CombineBatchescross-batch integration module in gatk-sv.SVStratify- tool for splitting out a VCF by variant class. Users pass in a configuration table (see tool documentation for an example) specifying one or more stratification groups classified by SVTYPE, SVLEN range, and reference context(s). The latter are specified as a set of interval lists using--context-nameand--context-intervalsarguments. All variants are matched with their respective group which is annotated in theSTRATINFO field. Optionally, the output can be split into multiple VCFs by group, which is a very useful functionality that currently can't be done efficiently with common commands/toolkits.GroupedSVCluster- a hybrid tool combining functionality fromSVStratifywithSVClusterto perform intra-stratum clustering. This tool is critical for fine-tuned clustering of specific variants types within certain reference contexts. For example, small variants in simple repeats tend to have lower breakpoint accuracy and are typically "reclustered" during call set refinement with looser clustering criteria.SVStratificationEngine- new class for performing stratification.CanonicalSVCollapserthat should improve breakpoint accuracy, particularly in larger call sets. Raw evidence support and variant quality are now considered when choosing a representative breakpoint for a group of clustered SVs.FlagFieldLogictype for customizing howBOTHSIDE_PASSandHIGH_SR_BACKGROUNDINFO flags are collapsed during clustering.RD_CNis now used as a backup ifCNis not available when determining carrier status for sample overlap.