-
Notifications
You must be signed in to change notification settings - Fork 77
/
Historic_ChangeLog
514 lines (364 loc) · 22.5 KB
/
Historic_ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
Version 18
1) Switched build process to use Apache Maven. Changed directory structure. Removed ant and Make files. Most jar files are now pulled directly from the Maven repository.
2) Upgraded Java version as Java 8 will be going out of support. Artemis will now only build/run with Java versions 9 to 11 (RT ticket #589103).
3) Upgraded to Picard 2.18.14 jar, which has a number of important bug fixes, including:
i) RT tickets #625730 - Loading bams into Artemis
ii) RT ticket #624271 - Problems with some bams loaded into Artemis causing data not to be viewable
4) Removed broken documentation functionality.
HTML/PDF manuals are now stored in Github gh-pages branch and exported from word docs (RT ticket #618907).
5) Added run menu option to execute an InterPro sequence search for a selected feature - RT ticket #643820.
6) Updated application scripts.
7) Fixed broken Amigo, GO term, Uniprot (RT ticket #639930) and UniprotKB URLs, used in the Artemis gene builder/feature edit windows.
8) Updated README.md
9) Fix for RT ticket #400288: "GO term warnings in Artemis" - we now no longer put out a warning for ISM, as it's optional in certain cases
[http://wiki.geneontology.org/index.php/Inferred_from_Sequence_Model_(ISM)].
10) Removed the INSTALL document as the content is now covered elsewhere.
11) Changed error message displayed when multiple features are selected for pfam/rfam search.
12) Added temporary Java 9+ --add-opens flags to art/act scripts. These will be removed when Ibatis is replaced.
13) Added unit tests for Chado access to enable Postgres 10 testing (RT ticket #632896).
14) Added Mockito for unit test mocking.
15) Updated/fixed run_blast<..> scripts in the etc folder. Running blast from the run menu now uses blast+. Removed options to run Smart, Jalview, fasta & fastx.
Results are displayed using View->Search Results.
Related RT ticket #596281 also closed.
16) Removed the old icons/Readme doc for local blast setup and added an updated version to the main README.md.
17) Updated setup_blast_dbs.sh script and added to etc folder, for setup of a local blast environment (detailed in the README.md and manuals).
18) Removed ability to run remote blast, fasta34 etc using SSH. This functionality is being retired.
19) Removed old ".sanger" and ".sanger.linux" scripts from etc folder - these were very out of date, with incorrect file paths etc.
20) Fixed RT ticket #642350: "GO term evidence codes in chado Artemis" - Added HTP, HDA, HMP, HGI, HEP GO evidence codes to Artemis.
21) Added fix for Java 9 JDK-8181568 issue across all apps, related to display of application icons on Macs.
22) Fixed incorrect Pfam URL, used in Feature Edit and Evidence Viewer screens.
23) Fixed error handling on Run->Pfam/Rfam search (RT ticket #420623).
24) Enabled ability to drag and drop data files on to Artemis and BamView apps [Mac OS X] for immediate display.
25) Fixed RT ticket #642944: "ACT crunch-m8 file generation". ACT can now handle tblastx hit table files produced from comparisons
on the NCBI Blast web site. This is an alternative to Web ACT / Double ACT.
[e.g. https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=tblastx&PAGE_TYPE=BlastSearch&BLAST_SPEC=blast2seq&LINK_LOC=blasttab&LAST_PAGE=tblastx&BLAST_INIT=blast2seq]
26) Removed the /lustre/scratch101/blastdb/Pathogen/Kineto_aa run->blast menu option. This database path no longer exists.
27) RT ticket #472601: Artemis RPKM and read count broken since version 16.0.6. Fixed bug related to intron region reads. New unit tests added.
Version 17.0.2-test - patch release for Sanger. Changes incorporated into GitHub master.
1) Added bamview_use_htsjdk_file_index_caching property to allow switching off of htsjdk caching for bam files.
This gets around a bug for large bams whereby parts of the bam fail to get loaded - RT ticket #624271.
See htsjdk GitHub issue 1127 for details.
Version 17
Artemis/ACT and associated applications have been upgraded to Java 8. Java 7/6/5 are now no longer supported - RT ticket #589103
BamView fixes/changes -
1) Improved mouse selection of reads.
2) Fixed exceptions thrown when zooming into an area with secondary reads or SNPs enabled - RT ticket #596489: Problem zooming in using isoseq data
3) Removed Samtools (htsjdk) jar and updated Picard jar (which includes htsjdk).
4) Upgraded Apache Commons Net jar - used for ftp'ing.
5) Added Cram file loading and fixed defects around that code area (also related RT ticket #560982: Artemis fails to build from source).
6) Added display of bam secondary & supplementary alignment flags to read details pop-up window.
7) Improved error handling/reporting, although this still needs more work.
8) Added buffer sizing to FTPSeekableStream.
9) Performance tweak for BAM file indexing.
10) Fixed issue in BamView whereby if you went to the end of a long contig and then switched to a shorter one (via the combo), you would end up off the end of the sequence
resulting in a negative array index exception. Code has been changed to reset display to the start of sequence when the combo is changed (i.e. same as Artemis).
11) Added bamview_perform_detailed_validation boolean property that can be set to perform more detailed sam validation during loading - this can be slow for large files.
Added EvoSuite unit tests and additional targeted tests (RT ticket #419534: Develop a test suite for Artemis).
Added Jacoco unit test coverage reporting - manual tests only currently. This may be switched to Clover in the future if this proves compatible with EvoSuite
- for the moment we are stuck with two coverage reports - evosuite and Jacoco.
Removed redundant Corba libraries.
Upgraded JUnit jar.
Changed Travis yml build file. RT ticket #597497: Set up artemis tests to run on travis build.
Changed all build scripts and startup scripts - RT ticket #598617. Removed building of artemis_mac.jar which is not used.
Removed etc/gene_builder script and the gff2embl script (and documentation references) - no longer supported.
Fixed RT ticket #606061 - Using EBI-Dbfetch on ARTEMIS. Change to DbfetchEntrySource.java due to incorrect regular expression and EBI URL.
Resized splash screen, as it was too small to the accommodate text info.
Updated splash screen logo.
Added error handling for dnaplotter template file loading in standalone mode.
Bioconda recipe prep work - RT ticket #341139.
Fixed RT ticket #467433 - Genbank DBSOURCE field was not recognised by Artemis. DBLINK was already added.
Fixed RT ticket #503254 - Act doesn't detect if file does not exist.
KNOWN ISSUES:
1) Java JDKs 1.8.0_131 and above have a Swing bug related to overlaying of modal dialogs for Mac OS X: https://bugs.openjdk.java.net/browse/JDK-8179335
2) There is a current bug in htsjdk whereby calls to the queryMate functionality can throw an exception on reads with secondaries and/or supplementals.
This affects looking up properties for a selected read on bamview - https://github.com/samtools/htsjdk/issues/1065
Version 16
Add 'Features Within Selection' option to the 'Select' menu to select
features that are contained by a selected base range.
Saving an entry as EMBL submission has an option now to remove products
from CDS features with a pseudogene qualifier.
From the 'View' menu added 'Adjust panel heights...' option in ACT
for adjusting their heights (BAM, VCF, plots, comparisons) by giving
them different weights in order to distribute the space between each
component.
Added ability to hide graph lines. Right click on the graph to get the
popup menu and select 'Configure...'. Then use the 'Line size' slider
to reduce the size of the line(s) that you want to hide to zero.
Labels can optionally be added to the header of base position plots
and these are used in the legend, e.g.
# colour 5:150:55 255:0:0 0:255:0 0:0:255 100:100:100 50:150:50
# label lab1 lab2 lab3 lab4 lab5 lab6
176 2204.8 848.23 0 0 0 536.04
...
In the Feature Editor there is now a button ('User Qualifiers') which
opens a tool for maintaining user defined lists of qualifiers (i.e.
qualifiers in the form tag = value pairs on separate lines) and the
option to read qualifiers from OBO formatted files or URLs. These can
be saved between sessions in the file '.artemis.qualifiers' in the
home directory.
Automatic addition of history qualifier in chado database mode.
Added RNASeq strand specific option to the BAM popup menu ('Colour By' ->
'RNASeq Strand Specific Tag (XS)'). Reads colours are based on the XS
tag (used by TopHat). The RNA strand is then used in the coverage plots
and in calculation of read counts and RPKM values.
Added the following flags:
-Dshow_snps Show SNP marks in BamView
-Dshow_snp_plot Open SNP plot in BamView
-Dshow_cov_plot Open coverage plot in BamView
Add validation checks. This will check the following:
All file types:
- CDS have no internal stop codon
- CDS have valid stop codon
GFF / Chado:
- check complete gene model
- check boundaries are valid
- check all features are on the same strand
- check CDS features have a phase
- check attribute column
- qualifiers have a value (not empty)
- only reserved tags start with uppercase
Validation can be run in two ways:
1. using the option in View->Feature Filters->Validation checks... which shows
'failed' features in feature list windows
2. selecting 'Validation report ...' option in the popup menu when right clicking
on the feature display. This produces a report with an option to auto-fix gene
boundaries and stop codons.
Version 15
Multiple BAM panels can be opened using the bamClone flag this is used with
the -Dbam flag:
art -Dbam='/pathToFile/file1.bam,/pathToFile/file2.bam' -DbamClone=n
(where n is an integer greater than 1). All BAM files are then shown in each
panel. Alternatively the following will open BAM files in separate panels
(using -Dbam[1,2,3...]):
art -Dbam1=fileA.bam -Dbam2=fileB.bam
Add SVG (scalable vector graphics) support for Artemis, ACT and DNAPlotter.
Option added to adjust VCF row height.
Add support for indexed user graphs using tabix. For example file.plot is a tab
delimited file with column 1 containing the sequence name and column 2 the
positions:
(grep ^"#" file.plot; grep -v ^"#" file.plot | sort -k1,1 -k2,2n) | bgzip > sorted.plot.gz ;
tabix -s 1 -b 2 -e 2 sorted.plot.gz
Option added to show or hide the average line in the graphs.
BAM coverage heatmap view added.
Add Rfam sequence search from the RUN menu.
Base similarity graph for each VCF added to the VCF view.
Sense and anti-sense read counts and RPKM values are now reported.
Option added to create features from BAM peaks, i.e. above a threshold of the
number aligned reads.
Option included for defining groups of BAM files so that they can be
switched on and off by their group.
Coverage plots from read alignments (BAM) can be plotted by their strand.
Addition of a Project File Manager used to group files together for
launching in Artemis. When a project has been added or updated the details
are saved at the end of each session in '.artemis.project.properties'
in the user's home directory.
Fix for saving user defined shortcuts between sessions on windows.
Add preliminary support for the CRAM format:
http://www.ebi.ac.uk/ena/about/cram_toolkit
This requires the cramtools.jar to be added to the start of the CLASSPATH.
Improved support for matching GFF feature coordinates to the correct
contig in a multiple FASTA sequences. This works now whether
the sequence is part of the GFF file or a separate multiple-FASTA file.
Add 'Feature Stack View' to visualise overlapping gene features.
Add support for read-only indexed GFF. Features in GFF format are
sorted and indexed with tabix:
http://samtools.sourceforge.net/tabix.shtml
Version 14
Add options in the navigator for searching the forward and
reverse strands individually for base / amino acid patterns.
Add an option to the navigator for searching for matches that
overlap a selected region or feature.
New translation table 24 (Pterobranchia mitochondrial) added.
If the BAM index file is missing then Artemis uses the picard library to
attempt to create the index.
Variant (VCF / BCF) filtering now uses the meta-data in the header to
enable filtering based on INFO, FILTER and FORMAT columns.
Shortcut changes made in the 'Preferences' menu are saved between sessions (the
shortcut_cache flag in the options file can be used to turn this on/off).
More support for GTF format to show CDS and exons as joined features.
BAM record list option added to display as a list the reads and their
properties.
Add options for loading graph, BAM and VCF files into ACT from the
command line. Numbers are used to associate the file with a particular
sequence. e.g. add a BAM to the first (top) sequence in ACT:
act -Dbam1=/pathToFile/file.bam
or, to add a userplot to the second sequence:
act -Duserplot2=/pathToFile/userPlot
Version 13.2.0
Added option to provide overview of the variation sites.
Change BamView filter to enable filtering in and out based on
reads flag.
Add read count and RPKM calculations to BamView.
Add option in BamView to clone the alignment panel.
Create features option for VCF records.
Unit tests added for writing VCF/BCF sequences.
Add new Coverage view to BamView. It automatically switches to this view
on zooming out.
Add options to write out or view FASTA sequences from VCF/BVF variation
data.
Version 13
Add support for reading in indexed BCF (Binary VCF) files.
Memory optimisation of codon caching, reducing the memory footprint.
It is now possible to add BAM, VCF and BCF files from the command line using the
JVM bam option, e.g. art -Dbam=/pathToFile/file.bam
and for multiple BAM's/VCF's this is comma separated
art -Dbam='/pathToFile/file1.bam,/pathToFile/file2.bam', it
can also read the BAM's from URL's.
Added option to display orientation of reads in BamView.
In Bamview, display reads that are split over introns so that the exon boundaries can
be identified by colouring the line between the aligned blocks grey.
Add VCF panel to ACT.
Support added to read indexed fasta sequence files. An index file is created
using SAMtools.
Add support for VCF v3.3 and v4.0.
Add option to show combined coverage plots for multiple BAMs.
Add show_forward_lines and show_reverse_lines as options for switching
frame lines on and off.
Add an option to delete qualifiers in the Find/Replace tool.
Artemis can read in a set of zipped search results (e.g. blastp/blastp.zip).
Add BamView panel to ACT.
Fix for writing EMBL files out from GFF entries.
An option has been added in BamView to colour reads by the colour
used in the coverage plot - useful when looking at multiple BAM files.
Plot multiple coverage plots in BamView when multiple BAM files are
loaded in.
Version 12
It is now possible to add user plots from the command line using the
JVM userplot option, e.g. art -Duserplot=/pathToFile/userPlot
and for multiple plots this is comma separated
art -Duserplot='/pathToFile/userPlot1,/pathToFile/userPlot2', it
can also read the plots from URL's.
Added support to be able to read BAM files to display read alignments.
It uses picard (http://picard.sourceforge.net/) to read from the BAM file
and so requires Artemis to be run with Java 1.6.
All graphs can now be configured to be plotted as line graphs
or heat maps.
Added support for BLAST tabular format (-m 8 option) and MSPcrunch
format to plot scores.
Added support for wiggle (variableStep/fixedStep) plots that can be
displayed as histograms or heat maps.
Added a new user plot file format with the first column specifying
a base position.
More configure options have been added to the graphs to enable
configuration of the graph line style and size.
GoTo directory option added to file manager to assist navigation.
The database manager is cached between sessions (this can is on by default
and can be switched off with -Ddatabase_manager_cache_off). There is an
option under the File menu to clear this cache.
A checkbox has been added to the window for adding ortholog/paralog links
in the Gene Builder. If this is selected it adds links between existing
ortholog/paralogs and the new ortholog/paralog. By default this is off.
An option has been added to the database manager to display polypeptide
domains in the feature display (as well as the protein map in the Gene
Builder).
An option has been added to the Run menu for doing a search of the Pfam
database.
Added an option to the Write menu for writing a combination of upstream
+ feature + downstream bases for selected features.
Option added in the View->Filter Features menu to search for Duplicate
Systematic Name Qualifier.
LookSeq analysis panel can be displayed by setting the lookseq
value in the options file. An option under the Display menu then
is used to shows the LookSeq read alignment panel in Artemis.
Added options to set the minimum and maximum values of the plots.
Transfer Annotation Tool (TAT) added to feature editor and Gene
Builder.
New graph popup menu option to show the values and average for
a selected range.
Added product_cv database option to define if the product is
stored as a controlled vocabulary or as a feature property (featureprop).
Version 11
Use black_belt_mode to suppress warnings when opening Artemis.
Script (writedb_entry) added to make it easier to write out
multiple entries as EMBL / GFF files from chado. This uses Artemis
read-write libraries and does not require each sequence to be
launched from the database. For command line help run:
etc/writedb_entry -help
On MacOSX - enable dropping files on Artemis application to open sequences
with file extensions: gff, embl, EMBL, genbank, gbk, fasta, seq, art and
dna (defined in Artemis.app/Contents/Info.plist).
Add NCBI search link to run menu. This transfers the sequence automatically
to the NCBI web page.
Add support for writing Sequin table format.
Added print to PostScript option in Artemis and ACT.
Optimisation of reading in user graphs.
New Edit->Selected Feature(s)->Convert Keys option to convert
keys of selected features.
New Edit menu option for finding and replacing qualifier text. This
has an option for boolean searches (e.g. and, or, &, |) of qualifier
text. This includes an option to search for duplicate qualifiers.
Implemented a commit manager for the database mode. This
highlights transactions that produce an error.
Make the chado transaction log messages more human readable.
Add ability to write file formats from Artemis in database mode.
With option to collapse the gene hierarchy (gene, transcript, exon)
into a CDS feature.
Graphs are now added to a split pane. So that their size can be
defined by dragging the divider at the bottom of the graphs.
Provide option to log transform user data plot.
Improved error reporting for contig reordering.
Fix for creating intergenic features for overlapping CDS's. Also
add note based on which one of the 4 cases with respect to the
flanking CDS it belongs to, i.e.:
IGR-F (forward): cds> IGR cds>
IGR-R (reverse): <cds IGR <cds
IGR-B (both): <cds IGR cds>
IGR-X: cds> IGR <cds
Add option to preferences for defining contig ordering features.
Added option to "Create features from graph peaks". For a graph
this creates features in regions above a given cut-off and above
a given feature size.
Add -Dread_only option for read only databases.
Add option to lazy load feature data from database.
Version 10
Add redo function to 'Edit' menu. Also enable/disable undo and
redo menu items when available/not available.
Add option to replace selected bases in 'Edit' menu.
Option to create features in intergenic regions added.
Feature editor now marks hyperlinks to SWALL, EMBL, UniProt,
PMID, PubMed, InterPro, OrthoMCLDB, Pfam that are opened in
the browser. Now configured in the options file.
Added "Convert Qualifier of Selected..." option to the Edit menu.
This allows the user to change the names of qualifiers for all
selected features.
Now using new release of j2ssh (0.2.9). This requires Java1.5+.
Implemented option for ORF creation to take into account boundaries
of multiple fasta sequences, so that they do not cross them.
Implemented the ability to run and store fasta and blast search
results for multiple databases... E.g. fasta searches on uniprot
and on user's own database, stored in multiple fasta_file qualifiers.
"Set Score Cutoffs" in Artemis popup menu uses existing /scores as
the initial min and max values (rather than just 0 and 100).
Added cache to store the entries retrieved for the object editor.
Implement log4j logging to be displayed in log viewer. Using colour
coding depending on level of logging.
Added -Doffset so that Artemis can be opened at a given base.
Version 9
Feature selector can be used to look for features with introns
that do no contain the GT/GC start or AG end.
Contig tool now checks for contigs that contain features that
span the boundaries of the contigs. These features have to be
removed or restricted to the contig boundary before it can
carry out contig reordering.
Fix for converting files from other file formats to
genbank format.
Fix rounding problem for long sequences when writting
out all bases in FASTA or raw format.
Cache the start codons (as per stop codon caching),
to speed their display.
Combine the extend to next exon and the fix stop codons
into one option.
The feature types that appear on the frame lines can be defined
by the user via an option ("Frame Line Features...") in the
feature display popup menu.
Added to File -> Preferences a user defined selection for display names
and systematic names. Also extended popup menu option in feature lists
to allow the user to be able to select multiple qualifiers to display.
For entries opened from the remote side of an SSH connection will search
for results on the remote file system if they are not found locally. They
are transferred via SSH and then stored locally.
For Mac users, the option to send search results to the browser will
display the results in the default browser.
When automatically generating gene names (under the Edit menu), the user
can specify the number of zeros to pad the numbering with. e.g if 5 digits are
selected the format will look like : 00001, 00002 etc.