Parallel lock free testing, remove potential deadlocks, cache static data, go to descriptor via test

doc/antlr-project-testing.md

@@ -87,6 +87,8 @@ And the result of testing the entire subdirectory:
 
 All test are run in parallel both via maven and via IDE.
 
+In IntelliJ, it's very easy to go to source by right-clicking on any test and pressing `Jump to source` (F4).
+
 ## Running test subsets
 
 From the `runtime-testsuite` dir
@@ -96,12 +98,14 @@ From the `runtime-testsuite` dir
 ```bash
 $ cd runtime-testsuite
 $ export MAVEN_OPTS="-Xmx1G"     # don't forget this on linux
-$ mvn -Dtest=JavaRuntimeTests test
+$ mvn -Dtest=java.** test
 -------------------------------------------------------
  T E S T S
 -------------------------------------------------------
+[INFO] Running org.antlr.v4.test.runtime.java.TestIntegerList
 [INFO] Running org.antlr.v4.test.runtime.java.JavaRuntimeTests
 ...
+[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 s - in org.antlr.v4.test.runtime.java.TestIntegerList
 [INFO] Tests run: 348, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.269 s - in org.antlr.v4.test.runtime.java.JavaRuntimeTests
 ...
 ```

doc/creating-a-language-target.md

@@ -6,19 +6,42 @@ This document describes how to make ANTLR generate parsers in a new language, *X
 
 Creating a new target involves the following key elements:
 
-1. For the tool, create class *X*Target as a subclass of class `Target` in package `org.antlr.v4.codegen.target`. This class describes language specific details about escape characters and strings and so on. There is very little to do here typically.
-1. Create *X*.stg in directory tool/resources/org/antlr/v4/tool/templates/codegen/*X*/*X*.stg. This is a [StringTemplate](http://www.stringtemplate.org/) group file (`.stg`) that tells ANTLR how to express all of the parsing elements needed to generate code.  You will see templates called `ParserFile`, `Parser`, `Lexer`, `CodeBlockForAlt`, `AltBlock`, etc... Each of these must be described how to build the indicated chunk of code. Your best bet is to find the closest existing target, copy that template file, and tweak to suit.
-1. Create a runtime library to support the parsers generated by ANTLR. Under directory runtime/*X*, you are in complete control of the directory structure as dictated by common usage of that target language. For example, Java has: `runtime/Java/lib` and `runtime/Java/src` directories. Under `src`, you will find a directory structure for package `org.antlr.v4.runtime` and below.
-1. Create a template file for runtime tests. All you have to do is provide a few templates that indicate how to print values and declare variables. Our runtime test mechanism in dir `runtime-testsuite` will automatically generate code using these templates for each target and check the test results.  It needs to know how to define various class fields, compare members and so on. You must create a *X*.test.stg file underneath [runtime-testsuite/resources/org/antlr/v4/test/runtime](https://github.com/antlr/antlr4/tree/master/runtime-testsuite/resources/org/antlr/v4/test/runtime). Again, your best bet is to copy the templates from the closest language to your target and tweak it to suit.
-1. Create test files under [/runtime-testsuite/test/org/antlr/v4/test/runtime](https://github.com/antlr/antlr4/tree/master/runtime-testsuite/test/org/antlr/v4/test/runtime). They will load defined test cases in each test descriptor. Also add the `/runtime-testsuite/test/org/antlr/v4/test/runtime/X/BaseXTest.java` which defines how test cases will execute and output.
-1. Create/edit shell scripts in [/.travis](https://github.com/antlr/antlr4/blob/master/.travis) and [/appveyor.yml](https://github.com/antlr/antlr4/blob/master/appveyor.yml) to run tests in CI pipelines.
+1. For the tool, create class *X*Target as a subclass of class `Target` in package `org.antlr.v4.codegen.target`.
+   This class describes language specific details about escape characters and strings and so on.
+   There is very little to do here typically.
+2. Create `*X*.stg` in directory `tool/resources/org/antlr/v4/tool/templates/codegen/*X*/*X*.stg`.
+   This is a [StringTemplate](http://www.stringtemplate.org/) group file (`.stg`) that tells ANTLR how to express
+   all the parsing elements needed to generate code.
+   You will see templates called `ParserFile`, `Parser`, `Lexer`, `CodeBlockForAlt`, `AltBlock`, etc...
+   Each of these must be described how to build the indicated chunk of code.
+   Your best bet is to find the closest existing target, copy that template file, and tweak to suit.
+3. Create a runtime library to support the parsers generated by ANTLR.
+   Under directory `runtime/*X*`, you are in complete control of the directory structure as dictated by common usage of that target language.
+   For example, Java has: `runtime/Java/lib` and `runtime/Java/src` directories.
+   Under `src`, you will find a directory structure for package `org.antlr.v4.runtime` and below.
+4. Create a template file for runtime tests.
+   All you have to do is provide a few templates that indicate how to print values and declare variables.
+   Our runtime test mechanism in dir `runtime-testsuite` will automatically generate code using these templates for each target and check the test results.
+   It needs to know how to define various class fields, compare members and so on.
+   You must create a `*X*.test.stg` file underneath [runtime-testsuite/resources/org/antlr/v4/test/runtime](../runtime-testsuite/resources/org/antlr/v4/test/runtime)
+   and `Test.*x*.stg` underneath [runtime-testsuite/resources/org/antlr/v4/test/runtime/helpers](../runtime-testsuite/resources/org/antlr/v4/test/runtime/helpers).
+   Again, your best bet is to copy the templates from the closest language to your target and tweak it to suit.
+6. Create test files under [/runtime-testsuite/test/org/antlr/v4/test/runtime](../runtime-testsuite/test/org/antlr/v4/test/runtime).
+   They will load defined test cases in each test descriptor.
+   Also add the `/runtime-testsuite/test/org/antlr/v4/test/runtime/X/BaseXTest.java` which defines how test cases will execute and output.
+7. Create/edit shell scripts in [/.github](../.github) and [/.circleci](../.circleci) to run tests in CI pipelines.
 
 ## Getting started
 
-1. Fork the `antlr/antlr4` repository at github to your own user so that you have repository `username/antlr4`.
-2. Clone `username/antlr4`, the forked repository, to your local disk.  Your remote `origin` will be the forked repository on GitHub. Add a remote `upstream` to the original `antlr/antlr4` repository (URL `https://github.com/antlr/antlr4.git`). Changes that you would like to contribute back to the project are done with [pull requests](https://help.github.com/articles/using-pull-requests/).
+1. Fork the `antlr/antlr4` repository at GitHub to your own user so that you have repository `username/antlr4`.
+2. Clone `username/antlr4`, the forked repository, to your local disk.
+   Your remote `origin` will be the forked repository on GitHub.
+   Add a remote `upstream` to the original `antlr/antlr4` repository (URL `https://github.com/antlr/antlr4.git`).
+   Changes that you would like to contribute back to the project are done with [pull requests](https://help.github.com/articles/using-pull-requests/).
 3. Try to build it before doing anything
+
 ```bash
 $ mvn compile
 ```
+
 That should proceed with success. See [Building ANTLR](building-antlr.md) for more details.

runtime-testsuite/pom.xml

@@ -30,7 +30,7 @@
 		<dependency>
 			<groupId>org.antlr</groupId>
 			<artifactId>ST4</artifactId>
-			<version>4.3.1</version>
+			<version>4.3.3</version>
 			<scope>test</scope>
 		</dependency>
 		<dependency>
@@ -72,19 +72,6 @@
 
 	<build>
 		<testSourceDirectory>test</testSourceDirectory>
-		<resources>
-			<resource>
-				<directory>resources</directory>
-			</resource>
-			<resource>
-				<directory>../runtime</directory>
-				<excludes>
-					<exclude>**/.build/**</exclude>
-					<exclude>**/target/**</exclude>
-					<exclude>Swift/*.xcodeproj/**</exclude>
-				</excludes>
-			</resource>
-		</resources>
 		<plugins>
 			<plugin>
 				<groupId>org.apache.maven.plugins</groupId>
@@ -93,18 +80,6 @@
 				<configuration>
 					<!-- SUREFIRE-951: file.encoding cannot be set via systemPropertyVariables -->
 					<argLine>-Dfile.encoding=UTF-8</argLine>
-					<includes>
-                        <include>**/csharp/Test*.java</include>
-                        <include>**/java/Test*.java</include>
-                        <include>**/java/api/Test*.java</include>
-                        <include>**/go/Test*.java</include>
-                        <include>**/javascript/Test*.java</include>
-                        <include>**/python2/Test*.java</include>
-                        <include>**/python3/Test*.java</include>
-                        <include>**/php/Test*.java</include>
-                        <include>**/dart/Test*.java</include>
-                        <include>${antlr.tests.swift}</include>
-                    </includes>
                 </configuration>
 			</plugin>
 			<plugin>

runtime-testsuite/resources/org/antlr/v4/test/runtime/helpers/RuntimeTestLexer.java

@@ -0,0 +1,10 @@
+import org.antlr.v4.runtime.CharStream;
+import org.antlr.v4.runtime.Lexer;
+
+public abstract class RuntimeTestLexer extends Lexer {
+	protected java.io.PrintStream outStream = System.out;
+
+	public RuntimeTestLexer(CharStream input) { super(input); }
+
+	public void setOutStream(java.io.PrintStream outStream) { this.outStream = outStream; }
+}

runtime-testsuite/resources/org/antlr/v4/test/runtime/helpers/RuntimeTestParser.java

@@ -0,0 +1,14 @@
+import org.antlr.v4.runtime.Parser;
+import org.antlr.v4.runtime.TokenStream;
+
+public abstract class RuntimeTestParser extends Parser {
+	protected java.io.PrintStream outStream = System.out;
+
+	public RuntimeTestParser(TokenStream input) {
+		super(input);
+	}
+
+	public void setOutStream(java.io.PrintStream outStream) {
+		this.outStream = outStream;
+	}
+}

runtime-testsuite/resources/org/antlr/v4/test/runtime/helpers/Test.java.stg

@@ -1,20 +1,34 @@
 import org.antlr.v4.runtime.*;
 import org.antlr.v4.runtime.tree.*;
 import org.antlr.v4.runtime.atn.*;
+
+import java.io.IOException;
+import java.io.PrintStream;
 import java.nio.file.Paths;
 import java.util.Arrays;
 
 public class Test {
 	public static void main(String[] args) throws Exception {
+		recognize(args[0], System.out, System.err);
+	}
+
+	public static void recognize(String inputFile, PrintStream outStream, PrintStream errorStream) throws IOException {
+		CustomStreamErrorListener errorListener = new CustomStreamErrorListener(errorStream);
 <if(lexerName)>
-		CharStream input = CharStreams.fromPath(Paths.get(args[0]));
+		CharStream input = CharStreams.fromPath(Paths.get(inputFile));
 		<lexerName> lexer = new <lexerName>(input);
+		lexer.setOutStream(outStream);
+		lexer.removeErrorListeners();
+		lexer.addErrorListener(errorListener);
 		CommonTokenStream tokens = new CommonTokenStream(lexer);
 <else>
 		CommonTokenStream tokens = null; // It's required for compilation
 <endif>
 <if(parserName)>
 		<parserName> parser = new <parserName>(tokens);
+		parser.setOutStream(outStream);
+		parser.removeErrorListeners();
+		parser.addErrorListener(errorListener);
 <if(debug)>
 		parser.addErrorListener(new DiagnosticErrorListener());
 <endif>
@@ -25,18 +39,36 @@ public class Test {
 <endif>
 		ParserRuleContext tree = parser.<parserStartRuleName>();
 <if(profile)>
-		System.out.println(Arrays.toString(profiler.getDecisionInfo()));
+		outStream.println(Arrays.toString(profiler.getDecisionInfo()));
 <endif>
 		ParseTreeWalker.DEFAULT.walk(new TreeShapeListener(), tree);
 <else>
 		tokens.fill();
-		for (Object t : tokens.getTokens()) System.out.println(t);
+		for (Object t : tokens.getTokens()) outStream.println(t);
 <if(showDFA)>
-		System.out.print(lexer.getInterpreter().getDFA(Lexer.DEFAULT_MODE).toLexerString());
+		outStream.print(lexer.getInterpreter().getDFA(Lexer.DEFAULT_MODE).toLexerString());
 <endif>
 <endif>
 	}
 
+	static class CustomStreamErrorListener extends BaseErrorListener {
+		private final PrintStream printStream;
+
+		public CustomStreamErrorListener(PrintStream printStream){
+			this.printStream = printStream;
+		}
+
+		@Override
+		public void syntaxError(Recognizer\<?, ?> recognizer,
+								Object offendingSymbol,
+								int line,
+								int charPositionInLine,
+								String msg,
+								RecognitionException e) {
+			printStream.println("line " + line + ":" + charPositionInLine + " " + msg);
+		}
+	}
+
 <if(parserName)>
 	static class TreeShapeListener implements ParseTreeListener {
 		@Override public void visitTerminal(TerminalNode node) { }

runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/Java.test.stg

@@ -1,6 +1,6 @@
-writeln(s) ::= <<System.out.println(<s>);>>
-write(s) ::= <<System.out.print(<s>);>>
-writeList(s) ::= <<System.out.println(<s; separator="+">);>>
+writeln(s) ::= <<outStream.println(<s>);>>
+write(s) ::= <<outStream.print(<s>);>>
+writeList(s) ::= <<outStream.println(<s; separator="+">);>>
 
 False() ::= "false"
 
@@ -44,7 +44,7 @@ ModMemberEquals(n,m,v) ::= <%this.<n> % <m> == <v>%>
 
 ModMemberNotEquals(n,m,v) ::= <%this.<n> % <m> != <v>%>
 
-DumpDFA() ::= "this.dumpDFA();"
+DumpDFA() ::= "this.dumpDFA(outStream);"
 
 Pass() ::= ""
 
@@ -186,9 +186,9 @@ protected static class PositionAdjustingLexerATNSimulator extends LexerATNSimula
 
 BasicListener(X) ::= <<
 @parser::members {
-public static class LeafListener extends TBaseListener {
+public class LeafListener extends TBaseListener {
 	public void visitTerminal(TerminalNode node) {
-		System.out.println(node.getSymbol().getText());
+		outStream.println(node.getSymbol().getText());
 	}
 }
 }
@@ -214,27 +214,27 @@ public static class MyRuleNode extends ParserRuleContext {
 
 TokenGetterListener(X) ::= <<
 @parser::members {
-public static class LeafListener extends TBaseListener {
+public class LeafListener extends TBaseListener {
 	public void exitA(TParser.AContext ctx) {
 		if (ctx.getChildCount()==2)
-			System.out.printf("%s %s %s\n",ctx.INT(0).getSymbol().getText(),
+			outStream.printf("%s %s %s\n",ctx.INT(0).getSymbol().getText(),
 				ctx.INT(1).getSymbol().getText(),ctx.INT());
 		else
-			System.out.println(ctx.ID().getSymbol());
+			outStream.println(ctx.ID().getSymbol());
 	}
 }
 }
 >>
 
 RuleGetterListener(X) ::= <<
 @parser::members {
-public static class LeafListener extends TBaseListener {
+public class LeafListener extends TBaseListener {
 	public void exitA(TParser.AContext ctx) {
 		if (ctx.getChildCount()==2) {
-			System.out.printf("%s %s %s\n",ctx.b(0).start.getText(),
+			outStream.printf("%s %s %s\n",ctx.b(0).start.getText(),
 				ctx.b(1).start.getText(),ctx.b().get(0).start.getText());
 		} else
-			System.out.println(ctx.b(0).start.getText());
+			outStream.println(ctx.b(0).start.getText());
 	}
 }
 }
@@ -243,26 +243,26 @@ public static class LeafListener extends TBaseListener {
 
 LRListener(X) ::= <<
 @parser::members {
-public static class LeafListener extends TBaseListener {
+public class LeafListener extends TBaseListener {
 	public void exitE(TParser.EContext ctx) {
 		if (ctx.getChildCount()==3) {
-			System.out.printf("%s %s %s\n",ctx.e(0).start.getText(),
+			outStream.printf("%s %s %s\n",ctx.e(0).start.getText(),
 				ctx.e(1).start.getText(), ctx.e().get(0).start.getText());
 		} else
-			System.out.println(ctx.INT().getSymbol().getText());
+			outStream.println(ctx.INT().getSymbol().getText());
 	}
 }
 }
 >>
 
 LRWithLabelsListener(X) ::= <<
 @parser::members {
-public static class LeafListener extends TBaseListener {
+public class LeafListener extends TBaseListener {
 	public void exitCall(TParser.CallContext ctx) {
-		System.out.printf("%s %s\n",ctx.e().start.getText(),ctx.eList());
+		outStream.printf("%s %s\n",ctx.e().start.getText(),ctx.eList());
 	}
 	public void exitInt(TParser.IntContext ctx) {
-		System.out.println(ctx.INT().getSymbol().getText());
+		outStream.println(ctx.INT().getSymbol().getText());
 	}
 }
 }
@@ -277,13 +277,13 @@ void foo() {
 >>
 
 Declare_foo() ::= <<
-	public void foo() {System.out.println("foo");}
+	public void foo() {outStream.println("foo");}
 >>
 
 Invoke_foo() ::= "foo();"
 
 Declare_pred() ::= <<boolean pred(boolean v) {
-	System.out.println("eval="+v);
+	outStream.println("eval="+v);
 	return v;
 }
 >>

runtime-testsuite/test/org/antlr/v4/test/runtime/CustomDescriptors.java

@@ -6,12 +6,18 @@
 
 package org.antlr.v4.test.runtime;
 
+import java.net.URI;
+import java.nio.file.Paths;
 import java.util.*;
 
 public class CustomDescriptors {
-	public static HashMap<String, RuntimeTestDescriptor[]> descriptors;
+	public final static HashMap<String, RuntimeTestDescriptor[]> descriptors;
+	private final static URI uri;
 
 	static {
+		uri = Paths.get(RuntimeTestUtils.runtimeTestsuitePath.toString(),
+						"test", "org", "antlr", "v4", "test", "runtime", "CustomDescriptors.java").toUri();
+
 		descriptors = new HashMap<>();
 		descriptors.put("LexerExec",
 				new RuntimeTestDescriptor[]{
@@ -40,7 +46,7 @@ private static RuntimeTestDescriptor getLineSeparatorLfDescriptor() {
 				"lexer grammar L;\n" +
 						"T: ~'\\n'+;\n" +
 						"SEPARATOR: '\\n';",
-				null, false, false, null);
+				null, false, false, null, uri);
 	}
 
 	private static RuntimeTestDescriptor getLineSeparatorCrLfDescriptor() {
@@ -61,7 +67,7 @@ private static RuntimeTestDescriptor getLineSeparatorCrLfDescriptor() {
 				"lexer grammar L;\n" +
 						"T: ~'\\r'+;\n" +
 						"SEPARATOR: '\\r\\n';",
-				null, false, false, null);
+				null, false, false, null, uri);
 	}
 
 	private static RuntimeTestDescriptor getLargeLexerDescriptor() {
@@ -88,7 +94,7 @@ private static RuntimeTestDescriptor getLargeLexerDescriptor() {
 				"",
 				grammarName,
 				grammar.toString(),
-				null, false, false, null);
+				null, false, false, null, uri);
 	}
 
 	private static RuntimeTestDescriptor getAtnStatesSizeMoreThan65535Descriptor() {
@@ -138,6 +144,6 @@ private static RuntimeTestDescriptor getAtnStatesSizeMoreThan65535Descriptor() {
 				grammarName,
 				grammar.toString(),
 				null, false, false,
-				new String[] {"CSharp", "Python2", "Python3", "Go", "PHP", "Swift", "JavaScript", "Dart"});
+				new String[] {"CSharp", "Python2", "Python3", "Go", "PHP", "Swift", "JavaScript", "Dart"}, uri);
 	}
 }