Merge https://github.com/influxdata/influxdb-java

fix compilation errors
influxdata · Jun 1, 2018 · f8cad96 · f8cad96
1 parent 2887f8f
commit f8cad96
Show file tree

Hide file tree

Showing 32 changed files with 3,595 additions and 875 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -1,16 +1,18 @@
-language: java
 sudo: required
 
-jdk:
-  - oraclejdk8
-
-addons:
-  apt:
-    packages:
-      - oracle-java8-installer # Updates JDK 8 to the latest available.
-
 services:
   - docker
+
+# We test against all influxdb versions with the most actual JDK.
+# Test only the most recent influxdb version with JDK8 which will be legacy soon.
+env:
+  - MAVEN_JAVA_VERSION=3-jdk-10-slim INFLUXDB_VERSION=1.5
+  - MAVEN_JAVA_VERSION=3-jdk-10-slim INFLUXDB_VERSION=1.4
+  - MAVEN_JAVA_VERSION=3-jdk-10-slim INFLUXDB_VERSION=1.3
+  - MAVEN_JAVA_VERSION=3-jdk-10-slim INFLUXDB_VERSION=1.2
+  - MAVEN_JAVA_VERSION=3-jdk-10-slim INFLUXDB_VERSION=1.1
+  - MAVEN_JAVA_VERSION=3-jdk-8-slim INFLUXDB_VERSION=1.5
+
 script: ./compile-and-test.sh
 
 after_success:

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,7 +1,37 @@
 # Changelog
 
-## 2.9 [unreleased]
+## 2.11 [unreleased]
 
+### Features
+
+- Allow write precision of TimeUnit other than Nanoseconds [PR #321](https://github.com/influxdata/influxdb-java/pull/321)
+- Support dynamic measurement name in InfluxDBResultMapper [PR #423](https://github.com/influxdata/influxdb-java/pull/423)
+
+## 2.10 [2018-04-26]
+
+### Fixes
+- Fix IllegalAccessException on setting value to POJOs, InfluxDBResultMapper is now more thread-safe [PR #432](https://github.com/influxdata/influxdb-java/pull/432)
+
+### Features
+
+- Support for parameter binding in queries ("prepared statements") [PR #429](https://github.com/influxdata/influxdb-java/pull/429)
+- Allow to figure out whether the Point.Builder has any field or not [PR #434](https://github.com/influxdata/influxdb-java/pull/434)
+
+### Improvements
+
+- Performance: use chained StringBuilder calls instead of single calls [PR #426](https://github.com/influxdata/influxdb-java/pull/426)
+- Performance: Escape fields and keys more efficiently [PR #424](https://github.com/influxdata/influxdb-java/pull/424)
+- Build: Speed up travis build [PR #435](https://github.com/influxdata/influxdb-java/pull/435)
+- Test: Update junit from 5.1.0 to 5.1.1 [PR #441](https://github.com/influxdata/influxdb-java/pull/441)
+
+## 2.9 [2018-02-27]
+
+### Features
+
+- New extensible API to configure batching properties.  [PR #409] 
+- New configuration property 'jitter interval' to avoid multiple clients hit the server periodically at the same time. [PR #409] 
+- New strategy on handling errors, client performs retries writes when server gets overloaded [PR #410]
+- New exceptions give the client user easier way to classify errors reported by the server. [PR #410] 
 
 ## 2.8 [2017-12-06]
 

diff --git a/README.md b/README.md
@@ -12,48 +12,77 @@ To connect to InfluxDB 0.8.x you need to use influxdb-java version 1.6.
 This implementation is meant as a Java rewrite of the influxdb-go package.
 All low level REST Api calls are available.
 
-## Usages
+## Usage
 
-### Basic Usages:
+### Basic Usage:
+
+This is a recommended approach to write data points into InfluxDB. The influxdb-java 
+client is storing your writes into an internal buffer and flushes them asynchronously 
+to InfluxDB at a fixed flush interval to achieve good performance on both client and 
+server side. This requires influxdb-java v2.7 or newer.
+
+If you want to write data points immediately into InfluxDB and synchronously process
+resulting errors see [this section.](#synchronous-writes)
 
 ```java
 InfluxDB influxDB = InfluxDBFactory.connect("http://172.17.0.2:8086", "root", "root");
 String dbName = "aTimeSeries";
 influxDB.createDatabase(dbName);
+influxDB.setDatabase(dbName);
 String rpName = "aRetentionPolicy";
 influxDB.createRetentionPolicy(rpName, dbName, "30d", "30m", 2, true);
+influxDB.setRetentionPolicy(rpName);
+
+influxDB.enableBatch(BatchOptions.DEFAULTS);
+
+influxDB.write(Point.measurement("cpu")
+	.time(System.currentTimeMillis(), TimeUnit.MILLISECONDS)
+	.addField("idle", 90L)
+	.addField("user", 9L)
+	.addField("system", 1L)
+	.build());
+
+influxDB.write(Point.measurement("disk")
+	.time(System.currentTimeMillis(), TimeUnit.MILLISECONDS)
+	.addField("used", 80L)
+	.addField("free", 1L)
+	.build());
 
-BatchPoints batchPoints = BatchPoints
-				.database(dbName)
-				.tag("async", "true")
-				.retentionPolicy(rpName)
-				.consistency(ConsistencyLevel.ALL)
-				.build();
-Point point1 = Point.measurement("cpu")
-					.time(System.currentTimeMillis(), TimeUnit.MILLISECONDS)
-					.addField("idle", 90L)
-					.addField("user", 9L)
-					.addField("system", 1L)
-					.build();
-Point point2 = Point.measurement("disk")
-					.time(System.currentTimeMillis(), TimeUnit.MILLISECONDS)
-					.addField("used", 80L)
-					.addField("free", 1L)
-					.build();
-batchPoints.point(point1);
-batchPoints.point(point2);
-influxDB.write(batchPoints);
 Query query = new Query("SELECT idle FROM cpu", dbName);
 influxDB.query(query);
 influxDB.dropRetentionPolicy(rpName, dbName);
 influxDB.deleteDatabase(dbName);
+influxDB.close();
+```
+
+
+Any errors that happen during the batch flush won't leak into the caller of the `write` method. By default, any kind of errors will be just logged with "SEVERE" level.
+If you need to be notified and do some custom logic when such asynchronous errors happen, you can add an error handler with a `BiConsumer<Iterable<Point>, Throwable>` using the overloaded `enableBatch` method:
+
+```java
+influxDB.enableBatch(BatchOptions.DEFAULTS.exceptionHandler(
+        (failedPoints, throwable) -> { /* custom error handling here */ })
+);
 ```
+
+With batching enabled the client provides two strategies how to deal with errors thrown by the InfluxDB server. 
+
+   1. 'One shot' write - on failed write request to InfluxDB server an error is reported to the client using the means mentioned above.        
+   2. 'Retry on error' write (used by default) - on failed write the request by the client is repeated after batchInterval elapses 
+       (if there is a chance the write will succeed - the error was caused by overloading the server, a network error etc.) 
+       When new data points are written before the previous (failed) points are successfully written, those are queued inside the client 
+       and wait until older data points are successfully written. 
+       Size of this queue is limited and configured by `BatchOptions.bufferLimit` property. When the limit is reached, the oldest points
+       in the queue are dropped. 'Retry on error' strategy is used when individual write batch size defined by `BatchOptions.actions` is lower than `BatchOptions.bufferLimit`.
+
 Note:
+* Batching functionality creates an internal thread pool that needs to be shutdown explicitly as part of a graceful application shut-down, or the application will not shut down properly. To do so simply call: ```influxDB.close()```
+* `InfluxDB.enableBatch(BatchOptions)` is available since version 2.9. Prior versions use `InfluxDB.enableBatch(actions, flushInterval, timeUnit)` or similar based on the configuration parameters you want to set. 
 * APIs to create and drop retention policies are supported only in versions > 2.7
 * If you are using influxdb < 2.8, you should use retention policy: 'autogen'
 * If you are using influxdb < 1.0.0, you should use 'default' instead of 'autogen'
 
-If your application produces only single Points, you can enable the batching functionality of influxdb-java:
+If your points are written into different databases and retention policies, the more complex InfluxDB.write() methods can be used:
 
 ```java
 InfluxDB influxDB = InfluxDBFactory.connect("http://172.17.0.2:8086", "root", "root");
@@ -63,7 +92,7 @@ String rpName = "aRetentionPolicy";
 influxDB.createRetentionPolicy(rpName, dbName, "30d", "30m", 2, true);
 
 // Flush every 2000 Points, at least every 100ms
-influxDB.enableBatch(2000, 100, TimeUnit.MILLISECONDS);
+influxDB.enableBatch(BatchOptions.DEFAULTS.actions(2000).flushDuration(100));
 
 Point point1 = Point.measurement("cpu")
 					.time(System.currentTimeMillis(), TimeUnit.MILLISECONDS)
@@ -83,53 +112,48 @@ Query query = new Query("SELECT idle FROM cpu", dbName);
 influxDB.query(query);
 influxDB.dropRetentionPolicy(rpName, dbName);
 influxDB.deleteDatabase(dbName);
+influxDB.close();
 ```
-Note that the batching functionality creates an internal thread pool that needs to be shutdown explicitly as part of a graceful application shut-down, or the application will not shut down properly. To do so simply call: ```influxDB.close()```
+ 
 
-If all of your points are written to the same database and retention policy, the simpler write() methods can be used.
-This requires influxdb-java v2.7 or newer.
+#### Synchronous writes
+
+If you want to write the data points immediately to InfluxDB (and handle the errors as well) without any delays see the following example: 
 
 ```java
 InfluxDB influxDB = InfluxDBFactory.connect("http://172.17.0.2:8086", "root", "root");
 String dbName = "aTimeSeries";
 influxDB.createDatabase(dbName);
-influxDB.setDatabase(dbName);
 String rpName = "aRetentionPolicy";
 influxDB.createRetentionPolicy(rpName, dbName, "30d", "30m", 2, true);
-influxDB.setRetentionPolicy(rpName);
-
-// Flush every 2000 Points, at least every 100ms
-influxDB.enableBatch(2000, 100, TimeUnit.MILLISECONDS);
-
-influxDB.write(Point.measurement("cpu")
-	.time(System.currentTimeMillis(), TimeUnit.MILLISECONDS)
-	.addField("idle", 90L)
-	.addField("user", 9L)
-	.addField("system", 1L)
-	.build());
-
-influxDB.write(Point.measurement("disk")
-	.time(System.currentTimeMillis(), TimeUnit.MILLISECONDS)
-	.addField("used", 80L)
-	.addField("free", 1L)
-	.build());
 
+BatchPoints batchPoints = BatchPoints
+				.database(dbName)
+				.tag("async", "true")
+				.retentionPolicy(rpName)
+				.consistency(ConsistencyLevel.ALL)
+				.build();
+Point point1 = Point.measurement("cpu")
+					.time(System.currentTimeMillis(), TimeUnit.MILLISECONDS)
+					.addField("idle", 90L)
+					.addField("user", 9L)
+					.addField("system", 1L)
+					.build();
+Point point2 = Point.measurement("disk")
+					.time(System.currentTimeMillis(), TimeUnit.MILLISECONDS)
+					.addField("used", 80L)
+					.addField("free", 1L)
+					.build();
+batchPoints.point(point1);
+batchPoints.point(point2);
+influxDB.write(batchPoints);
 Query query = new Query("SELECT idle FROM cpu", dbName);
 influxDB.query(query);
 influxDB.dropRetentionPolicy(rpName, dbName);
 influxDB.deleteDatabase(dbName);
 ```
 
-Also note that any errors that happen during the batch flush won't leak into the caller of the `write` method. By default, any kind of errors will be just logged with "SEVERE" level.
-
-If you need to be notified and do some custom logic when such asynchronous errors happen, you can add an error handler with a `BiConsumer<Iterable<Point>, Throwable>` using the overloaded `enableBatch` method:
-
-```java
-// Flush every 2000 Points, at least every 100ms
-influxDB.enableBatch(2000, 100, TimeUnit.MILLISECONDS, Executors.defaultThreadFactory(), (failedPoints, throwable) -> { /* custom error handling here */ });
-```
-
-### Advanced Usages:
+### Advanced Usage:
 
 #### Gzip's support (version 2.5+ required):
 
@@ -157,7 +181,6 @@ Query query = new Query("SELECT idle FROM cpu", dbName);
 influxDB.query(query, 20, queryResult -> System.out.println(queryResult));
 ```
 
-
 #### QueryResult mapper to POJO (version 2.7+ required):
 
 An alternative way to handle the QueryResult object is now available.
@@ -240,6 +263,36 @@ this.influxDB.query(new Query("SELECT idle FROM cpu", dbName), queryResult -> {
 });
 ```
 
+#### Query using parameter binding ("prepared statements", version 2.10+ required)
+
+If your Query is based on user input, it is good practice to use parameter binding to avoid [injection attacks](https://en.wikipedia.org/wiki/SQL_injection).
+You can create queries with parameter binding with the help of the QueryBuilder:
+
+```java
+Query query = QueryBuilder.newQuery("SELECT * FROM cpu WHERE idle > $idle AND system > $system") 
+        .forDatabase(dbName)
+        .bind("idle", 90)
+        .bind("system", 5)
+        .create();
+QueryResult results = influxDB.query(query);
+```
+
+The values of the bind() calls are bound to the placeholders in the query ($idle, $system). 
+
+#### Batch flush interval jittering (version 2.9+ required)
+
+When using large number of influxdb-java clients against a single server it may happen that all the clients 
+will submit their buffered points at the same time and possibly overloading the server. This is usually happening
+when all the clients are started at once - for instance as members of cloud hosted large cluster networks.  
+If all the clients have the same flushDuration set this situation will repeat periodically.
+
+To solve this situation the influxdb-java offers an option to offset the flushDuration by a random interval so that 
+the clients will flush their buffers in different intervals:    
+
+```java
+influxDB.enableBatch(BatchOptions.DEFAULTS.jitterDuration(500);
+```
+
 ### Other Usages:
 For additional usage examples have a look at [InfluxDBTest.java](https://github.com/influxdb/influxdb-java/blob/master/src/test/java/org/influxdb/InfluxDBTest.java "InfluxDBTest.java")
 
@@ -250,12 +303,12 @@ The latest version for maven dependence:
 <dependency>
   <groupId>org.influxdb</groupId>
   <artifactId>influxdb-java</artifactId>
-  <version>2.8</version>
+  <version>2.10</version>
 </dependency>
 ```
 Or when using with gradle:
 ```groovy
-compile 'org.influxdb:influxdb-java:2.8'
+compile 'org.influxdb:influxdb-java:2.10'
 ```
 For version change history have a look at [ChangeLog](https://github.com/influxdata/influxdb-java/blob/master/CHANGELOG.md).
 

diff --git a/checkstyle.xml b/checkstyle.xml
@@ -29,7 +29,6 @@
 
   <module name="TreeWalker">
     <property name="cacheFile" value="${checkstyle.cache.file}"/>
-
     <!-- Checks for Javadoc comments.                     -->
     <!-- See http://checkstyle.sf.net/config_javadoc.html -->
     <!--module name="JavadocMethod"/-->