Merge branch 'release-v1.8.1'

.github/ISSUE_TEMPLATE/bug_report.md

@@ -6,8 +6,8 @@ about: Report a bug
 
 <!--- Remove text and sections that do not apply -->
 The issue tracker is only for reporting bugs or submitting feature requests.
-If you need technical assistance for running a node please consult the #fullnode channel on Discord (https://discord.gg/jrxApWC) or https://forum.helloiota.com/Technology/Help.
-If you have general questions on IOTA you can go to https://iota.stackexchange.com/, https://helloiota.com/, or browse Discord channels (https://discord.gg/C88Wexg).
+If you need technical assistance for running a node please consult the #fullnode channel on Discord (https://discord.iota.org) or https://forum.helloiota.com/Technology/Help.
+If you have general questions on IOTA you can go to https://iota.stackexchange.com/, https://helloiota.com/, or browse Discord channels (https://discord.iota.org).
 
 ### Bug description
 A general description of the bug.

.gitignore

@@ -21,7 +21,9 @@ bin/
 *.jar
 src/main/ixi/
 dependency-reduced-pom.xml
-
+.checkstyle
+db-bench/
+db-log-bench/
 
 # Created by https://www.gitignore.io/api/osx,linux,windows
 

.travis.yml

@@ -54,20 +54,6 @@ after_success:
   - test $TRAVIS_PULL_REQUEST = "false" && test $TRAVIS_JDK_VERSION = "oraclejdk8" && wget -O codacy-coverage-reporter-assembly-latest.jar $(curl https://api.github.com/repos/codacy/codacy-coverage-reporter/releases/latest | jq -r '.assets[0].browser_download_url')
   - test $TRAVIS_PULL_REQUEST = "false" && test $TRAVIS_JDK_VERSION = "oraclejdk8" && java -jar codacy-coverage-reporter-assembly-latest.jar report -l Java -r target/site/jacoco/jacoco.xml
 
-deploy:
-- provider: releases
-  api_key:
-    secure: "j+KolvRrsinh6OTqjli1y5I4gdPdD5MT6RFahVRrznxtcb2t7y6x41PknkI/8BksxLJKKQ1f40L307Aquzih9FmLesWcrYrCOS/B40dlCc3uV8DEZKhZF+QWqvsDpZ0EfiluGFljqGpxwngb/miNObqQzQvzpdUXBpZcuzxgqziynJD6jt3TYbNn544m/i4cWbb++b1h2Bm+jwELzJcf6/CZ1TxUh49vdM4S6yTpvnKwg4NLeH6g8s9oYEN1iwYtQokGKlH/emBZDsqDtF4KH7kXPpEKTcho4nH504XWJpanVAQ02Pha4SAnyVu4rKiLH4e6v99xZOBLVTH/qkvlwzvkNYr2xS5q9T9G7DZBRE+oc2MIbcnuSkhHwrUcmf2Hc0zCIP4RLCXAUuVlNEmIQUKgTezVzaqaEQLsa+EFIoosS+8pT/kVnaLbwxrITUmgM246+gvzMcVg8SmyPvVj6iRxfR6spTd7V2M0ePdqWlCL8P9RY4YrtxR2yJ6/3C0XNf29t+s/r1jat8fwWp8Gf7GiCaJEYG9VO24Chhi+MZCKBJnmx5G/98zvLw3f6a1FsEGZrw9RhF2vJKK6sD3rIvNkUsYmf0OhM7bf0NceFMGTJ/mFzM/JX0OHMtIyAvUfyyPVbQkIHNcGBgQMZasx07K3fjcn2lqv4FbHVi3HbdM="
-  file_glob: true
-  file: target/*.jar*
-  skip_cleanup: true
-  before_deploy: openssl aes-256-cbc -K $encrypted_5a15fa813cca_key -iv $encrypted_5a15fa813cca_iv -in codesigning.asc.enc -out codesigning.asc -d && gpg --fast-import codesigning.asc
-  deploy: mvn package -P build-extras -Dlogging-level=INFO
-  on:
-    tags: true
-    repo: iotaledger/iri
-    branch:
-    - master
 env:
   global:
   - secure: oJutGSe5KK3ymwSvohFZ89UL7XaNlXH6fdO2nBEYbk5KyYuQ1Wm+h6+N/05EbwxwDK9dEiF73ZJh76sZ3IWKjRC8VWjUAz+8p38hGvQwByrVsJkrbs5SA4wA4XnyvEqyMV3yqbcgRNbMlIvj7ubUbxMizz7MA7plnumrnx9ejPG1WP4rNgzUA3XcBWHdITyOSg9rSMBJzMZ9CN1Dv+1SxZV+GlkyWN8n1Cshkq2TnNl1oRZqGKs7O0ma1dN0hUKtBa80AsCTjH8ojZFJU92gmkvAWDw5BTSZ8mhX9sZcbBYT8SnQxNDrf51HJ+K7hvdpFbsy5Z0g24cNJkA8lfUnzXFznZ/PH4hm8MeVWjRiWdE4VZr47iYzJcs0brOHSeab6fBoeTPS5899Q3V+yAw6CahfvOBthuF3Ymy+fZXoDO5242PSWXXsGwFqK7r/gT0iwJZxMBI/UUWsP/s7J/CWux8ftwbOdSbs4XdQZsuqxzKEKDDwU0FTytantsMS/vcgI41K5azyY4tbkixzydaBPvhimkflaWcuZb//mXcntitgyenaSwfkLX5pD5l1srVzSliyabnH7AprLOm1B9DBfk2leI2XgGJg/Q/GrVtcjcl1FU0aXBFhLrp3BkK7chlG2v/Eacr8Pb6NOQHORyq64n2prF6zvExX0aadOddefoA=

DOCKER.md

@@ -2,13 +2,17 @@
 
 Run the official iotaledger/iri container, passing the mandatory -p option:
 
-```docker run iotaledger/iri:vX.X.X-RELEASE -p 14265```
+```sh
+docker run iotaledger/iri:vX.X.X-RELEASE -p 14265
+```
 
-This will get your a running IRI with its API listening on port 14265, no neighbours and an empty database. The IRI Docker container by default expects data at /iri/data. Use the `-v` option of the `docker run` command to mount volumes so to have persistent data. You can also pass more command line options to the docker run command and those will be passed to IRI.
+This will get your a running IRI with its API listening on port 14265, no neighbours and an empty database. The IRI Docker container by default expects data at `/iri/data`. Use the `-v` option of the `docker run` command to mount volumes so to have persistent data. You can also pass more command line options to the docker run command and those will be passed to IRI.
 
 If you want to use a iri.ini file with the docker container, supposing it's stored under /path/to/conf/iri.ini on your docker host, then pass `-v /path/to/conf:/iri/conf` and add -c /iri/conf/iri.ini as docker run arguments. So for example the `docker run` command above would become:
 
-```docker run -v /path/to/conf:/iri/conf -v /path/to/data:/iri/data iotaledger/iri:vX.X.X-RELEASE -p 14265 -c /iri/conf/iri.ini```
+```sh
+docker run -v /path/to/conf:/iri/conf -v /path/to/data:/iri/data iotaledger/iri:vX.X.X-RELEASE -p 14265 -c /iri/conf/iri.ini
+```
 
 Please refer to the IRI documentation for further command line options and iri.ini options.
 
@@ -23,15 +27,29 @@ When building IRI via the Dockerfile provided, Docker 17.05 minimum is required,
 
 The built container assumes the WORKDIR inside the container is /iri/data: this means that the database directory will be written inside that directory by default. If a system administrator wants to retain the database across restarts, it is his/her job to mount a docker volume in the right folder.
 
-The docker conatiner supports the env variables to configure advanced options. These variables can be set but are not required to run IRI.
+### Advanced Configuration
+
+The docker container supports the env variables to configure advanced options. These variables can be set but are not required to run IRI.
+
+`JAVA_OPTIONS` these are the java options to pass right after the java command. It must not contain -Xms nor -Xmx. Defaults to a safe value.
+
+`JAVA_MIN_MEMORY` the value of -Xms option. Defaults to 2G
+
+`JAVA_MAX_MEMORY` the value of -Xmx option. Defaults to 4G
+
+`DOCKER_IRI_JAR_PATH` the directory where the IRI jar file is. Defaults to `/iri/target/` as pushed by the Dockerfile. This is useful if custom IRI binaries want to be executed and the default path needs to be overridden.
+
+`DOCKER_IRI_JAR_FILE` the IRI jar file name to execute. Defaults to `iri*.jar`
 
-`JAVA_OPTIONS`: these are the java options to pass right after the java command. It must not contain -Xms nor -Xmx. Defaults to a safe value
-`JAVA_MIN_MEMORY`: the value of -Xms option. Defaults to 2G
-`JAVA_MAX_MEMORY`: the value of -Xmx option. Defaults to 4G
-`DOCKER_IRI_JAR_PATH`: defaults to /iri/target/iri*.jar as pushed by the Dockerfile. This is useful if custom IRI binaries want to be executed and the default path needs to be overridden
-`DOCKER_IRI_REMOTE_LIMIT_API`: defaults to "interruptAttachToTangle, attachToTangle, addNeighbors, removeNeighbors, getNeighbors"
-`DOCKER_IRI_MONITORING_API_PORT_ENABLE`: defaults to 0. If set to 1, a socat on port 14266 directed to 127.0.0.1:DOCKER_IRI_MONITORING_API_PORT_DESTINATION  will be open in order to allow all API calls regardless of the DOCKER_IRI_REMOTE_LIMIT_API setting. This is useful to give access to restricted API calls to local tools and still denying access to restricted API calls to the internet. It is highly recommended to use this option together with docker networks (docker run --net).
+`DOCKER_IRI_REMOTE_LIMIT_API` defaults to "interruptAttachToTangle, attachToTangle, addNeighbors, removeNeighbors, getNeighbors"
 
+`DOCKER_IRI_MONITORING_API_PORT_ENABLE` defaults to 0. If set to 1, a socat on port 14266 directed to 127.0.0.1:DOCKER_IRI_MONITORING_API_PORT_DESTINATION  will be open in order to allow all API calls regardless of the DOCKER_IRI_REMOTE_LIMIT_API setting. This is useful to give access to restricted API calls to local tools and still denying access to restricted API calls to the internet. It is highly recommended to use this option together with docker networks (docker run --net).
+
+`DOCKER_IRI_REMOTE` defaults to "true". This sets the boolean value for the "--remote" command line option. Setting to "false" ensures that the API port binds only to the localhost interface. The setting of "true" lets the API listen on all the interfaces (0.0.0.0). Set this value to "false" if you plan on running the container on the host network and want the API to bind to the localhost interface only. Set this value to false if you want to bind the --api-host (API_HOST) to a specific interface.
+
+`DOCKER_JAVA_NET_PREFER_IPV4_STACK` defaults to "true". This sets the boolean value for the -Djava.net.preferIPv4Stack option. To be able to use IPv6 make sure to set this to false.
+
+### Entrypoint
 The container entry point is a shell script that performs few additional steps before launching IRI:
 - verifies if `DOCKER_IRI_MONITORING_API_PORT_ENABLE` is set to 1
 - launches IRI with all parameters passed as desired
@@ -40,7 +58,8 @@ It is important to note that other than --remote and --remote-limit-api "$DOCKER
 
 **At the time of writing, IRI requires -p to be passed either via INI or via command line. The entrypoint of this docker container does not do that for you.**
 
-Here is a systemd unit example you can use with this Docker container. This is just an example and customisation is possible and recommended. In this example the docker network iri must be created and the paths /mnt/iri/conf and /mnt/iri/data are used on the docker host to serve respectively the neighbors file and the data directory. No INI files are used in this example, instead options are passed via command line options, such as --testnet and --zmq-enabled.
+### Systemd Unit
+Here is a systemd unit example you can use with this Docker container. This is just an example and customisation is possible and recommended. In this example the paths `/mnt/iri/conf` and `/mnt/iri/data` are used on the docker host to serve respectively the neighbors file and the data directory. No INI files are used in this example, instead options are passed via command line options, such as `--testnet true` and `--zmq-enabled false`.
 
 ```
 [Unit]
@@ -53,22 +72,30 @@ TimeoutStartSec=0
 Restart=always
 ExecStartPre=-/usr/bin/docker rm %n
 ExecStart=/usr/bin/docker run \
---name %n \
---hostname iri \
---net=iri \
--v /mnt/iri/conf:/iri/conf \
--v /mnt/iri/data:/iri/data \
--p 14265:14265 \
--p 15600:15600 \
--p 14600:14600/udp  \
-iotaledger/iri:vX.X.X-RELEASE \
--p 14265 \
---zmq-enabled \
---testnet
+  --name %n \
+  --hostname iri \
+  --user=1000 \
+  --net=host \
+  --cap-drop=ALL
+  -v /mnt/iri/conf:/iri/conf \
+  -v /mnt/iri/data:/iri/data \
+  -e DOCKER_IRI_REMOTE=true \
+  iotaledger/iri:vX.X.X-RELEASE \
+  --port 14265 \
+  --zmq-enabled false \
+  --testnet false
 
 ExecStop=/usr/bin/docker stop %n
 ExecReload=/usr/bin/docker restart %n
 
 [Install]
 WantedBy=multi-user.target
 ```
+
+## Security Considerations
+
+It is highly recommended to run IRI with an unprivileged user and not as user `root`.
+
+An unprivileged user can be created on the host and the UID passed to the docker command (e.g. `--user 1001`). Directories that are mounted to the container from the host should be owned by this user.
+
+In addition the `--cap-drop=ALL` passed to docker restricts process capabilities and adheres to the principle of least privilege. See https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities for more information.

Dockerfile

@@ -10,29 +10,45 @@ RUN mvn clean package
 FROM iotacafe/java:oracle8u181.1.webupd8.1-1@sha256:21b0fb1e5b5be7cd239a742238f346e076a46dc0003670cd50f079780288773f
 
 RUN apt-get update && apt-get install -y --no-install-recommends \
-        jq curl socat \
+        socat \
     && rm -rf /var/lib/apt/lists/*
 
 COPY --from=local_stage_build /iri/target/iri*.jar /iri/target/
 COPY docker/entrypoint.sh /
 
-# Java related options. Defaults set as below
-ENV JAVA_OPTIONS="-XX:+UnlockExperimentalVMOptions -XX:+DisableAttachMechanism -XX:InitiatingHeapOccupancyPercent=60 -XX:G1MaxNewSizePercent=75 -XX:MaxGCPauseMillis=10000 -XX:+UseG1GC"
-ENV JAVA_MIN_MEMORY 2G
-ENV JAVA_MAX_MEMORY 4G
-
-# Additional custom variables. See DOCKER.md for details
-ENV DOCKER_IRI_JAR_PATH "/iri/target/iri*.jar"
-ENV DOCKER_IRI_REMOTE_LIMIT_API "interruptAttachToTangle, attachToTangle, addNeighbors, removeNeighbors, getNeighbors"
-
-# Setting this to 1 will have socat exposing 14266 and pointing it on
-# localhost. See /entrypoint.sh
-# !!! DO NOT DOCKER EXPOSE (-p) 14266 as the remote api settings
-#     will not be applied on that port !!!
-# You also have to maintain $DOCKER_IRI_MONITORING_API_PORT_DESTINATION
-# based on the actual API port exposed via IRI
-ENV DOCKER_IRI_MONITORING_API_PORT_ENABLE 0
-ENV DOCKER_IRI_MONITORING_API_PORT_DESTINATION 14265
+# Default environment variables configuration. See DOCKER.md for details.
+# Override these variables if required (e.g. docker run -e JAVA_OPTIONS="myoptions" ...)
+# `JAVA_OPTIONS`                                 Java related options
+# `JAVA_MIN_MEMORY` and `JAVA_MAX_MEMORY`        Settings for -Xms and -Xmx respectively.
+#                                                See https://docs.oracle.com/cd/E21764_01/web.1111/e13814/jvm_tuning.htm#PERFM161
+# `DOCKER_IRI_JAR_PATH`                          Path where the IRI jar file is located.
+# `DOCKER_IRI_JAR_FILE`                          IRI jar file.
+# `DOCKER_IRI_REMOTE_LIMIT_API`                  Sets the --remote-limit-api options.
+#                                                (Deprecation warning, see https://github.com/iotaledger/iri/issues/1500)
+# `DOCKER_IRI_MONITORING_API_PORT_ENABLE`        When using a docker bridged network setting this to 1 will have
+#                                                socat exposing 14266 and pointing it on localhost. See /entrypoint.sh
+#                                                Do not enable this option when running IRI's container on host network.
+#                                                !!! DO NOT DOCKER EXPOSE (-p) 14266 as the remote api settings will
+#                                                not be applied on that port !!!
+#                                                You also have to maintain $DOCKER_IRI_MONITORING_API_PORT_DESTINATION
+#                                                based on the actual API port exposed via IRI
+# `DOCKER_IRI_MONITORING_API_PORT_DESTINATION`   Set this to the actual IRI API port. This is used to map port 14266.
+# `DOCKER_IRI_REMOTE`                            When using a docker bridged network set this to true. Using host network
+#                                                you may choose to set it to false to make sure the API port listens on
+#                                                localhost only. If you want to bind your API (--api-host) to a specific interface
+#                                                you will have to set this option to false.
+# `DOCKER_JAVA_NET_PREFER_IPV4_STACK`            If set to true will allow usage of IPv4 only. Set to false to be able to use IPv6.
+#                                                See https://docs.oracle.com/javase/7/docs/api/java/net/doc-files/net-properties.html
+ENV JAVA_OPTIONS="-XX:+UnlockExperimentalVMOptions -XX:+DisableAttachMechanism -XX:InitiatingHeapOccupancyPercent=60 -XX:G1MaxNewSizePercent=75 -XX:MaxGCPauseMillis=10000 -XX:+UseG1GC" \
+    JAVA_MIN_MEMORY=2G \
+    JAVA_MAX_MEMORY=4G \
+    DOCKER_IRI_JAR_PATH="/iri/target" \
+    DOCKER_IRI_JAR_FILE="iri*.jar" \
+    DOCKER_IRI_REMOTE_LIMIT_API="interruptAttachToTangle, attachToTangle, addNeighbors, removeNeighbors, getNeighbors" \
+    DOCKER_IRI_MONITORING_API_PORT_ENABLE=0 \
+    DOCKER_IRI_MONITORING_API_PORT_DESTINATION=14265 \
+    DOCKER_IRI_REMOTE=true \
+    DOCKER_JAVA_NET_PREFER_IPV4_STACK=true
 
 WORKDIR /iri/data
 ENTRYPOINT [ "/entrypoint.sh" ]

README.md

@@ -27,7 +27,7 @@ The IOTA network is an independent peer-to-peer network with a first-user, frien
 - As a 'friend-to-friend' network, you have the privilege of joining new users into the network through your node
 by adding them to your approved neighbors list — ensuring that you both broadcast to them and also receive their broadcasts.
 
-You can **find neighbors** on the #nodesharing channel of our [[Discord server]](https://discord.gg/7Gu2mG5).
+You can **find neighbors** on the #nodesharing channel of our [[Discord server]](https://discord.iota.org/).
 
 Everyone will be welcoming and very happy to help you get connected.
 If you want to get tokens for your testcase, please just ask in one of the communication channels.

changelog.txt

@@ -1,3 +1,19 @@
+1.8.1
+
+- Feature: Improved CW Calculation (#1451)
+- Feature: Automatically request trunk and branch transactions when receiving a tx needed to solidify a milestone (#1545)
+- Change: removes tip solidifier code (#1547)
+- Fix: Read from the db in order to find the existence of spent-addresses (#1540)
+- Fix: adds recently requested txs set to TransactionRequester (#1543)
+- Fix: prevents certain HTTP API calls getting executed while the not is not synced (#1533)
+- Fix: Rewrites request queue logic  (#1530)
+- Fix: Up jackson-databind version to 2.9.9.1 (#1534)
+- Change: Fixed pmd test errors (#1526)
+- Change: Docker network option (#1496)
+- Fix: Changed TX requester removal log to Debug (#1515)
+- Change: Update undertow and jackson (#1511)
+- Fix: fixes self init. conn. being removed from reconnect pool too early (#1506)
+
 1.8.0
 
 - Feature: new networking infrastructure, new network rules, and batched BCT curl (#1393)

docker/entrypoint.sh

@@ -5,11 +5,18 @@ if [ "${DOCKER_IRI_MONITORING_API_PORT_ENABLE}" == "1" ]; then
   nohup socat -lm TCP-LISTEN:14266,fork TCP:127.0.0.1:${DOCKER_IRI_MONITORING_API_PORT_DESTINATION} &
 fi
 
+IRI_JAR_FILE=$(find "$DOCKER_IRI_JAR_PATH" -type f -name "$DOCKER_IRI_JAR_FILE" -print -quit)
+if [[ "${IRI_JAR_FILE}x" == "x" ]]
+then
+  >&2 echo "ERROR: File '$DOCKER_IRI_JAR_FILE' not found in path '$DOCKER_IRI_JAR_PATH'"
+  exit 1
+fi
+
 exec java \
   $JAVA_OPTIONS \
   -Xms$JAVA_MIN_MEMORY \
   -Xmx$JAVA_MAX_MEMORY \
-  -Djava.net.preferIPv4Stack=true \
-  -jar $DOCKER_IRI_JAR_PATH \
-  --remote true --remote-limit-api "$DOCKER_IRI_REMOTE_LIMIT_API" \
+  -Djava.net.preferIPv4Stack="$DOCKER_JAVA_NET_PREFER_IPV4_STACK" \
+  -jar "$IRI_JAR_FILE" \
+  --remote "$DOCKER_IRI_REMOTE" --remote-limit-api "$DOCKER_IRI_REMOTE_LIMIT_API" \
   "$@"