Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test dockerization #2234

Merged
merged 2 commits into from
Dec 10, 2018
Merged

Conversation

muros-ct
Copy link
Contributor

@muros-ct muros-ct commented Dec 6, 2018

Added support for integration tests with docker for Kapua deployment

Related Issue
None

Description of the solution adopted
Integration and/or functional tests can now be run either with embedded servers or with servers running in dockers. With servers I mean ES, DB, console, broker and API.

Screenshots
Not applicable.

Any side note on the changes made
Tests with embedded servers are still successfully run on Travis and Jenkins.

@@ -39,7 +39,7 @@ ENV ACTIVEMQ_OPTS "-Dcommons.db.schema.update=true \
-Dcommons.eventbus.url=\${SERVICE_BROKER_ADDR} \
-Dcertificate.jwt.private.key=file:///var/opt/activemq/key.pk8 \
-Dcertificate.jwt.certificate=file:///var/opt/activemq/cert.pem \
-Dbroker.ip=broker"
-Dbroker.ip=localhost"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @muros-ct,

Have you also tried a Docker Compose deployment with this setting? I suspect it may lead to some issues...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try another docker build and try it again. I think that it didn't work with broker. Because it sets device broker to broker instead of actual broker address.
What exactly does broker mean in this case? It should be ip or dns name. Right?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It actually gets resolved by Docker Compose to the ip address of the broker, since they're in the same Compose template

Copy link
Contributor Author

@muros-ct muros-ct Dec 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried with broker as broker.ip.
Problem is that broker name is matched to broker ip only inside docker. But if I connect external device to that broker then device tries to communicate with broker by this name. But because device itself is not inside docker it can't connect to broker as it doesn't exist outside, where device is.
Now to approach this issue?

I get this error in integration test:

[ERROR] Scenario: Send BIRTH message and then DC message Time elapsed: 0.017 s <<< ERROR!
org.eclipse.kapua.service.device.call.kura.exception.KuraMqttDeviceCallException: Error:
Caused by: org.eclipse.kapua.service.device.call.kura.exception.KuraMqttDeviceCallException: Error:
Caused by: org.eclipse.kapua.KapuaException: An internal error occurred: {0}.
Caused by: org.eclipse.kapua.transport.mqtt.MqttClientException: Error: tcp://broker:1883,KapuaPool-1544111357250-7243724233515380736,kapua-sys
Caused by: org.eclipse.paho.client.mqttv3.MqttException: MqttException
Caused by: java.net.UnknownHostException: broker
`

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LeoNerdoG can you please check my branch for this PR.
Does dockers still run? Especially, can a device connect to Kapua and can you issue commands on that device?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LeoNerdoG No need to check I reverted broker.ip back to broker.
@lorthirk You are right, I set the broker.ip to broker. The problem is that I want to have same tests for dockerized and local environment, and some just can't be run in combination.
Also some test scenarios are still written for embedded broker and are using localhost keyword in scenario itself. I will fix those in another PR.

@codecov
Copy link

codecov bot commented Dec 6, 2018

Codecov Report

Merging #2234 into develop will decrease coverage by 1.45%.
The diff coverage is 50%.

Impacted file tree graph

@@              Coverage Diff              @@
##             develop    #2234      +/-   ##
=============================================
- Coverage      57.42%   55.97%   -1.46%     
+ Complexity      1584     1555      -29     
=============================================
  Files           1071     1071              
  Lines          26010    26063      +53     
  Branches        2299     2301       +2     
=============================================
- Hits           14936    14588     -348     
- Misses         10093    10507     +414     
+ Partials         981      968      -13
Impacted Files Coverage Δ Complexity Δ
...service/datastore/internal/MessageStoreFacade.java 48.41% <0%> (-0.45%) 0 <0> (ø)
...datastore/internal/mediator/DatastoreMediator.java 87.83% <0%> (-3.72%) 0 <0> (ø)
...java/org/eclipse/kapua/qa/steps/EmbeddedJetty.java 0% <0%> (-100%) 0 <0> (-4)
...lipse/kapua/service/tag/steps/TagServiceSteps.java 90.38% <100%> (+0.8%) 11 <2> (+1) ⬆️
...ice/datastore/client/rest/RestDatastoreClient.java 58.86% <100%> (-0.1%) 45 <0> (-1)
...service/datastore/steps/DataStoreServiceSteps.java 89.63% <100%> (+0.02%) 195 <1> (ø) ⬇️
...pse/kapua/service/user/steps/UserServiceSteps.java 80.22% <100%> (+0.15%) 46 <0> (ø) ⬇️
...rg/eclipse/kapua/qa/steps/EmbeddedEventBroker.java 70% <20%> (-5.56%) 4 <2> (ø)
.../org/eclipse/kapua/qa/steps/EmbeddedDatastore.java 59.25% <20%> (-8.93%) 4 <2> (ø)
...ava/org/eclipse/kapua/qa/steps/EmbeddedBroker.java 60% <33.33%> (-4.45%) 4 <2> (ø)
... and 62 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 77e96f7...7fe81a1. Read the comment docs.

@Coduz Coduz added the Test Test related stuff. It's a dirty job, but someone needs to do that! label Dec 7, 2018
@muros-ct
Copy link
Contributor Author

muros-ct commented Dec 7, 2018

@Coduz and @lorthirk Do you think we can merge this.
@andrejn-ct and I would like to base a new branch based on develop. That new branch would cover refactoring of qa steps. For that we would need this changes.

Integration tests were refactored to a degree that they can be run
with dockerized environment of Kapua. Database, ES, console, broker, API
all are run in docker. Tests connect to those in run as full integration
or functional tests.

This refactoring didn't changed the integration tests with embedded infrastructure,
meaning that both approaches can be used.

Read RunTests.md for usage.

Signed-off-by: Uros Mesaric Kunst <uros.mesaric-kunst@comtrade.com>
integration tests.

Broker for docker is returned back to way it was before change, using
broker.ip as broker and not localhost.

Signed-off-by: Uros Mesaric Kunst <uros.mesaric-kunst@comtrade.com>
@muros-ct
Copy link
Contributor Author

If everyone is OK, I would merge this changes.
It effects only tests and changes are needed for further refactoring of qa-steps.

@muros-ct muros-ct merged commit 00359f2 into eclipse-kapua:develop Dec 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Test Test related stuff. It's a dirty job, but someone needs to do that!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants