power: Add device idle power management support #13382

ramakrishnapallala · 2019-02-14T04:15:17Z

device: Extend device_set_power_state API to support async requests
power: Add device idle power management support
samples: power: Add test for device Idle PM

ramakrishnapallala · 2019-02-14T04:17:20Z

These changes are the outcome of the discussion in Dev Meeting related to PR: #12172

zephyrbot · 2019-02-14T04:29:47Z

All checks are passing now.

Review history of this comment for details about previous failed status.
Note that some checks might have not completed yet.

codecov-io · 2019-02-14T04:56:45Z

Codecov Report

Merging #13382 into master will increase coverage by 0.03%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master   #13382      +/-   ##
==========================================
+ Coverage   51.99%   52.02%   +0.03%     
==========================================
  Files         308      308              
  Lines       45569    45569              
  Branches    10553    10553              
==========================================
+ Hits        23693    23709      +16     
+ Misses      17069    17057      -12     
+ Partials     4807     4803       -4

Impacted Files	Coverage Δ
include/device.h	`100% <ø> (ø)`	⬆️
kernel/device.c	`88.88% <ø> (ø)`	⬆️
drivers/timer/sys_clock_init.c	`0% <0%> (ø)`	⬆️
subsys/testsuite/ztest/src/ztest.c	`72.72% <0%> (+7.43%)`	⬆️
subsys/testsuite/ztest/include/ztest_assert.h	`77.77% <0%> (+38.88%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bcf3d8e...1604ed2. Read the comment docs.

pizi-nordic · 2019-02-14T10:17:08Z

include/device.h

@@ -208,6 +208,7 @@ extern "C" {

 struct device;

+typedef void (*device_pm_cb)(struct device *dev, int status);


Could we add user-defined context the the callback?

pizi-nordic · 2019-02-14T10:18:34Z

include/device.h

@@ -221,7 +222,7 @@ struct device_config {
 	int (*init)(struct device *device);
 #ifdef CONFIG_DEVICE_POWER_MANAGEMENT
 	int (*device_pm_control)(struct device *device, u32_t command,
-				 void *context);


What about something like this:

int (*device_pm_control)(struct device *device, u32_t command, void *arg, device_pm_cb cb, void *context) typedef void (*device_pm_cb)(struct device *dev, int status, void *context);

ramakrishnapallala · 2019-02-18T07:41:49Z

@pizi-nordic Can you look the PR once?

ramakrishnapallala · 2019-02-18T10:07:09Z

@pizi-nordic I have modified the device idle PM based in async device_set_power_state(). Please let me know if you any concerns with the design. Based on your feedback I will push the sample app.

ramakrishnapallala · 2019-02-19T05:50:13Z

@pizi-nordic Can you please review the PR? based on your feedback i will work on sample app.

pizi-nordic · 2019-02-19T15:04:00Z

doc/reference/power_management/index.rst

@@ -339,7 +339,7 @@ Device Set Power State

 .. code-block:: c

-   int device_set_power_state(struct device *device, u32_t device_power_state);
+   int device_set_power_state(struct device *device, u32_t device_power_state, device_pm_cb cb);


How I can pass custom data to the callback?
(We agreed on a slightly different API: #13382 (comment))

int (*device_pm_control)(struct device *device, u32_t command, void *arg, device_pm_cb cb, void *context)
typedef void (*device_pm_cb)(struct device *dev, int status, void *context);

I missed the arg parameter you mentioned in the comment.
How are you planning to use arg parameter?

The arg allows to pass some parameters specific for given operation. I not have exact scenario for its usage now, however many times I missed this functionality.

OK sure, will update the PR accordingly.

pizi-nordic · 2019-02-21T14:33:29Z

doc/reference/power_management/index.rst

@@ -339,7 +339,7 @@ Device Set Power State

 .. code-block:: c

-   int device_set_power_state(struct device *device, u32_t device_power_state);
+   int device_set_power_state(struct device *device, u32_t device_power_state, void *arg, device_pm_cb cb);


I suggest swapping cb and arg arguments. This would result in more convenient order.

do you want to send the arg to callback() from driver set power function?

Yes, This should be opaque value which is passed from device_set_power_state() caller to callback.

pizi-nordic · 2019-02-21T14:50:17Z

subsys/power/device_pm.c

@@ -0,0 +1,158 @@
+/*


This does not look like the FSM we agreed on during our call.
Do you need some kind of formal design proposal?

Before I implement the FSM I wanted some clarity on the state transitions.

Consider the following scenario.

device idle requested suspend

pm_state change from ACTIVE -> SUSPENDING

driver tries to suspend the device but fails to do so

driver calls the callback with an error return code.
In this case, what should we set the device state to?

Also if the pm_state is in SUSPENDING and new request comes for SUSPEND what we should do? should we call the device_set_power_state() with new SUSPEND requests for return?

what action we should for new requests if the current case is SUSPENDING or RESUMING? we can not skip the new requests(ex: SUSPEND) even if it is the same as the previous (ex: SUSPENDING) request because the previous request might fail. It would be better to rely only on the actual state for queuing the requests.

Also, if the current pm_state SUSPENDING and the device were not put suspend yet and but a new request comes for RESUME. How we should handle this case? do we need to change the pm state to RESUMING?

This commit now uses the actual device PM states as pm_states so we don't need any translation of FSM to device states in the device_pm code. I feel this simplifies the whole logic.

@pizi-nordic Please have look at my query above. Thanks.

You cannot use actual device state because it not covers transition state.
Here is algorithm I have in my mind. I hope it explain all the cases you described.
As you see, the algorithm is trying to reduce number of calls to device_set_power_state().

Assumptions:

Device could change power state only when it is idle. There is no "forcing" power state, as this might render the system non-operational (for example shutting down SPI bus during write to SPI flash will corrupt data). Failure of setting power mode is a failure of device - you cannot rely on it any more.

Suspending and resuming operations cannot be terminated until complete.

Variables:

FSM state: (ACTIVE, SUSPENDED, SUSPENDING, RESUMING)

Target state: (ACTIVE, SUSPENDED).

Pseudocode:

device_suspend() { if (target_state == SUSPENDED) { /* Device will eventually reach the SUSPSENDED state */ return; } target_state = SUSPENDED; trigger_fsm(); } device_resume() { if (target_state == ACTIVE) { /* Device will eventually reach the ACTIVE state */ return; } target_state = ACTIVE; trigger_fsm(); } trigger_fsm() { run_through_workqueue(fsm); } /* * As device_set_power_state() is asynchronous, this will no block workqueue. * Also, workqueue allow us to call device api from thread mode as well as handle devices * which calls callback directly from device_set_power_state(). */ fsm() { switch (fsm_state) { case SUSPENDING: case RESUMING: /* We have to wait */ break; case ACTIVE: if (target_state == SUSPENDED) { device_set_power_state(SUSPENDED, trigger_fsm) fsm_state = SUSPENDING; } break; case SUSPENDED: if (target_state == ACTIVE) { device_set_power_state(ACTIVE, trigger_fsm) fsm_state = RESUMING; } break; } broadcast_state(fsm_state); } device_wait_suspended() { while (recv_broadcasted_state() != SUSPENDED) { sleep() } } device_wait_active() { while (recv_broadcasted_state() != ACTIVE) { sleep() } } device_suspend_sync() { device_suspend(); device_wait_suspended(); } device_resume_sync() { device_resume(); device_wait_active(); }

This could be extended to cancel work queue job if is is not longer needed, but such shortcut must be carefully checked for race conditions.

I will look into it and get back to you.

FSM state: (ACTIVE, SUSPENDED, SUSPENDING, RESUMING)

Target state: (ACTIVE, SUSPENDED).

device_suspend() {
if (target_state == SUSPENDED) {
/* Device will eventually reach the SUSPSENDED state */
return;
}

target_state = SUSPENDED;
trigger_fsm();
}

device_resume() {
if (target_state == ACTIVE) {
/* Device will eventually reach the ACTIVE state */
return;
}

target_state = ACTIVE;
trigger_fsm();
}
trigger_fsm() {
run_through_workqueue(fsm);
}
Are you suggesting we should have workquque in the device_pm.c? If so, do we need a saperate thread or we should submit the work on system thread?

fsm() {
switch (fsm_state) {
case SUSPENDING:
case RESUMING:
/* We have to wait */
break;

case ACTIVE: if (target_state == SUSPENDED) { device_set_power_state(SUSPENDED, trigger_fsm) fsm_state = SUSPENDING; } break; case SUSPENDED: if (target_state == ACTIVE) { device_set_power_state(ACTIVE, trigger_fsm) fsm_state = RESUMING; } break;

}

broadcast_state(fsm_state);
}
is this the worked logic which maintains the states? also why broadcat fsm state?

Are you suggesting we should have workquque in the device_pm.c? If so, do we need a saperate thread or we should submit the work on system thread?

IMHO we can safely use system workqueue. The separate thread was needed earlier because synchronous device_set_power_state() could block for a long time blocking execution of other things. Now, with the asynchronous device_set_power_state() this problem is eliminated, so we can use system thread.

There is a possibility to implement this without any workqueue (fsm executed directly from the caller), but this would require using spinlocks which in my opinion will be overkill.

is this the worked logic which maintains the states?

I am not fully getting this question. If you are asking which entity holds the device state: In the proposed model this is tracked by FSM.

why broadcat fsm state?

Broadcasting the FSM state is needed for blocking version of the device_get/put. Whenever state is changed, a signal is sent. If FSM reached desired state, then you can exit from the blocking function.

Also, the application can track FSM state and run some action if device reached desired state (example: Power off the system if all devices are suspended).

addressed, at least when it comes to the system workqueue

ramakrishnapallala · 2019-02-25T15:10:42Z

@pizi-nordic I have updated the PR as your comments. Please look at the overall flow/design of the code. I will update PR with sample app once you are fine with it.

pizi-nordic · 2019-02-25T15:13:50Z

@ramakrishnapallala: OK. I will look at the code tomorrow morning.

pizi-nordic

Looks good from high level. I started detailed review (see my comments), but this is not finished yet. I will continue tomorrow.

pizi-nordic · 2019-03-11T15:27:20Z

subsys/power/device_pm.c

+	k_sem_take(&dev->config->pm->lock, K_FOREVER);
+	/* Bring up the device before disabling the Idle PM */
+	k_work_submit(&dev->config->pm->work);
+	dev->config->pm->enable = false;


I suggest moving dev->config->pm->enable = false; before k_work_submit(). If work queue thread has higher priority than caller thread, the work will be executed before enable flag change and the FSM will not update device state.

wil fix it.

pizi-nordic · 2019-03-11T15:48:42Z

subsys/power/device_pm.c

+		} else {
+			k_poll_signal_check(&dev->pm->signal,
+						&signaled, &result);
+			if (result != target_state) {


Still, you are waiting here for first signal, which might be not the you are waiting for.
Also, the signal state is not reset, so this will work only once.

ramakrishnapallala · 2019-03-12T07:47:56Z

@pizi-nordic I have updated the PR, please have a look. Thanks.

pizi-nordic

Looks really good. Needs only some polish to be perfect :)

pizi-nordic · 2019-03-12T14:26:42Z

subsys/power/device_pm.c

+	k_sem_take(&dev->config->pm->lock, K_FOREVER);
+	dev->config->pm->enable = true;
+
+	/* During the driver init, device can set the


IMHO we have to clearly write in the docs for devices, that all drivers has to boot the device in suspended state.
Now, all devices are active after initialization.

Adding in the DEVICE_DEFINE macro description.

pizi-nordic · 2019-03-12T14:32:03Z

subsys/power/device_pm.c

+LOG_MODULE_DECLARE(power);
+
+/* Device PM request type */
+#define DEVICE_PM_ASYNC			(1 << 0)


If you add #define DEVICE_PM_SYNC (0 << 0) and use this flag later, the code will be easier to read.

Ok will add the macro.

pizi-nordic · 2019-03-12T14:34:05Z

subsys/power/device_pm.c

+			"Invalid device PM state requested");
+
+	if (target_state == DEVICE_PM_ACTIVE_STATE) {
+		if (atomic_inc(&dev->config->pm->usage) < 0) {


I think that usage count management should go into get/put and the device_pm_request() should be called only when necessary. Now we are mixing different tasks in one function.

this will avoid code duplication but anyways will check it again.

pizi-nordic · 2019-03-12T14:35:47Z

subsys/power/device_pm.c

+			"Invalid device PM state requested");
+
+	if (target_state == DEVICE_PM_ACTIVE_STATE) {
+		if (atomic_inc(&dev->config->pm->usage) < 0) {


if (atomic_inc(&dev->config->pm->usage) < 0): Are we starting count from -1?
We only have to trigger transition when usage changes from 0 to 1 and from 1 to 0.

this is just to make sure we are also considering the case where some app/driver might call put() multiple times without calling get().

pizi-nordic · 2019-03-12T14:38:58Z

subsys/power/device_pm.c

+	k_work_submit(&dev->config->pm->work);
+
+	/* Incase of Sync request wait for completion event */
+	if (!(pm_flags & DEVICE_PM_ASYNC)) {


You can save one level of indentation and make code easier to read ;-)

if (pm_flags & DEVICE_PM_ASYNC) { return } /* Blocking code here */

pizi-nordic · 2019-03-12T14:44:40Z

subsys/power/device_pm.c

+
+fsm_out:
+	k_poll_signal_raise(&dev->config->pm->signal, pm_state);
+


Please remove this empty line.

pizi-nordic · 2019-03-12T14:45:45Z

subsys/power/device_pm.c

+	/* Set the fsm_state */
+	if (*((u32_t *)context) == DEVICE_PM_ACTIVE_STATE) {
+		atomic_set(&dev->config->pm->fsm_state,
+				DEVICE_PM_FSM_STATE_ACTIVE);


If you pass DEVICE_PM_FSM_STATE_ACTIVE/DEVICE_PM_FSM_STATE_SUSPENDED as arg, the decoding here could be replaced by single call to atiomic_set() saving some flash.

then again I will have to add temp local variable for storing target fsm state. Can we leave this as it for now.

pizi-nordic · 2019-03-12T14:56:41Z

samples/subsys/power/device_pm/src/dummy_driver.c

+	}
+
+	printk("Async wakeup request queued\n");
+	ret = k_poll(&async_evt, 1, K_FOREVER);


We have to wait in a loop until the state we are requesting is signalled. Other states might be signalled before that time.

pizi-nordic · 2019-03-12T14:57:41Z

samples/subsys/power/device_pm/src/dummy_driver.c

+			ret = dummy_suspend(dev);
+		}
+		break;
+	case DEVICE_PM_GET_POWER_STATE:


IMHO get power state API should be removed, but this might go into another PR.

pizi-nordic · 2019-03-12T15:04:07Z

drivers/counter/counter_qmsi_aonpt.c

+		cb(dev, ret, context, arg);
+	}
+
+	return ret;


We have to differentiate the device_set_power_state() return value and the return code passed to callback:

If device_set_power_state() return something other that 0, then callback will not be called and operation failed.

If device_set_power_state() return 0, the operation status is reported through callback which will be/was called.

Also, this has to be documented.

mnkp · 2019-03-13T01:25:30Z

I have a question regarding the first commit "device: Extend device_set_power_state API to support async requests". The commit message says:

The existing device_set_power_state() API works only in synchronous
mode and this is not desirable for devices(ex: Gyro) which take
longer time (few 100 mSec) to suspend/resume.

However, the mere fact that some devices may take long time to suspend/resume does not seem like a good enough reason to make device_set_power_state asynchronous. If I need to wait for a device to wake up then I can wait in a thread I'm calling the function from. It's after all what the threads are made for. The rest of the system is running and if there is anything to do in a meantime it should be done elsewhere. I have a trouble coming up with a scenario when I would like the thread to proceed before the device is fully suspended (or resumed).

Looking at the remaining implementation let's say I have an external audio chip, or the aforementioned gyro and the chip documentation says that after waking the device up by toggling a gpio pin I need to wait 10 ms for the internal power to stabilize before the device can be considered active. The chip does not provide any interrupt to let me know that it's active. I simply need to wait 10 ms. The implementation of the device_set_power_state() function must be non blocking as it's submitted on the system work queue. So how exactly do I wait 10 ms? I cannot call k_sleep() from within device_set_power_state() as this would be blocking (for the system workqueue). Am I supposed to use a kernel timer? These all sounds fairly complicated.

My question is: what kind of real life use case are we trying to solve that requires an async version of device_set_power_state() function that cannot be solved by its synchronous version + the standard thread support?

The implementation proposed in this PR is complex and seems to go against the very principles Zephyr is based on. According to the Synchronous Calls paragraph of the Device Drivers and Device Model:
"High-level calls accessed through device-specific APIs, such as i2c.h or spi.h, are usually intended as synchronous. Thus, these calls should be blocking."

Add framework for device Idle Power Management(IPM) for suspending devices based on device idle. This will help in saving power even while system(CPU) is active. The framework uses device_set_power_state() API set the device power state accordingly based on the usage count. Signed-off-by: Ramakrishna Pallala <ramakrishna.pallala@intel.com>

ramakrishnapallala · 2019-03-13T12:51:20Z

recheck

ramakrishnapallala · 2019-03-13T13:52:51Z

@pizi-nordic please review the updated the PR.

ramakrishnapallala · 2019-03-13T15:32:43Z

recheck

carlescufi

Most of the comments from @pizi-nordic seem to have been addressed, with some notable exceptions regarding memory usage when it comes to what gets stored in the pm structure. I will wait for another round of feedback from @ramakrishnapallala and then we can take a decision for code freeze.

carlescufi · 2019-03-13T16:14:32Z

samples/sensor/apds9960/src/main.c

@@ -79,11 +79,11 @@ void main(void)
 		u32_t p_state;

 		p_state = DEVICE_PM_LOW_POWER_STATE;
-		device_set_power_state(dev, p_state);
+		device_set_power_state(dev, p_state, NULL, NULL);


I agree, an __ASSERT() here would be safer

carlescufi · 2019-03-13T16:15:21Z

include/device.h

+static inline void device_pm_disable(struct device *dev) { }
+static inline int device_pm_get(struct device *dev) { return 0; }
+static inline int device_pm_get_sync(struct device *dev) { return 0; }
+static inline int device_pm_put(struct device *dev) { return 0; }


Any follow-up here @ramakrishnapallala ?

carlescufi · 2019-03-13T16:15:35Z

include/device.h

+ * @brief Disable device idle PM
+ *
+ * Called by a device driver to disable device idle power management.
+ *


carlescufi · 2019-03-13T16:15:44Z

include/device.h

+ *
+ * Called by a device driver to enable device idle power management.
+ *
+ * @param dev Pointer to device structure of the specific device driver


carlescufi · 2019-03-13T16:16:00Z

include/device.h

@@ -221,7 +222,7 @@ struct device_config {
 	int (*init)(struct device *device);
 #ifdef CONFIG_DEVICE_POWER_MANAGEMENT
 	int (*device_pm_control)(struct device *device, u32_t command,
-				 void *context);


carlescufi · 2019-03-13T16:19:30Z

subsys/power/device_pm.c

@@ -0,0 +1,158 @@
+/*


addressed, at least when it comes to the system workqueue

carlescufi · 2019-03-13T16:19:37Z

subsys/power/device_pm.c

+LOG_MODULE_DECLARE(power);
+
+/* Device PM request type */
+#define DEVICE_PM_ASYNC			(1 << 0)


carlescufi · 2019-03-13T16:19:57Z

subsys/power/device_pm.c

+	}
+}
+
+static void device_pm_callback(struct device *dev,


addressed by workqueue

carlescufi · 2019-03-13T16:20:15Z

subsys/power/device_pm.c

+
+static void pm_work_handler(struct k_work *work)
+{
+	struct device_pm *pm = CONTAINER_OF(work,


carlescufi · 2019-03-13T16:20:40Z

subsys/power/device_pm.c

+	k_sem_take(&dev->config->pm->lock, K_FOREVER);
+	dev->config->pm->enable = true;
+
+	/* During the driver init, device can set the


Added test for Device Idle Power Management to invoke device_pm_get/put API's. Signed-off-by: Ramakrishna Pallala <ramakrishna.pallala@intel.com>

ramakrishnapallala · 2019-03-14T01:51:19Z

@pizi-nordic @carlescufi I replied to your comments, can you check.

carlescufi

Approving with reservations due to the currently unresolved issues. I believe @pizi-nordic already owns a number of GitHub issues related to improving power management in general, I think we could add one more about device idle improvements.

ramakrishnapallala requested review from anangl, andrewboie, andyross, dbkinder, MaureenHelm, nashif, nniranjhana, spoorthik and tbursztyka as code owners February 14, 2019 04:15

ramakrishnapallala requested a review from pizi-nordic February 14, 2019 04:15

ramakrishnapallala force-pushed the dev-pm branch from 31c9497 to 38f940c Compare February 14, 2019 05:11

pizi-nordic reviewed Feb 14, 2019

View reviewed changes

ramakrishnapallala force-pushed the dev-pm branch from 38f940c to 2db68e2 Compare February 15, 2019 02:50

dbkinder approved these changes Feb 15, 2019

View reviewed changes

ramakrishnapallala changed the title ~~device: Extend device_set_power_state API to support async requests~~ power: Add device idle power management support Feb 18, 2019

pizi-nordic suggested changes Feb 19, 2019

View reviewed changes

ramakrishnapallala force-pushed the dev-pm branch 2 times, most recently from 60a61fa to 13a21f1 Compare February 21, 2019 10:11

pizi-nordic reviewed Feb 21, 2019

View reviewed changes

ramakrishnapallala force-pushed the dev-pm branch 3 times, most recently from d3d473f to 131e782 Compare February 25, 2019 14:35

ramakrishnapallala force-pushed the dev-pm branch from 489d8d2 to d14a116 Compare March 11, 2019 05:48

pizi-nordic reviewed Mar 11, 2019

View reviewed changes

ramakrishnapallala force-pushed the dev-pm branch from d14a116 to d6b7d61 Compare March 12, 2019 07:47

nashif approved these changes Mar 12, 2019

View reviewed changes

nashif requested review from pizi-nordic and removed request for nniranjhana March 12, 2019 10:01

pizi-nordic reviewed Mar 12, 2019

View reviewed changes

pizi-nordic mentioned this pull request Mar 12, 2019

Make device a pollable object #14311

Closed

galak added this to the v1.14.0 milestone Mar 12, 2019

galak added area: Power Management priority: high High impact/importance bug labels Mar 12, 2019

ramakrishnapallala force-pushed the dev-pm branch from d6b7d61 to 33e5e57 Compare March 13, 2019 10:35

carlescufi requested changes Mar 13, 2019

View reviewed changes

ramakrishnapallala force-pushed the dev-pm branch from 33e5e57 to 1604ed2 Compare March 13, 2019 16:22

samples: power: Add test for device Idle PM

1604ed2

Added test for Device Idle Power Management to invoke device_pm_get/put API's. Signed-off-by: Ramakrishna Pallala <ramakrishna.pallala@intel.com>

nashif requested review from carlescufi and pizi-nordic March 14, 2019 05:52

carlescufi approved these changes Mar 14, 2019

View reviewed changes

nashif merged commit d910aa6 into zephyrproject-rtos:master Mar 14, 2019

mnkp mentioned this pull request Mar 16, 2019

Device Idle Power Management #12003

Closed

pabigot mentioned this pull request May 27, 2020

device_pm: clarify and document usage #24653

Closed

pabigot mentioned this pull request Jul 14, 2020

Power Management Infrastructure #14307

Closed

12 tasks

pabigot mentioned this pull request Jul 22, 2020

subsys: power: refactor device runtime power management #26366

Closed

		@@ -208,6 +208,7 @@ extern "C" {

		struct device;

		typedef void (device_pm_cb)(struct device dev, int status);


		fsm_out:
		k_poll_signal_raise(&dev->config->pm->signal, pm_state);

power: Add device idle power management support #13382

power: Add device idle power management support #13382

Conversation

ramakrishnapallala commented Feb 14, 2019 • edited Loading

ramakrishnapallala commented Feb 14, 2019

zephyrbot commented Feb 14, 2019 • edited Loading

codecov-io commented Feb 14, 2019 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ramakrishnapallala commented Feb 18, 2019

ramakrishnapallala commented Feb 18, 2019

ramakrishnapallala commented Feb 19, 2019

Choose a reason for hiding this comment

ramakrishnapallala Feb 20, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pizi-nordic Feb 21, 2019 • edited Loading

Choose a reason for hiding this comment

ramakrishnapallala Feb 21, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pizi-nordic Feb 22, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pizi-nordic Feb 25, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ramakrishnapallala commented Feb 25, 2019

pizi-nordic commented Feb 25, 2019

pizi-nordic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ramakrishnapallala commented Mar 12, 2019

pizi-nordic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mnkp commented Mar 13, 2019

ramakrishnapallala commented Mar 13, 2019

ramakrishnapallala commented Mar 13, 2019

ramakrishnapallala commented Mar 13, 2019

carlescufi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ramakrishnapallala commented Feb 14, 2019 •

edited

Loading

zephyrbot commented Feb 14, 2019 •

edited

Loading

codecov-io commented Feb 14, 2019 •

edited

Loading

ramakrishnapallala Feb 20, 2019 •

edited

Loading

pizi-nordic Feb 21, 2019 •

edited

Loading

ramakrishnapallala Feb 21, 2019 •

edited

Loading

pizi-nordic Feb 22, 2019 •

edited

Loading

pizi-nordic Feb 25, 2019 •

edited

Loading