LoRaWAN LMICC library with FreeRTOS

[Ryann98]

Hello,
Is multi-tasking using (FreeRTOS) possible with the LoRaWAN LMICC library ?
I am running two tasks on my FeatherBoard:
-one to sendLoRa packets and then suspending it to run another task and do some computation
-and then resuming back the sendLora task but the transmission never resumes

Any suggestions or help would be appreciated.
Thank you.

[terrillmoore]

Yes, it's very possible. However, it requires care, because you must put the LMIC in its own task, and you must introduce trampoline functions to deal with synchronization. You cannot call the LMIC APIs directly because the job loop "knows" it's not going to be interrupted, and your call to the LMIC starts a copy of the framework running in a different thread. Result: crashes.

The right thing is three tasks: one to send packets; one for your other task; and a third for the LMIC. The LMIC task alternates between checking the run loop and checking for incoming requests.

Because the LMIC directly exposes the LMIC.* contents as part of the API, it's going to require a little work if you've got code that talks to LMIC.* entries directly. And you can't call LMIC_... functions at all from outside the LMIC thread; you have to have helper functions that synchronize and do the work. But it's mostly just typing, and less typing if you do this in C++ rather than pure C.

Is your feather a single core or multi-core system?

This is really hard to describe (based on lots of years trying to describe it) because a concrete description depends on how you've structured your app. It's comparatively easy to show by example. If you'll post a link to your repo, I can possibly fork it and send updates. I can certainly review and give more concrete suggestions.

[Ryann98]

Hello Terrill,

Thank you for your detailed explanation, I appreciate you getting back to me.
My code is still a mess and I wouldn't want to waste your time but I will post below a pseudo-code describing exactly what my code is intented to do.

I am using the nRF52840 feather with Bluetooth communication enabled and writing to a JSON file using "Arduino.JSON".

In my setup() function:

• I am writing to a JSON file
• Defining and configuring the Bluetooth peripherals
• Defined 3 tasks to run my code

Everything what is happening is inside my tasks, my loop() function is empty.

-The first (highest priority) task: LoraTransmit(): I set up the Lora configuration as mentioned in the "ttn-otaa-feather-us915"

-The second task ReceiveBluetooth(): I am waiting for a Bluetooth command from the "Bleuart".If a valid command is received,
I suspend the LoraTransmit() task and change a flag

-The third task (lowest priority) loopTask(): checks for the flag change and executes a function that has some computation and calls for a modbus sensor.
Once it is done, I resume the LoraTransmit() task.

Additional info:
I had to suspend the LoraTransmit() task because it was crashing my board, like you mentioned above. However, when I resume the LoraTransmit() in the third task [looptask()] , it never resumes. Referring to your previous reply, it seems like I am doing something wrong or not using the LMIC correctly.

Below is the implementation for my LoraTransmit task:

void LoraTransmit(void *)
{

//LMIC_setClockError(MAX_CLOCK_ERROR * 1 / 100);
delay(5000);
while (! Serial)
;
Serial.begin(9600);
Serial.println(F("Starting"));
// LMIC init
os_init();
// Reset the MAC state. Session and pending data transfers will be discarded.
LMIC_reset();

LMIC_setLinkCheckMode(0);

//LMIC.dn2Dr = DR_SF9; //
  
LMIC_setDrTxpow(DR_SF7,14);
LMIC_selectSubBand(1);

// Start job (sending automatically starts OTAA too)
do_send(&sendjob);

 for(;;)

{
//Serial.print("LoRa TASK RUNNING \n");
os_runloop_once();

// vTaskDelay(pdMS_TO_TICKS(DELAY_50_MS));
}
}

I would be happy to provide a more detailed implementation once I organize my code.
Thank you so much for your assistance !

[terrillmoore]

Sure.

A few questions for you.

• What development environment are you using?
• Can you post a link? (I can tell that you must be using an Arduino/FreeRTOS integration; you're using Serial.print and other Arduino APIs, and so there must be an Arduino board package that includes the synchronization to support multiple tasks.

If I can find the dev env, it will be easier for me to research and suggest changes / approaches.

Best regards,
--Terry

[Ryann98]

I am using Platform IO. I shared my repository with you and created a READme file describing briefly my issue.
Thank you !!

[Ryann98]

Hello again Terrill,

Is there a way that would allow me to suspend and resume Lora Transmission ? or the os_runloop_once() should always be running ?
Also, what if I wanted to change the “TX_INTERVAL” time or for example ask the user to set his desired transmission time.
In the code that I used it was declared as a constant “const unsigned TX_INTERVAL=30”, can we change it to a variable ?

[terrillmoore]

The TX_INTERVAL can be anything you like.

The actual flow you need (sorry that the examples are not clear) is this:

1. Initialize the LMIC. At this point, it will be idle. You should set a flag "LMIC is idle". (This could be a FreeRTOS event flag or semaphore.) The LMIC task can block after calling the runloop whenever the LMIC is idle flag is set.

2. At some point (whenever it makes sense) transmit data. From inside the LMIC task, use one of these APIs: LMIC_setTxData2(), LMIC_setTxData2_strict(), LMIC_sendWithCallback() or LMIC_sendWithCallback_strict(). (See table below for discussion of differences.) [This is a little tricky, because your other code is outside the LMIC task, and so there's a synchronization needed.] **Reset the LMIC is idle flag."

3. Until the transmit completes, don't try to send again.

4. Inside the LMIC task, when you detect that the LMIC is idle, set the idle flag again.

5. To avoid unnecessary compute-bound spinning on the LMIC, in v4.1.1, you have to use a hack at the top level of the LMIC thread.

6. Avoid, if possible, using an os_job_t for doing your delays -- that's really confusing, and makes it harder to figure out what is going on.

Detecting LMIC idle

If the LMIC is idle, you don’t need to call os_runloop_once. It’s currently not so easy to find out if the LMIC is idle. You can do it without changing code by a two-step process.

1. LMIC_queryTxReady() – if zero, then the LMIC is busy.
2. Otherwise, call os_queryTimeCriticalJobs(sec2osticks(1000)) – if it returns zero, then the LMIC is idle.

Discussion of send APIs

API name	requires buffer pointer	returns status code	will notify your routine when TX is done	will increase data rate if needed	Note
LMIC_setTxData()	no	no	no	yes	legacy API, do not use in new code
LMIC_setTxData_strict()	no	no	no	no	legacy API, do not use in new code
LMIC_setTxData2()	yes	yes	no	yes	you must detect TX done in the global LMIC event handler
LMIC_setTxData2_strict()	yes	yes	no	no	you must detect TX-done in the global LMIC event handler
LMIC_sendWithCallback()	yes	yes	yes	yes
`LMIC_sendWithCallback_strict()	yes	yes	yes	no

Recommended LMIC thread design

Assume that LMIC_thread() is started "somehow" -- it's the main function of the LMIC task. Use the following pseudocode. Some changes are needed to make this simpler; maybe in v5.

void LMIC_thread(void *arg) {
   // initialize the LMIC
   // other initialization for message queue ... app specific
   while (1) {
     // wait for something to do
     while (LMIC is idle && no message to send)
        /* block -- OS-specific */;

     // transmit a message if one is ready to be transmitted
     if (message to send && LMIC_queryTxReady()) {
         // consume message
         // ("consume" -- reset flags or adjust queues so that the message only gets sent once.)
         status = LMIC_sendWithCallback(...);
         // remember to check the status code!
     }

     // now keep running the LMIC until it goes idle, sleeping when possible.
     while (! LMIC is idle) {
       os_runloop_once();
       if (! os_queryTimeCriticalJobs(ms2osticks(10)) {
           // use an OS API to sleep the task for 10ms, as there's nothing happening soon
     } // end while
  // LMIC is now idle,... loop around and look for a message; if not found, sleep forever.
  } 

  // not reached
}

[terrillmoore]

The above pseudo-code is not necessarily complete; in particular, the callback functions all are issued directly from inside the LMIC task. The problem is that these callbacks can happen at time-critical moments. Experience suggests that it is best (in general) to have additional auxiliary tasks.

1. A callback task that takes care of delivering any user notifications. This task watches a queue; the LMIC posts to that queue and the callback task reads from that queue, and calls the user notification functions. (The queue elements typically contain a pointer to the user's callback function and any required data for the callback.) This runs at a lower priority than the LMIC. The LMIC must not block when posting, so the queue has to be "big enough" to never overflow. Generally, experience shows that a queue depth of 8 is much more than is needed.
2. A debug print task that outputs debug messages. This runs at a lower priority than the callback task (and maybe anything else). Often this is just processing a ring buffer. It's important to make sure that the debug prints don't block, if the ring buffer gets full.

Tasks other than the callback task and the debug print task should not #include "lmic.h". They must not call any of the LMIC APIs directly, and they should not access any of the fields of struct LMIC_t. Instead, you should provide client APIs that submit requests to the LMIC thread for processing. The augmented LMIC_thread() function would look like this:

typedef struct LMIC_request_s LMIC_request_t;   ///< the LMIC request block
typedef uint8_t LMIC_request_status_t;          ///< the status code.

struct LMIC_request_s {
  // to save memory, we put wider things first.
  LMIC_request_t *pNext, *pLast;   ///< forward and back links in the queue
  void *pClientData;               ///< arbitrary data for use by client
  void (*pClientDoneFn)(LMIC_request_t *pRequest);	///< function to call when complete

  uint8_t bRequestCode;           ///< the request code.
  LMIC_request_status_t bStatus;  ///< the request status.
  
  // this is just an example; in reality, this should be more complicated.
  union {
    struct {
      const uint8_t *pWriteBuffer;    ///< input: pointer to message to be queued
      size_t nWriteBuffer;            ///< input: size of write buffer
      lmic_tx_error_t txStatus;       ///< output: the transmit result.
    } SendData;                       ///< The parameters for LMIC_RQ_SEND_BUFFER
    struct {
      uint32_t FCnt;                  ///< output: the frame count.
    } GetFCnt;                        ///< The parameters for LMIC_RQ_GET_FCntDown, etc.
  } Rq;                               ///< The parameters for specific requests; in a union to save memory.
} LMIC_request_t;

bit_t LMIC_processInboundRequest() {
  LMIC_request_t *pRequest;
  // pull first request from input queue and put in pRequest. Do this
  // atomically. Set pRequest to NULL if queue is empty.
  
  if (pRequest == NULL)
	return 0;

  // figure out what to do with the request block. Operations that require the LMIC to be idle
  // should be deferred (use the message queue).
  switch (pRequest->bRequestCode) {
  default:
	// unknown request. Complete with error.
	pRequest->bStatus = LMIC_RQ_STATUS_UNKNOWN_REQUEST;
	// queue the request to the callback task to be completed.
	break;

  // we want to process most requests even while the LMIC is busy; but we can only
  // send when it's not busy. So move send requests to a secondary queue to process later.
  case LMIC_RQ_SEND_BUFFER:
	// append the request to the message-to-send queue.
	break;

  // this is an example of how to handle an immediate request; we can return a variety of
  // things immediately (even if the LMIC is busy) because we share the thread with the LMIC;
  // so it can't be changing anything. No further synch is needed, so we just do the work.
  case LMIC_RQ_GET_FCntDown:
        // return FCntDown in request block
	pRequest->Rq.GetFCnt.FCnt = LMIC.seqnoDn;
	pRequest->bStatus = LMIC_RQ_STATUS_OK;
	// queue the request to the callback list to be completed
	break;
  // ... other requests
  }

  // indicate that we did something.  
  return 1;
}

void LMIC_thread_v2(void *arg) {
   // initialize the LMIC
   // other initialization for inbound request queue ... app specific
   while (1) {
     // wait for something to do
     while (LMIC is idle && no request pending)
        /* block -- OS-specific */;

     // remove requests from the input queue and process.
     while (LMIC_processInboundRequest() != 0)
	/* drain the inbound queue */;

     // transmit a message if one is ready to be transmitted
     if (message to send && LMIC_queryTxReady()) {
         // consume message
         // ("consume" -- reset flags or adjust queues so that the message only gets sent once.)
         status = LMIC_sendWithCallback(...);
         // remember to check the status code!
     }

     // now keep running the LMIC until it goes idle, sleeping when possible.
     while (! LMIC is idle) {
       while (LMIC_processInboundRquest() != 0)
         /* drain the inbound queue */;
       os_runloop_once();
       if (! os_queryTimeCriticalJobs(ms2osticks(10)) {
           // use an OS API to sleep the task for 10ms, as there's nothing happening soon
	   // wake up early if the inbound queue becomes non-empty.
     } // end while
  // LMIC is now idle,... loop around and look for a message; if not found, sleep forever.
  } 

  // not reached
}

This style require a fair amount of typing if done in pure C (mostly in the header file), but the resulting code can be typesafe, fast and robust. (I claim it's even readable, once you learn the idioms.)

[Ryann98]

Hi Terrill.
Thank you so much for your detailed explanation, I will try to implement something similar to this !