Describe the architecture

BME680Setup.md

@@ -48,13 +48,4 @@ This is a Python client library to talk with Azure IoT Hub. There's also a C ver
 
 The SDK's documentation is here: https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-sdks (scroll down to the "Azure IoT Hub device SDK for Python" section).
 
-And that's it. You now have a working Zero, the sensor is working, and all the requirements are installed. The next step includes the remainig steps to [get the Zero to do readings in an interesting format and then upload them to Azure](DeviceUploadData.md)
-
-
-
-
-
-Also -- for another page, not here:
-- change the checkpointing interval
-- every 3 seconds
-- change 5ºC to 4ºC ??
+And that's it. You now have a working Zero, the sensor is working, and all the requirements are installed. The next step includes the remaining steps to [get the Zero to do readings in an interesting format and then upload them to Azure](DeviceUploadData.md).

DeviceUploadData.md

@@ -0,0 +1,10 @@
+# Get sensor readings in CSV format and upload to Azure
+
+
+
+
+
+Also -- for another page, not here:
+- change the checkpointing interval
+- every 3 seconds
+- change 5ºC to 4ºC ??

README.md

@@ -6,20 +6,41 @@ This is a sample of using the Bosch BME680 sensor connected to a Raspberry Pi Ze
 
 **To-add: Add photo of it with the connections**
 
-## Note on reading data from sensor
+## Architecture
 
-Bosch has a library (compiled C) to handle their BME680 sensor, including proprietary code to calculate the Interior Air Quality (IAQ) score adjusting for humidity/pressure. This led me to decide to not use either Adafruit's (https://github.com/adafruit/Adafruit_BME680) or Pimoroni's (https://github.com/pimoroni/bme680-python/) Python Libraries. What Bosch's code does exactly is not clear because the code is closed. You can also simply use either of the above libraries with the Adafruit breakout, both work.
+The process is divided in these steps:
+
+1. **Zero**: get readings from the sensor every 3 seconds and write a CSV file to the file system
+    - *Note*: as per various sources on the web, the frequency of readings seems to have an effect in the Gas reading values, due to the way the MOX sensor works, hence the short interval. (see here https://www.adafruit.com/product/3660 and https://cdn-shop.adafruit.com/product-files/3660/BME680.pdf and https://forums.pimoroni.com/t/bme680-observed-gas-ohms-readings/6608/5 for more information)
+2. **Zero**: a cron job calling a Python script regularly wakes up and pushes up the files to Azure IoT Hub in cronological order
+    - While this may seem inneficient vis-a-vis sending up the readings directly, I've actually found that having the readings as files helped me overcome development-time issues while setting up the processing in Azure, or situations where for some reason wifi drops or some kind of anomaly happens. Unless this is not at all possible, I'd also resort to this design in a production system.
+
+3. **Azure**: data received in an Azure IoT Hub is read by Azure Stream Analytics
+4. **Azure**: an Azure Stream Analytics job pushes the data into an Azure SQL Database
+5. **PowerPI** is used to do visualizations on the data read from the Azure SQL Database.
+
+This is the big picture overview as of writing. I've noticed from data I've already gathered that there are both "bad readings" (reported by the sensor) that have to be ignored, and anomalies that have to either removed from the data or smoothed out. I'll add steps above (probably in Stream Analytics) to handle these.
+
+**TBD** - do an architecture diagram
+
+### Note on reading data from sensor
+
+Bosch has a library (compiled C) to handle their BME680 sensor, including proprietary code to calculate the Interior Air Quality (IAQ) score adjusting for humidity/pressure. The fact that they are the makers of the sensor and that the calculations done are unknown l(it's close code) ed me to decide to not use Adafruit's (https://github.com/adafruit/Adafruit_BME680) nor Pimoroni's (https://github.com/pimoroni/bme680-python/) libraries to get readings. You can also simply use either of the above libraries with the Adafruit breakout, both work - Pimoroni's seems more refined.
 
 ## Base setup instructions (if you're starting from scratch)
 
 Start by following these instructions to set up the Zero: [Basic Device Setup](DeviceSetup.md) . If you already have one running, just remember to enable I2C. After this, you'll need to install some aditional libraries in the device, related either to the BME680 sensor or to Azure. To do this, follow the steps here: [Libraries setup](BME680Setup.md) . When you finish these two, you'll have a C application installed that prints out the sensor's readings, and all the required dependencies for what follows.
 
-## Get sensor readings and push them to Azure
+## Get formatted sensor readings and push them to Azure
 
 After the setup is done, you'll need to make make changes to make sure that:
 - the readings are captured in a format simple to process in Azure (I picked CSV, JSON would be another good option)
-- have code to send the readings to Azure IoT Hub
+- have code to send the readings to Azure IoT Hub.
+
+To do this, follow [these steps](DeviceUploadData.md)
 
 ## Process the incoming data in Azure
 
 And finally, what I'm doing in Azure, of course :)
+
+**TBD**