BME680 on Heltec LoRa 32(V3) with TTN and Home Assistant

I’m very new to LoRa and TTN, relatively new to Home Assistant, and been working with Arduino for several years, but still have a lot to learn.

I’m running HA OS on Raspberry Pi 4 with the TTN integration setup. I also have mqtt running and have sensors forwarding data from ESP8266 boards to mqtt over WiFi, but I’d like to get LoRa going to provide better power mangement and longer radio signal transmission distances. I curretnly have a Heltec LoRa Gateway connecting my TTN Stack to Home Assistant and to my LoRa end devices. I have a new 915 MHz Heltec Lora ESP32(V3) connected to an Adafruit BME680 board and have copied the sketch below.

I have been working for some time on getting my Home Assistant instance up and running with LoRa communication from remote sensors. For personal reasons I am working at getting this assembled with as few pre-made LoRa sensors as possible and have been trying to get the Heltec LoRa 32(v3) board running as a node to send (and eventually receieve) to the Heltec LoRa gateway, which is connected to my TheThingsStack. I have finally gotten this working in a bit of a rough form (see the image of HA dashboard data, all setup using Helpers),


but I have a few questions about the Arduino sketch that I have running on the Heltec board (copied below). I have not modified the credits yet since it is based on the “LoRaWAN_TTN” example from the Heltec_ESP32_LoRa_V3 library. I have also copied over some bits from the Adafruit BME680 library.

/**
 * 
 * FOR THIS EXAMPLE TO WORK, YOU MUST INSTALL THE "LoRaWAN_ESP32" LIBRARY USING
 * THE LIBRARY MANAGER IN THE ARDUINO IDE.
 * 
 * This code will send a two-byte LoRaWAN message every 15 minutes. The first
 * byte is a simple 8-bit counter, the second is the ESP32 chip temperature
 * directly after waking up from its 15 minute sleep in degrees celsius + 100.
 *
 * If your NVS partition does not have stored TTN / LoRaWAN provisioning
 * information in it yet, you will be prompted for them on the serial port and
 * they will be stored for subsequent use.
 *
 * See https://github.com/ropg/LoRaWAN_ESP32
*/


// Pause between sends in seconds, so this is every 15 minutes. (Delay will be
// longer if regulatory or TTN Fair Use Policy requires it.)
#define MINIMUM_DELAY 300 


#include <heltec_unofficial.h>
#include <LoRaWAN_ESP32.h>
#include <Wire.h>
#include <SPI.h>
#include <Adafruit_Sensor.h>
#include "Adafruit_BME680.h"

LoRaWANNode* node;

//create BME sensor
Adafruit_BME680 bme(&Wire1); // I2C on second Wire channel (Wire1)

#define SEALEVELPRESSURE_HPA (1013.25)

//Define the I2C pins for Wire1
#define BME_SDA 42
#define BME_SCL 41

//Setup sequatial counting for data transmissions
RTC_DATA_ATTR uint8_t count = 0; //for LoRa transmissions
RTC_DATA_ATTR uint8_t trans = 0; //for serial debugging

void setup() {
 //Not sure about the ordering of the next few things, but this works. It starts the heltec board
 //along with serial debugging and the BME680 board
 heltec_setup();

 Wire.begin();

bool wireStatus = Wire1.begin(BME_SDA, BME_SCL);
bme.begin(0x77, &Wire1);

 Serial.begin(115200); 

//Debugging at 115200 baud for BME680
 if (!bme.begin()) {
    Serial.println("Could not find a valid BME680 sensor, check wiring!");
    while (1);
    }else{
      Serial.print("BME680 found");
      }
  
  // Set up oversampling and filter initialization for BME680
  bme.setTemperatureOversampling(BME680_OS_1X);
  bme.setHumidityOversampling(BME680_OS_2X);
  bme.setPressureOversampling(BME680_OS_16X);
  bme.setIIRFilterSize(BME680_FILTER_SIZE_3);
  bme.setGasHeater(320, 150); // 320*C for 150 ms

 // Obtain directly after deep sleep
 // May or may not reflect room temperature, some correction needed 
  float temp = heltec_temperature();
  Serial.println();  
  Serial.println(F("-------------------------"));
  Serial.print("Consecutive Transmissions: ");
  Serial.print(trans++);
  Serial.println();
  Serial.printf("Heltec Temperature: %.1f °C\n", temp);

 //create floats and nicknames for the BME680 values
  float bmet = bme.readTemperature();
  float bmeh = bme.readHumidity();
  float bmep = bme.readPressure();
  float bmea = bme.readAltitude(SEALEVELPRESSURE_HPA);
  float bmeg = bme.readGas();

 //Perform reading of BME680 sensor to make data available for transmision
  bme.performReading();

 //Print BME680 data to serial for debugging, math to get values into indicated units
  Serial.printf("BME Temperature: %.1f °C\n", bmet);
  Serial.print("Pressure = ");
  Serial.print((bme.pressure) * 0.000010197162, 4);
  Serial.println(" atm");
  Serial.print("Humidity = ");
  Serial.print(bme.humidity);
  Serial.println(" %");
  Serial.print("Gas = ");
  Serial.print(bme.gas_resistance / 10000.0);
  Serial.println(" KOhms");
  Serial.print("Approx. Altitude = ");
  Serial.print(bme.readAltitude(SEALEVELPRESSURE_HPA));
  Serial.println(" m");
  Serial.println(F("-------------------------"));
 
 // initialize radio
  Serial.println("Radio init");
  int16_t state = radio.begin();
  if (state != RADIOLIB_ERR_NONE) {
    Serial.println("Radio did not initialize. We'll try again later.");
    goToSleep();
  }

  node = persist.manage(&radio);

  if (!node->isActivated()) {
    Serial.println("Could not join network. We'll try again later.");
    goToSleep();
  }

// If we're still here, it means we joined, and we can send something

// Manages uplink intervals to the TTN Fair Use Policy
  node->setDutyCycle(true, 1250);

//create LoRa data uplink structure and packet
  uint8_t uplinkData[7];
  uplinkData[0] = (count++); //sequantially adds one intiger to the value 'count' for each transmission, resets when RST button pushed
  uplinkData[1] = (temp * 4); //max value to multiply by and keep under cap of 256 for uint8_t value
  uplinkData[2] = (bmet * 5); //max value to multiply by and keep under cap of 256 for uint8_t value
  uplinkData[3] = (bmeh); //no math needed
  uplinkData[4] = (bmep * 0.0010197162); //converts to atm * 100 to keep two decimals
  uplinkData[5] = (bmea / 16.09); //converts to miles * 100 to keep decimals
  uplinkData[6] = (bmeg / 10000.0); //converts to kOhms

//define LoRa downlink size
  uint8_t downlinkData[256];
  size_t lenDown = sizeof(downlinkData);

//Send of the LoRa data and establish wait for downlink
  state = node->sendReceive(uplinkData, sizeof(uplinkData), 1, downlinkData, &lenDown);

  if(state == RADIOLIB_ERR_NONE) {
    Serial.println("Message sent, no downlink received.");
  } else if (state > 0) {
    Serial.println("Message sent, downlink received.");
  } else {
    Serial.printf("sendReceive returned error %d, we'll try again later.\n", state);
  }

  goToSleep();    // Does not return, program starts over next round

}

void loop() {
  // This is never called. There is no repetition: we always go back to
  // deep sleep one way or the other at the end of setup()
}

void goToSleep() {
  Serial.println("Going to deep sleep now");
  // allows recall of the session after deepsleep
  persist.saveSession(node);
  // Calculate minimum duty cycle delay (per FUP & law!)
  uint32_t interval = node->timeUntilUplink();
  // And then pick it or our MINIMUM_DELAY, whichever is greater
  uint32_t delayMs = max(interval, (uint32_t)MINIMUM_DELAY * 1000);
  Serial.printf("Next TX in %i s\n", delayMs/1000);
    Serial.println(F("*****-----*-----*-----*-----*-----*-----*-----*-----*-----*-----*-----*****"));
  Serial.println();
  delay(100);  // So message prints
  // and off to bed we go
  heltec_deep_sleep(delayMs/1000);
}

Here is my JavaScript payload formatter for my LoRa uplink data into TTN. I do a bit of math on the bytes to undo the processing I attempted in ArduinoIDE. This is all to convert units and preserve decimal places.

function Decoder(bytes, port) {
  var decoded = {};

  // Decode bytes to int
  var testShort0 = (bytes[0]);
  var testShort1 = ((bytes[1] / 4) + 21);
  var testShort2 = (((bytes[2] / 5) + (9/5)) + 32);
  var testShort3 = (bytes[3]);
  var testShort4 = (bytes[4] / 100);
  var testShort5 = ((bytes[5] * 16.09) * 3.28084);
  var testShort6 = (bytes[6]);
  
  // Decode int 
  decoded.count = testShort0;
  decoded.temperature = testShort1;
  decoded.bme_temp = testShort2;
  decoded.bme_hum = testShort3;
  decoded.bme_pres = testShort4;
  decoded.bme_alt = testShort5;
  decoded.bme_gas = testShort6;

  return decoded;
}

Here are my questions:
(1) Do I need to run both Wire and Wire1 on this board since the onboard OLED is hardwired into the primary I2C bus? I think I do, but would love to confirm.
(2) I am trying to send temperature data and keep two decimal places, but with uint8_t data it is only able to reach values of 256, or so I understand. I would like to multiply the raw value by 100 to preserve the decimals, this is what I see elsewhere, but then the values become too large. Is there a way to send them in a larger byte, uint16_t? or format them into two bytes of uint8_t and then recombine them in TTN using the uplink formatter?
(3) I have read a bit about CayenneLPP formatting and believe it might work better than JS since I believe it would allow me to assign units to each value and they would be preserved all the way to HA, removing the need to create a helper for each value to give it units. Is this correct? If so, I struggled with trying to merge the Cayenne formatting (from examples) into this Arduino sketch and gave up. Any pointers or small bits of sketches that would accomplish this would be greatly appreciated.
(4) I have seen that sometimes the formatting of the uplink data “cloggs” the airwaves with needless transmissions and confirmations, I want to confirm that I am not doing this. It appears not since my consumed airtime is 57 ms, but I’m also looking to keep airtime as short as possible to save power.
(5) Not really a question, but I am planning to add a downlink transmission to these boards to control a relay. I plan to simply start with an example code to add the relay, but I’m not sure how to setup that downlink. I’ll look into it on the HA side and work my way back to the Heltec board, but you may have a link to share already, if so I’d be in your debt.