WiFi LoRa 32 (V3) - Encode/decode packets to binary (shorter)

#1

Could you please help me how to encode data e.g. from temperature and humidity sensor into binary form - to make the LoRa packet length shorter? Similar to what LoRaWAN does.

I use the official Heltec library, I only transfer data via LoRa (not LoRaWAN), so it is necessary to make it work with the function Radio.Send()

Radio.Send( (uint8_t *)txpacket, strlen(txpacket) ); //send the package out

I was looking at something like this: https://create.arduino.cc/editor/LogMaker360/770f60c9-52c5-4d29-b417-4804b278db20/preview

I would appreciate any code example compatible with Heltec library.

Thanks

#2

It’s the exact same as the example, you create an array, put the data in to it but instead of calling ttn.sendBytes(data, sizeof(data)); you would, possibly no surprise here, call Radio.Send(data, sizeof(data));

You may find https://www.thethingsnetwork.org/docs/devices/bytes/ useful.

#3 
float temperature = 21.56;
float humidity = 76.21;

Ok, thanks. Could you please write me exapmle to send these two values to be able to send them by Radio.Send()?

#4

Nope, there’s one in the link above if you read it.

#5

The linked code looks fairly terrible as an example.

You can properly get away with reduced precision on humidity and temperature and can send it as one byte.
If humidity is 0…100 than that fits in the lower 128
Temperature -50 to 50 (or whatever range you supply) fit in the upper 128.

Just make a function that can pack that ine two byte, and a function to split that back into two floats.

Step one: Setup requirements
Step two: Make test code and unit test that
Step tree: implement

#6

Ok, but I don’t know how to encode into the buffer…

Could you please write me exapmle how to send these values by Radio.Send() Heltec compatible function?
Because the function requires parameters: RadioSend( uint8_t *buffer, uint8_t size )

#7

In the example you linked to, buffer is called data, otherwise it’s all the same.

#8

Ok, so I tried this code for sender:

void loop()
{
	if(lora_idle == true)
	{
    delay(1000);
		txNumber += 0.01;
		sprintf(txpacket,"%d",12);  //start a package


    float celsius =  12.19;
    //float pressure = 998.76 / 100.0F;
    //pressure * 100;
    float pressure = 97.32;
    float humaidity = 67.24;
    
    //translate float value to int 16 bit *100 get rid of the .
     int16_t temperature = (int16_t)(celsius * 100);
     int16_t barometer = (int16_t) (pressure *100) ;
     //serialMonitor.println(barometer);
     int16_t humidity = (int16_t)(humaidity * 100);

    byte data[6];
    data[0] = temperature >> 8;
    data[1] = temperature & 0xFF;
    data[2] = highByte(barometer);
    data[3] = lowByte(barometer);
    data[4] = humidity >> 8;
    data[5] = humidity & 0xFF;
   
		Serial.printf("\r\nsending packet \"%s\" , length %d\r\n",data, sizeof(data));

		Radio.Send( (uint8_t *)data, sizeof(data) ); //send the package out	

    
    lora_idle = false;
	}
  Radio.IrqProcess( );
}

and this for receiver:

void OnRxDone( uint8_t *payload, uint16_t size, int16_t rssi, int8_t snr )
{
    rssi=rssi;
    rxSize=size;
    memcpy(rxpacket, payload, size );
    rxpacket[size]='\0';
    Radio.Sleep( );
    Serial.printf("\r\nreceived packet \"%s\" with rssi %d , length %d\r\n",rxpacket,rssi,rxSize);
    lora_idle = true;


    int myVal1 = ((int)(payload[0]) << 8) + payload[1];
    int myVal2 = ((int)(payload[2]) << 8) + payload[3];

    Serial.println("");
    Serial.println(myVal1);
    Serial.println(myVal2);
}

This seems to be working - I get these values on the serial monitor:

image
image.png787×486 8.19 KB

The functions highByte() and lowByte() on the sender’s side are useful, but splitting the payload to individual values on the receiver’s side is annoying.

Aren’t there any functions for ESP32 that can automatically decode the payload if I know that, for example, each value is a decimal number of length 2 bytes?

Thank you

#9

Maybe check out the Union construct. I use this to automatically store/retrieve various ‘values’ into/from byte arrays before/after transmitting/receiving the latter. Just be aware of the default boundaries (byte sizes) of the various data types you may choose to use.

#10

Ok, Can you please explain first what is the difference between data sent as text and in float format? How many bytes for the Heltec library does a single character of a char type variable occupy?

Foe example I stick to the 2 values - temperature and humidity and I would like to see the length comparisson between chars and floats.

Thanks

#11

You’ve done it! Yay!

Awwwwwwww, poor you!

As you build up a toolkit, it’s copyNpasta, takes seconds.

This isn’t just an ESP32 question, it’s more of a general Arduino library question. There is some attempts around, but as you are dealing with the details of your device, getting familiar with possibilities using plain C is so much better before you give your data to someone to mess with & for you to debug.

#12

Fine, can you please explain me this? Or which type of data encoding would you recommend me for Heltec board LoRa communication?

Ok, Can you please explain first what is the difference between data sent as text and in float format? How many bytes for the Heltec library does a single character of a char type variable occupy?

For example I stick to the 2 values - temperature and humidity and I would like to see the length comparisson between chars and floats.

Thanks

#13

The one you’ve implemented is absolutely fine and to reiterate, there is nothing special about the Heltec board in combination with LoRa, these are the techniques used in many many different systems, there is no secret sauce that for the Heltec LoRa32 V3 that comes with magic fairy dust that is being withheld from you. By restricting your research to just this board you potentially miss out on many many examples in tutorials all over the web.

Now you’ve got some data moving, you should re-review the link to Working with Bytes I gave above - you’ll probably pick up some more info in the light of your success.

#14

This is probably the simplest form of the sort of data structure that I use:

static const int PH_atmospherePayloadSize = 6;
union PH_atmospherePayload {
  uint8_t payloadByte[PH_atmospherePayloadSize];
  struct payloadContent {
    int16_t temperature;
    uint16_t pressure;
    uint16_t humidity;
  } recorded;
};

But this is not a ‘Heltec’ issue, as @nmcc says, this is all general [C] programming and you will learn a whole lot more by writing a few test sketches to read and write data to/from the data structure(s) that you think might be appropriate, and searching the huge volume of resources that is available to us all on the Web these days [starting with the link that @nmcc provided] for answers to any programming questions that might arise as a result.

#15

Ok, thanks.

If I consider a decimal number of float datatype E.Q. float celsius = 12.19;, it takes 4 bytes if I’m right.

And if consider the same value but in char array like:

#define BUFFER_SIZE 30 // Define the payload size here
char txpacket[BUFFER_SIZE];

sprintf(txpacket,"12.19");  //start a package
   
Serial.printf("\r\nsending packet \"%s\" , length %d\r\n",txpacket, strlen(txpacket));

Radio.Send( (uint8_t *)txpacket, strlen(txpacket) ); //send the package out

How many bytes will the message take if transmitted as a char array? 5 bytes?
Does every character in ESP32 world take one byte?

If yes, why is sending data in binary form better when it’s not much shorter?

Thank you

#16

I think there’s a bit more homework to do there… You are printing out the characters “1”, “2”, “.”, “1”, “9”. Five characters. Five bytes. Not a floating point number at all. It may have started out that way, but you’ve just converted it to a character representation of a floating point number correct to two decimal places. If you had sent the value in binary, you would have sent a value that was correct to many more than two decimal places. This may or may not be of any value, depending on the accuracy of your measurement in the first place.

But computers work with integers. A floating point number is just an approximation. Recognising this, I generally convert my measurements to integers (first multiply by 10 or 100 for accuracy to one of two decimal places respectively) and only convert them back to a floating point number for display purposes.

None of this is specific to any processor—these are features of every programming language I’ve ever used.

If you want to know how many bytes a particular variable takes, just use the Union construct I suggested, setting the byte [uint8_t] array to something bigger than you should need (I think 10 should be more than any standard data type uses). Assign the relevant value to the variable of interest, then print out the byte array. If you initialise the bytes to zeros, and assign an appropriate [non-zero] value to your variable, you will be able to see how many bytes have non-zero values and thus how many bytes are used by the variable in question.

#17

Asking how many bytes does a Heltec library use for a single character of a char type is almost unanswerable in terms of the basics of programming. And it’s not specific to the Heltec library, but I’m repeating myself now! The web is full of tutorials on the fundamentals of data types and how they are represented. Suffice to say, turning any number in to text is very inefficient and actually harder to process at the receiving end.

Which bit of

is unclear - you have done it, apart from continuously assuming that there is something special about Heltec’s LoRa communications.

As @UniquePete explains above there are other ways of doing things but if you look at the code you linked to in your first post and to the code examples in the link to Working with Bytes, you’ll see you’ve got the basics figured out.

Have you re-read Working with Bytes? Have you re-reviewed the original sample? Why do you think Heltec & specifically Heltec’s LoRa communications is so special?

I’m happy to take time out to help, but it’s important that you accept the advice you receive - otherwise you may find your next question gets far less input.