WiFi LoRa with Arduino Sensors, RX/TX not working right?

#1

Hello everyone,

for my projects, I currently use sensors from DFRobot which I hooked onto a Heltec WiFi LoRa 32 V3. The measured data is sent via LoRaWAN. You can see the code here: https://github.com/JuliaSteiwer/IoT-Water-Quality-Monitoring/tree/main/Working%20Code. This code so far works reliably, there’s just one thing that has been an issue since day 1:

The sensors are actually all tested with Arduino UNO on DFRobot’s side, which means libraries and functions often do not work with the Heltec at all.

For me, that means lots of testing and debugging to get properly working code to control the sensors with. No “plug and play”, as you would have it with an Arduino. Just recently, I gave up frustrated on trying to run a sensor with the Heltec (spend > 5h trying to get somethin to work), and when I hooked the sensor to a Minima R4, everything worked perfectly immediately.

Now you might say “Julia, if your current code works fine, why bother changing anything?” but the thing is, with a new Heltec version available (Heltec WiFi LoRa 32 V4) and DFRobot releasing more and better sensors frequently, it’s neither efficient nor feasible to try and manipulate code for hours just to force it to work with a different chip. Improving the system doesn’t really work when more time is spent on fixing code compatibility rather than trying new things.

What do I want now? Proper deep sleep, LoRaWAN, and plug-and-play for the sensors, all while evading a cost explosion.

My ideas were the following:

  1. “Die eierlegende Wollmilchsau”: The MCU that has it all and solves it all, which… apparently does not exist (or maybe it’s just the enshittification of online search). You can get (a) Arduino without any of the other things, (2) Arduino with LoRa but not Deep Sleep, or (3) ESP32-based with Deep Sleep and LoRa but Arduino-specific libraries won’t work. But not something that has it all.
  2. Heltec + Arduino. Use the Arduino for triggering the sensors only. Use wire to get the sensor data from the Arduino to the Heltec. Use the Heltec to control Deep Sleep and communication. Using a cheap Arduino and HeltecV4, I’m at 5€ more with my usual shop than with just the V3.

So, I tried idea 2 with the LoRa V3 and the Arduino Minima R4 I currently have at home. Just for testing, I kept it as simple as possible: Have a code for the Arduino that prints something, sends that via RX/TX to the TX/RX of the Heltec, then the Heltec gets a code that prints the Arduino message. Despite how easy that should be, it’s not working. Looking at the pin-out, I should be using GPIO43 for RX and GPIO44 for TX on the Heltec. I wired the Arduino’s RX to the Heltec’s TX and the Arduino’s TX to the Heltec’s RX. It does not work. I made sure that their GNDs are connected and that both are supplied with power. At some point, my PC outright refused to connect with the Heltec anymore, resulting in the dreaded Failed to execute script 'esptool' due to unhandled exception! Failed uploading: uploading error: exit status 1 error message.

Out of despair I then tried asking “AI” chatbots (LLMs are essentially just glorified auto-correct but at some point in despair…). Outstanding results, they invited GPIOs that outright do not exist, completely ignoring pin-out diagrams, data-sheets and schematics.

In my plight, I return here with the question: How can I transfer data from an Arduino board to a Heltec board?

The logic I’d like to achieve is:
1. Heltec board (HTB) wakes up from deep sleep after time t.
2. HTB powers the Arduino board (AB).
3. AB powers the sensors and gets data from them.
4. AB sends data to HTB.
5. HTB sends received data via LoRaWAN.
6. HTB goes back to sleep, powering off the Arduino.

How would I implement such a solution? Or is there a better way?

Please let me know your ideas and insights, I’m looking forward to and being grateful for all of them!

#2

Avengers, assemble! Julia has returned :smile:
(I’m building a bathroom closet today so give me some time…)

You can watch this in the meantime:

1 Like
#3

Aha yea, with the questions I ask here you’d need them definitely… :sweat_smile:

If teleportation was a thing I’d just say, let’s switch the tasks, I build the closet, you can go wild with my hardware and code. Definitely watching the video later, must have been amazing presenting at TTI’s conference! Thanks a lot! :blush:

#4

What sensors do you wish to connect ? give links?
or what do you wish to measure? e.g. temperature (what range), voltage, current, pressure, pH, etc

#5

Hi, the GitHub repo I linked has the sensors linked in the READ.ME file.

#6

The reason why I linked that clip is because we really want you to use the default Arduino framework. Because yes, option #1 really exists and it can be the Heltec hardware without too much troubles. But it is the Heltec framework that sucks in so many unexpected places and always runs behind on the actual Arduino-ESP32 framework. Using the normal framework means you will have to get rid of LoRaWan_App and use RadioLib instead. A year ago you totally didn’t have the brain capacity for that. Given that you opened this topic shows that you now have. And RadioLib is sooo simple, as demonstrated in that video. Please watch it, please try RadioLib with the normal Arduino-ESP32 framework.

Try not to use it with the V4 as your first board but with a V3. The V4 needs more control for the additional RF components and should actually not be used at all in the EU right now. If you get stuck, feel free to open a discussion on the repository. We’re more than happy to help.

And you’ll find out that with the normal framework, using all sorts of sensors really is going to be easier. Maybe not fully Arduino-easy, but the ESP32 isn’t popular without reason.

#7

Thank you, I will take an in-depth look into the RadioLib library and try to implement it with my current sensor suite.

And I’d greatly appreciate it if next time you could not add such a comment. Regardless of your intention, it comes off as insulting.

#8

Sorry, didn’t mean to offend you! It’s something I regularly say about myself when I’m under pressure, didn’t think about how it would sound to you. I was actually trying to be positive by seeing you’re freshly revisiting the topic.

Not yet, not yet! Just try out the Starter example, try a simple BME280 if you have it around, leave it for a couple of days, try the Reference and/or deepsleep. Get comfortable with it before diving too deep into combining it all :slight_smile:

#9

I will start with a DS18B20 that I have around, it should cause the least of problems. I’ll have to wait for a LoRaWAN gateway, hopefully it’ll arrive by next weekend. I’m already using RadioLib persistence instead of LoRaWAN_App in my code, maybe by checking all of RadioLib’s options I’ll find out why some of the Arduino-based sensor libraries are still not working for me.

#10

I think he meant experience not capacity, but still, he’s a very naughty boy.

It is well supported but does have some wrinkles with the IO due to timing issues.

Huh? If you are using that, you are already using RadioLib as you can’t use that code without using RadioLib. I personally know the person that wrote that sketch and I believe the person that wrote config.h & notes.md is totally amaze-balls who in his spare time hand feeds orphaned kittens.

Create single sketches for each sensor to test them on the board. The implementation of LW along side is totally seperate.

Overall, I think one aspect of the development experience you may have not fully embedded is looking around, reading each line of the code you are implementing, standing back and looking at the landscape. The last item particularly as the code for RL persistence literally references RadioLib: https://github.com/radiolib-org/radiolib-persistence/blob/5559cead41445a7436dac04ff449546498143fe8/examples/LoRaWAN_ESP32/LoRaWAN_ESP32.ino#L36

CopyNPasta at speed makes for climbing the wrong mountain. Taking known good foundations, which is what is supplied there, and adding in your sensor reading code and then building a payload is exactly what @CaptainAwesome, responsible for config.h and Notes.md would want for the user base.

#11

Mea culpa, mea culpa. I was under the impression that you were still using the Heltec framework. I hadn’t looked at your code before responding, I thought I remembered it from a year ago. I now better understand why my messages must have come across the way they did. Sorry.

So a few questions to get things straight before I start saying more weird stuff:

  1. Is this currently achieved with the code you linked? What is the sleep current?
  1. I guess this is working properly now, or is it not?

…

  1. What sensor was this? This is bordering an unreasonable amount of time indeed if it works out-of-the-box on Arduino.

  2. What board do you select in the Arduino IDE? Please tell us the selection as per the Tools menu, not the search box in the top left. Because the Tools menu will also tell you which package you use (Arduino-ESP32 or Heltec-esp32).

I’ll tell you in advance that using an Arduino Uno (R4) for your sensors is going to ruin, obliterate and shatter any hopes of deep sleep. Have you Googled its power consumption?

Really, it can work with Heltec without an extreme amount of effort. Here’s my project using a Tracker and a buttload of sensors: https://github.com/Ichthus-College-IN/MJLO-firmware. It uses two SPI devices, a UART device, five I2C sensors and two interrupts. Admittedly that has pushed me into the danger-zone with the available pins, but with the Espressif table and the Heltec schematic open, it is quite clear which ones should and should not be used.

#12

@nmcc @bns

Thanks for your replies, I’ll try to keep my answer as short and concise as possible. 🫣

Am I already using RadioLib?

Yes. Back then, as detailed in that other mammoth thread, I went with Persistence after the both of you had recommended it. Mainly because we agreed that persistence prevents legal issues with duty-cycling that might arrive due to overly frequent reconnections rather than simple uplinks.

What is working?

The following things work beautifully, like a Swiss clock:

  • LoRaWAN: Once the gateway is set-up (can be an issue with some manufacturers who want you to use their [pay-walled] TTN „alternatives“), everything works smoothly. Package loss is only an issue when the WiFi the gateway is connected to is weak.
  • Deep Sleep: You can practically hear the board snoring. I have not yet measured the actual sleep current but with a 4400mAh battery and two 7.2 V 200 mA solar panels, it just kept going even here in cloudy Northern Germany. In a run-time of several months (read: over half a year; and with the node in a shadowy place), the people I worked with only had to swap for a freshly charged battery twice. Even before using persistence and just having deep sleep with the Heltec LoRaWAN library, power consumption was quite low. Combine RLP with that amazing 5V boost converter with true shutdown that Steven recommended, it works like a charm.

What am I missing?

Quick and simple sensor usage. I know, plug-and-play is something everyone always rolls their eyes at but it is possible and development-wise, one should strive for it. The specific sensor I was wrestling with is this one here from DFRobot but the problem has existed with multiple of their sensors, especially those relying on libraries provided by DFRobot. On average, this is what happened when I used their sensors (including the linked one, which would, if I got it to work with the Heltec board, solve several problems at once):

  1. I take the sensor and hook it up to an Arduino board to check general functionality.
  2. Everything works. The values are correct or in acceptable/plausible ranges (might need calibration). The full functionality from the DFRobot library is given.
  3. Take the exact same .ino code, make changes in required places (baud rate 9600 -> 112500, ADC 5000-> 3300 etc.).
  4. Nothing works anymore because the library is Arduino compatible only.
  5. Grab the usually <10 lines from the library that are required for the code to run and hard-code them into the main .ino file.
  6. Sensor now works with the Heltec board but some very neat functionalities (e.g. automatic calibration) are now missing.

With the linked sensor, it was especially frustrating: Whatever I tried did not work with the Heltec. Plugged into the Arduino though? Working immediately. (This made me so mad I had to walk away from the table to calm down tbh.)

What was my process arriving at my code?

  • Every sensor gets tested individually. One sketch for each sensor, really testing sensor function only. Nothing else.
  • Slowly bringing the sensor codes together. I usually start with temperature only, check that it works. It does? Add pH code and check again. That works? Add next sensor. Sensor by sensor until I have one code controlling all sensors.
  • RadioLib Persistence: Set up everything with the deep sleep, then start by sending constant values over LoRaWAN.
  • Once that worked, I added every sensor individually into the code. Starting with temperature. Its values appeared in TTN correctly? Cool, add the next sensor and continue like that.
  • Arrive at one big code that has RadioLib with persistence, deep sleep, and sensors.

Which board do I select?

Heltec WiFi LoRa 32(V3)

What is installed in the boards manager?

  • Arduino AVR Boards (by Arduino, v1.8.7)
  • Arduino UNO R4 Boards (by Arduino, v1.5.3)
  • Esp32 (by Espressif Systems, v3.3.7)
  • Heltec ESP32 Series Dev-boards (by Heltec Automation, v0.0.7)

Arduino ESP32 Boards (by Arduino) is not installed but I feel like it’s staring at me wanting to be installed…

So, what are the issues right now/what would help?

Sensors that work out-of-the-box with Arduino only work with Heltec after taking apart their libraries and sacrificing very neat functionalities just to have them running. There has to be an easier way. I would like to get closer to a plug-and-play experience so I can focus on analyzing data rather than fixing code bugs or electronics issues. A framework or working solution would help a lot. It’s easier for me to learn how something works by looking at a working example and taking it apart to check how the maker arrived there than trying to piece everything together without having a working example I could come back to.

That was not very short… perhaps not even concise.

Here’s your TL;DR:

:white_check_mark: LoRaWAN (RadioLib Persistence)
:white_check_mark: Deep Sleep & Low Energy Consumption
:x: Easy and flexible code implementation of new sensors.
🫵 I suck at guessing games. Please just tell me directly what you want me to know/do because I won’t guess it in a thousand years. Which means you’ll get frustrated with me and I feel like a fool–a lose-lose situation, if you will.

#13

This just isn’t a thing. Arduino compatible means it works with the Arduino system - not just ones with the official Arduino logo on them.

There is very little that I’ve found that is board specific - and certainly nothing that is particularly vendor specific given how hard it is to detect who made the board.

I suspect the issue is with pin choice on the ESP32, particularly with a non-mainstream sensor that works over ModBus. I have projects with multiple revisions of spreadsheets used for pin allocation plus lots of prototype boards with patch wires as I discover issues with what combos will work reliably.

The Arduino dream at the Uno level of providing a system that allows simple sensors to be dropped in to place and then flash an LED if a reading goes over a certain value is fine. But you aren’t on the ground floor, you are way up high, you’ve LoRaWAN, deep sleep and a massively more complex microcontroller with two cores and multiplexed IO with optimisation paths for the internal peripherals. Which is why I said the DS18B20 isn’t necessarily as simple as you’d like as some of the IO is going to be glitchy as it runs through the cross-connections whereas others will be fine. But then you have to look at all the other sensors you have.

I’d also note, if you have $200 Dissolved Oxygen sensor, resenting having a $5 daughter board to interface the sensors seems a bit incongruous.

So the short answer is to stop thinking that this stuff is easy, because you aren’t operating at the entry level by any means. And I suspect you need to stop hammering away at things to make them work - if they don’t work, take a step back to document & review and maybe ask a real person for some input, not AI which is an average of the published content of the internet of people that got an LED to flash on an Arduino once and present themselves as experts.

I’d definitely skip the whole ‘does this work on a totally different board’ and just test it out on the Heltec. Alternatively, you could interface a LoRa radio to a Minima R4 if you are finding that works well with your sensors.

#14

Hi Nick, that specific sensor does not have a specific library, that’s true. Some of the sensors I’m using/have used do have specific libaries, namely:

And for all of them, connecting them to an Arduino board and uploading the code resulted in everything working immediately, while using them with the Heltec board (and yes, I made sure to change relevant parts in the main code) had issues, mainly with functions from the library not being called right. It’s been ages since I last looked at the libraries, and there was a comparison logic checking what board you were using. One of those repos even had a list showing that they tested an ESP32, it failed, and they just left it at that.

Some more things I’d like to clarify:

For the $200 Oxygen sensor, I was not too cheap to get a converter; in fact, I have the one from DFRobot specifically made for their ModBus sensors. I connected everything as per plan, which worked on the Arduino. Changing code and pins for the Heltec, it did not work. Pins used: RX/TX (GPIO43/44, resp.), then GPIO47/48. Conv_RX to Heltec_TX, Conv_TX to Heltec_RX (and vice versa because there was no proper reaction and I was curious as to what it would do). I got more of a result for the Heltec board using GPIO16/17 (despite 16 not existing and 17 being for OLED) (some text was displayed but time-out for values), than with the RX/TX pins (nothing shown in the Serial Monitor).

I’m not talking about the high-level stuff right now. I’m talking about getting the sensors to work out-of-box the way they do for the Arduino boards. For testing the sensor to compare it to the other oxygen sensor, I do not need the high-level stuff right now; the problem is failure at the low-level (getting the sensor to run).

The 5 hours were about the time I spent working with this sensor before I opened this thread. In fact, I had spent almost 3 week-ends trying different things, reading through other forums, testing different approaches, reading through tutorials and knowledge bases, taking voltage measurements etc. before I finally caved out of frustration, asking „AI“ even though I knew it would yield no results–it was just a last attempt out of frustration, hoping to get either a new angle or such an obviously false answer that the solution to my problem would fall onto me like that apple on Newton’s head.

And the reason why opening this thread was last on my options: Every time I come here, whether it is for my problems or just to check what others do, there’s a way of answering that provides more of a brain-nudge than an answer. Sometimes it feels like text is even being ignored or others are assumed to be „too lazy“ to dive deep. I’m not saying this to offend or insult you but this sensor node is a means to end for me, namely getting data which I can used for modelling and simulation. Currently, sensor compatibility failing at low level is pushing back the data acquisition, which causes delays for all the other things. I’ve had issues with a supervisor because she believed all of this should work plug-and-play (despite giving others 10+ years to develop a worse Heltec board but alas), and so many stakeholders (especially end-product-users or those financing stuff) think the same way. I’m getting frustrated and starting to lose that development passion because of these issues, both with people who refuse to show an ounce of understanding towards the difficulties associated with development and due to these hardware issues. I know that in the ideal case, I would learn all of this over months and years and then maybe become a crack like you and Steven one day but right now, my priority is simplifying sensor implementation to the point that I can (1) easily make and deploy tons of nodes so that (2) I can get the data I need and start the work I actually want to do.

I hope this doesn’t sound like a tantrum but I’m seriously frustrated by these frequent sensor issues. What used to be fun now just feels like a chore. And other issues in life coming in on top of that really make me want to hurl all this stuff into a corner on some days.

#16

Trying to add and rephrase something about Tx and Rx and Serial and CDC but either I’m saying forbidden character combinations or the conference place WiFi is too bad. Some stuff is worded suboptimal but can’t edit the way I want it.

#17

Disclaimer: I’ve read your post in full and am not intentionally ignoring what you write. Keeping it short to prevent going on and on about the high-level stuff :slight_smile:

A good way to move forward could be to open a specific discussion about such an issue with the Modbus sensor. Maybe Nick has some Modbus stuff laying around, I can order something Modbus no problem, and we’ll get your problem fixed.

This may not be the solution that you want, because you want to be able to “fix” any sensor yourself. The problem however is that if you want a fix for any sensor, we need to teach you how to fish, i.e. nudge your brain. That is what Nick and I (try to) specialize in. Heck, it’s even my job as a teacher. But this high level stuff doesn’t directly translate into resolving the issues with the sensors on your desk, because it’s something that percolates only through the years unfortunately.

My recommendation to you is that you open up discussions one by one with specific problems. We can help you directly with the problem (like using Modbus / finding the appropriate pins) and along the way you can pick up the experience of how these problems are resolved. Pin 43/44 for instance aren’t easy as there’s a lot of things going on with Serial on the ESP32-S3. We can discuss that in a specific discussion and you’ll suddenly find yourself understanding much more about Tx/Rx/D+/D-/CDC/Serial.

Would such a way forward suit you? Or do you prefer something else?

1 Like
#18

I have done some stuff with the RS485 [Modbus] interface (see here) but not to date, unfortunately, with ESP32 processors (although that is on my To-Do list). While the discussion under that link only (for the moment) discusses the issues around using CubeCell (single UART) and CubeCell Plus (double UART) dev boards, it may provide a ‘brain-nudge’ in relation to what might be going on with the ESP32. I thought that I’d looked briefly at the ESP32 boards and that extending this work to them wasn’t going to be a big deal, but maybe I’m in for a surprise…

I do also have pH and turbidity sensors on my desk, waiting to move up my To-Do list and, if it helps, I’ll make a point of also testing an ESP32 configuration when they get their turn. Unfortunately, that probably won’t happen this week, but soon maybe…

I use a range of MCUs in my (just straight LoRa, not LoRaWAN) applications, including the Arduino Pro Mini, NodeMCU, Heltec CubeCell and various ESP32 boards, although mainly the Heltec WiFi LoRa 32 and Wireless Stick Lite boards. I too am interested in ‘portable code’ and, once the fundamental differences are sorted, generally don’t have major problems switching between these, the capabilities of individual MCUs notwithstanding.

1 Like
#19

@bns I think it’s for the best right now if we proceed with me opening threads issue by issue. I absolutely want to learn new things to become an ESP32 god myself (like you and Nick) but right now, frustration with this sensor is a bit in the way of the joy of learning (like the olden school days, when our parents said “you can [insert partaking in fun activity] after homework” :sweat_smile:).

I will open a new thread for the specific issue on Saturday, when I have access to my board and sensor again, so I can provide you with IDE outputs, screenshots, pictures of the set-up etc. to ensure that any information that is helpful is available.

@UniquePete I’ll check that out, thank you for the link!

Here’s a “fun” fact for anyone reading this thread: I asked the manufacturer about ESP32 compatibility. From their reply: “in principle it can be used with ESP32-based controllers. However, we currently do not provide dedicated example code for ESP32.” which is “fun” because (1) this has been an issue with multiple sensors, sometimes they even tested with ESP32, got failure and then just didn’t try fixing the error, and (2) they have dedicated ESP32 and even ESP32+LoRaWAN libraries, which tells me they might even sell something ESP32-based with LoRaWAN… :sweat_smile:

#20

I think you are still failing to get the message.

Why? My understanding is that you “focus on coastal morphology and river health”. This expertise does not speak to becoming proficient at developing firmware for a microcontroller over LoRaWAN. There is no reason you shouldn’t be able to hack on an existing setup, but building one from scratch has many moving parts requiring a considerable amount of prior experience and doing it frequently enough to be in the zone.

As I said in a prior post, Arduino marketing is good for punting the idea of accessibility to maker electronics for all. This leads to some myth that it’s just some sort of Lego and you can build a complex system by slotting things together. This is a common trap that many academics fall in to, usually a senior staff member who then puts pressure on to a colleague to get something done that is outside the skill set & experience of the discipline that they are all involved in. In some cases the expectations fall under the heading of bullying and illegal.

This issue has come up several times on the TTN forum. I don’t understand why a collaboration isn’t formed with a university Electronic Engineering dept.

I have built a water monitoring system. I point blanked refused to become involved in the calibration of most of the sensors which included Ion Selective Electrodes / Ion Exchange Membrane sensors for two very good reasons. The first is that I totally failed my Chemistry A Level (national exam for 18 year olds) and second, I had enough to do with mechanical mounts despite that not being my thing either, without getting involved in mixing liquids in certain ratios and sticking sensors in to them.

Figuring out the impedance of the different ISE probes and designing & building interface circuits to read the voltages via an ADC was interesting. But I’m more a digital electronics & firmware man, so the analog stuff was a handful.

The client had a really cool & helpful calibration recording device to give me the info I needed, partly because it was out of their understanding what I needed.

I did OK with my Maths A level, but if you gave me some stats to process, it wouldn’t go well.

If you put the info in context at the “explain it to me like I’m a five year old” I can grasp the usefulness of a monitoring device. I’m also old enough to get the why of monitoring nitrates in water courses, capturing water levels and rainfall data to then figure out if a farmers overuse of fertiliser during a period of rainfall resulted in a rise in the nitrate levels. And I can capture that data. And I can do a simple graph to show some correlation. But the refined details aren’t in my range.

So please cut yourself some slack. There’s no reason why you shouldn’t over time get good at building devices - perhaps even going on to teach other environmental scientists how to do it - but you can’t suddenly leapfrog your experience and you particularly can’t do that with someone breathing down your neck which probably reduces your effectiveness by as much as 50%.

Why should they? There are dozens of microcontroller platforms around they can’t possibly support all of them or even come close to testing on each of them. And realistically, ESP32 is one of the last choices I’d make for a LoRaWAN device as it is stupid power hungry, so would be last in the list for testing. But great for a powered device that needs internet or BlueTooth.

If you make a sensor, you can’t possibly test it against even the common boards. And if you make a board, you can’t possibly test it against all the common sensors. Not without spending a lot of money which then has to be built-in to the cost of the sensors & boards.

It is the job of the embedded developer to build these things - perhaps with some pre-selection of microcontroller based on sensor support. But mostly that matrix doesn’t exist.

They are lying. Heltec try to create a Board Support Platform with LoRaWAN stack. Most of us use the standard Espressif BSP and RadioLib. Both a BSP and a LW stack are extremely non-trivial and not something they’d keep to themselves because they’d want to recoup their investment.


Helping you on a more detailed basis, no problem at all.

But it will be smoother if you could stop hoping that this stuff just works. It doesn’t and economically, it can’t.

#21

Just out of interest, I ran my RS485 test sketch up on a WiFi LoRa 32 V3 board today and it pretty much worked out of the box (HardwareSerial, Rx 47, Tx 48). If it helps, the configuration and sketch are now available through that link I provided in my previous reply.