USB Power versus Battery Power

I have a prototype Heltec ESP32 Lorawan w/OLED and I am running on USB power.
I need to test with battery and measure the battery voltage over time.
Do I connect the battery through the little connector on the back of the board? Or can I run battery power directly into the #V pin on the chip?
I need to be able to switch back and forth fairly easy?
I am using a rechargeable CR124 battery.

It can be connected directly using the interface on the back.