Bluetooth looses data using library V3.0.4

[EricHarbers]

Board

ESP32

Device Description

The ESP32 chip is connected to a Bluetooth Serial device, eg a PC

Hardware Configuration

The ESP32 chip is connected to a serial device, using a UART

Version

v3.0.4

IDE Name

Arduino IDE

Operating System

Windows 10

Flash frequency

40 MHz

PSRAM enabled

yes

Upload speed

230400

Description

Today I installed the ESP32 library V3.0.4 for Arduino.
After I create a version of my project for the ESP32 project I use, my Bluetooth Serial communication looses data. The data will be transferred from a serial uart at 230400 Baud.
The problem started since the V3.0.4 version of the library. With previous versions I had no problems for Bluetooth.
I have seen that between the old version and the V3.0.4 version, the file BluetoothSerial.cpp has been changed significiantly.
With the V3.0.4 version, my bluetooth looses bytes at the speed of 230400 baud.

Sketch

/*
                              +------------------+
+-----------------------------| CPP SOURCE FILE  |-----------------------------+
|                             +------------------+                             |
|                                                                              |
|       project       : ESP32Control                                           |
|       filename      : Bluetooth.cpp                                          |
|       initiator     : Eric Harbers                                           |
|                                                                              |
+------------------------------------------------------------------------------+
Copyright (C) 2024, Enraf-Nonius, Rotterdam, Netherlands


 ________________________
| REVISION HISTORY       |
|________________________|_____________________________________________
|Version  |    Date      |     Revision by         |     Description
|_________|______________|_________________________|___________________
|  V1.00  |  2024-04-01  |  Eric Harbers           | * Original version
|  V1.01  |  2024-04-25  |  Eric Harbers           | * Check for 6 valid digits in BTConfirmRequestCallback
|         |              |                         | 
|         |              |                         | 
|         |              |                         | 
|_________|______________|_________________________|___________________

*/

#include "Bluetooth.h"
#include "MyomedProtocol.h"
#include "Message.h"

BluetoothSerial SerialBT;
t_Bluetooth bluetooth;
uint8_t ReceiveBuffer[512];

#ifdef MYOMEDPROTOCOL_ACTIVE
uint8_t mp_send_Buffer[1024];
uint8_t mp_receive_Buffer[1024];
MyomedProtocol mp_send(mp_send_Buffer, sizeof(mp_send_Buffer));
MyomedProtocol mp_receive(mp_receive_Buffer, sizeof(mp_receive_Buffer));
#endif

void bluetooth_connect();
void bt_spp_cb(esp_spp_cb_event_t event, esp_spp_cb_param_t *param);
void bluetooth_receiveBT(const uint8_t *buffer, size_t size);
void bluetooth_receiveSerial();
void bluetooth_receiveErrorSerial(hardwareSerial_error_t e);
uint8_t bluetooth_sendcommand(MyomedCommand &c);
uint8_t bluetooth_receivecommand(MyomedCommand &c);
uint8_t bluetooth_receivesample(MyomedSample &s);

void BTConfirmRequestCallback(uint32_t keyVal) {
  SerialBT.confirmReply(true);
  if (keyVal < 1000000) // must be max 6 digits
  {
    Serial1.printf("BTConfirmRequestCallback val %u\r\n", keyVal);
    bluetooth.keyVal = keyVal;
    bluetooth.status |= BLUETOOTH_STATUS_PIN;
  }
}

void BTAuthCompleteCallback(bool success) {
  bluetooth.status &= ~BLUETOOTH_STATUS_PIN;
  bluetooth.keyVal = (uint32_t)-1;
}

void bluetooth_init() {

  memset(&bluetooth, 0, sizeof(t_Bluetooth));
  bluetooth.keyVal = (uint32_t)-1;
  strcpy(bluetooth.localName, "Myomed");
  strcpy(bluetooth.remoteName, "raspberrypi");
  strcpy(bluetooth.pin, "1234");
  memcpy(bluetooth.check, "12345678", BLUETOOTH_CHECK_LEN);
  Serial.setRxBufferSize(1024);
  Serial.onReceive(bluetooth_receiveSerial);
  Serial.onReceiveError(bluetooth_receiveErrorSerial);
  SerialBT.register_callback(bt_spp_cb);
  SerialBT.onConfirmRequest(BTConfirmRequestCallback);
  SerialBT.onAuthComplete(BTAuthCompleteCallback);
  SerialBT.onData(bluetooth_receiveBT);
  SerialBT.enableSSP();
  SerialBT.begin(bluetooth.localName, true);
#ifdef MYOMEDPROTOCOL_ACTIVE
  mp_send.setCommandFunction(bluetooth_sendcommand);
  mp_receive.setCommandFunction(bluetooth_receivecommand);
  //mp_receive.setSampleFunction(bluetooth_receivesample);
#endif

  bluetooth_connect();
}

void bluetooth_connect() {
  if (SerialBT.connect(bluetooth.remoteName))
  {
    bluetooth.status |= BLUETOOTH_STATUS_CONNECTED;
  }
  else
  {
    bluetooth.status &= ~BLUETOOTH_STATUS_CONNECTED;
  }
  memcpy((void *)&bluetooth.macAddress, (void *)esp_bt_dev_get_address(), sizeof(bluetooth.macAddress));
}

void bluetooth_receiveBT(const uint8_t *buffer, size_t size) {
#ifdef MYOMEDPROTOCOL_ACTIVE
  mp_send.parse(buffer, size);
#endif
  Serial.write(buffer, size);
}

void bluetooth_receiveSerial() {
  size_t len;

  while ((len = Serial.read(ReceiveBuffer, sizeof(ReceiveBuffer))) > 0) {
#ifdef MYOMEDPROTOCOL_ACTIVE
    mp_receive.parse(ReceiveBuffer, len);
#endif
    SerialBT.write(ReceiveBuffer, len);
  }
}

void bluetooth_receiveErrorSerial(hardwareSerial_error_t e) {
  static char* errorList[] = {
    "UART_NO_ERROR",
    "UART_BREAK_ERROR",
    "UART_BUFFER_FULL_ERROR",
    "UART_FIFO_OVF_ERROR",
    "UART_FRAME_ERROR",
    "UART_PARITY_ERROR"
  };
  
//  Serial1.printf("error = %s\r\n", errorList[e]);
}

uint8_t bluetooth_sendcommand(MyomedCommand &c) {
  Serial1.printf("Send: ");
  Serial1.printf("Cmd = %X, ", c.getCmd());
  Serial1.printf("Data = %X\r\n", c.getData());
  return 0;
}

uint8_t bluetooth_receivecommand(MyomedCommand &c) {
  uint16_t cmd = c.getCmd();

  switch (cmd) {
    case 0x100:
      break;

    default:
      Serial1.printf("Receive: ");
      Serial1.printf("Cmd = %X, ", cmd);
      Serial1.printf("Data = %X\r\n", c.getData());
      return 1;
  }
  return 0;
}

uint8_t bluetooth_receivesample(MyomedSample &s) {
  static size_t cnt = 0;
  size_t i;
  bool res = true;

  for (i=0; i<2; i++) {
    unsigned int v = s[i];
    char low = v & 0xFF;
    char high = (v >> 8) & 0xFF;

    res &= (low == high);
  }
  if (!res) {
    cnt++;
  }
  else
    cnt = 0;
  return 0;
}

void bt_spp_cb(esp_spp_cb_event_t event, esp_spp_cb_param_t *param) {
  //Serial1.printf("Event = %u\r\n", event);
  switch (event) {
    case ESP_SPP_SRV_OPEN_EVT://Client connection open
      bluetooth.status |= BLUETOOTH_STATUS_CONNECTED;
      break;

    case ESP_SPP_CLOSE_EVT://Client connection closed
      bluetooth.status &= ~BLUETOOTH_STATUS_CONNECTED;
      break;
  }
}

void bluetooth_tick() {
}

Debug Message

I see no specific debug messages, but I loose data in my Bluetooth communication so that there will be a corrupted communication between my ESP32 and the PC. This was not the case with previous lib versions.

Other Steps to Reproduce

I have solved the problem by increasing the TX_QUEUE_SIZE in the V3.0.4 lib file BluetoothSerial.cpp. See line 44:
The original TX_QUEUE_SIZE was: 32
I have increased the TX_QUEUE_SIZE to: 128

This change solved my problem.

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[me-no-dev]

can you please provide a minimal sketch to reproduce the issue. One that we can compile and verify

[EricHarbers]

ESP32Control.zip
Attached the zip file containing my project.
In the zip file there is also the file sections.ld, in which I've added a section:
.version_info : ALIGN(0x10)
{

KEEP(*(.versionSection)) /* keep my variable even if not referenced */

} >default_rodata_seg
This ld file is placed in the arduino library map.

[EricHarbers]

MyoConnect.zip
Also the zip file MyoConnect.zip. Please use that on the pc and start MyoConnect.exe.
The PC bluetooth name can be any name.
In the Debug screen, you will see messages like: a 2-byte hex value and then: the command does not exist. (Maybe in dutch, i am not sure). This is the indication that the protocol has been disturbed.

[EricHarbers]

No, you miss the device that is serial connected to the ESP32.
It will not communicate now.
I'll change the code for you. Be patient.

[EricHarbers]

I don't know how to let you get our problem, because I miss a part of our device that is specific.
Can't you simply send data between the ESP32 and a PC using a bluetooth channel. The data should be read by the ESP32 from a serial input at 230400 bps. That is what my device does.

[EricHarbers]

You have to simply send data between the ESP32 and a PC using a bluetooth channel. The data should be read by the ESP32 from a serial input at 230400 bps. That is what my device does.

[EricHarbers]

The data loose is still coming, despite the bigger buffer I used
the problem started since version V3.0.4.
Can someone take a look for it?

[EricHarbers]

I've investigated some more time in the problem.
I'm back to ESP32 BSP V3.0.3. That solved all my problems with bluetooth and serial UART.

With BSP V3.0.4 I had the following problems:

• The serial UART looses data from time to time. (That was the problem mentioned above, for bluetooth (but it occurred on the UART))
• During Bluetooth communication, the ESP32 disconnected the bluetooth connection.
• When the ESP32 was paired with my PC, it couldn't connect with bluetooth. It gave Error 1 (unknown socket error) or Error 4 and stayed disconnected.
• Only with an unpaired ESP32 it was possible to communicate through bluetooth.

[me-no-dev]

there are no changes on our end between those versions, which suggests that maybe something in IDF changed. If so, maybe it's already fixed or will be soon fixed. Please try the new versions when they come out

[EricHarbers]

On your BluetoothSerial.cpp, I've seen a lot of changed code between V3.0.3 and V3.0.4.
I could see that because I have a copy of bluetoothserial.cpp V3.0.3.

I don't know the changes with serial uart.

[me-no-dev]

are you sure you've seen changes in BluetoothSerial between 3.0.3 and 3.0.4? 3.0.3 was released on July 17th and last change is in June 19th

[EricHarbers]

Dear me-no-dev,

Yes, I am absolutely sure that the code for BluetoothSerial has been changed between V3.0.3 and V3.0.4. That's why I initially came with the bufsize change.
With V3.0.3 it works fine.
Next week, at my work, I'll take an extra check.

[EricHarbers]

No, you have right.
I've checked the code of V3.0.3 and it looks the same like V3.0.4.
Sorry for the mistake.

But I don't know why V3.0.3 BT will work correctly and V3.0.4 BT not.

[me-no-dev]

Probably something in ESP-IDF driver changed. That is why I suggested for you to try new versions when we release them and see if the issue has been resolved