Skip to content

Node-to-companion frames fragmentation#2885

Open
HDDen wants to merge 3 commits into
meshcore-dev:devfrom
HDDen:maxframesize_fragmentation
Open

Node-to-companion frames fragmentation#2885
HDDen wants to merge 3 commits into
meshcore-dev:devfrom
HDDen:maxframesize_fragmentation

Conversation

@HDDen

@HDDen HDDen commented Jul 3, 2026

Copy link
Copy Markdown

Hello! This PR adds separation of packets transmitted from the node to the application.

Problem

We're using text type as an example, but it applies to all packet types: the firmware allows sending text payloads to channels with a limit higher than the original application (for example, with a senderName of 3 bytes, the allowed text portion in the official MeshCore app is 144 bytes; a third-party client can send 155 bytes), but its never have receive heard retranslations.
Messages with the maximum limit can be sent through third-party applications (for example, meshcore_open). However, if such a message is sent, it will actually go out into the network, be relayed, and reach the nodes, where it will be displayed by them. Sender will not hear the relays and will not know whether the message has actually propagated beyond their node.

However, even when sent from the official app, the actual byte limit at which it would receive a retransmission confirmation is 134 bytes (with a 3-byte name). If this limit is exceeded, no confirmation will be received. Again, the problem manifested itself in messages from channels—it's possible that the transition from 134 to 135 bytes of text could, due to the AES block boundary, immediately increase the encrypted payload by 16 bytes, pushing the companion frame into the problematic range.

The root of the problem is that when receiving its own packet from the air (for retransmission confirmation), its information didn't fit within the limits specified by MAX_FRAME_SIZE=176. The data packet was larger and physically didn't fit within this limit, and could then either arrive corrupted or simply be discarded. Simply increasing this limit in the firmware could likely disrupt BLE operation with some devices, which is likely why such a conservative limit was chosen.

This PR added to node firmware fragmentation of packets sent from the node to the application. This way, we maintain the previous limit and eliminate the issue with sending large amounts of data exceeding it. For the fix to work, you need to make changes in the MeshCore application, also upgrading the companion protocol version to 14, otherwise the node will return data in the old mode.

Additionally, the limits were out of sync: the firmware assumed that frames up to MAX_FRAME_SIZE = 176 bytes could be sent directly, while the application and its TCP/USB/BLE decoder actually expected a maximum of 172 bytes. Therefore, frames of 173..176 bytes fell into a gray area: the firmware did not fragment them, and the application could discard them.

How to reproduce

  1. On the original firmware and application, set the node name to 3 bytes, and try sending a 134-byte message to the channel. Observe the retry counter increasing.
  2. Then increase the message size to 135 bytes and send. Next, you can extend it to the full official length of 144 bytes. Make sure that messages are received and retransmitted on the other node, but there's no retransmission counter on the outgoing node.

How to test the fix:

  1. Build the firmware with the changes from the PR.
  2. Build the app with the changes to the companion protocol (I'm attaching the apk at https://hdden.ru/mcoa-180-frames-patch.apk , but you can build it yourself from https://github.com/HDDen/meshcore-open/tree/mcoa-framed-messages ).
  3. In the app settings, enable "Settings - App settings - Enable Message Tracing."
  4. Connect to the node and send a message to the channel with the maximum allowed limit (in my case, it was 155 bytes for the text). Wait and watch the retransmission counter increase.

I'm attaching screenshots from the original MeshCore app and my fork of meshcore_open, "meshcore_open advanced".
In the fork screenshot: 1 and 2 are sent from the original firmware; retransmissions are displayed for 134 bytes, but not for 135. 3 and 4 are with the modified firmware and app; retransmissions are displayed even for the maximum allowed limits.

Screenshots of the original app: 1 - text length is 144 bytes, no retransmissions are visible; 135 bytes - same. 134 bytes - retransmissions are present.

Solution

Companion frame fragmentation has been added for the new version of the companion protocol (the version has been increased from 13 to 14). This mechanism is enabled only if the application has declared target protocol version '14' via CMD_DEVICE_QUERY. For all current applications working with protocol 13, the old logic has been retained without breaking changes.

Fragmentation is applied to frames that the node returns to the application and that exceed the expected limit. Packet types subject to fragmentation:

  • queued contact/channel messages via CMD_SYNC_NEXT_MESSAGE;
  • channel data receive frames;
  • raw data receive frames;
  • control data receive frames;
  • RF log frames.

A safe limit has also been added:

  • MAX_FRAME_SIZE = 176 remains the internal firmware transport limit;
  • APP_SAFE_FRAME_SIZE = 172 is used as the app-facing limit for protocol v14;
  • frames greater than 172 bytes are fragmented into PACKET_FRAME_FRAGMENT (0x91);
  • fragment frames themselves also do not exceed 172 bytes.

For CMD_SYNC_NEXT_MESSAGE, protocol v14 now includes fragment acknowledgement: after receiving a fragmented queued frame, the application acknowledges each fragment in the next CMD_SYNC_NEXT_MESSAGE. If the ACK is not received or does not match, the node repeats the last fragment. For protocol v13, the behavior of CMD_SYNC_NEXT_MESSAGE remains unchanged.

Screenshots 2 - comparsion between original and patched 3 - official app

@HDDen

HDDen commented Jul 3, 2026

Copy link
Copy Markdown
Author

Why this is important: initially, I wanted to explore the possibility of implementing Unicode string compression via BOCU-1 (very useful for, for example, Cyrillic messages, where each character takes up two bytes instead of one in Latin). The plan was to decode the received text directly on the node, so that to the application will be passed the final UTF-8, thus avoiding the need to update/code the aplication. But I ran into this limit: if the decoded text was longer than the limit (and most likely it would be), it simply wouldn't be possible to send it to the application.

So, first I decided to try to get around this limitation.

@HDDen HDDen changed the title Node to companion frames fragmentation Node-to-companion frames fragmentation Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant