TCP connection not always reading data, data always being generated.

I have two TCP connections between matlab and Unity. I send data when certain events occur in unity. Connection 1 covers event 1, connection 2 covers event 2. These events occur multiple times. The two connections are over different ports. I've tried creating the connections using the IP address, "localhost", and "0.0.0.0" and my issue persists. I have also tried using Matlab 2023b and Matlab 2019a. Matlab is able to successfully read the data from most events, but it sometimes simply misses the data being sent. For example, I get data over connection one for events a, b, and d, but I do not get the data from event c. I have try/catch blocks set up but there is no error, the data is simply not detected. The data is being accurately generated on the unity side. I have explicitly set the buffer size to be high, and inserted pauses between event occurence, but the issue persists. I am happy to share my code if needed, but I think this issue stems from a fundamental misunderstanding I have of TCP connections. Is there ever a reason that the connection would miss receiving a piece of data? Thank you in advance for your help!

[Walter Roberson]

... There is always the possibility that the MATLAB TCP drivers are simply losing data :(

[Hassaan]

@Sophie Dewil What is the periodicity/frequency of sending data?. Try capturing a session using wireshark.

I once did a project communciating between a Microcontroller(STM32) - Client and MATLAB - Server with data sending frequency every 200ms and the results were good. It was MATLAB r2021b at that time. I am not saying issue is with r2019 or r2023. A code snippet of your codebase may also provide more insight.

In the end the problem could be with MATLAB TCP APIs themselves.

References:

Troubleshooting TCP/IP Server Interface - MATLAB & Simulink (mathworks.com)

TCP/IP Communication - MATLAB & Simulink (mathworks.com)

Communicate Between a TCP/IP Client and Server in MATLAB - MATLAB & Simulink (mathworks.com)

TCP/IP connection is very slow when using fscanf to read from other computer - MATLAB Answers - MATLAB Central (mathworks.com)

Tcpclient function very slow. - MATLAB Answers - MATLAB Central (mathworks.com)

-----------------------------------------------------------------------------------------------------------------------------------------------------

If you find the solution helpful and it resolves your issue, it would be greatly appreciated if you could accept the answer. Also, leaving an upvote and a comment are also wonderful ways to provide feedback.

It's important to note that the advice and code are based on limited information and meant for educational purposes. Users should verify and adapt the code to their specific needs, ensuring compatibility and adherence to ethical standards.

A multiverse of answers and solutions.

Professional Interests

• Technical Services and Consulting
• Embedded Systems | Firmware Developement | Simulations
• Electrical and Electronics Engineering

Feel free to contact me.

Answer 1

[Hassaan]

@Sophie Dewil Some of the possible troubleshooting steps:

1. Buffer Overflow

• If the receiving buffer in MATLAB is not read quickly enough, new incoming data can be lost if the buffer overflows. This can happen despite setting a high buffer size if there's a delay or block in your code that prevents timely reading of the buffer.
• Ensure that your MATLAB code continuously polls or reads from the TCP buffer frequently enough to prevent overflow. Using read operations in a loop or event-driven callbacks can help.

2. Timing Issues

• Sending and immediately closing a connection from Unity without ensuring the data has been fully sent or received can result in lost data. TCP connections require proper shutdown procedures to ensure all data is transmitted.
• Implement a handshake mechanism or a delay before closing the connection on the Unity side to ensure MATLAB has time to receive and process all sent data.

3. Blocking Operations

• If your MATLAB code is performing a blocking operation (e.g., waiting for a specific amount of data that never arrives), it might miss new data sent on another connection.
• Use non-blocking reads or specify timeouts when waiting for data. This approach allows your application to handle multiple streams of data more effectively.

4. Connection Interruptions

• Transient network issues or interruptions can disrupt TCP connections, although TCP tries to automatically recover and retransmit lost packets.
• Implement application-level acknowledgment messages. After receiving data, MATLAB can send a confirmation back to Unity. If Unity doesn't receive this acknowledgment within a certain timeframe, it can attempt to resend the data.

5. Application-Level Issues

• The issue might lie in how data is being interpreted or processed within your MATLAB application rather than the TCP connection itself.
• Add detailed logging on both the Unity and MATLAB sides to trace the data flow and identify where the loss seems to occur.

6. MATLAB TCP Object Configuration

• Incorrect configuration of the TCP object in MATLAB might lead to unexpected behavior.
• Review the properties of the TCP object in MATLAB, such as Timeout, InputBufferSize, and OutputBufferSize, to ensure they are set appropriately for your data transmission needs.

Debugging Steps

• Add detailed logging in both Unity and MATLAB to confirm that data is being sent as expected and to trace where it might be getting lost.
• Test with a simpler setup (e.g., sending constant or very simple data from Unity to MATLAB) to isolate the problem.
• Consider using a network packet sniffer (e.g., Wireshark) to verify that data is indeed being sent over the network. This can help determine if the issue is on the sending or receiving end.

References:

Troubleshooting TCP/IP Server Interface - MATLAB & Simulink (mathworks.com)

TCP/IP Communication - MATLAB & Simulink (mathworks.com)

Communicate Between a TCP/IP Client and Server in MATLAB - MATLAB & Simulink (mathworks.com)

TCP/IP connection is very slow when using fscanf to read from other computer - MATLAB Answers - MATLAB Central (mathworks.com)

Tcpclient function very slow. - MATLAB Answers - MATLAB Central (mathworks.com)

-----------------------------------------------------------------------------------------------------------------------------------------------------

If you find the solution helpful and it resolves your issue, it would be greatly appreciated if you could accept the answer. Also, leaving an upvote and a comment are also wonderful ways to provide feedback.

It's important to note that the advice and code are based on limited information and meant for educational purposes. Users should verify and adapt the code to their specific needs, ensuring compatibility and adherence to ethical standards.

A multiverse of answers and solutions.

Professional Interests

• Technical Services and Consulting
• Embedded Systems | Firmware Developement | Simulations
• Electrical and Electronics Engineering

Feel free to contact me.

[Walter Roberson]

By design, TCP is a "reliable" transport protocol. If packets get lost in transit, or packets get corrupted in transit, then TCP automatically retries until it receives the packet correctly (or declares the socket to be unrecoverable and closes the socket.)

This applies right to the packet buffering. If there is no room to accept packets (buffers are full) then TCP discards the new packets, and engages in recovery to get them back later.

A number of your proposed steps do not apply to TCP.

[Hassaan]

@Walter Roberson I do agree and I am not blaiming or targetting TCP only. There could be other application level issues and how the MATLAB code is written keeping in view the blocking and non-blocking nature of the existing code [MATLAB, Unity].

The steps are just a guideline and may not be applicable to each scenario. Few of the possiblities, I have faced myself and were helpful for my case.

If I missed something I do incourage to guide further. Positive feedback and recommendations are welcomed.

-----------------------------------------------------------------------------------------------------------------------------------------------------

If you find the solution helpful and it resolves your issue, it would be greatly appreciated if you could accept the answer. Also, leaving an upvote and a comment are also wonderful ways to provide feedback.

It's important to note that the advice and code are based on limited information and meant for educational purposes. Users should verify and adapt the code to their specific needs, ensuring compatibility and adherence to ethical standards.

A multiverse of answers and solutions.

Professional Interests

• Technical Services and Consulting
• Embedded Systems | Firmware Developement | Simulations
• Electrical and Electronics Engineering

Feel free to contact me.