OSDev.org

I have hard times understanding what the TCP really represent and what they originally intended.

First which buffer do they intend here? We have transient buffers inside the network stack (similar to Linux sk_buff) whose are then are assembled and should be delivered to the user. The user process have a buffer where the received data should be assembled. So the question what the TCP window represent, it is the amount of total headroom of the entire stack + user buffer or only the user buffer?

When it comes to the user buffer it actually becomes hard to actually calculate the window depending on the type of system. My system is very asynchronous where received data is pushed to the user process and the data ends up in a buffer allocated from a dedicated buffer. With ring buffers calculating the remaining capacity is simple but isn't used here because of the inconvenience of the wrapping. The buffers are allocated with extra meta data and alignment so depending on how many packets and their sizes it becomes impossible to calculate how much buffer space is reaming. Also fragmentation can lead to that suddenly there isn't enough space for a large data so suddenly the TCP window can change.

What did they originally intend for this TCP window?
What is a good strategy for buffering which makes it easy to calculate the TCP window?

Hi,

OSwhatever wrote:What did they originally intend for this TCP window?

For server, the buffer is intended to allow data to be sent again if it wasn't received. For client the buffer is intended to allow the TCP/IP stack to re-order received data so that it can be given to a process in the right order.

OSwhatever wrote:What is a good strategy for buffering which makes it easy to calculate the TCP window?

This is always "MTU * window_size" for both sender and receiver. I'm not sure why "window size" is typically described in terms of packets (convenience or historical?).

OSwhatever wrote:When it comes to the user buffer it actually becomes hard to actually calculate the window depending on the type of system. My system is very asynchronous where received data is pushed to the user process and the data ends up in a buffer allocated from a dedicated buffer. With ring buffers calculating the remaining capacity is simple but isn't used here because of the inconvenience of the wrapping. The buffers are allocated with extra meta data and alignment so depending on how many packets and their sizes it becomes impossible to calculate how much buffer space is reaming. Also fragmentation can lead to that suddenly there isn't enough space for a large data so suddenly the TCP window can change.

I'm not sure if this has anything to do with TCP itself. Typically, for client, TCP/IP stack reorders data it received from network and then sends it to a process in the correct order (or puts it into a "user buffer" in the correct order).

I suspect that you might have asynchronous messaging where the OS doesn't guarantee that messages arrive in the order they were sent; and that everything that uses messages (keyboard, sound, file IO, ...) has to deal with messages arriving out of order; and that you're trying to use "message re-ordering" for "TCP re-ordering". In this case you have to have something to ensure that there isn't an infinite number of messages sent from TCP/IP stack (or keyboard driver, or sound card driver, or ...) that haven't been received by the process yet.


Cheers,

Brendan

Brendan wrote:I suspect that you might have asynchronous messaging where the OS doesn't guarantee that messages arrive in the order they were sent; and that everything that uses messages (keyboard, sound, file IO, ...) has to deal with messages arriving out of order; and that you're trying to use "message re-ordering" for "TCP re-ordering". In this case you have to have something to ensure that there isn't an infinite number of messages sent from TCP/IP stack (or keyboard driver, or sound card driver, or ...) that haven't been received by the process yet.

So far in my stack reordering is done by that it holds on to all the network packet buffers (small buffers, <= MTU size) until the full entire has been received. Then it pushes the data to the user process. In my opinion this is bad because then you hold on to the network buffers for longer periods, for these small buffers you want them to go through the stack as fast as possible so that they can be reused. However, since I don't have a user process buffer available in forehand, the stack has nowhere to put the assembled reordered data.

This is a little bit of a problem as the network stack has no knowledge of the user process buffer in my case but just pushes the data when a full packet has arrived. One possible solution is to have a per socket buffer in the network stack side as well but then you have a "double copy" solution which I wanted to avoid but perhaps I can't.