Rebex TLS sockets - How to know if socket is disconnected?

Question

Hello,

I'm having an unusual problem with Rebex.Tls sockets.

The code:

private async Task StartServer() // Starts server
{
    try
    {
        var address = IPAddress.Parse("127.0.0.1");
        var port = Int32.Parse("5678");

        _server = new AbcSocketServer(new SocketSettings(), _logger, _tlsLogConfiguration, _context.ServerCertificate);
        _server.OnConnection += OnConnectionA; // add new connection
        _server.StartListen(new IPEndPoint(address, port), 100);

        _logger.Info($"Server started. Listening on {address}:{port} ...");
    }
    catch (Exception ex)
    {
        _logger.ErrorFormat("The application failed to start correctly. {0}", ex);
    }
}

private void OnConnectionA(object sender, SecureConnectionArgs e)
{
    Task.Run(async () =>
    {
        LogNewConnection(e.Ssl.Socket);
        SecureConnectionArgs client = e;

        try
        {
            while (e.Ssl.Connected)
            {
                ReceivedData received = await PullData(2048, e); // Exception

                if (received.BytesCount == 0)
                {
                    await e.Ssl.DisposeAsync();
                    break;
                }

                string req = Encoding.UTF8.GetString(received.MessageData).TrimEnd('\0');
                if (!string.IsNullOrEmpty(req))
                {
                    _logger.Info("Received: " + req);
                }
            }
        }
        catch (Exception ex)
        {
            _logger.Error($"Received unexpected error: {ex.Message} \n{ex.StackTrace}");
        }
        finally
        {
            LogDisconect(client.Ssl.Socket);
            client.Ssl.Dispose();
            var socketId = _manager.GetAllSockets().FirstOrDefault(s => s.Value == client).Key;

            _logger.Warn($"Disconnected {socketId}");
        }
    });
}

private async Task<ReceivedData> PullData(int chunkSize, SecureConnectionArgs e)
{
    byte[] messageData = new byte[0];
    byte[] dataChunk;
    int bytesReceived = 0;
    int available = int.MaxValue;

    while (available > 0)
    {
        dataChunk = new byte[chunkSize];
        bytesReceived += await e.Ssl.ReceiveAsync(dataChunk);
        messageData = messageData.Concat(dataChunk).ToArray();

        available = e.Ssl.Available;
    }

    return new ReceivedData(messageData, bytesReceived);
}

public ExampleResponse ExampleMethod(string socketId, string id)
{
    var client = _manager.GetAllSockets().FirstOrDefault(s => s.Key == socketId);   // Get socket from KeyValuePair<string, SecureConnectionArgs>
    try
    {
        if(client.Value == null || IsSocketAlive(client.Value)) // Check if socket is not null or is still connected
        {
            _log.Info("Offline");

            return new ExampleResponse
            {
                ErrorCode = ErrorCodes.Offline,
                ErrorMessage = $"Socket {socketId} is offline"
            };
        }
        else
        {
            var message = new ExampleMessage
            {
                text = "Sending a message"
            };

            _log.Info("Sending...");

            var xmlMsg = message.SerializeMessage();    // Serialize message to XML
            client.Value.Ssl.SendAsync(Encoding.UTF8.GetBytes(xmlMsg)); // Sends message to socket
        }
    }
    catch(Exception ex)
    {
        _log.Error($"Exception with socket {socketId}: {ex}");

        return new ExampleResponse
        {
            ErrorCode = ErrorCodes.InternalError,
            ErrorMessage = $"Internal error has occured with socket {socketId}"
        };
    }

    // If we came this far the message has been sent
    return new ExampleResponse
    {
        ErrorCode = ErrorCodes.Success,
        ErrorMessage = "None"
    };
}

private bool IsSocketAlive(SecureConnectionArgs client) // Checks if socket is still online
{
    return (client.Ssl.Poll(1, SocketSelectMode.SelectRead) && client.Ssl.Available == 0)
}

Now the problem with this code is that when the ethernet cable gets unplugged it does not fail to send a message to the socket and it returns that the message has been sent.
But after a couple of seconds we do get an Exeption OnConnectionA (See // Exception)
Received unexpected error: Connection was closed by the remote connection end. at gcrvw.qwujb.xslib() at gcrvw.nduus.mmyho(Task p0, Func`1 p1) at gcrvw.qwujb.xulmm() at gcrvw.qwujb.gmkrt(Byte[] p0, Int32 p1, Int32 p2, Boolean p3) at gcrvw.nduus.uapvn[TR](Task`1 p0, Func`1 p1) at gcrvw.qwujb.ofafp(Byte[] p0, Int32 p1, Int32 p2, Boolean p3) at gcrvw.npgii.iurrr(ArraySegment`1 p0) at Rebex.Net.TlsSocket.ReceiveAsync(ArraySegment`1 buffer)

How can we make sure that the socket is alive and receives the message from ExampleMethod?
Because it sends the message and returns that everything is ok.

How can we make sure that the socket before sending the message is still alive if it disconnects ungracefully (ethernet cable unplugged)?
When we disable the network adapter all statuses change and we are able to identify that the socket was disconnected only when disconnecting ungracefully (ethernet cable unplugged) we dont know because statuses do not change.

Scenario:
1. Socket connects and is added to _manager list
2. Unplug ethernet cable from computer
3. Initiate ExampleMethod() - Send to socket
4. client.Value.Ssl.SendAsync(Encoding.UTF8.GetBytes(xmlMsg)); - this line passes without errors even with disconnected ethernet cable
5. After ~10 seconds we get Exception in bytesReceived += await e.Ssl.ReceiveAsync(dataChunk);

We dont understand why 4. Did not throw exception? We need to know here if the message was sent successfully.

• .NET Core 3.1
• Rebex.Tls (5.0.7579)

Answer 1

Hello,
I don't see anything special in your description of the socket behavior. We use a .NET TCP socket as the underlying network layer.

1) You never detect that the connection is broken (for example cable unplugged) when you call only Receive.
2) You detect the broken connection when the Send operation is initiated.

The Send operation is indeed required to detect the broken connection. But it is not true that you have guaranteed that the Send operation immediately fails when the connection is already broken. Winsock on Windows (other TCP stacks are alike) happily accepts to be sent data in an internal buffer and returns 'success' - then Send operation on Socket returns success. Later - you don't know the exact time when because Winsock might delay the operation, retry the operation, etc. - when the Send operation fails in Winsock, the internal state of the Socket is set to 'broken' -> therefore Receive or one of the next Send operations fail - but again  - even the second/third. Send operation might succeed. You are lucky when the Receive operation fails. If you need special guarantees, design your application protocol to handle "guaranteed delivery", "synchronization of exchanged messages after reconnect" or similar requirements.

BTW:
TlsSocket.Connected property is also unreliable as documented.
"Gets the connection state of the TlsSocket. This property will return the latest known state of the SecureSocket. When it returns false, the socket was either never connected or no longer connected.

Note: There is no guarantee that the session is still connected even though Connected returns true."

The conclusion.
Behavior by TCP/OS TCP stack design.