Rebex.Net.Sftp.ConnectAsync is not completely asynchronous?

Question

Does the "ConnectAsync" method that is in Rebex.Net.Sftp class work completely asynchronously? I've tried to limit thread pool maximum worker threads to 10 and started 10 parallel executions of "ConnectAsync" method. In this case all the threads were blocked in Rebex.Net.ProxySocket.IR method until the timeout occurred. However if I decrease parallel executions to 9, all of them are processed successfully within few seconds.

It's critical issue for us, any help will be appreciated!

Answer 1

Unfortunately, ConnectAsync and other *Async methods are not truly asynchronous. They are in fact equivalent to their synchronous variants.

We are aware of this limitation and we plan to do something about it in the future, most likely after we deprecate .NET 2.0/3.5 platforms, which would enable us to use await/async (that would help us tremendously to bring make codebase up-to-date).

But unfortunately, this is going to be a huge amount of work that might take months to complete. We are currently busy improving other areas (such as ECDSA and ECDH support in SSH and TLS) and can't devote any resources to this yet.

---

However, in this particular case, the issue is most likely caused by a "none proxy" code (basically an implementation of our Rebex.Net.ISocket interface on top of System.Net.Sockets.Socket) which creates an additional thread. And this might actually be quite simple to solve.

Additionally, you can even fix this yourself by implementing a custom transport layer using ISocketFactory/ISocket interfaces and instruct the Sftp object to use it instead of the built-in layer by calling SetSocketFactory method before connecting.

Sample implementation:

public class CustomSocket : ISocket
{
    private class CustomSocketFactory : ISocketFactory
    {
        public ISocket CreateSocket()
        {
            return new CustomSocket(this);
        }
    }

    private static readonly ISocketFactory _factory = new CustomSocketFactory();

    public static ISocketFactory Factory
    {
        get { return _factory; }
    }

    private readonly Socket _socket;

    public CustomSocket(ISocketFactory factory)
    {
        _socket = new Socket(SocketType.Stream, ProtocolType.Tcp);
    }

    public bool Connected
    {
        get { return _socket.Connected; }
    }

    public EndPoint LocalEndPoint
    {
        get { return _socket.LocalEndPoint; }
    }

    public EndPoint RemoteEndPoint
    {
        get { return _socket.RemoteEndPoint; }
    }

    public int Timeout
    {
        get
        {
            return _socket.ReceiveTimeout;
        }
        set
        {
            _socket.ReceiveTimeout = value;
            _socket.SendTimeout = value;
        }
    }

    public void Close()
    {
        _socket.Close();
    }

    public void Connect(EndPoint remoteEP)
    {
        _socket.Connect(remoteEP);
    }

    public void Connect(string serverName, int serverPort)
    {
        _socket.Connect(serverName, serverPort);
    }

    public bool Poll(int microSeconds, SocketSelectMode mode)
    {
        return _socket.Poll(microSeconds, (SelectMode)mode);
    }

    public int Receive(byte[] buffer, int offset, int count, SocketFlags socketFlags)
    {
        return _socket.Receive(buffer, offset, count, socketFlags);
    }

    public int Send(byte[] buffer, int offset, int count, SocketFlags socketFlags)
    {
        return _socket.Send(buffer, offset, count, socketFlags);
    }

    public void Shutdown(SocketShutdown how)
    {
        _socket.Shutdown(how);
    }
}

Usage:

var sftp = new Sftp();
sftp.SetSocketFactory(CustomSocket.Factory);
sftp.Connect("test.rebex.net");
sftp.Login("demo", "password");
sftp.GetList();
sftp.Disconnect();

Comments

Thanks a lot for the information and implementation of CustomSocket. With this implementation when I call sftp.ConnectAsync, there are no calls of BeginXXX and EndXXX methods in CustomSocket, instead such methods as Connect, Poll, Send, Receive are called. Thus there is the same issue with blocked threads. Is there any possibility to make calls exactly to BeginXXX and EndXXX methods?

---

Unfortunately, we are unable to address this until we refactor the internals of our SFTP/SSH implementation as outlined in the beginning of my reply above. This would be a huge undertaking and I'm afraid we won't be able to devote our resources into this in the near term. We do plan to eventually address this limitation, but it's not yet in our near-term roadmap. Sorry!

---

Has the refactoring of the internals happened? Are ConnectAsync and other *Async methods truly asynchronous now?

---

This is a huge undertaking, and the work has only just begun recently (we have been very busy with other features - see https://www.rebex.net/total-pack/history.aspx). So far, we have a truly asynchronous TLS layer (still not enabled by default), and plan to implement asynchronous SSH layer next. Once finished, we could start upgrading higher-level assemblies such as Rebex.Sftp or Rebex.Ftp.
However, please note that it's quite possible this will only be available for modern platforms (.NET 6/7/8/...), because by the time we are done, .NET Framework 4.8 would only have several years of extended support left.