Are there options which can be tweaked to improve bandwidth/performance of an SSH tunnel?

Question

We are evaluating Rebex Terminal Emulation's SSH tunnel capabilities, but we have a strong requirement to support high bandwidth usage. In our testing with the Tectia SSH Client, we were able to establish a tunnel and obtain speeds upwards of 800 Mbps. However, in the same environment using the Rebex Terminal Emulation library for .NET, we are maxing out at around 100 Mbps.

Since the client technology appears to be the bottleneck, can you provide guidance about how we can increase performance? Thank you.

Answer 1

It might be possible to tweak SSH tunnels to improve the bandwidth a bit. However, there are also some bottlenecks inherent to the underlying design of Rebex Terminal Emulation's SSH core. The SSH core has been designed in 2006 and targeted .NET 1.0 on single-CPU machines, and although a major update (with scalability and speed as the goals) is already one of our top priorities, it has been postponed due to the ongoing pandemic and lockdowns. We expect to start updating the SSH core later this year.

In the meantime, in order to achieve the highest speed, make sure to use use the fastest cipher available (this depends on the platform you use - on Windows, AES/CBC and AES/GCM ciphers are the best) and set Ssh's Settings.SetNoDelayForTunnelSockets property to true.

Additionally, the internal buffer sizes we use for SSH tunnels are very low, so increasing them might provide a noticeable improvement in transfer speed. We'll look into this and see what we can do.

Also, please let us know what kind of tunnels you intend to use so we can make sure we are looking into the same scenario.

Comments

Hi, Lukas, and thanks for the info. I'm not 100% what you mean what you ask what "kind of tunnels" we intend to use, but I can say that we are establishing an SSH tunnel via a Tectia SSH server which uses certificate-based authentication. Here is a code snippet from what we've been testing so far.

```
var ssh = new Ssh();
ssh.Settings.SshParameters.SetEncryptionAlgorithms("aes128-ctr");           
ssh.Settings.SetNoDelayForTunnelSockets = true;

await ssh.ConnectAsync(args.GatewayServerName, args.GatewayServerPort);
var privateKey = new SshPrivateKey(cert); // from machine store
await ssh.LoginAsync(args.AuthUserName, privateKey);

var tunnel = await ssh.StartOutgoingTunnelAsync(
   "localhost", args.GatewayLocalPort,     
   args.GatewayDestinationServerName, args.GatewayDestinationServerPort);
```

As you can see, we are setting the encryption algorithm to "aes128-ctr" and are setting `SetNoDelayForTunnelSockets` to `true`. However, we are still not seeing the performance we desire. I understand that you plan for some performance improvements in the future. However, if you have any other guidance for the meantime, please let me know. Thanks.

---

Actually, AES/CTR ciphers such as "aes128-ctr" are quite slow with Rebex SSH. AES/CTR is not supported by Windows CNG API, so they are partially implemented in managed C# code, which makes them noticably slower than their AES/CBC or AES/GCM counterparts (both of which use fast implementations provided by CNG API). The only way to make our "aes128-ctr" as fast as Tectia's native implementation would be to use an external native DLL, which is not currently on our roadmap. (Contact us know at support@rebex.net to discuss this.)

We'll investigate the possibility of adding an option to use larger data buffers for SSH tunnels and let you know when there's a new build to try, but I'm afraid that AES/CTR is still going to be a big bottleneck.

Regarding "what kind of tunnels", I meant whether you are using outgoing tunnels, incoming tunnels, or tunnels via SOCKS5, and you answered that question. Thanks!

---

This is helpful. Thank you. We are now seeing that the AES/CBC and AES/GCM algorithms are yielding better performance, around 500 Mbps. We're tweaking and testing, but getting closer to what we need. We would love to see a performance upgrade in the future with larger data buffers, etc.

---

We would be very interested in your progress as you roll out your performance upgrades this year. Our research indicates that increasing the SSH buffer size would unlock a lot of performance potential, as shown by the HPN SSH project (https://www.psc.edu/research/networking/hpn-ssh/).

Can you say whether you plan to implement something similar, something which could push speeds to 1 Gbps and beyond?