(Rebex.FTP/SSL) Very slow download speed from within docker container.

0 votes
asked Feb 14 by lis (190 points)

Hello!

I have a very interesting issue here.

Shortly:
We have a simple .NET Core 2.1 application that downloads stuff from FTP server within same network (10 GBit network) and achieves maximum speed of ~16 Mbps when launched within docker container.

Same app reaches 200 Mbps when started from outside docker.

There is nothing special, here is our test app code:

var client = new Ftp();
Rebex.Licensing.Key = "...";

client.ValidatingCertificate += (sender, args) => {
 args.Accept();
};

var sslMode = SslMode.Explicit;
var ftpUri = new Uri("ftp://local-network-server/");

// Connect securely using explicit or implicit SSL.
if (ftpUri.Port > 0)
    client.Connect(ftpUri.Host, ftpUri.Port, sslMode);
else
    client.Connect(ftpUri.Host, sslMode);

client.SecureTransfers = true;
client.Timeout = (int) TimeSpan.FromMinutes(5).TotalMilliseconds;
client.Settings.KeepAliveDuringTransfer = true;
client.Settings.KeepAliveDuringTransferInterval = 60;
client.Settings.PauseBeforeUploadClose = true;
client.Settings.UseLargeBuffers = true;
client.UploadBufferLength = 256 * 1024; // 256 KB
client.Settings.SslReuseSessions = true;

// Connection is protected now, we can log in safely.
client.Login("user", "pass");

client.DownloadAsync("/path/to/folder", Directory.GetCurrentDirectory(), TraversalMode.Recursive, TransferMethod.Copy, ActionOnExistingFiles.OverwriteAll).Wait();

client.Disconnect();

Here is our Dockerfile:

FROM microsoft/dotnet:2.2-runtime

# Set current work dir.
WORKDIR /app

# Copy app build.
ADD ./bin/Debug/netcoreapp2.1/publish/ ./

# Start app.
ENTRYPOINT ["dotnet", "Expsz.dll"]

Rebex.FTP Nuget - 4.0.6930
Rebex.Common Nuget - 2.5.6930
Rebex.Networking Nuget - 3.5.6930

IMPORTANT
As an alternative I tried FluentFTP lib from within container, it doesnt have such issues and is able to achieve good speeds. Also fetching stuff via curl/wget also works good.

Any ideas? Might it be due to docker base image? Does Rebex.Ftp rely on some OS things?

Anything more I can provide to make things clear?
Any help is appreciated. Thanks!

Applies to: Rebex FTP/SSL
commented Feb 14 by Lukas Pokorny (102,170 points)
It's hard to tell without more information. Is this a Linux-based docker container? One feature that is known to be slow on .NET Core on Linux is TLS 1.2 with AES/GCM encryption (on Windows, a native CNG API is used instead). Please create a log using Ftp's LogWriter property (see https://rebex.net/kb/logging/ for more information) and either post it here or mail it to support@rebex.net for analysis - hopefully, this will provide some clues.

1 Answer

0 votes
answered Feb 14 by lis (190 points)
selected Feb 14 by lis
 
Best answer

UPDATE
I found out that this is caused by SecureTransfers property. If I set it to false then speed from container is the same as from host.

So as I understand, this SecureTransfers compresses/encrypts traffic on fly right? So bottleneck could be from either client or server side as I understand.

commented Feb 14 by Lukas Pokorny (102,170 points)
Setting SecureTransfers to false disables TLS/SSL for data transfers, which means that uploads, downloads and listings are transmitted unencrypted. This reveals that the slowdown is in fact caused by TLS/SSL encryption.
Please create a log (see my comment at http://forum.rebex.net/9818/rebex-very-slow-download-speed-from-within-docker-container?show=9821#c9821) and let us know whether TLS 1.2 with AES/GCM has actually been negotiated.
commented Feb 14 by lis (190 points)
Okay I see now from logs:

TLS: Data connection: Connection secured using cipher: TLS 1.2, RSA with ephemeral Elliptic Curve Diffie-Hellman, AES with 128-bit key in GCM mode, AEAD.

Just as you said. And no way to make it faster?
P.S Yes, this is Linux based container.
commented Feb 14 by Lukas Pokorny (102,170 points)
Thanks! We currently use a managed implementation of AES/GCM on non-Windows platforms, which is slow. We plan to optimize it to make it faster, although it's hard to tell how much faster it can get. Fortunately, .NET Core 3.0 is going to support AES/GCM, which will resolve the slowness.

In the meantime, consider disabling AES/GCM support in Rebex FTP's TLS/SSL layer. Or rather, only enable the AES/SHA-2 ciphers that are fast, widely-supported and sufficiently secure:
    client.Settings.SslAllowedSuites = TlsCipherSuite.RSA_WITH_AES_128_CBC_SHA256 |

Alternatively, enable DHE_ variant of these ciphers or additional ciphers as well - check out https://rebex.net/ftp-ssl.net/features/tls-ssl.aspx for the full list.
...