How to Optimize Transfer Performance with UploadBufferLength?

Question

In some performance tests, we're getting slower Ftp Upload times with our Rebex library than compared with FileZilla. Looking at the network traffic, I noticed all packet sizes from our system were the default 4096 bytes, while FileZilla seemed to do some dynamic changing of packet sizes while uploading (ranging from 2000-11000 bytes). I'm wondering if anyone has modified the UploadBufferLength and saw an increase in transfer speeds, or if there was a way to auto-size the UploadBufferLength to increase performance?

Any other ways to increase performance (I'm uploading jpeg images only) will be appreciated.

Answer 1

In case you use Rebex FTP or Rebex FTP/SSL:

Yes, setting Ftp object's UploadBufferLength to a larger value can result in increased performance in many cases. Try setting it to something like 64KB and let us know whether it helps.

Another trick FileZilla does is enabling larger TCP window size (it sets TCP send buffer size to 256KB and TCP receive buffer size to 4MB). However, we were quite reluctant to do this in Rebex FTP because this has been observed to only increase the speed in some scenarios and needs quite a lot of memory. However, if using a larger UploadBufferLength doesn't help, we are ready to work with you to give this a try as well.

---

In case you use Rebex SFTP:

Before connecting, enable Sftp object's UseLargeBuffers option:

sftp.Options |= SftpOptions.UseLargeBuffers; //(C#)

sftp.Options = sftp.Options Or SftpOptions.UseLargeBuffers; ' VB.NET

(where 'sftp' is an instance of Rebex.Net.Sftp)

This enables several FileZilla-like tricks (like larger TCP window size and some internal received data buffers).

---

Update: FtpOptions.UseLargeBuffers is now available in Rebex FTP(/SSL) v3.0.3793.0 as well.

Comments

Yes! This worked very well. We're seeing a big performance increase. I'm wondering if there will be any server side issues for setting large upload buffers- we don't really know who we're uploading too. Do you foresee any issues with servers not accepting 64kb buffer sizes? Thanks so much!

---

I don't expect any issues with FTP servers. TCP is a stream transport layer, so it should not matter to the server whether it receives small and large blocks. In fact, if the transfer speed is slow or the server is not willing to accept bigger blocks, the data will still be sent in smaller chunks by the TCP stack. Large buffer size might only cause problems at the client on very slow networks (like GPRS) where it takes a lot of time to send a block this large.