Timeout exception when connecting in passive mode to an FTP server via an HTTP proxy

Question

I'm getting

Rebex.Net.FtpException: Timeout exceeded.

when connecting through an HTTP proxy server with FTP passive mode to an FTP server.

The log reads:

[O] Connecting to www.example.de:21 using Ftp.
[O] Assembly: Rebex.Ftp 2019 R2 for .NET 4.0-4.7
[O] Platform: Windows 6.1.7601 64-bit; CLR: 4.0.30319.42000
[O] Using proxy HTTP CONNECT 172.15.11.9:8080.
[R] 220 ::ffff:5.38.228.58 FTP server ready
[S] USER ftpuser-348923094
[R] 331 Password required for ftpuser-348923094
[S] PASS **********
[R] 230 User ftpuser-348923094 logged in
[S] FEAT
[R] 211-Features:
[R]  MDTM
[R]  SSCN
[R]  TVFS
[R]  MFMT
[R]  SIZE
[R]  PROT
[R]  CCC
[R]  LANG en-US;es-ES;fr-FR;it-IT;ja-JP;ko-KR;ru-RU.UTF-8;ru-RU;zh-CN
[R]  PBSZ
[R]  AUTH TLS
[R]  MFF modify;UNIX.group;UNIX.mode;
[R]  REST STREAM
[R]  MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.mode*;UNIX.owner*;
[R]  UTF8
[R]  EPRT
[R]  EPSV
[R] 211 End
[S] OPTS UTF8 ON
[R] 200 UTF8 set to on
[S] OPTS MLST modify;perm;size;type;UNIX.group;UNIX.mode;UNIX.owner;
[R] 200 OPTS MLST modify;perm;size;type;UNIX.group;UNIX.mode;UNIX.owner;
[S] PWD
[R] 257 "/" is the current directory
[S] CWD /
[R] 250 CWD command successful
[S] PWD
[R] 257 "/" is the current directory
[S] MLST /www
[R] 250-Start of list for /www
[R]  modify=20200120151226;perm=flcdmpe;type=dir;UNIX.group=1168735;UNIX.mode=0770;UNIX.owner=10877227; /www
[R] 250 End of list
[S] CWD /
[R] 250 CWD command successful
[S] PWD
[R] 257 "/" is the current directory
[S] MLST /www/eins
[R] 250-Start of list for /www/eins
[R]  modify=20200121144633;perm=flcdmpe;type=dir;UNIX.group=1168735;UNIX.mode=0750;UNIX.owner=10877227; /www/eins
[R] 250 End of list
[S] CWD /
[R] 250 CWD command successful
[S] PWD
[R] 257 "/" is the current directory
[S] MLST /www/eins/
[R] 250-Start of list for /www/eins/
[R]  modify=20200121144633;perm=flcdmpe;type=dir;UNIX.group=1168735;UNIX.mode=0750;UNIX.owner=10877227; /www/eins
[R] 250 End of list
[S] TYPE I
[R] 200 Type set to I
[S] PASV
[R] 227 Entering Passive Mode (5,35,226,68,198,114).
[S] STOR /www/eins/2f2fbe94-dd58-4542-bf24-4c477c5cf8f0zp14.3-publish-test
[O] Timing out.
[O] Rebex.Net.FtpException: Timeout exceeded.

Could this be the same cause as this one?

Answer 1

According to the log, the FTP client failed because no response to its STOR command arrived from the server within a time period specified by Ftp object's Timeout property.

• One possible explanation for this is that the STOR command never reached the server. The most likely explanation for this would be a proxy server issue.

• Another explanation is that the server's response never reached the client. Again, a proxy server issue would be the most likely explanation in this case as well.

• It's also possible that the STOR command did actually reach the server, but the server was unable to respond yet because it was still waiting for the client to establish a data connection. Normally, Rebex FTP establishes the data connection after receiving the response to PASV command, and before sending the STOR command, but it is possible to tweak this using Settings.ConnectPassiveLater option. However, if the option was set to false, then again the proxy looks like the most likely culprit.

Based on the log, it's not possible to tell whether this could have the same cause as the other issue. Is the server supposed to accept data connections at 5.35.226.68? If it is, setting Settings.IgnorePassiveModeAddress to true might help. Otherwise, it most likely won't.

Are you able to connect to the same FTP server (possibly from different location) without using the proxy server, or using a different proxy server?

Comments

Thank you. The customer where this happens claims that he can use FileZilla successfully through his HTTP proxy.

I do think I add the two options `Settings.ConnectPassiveLater` and `Settings.IgnorePassiveModeAddress` as checkboxes to our application's GUI and let the customer try whether this helps.

---

Adding those options seems like a good idea, although please be aware that enabling them might cause issues with some FTP servers, proxies or FTP-aware firewalls.

A log form FileZilla might also be quite useful for comparison.

---

The customer sent me a FileZilla log file: https://pastebin.com/bZYJBU35

---

Thanks! According to the FileZilla log, the customer is apparently using FTP over TLS in FileZilla. According the Rebex logs you posted before, they have been using plain unencrypted FTP via Rebex FTP.

FileZilla logs also reveal that "EPSV" command has been used instead of "PASV" to start a passive data connection. With Rebex, you can enforce usage of this command using Settings.ForceExtendedDataChannelCommand option.

Based on this, please try using FTP over TLS with Rebex FTP with this server as well, and try enforcing the EPSV command. If it turns out this solves the issue, we'll consider adding Settings.PreferExtendedDataChannelCommand option that would use EPSV with servers that advertise support for it (which would correspond to FileZilla's behavior).

---

Thank you, Lukas. I'll discuss this with my customer and see what he tells me back.

---

We got feedback from the customer. Setting "Settings.IgnorePassiveModeAddress" to "true" did the trick. If you nevertheless consider adding a "PreferExtendedDataChannelCommand" property, it would still be helpful. We always have customers stating "it works in FileZilla, why doesn't it work in your application?", so being more like FileZilla would be a great thing!

---

Thanks for your feedback. We plan to add this property to 2020 R2 release. In a year or two, we will consider making it a default.

---

We figured out this actually belongs to Ftp.EnabledExtensions. Starting with Rebex FTP/SSL 2020 R2, you can enable this using the following code:

ftp.EnabledExtensions |= FtpExtensions.ExtendedPassiveMode;

(where "ftp" is an instance of Ftp class)

---

Thanks again, Lukas, will download and install right now.