Sftp GetList() Message with invalid length 2209797 was received.

Question

When I try to get list from specific path am getting this Sftp Exception.

SftpItemCollection sftpItemCollection = _sftp.GetList(path);

ErrorDescriptionForBug: Rebex.Net.SftpException, Message with invalid length 2209797 was received. ErrorStackTrace: at SftpComponent.GetDirectoryListing(String path)

In that folder contents more than 18000 files. This method works for ftp protocol fine but not sftp.

Any suggestion.

Answer 1

The exception occurred because reply of the server exceeded internal threshold of 2MB per reply.

The SFTP draft says this about packet size:

The maximum size of a packet is in
practice determined by the client ...
All servers SHOULD support packets of
at least 34000 bytes ...

In other words: clients should use max packet size = 34000B (and "polite" server should use the same).

Unfortunately, the packet size is not determined by the client in case of listing directories. The draft in this case says:

The files in a directory can be listed
using the SSHFXPOPENDIR and
SSHFXPREADDIR requests. Each
SSHFXPREADDIR request returns one or
more file names with full file
attributes for each file. The client
should call SSHFXPREADDIR repeatedly
until it has found the file it is
looking for or until the server
responds with a SSHFXPSTATUS message
indicating an error (normally
SSHFXEOF if there are no more files
in the directory).

This says that response to directory listing can be split among various number of SSHFXPREADDIR packets. The reason is obvious: split large directory listing into small packets (normally 34000B at max).

So, the suggested way to solve this issue is to configure your SFTP server to not send replies larger than 34000B (of course if it is possible depends on capabilities of your SFTP server).

If configuring your server or using another server is not possible for you, we can add an option to specify max packet size. But this will not solve the problem in all cases. What if the directory contains e.g. 1.000.000 files?

Other clients will probably also fail to list such directory (e.g. WinSCP have internal buffer limited to 102400B). If the client doesn't have the internal buffer size limitation it can ends with OutOfMemoryException shutting your system down.

Actually, what is your SFTP server? We can report this issue to the vendor and discussed it with him personally, because we treat this as bug of the SFTP server.