The exception occurred because reply of the server exceeded internal threshold of 2MB per reply.
The SFTP draft says this about packet size:
The maximum size of a packet is in
practice determined by the client ...
All servers SHOULD support packets of
at least 34000 bytes ...
In other words: clients should use max packet size = 34000B (and "polite" server should use the same).
Unfortunately, the packet size is not determined by the client in case of listing directories. The draft in this case says:
The files in a directory can be listed
SSH_FXP_READDIR requests. Each
SSH_FXP_READDIR request returns one or
more file names with full file
attributes for each file. The client
until it has found the file it is
looking for or until the server
responds with a
indicating an error (normally
SSH_FX_EOF if there are no more files
in the directory).
This says that response to directory listing can be split among various number of
SSH_FXP_READDIR packets. The reason is obvious: split large directory listing into small packets (normally 34000B at max).
So, the suggested way to solve this issue is to configure your SFTP server to not send replies larger than 34000B (of course if it is possible depends on capabilities of your SFTP server).
If configuring your server or using another server is not possible for you, we can add an option to specify max packet size. But this will not solve the problem in all cases. What if the directory contains e.g. 1.000.000 files?
Other clients will probably also fail to list such directory (e.g. WinSCP have internal buffer limited to 102400B). If the client doesn't have the internal buffer size limitation it can ends with
OutOfMemoryException shutting your system down.
Actually, what is your SFTP server? We can report this issue to the vendor and discussed it with him personally, because we treat this as bug of the SFTP server.