Change in behavior of Sftp.CreateDirectoryAsync or Sftp.IsConnected?

Question

[Short version]

I am wondering if there were any changes between builds 6821 and 7077 of Rebex SFTP that could affect the behavior of either of the following:

(1) Sftp.CreateDirectoryAsync, causing it to return false when it used to return true (or maybe when it used to throw an exception?)

and/or

(2) Sftp.IsConnected, causing it to return true when it used to return false

---

[Long version]

Our team uses Rebex SFTP to communicate with a third-party SFTP server, and we recently upgraded from build 6821 to 7077. Since deploying the new version, the third party has reported to us that they are intermittently seeing large numbers of failures on an Add Directory operation coming from our client. At the times when they see the failures on their end, our logs show a large number of Rebex.Net.SftpException with the message "Failure; General failure."

The basic logic in our SFTP client application is the following:
- Check that client is still connected using Sftp.IsConnected (if false, skip all of the following steps)
- Check if directory exists using Sftp.DirectoryExistsAsync
- If directory does not exist, create it using Sftp.CreateDirectoryAsync
- Upload file to directory
- Repeat (up to a couple hundred times)

On analyzing our logs, I believe Sftp.DirectoryExistsAsync sometimes returns false when the directory already exists. So then we call Sftp.CreateDirectoryAsync, but of course that fails. This issue is intermittent and the network infrastructure in this environment has been somewhat unstable in the past, so it's likely that the connection has failed in some way when all of this happens. My theory is that either Sftp.IsConnected used to return false and now it returns true, or Sftp.DirectoryExistsAsync used to return true and now it returns false. But I'm not sure how to work around this to fix the errors.

What I am trying to figure out is:

(1) Why all of these errors only started cropping up after we updated to a newer version of the library
(2) And more importantly, how we can eliminate them. (We have been told we are spamming the server's logs.)

Any ideas? Thanks in advance!

Answer 1

Hello,

between builds 6821 and 7077, there were actually few changes at the SFTP protocol level:

SFTP: Added SftpItemType.Fifo enum value.
SFTP: Fixed SftpItem.Owner/Group properties that returned null (contrary to the documentation) when using SFTP v3.
SFTP: GetItems() method changed to return items of unknown type instead of throwing an exception.
SFTP: Added support for 'fsync@openssh.com' SFTP extension (enable using Sftp.Settings.FileSyncOnUpload property).
SFTP: Added Sftp.CreateLink method (needs 'hardlink@openssh.com' extension support to create hard links).
SFTP: Fixed minor race condition in Dispose method.

Other changes, including those in the SSH core, should not affect the working in SFTP protocol.

This makes the apparent change in behavior quite puzzling. Have you ruled out the possibility that the server has been upgraded around the same time?

Also, it might be useful to know what's actually inside those spammed server-side logs - what kind of error does the server report?

Anyway, to be able to tell what is actually going on at the client side, create a client-side communication log using Sftp object's LogWriter property, and either post it here or mail it to support@rebex.net for analysis. The log will show those "Failure; General failure." errors in context of other commands and responses, hopefully making it possible to determine their meaning and cause.

---

As a side note, please note that your Rebex support contact has expired last year. We'll try to help you solve this regardless, but consider renewing your support contract to ensure continued support and upgrades.

Comments

Thanks for following up and alerting me to the expiration of the support contract, I was not aware. I will add the suggested logging.

Unfortunately for our ability to isolate variables, the third party actually did update their server around the same time. They say it was only an OS upgrade, so at first it seemed unlikely to be caused by their changes, but now I am wondering if could have affected the connection logic and thus the value returned to Sftp.IsConnected.

We are going to try rolling back and also adding the additional logging to see if that helps.

Thanks for your help!