Question

"Server reports current directory as a relative path. Multi-file operations cannot be used with this server."

I am getting this error when calling the GetItemsAsync.
I understand the meaning of it and it is true that this particular server from our customer is really weird and it is returning paths without the leading "/". BTW, the server is:
"215 MVS is the operating system of this server. FTP Server is running on z/OS."

Here is a partial log:
2022-03-02 09:09:00.540 INFO FileTransferClient(1)[4] Response: PROT
2022-03-02 09:09:00.542 INFO FileTransferClient(1)[4] Response: 211 End
2022-03-02 09:09:00.553 INFO FileTransferClient(1)[8] Command: PWD
2022-03-02 09:09:00.566 INFO FileTransferClient(1)[8] Response: 257 "'TSVT.'" is working directory.

Is there any way to make GetItemsAsync work anyway so I can retrieve the files and folders recursively?

Thanks,

Answer 1

The GetItemsAsync requires particular FTP server behavior. One of the requirements is absolute path reporting.

The second part of the exception says: Changing directory to an absolute path first could help to solve this problem.

Please, try to call client.ChangeDirectory("/TSVT"); before calling the GetItemsAsync() method.

If this does not help, there is one possible solution. You can plug in to communication layer and modify server response of the PWD command to provide absolute paths.

It can be done like this:

using (var client = new Ftp())
{
    client.SetSocketFactory(CustomProxySocket.Factory);

    client.Connect(...);

    var list = await client.GetItemsAsync("*", TraversalMode.NonRecursive);
}

You can download CustomProxySocket class here. It contains logic to detect PWD command and fix the reported path.
I don't know whether the correct name of the directory is 'TSVT.' or TSVT. or TSVT. So, please update the correction logic in the CustomProxySocket accordingly.

Comments

Thanks Lukas, this seems to be working.
I am curious, is this illegal to return a path without a leading "/" in response to PWD?

---

The RFC 959 does not tell much about PWD command/response:

>>  PWD
>>    Print the current working directory name.

and:

>>  Essentially because the PWD command returns the same type of
>>  information as the successful MKD command, the successful PWD
>>  command uses the 257 reply code as well.

This could indicate that the PWD may behave similar to MKD command.

For MKD command RFC 959 says:

>>  ... If it is a "relative" pathname (i.e., a pathname
>>  which is interpreted relative to the current directory), the user
>>  would need to be in the same current directory in order to reach
>>  the subdirectory.  Depending on the application, this may be
>>  inconvenient.  It is not very robust in any case.

As you can see, the RFC does not require absolute paths, but using relative paths is not recommended.

In any cases, the RFC is very vague about PWD return values.

---

Hi Lukas, thanks for the explanation.
We just got a reply from the customer but unfortunately, the problem persists.
Here is the log up to the response to the PWD command:

2022-03-10 14:48:54.479 Opening log file.
2022-03-10 14:48:54.480 INFO FileLogWriter(5)[1] Info: Assembly: Rebex.Common R6.1 for .NET 4.6-4.8
2022-03-10 14:48:54.480 INFO FileLogWriter(5)[1] Info: Platform: Windows 10.0.19042 64-bit; CLR: 4.0.30319.42000
2022-03-10 14:48:54.480 DEBUG FileLogWriter(5)[1] Info: Culture: en; Windows-1252
2022-03-10 14:48:54.606 INFO FileTransferClient(3)[8] Info: Connecting to 10.6.32.11:21 using Ftp.
2022-03-10 14:48:54.606 INFO FileTransferClient(3)[8] Info: Assembly: Rebex.Ftp R6.1 for .NET 4.6-4.8
2022-03-10 14:48:54.606 INFO FileTransferClient(3)[8] Info: Platform: Windows 10.0.19042 64-bit; CLR: 4.0.30319.42000
2022-03-10 14:48:54.613 DEBUG FileTransferClient(3)[8] Info: Culture: en; Windows-1252
2022-03-10 14:48:54.613 INFO FileTransferClient(3)[8] Info: Using proxy ‪‎‌‭‍‭‎‫‮‫‮‮‬‌​‏‏‭‍‌‭‏‮‏‪​‏‭‮+‎‪‫​‮‭‭‪‭​‬‫‌‭‭‬​‪‌‎‪‫‏‏‫‮​‮.
2022-03-10 14:48:54.615 DEBUG FileTransferClient(3)[8] Info: Connection succeeded.
2022-03-10 14:48:54.656 INFO FileTransferClient(3)[8] Response: 220-FTP 14:48:54 on 2022-03-10.
2022-03-10 14:48:54.664 INFO FileTransferClient(3)[8] Response: 220 Connection will close if idle for more than 8 minutes, 20 seconds.
2022-03-10 14:48:54.677 INFO FileTransferClient(3)[8] Command: AUTH TLS
2022-03-10 14:48:54.731 INFO FileTransferClient(3)[8] Response: 234 Security environment established - ready for negotiation
2022-03-10 14:48:54.731 DEBUG FileTransferClient(3)[8] Info: Upgrading control connection to TLS.
2022-03-10 14:48:54.731 DEBUG FileTransferClient(3)[8] TLS: Using classic TLS core.
2022-03-10 14:48:54.738 DEBUG FileTransferClient(3)[8] TLS: Enabled cipher suites: 0x000F3DF7EBE00640.
2022-03-10 14:48:54.738 DEBUG FileTransferClient(3)[8] TLS: Applicable cipher suites: 0x000F3DF7EBE00640.
2022-03-10 14:48:54.738 DEBUG FileTransferClient(3)[8] TLS: HandshakeMessage:ClientHello was sent.
2022-03-10 14:48:54.753 DEBUG FileTransferClient(3)[8] TLS: HandshakeMessage:ServerHello was received.
2022-03-10 14:48:54.753 INFO FileTransferClient(3)[8] TLS: Negotiating TLS 1.2, RSA, AES with 256-bit key in GCM mode, AEAD.
2022-03-10 14:48:54.753 DEBUG FileTransferClient(3)[8] TLS: The server supports secure renegotiation.
2022-03-10 14:48:54.754 DEBUG FileTransferClient(3)[8] TLS: HandshakeMessage:Certificate was received.
2022-03-10 14:48:54.754 DEBUG FileTransferClient(3)[8] TLS: HandshakeMessage:ServerHelloDone was received.
2022-03-10 14:48:54.754 DEBUG FileTransferClient(3)[8] TLS: Verifying server certificate ('CN=xxxxx, OU=xx, O=xxxx, L=xxx, S=xxx, C=xx').
2022-03-10 14:48:54.756 DEBUG FileTransferClient(3)[8] TLS: Certificate verification result: Accept
2022-03-10 14:48:54.757 DEBUG FileTransferClient(3)[8] TLS: HandshakeMessage:ClientKeyExchange was sent.
2022-03-10 14:48:54.757 DEBUG FileTransferClient(3)[8] TLS: CipherSpec:ChangeCipherSpec was sent.
2022-03-10 14:48:54.758 DEBUG FileTransferClient(3)[8] TLS: HandshakeMessage:Finished was sent.
2022-03-10 14:48:54.777 DEBUG FileTransferClient(3)[8] TLS: CipherSpec:ChangeCipherSpec was received.
2022-03-10 14:48:54.777 DEBUG FileTransferClient(3)[8] TLS: HandshakeMessage:Finished was received.
2022-03-10 14:48:54.777 INFO FileTransferClient(3)[8] TLS: Connection secured using cipher: TLS 1.2, RSA, AES with 256-bit key in GCM mode, AEAD.
2022-03-10 14:48:54.778 DEBUG FileTransferClient(3)[8] Info: Control connection upgraded to TLS 1.2.
2022-03-10 14:48:54.815 INFO FileTransferClient(3)[8] Command: USER xxxxxxx
2022-03-10 14:48:54.832 INFO FileTransferClient(3)[8] Response: 331 Send password please.
2022-03-10 14:48:54.841 INFO FileTransferClient(3)[8] Command: PASS **********
2022-03-10 14:48:54.888 INFO FileTransferClient(3)[8] Response: 230 xxxxxxx is logged on.  Working directory is "TSVT.".
2022-03-10 14:48:54.898 INFO FileTransferClient(3)[8] Command: FEAT
2022-03-10 14:48:54.915 INFO FileTransferClient(3)[8] Response: 211- Extensions supported
2022-03-10 14:48:54.921 INFO FileTransferClient(3)[8] Response:  AUTH TLS
2022-03-10 14:48:54.926 INFO FileTransferClient(3)[8] Response:  PBSZ
2022-03-10 14:48:54.931 INFO FileTransferClient(3)[8] Response:  PROT
2022-03-10 14:48:54.936 INFO FileTransferClient(3)[8] Response: 211 End
2022-03-10 14:48:54.948 INFO FileTransferClient(3)[4] Command: PWD
2022-03-10 14:48:54.958 INFO FileTransferClient(3)[4] Response: 257 "'TSVT.'" is working directory.

It seems the proxy wasn't being used for some reason.
From the log it does say "Using proxy .+"
(I think because the class name is obfuscated so it only shows the ".+")

Do you have any idea?

---

Is it because the data is already SSL encrypted at ISocket.Send so the check for "PWD" will not work?

---

To the "Using proxy .+" question:

I see this log message in my non-obfuscated build:

  "Using proxy Rebex.Samples.CustomProxySocket+CustomProxySocketFactory."

If you are using obfuscation, the class name can be unprintable resulting to the reported log message. However, the custom proxy socket is used if it was passed to the SetSocketFactory() method.

To the "PWD" not working issue:

You are right. The custom socket represents low-level transport layer, which means the data inside the ISocket.Send/Received methods are encrypted if TLS is in effect. If you print raw data passed to the ISocket.Send, you will not see "PWD", but something like this:

  ↨♥♥ P?3???#V??§?♣????2?♀=←?!??

Conclusion:

It would be very hard to workaround this issue when using TLS. The suggested  solution is to not use the GetItems() method, but to call GetList() recursively instead. I will post sample code as anoter answer later. Unfortunately, the multi-file (recursive) operations are not supported for your FTP server.

The last possible workaround:

1. If you are using TLS in EXPLICIT mode.
2. If you need to secure authentication and data transfers only, you can unprotect control connection after logging in. It can be done like this:

    client.Connect("test.rebex.net", SslMode.Explicit);
    client.Login("demo","password");
    client.ClearCommandChannel();

Answer 2

If recursive Ftp.GetItemsAsync() cannot be used, you can write your own traversal logic like this:

public static async Task<IEnumerable<FtpItem>> GetItemsAsync(Ftp client, string parentPath)
{
    var result = new List<FtpItem>();
    var list = await client.GetListAsync();
    foreach (var item in list)
    {
        // determine correct path
        string path = parentPath.TrimEnd('/') + "/" + item.Name;
        result.Add(new FtpItem(path, item.Name, item.Length, item.Type));

        if (item.IsDirectory)
        {
            // recursively traverse directories
            client.ChangeDirectory(item.Name);
            result.AddRange(await GetItemsAsync(client, path));
            client.ChangeDirectory("..");
        }
    }
    return result;
}

The code can be used like this:

using (var client = new Ftp())
{
    client.Connect("test.rebex.net", SslMode.Implicit);
    client.Login("demo", "password");

    // specify desired directory or "." for current dir
    string directoryToList = ".";

    // navigate to desired directory or resolve "."
    if (directoryToList == ".")
        directoryToList = client.GetCurrentDirectory();
    else
        client.ChangeDirectory(directoryToList);

    // get items recursively and print output
    var list = await GetItemsAsync(client, directoryToList);
    foreach (var item in list)
    {
        Console.WriteLine(item.Path);
    }
}

Comments

Thanks Lukas.
I just realized that there are so many differences in an MVS ftp server I need to handle. I assume Rebex does not currently support that?

For example, the listing format is completely different:
Volume Unit    Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname

And also subfolders are separated by "." instead of the "/"

---

For multi-file operations (such as GetItems/Upload/Download) only UNIX-like FTP servers are supported.

If you need to work with other kinds of server, you need to use single-file operations (GetList/PutFile/GetFile).

The GetItemsAsync method I posted here should work for you. You only need to replace  path adjustment routine like this:

  string path = parentPath.TrimEnd('.') + "." + item.Name;