SFTP SftpServerType.Unix+SftpTransferType.Ascii not working for Unicode file

Question

I am uploading a Unicode-encoded file from Windows to a Unix SFTP server via the following code:

using (Sftp sftp = new Sftp())
{
    // Settings
    sftp.Timeout = requestTimeout;
    sftp.ServerType = SftpServerType.Unix;
    sftp.TransferType = SftpTransferType.Ascii;

    // Connect to SFTP server
    sftp.Connect(serverHostname);

    // Login 
    SshPrivateKey privateKey = new SshPrivateKey(privateKeyPath, "");
    sftp.Login(username, privateKey);

    // Upload the file
    sftp.PutFile(localFilePath, remoteFilePath);
}

However, the file gets uploaded to the server with "ÿþ" at the beginning of the document, and "^M" at the end of each line--it appears it did not get encoded properly.

For example, I can create a file using the following code:

string tempPath = Path.GetTempFileName();
string[] lines = new string[] 
{
    "this is line 1",
    "this is line 1",
    "this is line 1",
    "this is line 4"
};
File.WriteAllLines(tempPath, lines, Encoding.Unicode);

When I upload it, what I see on the server is:

ÿþthis is line 1^M
this is line 1^M
this is line 1^M
this is line 4^M

What could be causing this to happen? I am using version 2.0.4086.0 of Rebex.Net.Sftp.dll

Answer 1

Hello,

The two strange characters are 0xFF,0xFE and represent a Unicode byte-order-mark (BOM). They were added to the file bacause you specified Encoding.Unicode, which instructs the File.WriteAllLines method (and others) to emit BOMs.

To disable BOM emiting, use new UnicodeEncoding(false, false) instead of Encoding.Unicode:

    File.WriteAllLines(tempPath, lines, new UnicodeEncoding(false, false));

This said, SftpTransferType.Ascii only works properly for ASCII-like charsets - this includes ASCII, windows-1252, iso-8859-1 and UTF-8, but not UTF-16 which is what you are using. Please use SftpTransferType.Binary instead and force Unix-style end-of-lines when creating the local file using the following code instead of File.WriteAllLines:

    using (var writer = new StreamWriter(tempPath, false, new UnicodeEncoding(false, false)))
    {
        writer.NewLine = "\n";
        foreach (string line in lines)
        {
            writer.WriteLine(line);
        }
    }

This should do the trick! :-)

Comments

Thanks, this is the answer I was looking to confirm, "SftpTransferType.Ascii does not support UTF-16".

---

Also, thanks for pointing out StreamWriter.NewLine, I did not realize that existed.