sftp component adding extra carriage return to the file after each line

0 votes
asked Jun 29, 2018 by Ritesh (350 points)

Hi There

I am using an SFTP component to download the file from the SFTP server. Its downloading the files however its adding an extra ascii carriage return after each line in the downloaded file. how can I remove the added carriage line (code 10) after each line.

I am setting the transfer type as
SFTPclient.TransferType = SftpTransferType.Binary

Could you please help me on this.


Applies to: Rebex SFTP

1 Answer

0 votes
answered Jun 29, 2018 by Lukas Matyska (60,770 points)

Can you please ensure that the carriage return (0x0D) is not already present in the source file on the remote? You can check it by executing xxd command on the server like this:

xxd /remote/file.txt

Do you experience the same behavior using PuTTY's psftp.exe utility?
You can download the file like this:

psftp.exe -pw password user@host
get /remote/file.txt c:/local/file.txt 
commented Jun 29, 2018 by Ritesh (350 points)
we used to download the files using our bespoke application and we used to get the content in a file in one line  like

 B000000000000033002018062820180628201806290000260033027940000250200000T3335    8989898989898989898  000000000000000000T3330500000000x1 02 011095438AATY1                          000000G443333000000000000000000000000000                                          TT0489    454590976913620180628185352    

where as when I download using Rebex SFTP I get content in separate line  as follows

G0xx4    00000000000                244420003320000000sss216005000000ddd000000F1
G0xx5    00000000000                244420003320000000sss2160050000dff00000000F1
G0xx1    00000000000                244420ddf003320000000sss216005000000000000F1
G0xx2    00000000000                244420003320dfd000000sss216005000000000000F1
G0xx 3   00000000000                24442dfd0003320000000sss216005000000000000F1
commented Jul 2, 2018 by Lukas Matyska (60,770 points)
The two samples you provided are from two different files. Can you please ensure that the file doesn't contain carriage return (0x0D) at the server already?

You can do it by `xxd` command if available on the server. Or by downloading using other SFTP tool (e.g.  PuTTY's psftp.exe utility, or WinSCP, or FileZilla).

Please note that in SftpTransferType.Binary mode, we do not manipulate transferred data. It is very probable that the carriage return (0x0D) is received in incoming data.
commented Jul 2, 2018 by Ritesh (350 points)
The sample are from  same files. However I have changed the content intentionally (Randomly).

I have downloaded the file using Filezilla and even that is adding a extra line feed.

I am downloading files from an Unix based platform to a windows based platform.
It seems that line feed in a file in  Unix is different to a file in windows based.

So it looks like our program which is running on windows platform to import the files is not happy because the line feed is Unix based.

How can I solve the line feed issue in two different platform.
commented Jul 2, 2018 by Ritesh (350 points)
Could someone please help me on this as soon as possible as I have a deadline. Your help is much appreciated.

commented Jul 2, 2018 by Lukas Matyska (60,770 points)
Our suggestions are:

1. Choose one format: either unix-style <LF> or windows-style <CR><LF> and use it on all platforms.

2. Always use BINARY mode to transfer files to prevent file changes (for both upload and download).
commented Jul 2, 2018 by Ritesh (350 points)
Thanks Lukas for a prompt reply. Really appreciate it.

The problem is we are downloading files from a third party server which is Unix based and we are downloading files on windows based platform. Is there any way we can change the new line feed character before downloading the file?
commented Jul 3, 2018 by Lukas Matyska (60,770 points)
You cannot modify data on server before download, but you can modify data after download on client.

The example solution is here http://forum.rebex.net/8688/removing-replacing-line-feed-from-text-file-sftp-memory-stream
commented Jul 3, 2018 by Lukas Pokorny (126,910 points)
Actually, they can modify data on the server before download by running a command such as fromdos/todos at the server, possibly executed using Rebex Terminal Emulation (see https://www.rebex.net/terminal-emulation.net/features/easy-to-use-api.aspx#simple-commands). However, this seems like a very odd solution. Downloading the file as-is and replacing the end-of-line sequences is definitely the preferred method.
commented Jul 4, 2018 by Ritesh (350 points)
Thanks Lukas for all your help.