Upload ascii files to mainframe based ftp server

0 votes
asked May 15, 2013 by CER (420 points)
edited Jul 29, 2013

We are using rebex.ftp.dll 4.0.4444.0 to upload small text files (actually swift messages) from a windows client to a mainframe based ftp server. TransferType = FtpTransferType.Ascii, EnabledExtensions = 0, TransferMode = Stream, Encoding = Default.

The file ends up on the mainframe side in EBCDIC as expected. But, if we send the same file 10 times in a row with distinct filenames, some of them ends up with invalid contents on the server. really cant figure out why!

We upload them one at the time using Ftp.PutFile(localfile, remotefile) The call succeeds, we dont get any exceptions when doing it.

I cant get access to the ftp server from my development environment, only through vpn at customer site where I have to remote desktop to, so its pretty hard to research these things. I blame it on the vpn connection, customer thinks we dont understand ftp or ebcdic, they say it worked fine in the old version, where we used darts ftp component.

dont think its related - but who does the character translation from ascii to ebcdic? if its the rebex component, then, what triggers this translation?

I cant think of anything which could disturb the transfer - anyone have a good idea? Perhaps Transfermode = Block makes a difference on a mainframe?

5 Answers

+1 vote
answered May 15, 2013 by CER (420 points)
edited May 15, 2013

Let me answer that myself.

Setting TransferMode=block solved the problem of some files being uploaded incorrectly. Never seen that problem before. After some investigation I discovered that out the old ftp component detected the servertype and automatically adjusted the transfermode accordingly.

But Im still not sure where the translation from ascii to ebcdic takes place or how / when. Im sure its inside the rebex client, but I cant find any documentation on this. how do I force translation to ebcdic during ascii transfer if needed?

+1 vote
answered May 15, 2013 by Tomas Knopp (58,890 points)
edited May 15, 2013

Ftp client sends data the same way either using FtpTransferType.Ascii or FtpTransferType.Binary.

Client only notifies the server about the transfer type (so that server know whether to perform conversions (in ASCII mode) or not (Binary mode).

According to RFC959. ASCII transfer type in FTP protocol works like this:

FTP client sends a file in ASCII encoding. It is up to the server to properly decode from ASCII and encode to the server's encoding (in your case translating to EBCDIC).

+1 vote
answered May 15, 2013 by CER (420 points)
edited May 15, 2013

Ok, thanks. I wasnt aware of that, always thought the client had to sort it out.

Now back to wondering why I wasnt told to ensure that transfermode was set to block instead of stream. Perhaps its just common knowledge when talking to a mainframe based ftp server.

0 votes
answered May 16, 2013 by Tomas Knopp (58,890 points)
edited May 16, 2013

Sorry for late response, we do not get notified of comment's edits. In fact, this is the first time we have a customer reporting that switching to FtpTransferMode.Block was needed for the server to accept files properly.

Rebex FTP/SSL uses FtpTransferMode.Stream by default and we do not perform any automatic TransferMode detection.

If you would like Rebex to have this detected automatically you can send us a log from the Dart component you previously used and a log from Rebex FTP/SSL component which can be created as described in this article. We would then be able to look into this issue. Thanks!

0 votes
answered May 23, 2013 by CER (420 points)
edited Jul 29, 2013

It was a series of unfortunate events which led to this problem. First of all, we didnt know they needed transfertype=ascii and transfermode=blockmode. But after that we actually got a couple of files though fine and we saw an error message from the server which made us reconnect and after the reconnect the transfermode was suddendly reset to binary. I had to change our use of the rebex ftp to ensure that transfertype was set correctly after reconnecting. so,

Ftp Client = null;

void Initialise() {
    Ftp Client = new Ftp { EnabledExtensions = FtpExtensions.All ^ ConnectionSettings.DisabledFeatures, Timeout = (int)BaseSettings.OperationTimeout.TotalMilliseconds, Passive = ConnectionSettings.Passive, TransferMode = ConnectionSettings.TransferMode, TransferType = ConnectionSettings.Binary ? FtpTransferType.Binary : FtpTransferType.Ascii };
}

void Connect() {
    var tlsParameters = new TlsParameters();
    Client.Connect(ConnectionSettings.Host, ConnectionSettings.Port, tlsParameters, FtpSecurity.Unsecure);
    Client.Login(ConnectionSettings.Username, ConnectionSettings.Password);
}

is now changed to

void Initialise() {
    Client = new Ftp { EnabledExtensions = FtpExtensions.All ^ ConnectionSettings.DisabledFeatures, Timeout = (int)BaseSettings.OperationTimeout.TotalMilliseconds, Passive = ConnectionSettings.Passive };
}

void Connect() {
    var tlsParameters = new TlsParameters();
    Client.TransferMode = ConnectionSettings.TransferMode;
    Client.Connect(ConnectionSettings.Host, ConnectionSettings.Port, tlsParameters, FtpSecurity.Unsecure);
    Client.Login(ConnectionSettings.Username, ConnectionSettings.Password);
    Client.TransferType = ConnectionSettings.Binary ? FtpTransferType.Binary : FtpTransferType.Ascii;
}

So, im always setting transfermode and transfertype on every connect and reconnect. Can anyone confirm that this is actually necessary?

commented May 24, 2013 by Lukas Pokorny (102,210 points)
edited May 24, 2013

Yes, I confirm this is actually necessary, although I would say this is actually a bug. Old versions of Rebex FTP had SetTransferType method instead of TransferType property, and the behavior you describe is most likely a leftover from that time.

commented Jul 29, 2013 by Lukas Pokorny (102,210 points)
edited Jul 29, 2013

This has been fixed in Rebex FTP/SSL 2013 R1.

...