Timeout exception using FTP component

Question

Hi

I have downloaded FTP component and I am trying to download files from the FTP server. I have got thousand of files on FTP server. Initially I am able to download few hundred files but after 5 minutes or so its throwing the below exception

"A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond."

what I don't understand is why it is throwing exceptions after downloading couple of hundred files. Am i missing something here

Comments

In FTP protocol, a new TCP/IP connection is established for each transfer. But this doesn't explain why it starts failing after few hundred files... Please create a communication log of your transfer using Ftp object's LogWriter property (as described at http://www.rebex.net/kb/logging.aspx) and mail it to support@rebex.net for analysis - we might spot something there!

Answer 1

Thanks for replying.

To explain my requirement in more details I have multiple folders in my root directory on FTP server and in each directory i have sub folders and files. I have wrote the following code

    Dim client As New Ftp
    client.Connect(ConfigurationManager.AppSettings("Host"), 21)
    client.Login(ConfigurationManager.AppSettings("Username"), ConfigurationManager.AppSettings("Password"))
    client.GetFiles("/*", "c:\FTP\Test", FtpBatchTransferOptions.Recursive, FtpActionOnExistingFiles.SkipAll)

As i have mentioned before I could download only few hundered files from the first folder and after 5-6 minutes it get disconnected by throwing the follwing exception.

"A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond."

Thanks Ritesh

Answer 2

Please add the following line your code (just above the client.Connect call):

client.LogWriter = new Rebex.FileLogWriter("c:\FTP\ftp.log", Rebex.LogLevel.Debug)

And mail the log it produces to us for analysis. That might give us some idea on what's going on! (Change the log path if needed, of course.) Thanks!

Answer 3

Thans for the reply Lukas

I am using Rebex FTP component to download files from FTP server. My requirement is as follows

I have multiple folders in my root directory on the FTP server and each folder has sub folders and hundreds of files. To download the files I wrote the following code:

Public Sub DownloadCallFiles()
Try
            Dim client As New Ftp
            client.LogWriter = New Rebex.FileLogWriter("c:\ftp\log.txt", Rebex.LogLevel.Debug)
            client.Connect(ConfigurationManager.AppSettings("Host"), 21)
            client.Login(ConfigurationManager.AppSettings("Username"),ConfigurationManager.AppSettings("Password"))
            client.GetFiles("/*", "c:\FTP\Test", FtpBatchTransferOptions.Recursive, FtpActionOnExistingFiles.OverwriteDifferentSize)
        Catch ex As Exception
             DownloadCallFiles()
        Finally

        End Try
End Sub

The problem I am facing is after downloading couple of folders and few hundred files (which is taking about 5-6 minutes) it’s getting disconnected by throwing the following exception

"A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond."

What I have done is I am calling the same subroutine in my catch block and that way I am able to download all the folders and files.

My second question is before downloading the file can I check some filters. i.e. check the remote file with the local file. I have to check two things, say the remote file name is test.csv

1) Check if the remote file has been downloaded then do not download again

After downloading the file we import the local files into database and add a extension to it say test.tkn so I have to check the same file twice

For example : If Test.csv exist on local folder or test.tkn exist on local folder then do not download

Do you have any example to accomplish such type of task.

Regards Ritesh

Answer 4

Hello Ritesh! The problem is probably not with your source code, but more probably in the server, or Rebex components. So we need to see the Rebex log. Create it directly in your program as described above! Add the line of code into your program which creates the LOG and post it here. Then we will be surely able to find out what is the problem. If you do not send the log file we will not be able to help you.

Answer 5

I have attached the log file to the email sent to support@rebex.net

Answer 6

Hello Ritesh! Thank you for the log file. We are not sure as to what is causing the exception. It looks like the server simply chokes from time to time. In the latest release there is a new class FileSet which enables to individually download files matching a filter. Please try the code below. Does it work better? Are there still exceptions thrown by the server after a few hundred times have been downloaded?

And please note the handling of already downloaded files:

Dim sourcePath = "/"
Dim targetPath = "c:\data"

Private Function GetClient() As Ftp
    Dim client = New Ftp()

    ''# connect and login
    client.Connect("server")
    client.Login("user", "pass")

    ''# change to desired directory
    client.ChangeDirectory(sourcePath)

    Return client
End Function

Sub Main()
    ''# connect to the ftp server
    Dim client = GetClient()

    ''# get the list of all remote items under the current directory
    Dim list As FtpItemCollection = client.GetItems("*")

    ''# download all files
    For index As Integer = 0 To list.Count - 1
        Dim remoteItem As FtpItem = list(index)
        If remoteItem.IsFile Then

            ''# prepare local path of the file
            Dim localPath As String = Path.Combine(targetPath, remoteItem.Path)
            Dim localDirectory As String = Path.GetDirectoryName(localPath)

            ''# create local directory if not exists
            If Not Directory.Exists(localDirectory) Then
                Directory.CreateDirectory(localDirectory)
            End If

            ''# try to transfer the file individually
            Try
                client.GetFile(remoteItem.Path, localPath)
            Catch ex As FtpException
                ''# reconnect client if server chunks
                If client.GetConnectionState().Connected Then client.Disconnect()
                client = GetClient()

                ''# resume failed transfer
                index -= 1
                Continue For
            End Try
        End If
    Next

    client.Disconnect()
End Sub

Does it help?

Comments

Thanks for the example. I could able to download one file at a time and also check for the conditions as well but I have another problem i.e. in my root folder i have multiple folders for example (Folder1,Folder2,Folder3,Folder4,Folder5) and each folder has subfolders. The above example is downloading files and sub folders only from Folder1, how do i get files from Folder2,Folder3 and so on all at one go.

Regards