Problem Connecting to ftp while transferring using Rebex.Net.Ftp

0 votes
asked Jul 20, 2010 by user-181 (google) (200 points)
edited Jul 12, 2012

Adding more threads to upload while transferring, i get these errors while setting properties or connecting:

Stack Traces:

Rebex.Net.FtpException: Another operation is pending. at Rebex.Net.Ftp.clXkcGZ() at Rebex.Net.Ftp.bNVhrT() at Rebex.Net.Ftp.set_UploadBufferLength(Int32 value)

Rebex.Net.FtpException: Another operation is pending. at Rebex.Net.Ftp.clXkcGZ() at Rebex.Net.Ftp.bNVhrT() at Rebex.Net.Ftp.set_Timeout(Int32 value)

Rebex.Net.FtpException: Another operation is pending. at Rebex.Net.Ftp.clXkcGZ() at Rebex.Net.Ftp.bNVhrT() at Rebex.Net.Ftp.set_AbortTimeout(Int32 value)

Rebex.Net.FtpException: Another operation is pending. at Rebex.Net.Ftp.clXkcGZ() at Rebex.Net.Ftp.bNVhrT() at Rebex.Net.Ftp.Connect(String serverName, Int32 serverPort, TlsParameters parameters, FtpSecurity security)

the properties are set with these values:

ftp.UploadBufferLength = 64000; ftp.Timeout = 120000; ftp.AbortTimeout = 1200000;

even if I don't set those properties, it throws the exception when connecting.

PS: The problem doesn't happen if I add a thread to upload while it isn't transferring yet.

Applies to: Rebex FTP/SSL

2 Answers

+2 votes
answered Jul 20, 2010 by Lukas Pokorny (93,630 points)
edited Jul 12, 2012
 
Best answer

This exception is thrown if you call a method or property of the Ftp object multiple times at the same time from several different threads. The Ftp object represents a single FTP session (or connection) that doesn't support multiple concurrent operations.

It's perfectly OK to use multiple threads to perform multiple simultaneous operations, but you have to use several instances of the Ftp object to make this possible. Using one Ftp instance per thread is the easiest way to achieve this (another is using a pool of Ftp objects, but this is more complicated and probably not needed in this case).


It's not entirely clear what is going on from your code snippet. I tried extending it to form the simplest possible console application I could use to reproduce the problem you describe, and this is what I ended up with:

using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;
using Rebex.Net;

namespace FtpThreads
{
    public class FtpThreadTest
    {
        // added to make it compile
        private static class ConnectionSettings
        {
            public static string proxy_host { get { return null; } }
            public static int proxy_port { get { return 1080; } }
            public static bool proxy_anonymous { get { return true; } }
            public static string proxy_username { get { return null; } }
            public static string proxy_password { get { return null; } }
            public static int proxy_type_index { get { return 0; } }
            public static FtpProxyType getProxyType(int index) { return FtpProxyType.None; }

public static string ftp_host { get { return "ftp.rebex.net"; } }
            public static int ftp_port { get { return 21; } }
            public static bool ftp_anonymous { get { return false; } }
            public static string ftp_username { get { return "anonymous"; } }
            public static string ftp_password { get { return "guest"; } }

public static bool ftp_passivemode { get { return true; } }
            public static int ftp_protocol_index { get { return 0; } }
            public static FtpSecurity getConnectSecurityType(int index) { return FtpSecurity.Unsecure; }
        }

// added to make it compile
        public class Transfer
        {

}

public void CreateNewThread(Transfer trans) // made public to make it possible to test this
        {
            Thread trd; // added

trd = new Thread(new ParameterizedThreadStart(this.LoadFtp));
            trd.IsBackground = true;
            trd.Priority = ThreadPriority.Normal;
            trd.Start(trans);
        }

private void LoadFtp(object type)
        {
            Ftp transfer;
            if ((Ftp)type == null)
            {
                transfer = new Ftp();

transfer.StateChanged += new FtpStateChangedEventHandler(client_StateChanged);
                transfer.TransferProgress += new FtpTransferProgressEventHandler(transfer_TransferProgress);
            }
            else
            {
                //transfer = (Transfer)type; // not sure what this was supposed to do
                transfer = (Ftp)type; // this at least compiles
            }

FtpState state = OpenConnection(transfer);

// do something else to else it works
            if (state == FtpState.Ready)
            {
                Console.WriteLine("Connected successfully from thread {0}.", Thread.CurrentThread.ManagedThreadId);

// get list of files - this takes some time
                transfer.GetList();
            }
            else
            {
                Console.WriteLine("Not connected successfully from thread {0}: {1}", Thread.CurrentThread.ManagedThreadId, state);
            }

// let the garbage collector dispose the object later
        }

// added to make it compile
        void transfer_TransferProgress(object sender, FtpTransferProgressEventArgs e)
        {

}

// added to make it compile
        void client_StateChanged(object sender, FtpStateChangedEventArgs e)
        {

}

private FtpState OpenConnection(Ftp handler)
        {
            //try // no matching catch or finally block was found
            {
                TlsParameters par;
                lock (handler)
                {
                    handler.Proxy.ProxyType = ConnectionSettings.getProxyType(ConnectionSettings.proxy_type_index);
                    handler.Proxy.Host = ConnectionSettings.proxy_host;
                    handler.Proxy.Port = ConnectionSettings.proxy_port;

if (ConnectionSettings.proxy_anonymous)
                    {
                        handler.Proxy.UserName = String.Empty;
                        handler.Proxy.Password = String.Empty;
                    }
                    else
                    {
                        handler.Proxy.UserName = ConnectionSettings.proxy_username;
                        handler.Proxy.Password = ConnectionSettings.proxy_password;
                    }

handler.Passive = ConnectionSettings.ftp_passivemode;

handler.UploadBufferLength = 65000;
                    handler.Timeout = 120000;
                    handler.AbortTimeout = 1200000;

handler.Options = FtpOptions.KeepAliveDuringTransfer;
                    handler.KeepAliveDuringTransferInterval = 90;

par = new TlsParameters();
                    //par.CertificateVerifier = new FingerprintVerifier();
                    par.CertificateVerifier = CertificateVerifier.AcceptAll; // just to make it work easily
                }
                try
                {
                    handler.Connect(ConnectionSettings.ftp_host, ConnectionSettings.ftp_port, par, ConnectionSettings.getConnectSecurityType(ConnectionSettings.ftp_protocol_index));
                }
                catch (StackOverflowException ex) // not sure why this is here - runtime stack overflows are usually fatal so the app would exit before your catch it
                {
                    //SetText(handler.ToString() + ": " + ex.Message + "\r\n", textBox1);
                    Console.WriteLine(ex); // just display it
                }
                finally { }
                if (!ConnectionSettings.ftp_anonymous)
                    handler.Login(ConnectionSettings.ftp_username, ConnectionSettings.ftp_password);
            }

return handler.State; // added
        }

}

// added to test the class
    class Program
    {

static void Main(string[] args)
        {
            FtpThreadTest threadCreator = new FtpThreadTest();

for (int i = 0; i < 8; i++)
            {
                Console.WriteLine("Creating new thread.", i);
                threadCreator.CreateNewThread(null);
            }

Console.WriteLine("Press any key to exit.");
            Console.ReadKey();

}
    }
}

When I run this, it works fine and "Another operation is pending" error doesn't occur. We are unable to reproduce the issue you describe. To make sure you are in fact using multiple instances of Ftp object, please add the following line right below the transfer = new Ftp():

// create a log file
transfer.LogWriter = new Rebex.FileLogWriter(@"c:\temp\ftplog.txt", Rebex.LogLevel.Debug); // modify the path if needed

This will create a communication log where different instances of Ftp object are clearly distinguishable.

We know people have been using multiple Ftp objects from multiple threads for years without any problems, so unless there is a bug in your code, it is something very hard to reproduce. Posting a code that triggers the error (or modifying the test code above) would really help.

For a sample code that uses multiple threads to upload a list of files, check out the FtpsMultiUploader project. It doesn't raise "Another operation is pending" either, or at least no one reported that yet.


<
commented Jul 20, 2010 by user-181 (google) (200 points)
I'm using several instances of the Ftp object. Look at my code: http://pastebin.com/pjyHQpvQ when creating a new thread I always pass null to create a new Ftp instance. I only pass the Ftp object to LoadFtp if I want to reconnect.
commented Jul 20, 2010 by Lukas Pokorny (93,630 points)
Thanks! I am still unable to reproduce the issue - please see my updated answer for more information.
commented Jul 20, 2010 by user-181 (google) (200 points)
Looks like its generating the threads fine. Here is the log of me adding a first thread and after starts uploading I add a second thread: http://pastebin.com/ggteJ49t
commented Jul 21, 2010 by Lukas Pokorny (93,630 points)
It looks like the first thread created an Ftp object (identified as Transfer(1)) and did its work without problems. The second thread created a new Ftp object (Transfer(2)) and did what it was supposed to do as well, but after it started closing the connection, an error occured - it looks like another thread tried using the same Ftp object to do something else. Perhaps you called the Disconnect method but then started re-using the object before it finished?
commented Jul 21, 2010 by Lukas Pokorny (93,630 points)
The log might be even more informative if it includeed thread IDs. The built-in FileLogWriter doesn't support this feature, but it's simple to extend it to add that. I updated my answer again to include a FileLogWriter2 class that can be used instead of FileLogWriter to produce a log that includes the trhead ID as well.
0 votes
answered Jul 21, 2010 by user-181 (google) (200 points)
edited Jul 21, 2010

I fixed. Thanks, I used a threadpool and fixed the issue...

Thanks very much for helping me out

commented Jul 22, 2010 by Lukas Pokorny (93,630 points)
Thanks for letting us know, I'm glad you fixed it!
...