Email crawling inside multiple threads

Question

Hello,

I am using Rebex.NET library for crawling different email accounts (Gmail/Yahoo/Hotmail). We are creating separate threads for each email address to crawl inside a windows service. But single email account gets crawled multiple times.

Code:

   ThreadPool.SetMinThreads(1, 1);
   ThreadPool.SetMaxThreads(maxThreads, 1);

   for (int usersIndex = 0; usersIndex < userAccountsToCrawl.Count(); usersIndex++)
            {
                ThreadPool.GetMaxThreads(out maxThreads, out placeHolder);
                ThreadPool.GetAvailableThreads(out availableThreads, out placeHolder);

                if (availableThreads == 0)
                {
                    Thread.Sleep(TimeSpan.FromMilliseconds(Convert.ToDouble(Itemize.BussinessLayer.EmailCrawlingBL.EmailCrawlerSetting.Default.ThreadSleepTimeMilliSecs)));
                    usersIndex--;
                }
                else
                {
                    ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessUser), userAccountsToCrawl[usersIndex]);
                }
            }

Please let me know whether this is any issue with Rebex.NET while multiple threads trying to crawl email account.

Regards Sanjay

Answer 1

There is no interaction between instances of rebex mail objects (Rebex.Net.Pop3 or Rebex.Net.Imap). Each instance runs indenpendently on the other. No credentials or connections are shared or cached.

I'm afraid the problem should be within your ProcessUser method.

We have not encountered some issue similar to this.

Comments

Thanks Jan for reply. We are monitoring our multithreading related code and will update you accordingly.