Email crawling inside multiple threads

0 votes
asked Apr 20, 2011 by Radioactive (270 points)
edited Apr 21, 2011

Hello,

I am using Rebex.NET library for crawling different email accounts (Gmail/Yahoo/Hotmail). We are creating separate threads for each email address to crawl inside a windows service. But single email account gets crawled multiple times.

Code:

   ThreadPool.SetMinThreads(1, 1);
   ThreadPool.SetMaxThreads(maxThreads, 1);

   for (int usersIndex = 0; usersIndex < userAccountsToCrawl.Count(); usersIndex++)
            {
                ThreadPool.GetMaxThreads(out maxThreads, out placeHolder);
                ThreadPool.GetAvailableThreads(out availableThreads, out placeHolder);

                if (availableThreads == 0)
                {
                    Thread.Sleep(TimeSpan.FromMilliseconds(Convert.ToDouble(Itemize.BussinessLayer.EmailCrawlingBL.EmailCrawlerSetting.Default.ThreadSleepTimeMilliSecs)));
                    usersIndex--;
                }
                else
                {
                    ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessUser), userAccountsToCrawl[usersIndex]);
                }
            }

Please let me know whether this is any issue with Rebex.NET while multiple threads trying to crawl email account.

Regards Sanjay

Applies to: Rebex Secure Mail

1 Answer

0 votes
answered Apr 21, 2011 by Jan Sotola (16,920 points)
edited Apr 21, 2011

There is no interaction between instances of rebex mail objects (Rebex.Net.Pop3 or Rebex.Net.Imap). Each instance runs indenpendently on the other. No credentials or connections are shared or cached.

I'm afraid the problem should be within your ProcessUser method.

We have not encountered some issue similar to this.

commented Apr 21, 2011 by Radioactive (270 points)
edited Apr 21, 2011

Thanks Jan for reply. We are monitoring our multithreading related code and will update you accordingly.

...