0 votes
by (190 points)
edited

I'm using Rebex.Net.Scp to read in groups of files. I was hoping that I could use BeginGetFile to start 10 asynchronous downloads, and then wait until all 10 have completed. I can do this, but it is the same amount of time as downloading each file serially with ReadMemory.

Is there a way to read multiple files simultaneously to speed up the transfer?

Thanks - Brad

by (73.5k points)
edited

We are able to reproduce the issue, but have not the cause yet. We will keep you updated. Thanks for bringing it to our attention.

by (190 points)
edited

I was able to get it working, but I get errors.
I'm trying to download a group of 10 files at a time. I created a new SCP object for each file I am downloading in the group. I have one SSH object that I Bind to each SCP. By doing this I can get it simultaneously download 10 files at a time. It is quite a bit faster, but I keep getting errors. I'm wondering if the SCP object is thread safe?

by (73.5k points)
edited

Both Scp and SshSession should be thread safe. Unfortunately we have found some threading bugs. I am sending you current beta build at your email. Please give it a try.

1 Answer

0 votes
by (73.5k points)
edited

We have performed speed tests against several servers and we have come to the conclusion that the performance depends on the server itself. With the same code, we got totally different results for different servers. Simply some servers can use a session at full and some cannot. You should do your own testing to determine which method is the fastest against your server. The offered methods are following:

  1. Make it whole synchronously (use one Scp object and call GetFile for each file)
  2. Use one Scp for all files (uses one SshSession), call BeginGetFile for each file
  3. Use one Scp for each file over one shared SshSession, call BeginGetFile on each Scp object
  4. Use one Scp for each file every with its own SshSession, call BeginGetFile on each Scp object
...