GetDownloadStream vs Download SFTP

Question

I am looking for the pros and cons of reading each individual file as a stream vs. pulling all the files to a local disk and reading them from there.

In my situation I need to:

1. Read the files from a remote server,
2. Update a database with information from the files
3. Save those files to another remote location,
4. Delete them from the original SFTP server.

I know myClient.Download has the ability to perform Tasks 1,3 and 4
in one line of code but then I have to read from another hard-disk (not using SFTP).

so I am wondering of these two methods, is one more performant or reliable with multiple files? Is there anything else I should consider?

https://www.rebex.net/doc/api/Rebex.Net.Sftp.html

Answer 1

You can download files from SFTP server using 3 methods:

1. Sftp.Download()
2. Sftp.GetFile()
3. Sftp.GetDownloadStream()

There should be no significant performance differences between the methods for large files (the code for transferring data is same).

The methods offer different functionality:

1. Sftp.Download() - High level API for downloading multiple files or directories, use can use wildcards in path, you can specify Overwrite mode, etc.
2. Sftp.GetFile() - Simple API for downloading single file (to file or stream).
3. Sftp.GetDownloadStream() - Low level API which enables you to access remote file on stream basis. You can seek to specific location, read block by block, etc.

It depends on your needs:

1. If you want to download multiple files in single line of code, I suggest you to use Sftp.Download().
2. If you want to download single file (or one part of it), I suggest you to use Sftp.GetFile() method.
3. If you want access remote file on stream basis or you want to download more than one part of it, I suggest you to use Sftp.GetDownloadStream().

Comments

I appreciate the clarification. In my situation I am not dealing with a few large files but many small ones. I was actually wondering if there was any kind of magic-batch-processing that Sft.Download() might be doing to reduce server load when downloading multiple files.

---

No, there is no magic-batch-processing in Sft.Download(). Performance should be comparable for single Sftp.Download() call or multiple Sftp.GetFile() calls.