When I set the parallelism to 1 (only one thread) I pretty much can get end to end in about 2 seconds, see the Console output from code below:
Please notice, the time between file uploaded and command executed is only about 1 second or so.
However, if I set the parallelism to 10 (10 concurrent thread, I have I7 CPU, 4 cores, Windows 7), I got something like following
Thread 0, 11:03:09:6928, request Id 3921126483 received.
Thread 6, 11:03:09:6968, request Id 1444956043 received.
Thread 2, 11:03:09:6978, request Id 5699697281 received.
Thread 8, 11:03:09:6968, request Id 1908922957 received.
Thread 4, 11:03:09:6978, request Id 2977101433 received.
Thread 6, 11:03:09:9088, File 1444956043 TARed.
Thread 8, 11:03:09:9088, File 1908922957 TARed.
Thread 2, 11:03:09:9098, File 5699697281 TARed.
Thread 4, 11:03:09:9178, File 2977101433 TARed.
Thread 0, 11:03:09:9178, File 3921126483 TARed.
Thread 1, 11:03:10:4948, request Id 3077438486 received.
Thread 1, 11:03:10:5168, File 3077438486 TARed.
Thread 3, 11:03:11:0478, request Id 8839964099 received.
Thread 0, 11:03:11:0568, 3921126483 session connected.
Thread 6, 11:03:11:0568, 1444956043 session connected.
Thread 8, 11:03:11:0568, 1908922957 session connected.
Thread 1, 11:03:11:0598, 3077438486 session connected.
Thread 2, 11:03:11:0618, 5699697281 session connected.
Thread 4, 11:03:11:0638, 2977101433 session connected.
Thread 3, 11:03:11:0758, File 8839964099 TARed.
Thread 1, 11:03:11:1938, 3077438486 session authenticated.
Thread 8, 11:03:11:1948, 1908922957 session authenticated.
Thread 2, 11:03:11:2168, 5699697281 session authenticated.
Thread 0, 11:03:11:2178, 3921126483 session authenticated.
Thread 4, 11:03:11:2568, 2977101433 session authenticated.
Thread 6, 11:03:11:2768, 1444956043 session authenticated.
Thread 3, 11:03:11:3668, 8839964099 session connected.
Thread 8, 11:03:11:3998, 1908922957 sftp session bound.
Thread 6, 11:03:11:3998, 1444956043 sftp session bound.
Thread 1, 11:03:11:3998, 3077438486 sftp session bound.
Thread 4, 11:03:11:4038, 2977101433 sftp session bound.
Thread 0, 11:03:11:3998, 3921126483 sftp session bound.
Thread 2, 11:03:11:3998, 5699697281 sftp session bound.
Thread 3, 11:03:11:4588, 8839964099 session authenticated.
Thread 6, 11:03:11:5228, File 1444956043 uploaded.
Thread 1, 11:03:11:5238, File 3077438486 uploaded.
Thread 4, 11:03:11:5298, File 2977101433 uploaded.
Thread 6, 11:03:11:5328, 1444956043 ssh client session bound.
Thread 4, 11:03:11:5338, 2977101433 ssh client session bound.
Thread 1, 11:03:11:5328, 3077438486 ssh client session bound.
Thread 0, 11:03:11:5308, File 3921126483 uploaded.
Thread 0, 11:03:11:5398, 3921126483 ssh client session bound.
Thread 8, 11:03:11:5238, File 1908922957 uploaded.
Thread 8, 11:03:11:5418, 1908922957 ssh client session bound.
Thread 2, 11:03:11:5508, File 5699697281 uploaded.
Thread 2, 11:03:11:5518, 5699697281 ssh client session bound.
Thread 3, 11:03:11:5788, 8839964099 sftp session bound.
Thread 3, 11:03:11:7238, File 8839964099 uploaded.
Thread 3, 11:03:11:7258, 8839964099 ssh client session bound.
Thread 5, 11:03:12:0008, request Id 1707773597 received.
Thread 5, 11:03:12:0448, File 1707773597 TARed.
Thread 5, 11:03:12:4028, 1707773597 session connected.
Thread 5, 11:03:12:6458, 1707773597 session authenticated.
Thread 5, 11:03:12:7648, 1707773597 sftp session bound.
Thread 5, 11:03:12:8938, File 1707773597 uploaded.
Thread 5, 11:03:12:8958, 1707773597 ssh client session bound.
Thread 7, 11:03:12:9978, request Id 7507612201 received.
Thread 7, 11:03:13:0348, File 7507612201 TARed.
Thread 7, 11:03:13:3238, 7507612201 session connected.
Thread 7, 11:03:13:4318, 7507612201 session authenticated.
Thread 7, 11:03:13:5528, 7507612201 sftp session bound.
Thread 7, 11:03:13:6458, File 7507612201 uploaded.
Thread 7, 11:03:13:6468, 7507612201 ssh client session bound.
Thread 9, 11:03:13:9978, request Id 3423911186 received.
Thread 9, 11:03:14:0368, File 3423911186 TARed.
Thread 9, 11:03:14:2968, 3423911186 session connected.
Thread 9, 11:03:14:4118