Update: Rebex File Server 2016 R1 adds workaround for Windows CE with missing Enhanced DSS and Diffie-Hellman Cryptographic Provider - managed and slow implementation of Diffie-Hellman is used in this case. however, please note that this may be too slow in many scenarios.
Rebex File Server uses Diffie-Hellman algorithm when negotiating a secure SSH session. However, the exception indicates that your Windows CE device lacks "Microsoft Enhanced DSS and Diffie-Hellman Cryptographic Provider" CSP provider we use for Diffie-Hellman calculations.
The most common reason for the missing CSP provider is a mistake while creating the OS image - not checking the appropriate checkbox in Windows CE Platform Builder will result in an OS with missing DSS and DH provider support.
To check whether this is indeed the case, check the "HKEY_LOCAL_MACHINE\Comm\Security\Crypto\Defaults\Provider"
registry key to see the list of installed CSP providers. (Use PHM Registry Editor if the system lacks a registry editor.)
Unfortunately, the only solution to this is to install an updated OS image that contains the DSS and DH CSP provider.
Although we have a .NET-only implementation of Diffie-Hellman algorithm as well, it's too slow in low-power Windows CE devices because Diffie-Hellman calculation are very CPU-intensive.