Depends on what you want to log, for your case anywhere after creating new Sftp object and before calling the method which results in the "Inavlid decoder state" exception.
e.g.
Sftp sftp = new Sftp();
sftp.Connect(...);
sftp.Login(...);
//...
sftp.LogWriter = new Rebex.FileLogWriter(@"c:\temp\log.txt", Rebex.LogLevel.Debug);
// failing method:
sftp.Download("largeFileName", "C:\\temp\\largeFileName");
//...
sftp.Disconnect();
Thank you!