Your original code most likely used CertificateChain.LoadPfx to load the client certificate (and associated private key) in order to use it for client certificate authentication. If you no longer want to perform client certificate authentication, then you need to remove (or disable) the relevant code. Replacing the client certificate with the server certificate is not going to work. Instead, just set Ftp object's Settings.SslClientCertificateRequestHandler to null and see whether this works.
But what about the CRT file? If the server's certificate was issued by a certification authority that is trusted by the client, you don't actually need this at all. The server sends its certificate chain to the client during TLS/SSL negotiation anyway and Rebex FTP/SSL validates it using Windows CryptoAPI by default. Unless you would like to validate the server's certificate chain yourself (using a custom certificate validator), the only thing you might have to do is to add the certificate of the chain's root CA to the trusted root certification authorities store of the user account under which the application runs.
The "Invalid Base-64 encoding of a certificate" error indicates that the certificate appeared to be Base-64 encoded, but the library was unable to decode the Base64-based data for some reason. Would it be possible to send us the CRT file for analysis? (Unlike private keys, certificates are supposed to be publicly available.)