HttpRequest repeats POST request (depending on settings)

Question

We are using Rebex HTTPS Legacy 2019R3.3 in different Windows Embedded Handheld 6.5.3 devices. When sending POST data and loosing the connection, we found some unxpected behaviour depending on HttpRequest.AllowWriteStreamBuffering value.

With AllowWriteStreamBuffering set to true, a WebException with the status SecureChannelFailure is thrown in the application. In addition, Rebex performs the same request three times (unexpectedly). Here an extract of the Rebex log:

2020-02-04 09:39:16 DEBUG HttpRequest(6) HTTP: Using new HTTP session (5) provided by Rebex.Net.HttpRequestCreator(1).
09:39:16 INFO HttpRequest(6) HTTP: Connecting to 'https://…:443'...
…
09:39:17 INFO HttpRequest(6) HTTP: Sending request: POST …
09:39:17 DEBUG HttpRequest(6) HTTP: PreAuthenticate: False
09:39:18 DEBUG HttpRequest(6) HTTP: Request Connection: keep-alive.
09:39:18 DEBUG HttpRequest(6) HTTP: Sending request (552 bytes).
09:39:18 INFO HttpRequest(6) HTTP: Received response: 100 Continue.
09:39:18 DEBUG HttpRequest(6) HTTP: Received 0 headers.
09:39:18 DEBUG HttpRequest(6) HTTP: Sending 1447 bytes of data.
09:40:18 DEBUG HttpRequest(6) TLS: TLS socket was closed, 0 bytes of data were received.
09:40:18 DEBUG HttpRequest(6) TLS: Closing TLS socket.
09:40:18 INFO HttpRequest(6) HTTP: Connection lost.
09:40:18 DEBUG HttpRequest(6) HTTP: Closing HTTP session (5).
09:40:18 INFO HttpRequest(6) HTTP: Reconnecting to 'https://…:443'...
…
09:40:19 INFO HttpRequest(6) HTTP: Sending request: POST …
09:40:19 DEBUG HttpRequest(6) HTTP: PreAuthenticate: False
09:40:19 DEBUG HttpRequest(6) HTTP: Request Connection: keep-alive.
09:40:19 DEBUG HttpRequest(6) HTTP: Sending request (552 bytes).
09:40:20 INFO HttpRequest(6) HTTP: Received response: 100 Continue.
09:40:20 DEBUG HttpRequest(6) HTTP: Received 0 headers.
09:40:20 DEBUG HttpRequest(6) HTTP: Sending 1447 bytes of data.
09:41:20 DEBUG HttpRequest(6) TLS: TLS socket was closed, 0 bytes of data were received.
09:41:20 DEBUG HttpRequest(6) TLS: Closing TLS socket.
09:41:20 INFO HttpRequest(6) HTTP: Connection lost.
09:41:20 DEBUG HttpRequest(6) HTTP: Closing HTTP session (5).
09:41:20 INFO HttpRequest(6) HTTP: Reconnecting to 'https://…:443'...
…
09:41:22 INFO HttpRequest(6) HTTP: Sending request: POST …
09:41:22 DEBUG HttpRequest(6) HTTP: PreAuthenticate: False
09:41:22 DEBUG HttpRequest(6) HTTP: Request Connection: keep-alive.
09:41:22 DEBUG HttpRequest(6) HTTP: Sending request (552 bytes).
09:41:22 INFO HttpRequest(6) HTTP: Received response: 100 Continue.
09:41:22 DEBUG HttpRequest(6) HTTP: Received 0 headers.
09:41:22 DEBUG HttpRequest(6) HTTP: Sending 1447 bytes of data.
09:42:22 DEBUG HttpRequest(6) TLS: TLS socket was closed, 0 bytes of data were received.
09:42:22 DEBUG HttpRequest(6) TLS: Closing TLS socket.
09:42:22 INFO HttpRequest(6) HTTP: Connection lost.
09:42:22 DEBUG HttpRequest(6) HTTP: Closing HTTP session (5).
09:42:22 ERROR HttpRequest(6) HTTP: Error while sending request: qhir: Connection closed.

With AllowWriteStreamBuffering set to false, a WebException with the status RequestCanceled is thrown in the application. Here an extract of the Rebex log:

08:23:29 DEBUG HttpRequest(1578) HTTP: Using new HTTP session (192) provided by Rebex.Net.HttpRequestCreator(1).
08:23:29 INFO HttpRequest(1578) HTTP: Connecting to 'https://…:443'...
…
08:23:33 INFO HttpRequest(1578) HTTP: Sending request: POST …
08:23:33 DEBUG HttpRequest(1578) HTTP: PreAuthenticate: False
08:23:33 DEBUG HttpRequest(1578) HTTP: Request Connection: keep-alive.
08:23:33 DEBUG HttpRequest(1578) HTTP: Sending request (558 bytes).
08:23:33 INFO HttpRequest(1578) HTTP: Received response: 100 Continue.
08:23:33 DEBUG HttpRequest(1578) HTTP: Received 0 headers.
08:23:33 DEBUG HttpRequest(1578) HTTP: Sending non-buffered data in chunked mode.
08:24:34 DEBUG HttpRequest(1578) TLS: TLS socket was closed, 0 bytes of data were received.
08:24:34 DEBUG HttpRequest(1578) TLS: Closing TLS socket.
08:24:34 INFO HttpRequest(1578) HTTP: Connection lost.
08:24:34 DEBUG HttpRequest(1578) HTTP: Closing HTTP session (192).
08:24:34 ERROR HttpRequest(1578) HTTP: Error while sending request: qhir: Unable to send request content again when stream buffering is disabled.
at qhio.lxrk()
at Rebex.Net.HttpRequest.mxyh()
at Rebex.Net.HttpRequest.bjwi.pwjx()
at qhvi.rnjr(Object ejo)
at qhyj.aucr.xyrn()
at qhwo.wuib()

The term unable to send request content again indicates Rebex would like to repeat the request as well, but fails here.

We have some questions here:

1. Can we configure whether the Rebex HTTPS library will repeat a failed request (and how often)?
2. Is the different WebExceptionStatus depending on AllowWriteStreamBuffering value intended?
3. Could the Rebex HTTPS library repeat a failed request with AllowWriteStreamBuffering set to false, when the underlying stream is seekable (like a MemoryStream)?

Answer 1

We wanted to mimic system HTTP request behavior, which internally repeats failed requests. However, we realized that system HTTP request does not repeat failed requests in case data are sent to the server (e.g. using POST method).

We considered our behavior to be a bug. It will be fixed in the next version.

Please, let me know, whether you want to receive BETA version with the new behavior.

To your questions:

1. No - you cannot configure whether failed requests will be repeated. We will discuss it and very probably we will add an option to configure this behavior.

2. Yes - both reports the current cause of a problem. We can discuss whether it would be better to report SecureChannelFailure or ConnectionClosed or ReceiveFailure. Anyway, in the next version the error will not depend on AllowWriteStreamBuffering value.

3. No - when AllowWriteStreamBuffering is set to false, the request data is discarded immediately when sent to the server. In case of failed request, you would have to write the data to the request stream again (from your MemoryStream) but it would require different kind of API.

---

UPDATE:
Rebex HttpRequest now don't attempt to retry failed requests if request data was sent to the server already (same behavior as system's HttpWebRequest on .NET 4.5) - this was changed in 2020 R4.

Comments

Thanks for investigating here! We would really welcome an option to configure whether failed requests will be repeated, as it makes behavior and timing more predictable.

We do not need a hotfix version, thank you.

---

OK. I have created ticket for this issue. It is scheduled for the next version.
I will post here a note, when the new behavior is released publicly.

---

There is no behavior change in the new version 2019 R3.4 (build 7350) from 2020-03-27, right?

---

2019 R3.4 is meant as a drop-in replacement for 2019 R3.3 and earlier releases.

However, stricly speaking, every bugfix and almost every improvement is a behavior change. For example, R3.4 improves client certificate selection logic in HttpRequest.ClientCertificates collection, which is a slight change of behavior.  As another example, addition of CHACHA20_POLY1305 ciphers to TlsCipherSuite implies that applications using TlsCipherSuite.All will announce support for these ciphers after upgrading to R3.4.