Https "Unexpected data appeared after end of response" Error

0 votes
asked Mar 27 by lanopk (1,220 points)

I'm curious when and why the "Unexpected data appeared after end of response" error occurs.

When downloading files, this error occurs quite occasionally.
However, we have not yet found the cause of it.

It seems to be due to more read requests than the given Content-Length.

It is common to return 0 if there is no data when requesting Read.

How do I handle this exception?

I am using the latest fixed version (5.0.7733.0) that you sent me.

Thank you.

Best regards.

BooKyung Oh.
OpenBoxLab Inc.

Applies to: Rebex HTTPS

1 Answer

0 votes
answered Mar 29 by Lukas Pokorny (123,810 points)

This error occurs when the server sends more data that it was expected to. A mismatch between the number of bytes reported in Content-Length and actual number of bytes received is the most common cause.

We choose to fail in this case because silently ignoring the problem could lead to data corruption. Sending more data than announced indicates a serious server-side issue that should not be hidden from the caller.

commented Mar 30 by lanopk (1,220 points)
I understood well.

In this case, if I read data as much as Content-Length size, does not an exception occur?

Thank you.
commented Mar 30 by lanopk (1,220 points)
I don't know the details inside Http yet, so there's always something to worry about.
When I use it in KeepAlive mode
Doesn't there conflict with the previous data when I cancel the download and download a new one?
commented Mar 30 by lanopk (1,220 points)
Well. That's a little weird :
Http Request : 1105
Http Response : Content-length : 1105 (OK)
Read Stream Start :  stream.read(buffer, 0, 1105)
Exception (Unexpected data..)

stack trace :
uspns.imxim.Read(Byte[] buffer, Int32 offset, Int32 count)
uspns.dhhhe.twrde(Byte[] p0, Int32 p1, Int32 p2)
uspns.dhhhe.pewak[T](Func`1 p0)
RaiDrive.Service.ThrottledStream.Read(Byte[] buffer, Int32 offset, Int32 count)
commented Mar 30 by lanopk (1,220 points)
There are two cases in my download code.
I always use KeepAlive.
One is when Http is aborted by using Cancellation(request.abort call),
the other is when I get enough data while downloading and close Response and Response stream without reading the rest.
Then it continues to download the next file or another location of the same file.
These tasks are operated in multi-thread.
If I don't properly call request abort, wouldn't it be possible for this to happen?
Is it normal to have to abort request all the time when I haven't read the data completely?
commented Mar 30 by Lukas Pokorny (123,810 points)
The error will occur even if you only read as much data as indicated by Content-Length.

But it should not affect other downloads - when an error occurs, the HTTP session is discarded and not reused for subsequent requests.

This also applies to HTTP sessions from which not all data has been read - these are discarded and not reused. In this case, it might be better to actually read the rest of the data as well and discard it, because that would make it possible to reuse the session.

As usual, a communication log at LogLevel.Debug might make it possible to tell what is actually going on.
commented Mar 30 by lanopk (1,220 points)
I sent a log file.
commented Mar 30 by Lukas Pokorny (123,810 points)
Thanks! The log confirms that the server really sent more data than it should. So this is a server-side bug.

We will modify Rebex HTTPS to return the data up to the number of bytes of Content-Length, and only fail in the next Stream.Read call.
...