Why I got System.Net.WebException : More compressed data expected.

0 votes
asked Apr 12 by Lukas Matyska (42,060 points)

(Note: this was originally asked at comment)

Hi Lukas,

  • below is the response am getting from server - seems the Request XML size is big?
2018-04-11 12:28:33 VERBOSE HttpRequest(1)[105316382] HTTP: Received data:
 0000 |48-54-54-50-2F-31-2E-31 20-32-30-30-20-4F-4B-0D| HTTP/1.1 200 OK.
 0010 |0A-43-61-63-68-65-2D-43 6F-6E-74-72-6F-6C-3A-20| .Cache-Control: 
 0020 |6E-6F-2D-63-61-63-68-65 2C-20-6D-75-73-74-2D-72| no-cache, must-r
 0030 |65-76-61-6C-69-64-61-74 65-2C-20-6D-61-78-2D-61| evalidate, max-a
 0040 |67-65-3D-30-0D-0A-50-72 61-67-6D-61-3A-20-6E-6F| ge=0..Pragma: no
 0050 |2D-63-61-63-68-65-0D-0A 54-72-61-6E-73-66-65-72| -cache..Transfer
 0060 |2D-45-6E-63-6F-64-69-6E 67-3A-20-43-68-75-6E-6B| -Encoding: Chunk
 0070 |65-64-0D-0A-43-6F-6E-74 65-6E-74-2D-54-79-70-65| ed..Content-Type
 0080 |3A-20-74-65-78-74-2F-78 6D-6C-3B-63-68-61-72-73| : text/xml;chars
 0090 |65-74-3D-55-54-46-2D-38 0D-0A-43-6F-6E-74-65-6E| et=UTF-8..Conten
 00A0 |74-2D-45-6E-63-6F-64-69 6E-67-3A-20-64-65-66-6C| t-Encoding: defl
 00B0 |61-74-65-0D-0A-53-65-72 76-65-72-3A-20-4D-69-63| ate..Server: Mic
 00C0 |72-6F-73-6F-66-74-2D-49 49-53-2F-37-2E-30-0D-0A| rosoft-IIS/7.0..
 00D0 |5F-63-68-61-72-73-65-74 3A-20-55-54-46-2D-38-0D| _charset: UTF-8.
 00E0 |0A-58-2D-50-6F-77-65-72 65-64-2D-42-79-3A-20-41| .X-Powered-By: A
 00F0 |53-50-2E-4E-45-54-0D-0A 58-2D-46-72-61-6D-65-2D| SP.NET..X-Frame-
 0100 |4F-70-74-69-6F-6E-73-3A 20-53-41-4D-45-4F-52-49| Options: SAMEORI
 0110 |47-49-4E-0D-0A-44-61-74 65-3A-20-57-65-64-2C-20| GIN..Date: Wed, 
 0120 |31-31-20-41-70-72-20-32 30-31-38-20-31-36-3A-32| 11 Apr 2018 16:2
 0130 |38-3A-33-35-20-47-4D-54 0D-0A-0D-0A            | 8:35 GMT....
2018-04-11 12:28:34 INFO HttpRequest(1)[105316382] HTTP: Received response: 200 OK.
2018-04-11 12:28:34 DEBUG HttpRequest(1)[105316382] HTTP: Received 10 headers.
2018-04-11 12:28:34 DEBUG HttpRequest(1)[105316382] HTTP: Response Content-Length not specified.
2018-04-11 12:28:34 DEBUG HttpRequest(1)[105316382] HTTP: Response Transfer-Encoding: Chunked.
2018-04-11 12:28:34 VERBOSE HttpRequest(1)[105316382] HTTP: Received data:
 0000 |32-33-64-0D-0A-84-54-5D 73-DA-30-10-FC-2B-8C-DF| 23d...T]s.0..+..
 0010 |8D-2C-63-13-C8-08-65-DC 38-9D-32-93-06-26-5F-D3| .,c...e.8.2..&_.
 0020 |B7-8E-B0-8F-E0-A9-2D-39 92-1C-A0-BF-BE-67-30-C6| ......-9.....g0.
 0030 |40-D3-EA-C5-BA-DD-DB-D3 EA-64-89-DD-6C-8A-BC-F7| @........d..l...
 0040 |01-DA-64-4A-4E-1C-DA-F7 9C-1E-C8-44-A5-99-7C-9B| ..dJN......D..|.
 0050 |38-2F-CF-5F-DD-91-73-C3 D9-D3-2C-9A-BB-77-0F-AF| 8/._..s...,..w..
 0060 |D7-77-F2-03-72-55-42-0F 65-D2-5C-1F-F0-89-B3-B2| .w..rUB.e.\.....
 0070 |B6-BC-26-C4-24-2B-28-84 E9-23-6D-94-28-FB-4A-BF| ..&.$+(..#m.(.J.
 0080 |91-7A-42-A0-11-12-A7-91 6E-4C-D6-AA-D6-EB-75-7F| .zB.....nL....u.
 0090 |3D-D8-25-FB-9E-47-C9-8F EF-F7-4F-BB-42-6E-26-8D| =.%..G....O.Bn&.
 00A0 |15-32-81-A3-2A-FD-BF-CA E9-18-FE-A2-D2-2D-67-A8| .2..*........-g.
 00B0 |9C-57-3A-59-09-03-73-AD D2-2A-B1-3F-67-95-2D-2B| .W:Y..s..*.?g.-+
 00C0 |DB-94-95-E6-B8-83-0C-16 90-F7-13-55-90-DB-CA-58| ...........U...X
 00D0 |55-BC-4C-B1-DE-99-FA-11 4C-A9-A4-69-DA-70-E2-A8| U.L.....L..i.p..
 00E0 |A3-47-92-7C-22-C4-92-1A CA-7C-1B-0B-2B-38-33-60| .G.|"....|..+83`
 00F0 |EA-FE-9F-04-D3-94-07-4B 18-FA-02-AE-DC-E5-55-32| .......K......U2
 0100 |74-83-90-0A-77-B4-18-09 77-11-78-10-C2-D0-4B-FC| t...w...w.x...K.
 0110 |31-30-72-CC-67-F0-5E-65 65-01-D2-62-10-7A-BB-31| 10r.g.^ee..b.z.1
 0120 |64-A4-8B-B6-E9-FB-B5-74 63-07-23-E0-5E-40-28-C5| d......tc.#.^@(.
 0130 |56-D2-11-23-27-04-C3-33 B0-95-B9-55-29-26-E1-C0| V..#'..3...U)&..
 0140 |22-47-80-81-D6-4A-4F-E5 52-F1-A7-2A-49-B0-3A-AE| "G...JO.R..*I.:.
 0150 |D8-42-4C-E9-14-F4-34-E6 D4-FD-16-8F-87-31-DA-39| .BL...4......1.9
 0160 |20-7B-EA-41-21-45-BD-10 57-1D-D1-30-68-68-44-59|  {.A!E..W..0hhDY
 0170 |9E-19-3B-5B-CE-C5-B6-B6 1E-55-76-A5-74-F6-5B-58| ..;[.....Uv.t.[X
 0180 |F4-CE-99-E8-86-36-86-E5 D9-56-CE-37-B0-33-D4-F8| .....6...V.7.3..
 0190 |6F-EC-9D-9B-8F-E6-F3-C7 D9-6B-74-7F-E2-7E-21-E4| o........kt..~!.
 01A0 |AF-93-A5-77-2A-4A-7D-6F 74-C5-C8-DF-59-74-F2-5E| ...w*J}ot...Yt.^
 01B0 |81-B1-51-A1-2A-69-B9-D7 F7-6B-37-5D-A8-B5-0F-69| ..Q.*i...k7]...i
 01C0 |83-D4-39-17-20-CB-85-B1 41-9C-BD-65-D6-CC-96-B7| ..9. ...A..e....
 01D0 |42-A7-7C-D3-19-E3-70-1C 32-72-99-C3-A4-28-EA-0E| B.|...p.2r...(..
 01E0 |EC-3F-76-23-8F-DD-8F-18 D9-C7-E8-B1-BE-F3-22-7F| .?v#..........".
 01F0 |DE-87-E4-2C-16-D6-EA-6C E1-51-4E-D1-D7-61-7E-40| ...,...l.QN..a~@
 0200 |7D-1E-95-A5-56-1F-90-B6 A4-DF-92-03-4E-F1-2C-5B| }...V.......N.,[
 0210 |62-D0-12-01-8F-61-91-D9 5E-ED-B1-A5-83-96-0E-79| b....a..^......y
 0220 |0B-EE-A6-97-07-4C-FE-F5 43-90-CE-6D-FA-EC-CE-D5| .....L..C..m....
 0230 |3D-F9-EC-1D-40-EE-EC-C9 20-17-6F-1E-FF-03-00-00| =...@... .o.....
 0240 |FF-FF-0D-0A-30-0D-0A-0D 0A                     | ....0....
2018-04-11 12:28:34 INFO HttpRequest(1)[105316382] HTTP: ZLIB header check failed. Using DEFLATE fallback.
2018-04-11 12:28:34 DEBUG HttpRequest(1)[105316382] HTTP: Closing HTTP session (1).

1 Answer

0 votes
answered Apr 12 by Lukas Matyska (42,060 points)

I recreated server response from the included Verbose log and I found two issues:

  1. Reported Content-Encoding is deflate, which should use "zlib" format according to RFC 7230, section 4.2.2.
    However, "zlib" wrapper was not used. It is ok, because we are aware of this and we have workaround for this, but it is first violation.

  2. More serious issue is, that the compressed data are not finished correctly. The last block in deflate data should be marked as final, but it is not in your case. I am able to decompress all 1325 bytes of data which forms complete <SOAP-ENV:Envelope>, but because of unfinished deflate data we reported "More compressed data expected."

Please, report this issue to your server vendor. It seems that Flush() or Close() was not called when created deflate data.

However, I also tried this with system WebRequest and it doesn't complain. So we will add workaround for this as well.

commented 3 days ago by RajaK (110 points)
hi Lukas,
Thanks for the guidance on this  - I checked with My Server team , since this  is  wotking in http mode - wcf .Net3.5CF web request  - they requested to check in in Rebex library side.
1.for first  .Zlib format
   am  using  below attributes while adding header for request.

public Message Request(Message message, TimeSpan timeout)
        {
            ThrowIfDisposedOrNotOpen();

            var request = this.binding.RequestCreator.Create(this.via);
            request.Timeout = (int)Math.Ceiling(timeout.TotalMilliseconds);

            string action = message.Headers.Action;

            request.Method = "POST";
            request.Headers.Add("SOAPAction", '"' + action + '"');
            request.Headers.Add("Content-Type", "text/xml; charset=utf-8");

 - do I need to any other header like  
 request.Headers.Add("Content-Encoding","XXX");    - ?

2.For Compressed data  are not finished issue   - do we have any hotfix on Rebex side ?? am in the position to show the Prrof of concepts to occomdate the TLS1.2 with Rebex to proceed for Production deployment.
Kindly help on this.
commented 2 days ago by Lukas Matyska (42,060 points)
Hello,

1. I don't think any client headers can change server behavior in this case.
2. You can try the current build with workaround for the unfinished deflate data. It is available at https://www.rebex.net/getfile/80aa6d4c569d4060aac92e84ab594b7c/RebexHttps-HotfixBuild6682-Trial-Binaries.zip

If you purchase Rebex HTTPS solution, let us know - we will send you full version with the workaround.
commented 2 days ago by RajaK (110 points)
Hi Thanks for the build, can we get the FTP.dll as well , since am using the Rebex FTP also in same project and getting

he type 'Rebex.Net.NetworkSession' is defined in an assembly that is not referenced. You must add a reference to assembly 'Rebex.Networking, Version=3.5.6666.0, Culture=neutral, PublicKeyToken=1c4638788972655d

 - due to version mismatch  , the build which I got not containing  FTL dll ,
commented 1 day ago by Lukas Matyska (42,060 points)
...