Separating serialization of headers from body

0 votes
asked Oct 22, 2012 by Jonathan Gilbert (120 points)
edited Oct 23, 2012

Good day :-)

I am looking into using the Rebex MIME components (part of the Rebex IMAP/SSL pack) to implement AS2 communications. AS2 is a standard (RFC 4130) for business-to-business communication that consists of recommendations on the use of other pre-existing Internet standards. It uses HTTP or HTTPS (RFCs 2616, 2818) for delivery of messages and S/MIME (RFCs 2045, 2046, 3851) for the formatting of messages. Non-repudiation is achieved by Message Disposition Notifications, or MDNs (RFC 3798).

What's odd about AS2 is the way it merges these standards together. HTTP and MIME have a superficially similar structure: headers, blank line, body. When AS2 was being designed, from what I've understood, they decided that the S/MIME entity containing the content would actually be the HTTP body. A sample request in the RFC, for instance, looks like this:

POST /receive HTTP/1.0
Host: 10.234.160.12:80
User-Agent: AS2 Company Server
Date: Wed, 31 Jul 2002 13:34:50 GMT
From: mrAS2@example.com
AS2-Version: 1.1
AS2-From: "\"  as2Name  \""
AS2-To: 0123456780000
Subject: Test Case
Message-Id: <200207310834482A70BF63@\"~~foo~~\">
Disposition-Notification-To: mrAS2@example.com
Disposition-Notification-Options: signed-receipt-protocol=optional,
     pkcs7-signature; signed-receipt-micalg=optional,sha1
Content-Type: multipart/signed; boundary="as2BouNdary1as2";
     protocol="application/pkcs7-signature"; micalg=sha1
Content-Length: 2464

--as2BouNdary1as2
Content-Type: application/edi-x12
Content-Disposition: Attachment; filename=rfc1767.dat

[ISA ...EDI transaction data...IEA...]

--as2BouNdary1as2
Content-Type: application/pkcs7-signature

[omitted binary pkcs7 signature data]
--as2BouNdary1as2--

You can see the HTTP headers ("Host", "User-Agent") mingling with MIME headers ("From", "Subject", "Message-Id").

My implementation platform is ASP.NET. ASP.NET parses out the HTTP request and headers before it hands off processing to an HTTP handler implementation. This makes the headers inaccessible to the MimeEntity's Load method, but that's easy enough to work around because, according to Microsoft's documentation, the Server Variable called "ALL_RAW" stores the raw text of the headers. I can prepend that to the POST data before passing it on to the MimeEntity's Load method.

Where I'm running into difficulties is at the opposite end of things. An AS2 request can ask that the recipient supply an MDN -- an electronic form of proving that you really did get the message -- and that MDN is also an S/MIME entity. It is merged similarly with the HTTP response:

HTTP/1.0 200 OK
AS2-From: 0123456780000
AS2-To: "\"  as2Name  \""
AS2-Version: 1.1
Message-ID: <709700825.1028122454671.JavaMail@ediXchange>
Content-Type: multipart/signed; micalg=sha1;
      protocol="application/pkcs7-signature";
      boundary="----=_Part_57_648441049.1028122454671"
Connection: Close
Content-Length: 1980

------=_Part_57_648441049.1028122454671
Content-Type: multipart/report;
       Report-Type=disposition-notification;
       boundary="----=_Part_56_1672293592.1028122454656"

------=_Part_56_1672293592.1028122454656
Content-Type: text/plain
Content-Transfer-Encoding: 7bit

MDN for -
 Message ID: <200207310834482A70BF63@\"~~foo~~\">
  From: "\"  as2Name  \""
  To: "0123456780000"
  Received on: 2002-07-31 at 09:34:14 (EDT)
 Status: processed
 Comment: This is not a guarantee that the message has
  been completely processed or &understood by the receiving
  translator

------=_Part_56_1672293592.1028122454656
Content-Type: message/disposition-notification
Content-Transfer-Encoding: 7bit

Reporting-UA: AS2 Server
Original-Recipient: rfc822; 0123456780000
Final-Recipient: rfc822; 0123456780000
Original-Message-ID: <200207310834482A70BF63@\"~~foo~~\">
Received-content-MIC: 7v7F++fQaNB1sVLFtMRp+dF+eG4=, sha1
Disposition: automatic-action/MDN-sent-automatically;
       processed

------=_Part_56_1672293592.1028122454656--
------=_Part_57_648441049.1028122454671
Content-Type: application/pkcs7-signature; name=smime.p7s
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=smime.p7s

MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQ
cp24hMJNbxDKHnlB9jTiQzLwSwo+/90Pc87x+Sc6EpFSUYWGAAAAAAAA
------=_Part_57_648441049.1028122454671--

This is where things get tricky. As with requests, in responses ASP.NET wants to handle the headers for you. The code I'm writing is only allowed to supply headers in name/value form, and the content must be independently written.

Looking over the Rebex MimeEntity class, I don't see an obvious way of doing an otherwise-normal serialization pass, but entirely omitting the headers and the blank line separating them from the body. Complicating matters is the obfuscation that I understand you've needed to apply in the wake of the "ComponentForge" nastiness -- ordinarily, I'd be able quickly to tell in a general sense whether it is possible with existing code, but the obfuscation slows things down enough that it is a better use of my time simply to ask. :-)

At this point in time, I am considering work-arounds such as serializing the MimeEntity to a buffer, scanning for the blank line, and then emitting only those bytes that come thereafter. Obviously, though, this is inelegant and inefficient, and if there is a way to achieve my goal more directly (which, in this case, means emitting the serialized parts of a multipart message, boundaries and all, but omitting the headers), it would be greatly preferable to do it that way.

So, can it be done? :-)

Applies to: Rebex Secure Mail

1 Answer

0 votes
answered Oct 23, 2012 by Lukas Pokorny (100,710 points)
edited Oct 23, 2012

This can be done quite efficiently using MimeEntity object's little-known ToStream method. It returns a read-only Stream that provides the whole message content (in MIME format). It is efficient internally - it doesn't serialize the whole entity into a buffer, but returns the data on-the-fly.

You would still have to scan for the blank line (marking the end of headers) when using this approach, but at least you won't have to serialize the whole entity into a buffer (although it might be a good idea to retrieve the headers this way). Once you reach the blank line, just start copying the data data into HttpWebRequest's GetRequestStream() (or whatever object you use for HTTP requests).

I don't think there is any easier way than this, but we might consider adding it - it would be trivial to enhance the method to provide two read-only streams, one for the headers and another for the rest.

...