An internal server error occurred. The operation failed using EWS and OAuth with Office 365

Question

We have recently converted to using OAuth with Exchange Online from Basic Authentication.

Just starting today, a number of customer sites reported the following errors, sometime intermittently:

Rebex.Net.EwsException: Internal server error. An internal server error occurred. The operation failed. ---> Rebex.Net.EwsException: Internal server error. An internal server error occurred. The operation failed. ---> Rebex.Net.EwsException: Internal server error. An internal server error occurred. The operation failed.

In some cases this is intermittent and in other cases it is regularly reproducible. The strange thing is that this has been working properly for the last week or two since our customer started using OAuth.

Any assistance would be much appreciated.

Comments

This doesn't look like any common issue we are aware of. It might be a firewall or router issue, but we can't really tell based on the little information above.

Would it be possible to create a log file using Ews object's LogWriter property (as described at https://www.rebex.net/kb/logging/), and either post it here or mail it to support@rebex.net for analysis? Or at least post the output of Exception's ToString() method.

---

We sent the log file to your support email address.

---

Hi, thanks for the log!

However, would it be possible to try once more with LogLevel.Verbose, and post the full response between the last “HTTP: Sending request: POST /EWS/Exchange.asmx” and “HTTP: Request failed: 500” entries? That part is supposed to contain some information about the error.

Alternatively, can you provide instructions to reproduce the issue?

---

Hi,

We sent the verbose log over as well. Please advise as soon as possible since we need to get this working quickly before the cutover to OAuth takes place.

Thanks

---

It looks like Exchange Online returns a "500" HTTP error (with "ErrorInternalServerError" status code in the response) when a mailbox is accessed by EWS for the first time. After several minutes (possibly after Exchange online prepares the data structures needed for EWS access), the error stops occurring.

So it looks like a server-side issue - if it was by design, the server would return a relevant error code instead. Please contact Microsoft for further clarification.

As a workaround, consider making the application wait a minute or two before retrying, while informing the user that the mailbox is not yet ready. If the problem persists, display a different error after a minute or two saying that the mailbox could not be accessed due to internal server error.

---

Hi,

We opened a support incident with Microsoft and they are looking into this issue. They also pointed us to this article on the web, which seems to be related to the same issue we are having (note the date on the article - August 24th, which is about the same time our errors popped up:

https://techcommunity.microsoft.com/t5/exchange/ews-with-oauth-quot-an-internal-server-error-occurred-the/m-p/3609047

One the solutions was as follows:

I've found out that the problem can be solved by added a header after setting ImpersonalteUserId

service.HttpHeaders.Add("X-AnchorMailbox", mailbox.Mailbox);

Is this a property or setting that can be applied in your EWS SecureMail API?

---

This was one of the first things we checked. Please check rebex.debug.txt log file you sent us - the anchor header is there. Rebex EWS puts adds it by default. In the log, you can see it just several lines up from "DEBUG Ews(1)[16] HTTP: Sending 759 bytes of data" log entry.

---

Our customers started to report the same problem from the beginning of this week. I'll open a ticket, but despite this is not a "repro" you can take for sure that the problem is not specific to "ITM", we have the same problem from customers using our product as well.

---

Yes, we have had numerous customer reports starting from around the 22nd or 23rd of August. I have opened a support incident with Microsoft and they are currently looking into it on their end as well.

---

Interestingly, I noticed the same 500 error pop up once in our automated tests for O365 access overnight. It only occurred once when the test run started, but all subsequent tests passed without issues. The mailbox used by those tests is not new, but those particular tests have not been run for over a month (and the mailbox has not been accessed by any user or app in the meantime), so perhaps Exchange Online decided there is no need to keep the mailbox ready for instant EWS access, and reverted it back to "not accessed by EWS" mailbox. (Just speculating here... Hopefully Microsoft would clarify this, if you are able to get through all their layers of support.)

---

Jiří Zídek: We have actually been able to reliably reproduce this exact issue yesterday. It was actually very simple: The error occurs whenever we triy accessing a mailbox using EWS that has not been accessed before and that had lot of e-mail in it. However, so far this appears to be a server-side error - we can suggest workarounds for it, but only Microsoft can fix it.

---

We have reports from customers using shared box like "info@company.cz" being accessed by our contact center system, so "not using" scenario is not very likely. These are high traffic boxes (1000 emails/day) and accessed every 1-2 minutes.

---

Jiří Zídek: It's quite possible that the same error is triggered by multiple causes. We experimentally confirmed yesterday that "first EWS access to the mailbox" is one of them. Now it seems that "mailbox not accessed by EWS for a while" is another one. Perhaps there are even more reasons, but you really have to contact Microsoft to find out. Sorry!

---

I'll send you a verbose log once colleagues get it, so we might get further. Since there is no diagnosis it might be problem in Rebex or in Microsoft API, so we have to work on both ends :-). In fact we can serve only as mediator between you and Microsoft - we are not experts on EWS or O365 - that's why we use Rebex lib - otherwise we could use directly MSFT libs.

---

Sent logs to support email address. Problem has 100% repro on our internal company test account.

---

It really looks like this is a Microsoft issue. Non-Rebex users are encountering the same problem as well: https://docs.microsoft.com/en-us/answers/questions/980333/internal-server-error-connecting-to-o365-exchange.html

---

I checked Jiri's log, and it appears the same as ITM's log and our logs... App-only authentication, "500 " HTTP error on first GET request in the Login method, with "ErrorInternalServerError" code in the SOAP response (just like in the link above).

---

What has been the longest period observed between the first appearance of "ErrorInternalServerError" for a particular mailbox and its disappearance? In case of mailboxes where we reproduced the issue, the outage only lasted for a moment. But for the users of Microsoft's library at the link above, it seems to persist for longer periods as well.

Answer 1

This seems to be server-side issue. Microsoft's EWS service has been apparently having some troubles recently, and non-Rebex EWS users are affected as well. More information: https://docs.microsoft.com/en-us/answers/questions/980333/internal-server-error-connecting-to-o365-exchange.html