There are no current issues.
ITI Systems was notified that message queues were backing up on muexch04. It was determined that due to low disk space muexch04 was rejecting mail submission. The remaining 7 exchange servers were processing mail as expected. If any messages were rejected or delayed they should be retried shortly. We have corrected the low disk space issue on muexch04 and queues are behaving as expected. We will continue to monitor for 24 hours.
Affected Applications: Exchange
ITI Systems was notified of a failed exchange server (muexch03). The server is rebooting. We are investigating the cause.
Affected Applications: Exchange Mail (on-prem), Calendar, UM
With the migration of the Marshall Email System's Spam Security service to Barracuda Cloud, the on-site appliances had become redundant. At 10AM 12/17/2019, we shut down the 3 on-premises barracuda appliances. The units are still in place in the highly unlikely event that we need to reactivate them for some reason. These units will be removed later after a period of time to verify that there are no unknown services attempting to use them.
Impact: Little to none. There were no known services routing mail through the on-premises appliances.
Affected Applications: None known
At 8:35, the Systems group discovered that mail flow was interrupted. This started at 8:00 AM. All mail that was interrupted was captured in a system mailbox. We are investigating the cause and means to redirect the messages to the intended recipients.
Impact: All mail that flowed through on-premises Exchange servers between 8:00 and 8:35 this morning.
Affected Applications: Exchange Servers, on-premises email
Due to several compromised accounts that started sending out SPAM this has caused interruption with our out bound mail and our mail reputation.
Impact: interruptions with outbound mail
Affected Applications: Exchange On-Prem mail service.
Due to an unexpected issue, our email flow has been changed back to flow through our on-premises exchange servers instead of the Barracuda ESS. This was done to address an issue where email relayed from some trusted servers/domains was being rejected by Barracuda.
Impact: Email that had been relayed through our email servers from trusted servers was being rejected and held in queue. These messages were eventually processed when the original email flow was reestablished. No email was lost due to this issue.
Affected Applications: -----
this has been enabled again. all outbound mail is going through barracuda cloud.
The ITI Systems team has reconfigured the outbound Exchange send connector to direct all outbound email through the Barracuda Essentials cloud appliance for scanning. Please report any issues with mail flow to the Service desk.
Impact: All outbound email
Affected Applications: Exchange
When users attempt to preview MS Office attachments in Outlook Web while outside of Marshall's network they will receive an error that muexchwac.marshall.edu could not be found. ITI Systems is working to identify the cause. Users on Marshall's networks will not experience this issue.
Affected Applications: OWA
A problem was discovered with Outlook Web App failing to display messages. This seems to only be a problem displaying message in Outlook Web App and does not impact mail flow. ITI Systems is investigating the problem.
Impact: Some users with mailboxes hosted on-premises that use the Outlook Web App may experience this problem.
Affected Applications: Outlook Web App.
The ITI Systems team has successfully disabled SSL Offloading on our load balancer and enabled SSL Bridging in an effort to resolve a few different issues including Free/Busy sharing from O365 to on-prem.
Please let the service desk know if you are experiencing any issues.
Impact: Email System
Affected Applications: Outlook Clients
Currently, the Systems team is investigating authentication issues with Outlook. Currently, users are being repeatedly prompted for credentials and we are actively investigating. All mail is flowing normal at this time; this only affects client access. Outlook Web is functional as a workaround during this time.
Impact: End users with the Outlook client are having issues accessing email. ActiveSync is working as expected during this time.
Affected Applications: Microsoft Outlook
ITI Systems has received reports that email delivery is slow for some users. We are rebooting an Exchange host which appears to be sluggish.
Impact: Users whose email is hosted on this system may experience slowness as the databases hosted on muexch05 are activated on other nodes during the reboot process.
Affected Applications: Exchange mail
A server hosting Exchange 2016 mailbox databases had a failure and rebooted. ITI Systems is investigating the cause of the failure.
Impact: Minimal impact. The mailbox databases failed over on surviving peers
Affected Applications: Exchange 2016
Due to several compromised accounts sending SPAM and phishing emails targeting unknown users at AOL and Yahoo, Marshall.edu has earned a bad sender reputation with Yahoo’s sender reputation service. This service forces all emails coming from marshall.edu to be deferred when sending emails to AOL.com and Yahoo.com. There is no way to request a delisting but Yahoo claims that “typically” the listing should only last for 4 hours unless there are continued reports. The Systems team will be closely monitoring our Exchange queues and will update the ITServiceStatus web site once the ban has lifted.
Affected Applications: -----
In an effort to improve recipient verification on mass mailings coming in to the University email systems, the ITI Systems team has made modifications to our Barracuda_Recipient_Verification and Barracuda receive connectors in Exchange. These changes have eliminated the restrictions on the number of concurrent recipient verification requests. Please contact the service desk and report any undesired issues with email due to this change.