Everything you wanted to know about Security and Audit Logging in Office 365

There are four primary audit log locations in Office 365. Depending on license level, these logs have varying lengths of retention.

  1. Office 365 “Unified Access Log”
    1. Enabled by ‘opt in’ (The first time you visit the log page, it asks if you want to enable it.)
    2. Goes back 90 days
    3. Accessible here: https://compliance.microsoft.com/auditlogsearch?viewid=Test%20Tab
    4. Documentation: Search the audit log in the Microsoft 365 compliance center – Microsoft 365 Compliance | Microsoft Docs
    5. There are four options for extending the logs beyond 90 days
    6. Option 1: purchase M365 E5 (or other license) “Advanced Audit” which can extend this log to 1 year
    7. Option 2: purchase ‘10-Year Audit Log Retention Add On’ (this add-on first became available for purchase in March 2021). Note: This policy is *not* retroactive.
    8. Option 3: Extend this into Sentinel to get correlation and default query templates
    9. Option 4: Use a 3rd party SIEM to query the Office 365 Management API.
      TIP: When you purchase the “Advanced Audit” license, it will reduce the throttling that occurs when querying the API (so for example you will see data in Splunk much faster!).
    10. Option 5: PowerShell
  2. Azure AD Audit Log
    1. Enabled by Default
    2. Goes back 30 days with an Azure AD P1 license (or 7 days with an Azure AD Free)
    3. Accessible here: Azure Audit Log
    4. Documentation: Audit logs in Azure Active Directory | Microsoft Docs
    5. Latency. Audit logs have a latency ranging from 15 minutes to an hour
    6. Activities. A complete list of each activity audited is available (here)
    7. Limits. The export limit from the web interface is 5,000 records. You can get around this by exporting the logs through one of the options below, which can also be used to extend retention.
    8. Option 1: Extend this log into Azure Log Analytics (aka Azure Monitor) to go beyond 30 days (learn how here)
    9. Option 2: Extend this log into Sentinel to get correlation and default query templates (learn how here)
    10. Option 3: PowerShell or Graph API.
  3. Azure AD Sign-in log
    1. Enabled by Default
    2. Goes back 30 days with an Azure AD P1 license (or 7 days with an Azure AD Free)
    3. Accessible here: Azure Sign-in Log
    4. Documentation: Sign-in logs in Azure Active Directory | Microsoft Docs
    5. Latency. Sign-in activity logs can take from 15 minutes to up to 2 hours for some records. According to the documentation, 95% of all logs will show up in 2 minutes.
    6. Limits. The export limit from the web interface is 5,000 records. You can get around this by exporting the logs through one of the options below, which can also be used to extend retention.
    7. Option 1: Extend this log into Azure Log Analytics (aka Azure Monitor) to go beyond 30 days (learn how here)
      NOTE: Unlike the Azure Audit Log, the Azure AD Sign-in logs require an Azure AD P1 or higher license to export into Log Analytics.
    8. Option 2: Extend this log into Sentinel to get correlation and default query templates (learn how here)
    9. Option 3: PowerShell or Graph API.
  4. Microsoft Cloud App Security
    1. Goes back 6 months
    2. Not enabled by default, requires configuration. You must go to “Connected Apps” then click the three dots to make it include the additional log sources as shown here:
      clip_image002
    3. Requires an M365 E5 license or O365 E5 license (or available via Stand Alone)
    4. Accessible here: https://portal.cloudappsecurity.com/#/audits/
    5. Option 1: Extend this log into Sentinel to go beyond 6 months

Reporting

There are a few really useful built-in reports that analyze the logs and produce findings.

  • Risky sign-ins – A risky sign-in is an indicator for a sign-in attempt that might have been performed by someone who is not the legitimate owner of a user account.

    Latency can range from as little as 5 minutes, to a maximum of 2 hours.

  • Users flagged for risk – A risky user is an indicator for a user account that might have been compromised
    Latency can range from as little as 5 minutes, to a maximum of 2 hours.
  • Risk Detections. Azure AD uses adaptive machine learning algorithms and heuristics to detect suspicious actions that are related to your user accounts. Each detected suspicious action is stored in a record called a risk detection.
    Here are the latencies associated with when the risk detections will appear:
    image

Advanced Audit License

As noted above, the new Advanced Audit License extends the retention of the UAL audit log to 1 year and speeds up 3rd party API throttling. A 3rd useful capability is the additional fields that get audited when this license is applied to mailboxes. The ability to log the ‘MailItemsAccessed’ (some may know this as MessageBind, that is what it was called in on-premises Exchange). Additional entries including exactly which items were sent from the compromised account are also logged.

The Send event is also a mailbox auditing action and is triggered when a user performs one of the following actions:

  • Sends an email message
  • Replies to an email message
  • Forwards an email message

Investigators can use the Send event to identify email sent from a compromised account. The audit record for a Send event contains information about the message, such as when the message was sent, the InternetMessage ID, the subject line, and if the message contained attachments. This auditing information can help investigators identify information about email messages sent from a compromised account or sent by an attacker. Additionally, investigators can use a Microsoft 365 eDiscovery tool to search for the message (by using the subject line or message ID) to identify the recipients the message was sent to and the actual contents of the sent message. You can also run the Search-UnifiedAuditLog -Operations Send or Search-MailboxAuditLog -Operations Send commands in Exchange Online PowerShell.

The MailItemsAccessed mailbox auditing action covers all mail protocols: POP, IMAP, MAPI, EWS, Exchange ActiveSync, and REST.

This is useful in a forensic investigation because it logs which emails were accessed. Imagine the relief of a Legal team when a hacker only accessed 10 items instead of a million items, and none of those 10 items contained PII or PHI data.

Note: When a protocol such as POP, IMAP, or MAPI over HTTPS (aka Outlook Anywhere) syncs a folder, then a single audit event is logged that the folder contents were synced rather than an entry for each item in the folder. (Reference).

Note: If an attacker generates more than 1,000 audit records in 24 hours in a mailbox, then this audit log is paused for 24 hours =( So a crafty hacker could overwhelm the log in order to hide activities (the pause occurs for 24 hours). (Reference)

SearchQueryInitiatedExchange

The SearchQueryInitiatedExchange event is triggered when a person uses Outlook to search for items in a mailbox. Events are triggered when searches are performed in the following Outlook environments:

  • Outlook (desktop client)
  • Outlook on the web (OWA)
  • Outlook for iOS
  • Outlook for Android
  • Mail app for Windows 10
  • Investigators can use the SearchQueryInitiatedExchange event to determine if an attacker who may have compromised an account looked for or tried to access sensitive information in the mailbox. The audit record for a SearchQueryInitiatedExchange event contains information such as the actual text of the search query. The audit record also indicates the Outlook environment the search was performed in. By looking at the search queries that an attacker may have performed, an investigator can better understand the intent of the email data that was searched for.

Similar to searching for mailbox items, the SearchQueryInitiatedSharePoint event is triggered when a person searches for items in SharePoint. Events are triggered when searches are performed in the following types of SharePoint sites:

  • Home sites
  • Communication sites
  • Hub sites
  • Sites associated with Microsoft Teams

Investigators can use the SearchQueryInitiatedSharePoint event to determine if an attacker tried to find (and possibly accessed) sensitive information in SharePoint. The audit record for a SearchQueryInitiatedSharePoint event contains also contains the actual text of the search query. The audit record also indicates the type of SharePoint site that was searched. By looking at the search queries that an attacker may have performed, an investigator can better understand the intent and scope of the file data being searched for.You can also run the Search-UnifiedAuditLog -Operations SearchQueryInitiatedSharePoint in Exchange Online PowerShell. You must enable SearchQueryInitiatedSharePoint to be logged so you can search for this event in the audit log. For instructions, see Set up Advanced Audit.

In additional to the events listed above, there are also unique audit events that are only audited when the Advanced Audit license is owned:

Alerting

You can configure Alert Policies to notify you when certain things happen. This can be done in M365, Azure Monitor, MCAS, or Sentinel.

Audit Log Bypass

This article describes how it is possible for a user with administrative rights to bypass Mailbox audit logging, so be sure to document the configuration, and any changes to this configuration during a forensic investigation.

Manage mailbox auditing – Microsoft 365 Compliance | Microsoft Docs

PowerShell Modules

There are a variety of PowerShell modules available, designed to automate gathering the logs or searching them for use in forensic investigations. If you find any more, send me a DM on Twitter at @ITGuySocal

1. Hawk

2. DFIR-O365RC

3. Azure AD Toolkit (This is what Microsoft’s DART team uses)

4. CrowdStrike Reporting Tool for Azure (CRT)

5. Sparrow (this is what the US Government’s CISA’s Cloud Forensics team wrote back in December 2020 to identify activity in a tenant associated with the TTPs used by the hackers who compromised SolarWinds).

6. 365BlueTeamKit by Chaim Black

7. Office 365 Extractor by Joey Rentenaar and Korstiaan Stam from PwC Netherlands Incident Response team

8. Mandiant Azure AD Investigator This is similar to Sparrow in that it was built to  detecting artifacts that may be indicators of Nobelium/UNC2452/Sunburst or other threat actors that use those same techniques. 

Leave a comment