Category Archives: Identity

Everything you wanted to know about Security and Audit Logging in Office 365

There are four primary audit log locations in Office 365. Depending on license level, these logs have varying lengths of retention.

  1. Office 365 “Unified Access Log”
    1. Enabled by ‘opt in’ (The first time you visit the log page, it asks if you want to enable it.)
    2. Goes back 90 days
    3. Accessible here: https://compliance.microsoft.com/auditlogsearch?viewid=Test%20Tab
    4. Documentation: Search the audit log in the Microsoft 365 compliance center – Microsoft 365 Compliance | Microsoft Docs
    5. There are four options for extending the logs beyond 90 days
    6. Option 1: purchase M365 E5 (or other license) “Advanced Audit” which can extend this log to 1 year
    7. Option 2: purchase ‘10-Year Audit Log Retention Add On’ (this add-on first became available for purchase in March 2021). Note: This policy is *not* retroactive.
    8. Option 3: Extend this into Sentinel to get correlation and default query templates
    9. Option 4: Use a 3rd party SIEM to query the Office 365 Management API.
      TIP: When you purchase the “Advanced Audit” license, it will reduce the throttling that occurs when querying the API (so for example you will see data in Splunk much faster!).
    10. Option 5: PowerShell
  2. Azure AD Audit Log
    1. Enabled by Default
    2. Goes back 30 days with an Azure AD P1 license (or 7 days with an Azure AD Free)
    3. Accessible here: Azure Audit Log
    4. Documentation: Audit logs in Azure Active Directory | Microsoft Docs
    5. Latency. Audit logs have a latency ranging from 15 minutes to an hour
    6. Activities. A complete list of each activity audited is available (here)
    7. Limits. The export limit from the web interface is 5,000 records. You can get around this by exporting the logs through one of the options below, which can also be used to extend retention.
    8. Option 1: Extend this log into Azure Log Analytics (aka Azure Monitor) to go beyond 30 days (learn how here)
    9. Option 2: Extend this log into Sentinel to get correlation and default query templates (learn how here)
    10. Option 3: PowerShell or Graph API.
  3. Azure AD Sign-in log
    1. Enabled by Default
    2. Goes back 30 days with an Azure AD P1 license (or 7 days with an Azure AD Free)
    3. Accessible here: Azure Sign-in Log
    4. Documentation: Sign-in logs in Azure Active Directory | Microsoft Docs
    5. Latency. Sign-in activity logs can take from 15 minutes to up to 2 hours for some records. According to the documentation, 95% of all logs will show up in 2 minutes.
    6. Limits. The export limit from the web interface is 5,000 records. You can get around this by exporting the logs through one of the options below, which can also be used to extend retention.
    7. Option 1: Extend this log into Azure Log Analytics (aka Azure Monitor) to go beyond 30 days (learn how here)
      NOTE: Unlike the Azure Audit Log, the Azure AD Sign-in logs require an Azure AD P1 or higher license to export into Log Analytics.
    8. Option 2: Extend this log into Sentinel to get correlation and default query templates (learn how here)
    9. Option 3: PowerShell or Graph API.
  4. Microsoft Cloud App Security
    1. Goes back 6 months
    2. Not enabled by default, requires configuration. You must go to “Connected Apps” then click the three dots to make it include the additional log sources as shown here:
      clip_image002
    3. Requires an M365 E5 license or O365 E5 license (or available via Stand Alone)
    4. Accessible here: https://portal.cloudappsecurity.com/#/audits/
    5. Option 1: Extend this log into Sentinel to go beyond 6 months

Reporting

There are a few really useful built-in reports that analyze the logs and produce findings.

  • Risky sign-ins – A risky sign-in is an indicator for a sign-in attempt that might have been performed by someone who is not the legitimate owner of a user account.

    Latency can range from as little as 5 minutes, to a maximum of 2 hours.

  • Users flagged for risk – A risky user is an indicator for a user account that might have been compromised
    Latency can range from as little as 5 minutes, to a maximum of 2 hours.
  • Risk Detections. Azure AD uses adaptive machine learning algorithms and heuristics to detect suspicious actions that are related to your user accounts. Each detected suspicious action is stored in a record called a risk detection.
    Here are the latencies associated with when the risk detections will appear:
    image

Advanced Audit License

As noted above, the new Advanced Audit License extends the retention of the UAL audit log to 1 year and speeds up 3rd party API throttling. A 3rd useful capability is the additional fields that get audited when this license is applied to mailboxes. The ability to log the ‘MailItemsAccessed’ (some may know this as MessageBind, that is what it was called in on-premises Exchange). Additional entries including exactly which items were sent from the compromised account are also logged.

The Send event is also a mailbox auditing action and is triggered when a user performs one of the following actions:

  • Sends an email message
  • Replies to an email message
  • Forwards an email message

Investigators can use the Send event to identify email sent from a compromised account. The audit record for a Send event contains information about the message, such as when the message was sent, the InternetMessage ID, the subject line, and if the message contained attachments. This auditing information can help investigators identify information about email messages sent from a compromised account or sent by an attacker. Additionally, investigators can use a Microsoft 365 eDiscovery tool to search for the message (by using the subject line or message ID) to identify the recipients the message was sent to and the actual contents of the sent message. You can also run the Search-UnifiedAuditLog -Operations Send or Search-MailboxAuditLog -Operations Send commands in Exchange Online PowerShell.

The MailItemsAccessed mailbox auditing action covers all mail protocols: POP, IMAP, MAPI, EWS, Exchange ActiveSync, and REST.

This is useful in a forensic investigation because it logs which emails were accessed. Imagine the relief of a Legal team when a hacker only accessed 10 items instead of a million items, and none of those 10 items contained PII or PHI data.

Note: When a protocol such as POP, IMAP, or MAPI over HTTPS (aka Outlook Anywhere) syncs a folder, then a single audit event is logged that the folder contents were synced rather than an entry for each item in the folder. (Reference).

Note: If an attacker generates more than 1,000 audit records in 24 hours in a mailbox, then this audit log is paused for 24 hours =( So a crafty hacker could overwhelm the log in order to hide activities (the pause occurs for 24 hours). (Reference)

SearchQueryInitiatedExchange

The SearchQueryInitiatedExchange event is triggered when a person uses Outlook to search for items in a mailbox. Events are triggered when searches are performed in the following Outlook environments:

  • Outlook (desktop client)
  • Outlook on the web (OWA)
  • Outlook for iOS
  • Outlook for Android
  • Mail app for Windows 10
  • Investigators can use the SearchQueryInitiatedExchange event to determine if an attacker who may have compromised an account looked for or tried to access sensitive information in the mailbox. The audit record for a SearchQueryInitiatedExchange event contains information such as the actual text of the search query. The audit record also indicates the Outlook environment the search was performed in. By looking at the search queries that an attacker may have performed, an investigator can better understand the intent of the email data that was searched for.

Similar to searching for mailbox items, the SearchQueryInitiatedSharePoint event is triggered when a person searches for items in SharePoint. Events are triggered when searches are performed in the following types of SharePoint sites:

  • Home sites
  • Communication sites
  • Hub sites
  • Sites associated with Microsoft Teams

Investigators can use the SearchQueryInitiatedSharePoint event to determine if an attacker tried to find (and possibly accessed) sensitive information in SharePoint. The audit record for a SearchQueryInitiatedSharePoint event contains also contains the actual text of the search query. The audit record also indicates the type of SharePoint site that was searched. By looking at the search queries that an attacker may have performed, an investigator can better understand the intent and scope of the file data being searched for.You can also run the Search-UnifiedAuditLog -Operations SearchQueryInitiatedSharePoint in Exchange Online PowerShell. You must enable SearchQueryInitiatedSharePoint to be logged so you can search for this event in the audit log. For instructions, see Set up Advanced Audit.

In additional to the events listed above, there are also unique audit events that are only audited when the Advanced Audit license is owned:

Alerting

You can configure Alert Policies to notify you when certain things happen. This can be done in M365, Azure Monitor, MCAS, or Sentinel.

Audit Log Bypass

This article describes how it is possible for a user with administrative rights to bypass Mailbox audit logging, so be sure to document the configuration, and any changes to this configuration during a forensic investigation.

Manage mailbox auditing – Microsoft 365 Compliance | Microsoft Docs

PowerShell Modules

There are a variety of PowerShell modules available, designed to automate gathering the logs or searching them for use in forensic investigations. If you find any more, send me a DM on Twitter at @ITGuySocal

1. Hawk

2. DFIR-O365RC

3. Azure AD Toolkit (This is what Microsoft’s DART team uses)

4. CrowdStrike Reporting Tool for Azure (CRT)

5. Sparrow (this is what the US Government’s CISA’s Cloud Forensics team wrote back in December 2020 to identify activity in a tenant associated with the TTPs used by the hackers who compromised SolarWinds).

6. 365BlueTeamKit by Chaim Black

7. Office 365 Extractor by Joey Rentenaar and Korstiaan Stam from PwC Netherlands Incident Response team

8. Mandiant Azure AD Investigator This is similar to Sparrow in that it was built to  detecting artifacts that may be indicators of Nobelium/UNC2452/Sunburst or other threat actors that use those same techniques. 

Learning SAML in Azure AD

In my opinion, the best way to learn a technology such as SAML is building a hands-on lab environment. In this blog we’ll explore how to create a SAML Application in Azure AD, using the “Azure AD SAML Toolkit” app.

There are two types of SAML Applications:

– Service Provider Initiated (aka “SP Initiated”). This means that the end-user will navigate to the web application first and then be directed back to the IDP to authenticate, and then the end-user will go back to the application to present the authentication token.

– Identity Provider Initiated (aka “IDP Initiated”). This means that the end-user will navigate to the IDP, in this case Azure is the IDP, so the list of applications will be found at “myapps.microsoft.com.”

Experts may read the simplistic explanations above and notice there are exceptions to these definitions, but to keep things simple we are going with those definitions.

The “Azure AD SAML Toolkit” was written to support SP Initiated SSO.

Getting Started

If you already have a test environment or if you are brave enough to test in your production environment, then skip step 1 and proceed to step 2.

Step 1) Sign up for a Office 365 trial tenant (here)

Step 2) Sign into the Azure Portal (this is the IDP) here: https://portal.azure.com

Step 3) In the search box type in “Azure Active Directory”

image

Step 4) Click Add > Enterprise Application

SNAGHTMLbf612fd

Step 5) Type “Azure AD SAML Toolkit” and then click on the application

image

Step 6) Click Create

image

Step 7) Click Single sign-on in the left navigation then click SAML on the right

image

Step 8) In Step One of “Basic SAML Configuration”  Click Edit

image

Step 9) In the Sign on URL enter: https://samltoolkit.azurewebsites.net/

image

PRO TIP: If you leave this field blank, you are configuring an app for “IDP Initiated” SSO instead of “SP Initiated SSO.” This will help you in the future when troubleshooting why an IDP Initiated app is not working, clear this field and it may start working.

Step 10) In the Reply URL (otherwise known as the Assertion Consumer Service URL) enter:  https://samltoolkit.azurewebsites.net/SAML/Consume

image

Leave the Entity ID with the pre-populated value, it should be:
https://samltoolkit.azurewebsites.net

Step 11)  Click Save
image

Step 12) On step three click Download next to “Certificate (Raw)”

The Certificate download link

Step 13) On Step four “Set up Azure AD SAML Toolkit” copy the values shown into Notepad or your favorite text editor. You’ll need these in a future step

Copy configuration URLs

Step 14) On the left navigation select “Users and Groups” and then add the user you are signed in with (this user will be used in a future step)

image

Step 15) Browse to the Azure AD SAML Toolkit website here: https://samltoolkit.azurewebsites.net

Step 16) Click Register

Azure AD SAML Toolkit Register

Step 17) Enter the email address associated with the user account that you added to the SAML Account in step 14 above

Step 18) Click SAML Configuration on the top navigation

Azure AD SAML Toolkit SAML Configuration

Step 19) Click Create

image

Step 20) Paste the values that you copied from step 13 above, and upload the certificate from step 12 above

image

Step 21) Copy Sign-on URL, Identifier and ACS URL values on SAML Toolkit SSO configuration page and paste into respected textboxes in the Basic SAML Configuration section in the Azure portal

image

Step 22) Test it out! Sign into Myapps.Microsoft.com and find the Azure AD SAML Toolkit. Click on it and then you should see a button to Log in. Click on this button to initiated an SP-Initiated sign-in.

image

If it works, you’ll be brought back to the same configuration page you used to create the app. This is just a simple app to show how SSO works with Azure AD.

References