Importing Legacy Emails From O365
20 Jun 2024 - sj, tags: archiving, insights, news, product
Microsoft 365 (formerly office365) is a popular email service provider. It’s often a requirement to archive the legacy emails already in o365.
The latest release of piler enterprise features a tool that downloads all emails from o365 using the GraphQL API that you can import.
Setup
Create a new application with a secret in Azure portal
Grant the following application type permissions:
- Mail.Read
- User.ReadBasic.All
If everything is set properly, you see a similar output for the decoded access token. Notice the “roles” section.
{
"aud": "https://graph.microsoft.com",
"iss": "https://sts.windows.net/fe3b7b62-7d1a-4bcc-a513-e3b0798e601f/",
"iat": 1739172947,
"nbf": 1739172947,
"exp": 1739176847,
"aio": "k2RgYGjT0svi6ArYtthhkX+BhEkbAA==",
"app_displayname": "imap-import",
"appid": "4bc3a597-1602-4d3f-b07e-264374805380",
"appidacr": "1",
"idp": "https://sts.windows.net/fe3b7b62-7d1a-4bcc-a513-e3b0798e601f/",
"idtyp": "app",
"oid": "456bad53-c1db-4ab2-b149-5c95e47c556c",
"rh": "1.AVIAYns7_hp9zEulE-OweY5gHwMAAAAAAAAAwAAAAAAAAAC6AABSAA.",
"roles": [
"User.ReadBasic.All",
"Mail.Read"
],
"sub": "456bad53-c1db-4ab2-b149-5c95e47c556c",
"tenant_region_scope": "NA",
"tid": "fe3b7b62-7d1a-4bcc-a513-e3b0798e601f",
"uti": "5hddMVdLBEiufY9lQN7YAA",
"ver": "1.0",
"wids": [
"0997a1d0-0d1d-4acb-b408-d5ca73121e90"
],
"xms_idrel": "7 10",
"xms_tcdt": 1669093587
}
Install the msal python3 package
pip install msal
Create config.py
Create config.py, and set the values for client_id, client_secret and tenant_id. Feel free to customise the list of SKIP_FOLDERS. These are IMAP folders that won’t be processed.
config.py:
DEBUG = False
SKIP_FOLDERS = ['Deleted Items', 'Drafts', 'Junk Email']
client_id = ".........."
client_secret = "......"
tenant_id = "......."
Usage
You can download emails from a single mailbox
download-emails-o365.py --email user@example.com
Or you can download all users’ emails by omitting the –email argument.
The tool creates a directory called ‘email’ in the current directory. Each mailbox gets its own directory, eg. 8f7cd945-c816-491b-8c8c-577bd664189a
The tool preserves the IMAP folder structure, eg.
ls -la emails/8f7cd945-c816-491b-8c8c-577bd664189a
total 12
drwxr-xr-x 2 piler piler 4096 May 30 09:29 Inbox/
drwxr-xr-x 3 piler piler 4096 May 30 09:29 folder2/
drwxr-xr-x 2 piler piler 4096 May 30 09:29 'test folder 1'/
When you have downloaded all emails, run pilerimport to import them. The command below processes all emails in the “emails” directory, and removes the processed emails:
pilerimport -d emails -r