Considerations For Saas Archiving
01 Nov 2023 - sj, tags: insights, product, archiving
Recently, I got an inquery about designing a saas archiving service for customers. In this blog post we’ll cover some best practices and recommendations.
Multi nodes or single node?
It depends on the future expectations. If you expect a huge data volume over time, it’s best to start with a multinode setup, 1 gui node + 1 worker node. Then you may expand the setup with additional worker nodes later.
If Multi node what would the requirements per node be?
The gui/master node can be a small VM, it doesn’t really store much data, and serves only the frontend.
Is there a cost difference on the licensing between the single and multi node?
For starting with the above mentioned layout (1+1 nodes), no. The gui node needs no license, only the worker nodes.
What are the limitations if you setup a single node or would it more be a limit of CPU and memory?
A single node may store up to 10^12 emails per tenant in theory. So far we’ve seen nodes with ~100 million messages. With that scale it’s not only cpu and memory (for mysql and manticore), but also disk i/o. Especially if your users will use the archive heavily.
Where to store the archived emails?
Piler enterprise supports various storage solutions. I’ve seen some archives in AWS storing data on a mounted EFS (=NFS) volume.
Another (remote) solution could be iSCSI.
S3 compatible object stores are also supported, eg. AWS S3, Wasabi, Exascale, IBM cloud, Minio, etc.
Also, a classic, local storage is also supported.
When using a remote storage, eg. NFS, object store, then you may allocate much less local storage to the archive. In this case only the MySQL databases, and the Manticore index data is stored locally.
An additional benefit of S3 storage could be the replication and versioning of the stored data.
The drawback of S3 storage is the somewhat slower access to the emails than the local storage.
Dealing with spam
Piler doesn’t feature anti-spam properties, and the archive should receive clean emails only. You may decide to block spam on your MX servers, but there’s a chance to block legit emails as well. One option might be move marked spam to a quarantine accessible for your users, and when they release false positives (ie. legit emails marked as spam incorrectly), then those emails should be forwarded to the archive.
Another option could be simply marking spam, and let them hit the archive. Then set a short retention policy for recognized spam, and enable purging messages. Thus spam wouldn’t occupy disk space long time in the archive.
Putting the archive behind WAF like Cloudflare
It’s a useful practice to protect the web traffic to the archive. Note that your smtp servers still need to access the archive directly on port 25/tcp.
Using subdomains for organizations
In a multi-tenant setup each tenant (=organization) has their own URI where they can access their emails.
The domain name could be in either in your (=the service provider’s) domain, eg. exampleprovider.com, ie. org1.exampleprovider.com, org2.exampleprovider.com, etc.
Or the organization may require to use their own domain name, eg. archive.example.com.
Piler supports both use cases. The only requirement is that the archive’s domain name points to the archive’s IP-address.
Piler enterprise ships and relies on nginx. When deploying nginx to the localhost, you may run traefik edge router on the public (or private) address of the archive. Then if your domain is hosted at a supported provider, then traefik is able to acquire wildcard https certificate, so the *.exampleprovider.com scheme is very convenient in this case.
Track how many mailboxes each client is using
Piler doesn’t support the concept of mailboxes. All it sees is email addresses. It doesn’t know that a user has 2 more email aliases.
Recent versions of piler keep tracking a sender table with the customer’s email addresses seen by piler. If may serve as a starting point to calculate how many mailboxes they might have.
Because it may not be accurate (the client may argue they have 2 users with 150 email aliases), you may adapt a different pricing model as well, based on the archived data volume. It’s a precise hard fact. You basically query the counters in piler’s database for the given client, and charge them based on.