$ ls topics/{releases,shield,deployment,microsoft-365}
·10 min read
Deployment Control: deploy today, roll out at your pace
The engineering story behind Shield's Deployment Control release: how deployment and rollout became separate steps, how health checks gate enablement, and the routing redesign that removed DNS from the process.
Ben Hathaway
Chief Technology Officer
We have been deploying Shield into Microsoft 365 tenants for about three years, and for most of that time the experience was a wizard: click the button, watch the progress indicator, get green or red. That design came from the same philosophy as Shield’s end-user experience, where removing complexity is the whole point. Carried over to the administrator’s side of the product, it was the wrong tradeoff, and this release moves in the other direction.
The release is called Deployment Control. It covers four capabilities: transparent deployment, health monitoring, staged rollout, and zero-configuration routing. Here is what actually changed in how Shield gets into a tenant, and the design decisions underneath.
Deployment and rollout are different jobs#
The release starts with a definition, because our own language had been getting in the way. “Deploying Shield” used to mean both installing the product and turning it on, in one trip through a wizard, and that coupling made the first click carry all of the risk.
The two jobs are now separate, with separate names. Deployment is the one-time setup: it registers Shield’s enterprise application with the permissions the administrator consents to, creates the mail connectors, and puts in place the configuration Shield needs to work with the tenant. Running it changes nothing about mail flow. Rollout is enabling Shield for users, and it is the only step that changes whose mail Shield handles. Under the hood, enabling or disabling a user changes membership in the security groups that the tenant’s mail routing keys on. Nothing is redeployed in either direction; membership alone determines whether Shield touches that user’s mail.
Everything else in the release builds on that split. An administrator can deploy in the middle of a workday, verify the result, and enable users whenever it makes sense: that evening, next week, or one department at a time.
Transparent deployment#
Deployment runs as a series of tasks against the Microsoft 365 tenant, and the new deployment view shows each task as the deployment engine runs it: the command, live progress, timestamped logs, and the complete API responses coming back from Microsoft. The history is kept after the run, so an administrator and our support team can open the same deployment and read the same output. If a task fails, the administrator can correct the tenant-side problem and run it again from the same screen.
The design borrows from CI/CD tooling, the run views in GitHub Actions or a Cloudflare Pages deploy, which settled on the right pattern years ago: a remote operation should show its progress while it runs, preserve its output afterward, and let the operator retry a failed step once the underlying cause is fixed.
This matters for Shield because a deployment’s outcome depends heavily on the tenant it runs against. A tenant setting that is incompatible with Shield is not something we can fix from our side, but the administrator has to be able to see it before they can. The clearest example is permission propagation. Grant a permission to an application in Microsoft 365 and it may take effect in seconds, or hours, or days; Microsoft’s own admin tools sometimes estimate “a few hours to a few days” for changes to apply. Nearly every deployment failure our team hit during beta traced back to permissions that had been granted but had not finished propagating. In the old wizard, that failure looked like Shield was broken. In the new view, it reads as what it is: a specific request that Microsoft rejected, waiting on Microsoft’s own timeline. We are also feeding those captured responses back into the deployment engine to tune its retry timing, so waiting out propagation becomes the engine’s job instead of the administrator’s.
A full deployment run takes around ten minutes, and nearly all of that is waiting on Microsoft to respond. Nothing requires babysitting; start it, do something else, and read the results when it finishes.
The visibility also answers a fair question administrators have been asking since Shield first shipped: what changes is Shield making to my tenant? Consenting to Shield’s enterprise application means approving a long list of tenant permissions up front, and approving a list is very different from understanding it. Watching the deployment execute shows what those permissions are actually used for, command by command.
One more piece rides along with this. Shield has had three generations of mail flow architecture, Alpha, Bravo, and Charlie, and some longstanding tenants are still on Alpha or Bravo because there was never a clean mechanism to upgrade. The organization page now identifies a legacy configuration with a banner and lets the administrator move to Charlie themselves, using the same transparent deployment machinery, without a support ticket.
Health monitoring#
A deployed tenant does not stay deployed on its own. Anyone with sufficient rights can change a connector, revoke a permission, or alter configuration Shield depends on, and Microsoft can change behavior on its side too. We have had cases where someone spotted the unfamiliar Shield application in their Microsoft admin panel days after deployment and deleted it, which removes every permission Shield has. In the old world, all of these surfaced the same way: as an apparent Shield bug, reported through a ticket, escalating to our engineering team before anyone could see what had actually changed.
Health monitoring makes that state inspectable. Health Pages are built from individual checks we call Pulses. Each Pulse runs commands against Microsoft to compare the state a piece of configuration should be in against the state it is actually in, at the organization, domain, or user level, since Shield has requirements at all three. An administrator runs the checks on demand, most often right after a deployment. During setup, a failing Pulse gates enablement: users cannot be activated onto a configuration Shield already knows is broken.
When a Pulse fails, the page shows the expected state next to the found state and explains the consequence in plain language. Where Shield can fix the problem, there is a one-click repair; where it can’t, there are instructions for the change the administrator needs to make. Either way, re-running the Pulse confirms the result without a ticket.
Repair is deliberately a button rather than an automatic correction. The tenant belongs to the administrator, and a security product silently rewriting its configuration is exactly the behavior administrators distrust. Just as important, drift is evidence. A setting that keeps changing back is a signal worth investigating, maybe a script, maybe a person with global admin rights making changes nobody knows about, and auto-repair would erase that trail before anyone saw it.
One boundary to be clear about: in this release, health checks run on demand. Shield does not yet notify anyone when an organization moves from healthy to unhealthy. Notifications on state change are planned work; making the state visible, explainable, and repairable came first.
Staged rollout#
Once a tenant is deployed and healthy, rollout happens at whatever pace the administrator chooses: one user, a group, or the whole organization at once. Each user is an independent toggle, and the toggle works in both directions.
Disabling a user removes Shield from their mail entirely; nothing passes through unfiltered because nothing passes through at all. Handling returns to what it was before, which includes Microsoft’s Exchange Online Protection, the platform-level filtering that cannot be turned off. A user Shield is not handling yet, or has been pulled back from, keeps that baseline protection.
The per-user switch covers a set of situations that used to have only organization-sized answers. A pilot user can run Shield before the rest of the company, in a ten-seat shop as usefully as in a thousand-seat one. A mailbox with an unusual rule or integration can be pulled out of Shield to isolate a problem, then re-enabled once it’s understood, without disturbing anyone else. Service mailboxes feeding ticketing systems and CRMs can be excluded individually instead of engineering around them.
It also changes migrations. Because activation is per-user, an organization moving from an API-based security product can run both during the transition, split user by user. That does not extend to gateway products, CloudFilter included, because a gateway takes the domain’s MX records and filters everything behind them; there is no per-user split of an MX record. Shield can still be deployed while the gateway is in place, though, so the cutover itself can be quick.
Zero-configuration routing#
The largest engineering change in the release is the one administrators will notice least, because it shows up as an absence. For a tenant already routing mail through Microsoft 365, deploying Shield changes no DNS at all: no MX records, no SPF, no DKIM, no DMARC.
The change behind that absence is Charlie, the third generation of Shield’s mail flow. The evolution of Shield’s mail flow design, from Alpha’s hand-edited MX records through Bravo’s connectors and shared certificate, is its own story. The part that matters for deployment is that Shield no longer does final delivery. Outbound mail is inspected and handed back to the customer’s own tenant, where Microsoft delivers it from its own infrastructure and DKIM-signs it with the customer’s existing keys. Because Shield never sends as the customer, there is nothing to authorize in anyone’s SPF, no DKIM records to publish per domain, and no propagation to wait out. If a domain’s records were right before Shield, they are right after; if they were wrong before, the health checks will say so, but that misconfiguration was there before Shield was.
Inbound needs no DNS either. The MX record stays pointed at Microsoft 365, and transport rules inside the tenant route mail to Shield by user rather than by domain. Each direction keys on its own security group, one inbound and one outbound; rollout keeps the two in step today, but they are deliberately separate mechanisms, so each direction stays independently controllable.
The relay back into the tenant rides on a certificate identity unique to each tenant, provisioned during deployment without customer DNS changes. That design came out of our internal testing, which tripped an undocumented Exchange Online attribution rule and took our own tenant’s outbound down for two days; the postmortem covers how Microsoft 365 decides which tenant email belongs to.
Handing final delivery to Microsoft has a second effect: shared IP reputation stops being a coupling point between customers. When Shield delivered mail itself, a compromised account at one customer could land a shared delivery IP on a blocklist and degrade delivery for everyone using it. Now each tenant’s mail leaves from Microsoft’s infrastructure under that tenant’s own identity, and there is no shared reputation pool for us to manage or for customers to inherit from each other.
The one caveat on “no DNS changes”: an organization leaving a traditional gateway, ours included, still has to point its MX records back at Microsoft 365. That change belongs to the gateway migration itself, and it can happen ahead of time, with Shield deployed and verified before any users are enabled.
Available today#
Internal testing has been running since late June, and it is what hit the attribution incident above: we rolled back, rebuilt the identity model, and started again. Partner beta testing began at the beginning of July. Earlier this month we moved our own corporate tenant onto the new deployment without announcing it, and nobody noticed, which for a mail-flow change is the result we wanted. As of today, Deployment Control is generally available.
The whole release comes down to a sequence: deploy without touching mail flow, verify the tenant is healthy, then enable the right users at the right time, with a way back for any one of them. Every capability above exists to make some part of that sequence visible, checkable, or reversible. The measure of success is the one from our own tenant’s migration: mail kept moving, and nobody had a reason to ask what changed.