Take a requirement that sounds unremarkable: upgrade twelve web servers with no downtime. One at a time, drained from the load balancer first, health-checked before returning to service, and the whole thing stops if more than a quarter of them fail.
Every word of that is about coordination between machines. And that is precisely the thing a per-node convergence model has no vocabulary for.
This is, in my experience, the actual reason people move to Ansible — not the YAML, not the agentless SSH thing, not any of the arguments that get made in comparison posts. It is that one class of problem is expressible in one tool and structurally absent from the other.
Why Puppet cannot say it
Puppet's model is that each node independently compiles a catalogue describing its desired state, then converges toward it. The agent wakes on its own schedule, applies the catalogue, sleeps. Ordering exists — require, before, notify, the -> chaining arrow — but it is ordering within one node's catalogue. There is no coordinator, because the design goal was explicitly not to need one.
That design has real virtues. Nodes self-heal without anyone running anything. Drift is corrected continuously. A machine that was offline for a week fixes itself when it comes back. None of that requires a control host to be reachable, or a human to be awake.
What it cannot express is sequence across nodes. Exported resources and PuppetDB let nodes share data — this is how you get a load balancer config that knows about its backends — but sharing facts is not sequencing actions. Nothing in the model says "node B must not begin until node A has finished and passed a health check," because there is no vantage point from which that sentence could be evaluated.
To be fair rather than rhetorical: Puppet does have an answer, and it is Bolt, with the orchestrator in Puppet Enterprise. Both work. But note what they are — imperative, push-based task runners that step outside the catalogue model entirely. The orchestration story required leaving the paradigm, which is itself the point being made here.
Ansible starts from the other end. It is push-based and centrally sequenced by design, so cross-node coordination is not a bolt-on. It is the base case.
The base model, in three sentences
A playbook is a list of plays. Each play targets a group of hosts, and plays execute strictly in order. Within a play, tasks execute in order, and each task runs across all targeted hosts in parallel before the next task begins.
That last clause is the one to internalise, because it is a synchronisation barrier and everything else builds on it. Task three does not begin on any host until task two has completed on every host. You get lockstep progression across your fleet for free, without asking for it.
One practical footnote: the parallelism is capped by forks, which defaults to 5. On a fleet of fifty machines, an unconfigured Ansible is doing ten sequential rounds of five, and people routinely conclude Ansible is slow when what they have actually found is a default. Raise it in ansible.cfg.
serial: turning a fleet into batches
By default a play hits every host at once, which for a deployment means every host is broken simultaneously if the deployment is broken. serial splits the play into batches:
- hosts: web
serial: 1
The critical semantic — and the one most misread — is that the entire play runs to completion for batch one before batch two starts. Not task by task across batches. All tasks, all handlers, one batch at a time. That is what makes it a rolling deploy rather than a slow parallel one.
Percentages work, and so do ramps:
serial: "25%"
serial: [1, 5, "30%"]
The ramp is the form worth knowing. One host first — a canary, where a broken deploy costs you one machine and you find out immediately. Then five. Then 30% of the remainder per batch. This is the shape of every careful production rollout, and it is one line.
Failure thresholds
Ansible's default failure behaviour surprises people: a host that fails is removed from the play, and the play continues on the survivors. With one host left standing out of twelve, it will carry on.
Two keywords fix this, and they do different things.
max_fail_percentage aborts the play when the proportion of failed hosts in the current batch exceeds the threshold:
- hosts: web
serial: 4
max_fail_percentage: 25
Two details. It is evaluated per batch, not against the whole inventory — with batches of four, one failure is 25%, which does not exceed 25, so the play continues. And max_fail_percentage: 0 means any failure at all aborts, since one failure is always greater than zero percent.
any_errors_fatal is the blunter instrument: the play stops the moment any host fails any task, with no threshold and no waiting for the rest of the batch.
- hosts: database
any_errors_fatal: true
Use any_errors_fatal where partial completion is worse than no completion — cluster reconfiguration, coordinated schema changes. Use max_fail_percentage for rollouts where a couple of dead machines are tolerable but a systemic problem is not.
delegate_to: acting on a host you are not configuring
This is the keyword that makes real orchestration possible, and it is more subtle than it first appears.
delegate_to runs a task on a different machine while keeping the variable context of the host currently being iterated. So while Ansible is working through web03, you can execute something on the load balancer — and inventory_hostname still means web03.
- name: Drain this host from the load balancer
community.general.haproxy:
state: disabled
host: "{{ inventory_hostname }}"
backend: app
socket: /var/run/haproxy.sock
wait: true
delegate_to: "{{ groups['loadbalancer'][0] }}"
Read that carefully. The task runs on the load balancer. The host it talks about is the web server being deployed. That separation between execution context and variable context is the whole mechanism, and once it clicks, most orchestration patterns become obvious: draining and restoring, DNS updates, monitoring silences, notifying an external API, waiting for a health check from somewhere that is not the machine you just restarted.
delegate_to: localhost — or its shorthand local_action — covers the common case of running something on the control node.
run_once, and its trap
Some things must happen exactly once across a group. Database migrations, obtaining a shared certificate, seeding a cluster.
- name: Apply database migrations
ansible.builtin.command: /opt/app/bin/migrate
run_once: true
The task executes on the first host of the batch, and its registered result is made available to all hosts. Straightforward — except for the interaction that catches everyone:
run_once combined with serial runs once per batch, not once per play. A rolling deploy with serial: 1 across twelve hosts runs your "run once" migration twelve times.
The reliable fix is not to rely on the keyword at all. Put anything that must happen exactly once in its own play, before the rolling one:
- name: Migrate the database
hosts: web[0]
tasks:
- name: Apply migrations
ansible.builtin.command:
cmd: /opt/app/bin/migrate
creates: "/opt/app/.migrated-{{ app_version }}"
- name: Roll out the application
hosts: web
serial: [1, 3, "50%"]
# ...
Plays are the natural unit for "this, then that." Reaching for run_once inside a batched play is usually a sign the work belongs in a separate play.
Handlers, and when they actually fire
Handlers run at the end of a play — or with serial, at the end of each batch. That default is wrong for rolling deployments, and wrong in a way that produces a passing playbook and a broken deploy.
The sequence you want is: change config, restart service, verify health, return to load balancer. What you get by default is: change config, verify health against the old still-running service, return to load balancer, restart service. The health check passes because it tested the wrong thing.
- meta: flush_handlers
That forces pending handlers to run immediately. In a rolling deploy it belongs between the deployment tasks and the health check, and it is one of the few pieces of Ansible where getting the ordering wrong produces a silent false pass rather than an error.
Putting it together
The requirement from the top of the article, in full:
- name: Rolling application deployment
hosts: web
serial: [1, 3, "50%"]
max_fail_percentage: 25
become: true
pre_tasks:
- name: Drain from the load balancer
community.general.haproxy:
state: disabled
host: "{{ inventory_hostname }}"
backend: app
socket: /var/run/haproxy.sock
wait: true
delegate_to: "{{ groups['loadbalancer'][0] }}"
- name: Allow connections to finish
ansible.builtin.wait_for:
timeout: 15
roles:
- app
post_tasks:
- name: Restart now, not at the end of the batch
meta: flush_handlers
- name: Wait for the health endpoint
ansible.builtin.uri:
url: "http://{{ inventory_hostname }}:8080/health"
status_code: 200
register: health
until: health.status == 200
retries: 30
delay: 2
delegate_to: localhost
- name: Return to the load balancer
community.general.haproxy:
state: enabled
host: "{{ inventory_hostname }}"
backend: app
socket: /var/run/haproxy.sock
delegate_to: "{{ groups['loadbalancer'][0] }}"
Thirty-odd lines, and every element of the original requirement is present and enforced. One canary host, then three, then half the remainder. Drained before touching, health-checked before restored. Abort if a quarter of any batch fails.
Note where the delegated tasks sit: the health check runs from the control node, not from the machine being deployed, because a service asking itself whether it is up is not a useful test. Both load balancer tasks execute on the load balancer while referring to the web server. And pre_tasks and post_tasks exist specifically to bracket roles — pre_tasks, then roles, then post_tasks, with handlers notified along the way.
A few smaller things worth knowing
throttle limits concurrency for a single task independently of forks and serial. Useful when one step hits a rate-limited API and the rest can run wide.
meta: end_host stops the play for the current host only, leaving the others running — the clean way to say "this machine is already at the target version, skip it" without failing anything.
--limit is your recovery tool. When a rollout aborts at batch three, you fix the problem and rerun against only the hosts that did not complete, rather than starting over.
What you give up
This would be a dishonest article if it stopped there, because the trade is real.
Ansible only runs when something runs it. There is no agent, so there is no continuous convergence — between playbook runs, drift accumulates and nothing corrects it. A machine that was offline during your rollout stays out of date until you notice. Someone edits a config by hand at 2am and it stays edited. Puppet's model handles all of that for free, and that is not a small thing to give up.
You also acquire a control node, which is a dependency and a bottleneck. If deployment requires a working CI runner with SSH keys and network reach to everything, then that runner is now production infrastructure.
The usual answer is scheduling: run the playbook from CI on a timer as well as on demand, which recovers approximate convergence at a coarser granularity. The more interesting answer is ansible-pull, which inverts the model — each node clones the repository and applies it to itself on a cron, giving you Puppet's pull-based convergence with Ansible's syntax. You lose orchestration entirely in that mode, which is a reasonable trade for a fleet of identical stateless machines and a bad one for anything needing coordinated deployment.
Which is the honest summary of the whole comparison. Puppet's model is better at keeping a machine correct. Ansible's is better at changing a fleet safely. Most infrastructure needs both, and choosing is mostly a question of which one you would rather solve by hand.