Introduction
For decades, SSH has been the universal tool for managing Linux servers.
Need to restart a service?
SSH.
Need to update PHP?
SSH.
Need to fix a configuration file?
SSH.
Need to investigate a production problem at 2 a.m.?
SSH.
There is nothing inherently wrong with SSH.
In fact, it remains one of the most important tools in a Linux administrator's toolbox.
The problem begins when SSH evolves from an emergency access mechanism into the primary method of managing production infrastructure.
At that point, every administrator is quietly changing reality.
One command at a time.
One forgotten configuration file at a time.
One "temporary" fix that somehow survives for the next three years.
The result is not simply inconsistent servers.
The result is infrastructure that nobody completely understands anymore.
This article argues that the real danger is not SSH itself.
The danger is using interactive shell sessions as a configuration management system.
SSH Is a Fantastic Tool
Let's begin with an unpopular statement.
SSH is not the problem.
SSH is secure.
Fast.
Reliable.
Universally supported.
Without it, operating Linux systems would be significantly harder.
There are many situations where SSH is exactly the right tool:
- investigating an outage
- reading log files
- collecting diagnostics
- examining processes
- testing connectivity
- emergency access when automation has failed
Notice the common theme.
Every example is about understanding a system.
Not changing it.
That distinction matters.
The First Manual Change
Almost every configuration drift story begins the same way.
Someone discovers a production issue.
Perhaps an application is running out of memory.
Maybe Nginx needs another worker.
Perhaps a firewall rule blocks legitimate traffic.
The engineer logs in.
Runs a few commands.
Edits a configuration file.
Restarts a service.
The problem disappears.
Success.
Except…
Was the change documented?
Was it reviewed?
Will staging receive the same configuration?
Will the replacement server receive that modification?
Will the disaster recovery environment behave the same way?
Usually the answer is:
"We'll fix the automation later."
Later rarely arrives.
Every SSH Session Changes History
One fascinating aspect of production systems is that every manual modification creates a new version of reality.
The infrastructure repository still describes one configuration.
Production now contains another.
Nobody notices immediately because everything continues working.
Weeks later another engineer investigates an unrelated problem.
They compare the configuration in Git with the configuration on the server.
They no longer match.
Now the engineer faces an impossible question.
Which version is correct?
The one in Git?
Or the one running production?
Nobody knows.
This uncertainty is far more dangerous than the original problem.
The Birth of Configuration Drift
Configuration drift rarely happens because people are careless.
It happens because people are trying to solve problems.
Production is down.
Customers are waiting.
Management wants answers.
Nobody wants to stop and update an Ansible role while an outage is ongoing.
So the immediate fix happens first.
The automation is updated later.
Except "later" becomes tomorrow.
Tomorrow becomes next month.
Eventually production consists of hundreds of undocumented differences.
Every one of them started with a reasonable decision.
Snowflake Servers
Infrastructure engineers often describe manually maintained servers as snowflakes.
Every server becomes unique.
One machine has a newer OpenSSL version.
Another has an extra PHP extension.
One still contains an old SSL certificate.
Another has debugging enabled.
One has a cron job nobody remembers creating.
Each difference seems insignificant.
Collectively they create an environment where behavior depends on which server handles the request.
Production should never be that interesting.
Good infrastructure is deliberately boring.
SSH Should Observe, Not Configure
A useful rule is remarkably simple.
SSH is for observation.
Automation is for change.
Need to inspect log files?
SSH.
Need to investigate CPU usage?
SSH.
Need to analyze memory consumption?
SSH.
Need to understand why a deployment failed?
SSH.
Need to install software?
Use automation.
Need to change Nginx?
Use automation.
Need to update PHP?
Use automation.
Need to create users?
Use automation.
Need to change firewall rules?
Use automation.
Once this mental model becomes normal, infrastructure becomes dramatically more predictable.
The Psychology of "Quick Fixes"
Every administrator has experienced the temptation.
"It only takes thirty seconds."
"And I'll remember to update Ansible afterwards."
Sometimes you even do.
But the pressure of production encourages optimization for speed rather than repeatability.
That is perfectly understandable.
It is also how technical debt accumulates.
Infrastructure debt rarely arrives through major mistakes.
It arrives through hundreds of successful shortcuts.
Git Should Describe Reality
Imagine cloning your infrastructure repository onto a new laptop.
Without contacting another administrator…
Could you answer:
- Which packages are installed?
- Which services run?
- Which ports are open?
- Which users exist?
- Which TLS configuration is active?
- Which applications are deployed?
If not, the repository is documentation.
It is not the source of truth.
The source of truth should always be the code that defines production.
Everything else is merely a temporary state.
The Break-Glass Exception
Of course, reality is messy.
Sometimes production is on fire.
Customers cannot log in.
Revenue is being lost every minute.
In those situations, manual intervention is justified.
The important part comes afterwards.
Every emergency change should immediately become automation.
Otherwise the emergency becomes permanent architecture.
Good engineering is not avoiding manual fixes.
Good engineering is ensuring they never remain manual.
Conclusion
SSH has transformed Linux administration.
It remains one of the most powerful tools available to infrastructure engineers.
But powerful tools are often misused.
When SSH becomes the primary mechanism for changing production, every administrator begins creating a slightly different version of reality.
Configuration drift grows.
Documentation becomes unreliable.
Servers become unique.
Recovery becomes uncertain.
The goal is not to eliminate SSH.
The goal is to return SSH to its original purpose:
Understanding systems.
Not defining them.
Production infrastructure should be built from code, reviewed through Git, deployed by automation, and reproduced whenever necessary.
If an SSH session changes production permanently, the change is incomplete until it exists in your Infrastructure as Code repository.
Ultimately, the healthiest production environments are not the ones with the fewest SSH sessions.
They are the ones where SSH is rarely needed because automation already knows how the infrastructure should behave.