💻 “Works on My System” — An Engineer’s Most Dangerous Phrase

It was one of those typical Tuesdays. Mid-sprint, tasks are moving on the board, and our QA team is diligently validating builds for the upcoming release. Suddenly, our release Slack channel lit up. A bug had been reported — a critical one. The checkout module was failing silently during a key transaction flow. The immediate response from the engineer responsible was something many of us have heard too often:

"Hmm… works on my system."

Everyone in the room exchanged knowing glances. This wasn't the first time. And unless we tackled it fundamentally, it wouldn't be the last.

🤔 What Does “Works on My System” Really Mean?

The phrase “works on my system” is a red flag. It doesn't mean the code is flawless. It means the local development environment is out of sync with the environments used for testing, staging, or production.

Here are the usual suspects:

• Different Node.js, Java, or Python versions
• Discrepancies in environment variables
• Uncommitted local config files
• Missing secrets
• OS-level file system behavior (case sensitivity, line endings, etc.)

Worse, it's often a cultural signal: a subtle blame shift to the environment rather than owning the code behavior.

Over the years, I encountered this issue in multiple companies, teams, and tech stacks. It was clear we needed more than best intentions. We needed a system.

🏋️ My Playbook: A Story of Standardization and Scale

1. Start by Containerizing Everything

We began with Docker. Every microservice, every monolith, and even static frontends, got Dockerized.

# Example Dockerfile snippet for a Spring Boot app
FROM openjdk:17-jdk-slim
COPY build/libs/app.jar app.jar
ENTRYPOINT ["java", "-jar", "app.jar"]

This ensured all dependencies, JDK versions, and runtime behaviors were frozen in time.

But we didn’t stop there. Docker Compose lets us define entire ecosystems:

version: '3.8'
services:
  api:
    build: ./api
    environment:
      - SPRING_PROFILES_ACTIVE=dev
    ports:
      - "8080:8080"
  db:
    image: postgres
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: pass

We shipped these definitions with the codebase. Developers could spin up a full stack with a single command: docker-compose up.

2. Developer Experience Must Remain Seamless

Early adoption was rocky. Devs complained:

• “I can’t debug inside a container.”
• “It’s slower than running locally.”

To counter this, we introduced:

• IDE-integrated Docker plugins (VS Code Dev Containers, IntelliJ Docker integration)
• Pre-configured remote debugging ports and launch profiles
• Named volumes to preserve state

We even wrote scripts to auto-generate local .env files from Vault or SSM.

Outcome: Developers stopped feeling like containers were a tax. Instead, they became enablers.

3. Push Containers into CI and Production

Containers in dev are only valuable if they match what runs in prod. We standardized builds to emit immutable Docker images.

These images were deployed to:

• ECS with Fargate for AWS-native teams
• EKS or AKS for Kubernetes-first orgs
• Nomad in one legacy case

We used docker-compose.override.yml to handle local-only overrides (e.g., mounting code for hot reload).

This architecture eliminated the mystery. If it worked in CI and QA, we could safely promote it to prod.

4. Branch-Based, Ephemeral Environments

Next came the game-changer: preview environments on demand.

• Every PR to main triggered a deploy to qa/$branch.
• Infrastructure (API + DB + UI) spun up in a separate namespace.
• Each env autodestructed after 8 hours.

Developers could:

• Test features collaboratively
• Get feedback from PMs and QA
• Demo to stakeholders

Cost Savings?

• By using autoscaling and sleep schedules, we cut staging costs by ~60%.
• Environments only lived when needed.

🚀 Benefits Beyond Standardization

✅ Quick Onboarding: New developers clone the repo, run docker-compose up, and they’re productive in minutes. No need to install half a dozen tools.

✅ Rapid Prototyping: Spinning up a POC becomes trivial. Containers are disposable.

✅ Infrastructure Cost Reduction: Ephemeral environments spin down automatically, avoiding idle costs.

✅ Low Demo Barrier: Need to show progress to a stakeholder? Push a branch. Let the CI/CD pipeline spin up a preview environment with a public link.

✅ Strong Security Posture: Fewer unknowns mean fewer surprises in production. Containers offer tighter resource isolation.

📈 Pitfalls (and How to Avoid Them)

Despite its benefits, the container-driven model had some caveats:

❌ Overhead in Setup

Initial containerization took weeks. CI/CD refactors needed buy-in. Not all teams had Docker expertise.

Mitigation:

• Invest in internal dev enablement docs
• Run workshops, demos, and pair sessions

❌ Divergent Base Images

Devs sometimes used different Dockerfiles. Result: subtle bugs due to mismatched glibc or OpenSSL versions.

Mitigation:

• Maintain a shared base-image per language stack
• Lint and validate Dockerfiles with CI rules

❌ Debugging in Ephemeral Environments

Once an env is gone, logs are too.

Mitigation:

• Integrate log aggregation (e.g., Loki, Datadog)
• Use tools like kubectl cp to extract logs before shutdown

🔄 Alternatives We Explored (and Moved On From)

Only Docker and ephemeral cloud environments gave us predictability, reproducibility, and scalability.

🌟 Today: A Culture of Confidence

Today, when someone says "Works on my system," it’s often followed by:

"...and here's the PR preview at qa/my-feature. Logs are here, and you can test the flow end-to-end."

That’s the culture we’ve built: confident, collaborative, reproducible.

No more wild goose chases. No more finger-pointing. Just software that behaves the same everywhere.

🤔 Final Thought

"Works on my system" isn’t just a debugging issue. It’s a symptom of misaligned environments, missing infrastructure, and inconsistent developer workflows.

Fixing it isn’t about blaming developers. It’s about empowering them.

If your engineering team is still debugging differences between local and QA, maybe it’s time to build your playbook.