Cross-Environment Configuration Strategy, Drift Prevention, and Multi-Cluster State Management: Best Practices¶

Introduction¶

Configuration governance is the foundation of reliable, reproducible distributed systems. Without proper configuration management, environments drift, deployments fail, and systems become unpredictable. This guide provides a complete framework for managing configuration across all environments and deployment targets.

What This Guide Covers: - The problem space: why configuration governance is critical - Core principles of environment configuration strategy - Patterns for multi-environment configuration design - Anti-patterns and real-world pitfalls - Environment promotion pipelines - Automated drift detection and enforcement - Configuration metadata and versioning - Secrets lifecycle governance - Tooling and implementation examples - Agentic LLM integration for configuration

Prerequisites: - Understanding of distributed systems and Kubernetes - Familiarity with configuration management and GitOps - Experience with multi-environment deployments

The Problem Space: Why Config Governance Is Critical¶

Configuration Drift Types¶

Schema Drift: Database schemas diverge between environments.

Example:

-- Dev: Has new column
ALTER TABLE users ADD COLUMN email_verified BOOLEAN;

-- Prod: Missing column
-- Schema drift: dev has email_verified, prod doesn't

Impact: Application code works in dev, fails in prod.

Secrets Drift: Secrets differ between environments.

Example:

# Dev: Test credentials
database_password: "test123"

# Prod: Production credentials (different)
database_password: "prod-secret-xyz"

Impact: Security vulnerabilities, authentication failures.

Runtime Drift: Runtime configuration differs.

Example:

# Dev: Low resource limits
resources:
  requests:
    cpu: 100m
    memory: 256Mi

# Prod: High resource limits
resources:
  requests:
    cpu: 1000m
    memory: 2Gi

Impact: Performance differences, resource contention.

Cluster State Drift: Kubernetes cluster state diverges.

Example:

# Dev: 3 replicas
kubectl get deployment app -n dev
# replicas: 3

# Prod: 5 replicas (drifted)
kubectl get deployment app -n prod
# replicas: 5

Impact: Inconsistent scaling, unpredictable behavior.

API Contract Drift: API contracts differ between environments.

Example:

# Dev: API v2 with new field
class UserV2(BaseModel):
    id: int
    name: str
    email: str  # New in v2

# Prod: Still on API v1
class UserV1(BaseModel):
    id: int
    name: str

Impact: Integration failures, breaking changes.

Operator/Default Drift: Operator defaults change over time.

Example:

# Old operator version: default replicas = 1
# New operator version: default replicas = 3
# Deployments drift without explicit config

Impact: Unintended scaling, resource consumption.

Environment Variable Entropy: Environment variables become inconsistent.

Example:

# Dev
DATABASE_URL=postgresql://dev-db:5432/app
LOG_LEVEL=DEBUG

# Prod
DATABASE_URL=postgresql://prod-db:5432/app
LOG_LEVEL=INFO
# Missing: LOG_LEVEL in some pods

Impact: Inconsistent behavior, debugging difficulties.

Config Shadowing: Configs override each other unexpectedly.

Example:

# base.yaml
database:
  host: "localhost"
  port: 5432

# override.yaml
database:
  host: "remote-host"
  # port not specified, but might be overridden by env var
  # Config shadowing: unclear which value wins

Impact: Unpredictable configuration resolution.

Version Skew Between Environments: Different versions deployed.

Example:

# Dev: Latest version
image: app:v1.5.0

# Prod: Older version
image: app:v1.3.0
# Version skew: features available in dev not in prod

Impact: Feature inconsistencies, testing gaps.

Consequences of Configuration Drift¶

Inconsistent Correctness Guarantees: - Dev works, prod fails - Different behavior across environments - Unpredictable outcomes

Broken Reproducibility: - Can't reproduce production issues in staging - Different configurations yield different results - Testing becomes unreliable

Unstable Deployments: - Deployments fail due to config mismatches - Rollbacks don't work because configs differ - Recovery procedures fail

Unpredictable Performance Behavior: - Different resource limits cause performance differences - Caching configurations differ - Load balancing behaves differently

Security Gaps: - Secrets drift creates vulnerabilities - Access controls differ - Audit trails incomplete

Misaligned Audit Trails: - Config changes not tracked - Drift not detected - Compliance violations

"Can't Reproduce in Staging" Anti-Patterns: - Issues only occur in production - Staging doesn't match production - Debugging becomes impossible

Core Principles of Environment Configuration Strategy¶

"Configuration as a Product"¶

Principle: Treat configuration as a first-class product with: - Versioning - Testing - Documentation - Lifecycle management

Example:

# config/product.yaml
apiVersion: v1
kind: ConfigProduct
metadata:
  name: app-config
  version: 1.2.3
spec:
  environments:
    - name: dev
      version: 1.2.3-dev
    - name: prod
      version: 1.2.3

Declarative Configuration First¶

Principle: All configuration should be declarative, not imperative.

Example:

# Declarative: Desired state
apiVersion: apps/v1
kind: Deployment
spec:
  replicas: 3
  template:
    spec:
      containers:
      - name: app
        image: app:v1.2.3

# Not: kubectl scale deployment app --replicas=3

Immutable Infrastructure and Reproducible Builds¶

Principle: Infrastructure and builds should be immutable and reproducible.

Example:

# Immutable: Pinned base image
FROM python:3.11-slim@sha256:abc123...

# Immutable: Pinned dependencies
COPY requirements-lock.txt .
RUN pip install --no-cache-dir -r requirements-lock.txt

Fail-Fast Configuration Validation¶

Principle: Validate configuration early and fail fast.

Example:

# Fail-fast validation
from pydantic import BaseModel, ValidationError

class AppConfig(BaseModel):
    database_url: str
    redis_url: str
    log_level: str

    @validator('log_level')
    def validate_log_level(cls, v):
        if v not in ['DEBUG', 'INFO', 'WARNING', 'ERROR']:
            raise ValueError(f"Invalid log level: {v}")
        return v

# Validate at startup
try:
    config = AppConfig(**load_config())
except ValidationError as e:
    raise SystemExit(f"Configuration validation failed: {e}")

Explicit Environment Boundaries¶

Principle: Make environment boundaries explicit and enforced.

Example:

# Explicit environment boundaries
environments:
  dev:
    cluster: dev-cluster
    namespace: dev
    config_source: config/dev/

  prod:
    cluster: prod-cluster
    namespace: prod
    config_source: config/prod/

Thin Environment Differences (TED Principle)¶

Principle: Minimize differences between environments.

Example:

# Base config (shared)
base:
  database:
    pool_size: 10
    timeout: 30

  # Environment-specific overrides (thin)
  dev:
    database:
      pool_size: 5  # Only difference

  prod:
    database:
      pool_size: 20  # Only difference

Component Autonomy & Isolation¶

Principle: Components should be autonomous and isolated.

Example:

# Autonomous component config
components:
  user-service:
    config:
      database_url: ${USER_DB_URL}
      redis_url: ${REDIS_URL}
    # No dependencies on other component configs

Configuration Layering¶

Principle: Layer configuration from base to runtime.

Layers: 1. Base: Common configuration 2. Environment: Environment-specific overrides 3. Secrets: Secret values 4. Runtime: Runtime-specific adjustments

Example:

# Layer 1: Base
base:
  app:
    name: myapp
    version: 1.0.0

# Layer 2: Environment
environments:
  dev:
    app:
      debug: true

  prod:
    app:
      debug: false

# Layer 3: Secrets
secrets:
  database_password: ${DB_PASSWORD}

# Layer 4: Runtime
runtime:
  replicas: ${REPLICAS}

Version-Locked Everything¶

Principle: Lock versions of all dependencies.

Example:

# Version-locked config
versions:
  app: "1.2.3"
  postgres: "15.3"
  redis: "7.2.0"
  python: "3.11.5"

  dependencies:
    fastapi: "0.104.0"
    pydantic: "2.5.0"

Patterns for Multi-Environment Configuration Design¶

Canonical Configuration Hierarchy¶

Structure:

config/
├── base/
│   ├── app.yaml
│   ├── database.yaml
│   └── redis.yaml
├── environments/
│   ├── dev/
│   │   ├── app.yaml
│   │   └── database.yaml
│   ├── staging/
│   │   ├── app.yaml
│   │   └── database.yaml
│   └── prod/
│       ├── app.yaml
│       └── database.yaml
└── secrets/
    ├── dev/
    └── prod/

Resolution:

# config_resolver.py
class ConfigResolver:
    def resolve(self, environment: str) -> dict:
        """Resolve configuration for environment"""
        # Load base
        base_config = load_config("config/base/")

        # Override with environment
        env_config = load_config(f"config/environments/{environment}/")

        # Merge
        config = deep_merge(base_config, env_config)

        # Inject secrets
        secrets = load_secrets(f"config/secrets/{environment}/")
        config = inject_secrets(config, secrets)

        return config

Dedicated Schema Repository and Environment Diffs¶

Schema Repository:

schemas/
├── postgres/
│   ├── v1/
│   │   └── schema.sql
│   └── v2/
│       └── schema.sql
├── api/
│   ├── v1/
│   │   └── openapi.yaml
│   └── v2/
│       └── openapi.yaml
└── config/
    ├── v1/
    │   └── config-schema.json
    └── v2/
        └── config-schema.json

Environment Diffing:

# diff_tool.py
class EnvironmentDiffer:
    def diff_environments(self, env1: str, env2: str) -> dict:
        """Diff configurations between environments"""
        config1 = load_config(env1)
        config2 = load_config(env2)

        diff = {
            'added': find_added_keys(config1, config2),
            'removed': find_removed_keys(config1, config2),
            'changed': find_changed_values(config1, config2)
        }

        return diff

Context-Aware Config Resolvers¶

Resolver:

# context_aware_resolver.py
class ContextAwareConfigResolver:
    def resolve(self, context: dict) -> dict:
        """Resolve config based on context"""
        environment = context.get('environment')
        cluster = context.get('cluster')
        region = context.get('region')

        # Load base
        config = load_base_config()

        # Apply environment overrides
        config = apply_environment_overrides(config, environment)

        # Apply cluster overrides
        config = apply_cluster_overrides(config, cluster)

        # Apply region overrides
        config = apply_region_overrides(config, region)

        return config

Strict Type-Checked Configuration Loading¶

Type Checking:

# typed_config.py
from pydantic import BaseModel, Field
from typing import Literal

class DatabaseConfig(BaseModel):
    host: str = Field(..., description="Database host")
    port: int = Field(5432, ge=1, le=65535, description="Database port")
    database: str = Field(..., description="Database name")
    pool_size: int = Field(10, ge=1, le=100, description="Connection pool size")

class AppConfig(BaseModel):
    name: str
    environment: Literal["dev", "staging", "prod"]
    database: DatabaseConfig
    log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR"] = "INFO"

# Load with validation
config = AppConfig(**load_raw_config())

Single-Source-of-Truth Values¶

Pattern:

# Single source of truth
# config/base/app.yaml
app:
  name: myapp
  version: 1.2.3

# Environments reference base
# config/environments/dev/app.yaml
app:
  # Inherit from base
  name: ${base.app.name}
  version: ${base.app.version}
  # Only override what's different
  debug: true

"Configuration Contracts" with Versioned Validation Rules¶

Contract Definition:

# config-contract.yaml
apiVersion: config.example.com/v1
kind: ConfigContract
metadata:
  name: app-config-contract
  version: 1.0.0
spec:
  schema:
    type: object
    properties:
      database_url:
        type: string
        pattern: "^postgresql://"
      log_level:
        type: string
        enum: ["DEBUG", "INFO", "WARNING", "ERROR"]
    required: ["database_url", "log_level"]

  validation_rules:
    - name: "database_url_format"
      rule: "must match postgresql:// pattern"
    - name: "log_level_enum"
      rule: "must be one of DEBUG, INFO, WARNING, ERROR"

Feature Flags Aligned with Environment Tiers¶

Feature Flag Strategy:

# feature-flags.yaml
feature_flags:
  new_ui:
    dev: true
    staging: true
    prod: false  # Not ready for prod

  experimental_api:
    dev: true
    staging: false  # Too risky for staging
    prod: false

  performance_optimization:
    dev: true
    staging: true
    prod: true  # Safe for all environments

Temporal Configuration Behavior¶

Temporal Configs:

# temporal-config.yaml
temporal_configs:
  - name: "maintenance_window"
    schedule: "0 2 * * 0"  # Sunday 2 AM
    config:
      maintenance_mode: true

  - name: "high_traffic_period"
    schedule: "0 9 * * 1-5"  # Weekdays 9 AM
    config:
      replicas: 10
      cache_ttl: 300

Docker Compose Overrides¶

Pattern:

# docker-compose.base.yaml
services:
  app:
    image: app:latest
    environment:
      - DATABASE_URL=${DATABASE_URL}
      - LOG_LEVEL=INFO

# docker-compose.dev.yaml
services:
  app:
    environment:
      - LOG_LEVEL=DEBUG
    volumes:
      - ./src:/app/src

# docker-compose.prod.yaml
services:
  app:
    environment:
      - LOG_LEVEL=INFO
    deploy:
      replicas: 3

Usage:

# Dev
docker-compose -f docker-compose.base.yaml -f docker-compose.dev.yaml up

# Prod
docker-compose -f docker-compose.base.yaml -f docker-compose.prod.yaml up

Helm Charts / Kustomize Overlays¶

Helm Pattern:

# values-base.yaml
replicas: 1
image:
  repository: app
  tag: latest
resources:
  requests:
    cpu: 100m
    memory: 256Mi

# values-dev.yaml
replicas: 1
image:
  tag: dev
resources:
  requests:
    cpu: 100m
    memory: 256Mi

# values-prod.yaml
replicas: 5
image:
  tag: v1.2.3
resources:
  requests:
    cpu: 1000m
    memory: 2Gi

Kustomize Pattern:

kustomize/
├── base/
│   ├── kustomization.yaml
│   └── deployment.yaml
├── dev/
│   ├── kustomization.yaml
│   └── patches/
│       └── replicas.yaml
└── prod/
    ├── kustomization.yaml
    └── patches/
        └── replicas.yaml

Postgres Config Files¶

Postgres Config:

# postgresql.base.conf
shared_buffers = 256MB
max_connections = 100
work_mem = 4MB

# postgresql.dev.conf (override)
shared_buffers = 128MB
max_connections = 50

# postgresql.prod.conf (override)
shared_buffers = 1GB
max_connections = 200

Prefect Flows¶

Prefect Config:

# prefect_config.py
from prefect import flow
from pydantic import BaseSettings

class PrefectConfig(BaseSettings):
    environment: str
    database_url: str
    redis_url: str

    class Config:
        env_file = f".env.{environment}"

@flow
def etl_flow(config: PrefectConfig):
    """ETL flow with environment-aware config"""
    # Use config
    pass

Air-Gapped Configs¶

Air-Gapped Pattern:

# Package configs for air-gapped deployment
tar -czf config-bundle-$(date +%Y%m%d).tar.gz \
    config/ \
    schemas/ \
    secrets-encrypted/ \
    checksums.txt

# Verify on air-gapped cluster
sha256sum -c checksums.txt

Anti-Patterns and Real-World Pitfalls¶

Overloaded .env Files¶

Problem: Single .env file with everything.

Example:

# .env (overloaded)
DATABASE_URL=...
REDIS_URL=...
API_KEY=...
SECRET_KEY=...
LOG_LEVEL=...
DEBUG=...
# 100+ variables, no organization

Fix: Organize by domain and environment.

# .env.database
DATABASE_URL=...
DATABASE_POOL_SIZE=...

# .env.redis
REDIS_URL=...
REDIS_TTL=...

# .env.app
LOG_LEVEL=...
DEBUG=...

Copy-Pasted YAML¶

Problem: YAML files copied between environments.

Example:

# config/dev/app.yaml
app:
  name: myapp
  database_url: postgresql://dev-db/app

# config/prod/app.yaml (copy-pasted, forgot to change)
app:
  name: myapp
  database_url: postgresql://dev-db/app  # Wrong! Still pointing to dev

Fix: Use inheritance and templating.

# config/base/app.yaml
app:
  name: myapp

# config/environments/dev/app.yaml
app:
  database_url: postgresql://dev-db/app

# config/environments/prod/app.yaml
app:
  database_url: postgresql://prod-db/app

Environment-Specific Logic Hidden in Code¶

Problem: Environment checks in application code.

Example:

# Bad: Environment logic in code
if os.getenv("ENVIRONMENT") == "dev":
    database_url = "postgresql://dev-db/app"
elif os.getenv("ENVIRONMENT") == "prod":
    database_url = "postgresql://prod-db/app"

Fix: Externalize to configuration.

# Good: Config-driven
config = load_config()
database_url = config.database_url

Divergent Secrets Management Systems¶

Problem: Different secret management in each environment.

Fix: Standardize on one system.

# Standardized secrets management
secrets_management:
  system: "vault"  # Same for all environments
  backend: "vault-backend"
  paths:
    dev: "secret/data/dev"
    prod: "secret/data/prod"

Stale or Undocumented Toggles¶

Problem: Feature flags never cleaned up.

Fix: Document and audit feature flags.

# feature-flags.yaml
feature_flags:
  old_feature:
    enabled: false
    deprecated: true
    removal_date: "2024-06-01"
    owner: "team-a"

  new_feature:
    enabled: true
    description: "New feature description"
    owner: "team-b"

Per-Developer Configuration Mutations¶

Problem: Each developer has different local config.

Fix: Standardize local development.

# .devcontainer/devcontainer.json
{
  "name": "Standard Dev Environment",
  "dockerComposeFile": "docker-compose.yml",
  "service": "app",
  "workspaceFolder": "/workspace"
}

Configuration with Implicit Default Changes¶

Problem: Defaults change without notice.

Example:

# Old version: default replicas = 1
# New version: default replicas = 3
# Deployments change without explicit config

Fix: Always specify values explicitly.

# Explicit values
replicas: 3  # Never rely on defaults

Manual Patching of RKE2 Nodes¶

Problem: Manual changes to nodes cause drift.

Fix: Use GitOps and automation.

# GitOps: All changes through Git
apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
  source:
    repoURL: https://github.com/org/config-repo
    path: k8s/

Drift Introduced by "Snowflake" Servers¶

Problem: Servers configured differently.

Fix: Use infrastructure as code.

# Infrastructure as code
resources:
  - type: server
    name: app-server-1
    config: standard-app-server-config

  - type: server
    name: app-server-2
    config: standard-app-server-config  # Same config

Environment Promotion Pipelines¶

Dev → Test Promotion¶

Pipeline:

graph LR
    Dev["Dev Environment"]
    Validate["Validate Config"]
    Test["Test Environment"]
    Verify["Verify Deployment"]

    Dev --> Validate
    Validate --> Test
    Test --> Verify

    style Dev fill:#e1f5ff
    style Validate fill:#fff4e1
    style Test fill:#e8f5e9
    style Verify fill:#f3e5f5

Implementation:

# .github/workflows/promote-dev-to-test.yml
name: Promote Dev to Test
on:
  workflow_dispatch:

jobs:
  promote:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Validate dev config
        run: |
          python scripts/validate_config.py config/environments/dev/

      - name: Diff dev and test
        run: |
          python scripts/diff_configs.py dev test

      - name: Promote to test
        run: |
          python scripts/promote_config.py dev test

      - name: Deploy to test
        run: |
          kubectl apply -f config/environments/test/

Test → Staging Promotion¶

Pipeline:

# .github/workflows/promote-test-to-staging.yml
name: Promote Test to Staging
on:
  workflow_dispatch:
    inputs:
      approval:
        description: 'Approve promotion'
        required: true
        type: boolean

jobs:
  promote:
    if: github.event.inputs.approval == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Validate test config
        run: |
          python scripts/validate_config.py config/environments/test/

      - name: Schema promotion check
        run: |
          python scripts/check_schema_promotion.py test staging

      - name: Promote to staging
        run: |
          python scripts/promote_config.py test staging

Staging → Prod Promotion¶

Pipeline:

# .github/workflows/promote-staging-to-prod.yml
name: Promote Staging to Prod
on:
  workflow_dispatch:
    inputs:
      approval:
        description: 'Approve production promotion'
        required: true
        type: boolean

jobs:
  promote:
    if: github.event.inputs.approval == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Validate staging config
        run: |
          python scripts/validate_config.py config/environments/staging/

      - name: Production readiness check
        run: |
          python scripts/check_prod_readiness.py staging

      - name: Promote to prod
        run: |
          python scripts/promote_config.py staging prod

      - name: Deploy to prod
        run: |
          kubectl apply -f config/environments/prod/

Automated Diffing¶

Diff Tool:

# scripts/diff_configs.py
import yaml
from deepdiff import DeepDiff

def diff_configs(env1: str, env2: str):
    """Diff configurations between environments"""
    config1 = load_config(env1)
    config2 = load_config(env2)

    diff = DeepDiff(config1, config2, ignore_order=True)

    if diff:
        print(f"Differences between {env1} and {env2}:")
        print(diff.pretty())
        return False
    else:
        print(f"No differences between {env1} and {env2}")
        return True

Schema Promotion Policies¶

Policy:

# schema-promotion-policy.yaml
schema_promotion:
  rules:
    - name: "backward_compatible_only"
      rule: "Only backward-compatible schema changes allowed"
      enforcement: "strict"

    - name: "migration_required"
      rule: "Breaking changes require migration plan"
      enforcement: "strict"

    - name: "version_bump"
      rule: "Schema changes require version bump"
      enforcement: "strict"

Config Gates and Approval Workflows¶

Gate Configuration:

# config-gates.yaml
gates:
  - name: "config_validation"
    type: "automated"
    checks:
      - "schema_validation"
      - "type_validation"
      - "constraint_validation"

  - name: "security_review"
    type: "manual"
    approvers:
      - "security-team"

  - name: "architecture_review"
    type: "manual"
    approvers:
      - "architecture-team"

Reproducible Environment Snapshots¶

Snapshot Creation:

# scripts/create_snapshot.py
def create_environment_snapshot(environment: str) -> dict:
    """Create reproducible environment snapshot"""
    snapshot = {
        'timestamp': datetime.now().isoformat(),
        'environment': environment,
        'config': load_config(environment),
        'schemas': load_schemas(environment),
        'secrets_hash': hash_secrets(environment),
        'images': get_image_versions(environment),
        'checksum': None
    }

    # Calculate checksum
    snapshot['checksum'] = calculate_checksum(snapshot)

    return snapshot

Config Freeze Periods¶

Freeze Policy:

# config-freeze-policy.yaml
freeze_periods:
  - name: "year_end_freeze"
    start: "2024-12-15"
    end: "2025-01-05"
    environments: ["prod"]
    exceptions:
      - "critical_security_patches"
      - "emergency_fixes"

Automated Config Migration Scripts¶

Migration Script:

# scripts/migrate_config.py
class ConfigMigrator:
    def migrate(self, source_env: str, target_env: str):
        """Migrate configuration between environments"""
        source_config = load_config(source_env)
        target_config = load_config(target_env)

        # Apply migrations
        migrations = load_migrations()
        for migration in migrations:
            if migration.applies_to(source_config, target_config):
                target_config = migration.apply(target_config)

        # Save migrated config
        save_config(target_env, target_config)

Config Rollback Procedures¶

Rollback Script:

#!/bin/bash
# scripts/rollback_config.sh

ENVIRONMENT=$1
VERSION=$2

echo "Rolling back ${ENVIRONMENT} to version ${VERSION}"

# Restore config from version
git checkout ${VERSION} -- config/environments/${ENVIRONMENT}/

# Validate restored config
python scripts/validate_config.py config/environments/${ENVIRONMENT}/

# Apply restored config
kubectl apply -f config/environments/${ENVIRONMENT}/

echo "Rollback complete"

Automated Drift Detection & Enforcement¶

GitOps Enforcement Patterns¶

ArgoCD Configuration:

# argocd-app.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: app-config
spec:
  source:
    repoURL: https://github.com/org/config-repo
    path: k8s/
    targetRevision: main
  destination:
    server: https://kubernetes.default.svc
    namespace: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Temporal Auditing of Runtime Configs¶

Audit Script:

# audit/runtime_config_audit.py
class RuntimeConfigAuditor:
    def audit_runtime_configs(self):
        """Audit runtime configurations"""
        # Get desired config from Git
        desired_config = load_config_from_git()

        # Get actual config from cluster
        actual_config = get_config_from_cluster()

        # Compare
        drift = compare_configs(desired_config, actual_config)

        if drift:
            # Alert on drift
            alert_on_drift(drift)

            # Optionally auto-remediate
            if self.auto_remediate_enabled():
                self.remediate_drift(drift)

        return drift

Kubernetes Cluster State Diffing¶

State Diff Tool:

# drift/k8s_state_diff.py
class KubernetesStateDiffer:
    def diff_cluster_state(self, cluster: str):
        """Diff Kubernetes cluster state"""
        # Get desired state from Git
        desired_state = load_kubernetes_manifests()

        # Get actual state from cluster
        actual_state = get_cluster_state(cluster)

        # Diff
        diff = kubectl_diff(desired_state, actual_state)

        return diff

Postgres Config/State Hash Monitors¶

Postgres Monitor:

# monitoring/postgres_config_monitor.py
class PostgresConfigMonitor:
    def monitor_postgres_config(self, instance: str):
        """Monitor Postgres configuration"""
        # Get config hash
        config_hash = get_postgres_config_hash(instance)

        # Compare with expected
        expected_hash = get_expected_config_hash(instance)

        if config_hash != expected_hash:
            alert_on_config_drift(instance, config_hash, expected_hash)

Redis Config Fingerprinting¶

Redis Fingerprint:

# monitoring/redis_fingerprint.py
class RedisConfigFingerprinter:
    def fingerprint_redis_config(self, instance: str) -> str:
        """Generate Redis config fingerprint"""
        config = get_redis_config(instance)

        # Create fingerprint
        fingerprint = hashlib.sha256(
            json.dumps(config, sort_keys=True).encode()
        ).hexdigest()

        return fingerprint

FDW Target Drift Detection¶

FDW Drift Detection:

-- Check FDW target consistency
SELECT 
    srvname,
    srvoptions,
    md5(srvoptions::text) AS config_hash
FROM pg_foreign_server
WHERE srvname LIKE 's3_%';

-- Compare hashes across environments
-- Alert if hashes differ

Docker Image SBOM Diffing¶

SBOM Diff Tool:

# drift/sbom_diff.py
class SBOMDiffer:
    def diff_sboms(self, image1: str, image2: str) -> dict:
        """Diff SBOMs between images"""
        sbom1 = get_sbom(image1)
        sbom2 = get_sbom(image2)

        diff = {
            'added_packages': find_added_packages(sbom1, sbom2),
            'removed_packages': find_removed_packages(sbom1, sbom2),
            'version_changes': find_version_changes(sbom1, sbom2)
        }

        return diff

Runbook Automation for Reconciliation¶

Reconciliation Runbook:

#!/bin/bash
# runbooks/reconcile_config_drift.sh

echo "=== Config Drift Reconciliation ==="

# 1. Detect drift
DRIFT=$(python scripts/detect_drift.py)

if [ -z "$DRIFT" ]; then
    echo "No drift detected"
    exit 0
fi

# 2. Analyze drift
echo "Analyzing drift..."
python scripts/analyze_drift.py "$DRIFT"

# 3. Reconcile
echo "Reconciling drift..."
python scripts/reconcile_drift.py "$DRIFT"

# 4. Verify
echo "Verifying reconciliation..."
python scripts/verify_reconciliation.py

echo "=== Reconciliation complete ==="

Alerting Patterns¶

Prometheus Alerts:

# alerts/config_drift.yaml
groups:
  - name: config_drift
    rules:
      - alert: ConfigDriftDetected
        expr: config_drift_score > 0.1
        for: 5m
        annotations:
          summary: "Configuration drift detected"
          description: "Config drift score: {{ $value }}"

      - alert: PostgresConfigDrift
        expr: postgres_config_hash != postgres_expected_config_hash
        for: 1m
        annotations:
          summary: "Postgres configuration drift"
          description: "Postgres config hash mismatch"

Configuration Metadata & Versioning¶

Config Lineage¶

Lineage Tracking:

# metadata/lineage.py
class ConfigLineageTracker:
    def track_lineage(self, config: dict) -> dict:
        """Track configuration lineage"""
        lineage = {
            'config_id': generate_config_id(config),
            'source': 'git',
            'commit': get_git_commit(),
            'author': get_git_author(),
            'timestamp': datetime.now().isoformat(),
            'parent_configs': get_parent_configs(config),
            'derived_configs': get_derived_configs(config)
        }

        return lineage

Environment Tagging¶

Tagging Strategy:

# config-tags.yaml
tags:
  environment: "prod"
  cluster: "prod-us-east-1"
  region: "us-east-1"
  version: "1.2.3"
  deployment_id: "deploy-20240115-120000"

Build-ID Embedding¶

Build ID Injection:

# build_id_injection.py
class BuildIDInjector:
    def inject_build_id(self, config: dict) -> dict:
        """Inject build ID into configuration"""
        build_id = os.getenv("BUILD_ID") or generate_build_id()

        config['metadata'] = {
            'build_id': build_id,
            'build_timestamp': datetime.now().isoformat(),
            'git_commit': get_git_commit(),
            'git_branch': get_git_branch()
        }

        return config

Schema Versioning¶

Schema Version Strategy:

# schema-versioning.yaml
schemas:
  postgres:
    current_version: "2.0.0"
    versions:
      - version: "1.0.0"
        migration: "migrations/001_initial.sql"
      - version: "2.0.0"
        migration: "migrations/002_add_users.sql"

  api:
    current_version: "1.5.0"
    versions:
      - version: "1.0.0"
        openapi: "api/v1/openapi.yaml"
      - version: "1.5.0"
        openapi: "api/v1.5/openapi.yaml"

Config Fingerprinting¶

Fingerprint Generation:

# fingerprinting.py
class ConfigFingerprinter:
    def generate_fingerprint(self, config: dict) -> str:
        """Generate configuration fingerprint"""
        # Normalize config (remove metadata, sort keys)
        normalized = self.normalize_config(config)

        # Generate hash
        fingerprint = hashlib.sha256(
            json.dumps(normalized, sort_keys=True).encode()
        ).hexdigest()

        return fingerprint

Artifact Provenance¶

Provenance Tracking:

# provenance.yaml
provenance:
  config_id: "config-abc123"
  source:
    repo: "https://github.com/org/config-repo"
    commit: "abc123def456"
    path: "config/environments/prod/"
  build:
    build_id: "build-xyz789"
    build_timestamp: "2024-01-15T12:00:00Z"
  deployment:
    deployed_by: "ci-system"
    deployed_at: "2024-01-15T12:05:00Z"
    deployed_to: "prod-cluster"

In-Cluster Validation Hooks¶

Validation Webhook:

# validation/webhook.py
from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/validate", methods=["POST"])
def validate_config():
    """Kubernetes validation webhook"""
    admission_review = request.json

    # Extract config
    config = admission_review["request"]["object"]

    # Validate
    try:
        validate_configuration(config)
        allowed = True
        message = "Configuration valid"
    except ValidationError as e:
        allowed = False
        message = str(e)

    # Return response
    return jsonify({
        "apiVersion": "admission.k8s.io/v1",
        "kind": "AdmissionReview",
        "response": {
            "uid": admission_review["request"]["uid"],
            "allowed": allowed,
            "status": {"message": message}
        }
    })

Environment-Aware Dashboards¶

Dashboard Configuration:

id=__codelineno-74-1 name=__codelineno-74-1 href=#__codelineno-74-1>{ "dashboard": { "title": "Configuration Health by Environment", "panels": [ { "title": "Config Drift by Environment", "targets": [ { "expr": "config_drift_score{environment=\"dev\"}", "legendFormat": "Dev" }, { "expr": "config_drift_score{environment=\"prod\"}", "legendFormat": "Prod" } ] }, { "title": "Config Version Distribution", "targets": [ { "expr": "count by (version) (config_versions)", "legendFormat": "{{version}}" } ] } ] } class=p>}

Secrets Lifecycle Governance¶

Sealed Secrets¶

Sealed Secrets Pattern:

# sealed-secret.yaml
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: database-credentials
spec:
  encryptedData:
    password: AgBy3i4OJSWK+PiTySYZZA9rO43cGDEQAx...

Age/SOPS Tooling¶

SOPS Configuration:

# .sops.yaml
creation_rules:
  - path_regex: secrets/dev/.*
    kms: "arn:aws:kms:us-east-1:123456789012:key/abc123"

  - path_regex: secrets/prod/.*
    kms: "arn:aws:kms:us-east-1:123456789012:key/xyz789"

Usage:

# Encrypt
sops -e -i secrets/dev/database.yaml

# Decrypt
sops -d secrets/dev/database.yaml

HashiCorp Vault Patterns¶

Vault Integration:

# vault-config.yaml
vault:
  address: "https://vault.example.com"
  auth:
    method: "kubernetes"
    role: "app-role"
  paths:
    dev: "secret/data/dev"
    prod: "secret/data/prod"

External Secrets Operator:

# external-secret.yaml
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: database-credentials
spec:
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: database-credentials
  data:
    - secretKey: password
      remoteRef:
        key: secret/data/prod/database
        property: password

Rotation Schedules¶

Rotation Policy:

# rotation-policy.yaml
rotation:
  schedules:
    database_passwords:
      frequency: "90 days"
      environments:
        - prod
        - staging

    api_keys:
      frequency: "180 days"
      environments:
        - prod

    tls_certificates:
      frequency: "365 days"
      environments:
        - prod

KMS Integration¶

KMS Configuration:

# kms-config.yaml
kms:
  provider: "aws"
  region: "us-east-1"
  key_id: "arn:aws:kms:us-east-1:123456789012:key/abc123"
  encryption_context:
    environment: "prod"
    service: "database"

Ephemeral Secrets for Ephemeral Environments¶

Ephemeral Secret Pattern:

# ephemeral_secrets.py
class EphemeralSecretManager:
    def create_ephemeral_secret(self, environment: str, ttl: int = 3600):
        """Create ephemeral secret for temporary environment"""
        secret = generate_secret()

        # Store with TTL
        store_secret_with_ttl(environment, secret, ttl)

        return secret

Redaction Rules¶

Redaction Configuration:

# redaction-rules.yaml
redaction:
  rules:
    - pattern: "password.*"
      action: "redact"
      replacement: "***REDACTED***"

    - pattern: "api_key.*"
      action: "redact"
      replacement: "***REDACTED***"

    - pattern: "secret.*"
      action: "redact"
      replacement: "***REDACTED***"

Integration with Postgres pgaudit¶

PgAudit Integration:

-- Configure pgaudit for secret access
ALTER SYSTEM SET pgaudit.log = 'all';
ALTER SYSTEM SET pgaudit.log_catalog = off;
ALTER SYSTEM SET pgaudit.log_parameter = on;

-- Audit secret table access
CREATE TABLE secrets (
    id SERIAL PRIMARY KEY,
    key VARCHAR(255),
    value TEXT,
    environment VARCHAR(50)
);

-- Enable auditing
ALTER TABLE secrets ENABLE ROW LEVEL SECURITY;

Tooling & Implementation Examples¶

Helm Hierarchical Values¶

Values Structure:

# values.yaml (base)
replicas: 1
image:
  repository: app
  tag: latest

# values-dev.yaml
replicas: 1
image:
  tag: dev

# values-prod.yaml
replicas: 5
image:
  tag: v1.2.3

Usage:

helm install app ./chart -f values.yaml -f values-dev.yaml

Kustomize Overlays¶

Overlay Structure:

kustomize/
├── base/
│   ├── kustomization.yaml
│   └── deployment.yaml
├── dev/
│   ├── kustomization.yaml
│   └── patches/
│       └── replicas.yaml
└── prod/
    ├── kustomization.yaml
    └── patches/
        └── replicas.yaml

Base Kustomization:

# base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml

Dev Overlay:

# dev/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../base
patches:
  - path: patches/replicas.yaml

Python Pydantic Settings¶

Pydantic Config:

# config.py
from pydantic import BaseSettings, Field
from typing import Literal

class DatabaseSettings(BaseSettings):
    host: str = Field(..., env="DATABASE_HOST")
    port: int = Field(5432, env="DATABASE_PORT")
    database: str = Field(..., env="DATABASE_NAME")
    pool_size: int = Field(10, ge=1, le=100)

class AppSettings(BaseSettings):
    environment: Literal["dev", "staging", "prod"]
    database: DatabaseSettings
    log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR"] = "INFO"

    class Config:
        env_file = f".env.{environment}"
        env_file_encoding = "utf-8"

# Load config
settings = AppSettings(environment="prod")

Go fx/Config Loader Patterns¶

Go Config:

// config.go
package config

import (
    "github.com/spf13/viper"
)

type Config struct {
    Environment string `mapstructure:"environment"`
    Database    DatabaseConfig `mapstructure:"database"`
    LogLevel    string `mapstructure:"log_level"`
}

type DatabaseConfig struct {
    Host     string `mapstructure:"host"`
    Port     int    `mapstructure:"port"`
    Database string `mapstructure:"database"`
}

func LoadConfig(env string) (*Config, error) {
    viper.SetConfigName("config")
    viper.SetConfigType("yaml")
    viper.AddConfigPath(fmt.Sprintf("config/environments/%s/", env))

    if err := viper.ReadInConfig(); err != nil {
        return nil, err
    }

    var config Config
    if err := viper.Unmarshal(&config); err != nil {
        return nil, err
    }

    return &config, nil
}

Rust figment/Config Crates¶

Rust Config:

// config.rs
use figment::{Figment, providers::{Format, Yaml, Env}};
use serde::{Deserialize, Serialize};

#[derive(Debug, Deserialize, Serialize)]
struct Config {
    environment: String,
    database: DatabaseConfig,
    log_level: String,
}

#[derive(Debug, Deserialize, Serialize)]
struct DatabaseConfig {
    host: String,
    port: u16,
    database: String,
}

impl Config {
    fn load(env: &str) -> Result<Self, figment::Error> {
        Figment::new()
            .merge(Yaml::file(format!("config/environments/{}/config.yaml", env)))
            .merge(Env::prefixed("APP_"))
            .extract()
    }
}

Prefect Block Configurations¶

Prefect Blocks:

# prefect_blocks.py
from prefect.blocks.system import Secret
from prefect import flow

@flow
def etl_flow():
    """ETL flow with Prefect blocks"""
    # Load secret block
    db_password = Secret.load("database-password")

    # Use in flow
    database_url = f"postgresql://user:{db_password.get()}@host/db"
    # ...

Postgres Config Templating via Jinja¶

Jinja Template:

# postgresql.conf.j2
# Auto-generated from template
shared_buffers = {{ shared_buffers }}
max_connections = {{ max_connections }}
work_mem = {{ work_mem }}

# Environment-specific
{% if environment == "prod" %}
shared_buffers = 1GB
max_connections = 200
{% else %}
shared_buffers = 256MB
max_connections = 100
{% endif %}

Template Rendering:

# render_postgres_config.py
from jinja2 import Template

def render_postgres_config(environment: str) -> str:
    """Render Postgres config from Jinja template"""
    template = Template(open("postgresql.conf.j2").read())

    context = {
        'environment': environment,
        'shared_buffers': get_shared_buffers(environment),
        'max_connections': get_max_connections(environment),
        'work_mem': get_work_mem(environment)
    }

    return template.render(**context)

GitHub Actions + Mike Versioned Docs¶

GitHub Actions:

# .github/workflows/deploy-docs.yml
name: Deploy Docs
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: |
          pip install mkdocs-material mike

      - name: Deploy docs
        run: |
          mike deploy latest
          mike set-default latest

Air-Gapped Config Packaging¶

Packaging Script:

#!/bin/bash
# scripts/package_airgap_config.sh

PACKAGE_NAME="config-bundle-$(date +%Y%m%d).tar.gz"

# Package configs
tar -czf "$PACKAGE_NAME" \
    config/ \
    schemas/ \
    secrets-encrypted/ \
    checksums.txt

# Sign package
gpg --sign "$PACKAGE_NAME"

# Create manifest
cat > manifest.json <<EOF
{
  "package_name": "$PACKAGE_NAME",
  "created_at": "$(date -Iseconds)",
  "checksum": "$(sha256sum $PACKAGE_NAME | cut -d' ' -f1)",
  "environments": ["dev", "staging", "prod"]
}
EOF

echo "Package created: $PACKAGE_NAME"

Agentic LLM Integration Hooks¶

Generate Safe Environment Configurations¶

LLM Config Generator:

# llm/config_generator.py
class LLMConfigGenerator:
    def generate_config(self, requirements: dict) -> dict:
        """Generate safe environment configuration using LLM"""
        prompt = f"""
        Generate safe environment configuration for:

        {json.dumps(requirements, indent=2)}

        Requirements:
        1. Follow security best practices
        2. Use appropriate resource limits
        3. Include proper validation
        4. Document all settings
        """

        response = self.llm_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {"role": "system", "content": "You are a configuration generation expert."},
                {"role": "user", "content": prompt}
            ]
        )

        return json.loads(response.choices[0].message.content)

Validate Configs for Inconsistencies¶

LLM Config Validator:

# llm/config_validator.py
class LLMConfigValidator:
    def validate_config(self, config: dict) -> List[dict]:
        """Validate config for inconsistencies using LLM"""
        prompt = f"""
        Validate this configuration for inconsistencies:

        {json.dumps(config, indent=2)}

        Check for:
        1. Inconsistent values
        2. Security issues
        3. Performance problems
        4. Missing required fields
        """

        response = self.llm_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {"role": "system", "content": "You are a configuration validation expert."},
                {"role": "user", "content": prompt}
            ]
        )

        return json.loads(response.choices[0].message.content)

Surface Hidden Drift¶

LLM Drift Detector:

# llm/drift_detector.py
class LLMDriftDetector:
    def detect_hidden_drift(self, configs: dict) -> List[dict]:
        """Detect hidden configuration drift using LLM"""
        prompt = f"""
        Analyze these configurations for hidden drift:

        {json.dumps(configs, indent=2)}

        Identify:
        1. Subtle differences
        2. Implicit dependencies
        3. Configuration patterns
        4. Potential drift sources
        """

        response = self.llm_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {"role": "system", "content": "You are a drift detection expert."},
                {"role": "user", "content": prompt}
            ]
        )

        return json.loads(response.choices[0].message.content)

Propose Environment Diffs¶

LLM Diff Proposer:

# llm/diff_proposer.py
class LLMDiffProposer:
    def propose_diff(self, env1: str, env2: str) -> dict:
        """Propose environment diff using LLM"""
        prompt = f"""
        Propose configuration diff between {env1} and {env2}:

        Environment 1 ({env1}):
        {json.dumps(load_config(env1), indent=2)}

        Environment 2 ({env2}):
        {json.dumps(load_config(env2), indent=2)}

        Provide:
        1. Diff summary
        2. Rationale for differences
        3. Recommendations
        """

        response = self.llm_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {"role": "system", "content": "You are a configuration diff expert."},
                {"role": "user", "content": prompt}
            ]
        )

        return json.loads(response.choices[0].message.content)

Generate Config Promotion Plans¶

LLM Promotion Planner:

# llm/promotion_planner.py
class LLMPromotionPlanner:
    def generate_promotion_plan(self, source_env: str, target_env: str) -> dict:
        """Generate config promotion plan using LLM"""
        prompt = f"""
        Generate promotion plan from {source_env} to {target_env}:

        Source environment:
        {json.dumps(load_config(source_env), indent=2)}

        Target environment:
        {json.dumps(load_config(target_env), indent=2)}

        Provide:
        1. Promotion steps
        2. Validation checks
        3. Rollback plan
        4. Risk assessment
        """

        response = self.llm_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {"role": "system", "content": "You are a configuration promotion expert."},
                {"role": "user", "content": prompt}
            ]
        )

        return json.loads(response.choices[0].message.content)

Detect Misunderstood Settings¶

LLM Setting Analyzer:

# llm/setting_analyzer.py
class LLMSettingAnalyzer:
    def analyze_settings(self, config: dict, system_info: dict) -> List[dict]:
        """Detect misunderstood settings using LLM"""
        prompt = f"""
        Analyze these settings for potential issues:

        Configuration:
        {json.dumps(config, indent=2)}

        System Info:
        {json.dumps(system_info, indent=2)}

        Identify:
        1. Settings that don't match system capacity
        2. Incompatible settings
        3. Performance issues
        4. Security concerns
        """

        response = self.llm_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {"role": "system", "content": "You are a configuration analysis expert."},
                {"role": "user", "content": prompt}
            ]
        )

        return json.loads(response.choices[0].message.content)

Produce Diagrams of Config Layering¶

LLM Diagram Generator:

# llm/diagram_generator.py
class LLMDiagramGenerator:
    def generate_layering_diagram(self, config_structure: dict) -> str:
        """Generate Mermaid diagram of config layering"""
        prompt = f"""
        Generate a Mermaid diagram showing configuration layering:

        {json.dumps(config_structure, indent=2)}

        Show:
        1. Base layer
        2. Environment layer
        3. Secrets layer
        4. Runtime layer
        """

        response = self.llm_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {"role": "system", "content": "You are a diagram generation expert."},
                {"role": "user", "content": prompt}
            ]
        )

        return response.choices[0].message.content

Audit Kubernetes Manifests for Environment Divergence¶

LLM K8s Auditor:

# llm/k8s_auditor.py
class LLMKubernetesAuditor:
    def audit_manifests(self, manifests: List[dict]) -> dict:
        """Audit Kubernetes manifests for environment divergence"""
        prompt = f"""
        Audit these Kubernetes manifests for environment divergence:

        {json.dumps(manifests, indent=2)}

        Identify:
        1. Environment-specific differences
        2. Inconsistencies
        3. Security issues
        4. Best practice violations
        """

        response = self.llm_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {"role": "system", "content": "You are a Kubernetes audit expert."},
                {"role": "user", "content": prompt}
            ]
        )

        return json.loads(response.choices[0].message.content)

Auto-Patch and Auto-Refactor Configuration Structure¶

LLM Config Refactorer:

# llm/config_refactorer.py
class LLMConfigRefactorer:
    def refactor_config(self, config: dict, issues: List[str]) -> dict:
        """Auto-refactor configuration structure using LLM"""
        prompt = f"""
        Refactor this configuration to address issues:

        Configuration:
        {json.dumps(config, indent=2)}

        Issues:
        {json.dumps(issues, indent=2)}

        Provide refactored configuration that:
        1. Addresses all issues
        2. Maintains backward compatibility
        3. Follows best practices
        """

        response = self.llm_client.chat.completions.create(
            model="gpt-4",
            messages=[
                {"role": "system", "content": "You are a configuration refactoring expert."},
                {"role": "user", "content": prompt}
            ]
        )

        return json.loads(response.choices[0].message.content)

Configuration Scoring Rubric¶

Scoring Criteria¶

Rubric:

# config-scoring-rubric.yaml
scoring_criteria:
  versioning:
    weight: 0.15
    criteria:
      - "All configs versioned": 10
      - "Most configs versioned": 7
      - "Some configs versioned": 4
      - "No versioning": 0

  validation:
    weight: 0.20
    criteria:
      - "Comprehensive validation": 10
      - "Basic validation": 7
      - "Minimal validation": 4
      - "No validation": 0

  documentation:
    weight: 0.15
    criteria:
      - "Complete documentation": 10
      - "Good documentation": 7
      - "Basic documentation": 4
      - "No documentation": 0

  drift_detection:
    weight: 0.20
    criteria:
      - "Automated drift detection": 10
      - "Manual drift detection": 7
      - "Periodic checks": 4
      - "No drift detection": 0

  secrets_management:
    weight: 0.15
    criteria:
      - "Centralized secrets management": 10
      - "Standardized secrets": 7
      - "Some secrets management": 4
      - "No secrets management": 0

  promotion_process:
    weight: 0.15
    criteria:
      - "Automated promotion": 10
      - "Standardized promotion": 7
      - "Manual promotion": 4
      - "No promotion process": 0

Scoring Calculation¶

Calculator:

# scoring/calculator.py
class ConfigScorer:
    def calculate_score(self, config: dict) -> float:
        """Calculate configuration score"""
        rubric = load_scoring_rubric()

        scores = {}
        for criterion, details in rubric['scoring_criteria'].items():
            score = self.evaluate_criterion(config, criterion, details)
            scores[criterion] = score * details['weight']

        total_score = sum(scores.values())
        return total_score

Template for Environment Configuration Governance¶

Governance Template¶

Template:

# governance-template.yaml
configuration_governance:
  policy:
    name: "Environment Configuration Governance Policy"
    version: "1.0.0"
    effective_date: "2024-01-01"

  principles:
    - "Configuration as a Product"
    - "Declarative Configuration First"
    - "Immutable Infrastructure"
    - "Fail-Fast Validation"
    - "Explicit Environment Boundaries"
    - "Thin Environment Differences"
    - "Version-Locked Everything"

  standards:
    versioning:
      required: true
      format: "semver"

    validation:
      required: true
      tools: ["pydantic", "jsonschema"]

    documentation:
      required: true
      format: "markdown"

    drift_detection:
      required: true
      frequency: "daily"

  processes:
    promotion:
      - "dev → test: Automated"
      - "test → staging: Automated + Approval"
      - "staging → prod: Automated + Approval + Review"

    review:
      frequency: "quarterly"
      participants: ["architecture-team", "platform-team"]

  metrics:
    - "Config drift score"
    - "Promotion success rate"
    - "Validation failure rate"
    - "Documentation coverage"