A production Python project juggles several kinds of configuration: database connections, API keys, model hyperparameters, and different environments (development, testing, production). This is how I think about the trade-offs.

1. Environment variables (the baseline)

Read with os.environ. Simple, but weakly typed and awkward to manage.

import os
                        DB_URL = os.environ.get("DATABASE_URL", "sqlite:///dev.db")

2. .env files + python-dotenv

Put environment variables in .env so switching is easy during development. Always add .env to .gitignore. 

# .env
                        DATABASE_URL=postgres://...
                        SECRET_KEY=xxx

3. YAML / JSON / TOML config files

Good for structured configuration, such as machine-learning hyperparameters.

# config.yaml
                        model:
                          name: resnet50
                          lr: 0.001
                          batch_size: 64
                        data:
                          train: data/train
                          val: data/val

4. Pydantic Settings (recommended)

Type safety, automatic validation, and merging across sources (env > file > default). My default choice for new projects. 

from pydantic_settings import BaseSettings

                        class Settings(BaseSettings):
                            db_url: str
                            secret_key: str
                            debug: bool = False

                            class Config:
                                env_file = ".env"

                        settings = Settings()

5. Secrets

Never put secrets in code, config files, or git. Team projects should use Vault / AWS Secrets Manager / GCP Secret Manager; personal projects should at least use .env plus .gitignore.

If a secret was ever committed, rotate it immediately and clean up the git history.

What I do in practice

  1. Small scripts: os.environ plus .env.
  2. Medium projects: Pydantic Settings plus .env.
  3. Training runs: Hydra or OmegaConf for composing configs.
  4. Deployment: 12-factor, all configuration via environment variables, secrets handled by the cloud provider.