defensive-error-handling
verified3a74d39c-2a1d-4bf8-ba93-4b869bbbd826
Handle errors explicitly — typed/structured errors, no bare except, fail loud with context, and convert exceptions at boundaries. Use when writing or reviewing code that touches I/O, networks, or user input.
Metadata
Skill file
# Defensive Error Handling
Use when writing or reviewing code that touches I/O, networks, databases, or user
input — the places exceptions actually happen. The goal: handle errors explicitly
and loudly, with context, and convert exceptions at the boundary between layers.
## 1. The decision table: what to do with an exception
For every `except` clause, pick exactly one strategy. If you cannot name which one
you are using, the handler is wrong.
| Strategy | Use when | Example |
|---|---|---|
| **Retry** | Transient failure, idempotent, bounded | network timeout, rate-limit |
| **Propagate** | Caller can decide better, or it is a real bug | programming error, logic error |
| **Convert** | Crossing a layer boundary; hide library details | catch `requests.ConnectionError` → raise `PaymentGatewayError` |
| **Fail** | Unrecoverable, no safe fallback | config missing, DB down at startup |
```python
# RETRY — transient, idempotent, bounded
for attempt in range(3):
try:
return http_client.get(url, timeout=5)
except requests.Timeout:
if attempt == 2:
raise
time.sleep(2 ** attempt)
# CONVERT — at the boundary, keep domain language
def charge_card(amount: int) -> ChargeResult:
try:
return gateway.charge(amount)
except (gateway.NetworkError, gateway.ProviderError) as exc:
raise PaymentFailedError(amount=amount) from exc
```
## 2. Anti-patterns — never do these
| Anti-pattern | Why it is wrong | Correct version |
|---|---|---|
| `except:` (bare) | Catches `KeyboardInterrupt`/`SystemExit`, hides everything | `except ValueError:` (specific) |
| `except Exception: pass` | Swallows the error, code continues in a broken state | `except Exception as e: log + re-raise or handle` |
| Catch without re-raise or handling | Half a fix; the error vanishes | either handle or `raise` |
| Stringly errors `if err == "timeout"` | Fragile, no type safety | typed exception classes |
| `raise "some string"` | Invalid in Py3; loses stack/type | `raise ValueError("...")` |
## 3. The boundary pattern
Catch library/network exceptions at the **edge** of your code, convert them to
**domain errors**, and log the **context** — what was attempted, the input, and
the retry count.
```python
def fetch_invoice(invoice_id: str) -> Invoice:
try:
resp = http_client.get(f"/invoices/{invoice_id}", timeout=5)
resp.raise_for_status()
return Invoice.from_dict(resp.json())
except requests.HTTPError as exc:
if exc.response.status_code == 404:
raise InvoiceNotFoundError(invoice_id) from exc
logger.error("invoice fetch failed",
extra={"invoice_id": invoice_id,
"status": exc.response.status_code,
"retries": 0})
raise InvoiceServiceError(invoice_id) from exc
```
Key rules:
- **Convert** at the boundary so callers never import `requests` just to catch an error.
- **Log context** (`extra={...}` with structured fields), not just "an error happened".
- **Always chain** with `from exc` to preserve the original traceback.
## 4. When to fail fast vs recover
| Situation | Action |
|---|---|
| Config missing at startup | **Fail** — crash with a clear message; do not limp |
| One of N requests fails | **Recover** — log and continue, or retry |
| Data corruption / invariant broken | **Fail** — this is a bug; surface it, don't mask it |
| User gave bad input | **Return a validation error**, do not raise |
Failing fast on unrecoverable states is a feature: a service that crashes with a
clear message is more debuggable than one that silently returns wrong answers.
## 5. Worked example: wrapping a flaky HTTP call
```python
import logging, time, requests
logger = logging.getLogger(__name__)
class InvoiceServiceError(Exception): ...
class InvoiceNotFoundError(InvoiceServiceError): ...
def fetch_invoice(invoice_id: str, *, retries: int = 3) -> dict:
for attempt in range(retries):
try:
resp = requests.get(f"{BASE}/invoices/{invoice_id}", timeout=5)
resp.raise_for_status()
return resp.json()
except requests.HTTPError as exc:
if exc.response.status_code == 404:
raise InvoiceNotFoundError(invoice_id) from exc # CONVERT (terminal)
logger.error("invoice fetch HTTP error",
extra={"invoice_id": invoice_id,
"status": exc.response.status_code,
"attempt": attempt})
raise InvoiceServiceError(invoice_id) from exc # CONVERT + propagate
except requests.Timeout as exc:
if attempt == retries - 1:
raise InvoiceServiceError(invoice_id) from exc # CONVERT (exhausted)
logger.warning("invoice fetch timeout, retrying",
extra={"invoice_id": invoice_id, "attempt": attempt})
time.sleep(2 ** attempt) # RETRY (bounded backoff)
raise InvoiceServiceError(invoice_id) # unreachable, but explicit
```
Notice: specific exception types, `from exc` chaining, structured `extra=` context,
and a bounded retry only on the *transient* failure (timeout), never the terminal
one (404).
## 6. Custom exception hierarchy
For anything beyond a script, define a small hierarchy so callers can catch at the
right level:
```python
class AppError(Exception):
"""Base for all domain errors."""
class NotFoundError(AppError): ...
class InvalidInputError(AppError): ...
class ExternalServiceError(AppError): ...
class InvoiceNotFoundError(NotFoundError): ...
class PaymentFailedError(ExternalServiceError): ...
```
Callers catch `NotFoundError` to handle any missing resource, or the specific
subclass when they care. This beats both `except Exception` (too broad) and a
sprawl of unrelated exception types.
## Guardrails
- Do **not** use bare `except:` or `except Exception: pass`. Ever.
- Do **not** catch `KeyboardInterrupt` or `SystemExit` — these are control flow,
not errors.
- Do **not** lose the original traceback — chain with `from exc`.
- Do **not** catch broadly and "handle" by logging alone; either recover or re-raise.
- Do **not** swallow errors at a boundary — convert them to domain errors.
- Prefer typed, specific exception classes over strings and over a single generic
`Exception`.
## Pitfalls
- **Over-catching** — wrapping a whole function in `try/except Exception` hides real
bugs behind a friendly log line.
- **Catching `KeyboardInterrupt`/`SystemExit`** — breaks Ctrl-C and `sys.exit()`,
making the program hard to stop.
- **Losing the traceback** — `raise NewError("...")` without `from exc` discards the
original cause, making the log useless.
- **Logging without context** — "error: failed" tells you nothing; include input,
id, status, retry count.
- **Stringly errors** — comparing error strings is fragile across versions and
libraries; use exception types.
## Verify / Checklist
- [ ] Every `except` clause uses a specific exception type (no bare `except:`).
- [ ] No `except Exception: pass` or silent swallowing anywhere.
- [ ] Boundary conversions use `raise DomainError(...) from exc` to preserve cause.
- [ ] Error logs include structured context (id, input, status, retries), not just a message.
- [ ] Retry logic is bounded (max attempts + backoff) and only for idempotent/transient ops.
- [ ] `KeyboardInterrupt`/`SystemExit` are not caught.
- [ ] Each `except` handler maps to one strategy from the decision table.Attached files
No attached files.