Survey-weighted Difference-in-Differences: combining balance and diff-diff on a BRFSS-style smoking-ban policy

This tutorial shows an end-to-end survey-weighted causal-inference workflow in pure Python, using two complementary open-source tools, bridged by a thin adapter:

1. balance to make a complex survey sample representative of a known population frame (ACS demographics) via poststratification / raking — and to carry a design-consistent weight column with valid effective-sample-size and standard-error accounting.
2. diff_diff to estimate a modern staggered Callaway–Sant'Anna Difference-in-Differences with built-in survey-design variance and HonestDiD sensitivity.
3. The thin adapter balance.interop.diff_diff (added in balance 0.21) that hands a balance.Sample straight to a diff-diff estimator — building the SurveyDesign, aggregating respondents to a geo-period panel, and stripping balance's book-keeping columns for you.

The running question. Did state indoor-smoking bans (adopted 2020 and 2022) reduce adult asthma prevalence relative to states without bans, 2018–2024? The microdata mirrors the public-use BRFSS file (CDC BRFSS), but we generate it synthetically so the notebook is self-contained and deterministic. The one cell that builds the synthetic frame is exactly the cell you would replace with a real pyreadstat.read_xport(...) call.

Two lessons, one pipeline. BRFSS response rates have declined for decades, and the decline is concentrated among younger adults — so the realized sample ages over time even though the design weights are nominally correct. We will see that this differential non-response:

• biases a descriptive estimand (the national prevalence trend) badly — and balance fixes it; and
• largely cancels in the DiD — so the causal ATT is reassuringly robust to composition, while balance still supplies the correct design-based standard errors. That robustness is a feature, and the survey weights are what let you verify it.

Pre-requisites. pandas, basic causal-inference vocabulary (treatment, control, parallel trends), and ideally the balance quickstart. No R or Stata assumed.

Setup

You need a build of balance that ships balance.interop.diff_diff (v0.21+) and diff-diff (>= 3.3). The simplest install is:

pip install "balance[did]"

which pulls diff-diff>=3.3.0,<4 as an optional extra. If you already have balance, pip install diff-diff is sufficient.

import logging
import warnings

import matplotlib.pyplot as plt
import numpy as np
import pandas as pd

# balance - survey reweighting against a population frame
import balance
from balance import Sample

# balance.interop.diff_diff - the thin adapter (added in balance 0.21)
from balance.interop import diff_diff as bd
from balance.interop.diff_diff import as_balance_diagnostic, fit_did

# diff-diff - survey-aware DiD estimators + sensitivity
import diff_diff as dd
from diff_diff import compute_honest_did

warnings.filterwarnings("ignore", category=FutureWarning, module="balance")
# Keep the tutorial output focused on results, not balance's INFO/WARNING logs.
logging.getLogger("balance").setLevel(logging.ERROR)

print(f"balance:   {balance.__version__}")
print(f"diff-diff: {dd.__version__}")

INFO (2026-08-06 09:40:21,383) [__init__/<module> (line 77)]: Using balance version 0.23.0

INFO (2026-08-06 09:40:21,384) [__init__/<module> (line 82)]: 
balance (Version 0.23.0) loaded:
    📖 Documentation: https://import-balance.org/
    🛠️ Help / Issues: https://github.com/facebookresearch/balance/issues/
    📄 Citation:
        Sarig, T., Galili, T., & Eilat, R. (2023).
        balance - a Python package for balancing biased data samples.
        https://arxiv.org/abs/2307.06024

    Tip: You can view this message anytime with balance.help()

balance:   0.23.0
diff-diff: 3.8.0

The dataset

We build a respondent-level synthetic frame that mirrors BRFSS 2018–2024: 50 states × 7 years × ~150 adult respondents per state-year (~54,000 rows). Each respondent has:

• demographics age_band, educ_cat, sex, race (the variables we poststratify on);
• a binary outcome asthnow ∈ {0, 1} (currently has asthma), drawn from a logistic model with a baseline prevalence near 9%;
• a complex survey design: stratum (region type), psu (county cluster), fpc, and a design_weight standing in for BRFSS _LLCPWT;
• g, the year a state's smoking ban took effect (0 = never-treated).

Two things are baked into the data generator. First, the smoking ban lowers asthma prevalence by about 3 percentage points in treated states after adoption (parallel trends hold in the population). Second — the realistic wrinkle — non-response grows over time and is concentrated among younger adults, so the realized sample ages even though the design weights only correct for the sampling design. That gap is exactly what balance is for.

The generator is deterministic given its seed, so the notebook re-runs identically. Replace this one cell with pyreadstat.read_xport(...) calls to run on real BRFSS XPT files.

# --- Synthetic BRFSS-style respondent microdata --------------------------
# Swap this entire cell for `pyreadstat.read_xport(...)` on real BRFSS files.
SEED = 20260430
N_STATES = 50
YEARS = list(range(2018, 2025))            # 2018..2024
COHORTS = {2020: 10, 2022: 10}             # staggered adoption; 30 never-treated
N_POP_PER_CELL = 230                       # pre-non-response draws per state-year
BASE_PREV = 0.09                           # baseline asthma prevalence
SIGMA_STATE = 0.10                         # between-state variation (log-odds)
B_AGE, B_EDUC = 0.60, -0.30               # older / less-educated -> more asthma
TAU_LOGIT = -0.35                          # smoking-ban effect (log-odds), ~ -3 pp
RESP_BASE = 0.70                           # baseline response rate
RESP_AGE_DRIFT = 0.30                      # /yr: young adults increasingly drop out
N_STRATA = 5
AGE_BINS = [-99, -1.0, -0.3, 0.3, 1.0, 99]
AGE_LBL = ["18-29", "30-44", "45-59", "60-74", "75+"]
EDUC_BINS = [-99, -0.5, 0.5, 99]
EDUC_LBL = ["hs_or_less", "some_college", "college_plus"]
RAKE_VARS = ["age_band", "educ_cat", "sex", "race"]


def _expit(x):
    return 1.0 / (1.0 + np.exp(-x))


def make_brfss_microdata(seed=SEED):
    """Return (micro, acs, true_att_pp, true_prev): respondent microdata, an
    ACS-style population target, the planted ATT (pp), and the true national
    prevalence by year (pre-non-response)."""
    rng = np.random.default_rng(seed)
    b0 = np.log(BASE_PREV / (1 - BASE_PREV))
    g = np.zeros(N_STATES, dtype=int)
    order = rng.permutation(N_STATES)
    i = 0
    for yr, k in COHORTS.items():
        g[order[i:i + k]] = yr
        i += k
    alpha = rng.normal(0, SIGMA_STATE, N_STATES)          # state fixed effects
    stratum = rng.integers(0, N_STRATA, N_STATES)
    stratum_w = np.linspace(0.7, 1.4, N_STRATA)

    rows, num, den = [], {}, {}
    for s in range(N_STATES):
        for yr in YEARS:
            n = N_POP_PER_CELL
            age = rng.normal(0.0, 1.0, n)
            educ = rng.normal(0.0, 1.0, n)
            sex = rng.choice(["male", "female"], n)
            race = rng.choice(["white", "black", "hispanic", "asian", "other"],
                              p=[0.60, 0.15, 0.18, 0.05, 0.02], size=n)
            post = int(g[s] > 0 and yr >= g[s])
            eta = b0 + alpha[s] + B_AGE * age + B_EDUC * educ + TAU_LOGIT * post
            p1 = _expit(eta)
            y = rng.binomial(1, p1)
            num[yr] = num.get(yr, 0.0) + p1.sum()         # true population prevalence
            den[yr] = den.get(yr, 0) + n
            # differential non-response: age-response gradient grows over time
            grad = RESP_AGE_DRIFT * (yr - 2018)
            responded = rng.binomial(
                1, _expit(np.log(RESP_BASE / (1 - RESP_BASE)) + grad * age)
            ).astype(bool)
            p0 = _expit(eta - TAU_LOGIT * post)
            sub = pd.DataFrame({
                "state": s, "year": yr, "asthnow": y, "post": post,
                "age": age, "educ": educ, "sex": sex, "race": race,
                "stratum": stratum[s], "psu": s * 100 + rng.integers(0, 8, n),
                "fpc": 200.0,
                "design_weight": stratum_w[stratum[s]] * rng.uniform(0.8, 1.2, n),
                "g": g[s], "_p1": p1, "_p0": p0,
            })
            rows.append(sub[responded])
    micro = pd.concat(rows, ignore_index=True)
    micro["age_band"] = pd.cut(micro["age"], AGE_BINS, labels=AGE_LBL).astype(str)
    micro["educ_cat"] = pd.cut(micro["educ"], EDUC_BINS, labels=EDUC_LBL).astype(str)
    micro["id"] = np.arange(len(micro))
    tp = micro[micro["post"] == 1]
    true_att_pp = float((tp["_p1"] - tp["_p0"]).mean()) * 100
    true_prev = pd.Series({y: num[y] / den[y] * 100 for y in YEARS})
    micro = micro.drop(columns=["_p1", "_p0"])  # internal truth cols, not real data

    # ACS-style population target: stable demographic marginals.
    nt = 40000
    acs = pd.DataFrame({
        "age": rng.normal(0, 1, nt), "educ": rng.normal(0, 1, nt),
        "sex": rng.choice(["male", "female"], nt),
        "race": rng.choice(["white", "black", "hispanic", "asian", "other"],
                           p=[0.60, 0.15, 0.18, 0.05, 0.02], size=nt),
        "weight": 1.0,
    })
    acs["age_band"] = pd.cut(acs["age"], AGE_BINS, labels=AGE_LBL).astype(str)
    acs["educ_cat"] = pd.cut(acs["educ"], EDUC_BINS, labels=EDUC_LBL).astype(str)
    acs["id"] = np.arange(nt)
    return micro, acs, true_att_pp, true_prev

micro, acs, true_att_pp, true_prev = make_brfss_microdata()

print("Microdata shape:", micro.shape)
print("Outcome values:", sorted(micro["asthnow"].unique()),
      "| overall prevalence: %.1f%%" % (100 * micro["asthnow"].mean()))
print("Planted true ATT: %+.2f pp" % true_att_pp)
micro.head()

Microdata shape: (54072, 16)
Outcome values: [np.int64(0), np.int64(1)] | overall prevalence: 11.1%
Planted true ATT: -3.15 pp

	state	year	asthnow	post	age	educ	sex	race	stratum	psu	fpc	design_weight	g	age_band	educ_cat	id
0	0	2018	0	0	-0.544475	-0.071436	male	white	4	1	200.0	1.292769	0	30-44	some_college	0
1	0	2018	0	0	1.356305	0.042664	female	black	4	4	200.0	1.656150	0	75+	some_college	1
2	0	2018	1	0	1.543988	-0.998996	male	asian	4	3	200.0	1.173507	0	75+	hs_or_less	2
3	0	2018	0	0	0.051504	1.774550	male	white	4	5	200.0	1.309412	0	45-59	college_plus	3
4	0	2018	0	0	-0.728620	-1.443948	female	white	4	1	200.0	1.181719	0	30-44	hs_or_less	4

Step 1 — Inspect the panel

Three things matter before any weighting or DiD: the staggered-adoption structure (g), the cohort sizes, and the raw outcome trends. Note in the plot below that the design-weighted prevalence appears to rise over time in both arms — a red flag we will explain (and fix) in Step 2.

print("First-treat-year (g) across states (0 = never-treated):")
print(micro.drop_duplicates("state")["g"].value_counts().sort_index())

fig, ax = plt.subplots(figsize=(7, 4))
labelled = micro.assign(
    cohort=np.where(micro["g"] > 0, "treated (ban)", "control"))
for lab, sub in labelled.groupby("cohort"):
    series = sub.groupby("year").apply(
        lambda d: np.average(d["asthnow"], weights=d["design_weight"]) * 100)
    series.plot(ax=ax, marker="o", label=lab)
ax.set(ylabel="asthma prevalence (%)", xlabel="year",
       title="Design-weighted asthma prevalence by cohort (raw)")
ax.legend(title="cohort")
plt.tight_layout()
plt.show()

First-treat-year (g) across states (0 = never-treated):
g
0       30
2020    10
2022    10
Name: count, dtype: int64

Step 2 — Make the survey representative (the descriptive estimand)

BRFSS design weights correct for the sampling design, not for non-response. Because younger adults increasingly drop out, the realized sample ages year over year, and — since older adults have more asthma — the design-weighted national prevalence drifts upward even when the true prevalence is flat or falling.

The standard fix is poststratification / raking: reweight each survey wave so its demographic margins match the population (here, ACS). balance does this with adjust(method="rake", ...). We rake each year separately — the correct practice for a repeated cross-section — and compare the result against the true population prevalence.

def poststratify_by_wave(micro, acs):
    """Rake each annual wave to the ACS demographic margins. Returns the
    per-respondent weights and a dict {year: adjusted Sample} (one balance
    Sample per wave), used for the per-wave diagnostics in Step 6."""
    target = Sample.from_frame(acs, id_column="id")
    w = pd.Series(index=micro.index, dtype=float)
    adjs = {}
    for yr, idx in micro.groupby("year").groups.items():
        wave = micro.loc[idx]
        adj = (
            Sample.from_frame(wave, id_column="id", weight_column="design_weight",
                              outcome_columns=["asthnow"])
            .set_target(target)
            .adjust(method="rake", variables=RAKE_VARS)
        )
        # balance casts the id column to str in from_frame(); match on the str id.
        # The assert turns any future id-alignment change into a loud failure
        # instead of silent NaN weights.
        wt = (adj.df.set_index("id")[adj.weight_column]
              .reindex(wave["id"].astype(str).values).to_numpy())
        assert not np.isnan(wt).any(), "weight lookup produced NaNs (id alignment changed?)"
        w.loc[idx] = wt
        adjs[yr] = adj
    return w, adjs


w_py, adjs = poststratify_by_wave(micro, acs)
micro["w"] = w_py.values
# Latest wave (2024) is the most non-response-distorted here (drift grows over time).
adj_last = adjs[max(YEARS)]

trend = pd.DataFrame({
    "true_population": true_prev,
    "survey_design_weighted": micro.groupby("year").apply(
        lambda d: np.average(d["asthnow"], weights=d["design_weight"]) * 100),
    "survey_poststratified": micro.groupby("year").apply(
        lambda d: np.average(d["asthnow"], weights=d["w"]) * 100),
}).round(2)
print(trend)

ax = trend.plot(marker="o", figsize=(7, 4))
ax.set(ylabel="asthma prevalence (%)", xlabel="year",
       title="Design weights drift upward; poststratification tracks the truth")
plt.tight_layout()
plt.show()

      true_population  survey_design_weighted  survey_poststratified
2018            10.51                   10.81                  10.78
2019            10.55                   11.28                  10.77
2020             9.95                   10.61                   9.74
2021            10.02                   11.61                  10.32
2022             9.36                   10.47                   9.26
2023             9.41                   11.49                   9.76
2024             9.33                   12.23                  10.03

The design-weighted series climbs while the true population prevalence is flat-to-declining (the bans are working) — the naive survey gets even the direction wrong. Poststratification recovers the truth.

balance also reports the diagnostics survey methodologists expect: covariate balance before/after (ASMD), the design effect, and the effective sample size. The Love plot is the canonical before/after balance visual.

print(adj_last.summary())
adj_last.covars().love_plot()

Adjustment details:
    method: rake
Covariate diagnostics:
    Covar ASMD reduction: 90.9%
    Covar ASMD (21 variables): 0.049 -> 0.004
    Covar mean KLD reduction: 0.3%
    Covar mean KLD (13 variables): 11.046 -> 11.010
Weight diagnostics:
    design effect (Deff): 1.541
    effective sample size proportion (ESSP): 0.649
    effective sample size (ESS): 4762.3
Outcome weighted means:
            asthnow
source             
self          0.100
unadjusted    0.122

Step 3 — Survey-weighted Callaway–Sant'Anna via the adapter

Modern staggered-DiD estimators operate on a unit × period panel, not raw microdata. balance.interop.diff_diff bridges the two:

• bd.to_panel_for_did(sample, ...) builds the respondent-level SurveyDesign from the active balance weight, drops balance's book-keeping columns, and calls diff_diff.aggregate_survey to collapse respondents into a state-year panel of weighted prevalence (with design-based cell variances) — returning a (panel_df, second_stage_design) pair.
• fit_did(...) then builds the second-stage SurveyDesign and fits the chosen estimator, routing keyword arguments to __init__/fit for you.

We use Callaway & Sant'Anna (2021) with estimation_method="reg" (an outcome-regression DiD): balance has already handled covariate balance via poststratification, so the estimator only does the identification. We cluster by state and use a universal base period. We wrap the call so we can reuse it for the robustness check in Step 5.

def run_did(weight_col):
    """Aggregate a weighted respondent sample to a state-year panel and fit
    Callaway-Sant'Anna, all through the balance<->diff-diff adapter."""
    sample = Sample.from_frame(micro, id_column="id", weight_column=weight_col,
                               outcome_columns=["asthnow"])
    panel_df, second_stage = bd.to_panel_for_did(
        sample, by=["state", "year"], outcomes="asthnow",
        second_stage_weights="pweight")
    # carry the cohort (g) onto the panel (one row per state-year)
    gmap = micro.drop_duplicates("state")[["state", "g"]]
    panel_df["state"] = panel_df["state"].astype(int)
    panel_df = panel_df.merge(gmap, on="state", how="left")
    panel_df["g"] = panel_df["g"].fillna(0).astype(int)
    panel_df["id"] = np.arange(len(panel_df))
    panel_sample = Sample.from_frame(
        panel_df, weight_column=second_stage.weights,
        outcome_columns=["asthnow_mean"])
    res = fit_did(
        panel_sample, estimator="CallawaySantAnna", outcome="asthnow_mean",
        time="year", unit="state", treatment_first="g",
        estimation_method="reg", control_group="not_yet_treated",
        base_period="universal", cluster="state", aggregate="all")
    return res, panel_df


res, panel_df = run_did("w")
print("Panel shape:", panel_df.shape)
print(res.summary())

ax = dd.plot_event_study(res)
ax.set_title("Callaway-Sant'Anna event study (survey-weighted)")
plt.tight_layout()
plt.show()

/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/diff_diff/prep.py:1664: UserWarning: pweight weights normalized to mean=1 (sum=54072). Original sum was 2.8e+05.
  full_resolved = effective_design.resolve(data)
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/diff_diff/survey.py:1263: UserWarning: pweight weights normalized to mean=1 (sum=350). Original sum was 5.407e+04.
  resolved = survey_design.resolve(data)

Panel shape: (350, 13)
=====================================================================================
            Callaway-Sant'Anna Staggered Difference-in-Differences Results           
=====================================================================================

Total observations:                   350
Treated units:                         20
Never-treated units:                   30
Treatment cohorts:                      2
Time periods:                           7
Control group:                 not_yet_treated
Base period:                    universal


-------------------------------------------------------------------------------------
                                    Survey Design                                    
-------------------------------------------------------------------------------------
Weight type:                      pweight
PSU/Cluster:                           50
Effective sample size:               47.9
Kish DEFF (weights):                 1.04
Survey d.f.:                           49
-------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------
                   Overall Average Treatment Effect on the Treated                   
-------------------------------------------------------------------------------------
Parameter           Estimate    Std. Err.     t-stat      P>|t|   Sig.
-------------------------------------------------------------------------------------
ATT                  -0.0300       0.0074     -4.079     0.0002    ***
-------------------------------------------------------------------------------------

95% Confidence Interval: [-0.0448, -0.0152]
CV (SE/abs(ATT)):             0.2451

-------------------------------------------------------------------------------------
                            Event Study (Dynamic) Effects                            
-------------------------------------------------------------------------------------
Rel. Period         Estimate    Std. Err.     t-stat      P>|t|   Sig.
-------------------------------------------------------------------------------------
-4.0                 -0.0136       0.0140     -0.972     0.3360       
-3.0                 -0.0167       0.0132     -1.260     0.2137       
-2.0                 -0.0025       0.0090     -0.284     0.7778       
-1.0                  0.0000          nan        nan        nan       
0.0                  -0.0377       0.0089     -4.230     0.0001    ***
1.0                  -0.0242       0.0082     -2.932     0.0051     **
2.0                  -0.0407       0.0082     -4.940     0.0000    ***
3.0                  -0.0107       0.0132     -0.812     0.4208       
4.0                  -0.0252       0.0104     -2.428     0.0189      *
-------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------
                             Effects by Treatment Cohort                             
-------------------------------------------------------------------------------------
Cohort              Estimate    Std. Err.     t-stat      P>|t|   Sig.
-------------------------------------------------------------------------------------
2020.0               -0.0257       0.0090     -2.862     0.0062     **
2022.0               -0.0376       0.0109     -3.441     0.0012     **
-------------------------------------------------------------------------------------

Signif. codes: '***' 0.001, '**' 0.01, '*' 0.05, '.' 0.1
=====================================================================================

<Figure size 640x480 with 0 Axes>

The event study shows flat, non-significant pre-treatment coefficients (parallel trends hold) and a clear post-treatment drop — an overall ATT of about −3 percentage points that recovers the planted effect.

Step 4 — HonestDiD sensitivity to parallel-trends violations

Parallel trends is an assumption, not a fact. The Rambachan & Roth (2023) HonestDiD framework asks: how large could post-treatment violations be — relative to the largest pre-treatment violation (the relative-magnitude parameter M) — before the effect could be zero? diff_diff.compute_honest_did returns robust confidence intervals that widen with M. We sweep a few values to find the breakdown point rather than reporting a single pass/fail.

print("HonestDiD (Rambachan & Roth 2023) relative-magnitude sensitivity:")
for m in (0.25, 0.5, 1.0, 1.5):
    h = compute_honest_did(res, method="relative_magnitude", M=m)
    excludes_zero = not (h.ci_lb <= 0 <= h.ci_ub)
    flag = "  <- still excludes 0" if excludes_zero else ""
    print(f"  M={m:>4}:  robust 95% CI = "
          f"[{h.ci_lb * 100:+.2f}, {h.ci_ub * 100:+.2f}] pp{flag}")

HonestDiD (Rambachan & Roth 2023) relative-magnitude sensitivity:
  M=0.25:  robust 95% CI = [-5.31, -0.23] pp  <- still excludes 0
  M= 0.5:  robust 95% CI = [-6.37, +0.83] pp
  M= 1.0:  robust 95% CI = [-8.49, +2.95] pp
  M= 1.5:  robust 95% CI = [-10.60, +5.07] pp

With only ~50 clusters the effect is robust to modest pre-trend violations but not unlimited ones — exactly the kind of honest caveat HonestDiD is designed to surface.

Step 5 — Robustness: does the reweighting change the ATT?

A natural question is how much the survey weighting moves the causal estimate. We re-run the identical pipeline on the raw design weights (no poststratification) and compare. Because a DiD differences out time-invariant composition, the common non-response drift largely cancels: the ATT is robust. The weighting's payoff is elsewhere — an unbiased descriptive trend (Step 2) and design-consistent standard errors — and the agreement here is reassurance, not redundancy.

res_design, _ = run_did("design_weight")

print("Planted true ATT      : %+.2f pp" % true_att_pp)
print("Design-weighted ATT   : %+.2f pp" % (res_design.overall_att * 100))
print("Poststratified ATT    : %+.2f pp" % (res.overall_att * 100))

Planted true ATT      : -3.15 pp
Design-weighted ATT   : -3.44 pp
Poststratified ATT    : -3.00 pp

/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/diff_diff/prep.py:1664: UserWarning: pweight weights normalized to mean=1 (sum=54072). Original sum was 5.653e+04.
  full_resolved = effective_design.resolve(data)
/opt/hostedtoolcache/Python/3.12.13/x64/lib/python3.12/site-packages/diff_diff/survey.py:1263: UserWarning: pweight weights normalized to mean=1 (sum=350). Original sum was 5.407e+04.
  resolved = survey_design.resolve(data)

Step 6 — Combined and per-wave diagnostics

bd.as_balance_diagnostic(sample, did_results) joins balance's pre-fit diagnostics (ASMD, Kish ESS, design effect) for a given adjusted Sample with diff-diff's post-fit survey-design diagnostics for the fitted DiD, into one flat dict.

Because we poststratify each wave separately, there is no single pipeline-wide adjusted Sample — the balance-side metrics are inherently per wave. So we report two things: a per-wave covariate-balance table across all seven waves, and the combined as_balance_diagnostic dict for the most non-response-distorted wave (2024) paired with the full-panel DiD result. Read the combined dict carefully: its diff_diff_*/att entries describe the whole 2018–2024 panel, while its balance_* entries describe only the 2024 wave.

# Per-wave covariate-balance diagnostics (balance side), via the adapter helper.
BAL_KEYS = ["balance_asmd_mean_post", "balance_asmd_max_post",
            "balance_kish_ess", "balance_design_effect"]
per_wave = pd.DataFrame(
    {yr: {k: as_balance_diagnostic(a, res)[k] for k in BAL_KEYS}
     for yr, a in adjs.items()}
).T
per_wave.index.name = "year"
print("Per-wave balance diagnostics (all waves):")
print(per_wave.round(3))

# Combined dict: balance side = 2024 wave (most distorted); diff-diff side = full panel.
print("\nCombined diagnostic (balance=2024 wave, diff-diff=full 2018-2024 panel):")
diag = as_balance_diagnostic(adj_last, res)
pd.DataFrame([diag]).T.rename(columns={0: "value"})

Per-wave balance diagnostics (all waves):
      balance_asmd_mean_post  balance_asmd_max_post  balance_kish_ess  \
year                                                                    
2018                   0.000                  0.004          7589.334   
2019                   0.001                  0.009          7568.501   
2020                   0.001                  0.010          7125.488   
2021                   0.002                  0.025          6676.885   
2022                   0.003                  0.035          6262.412   
2023                   0.004                  0.037          5451.842   
2024                   0.004                  0.050          4762.253   

      balance_design_effect  
year                         
2018                  1.063  
2019                  1.068  
2020                  1.098  
2021                  1.157  
2022                  1.217  
2023                  1.358  
2024                  1.541  

Combined diagnostic (balance=2024 wave, diff-diff=full 2018-2024 panel):

	value
att	-0.030014
se	0.007357
conf_int	(-0.04479875661771691, -0.015228739590515752)
n_obs	350
diff_diff_design_effect	1.044851
diff_diff_effective_n	47.853733
diff_diff_sum_weights	50.0
balance_kish_ess	4762.253361
balance_design_effect	1.541497
balance_asmd_mean_post	0.004465
balance_asmd_max_post	0.049575

Discussion

What this notebook demonstrated:

• A respondent-level BRFSS-shaped survey with realistic differential non-response is made population-representative with balance poststratification — fixing a descriptive prevalence trend that the raw design weights get backwards.
• The reweighted sample is handed to diff-diff through a single bd.to_panel_for_did + fit_did pair: respondents are aggregated to a state-year panel and a survey-weighted Callaway–Sant'Anna ATT is fit with design-based standard errors.
• Parallel-trends sensitivity (HonestDiD) and a combined diagnostic are each one line.
• The causal ATT is robust to the reweighting (composition differences out in a DiD), while the survey weights remain essential for the descriptive estimand and for valid inference.

What to try next:

1. Real BRFSS data: replace the data-generating cell with pyreadstat.read_xport(...) per year and concatenate; the rest is unchanged.
2. Other estimators: swap estimator="CallawaySantAnna" for "SunAbraham", "ImputationDiD", or "WooldridgeDiD" — the adapter resolves any estimator exported by diff_diff.
3. Other reweighting: try method="ipw" or method="cbps" in poststratify_by_wave — the adapter is weighting-method agnostic.
4. Continuous treatment: dd.ContinuousDiD works for dose-of-policy designs.
5. Cross-language replication: compare against R's survey::svydesign + did::att_gt for a JSS / Epidemiology-Methods note.

References

Methodology

• Callaway, B., & Sant'Anna, P. H. C. (2021). Difference-in-differences with multiple time periods. Journal of Econometrics, 225(2), 200-230. — the ATT(g, t) estimator behind CallawaySantAnna.
• Sant'Anna, P. H. C., & Zhao, J. (2020). Doubly robust difference-in-differences estimators. Journal of Econometrics, 219(1), 101-122. — the doubly-robust DiD operationalised by estimation_method="dr".
• Rambachan, A., & Roth, J. (2023). A more credible approach to parallel trends. Review of Economic Studies, 90(5), 2555-2591. — the HonestDiD sensitivity framework used in Step 4.
• Sarig, T., Galili, T., & Eilat, R. (2023). balance - a Python package for balancing biased data samples. arXiv 2307.06024. — the balance package paper.
• Ghandour, K., & Reece, A. (2025). diff-diff: Modern Difference-in-Differences in Python. Zenodo, DOI 10.5281/zenodo.19646175. — the diff-diff package citation.

Data

• CDC Behavioral Risk Factor Surveillance System (BRFSS). — the public-use file this notebook's synthetic frame mirrors.
• Census ACS 1-year PUMS. — the demographic-margin target frame.

Related tutorials

• balance_quickstart
• balance_quickstart_cbps