{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# Synthetic Control\n",
    "\n",
    "This notebook demonstrates **synthetic control** impact estimation via [pysyncon](https://github.com/sdfordham/pysyncon) [`Synth`](https://sdfordham.github.io/pysyncon/synth.html).\n",
    "\n",
    "The synthetic control method constructs a weighted combination of control units as a counterfactual for the treated unit, then estimates the causal effect as the difference between observed and synthetic outcomes in the post-treatment period.\n",
    "\n",
    "## Workflow overview\n",
    "\n",
    "1. User provides `products.csv` (each product = a \"unit\" in the panel)\n",
    "2. User configures `DATA.ENRICHMENT` for treatment assignment\n",
    "3. User calls `measure_impact(config.yaml)`\n",
    "4. Engine handles everything internally (adapter, enrichment, transform, model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1",
   "metadata": {},
   "source": [
    "## Initial setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "import pandas as pd\n",
    "from impact_engine_measure import measure_impact, load_results\n",
    "from impact_engine_measure.core.validation import load_config\n",
    "from impact_engine_measure.core import apply_transform\n",
    "from impact_engine_measure.models.factory import get_model_adapter\n",
    "from online_retail_simulator import enrich, simulate"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3",
   "metadata": {},
   "source": [
    "## Step 1 — Product Catalog\n",
    "\n",
    "We use a small catalog (20 products) because synthetic control treats each product as a separate unit in the donor pool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4",
   "metadata": {},
   "outputs": [],
   "source": [
    "output_path = Path(\"output/demo_synthetic_control\")\n",
    "output_path.mkdir(parents=True, exist_ok=True)\n",
    "\n",
    "catalog_job = simulate(\"configs/demo_synthetic_control_catalog.yaml\", job_id=\"catalog\")\n",
    "products = catalog_job.load_df(\"products\")\n",
    "\n",
    "print(f\"Generated {len(products)} products\")\n",
    "print(f\"Products catalog: {catalog_job.get_store().full_path('products.csv')}\")\n",
    "products.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5",
   "metadata": {},
   "source": [
    "## Step 2 — Engine configuration\n",
    "\n",
    "Configure the engine with the following sections.\n",
    "- `ENRICHMENT` — Quality boost applied to ~10% of products starting Nov 15\n",
    "- `TRANSFORM` — `prepare_for_synthetic_control` adds a time-aware `treatment` column\n",
    "- `MODEL` — `synthetic_control` with one designated treated unit\n",
    "\n",
    "The panel structure is 20 products x 30 days (Nov 1–30). Enrichment starts Nov 15, giving 14 pre-treatment and 16 post-treatment periods.\n",
    "\n",
    "With `seed=42` and `enrichment_fraction=0.1`, product `BU9XOOP3LG` is the treated unit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6",
   "metadata": {},
   "outputs": [],
   "source": [
    "config_path = \"configs/demo_synthetic_control.yaml\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7",
   "metadata": {},
   "source": [
    "## Step 3 — Impact evaluation\n",
    "\n",
    "A single call to `measure_impact()` handles everything.\n",
    "- Engine creates `CatalogSimulatorAdapter`\n",
    "- Adapter simulates daily metrics (30-day panel)\n",
    "- Adapter applies enrichment (quality boost to treated product after Nov 15)\n",
    "- `prepare_for_synthetic_control` transform adds the `treatment` column\n",
    "- `SyntheticControlAdapter` builds a `Dataprep` object and fits via pysyncon's `Synth`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8",
   "metadata": {},
   "outputs": [],
   "source": [
    "job_info = measure_impact(config_path, str(output_path), job_id=\"results\")\n",
    "print(f\"Job ID: {job_info.job_id}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9",
   "metadata": {},
   "source": [
    "## Step 4 — Review results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10",
   "metadata": {},
   "outputs": [],
   "source": [
    "result = load_results(job_info)\n",
    "\n",
    "data = result.impact_results[\"data\"]\n",
    "model_params = data[\"model_params\"]\n",
    "estimates = data[\"impact_estimates\"]\n",
    "summary = data[\"model_summary\"]\n",
    "\n",
    "print(\"=\" * 60)\n",
    "print(\"SYNTHETIC CONTROL RESULTS\")\n",
    "print(\"=\" * 60)\n",
    "\n",
    "print(f\"\\nModel Type: {result.model_type}\")\n",
    "print(f\"Treated Unit: {model_params['treated_unit']}\")\n",
    "print(f\"Treatment Time: {model_params['treatment_time']}\")\n",
    "\n",
    "print(\"\\n--- Impact Estimates ---\")\n",
    "print(f\"ATT:               {estimates['att']:.4f}\")\n",
    "print(f\"Standard Error:    {estimates['se']:.4f}\")\n",
    "print(f\"CI Lower:          {estimates['ci_lower']:.4f}\")\n",
    "print(f\"CI Upper:          {estimates['ci_upper']:.4f}\")\n",
    "print(f\"Cumulative Effect: {estimates['cumulative_effect']:.4f}\")\n",
    "\n",
    "print(\"\\n--- Model Summary ---\")\n",
    "print(f\"Pre-treatment periods:  {summary['n_pre_periods']}\")\n",
    "print(f\"Post-treatment periods: {summary['n_post_periods']}\")\n",
    "print(f\"Control units:          {summary['n_control_units']}\")\n",
    "print(f\"MSPE:                   {summary['mspe']:.4f}\")\n",
    "print(f\"MAE:                    {summary['mae']:.4f}\")\n",
    "\n",
    "print(\"\\n--- Control Unit Weights ---\")\n",
    "for unit, weight in summary[\"weights\"].items():\n",
    "    if weight > 0.001:\n",
    "        print(f\"  {unit}: {weight:.4f}\")\n",
    "\n",
    "print(\"\\n\" + \"=\" * 60)\n",
    "print(\"Demo Complete!\")\n",
    "print(\"=\" * 60)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11",
   "metadata": {},
   "source": [
    "## Step 5 — Model validation\n",
    "\n",
    "Compare the model's ATT estimate against the **true per-period revenue difference** for the treated unit (enriched vs counterfactual)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "12",
   "metadata": {},
   "outputs": [],
   "source": [
    "def calculate_true_effect(\n",
    "    baseline_metrics: pd.DataFrame,\n",
    "    enriched_metrics: pd.DataFrame,\n",
    "    treated_unit: str,\n",
    "    treatment_time: str,\n",
    ") -> dict:\n",
    "    \"\"\"Calculate TRUE per-period effect for the treated unit.\"\"\"\n",
    "    treatment_date = pd.Timestamp(treatment_time)\n",
    "\n",
    "    baseline_unit = baseline_metrics[\n",
    "        (baseline_metrics[\"product_id\"] == treated_unit) & (pd.to_datetime(baseline_metrics[\"date\"]) >= treatment_date)\n",
    "    ]\n",
    "    enriched_unit = enriched_metrics[\n",
    "        (enriched_metrics[\"product_id\"] == treated_unit) & (pd.to_datetime(enriched_metrics[\"date\"]) >= treatment_date)\n",
    "    ]\n",
    "\n",
    "    baseline_mean = baseline_unit[\"revenue\"].mean()\n",
    "    enriched_mean = enriched_unit[\"revenue\"].mean()\n",
    "    mean_effect = enriched_mean - baseline_mean\n",
    "\n",
    "    return {\n",
    "        \"baseline_mean\": float(baseline_mean),\n",
    "        \"enriched_mean\": float(enriched_mean),\n",
    "        \"mean_effect\": float(mean_effect),\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13",
   "metadata": {},
   "outputs": [],
   "source": [
    "baseline_metrics = catalog_job.load_df(\"metrics\").rename(columns={\"product_identifier\": \"product_id\"})\n",
    "\n",
    "enrich(\"configs/demo_synthetic_control_enrichment.yaml\", catalog_job)\n",
    "enriched_metrics = catalog_job.load_df(\"enriched\").rename(columns={\"product_identifier\": \"product_id\"})\n",
    "\n",
    "print(f\"Baseline records: {len(baseline_metrics)}\")\n",
    "print(f\"Enriched records: {len(enriched_metrics)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14",
   "metadata": {},
   "outputs": [],
   "source": [
    "treated_unit = model_params[\"treated_unit\"]\n",
    "true_effect = calculate_true_effect(baseline_metrics, enriched_metrics, treated_unit, \"2024-11-15\")\n",
    "\n",
    "true_me = true_effect[\"mean_effect\"]\n",
    "model_me = estimates[\"att\"]\n",
    "\n",
    "if true_me != 0:\n",
    "    recovery_accuracy = (1 - abs(1 - model_me / true_me)) * 100\n",
    "else:\n",
    "    recovery_accuracy = 100 if model_me == 0 else 0\n",
    "\n",
    "print(\"=\" * 60)\n",
    "print(\"TRUTH RECOVERY VALIDATION\")\n",
    "print(\"=\" * 60)\n",
    "print(f\"True mean effect:  {true_me:.4f}\")\n",
    "print(f\"Model estimate:    {model_me:.4f}\")\n",
    "print(f\"Recovery accuracy: {max(0, recovery_accuracy):.1f}%\")\n",
    "print(\"=\" * 60)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15",
   "metadata": {},
   "source": [
    "### Convergence analysis\n",
    "\n",
    "How does the estimate improve as the number of control units increases? We vary the catalog size (keeping 1 treated unit) and observe convergence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16",
   "metadata": {},
   "outputs": [],
   "source": [
    "control_sizes = [3, 5, 8, 12, 15, 18]\n",
    "estimates_list = []\n",
    "truth_list = []\n",
    "\n",
    "parsed = load_config(config_path)\n",
    "measurement_params = parsed[\"MEASUREMENT\"][\"PARAMS\"]\n",
    "\n",
    "# enrichment_start is auto-injected during measure_impact() but not\n",
    "# available when calling apply_transform directly — supply it explicitly.\n",
    "transform_config = {\n",
    "    \"FUNCTION\": \"prepare_for_synthetic_control\",\n",
    "    \"PARAMS\": {\"enrichment_start\": \"2024-11-15\"},\n",
    "}\n",
    "\n",
    "# Determine which product is enriched in the retrieved metrics\n",
    "# (enrichment assignment may differ from the measure_impact run).\n",
    "enriched_ids = enriched_metrics[enriched_metrics[\"enriched\"]][\"product_id\"].unique()\n",
    "convergence_treated = enriched_ids[0]\n",
    "all_ids = enriched_metrics[\"product_id\"].unique()\n",
    "control_pool = [pid for pid in all_ids if pid != convergence_treated]\n",
    "\n",
    "for n_controls in control_sizes:\n",
    "    subset_ids = [convergence_treated] + control_pool[:n_controls]\n",
    "    enriched_sub = enriched_metrics[enriched_metrics[\"product_id\"].isin(subset_ids)]\n",
    "    baseline_sub = baseline_metrics[baseline_metrics[\"product_id\"].isin(subset_ids)]\n",
    "\n",
    "    true = calculate_true_effect(baseline_sub, enriched_sub, convergence_treated, \"2024-11-15\")\n",
    "    truth_list.append(true[\"mean_effect\"])\n",
    "\n",
    "    transformed = apply_transform(enriched_sub, transform_config)\n",
    "    model = get_model_adapter(\"synthetic_control\")\n",
    "    model.connect(measurement_params)\n",
    "    result = model.fit(\n",
    "        data=transformed,\n",
    "        treatment_time=\"2024-11-15\",\n",
    "        treated_unit=convergence_treated,\n",
    "        outcome_column=\"revenue\",\n",
    "        unit_column=\"product_id\",\n",
    "        time_column=\"date\",\n",
    "    )\n",
    "    estimates_list.append(result.data[\"impact_estimates\"][\"att\"])\n",
    "\n",
    "print(\"Convergence analysis complete.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17",
   "metadata": {},
   "outputs": [],
   "source": [
    "from notebook_support import plot_convergence\n",
    "\n",
    "plot_convergence(\n",
    "    control_sizes,\n",
    "    estimates_list,\n",
    "    truth_list,\n",
    "    xlabel=\"Number of Control Units\",\n",
    "    ylabel=\"ATT\",\n",
    "    title=\"Synthetic Control: Convergence of Estimate to True Effect\",\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}