Schema Selection Evaluator · 可复现实验

方法	平均必要列召回	平均候选列数	完整策略	参考 SQL 结果匹配	边界状态符合预期
仅候选检索	58.2%	4.8	1 / 5	1 / 5	1 / 4
检索 + 路径补全	86.0%	8.4	4 / 5	4 / 5	4 / 4

orders_zh

按客户编号汇总下单时间为上海9月1日至7日、排除取消订单的未发金额和件数

联合租户/订单键补全；时间半开区间、取消与粒度

仅候选检索

ready · 6 列

必要列召回 60.0% · 完整策略否

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "order_lines",
    "orders"
  ],
  "columns": [
    "order_lines.ordered_qty",
    "order_lines.shipped_qty",
    "order_lines.unit_price_cents",
    "orders.created_at",
    "orders.customer_id",
    "orders.status"
  ],
  "relationships": [],
  "candidate_columns": 6,
  "notes": [],
  "coverage": {
    "strategy": "aggregate-lines-per-order",
    "tables_recall": 1.0,
    "missing_tables": [],
    "columns_recall": 0.6,
    "missing_columns": [
      "order_lines.order_id",
      "order_lines.tenant_id",
      "orders.order_id",
      "orders.tenant_id"
    ],
    "relationships_recall": 0.0,
    "missing_relationships": [
      "order_lines(tenant_id,order_id)->orders(tenant_id,order_id)"
    ],
    "element_recall": 0.6153846153846154,
    "complete": false,
    "extra_columns": []
  },
  "reference_query": {
    "status": "not_run_incomplete_context",
    "expected_rows": [
      [
        "C1",
        2,
        10,
        7000
      ],
      [
        "C2",
        1,
        2,
        4000
      ]
    ]
  }
}

路径补全后

ready · 10 列

必要列召回 100.0% · 完整策略是

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "order_lines",
    "orders"
  ],
  "columns": [
    "order_lines.order_id",
    "order_lines.ordered_qty",
    "order_lines.shipped_qty",
    "order_lines.tenant_id",
    "order_lines.unit_price_cents",
    "orders.created_at",
    "orders.customer_id",
    "orders.order_id",
    "orders.status",
    "orders.tenant_id"
  ],
  "relationships": [
    "order_lines(tenant_id,order_id)->orders(tenant_id,order_id)"
  ],
  "candidate_columns": 10,
  "notes": [
    "Added FK endpoints: order_lines.order_id, order_lines.tenant_id, orders.order_id, orders.tenant_id"
  ],
  "coverage": {
    "strategy": "aggregate-lines-per-order",
    "tables_recall": 1.0,
    "missing_tables": [],
    "columns_recall": 1.0,
    "missing_columns": [],
    "relationships_recall": 1.0,
    "missing_relationships": [],
    "element_recall": 1.0,
    "complete": true,
    "extra_columns": []
  },
  "reference_query": {
    "status": "matched",
    "expected_rows": [
      [
        "C1",
        2,
        10,
        7000
      ],
      [
        "C2",
        1,
        2,
        4000
      ]
    ],
    "actual_rows": [
      [
        "C1",
        2,
        10,
        7000
      ],
      [
        "C2",
        1,
        2,
        4000
      ]
    ]
  }
}

orders_en

By customer id, sum remaining amount and quantity for order date September 1–7 Shanghai, excluding cancelled orders

同一业务问题的英文词表覆盖

仅候选检索

ready · 6 列

必要列召回 60.0% · 完整策略否

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "order_lines",
    "orders"
  ],
  "columns": [
    "order_lines.ordered_qty",
    "order_lines.shipped_qty",
    "order_lines.unit_price_cents",
    "orders.created_at",
    "orders.customer_id",
    "orders.status"
  ],
  "relationships": [],
  "candidate_columns": 6,
  "notes": [],
  "coverage": {
    "strategy": "aggregate-lines-per-order",
    "tables_recall": 1.0,
    "missing_tables": [],
    "columns_recall": 0.6,
    "missing_columns": [
      "order_lines.order_id",
      "order_lines.tenant_id",
      "orders.order_id",
      "orders.tenant_id"
    ],
    "relationships_recall": 0.0,
    "missing_relationships": [
      "order_lines(tenant_id,order_id)->orders(tenant_id,order_id)"
    ],
    "element_recall": 0.6153846153846154,
    "complete": false,
    "extra_columns": []
  },
  "reference_query": {
    "status": "not_run_incomplete_context",
    "expected_rows": [
      [
        "C1",
        2,
        10,
        7000
      ],
      [
        "C2",
        1,
        2,
        4000
      ]
    ]
  }
}

路径补全后

ready · 10 列

必要列召回 100.0% · 完整策略是

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "order_lines",
    "orders"
  ],
  "columns": [
    "order_lines.order_id",
    "order_lines.ordered_qty",
    "order_lines.shipped_qty",
    "order_lines.tenant_id",
    "order_lines.unit_price_cents",
    "orders.created_at",
    "orders.customer_id",
    "orders.order_id",
    "orders.status",
    "orders.tenant_id"
  ],
  "relationships": [
    "order_lines(tenant_id,order_id)->orders(tenant_id,order_id)"
  ],
  "candidate_columns": 10,
  "notes": [
    "Added FK endpoints: order_lines.order_id, order_lines.tenant_id, orders.order_id, orders.tenant_id"
  ],
  "coverage": {
    "strategy": "aggregate-lines-per-order",
    "tables_recall": 1.0,
    "missing_tables": [],
    "columns_recall": 1.0,
    "missing_columns": [],
    "relationships_recall": 1.0,
    "missing_relationships": [],
    "element_recall": 1.0,
    "complete": true,
    "extra_columns": []
  },
  "reference_query": {
    "status": "matched",
    "expected_rows": [
      [
        "C1",
        2,
        10,
        7000
      ],
      [
        "C2",
        1,
        2,
        4000
      ]
    ],
    "actual_rows": [
      [
        "C1",
        2,
        10,
        7000
      ],
      [
        "C2",
        1,
        2,
        4000
      ]
    ]
  }
}

customer_product

按客户名称和商品名称汇总下单时间为上海9月1日至7日、排除取消的未发金额

沿客户—订单—明细—商品补齐关系与键

仅候选检索

ready · 7 列

必要列召回 41.2% · 完整策略否

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "customers",
    "order_lines",
    "orders",
    "products"
  ],
  "columns": [
    "customers.name",
    "order_lines.ordered_qty",
    "order_lines.shipped_qty",
    "order_lines.unit_price_cents",
    "orders.created_at",
    "orders.status",
    "products.name"
  ],
  "relationships": [],
  "candidate_columns": 7,
  "notes": [],
  "coverage": {
    "strategy": "customer-order-line-product",
    "tables_recall": 1.0,
    "missing_tables": [],
    "columns_recall": 0.4117647058823529,
    "missing_columns": [
      "customers.customer_id",
      "customers.tenant_id",
      "order_lines.order_id",
      "order_lines.product_id",
      "order_lines.tenant_id",
      "orders.customer_id",
      "orders.order_id",
      "orders.tenant_id",
      "products.product_id",
      "products.tenant_id"
    ],
    "relationships_recall": 0.0,
    "missing_relationships": [
      "order_lines(tenant_id,order_id)->orders(tenant_id,order_id)",
      "order_lines(tenant_id,product_id)->products(tenant_id,product_id)",
      "orders(tenant_id,customer_id)->customers(tenant_id,customer_id)"
    ],
    "element_recall": 0.4583333333333333,
    "complete": false,
    "extra_columns": []
  },
  "reference_query": {
    "status": "not_run_incomplete_context",
    "expected_rows": [
      [
        "Acme",
        "Bolt",
        7000
      ],
      [
        "Beta",
        "Nut",
        4000
      ]
    ]
  }
}

路径补全后

ready · 17 列

必要列召回 100.0% · 完整策略是

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "customers",
    "order_lines",
    "orders",
    "products"
  ],
  "columns": [
    "customers.customer_id",
    "customers.name",
    "customers.tenant_id",
    "order_lines.order_id",
    "order_lines.ordered_qty",
    "order_lines.product_id",
    "order_lines.shipped_qty",
    "order_lines.tenant_id",
    "order_lines.unit_price_cents",
    "orders.created_at",
    "orders.customer_id",
    "orders.order_id",
    "orders.status",
    "orders.tenant_id",
    "products.name",
    "products.product_id",
    "products.tenant_id"
  ],
  "relationships": [
    "order_lines(tenant_id,order_id)->orders(tenant_id,order_id)",
    "order_lines(tenant_id,product_id)->products(tenant_id,product_id)",
    "orders(tenant_id,customer_id)->customers(tenant_id,customer_id)"
  ],
  "candidate_columns": 17,
  "notes": [
    "Added FK endpoints: customers.customer_id, customers.tenant_id, order_lines.order_id, order_lines.product_id, order_lines.tenant_id, orders.customer_id, orders.order_id, orders.tenant_id, products.product_id, products.tenant_id"
  ],
  "coverage": {
    "strategy": "customer-order-line-product",
    "tables_recall": 1.0,
    "missing_tables": [],
    "columns_recall": 1.0,
    "missing_columns": [],
    "relationships_recall": 1.0,
    "missing_relationships": [],
    "element_recall": 1.0,
    "complete": true,
    "extra_columns": []
  },
  "reference_query": {
    "status": "matched",
    "expected_rows": [
      [
        "Acme",
        "Bolt",
        7000
      ],
      [
        "Beta",
        "Nut",
        4000
      ]
    ],
    "actual_rows": [
      [
        "Acme",
        "Bolt",
        7000
      ],
      [
        "Beta",
        "Nut",
        4000
      ]
    ]
  }
}

stock

列出商品名称和库存

单表问题不需要路径扩展

仅候选检索

ready · 2 列

必要列召回 100.0% · 完整策略是

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "products"
  ],
  "columns": [
    "products.available_qty",
    "products.name"
  ],
  "relationships": [],
  "candidate_columns": 2,
  "notes": [],
  "coverage": {
    "strategy": "scoped-stock",
    "tables_recall": 1.0,
    "missing_tables": [],
    "columns_recall": 1.0,
    "missing_columns": [],
    "relationships_recall": 1.0,
    "missing_relationships": [],
    "element_recall": 1.0,
    "complete": true,
    "extra_columns": []
  },
  "reference_query": {
    "status": "matched",
    "expected_rows": [
      [
        "Bolt",
        30
      ],
      [
        "Nut",
        40
      ]
    ],
    "actual_rows": [
      [
        "Bolt",
        30
      ],
      [
        "Nut",
        40
      ]
    ]
  }
}

路径补全后

ready · 2 列

必要列召回 100.0% · 完整策略是

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "products"
  ],
  "columns": [
    "products.available_qty",
    "products.name"
  ],
  "relationships": [],
  "candidate_columns": 2,
  "notes": [],
  "coverage": {
    "strategy": "scoped-stock",
    "tables_recall": 1.0,
    "missing_tables": [],
    "columns_recall": 1.0,
    "missing_columns": [],
    "relationships_recall": 1.0,
    "missing_relationships": [],
    "element_recall": 1.0,
    "complete": true,
    "extra_columns": []
  },
  "reference_query": {
    "status": "matched",
    "expected_rows": [
      [
        "Bolt",
        30
      ],
      [
        "Nut",
        40
      ]
    ],
    "actual_rows": [
      [
        "Bolt",
        30
      ],
      [
        "Nut",
        40
      ]
    ]
  }
}

underspecified

未发金额

故意缺少分组/日期/状态描述，图扩展不能补出业务口径

仅候选检索

ready · 3 列

必要列召回 30.0% · 完整策略否

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "order_lines"
  ],
  "columns": [
    "order_lines.ordered_qty",
    "order_lines.shipped_qty",
    "order_lines.unit_price_cents"
  ],
  "relationships": [],
  "candidate_columns": 3,
  "notes": [],
  "coverage": {
    "strategy": "aggregate-lines-per-order",
    "tables_recall": 0.5,
    "missing_tables": [
      "orders"
    ],
    "columns_recall": 0.3,
    "missing_columns": [
      "order_lines.order_id",
      "order_lines.tenant_id",
      "orders.created_at",
      "orders.customer_id",
      "orders.order_id",
      "orders.status",
      "orders.tenant_id"
    ],
    "relationships_recall": 0.0,
    "missing_relationships": [
      "order_lines(tenant_id,order_id)->orders(tenant_id,order_id)"
    ],
    "element_recall": 0.3076923076923077,
    "complete": false,
    "extra_columns": []
  },
  "reference_query": {
    "status": "not_run_incomplete_context",
    "expected_rows": [
      [
        "C1",
        2,
        10,
        7000
      ],
      [
        "C2",
        1,
        2,
        4000
      ]
    ]
  }
}

路径补全后

ready · 3 列

必要列召回 30.0% · 完整策略否

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "order_lines"
  ],
  "columns": [
    "order_lines.ordered_qty",
    "order_lines.shipped_qty",
    "order_lines.unit_price_cents"
  ],
  "relationships": [],
  "candidate_columns": 3,
  "notes": [],
  "coverage": {
    "strategy": "aggregate-lines-per-order",
    "tables_recall": 0.5,
    "missing_tables": [
      "orders"
    ],
    "columns_recall": 0.3,
    "missing_columns": [
      "order_lines.order_id",
      "order_lines.tenant_id",
      "orders.created_at",
      "orders.customer_id",
      "orders.order_id",
      "orders.status",
      "orders.tenant_id"
    ],
    "relationships_recall": 0.0,
    "missing_relationships": [
      "order_lines(tenant_id,order_id)->orders(tenant_id,order_id)"
    ],
    "element_recall": 0.3076923076923077,
    "complete": false,
    "extra_columns": []
  },
  "reference_query": {
    "status": "not_run_incomplete_context",
    "expected_rows": [
      [
        "C1",
        2,
        10,
        7000
      ],
      [
        "C2",
        1,
        2,
        4000
      ]
    ]
  }
}

denied_salary

列出员工工资 salary

禁止表在排序前被排除

仅候选检索

no_candidates · 0 列

边界用例：单独验收状态

查看候选、遗漏、路径与 SQL 结果

{
  "status": "no_candidates",
  "tables": [],
  "columns": [],
  "relationships": [],
  "candidate_columns": 0,
  "notes": [],
  "status_matches_expectation": true
}

路径补全后

no_candidates · 0 列

边界用例：单独验收状态

查看候选、遗漏、路径与 SQL 结果

{
  "status": "no_candidates",
  "tables": [],
  "columns": [],
  "relationships": [],
  "candidate_columns": 0,
  "notes": [],
  "status_matches_expectation": true
}

ambiguous_warehouse

列出物流跟踪号和仓库名称

起始仓与目的仓两条关系需要澄清

仅候选检索

ready · 2 列

边界用例：单独验收状态

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "shipments",
    "warehouses"
  ],
  "columns": [
    "shipments.tracking_number",
    "warehouses.name"
  ],
  "relationships": [],
  "candidate_columns": 2,
  "notes": [],
  "status_matches_expectation": false
}

路径补全后

ambiguous_path · 2 列

边界用例：单独验收状态

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ambiguous_path",
  "tables": [
    "shipments",
    "warehouses"
  ],
  "columns": [
    "shipments.tracking_number",
    "warehouses.name"
  ],
  "relationships": [],
  "candidate_columns": 2,
  "notes": [
    "shipments(tenant_id,origin_id)->warehouses(tenant_id,warehouse_id)",
    "shipments(tenant_id,destination_id)->warehouses(tenant_id,warehouse_id)"
  ],
  "status_matches_expectation": true
}

disconnected_invoice

按客户名称汇总发票金额

没有维护过的关系时不按同名字段猜连接

仅候选检索

ready · 2 列

边界用例：单独验收状态

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "customers",
    "invoices"
  ],
  "columns": [
    "customers.name",
    "invoices.amount_cents"
  ],
  "relationships": [],
  "candidate_columns": 2,
  "notes": [],
  "status_matches_expectation": false
}

路径补全后

no_authorized_path · 2 列

边界用例：单独验收状态

查看候选、遗漏、路径与 SQL 结果

{
  "status": "no_authorized_path",
  "tables": [
    "customers",
    "invoices"
  ],
  "columns": [
    "customers.name",
    "invoices.amount_cents"
  ],
  "relationships": [],
  "candidate_columns": 2,
  "notes": [
    "customers → invoices: no approved, authorized path"
  ],
  "status_matches_expectation": true
}

denied_join_key

按客户编号汇总下单时间为上海9月1日至7日、排除取消订单的未发金额和件数

补全不能重新引入被禁止的联合键列

仅候选检索

ready · 6 列

边界用例：单独验收状态

查看候选、遗漏、路径与 SQL 结果

{
  "status": "ready",
  "tables": [
    "order_lines",
    "orders"
  ],
  "columns": [
    "order_lines.ordered_qty",
    "order_lines.shipped_qty",
    "order_lines.unit_price_cents",
    "orders.created_at",
    "orders.customer_id",
    "orders.status"
  ],
  "relationships": [],
  "candidate_columns": 6,
  "notes": [],
  "status_matches_expectation": false
}

路径补全后

no_authorized_path · 6 列

边界用例：单独验收状态

查看候选、遗漏、路径与 SQL 结果

{
  "status": "no_authorized_path",
  "tables": [
    "order_lines",
    "orders"
  ],
  "columns": [
    "order_lines.ordered_qty",
    "order_lines.shipped_qty",
    "order_lines.unit_price_cents",
    "orders.created_at",
    "orders.customer_id",
    "orders.status"
  ],
  "relationships": [],
  "candidate_columns": 6,
  "notes": [
    "order_lines → orders: no approved, authorized path"
  ],
  "status_matches_expectation": true
}

实验边界

[
  "Hand-authored aliases and fixtures; no model or embedding retrieval, no held-out benchmark.",
  "Fixed reference SQL only: result matches are not generated-SQL accuracy.",
  "Graph expansion checks approved connectivity, not automatic business intent or cardinality inference.",
  "English and Chinese order cases are translations, not independent business samples.",
  "underspecified uses a known target as a diagnostic; its missing intent cannot be inferred from the question."
]

完整机器可读结果：report.json；复现：python3 evaluate.py --top-k 8