Simulatte's synthetic India general population tested against 15 published Pew India survey questions spanning democracy satisfaction, party approval, governance preferences, institutional trust, religion, gender norms, and climate. The most technically complex study in the program.
40 personas calibrated across religion (Hindu/Muslim/Sikh/Christian), caste (General/OBC/SC/ST), region (North/South/West/East), and political lean. 22 development sprints from a 45.9% baseline. Final accuracy: 85.3% — 5.7 pp from the theoretical human ceiling.
India's political landscape requires calibration across four intersecting dimensions simultaneously. Unlike the US study where demographic and political lean correlate more predictably, Indian opinion is shaped by religion, caste, region, and political lean in combinations that do not reduce to a single axis.
Calibrated to Pew Global Attitudes Spring 2023 BJP favorability data. 14 bjp_supporter personas (35%) are required to reach Pew's ~42% BJP approval distributions. Original 7 bjp_supporter pool (18%) produced a structural ceiling on in02/in03.
| Religion | Approx. share | Calibration source |
|---|---|---|
| Hindu | 75% | Census + Pew 2023 |
| Muslim | 13% | Census 2011 |
| Sikh | 5% | Census 2011 |
| Christian | 7% | Census 2011 |
| Caste category | Approx. share |
|---|---|
| General | 25% |
| OBC | 40% |
| SC (Dalit) | 20% |
| ST (Tribal) | 15% |
At n=40 each persona represents 2.5% of the simulated distribution. A change of one persona's political lean shifts every question's distribution by 2.5 pp. Pool composition is therefore both a calibration tool and a source of structural constraint — certain question accuracies are mathematically bounded by how many BJP-voting vs. opposition-voting personas are in the pool.
15 questions from Pew Global Attitudes 2023, Pew Religion in India 2021, and Pew Gender Roles India 2022. Questions marked RLHF are constrained by model alignment training independent of persona calibration.
Green bar = ≥90% · Standard bar = 80–90% · Grey bar = <80% · RLHF = alignment-constrained floor
Six questions showing the range from near-perfect calibration to RLHF-constrained floors. Green bars = Simulatte. Light bars = Pew Research ground truth.
A overshoot (+19pp) is an RLHF structural floor — 14 bjp_supporters treat "no parliament" as abstractly good. 22 sprints of calibration could not reduce A below 62.5%.
A overshoot driven by bjp_supporter institutional trust floor. B-modal pattern in Pew data (48% B) reflects nuanced trust/concern split that BJP voters collapse into trust.
Before A-22: B=2% (Pew 11%). Behavioral anchoring ("daily schedule" vs. "secular identity") moved B to 8% — +5pp gain.
Study 1B required more than twice as many sprints as Study 1A to reach a comparable accuracy level, reflecting the greater structural complexity of the India dataset and the impact of the A-9 root cause fix — the largest single-sprint gain in the entire program.
| Sprint | Score | Δ | Key event |
|---|---|---|---|
| A-2 | 45.9% | — | Broken archetype mapping (all personas → moderate) |
| A-9 | 83.3% | +29.9 | Root cause fix: India archetype mapping bug |
| A-10 | 84.6% | +1.3 | Spread notes: in14/in06/in11/in02/in03/in12 |
| A-11 | 84.8% | +0.2 | in01/in08 spread notes; in14/in06 strengthened |
| A-12 | 85.0% | +0.2 | Pool rebalance: bjp_supporter 18%→35% |
| A-14 | 80.8% | −4.2 | First true bjp_supporter pool — new calibration challenges |
| A-15 | 81.6% | +0.8 | INC conviction; in07/in13 spread notes |
| A-17 | 79.9% | −0.2 | Trust 0.68 → bimodal collapse on in09 |
| A-18 | 83.4% | +3.5 | Trust fix; in15 "major threat ≠ development priority" (+25pp) |
| A-20 | 83.8% | +1.5 | in13 rebalanced; structural ceiling identified |
| A-21 | 83.1% | −0.7 | bjp_lean democratic narrative; sampling variance |
| A-22 | 85.3% | +2.2 | Pool recomposition (opposition_lean 6→3); in11 behavioral anchor |
Study 1B produced a systematic finding with implications beyond Simulatte. Anthropic's Constitutional AI and RLHF training creates behavioural blocks on outputs that endorse bypassing democratic accountability or gender discrimination. These blocks operate downstream of persona stance fields — the model reads the persona's position, then produces an output inconsistent with it.
This is not a Simulatte-specific finding. It applies to any LLM used as a survey respondent where the question content conflicts with alignment training. Cross-cultural social science applications should audit their question set for RLHF-blocked constructs before reporting accuracy claims.
Anthropic's Constitutional AI training creates two independent blocks:
Endorsing governance by a strong leader who bypasses parliament. Affects in07. The model reads the bjp_supporter persona's support for this arrangement, then refuses to endorse it and explains why parliamentary oversight is important. Robust across all 22 sprints — no narrative framing produced consistent breakthrough.
Endorsing gender-discriminatory norms (in12, in13, in14). This block is softer than Block 1 and partially addressable. When gender norms are framed at narrative generation time as dharma, Islamic teaching, or cultural community belonging — rather than as "discrimination" — partial breakthrough is possible. The in14 ceiling (always 100% very important) is not addressable.
Narrative-level intervention worked where survey-time framing failed.
Embedding traditional gender norms at narrative generation time as a genuine cultural or religious identity — framed through dharma, Islamic teaching, or joint family obligation — moved the score from 0% to 55–65%. The persona narrative must contain the stance as an identity claim, not as a policy position. When the survey model encounters "discrimination," it triggers the RLHF block. When it encounters "my family follows dharma," it can engage authentically.
23 separate narrative framings were tested for the strong leader question. Democratic mandate framing, national security framing, development-efficiency framing, and historical context framing were all attempted. None moved A below 62.5%. The RLHF block on "endorsing non-democratic governance" is more robust than the gender-norms block.
The single largest gain across both studies. _ARCHETYPE_TO_LEAN in attribute_filler.py lacked India archetype entries, so all India personas silently mapped to political_lean="moderate" for the first 8 sprints. Every political lean gate, stance field, and narrative constraint returned neutral values — equivalent to running with no political calibration at all. Fix: _get_political_lean() now reads directly from the demographic anchor's political profile for India personas, bypassing the broken lookup table entirely.
India's political distribution has no US equivalent. With 7 bjp_supporter personas (18%), the maximum achievable A+B on in02 (Modi approval) was mathematically bounded below Pew's 56% A. Rebalancing to 14 bjp_supporter personas (35%) added 17–22 pp to in02, in03, and in12 simultaneously in a single sprint. The A-22 further converted 3 opposition_lean personas (Birsa Munda, Ramesh Chamar, Thomas Mathew) to neutral — each represents a demographic with genuinely mixed BJP-era alignment — reducing the in09 C-floor by 7.5 pp.
The largest single-question gain after the root cause fix. India personas systematically chose B ("somewhat of a threat") because they interpreted "major threat" as implying climate should be prioritised over development. The spread note fix explicitly decoupled the two concepts: a farmer voting BJP can say "major threat" because his crops fail from monsoon disruption — this does not mean he wants development paused. The fix is qualitatively different from option-vocabulary anchoring: it addresses what the question means in context, not just what the response options say.
Before A-22, spread notes framed religion importance in terms of "secular identity" — causing personas to assert their religious identity and override the note (B stuck at 2% vs Pew 11%). The distinction: identity-level framing activates narrative override because the persona's narrative says "I am a devout Hindu" and the model interprets the note as questioning that identity. Rewriting the note around behavioral patterns — "is your daily routine primarily structured around religious observance, or around career and family?" — avoids triggering the override and moved B from 2% to 7.5%.
In02/in03 (Modi and BJP approval) were relatively straightforward to calibrate once the pool had sufficient BJP supporters. In04 (INC approval) was harder: bjp_supporter personas needed to express strong negative conviction about Congress — not just weak preference for BJP. Adding explicit INC conviction language ("frustrated with Congress's legacy", "believes Congress failed India's development") as a dedicated narrative field gave bjp_supporters the strong negative INC signal they needed without affecting their positive BJP signals. Without this asymmetry, BJP supporters tended to produce moderate INC approval instead of the D-modal distribution Pew observed.
# Study 1B — India Pew Replication cd study_1b_pew_india python3 run_study.py --simulatte-only # Reference cohorts (Sprint A-22) # Cohort 1: 6025615a # Cohort 2: 01de2a63 # Check audit manifests ls results/audit_manifest_a*.json