Methodology

How to read message test results

What is a message test?

A message test measures whether a specific message changes how people feel about an issue. Respondents are randomly assigned to one of three groups: two see different messages, while a control group sees no message. We compare responses to measure effect.

What do the colors mean?

Each lift score is color-coded by strength and direction:

Persuasive (+5 or more) — Statistically significant positive shift
Directional positive (+1 to +4) — Positive trend, not yet significant
No effect (0) — No measurable impact
Directional negative (-1 to -4) — Negative trend, not yet significant
Backlash (-5 or worse) — Statistically significant negative shift

What does "gray" / no effect mean?

A gray score means the message didn't move the needle. This doesn't mean the message failed — awareness, for example, is hard to shift with a single exposure. It simply means we can't attribute a change to this message on this outcome.

Sample & confidence

Respondents are drawn from a nationally representative online panel. With ~669 per group, we detect effects of ±3–5 points at 90% confidence. Subgroup results have smaller samples and wider margins.

How to use subgroup data

Subgroup breakdowns show how audiences responded. Large positive lifts mean a message resonates; large negatives indicate backlash. Use these for targeting, but note subgroup samples are smaller so interpret directionally.

Message Tests › Clean Energy Initiative Round 2

Clean Energy Initiative Round 2

● Complete Fielded Jan 10 – 11, 2026 2,006 respondents 2 messages
📊 2,006 nationally representative respondents · ~669 per group · ±3–5pt margin at 90% confidence Learn more
Ask about this test
Query across all KPIs, subgroups, and messages with natural language.
Which subgroups had the biggest backlash? Compare Group 1 vs Group 2 among moderates What drove the affordability shift? Show me confidence intervals for job creation
Message Performance
Group 1 drives persuasive lifts on Job Creation (+6) and Energy Costs (+5), while Group 2 triggers backlash on Grid Reliability. Neither message moved overall awareness.
OutcomeBaselineGroup 1Group 2
Awareness38%+0No measurable effect+0No measurable effect
Job Creation31%+6Persuasive — significant lift+2Directional positive
Affordability27%+3Directional positive-1Directional negative
Energy Costs40%+5Persuasive — significant lift+3Directional positive
Local Impact48%+2Directional positive-1Directional negative
Grid Reliability44%-2Directional negative-5Backlash — significant negative shift
Backlash
(-) Dir.
None
(+) Dir.
Persuasive
Subgroup Results
Group 1 resonates strongly with suburban, young, and parent audiences (+10pp each). Group 2 backfires with higher-income and conservative segments, losing up to 9 points.
Group 1
Overall +2
Top Performing
Parents
+10
18 – 34
+10
Suburban
+10
Some College
+8
Urban
+7
Independent
+5
Group 2
Overall -2
Top Performing
Suburban
+9
Some College
+6
Underperforming
$100k+
-9
Bachelor's+
-7
Conservative
-7
Men
-5
Rural
-3
🏆
Recommended Next Step
Prioritize Group 1 for your next campaign
It was the strongest message across your success KPIs, with a 92% probability of being your best performer on Job Creation and consistent positive effects across multiple subgroups.
Target These Audiences for Maximum Impact
Suburban
+10
pp lift
View in Analysis →
Parents
+10
pp lift
View in Analysis →
18 – 34
+10
pp lift
View in Analysis →
Urban
+7
pp lift
View in Analysis →
Independent
+5
pp lift
View in Analysis →
Caution: Group 2 showed backlash among higher-income ($100k+), older (55+), and conservative-leaning audiences. Consider avoiding that message with those segments, or explore the Analysis tab to investigate further.
KPI
Subgroup
Group 1 (n≈669)
Group 2 (n≈669)
Control (n≈668)
Ask about the data
Query across all KPIs, subgroups, and messages
AI
847 responses · All Groups · All Respondents
AI-Generated Themes
Responses automatically clustered by topic. Click a theme to see its responses.