TL;DR
Get everyday essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Artificial Analysis scores Meta’s Muse Ember 36 on its Intelligence Index and estimates an average cost of $0.14 per index task, compared with $0.41 for Claude Haiku 5.5 at the same score. The evaluation also finds low factual accuracy alongside a high rate of refusals and a lower hallucination rate; performance varies by task.
Meta released Muse Ember on November 9, and Artificial Analysis reports that its max-effort version scored 36 on the Intelligence Index at an estimated $0.14 per index task. Claude Haiku 5.5 also scored 36 in the comparison, at $0.41 per task, while separate tests found Muse Ember often declines questions and has low factual accuracy.
Artificial Analysis describes Muse Ember as a proprietary model available through Meta’s API and Muse Code. It accepts text and images and generates text. In the index evaluation, the max setting scored 36, while the xhigh setting scored 34 at an estimated $0.12 per task. These costs represent the average API cost of one task in the index, not a universal price for every user request.
The report lists input pricing at $0.10 per million tokens and output pricing at $0.50 per million, matching the listed rates for Claude Haiku 5.5 and GPT-6 Luna. Artificial Analysis attributes Muse Ember’s lower index-task cost partly to cached input priced at $0.004 and partly to using about a third as many output tokens as Haiku 5.5 to complete the index. It reports output speed of 440 tokens per second for the max setting and a one-million-token context window.
Results varied across other evaluations. Muse Ember scored 68.1% on AutomationBench-AA, level with Meta’s more expensive Muse Wonder in that test. It scored 22.8% on GDP.pdf, compared with Haiku 5.5’s 20.8%, but trailed Haiku on the AA-Briefcase knowledge-work test. Coding results were mixed: Muse Ember scored higher on Terminal-Bench 4.0 and lower on FrontierCode 2.0. In Meta’s Muse Code tool, it scored 46 on Artificial Analysis’s Coding Agent Index.
Muse Ember’s benchmark score is 36 with a factuality trade-off
Meta’s model matches Claude Haiku 5.5 on the Intelligence Index at a lower estimated task cost. Separate testing found frequent refusals and low factual accuracy.
Same index score, lower estimated cost
The benchmark task estimate is about one-third of Haiku 5.5’s; actual costs depend on workload and API use.
| Model | Index score | Cost per index task |
|---|---|---|
| Muse Ember (max) | 36 | $0.14 |
| Claude Haiku 5.5 (max) | 36 | $0.41 |
| Muse Wonder | 44 | $6.95 |
Factual answers come with caveats
In AA-Omniscience, Muse Ember declined more often and hallucinated less than Haiku 5.5.
- 21.1% answered correctly
- 64% declined
- 17.4% hallucination rate
- 34.7% answered correctly
- 34.2% hallucination rate
Results vary by task
Strong results in some evaluations do not establish performance on a company’s own work.
Competitive result
Scored 68.1% on AutomationBench-AA, level with the more expensive Muse Wonder.
Mixed comparison
Scored 22.8% on GDP.pdf, ahead of Haiku 5.5’s 20.8%, but trailed it on AA-Briefcase.
Mixed results
Scored higher on Terminal-Bench 4.0 and lower on FrontierCode 2.0 than Haiku 5.5.
Test it on real workflows
Benchmark estimates are a starting point; business costs and quality depend on the work being done.
- Compare models on representative prompts and company data.
- Track accuracy, refusals, latency, and total API cost.
- Check whether the lower task cost fits the workflow’s quality needs.
Where the Lower Task Cost Matters
The reported price difference may matter to organizations running high volumes of repeated model steps, such as extraction, tagging, routing, and form filling. Artificial Analysis’s index estimate puts Muse Ember at roughly one-third of Haiku 5.5’s per-task cost for this benchmark. Actual costs will depend on the workload, token use, caching, and API configuration.
The quality trade-off is material. The report’s AA-Omniscience results show Muse Ember answered only 21.1% of questions correctly, while declining 64%. Its reported hallucination rate was 17.4%, below Haiku 5.5’s 34.2% in the same evaluation. A system that often refuses may be useful where unsupported answers carry risk, but less useful when users need broad factual coverage. The benchmark figures do not establish how the model will perform on any particular company’s data or tasks.
Popular right now
De'Longhi Rivelia Automatic Espresso Machine with Grinder and Milk Frother, Beige
Vitamix Touch & Go Advance 32-Ounce Bar Blender, White
Ninja Crispi Microwave and Air Fryer with FusionCrisp, Silver, MW101
As an affiliate, we earn on qualifying purchases.
How Muse Ember Compares
Artificial Analysis compares models across nine evaluations in its Intelligence Index. Its November 9–10 data places Muse Ember (max) at 36 and Haiku 5.5 (max) at the same score. The report lists Muse Wonder at 44 for $6.95 per index task, illustrating a much higher estimated cost for Meta’s larger model in this benchmark. Those figures describe the index setup and should not be read as direct measures of every use case.
The factual-question test adds a different dimension to the headline score. Artificial Analysis says AA-Omniscience allows a model to decline rather than guess. In its results, Haiku 5.5 answered 34.7% correctly, compared with Muse Ember’s 21.1%; Muse Ember declined more often and had a lower measured hallucination rate. These are test-specific outcomes, not a guarantee that the model will refuse or avoid errors in every interaction.
What the Benchmarks Cannot Show
The source material does not provide details on independent replication, confidence intervals, or how much results may vary across prompts and configurations. Artificial Analysis’s cost-per-task figure is an estimate for its index workload; the cost on a business’s own tasks remains unknown. The reported benchmark scores also do not establish whether the model will meet a specific organization’s accuracy, reliability, or latency requirements.
The low AA-Omniscience accuracy and high refusal rate leave open how Muse Ember handles questions that mix known and unknown information. The reported tests do not show whether those refusals can be adjusted, how performance changes with additional context, or how the model compares on a particular company’s documents and workflows.
Testing Muse Ember in Workflows
Businesses considering Muse Ember can compare it with other models on representative tasks, tracking accuracy, refusal rates, latency, and total API cost. The Artificial Analysis results provide an initial comparison, but they do not replace testing with an organization’s prompts, data, and quality requirements. Further evaluations or product updates may clarify how the model performs across workloads beyond the published benchmarks.
Key Questions
What score did Muse Ember receive?
Artificial Analysis reported a score of 36 for Muse Ember at max effort on its Intelligence Index. The xhigh setting scored 34.
How does its reported task cost compare with Haiku 5.5?
The index estimate is $0.14 per task for Muse Ember (max) and $0.41 for Claude Haiku 5.5 (max), with both scoring 36. These are costs for the benchmark tasks, not a fixed cost for every request.
Does Muse Ember answer factual questions accurately?
In Artificial Analysis’s AA-Omniscience test, it answered 21.1% correctly and declined 64% of questions. The results apply to that test and do not establish accuracy on other tasks.
What did the evaluation report about hallucinations?
Artificial Analysis measured a 17.4% hallucination rate for Muse Ember in AA-Omniscience, compared with 34.2% for Claude Haiku 5.5. The report also found Muse Ember declined questions more often.
Where did Muse Ember perform well?
It scored 68.1% on AutomationBench-AA and 22.8% on GDP.pdf. Its coding results were mixed, with a higher score than Haiku 5.5 on Terminal-Bench 4.0 and a lower score on FrontierCode 2.0.
Source: Thorsten Meyer AI
