CCARF Practice Question

Paper 4 · Question 26

D5 Context Management & Reliability · Paper 4

A multi-agent research system finds conflicting data on a pharmaceutical compound's efficacy. A 2022 peer-reviewed clinical trial reports 78% efficacy, while a 2024 preprint reports 45% efficacy with a larger sample size. The synthesis agent must produce a recommendation for a medical advisory board. Which approach correctly handles the conflicting sources?

• A.Average the two values (61.5%) and present it as the best estimate with a note about variance between studies
• B.Use the 2024 preprint as it has a larger sample size and is more recent, noting it supersedes the 2022 study
• C.Present both findings with full provenance: source identity, publication date, peer-review status, sample size, methodology, and any methodological differences that may explain the discrepancy. Let the advisory board weigh the evidence
• D.Exclude both findings and report that no reliable consensus exists, recommending further research before any conclusions

Show answer & explanations

Correct answer: C

A: Averaging conflicting scientific results is methodologically invalid. The studies may have different populations, methodologies, or endpoints. The average has no scientific meaning and obscures the actual disagreement.

B: Recency and sample size alone do not determine reliability. The preprint has not undergone peer review, and simply picking one source discards valuable context that the advisory board needs for informed decision-making.

C: For a medical advisory board, preserving full provenance is critical. The board needs to weigh peer-review status, sample size, methodology differences, and temporal context. Annotating both sources with complete metadata enables informed human judgment on conflicting scientific evidence.

D: Excluding available evidence is unhelpful. The advisory board can make informed decisions with properly attributed conflicting data. Withholding findings because they conflict reduces the board's ability to assess the situation.

Related Context Management & Reliability questions

More: Paper 4 · All practice questions