Evaluation report — course pipeline
Generated by python -m parsrag.eval run. Offline: FakeLLM (extractive) + HashingEmbedding.
original (66 questions)
| metric |
value |
95% CI |
| retrieval_r@1 |
0.881 |
[0.797, 0.949] |
| retrieval_r@3 |
1.000 |
|
| retrieval_mrr |
0.941 |
[0.898, 0.975] |
| answer_f1 |
0.276 |
[0.210, 0.344] |
| answered_f1 |
0.285 |
[0.219, 0.353] |
| citation_precision |
0.842 |
[0.737, 0.930] |
| faithfulness |
1.000 |
[1.000, 1.000] |
| citation_hit |
0.842 |
|
| abstention_precision |
0.714 |
|
| abstention_recall |
0.714 |
|
| abstention_f1 |
0.714 |
|
| answered_rate |
0.966 |
|
paraphrase (51 questions)
| metric |
value |
95% CI |
| retrieval_r@1 |
0.370 |
[0.239, 0.500] |
| retrieval_r@3 |
0.543 |
|
| retrieval_mrr |
0.487 |
[0.373, 0.600] |
| answer_f1 |
0.016 |
[0.005, 0.031] |
| answered_f1 |
0.075 |
[0.031, 0.122] |
| citation_precision |
0.800 |
[0.500, 1.000] |
| faithfulness |
1.000 |
[1.000, 1.000] |
| citation_hit |
0.800 |
|
| abstention_precision |
0.100 |
|
| abstention_recall |
0.800 |
|
| abstention_f1 |
0.178 |
|
| answered_rate |
0.217 |
|