In this paper, motivated by a recently proposed bias-variance based evaluation, we adopt a strong and unbiased "baseline", which is a virtual target model ...

ABSTRACT. Recent research has shown that the improvement of mean re- trieval effectiveness (e.g., MAP) may sacrifice the retrieval.

We will propose a generalized formulation of the bias- variance analysis with respect to IR performance metrics. (such as MAP and ERR) and construct a ...

In this paper, motivated by a recently proposed bias-variance based evaluation, we adopt a strong and unbiased “baseline”, which is a virtual target model ...

In this paper, motivated by a recently proposed bias-variance based evaluation, we adopt a strong and unbiased "baseline", which is a virtual target model ...

We also propose generalized bias-variance metrics, based on which a systematic and quantitative evaluation of the effectiveness-stability tradeoff is carried ...

Peng Zhang, Linxue Hao, Dawei Song, Jun Wang, Yuexian Hou, Bin Hu Generalized Bias-Variance Evaluation of TREC Participated Systems CIKM, 2014. CIKM 2014 ...

The evaluation of both effectiveness and stability are often based on a baseline model, which could be weak or biased. In addition, the effectiveness-stability ...

Generalized bias-variance evaluation of trec participated systems. In Proceedings of the 23rd ACM Inter- national Conference on Conference on Information ...

In general, the bias represents the gap between the expectation (i.e., mean) of estimated values and the true target value, while the variance represents the ...