To assess the quality of the reviewing work by the program committee, ICDE 2010 obtained feedback from authors of submitted papers. To recognize the thorough and thoughtful work of PC members, we introduced an Outstanding Reviewer Award and an Outstanding Area Chair Award. ICDE will recognize 20 outstanding reviewers and 1 outstanding area chair at Los Angeles during the first week of March 2010.
During the author-reply phase, we solicited author feedback on the perceived quality of reviews as an optional question. About 50 percent of the authors provided feedback, by assigning each review one of the following three grades:
3=good (thoughtful, thorough and constructive),
2=satisfactory (competent and fair),
1=poor (superficial or biased).
We were very pleased to find that the average rating per review is about 2.2, clearly above the “satisfactory (2)” level. Moreover, we also found the variance to be fairly low.
Outstanding Reviewer Awards
To reward the thoroughness, thoughtfulness, and constructiveness of reviews, ICDE 2010 is giving out Outstanding Reviewer Awards, based on the following Normalized Care metric: the average grade given by authors to the PC member’s reviews, divided by the average score that the PC member gave to the papers. We realize that such a metric is merely a syntactic proxy for reviewing quality, and as such can by no means comprehensively capture the true performance of each reviewer. But as such an award is given out for the very first time at any database conference, we limited our evaluation to this objectively measurable and reasonably meaningful metric.
Based on this measure, those reviewers with a Normalized Care of 1.3 or higher are winners of the ICDE 2010 Outstanding Reviewer Award:
Alex Delis (U Athens, Greece)
Andrey Balmin (IBM, USA)
David Lomet (Microsoft, USA)
George Kollios (Boston U, USA)
Goetz Graefe (HP, USA)
Jeff Naughton (U Wisconsin, USA)
Jianliang Xu (Hong Kong Baptist U, China)
Jiawei Han (UIUC, USA)
Lei Chen (HKUST, China)
Raghav Kaushik (Microsoft, USA)
Samuel Madden (MIT, USA)
Sarah Cohen-Boulakia (LRI, France)
Sergey Melnik (Google, USA)
Shivakumar Vaithyanathan (IBM, USA)
Thomas Neumann (MPI Informatics, Germany)
Thomas Seidl (RWTH Aachen, Germany)
Ulf Leser (Humboldt U, Germany)
Wang-Chiew Tan (UC Santa Cruz, USA)
Wei-Shinn Ku (Auburn U, USA)
Yoshiharu Ishikawa (Nagoya U, Japan)
The highest scorer on the Normalized Care metric is Sarah Cohen-Boulakia. Her prize is a free registration for ICDE 2010. Congratulations to all award winners, and many thanks to all PC members for their excellent work.
Outstanding Area Chair Award
To reward an area chair who has shown exemplary leadership in guiding his or her team to timely and conscientious reviewing, ICDE is recognizing an Outstanding Area Chair with the prize of a free registration for ICDE 2010. The metric used to determine the winner was the product of the Normalized Return (fraction of the completed number of reviews at the official review due date) and the area’s Total Normalized Care (aggregated over all review-quality ratings in the area). Based on this metric, the winner of the ICDE 2010 Outstanding Area Chair Award is:
Susan Davidson (U Pennsylvania, USA)
Congratulations to our outstanding area chair and many thanks to all area chairs for their excellent work.