Commit | Line | Data |
---|---|---|
affb6e6a MM |
1 | For funding agencies such as NSF program directors |
2 | that co-ordinate panels, assigning proposals to | |
3 | reviewers is a major challenge. It is important that each proposal be | |
4 | reviewed by qualified experts, and at the same time we would like the | |
5 | workload across different reviewers to be roughly balanced. The same | |
6 | issue arises for a program committee chair, who may have to assign | |
7 | literally hundreds of papers to a program committee consisting of | |
8 | thirty to forty program committee members. | |
9 | ||
10 | What does CMT use? What does Easychair use? | |
11 | ||
12 | From now on we will focus on the problem of assigning papers to | |
13 | reviewers. | |
14 | We assume that each reviewer is given access to the | |
15 | list of papers to be reviewed, and provides input on their | |
16 | preferences by giving a ``desirability'' score to each paper. | |
17 | We also assume that each paper has to be reviewed by $at least r$ | |
18 | reviewers. | |
19 | ||
20 | List of codes. | |
21 | ||
22 | We do not consider stable marriage type preference lists, | |
23 | because a strict ranking of papers would be rather tedious | |
24 | to produce. In this scheme, the papers are essentially grouped | |
25 | into a few categories. | |
26 | ||
27 | ||
28 | Ideally, from the perspective of the papers, we would like to | |
29 | assign each paper the $r$ ``best'' reviewers for the paper. | |
30 | Ofcourse, this would lead to a load imbalanced solution where | |
31 | the load on some program committee members is very high, and the | |
32 | load on others is low. On the other hand, we could insist | |
33 | on a perfectly load balanced solution in which the number | |
34 | of papers assigned to each program committee member do not | |
35 | exceed $\lceil rN/P \rceil$, where $N$ is the number of | |
36 | submissions and $P$ is the number of program committee members. | |
37 | Recall that each paper needs $r$ reviewers, so a total of $rN$ | |
38 | reviews need to be generated. However this may lead to a solution | |
39 | which is not optimal from the perspective of the papers. | |
40 | ||
41 | One of our goals is to study precisely this tradeoff, and allow each | |
42 | reviewer to have upto $\lceil rN/P \rceil + C$ papers assigned to | |
43 | them, where $C$ is the {\em imbalance factor}. The main question | |
44 | we consider is: is it possible to obtain a high quality | |
45 | assignment with a fairly low value of $C$? | |
46 | ||
47 | One can ask the same question from the perspective of the reviewers | |
48 | as well. Each reviewer would ideally like papers | |
49 | that are the ``most desirable'' from their point of view. | |
50 | ||
51 | {\em Stinkers} are papers that pretty much no-one wanted to review. | |
52 | We would like to spread the load of the stinkers as evenly as possible. |