| 1 | For funding agencies such as NSF program directors |
| 2 | that co-ordinate panels, assigning proposals to |
| 3 | reviewers is a major challenge. It is important that each proposal be |
| 4 | reviewed by qualified experts, and at the same time we would like the |
| 5 | workload across different reviewers to be roughly balanced. The same |
| 6 | issue arises for a program committee chair, who may have to assign |
| 7 | literally hundreds of papers to a program committee consisting of |
| 8 | thirty to forty program committee members. |
| 9 | |
| 10 | What does CMT use? What does Easychair use? |
| 11 | |
| 12 | From now on we will focus on the problem of assigning papers to |
| 13 | reviewers. |
| 14 | We assume that each reviewer is given access to the |
| 15 | list of papers to be reviewed, and provides input on their |
| 16 | preferences by giving a ``desirability'' score to each paper. |
| 17 | We also assume that each paper has to be reviewed by $at least r$ |
| 18 | reviewers. |
| 19 | |
| 20 | List of codes. |
| 21 | |
| 22 | We do not consider stable marriage type preference lists, |
| 23 | because a strict ranking of papers would be rather tedious |
| 24 | to produce. In this scheme, the papers are essentially grouped |
| 25 | into a few categories. |
| 26 | |
| 27 | |
| 28 | Ideally, from the perspective of the papers, we would like to |
| 29 | assign each paper the $r$ ``best'' reviewers for the paper. |
| 30 | Ofcourse, this would lead to a load imbalanced solution where |
| 31 | the load on some program committee members is very high, and the |
| 32 | load on others is low. On the other hand, we could insist |
| 33 | on a perfectly load balanced solution in which the number |
| 34 | of papers assigned to each program committee member do not |
| 35 | exceed $\lceil rN/P \rceil$, where $N$ is the number of |
| 36 | submissions and $P$ is the number of program committee members. |
| 37 | Recall that each paper needs $r$ reviewers, so a total of $rN$ |
| 38 | reviews need to be generated. However this may lead to a solution |
| 39 | which is not optimal from the perspective of the papers. |
| 40 | |
| 41 | One of our goals is to study precisely this tradeoff, and allow each |
| 42 | reviewer to have upto $\lceil rN/P \rceil + C$ papers assigned to |
| 43 | them, where $C$ is the {\em imbalance factor}. The main question |
| 44 | we consider is: is it possible to obtain a high quality |
| 45 | assignment with a fairly low value of $C$? |
| 46 | |
| 47 | One can ask the same question from the perspective of the reviewers |
| 48 | as well. Each reviewer would ideally like papers |
| 49 | that are the ``most desirable'' from their point of view. |
| 50 | |
| 51 | {\em Stinkers} are papers that pretty much no-one wanted to review. |
| 52 | We would like to spread the load of the stinkers as evenly as possible. |