Tags: * hire more staff, not only for ARR * for subreviewers * improve management of junior reviewers
" Consider broadening the scope of the FTE position beyond ARR. ACL relies unusually heavily on the
volunteer efforts of researchers. Many analogous and smaller academic organizations have full-time staff
who ensure continuity. For example, the American Musicological Society (definitely smaller and poorer
than ACL) has a paid executive director who serves on its board, as well as three other paid staff members.
For rhe right person, an Executive Director role at ACL would be mutually beneficial. This would bring
us better into line with other academic organizations. The changing demographics of the conference mean
that the majority of authors are too inexperienced to be trusted with reviews. The status quo ante was
that senior researchers (faculty and their equivalents in industry) used trusted graduate students (and
their equivalents in industry) as sub-reviewers when they deemed it appropriate. Personally, I did this
only for papers where I knew that: a) the subject matter is well-aligned with the interests of the sub-reviewer
b) the sub-reviewer is adequately knowledgable in the field. For example, they might have passed a qualifying
exam c) the sub-reviewer is adequately acculturated to the publication norms of computational linguistics.
They might have submitted one or more reasonable papers as first author, and experienced the review
feedback process. d) the sub-reviewer will probably produce a reasonable review without extensive supervision
(but I always checked, and used any deficiencies as teaching opportunities) All of these things are
judgment calls. The pool of people capable of executing a research project and writing an acceptable
ACL paper is larger than the pool of suitable reviewers. A rough guideline is that Ph.D students should
not review until they have had a research proposal accepted and the dissertation in progress. Industrial
researchers whose qualification is a Masters' degree need at least one year's experience of doing submittable
research before they should review. If these criteria mean that the pool of reviewers is too small,
authors not meeting the criteria should be allowed to review if and only if someone more qualified agrees
to mentor them. "