Monday, October 23, 2006

Making the case for an ACM Fellow

ACM's highest membership grade is Fellow, intended to recognize and honor outstanding ACM members for their achievements in computer science and information technology and for their significant contributions to the mission of ACM. The goal is for Fellows to represent the top 1% of ACM's membership. (The other advanced membership grades are Distinguished and Senior.)

I served a five-year term on the ACM Fellows Committee, including chairing it one year. And as co-chair of the ACM Awards Committee, I have been a non-voting observer for the past five years. In those ten years, I have observed a number of commonalities that I think I can discuss without violating the confidentiality of the committee's discussions. [1]

First, I should say that this is a very dedicated and hardworking committee, who take their task very seriously. They are unpaid volunteers who each year read and consider some 500 nomination and endorsement letters. Although their collective expertise covers an impressive portion of the discipline, it is inevitable that no member of the committee will be at home in the areas of all of the candidates. But they each try to properly evaluate all of the cases presented, prior to a face-to-face meeting where they discuss each case.

Second, cases fall into three groups of roughly equal size:
  • Cases generally agreed to demonstrate that the candidate meets the criteria.
  • Cases generally agreed to not do so.
  • Cases where there is either disagreement or uncertainty within the committee.
Not surprisingly, the bulk of the committee's discussion is devoted to the latter group, and this is where the chance of mistakes is greatest.

Every year there are surely a few candidates who are rejected, not because they are not qualified, but because their nomination and endorsements fail to make the case as effectively as possible. There is no way for the committee to know which cases these are. When the discussion is inconclusive, the committee tends to make the conservative choice: A candidate who is not accepted, but should have been, can be nominated again, but a candidate who is accepted will not be re-evaluated. [2]

Here are some suggestions for nominators and endorsers about things they can do to strengthen (or avoid unnecessarily weakening) the cases for their candidates.
  1. Nominate the candidate: Probably the greatest single reason for people who should be ACM Fellows not being accepted is that they have not been nominated. The committee is not empowered to select candidates on its own, it can only judge the nominations it receives. [3]
  2. Follow the rules scrupulously: ACM Headquarters staff check all candidate packages strictly. If a nomination or endorsement arrives after the deadline, the committee doesn't see it. If a nominator or endorser isn't an ACM member as of the deadline, the committee doesn't see their input. If there are not five endorsements, the committee doesn't see the case at all. Etc.
  3. Summarize the case in the nomination: In addition to supplying the required data, use the 7,500 characters to describe the candidate's accomplishments briefly, but specifically.
    • Avoid flowery adjectives, multiple superlatives, and overworked words such a "seminal" and "unique." Instead, indicate the importance of the work by describing its specific consequences (e.g., a new line of research taken up by others, a system or product in widespread use, an important conjecture resolved, a popular textbook) in terms understandable to a non-specialist.
    • The ideal candidate will have distinguished personal technical accomplishments, distinguished technical leadership, and distinguished ACM and community service. The more "concentrated" the accomplishments are into one or two areas, the more outstanding they need to be in those areas.
    • Although there is no formal age or experience requirement for Fellow, evidence is needed that the candidate's accomplishments are already--not just potentially--in the top 1% of ACM's membership.
    Cases with a weak nomination letter usually end in the bottom third and get little discussion.
  4. This is not a tenure case: Performing acceptably for a sufficient number of years is not enough. Having hundreds of refereed publications is not enough. There is no "up or out"; many distinguished members will go their entire careers without becoming Fellows.
  5. Choose the endorsers carefully: Since it is assumed that likely candidates will all have strong nominations, much of the committee's discussion focuses on the endorsements.
    • Each endorser should be able to personally attest to portions of the candidate's accomplishments.
    • An endorsement from an ACM Fellow carries just a little more weight, provided that the Fellow is actually familiar with the accomplishments discussed.
    • An endorsement by someone known to members of the committee can be more readily calibrated.
    • It is helpful to have endorsers from several different organizations, and endorsers who can address each of the areas of accomplishment outlined in the nomination.
    • With few exceptions, those who endorse more than two or three candidates will not strengthen their cases. Nor will cycles in the endorsement graph. [4]
    • It is acceptable to seek more than the minimum of 5 endorsements, to guard against the contingency of an endorsement arriving late. But 5 strong endorsements will make a more convincing case than those same 5 endorsements plus 3 weaker ones.
  6. Write the endorsements as carefully as the nomination: An endorsement should focus on the accomplishments that you can personally attest to, and place them in context. Use this focus to include more specifics than are in the nomination, if you can. Merely repeating portions of the nomination letter will actually weaken the case.
    If you are endorsing multiple candidates, it is very important to rank them, and useful to compare them. (If you cannot do this, you probably don't know all of them well enough to endorse.)
Notes:
[1] These are my personal observations, and should not be interpreted as representing the views or policies of any other person or organization.
[2] Nominators of rejected candidates should seriously consider whether a stronger case could have been made; if they are convinced it could have been, it is acceptable to re-nominate the next year (but not effective to do so with the same letters). If a candidate is rejected two years in a row, a third consecutive nomination is not allowed. Wait a couple of years; if the candidate then has further accomplishments to strengthen the case, consider re-nominating.
[3] And, as a matter of policy, committee members recuse themselves from nominating or endorsing candidates, as do the Awards Committee co-chairs.
[4] There are over 500 ACM Fellows; why do we always get endorsements from the same 50?

Labels: ,

7 Comments:

Comment by Blogger Marc:

"This is not a tenure case: Performing acceptably for a sufficient number of years is not enough. Having hundreds of refereed publications is not enough."

This is an offensive description of the tenure process. Most universities will claim and many will enforce stricter criteria than those.

8:37 PM  
Comment by Blogger Jim Horning:

Sorry, it was not my intention to give offense. Having both received tenure, and sat on tenure committees, I realize that tenure criteria aren't simple. But they are different, yet the Fellows Committee gets suprisingly many nominations that look like recycled tenure packages.

9:17 PM  
Comment by Blogger Peter S Magnusson:

Where is the current membership of the ACM Awards Committee listed? I couldn't find it on ACM's web site. In case you happen to know, and thanks for a most useful posting.

10:55 PM  
Comment by Blogger Jim Horning:

I agree that this information isn't prominent. However, if you read the Annual Report on the Awards site, you will find

"1.3 Committee Organization

"The Awards Committee is a standing committee of Council, reporting through the President.
The ACM Awards Committee consists of the ACM President, the CEO/Executive Director (ex officio),
the Co-Chairs of the Awards Committee, the current chairs of the individual ACM award
selection committees, and the ACM SIG Chairs Liaison with the Awards Committee."

The committee chairs are also listed for each committee.

2:48 AM  
Comment by Blogger Craig:

I typically nominate two people each year for ACM fellow (and have done so since the inception of the program). As an outsider to the process who simply gets to observe what the black box selection process results are, I have to say that Jim's comments largely ring true with one caveat:

You have to work doubly hard to get a non-North American person made a Fellow. As best I can tell, the committee doesn't understand the non-US research milieu, with its different research emphasis and different degrees and honors. I don't know why this is true, but what evidence I have says it is. And it is very frustrating, given all the work ACM does to expand its international footprint.

And just for context: I'm a North American, a Fellow, and long-time ACM volunteer.

12:08 PM  
Comment by Anonymous S:

"There are over 500 ACM Fellows; why do we always get endorsements from the same 50?"

In part, you've answered your own question:
"An endorsement by someone known to members of the committee can be more readily calibrated."

3:41 PM  
Comment by Blogger Jim Horning:

@S:

I'll bet that every member of the Fellows Committee personally knows more than 50 Fellows (and not the same 50, either).

9:41 PM  

Post a Comment

<< NIASAWHIWB Home