An independent appeals process

[Aureliano Buendia]

It would be nice if grant applicants could request and receive an independent review of funding denials. Right now, the decisions are made by program officers (POs), who are professional yet human. It is hard to imagine that POs are completely unbiased in their decisions.

Perhaps this already exists at NSF. Does anyone know?

[Unsub]

I am not so sure that funding decision rests with the Program officers. They can only fund projects that the panel and reviewers recommended as being Highly Competitive. I have never received any funding from NSF and I have tried many times. However being a part of the panel a few times, I have realized that the panel formation and the ensuing discussion are themselves completely random processes and if one of the panelists does not strongly support the project, then even a very good proposal can go unfunded. And many of us are not so vocal and pushy and being quiet means some one with more muscle can talk big about another good proposal and then if no one pushes the other good one it just gets left behind.

Either way, I don't think the POs can pick one from a non-competitive rank and decide to fund it because they know the PI or some such thing. Of course if there is not enough funds available (which is more often the case) even a highly competitive grant will go unfunded. More than just blaming POs and NSF the problem is with the state of funding for science. Common man thinks science is a waste of money and so we have to justify how each thing we want to do benefits the society and as we all know the benefits are not always immediate.

And yes I think NSF has an appeals process. if you poke around long enough in the GPG you will find the way to do it.

[Raven]

The following is excerpted from the NSF Grant Proposal Guide, available on-line:

D. Reconsideration

1. Overview

a. A proposer whose proposal has been declined may ask the cognizant NSF Program Officer or the cognizant Division Director for information over and above the explanatory materials received with the declination notice. If the PI/PD is not satisfied that the proposal was fairly handled and reasonably reviewed, he/she may request reconsideration by the responsible Assistant Director (AD) or Office Head. An organization (or an unaffiliated PI/PD) still not satisfied after reconsideration by the responsible AD/Office Head may request further reconsideration by the Deputy Director of the Foundation. The decision made by the Deputy Director is final.

b. If a proposal has been declined after review by the NSB, only an explanation will be available.

c. The aim of any reconsideration is to ensure that NSF’s review has been fair and reasonable, both substantively and procedurally. The scientific and technical merits may be examined within the context of budget availability and program priorities. Reconsideration also may address any procedural errors in peer review or other aspects of proposal review, including unaccounted-for conflict of interests or inappropriate consideration of records, information or rumor.

d. Award of NSF assistance is discretionary and reconsideration is not an adversarial process. A formal hearing, therefore, is not provided. Because factors such as program budget and priorities factor into the decision on a proposal, NSF cannot ensure proposers that reconsideration will result in an award even if error is established in connection with the initial review

e. No revisions made to the proposal after declination will be considered in connection with the original proposal. A substantially revised proposal, however, may be submitted for review as a new proposal under standard procedures. NSF reserves the right to return without review a proposal that is substantially the same as one that was previously reviewed and declined whether or not a request for reconsideration was made.

2. Applicability

NSF's reconsideration process is available to individuals and organizations concerning proposals for grant funding. It does not apply to:

a. “discourage” (i.e., non-binding) decisions resulting from submission of a preliminary proposal;

b. proposals for:

(1) fellowships;

(2) travel grants;

(3) Small Grants for Exploratory Research (SGER);

(4) Rapid Response Grants (RAPID);

(5) EArly-concept Grants for Exploratory Research (EAGER); or

(6) Phase I proposals submitted under the Small Business Innovation Research (SBIR) program.

c. proposals returned without review by NSF for failure to:

(1) be submitted with sufficient lead time before the activity is to begin;

(2) meet an announced proposal deadline date; or

(3) meet NSF proposal preparation requirements, such as page limitations, formatting instructions, and electronic submission, as specified in the Grant Proposal Guide or program solicitation.

3. Reconsideration Process

The following paragraphs highlight the various stages of the NSF Reconsideration Process, including the necessary procedural aspects of each stage of the process:

a. Explanations by the NSF Program Officer or Division Director

When a proposal is declined, the PI/PD receives verbatim but unattributed copies of any ad hoc reviews and the panel summary (if applicable), a description of how the proposal was reviewed, and, if not otherwise provided in the panel summary, an explanation (written or telephoned) of the basis for the declination. A returned proposal also will be accompanied by an explanation. A PI/PD who is considering asking for reconsideration should first contact the cognizant NSF Program Officer or Division Director, who will afford the PI/PD an opportunity to present his/her point of view, provide additional information if any exists, and take any further action that seems appropriate.

b. Reconsideration by the Cognizant Assistant Director

(1) If dissatisfied with the explanation provided by the NSF Program Officer or Division Director, the PI/PD may request in writing that NSF reconsider its action. Such a request will be considered only if the PI/PD has first sought and obtained an explanation from the cognizant NSF Program Officer or Division Director, and only if the request is received by the Foundation within 90 days after the declination or the return. The request should be addressed to the AD/Office Head for the directorate or office that handled the proposal and should explain why the PI/PD believes that the declination or return was unwarranted.

(2) The AD/Office Head will reconsider the record to determine whether NSF’s review of the declined proposal was fair and reasonable, substantively and procedurally, taking into account availability of funds and the policies and priorities of the program and NSF. In the case of a returned proposal, the record will be reviewed to determine whether the proposed project was inappropriate for NSF consideration. The AD/Office Head may request additional information from the PI/PD and may obtain additional reviews. If additional reviews are sought, they are subject to standard review procedures (e.g., instructions must be provided to reviewers and conflicts-of-interest policies must be followed). The AD/Office Head may conduct the reconsideration personally or may designate another NSF official who had no part in the initial review to do so. As used here, “AD/Office Head” includes such a designated official.

(3) Within 45 days after the date of the request, the AD/Office Head will furnish the results of the reconsideration, in writing, to the PI/PD. If results cannot be furnished within 45 days, the AD/Office Head will send the PI/PD a written explanation of the need for more time, indicating the date when the results can be expected. If the AD/Office Head reaffirms the declination or return, he/she will inform the PI/PD that the PI/PD’s organization may obtain further reconsideration by the Deputy Director of NSF as provided below.

c. Further Reconsideration by the Deputy Director

(1) Within 60 days after the AD/Office Head has notified the PI/PD of the results of the reconsideration, the proposing organization or an unaffiliated PI/PD may request further reconsideration by the Deputy Director of NSF.

(2) A request for further reconsideration need not be in any particular format, but it must be in writing, and must be signed by the organization’s president or other chief executive officer and by the PI/PD. For declinations, it should explain why the organization believes that an error may have occurred in the initial evaluation and why it is not entirely satisfied with the reconsideration by the responsible AD/Office Head. For returned proposals, it should explain why the organization believes that an error may have occurred in the initial determination that the proposal was inappropriate for NSF consideration.

(3) The Deputy Director will review the request for further reconsideration and the record of earlier NSF actions, including the original review and the reconsideration by the AD/Office Head, to determine whether NSF’s review of the declined proposal was fair and reasonable, or, in the case of a returned proposal, whether the proposed project was inappropriate for NSF consideration. The Deputy Director may request additional information from the PI/PD or the proposing organization and may obtain additional reviews. If additional reviews are sought, they are subject to standard review procedures (e.g., instructions must be provided to reviewers and conflicts-of-interest policies must be followed).

(4) The Deputy Director may conduct the further reconsideration personally or may designate another NSF official who had no part in the initial evaluation of the proposal or the earlier reconsideration to do so. As used here, “Deputy Director” includes such a designated official.

(5) Within 30 days after a request for further reconsideration is received at NSF, the Deputy Director will furnish the results of the further reconsideration, in writing, to the organization. If results cannot be furnished within 30 days, the Deputy Director will send the organization a written explanation of the need for more time, indicating the date when the results can be expected.

(6) The decision made by the Deputy Director is final.

[Aureliano Buendia]

Thanks for the replies. Is anyone aware of attempts to use this appeals process? Are there any stats on its use and success? I'll bet both are rare to non-existant.

[Aureliano Buendia]

And yes, program officers have wide latitude to decide who gets funded and who doesn't. In my previously funded grants, I've had interactions with program officers that have made this clear, as well as interactions that show their power over budgets.

And the comment above: "And many of us are not so vocal and pushy and being quiet means some one with more muscle can talk big about another good proposal and then if no one pushes the other good one it just gets left behind," gives me NO confidence in the process at all.

[Unsub]

"...gives me NO confidence in the process at all."
I totally agree and I have never been funded so far but beign a part of the review process, I saw with my own eyes the 'randomness' or the biased nature of the whole process. It has made me even more disillusioned. Worse off for me, 3 yrs after one of my proposals got reviews which made me doubt if PhD and over 11 papers in international journals mean anything about my abilities, I see the big guys in my field doing exactly similar experiments I'd proposed and publishing the results.

The system is broken and I dont know how it can be fixed. Only the 'big' people in the field of the students and post docs of these people can get grants because the reviewers know them and dont feel threatened by them because after all they are all one and the same - each one is scratching the other one's back.
The system does not let a new researcher enter - if you have a doctorate from a developing country and do not know the eminent researchers in the US then forget breaking into this NSF granting system. Spend time sucking up to some big names in the field before submitting a grant. this is the suggestion of a PO who did not use the phrase 'suck up' but said 'get to know' the big reseachers in the field.

[backup]

I agree with unsub on the elite nature, or insiders' club, of those that get funded.

And the CAREER program, which is ostensibly targeted at young researchers, is a tough nut as well. Success often requires 3-4 submissions, the terms are five years (long by NSF standards) and thus require huge budgets again. It seems like "normal" people - that is, those of us who teach more than 2 courses a year, are highly disadvantaged. And don't even get me started on the RUI program. How the heck does it even work? No one has ever given me a coherent explanation.

I like the concept of CAREER grants, that is designating a pot of money for young researchers at elite institutions. But how about creating other designated pots of money for beginning researchers at schools with heavier teaching loads, or small or no grad programs?

[anonomy]

Posted by backup on Jun 16, 2010, 8:28am
"And don't even get me started on the RUI program. How the heck does it even work? No one has ever given me a coherent explanation.

I like the concept of CAREER grants, that is designating a pot of money for young researchers at elite institutions. But how about creating other designated pots of money for beginning researchers at schools with heavier teaching loads, or small or no grad programs?"

CAREER grants generally don't have much to do with being a hotshot young researcher at an elite institution. The program isn't for "pure" research. It is for (warning, buzz word) integration of research and teaching. So, if your research program has little to do with the courses you teach, it's not a good fit. If you're a researcher who is not very pedagogically inclined, it's not a good fit. It's unfortunate that many (even folks in Congress and other high level decision-makers based on the press releases around budget season) seem to equate this program to the various NIH "get a promising young researcher started" grants.

On the other hand, RUI IS for "schools with heavier teaching loads, or small or no grad programs." An RUI designation basically gives the proposer a few extra pages to make their case to the reviewers and PO especially the "broader impacts" of the work as it relates to improving your institution, but it's not really a program on it's own (neither is CAREER). These proposals come in identified as RUI, CAREER, etc. and there are additional considerations related to their review but they are reviewed in the appropriate panel for their scientific content.
[If you think current panels are "random" then just try to picture one to distribute a pot of money for CAREER or RUI proposals from across the foundation!?! Individual programs don't get enough of these to justify stand-alone panels.]
And RUIs do pretty well- at least for the programs I've talked to where folks have looked into it- RUIs wind up being funded at the same rate as those from "big institutions."