Proposal Review Process

« Return to page index

1. Overview

Proposals are evaluated on the basis of scientific merit. Proposals may also be reviewed for their feasibility by Observatory staff. A feasibility review may include a technical review, a data management review, or both. Unless otherwise noted in the Call for Proposals, semester proposals utilize the Panel Proposal Review process and DDT proposals utilize the Observatory Site Review process. The review process for special calls will be provided in the notice for the special call. Details about the Panel Proposal Review process are available online at https://science.nrao.edu/observing/proposal-types/proposal-review-system. The Observatory Site Review process is a less formal review process whereby primarily Observatory staff will provide information for the scientific, technical, and/or data management components of the proposal review.

For the Panel Proposal Review Process, the Observatory is responsible for recruiting reviewers for the Science Review Panels (SRPs) and TAC. The Observatory is committed to the principles of equity and inclusion in the peer review process. As such, it is a goal of recruiting that the demographics of the review panels and TAC reflect the gender demographics of the astronomical community4.

The Observatory welcomes volunteers to be a science reviewer for the Panel Proposal Review process. To apply to volunteer, please submit the online form at https://science.nrao.edu/observing/proposal-types/srp-volunteer-form. For more information about the volunteering and the review process, see online at https://science.nrao.edu/observing/proposal-types/volunteer_review.

Membership of the SRPs is kept confidential by the Observatory, but SRP members are welcome to note their participation (in their curriculum vitae, for example). Membership of the TAC is made public by the Observatory. For the most current semester, the TAC membership is available online https://science.nrao.edu/observing/proposal-types/time-allocation-committee.

4 The specific recruitment goals are determined by the demographics of the Observatory’s user base, which relies on users to provide their information. Gender-related reports are available at https://science.nrao.edu/science/reports/gender-related-systematics-in-the-nrao-and-alma-proposal-review-process

2. Proposal Assessment

The purpose of the proposal-selection process for Observatory telescopes is to select the proposals that potentially are most valuable for the advancement of scientific knowledge. This does not necessarily mean giving high marks to those proposals that will provide sure results; it also includes a careful consideration of well-reasoned proposals that may be unconventional, but provide opportunities for new discoveries. In the evaluation of proposals, reviewers consider how best to exploit the full capability of the unique scientific instruments that the Observatory operates for the community. Reviewers may also consider other factors such as

Publication record

Due recognition should be given to teams who have an established publication record for past related proposals. In contrast, some observers may have considerable data that clearly has not been analyzed and published yet.

Possibility of acquiring more appropriate data

Reviewers may also wish to consider the possibility that other data may be more appropriate for reaching the scientific goals of the proposal. For instance, the relevant data might already exist in the Archive (available through the Archive Access Tool).

Selection of resources

Reviewers may judge whether the requested observations are an appropriate approach to meeting the stated science objectives.

Resource requirements

Reviewers may take into account the amount of resources requested. For example, a particular proposal or scientific goal may be viewed as a valuable use of 4 hours of telescope time, but may not be as valuable if it requires 80 hours of time.

Student status

The Observatory encourages the use of its facilities for student research, particularly for PhD dissertations. In this case, it is most helpful if the proposal will say in some manner how the proposed observations will be used in the dissertation, and whether they are a sidelight or a main focus of the thesis. Reviewers are encouraged to comment on this topic and may choose to support proposals more strongly if there appears to be a well-thought-out program of student research.

3. Outcomes and Disposition Letters

A Disposition Letter is sent to the PI and co-Is for semester and DDT proposals. Disposition Letters are also available in the PST for proposers to review. For the semester proposals, this letter contains comments from the cognizant SRP, a linear-rank score from the SRP, comments from the feasibility review (if available), comments from the TAC (if available), and the scheduling priorities. For DDT proposals, this letter contains a brief notice of approval or justification if declined.

A scheduling priority is assigned to each session of a proposal, which is guided by the outcome of the SRPs (e.g., linear-rank score) or the Observatory Site Review, and takes into account the time available on a facility. It is the policy of the Observatory to not allocate all of the available observing time in any LST/GST range during any semester to a single proposal; this includes Joint Proposals. There are five different scheduling priorities:

Priority A This is the highest scheduling priority; the observations will almost certainly be scheduled. For the GBT and the VLBA, the observations will be considered for scheduling for up to two semesters. For the VLA, the observations will be considered for scheduling for two VLA configuration cycles5.
Priority B This is the next highest scheduling priority; the observations will be scheduled on a best effort basis. For the GBT and VLBA, the observations will only be considered for scheduling for one semester. For the VLA, the observations becomes ineligible when the associated configuration ends.
Priority C A lower scheduling priority; the observations are not guaranteed to be scheduled and are used to supplement gaps (e.g., filler time) in LST/GST ranges not filled by higher priority proposals. For the GBT and VLBA, the observations will only be considered for scheduling for one semester. For the VLA, the observations becomes ineligible when the associated configuration ends.
Priority D The lowest scheduling priority; the observations have a low probability of being scheduled and are used as secondary filler when higher scheduler priorities cannot be scheduled. For the GBT and VLBA, the observations will only be considered for scheduling for one semester. For the VLA, the observations becomes ineligible when the associated configuration ends.
Priority N The observations will not be scheduled.

The scheduling terms described above apply unless explicitly stated otherwise in a Disposition Letter. Disposition Letters may include a Scheduling Priority marked with an asterisks (∗; e.g., N∗); this is used to indicate that there are specific comments from the SRP, TAC, or Observatory Staff to the proposing team regarding the requested resources. Proposers should review the comments that are returned in the Disposition Letter before proceeding to observation preparation (Section 5.1). In the event that clarification is needed about technical comments, please submit a ticket to the NRAO Science Helpdesk and select the “VLA/GBT/VLBA Proposal Review” topic.

The outcome of the proposal review process is final. If an unfavorable disposition is received, the Observatory strongly encourages authors to prepare a new proposal for a future semester that addresses the returned comments. It is the policy of the Observatory to not allow communication between science reviewers and authors, even mediated through a third party such as the Observatory staff.

In case of questions about details in the Disposition Letter, please submit a ticket to request clarification through the NRAO Science Helpdesk and select the “VLA/GBT/VLBA Proposal Review” topic. However, in no case will such a request lead to a revision of the linear-rank or scheduling priority assigned to the proposal as a result of the review process.

 5 See the definition of a VLA configuration cycle in Section 7.1

4. Conflicts and Confidentiality in the Review Process

The review process relies on the scientific community to evaluate proposals based on their scientific merit and to make recommendations regarding time allocation. The process must be robust and free of real or perceived conflicts of interest and must maintain confidentiality. This section details conflicts of interest and confidentiality policies associated with the proposal evaluation and time allocation process.

For the purposes of the Observatory proposal evaluation and time allocation process, an individual is considered to have a potential conflict of interest if one or more of the following is true:

  1. They are a PI or a co-I on a proposal under consideration.

  2. They are affiliated with the same department as the PI or a co-I on a proposal under consideration. This may be relaxed depending on the nature of the relationship between the reviewer and PI/co-I (e.g., a large department with little interaction).

  3. They are a spouse, partner or other family member of a proposal author.

  4. They are a current or recent collaborator of a proposal author (within the last five years).

  5. They are a former student or advisor of a proposal author (within the last five years).

  6. They have any other reason to believe they cannot render a fair and impartial judgment on the scientific merit of the proposal.

In order to ensure that all proposals are treated fairly and without bias, such potential conflicts must be identified and declared. The response to a conflict depends on the role of the participant. The details are available online at https://science.nrao.edu/observing/proposal-types/coi.

5. Proposal Conflicts Policy

With the increasingly flexible capabilities being offered on Observatory facilities, there may more frequently be partial conflicts between proposals in terms of source(s), spectral line(s), array configuration (VLA), observing band, correlator/spectrometer set-up, and science goals. For proposals submitted on the semester submission deadlines, proposal conflicts are handled by the normal proposal evaluation and time allocation process. Proposals for DDT, however, may be submitted at any time and may conflict with a proposal that has already been approved through the standard time allocation process, or has previously been observed with the data still proprietary. Under these circumstances, for the overlapping science goals,

  • Any proposal that has been previously approved through the normal time allocation process with priority A or B, but has not yet been observed, will take precedence over the DDT proposal.

  • Proprietary data that have already been observed, including that for priority C and D projects, at the time of the DDT proposal submission will also take precedence over the DDT proposal.

The DDT proposal will be evaluated only on the part of its science goals that cannot be derived from the observations associated with the previously-approved proposal or previously observed proprietary data conflicted with the DDT proposal. If a DDT proposal is approved on these grounds, the DDT team may be given the option of either:

  • employing an observing set-up that does not conflict with the previously-approved or previously-observed proposals, or

  • to use the previously-approved proposal’s proposed set-up; in this case the conflicting/overlapping data will be made immediately available to the proposers of the previously-approved proposal.

The definition of the conflicting/overlapping data is defined in Appendix A.3, and the Observatory utilizes the source list and resources provided in the proposal to determine potential conflicts.

There may be cases where more than one DDT or ToO programs are triggered on the same object nearly simultaneously. Should such a situation arise, the NRAO Director or designee will take the final decision on which program will be observed. Typically, it will be based on the order of proposal submission, with priority given to the first received proposal. Other factors may be considered, including the requested observing frequency and the prevalent weather conditions.

Staff | Policies | Diversity