PIs demonstrate their success in training students to inventory biota. Their ladybug project was NSF funded and supports Criterion II.

CRITERION I (Intellectual Merit):

• Strengths: The proposal fell short of meeting the requirements of the Macrosystems Biology program. Connection to NEON databases combined with photographic data with taxonomic identification collected from additional sites are strengths of this proposal. Authors tie their proposal to the successful ladybug project that shows large scale inventories and monitoring by undergraduate students can be successful. This was good "proof of concept"?

• Weaknesses: Science questions with specific projects that test those questions were not well-developed. The proposal would have benefited from a discussion of how this program compares to other monitoring programs and how or whether it would partner with them. In addition, the proposal does not provide sufficient motivation for selecting the target species for monitoring Research questions and/or hypotheses need to be developed and tested with quantitative methods. For example, it was not clear how the design of data collection within sites will be used to address scientific questions with quantitative rigor. It was also unclear how the quality of the data (QC) will be controlled and who will use the data. Scaling was not well addressed nor was integration across disciplines. Reviewers suggest collaboration with atmospheric, hydrological or mathematical professionals.

CRITERION II (Broader Impacts):

• Strengths: The broader impacts of the project are strong. Specifically, the development of an infrastructure to support spatial and temporal data collection on biotic components of the environment over a broad spatial scale using graduate students, hundreds of undergraduates, > 150 high school teachers, and > 500 high school students. PIs and collaborators have extensive experience recruiting and training students. Leveraging collaboration with Discover Life is also noteworthy.

• Weaknesses: It is questionable if photo essays from Discover Life will show biotic changes over time. Dissemination of data was not clear. Discussion on products and dissemination of findings to scientific and lay groups needs to be further developed. There was also insufficient discussion regarding undergraduate training to ground the work. For example, there was no discussion of how their research efforts relate to this project and why such data collection is important.

SPECIFIC SOLICITATION CRITERIA

• Data Management and Access Plan
  
  • Strengths: The data management plan is expansive and detailed.
  • Weaknesses: It was unclear on who will be in charge of QC.

• Project Management Plan
  
  • Strengths: This plan is well conceived and supported by a logical administrative structure.
  • Weaknesses:

• Postdoctoral Mentoring Plan
  
  • N/A

SYNTHESIS AND RECOMMENDATION

The PIs propose to develop a species distribution database in NEON domains and additional sites using citizen groups. Authors state data collected at the various sites will be used to address a number of broad topics including temperature-driven plant, fungi, and animal phenology, inter-annual variation in pollinator abundance, and using lichens as bio-indicators of pollution and drought. Recommendations to strengthen this proposal include developing science questions with analyses. These were not clear or developed. In addition, discussion on how data will be used by scientists and others is needed.

The panel recommendation is: Not Competitive.

Reviewer #1

• This proposal primarily develops the physical and human infrastructure that is needed to remotely collect data via camera traps on the occurrence of species at 95 sites distributed across the temperate US, as well as at a few tropical sites (e.g., Hawaii, Panama, Costa Rica and Puerto Rico). A broad array of taxa, ranging from fungi and lichens, to flowering plants and slime molds, to flies, moths, and other insects will be the targets of investigation. The PI and co-PIs are well qualified to guide the project, and have assembled a team of science, education, outreach, and technology experts to assist with various activities. Moreover, the principals have demonstrated impressive results from prior NSF support.

• The data collected at the various sites will be used to address a number of broad topics such as (1) the way in which temperature drives phenology of plants, fungi, and animals; (2) relationships between alpha and beta diversity across various NEON domains for a number of different taxa; (3) inter-annual variation in pollinator abundance and disruptions of pollination synchrony; (4) the relationship between precipitation patterns and fruiting phenology in selected mushroom species; (5) vegetation associations and effects of climatic variation on myxomycete species; and (6) lichens as bioindicators of pollution and drought.

• Although it is clear that the collected data will pertain to these large scientific issues, the proposal does not make clear how the design of data collection within sites will be used to address such questions with quantitative rigor. Similarly, it provides very little insight about the actual analyses (statistical tests, simulation approaches, or modeling) that will be executed or the challenges that might arise (e.g., biases, psuedoreplication, phylogenetic non-independence, or spatial autocorrelation) from the use of data collected from camera traps for this purpose. Consequently, it is unclear if predictive understanding will result from this grand collection scheme, at least based on the exposition in the proposal. Moreover, measurements of the drivers of biotic change are often at scales that are inappropriate or un-informative for understanding spatial and temporal dynamics of species presences (and absences). Intimate association of these kinds of environmental data (measured and estimated from models) would have improved the scientific rigor of the research and likely expanded the disciplinary ramifications of the project to expand far outside the domain of biology.

• The broader impacts of the project are immense in terms of developing an infrastructure to support broad scale data collection on biotic components of the environment over broad expanses of space and over time. The project will support 3 graduate students per year. In addition, the five-year project will support hundreds of undergraduates, over a 150 high school teachers, and 500 high school students. Species identification will be facilitated via the production of field guides and videos; these will also stimulate local interests in biodiversity and natural history.

• The project management plan is well conceived and supported by a logical administrative structure. Nonetheless the amount of communication and coordination to effectively lead such a project are considerable. Biosketches for some of the senior technical and outreach staff, especially those substantively supported by the project budget, would have been usefully included in the proposal.

• The data management plan is clear, expansive, sufficiently detailed, and appropriate in scope for a project of this nature. This is a strength of the proposal.

ADDITIONAL REVIEW CRITERIA FOR MACROSYSTEMS BIOLOGY

• The project does not cast the research in the framework of theory and does not produce novel approaches for advancing a theoretical framework for predictive understanding?

• The project team represents a number of sub-disciplines in biology, but does not include atmospheric, hydrological or mathematical perspectives, so it is not as multidisciplinary as most projects. Ultimately, the utility of the collected data will come from its integration with other types of data, so it was disappointing to see a narrow bio-centric focus.

• The project involves recruitment, education, and training of a workforce related to basic research on regional to continental scale biology.

Reviewer #2

What is the intellectual merit of the proposed activity?

• Asking for funding of a large, citizen-science (but paid) project that may be useful to answer several questions that might be macrosystems biology. It could add a component that is not addressed very well in the NEON plans by increasing the geographic scope of organismal biology. There is very little in this proposal that addresses ecological theory beyond these questions, particularly the development of macrosystems biology theory.

• All the taxonomic work will be on terrestrial macroorganisms from large, interesting insects and lichens to large plants and animals but especially focused on moths, plants, mushrooms. This focus misses an enormous amount of biodiversity in aquatic systems, in the microbiota, and in rare animals and plants that are not likely to be found by photographers.

• The team is not planning to "write traditional publications per se, but to provide researchers, land managers, policy makers and the general public with up-to-date, high quality biological data." Well thought out selection of field sites for the work. Already established positive connections with most of them. The team has also gathered a good group of taxonomic specialists for identifying difficult specimens.

• No indirect costs! But, they ask for $250,000 for administration, $300,000 for an outreach coordinator, $2 million to pay stipends for the undergraduates, a small amount for "essential business services" and OMB auditing requirements, over $1 million for undergraduates room and board, and a half million for "Contingency support funds." All told, they are asking for nearly 40% of the program's total projected budget to mobilize an army of undergraduates to photograph and identify as many animals and plants as they can in 95 sites, mostly near biological field stations.

• Discover Life of the Polistes Foundation seems to be conducting an excellent program of public education about biodiversity and organismal biology. I enjoyed looking over the web site and am glad to learn about the program.

What are the broader impacts of the proposed activity?

• This proposal is mostly "broader impacts," with education of high-school students and postdocs one of the primary objectives, and the scientific questions emphasized a bit less.

Summary Statement

• This project is not appropriate for Macrosystems Biology funding, but should seek funds from private foundations or perhaps educational directorates.

Reviewer #3

• This proposal aims to develop a species distribution database in NEON domains and additional sites. The structure and organization of this proposal undermines it's potential. Although the goals are described, this reviewer had difficulty finding a compelling reason for the project. There is insufficient discussion regarding key questions, target species, and undergraduate training to ground the work. For example, there is reference to undergraduates engaging in independent research as well as inventory activities but there is no discussion of the research activities, nor is there a connection made between their research and the larger reason this sort of data collection is important. As a consequence, the undergraduate component, which is highlighted as fundamentally important, reads as if it is little more than a science "jobs program." Both of the primary aims: developing the database and developing a significant undergraduate research program are insufficiently described and evidenced in terms of need, impact, and longevity.

What are the broader impacts of the proposed activity?

• The PIs and collaborators have significant experience in recruiting and training students, particularly those from underrepresented groups. The project would likely have a positive impact on those groups.

Summary Statement

• This project has an excellent network of collaborators relative to all the practical aspects of the project. However, the underlying need and potential impact are poorly described.

Reviewer #4

What is the intellectual merit of the proposed activity?

• This is a well-written and well-organized proposal that aims to establish a network of individual groups (largely made up of students and teachers) that will inventory various plant and insect species across a number of field sites.

• Strengths: The PIs have obviously developed detailed proposals for conducting this work and appear capable of getting this large project off the ground (although, I think the scope could be narrowed considerably). This is an excellent teaching and community outreach project.

• Weakness: I really wanted to see well thought out science questions with specific projects that test those questions. How will this ecological monitoring advance the way we think about ecology? How is it different from the already existing monitoring system (NEON) that NSF is putting in place? Perhaps an educational or more applied program would be a better fit?

• There is not enough information on how the data will be collected and used. Data collection will drive how the data can be analyzed. Will all of these things be standardized across sites? Who will use the data and will it be in a format that is usable by researchers? Will the data be collected in such a way that cross-site analysis will be possible? Who will be in charge of analyzing the weather station data with the phenology data?

A few other things:

1. This proposal seems awfully ambitious in scope. Why not just focus on the temperate or tropical sites with explicit questions that will be asked? The size of this project gives me pause about its ability to be conducted.

2. There needs to be more discussion about what models the PIs plan on using to understand how climate affects phenology etc across their sites.

3. How will the quality of the data (QC) be controlled and who will be in charge of QC?

4. Who will use the data and how will they use it?

5. Fungal fruiting bodies are a poor predictor of fungal presence.

6. Where are the weather stations you propose to use and who is in charge of those data? How close are the weather stations to the collection sites (page 12)?

7. Who will be conducting the data analysis and who will write up these papers?

8. I didn't see the letters of support from the other field stations.

9. Your budget pages are incorrect.

What are the broader impacts of the proposed activity?

• Broader impacts are a strength of this proposal. The PIs will use teams of teachers, high school students, and undergraduates to achieve their objectives.

Summary Statement

• While very well written, this proposal needs more developed science questions. In addition, it needs to have a better-developed data collection and analysis section with explicit examples of how the collected data will be used by scientists. Scaling back the scope of the project to focus on more defined science questions would make this proposal more interesting.

• Broader impacts are clearly a strength of this proposal as they are integrated throughout.