SEACOOS External Evaluation Evaluation held: January 17-19, 2005 St. Petersburg, FL Response drafted: July 2005 Panel Members: Mary Altalo, SAIC, Chair [governance] Carolyn Thoroughgood, U. Delaware [E&E] Eric Lindstrom, NASA [Fed relations] Steve Hankin, NOAA [IM] Wendell Brown, U. Massachusetts [obs and modeling]

Preparations Two binders of material developed for evaluation, presented over 2 days to panel –Binder 1: supporting information for presentations (background and WG docs) –Binder 2: proposals and reports yrs 1-3. The evaluation reflects not only the evaluators’ perceptions of SEACOOS and its merits and deficiencies, but also the ability of SEACOOS to concisely convey what the program is about

This review was requested by the SEACOOS The findings are guidance for the “fine tuning” of the future direction of the program (37 recommendations/findings) Format: panel, response General findings: SEACOOS unsure of its “Mission”. SEACOOS mission has evolved as IOOS has become better defined, and as SECOORA was initiated and began to identify its mission. SEACOOS is a research effort to better understand what will be required of an RCOOS.

Confusion about the distinction between SECOORA and SEACOOS -The evaluation has helped SEACOOS recognize role as a research effort prototyping various technical elements of the RCOOS. -It is not our intention that SEACOOS assume the responsibilities of a RA. -Has provided a foundation for some essential SECOORA elements, e.g. RCOOS, data management hub, spin-up an Educational & Extension activity for SECOORA. -Financial strategies, full user characterizations, and long-term strategic planning are crucial activities for the RA to pursue.

Having technology goals, research goals and societal goals requires multiple sets of performance metrics... needs to be requirements-driven system…need ConOPS and design plans and plans for transitioning to operations. Need systems engineering approach for rigor, accountability (not fiscal) and demonstrable design and build targets Recurring comment about the need for a systems engineering approach Begun to investigate…relevant and will be carefully considered. Because performance metrics, enumeration of user needs, credible cost estimates can be captured within systems engineering development we will pursue this approach over the next few years.

Need “credible” estimate of what this operational system, fully scoped and transitioned to meet diverse user needs will cost. Fear operational costs are significantly undervalued We agree - critical statement. COOS is significantly undervalued. Know individual SEACOOS team member’s costs, need more complete cost analysis. Suspect OCEAN.US and USCOP figures are not realistic

Governance and planning Need rigorous long term strategic plan, reviewed semiannually, because the sponsorship is changing and evolving rapidly Agree but SEACOOS has already spent considerable time on a strategic plan and an implementation plan…SEACOOS efforts are starting points for similar planning efforts within SECOORA. Need to think about a long-term vision for transition to an operational program, within the context of the RA

Conform to project management software – possibly with the help of a “translator” – create action items and issues list weekly. Celoxis…we have yet to identify a suitable project management application. Observations and Modeling SEACOOS has established three HR radar evaluation sites - duplicative evaluation effort? HF radar is being strategically implemented, with appropriate staging and without ”duplication”. Purchased two types of systems, evaluate their performance independently. Able to identify strengths and weaknesses of the different systems before decisions on how to cover the southeast.

SEACOOS array designed to serve physical oceanographic researchers - need to identify a significant non-“super user” demonstrate how their needs influence the evolution of the system. The current configuration reflects history more than planning. Significant engagement of casual information users requires development of highly tailored products. -Products require focus groups and testing. -Effort is outside the realm of the SEACOOS development effort, encroaches on private sector. -Focus instead on super-users. Teams will bring together multidisciplinary and multiagency reps to approach a few well defined societal problems.

Why three models instead of one regional? Need some SEACOOS-wide modeling efforts. One model not sufficient, need an ensemble. Starting point was three subregional models already in existence, but this was never conceived as being our end state model design. Implement and run a prototype near real-time application of a single, full-region prognostic circulation model, with prototype open boundary conditions and some data assimilation. Agree. Challenging goal and will require partnerships with well thought out collaborations. Such partnerships are presently being pursued along at least two different fronts.

Use Coastal Ocean Data Assimilation Experiments (CODAE) to help scientifically guide future deployment of observational resources. Use model skill to inform the optimal configuration of the observing system design. CODAE is an important activity, in which we must engage. However, even without a CODAE, we can design many of the observing system attribute Data Management SEACOOS should identify clear target user groups and design the next generation of information management system to meet those requirements Primary emphasis to date on “aggregation” and the identification of necessary standards and protocols to accomplish that aggregation

The new “Teams” approach will help SEACOOS member scientists should be viewed as one of the primary test user groups of IM system Need to do this. SEACOOS IM standards drafted are of high quality, yet must engage in community-wide efforts to define data standards. Become an active participant in the standards bodies that are responsible for emerging community standards Consistent with our near-term planning for IM The regional IM should enable the creation of products but not be the authoritative creator of the products. Need to distinguish between the roles of the value-added information provider vs. regional data provider.

Distinction is not always clear. Need to develop a strategy and guidelines. Best done with SECOORA, soon. Requires ongoing dialogue with industry partners. In next phase of development IM should address its more limited successes as a distributor of (“raw”) data. Enhanced access to data is a priority for the coming project year. Need to identify appropriate data formats and protocols Embrace a 4-dimensional view of data Will do, in coordination with the Modeling and Observations WGs On-line documentation of GIS portal is of extraordinary quality and thoroughness. Apply same to data access Will do.

In SEACOOS archives the responsibility of individual data providers. Reconsider. Foster cooperative relationship with an IOOS archive facility. A regional IM system should be a conduit to long term archival ensuring the integrity and consistency of the data streams as they pass into the archive. complex issue…. Federal Partners SEACOOS temporary. Develop a forward-looking budget and plan for sustaining an approved SE Atlantic integrated observing system. Negotiation must begin immediately with Federal agencies for sustained funding in the 2007 budget and beyond. SECOORA is the appropriate body to engage in this discussion. Summer 2005 Joint SEACOOS/SECOORA Workshop

The common interests of SEACOOS and federal agencies should be documented element-by element throughout the SEACOOS project. Present list at annual IOOS Development plan conference and bring relevant federal. This practice should continue on an annual basis. Novel and will be pursued SEACOOS needs to clearly identify itself as a research activity with the Ocean.US organizational rubric and develop a roadmap for moving through the further stages of development. Adopt a more rigorous system engineering approach We have begun to investigate how this might be implemented in the next phase of the program

Extension and Education Engage extension personnel and an external advisory group to define the roles, responsibilities, and costs for university-based extension activities vs. an incorporated RA or the private sector. A task for SEACOOS or SECOORA? A joint SEACOOS/SECOORA work group could undertake this task. Professionals with expertise and skills in information/data management and technology transfer to prospective users must be entrained. Will explore new skill sets and team approaches to user engagement to augment traditional Sea Grant Extension audiences - coastal management, fisheries, coastal natural hazards, and K-12 education

Prepare a detailed inventory of specific regional and state governmental entities and business and industry clientele in the SEACOOS domain Inventory begun by Sandy in her SEACOOS user characterization report, continues with SECOORA. E&E has pursued a number of way to do this: collaborative workshops; targeted super-user meetings; market analyses. (Did they not see this?) Where major user needs are not addressed by the SE observing system the extension effort should broker necessary contacts to address high priority needs. Novel use of Extension? Develop specific educational objectives and defined target audiences and develop a funding plan that ensures comprehensive education program

First objective is development of OOS awareness among educators. Target audience is K-16 teachers and instructors, aligns with COSEEs and work plans of Sea Grant marine education specialists in the region. Definition of the Education programming is too limited considering the broad societal mandate for ocean observing. Educational programming for future business managers/leaders to familiarize them with the potential benefits of incorporating environmental information into business planning and operations should be developed. An excellent suggestion, but presupposes vastly increased funding.