1. NASA MSFC Mission Operations Laboratory MSFC NASA MSFC Mission Operations Laboratory Kevin Kasperitis EO10 (256) 961-1053 Kevin.Kasperitis@nasa.gov OC Review Processes and Knowledge Management OC 205

2. NASA MSFC Mission Operations Laboratory MSFC Page 2 Course Objectives The Operations Controller (OC) Knowledge Management and Review Processes course provides an overview of OC knowledge sharing standards and consistent product review requirements within the team and with other POIC Cadre. The material is designed to provide a top- level introduction to non-console related preparation checklists and knowledge transfer techniques. Product review is addressed through discussion with other Operations Controllers, mentoring, and by completing the OC training flow. Upon completion of this course, the trainee should acquire an understanding of: –OC product review checklists and processes Technical review standards –OC knowledge management best practices Data sharing within the team Handover and back-up reviewers –OC Team consistency standards for: Multi-level reviews Multiple reviewer coordination –OC interface with operations product developers

3. NASA MSFC Mission Operations Laboratory MSFC Page 3 Checklists & Process Basics The OC Team is responsible for reviewing Operations Products that are used to support real-time knowledge for the entire team. The typical Operations Products development process is complex and the template for review and approval of numerous products prohibits a detailed review by all OCs for all products. –All OCs are assigned a list of payloads for which they are the primary product reviewers. –OCs, by virtue of certification and training, are uniquely qualified to identify product specific operational elements, requirements, and constraints that affect Payload and ISS system-to-system interfaces. –Consistent payload assignments within the OC Team allows individual OCs to integrate payload specific knowledge across multiple product reviews. –The OC team uses checklists as a team process to provide a standard level of review by all OCs. –The OC ability to review for operations feasibly is dynamic, knowledge and experience based and comprehensive enough to review beyond checklist line items when products have unique operating constraints. –The review checklists presented on the next slides were developed to encompass a holistic representation of OC real-time operations knowledge and perspective. The checklists outline basic and common review elements required by all OC’s during product evaluations.

4. NASA MSFC Mission Operations Laboratory MSFC Page 4 Checklists How to use a checklist to review a product? The OC Team goal is to ensure operational usability, increase execution success, and ensure comprehensive details support final product effectiveness. Remember – first and foremost – As a Certified OC: You review operations products so that baseline products will satisfy the needs of all OCs responsible for real-time operations. –Read and evaluate the details of products or documents for: Completeness Usability / Operations Feasibility / Safety Consistency with other operations products Adherence to systems engineering principles –Mark a checklist when an item is clearly explained and is pertinent to OC knowledge requirements –Identify any checklist item that is Not Applicable (N/A) –Identify checklist items that are missing, not fully explained or create operational questions. (Leave blank or mark with a “?” for follow-up). –Ask document owner/author questions, before submitting comments, to clarify missing essential elements (i.e. anything that is not obviously an N/A item) –Officially submit comments, according to baseline/review processes, to: Request additional information Add clarifying words Improve readability / usability Ask questions

5. NASA MSFC Mission Operations Laboratory MSFC Page 5 The OC team uses checklists developed and reviewed by the entire OC Team for training products and safety data packages. As an essential element to new OC training we systematically teach timeline and procedure review processes. The checklists and processes discussed in this presentation include: Training Product review checklists apply to: –Ops Lead and PD produced Self Study courses / Ops Summaries –Other Discipline Specific Training Products Safety Data Package review checklists apply to: –Published Safety Data Packages (prior to product approval) –Hazard Control documents and databases Timeline Review Process Overview Procedure Review Process Overview Checklists

6. NASA MSFC Mission Operations Laboratory MSFC Page 6 Training Product Review Checklist Training Products are available for reference during payload operations and all OCs should habitually review published training products when preparing for console operations. The content of all training products should provide enough detail to understand the basic operations and interfaces required to effectively *conduct, collect and transfer science to the end users. (*Note: During real-time operations one of the OC’s primary responsibilities is to execute payload operations that conduct, collect and transfer science to the end users.) Training Product Review Checklist Operations Concept Science Objectives Science Principles Science return – Samples, Data, Photos, etc. General Narrative of Operations – Known/Expected points. Planned Operations Crew Activities – Summary of Ops Ground Activities – Summary of Impacts to Crew Ops / Data Sequence of Activities (End to End) Hardware Concept Payload location Photographs / Diagrams Use of Station Support Equipment Stowage Constraints (Payload / Parts / Accessories)

7. NASA MSFC Mission Operations Laboratory MSFC Page 7 Training Product Review Checklist (Continued) Visiting Vehicle Transport / Transfer Transfer Constraints Special Tools for Transfer/Installation/Stowage Power Requirements Temp Status / Conditions Rack Interfaces Basic Operations Activities Crew / Ground Interaction Interaction with Other Payloads or Facilities Voice / Comm. / Data choreography Execution / Ops Notes Other data (linked from the CBT) Consumables / Re-supply Resources / Operations conditions Power requirements / limits Thermal Air Circulation - Front / Rear Breather MTL / LTL Thermal Protection Contingency for changes in Thermal Status Uses for VRS/VES - Materials evacuated / Timing in Ops Flow Uses for Nitrogen (N2)

8. NASA MSFC Mission Operations Laboratory MSFC Page 8 Training Product Review Checklist Command and Data Payload Software Shared Software Ground Commanding (Who / When / Where) Data Collection / Storage / Transfer Data Flow Path (ExPRESS / Ethernet / Low Rate) to the ground Health and Status Data Piece of PL hardware that generate data Critical or Hazardous Commanding Photo / Video Requirements Video Routing Requirements Operations Constraints (Ensure training considers the following) Physical Constraints Guidelines / PL Regs / Flight Rules (summary) Hazards (and controls) Contingency Operations (Crew or Ground) Planned Contingencies / Malfunctions / Maintenance Emergency Caution / Warning Parameters Monitoring Expected Responses Safety Data – Summary of Safety and Controlled Hazards Ensure training considers / addresses past anomalies and documented resolutions

9. NASA MSFC Mission Operations Laboratory MSFC Page 9 Safety Product Review Checklist Safety Data Packages are available for reference during payload operations and all OCs should habitually review published data packages when preparing for console operations. In addition, the Safety Hazard documents and databases (PHCM/HCL/LISA Tables/etc.) are required references during real-time operations. The content of safety data products should provide enough detail to understand all hazard controls or hazardous materials. A basic understanding of the hardware design and required controls – including hazardous material containment – should be clear before the product is approved for OC Team Training. (Note: During real-time operations one of the OC’s primary responsibilities is to ensure that safety controls are comprehensive for payload operations that conduct, collect and transfer science to the end users.) Safety Data Package Review Checklist General Description Operations Concept / Interfaces Activation and Checkout Nominal Operations Off-Nominal Operations Malfunctions Software / GUI

10. NASA MSFC Mission Operations Laboratory MSFC Page 10 Safety Product Review Checklist Structural Interfaces Power System Description Nominal Peak Batteries Circuit Protection Power Sequence Vacuum Requirements N2 Requirements Thermal / Cooling Requirements MTL LTL Air Circulation Front / Rear Breather Hazardous Commanding Commanding Details Hazard Controls Pertinent Types (Shock, Containment, Sharp Edges) Support Equipment (i.e. Gloves/Power connections) Procedures Science Products Processed Samples D/L Data / Telemetry D/L Video Video Tape Stowage / Containment

11. NASA MSFC Mission Operations Laboratory MSFC Page 11 Timeline Review Process Overview Timeline reviews require the sum of all OC knowledge and experience. All the product review efforts, skills and experience are integrated and essential in the timeline review process. Timelines are evaluated for operational feasibility, resource management, product readiness and adherence to rules, regulations and safety. Reviews are conducted frequently and for a variety of reasons: -Increment timelines (OOS - ref POH Volume 1) -Preparation for Console (WLP/STP/OSTP – reference POH Volume 2) -Execution and approval for operations (E-7, E-3, E-1, etc. reference the FCOH) The process for reviewing a timeline can be as dynamic as the timelines themselves. Although the planning details are vetted and incorporated with the best situational awareness and operations standards known to the planning community; operations feasibility may be impacted by unforeseen details including: –Crew requests / unexpected conditions (delays, anomalies, etc.) –Safe / Unsafe environments –Plan changes due to operational events or anomalies –Unexpected change requests from various sources (JSC/ESA/JAXA/etc.)

12. NASA MSFC Mission Operations Laboratory MSFC Page 12 Timeline Review Process Overview There are multiple timeline review methods and no single method constitutes an OC process or is appropriate for all timelines. All OCs are encouraged to find / create / mimic a system for comprehensive timeline reviews that works for them in the most efficient and effective way. Timeline reviews are not considered comprehensive if the sum of all OC responsibilities is not considered during the review – this is what we do – execute timelines! The following example constitutes one review process flow: Identify Crew (or Ground) Activity or Sequence Verify Existence or Accuracy of Ops Product *Details Identify any associated activities further down the timeline (i.e. other crew or ground activity) Review all applicable Flight Rules, Payload Regulations, and Safety Data *Procedures, Execute / Ops Notes, associated attachments, messages, support products (i.e. stowage notes) Move to next Crew (or Ground) Activity or Sequence until all activities are reviewed 2 1 3 4 3A

13. NASA MSFC Mission Operations Laboratory MSFC Page 13 Timeline Review Process Overview 1.Start at the top* of the timeline, review an activity (or sequence), and ensure operations details are accurate and complete. a)All available TPS and PPO Reports (for all payloads) should be reviewed prior to a timeline review. Considering these reports first will answer many questions and ensure that planning models are accurately and completely represented and valid (PPO) and that previous reviews, planning deviations & added situational awareness is considered (TPS). b)Review execution / operations notes to identify unique actions and situational awareness c)Review procedures and ensure the ops steps are feasible for the timeline environment d)Identify all required resources during the procedure review ( Pwr/Data/Video/Water/Air/Vacuum/etc.) e)Ensure that every identified resource has a corresponding activity in the timeline then review as a connected activity or as part of the required sequence of activities. 2.Check for adherence to Flight Rules, Payload Regulations, Safety Controls as part of the procedure, activity or sequence review. 3.Exhaust the review of an activity by identifying all known connections to other activities (i.e. Ground Commanding, resource bands identified, etc.) 4.Move to the next activity. 5.Review systems activities for impacts to Payload Operations. Note: *Top is relative because all timelines can be arranged according to user preferences. The process assumes you will start with crew activities and the procedure / execution note / sequence details will connect to other activities and lead to a cyclical review – or – the review stops when all connections are reviewed. PPOs or Operations Plans (i.e. PRO PLANS) should be consulted to ensure completeness of scheduled activities – identify any items NOT in the plan that are not obviously missing. When all crew activities are reviewed then all ground activities (not associated with crew actions) should likewise be reviewed. Eventually all Payload activities will be reviewed if the reviewer proceeds from top-to- bottom and left-to-right until the review period is exhausted. If flight rules, payload regulations, required resources, and safety data are inherently considered, verified, validated, etc. during the review AND the OC understands the content and requirement for EVERY ACTIVITY on the timeline the review is complete.

14. NASA MSFC Mission Operations Laboratory MSFC Page 14 Another Example of a Timeline Review Process (Based on the POH Vol. 2 SOP Requirements)

15. NASA MSFC Mission Operations Laboratory MSFC Page 15 Crew Procedure Review Process Overview There are multiple procedure review methods and no single method constitutes an OC process or is appropriate for all crew procedure reviews. All OCs are encouraged to find / create / mimic a system for comprehensive crew procedure reviews that works for them in the most efficient and effective way. Crew procedure reviews are not considered comprehensive if the sum of all OC knowledge and responsibilities is not considered during the review. The basics of crew procedure reviews are similar to the timeline review requirements the process flow is top to bottom with requirements for cyclical checks and validations. Read procedure for objectives and usability Validate ‘Operational Feasibility’ Step-by- Step Identify any missing resources, safety hazards. Apply logic for resources management and any applicable (ODF) data standards, Safety Controls, Rules/Regulations Move to the next step. 2 1 3 4 5

16. NASA MSFC Mission Operations Laboratory MSFC Page 16 Knowledge Management What is It? Knowledge management (KM) is the process of capturing, developing, sharing, and effectively using organizational knowledge. It refers to a multi- disciplined approach to achieving organizational objectives by making the best use of knowledge Information management, on the other hand, is mainly concerned with people managing information sources, that is, auditing them, acquiring and storing them for easy retrieval and dissemination of information. The Operations Controller (OC) Knowledge Management and Review Process goals include: Develop consistency between OCs Share knowledge within the team and with other disciplines Data sharing within the team Handover and back-up reviewers

17. NASA MSFC Mission Operations Laboratory MSFC Page 17 On console, we transfer information and experience in one-on-one shift handovers, log entries, specific handover worksheets and through personal discussion. The ultimate goal is to create a seamless transition from shift to shift so qualified OC’s can continue operations without interruption in data, resources, and knowledge. In the office environment we rely on assignment lists to designate primary reviewers that follow connected products through the development life cycle and baseline process. A primary OC reviewer uses knowledge and experience to gather and understand information as well as manage the transfer of information from meetings, reviews or other events. Ultimately, we hand over the benefit of our efforts to the entire team through comments incorporated into usable operations products. The published operational products are eventually assigned to the entire OC team and a qualified OC perspective should have been considered in the development process. Notably … it is not possible for all OC’s to review all documents prior to baseline and publication templates. All OC’s eventually have supported the production of finish products that meet the needs of the team. Individual efforts may affect final products but all reviews are conducted by qualified OCs. Handover and Backup Reviewers

18. NASA MSFC Mission Operations Laboratory MSFC Page 18 All certified OC’s realize that handover information is a sharing of both knowledge and experience. As a Team we are asked to produce comprehensive meeting notes, trip reports, and status updates for special assignments. The following best practices are required: -Support ALL product development reviews -Present standard meeting briefings, tiger team statuses, and trip reports at the weekly OC Team Meeting -Consult Increment Lead OC’s for generic questions before commenting to documents -Coordinate comments between OC’s when multiple reviewers are assigned and submit one team input -Expedite reviews of large documents or short template deadlines with a sub-team/round table of OC’s -Include OCiTs (OC’s in Training) in the review, whenever possible, as a mentor and training function Data Sharing Within the Team

19. NASA MSFC Mission Operations Laboratory MSFC Page 19 KM External to the Team The OC Team creates very few operational products. We do however review almost all products and processes used to create operational products. We balance Console Operations with Office Tasks -Flight Control obligations are our first priority but are generally balanced with time and tasks in the office. -We use team leadership to assign and divide tasks based on OC inherent interests, skills, and experience. -We rely on team leadership to provide back-up resources when tasking overlaps and creates conflict in daily work loads. (e.g. console shifts, product reviews, special assignments, and tiger teams). -Some, but not all, OC tasks are: -shared by two primary OCs -or assigned to a primary and back-up OC

20. NASA MSFC Mission Operations Laboratory MSFC Page 20 Primary – Backup OC’s The OC Team Supports large efforts with duplicate and sometimes multiple OC’s. Some examples of large tasks requiring more than one OC include: Designated Increment Lead OC’s Visiting Vehicle Process and Product Support Payload Regulation and Flight Rule development, review and analysis SharePoint/Website content maintenance and development Handbook and Console Tools Maintenance Console Transition Activities Major Payloads or Payload Facilities –External Payloads –HRF / MSG type facilities with multiple payloads –Special case payloads (e.g. Nanoracks) Training: –Proficiency Training –SME Training (e.g. Visiting Vehicle) –Timeline Review / Log Training

21. NASA MSFC Mission Operations Laboratory MSFC Page 21 Primary Assignments (with no designated Backup OC) The OC Team typically assigns one OC per payload. Assigned OC’s will review all products and processes through development. Sometimes, conflicts happen and we use the team resources as a flexible back-up system. As an example, the following flow demonstrates the OC process for product review, meeting support, or document development. The highlight of the flow demonstrated the team mechanism for backup or alternate reviewers: OC is assigned a Payload by OC Sub Team Lead (OC Payload Assignment Matrix updated by Sub-Team Lead) POA Team Lead notified when products are assigned for review (and response) OC Reviews product(s) and supporting material and maintains notes or defers to forum notes OC – notified of review OC notifies Team Lead, NASA Lead, Sub-Team Lead of completed review (usually done as a cc: to official response) OC notifies Team Lead, NASA Lead, Sub-Team Lead of completed review (usually done as a cc: to official response) OC reviews and responds - by the due date - to product review requests (on behalf of Team Lead) Alternate OC assigned to review product for another responsible OC Alternate OC retrieves and reviews supporting material including OC specific notes and hands over (verbally if possible) a status report from the OC assigned to the Payload. OC Review Complete Alternate OC reviews product and responds to the review request (on behalf of Team Lead – AND – assigned OC)