Presentation is loading. Please wait.

Presentation is loading. Please wait.

National Fighter Procurement Secretariat Secrétariat national d’approvisionnement en chasseurs Industry Engagement Request Questionnaire 1 Assessment Methodology.

Similar presentations


Presentation on theme: "National Fighter Procurement Secretariat Secrétariat national d’approvisionnement en chasseurs Industry Engagement Request Questionnaire 1 Assessment Methodology."— Presentation transcript:

1 National Fighter Procurement Secretariat Secrétariat national d’approvisionnement en chasseurs Industry Engagement Request Questionnaire 1 Assessment Methodology Meeting with Companies May 03, 2013

2 2 To provide companies with details on the assessment methodology that will be used in assessing your responses to Questionnaire 1 To provide clarifications and provide answers to questions To provide companies with information on next steps We are committed to an open, fair and transparent process Purpose

3 Industry Engagement Request Questionnaire 1 Canadian fighter mission and threat information across two time horizons ( and 2030+). Responses to include defined mission configurations: – Air-to-Air; – Air-to-Ground; and – Air-to-Surface. Responses to include capability developments (planned upgrades). Responses to include descriptions of capability, production and supportability. 3

4 PRINCIPLES The following Principles will guide all participants involved in the assessment of the responses: Consistency – Team leads will ensure that the assessment is well documented, including all qualitative and professional military judgement aspects. – Individual aircraft are being assessed against each Canada First Defense Strategy (CFDS) mission. – No aircraft will be screened out. All options will remain on the table. Final product will be a high level risk assessment. Objective – The National Fighter Procurement Secretariat will provide oversight through all steps of the assessment process Use of Third Parties – Responses will be shared with the Independent Review Panel (i.e. Panel) members and any significant differences of opinion between assessment teams will be brought to the attention of the Panel. 4

5 Overview of Q1 Assessment The “Capabilities” questions of the Industry Engagement Request will be assessed using a 3-step assessment process: – Step 1 – Assessment of Measures of Performance (MOP); – Step 2 – Assessment of Measures of Effectiveness (MOE); and – Step 3 – Assessment of Aerospace Capabilities weighted against Canada First Defense Strategy (CFDS) Mission and criticality to result in “Operational Risk” assessment of each aircraft for each of the missions. The “Production and Supportability” questions will be assessed using a 2- step assessment process: – Step 1 – Assessment of responses by technical Subject Matter Experts teams on Military Strategic Assessment Factors – Impact Statements; and – Step 2 – Assessment of responses by the Royal Canadian Air Force Senior leadership level from a missions perspective to result in “Strategic Risk” assessment of each aircraft for each of the missions. Both the “Operational Risk” and “Strategic Risk” will then be consolidated into an overall risk profile for each aircraft against each mission. 5

6 Step 1 of Capabilities Assessment Assessments being performed using responses to the 17 areas of « Capabilities » contained in the Industry Engagement Request. National Defence teams assessing specific areas, with oversight from the National Fighter Procurement Secretariat. Qualitative assessment - independent of missions. Results in Measures of Performance assessment for each aircraft. 6 SensorsRadio Frequency Sensors Electro-optical or Infrared Sensors Air-to-Air Weapons WeaponsAir-to-Ground Weapons Air-to-Surface Weapons Non-Kinetic Weapons (Electronic Attack) Self Protection Radio Frequency Self Protection Infrared Self Protection Countermeasures AvionicsData Link Pilot Workload Communications Sensor Integration PerformanceRadio Frequency Signature Infrared Signature Engine/Airframe Combat Radius/Endurance

7 Evaluation Grid for Step 1 Performance rating of individual systems Rating scale from 0-10 to allow sufficient fidelity Classified systems information represented by a number (supported by classified references) RatingDefinition 10 Excellent Without appreciable deficiencies 9 8 Very Good Limited only by minor deficiencies 7 6 Good Limited by moderate deficiencies 5 4 Poor Limited by major deficiencies 3 2 Very Poor Significantly limited by major deficiencies 1 0 Non-Existant 7

8 Step 2 of Capabilities Assessment Three Royal Canadian Air Force teams (four members each) performing this assessment, with oversight from the National Fighter Procurement Secretariat: Including fighter pilots and at least one Air Force Air Weapon Controller on each team. Significant differences of opinion between assessment teams will be brought to the attention of the Panel. The following inputs will be used in this assessment: Scores and input from the results of Step 1; Companies responses to the Industry Engagement Request, including the mission configuration systems; Vignettes detailed in Appendix “A” of the Industry Engagement Request; Detailed threat capabilities from Canadian Forces Threat Assessment; Detailed fighter task lists; and Other information available to Government. Fighter Measures of Effectiveness areas of consideration detailed in the Industry Engagement Request. 8

9 Information on Missions as per the Canada First Defence Strategy (CFDS) 1 – Conduct daily domestic and continental operations, including in the Arctic and through NORAD (Vignette 1) 2 – Support a major international event in Canada, such as the 2010 Olympics (Vignette 2) 3 – Respond to a major terrorist attack (9/11-type scenario under CFDS Mission 1 through NORAD) (Vignette 5) 4 – Support civilian authorities during a crisis in Canada such as a natural disaster (No Vignette) 5 – Lead and/or conduct a major international operation for an extended period [including complex peace enforcement operation, and coalition state-on-state war- fighting] (Vignettes 3 & 4) 6 – Deploy forces in response to crises elsewhere in the world for shorter periods (Vignette 6) Of the 6 Canada First Defense Strategy missions above, only 5 relate to the 6 vignettes. Mission 4 (Support civilian authorities during a crisis in Canada such as a natural disaster) will be analyzed together with Mission 2, understanding the fighter’s role in this mission is generally minor. 9

10 Information on Aerospace Capabilities 1.Defensive Counter Air (DCA) 2.Offensive Counter Air (OCA) 3.Strategic Attack 4.Close Air Support (CAS) 5.Land Strike 6.Tactical Support to Maritime Operations (TASMO) 7.Intelligence, Surveillance, and Reconnaissance (ISR) 10

11 Information on Fighter Measures of Effectiveness 1.Awareness 2.Survivability 3.Reach and Persistence 4.Responsiveness 5.Lethality 6.Interoperability 11

12 Step 2 of Capabilities Assessment Overview Step 2 will first generate “raw” scores for Measures of Effectiveness (MOEs). Raw results will then be weighted against the applicable mission to come up with an Operational Level Aerospace Capability Score for each aircraft against each mission over the 2 timeframes. Mission weightings are sensitive and not included as part of this presentation. 12

13 Evaluation Grid for Step 2 Raw Scores Effectiveness rating of contributing system performances. Multiple systems will contribute to each measure of effectiveness. Will vary across all missions. Rating scale from 1-10 to allow sufficient fidelity. Qualitative, and thus requires Subject Matter Expert input. Will result in “Raw” scores for Step RatingDefinition 10 Excellent No appreciable limitations in delivering the desired effect 9 8 Very Good Only minor limitations in delivering the desired effect 7 6 Good Moderate limitations in delivering the desired effect 5 4 Poor Major limitations in delivering the desired effect 3 2 Very Poor Severe limitations in delivering the desired effect 1

14 14 Step 2 Example CFDS Mission Aerospace Capability 1. Defence of Canada – Baseline and Arctic Defensive Counter AirVignette 1 Threat The Canadian Fighter is conducting normal daily and contingency North American Aerospace Defence (NORAD) missions at normal alert levels, and is prepared to react to elevated alert levels. Operations are conducted from main operating bases as well as Forward Operating Locations and missions may be over land and over water. Operations are being conducted in a combined and/or joint environment with NORAD forces. The threats for these missions can be air and maritime-based : Non-state civilian aircraft (including low/slow) State long-range aviation Classified Threat List (NORAD) Tasks - Rapidly establish and maintain air presence/policing/deterrence for a tactically significant period of time in an area of operations. - Detect, validate, characterize, assess, and warn of unwanted and unauthorized air activity approaching or operating within North American domestic airspace including potential attacks against North America. (NCI , 1.2.1) - Detect, Identify, Determine Intent, Monitor, Intercept, Report, Negate, or Destroy potential air threats : Non-state civilian aircraft (including low/slow) State long-range aviation and escorts Non-state use of stand-off missile technologies Classified Threat List (NORAD)

15 15 Defensive Counter Air (MOEs) using Vignette 1 in 2020 to 2030 timeframe Aircraft X Aircraft Y Lethality 29 Survivability 49 Responsiveness 96 Reach and Persistence 96 Interoperability 58 Awareness 68 Tech AreaContributing SystemAircraft XAircraft Y SensorsRadio Frequency Sensors 38 Electro-optical or Infrared Sensors 77 WeaponsAir-to-Air Weapons 28 Air-to-Ground Weapons Air-to-Surface Weapons Non-Kinetic Weapons 05 Self ProtectionRadio Frequency Self Protection 29 Infrared Self Protection 57 Countermeasures 58 AvionicsData Link 510 Communications 77 Sensor Integration 28 Pilot Workload 89 PerformanceRadio Frequency Signature 49 Infrared Signature 36 Engine/Airframe 96 Combat Radius/Endurance 97 Step 2 Example – Assessment of Raw Scores

16 16 Step 2 Example – Capability Raw Scores Mission Aerospace Capability Measures of Effectiveness (Aircraft X) LethalitySurvivability Reach and Persistence ResponsivenessInteroperabilityAwareness 1. Defence of Canada (Baseline and Arctic) Defensive Counter Air Offensive Counter Air Strategic Attack Close Air Support Land Strike TASMO ISR 5.8 Example Only

17 17 Step 2 Example – Capability Raw Scores Mission Aerospace Capability Measures of Effectiveness (Aircraft Y) LethalitySurvivability Reach and Persistence ResponsivenessInteroperabilityAwareness 1. Defence of Canada (Baseline and Arctic) Defensive Counter Air Offensive Counter Air Strategic Attack Close Air Support Land Strike TASMO ISR 7.66 Example Only

18 18 Step 2 - Capability (Weighted) Mission Aerospace Capability (Weighted) Measures of Effectiveness (Aircraft X) ( ) LethalitySurvivability Reach and Persistence ResponsivenessInteroperabilityAwareness 1. Defence of Canada (Baseline and Arctic) Defensive Counter Air Offensive Counter Air Strategic Attack Close Air Support Land Strike TASMO ISR Applying weighting function to each measure of effect allows capability to be properly quantified within that Mission 6.7 Example Only

19 19 Step 2 - Capability (Weighted) Mission Aerospace Capability (Weighted) Measures of Effectiveness (Aircraft Y) ( ) LethalitySurvivability Reach and Persistence ResponsivenessInteroperabilityAwareness 1. Defence of Canada (Baseline and Arctic) Defensive Counter Air Offensive Counter Air Strategic Attack Close Air Support Land Strike TASMO ISR Applying weighting function to each measure of effect allows capability to be properly quantified within that Mission 7.1 Example Only

20 20 Step 3 of Capabilities Assessment This last step of the Capabilities Assessment takes into account: a) the weighting of each Fighter Aerospace Capability within a particular mission and b) criticality as follows: DCA – XX%, OCA – XX%, TASMO – XX%, ISR – XX%; Mission Critical (MC), Mission Essential (ME), or Mission Routine (MR); The Operational Level Score obtained at the end of Step 2 is aggregately weighted and assessed by Royal Canadian Air Force against the mission criticality matrix. Aerospace capability limitations will carry risk forward dependant on their mission criticality (translation matrix on slide 22 expands).

21 CFDS Mission 1 (example) 6.1 TASMOISRLand StrikesCAS Strategic Attack OCADCA Aircraft Y Aircraft D Aircraft C Aircraft B Aircraft A ISRTASMOLd Strk.CAS Strat Atk. OCADCA CFDS Mission 1 (example) ECC Mission criticality (C/E/R) X%X%X%X%X%X% Weighting roles VS CFDS Weighting of Fighter Roles / CFDS missions + Mission Criticality Step 3a – Operational Mission Weighting 21 Operational level scoring (before Mission criticality) CFDS 1 CFDS 2/4 CFDS 3CFDS 5 CFDS6 V1V2V5V3V4V6 Aircraft A Aircraft B Aircraft C Aircraft D Aircraft E Aircraft Y7.0 Operational Risk CFDS 1 CFDS 2/4 CFDS 3CFDS 5 CFDS 6 V1V2V5V3V4V6 Aircraft A Aircraft B Aircraft C Aircraft D Aircraft E Aircraft Y Aircraft E

22 Step 3b - Mission Criticality Translation Matrix 22 Definitions Mission Critical: A capability that delivers a direct effect as part of its primary function. This effect Is evaluated as critical to mission success and will pose risk to mission success if it is not employed. Mission Essential: A capability that is an essential enabler to Mission Critical Capabilities; the lack of one or more Mission Essential Capabilities will pose risk to mission success. Mission Routine: A capability that is required for the mission but either as a routine supporting function or a very low likelihood of employment. Only in cases where multiple Mission Routine Capabilities constitute a systems failure will any significant risk be posed to the mission.

23 Operational level scoring (before Mission criticality) CFDS 1CFDS 2/4CFDS 3CFDS 5 CFDS 6 Vignette 1 Vignette 2Vignette 5Vignette 3Vignette 4Vignette 6 Aircraft A Aircraft B Aircraft C Aircraft D Aircraft E Example Step 3 – “Operational Risk” Assessment Mission Criticality Translation Matrix Operational Risk CFDS 1CFDS 2/4CFDS 3CFDS 5 CFDS 6 Vignette 1 Vignette 2 Vignette 5 Vignette 3 Vignette 4 Vignette 6 Aircraft A Aiircraft B Aircraft C Aircraft D Aircraft E Example Step 3 - End State “Operational Risk” RatingDefinition 10 Insignificant limitations in accomplishing the mission. 9 8 Minor limitations in accomplishing the mission 7 6 Moderate limitations in accomplishing the mission. 5 4 Major limitations in accomplishing the mission 3 2 Severe limitations in accomplishing the mission. 1 Definition Low operational risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. Medium operational risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. Significant operational risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. High operational risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. Very high operational risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. 23

24 24 Assessment of the “Production and Supportability” Questions Step 1 – Review of responses by technical Subject Matter Expert teams – no scoring. Step 2 – Assessment of responses by the Royal Canadian Air Force Senior leadership level from a Canada First Defense Strategy missions perspective to result in “Military Strategic Risk” assessment of each aircraft against the missions. Both Steps will be overseen by the National Fighter Procurement Secretariat. Significant differences of opinion between assessment teams will be brought to the attention of the Panel.

25 Factor Analysis through Deductive Reasoning Where does the Industry response fit inside the framework? 25 Strategic Assessment Definition: There is a possible risk that… Factors The mission need is… The threat is assessed as… CFDS Likelihood Possible Impacts Required mitigations Considerations So What? Industry responses to Industry Engagement Request Open source study Government-to-Government Information Other products from the Evaluation of Options Source Documentation So What? Strategic Risk Assessment Deductions (step 1)(step 2) (used during step 2 only)

26 Acquisition This Strategic Assessment Factor (SAF) is an assessment of factors, other than cost, that would affect acquisition of aircraft types. This includes: planned production periods of aircraft type; the manufacturer’s ability to continue or re-establish production periods; the ability of the manufacturer to offer a complete “cradle to grave” program, including; training, in-service / life cycle support and disposal; the manufacturer’s successful completion of any required developmental work before the aircraft are ready for acquisition; and the ability of aircraft type to be certified for airworthiness under Canadian Forces regulations. 26

27 Supportability and Force Management An assessment of the overall / long-term supportability and aspects related to management of the fighter force. This includes factors such as: the required quantity of aircraft; managed readiness postures and sustainment rations; the training system and production of appropriately trained pilots and maintenance / support personnel (including use of simulators); training considerations for maintenance / aircrew personnel; aircraft mission availability rates and aircrew / technician workload; long-term availability of components needed for aircraft maintenance; implementation and sustainment of supporting infrastructure; the ability to operate from required locations; weapons compatibility and support; and the suitability of a reprogramming capability including its flexibility and responsiveness for new threats and/or theatres of operation or new capabilities. 27

28 Integration This Strategic Assessment Factor assesses broad interoperability within the Canadian Forces and with allied forces. This includes interoperability with air-to-air refuelling services (other than Canadian Forces), common ground/spares support with allies and their supply lines, training systems, data sharing, communications, STANAGs, and the ability to feed data into Canadian Forces and Government of Canada networks taking into consideration national security requirements. 28

29 Growth Potential This Strategic Assessment Factor assesses the growth potential and technological flexibility to respond to unforeseen future advances in threat capabilities, to implement required enhancements to fighter technology, and to evolve as needed to meet the Canadian Forces’ needs. Factors include analysis of the architecture of aircraft types, power and cooling capability for new systems. 29

30 30 Response to IER Strategic Assessment Factors Acquisition Supportability & Force Management Integration Growth Potential Future Security Env’t Other reports resulting from evaluation of options Step 1 Aircraft - SAF Assessment - Technical SME level working groups - Problem space defined and scoped by the use of a “possible risk” based questionnaire - SME comment on likelihood; then -Comment on possible impacts and required mitigations to the overall fighter program -Information to be collated and considered in Step 2. Step 2 SAF - CFDS assessment - Senior leadership level Working group (fighter heavy) - Problem space defined and scoped by the Task 2 report - “Considering the likelihood of possible impacts to the overall Fighter Program, and based on your Professional Military Judgment, how would you rate the strategic risk to the RCAF ability to successfully fulfill the fighter contribution to each CFDS mission?” Open Source/Other Info and Gov-to-Gov Info End State - Strategic Risk Strategic Risk CFDS 1 CFDS 2/4 CFDS 3CFDS 5 CDFDS 6 V1V2V5V3V4V6 Aircraft A Aircraft B Aircraft C Aircraft D Aircraft E Example Definition Low strategic risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. Medium strategic risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. Significant strategic risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. High strategic risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. Very high strategic risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X.

31 Final Stage – Overall Mission Risk Assessment End State: Mission Risk Strategic Risk Operational Risk 31

32 End State Mission Risk Operational Risk CFDS 1 CFDS 2/4 CFDS 3CFDS 5 CFDS 6 V1V2V5V3V4V6 Aircraft A Aircraft B Aircraft C Aircraft D Aircraft E Aircraft Y Mission Risk (fusion of Operational and Strategic risk) RCAF to apply professional military judgement Strategic Risk CFDS 1 CFDS 2/4 CFDS 3CFDS 5 CFDS 6 V1V2V5V3V4V6 Aircraft A Aircraft B Aircraft C Aircraft D Aircraft E Aircraft Y C Air Force endorsement of Final Report Definition Low risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. Medium risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. Significant risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. High risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. Very high risk to the RCAF ability to successfully fulfill the fighter contribution to CFDS mission X. 32 Mission Risk CFDS 1CFDS 2/4CFDS 3CFDS 5 CFDS 6 V1V2V5V3V4V6 Aircraft A Aircraft B Aircraft C Aircraft D Aircraft E Aircraft Y

33 33 Mission Risk Mission Risk will be an outcome of the fusion of Strategic and Operational risk by the same RCAF senior leaders assessment teams who took part in step 2 of the Strategic Assessments. Decision support tools, such as but not limited to a mission risk translation matrix and the DND/CF Integrated Risk Management Guidelines will be used to determine a rating. Final assessment will be informed by, but not limited to the decision support tools provided. Assessors will be allowed to deviate from the mission risk translation matrix used by + or – one risk level. Rationale behind a final assessment that deviates from the mission risk translation matrix used will be provided to the Independent Review Panel for review.

34 Summing Up We are committed to an open, fair and transparent process – IER and methodology posted publicly on Government website – All questions and answers during IER shared simultaneously with all companies – Details of assessment methodology shared with companies in advance of IER response deadline – NFPS overseeing entire assessment process – Significant differences of opinion between assessment teams will be brought to the attention of the Panel – The process will result in an assessment of benefits and risks associated with each aircraft against each mission. Canada will release a summary report that respects commercial sensitivities and classified information restrictions. 34


Download ppt "National Fighter Procurement Secretariat Secrétariat national d’approvisionnement en chasseurs Industry Engagement Request Questionnaire 1 Assessment Methodology."

Similar presentations


Ads by Google