Back to Top

Educating the Aerospace Industry

Blog Archives

Major Reviews

System Requirements Review (SRR)

 

A System Requirements Review (SRR) is a formal review conducted to ensure that system requirements have been completely and properly identified and that a mutual understanding between the government and contractor exists. It ensures that the system under review can proceed into initial systems development and that all system and performance requirements derived from the Initial Capabilities Document (ICD) or draft Capability Development Document (CDD) are defined and testable, and are consistent with cost, schedule, risk, technology readiness, and other system constraints.

 

Checklist: SRR Risk Assessment Pre-Award

Checklist: System Requirements Review Completion Checklist

 

A SRR assesses the system requirements captured in the system specification and ensures that the system requirements are consistent with the approved materiel solution, ICD, enabling concepts, and available technologies identified in the Materiel Solutions Analysis (MSA) phase. An SRR is important in understanding the system performance, cost, and scheduling impacts that the defined requirements will have on a system.

 

Completion of the SRR should provide the following:

  1. An approved system performance specification with sufficiently conservative requirements to provide for design trade space for the Engineering, Manufacturing, and Development (EMD) phase,
  2. A preliminary allocation of system requirements to hardware, human, and software subsystems,
  3. A preliminary Identification of all software components (tactical, support, deliverable, non-deliverable, etc.),
  4. A comprehensive risk assessment for EMD,
  5. An approved EMD phase Systems Engineering Plan (SEP) that addresses cost and critical path drivers, and
  6. An approved Life-Cycle Sustainment Plan (LCSP) defining the product support plan and sustainment concepts with the corresponding metrics.

 

The SRR reviews and evaluates the draft functional baseline and requirements analysis. All relevant documentation should be reviewed, including:

 

Note: IEEE 5288.2 “Standard for Technical Reviews and Audits on Defense Programs” is the standard for technical reviews and audits to be performed throughout the acquisition life cycle for the US Department of Defense (DoD) and other defense agencies. This standard guides the DoD and contractor on what is required during an SRR.

 

AcqLinks and References:

Updated: 4/11/2018

Major Reviews

Physical Configuration Audit (PCA)

Major Reviews


The Physical Configuration Audit (PCA) examines the actual configuration of an item being produced and is conducted around the time of the Full-Rate Production Decision. It verifies that the related design documentation matches the Configuration Item (CI) as specified in the contract and confirms that the manufacturing processes, quality control system, measurement, and test equipment, and training are adequately planned, tracked, and controlled. The PCA validates many of the supporting processes used by the contractor in the production of the item and verifies other elements of the item that may have been impacted or redesigned after completion of the System Verification Review. The PCA is also used to verify that any elements of the CI that were redesigned after the completion of the Functional Configuration Audit (FCA) also meet the requirements of the CI’s performance specification. [1]

Detailed instructions and guidance for conducting PCA can be found in the:

Handbook: NAVAIR ILSM Configuration Management – Section 3

Checklist: PCA Risk Assessment

A PCA is normally conducted when the government plans to control the detailed design of the item it is acquiring via the Technical Data Package. When the government does not plan to exercise such control or purchase the item’s Technical Data Package, the contractor should conduct an internal PCA to define the starting point for controlling the detailed design of the item and establishing a product baseline. The PCA is complete when the design and manufacturing documentation match the item as specified in the contract. If the PCA was not conducted before the Full-Rate Production Decision (FRPD), it should be performed as soon as production systems are available. [1]

The PCA for a CI shall not be started unless the FCA has already been accomplished. After successful completion of the audit and the establishment of a Product Base Line (PBL), all subsequent changes are processed by formal engineering change action.

Software PCA [2]
The Software PCA is an examination of the as-coded total system software against its design or deliverable documentation. For Commercial off-the-Shelf (COTS) software this involves verification of correct documentation to support use of the software versions actually being delivered on the applicable media. Adequacy of identification and marking of all deliverable software media in accordance with contract requirements or best commercial practice is included in the PCA. The software PCA is normally conducted by government personnel at the trainer site after Government Final Inspection as soon as all final software corrective actions have been implemented.

Hardware PCA [2]
The Hardware PCA is an examination of the as-built system against its design documentation. A prerequisite for the PCA is the successful completion of an FCA. Preliminary Operation and Maintenance manuals and red-lined preliminary engineering drawings may be used when appropriate. Audits of Commercial off-the-Shelf (COTS) equipment should be limited to verification of agreement of Model/Part Numbers with system drawing parts lists and correct vendor documentation supplied. The hardware PCA is conducted jointly by the contractor and the government at the contractor’s plant prior to shipment for new trainer acquisitions.

Preliminary PCA [2]
A Preliminary PCA is normally conducted by the government during new trainer acquisitions, typically at the Critical Design Review time frame, to review reference designator number assignment plans, drawing/lists expectations, mutual understanding of contract specifications relative to the total hardware system, part identification, and markings, in-process inspections, review of contractor’s CM/QA program and evidence thereof. The hardware PCA may also be conducted incrementally when the contract involves a large system. In that case the final PCA should be conducted prior to the completion of GFI.

AcqLinks and References:

Major Reviews

Critical Design Review (CDR)

 

A Critical Design Review (CDR) is a multi-disciplined technical review to ensure that a system can proceed into fabrication, demonstration, and test and can meet stated performance requirements within cost, schedule, and risk.  A successful CDR is predicated upon a determination that the detailed design satisfies the Capabilities Development Document (CDD).  Multiple CDRs may be held for key Configuration Items (CI) and/or at each subsystem level, culminating in a system-level CDR. The CDR is conducted during the Engineering, Manufacturing, and Development (EMD) phase and when the product baseline has been achieved and the CDR entrance criteria detailed in the Systems Engineering Plan (SEP) have been met, allowing fabrication of hardware and coding of software deliverables to proceed.

 

Checklist: DoD Critical Design Review (CDR)

Checklist: Critical Design Review Checklist – Jan 2015

 

A CDR assesses the system’s final design as captured in product specifications for each Configuration Item in the system’s product baseline and ensures that each configuration item has been captured in the detailed design documentation. The resulting set of detailed drawings and specifications establish an initial product baseline, with a final baseline incorporating any design changes resulting from System Demonstration and Initial Operational Test and Evaluation (IOT&E).  The completion of CDR usually initiates the start of formal Configuration Management (CM) by the Contractor of the Technical Baseline.  Any changes to that baseline can only be accomplished with the approval of the Government.

 

A CDR should: [1]

  • Determine that detailed design of the configuration item under review satisfies cost (for cost-type contracts), schedule, and performance requirements.
  • Establish detail design compatibility among the configuration item and other items of equipment, facilities, computer software, and personnel.
  • Assess configuration item risk areas (on a technical, cost, and schedule basis).
  • Assess the results of producibility analyses conducted on system hardware.
  • Review preliminary hardware product specifications.
  • Determine the acceptability of the detailed design, performance, and test characteristics, and on the adequacy of the operation and support documents.

 

Completion of CDR should provide: [1]

  1. A system initial Product Baseline,
  2. An updated risk assessment for Engineering, Manufacturing, and Development (EMD),
  3. An updated Cost Analysis Requirements Description (CARD) based on the system product baseline,
  4. An updated program development schedule including fabrication, test, and evaluation, and software coding, critical path drivers, and
  5. An approved Life-Cycle Sustainment Plan (LCSP) updating program sustainment development efforts and schedules based on current budgets, test evaluation results, and firm supportability design features.

 

Post-CDR Assessment
For Major Defense Acquisition Programs (MDAP), the Deputy Assistant Secretary of Defense (Systems Engineering) “DASD(SE)” will assess the design maturity and technical risks evident in the CDR. Program Managers (PM) of these programs are required to invite DASD(SE) engineers to their system-level CDRs and make available CDR artifacts. These will include all CDR briefings and those artifacts that constitute the initial product baseline.
[1]

 


AcqTips:

  • The CDR chairperson should tailor the review to the technical scope and risk of the system, and address specifics of the CDR in the Systems Engineering Plan (SEP).
  • See Major Reviews

AcqLinks and References:

Update: 12/26/2017

Major Reviews

System Functional Review (SFR)

 

Major Reviews


 

The System Functional Review (SFR) is a technical review to ensure that the system’s functional baseline is established and can satisfying the requirements of the Initial Capabilities Document (ICD) or draft Capability Development Document (CDD) within the currently allocated budget and schedule. It also determines whether the system’s lower-level performance requirements are fully defined and consistent with the system concept and whether lower-level systems requirements trace to top-level system performance requirements.  The SFR is conducted during the Technology Maturation and Risk Reduction (TMRR) Phase of a program.

 

A critical component of an SFR review is the development of representative operational use cases for the system. System performance and the anticipated functional requirements for operations maintenance, and sustainment are assigned to sub-systems, hardware, software, or support after detailed analysis of the architecture and the environment in which it will be employed. The SFR determines whether the system’s functional definition is fully decomposed to its lower level, and that Integrated Product Teams (IPT) are prepared to start preliminary design.

 

The system’s lower-level performance requirements are evaluated to determine whether they are fully defined and consistent with the system concept and whether traceability of lower-level systems requirements to top-level system performance and the CDD is maintained. This activity results in two major systems engineering products: the final version of the system performance specification and draft version of the performance specifications, which describe the items below system level (item performance specifications).

 

Completion of the SFR should provide the following:

  1. An established system functional baseline with traceability to lower-level performance requirements,
  2. An updated risk assessment for the Engineering, Manufacturing and Development (EMD) Phase,
  3. An updated Cost Analysis Requirements Description (CARD) or a CARD-like document based on the system functional baseline,
  4. An updated program development schedule including system and software critical path drivers, and
  5. A preliminary system level maintenance plan with updates applicable to this phase.

 

AcqTips:

  • The SFR is the first review that begins to allocate requirements to separated sub-systems and organizational IPTs
  • An SFR review is where the need for Interface Control Documents becomes necessary to define areas of responsibility and constraints requiring coordination across IPTs.

AcqLinks and References:

Updated: 9/19/2017

Major Reviews

Post-Deployment Review (PDR)

 

Major Reviews


 

A Post-Deployment Review (PDR) is used by a Program Manager (PM) to review a system, beginning at Initial Operational Capability (IOC), to verify whether the fielded system continues to meet or exceed thresholds and objectives for cost, performance, and support parameters approved at full-rate production. DoD policy requires that “The Services shall conduct periodic assessments of system support strategies vis-à-vis actual vs. expected levels of performance and support. These reviews occur nominally every three (3) to five (5) years after IOC or when precipitated by changes in requirements/design or performance problems, and should include, at minimum:

  • Product Support Integrator/Provider performance;
  • Product improvements incorporated; and
  • Configuration control.

 

PDR continue as operational support plans execute (including transition from organic to contract support and vice versa, if applicable), and should be regularly updated depending on the pace of technology. The program manager should use existing reporting systems and operational feedback to evaluate the fielded system whenever possible.

 

AcqLinks and References:

Major Reviews

Major Reviews Overview

Major Reviews


 

A Major Program Review is a time-based event where all key stakeholders on an acquisition program gather to discuss the progress of their program. Usually the review decides if there is sufficient data to suggest that a program can proceed to the next phase of the program. There are two main categories of reviews that take place during the life-cycle of an acquisition program.  These are:

  1. Program
  2. Technical

Program Reviews
Program Reviews are a fundamental part of an acquisition program that focuses on cost, schedule, and performance. They’re an important oversight tool that the Program Manager (PM) and key Stakeholders can use to review and evaluate the state of the program, re-directing activity if necessary. The majority of program reviews focus on determining if a program has achieved its objectives to date and will be allowed to proceed into the next acquisition phase.

 

The main program reviews during an acquisition program are:

 

Technical Reviews
Technical Reviews are a fundamental part of the Systems Engineering Technical Assessment Process for the PM. They’re an important oversight tool that the PM can use to review and evaluate the state of the system and the program, re-directing activity if necessary. They examine if a program is achieving its technical objectives as detailed in the acquisition strategy. Technical reviews can include an assessment of systems engineering, software, risk management, requirements, technology readiness & development, and the technical baseline. 

 

The most commonly used technical reviews during most acquisition programs are:

AcqLinks and References:

Updated: 1/27/2018

Major Reviews

Functional Configuration Audit (FCA)

 

Major Reviews


 

A Functional Configuration Audit (FCA) examines the functional characteristics of the configured product and verifies that the product has met the requirements specified in its Functional Baseline documentation approved at the Preliminary Design Review (PDR) and Critical Design Review (CDR).  It has to do more with systems engineering and program management than official auditing. The FCA is a review of the configuration item’s test and analysis data to validate the intended function meets the system performance specification.  The FCA is normally performed prior to Low-Rate Initial Production (LRIP) and prior to or in conjunction with a Physical Configuration Audit (PCA). A successful FCA typically demonstrates that Engineering and Manufacturing Development product is sufficiently mature for entrance into LRIP.  A FCA may also be conducted concurrently with the System Verification Review (SVR).

 

The issues that are addressed during a FCA are:

  • Readiness issues for continuing design, continuing verifications, production, training, deployment, operations, support, and disposal have been resolved.
  • Verification is comprehensive and complete
  • Configuration audits, including completion of all change actions, have been completed for all CIs
  • Risk management planning is/has been updated for production
  • Systems Engineering planning is updated for production
  • Critical achievements, success criteria, and metrics have been established for production.

 

In large system with complex Configuration Items (CI), the FCAs may be accomplished in increments. Each increment may address a specific functional area of the system and will document any discrepancies that are found in the performance capabilities of that increment. After all of the increments have been completed, a final (summary) FCA may be held to address the status of all of the action items that have been identified by the incremental meetings and to document the status of the FCA for the system or CI in the minutes and certifications. In this way, the audit is effectively accomplished with a minimum of complications. [1]

 

The Program Management Office (PMO) is ultimately responsible for the performance of audits.  The Program Manager (PM) has overall disposition authority on audit results and reports. The PM’s designee, who may be the System Engineer (SE) or Logistics Management Specialist (LMS), will ensure audits requirements are properly delineated in the contract and the FCA is properly executed.

– See Software Functional Configuration Audit

AcqTips:    

 

AcqLinks and References:

 

Major Reviews

Flight Readiness Review (FRR)

 

A Flight Readiness Review (FRR) is a sub-set of the Test Readiness Review (TRR) and is applicable only to aviation programs. It assesses the readiness to initiate and conduct flight tests or flight operations. Typically, FRR approval requires the aviation system to be under Configuration Management (CM), have a flight clearance issued by the technical authority, approved flight test plan(s), discrepancy tracking, and Risk Assessment processes in place.

 

The FRR is a technical assessment establishing the configuration to be used in-flight testing to ensure that the system has a reasonable expectation of being judged operationally effective and suitable.  This review assesses a system test environment to ensure that the system under review can proceed into flight test with airworthiness standards met, objectives clearly stated, flight test data requirements clearly identified, and an acceptable Risk Management Plan (RMP) defined and approved. [1]

 

An FRR shall be conducted prior to the first flight of any new air vehicle.  For complex systems, an FRR shall be conducted with an assessment of each subsystem or Configuration Item (CI) prior to flight.  An FRR is also required prior to the first flight of any major changes to hardware, software, envelope, or objectives not covered in a previous FRR. [1]

 

AcqLinks and References:

Updated: 2/17/2017

Major Reviews

System Verification Review (SVR)

 

Major Reviews


The System Verification Review (SVR) is a product and process assessment to ensure the system under review can proceed into Low-Rate Initial Production (LRIP) and Full-Rate Production (FRP) within cost, schedule, risk, and other system constraints during the Engineering, Manufacturing and Development (EMD) Phase.  It assesses the system functionality and determines if it meets the functional requirements in the Capability Development Document (CDD) and draft Capability Production Document (CPD) documented in the functional baseline. The SVR establishes and verifies final product performance and provides inputs to the CPD.

Checklist: System Verification Review (SVR) – 27 Sept 2010

Typical SVR success criteria include affirmative answers to the following exit questions:

  1. Does the status of the technical effort and system indicate operational test success (operationally effective and suitable)?
  2. Can the system satisfy the Capability Development Document (CDD) and draft Capability Production Document (CPD)?
  3. Are adequate processes and metrics in place for the program to succeed?
  4. Are the risks known and manageable?
  5. Is the program schedule executable within the anticipated cost and technical risks?
  6. Are the system requirements understood to the level appropriate for this review?
  7. Is the program properly staffed?
  8. Is the program’s non-recurring engineering requirement executable with the existing budget?
  9. Is the system producible within the production budget?

The SVR is often conducted concurrently with the Production Readiness Review (PRR) and Functional Configuration Audit (FCA). Product support IPT members should participate in the review to:

  • Address system supportability and, based on developmental testing or analysis whether the sustainment features will satisfy the Capability Development Document/draft Capability Production Document and Sustainment Key Performance Parameters (KPP) / Key System Attributes (KSA).
  •  Adequate processes are in place so the sustainment performance metrics can be used to help the program to succeed in meeting user needs.
  • Ascertain if the system is supportable within the procurement, operations, and support budgets.

The SVR risk assessment checklist is designed as a technical review preparation tool and should be used as the primary guide for assessing risk during the review. This checklist is attached below.

AcqTips:

AcqLinks and References:

Major Reviews

Alternative Systems Review (ASR)

 

Major Reviews


An Alternative Systems Review (ASR) is a technical review that assesses the preliminary materiel solutions that have been developed during the Materiel Solution Analysis (MSA) Phase.  The review ensures that one or more proposed materiel solution(s) have the best potential to be cost effective, affordable, operationally effective and suitable, and can be developed to provide a timely solution to at an acceptable level of risk to satisfy the capabilities listed in an Initial Capabilities Document (ICD).

Checklist: ASR Risk Assessment Checklist

A successful ASR is predicated on the review team’s determination that the operational capabilities, proposed solution(s), available technologies, and program resources (funding, schedule, staffing, infrastructure, and processes) form a satisfactory basis for proceeding into the Technology Development (TD) Phase.  An ASR should provide information to the Milestone A review before proceeding into the TD Phase.

Completion of the ASR should provide the following: [1]

  1. An agreement on the proposed materiel solution(s) (including the corresponding product support concept) to take forward into the milestone decision and subsequent Technology Development phase.
  2. Hardware and software architectural constraints/drivers to address all Key Performance Parameters (KPPs)
  3. An assessment of the full system software concept to include conceptual definition of the complete deliverable/non-deliverable software, scope, and risk.
  4. A comprehensive rationale for the proposed materiel solution(s), based upon the Analysis of Alternatives (AoA) that evaluated relative cost, schedule, performance (hardware, human, software), and technology risks.
  5. A comprehensive assessment of the relative risks associated with including Commercial off-the-Shelf (COTS)  items in the program, with emphasis on host platform environmental design, diagnostic information integration, and maintenance concept compatibility.
  6. A comprehensive risk assessment for the Technology Development (TD) phase.
  7. Results of Trade Studies/technical demonstrations for concept risk reduction.
  8. Joint requirements for the purposes of commonality, compatibility, interoperability, and integration.
  9. Refined thresholds and objectives initially stated as broad measures of effectiveness and suitability (e.g., Key Performance Parameters (KPP) / Key System Attributes (KSA)).
  10. Completed, comprehensive planning for the Technology Development phase (hardware, software and human systems), including critical components to be developed, prototyped, and demonstrated, their cost, and critical path drivers. This planning could include an Integrated Master Plan (IMP) and Integrated Master Schedule (IMS).
  11. Initial planning for the Engineering and Manufacturing Development (EMD) phase.
  12. A draft System Requirements Document (SRD)  if one does not already exist. (This is a high-level engineering document that represents the customer/user capability needs as system requirements.) This systems requirement document should include a system level description of all software elements required by the preferred system concept.

AcqTips:

  • The ASR risk assessment checklist is designed as a technical review preparation tool, and should be used as the primary guide for assessing risk during the review.

AcqLinks and References: