Publicly Released: August 14, 2020
Objective
The objective of this audit was to determine whether the U.S. Special Operations Command (USSOCOM) fielded Special Operations–Peculiar (SO-P) equipment that met performance requirements during test and evaluation (T&E).
Background
SO-P equipment is unique to USSOCOM units and funded and managed by USSOCOM. SO-P equipment is defined as equipment, material, supplies, and services required for special operations missions for which there is no Military Service common requirement. Examples of SO-P equipment include specialized communications systems; intelligence, surveillance, and reconnaissance equipment; vehicles to meet specific special operations mission requirements; and specialized aircraft with precision strike capability. During the SO-P equipment validation and approval process, USSOCOM officials identify mission requirements and the type of equipment needed to meet these mission requirements. USSOCOM officials record these mission requirements in capability documents, which identify the key performance parameters (KPPs) for the equipment. KPPs are those system attributes considered most critical or essential for an effective military capability.
USSOCOM Directive 71-5 assigns responsibility to the USSOCOM program managers to develop the T&E master plan for each program, which includes the requirements correlation matrix. The requirements correlation matrix identifies the KPPs and the T&E required to document the equipment’s performance. The USSOCOM Directorate of Operational Test and Evaluation is responsible for testing SO-P equipment to ensure that the equipment is operationally effective, operationally suitable, and safe for use by Special Operations Forces (SOF). The T&E also verifies that defined KPPs are met. T&E is conducted by qualified and trained SOF who represent the intended users, and is based on the KPP minimum requirements established in the validated capability documents. The T&E process culminates with a Fielding and Deployment Release (F&DR), which certifies that the SO-P equipment is operationally effective, suitable, and safe for use. According to USSOCOM officials, if SO-P equipment does not meet all KPPs, a conditional F&DR can be issued. USSOCOM Directive 70-1 states that conditional F&DRs certify the system or equipment is operationally effective, safe, and suitable for limited use for a specified period.
Finding
USSOCOM officials did not verify that all SO-P equipment met performance requirements during T&E prior to fielding for the programs reviewed. We reviewed a non-statistical sample of 10 of 28 USSOCOM SO-P programs which received full or conditional F&DRs during FYs 2017 through 2019. We determined that USSOCOM personnel followed command policy to verify KPPs passed T&E or issue a conditional F&DR that identified the KPPs that did not pass T&E prior to fielding for four programs reviewed, valued at $494.1 million. For the remaining six programs, valued at $815.8 million, USSOCOM officials did not verify that SO-P equipment passed all required T&E. For example, the Tactical Local Area Network program, which provides network communication equipment for SOF users, had 4 KPPs, but USSOCOM officials did not verify that 3 of the 4 KPPs passed T&E prior to fielding. Despite not verifying the SO-P equipment met KPPs via T&E, USSOCOM personnel issued full or conditional F&DRs that did not identify unverified KPPs for the six programs.
For the six programs, USSOCOM did not verify that the SO-P equipment passed required T&E because USSOCOM program managers did not consistently follow USSOCOM Directives 70-1 and 71-5 to identify T&E required to determine whether the SO-P equipment met the required KPPs, and document this in a requirements correlation matrix. In addition, USSOCOM guidance did not require SO-P program officials to verify that the SO-P equipment met the required KPPs prior to authorizing fielding.
As a result, USSOCOM purchased and fielded SO-P equipment for the six programs, valued at $815.8 million, without verifying that the equipment meets user needs. KPPs are the equipment attributes that are most critical for mission effectiveness. Without verification through T&E, USSOCOM has no assurance that the SO-P equipment for these six programs met the KPPs. USSOCOM officials did not verify that the SO-P equipment met the KPPs during T&E, and still issued a conditional F&DR for this program. Based on user comments, the equipment may not meet the requirements established in the KPPs, and therefore would not be operationally effective.
Recommendation
We recommend that the USSOCOM Commander develop internal controls to ensure USSOCOM program managers develop and maintain a requirements correlation matrix for each program that clearly matches KPPs to T&E, integrate the requirements correlation matrix into T&E, and document the results in the requirements correlation matrix. We also recommend that the USSOCOM Commander update U.S. Special Operations Command directives to require a completed requirements correlation matrix prior to issuing an F&DR for SO-P programs, and require that conditional F&DRs identify the KPPs that were not met, outline any limitations on how the equipment is used, and the work required prior to issuing a full F&DR.
Management Comments and Our Response
The USSOCOM Chief of Staff, responding for the USSOCOM Commander, agreed with the recommendation, stating it is essential to verify that a fielded system meets its validated KPPs, as well as achieving mission effectiveness and suitability prior to fielding. To ensure the requirements correlation matrix is utilized and archived efficiently, USSOCOM will now require program managers to provide a requirements correlation matrix at the start of test planning. This will be used to develop the evaluation framework by the operational test organizations. At the conclusion of operational testing the operational test organizations will create a requirements validation matrix documenting the results in their test report.
The Chief of Staff also stated future F&DRs will identify whether or not KPPs were met. If any KPPs were not met, but the system is still determined to be operationally effective, USSOCOM will issue a conditional F&DR with specific restrictions on its use and state the actions needed to close the restrictions. When all conditions are closed, USSOCOM will issue an unrestricted F&DR.
Comments from the Chief of Staff addressed the specifics of the recommendation; therefore, the recommendation is resolved but remain open. Although the Chief of Staff did not provide a completion date, we reached out to USSOCOM officials to determine whether the command would be updating its guidance to include the new policies and if so, when the command expects to complete the update. A USSOCOM official stated in an email response that the command did plan to update the criteria and that it would take approximately one year to collaborate, write, staff, and sign the directives. We will close the recommendation once USSOCOM updates the applicable criteria.
This report is the result of Proj. No. D2019-D000RG-0214.000