For the final selection of defence equipment, field trials are the most important
Maj Gen Mrinal Suman
Defence equipment is procured to perform specific operational missions. To ensure that the equipment complies with all performance parameters to fulfill the envisaged mission, the user service lays down the criteria in terms of qualitative requirements (QR). Validation of performance is carried out through technical appraisal of the equipment on offer. It entails field trials and staff evaluation. Field trials are conducted to obtain, verify and provide data for evaluation of performance and operational capability of systems, sub-systems and components to enable ascertainment of their mission accomplishment potential. Staff evaluation is the process of review and analysis of data produced during field trials to determine whether a system satisfies its acceptance criteria or not.
As is apparent, staff evaluation follows field trials. Hence, field trials are central to the final selection of equipment for induction into the services. Criteria for technical evaluation and the trial methodology is made known to the vendors upfront in the Request for Proposals (RFP) itself. Each parameter is explained in detail, so that the vendors fully understand the evaluation process.
A word about the trial procedure will be in order here. The manufacturers of the short-listed equipment are asked to send the desired number of units of the equipment/weapon system for field trials. Any vendor failing to produce equipment for trials by due date is normally given a grace period of 15 days to produce the equipment for trials. An additional grace period of up to 30 days may be granted by the Vice Chief of the user service. Failure to field the equipment within the permitted time period renders a vendor ineligible for further trials.
The Defence Procurement Procedure (DPP) assigns the responsibility of conducting field trials to the user service which formulates the trial directive in conformity with the trial methodology given in the Request for Proposal (RFP). It also constitutes the trial team. The trial directive contains a detailed trial plan, specifying the fundamental points that need to be addressed for validating the essential parameters, and the Enhanced Performance Parameters (if applicable), along with the evaluation score credits thereof. Parameters not mentioned in the RFP cannot be considered for trials.
Trial units are detailed on the basis of their location, suitability of the officers and availability of the necessary infra-structure. As regards the trial team, representatives of Defence Research and Development Organisation (DRDO) and QA agency may also be included, based on the requirement. A representative of the Acquisition Wing can participate as an observer. For equipment being procured for more than one service, representatives of each service for whom equipment is being procured is included. Such trial team is headed by the representative from the service nominated as the lead service. For trials of equipment involving transfer of technology, representative of the nominate production agency is included as an associate member.
Trials are conducted with respect to the conditions where the equipment is most likely to be deployed. In other conditions where the probability of deployment is not high, appropriate certifications confirming the functional effectiveness under such conditions can be obtained. In cases, where applicable, simulation-based testing can also be resorted to. All weather testing is carried out in cases where considered absolutely essential.
After each stage of the trials, a debriefing of all the vendors is carried out in a common meeting (wherever feasible) as regards the performance of their equipment. Compliance or otherwise, vis-à-vis the RFP parameters, is specifically communicated to all the vendors at the trial location itself. It is also ensured that all verbal communications with the vendors is confirmed in writing within a week and all such correspondence is duly recorded and documented.
Need for Fine-tuning the Trial Plan
As field trials are expensive in terms of time, effort and resources; it is essential that the trial plan is made with due diligence. It should be logical, comprehensive and explicit. Every parameter should be spelt out in precise and verifiable terms. Inexact and ambiguously defined terms always lead to multiple interpretations with subsequent misunderstandings and squabbling. Aggrieved vendors appeal to the government and the whole process gets aborted. The primary objective of field trials is to generate data to enable evaluating staff to take well-considered decisions. If the data is incomplete or insufficient, the evaluating authority is constrained to demand repeat/supplementary trials, with consequent loss of time.
To make field trials more productive, they need to be in phases. For that, key QR should be identified at the outset for according precedence. Although all QR are essential to the task/tasks to be performed by the equipment, there are invariably one or more QR that are of overriding importance. They need to be accorded primacy. Such key parameters without whose compliance procurement becomes meaningless should be termed as vital performance attributes (VPA) – an attribute is an assessable characteristic that describes an aspect of a system or capability.
For example, bullet proof vehicles are procured primarily for protection against designated hostile fire. Therefore, all other QR become secondary to bullet proofing. Hence, it is essential to first ascertain that the vehicle satisfies VPA of protection before undertaking validation of other QR. Howsoever, good a vehicle may be with respect to all other parameters, it cannot be considered for acceptance if it fails to provide the required degree of protection.
The case of the field trials of anti-mine shoes testifies to the need for identifying VPA. The said shoes were needed for providing protection to the wearer against mine blasts. Three major QR were ‘protection’, ‘weight’ and ‘comfort’. Field trials were ordered without identifying ‘protection’ as the key parameter or VPA. Although weight, comfort and other QR were of importance, ‘protection’ was the sole raison d’être for their procurement. Unfortunately, field trials were conceived in a routine manner and this vital aspect was totally disregarded.
There were three vendors in contention. To start with, the nominated trial units were asked to check weight and comfort aspects. Thereafter, the shoes were sent to a laboratory to check the degree of protection they provided. The trial units approved two samples and rejected the third one, being uncomfortable and cumbersome. During subsequent blast tests, the rejected sample was the only one found to be complying with the protection parameter. Staff evaluation became difficult. A great deal of time and effort would have been saved, had a proper trial plan been evolved and the blast test sought at the outset. Only those samples that met protection criteria should have been subjected to further trials.
As seen above, by restricting detailed trials only to VPA-compliant equipment, considerable infructuous effort and time can be saved. Therefore, field trials should be held in two phases. In Phase 1, all competing equipments should be tested for VPA compliance. Thereafter, only successful equipment should be subjected to detailed trials in Phase 2. The trial plan should, therefore, be based on identification of VPA and phasing of trials.
The Way Forward
Although field trials in India enjoy high credibility for being rigorous and thorough, more attention needs to be paid to the preparation of the trial plans. For, it has been seen that the root cause of most of the infirmities that afflict field trials lies in indifferently prepared trial plans.
The sole aim of the entire technical evaluation process is to identify equipment that is technically acceptable for introduction into service. To enable the general staff to take well-considered decisions, all required inputs should be made available to them. There should be no gaps at all. Therefore, it is essential that the trial plan is evolved in close consultations with the evaluating general staff.
Finally, quality of field trials is totally dependent on the diligence and expertise of the functionaries involved. For example, determination of sequence of trials requires a thorough knowledge of primary tasks for which equipment is being procured. Similarly, trial of new equipment with differing technologies is a highly specialised task. The trial teams must possess the necessary expertise to evaluate equipment with diverse technologies. Therefore, selection of concerned functionaries requires careful consideration.