Factory Acceptance Testing
Abstract
Factory acceptance testing is a vital practice, which every organization should practice, every time it engages in procurement of equipment and installation of plant and machinery. This is because it allows an organization to ensure that equipment and plant are of the right quality and that they are fully operational. This essay defines what factory acceptance testing and explains why it is important that organizations carry out factory acceptance tests before allowing installed plant to operate or allowing for shipment of purchased equipment into their premises. The essay also explains who should be responsible for factory acceptance testing. In addition, the essay reviews literature on factory acceptance testing, in which the process of carrying out a factory acceptance test is outlined and explained. Various tools, software, techniques and technologies that are used in factory acceptance testing are explained in this review. Application of factory acceptance test and its benefits to three Singaporean companies, SGS, TUV Rheidland and Trisen Asia Control Pte Ltd, is also explained.
1: Introduction
1.1: Background
Business organizations are always engaging in acquisitions of machinery, equipment, software and other appliances that are relevant in the running of their day to day business activities. Prior to the delivery or the final installation of such machinery, equipment, appliances or software, every client or buyer in this case wants to ensure that the things they acquired are operational as initially intended, and that there shall not be any disturbances occurring in site after installation/delivery. It is this requirement that necessitates the verification that all contractual requirements and specifications have been met. Any equipment or component that fails to meet the contractual requirements or specifications laid out could give rise to contractual consequences, or projects could run out of budget before completion. Preliminary evaluation of any equipment involves factory acceptance testing before the equipment becomes operational (Jain, 2011).
1.2: Factory Acceptance Testing and IEC 61551
In general, a significant financial payment made to any vendor is triggered by a FAT (Factory Acceptance Test), and therefore the procedure should be treated with a high level of formality. Witnessing of the FAT must be done by the owner of the system or the project manager, with a formal record of any inconsistencies and non conformities and methods on how they shall be handled. Vendors are often ill prepared to handle FAT and they end up rushing through the process in a bid to ship the equipment as fast as could be. Others do it fast without paying much attention so that they may realize milestones in their financial reporting period, or to take out expensive inventory from their books. The consequences of such an occurrence are that the observation of non conformities is delayed until such a time after installation has been done. This makes it rather difficult to correct the anomalies without subjecting the project schedule to some negative consequences. Sometimes, it may even be too late and the flexibility that comes with offering technical solutions could already be overtaken by events. Normally, the customer or an inspection agency conducts a Factory Acceptance Tests to ensure that products meet specified requirements (Dahl, 2013).
In line with the standard IEC 61551, factory acceptance tests (FAT) are not a legal requirement and a business person may omit the procedure, all the same. However, it is necessary to carry out a check to avoid or reduce the occurrences discussed earlier. The concept of factory acceptance tests applies to machinery, plants, electrical equipment, software and any other installations made to a business premises before they are installed or delivered to the buyer. The specifications of the particular purchase must be included in the contract or purchase order, and as stated earlier, the FAT must be witnessed by the owner of the system, the project manager or a designated person. The equipment must also be fully pre-tested by the supplier before the witnessed test, and must be carried out on all equipment that the supplier is to deliver or install to the buyer.
1.3: Research Objectives
The paper aims to shed light on the significance of the FAT process for project managers and business owners. The steps followed to ensure thorough tests shall also be profiled in the paper as well as the consequences of skipping the process, which may occur out of negligence, ignorance or the assumption that it has been carried out by another party. This is done in line with what has been learnt from three Singaporean companies, SGS, TUV Rheidland and Trisen Asia Control Pte Ltd that have applied factory acceptance testing.
2: Literature Review
For a long time now, several researchers have emphasized why it is ultimately necessary to carry out factory acceptance testing during addition, amendment or improvement process of a system within an organization. Farquharson and Wiseman (2011) state that the whole process incorporates conducting a customized testing procedure in all systems and the procedure must be completed before they can be installed to begin operations at the facility. They go ahead to state that the major rationale behind this procedure is the fact that predicting the correct operation of the instrumented systems or the consequences that may result due to failure is a difficult. For this reason, providing a valuable check against these safety issues before hand provides a cushion for the eventualities. For software and ICT systems require factory acceptance testing (Sustainable Energy Authority of Ireland, 2014).
2.1: Integrated Factory Acceptance Testing
IFAT is a factory acceptance test that incorporates the core elements that control systems suppliers and plant installation and maintenance staff deal with. These two parties then go ahead to test and validate a subset of the control system network as well as the security application. An IFAT is crucial in that it provides a number of benefits to the system owner being time savings; savings on costs; increased capability in meeting compliance requirements; and improved comfort levels that come with integrating security solutions.
FAT essentially involves assuring and proving information to the end user that the software, equipment or their components are correct and that their functioning is in line with the DFS. In addition to this, a factory acceptance test is responsible for permitting a major stage payment to the vendor, despite the fact that all this depends on the level of arrangements that had been made previously. It is logical to take this step at this stage because already, the systems have been developed, and the testing has been conducted to precision. Customers are able to assess and recommend for improvements of software to the developers (Liskin, 2012).
2.1.1: System Acceptance
System acceptance takes place after the FAT has been conducted, and is the process of demonstrating that the systems are functioning as stipulated in the contract. This ensures eventual effective operation of the process that it was originally designed for. Formal acceptance takes place after the commissioning of the application software. At the time of commissioning, it is important to note that the plant is still at its teething stage, and that the control system still requires fine tuning. Acceptance testing determines whether the requirements of specified equipment or plant have been met (BMT, 2014). An acceptance test represents the interests of a customer, giving them confidence that the application possesses the features that are required, and that these features are functioning correctly.
Theoretically, the passing of all acceptance tests signify the completion of a project, especially in physically involving projects like constructions which require physical plants and machinery to perform. In software engineering though, acceptance testing consists of executing a series of tests constituting a particular functionality that is based on the requirements that have been specified by the user of the software. System engineering, on the other hand, could consist of testing the black-box of a system, otherwise referred to as functional testing, validation testing, and factory acceptance testing or final testing.
2.1.2: Dynamic Development
Dynamic development is a procedure conducted to uncover any features of the system that are hidden, and it is done with the application of various tools such as formal verification, real time animator or model checker. It acts as an iterative way of acquiring more requirements in accordance with the stakeholders as well as in verification of the system’s correctness. Real-time animators are used in the earlier stages of system development and are based on more formal methods, while prototyping is applied for risk analysis (RCI, 2008). The use of formal methods is however useful in improving the quality of prototypes. After the code generation stage of acceptance testing, the final system may be utilized in verifying the system against the requirements of the informal user, as well as those of the end user (Farquharson & Wiseman, 2011).
2.1.3: Industrial Control System
ICS (Industrial Control System) encompasses a number of control systems that are used in industrial production. They include supervisory control and data acquisition (SCADA) systems, distributed control systems (DCS) and other minor configurations of control systems such as PLCs (Programmable Logic Controllers) (Farquharson & Wiseman, 2011). These systems are mainly found in fundamental infrastructures such as gas, water and electricity utility systems. As stated earlier, ICS in conjunction with major control system vendors are crucial in validating and testing of a division of the control system network and security application environment in the process of factory acceptance testing (Farquharson & Wiseman, 2011). During the last thirty years, there have been major developments in LAN and WAN based communications protocols. These developments have been carried out in industries such as management of waste water and distribution of electrical components. Some of them include MODBUS, Foundation Field bus and Process Field Bus among others. Their choice is dependent upon the requirements of operation being industry preference, system design history and vendor design history (Farquharson & Wiseman, 2011).
2.2: Advantages of Carrying out Factory Acceptance Testing
The benefits and advantages of factory acceptance tests to the industry owners and vendors have also been discussed at great length by scholars. Farquharson & Wiseman (2011) outline a number of benefits being that IFAT avoids expensive redesigning and troubleshooting in the course of outage operations. The main reason behind this is that relevant, organizational personnel are able to authenticate the existing communication between systems. This contributes to the realization of any latent troubles before new equipment is purchased or new plant is installed. Consequently, delays and reinstallation troubles are avoided because problems are discovered early enough and rectifications are made, with no hurry to meet imminent deadlines. In addition, the procedures, in conjunction with other tests that are carried out in the course of the installation provide verification and oversight that assists in meeting regulatory obligations for the site. This therefore means that carrying out a factory acceptance test is useful in saving resources of time and costs.
Allowing the vendors and customers to acquire knowledge and comfort with the solution of the integrated security is another benefit that comes with factory acceptance testing. This is due to the fact that the tests have been designed in such a way that they verify that applications and systems perform in accordance with the set requirements, and do not in any way affect the operations of the system in a negative way. As such, upon completion of the exercise, both the suppliers and the customers have gained confidence that the solution may be implemented without the risk of unfavorable effects to the vendor’s systems, as well as the assurance that the functionality will be in line with the needs of the customer.
Without integrating FAT into the implementation stage, it is possible to uncover issues that relate to the maintenance and operation during installation. Such issues could be related to configuration alterations, or a number of cyber assets having been omitted by the vendor security solution. Such issues could require massive rework by the site personnel in the course of difficult outage operations, which would have been controlled had an IFAT been conducted before the installation was initiated.
Table 1: Benefits of Carrying out Factory Acceptance Testing
Factory Acceptance Testing Benefits
To the Client; To the Vendor/Supplier;
Provides assurance that software, equipment or components are correct Permits payment to the vendor/supplier
Provides assurance that equipment or plant is standard and functions as per the expectations/requirements Allows acquisition of knowledge and comfort with the solution of the integrated security
Determines whether the vendor has met all the specifications
Provides confidence to the user or client
2.3: Parties to Factory Acceptance Testing
A successful FAT requires the attention of the customer (project owner), the vendor (supplier) and a neutral third party who may act as the witness. Representatives of the customer are necessary in that they provide the customers with oversight, technical knowledge and information useful in decision making. They also benefit from acquiring knowledge about the product because in most instances, they are part of the operators at the project site. The control system vendors and the security solution vendors also require being present for the procedure so as to verify that the systems have been configured as per their requirements, and also to provide technical insight in troubleshooting the security solution (Hedberg, 2006; American Association of Physicists in Medicine, 2010).
Figure 1: Parties to Factory Acceptance Testing
Client Representatives
Vendor Representatives
Vendor (Supplier)
Neutral Third Party
Securing Solution Vendors
System Vendors
Decision Making
Technical Knowledge
Oversight
Client (Project Owner)
Factory Acceptance Testing
As stated earlier, a neutral third party is critical for the procedure so as to benefit all the parties to the IFAT. Their duties in the entire transaction entail defining and running the testing activities, while acting as the host for the representatives of the vendor as well as those of the customers. This is usually carried out with the intent of creating a productive and neutral environment where the interests of both parties are taken care of. The vendors, the customers and the neutral third party representatives jointly ensure the success of the solution and at the same time garner knowledge from each other through the course of the procedure.
2.4: Conducting a Factory Acceptance Test
As earlier stated, a factory acceptance test is basically aimed at testing software and hardware functionality as one integrated unit, with the goal of identifying and resolving any problems that are in the system before its arrival into the plant site. In some instances, some of the problems may not be corrected until that time when the equipment is at the plant though it is necessary to keep such problems at minimum. The factory acceptance test could be conducted by either the vendor or the user, or a combination of the two, with the latter being more appropriate. Before it can be conducted though, apt planning and preparation is recommended. Following the guidance of the neutral third party, the users and the vendors must settle on the design as well as the systems that will be required by the procedure. For optimal benefit of the procedure, the actual systems up for installation at the user’s plant site should be present at the time of carrying out the test. If for some reason they cannot be availed, then equivalent systems should be used in place.
After the design of an integrated network has been made and systems identified, then it is time for developing a test plan. The third party, who should be impartial is responsible for writing the test plan. In writing the test plan, the equipment or plant supplier’s requirements and those of the buyer provide direction and descriptions of the tests to be carried out; security application tests and integrated network tests that are required. The test has to be reviewed by all the participants to ensure that they all understand and agree with its requirements. After the relevant parties have agreed, development of detailed test plans is instituted. Such test plans incorporate every step that is to be followed. Each step outlines both the guidelines and the criteria, which will be used to determine success or failure of every test that will be carried out. These are the specifications that are followed in carrying out the actual test activities. The actual testing is conducted by the neutral party, who follows the test plan to the letter to avoid deviating from the agreed upon plans. The supplier of equipment or plant and the customer should participate actively. This will guarantee accuracy and equality in the tests to be carried out. Whenever there is a system or application failure in terms of work or meeting the test requirements, there has to be a FAT variation document. The FAT variation document reports failure in the test and provides recommendations for solutions. It also identifies the party which will be charged with the responsibility of carrying out rectifications. It is imperative to ensure that all the involved parties have sufficient information pertaining to the progress and all other issues, which relevant parties may encounter during testing.
Completion of the testing activities gives way for the distribution of the final results package to all parties. The package includes test results, final configurations as well as proposed action plans for the issues that remain unresolved. This documentation provides the user and vendor with the necessities for an uninterrupted installation process.
