Building a Centre of Excellence #10 - Stage 5 : Testing
It isn't strictly true to say no testing is done before stage 5 as naturally sections of code and logic are verified during the build phase. Once the main development is complete, end to end testing begins.
Stepping Through
During the first test cycle, we step through the process, ensuring that each action of the flow behaves as expected. The process definition document is essential and acts as a reference point. The inputs and outputs of each step are recorded. The first test cycle usually picks up failures in process flow and inaccuracies in some of the convoluted logic and calculation stages.
Running in Debug Mode
With a successful step through, it is time to run the process in debug mode at the slowest speed to check it flows and performs correctly. Inserting breakpoints in strategic sections is helpful. The process runs several times, increasing the bot speed each time until we reach normal setting.
Scenario testing
With the process running fine, scenario testing using a range of sample test data is conducted. This approach ensures the process flexes and adapts for any known eventually. For example, in the case of patient data, we may use a range of test data sets. This may include data for adults (male and female), paediatrics, a variety of specialities, various presenting complaints and whatever mix of data gives us confidence that the process is optimised before go-live.
Exception Testing
Real-world processes do not always follow a happy path. During the process definition document stage, we try to predict any problems that may occur, for example, receipt of a duplicate referral, patient record not found, or invalid NHS number. Our bots are programmed to deal with these eventualities, and so we strive to test these before go-live. These types of exceptions are called business exceptions and are routed to the originating department (e.g., finance team, medical secretaries or booking centre staff) to resolve.
It is all-important to test and pre-empt technical failures. For example, the system is unavailable, cannot logon, shared drive not accessible, or config file does not exist.
The more testing you do, the lower the exception rate will be at go-live.
For go-live, we aim for less than 2% for business and technical exceptions combined. With a period of post-go-live stabilisation, exception analysis and process enhancements, this should fall below 1%.
For example, our GP referral process has only one or two referrals that need human interaction every 24 hours.
Full Steam Ahead
The process is now run at full speed in debug mode. The test will pick up any issues with attach, activate mode, and wait stages.
Control Room
The final test is now to run the process unattended in the control room at maximum speed with a single bot. If successful, we test with multiple bots running the same process to ensure queue management is 100% correct.
This approach of testing gives reassurance and confidence that the process is safe to deploy into the live environment to handle real-world transactions.