Preparation guide for User Acceptance Testing

What is User Acceptance Testing?

User Acceptance Testing (UAT) is conducted by the end-user to determine whether an application meets the business requirements in real-world scenarios. It is also known as beta testing or end-user testing. This type of testing is usually performed in the final phase of testing. UAT acts as the final round of testing before releasing a product into the production environment.

Why is User Acceptance Testing necessary?

Before getting into the purpose of UAT, let's consider some questions:

• Is any product built for end-users?
• Is the ultimate goal when a product is developed to fulfill business users' needs?

The answer will most likely be "Yes." But then, another question would interest you: "Why shouldn't we get end-users to validate the product against their actual requirements?"

Yes, UAT allows target users to interact with the application and experience it with real-world test data. The goal of UAT is to ensure that the system can perform correctly and align with day-to-day business.

Unlike other testing types like functional testing or unit testing, UAT is crucial because it helps to answer some questions that other testing types can not:

• Can the user use the application easily?
• Can the user attain their business goal via using the product?
• Is there any obstacle when the user is using it?
• Does it work as expected?

The primary purpose of UAT is to validate that the software aligns with the needs of users who use the product daily. Besides, end users can provide valuable feedback during UAT sections to improve the product. And needless to say, it's more time-saving to fix any issues that appear in the early stage of development rather than let them leak into the production environment.

User Acceptance Testing process

1. Entry Criteria

As mentioned in the above section, the main goal of UAT is not to find bugs. Instead, the main goal is to verify that the features meet the business requirements. Furthermore, end-users who use the features daily perform UAT to support their workflow. Thus, end-users must do the following tasks before conducting UAT:

• Features must be fully implemented and functioning.
• All other low-level testing activities such as Unit Testing, Integration Testing,  System Testing & Regression Testing must be done.
• All critical and major bugs found must be resolved.
• All critical bugs must be resolved before UAT. Why? Because if UAT users find a major bug that blocks an entire feature, then the UAT cannot be continued until the bug is fixed.

2. User Acceptance Testing Process

A dedicated person usually leads UAT (e.g., Product Manager or Business Analyst). On the other hand, some other companies can have different people or roles to run the UAT depending on the features that need to be tested. A UAT leader role needs to be assigned to plan and lead the UAT session in such a case.

Once a UAT leader is assigned, they will go through the following process:

2.1 Requirement Analysis

n this step, they need to analyze and learn about the business requirements and about features to test. The leader should have a thorough understanding of the features to prepare the test scenarios, assist, and answer any testers' questions during the test execution. They also need to collaborate with the development team to triage the issues and feedback reported by testers.

2.2 Test planning

Timeline Setup

First, the UAT leader needs to know when they can perform the UAT. Second, UAT needs to work with different stakeholders to understand when entry criteria are met and the release plan so that they can finish the UAT within this time frame. Third, once the timeline is figured out, the UAT leader will send the invitation to the UAT testers and finalize the list of testers when they respond.

Test scenario/case creation

The next step would be to prepare test scenarios and test cases. Remember that UAT testers are different from the testers in a development team. Professional testers can reduce some of the test steps (e.g., precondition how to log in and navigate to the page that needs to be tested). But UAT testers are real users who only know about their business workflow and don't have much knowledge of the entire application under test. Hence, the test steps should be very clear that can work them through the test scenarios. Another important factor when creating UAT test cases is that it has to cover all the workflows defined in the business/use case requirements.

Test Specifications & Assignment

Test specifications include test data and test environment.

Test data is not available for UAT users to use in most cases. Therefore, the UAT leader needs to work with development to prepare test data based on the test scenarios/cases defined above. We highly recommend that test data be actual, like production data, so that the test results are reliable. The test data needs to be attached to the test steps so that UAT testers can easily see and use

After test data is created, the leader must specify where each tester should execute the test. For example, do you want to test on different browsers, locations, OS, etc.? If these parameters are needed, they must be assigned explicitly in the test plan.

Test Logistics

When the UAT testers run the tests, how and where should they mark the test status? If they encounter bugs, where should they report these issues? If the team uses a test management tool, can the UAT testers access this tool to report bugs, or should they report it on a spreadsheet or excel file?

Which tools should they use to record bugs? Is it easy to install and use during the UAT session, which usually has a limited time?

UAT needs to answer these questions to have a good preparation for a couple of hours of UAT session.

2.3 Test Execution

• UAT leader will briefly describe the goal, process, and what features need to be tested to testers.
• Testers perform the testing based on the assignment and report the issues found to the Issue list.
• UAT leader will be available to assist testers.

2.4 Evaluation

• QA, Product Owner, and developer will have a triage meeting after the UAT section to evaluate and prioritize issues reported by testers.
• The team will conduct a short retrospective meeting to improve for the next UAT section.

Exit Criteria

This testing takes place before the release of new features to users. So, the first exit criterion is that there should be no critical or major bugs. If bugs are found and require fixing, it needs to go back to the previous phase to fix. Then testers need to retest the fix, and perform regression testing before they can do UAT again. Fixing bugs at this stage is very expensive. Therefore, we highly recommend that features be bug-free before moving to UAT. If there is no bug found, then it’s considered as passing this criterion.

The main goal of the UAT is to make sure the developed feature fulfills the business needs. Sometimes, the UAT testers are happy with the features. Still, they will request extra functionalities. These requests must be passed to the Product Owner and triaged for the upcoming development cycle.

Once all the exit criteria are met, there should be a sign-off meeting with all stakeholders to get the formal approval for the release candidate. This meeting will communicate the status of UAT and the following steps to make sure everyone is on the same page.

Bonus: Using Bird to optimize your UAT process

When it comes to performing UAT, it's essential to report all issues during testing. As in real-world testing, various kinds of technical logs, environments, browsers, and devices would be necessary for developers to debug issues. However, for non-tech savvy users, it is not easy to collect these kinds of data.

Bird Eats Bug, or Bird for short is a bug report tool that automatically captures such technical data and screenshots and video recordings. Non-technical users can enjoy their testing journey and use their mic and/or camera to describe issues quickly and give feedback easily.

Bird will help ease the burden of running from place to place to compile all testing results and synchronize them to issue tracking systems. In addition, all bug reports will be auto-posted to common tools like Zapier, Jira, Trello, or Github with Bird’s integration features. Your team will be in the loop with a simple click.

Conclusion

Each product might have a different process in terms of development and testing process. But all of them will have the final goal to bring value to end-users. Thus, UAT is vital to ensure that a product is built with the business objective for actual users. Whether we conduct UAT in a formal or informal approach, UAT reduces the chance of issues leaking into the production environment. In addition, it improves the product usability close to users' expectations.