Quality of construction: how does the QA (quality assurance) of Redmadrobot work
How can we rate the product’s goodness, check all the bugs and spots. Why do we need testers, when there is no code and what the reason of Crimea – says the Head of QA robots department – Marina Kulikova.
It seems, who can find all the disadvantages of product besides of users or developers. You just need to run the soft, try every scenario, click all buttons in each combination. In that case researching and testing comes to beta-test – intensive using the product by team before it will release to the market.
This is just misconception which have it’s own consequences like reviews in soft-stores. The main disadvantage is incomplete coverage by test-cases, so it’s impossible to foresee some things during the using of product by team.
Not to be san and hurt by loosing time aimlessly, testing by robots starts with the project. It can helps to appear some atypical problems. For example, Google didn’t recognize the Crimea as a part of Russian Federation. We found it during our requirement’s analysis, took into account in the development and saved Beeline users from roaming in leave and the company – from the wave of claims.
There is much more such cases have appeared in recent months - because of Roskomnadzor (Federal Supervision Agency for Information Technologies and Communications) blocking google services can’t work normal including geocoder, which is used by many applications. This happened during the testing. Of course, services are working now, but soft uses Yandex.Maps, because it is more reliable.
Testing of the product starts from the projecting stage – robots-testers take a research testing, try out some hypothesis and then check requirements for completeness, understanding and consistency, eliminating defects in the early development phase.
The first “pain” of testing – communications. When developers write one thing, testers check other, so some things developers have to fix on production. Really, that sounds and looks funny, but not for the user.
So with the pain of aimlessly lost time we have worked out and debugged the process of testing and relation to development.
All the process starts with test design:
1. All written requirements are testing by the team of developers and testers.
2. After that we start to write check-lists for Mock-testing and test-cases. (test case design phase)
3. Each test-case going through review.
Our team try to predict all possible scenarios and behavior of product/system and just after that we test and fix all little problems (if they are). Right after research testing and scenario preparing the main functional tests are starting. Cycle-by-cycle, feature-by-feature is remaking before it will be perfect.
Now we have exact algorithm of transferring assemblies to testing and only after that – release.
1. As soon as feature is ready, it is going to testing;
2. After first cycle of test the bugs appears;
3. Prioritize bugs and deciding, what should we take to next iteration;
4. Developers fix bugs;
5. Our QA team verify all bugfixes;
6. In case we again have bugs, we will repeat steps 1-5;
7. After the testing, soft is ready to be in Release Candidate list – here can be only verified features;
8. All tested/verified functions comes complete in release build;
9. Assembly is sent to QA department and then to installation testing, regression testing and then to compliance check with requirements in App Store and Google Play
10. QA department makes a report and then final build is going to market.
To optimize the testing, we automate API system, mobile applications work with their own middleware – we test it for full. The reason is that robots often work with the closed bank infrastructure and insurance companies, we don’t have constant access to the customer’s backend system, in such cases to test end-to-end scenarios we write tests for API and sometimes some load testing.
Despite robot’s love to algorithms, there is a sense in point automation, moreover, we have talented QA-engineers, we gave them more than a hundred devices. Now qc engineers can change their smartphones and tablets every day. The stock will last for four months.
We don’t test on emulators – who can give a guarantee, that it has no bugs, so devices themselves have a lot of differences: different operation systems with Android shell (such as Huawei with it’s EMIU) resolution and screen size, margins of the screen (there can be a very specific frame), and sometimes the differences in hardware. Emulators, alas, not take into account these "unexpected" features.
When the application is ready, the bugs are fixed, it remains to check the product for compliance the specific requirements of the stores. Apple traditionally follows strict rules in relation to App Store apps-the testing itself may take a couple of days, and if build is wrapped, to the planned release date you can add 3-4 days. Google traditionally easier and faster, but that doesn't mean that the requirements can be ignored.
How to avoid that? To test the UI/UX and gain expertise in the field of publication apps by stores. For example, if the application has a pop-up action, but at the time the checks it does not take effect, the Store can wrap the build. Tested by robots.
Responsibility for the result is a good stimulus and motivator. Test engineers — the last stage on the way to the final release of the product, the main filter from all bugs and spots. Especially if they don’t connect to the work in last moment.
In conclusion I would quote the QA “classic” Michael Fritzius:
«If this seems like a huge unsolvable problem, just take courage in the fact that being successful requires great software as the cornerstone of your company. If you have that, everything else will fall into place. Great QA is the key to having that»(с)