Introduction: In the fast-paced world of ecommerce, scaling up to meet growing customer demands can be both exhilarating and challenging. Our case study focuses on the lessons learned while implementing an automation framework for a leading ecommerce company. However, during this scale-up phase, the company encountered several testing challenges that impacted its ability to deliver quality solutions efficiently. This case study delves into the experiences of a testing team involved in the scale-up and highlights the lessons learned from the testing process.
We don’t talk enough about how we learned to create solutions which now can be considered successful so this case study focuses on a failure scenario and outlines the lessons we learned.
Key challenges:
The company encountered several testing challenges during the scale-up phase. Clients expressed dissatisfaction with project delays, product availability issues, crashes, and downtimes. These issues resulted in a lack of confidence in the company’s products among both developers and clients. Moreover, the absence of test automation and skilled automation engineers further hindered the testing process.
Recognizing the need for improvement, we proposed a test automation framework tailored to the company’s specific requirements. The framework aimed to reduce repetitive manual tasks, allowing test engineers to focus on areas where automation would yield better results. Additionally, it presented an opportunity for skill development among test engineers.
However, despite its potential benefits, the proposal faced resistance and was ultimately rejected by most team members.
Lessons Learned:
From this experience, an important lesson emerged: the success of a testing process or automation framework is contingent on the adoption and support of the teams involved.
Out of this, we came with the following points to serve us and others going through a similar journey:
- the success of a framework is not really related to its ability to solve a technical challenge
- excluding team members from decision-making and development stages leads to shorter time to deliver a framework but they won’t adopt it – won’t use it and won’t enhance it
- regardless of how amazing a test automation framework is, if it doesn’t address the teams pains , it won’t be relevant for them
- failure to include the team strengths and skills and preferences in the automation approach will lead to a framework on a shelf somewhere gathering dust or even worse, a belief that automation won’t work for them
- the lack of collaboration and engagement in the process of designing and development of the automation will alienate the teams
- and to top these, not including the development and testing teams in the decision making process, it is nearly impossible to find that sweet spot in the level of automation which yields the right results for this particular team and organization
We’ve seen it time and time again, in most of the organizations inquiring for our automation services, automation is not the silver bullet everyone expects it to be.
We’ve seen so many automation efforts gathering dust on various servers but more painful, we’ve seen so many organizations’ trust in automation decrease considerably in the past years. But there are KPIs from somewhere and they just inquire for automation to check some boxes.
This makes it difficult to lead to a successful automation result because all the parties come to the table and they don’t even try to ask the right questions.
Adding the scenario-based development techniques in the early stages of diagnosis we manage to avoid the mistakes we did before and we also help our clients define their scenarios in relation to automation.
Conclusion:
This case study highlights the lessons we learned while failing to deliver a successful test automation framework. While the framework met all the technical requirements and would have actually addressed most the challenges the company was facing, not considering the team, we failed to deliver an actual value to them. This goes beyond testing and automation. Clients come to us with specific requests but some of them won’t fit in their teams.
Learning from this experience and some others, we consider the teams in our processes, we consider their wants and pains as well as their skills and strengths.
