Introduction: In the fast-paced realm of ecommerce, the challenge of scaling up operations while maintaining quality and efficiency is ever-present. This case study sheds light on the success story of an ecommerce company implementing the right amount of automation. Faced with challenges that impeded project delivery and affected product availability, the company turned to test automation as a solution. Through effective implementation and active team involvement, the company achieved remarkable enhancements in testing efficiency, product quality, and overall business growth.

Key Challenges: 

The company faced several testing challenges, including 

• client dissatisfaction with project delays
• product availability issues
• increased team turnover rate as they were overworked and overwhelmed
• and a lack of confidence in the company’s offerings

Additionally, the manual testing workload was repetitive and time-consuming, resulting in limited focus on exploratory testing and frequent UI-related issues.

The solution identified: 

We started evaluating the pains of the teams by paying attention to the emerging patterns in our discussions. We recognized the need for a more efficient and effective testing process, but we mainly identified a short path to improve the situation. We then spearheaded the development of a tailored test automation framework. 

We considered the shortest path to yield results initially and we aimed to:

• reduce repetitive manual tasks
• decrease QE fatigue
• empower the testing team to focus on critical areas that required human expertise

As automation is not a silver bullet for all companies and products and teams, we focused on the level of automation sufficient and necessary for this particular organization.

Importantly, the development of the automation actively involved team members in the development process, incorporating their feedback and ensuring the framework aligned with their needs and motivations.

We also added the knowledge gained from the company’s portfolio as to include in our framework the scenarios relevant to customers. 

Focusing our efforts in the development of a bespoke test automation framework we included the relevant repeating issues gathered from the client base, development and testing teams.

Manual testing efforts decreased as team members transitioned to automation tasks, allowing for greater productivity. 

With reduced regression issues, the team delivered more products to clients within shorter time frames. 

The improved overall quality and enhanced user interface led to increased client satisfaction and confidence in the company’s products. 

The automation framework facilitated exploratory testing, resulting in valuable insights and an overall uplift in product quality.

During the scale-up, the company primarily employed test automation for regression testing. By automating repetitive test cases, the testing team was able to free up valuable time and resources to focus on other critical aspects, such as exploratory testing and system performance improvements.

By identifying and resolving issues in test environments, major system failures were prevented from reaching production. Additionally, the automation framework facilitated the swift discovery and inclusion of client-reported issues into the regression suite, eliminating recurring problems.

Key Lessons Learned and Advice: 

A crucial lesson derived from this experience was the significance of team adoption and support for testing processes and automation frameworks. The active involvement of team members throughout the development cycle, coupled with their motivation and input, played a vital role in the success of the automation framework. 

Another very important lesson was to rely on knowledge from similar customers and the experience the teams already had in the organization. 

Focusing our efforts to tailor our framework to consider the teams strengths while solving their immediate problems was the key in getting the resulting automation adopted and supported by the teams. 

To ecommerce companies undergoing similar scale-ups, this case study underscores the importance of fostering collaboration and leveraging the expertise of team members.

Establishing an automation framework is merely the initial step; actively involving the team and ensuring their buy-in and ongoing adoption are imperative for true success.

Conclusion: 

This success story emphasizes the transformative power of the right level of automation in an ecommerce scale-up. Through the strategic implementation of a tailored automation framework and active team involvement, the company achieved remarkable improvements in testing efficiency, product quality, and customer satisfaction. 

By embracing effective testing practices and involving the team throughout the process, ecommerce companies can overcome scale-up challenges, optimize their testing processes, and drive business growth in the competitive ecommerce landscape.

Introduction: 

In this case study, we examine the journey of implementing scenario-based development in an ecommerce company specializing in developing ecommerce solutions for other enterprises. The company experienced significant growth during its scale-up phase by expanding its client base and venturing into new markets. 

Growth Trajectory and Expansion: As the company acquired more clients, it concurrently developed new ecommerce applications and functionalities to meet their diverse needs. The scale-up involved managing live products in production, delivering enhancements and fixes to existing clients, as well as developing solutions for new clients.

In this case study we will focus on the improvement of the development lifecycle by implementing a scenario-based development framework to address their challenges.

Key Challenges: 

The ecommerce organization encountered several challenges during the scale-up phase. These challenges included:

• changing or evolving client requirements during the development phase
• miscommunication leading to different client expectations
• client dissatisfaction with the delivered products
• and extended project timelines due to rework during development
• loss of business from existing customers

Solution identified: 

While the company was using agile development, we focused on the testing phases of the development to discover the pains. We involved the development teams as well as product and scrum teams to create a more comprehensive understanding of the issues the product teams were facing. 

We soon discovered that the majority of the rework was due to changing requirements too frequently. While it is generally fine to change directions, in this case, the requirements were actually misunderstood by both the development teams as well as the customers.  

To tackle these challenges, we recognized the need for improved communication of requirements and expectations from clients. 

Based on the team’s skills and preferences we implemented a scenario-based framework.  Looking at the entire portfolio and discussing with team members and sales and customers, we created the following: 

• different templates for scenarios – focusing on a few key functionalities of their applications
• we tested these with additional teams teaching them what paths needed to be included and what level of detail is necessary
• we refined the framework based on their feedback 
• we shared it with customers and product to employ

Additionally, with the customers involved, we introduced wireframes to provide them with a visual representation of the expected user experience. 

This approach enhanced their understanding, encouraged active participation, and facilitated a comprehensive review of the ecommerce applications.

Product, development and testing teams utilized the scenarios in the analysis and design phases and continued to refer to them throughout the development cycle. 

They were used in the communication with the customers as well to align development with client expectations, minimize assumptions, and identify any gaps in requirements early on.

The implementation of scenario-based development framework resulted in notable cost savings and operational efficiencies for this ecommerce organization. By identifying issues early in the analysis and design phases, we prevented major system failures from reaching the development stage. This proactive approach reduced the need for costly post-production fixes and shortened the time to market for fixes and updated functionalities. Additionally, the streamlined testing process allowed teams to allocate more time to testing delivered functionality, resulting in higher quality products and increased operational efficiencies.

Key Lessons Learned and Future Recommendations: 

This experience taught us lessons we added to our current scenario-based development approach:

• different skillsets had completely different assumptions when considering the same functionality
• identifying issues early in the analysis and design phases, we prevented major system failures from reaching the development stage
• the introduction of visual representations in the scenario-based development framework facilitated clarity and alignment throughout the development cycle
• automation using the scenarios resulted covers the relevant functionality and it is easy to maintain
• testing and development efforts focused on quality once the functionality was clarified enough by the scenarios resulted from the scenario-based development

For future scale-ups, it is imperative to embrace comprehensive communication practices and actively involve clients to ensure a shared understanding of requirements and expectations.

Conclusion: 

Collaborative efforts enabled by the scenario-based development, led to enhanced quality and reduced frustrations, resulting in improved client satisfaction. By addressing communication gaps, we significantly reduced rework and shortened delivery time for clients from over a year to less than six months. 

Moreover, the implementation of scenario-based development instilled confidence in customers, attracting more business and contributing to the company’s expansion into new markets.

Finding a scenario-based development framework that suits both your organization, teams and customers is fun and challenging and so worth it.

Introduction: 

In this case study, we delve into the implementation of scenario-based development in a fintech insurance company during its scale-up phase. The primary focus is on leveraging scenario-based development to drive automated testing, addressing the challenges faced by the testing team in handling a large volume of functionalities and diverse data combinations related to customer communications.

Pension and Insurance Company Context: The company had an in-house development and manual testing team responsible for testing their products and internal applications. However, the testing team struggled to cope with the extensive volume of functionalities, prompting the need for automated testing to cover a vast array of data combinations for customer letters. The scope of testing also encompassed different types of content for PDF files, emails, web applications, and internal applications.

Key challenges: 

During the diagnosis phase, it was discovered that the testing team relied solely on manual testing, using a limited amount of data. However, the testers displayed a strong willingness to understand and implement complex database queries, despite lacking coding skills.

Company’s challenges: The main goals were to develop an automation framework that would lead to:

• The creation of a customizable and user-friendly automated testing solution
• Empowering the client’s testing team to easily add or modify test scenarios without relying on development skills
• Achieving high test scenario coverage across various technologies and data sources
• Establishing a dynamic solution that leverages extensive information and logic stored in databases, coupled with customization capabilities offered by the testing framework.

Our challenges: 

The primary challenge involved developing a custom automated testing framework, as this approach was relatively uncommon in the industry, resulting in limited documentation. 

The framework aimed to intelligently navigate different paths based on data stored in the database, necessitating the use of reverse engineering techniques. 

Additionally, ensuring ease of use and expandability by the client’s testing team posed an additional challenge.

Solution identified: 

We employed our scenario-based development approach to clearly define the scenarios the automation would have to consider.

In this, we met with product, development and testing teams to understand their pains and their needs. We evaluated their skills, strengths and wants moving forward. We identified supporters of the automation and we involved them in the decision making process and the development phases. 

Together, we leveraged their strengths and included all of this in our framework. 

We defined clearly the scenarios they needed covered and we discovered the patterns which helped in our design.  

We then spearheaded the development of customized automation framework focusing on the following: 

• navigation in the web interfaces of both internal and external applications – utilizing Selenium WebDriver and custom Java libraries 
• validation of the content of the web application, PDF files, and emails – a method was devised to dynamically generate expected content based on static text and dynamic data retrieved from the database at runtime
• the generated content was then compared with the actual content in the tested application using the automated framework
• navigation in web application was based on generated data to enhance the paths took by users – decisions of paths were made during runtime depending on the generated data

The implementation of this framework significantly reduced testing time from days or weeks to just a few hours, while also increasing test scenario coverage from 5-10% to 90%. 

Following the implementation, the testing team’s responsibilities primarily focused on testing edge cases and creating new and complex database queries for new functionalities.

Key lessons learned: 

• Effective Communication: Improving communication of requirements and expectations from clients is crucial to prevent misunderstandings and frequent changes during the development phase.
• Scenario-Based Development: Implementing a scenario-based development approach allows for a comprehensive understanding of the product, facilitates alignment with client expectations, minimizes assumptions, and identifies requirements gaps early on.
• Early Issue Identification: By identifying issues early in the analysis and design phases, major system failures can be prevented from reaching the development stage, saving costs and reducing rework.
• Test Automation: Developing a custom automated testing framework tailored to the specific needs of the company significantly reduces testing time, increases test scenario coverage, and enables the testing team to focus on edge cases and new functionality.
• Collaboration and Involvement: Collaborative efforts between development, testing, and product teams, as well as active involvement of clients, lead to enhanced quality, reduced frustrations, and improved client satisfaction.
• Comprehensive Testing Practices: Embracing comprehensive testing practices, including scenario-based development and test automation, helps ensure a shared understanding of requirements and expectations, leading to higher operational efficiencies and increased confidence in software products.

These lessons highlight the importance of effective communication, scenario-based development, test automation, collaboration, and comprehensive testing practices in successfully navigating the challenges faced during a scale-up phase in the fintech insurance industry.

Conclusion: 

By harnessing scenario-based development, the fintech insurance company successfully integrated automated testing into its scale-up phase. The customized test automation framework provided extensive coverage and rapid testing for diverse functionalities across various applications. Automation became the cornerstone of the testing process, reducing manual efforts and ensuring comprehensive test coverage. This case study demonstrates the value of scenario-based development in driving automated testing, resulting in notable operational efficiencies and increased confidence in software products. 

Introduction: 

In this case study, we explore the journey of an ecommerce company specializing in systems integration services and solutions. During its scale-up phase, the company experienced remarkable growth, expanding its client base in the United States and venturing into European markets. 

In this article we’ll focus on the testing ideas and practices we implemented in this organization to support them during their scale up.  This, of course, will translate in company’s reputation, customer satisfaction, and overall business growth.

Growth Trajectory and Expansion: The company started with a few clients in the United States and was trying to increase its customer base while expanding into new markets. 

During the scale-up phase, the company developed new ecommerce applications  for new customers and introduced new functionalities to their existing clients and applications. 

They also offered support for various clients with their applications live in production, releasing enhancements and fixes to address customer feedback.

Key Challenges: 

The company encountered several challenges during the scale-up phase and they all could be sum up to the clients being unhappy with their products and their delivery times. 

Client dissatisfaction arose from:

• delayed project deliveries
• frequent product unavailability due to crashes and downtimes
• end customer journey was far from flawless

We traced these issues to the testing practices as well as development and devops. 

We will continue to focus on the testing parts although changes happened in all the teams to address these challenges. 

For the testing discipline, we identified the following challenges: 

• the existing processes and documentation were inadequate to handle the increasing demands and complexities of the scale-up
• collaboration between projects and testing team members was minimum
• each application in development for a client was assigned a small testing team (1-2 test engineers) focused on the application from inception to the go live
• repeating issues were discovered on various products
• development and testing teams were overwhelmed and overworked – fixing issues after promoting to production, leading to decreased confidence among both developers and clients.
• the absence of dedicated testing teams supporting live clients and the lack of test artifacts, both manual and automated, posed significant obstacles to ensuring quality control.

The solution identified: 

Looking at the challenges expressed both by the company and individual team members such as developers and scrum masters, we identified testing improvements to put these projects on the path to success.  

To address these testing challenges, we implemented specific strategies aimed at enhancing collaboration, knowledge sharing, and reusability of testing artifacts. 

Initially, we created smoke test suites for all live clients. 

Working with the teams involved in the maintenance of these products, we then identified the areas most impacted by the releases and gradually expanded the range of test artifacts in subsequent increments. 

We brainstormed to get to a fast and efficient way to create these artifacts and discover what other artifacts might support them better. By utilizing reverse engineering techniques and gathering information from team members, we reconstructed testing artifacts from existing projects, minimizing trial and error. 

We documented everything, we created test procedures thus enabling knowledge transfer. 

We leveraged the collaborative tools already available in the company – we used Confluence and Jira with add-on for testing artifacts (Zephyr) for the following: 

• confluence – documentation of testing process and practices 
• confluence – knowledge sharing like how to use various tools, what areas to test, investigation and proof of concepts of tools and practices 
• zephyr – to store test cases – both project specific as well as reusable ones
• jira – for defects – using them as a knowledge sharing base to prevent issues from old projects on the new ones

The tools are relevant but what we consider more important is that we enabled team members to work together efficiently and we nurtured a culture of knowledge exchange. 

With the repositories of reusable artifacts and the knowledge base we managed to create rapid onboarding of testing engineers, including contractors who could seamlessly augment resources when needed. 

Moreover, we facilitated smoother transitions for engineers moving across projects, minimizing ramp-up time and maximizing productivity.

What was in for the company with the solution above, you ask? 

• the company extended its reach into European markets and witnessed increased revenue from ongoing client support post-launch
• new projects from existing clients as they were happier with their live products
• reduced the number of issues discovered in production or later stages of development
• decreased frustrations within the development and testing teams, leading to more efficient project deliveries

Key Lessons Learned and Future Recommendations: 

The testing process during the ecommerce scale-up provided valuable insights and lessons. 

• an external perspective proved instrumental in identifying untapped potential within the teams and streamlining processes 
• focusing on what is already created by the teams is more relevant than focusing on what is missing
• the knowledge was already there, it just needed to be organized and shared
• focusing on the pains of the teams, it was easy to trace them to the pains of the clients and fix all of them together

Effective communication, team nurture and collaboration are pivotal to testing success, and the early adoption of these practices is crucial for future scale-ups.

Conclusion: 

One might consider that te implementation of robust testing practices played a vital role in the successful ecommerce scale-up. 

But we believe the star here was identifying the pains of the teams and their invaluable knowledge. Figuring out what they were doing well led us to the improvements we brought. 

Optimizing collaboration, and promoting knowledge sharing, in a manner that focused both on the individual team members strengths and the company and individual challenges, we got to the point where the company experienced accelerated growth, increased client satisfaction, and expanded into new markets. 

Of course, the cost savings and operational efficiencies achieved further solidified the company’s reputation and supported its overall business growth.

Introduction: In the fast-paced world of ecommerce, scaling up to meet growing customer demands can be both exhilarating and challenging. Our case study focuses on the lessons learned while implementing an automation framework for a leading ecommerce company. However, during this scale-up phase, the company encountered several testing challenges that impacted its ability to deliver quality solutions efficiently. This case study delves into the experiences of a testing team involved in the scale-up and highlights the lessons learned from the testing process.

We don’t talk enough about how we learned to create solutions which now can be considered successful so this case study focuses on a failure scenario and outlines the lessons we learned. 

Key challenges: 

The company encountered several testing challenges during the scale-up phase. Clients expressed dissatisfaction with project delays, product availability issues, crashes, and downtimes. These issues resulted in a lack of confidence in the company’s products among both developers and clients. Moreover, the absence of test automation and skilled automation engineers further hindered the testing process.

Recognizing the need for improvement, we proposed a test automation framework tailored to the company’s specific requirements. The framework aimed to reduce repetitive manual tasks, allowing test engineers to focus on areas where automation would yield better results. Additionally, it presented an opportunity for skill development among test engineers. 

However, despite its potential benefits, the proposal faced resistance and was ultimately rejected by most team members.

Lessons Learned: 

From this experience, an important lesson emerged: the success of a testing process or automation framework is contingent on the adoption and support of the teams involved. 

Out of this, we came with the following points to serve us and others going through a similar journey:  

• the success of a framework is not really related to its ability to solve a technical challenge 
• excluding team members from decision-making and development stages leads to shorter time to deliver a framework but they won’t adopt it – won’t use it and won’t enhance it
• regardless of how amazing a test automation framework is, if it doesn’t address the teams pains , it won’t be relevant for them
• failure to include the team strengths and skills and preferences in the automation approach will lead to a framework on a shelf somewhere gathering dust or even worse, a belief that automation won’t work for them
• the  lack of collaboration and engagement in the process of designing and development of the automation will alienate the teams
• and to top these, not including the development and testing teams in the decision making process, it is nearly impossible to find that sweet spot in the level of automation which yields the right results for this particular team and organization

We’ve seen it time and time again, in most of the organizations inquiring for our automation services, automation is not the silver bullet everyone expects it to be. 

We’ve seen so many automation efforts gathering dust on various servers but more painful, we’ve seen so many organizations’ trust in automation decrease considerably in the past years. But there are KPIs from somewhere and they just inquire for automation to check some boxes. 

This makes it difficult to lead to a successful automation result because all the parties come to the table and they don’t even try to ask the right questions. 

Adding the scenario-based development techniques in the early stages of diagnosis we manage to avoid the mistakes we did before and we also help our clients define their scenarios in relation to automation. 

Conclusion: 

This case study highlights the lessons we learned while failing to deliver a successful test automation framework. While the framework met all the technical requirements and would have actually addressed most the challenges the company was facing, not considering the team,  we failed to deliver an actual value to them. This goes beyond testing and automation. Clients come to us with specific requests but some of them won’t fit in their teams. 

Learning from this experience and some others, we consider the teams in our processes, we consider their wants and pains as well as their skills and strengths.