Introduction 

In this case study, we examine how a fintech company tackled development bottlenecks and team frustrations by improving integration testing strategies. The product had been on the market for over three years, with evolving functionalities to stay competitive and comply with industry regulations. However, the development team had changed over time, leaving only a few of the original members. Those who initially built the system had either transitioned to other projects or left the company entirely. 

Our goal was to identify obstacles in the development process and address the primary source of frustration: the team’s lack of confidence in their releases. Productivity is multifaceted, but we focused on improving the stability of integration tests to ensure developers could deploy changes without fear of regression. To achieve this, we leveraged scenario-based development, allowing us to methodically identify the most impactful changes to improve team confidence and product stability. 

Growth Trajectory and Challenges 

With a live product, the company needed to continuously enhance its functionalities. However, the testing structure had become increasingly difficult to manage, making confident deployments a challenge. The testing process consisted of both API-level and UI-level tests, but key issues arose: 

• UI tests covered only a fraction of the system’s full functionality, though they provided good coverage for the UI itself. 

• API tests were divided into two categories: those testing exposed microservice endpoints and those testing endpoints accessed within microservices. 

• The extent of actual functionality coverage was unclear. 

• Tests within each service mixed unit tests with component tests, making it difficult to assess coverage and reliability. 

• Many integration tests for the full system were undocumented and unstable, yet they were considered the baseline for existing functionality. 

• These tests frequently failed due to specific configuration issues and lacked proper integration with deployed development and testing environments. 

• The development team lacked confidence in their ability to introduce changes without breaking existing functionality. 

Solution Identified Using Scenario-Based Development 

To address these challenges, we used scenario-based development to systematically assess pain points and prioritize improvements. This approach helped us understand the broader impact of changes and determine the most effective solutions that would yield maximum benefits while minimizing disruption. 

Rather than replacing the entire testing framework, which would have been costly and time-intensive, we focused on stabilizing existing tests and establishing a clear baseline for future development: 

1. Identifying Frequent Failures: We analyzed recurring test failures to understand their root causes. 

1. Defining a Baseline for Test Setup: By documenting and standardizing testing configurations, we reduced inconsistencies across environments. 

1. Streamlining Deployment and Testing Processes: We integrated tests more effectively within the development pipeline, ensuring they were reliable indicators of system stability. 

1. Prioritizing Fixes Based on Scenario Outcomes: By mapping out realistic development and testing scenarios, we determined which improvements would have the most immediate impact on developer confidence and product reliability. 

The scenario-based development approach allowed us to build trust in the test suite so that developers could confidently implement changes without introducing defects. Over time, this foundational work could support further improvements in testing frameworks, but our immediate focus was on enabling steady and predictable releases. 

Key Lessons Learned and Future Recommendations 

This experience reinforced several key takeaways: 

• Scenario-based development is a valuable tool for identifying high-impact changes. By mapping real-world use cases, we could determine the most urgent fixes. 

• Stabilizing existing tests is often more effective than rebuilding from scratch. A complete overhaul may be tempting, but addressing the most immediate concerns first can provide immediate benefits. 

• Clear documentation of test coverage is crucial. Without clarity on what tests cover, teams cannot confidently rely on them for validation. 

• Integration tests should be reliable and properly configured in all environments. Frequent failures erode confidence and slow development. 

• Empowering the team leads to long-term success. Providing developers with reliable tests enabled them to focus on building new features rather than debugging fragile tests. 

Conclusion 

By applying scenario-based development, we identified key issues and determined a practical solution that maximized benefits for both the team and their product. Addressing instability in the testing process helped the development team regain confidence in their releases, allowing them to implement changes with a clear understanding of their impact. This improved overall productivity and morale. 

In the future, further refinements in test coverage and framework optimization could enhance efficiency even more. However, at this stage, our focus on stabilization—guided by scenario-based development—proved to be the right step toward building a stronger and more resilient development process. 

Introduction:  

In this case study, we examine the journey of identifying the cause of the team demotivation and release delays by applying scenario-based development in a fintech company for one of their teams. The team was created to modify one of the company’s products to allow for deployment and customizations for different countries and legislations.  

Growth Trajectory and Expansion: The initial product was implemented to be used in a specific country based on specific requirements. At the time, nobody thought about the potential expansion into other markets. And as it usually happens, some of the implemented code as well as various tools were tied to legislation particular to the initial market. When the opportunity to expand into other markets arose, the company employed a specific team to deal with various refactoring needs, replatforming and customizations, while the original team focused on the original market’s new requirements.  

In this case study we will focus on the improvement of the development process of the new team by implementing a scenario-based development framework to address their challenges. 

Key Challenges:  

The team encountered several challenges during their tenure. These challenges included: 

• unclear, unorganized, outdated and repetitive technical debt items 

• miscommunications between the original and new team 

• extended project timelines due to unnecessary rework during development 

• team dissatisfaction due to the small or inexistent gains compared to the amount of work from tacking items in the technical debt log  

All these things happen in most products because most of us focus on delivering new functionalities to keep up with the market demands. And tacking technical debt is one of the efforts we admire and we are happy to support those embarking on this journey. The gains long term are difficult to quantify but they well exceed the effort.   

Solution identified:  

We started by analysing the technical debt backlog and the strategy of the team while observing each individual team member’s skills and interests. We involved the team as well as product and scrum teams to identify priorities and pains. 

We soon discovered that the majority of the issues were due to improper prioritization and analysis of the items in the backlog. The effort was tackled from the product functionality perspective instead of the product architecture and coding practices and needs. The logic made sense, in theory, because any development effort usually concerns the quality of the product. But considering the majority of the items in the technical debt, they required a different logical grouping and prioritizations.  

The point was to address as much technical debt as possible while avoiding duplication of effort and keeping the product functional in this process.  

To tackle these challenges, we recognized the need for improved communication of requirements and proper prioritisation in line with the actual expectations. 

Based on the team’s skills, interests and preferences we implemented a scenario-based framework.  Looking at the technical debt backlog, analysing the product’s needs and a logical grouping of the items, and discussing with team members and product owners and stakeholders, we created the following:  

• a new organized and prioritized technical debt backlog 

• dependencies added between various item groups 

• prioritized based on the bigger gains for a group of items 

• assigned them per sprints according to the skills and interests of each individual team member 

• we refined the framework based on the team feedback 

This approach enhanced their understanding, encouraged active participation, and facilitated a comprehensive review of the requirements. 

Additionally, the team felt empowered as they were contributing to the overall increase in quality of the product. Their efforts also supported the original team in gaining more confidence in the code while adding new functionalities. The practices created by addressing the technical debt were adopted by the original team as well.  

But the most important gain we noticed was the improved team satisfaction. They didn’t only address the technical as a chore but they did improve the software development practices for the entire product and they improved the quality and stability and reliability of the product.  

The implementation of a scenario-based framework for addressing technical debt transformed the team’s approach to managing and improving the product. By reorganizing the backlog, prioritizing tasks for maximum impact, and fostering collaboration, the team not only enhanced product quality but also set a precedent for effective software development practices. 

Key Lessons Learned and Future Recommendations:  

This experience taught us lessons we added to our current scenario-based development approach: 

• tackling technical debt without proper grouping or prioritization can result in duplication of effort and a lack of measurable progress 

• prioritizing items based on product architecture and coding needs rather than immediate functionality proved more effective for long-term gains 

• assigning tasks based on individual skills and interests increased motivation and ownership 

• active participation in decision-making improved team morale and resulted in better quality output 

• scenario-based frameworks help address complex challenges – a structured approach to tackling technical debt enabled the team to identify dependencies and logical groupings of tasks. 

It is imperative to create and maintain an updated technical debt backlog organized by architecture needs and logical groupings, rather than functionality alone.  

It is important to celebrate and communicate the value of addressing technical debt:  

• Regularly measure and communicate the impact of resolved technical debt (e.g., improved performance, reduced bugs). 

• Share success stories across the organization to foster a culture of quality and continuous improvement. 

Conclusion:  

This case study underscores the value of empowering teams to tackle technical debt with a structured, feedback-driven process. Beyond resolving existing issues, the initiative fostered higher team morale, stronger cross-team collaboration, and better long-term stability and reliability for the product. These lessons provide a blueprint for effectively managing technical debt and improving development processes in similar contexts. 

Finding a scenario-based development framework that suits both your organization, teams and users is fun and challenging and so worth it. 

Introduction:  

In this case study, we examine the journey of identifying the cause of the team demotivation and release delays by applying scenario-based development in a fintech company for one of their teams. The team was created to modify one of the company’s products to allow for deployment and customizations for different countries and legislations.  

Growth Trajectory and Expansion: The initial product was implemented to be used in a specific country based on specific requirements. At the time, nobody thought about the potential expansion into other markets. And as it usually happens, some of the implemented code as well as various tools were tied to legislation particular to the initial market. When the opportunity to expand into other markets arose, the company employed a specific team to deal with various refactoring needs, replatforming and customizations, while the original team focused on the original market’s new requirements.  

In this case study we will focus on the improvement of the development process of the new team by implementing a scenario-based development framework to address their challenges. 

Key Challenges:  

The team encountered several challenges during their tenure. These challenges included: 

• changing or unclear requirements during the development phase 

• miscommunication between the original and new team 

• team dissatisfaction due to wasted entire implemented features because the requirements were not ready 

• extended project timelines due to unnecessary rework during development 

Solution identified:  

While the company was using agile development, we focused on the testing and release phases of the development to discover the pains. We involved the development teams as well as product and scrum teams to create a more comprehensive understanding of the issues the product teams were facing.  

We soon discovered that the majority of the rework was due to ambiguous requirements. While it is generally fine to have a direction and involve the various skills in the team to define a more clear path, in this case, even when the team aligned on a path, they often couldn’t align with the different market requirements. At times because of the novelty of the product in the market, there were no clear expectations. At times because the other products this product was integrating were not actually ready.  

To tackle these challenges, we recognized the need for improved communication of requirements and expectations from stakeholders.  

Based on the team’s skills and preferences we implemented a scenario-based framework.  Looking at their previous deliverables and discussing with team members and product owners and stakeholders, we created the following:  

• in backlog refinement meetings, we defined a clear path for each story with clear expectations of the behaviour upon completion of the story 

• in the same meetings, we identified dependencies for each story 

• we added follow-ups with dependencies and stakeholders to ensure the path identified matches the expectations 

• we defined a decision matrix to start implementation by considering release dates from dependencies availability, requirements compatibility with the product, requirements readiness, etc. 

• we refined the framework based on the team feedback 

This approach enhanced their understanding, encouraged active participation, and facilitated a comprehensive review of the requirements. 

Additionally, with some of the proposed requirements not being clarified enough to be implemented, the team had more time to improve the testing coverage of the product as well as refactor outdated code and therefore improve the overall quality of the product.  

But the most important gain we noticed was the improved team satisfaction. The fact that they were creating new functionality and then reworked on removing it was taking a toll on the team satisfaction and on the quality of their work.  

The implementation of scenario-based development framework resulted in notable cost savings and operational efficiencies for this team. By identifying issues early in the analysis and design phases, we prevented major system failures from reaching the development stage. This proactive approach reduced the need for costly post-production fixes and shortened the time to market for fixes and updated functionalities. Additionally, the streamlined testing process allowed teams to allocate more time to testing delivered functionality, resulting in higher quality products and increased operational efficiencies. 

Key Lessons Learned and Future Recommendations:  

This experience taught us lessons we added to our current scenario-based development approach: 

• mapping scenarios improved communication and allowed the team to visualize the flow and interdependencies between components 

• identifying dependencies and tracking their progress in early stages helped prevent misalignment and delays 

• allowing team members to participate in defining paths and refining processes increased their sense of ownership and job satisfaction 

• improved morale directly contributed to higher-quality work and shorter development cycles 

• by integrating scenario-based development, the team gained clarity and focus on deliverables, reducing unnecessary work 

It is imperative to introduce retrospectives to gather input on the process and improve it based on the team’s experience.  

It is relevant to stay open to evolving methodologies as the market, product, and company grow and choosing the options which fit your needs best.  

Conclusion:  

The implementation of a scenario-based development framework significantly improved the development process, team morale, and product quality for the fintech company’s new team. By addressing ambiguous requirements, enhancing communication, and proactively managing dependencies, the team minimized wasted effort and reduced release delays. Additionally, focusing on iterative refinement and fostering collaboration empowered the team to deliver scalable, high-quality solutions while maintaining satisfaction and motivation. This case study highlights the critical importance of clear requirements, structured processes, and proactive engagement in overcoming common challenges in software development, paving the way for future scalability and efficiency. 

Finding a scenario-based development framework that suits both your organization, teams and customers is fun and challenging and so worth it. 

Introduction: In the fast-paced realm of ecommerce, the challenge of scaling up operations while maintaining quality and efficiency is ever-present. This case study sheds light on the success story of an ecommerce company implementing the right amount of automation. Faced with challenges that impeded project delivery and affected product availability, the company turned to test automation as a solution. Through effective implementation and active team involvement, the company achieved remarkable enhancements in testing efficiency, product quality, and overall business growth.

Key Challenges: 

The company faced several testing challenges, including 

• client dissatisfaction with project delays
• product availability issues
• increased team turnover rate as they were overworked and overwhelmed
• and a lack of confidence in the company’s offerings

Additionally, the manual testing workload was repetitive and time-consuming, resulting in limited focus on exploratory testing and frequent UI-related issues.

The solution identified: 

We started evaluating the pains of the teams by paying attention to the emerging patterns in our discussions. We recognized the need for a more efficient and effective testing process, but we mainly identified a short path to improve the situation. We then spearheaded the development of a tailored test automation framework. 

We considered the shortest path to yield results initially and we aimed to:

• reduce repetitive manual tasks
• decrease QE fatigue
• empower the testing team to focus on critical areas that required human expertise

As automation is not a silver bullet for all companies and products and teams, we focused on the level of automation sufficient and necessary for this particular organization.

Importantly, the development of the automation actively involved team members in the development process, incorporating their feedback and ensuring the framework aligned with their needs and motivations.

We also added the knowledge gained from the company’s portfolio as to include in our framework the scenarios relevant to customers. 

Focusing our efforts in the development of a bespoke test automation framework we included the relevant repeating issues gathered from the client base, development and testing teams.

Manual testing efforts decreased as team members transitioned to automation tasks, allowing for greater productivity. 

With reduced regression issues, the team delivered more products to clients within shorter time frames. 

The improved overall quality and enhanced user interface led to increased client satisfaction and confidence in the company’s products. 

The automation framework facilitated exploratory testing, resulting in valuable insights and an overall uplift in product quality.

During the scale-up, the company primarily employed test automation for regression testing. By automating repetitive test cases, the testing team was able to free up valuable time and resources to focus on other critical aspects, such as exploratory testing and system performance improvements.

By identifying and resolving issues in test environments, major system failures were prevented from reaching production. Additionally, the automation framework facilitated the swift discovery and inclusion of client-reported issues into the regression suite, eliminating recurring problems.

Key Lessons Learned and Advice: 

A crucial lesson derived from this experience was the significance of team adoption and support for testing processes and automation frameworks. The active involvement of team members throughout the development cycle, coupled with their motivation and input, played a vital role in the success of the automation framework. 

Another very important lesson was to rely on knowledge from similar customers and the experience the teams already had in the organization. 

Focusing our efforts to tailor our framework to consider the teams strengths while solving their immediate problems was the key in getting the resulting automation adopted and supported by the teams. 

To ecommerce companies undergoing similar scale-ups, this case study underscores the importance of fostering collaboration and leveraging the expertise of team members.

Establishing an automation framework is merely the initial step; actively involving the team and ensuring their buy-in and ongoing adoption are imperative for true success.

Conclusion: 

This success story emphasizes the transformative power of the right level of automation in an ecommerce scale-up. Through the strategic implementation of a tailored automation framework and active team involvement, the company achieved remarkable improvements in testing efficiency, product quality, and customer satisfaction. 

By embracing effective testing practices and involving the team throughout the process, ecommerce companies can overcome scale-up challenges, optimize their testing processes, and drive business growth in the competitive ecommerce landscape.

Introduction: 

In this case study, we examine the journey of implementing scenario-based development in an ecommerce company specializing in developing ecommerce solutions for other enterprises. The company experienced significant growth during its scale-up phase by expanding its client base and venturing into new markets. 

Growth Trajectory and Expansion: As the company acquired more clients, it concurrently developed new ecommerce applications and functionalities to meet their diverse needs. The scale-up involved managing live products in production, delivering enhancements and fixes to existing clients, as well as developing solutions for new clients.

In this case study we will focus on the improvement of the development lifecycle by implementing a scenario-based development framework to address their challenges.

Key Challenges: 

The ecommerce organization encountered several challenges during the scale-up phase. These challenges included:

• changing or evolving client requirements during the development phase
• miscommunication leading to different client expectations
• client dissatisfaction with the delivered products
• and extended project timelines due to rework during development
• loss of business from existing customers

Solution identified: 

While the company was using agile development, we focused on the testing phases of the development to discover the pains. We involved the development teams as well as product and scrum teams to create a more comprehensive understanding of the issues the product teams were facing. 

We soon discovered that the majority of the rework was due to changing requirements too frequently. While it is generally fine to change directions, in this case, the requirements were actually misunderstood by both the development teams as well as the customers.  

To tackle these challenges, we recognized the need for improved communication of requirements and expectations from clients. 

Based on the team’s skills and preferences we implemented a scenario-based framework.  Looking at the entire portfolio and discussing with team members and sales and customers, we created the following: 

• different templates for scenarios – focusing on a few key functionalities of their applications
• we tested these with additional teams teaching them what paths needed to be included and what level of detail is necessary
• we refined the framework based on their feedback 
• we shared it with customers and product to employ

Additionally, with the customers involved, we introduced wireframes to provide them with a visual representation of the expected user experience. 

This approach enhanced their understanding, encouraged active participation, and facilitated a comprehensive review of the ecommerce applications.

Product, development and testing teams utilized the scenarios in the analysis and design phases and continued to refer to them throughout the development cycle. 

They were used in the communication with the customers as well to align development with client expectations, minimize assumptions, and identify any gaps in requirements early on.

The implementation of scenario-based development framework resulted in notable cost savings and operational efficiencies for this ecommerce organization. By identifying issues early in the analysis and design phases, we prevented major system failures from reaching the development stage. This proactive approach reduced the need for costly post-production fixes and shortened the time to market for fixes and updated functionalities. Additionally, the streamlined testing process allowed teams to allocate more time to testing delivered functionality, resulting in higher quality products and increased operational efficiencies.

Key Lessons Learned and Future Recommendations: 

This experience taught us lessons we added to our current scenario-based development approach:

• different skillsets had completely different assumptions when considering the same functionality
• identifying issues early in the analysis and design phases, we prevented major system failures from reaching the development stage
• the introduction of visual representations in the scenario-based development framework facilitated clarity and alignment throughout the development cycle
• automation using the scenarios resulted covers the relevant functionality and it is easy to maintain
• testing and development efforts focused on quality once the functionality was clarified enough by the scenarios resulted from the scenario-based development

For future scale-ups, it is imperative to embrace comprehensive communication practices and actively involve clients to ensure a shared understanding of requirements and expectations.

Conclusion: 

Collaborative efforts enabled by the scenario-based development, led to enhanced quality and reduced frustrations, resulting in improved client satisfaction. By addressing communication gaps, we significantly reduced rework and shortened delivery time for clients from over a year to less than six months. 

Moreover, the implementation of scenario-based development instilled confidence in customers, attracting more business and contributing to the company’s expansion into new markets.

Finding a scenario-based development framework that suits both your organization, teams and customers is fun and challenging and so worth it.

Introduction: 

In this case study, we delve into the implementation of scenario-based development in a fintech insurance company during its scale-up phase. The primary focus is on leveraging scenario-based development to drive automated testing, addressing the challenges faced by the testing team in handling a large volume of functionalities and diverse data combinations related to customer communications.

Pension and Insurance Company Context: The company had an in-house development and manual testing team responsible for testing their products and internal applications. However, the testing team struggled to cope with the extensive volume of functionalities, prompting the need for automated testing to cover a vast array of data combinations for customer letters. The scope of testing also encompassed different types of content for PDF files, emails, web applications, and internal applications.

Key challenges: 

During the diagnosis phase, it was discovered that the testing team relied solely on manual testing, using a limited amount of data. However, the testers displayed a strong willingness to understand and implement complex database queries, despite lacking coding skills.

Company’s challenges: The main goals were to develop an automation framework that would lead to:

• The creation of a customizable and user-friendly automated testing solution
• Empowering the client’s testing team to easily add or modify test scenarios without relying on development skills
• Achieving high test scenario coverage across various technologies and data sources
• Establishing a dynamic solution that leverages extensive information and logic stored in databases, coupled with customization capabilities offered by the testing framework.

Our challenges: 

The primary challenge involved developing a custom automated testing framework, as this approach was relatively uncommon in the industry, resulting in limited documentation. 

The framework aimed to intelligently navigate different paths based on data stored in the database, necessitating the use of reverse engineering techniques. 

Additionally, ensuring ease of use and expandability by the client’s testing team posed an additional challenge.

Solution identified: 

We employed our scenario-based development approach to clearly define the scenarios the automation would have to consider.

In this, we met with product, development and testing teams to understand their pains and their needs. We evaluated their skills, strengths and wants moving forward. We identified supporters of the automation and we involved them in the decision making process and the development phases. 

Together, we leveraged their strengths and included all of this in our framework. 

We defined clearly the scenarios they needed covered and we discovered the patterns which helped in our design.  

We then spearheaded the development of customized automation framework focusing on the following: 

• navigation in the web interfaces of both internal and external applications – utilizing Selenium WebDriver and custom Java libraries 
• validation of the content of the web application, PDF files, and emails – a method was devised to dynamically generate expected content based on static text and dynamic data retrieved from the database at runtime
• the generated content was then compared with the actual content in the tested application using the automated framework
• navigation in web application was based on generated data to enhance the paths took by users – decisions of paths were made during runtime depending on the generated data

The implementation of this framework significantly reduced testing time from days or weeks to just a few hours, while also increasing test scenario coverage from 5-10% to 90%. 

Following the implementation, the testing team’s responsibilities primarily focused on testing edge cases and creating new and complex database queries for new functionalities.

Key lessons learned: 

• Effective Communication: Improving communication of requirements and expectations from clients is crucial to prevent misunderstandings and frequent changes during the development phase.
• Scenario-Based Development: Implementing a scenario-based development approach allows for a comprehensive understanding of the product, facilitates alignment with client expectations, minimizes assumptions, and identifies requirements gaps early on.
• Early Issue Identification: By identifying issues early in the analysis and design phases, major system failures can be prevented from reaching the development stage, saving costs and reducing rework.
• Test Automation: Developing a custom automated testing framework tailored to the specific needs of the company significantly reduces testing time, increases test scenario coverage, and enables the testing team to focus on edge cases and new functionality.
• Collaboration and Involvement: Collaborative efforts between development, testing, and product teams, as well as active involvement of clients, lead to enhanced quality, reduced frustrations, and improved client satisfaction.
• Comprehensive Testing Practices: Embracing comprehensive testing practices, including scenario-based development and test automation, helps ensure a shared understanding of requirements and expectations, leading to higher operational efficiencies and increased confidence in software products.

These lessons highlight the importance of effective communication, scenario-based development, test automation, collaboration, and comprehensive testing practices in successfully navigating the challenges faced during a scale-up phase in the fintech insurance industry.

Conclusion: 

By harnessing scenario-based development, the fintech insurance company successfully integrated automated testing into its scale-up phase. The customized test automation framework provided extensive coverage and rapid testing for diverse functionalities across various applications. Automation became the cornerstone of the testing process, reducing manual efforts and ensuring comprehensive test coverage. This case study demonstrates the value of scenario-based development in driving automated testing, resulting in notable operational efficiencies and increased confidence in software products. 

Introduction: 

In this case study, we explore the journey of an ecommerce company specializing in systems integration services and solutions. During its scale-up phase, the company experienced remarkable growth, expanding its client base in the United States and venturing into European markets. 

In this article we’ll focus on the testing ideas and practices we implemented in this organization to support them during their scale up.  This, of course, will translate in company’s reputation, customer satisfaction, and overall business growth.

Growth Trajectory and Expansion: The company started with a few clients in the United States and was trying to increase its customer base while expanding into new markets. 

During the scale-up phase, the company developed new ecommerce applications  for new customers and introduced new functionalities to their existing clients and applications. 

They also offered support for various clients with their applications live in production, releasing enhancements and fixes to address customer feedback.

Key Challenges: 

The company encountered several challenges during the scale-up phase and they all could be sum up to the clients being unhappy with their products and their delivery times. 

Client dissatisfaction arose from:

• delayed project deliveries
• frequent product unavailability due to crashes and downtimes
• end customer journey was far from flawless

We traced these issues to the testing practices as well as development and devops. 

We will continue to focus on the testing parts although changes happened in all the teams to address these challenges. 

For the testing discipline, we identified the following challenges: 

• the existing processes and documentation were inadequate to handle the increasing demands and complexities of the scale-up
• collaboration between projects and testing team members was minimum
• each application in development for a client was assigned a small testing team (1-2 test engineers) focused on the application from inception to the go live
• repeating issues were discovered on various products
• development and testing teams were overwhelmed and overworked – fixing issues after promoting to production, leading to decreased confidence among both developers and clients.
• the absence of dedicated testing teams supporting live clients and the lack of test artifacts, both manual and automated, posed significant obstacles to ensuring quality control.

The solution identified: 

Looking at the challenges expressed both by the company and individual team members such as developers and scrum masters, we identified testing improvements to put these projects on the path to success.  

To address these testing challenges, we implemented specific strategies aimed at enhancing collaboration, knowledge sharing, and reusability of testing artifacts. 

Initially, we created smoke test suites for all live clients. 

Working with the teams involved in the maintenance of these products, we then identified the areas most impacted by the releases and gradually expanded the range of test artifacts in subsequent increments. 

We brainstormed to get to a fast and efficient way to create these artifacts and discover what other artifacts might support them better. By utilizing reverse engineering techniques and gathering information from team members, we reconstructed testing artifacts from existing projects, minimizing trial and error. 

We documented everything, we created test procedures thus enabling knowledge transfer. 

We leveraged the collaborative tools already available in the company – we used Confluence and Jira with add-on for testing artifacts (Zephyr) for the following: 

• confluence – documentation of testing process and practices 
• confluence – knowledge sharing like how to use various tools, what areas to test, investigation and proof of concepts of tools and practices 
• zephyr – to store test cases – both project specific as well as reusable ones
• jira – for defects – using them as a knowledge sharing base to prevent issues from old projects on the new ones

The tools are relevant but what we consider more important is that we enabled team members to work together efficiently and we nurtured a culture of knowledge exchange. 

With the repositories of reusable artifacts and the knowledge base we managed to create rapid onboarding of testing engineers, including contractors who could seamlessly augment resources when needed. 

Moreover, we facilitated smoother transitions for engineers moving across projects, minimizing ramp-up time and maximizing productivity.

What was in for the company with the solution above, you ask? 

• the company extended its reach into European markets and witnessed increased revenue from ongoing client support post-launch
• new projects from existing clients as they were happier with their live products
• reduced the number of issues discovered in production or later stages of development
• decreased frustrations within the development and testing teams, leading to more efficient project deliveries

Key Lessons Learned and Future Recommendations: 

The testing process during the ecommerce scale-up provided valuable insights and lessons. 

• an external perspective proved instrumental in identifying untapped potential within the teams and streamlining processes 
• focusing on what is already created by the teams is more relevant than focusing on what is missing
• the knowledge was already there, it just needed to be organized and shared
• focusing on the pains of the teams, it was easy to trace them to the pains of the clients and fix all of them together

Effective communication, team nurture and collaboration are pivotal to testing success, and the early adoption of these practices is crucial for future scale-ups.

Conclusion: 

One might consider that te implementation of robust testing practices played a vital role in the successful ecommerce scale-up. 

But we believe the star here was identifying the pains of the teams and their invaluable knowledge. Figuring out what they were doing well led us to the improvements we brought. 

Optimizing collaboration, and promoting knowledge sharing, in a manner that focused both on the individual team members strengths and the company and individual challenges, we got to the point where the company experienced accelerated growth, increased client satisfaction, and expanded into new markets. 

Of course, the cost savings and operational efficiencies achieved further solidified the company’s reputation and supported its overall business growth.

Introduction: In the fast-paced world of ecommerce, scaling up to meet growing customer demands can be both exhilarating and challenging. Our case study focuses on the lessons learned while implementing an automation framework for a leading ecommerce company. However, during this scale-up phase, the company encountered several testing challenges that impacted its ability to deliver quality solutions efficiently. This case study delves into the experiences of a testing team involved in the scale-up and highlights the lessons learned from the testing process.

We don’t talk enough about how we learned to create solutions which now can be considered successful so this case study focuses on a failure scenario and outlines the lessons we learned. 

Key challenges: 

The company encountered several testing challenges during the scale-up phase. Clients expressed dissatisfaction with project delays, product availability issues, crashes, and downtimes. These issues resulted in a lack of confidence in the company’s products among both developers and clients. Moreover, the absence of test automation and skilled automation engineers further hindered the testing process.

Recognizing the need for improvement, we proposed a test automation framework tailored to the company’s specific requirements. The framework aimed to reduce repetitive manual tasks, allowing test engineers to focus on areas where automation would yield better results. Additionally, it presented an opportunity for skill development among test engineers. 

However, despite its potential benefits, the proposal faced resistance and was ultimately rejected by most team members.

Lessons Learned: 

From this experience, an important lesson emerged: the success of a testing process or automation framework is contingent on the adoption and support of the teams involved. 

Out of this, we came with the following points to serve us and others going through a similar journey:  

• the success of a framework is not really related to its ability to solve a technical challenge 
• excluding team members from decision-making and development stages leads to shorter time to deliver a framework but they won’t adopt it – won’t use it and won’t enhance it
• regardless of how amazing a test automation framework is, if it doesn’t address the teams pains , it won’t be relevant for them
• failure to include the team strengths and skills and preferences in the automation approach will lead to a framework on a shelf somewhere gathering dust or even worse, a belief that automation won’t work for them
• the  lack of collaboration and engagement in the process of designing and development of the automation will alienate the teams
• and to top these, not including the development and testing teams in the decision making process, it is nearly impossible to find that sweet spot in the level of automation which yields the right results for this particular team and organization

We’ve seen it time and time again, in most of the organizations inquiring for our automation services, automation is not the silver bullet everyone expects it to be. 

We’ve seen so many automation efforts gathering dust on various servers but more painful, we’ve seen so many organizations’ trust in automation decrease considerably in the past years. But there are KPIs from somewhere and they just inquire for automation to check some boxes. 

This makes it difficult to lead to a successful automation result because all the parties come to the table and they don’t even try to ask the right questions. 

Adding the scenario-based development techniques in the early stages of diagnosis we manage to avoid the mistakes we did before and we also help our clients define their scenarios in relation to automation. 

Conclusion: 

This case study highlights the lessons we learned while failing to deliver a successful test automation framework. While the framework met all the technical requirements and would have actually addressed most the challenges the company was facing, not considering the team,  we failed to deliver an actual value to them. This goes beyond testing and automation. Clients come to us with specific requests but some of them won’t fit in their teams. 

Learning from this experience and some others, we consider the teams in our processes, we consider their wants and pains as well as their skills and strengths.