LeanFT comes out-of-the-box with all you need to start automation efficiently and with an acceptable learning curve to balance between learning and productivity. It already has the framework that supports end-to-end automation of multiple technologies, not just web.

We think that a Selenium framework would not make a good investment at this point in time. It can be a short-term compromise in case:

• you did not purchase HPE leanFT (or UFT) already.
• you only need to do some web tests and you do not plan to put together big end-to-end regression automated suites.
• you have a highly skilled development engineers that want to write a lot of boilerplate code just to be able to provide some basic functional automation functionality

Why do we think this? Find out in the infographic bellow:

What are taxonomies?

Nothing too complicated. They are hierarchical lists with root causes, failure signs, and other defect-related elements. Some find it easier to think of them as classifications.

They tend to vary in level of detail: from broad to specific (from incomplete or missing parameters to missing descriptions). Even though you might not know it, even your day-to-day software testing is influenced by some industry standard taxonomy. One of the most popular ones is Dr. Boris Bazier’s (if you’re curious consider this book.

Why use them? 

They make our (software testing) lives easier. Having a defect taxonomy allows us to both classify failures and determine the type of bugs we should test for. We can focus on a specific element and constantly test for it. Also, taxonomies can be linked with risk scenarios that need to be addressed while testing.

Taxonomies present an advantage when it comes to teamwork. Less experienced testers can be given test cases based on them – this will give them less room for error.

Simply put, a tester with taxonomy at his disposal has higher chances of success.

Other aspects

We might think that the better/ larger the taxonomy the more effective the testing will be.  Yet, as we mentioned in other articles (https://blog.euro-testing.ro/testing-errors/), we should always keep our eyes open for items that are not in the taxonomy. Watch out for invalid characters, “correct” error messages or check how the software handles error corrections (do I have to start the registration process from zero if I i entered something wrong at a certain point?). Experienced testers usually develop a “nose” for where things in software could break.

No taxonomy has a one-fits-all property – it’s  likely to require some modifications to fit the product your testing for. Consider the defects you want to target and their level of detail. If you had a similar software testing project you can get additional inspiration from it.  Usually, a decision has to be made between the level of detail and the redundancies in the list.

Defect taxonomies should be frequently updated. If they are used in similar circumstances, an additional benefit to having a taxonomy is that later on, test cases can be built on them.

Final considerations

Remember that for innovative software there may be few if any industry standard taxonomies available. Will you tweak an existing one or start from scratch?  How many items should the taxonomy contain? Should there be 5 test cases for one item or 50? Always keep your eyes (and your instinct) open when it comes to adequate coverage. If you want to hear more about our approach on these matters let’s get in touch.

The false positive – our tests are marked as failed even if they actually passed and the software functions as it should.  We report errors even though they don’t exist. Data tells us the software should not work as intended yet it does.

From our experience, this type of error has an insidious impact. While it doesn’t affect the software itself, they tend to upset the dev’s trust in the software testing process.

Some can even begin to question the software testing company’s expertise. However, it’s usually uninspired to penalize testers for false positives (or even base KPIs on this) because it can only lead to an undesired situation – testers being scared to report them because of possible backlash. Also, keep in mind that most false positives are related to unclear situations – e.g. missing documentation. As cliché as it might sound – it’s better to be safe than sorry.

The false negative - our test are marked as passed even though they failed. We detected no problems at the moment of the test, yet they were present. The software will continue to run with glitches embedded even though it shouldn’t have.

What can happen? In a best case scenario, we detect them at a later stage of tests and fix them. Bad case: we notice them after the software has been deployed.  Worse case: the bugs remain in the software for an indeterminate amount of time.

The main problem with these errors is that they can affect the business bottom line by “breaking” the software.

We think that one of the best ways of detecting false negatives is to insert errors into the software and verify if the test case discovers them (linked with mutation testing).

What can we do about it?

Some argue that reporting false positives is somewhat preferable to missing false negatives. This is because while the first keep things “internal” the second have wider business implications: from bad software to unhappy end-users.

We should keep in mind is that they are by nature hard to detect. Their causes can vary:  from the way we approached the test to the automation scripts we used and even to test data integrity.

From our experience, having test case traceability in place works best to prevent both them. When was the first time the failure showed itself? Can we track it back in time? Was it linked with extra implementations? Did some software functionalities change? Does the test data look suspicious? These questions usually help us figure out which test cases were most likely affected.

All things considered, we believe it all comes down being responsible in software testing. It’s important to actually care about the test and not just do a superficial track & report. 

If you think you might be dealing with false positives and negatives errors in your software tests and need some guidance, drop us a line.

While it would be wonderful if we could have unlimited resources for testing – from our experience this is wishful thinking. Choices have to be made, and most of the time we go after issues that could prove critical for the business. When we define risk, we look at two dimensions as defined by HPE ALM (https://saas.hpe.com/en-us/software/alm): Business Criticality and Failure Probability. The first measures how crucial a requirement is for the business and the second indicates how likely a test based on the requirement is to fail.

Assessing risks 

While there are many ways to approach risk assessment, we usually use HPE ALM because it’s a reliable tool and saves us a lot of time. It has an integrated questionnaire that allows us to determine the risk and functional complexity of a requirement and give possible values for each criterion plus a weight assigned to each value. This allows us to evaluate the testing effort and determine the best testing strategy.

In assessing risk, comparing the changes between two releases or versions is fundamental for quality assurance to identify the risk areas, reducing the total testing efforts, managing project risks, bringing lots of value with less effort and more efficient testing.

The testing team can explore the risks and provide their feedback on the test execution and whether or not to continue testing.

Advantages vs Disadvantages

For some projects, the big challenge is to accommodate the need to reduce development time, while maintaining the scope. Under these conditions, a smart risk testing approach is key in allowing the testing team to develop their software in a timely manner, making the testing effort more efficient and effective.

Dealing with the most critical areas of the system first will counteract the additional time and costs of solving those issues at a later stage in the project. And maximize on the fact that the time is spent according to the risk rating and original mitigation plans.

A faster time to market and reduction of cost per quality are more easily achievable with this risk-oriented approach.

Proper risk identification in the analysis process, prevents the negative impact that assessing a risk as too low or based on too subjective criteria, could have.

Identifying potential issues that could affect the project’s cost or outcome, create an efficient risk-based testing work and ensure better product quality.

Overall Benefits

Using a testing approach that takes risk into account, promotes some of the best practices in risk management, while conducting fewer tests with a more focused view on critical areas, higher testing efficiency, and increased cost-effectiveness.

We invite you to test these benefits out for yourself and try on this software testing approach for size. If the size fits don’t hesitate to share some of your best practices in risk assessment software with us at Euro-Testing.

Or if you are not sure what testing approach would suit you best, let us know here! And we will tailor the best solution for your needs.

The context

One of our banking sector clients, asked us to help them with a solution which implied the upgrade of JIRA.  We had to upgrade from JIRA 313 to JIRA 7010 while going through all four intermediary versions.  All this, while the three environments (DEV, QA and PROD) had to stay synchronized.  

Needless to say, this task was labeled as mission critical by the client.

The problem

The main challenge was not losing data along the migration and maintaining the client’s current workflows. While you might say that this is something typical in the day-to-day activity of a software testing company, it’s also where the problems started showing up.

By default, Jira does not come with a built-in configuration porting mechanism from one environment to another. This means we have to rely on an JIRA add-on. Yet add-ons sometimes come with bugs and/or compatibility issues of their own. We got a first-hand taste of this when, with only three weeks left before the project deadline, we noticed that part of the data we tried to migrate became corrupt. Also, something was blocking the synchronization of environments. While we managed to fix part of the problems the main issue remained: customer IDs were not in synch.

The road to the solution

From what we could tell, the problem pointed to configuration management actions – the JIRA add-ons we used were not working as expected. Quite a predicament – it seemed that the add-ons were both the problem and the solution at the same time.  

We had to look for a new add-on which would offer us a good analysis of all the changes we were about to make during configuration deployments. Also, it had to shows how they would impact the existing configuration.

After some research, we came across Botron’s Configuration Manager Plugin (http://www.botronsoft.com/products/configuration-manager/ ). Simply put, it enables automated deployment of configurations across JIRA instances. It also allowed us to take system configuration snapshots: basically the whole configuration of the JIRA instance (projects, workflows, and custom fields). These could then be used to merge the configuration of one JIRA instance to another.

This may come as a surprise, but it was the only tool in the market capable of doing this the right way.

Long story short, it fixed our synchronization issues. We successfully migrated all relevant elements from user IDs to validations and other functions.

Last but not least, it also comes with a friendly user interface and a reliable (and fast) team behind it.

All in all, it’s a time saver.

Testing Automation

As new and improved tools are released, testing automation is becoming the rule rather than the exception. Manual testing won’t be completely replaced because you need it for certain types of applications or creative solutions. However, testing automation is great in optimizing testing processes and in reducing costs while improving product quality quickly and without too much hassle. You should consider it for all your projects.

Software Testing Outsourcing

Software testing cost companies a lot of resources, in terms of time and money. To do good testing, you need know-how, specialized staff, tools and project management. Can companies somehow avoid dealing with these issues? The answer is “yes” and more and more businesses are contracting suppliers for their testing needs. The main advantages of testing outsourcing are cost reduction, less complex operations, better testing and ability to focus on core activities.

Specialization and Experience

Testers become more specialized through education and certification. They also require years of experience that allow them to tackle complex projects quickly, professionally and with good results. This is great news for companies because they can have improved products and user experience. What you need to know though is that these specialists are usually part of third party testing services, and you can rarely find them in companies where they would have little room to evolve. So if you’re looking to reap the benefits of good testing, look at teams that have extensive experience with testing, in general, and knowledge of your industry, in particular.

User Experience

When it comes to software development, the user experience is taking a front row. Everything has to work seamlessly and intuitively for the user, or else they will look for an alternative. In a world in which people have little or no patience for errors, bugs or intricate paths, testing is more and more focused on improving the user experience. Companies need to understand this and invest in UX. The problem is that few testing professionals are truly experienced with this, so make sure your business is on of those who gets their hands on people who know how to improve user experience through testing.

Agile Environment

Working in an agile environment means more flexibility and openness, more efficiency and shorter release cycles – all essential ingredients if you want to remain competitive in the marketplace. It’s no wonder that agile testing is a trend to follow. Agile testing recognizes that testing is an integral part of the software development process. Testers on agile teams collaborate closely with developers to find solutions and create products that are as close to consumer needs as possible.

We hope that you found this information useful. For more in-depth analyses on how you can make this trends work for your business and on what type of testing is best suited for your project, contact Euro-Testing and we will give you an assessment of your needs and provide the best solutions for your particular situation.

So testing is a necessity for any business that hopes to optimize its operations. However, testing is not a core competency for most companies. It is often performed by a dedicated part of the team, by someone who is specialized in something different but does this out of necessity, or it is outsourced. Which one is better and why?

In-house testing vs. outsourcing

What is the difference between having an in-house tester and outsourcing the service? Regarding operations, you either have people on your team doing the testing, or you give your requirements to a third party that takes over the testing part of the project. This third party will keep in contact with you and your team for updates and progress reports.

We discuss further some of the advantages of outsourcing and explain why this might be a better option for your company than doing software testing in-house.

Quality of testing

There is a difference in skill level and quality of testing when it comes to outsourcing. Often, your team simply lacks the know-how to perform software testing at the highest standards. Your alternatives are either to invest in hiring new people with the required know-how or to spend resources on training your existing staff. Any of these two distracts your team from their main tasks, on the one hand, and, on the other hand, results in extra costs.

Outsourcing also implies a cost, but you work with specialists who have a lot of experience with similar projects and are almost always guaranteed to work better and more efficiently than an in-house team.

Standardized solutions

What happens a lot is that for every project companies develop, they have to come up with testing solutions. These customized solutions take resources to develop, and they quickly become obsolete. What’s more, new team members and managers have to spend precious time learning the ropes of the custom solutions. Outsourcing eliminates this problem by providing standardized solutions, that have been proven to work and that remain updated and constant no matter how the in-house team changes.

Spee-up projects

A result of outsourcing is getting things done quicker. Because of standardization, specialization and years of experience, companies that specialize in testing can do a better job in less time. In this way, your project is ready for the marker with no bugs or errors and with the optimal user experience in record time. No need to stress how important it is for project efficiency and for the coordination between departments not to go over established deadlines.

Decreased operations complexity

With outsourcing, you simply have less day to day hassle in managing your team and the output. Outsourcing spears you of all the trouble of adding people to your team or arranging for special training. It also takes away the need to manage this essential yet complex activity and supervise the quality of testing.

Reduced costs

You might think that having someone in-house to do the testing will surely save you some money, compared to contracting a third party service. While in the short run you might be right, in the medium and long run you risk inducing far greater costs on your department and company.

Remember that the main idea behind outsourcing of any kind is cost reduction. When you outsource, you transfer part of your operations to another company, reducing complexity for your own. Moreover, this third party is a specialist in something that you’re not, so they will be more efficient in their operations, delivering the same or better quality, at a fraction of the cost that it would take for your company to do it.

Not least, better quality testing means better quality products. And while customers will pay more for more quality, they will stop buying and spread the word if they encounter a faulty product.

For all these advantages, we encourage you to give software testing outsourcing a thought. Contact us at Euro-Testing to get more information about the solutions that would fit your company. We can help you figure out whether outsourcing is suited for your needs, as well as provide you with costs estimates and potential benefits, so you make an informed decision.