LeanFT comes out-of-the-box with all you need to start automation efficiently and with an acceptable learning curve to balance between learning and productivity. It already has the framework that supports end-to-end automation of multiple technologies, not just web.

We think that a Selenium framework would not make a good investment at this point in time. It can be a short-term compromise in case:

• you did not purchase HPE leanFT (or UFT) already.
• you only need to do some web tests and you do not plan to put together big end-to-end regression automated suites.
• you have a highly skilled development engineers that want to write a lot of boilerplate code just to be able to provide some basic functional automation functionality

Why do we think this? Find out in the infographic bellow:

What are taxonomies?

Nothing too complicated. They are hierarchical lists with root causes, failure signs, and other defect-related elements. Some find it easier to think of them as classifications.

They tend to vary in level of detail: from broad to specific (from incomplete or missing parameters to missing descriptions). Even though you might not know it, even your day-to-day software testing is influenced by some industry standard taxonomy. One of the most popular ones is Dr. Boris Bazier’s (if you’re curious consider this book.

Why use them? 

They make our (software testing) lives easier. Having a defect taxonomy allows us to both classify failures and determine the type of bugs we should test for. We can focus on a specific element and constantly test for it. Also, taxonomies can be linked with risk scenarios that need to be addressed while testing.

Taxonomies present an advantage when it comes to teamwork. Less experienced testers can be given test cases based on them – this will give them less room for error.

Simply put, a tester with taxonomy at his disposal has higher chances of success.

Other aspects

We might think that the better/ larger the taxonomy the more effective the testing will be.  Yet, as we mentioned in other articles (https://blog.euro-testing.ro/testing-errors/), we should always keep our eyes open for items that are not in the taxonomy. Watch out for invalid characters, “correct” error messages or check how the software handles error corrections (do I have to start the registration process from zero if I i entered something wrong at a certain point?). Experienced testers usually develop a “nose” for where things in software could break.

No taxonomy has a one-fits-all property – it’s  likely to require some modifications to fit the product your testing for. Consider the defects you want to target and their level of detail. If you had a similar software testing project you can get additional inspiration from it.  Usually, a decision has to be made between the level of detail and the redundancies in the list.

Defect taxonomies should be frequently updated. If they are used in similar circumstances, an additional benefit to having a taxonomy is that later on, test cases can be built on them.

Final considerations

Remember that for innovative software there may be few if any industry standard taxonomies available. Will you tweak an existing one or start from scratch?  How many items should the taxonomy contain? Should there be 5 test cases for one item or 50? Always keep your eyes (and your instinct) open when it comes to adequate coverage. If you want to hear more about our approach on these matters let’s get in touch.

The false positive – our tests are marked as failed even if they actually passed and the software functions as it should.  We report errors even though they don’t exist. Data tells us the software should not work as intended yet it does.

From our experience, this type of error has an insidious impact. While it doesn’t affect the software itself, they tend to upset the dev’s trust in the software testing process.

Some can even begin to question the software testing company’s expertise. However, it’s usually uninspired to penalize testers for false positives (or even base KPIs on this) because it can only lead to an undesired situation – testers being scared to report them because of possible backlash. Also, keep in mind that most false positives are related to unclear situations – e.g. missing documentation. As cliché as it might sound – it’s better to be safe than sorry.

The false negative - our test are marked as passed even though they failed. We detected no problems at the moment of the test, yet they were present. The software will continue to run with glitches embedded even though it shouldn’t have.

What can happen? In a best case scenario, we detect them at a later stage of tests and fix them. Bad case: we notice them after the software has been deployed.  Worse case: the bugs remain in the software for an indeterminate amount of time.

The main problem with these errors is that they can affect the business bottom line by “breaking” the software.

We think that one of the best ways of detecting false negatives is to insert errors into the software and verify if the test case discovers them (linked with mutation testing).

What can we do about it?

Some argue that reporting false positives is somewhat preferable to missing false negatives. This is because while the first keep things “internal” the second have wider business implications: from bad software to unhappy end-users.

We should keep in mind is that they are by nature hard to detect. Their causes can vary:  from the way we approached the test to the automation scripts we used and even to test data integrity.

From our experience, having test case traceability in place works best to prevent both them. When was the first time the failure showed itself? Can we track it back in time? Was it linked with extra implementations? Did some software functionalities change? Does the test data look suspicious? These questions usually help us figure out which test cases were most likely affected.

All things considered, we believe it all comes down being responsible in software testing. It’s important to actually care about the test and not just do a superficial track & report. 

If you think you might be dealing with false positives and negatives errors in your software tests and need some guidance, drop us a line.

The context

One of our banking sector clients, asked us to help them with a solution which implied the upgrade of JIRA.  We had to upgrade from JIRA 313 to JIRA 7010 while going through all four intermediary versions.  All this, while the three environments (DEV, QA and PROD) had to stay synchronized.  

Needless to say, this task was labeled as mission critical by the client.

The problem

The main challenge was not losing data along the migration and maintaining the client’s current workflows. While you might say that this is something typical in the day-to-day activity of a software testing company, it’s also where the problems started showing up.

By default, Jira does not come with a built-in configuration porting mechanism from one environment to another. This means we have to rely on an JIRA add-on. Yet add-ons sometimes come with bugs and/or compatibility issues of their own. We got a first-hand taste of this when, with only three weeks left before the project deadline, we noticed that part of the data we tried to migrate became corrupt. Also, something was blocking the synchronization of environments. While we managed to fix part of the problems the main issue remained: customer IDs were not in synch.

The road to the solution

From what we could tell, the problem pointed to configuration management actions – the JIRA add-ons we used were not working as expected. Quite a predicament – it seemed that the add-ons were both the problem and the solution at the same time.  

We had to look for a new add-on which would offer us a good analysis of all the changes we were about to make during configuration deployments. Also, it had to shows how they would impact the existing configuration.

After some research, we came across Botron’s Configuration Manager Plugin (http://www.botronsoft.com/products/configuration-manager/ ). Simply put, it enables automated deployment of configurations across JIRA instances. It also allowed us to take system configuration snapshots: basically the whole configuration of the JIRA instance (projects, workflows, and custom fields). These could then be used to merge the configuration of one JIRA instance to another.

This may come as a surprise, but it was the only tool in the market capable of doing this the right way.

Long story short, it fixed our synchronization issues. We successfully migrated all relevant elements from user IDs to validations and other functions.

Last but not least, it also comes with a friendly user interface and a reliable (and fast) team behind it.

All in all, it’s a time saver.

With so many options out there, you need to take some time to pick the automated testing tool best suited for your business. Before choosing a software testing automation tool, let’s take a look at some of the factors involved in the selection process.

Get specific

Break down your goals, and make sure the tool you’re considering can handle your needs. Analyze the current non-automated testing situation to determine where testing is not being done or does not appear to be sufficient. Does your team frequently run into the same problems when testing? Are you noticing certain gaps in the testing process?

Check functions

The right solution must live up to your business standards and deliver the functions users need. Look at past projects and talk to your team to determine testing needs. Make a list with the most important features that the testing automation tool should have.

Consider all parameters

Take into account the platforms and technologies already in use within your project. What are the OS your application is running on? Is it for desktop, web or mobile? You might require a desktop app that is supported by a mobile app. Consider all those parameters and check them against various tools. Choose a testing automation tool that integrates well with your current technology to avoid additional costs.

Make sure the tool is reliable

Software testing is all about reliability. You need good results, all the time. The automation tool you choose should be stable, error-free, and periodically updated. Also, will it likely continue to be on the market and provide support to users?

Check for upgrades

Find out if the tool has an upgrade soon to be released. Don’t invest money if there will be a better version of it shortly. You might want to negotiate with the provider a free or discounted upgrade, in case they come up with a superior product.

Check staff knowledge base

A solution can look good on paper, but be a headache to implement. Make sure your team has the required technical background to use the testing automation tool. Check if the solution comes with documentation and resources for your team to quickly learn its ropes. Finally, do you have enough time to train your team before implementing the solution?

Determine the costs

Make sure your budget can cover the entire costs demanded by the new tool:  the costs of learning, training, and the tool itself. Check with the provider if there will be any ongoing costs with using the tool, such as for upgrades and support.

Get feedback

Invest time in researching both online and offline and find out what others think of the tool you are interested in buying. Also, arrange a trial period so your team can get a hands-on experience with the testing automation tool. Ask for feedback on ease of use, results, and overall value provided.

These are some of the aspects and questions you should think about before choosing a testing automation tool. It should meet your needs both now and when scaling for the future, so it is crucial that you properly evaluate alternatives before making a decision.

For an in-depth analysis of your business needs and available solutions, contact us at Euro-Testing. We would be glad to give you all the information and assistance you need in picking the testing automation tool that best fits your requirements.

Better Quality Products

Better quality testing means better quality software, which translates into products that add more value to the customer. Remember that customers are willing to pay more for increased value. What’s more, when you sell a high quality product, you gain in reputation and brand image, things that are invaluable on the long term for the growth of any business.

Happier Customers

As you well know, the sale of a product doesn’t actually end with the sale. The client can ask for a refund if he or she is unhappy with the product. In addition, if the product is not reliable, you will have to spend money fixing or replacing it. When you add the costs, the bottom line is that it pays to produce a higher quality product, and software testing done right is the only way in which you can guarantee that what you’re offering is valuable and reliable to the customers.

Increase Sales

A good product needs less promotion than one with problems, because people recommend it and word-of-mouth is the most important promotion tool there is. Offering your clients a product that has been rigorously tested and quality checked means that you value them and go the extra mile to offer the best on the market. This will go a long way into not only getting new customers, but also retaining them.

Cut Costs

How do you cut costs with software testing when testing in itself is a service you have to pay for? First, software testing saves you money on the long run because it makes sure you use and sell software that is reliable and that doesn’t need constant fixing and patch-work. Think of the last time you compromised on a service or product and you’ll realize that you probably ended up spending more on the long run, either in actual money or in inconvenience.

Second, as mentioned in the previous paragraphs, a major benefit of software testing is that it allows you to remove errors and problems before the products get on the market. This can spare you of great headaches later when unsatisfied clients come knocking on your door. Customer support can be very expensive.

Third, by using automated software testing solutions where applicable, you gain in quality and decrease the costs of the service. Another advantage is that automated solutions produce more consistency, which means more quality for your software.

Improve User Experience

Whether the software is used internally or sold to clients, it has to be easy to use and understand. Only testers with experience can make sure that the software is designed in a way that allows users to follow a logical and intuitive path. Improved user experience also means that the software should be free of bugs and errors, which can be a source of frustration and inconvenience for users.

Choosing a professional software testing service, with a team that has the experience and know-how to produce the best possible quality for you, is guaranteed to significantly improve user experience and, as a result, to result in more satisfied clients.

Business Optimization

The bottom line is that software testing leads to business optimization. This is the biggest benefit of all, and it encompasses all the other benefits:

• More satisfied clients
• Customer retention
• Less costs with customer service
• Less costs with fixing
• Automation of processes
• Better quality and more reliable products
• Improved reputation and brand image

Euro-Testing has almost ten years of hands on experience with helping businesses optimize their operations through professional software testing solutions and tools. Our consultants are specialized in different areas of expertise, making sure that we can offer you the best service possible. Contact Euro-Testing for a consultation. We only take on projects that we are 100% sure that we can deliver successfully, so you can be sure that you will be able to benefit from all the advantages listed above.

One of the most common challenges for a business is to decide what type of software testing it needs based on different factors like project requirements, budget, timeline, expertise, and suitability. Since both manual and automated software testing offer advantages and disadvantages, this article will give you a glimpse into which of the two is best suited for your needs.

What is the difference?

Manual software testing means using a program or product as an end user would, making sure that all features work appropriately. It basically means that a tester has to run each individual program and set of tasks to see whether the outcomes are in accordance with the expectations.

Automated software testing uses pre-defined tools to run test cases in order to compare the results to the expected behavior. Once automated tests are created, they can easily be repeated and extended to perform complex tasks, which take longer or are impossible to run through manual testing.

When should you use manual or automated software testing?

When does it make sense to use an automated software testing tool over a manual tool? What can you gain by using automated software testing tools? Here’s a list of arguments that will help you better decide what method works best for your project.

Project requirements

Manual testing usually makes sense when you deal with small projects or have budget limitations. One of the most important advantage of manual testing is that the person performing the testing can pay more attention to details. Unlike a pre-defined script, this option allows the software that is being developed to be used as it would be upon launch, by real users. However, if you’re facing a large scale project, manual testing can turn into a tedious and time consuming process, not to mention budget consuming. In this case, automated testing proves to be both quicker and more effective because it streamlines the process and frees up resources for other core activities.

Financial implications

Manual testing might sound like a great solution cost wise. Nevertheless, while automation tools can be expensive in the short-term, they turn out to be a long-term investment. Not only do they achieve more than a human can in a given amount of time, but they also discover bugs faster. This allows your team to react more quickly, saving you both precious time and money. Moreover, automation provides greater consistency and less errors, which results in a more valuable final product, with less need for customer service.

Timeline

The main goal in software development processes is a timely release. Manual testing is practical when the test cases are run once or twice and there is no need for frequent repetition. On the other hand, if you need frequent testing, automated tests can be run over and over again at no additional cost and they are much faster than manual tests. Automated software testing can reduce the time to run repetitive tests from days to hours. This time savings translate into cost cuts for your business.

Sometimes, manual testing isn’t enough

Automated tests also allow you to test things that aren’t manually possible. For example, answering a question like ‘what if I had 500 accounts’, or ‘what if I processed ten transactions simultaneously’ can only be answered efficiently by using automated tests.

Final thoughts

Both software testing options present pros and cons. Before deciding, make sure that you consider your time, your resources, and the size of your project as well as the quality of the automated tools you’ll be using. Always remember though that combining both methods is an option. In fact, combining the two may be optimal for canceling out the others’ cons and developing the best software possible. For more in depth information on what type of testing is best suited for your project, contact Euro-Testing and we will give you an assessment of your needs and provide the best solutions for your particular situation.