How to Choose the Right Test Automation Tool, Brought to You by Provar

* This is part one of a two-part series that takes a deep dive into choosing the right test automation solution for your team’s needs. Part One focuses on what you should look for in the technology itself, while Part Two will focus on looking beyond the technology and evaluating the vendor’s business model.

The guide below will provide clarity on what to look for, as well as what questions to ask to ensure you select the right solution for your team’s needs. We will start by discussing the technology, of course.

The key considerations for a product evaluation of a Salesforce-specific testing solution fall into four categories: 

Let’s break this down in a little more detail.

Test Resilience or Fragility

Salesforce automatically updates its platform multiple times per year, which can cause automated tests to break unless the testing system was designed to operate with Salesforce. And when tests break, you’re in trouble.

Here are some questions you can ask to decide if a test automation solution has the test resilience or fragility you need:

  • Can a test created for Visualforce run without modification under Aura (Classic and Lightning)?

  • If Salesforce makes a change to the way a page is rendered (changes to the HTML, CSS, Javascript, etc.), will a test continue to work without modification?

  • If we make a change to a Salesforce layout, will a test created for the previous page layout work without modification?

  • If Salesforce changes an API used in a test, will it continue to work without modification?

Polymorphism and Reusability

The technology that makes tests resilient also enables polymorphism, where a single test can run across numerous contexts, and reusability, which reduces the number of tests you need to create and maintain. Reusable polymorphic tests make for a higher quality solution that evolves more quickly and easily.

Here are some questions you can ask to gauge a test automation solution’s polymorphism and reusability:

  • Can I create a single test and run the test for several different user profiles or page layouts without modification or creating additional tests?

  • Will a test that I create in English run without modification and without creating duplicate tests in another language like French or Japanese?

  • Can a single test run without modification or creation of additional tests across PC, Mac, and mobile users on several different browsers?

  • Can a test be used to validate field visibility and accessibility based on user permissions for several profiles/permission sets without modification and without creating additional tests?

  • Can a test created in the development environment run without modification and without creating additional tests in the validation or production environments?

Ease of Use and Learning

Ease of use and learning is another big factor to consider in your evaluation. Many Salesforce users are considered “citizen developers,” meaning they may not be experts in programming languages and frameworks. It’s essential that your test automation solution is usable by your entire team so you can deploy your tests with confidence.

Here are some questions you can ask to see if a test automation solution champions ease of use and learning:

  • Are test steps added from a Salesforce screen (“right-click to add test step”)? Can you build your test in one location without having to toggle back and forth between multiple platforms or screens?

  • Can I add a test step and see the results before adding the next step? Can I step back to remove a prior step? Can I pause and resume a test as I run it?

  • If I select a button in the Salesforce user interface and right-click to add a test step, does the test step automatically default to click? Does it default to “set” for an input field? What other default actions should we capture? Does your approach automatically detect the locators, such as elements you want to test, before you test, or do I have to label these elements? When running a test script, do you see the test pass/fail in real time, or do you need to fix it and run it over again to confirm the issue is resolved?

Testing Adjacent Systems

Finally, when it comes to the technology itself, it’s essential that you evaluate a test automation solution’s ability to test adjacent systems. Every Salesforce platform connects to other enterprise systems and custom integration points, so it’s important that your testing solution works well with other systems so that you can test your workflows end-to-end.

Here are some questions that’ll help you evaluate whether your test automation solution is ideal for testing adjacent systems:

  • Can I test that a triggered email was sent? Can I test email to case flow?

  • Can I verify that a Salesforce action triggered a connected action in an external system, such as creating an invoice in an ERP? Can I verify that an action on our website created the correct records/activities in Salesforce?

  • Can I test that an external API has been called from a Salesforce customization?

SponsorsMarisa Hambleton