DeepCode and Snyk joined forces. Find out more in our blog.

SummerAcademy on Debugging - 2.1.10 Dynamic Testing

Thursday, September 10th, 2020

In contrast to static testing, dynamic testing runs the software. In order to complete a successful dynamic test campaign:

  1. The software needs to compile and start - at least in parts
  2. You need at least parts of the infrastructure like databases, message queues, app servers, CICD …
  3. You need a test plan

There are two basic types of dynamic tests: Manual and Automated. Automation takes away the human effort (and source of error) but is a substantial work. Also, dynamic tests are prone to break due to changes in UI or server setup. Manual tests are easier to set up but are taunting (redo the same thing thousands of times … see) and humans lose concentration.

Here is a list of 9 tips for your next dynamic test:

  1. A test plan should be informed by (1) what parts of the application saw change recently, (2) the top use-cases of your app, (3) possible threats to safety, security, legality, compliance. It is actually a good practice to show that you take your duties for real when covering them with test cases.
  2. Try to automate as much as possible. Use frameworks like Selenium to automate. Programming Selenium scripts that do not easily break and can be maintained is a job in itself. It is a good practice to start with manual testing and if the test case works and is important, automate it.
  3. Make your code automatically testable. One example is to have unique IDs on every control in your website so that your testing framework can easily inject click events.
  4. Use a test management tool (or at least use your task management like JIRA). You will end up having sequences of test cases that need to be followed. As an example, you need to navigate to the dashboard and log in before you could do anything (and also a good thing to automate). Also, running test cases will alter the system status (adding new database entries as an example).
  5. Writing good test cases is not simple. Make sure your test description has a common form, are easy to follow. See in the resources on best practices.
  6. It is a good practice, to chain test cases so they follow a typical process within the application which is called an End-to-End or E2E test. This helps to find issues beyond simple issues but also integration problems with internal and external service providers.
  7. For load testing / functional testing, check out Artillery
  8. Extract and automate test cases to be used on production on a regular basis. If one of those cases catch wrongly issued invoices before they get sent to customers, any manager in your house will be a fan of it.
  9. E2E tests normally need sub-tests that provide a certain system state as a result (see example below). So, other than unit tests, these subtests have a certain order as they leave the system in a different state (you might be able to push your system into certain states, so literally mocking a passed test). This also means that failing one test means to stop testing as all subsequent tests can only fail.
    Example of an E2E for an e-Commerce-Shop:
    1. The user navigates to the landing page
    2. The user browses the catalog
    3. The user selects 3rd item on browse and clicks on details
    4. The user adds 3 times of this item to the shopping cart
    5. The user selects checkout and is shown cart
    6. The user is asked to log in and logs in
    7. The user pays by credit card
    8. The user receives order acknowledgment on the website and email
    9. The user closes the browser window

Key Take-Aways

It all starts with a test plan and well-written test cases. Manual tests are fast to set up and provide feedback on the test plan and test cases but are expensive and exhaustive long-term. Think replacing manual tests by automated tests where it makes sense.

Resources

Selenium - On test automation API test automation with Postman Top Test Management tools Writing good Test Cases

Testing JavaScript Course