After creating some automated tests it is commonly discovered that the IDE (Interactive Development Environment) you chose to write the tests has a pretty good test running/reporting tool. It is easy to stop there and declare that whenever we need to run any tests we can simply let someone on the automation team know and they can run them on their computer. You may wonder why some people do the extra work of getting the tests to run somewhere else. Here are 5 reasons to not run automated tests on your machine:
- Test results access
The purpose of a test is to find out if something is working or if there is a problem. A test is useless if you cannot view the results. Common questions after a test is ran: “What tests were ran?”, “What are the results of the tests?”, “When did you run the tests?”, “What were the errors from the failed tests”. When a test is ran on a private machine, that puts the burden on the person who ran them to communicate the results and answer all the questions. This may not seem too daunting until you consider you don’t know which people will be concerned about the result for that specific test run.
When the tests are ran from a central location that everyone has access too, it allows everybody to have access to the results. Any interested parties can see for themselves what the current results are.
- Multiple user access
There may be someone working on a certain issue to correct a failing test, or someone who wants to verify that a certain part of the system doesn’t have any problems. These examples do not require a run of the full test suite, but does require certain specific tests to be ran. It could turn into someones full time job to run the tests and report results.
Making the tests available for everyone on a central machine allows for a self-serve environment. If someone needs to verify a certain test is still working they can easily check it. Simple as that, they don’t need to pull other people to help or try to get the tests downloaded and running on their machine.
- Running tests in parallel
In the beginning stages of making automated tests you may have 5 tests. At this stage the idea of running them in parallel does not seem like a big deal. It does not take long for a test suite to grow from 5 to 100. If each test takes about 2 minutes then you are looking at over 3 hours of tests. You could try to run them in parallel on your local machine but with tools like selenium running more than one at a time causes it to trip over itself (it does not workout).
By running the tests on a build server it will be able to spread the load of tests over several computers. This allows for much faster results, and build servers allow for adding compute nodes if the current amount is not enough. If we approach those 100 tests again but we distribute them over 10 computers, we will have our results of all the tests in a mere 20 minutes.
- Free your computer
There are many different types of tests. Many of the UI testing tools do a decent job at not letting other actions on the computer affect the test, but they cannot completely isolate. Something as simple as moving your mouse over a browser, or scrolling on a different part of the screen could affect the test and cause it to fail. Many people simply do not touch their computer while the test is running to prevent any false failures. Wouldn’t it be great if you could run a test without it locking up your computer?
Tests running on the cloud have their own dedicated machine. You can run tests all day long and it will never slow you down.
- Continue to build the automation project
When you run the test locally on your own machine, you have to have the current code on your machine ready to run. This means if you are working on the project, you may have the code in a non-runnable state when tests need to be ran, causing you to revert some of your changes or make a way around what you are working on till you have finished running the tests.
The build server will pull the code from wherever it is being saved, not your local machine. You can do anything you want with the code locally and it will not affect the test runs till you are ready to push the updates to where the server is getting code.
Bottom line: to make your tests more effective, use the cloud.