Automation Testing vs Manual Testing

The general rule of thumb has always been to use common sense. If you’re only going to run the test one or two times or the test is really expensive to automate, it is most likely a manual test. But then again, what good is saying “use common sense” when you need to come up with deterministic set of guidelines on how and when to automate?

Pros of Automation:

If one has to run a set of tests repeatedly, automation is a huge win.
It gives one the ability to run automation against code that frequently changes to catch regressions in a timely manner
It gives one the ability to run automation in mainstream scenarios to catch regressions in a timely manner.
Aids in testing a large test matrix (different languages on different OS platforms).
Automated tests can be run at the same time on different machines, whereas the manual tests would have to be run sequentially.

Cons of Automation:
It costs more to automate. Writing the test cases and writing or configuring the automate framework one are using costs more initially than running the test manually.
Can’t automate visual references, for example, if you can’t tell the font color via code or the automation tool, it is a manual test.

Pros of Manual:

If the test case only runs twice a coding milestone, it most likely should be a manual test. Less cost than automating it.
It allows the tester to perform more ad-hoc (random testing).In my experiences, more bugs are found via ad-hoc than via automation. And, the more time a tester spends playing with the feature, the greater the odds of finding real user bugs.

Cons of Manual:

Running tests manually can be very time consuming
Each time there is a new build, the tester must rerun all required tests - which after a while would become very mundane and tiresome.