All About API Testing !!!

All About API Testing !!!

What is API Testing:

API testing is entirely different from GUI testing and mainly concentrates on the business logic layer of the software architecture. This testing won't concentrate on the look and feel of an application.
Instead of using standard user inputs(keyboard) and outputs, in API Testing, you use software to send calls to the API, get output, and note down the system's response.

API Testing requires an application to interact with API. In order to test an API, you will need to

•     Use Testing Tool to drive the API
•     Write your own code to test the API

Set-up of API Test environment:
    API testing is different than other testing as GUI is not available, and yet you are required to setup initial environment that invoke API with required set of parameters and then finally examine the test result. Setting up testing environment for API testing seems little complex.
    Database and server should be configured as per the application requirements.
    Once installation is done, API Function should be called to check whether that API is working.

Types of Output of an API:

•     Any type of data
•     Status (say Pass or Fail)
•     Call to another API function.

Test cases of API testing are based on

•     Return value based on input condition: it is relatively easy to test, as input can be defined and results can be authenticated
•     Does not return anything: When there is no return value, behavior of API on the system to be checked
•     Trigger some other API/event/interrupt: If output of an API triggers some event or interrupt, then those events and interrupt listeners should be tracked
•     Update data structure: Updating data structure will have some outcome or effect on the system, and that should be authenticated
•     Modify certain resources: If API call modifies some resources then it should be validated by accessing respective resources

Approach of API Testing:

Following points helps the user to do API Testing approach:

•     Understanding the functionality of the API program and clearly define the scope of the program
•     Apply testing techniques such as equivalence classes, boundary value analysis and error guessing and write test cases for the API
•     Input Parameters for the API need to be planned and defined appropriately
•     Execute the test cases and compare expected and actual results.

What to test for in API testing

API testing should cover atleast following testing methods apart from usual SDLC process
    Discovery testing: The test group should manually execute the set of calls documented in the API like verifying that a specific resource exposed by the API can be listed, created and deleted as appropriate
    Usability testing: This testing verifies whether the API is functional and user-friendly. And does API integrates well with another platform as well
    Security testing: This testing includes what type of authentication is required and whether sensitive data is encrypted over HTTP or both
    Automated testing: API testing should culminate in the creation of a set of scripts or a tool that can be used to execute the API regularly
    Documentation: The test team has to make sure that the documentation is adequate and provides enough information to interact with the API. Documentation should be a part of the final deliverable 

Best Practices of API Testing:

•     Test cases should be grouped by test category
•     On top each test, you should include the declarations of the APIs being called.
•     Parameters selection should be explicitly mentioned in the test case itself
•     Prioritize API function calls so that it will be easy for testers to test
•     Each test case should be as self-contained and independent from dependencies as possible
•     Avoid "test chaining" in your development
•     Special care must be taken while handling one time call functions like - Delete, CloseWindow, etc...
•     Call sequencing should be performed and well planned
•     To ensure complete test coverage, create test cases for all possible input combinations of the API.

Types of Bugs that API testing detects

•     Fails to handle error conditions gracefully
•     Unused flags
•     Missing or duplicate functionality
•     Reliability Issues. Difficulty in connecting and getting response from API.
•     Security Issues
•     Multi-threading issues
•     Performance Issues. API response time is very high.
•     Improper errors/warning to caller
•     Incorrect handling of valid argument values
•     Response Data is not structured correctly (JSON or XML)

 

Tools for API testing:

Since API and unit testing both target source code, similar tools can be used for testing both.

•     SOAPUI
•     Chrome Extension POSTMAN
•     Python Scripting with Request Library
•     Java with RestAPI Lib
•     Runscope
•     Postman with jetpacks
•     Postman with newman
•     Curl
•     Cfix
•     Check
•     CTESK
•     dotTEST
•     Eclipse SDK tool- Automated API testing 
•     Robot Framework (key word driven framework)

Challenges of API testing includes:

•     Main challenges in API testing is Parameter Combination, Parameter Selection, and Call Sequencing
•     There is no GUI available to test the application which makes difficult to give input values
•     Validating and Verifying the output in different system is little difficult for testers
•     Parameters selection and categorization required to be known to the testers
•     Exception handling function needs to be tested
•     Coding knowledge is necessary for testers

Conclusion:

API consists of set of classes / functions / procedures which represent the business logic layer. If API is not tested properly, it may cause problems not only the API application but also in the calling application.

Best Practices for Successful Automation !!!

Some points worth considering for Successfull Automation !!!!!!!

Automation testing of repetitive tasks can be cumbersome and time consuming. Here automation come to rescue. There are lot of benefits of using automation which are out of scope for this article. Here we will be discussing the factors which make automation effective and efficient.
To automate the repetitive steps that a tester would take to exercise functions through the user interface of a system in order to verify its functionality. I am sure you have all seen tools like Selenium, WebDriver, Eggplant or other proprietary solutions, and that you learned to love them.

On the downside, we observe problems when we employ these tools:

•     Scripting your manual tests this way takes far longer than just executing them manually.
•     The UI is one of the least stable interfaces of any system, so we can start automating quite late in the development phase.
•     Maintenance of the tests takes a significant amount of time.
•     Execution is slow, and sometimes cumbersome.
•     Tests become flaky.
•     Tests break for the wrong reasons.

None of these problems is particularly bad, and the advantages of automation still outweigh the cost. This might well be true. We learned to accept some of these problems as 'the price of automation', whereas others are met by some common-sense workarounds:

•  It takes long to automate a test—Well, let's automate only tests that are important, and will be executed again and again in regression testing.
•  Execution might be slow, but it is still faster than manual testing.
•  Tests cannot break for the wrong reason—When they break we found a bug.
• While evaluating tool make sure that the limitation of the tool are affecting least of  the  functionality of Application under test.

How to overcome these common issues:

Most of these problems are rooted in the fact that we are just automating manual tests. By doing so we are not taking into account whether the added computational power, access to different interfaces, and faster execution speed should make us change the way we test systems.

Considering the fact that a system exposes different interfaces to the environment—e.g., the user-interface, an interface between front-end and back-end, an interface to a data-store, and interfaces to other systems—it is obvious that we need to look at each and every interface and test it. More than that we should not only take each interface into account but also avoid testing the functionality in too many different places.

For example a store-administration system which allows you to add items to the store, see the current inventory, and remove items. One straightforward manual test case for adding an item would be to go to the 'Add' dialogue, enter a new item with quantity 1, and then go to the 'Display' dialogue to check that it is there. To automate this test case you would instrument exactly all the steps through the user-interface.

One way to avoid them in the first place would have been to figure out how this system looks inside.

•     Is there a database? If so, the verification should probably not be performed against the UI but against the database.
•     Do we need to interface with a supplier? If so, how should this interaction look?
•     Is the same functionality available via an API? If so, it should be tested through the API, and the UI should just be checked to interact with the API correctly.

This will probably yield a higher number of tests, some of them being much 'smaller' in their resource requirements and executing far faster than the full end-to-end tests. Applying these simple questions will allow us to:

•     write many more tests through the API, e.g., to cover many boundary conditions,
•     execute multiple threads of tests on the same machine, giving us a chance to spot race-conditions,
•     start earlier with testing the system, as we can test each interface when it becomes 'quasi-stable',
•     makes maintenance of tests and debugging easier, as the tests break closer to the source of the problem,

    require fewer machine resources, and still execute in reasonable time.

The user interface is just another interface, and so it deserves attention too. However I do think that we are currently focusing most of our testing-efforts on the UI. The common attitude, that the UI deserves most attention because it is what the user sees, is flawed. Even a perfect UI will not satisfy a user if the underlying functionality is corrupt.

Neither should we abandon our end-to-end tests. They are valuable and no system can be considered tested without them. Again, the question we need to ask ourselves is the ratio between full end-to-end tests and smaller integration tests.

Unfortunately, there is no free lunch. In order to change the style of test-automation we will also need to change our approach to testing. Successful test-automation needs to:

•     start early in the development cycle,
•     take the internal structure of the system into account,
•     have a feedback loop to developers to influence the system-design.

Some of these points require quite a change in the way we approach testing. They are only achievable if we work as a single team with our developers. It is crucial that there is an absolute free flow of information between the different roles in this team. In previous projects we were able to achieve this by removing any spatial separation between the test engineers and the development engineers. Sitting on the next desk is probably the best way to promote information exchange,
    using the same tools and methods as the developers,
    getting involved into daily stand-ups and design-discussions.

This helps not only in getting involved really early (there are projects where test development starts at the same time as development), but it is also a great way to give continuous feedback. Some of the items in the list call for very development-oriented test engineers, as it is easier for them to be recognized as a peer by the development teams.

To summarize,  a successful automation project needs:

•     to take the internal details and exposed interface of the system under test into account,
•     to have many fast tests for each interface (including the UI),
•     to verify the functionality at the lowest possible level,
•     to have a set of end-to-end tests,
•     to start at the same time as development,
•     to overcome traditional boundaries between development and testing (spatial, organizational and process boundaries), and  to use the same tools as the development team. 
• Start with most important functonality and write the code such that it can be re-used wherever required.
•  Make sure that if script is not handling individual testcases or the dependency on other testcases then the execution sequence is very important. for eg. If we execute testcase which Delete user before executing create user then it will fail. If time is constraint then instead of writing individual testcase we can use the testcases to be executed in sequence. And if possible remove the dependancy of one script on other. for eg: we can put a validation where we check if the user doesn't exist then first create a user by simply calling create user testcase and then proceed with the script to delete the user. This will be also important when we want to execute only selective testcases for smoke or sanity testing.