When you have been maintaining your Oracle BRM (Billing and Revenue Management) configuration for some time, you begin to realize that the amount of configured entities keeps growing. By this time, it is very likely that you also developed some customizations and integrated with additional solutions. The evolution of your environment leads to a complex cluster of intertwined objects. There comes the point where you, or your client, decide to change something. The new feature might seem easy to develop at first. The truth, however, is that it will very likely collide with an existing feature, or will require the update of existing configuration as well. What seemed a simple change, then affects the whole system. Such occurrences are sometimes challenging to avoid, but being prepared for them can save you a lot of trouble. Functional test scenarios and testing versatility can be a game-changer.
Table of contents
Continuous Development Flow
Here at Tridens, we have established a continuous development flow. All our BRM configurations, customizations, and other integrated components are properly versioned and tracked on a distributed version control system (Git). This practice allows our developers to check out required component versions at any time and begin their work. Everything is fine to this point, but how do we tackle the possibility of new features breaking the current build?
We have a deployment and testing flow set up, which tracks the remote repositories of our sources. Consequently, each time a developer commits and pushes some changes to version-control, this initiates the testing process. The deployment flow builds docker images for all relevant components with their latest versions on a local (or remote) environment, and automated tests are executed to ensure that the build is stable. Only when the tests are successful, can we release the new versions to production. This process helps us ensure that new features work seamlessly with existing ones. The executed tests are a part of the Oracle BRM TestToolkit, which we developed. The BRM TestToolkit has predefined steps and scenarios with background glue code. The steps are very descriptive and user-friendly, allowing our developers or any other user to write new test scenarios quickly. Find an example of test scenarios in the image below.
BRM TestToolkit for Automated Testing
What started as simple test scenarios for account creations and product purchases, represents today, the mainframe of our testing BRM TestToolkit. To clarify, we now have several different test cases and fringe-case scenarios to ensure maximum coverage of implemented features. Automated tests allow our developers to focus on the task at hand and not spend vast amounts of time for manual testing. Of course, sometimes, we find ourselves developing a new feature not yet covered by test scenarios. In such cases, test step definitions and their background code are implemented as a part of that feature – to be used in the future as well.
The TestToolkit can communicate with different components, enabling the possibility of quick adjustments for other systems as well. We also use redesigned versions of our BRM TestToolkit for our other components. One of the latest features we added was modularity, where developers can tag different test scenarios as parts of a group. Each group then has its scripts and separated logic, allowing, if desired, only a specific number of tests to be executed at a time and in parallel. Furthermore, modularity enables us to separate prepaid rating tests from postpaid rating tests or addon purchase tests from miscellaneous purchases. You can see the test flow in the image below.
Test Scenario Coverage
The BRM TestToolkit communicates with our API, which we integrated with Oracle BRM for simplification of more complex operations. The TestToolkit can also interact with the BRM RestBridge, our BRM wrapper solution, which eases communication with BRM by allowing the use of JSON or XML format via a REST API. The test step definitions support, among many others, the following operations:
- Account creation and management
- Deal purchase and other deal operations
- Balance retrieval and check
- Traffic generating – Usage events
- API calls
- CDR drop-off
- real-time traffic tools (diameter protocol)
- Rating checks
- The BRM TestToolkit compares different usage events and purchases against expected values
- each different plan or product can have its own set of rating files, ensuring versatility in complex product setups
By composing the user-friendly test steps, each developer can write the scenarios for the feature in development. In many cases, our developers choose to set up the test scenarios in advance – before developing the feature. Preparing scenarios in advance follows the Behaviour Driven Development (BDD) pattern. BDD essentially means that the scenarios dictate the development flow and should exist in advance. These scenarios describe how the system should behave, and developers must develop new features in a way to fit these scenarios. We can execute each test-scenario can be separately, resulting in a report in a specified format. Below is the example of an HTML report.
Reporting
When the system runs the test-scenarios as a part of a new development build, it usually executes them in their full potential – including all tests. From environment to environment, this can take some time, but it provides you with a broad picture of how the new feature might have affected any other existing implementations. We can assess build stability from the test report, which the BRM TestToolkit generates at the end of a test run. Test Reports contain details for each test scenario, but also provide some statistics and analytics about all tests in general. We can use this analysis to find differences between builds and to ensure that a build is working correctly. See an example of a report below.
Conclusion
Having the right testing solution can save vast amounts of time and prevent any unnecessary bugs from occurring. Therefore, if you are struggling with any difficulties mentioned in this article, or if you are interested in our Testing BRM Toolkit, feel free to contact us. We will gladly tell you more about our solution and devise an optimal plan on how to improve your deployment and testing flow.
