Performance & Scalability Testing: Fantastic Broadband Platform V3.0
The Client
Performance & Scalability Testing
The Business Challenge
The Fantastic Corporation’s broadband multimedia distribution solution enables the controlled delivery of rich, compelling, multimedia content (any combination of audio, video, HTML, XML, Java Script, Flash, text, data) to businesses and consumers over broadband networks such as satellite, cable television, xDSL and “wireless” networks. Fantastic needed to validate the performance, throughput and scalability of their Broadband Platform using an independent IT Service provider prior to launch.
The Challenge
Although the target audience for this product had been identified as corporations that need to deliver large amounts of content via intranet and satellite, Fantastic could not focus it further without conducting field tests outside of their R&D facility to determine actual performance, throughput and scalability. Fantastic’s highly skilled R&D staff could complete some testing in house; however, they did not have experience in high-end scalability testing or the resources to conduct the in-depth performance testing they required. Additionally, Fantastic wanted real-world data on their product performance, independent of their R&D facility in Manno, Switzerland.
The Solution
Due to our extensive experience in testing complex,
Enterprise-level applications in the wireless and Telecommunications
domains, Fantastic selected Wability as their QA Partner for their
Broadband Platform testing project. As part of our hands-on partnering
approach, the Wability test team spent two weeks training with Fantastic
developers in Switzerland to master the technical and functional
infrastructure of the Fantastic Broadband system. This allowed us to
support the Fantastic Engineers when setting-up a similar environment in
the Ballina facility, and greatly reduced the cost of supporting the
Wability team during the test cycle.
The training period also provided the opportunity for the Wability and
Fantastic teams to develop strong working relationships, ensuring smooth
communication throughout the project. We determined that using a
performance-testing tool would save time and money during testing.
During the first phase of the project, Wability and Fantastic jointly
performed a proof of concept test using selected tools, and decided that
the Rational Test Tool suite was the most appropriate and cost-effective
to create the virtual users for this broadband test. Having successfully
emulated the broadband infrastructure in the Wability European
Software Test Center in Ballina, a trial test was executed to verify
that the environment was operating correctly and that the tools selected
fully met Fantastic’s expectations. Wability then tested the solution
under specific stress/load configurations on a variety of operating
systems and hardware platforms. We identified bottlenecks and
under-performing areas.
The cross-platform environment for this broadband test included the Unix
HP-UX 11.0 operating system as well as Microsoft Windows 2000, NT, ME,
and 9X. A variety of Netscape and IE browsers were running on the
client-side.
Response Time Testing
Wability executed tests for six individual activities to verify the response times between CEC (Channel Editorial Center) and CMC (Channel Management Center) for 60, 100 and 200 concurrent connections.
Throughput Testing
To test this feature, a set of ‘schedules’ was defined so that the CMC would broadcast 20mbps during a 2-hour period. The goal was to verify that the CMC was able to broadcast all the transmissions without generating any delay. For this test, Wability used Rational Robot to design and develop test scripts that would generate a 2-hour transmission consisting of several schedules.
Conclusion
Wability’s project team’s technical ability was the foremost reason for the success of this project. We were able to recreate the test environment and execute the tests with limited support from Fantastic during the test phase. The initial investment by Wability in undergoing training at Fantastic produced an overall savings of time and money, as it provided an excellent understanding of the product features and the overall project requirements. Communication between Fantastic and Wability was excellent.
The relationship between the Wability and Fantastic teams was initiated during the pre-test training in Switzerland and gave both parties an ideal opportunity to develop good peer-to peer contact from a very early stage in the project lifecycle. This assisted greatly during the actual test phase. Wability’s expertise with various relevant automated testing tools made it easy to select the most appropriate tool for this testing project, thereby further saving time. Finally, Wability’s extensive library of software and cross-platform expertise make the outsource model a very attractive alternative for companies that require comprehensive test results but that do not have the infrastructure, the tools, and/or the expertise to execute the tests in-house. This hands-on expertise was effectively channeled to help Fantastic deliver independent, on time, quality results for their Broadband Multimedia Solution.
Results
During the last 6 months Fantastic has had the pleasure of working with the Wability team in Ireland. The results presented by the team generated a lot of interesting internal discussions and provided ideas to perform further tests to investigate some of the more complex results presented.
Wability Services
Wability is the premium provider of testing services that enable technology companies to release proven enterprise-scale applications on a worldwide basis. To learn more about Wability, email info@Wability.com, or visit www.Wability.com www.Wability.com.