Quality reports are crucial if you want to know how you are doing. Inés Smith, QA director at a major bank assesses the benefits of some reporting tools and what they can deliver in the software arena.

Reporting has always been a standard and perhaps the most salient requirement for any business. In recent years such requirements have been led by legal necessities such as in the Sarbanes–Oxley (SOx), PCI legislations or Audit where the impacts of non-compliance range from fines to jail terms, and includes the harsh reality that failure to comply will ultimately impact the organisation’s public image.

It has been statistically proven that 75 percent of organisations must comply with two or more regulations and corresponding audits and more than 40 percent must comply with three or more regulations. However, this was not the sole reason to increase the profile of the reporting; put simply, the increased growth of information technology resulted in the increase in demand for unified reports which join different views in one place. This allows the corporations, the divisions and the departments within them to take a pro-active rather than re-active approach based on the simple assumption that the earlier a problem is known the sooner it can be addressed.

Risk mitigation and management

In the early years, IT reporting stemmed not only from a desire for companies to take a proactive approach, but from the dire need to demonstrate the applied due diligence in order to avert or mitigate the impact of a potential legal action. In simple terms if a company was not checking their product or service quality they could be found guilty of negligence under the Tort law. Reporting mechanisms were created to quality check the product before its release and the industry/art of ‘Quality Assurance’ (QA) was born. The results were remarkable – reports were supplemented by the industry standards stipulating how the information must be displayed. The detail of the report was driven by the law, legal precedents and a QA standards body driven by IEEE.

The benefits of reporting

The obvious question from management is “How are we doing” and the obvious area for reporting is in IT aka Management Information Systems (MIS). These systems hold all the information which is the main ingredient for any report. The data is specifically held in databases which are designed to hold and retrieve information effectively and efficiently. It did not take long for generic reporting IT application to start appearing on the market.

One of the market leaders today is a product called Business Objects. This product connects to existing databases and then provides for reports to be configured / developed to retrieve, calculate and manipulate data into whatever report is required. As expected, every product comes with its limitations. The most common generic reporting limitation is the complexity of the data ‘as it is stored’. Some database storage schemas are just not friendly to standard requests to retrieve the data in a simple manor.

From text graphics

There was a time when only text reports were available, report images were not normal and people read books. Today things have changed. Most people prefer picture presentations with a small amount of detail. Presentations in some areas have even moved over to videos. Certainly the media web sites now have movies on all the important news events.

Studies have shown that people mostly read headings, captions above or below a picture and bullet points. Details in the text are often skimmed over or not read at all.

Reporting has also followed this trend. The requirement is for graphical presentations of information or even interactive graphical presentations to cover a point with a small amount of text. Management prefer dashboards which are colour coded to show up only on warnings and not bother them if things are running ‘business as usual’.

In 1989 a testing company saw the market need for a testing and audit tool. The product was the only one available on the market of its kind and remains so today. The test suite became the industry standard, the company was Mercury Interactive. Their test suite product range has dominated the market for many years. The company started with the goal to fill the market need for a test tool. At that stage there was no need or legal requirement for the reporting. The report’s functionality for the test tool suite left much to be desired.

New era for IT product quality

As time went on from the 1989 start for Mercury new legislation was introduced into the market which drove the need for companies to have better reports and audits. The legislation was first and foremost for the finance industry but as most companies use financial transactions the legislation also applied to them. Unfortunately QC did not respond to this need specifically, however, the product did change to allow for self make and customised reports; though these came with the need for highly technical skills to develop reports with in-depth knowledge of the QC database. In spite of this the reports were still created and constantly maintained. Resources were set aside to continue to extract and maintain the reports. Extraction or compiling of the latest up-to-date report was usually the full time job of a few people. This is an expensive overhead of niche market skill for any company.

Industry standards

Many of the large, and often rich, companies started using the tool somewhere in their IT division. In the normal operations of IT it’s new and evolving software was tested for quality. Reports were generated off that test information to meet the finance legislation and also their own management. It became the standard also for testers in that they were familiar with the Mercury test suite and no other test tools. Hiring was then easier for those who knew and would not need training. The market recognised the potential of Mercury Interactive and HP responded with a buyout in July 2006. The need for QA was clear.

As QA on IT product delivery evolved so did its reporting needs. The industry currently seems obsessed with the desire to have ‘product and business requirements’ traced through to the ‘delivered product’. Simply put: “Can the reports we have show that what we produced is what we wanted and meets our business requirements?” Sadly, the answer is no. There was nothing out there to show the trace, in a report, to the delivered product. Many hours of manual tracing was done. The tool of choice was Microsoft Excel.

A new approach

In the years that followed a new report product was created for the market. The product works exclusively with HP Quality Center. It filled the market need for companies to have expensive resources with niche market skills to develop and maintain reports, and interactive graphs, for Quality Center. The QCReporting product is a zero-maintenance overhead for all employees to see the details of the project.

Now companies have visibility of exactly what is happening on their quality front, whenever they want, without having to ask anyone and if there is another report they want they do not need to pay someone, or assign time to deliver it. Now there are no ‘altered’ reports and no way to hide the project’s dirty laundry.

A huge saving appeared on some budgets as they were able to take evasive action from a bad situation at the very first sign of it instead of being told that “everything is ok” until the last minute.

The offshoring trend

There have been trends with companies to move operations offshore. This may have started with a few call centres but has increased to every market sector over the years. Some companies have moved operations back to their country as they have perhaps not seen the savings they had accounted for.

There seems to be a division in response to offshoring. There are the companies who know exactly what transpires away from home and those who don’t. Logically, if management has very clear visibility then they are better able to manage. In essence visibility means better reporting; both accurate and up-to-date. Some companies don’t believe they will get the reports they need for proper management so have chosen an approach of better QA of the delivered product and better reports on the results of the QA outcome. Ironically the savings made offshore seem to be offset by the increase in QA onshore. The best approach is better reporting on both sides.

Meeting market demands

In the current market there are few products which can fulfil all corporate needs for reports. Some have tried and made a good common denominator approach such as business objects. However if it’s IT QA reporting and the company takes their product delivery seriously then testing products such as HP QC and QCReporting would be the best match for an overall solution. Currently these two products, working together, meet most of the market demands.

Inés Smith
QA director