It’s right out of Business 101: Decide whether to invest in a new IT project by figuring out the return on investment (ROI). But calculating the ROI of a given project is increasingly challenging, leading some to propose eliminating ROI altogether.

“Despite combining powers with the agency coin counters, CIOs say they still struggle to communicate cost savings associated with successful IT projects and to measure the risks of future projects,” writes Jack Moore in Nextgov, reporting on the results of a survey by the Association of Government Accountants. “CIOs say it’s often difficult to communicate success stories to other parts of their organization because of the difficulty of calculating cost savings. More than half of the CIOs surveyed–63 percent–said they struggle to calculate the return-on-investment of successful IT projects or to communicate those cost savings to other parts of the agency.”

Government CIOs aren’t alone. CIOs in a variety of industries report that they find it difficult to calculate ROI as new technologies and methodologies take over.

The typical ROI formula is simple enough: ROI % = (Return – Cost of Investment) divided by the Cost of Investment x 100. But like trying to figure out how many angels can dance on the head of a pin—“Measure area of pin. Measure area of angels’ feet. Divide.”—it’s not always as easy as it seems.

What are some of the challenges in calculating ROI?

Challenge #1: New Development Methodologies

Back in the day of “waterfall” development, where an entire project was developed at once, it was easier to determine what a project actually consisted of. But in today’s “agile” development of continuing improvement, how do you know when to stop counting? How do you know when your project is actually done?

“The whole idea of measuring progress as you go and keeping on top of these investments becomes interestingly complicated if you do agile, because the whole point of agile is that we don’t know what the end result is,” Rob Klopp, CIO and deputy commissioner of systems at the Social Security Administration tells Moore.

That said, some methodologies have been developed for measuring ROI of agile projects. For example, calculate an initial “velocity” for how many points per ideal day of work your team can complete in a day, and then use that to estimate an ROI, writes Narendra Gupta,  Agile Program Manager at SS&C Technologies. However, be prepared to revisit it frequently, he cautions.

Challenge #2: Calculating Legacy Costs

“Nearly two-thirds of the [US] government’s $80 billion IT spending annually goes toward operations and maintenance of so-called legacy systems,” Moore writes. Because these costs were often tied into the legacy systems originally, it’s not easy to figure out how much they cost now nor how much you could save, or how much it would cost to replace them with other systems, he explains.

David Kubersky of SimCorp has an interesting framework for calculating this, however. Identifying and describing the root problems and key objectives against which the company should measure the existing legacy system, identify the key performance indicators (KPIs) it has, and then align those KPIs to the drivers, he writes. For instance, the main drivers might include reducing operational risk, increasing operational efficiency, and supporting growth opportunities.

For example, when Canadian Seed Growers’ Association (CSGA) invested in document management, it wanted to be able to show its before-and-after story in a tangible way. Each year, CSGA processes 16,000 to 18,000 fields (approximately 1.2 million acres) to certify seed crops of 2,000 different varieties and more than 50 different species. Inspecting all these crops during a four month window results in thousands of incoming forms and documents from over 200 inspectors across the county—an intensely paper-heavy process.

CSGA kept the 56,000 staples removed from the 150,000 documents that are now processed electronically to represent the “sweat equity”—or the amount of employee effort required to do the job—using its legacy system. This worked out to:

  • 300 man-hours spent mailing forms
  • More than three weeks’ worth of transit and processing time per application
  • Significant postage costs to mail forms to inspectors

Doug Miller holding a tin can of staples

Doug Miller, CSGA Operations Manager, revealing CSGA’s “sweat equity” ROI – a coffee can containing four pounds of staples

Challenge #3: Calculating Returns

Particularly in a day and age when customer experience is so significant, it can be difficult to calculate an actual return on intangible values such as competitive advantage, product differentiation, and customer service, writes Jason Charvat in TechRepublic.

And some contend that intangible values shouldn’t be included in an ROI calculation at all. “Whilst they are often as important as tangible benefits, they are very difficult to financially quantify,” writes the UK consulting firm Axia, in describing how to calculate ROI. Instead intangible benefits such as increased customer and user satisfaction, improved customer service, and increased usability should instead be explained within the business case, the organization writes.

Anne Hartman, IT Director at Oneida County, looks at ROI from this perspective. Her end goal is to keep legislators happy, which ultimately requires keeping constituents happy because legislators listen to constituents. This is a scenario-based ROI and not a hard numbers based system. So, instead of tying return to a hard revenue or cost saving number, the return is measured by accomplishing both quantitative and qualitative goals—such as citizen satisfaction.

Challenge #4: Increasing Complexity

Particularly in marketing programs, it can be difficult to tell which of several different efforts are yielding results, writes Tony Popowski of Grass Roots Marketing Inc. “It used to take between 3-5 marketing ‘touches’ to obtain a customer. Example: an individual visits your website (marketing touch #1), he/she receives your quarterly newsletter (marketing touch #2), and then a salesperson would make a follow-up call to close the deal (marketing touch #3),” he writes.

But today, it can take from 12-25 marketing “touches,” ranging from social media to newsletters and websites.  “It can take months, or even years, for an individual to make a purchasing decision,” Popowski writes. “Since so many marketing ‘touches’ influence your audience’s purchasing decision, it’s becoming more difficult to accurately calculate the ROI of your marketing tactics.”

It can be done, however; there are methodologies intended for determining the distinction between “first touch” and “multi touch” attribution, which lets you calculate the ROI for these activities.

Even with all of these challenges, experts encourage IT staff to continue to do their best to calculate project costs, even when it’s challenging, because it helps build a business case for IT.

In particular, with so many high-profile security breaches, it’s important to figure out how much such a breach could cost your company, writes J. Peter Bruzzese in InfoWorld. “if you want to convince management that additional security is needed from a third-party vendor, you need to be able to provide clear examples from vendor-neutral sources to demonstrate the risk and both the cost and value of reducing it,” he writes. “Even security is an ROI calculation, in the form of risk avoidance.”

And yes, this is even true for those intangible costs. “You may say, ‘Hold it. Some IT investments are nearly impossible to quantify,’” Charvat writes. “I agree, but there’s always a way. A typical example is a business intelligence (BI) project. Even intangible benefits can be quantified with some creativity, and it is important to find a way to do this.”

Building a technology business case doesn’t have to be complicated. For examples, check out the Document Management Software Justification Toolkit to discover how organizations have successfully built the business case for Document Management software.


Related Posts