Return on investment is the Holy Grail of meetings and events, but the question has always been how to effectively track and measure it, compile the evaluation in a form that is appreciated by the C-suite, and use the information to improve next year's meeting. Today, the ongoing technological revolution is putting automated tools into planners' hands, tools that promise to give them a better grasp than ever on their meetings ROI.

"Technology is overcoming a lot of previously thorny issues," says Jack Phillips, Ph.D., head of the ROI Institute in Birmingham, AL, whose research and methods form the basis of many of the most prominent technological tools available to planners today. "First, there has been an inherent resistance to ROI measurement; meetings attract the kind of people that don't want to get into measurement. They're very busy, I understand, but they have clients asking for this."

Phillips has become a guru among those serious about meetings ROI, and his five levels of measurement inform the most advanced of them. The levels are: One—measuring reaction to a meeting, including "enjoyment;" Two—tracking the learning that attendees actually take away from the meeting; Three—detailing attendee actions that took place as a result of the meeting; Four—gauging subsequent impact of the meeting, such as greater productivity or sales, or reduced errors; and Five—arriving at an actual return on investment garnered via the previous four levels.

"There's also a sixth level, if you will, which is the impact that can't be converted to money," says Phillips. "This could include better branding, cooperation, teamwork, and a variety of intangibles. Every meeting has intangibles."

While results might be attached to specific meetings in theory, it's not easy in practice, which may be why so many planners fudge things by talking about ROO—Return on Objectives.

"That's probably a nice way of saying, 'I can't possibly put numbers on this,' " says Steve Mapes, vice president of creative services with Dayton, NJ-based Impact Unlimited, whose Eventrends technology is a player in the field. "But you could say you met 20 new customers who might buy your product, or educated 300 physicians on how to effectively administer a particular drug."

Nevertheless, planner focus remains solidly on the "I" in ROI, and there are several technologies today that nudge ever closer to that mecca of meetings measurement. Let's consider some of the automated tools out there that promise, to greater or lesser degrees, a means of getting to Jack Phillips's vaunted Level Five— the actual ROI of a meeting.

COMPREHENSIVE SERVICES

ROI tech tools can be broken up into two broad categories: the "one-stop shop" and the "niche measurers." Let's start with the technologies that aim to do it all.

MeetingMetrics

The New York-based GuideStar Research's MeetingMetrics is one of the leading technologies in the field, simply because it does it all, bringing the power of its technology, based on Phillips ROI methodology, to bear on meetings.

"Before we met them, they had been doing a great job with Level One measurement," says Phillips. "They were doing pre-meeting assessments to find out the content that needed to be covered and that required assessment, and in fact that's been their strength. We've been working with them for three years now to add Levels Two through Five, in capturing learning data, and doing follow-up surveys on the application and impact of meeting content. And they've now got the tools to convert everything to money, capturing the ROI."

MeetingMetrics' online application was launched at the 2007 Professional Convention Management Association (PCMA) meeting in Toronto. According to Ira Kerns, GuideStar and MeetingMetrics managing director, the Phillips-inspired ROI component was added just last fall.

"ROI is just part of it," Kerns says. The application, which is intensely survey-driven based on templates with thousands of questions, also measures knowledge, opinions, attitudes, enhanced abilities, attendee intentions, and behaviors, he says. The modular solution—planners can choose which tools to use and which meeting attributes to measure—establishes benchmarks before meetings against which follow-up surveys are compared. Individual conference-session surveys are paperless, received by attendee PDAs or notebook computers following conference sessions.

Follow-up surveys can go out two weeks after meetings to help determine return on event, or ROE, but Kerns notes that ROI can be a bit more elusive. "ROI is the way to translate the ingredients of the meeting into financial values, and thus the financial survey doesn't get run until two to four months after the meeting," he says. "You need time for the financial results to accumulate."

Planners can go through Web-based wizards to help them establish benchmarking and follow-up questions. Pre-meeting surveys tend to inform the content of the actual meeting, Kerns says, and can be considered market research of attendee attitudes and skills. Post-meeting surveys show how far the needle has moved.

Kerns, like many others in the field, stresses that the financial benefits of meetings often have to be extrapolated; it's not always evident. For example, a truck manufacturer might have a particular diesel engine that continually breaks down, leading to unhappy dealers and customers. While manufacturing struggles with making a better engine, sales and customer relations people can attend a training meeting to learn how to deal with an unhappy public in the meantime.

"And fixing customer complaints translates into financial value. It's not highly technical," Kerns says.

Metrics That Matter

KnowledgeAdvisors' Web-based application Metrics That Matter can handle multitrack programs for conferences over multiple days, with default session evaluations and benchmarks developed from thousands of meetings multiplied by thousands of attendees.

"Our organization uses this for training, coaching, leadership meetings, sales meetings, new hire orientations, and so forth, and all our evaluations get stored in our system," says Jeffrey A. Berk, chief operating office of KnowledgeAdvisors, based in Chicago. "That data is benchmarkable, so when clients go into the system and pull their own data, they can pull measurement criteria that matches their own meeting."

Berk stresses that Metrics That Matter isn't "just a bunch of questions," like Web-based survey tools by Survey Monkey or Zoomerang. Evaluations are based on aggregated industry benchmarks, which Berk estimates now total about half a billion. A new meeting can measure its own results against these existing benchmarks, or devise new ones with KnowledgeAdvisors.

Berk cites eight business outcomes that Metrics That Matter addresses: sales, costs, productivity, cycle time, quality, customer satisfaction, employee satisfaction, and elimination of on-the-job risk. Like some other applications, Metrics That Matter offers predictions of business results in line with the Phillips model, along with surveys that go out 60 to 90 days after the meeting, to help confirm those predictions.

"The technology, for the most part, continues to get more sophisticated, with a greater emphasis on measurement," Berk says. "I look at it as a post Sarbanes-Oxley world, where regulations have changed the way we do business. What we're also saying is that most people don't have the time and resources to do this on their own. They want tools to press buttons and get ROI figures that can be benchmarked."

MeetingMonitor

MeetingMonitor is a Netherlands-based company with an online ROI application designed for post-event surveys. It, too, is based on Phillips' ROI Institute methods, but addresses only Phillips' first two levels, satisfaction and learning. But that may be enough, according to Director Nico Meyer.

"It's all the majority of meeting organizers require most of time," he says. "By offering an easy-to-use tool that doesn't require much time to execute the surveys and create reports, the planner can focus on analyzing the results, proving the success and value of the meetings, and taking improvement measures."

Meyer notes the application has a type of text-mining capability that can analyze attendee sentiment as expressed in a survey's open-ended questions, an interesting and advanced technology.

KEYPAD SYSTEMS

Offering attendees the ability to reply directly to session leaders' polling or voting questions via keypad devices is not new; using their capabilities to pin down the ROI of meetings is.

iDNA Information Systems

Based in New York City, iDNA Information Systems offers a wireless keypad system that can provide real-time data for meetings of up to 1,500 attendees. Further, it has also affiliated itself with Phillips' ROI Institute, to add bottom-line measurement of meeting results.

The company's product uses interactive response technology for people in meeting rooms, "to measure their thinking, collaborate, participate, and leverage the power of their collective wisdom," according to iDNA Senior Strategist Mark Fite. While keypad systems have been doing something similar for quite some time in single sessions, the capabilities are growing.

Fite posits the example of physicians in a specialty discipline—say, cardiology—assembled in 25 different cities. Combined with broadcast capabilities where attendees can witness presentations, they can also ask benchmarking and case-study questions, and tap into the experience of peers.

Meanwhile an audit trail of attendance can be tracked via the keypad process, Fite notes, which has become increasingly important with the advent of more rigorous business accountability.

Beyond tracking, iDNA offers what it calls an ROI Toolkit, developed with the ROI Institute and its five levels of measurement. Via surveys delivered sometimes months after a meeting, Fite says, the iDNA system can address the first three of Phillips' ROI levels directly. And by incorporating surveys that ask for projections, he says, the product can actually get to Level Five, return on investment.

Importantly, says Fite, using the ROI Institute methodology, you can use technology to go back and attach changes that are attributable to a particular meeting.

Extreme Process

The Extreme Group, in Tampa, FL, offers Extreme Process, another keypad-oriented technology that measures audience response. Its goal, says President John Collins, is "to work with clients to make their meeting a success, and where they can have verifiable data to say, 'My message got through.'"

Extreme Process, he says, entails working with the meeting planner to understand a meeting's goals and objectives, and figuring out questions to ask attendees that will measure to what extent those goals were attained. Essentially an automated delivery system for surveys, the process can establish benchmarks with pre-meeting questionnaires, and follow up after sessions with surveys that measure learning change.

"We can instantly evaluate what people thought of the meeting," Collins says. "We can also look at sales down the road, or quality control measures." Extreme Group provides pre- and post-meeting reports, and provides post-meeting consultations. A strength of this system, says Collins, is the ability to measure the effectiveness of a meeting's content.

"That's maybe the most important part of the meeting," he says. "If the attendees don't get the message, who cares what the food was like?"

NETWORKING AND EXHIBITING

A series of new and emerging technologies are making exhibiting at trade shows more productive, and by extension improving exhibition ROI. While some are essentially lead-generation products—e.g., enhanced badge-scanning products—others show the same kind of promise as do the new conference session technologies that measure learning, retention, and impact.

Eventrends

Impact Unlimited's Eventrends is, according to Mapes, a trade show-oriented technology that works directly with sales, often with a company's customer relationship management (CRM) system. The technology, he says, is an outgrowth of the pharmaceuticals world, which attempts to identify "key opinion leaders" for their influence in promoting certain drugs.

The Eventrends technology, he says, does this within an exhibit booth, via badge-scanning technology, with the feedback coming to (not from) a rep's PDA, tablet PC, or computer.

"We have an 'attendee alert,' which would notify the rep that a visitor is a 'K,' meaning key opinion leader," Mapes says. "That enables the rep to take the conversation to a different level." The technology would work in any milieu, Mapes notes, including receptions.

"It reports data in its own logical fashion, but people have to decide on their own ROI," he says. "I know that in the pharma space, they want to know what the sales call costs in the field versus at a conference or trade show. So, what this could do is detail your total expenditure, the number of customers talked to, divided by your investment, to get to your cost per customer. In the end, it may not be ROI in its true sense, but you could assess whether your face-to-face expenditures could be more effective."

Fish Software

Radio frequency identification devices abound lately. One player is Fish Software, which also identifies attendees who come to an exhibit booth. A pre-event gift with an imbedded RFID tag, for example, can immediately identify the booth visitor and alert sales reps to his interests via the software's ongoing information acquisition system. Fish also can analyze booth traffic patterns to help identify which displays snared the most interest.

The payoff in ROI? It might take some analysis.

"People don't make buying decisions at meetings, but you can use RFID tags to measure dwell time in front of a booth," says Corbin Ball, a meetings technology consultant with Corbin Ball Associates, in Bellingham, WA. "The assumption here is that if someone spends 30 minutes in your booth rather than two minutes, he's a much better lead to start tracking."

For tips on determining ROI potential before an event and the role Wi-Fi can play in ROI visit www.mimegasite.com go to "Articles by Topic" and click on "Technology."

GOING MOBILE

One of the problems with surveys, even when delivered via e-mail, is that the attendee's sense of the meeting may have faded by the time she gets back to her computer. Thus, technologies that beam surveys to handheld devices, such as PDAs and smart phones, are gaining ground. Companies like Spotme and nTag are leaders here, but they are not alone.

Other players include VisionTree Conference Enterprise in San Diego, which collects data from audience input into the company's smart phones. The technology supports audience polling, live Q&As, surveys, and real-time graphical presentation of group data.

Poll Everywhere, a Chicago-based company, offers a simple text message voting application for live audiences. People vote by sending text messages to options displayed on-screen, with a PowerPoint or Web page display updated in real time. Advanced uses include text-to-screen for large general assemblies, texting questions to a presenter within sessions, and Web voting.

"How you manage this data is an entire other thing," notes meetings technology consultant Corbin Ball. "That's where companies like MeetingMetrics and others will help you. But you need to ask questions to begin with."

Originally published Aug. 1, 2008

For more ideas, tips, and tools for better meetings and events, get Successful Meetings' weekly e-newsletter delivered to your inbox. This page is protected by Copyright laws. Do Not Copy