Case Studies

The book contains several case studies. This page hosts copies of all of the case studies, longer versions of some of them, and additional case studies.

Title Case
Using Location Analytics and In-Store Mobile Surveys to Understand 4th July Shopping

Locately, a US-based shopper insights firm specializing in location analytics and location-targeted mobile surveys, conducted a study of US shoppers in Summer 2013, timed to coincide with the US 4th July celebrations. The study included 1152 participants who had opted to share their GPS location via Locately’s smartphone app, and generated 918 in-store mobile survey completes.

The research triggered in-store interviews when Locately’s location analytics technology detected that the shopper was in a store of interest, which for this study included mass merchandisers, grocery retailers, and warehouse club stores. Location analytics captured data on the shopper journey (passively) and gathered information about what happened inside store (via mobile surveys).

The data and insight generated by the research operates at two levels, the macro and the micro. At the macro level, the data answered questions around the awareness and impact of in-store shopper marketing activations, dollars spent to prepare for key 4th July events (such as grilling), and location-specific metrics such as stores visited, store drive-pasts, time spent in the store, and distance travelled to reach the store. At the micro level, the data allowed a single participant’s journey to be illustrated on a map and annotated with survey feedback from the stores they visited.

The study showed the power of integrating location analytics (mining passively-captured location data) and combining the behavioural data with survey responses to gain insight into the ‘Why?’

The Handbook of Mobile Market Research, Chapter 2.

UK Mappiness Project

The Mappiness project sought to find out what factors affected how happy people in the UK were. The project was based on an app which was signalled twice a day asking people to rate how happy they were. The data was combined with phone coordinates using GPS.

The Mappiness project was launched in the UK in 2010, and the initial report was based on over one million responses, gathered from almost twenty-two thousand participants.

In addition to the happiness rating, the GPS data allowed the project team to determine the weather, whether it was daytime or not, and area (divided into one of nine habitat types, such as ‘Marine and coastal margins’ and ‘Continuous urban’). The app also asked who they were with and what they were doing.

The general findings were not a surprise: being out of urban areas was associated with more happiness than being inside the urban areas, and being by the seaside was best of all.

The published paper is a useful resource in terms of methodology and approaches to analysis.

The Handbook of Mobile Market Research, Chapter 2. For more information see MacKerron, G and Mourato, S (2013) ‘Happiness is greater in natural environments’.

Evaluating Mobile Ad Testing

A number of organizations have been testing and evaluating methods of using mobile devices to test ads. The two cases below are both covered in the Research-on-Research section of the accompanying website.

In 2012 Ipsos ASI with MobileMeasure created and tested AdShout with three TV commercials that had been shown in Australia during the 2012 Olympics. The mobile test concentrated on key measures from Ipsos ASI standard testing method and produced very similar results.

In 2011 and 2012 Luma and Research Now tested print ads and TV commercials with a slightly modified version of Luma’s add+impact test. The results of the TV commercials were highly comparable. The print ad results were broadly comparable, but with one interesting and relevant difference, the mobile results produced lower scores for attention grabbing.

The Handbook of Mobile Market Research, Chapter 2.

How Many Cereal Packs?

As an illustration of the difference a picture can make, MMR CEO Mat Lintern highlighted a study they conducted in 2013 with UK housewives. In an online survey the average claimed number of cereal packets stored at home was lower than expected. So to check the accuracy of the online data, around 150 participants were asked to take a photo of the place where they kept their cereal and send this via MMS to the agency. This showed that on average that people had almost twice as many packs of cereal (9 versus 5) than they had claimed based on recall, and it also showed the proportion of own label was much higher than had been claimed (35% versus 20%).

Mat commented ‘The recall of the market leading brands in the survey was reasonably accurate, but consumers consistently failed to recall many of their lesser used cereals – which often tended to be smaller brands and own label products. The use of photos completely removed any ambiguity, providing accurate data with minimal input required by the respondents.’

The Handbook of Mobile Market Research, Chapter 6.

SMS to Blog: Capturing ‘Sweaty Moments’

This project was conducted in the UK in 2007 on behalf of a leading deodorant brand by UK agency Join the Dots, to find out more about women’s sweaty moments. Sweaty moments are those times of the day when women become aware they are sweating. The research needed to be in the moment, as recall has been shown to be a poor method of researching sweaty moments in terms of issues such as: how many times it happens, what the main causes are, how it made women feel, what they did as a consequence.

The project utilized SMS as the core triggering element for the study. During the first week of the project, the participants (20 women, aged 20 to 40 years old, with a mobile phone, and home PC access to the internet) were asked to send a text every time they experienced a sweaty moment. This text was logged on each participant’s personal blog, which had been created for the research project. Participants were encouraged to access their blog later in the day to expand the short text message, to add some context and further information. The participants edited their blog via a PC.
At the end of the week, the SMS phase was complete and a second phase, based on asking the women to review and comment on each other’s blogs, took place.

The research captured quantitatively the number of times the participants experienced sweaty moments and the times of day those moments happened. The research captured qualitatively descriptions of the triggers for the moment, how it made the women in the study feel, and the sorts of strategies they employed to deal with these moment, for example headed home for a shower, popped into an air conditioned shop, and sprayed on some deodorant.

The Handbook of Mobile Market Research, Chapter 6.

Gatorade, real-time tracking of interactions with experiential touchpoints via mobile phone

Gatorade is a global sports drink brand owned by PepsiCo, which has a strong heritage and position in Latin America. In 2011 Gatorade G-Series was launched in Mexico, creating more of a specialist sports nutrition position. PepsiCo had a substantial amount of research on the global messaging associated with this move, but the decision to spend money on sports related experiential touchpoints, such as parks and gyms, was supported more by market experience than specific research, which meant that Gatorade were keen to estimate the value of their channel investment.

One of the problems that faced Gatorade is that the sort of activities they wanted to research are not easily picked up by traditional research, for example surveys asking people to think back over the last week and recall marginal, fleeting events. Gatorade turned to a MESH Planning and their mobile phone-based Real-time Experience Tracking.
Gatorade’s study involved 400 participants, 200 each from two key target segments. The study measured interactions with Gatorade and benchmarked the results against three other brands, a sports drink, a brand of bottled water, and a sports brand.

The MESH Planning method is a mixed-mode approach, with the project starting with a survey, then utilizing SMS for a week, followed by a survey at the end of the week. The SMS phase identified interactions with branded experiential touchpoints such as material positioned in gyms. Later in the day the participants can expand on their SMS entries via an online access point. The interface has been optimized so that participants can typically enter their feedback in just a few key presses. Table X shows the sort of schema a project might use to allow participants to enter their encounter with experiential touchpoints (note, for commercial reasons this is NOT the schema that was used in the Gatorade study):


Mesh Code Sheet

A participant might text BB54 to say they saw a store promotion about Kenco, which was very positive and which made them slightly more likely to choose it. The system records the time of day of the entry.

Results
The research showed that experiencing a brand touchpoint was associated with shifts in the educational image statements Gatorade were aiming to shift. The research also showed that the two segments responded to experiential touchpoints differently. One group was more influenced by messaging and communications in places like gyms and fitness centres, the other group were more impacted by social messaging.

The research identified the interaction between TV advertising and experiential touchpoints and as a consequence Gatorade shifted some of its funding from TV advertising to experiential touchpoints. The research also provided guidance on the balance of messages between the three varieties of the G-Series: Prime, Perform, and Recover.

In summary, the research enabled Gatorade to spend more on experiential touchpoints, tailored outdoor advertising, and more on the Prime variant, and to spend less on traditional media, generic outdoor materials, and on the Perform variant. The research and product have been rolled out to other markets, including Brazil.

Key General Learning Points

  1. ‘In the moment’ research can reveal information and detail about events that occur during participants’ busy lives which would tend to be invisible to traditional, retrospective surveys.
  2. SMS is a good medium for this sort of real-time, high frequency feedback. For participants, it is typically always available and users are familiar with it.
  3. SMS can be made more useful by making it easier for participants to respond.
  4. The study benefits from combining mobile with different elements; capturing depth via one mode and ‘in the moment’ with mobile.

The Handbook of Mobile Market Research, Chapter 8.

Designing for Mobile

The agency, MMR, needed to research out of home breakfast habits in the UK, and wanted to conduct it in the moment, which meant mobile. Key items in their implementation were:

  • An online survey was used to profile the participants on key behavioural and attitudinal measures.
  • An app was used for the ‘in the moment’ research, to ensure that the data collection would be always on and respond quickly.
  • The participants kept a breakfast diary for seven days, using the app at meal time.
  • By providing extensive support for participants, 53% of those eligible downloaded the app and 83% of these completed all seven tasks.
  • Initiatives included phased incentives (making it worthwhile to do all seven tasks) and push notifications each morning to remind participants.

The survey was kept very short, asking key details only, such as where they were, who they were with, and what they ate.

MMR utilized the benefits of mobile by asking participants to take a photo of their meal, ensuring they knew exactly what people had chosen and not what they claimed, and also the ability it gives for people to complete the survey in the moment, maximizing data quality and providing the ability for people to accurately convey how they actually felt at that specific point in time.

Working with their supplier, MobileMeasure, the data entry was modified to make it intuitive and fast for mobile users, for example using simple taps instead of the drag and drop used in their online version. The bullseye data entry shown below, based on using phones in landscape mode, simply requires the participant to tap once on the screen, at which point the next statement is displayed.

Target Question

MMR’s CEO Mat Lintern is clear that when creating a mobile solution the benefits of the new mode should be maximized and makes the following recommendation: ‘When mobilising an existing system redesign it from a mobile user’s perspective and then that becomes the new standard.’

The Handbook of Mobile Market Research, Chapter 9.

Using mobile research to implement one of the UK’s largest voices of the customer programmes

Tesco is the UK’s largest retailer and one of the largest in the world. Its history is in grocery retailing but in recent years it has expanded into general retailing, and internationally. In 2012, Tesco announced it was going to invest £1 billion (about US $1.5 billion) in improving its UK shopper experience. To support this expenditure, Tesco needed a customer experience programme to monitor customer satisfaction, at store level, in ways that facilitated action.

Previously Tesco had used a mystery shopping programme, but it was felt the focus should not be on specific standards, but that it should be on customers. And the best way to get customer feedback about the shopping experience was face-to-face, at the stores themselves.

The scale of the project that UK agency Marketing Sciences was commissioned to deliver was enormous:

  • 100 interviews per store at over 950 Tesco stores, over a two-week period each month including a number in rural and hard to reach locations.
  • The store level reports were needed two weeks after fieldwork, available to store managers.

The solution was to use tablets loaded with research apps from Confirmit, meeting the following criteria:

  • Battery life that lasted a full interviewer shift.
  • Light enough for an interviewer to hold for about six hours.
  • Cost effective.
  • The screen needed to be as big and bright as possible (consistent with being as lightweight as needed and with a good battery life).
  • Not too attractive to potential thieves!
  • Able to run offline (when no internet available).
  • Capable of handling questionnaire updates and live messaging.

To get the project up and running, over 300 interviewers were trained, across different locations around the country. Once the system was live, 1900 shifts needed to be logged each month. If the weather was bad, or interviewers were ill, alternatives needed to be put in place immediately.

The quality control procedures included the following:

  • Were the interviews done on the right day, at the right time, and across the full shift?
  • Did interviewers take any strange or long breaks during a shift?
  • How did the patterns of interviewers compare with other interviewers?
  • GPS could be switched on to show where interviews were conducted, but battery drain was a concern in these cases.

The dashboards and reports provide information that is local, regional, and national and give a common language between store managers and those further up the chain of command.

By November 2013 the project had collected over 1.5 million interviews and saved 581 trees, by not using 24 tons of paper. For the client it meant getting very close to customers, at every store, with a rapid delivery of meaningful results. For Marketing Sciences it meant delivering one of the UK’s largest voice of the customer programmes.

The Handbook of Mobile Market Research, Chapter 9.

mCAPI in India – improving the efficiency of data collection

This case study looks at how Indian market research company Market Xcel Data Matrix Pvt. Ltd. and the German creator of the mQuest mCAPI system, cluetec GmbH, partnered to provide research services in India.

The scale and complexities of India are beyond those of most, possibly all, other markets. India has 137 million internet users, however, there are over 1 billion who do not have access to the internet. There are about 900 million mobile phone subscriptions in India, but the majority are feature phones. The complexity of India can be highlighted with the following statistics: 69% of the population is rural, 22 languages, nearly 8,000 towns, over half-a-million villages.

The net result of all these challenges is that online is only suitable in some situations and even CATI can be a challenge. A very large part of fieldwork in India is conducted as face-to-face interviews, using paper questionnaires. Market Xcel have estimated that as much as 90% of all interviews conducted in India are conducted via face-to-face. This means there are opportunities to significantly improve the speed, cost, and quality of fieldwork in India.

Reverse Outsourcing
Market Xcel is a forward looking full service Indian agency, with ten offices in India, the largest proprietary online panel in India, and a CATI centre. Market Xcel were keen to improve data quality and efficiency, particularly in a market where fieldwork is often outsourced multiple times, with many freelancers at the bottom of the pyramid.

Market Xcel’s solution was to move from paper to mCAPI, to use their own in-house interviewers (initially 275 of them), to implement a training module, and to utilise cluetec’s mQuest mCAPI system. This was an unusual case of an Indian firm looking to Germany for its software.

mCAPI Benefits in India
The use of mCAPI gave a focus to Market Xcel’s training programme. Data synchronisation removed the need to post questionnaires to regional centres, and to transport them to interviewers, and the problems of gathering in the completed surveys. GPS data is used to track and validate the interviewing process – along with paradata, such as date stamps. In addition, Market Xcel is able to offer its clients real-time reporting via a web interface – something that was quite impossible with traditional paper questionnaires. Photographs can be uploaded, not only as part of the research, but also as a part of the quality control process, i.e. as respondent and location validation.

Specific benefits of using mCAPI in India include: centralised translation of questionnaires, on a central server, dynamic quota control data and instructions, mCAPI supports more survey complexity than paper (for example, routing, piping, and adaptive processes), and third-party data entry is no longer required. The net effects being faster turnarounds and better quality control processes.

A Solution to Match the Market
In India the coverage of 3G is patchy, and communication costs can be expensive, and public Wi-Fi hotspots are too rare to be depended on. The system that was selected is capable of being used offline (without a connection the internet) and utilised 450 locally sourced Android tablet devices, which were easy and practical to service locally. Although, when needed, the system can be used online, especially in cities where the mobile operators have adequate coverage.

The system handles seven of the local languages, with plans for more soon. Studies can be set up so that respondents read the survey, or so that the questions are read by the interviewer – which makes the system flexible in areas where literacy levels are lower.

Plans for the Future
The success of the first two years with mCAPI has convinced Market Xcel to expand their mCAPI operation. They are looking to double the number of interviews completed with tablets in the coming year, adding more languages to those they are already using.

The Wider Lessons
The majority of the world’s population are not connected to the internet. Although a majority of the world’s population have a mobile phone, many of those phones are not necessarily ready for all the types of research that brands wish to conduct. Perhaps this explains why ESOMAR estimate that 11% of global spend is still on face-to-face interviewing (ESOMAR Global Market Research Report, 2012). For example, in Bolivia, Bangladesh, and Georgia, over 70% of research spend utilises face-to-face data collection.

mCAPI provides a real alternative to paper-based face-to-face research in less developed markets, leveraging the power of modern research approaches with the increasingly affordable mobile devices. The future may be online, or self-completion on personal mobile devices, but for the next few years there are many projects that would benefit from moving from paper questionnaires to mCAPI.

In markets like India, the majority of mCAPI projects require solutions that can work offline as well an online, given that online connectivity is unreliable and potentially expensive.

A shorter version of this case study is in The Handbook of Mobile Market Research, Chapter 12.

The power and perils of tracking device usage

Marcelo Ballve provides an example of the power and the perils of using device usage tracking devices. He compared the data from Flurry and Experian in terms of the number of hours a day spent using apps (Business Insider, June 2013) and found an interesting difference. Flurry produced an estimate of 2 hours’ app usage a day amongst US smartphone owners, but Experian produced an estimate of just under 1 hour.

Ballve explored the differences and at the same time provided some insight into the need to contextualize passive data.

Flurry were measuring apps on tablets and phones, but Experian were just measuring them on phones – so that could create some differences. However, the main point Ballve was making related to the context for the passive measurement.

Flurry provide app developers with a service. App developers can embed code from Flurry in their apps and every time the app is run it sends a signal to Flurry, who collect aggregated market information and provide the app developer with detailed and comparative information. From the app user’s point of view this is truly passive, in the sense that most app users do not know it is happening.

Experian have a panel of people who have downloaded a monitor app onto their device that records what they do; panel members have given their consent for this to happen. Experian monitor things like calls, games, internet usage and then average it across the sample to get an overall picture.

So, the Flurry figure is based on a much bigger sample size, but a sample of people who are using apps. The Experian figure is based on a more representative but smaller sample, and includes people who don’t use apps, and people who hardly use apps.

The lesson is that usage monitoring can measure something accurately, but a researcher has to make sure they know what the something is, and how it relates to their business question, before using it.