Quantitative (quant.) studies can be overwhelming. When there are pages and pages of numbers in front of you, it can be comforting to have some indication as to which of those numbers are the most “important.” Some people look to statistical significance as a way to find those key numbers, but in my opinion statistical significance is not always the most appropriate (or cost effective) way to look at quant. research findings. Let me explain…
When analyzing quantitative data, there are two primary methods used to examine the data: descriptive and inferential statistics. Neither one of these types of statistical analyses are better than the other, but which one is the best fit for your project depends on the problem you’re trying to solve. Additionally, your quant. research design determines whether one or both types of analyses are even possible.
The most commonly used method for analysis in market research is descriptive statistics. This type of analysis is almost always appropriate and will provide information about a group such as the average response, the frequency of responses, standard deviation, and range of responses. Examples of this type of analysis include cross-tabulation, percentage breakdown, and frequency distributions.
For many research questions, descriptive statistics are sufficient to provide you with an answer to support decision-making. For instance, if you want to know how a group of customers feel about several packaging options, you can show your customers all of the available options and ask for their feedback. Data yielded would be analyzed using descriptive statistics because there is only one group of participants being questioned. When determining opinion on a product, descriptive statistics will inform you as to how the population feels about your product, package design, or performance. For descriptive statistics to be useful, you will want to collect data from a sample, or group of participants, that is as similar as possible to your overall population of potential customers.
Inferential statistics are a better way to generalize quant. findings to a broader population, but require that your sample be randomly selected and that you have multiple groups of research participants to test. Inferential statistics are often used to measure the difference between two or more groups of participants and determine how likely that difference between these groups is due to chance.
If we take the package design example from above, and instead show individuals only one of the package designs, we will have discrete groups of individuals instead of one large group. We can then make inferences about the differences between these groups to see which package design is preferred. Inferential statistics will allow us to determine if the differences between package preference in these groups is significant or more likely to be due to chance. In order to apply this method appropriately, the sample must be randomly assigned to the groups, be representative of the overall population, and test a specific hypothesis (e.g. Package A is more likeable than Package B). The most commonly used inferential statistical tests used in market research are independent t-tests, chi-square analyses, and ANOVAs.
If you do not have multiple groups in your survey, inferential statistics would be inappropriate. You cannot compare the means to two groups with a t-test or ANOVA if you don’t have multiple groups. These tests are specifically designed to determine if one group behaves differently from another due to a specific variable.
While descriptive statistics alone will not provide you with a significance level, a significance level is not always necessary in market research. Let’s take a variation from the example used above. If 75% of your participants in Group 1 rate package A “very effective” and only 40% of participants in Group 2 rate package B “very effective”, the difference between A and B package ratings are obvious and doesn’t require a significance level test to tell you that – provided the sample is large enough and representative of the general population in question.
One of these methods for statistical analysis is not necessarily “better” than the other. The best one for your project depends entirely upon the research question(s) you’re needing to answer and your survey design.
I recently celebrated my 6 month anniversary at Sylver Consulting . . . 6 months since I officially crossed over from the world I was professionally raised in (market research consulting) to the world I now, happily, call home (design-based innovation research). Innovation Research is not new to me. I’ve been an avid fan and occasional practitioner of its methods for more than a decade and, from the beginning, I felt a deep kinship to its particular approach to problem-solving. Indeed, that’s how I met Sylver’s founder, Brianna Sylver, years ago and how (she and partner Adriano Galvao tell me) I came to mind when the position of Innovation Research Director at Sylver opened up. But having an outside-in, peripheral appreciation of something is different than being immersed in it and being able to claim it as my own. Now I believe I can.
These last few months the “light bulbs” have been going off and I now have a richer, deeper, more visceral-level appreciation for this unique “brand” of research – at least the way that it is practiced here at Sylver Consulting. Recently a new client of Sylver’s articulated a question that I have grappled with myself for many years in reaction to the proliferation and blurring of the lines between many research disciplines, specialties and practices. Her question was (in so many words) “what the heck is innovation research and how is it different from market research”?
Here at Sylver, we candidly acknowledge that “innovation” is an oft-overused word – thus, depending on how it’s used, it can stand for everything and nothing at all. But we use it because it represents a territorial space and set of practices that both overlaps with market research and distinguishes itself from it in some important ways. Coming from a more-or-less “traditional” market research background, I needed to experience these similarities and differences first-hand in order to truly understand and appreciate the distinctions.
The next time a client asks me this question, here’s my 4-point reply:
As innovation researchers . . .
We focus on the issues of “where to play” and “what to build”. Innovation research, of the kind practiced here at Sylver, is rooted in human-centered user experience and design strategy methods and training. These disciplines evolved from the need to create effective solutions to emerging human-centered (vs. business-centered) needs using adaptive technology. Before Innovation Research came along, Market Research “owned” the new product development process. Historically the job of market research was to utilize its understanding of consumers and its expertise in querying them to assess the value proposition of a new product idea and/or to provide direction for its development. This “classic” approach to new product development research worked for a good many years. But as the need for bigger competitive breakthroughs arose and the success rate of new product introductions declined, it became apparent that a new way of identifying opportunity was needed. Enter “innovation research specialists” – researchers whose training includes a mix of business strategy, design-thinking, social theory and creative problem solving. As it happens, this unique kind of training and skill set (one which emphasizes rigor, creativity and analytic outcome) is what’s needed to tackle “fuzzy front end” problems. Not only do you get a good sense for where there is opportunity to play with innovation research, but you also get design/success criteria that help you identify how best to go after that opportunity.
We “design” research with the end (and the end user) in mind. A key criticism of traditional market research is the idea that it doesn’t go far enough beyond the “what” of the data into providing the “so what” and “now what” action steps that are needed. The market research industry is well aware of this and has gone to great lengths to provide added value to its clients, whether internal or external, via an increasingly greater emphasis on “insights” versus data. I believe the industry as a whole has made good progress in this direction. But what I’ve come to experience first-hand is that, for research to be truly actionable, it’s not enough to dig deep in to the results and “pull out” rich insights that might (or might not) be there. The research itself needs to be designed with the end in sight – meaning there needs to be a plan in place for how the data will ladder up to actionable outcomes. This requires a unique way of planning and designing the research (and particularly the analytic process) in such a way that it is guaranteed to deliver against these criteria in a clear, articulate, and unambiguous manner. This is a skill set that design-trained professionals bring to the table.
We are first and foremost researchers (not designers or marketers or product developers). This means that our attention is on the rigor that we bring to the research process as the means to inform the larger strategic questions being asked. In doing so, it’s important to be method-agnostic, allowing the objectives of the investigation to drive the approach vs. having a favorite method or tool that we advocate using for every project. Unlike market research which (in my experience) tends to be siloed into either quant or qual specializations or “design research” which is often based solely, if not primarily, in ethnographic research methods, true innovation research needs to be capable of employing a wide variety of qualitative and quantitative methods according to the needs of the project.
We are active users of collaboration, iteration and ideation practices. Like the word “innovation” itself, lots of lip service gets paid to these process-oriented activities but, in my experience, they rarely get used to full advantage in classic market research projects. Why? I think it’s a chicken and egg kind of situation. Historically these activities have been “hit or miss”, delivering an inconsistent experience and value that’s hard to see or measure in the final product or deliverable. As a result, clients tend to push back against them, providers are reluctant to insist on them, and both groups are willing to consider them expendable when time is tight – and when isn’t it? What diminishes in their absence, of course, is a level of buy-in, creativity and quality of synthesis that only well-designed group participatory activities like these can provide. Here at Sylver, these practices are designed for efficiency, productivity and are non-negotiable. They are understood to be so essential that they are built into every key step (planning, execution, analysis) of every research project.
Can traditional market research practitioners/companies play in the “innovation” space and do these things? Certainly there are many who do and some who do it well – but probably more who only think they do. As a strategic, forward-thinking qual expert, I thought I had everything it took to be a truly “innovative” researcher. But I’m glad to be immersed in a world where I’m challenged to new ways of thinking, exposed to new ideas, practices and tools for doing good research, and given the opportunity to help solve some very cool and interesting new problems.
Research at Sylver is most challenging (and yields the greatest reward) for me when it involves familiar, everyday products and services that compose our day-to-day living. (Mobile devices, air freshener, sunflower seeds, etc.)
These seemingly mundane objects and experiences contain depth and richness in their stories not always immediate with use. Exploring challenges related to these objects allows us to render the invisible visible, revealing meaning to the client and ourselves. This post explores the continually evolving public restroom, a favorite subject of mine.
Often messy, impersonal, and full of odors none of us enjoy (among other unpleasant characteristics); public restrooms perform an important role in many of our everyday routines. Each restroom is a unique system composed of hardware, interface design, layout, location, technology, space, people (!), and institutional policies. At the crux of each space (and my interest) are touchless technologies and entry/exit door(s). The latter being my nemesis.
A brief Twitter exchange between the author and Bradley Corporation
Public restrooms are shared communal spaces that usually begin the day clean and orderly then decline quickly into an unwelcoming state most of us limit our exposure to. Restrooms may be touched-up throughout the day, but rarely return to their morning lemon-scented purity. Maintaining a clean environment can be accomplished through an unspoken and informal alliance between the restroom’s custodial staff and its patrons. A simple idea, but patrons have different — and often conflicting — standards of cleanliness. Rubbish is disposed of on the floor; toilets are not flushed; seats are sprinkled on; and hands are not washed. Luckily, many of these inconveniences can be minimized through the use of touchless devices (flush, sink, soap, dry).
As sensor-based technologies become fully integrated into the everyday public restroom experience, opportunities for our hands to have contact with contaminated surfaces are reduced, enabling a more hygienic experience for patrons. Almost.
Emerging touchless technologies help improve hygiene, but fail at the door
William Winsor Elementary School (Greenville, Rhode Island) was the site of my earliest experience using touchless restroom technologies. Well, almost touchless. Outside of the first floor’s main restrooms sat a half-moon basin (possibly a Bradley sink) that was controlled by a single foot pedal running along the contour of the hardware’s base. A manual soap container (dispensing damp crumbly pink stuff) and a hand-crank paper towel dispenser were located on either side of the basin, which was shared between boys and girls. Puddles were unavoidable and excessive use of paper towels was the norm. Bathroom breaks were group activities that concluded with students clustered around the sink while multiple kids attempted to control the water flow (frequently leading to mischief and puddles).
Detail of a public notice posted within an Illinois Tollway Oasis restroom. (Photo Credit: Brianna Sylver @BriannaSylver)
Technologies present in today’s public restrooms have developed beyond Harry Bradley’s original pedal-triggered “Lavatory” washroom sink and my early 1980s K-6 facility. Whether you’re using a restroom at your place of work, a gas station along the way or at restaurant with friends, it’s not uncommon to find a suite of sensor-based touchless devices managing some or all of a facility’s material assets. Smart implementation of these tools can help conserve resources (electricity, paper, water, soap), reduce manual labor, minimize negative impact on the environment, eliminate specific types of vandalism, save money, and—often as a byproduct of the previous incentives—provide a more hygienic experience for patrons.
As emerging technologies continue to evolve restroom experiences (see Dyson’s faucet/dryer and Bradley’s Advocate system) and facilities shift from mixed touch-and-touchless systems to 100% touchless devices, a single obstacle remains in the way of the patron participating in a completely touchless experience — the exit door. Restrooms often require the patron to open a door manually to exit. This function immediately negates any actual or perceived health benefits achieved from an otherwise touchless experience.
The American Restroom Association states, “Restroom doors should be designed so that after one has washed their hands, exit is possible without touching a surface.” Door-free restrooms are commonly found in airports, malls, stadiums and similar-sized venues that need to manage large volumes of people. For smaller, more intimate spaces, door-free restrooms can be costly to install, cause unwanted noise pollution, and breach privacy.
Installing inward opening manual doors is a rational decision from a business’ point of view. Doors opening into hallway traffic have the potential to cause more accidents than doors that open into a restroom. For the patron, this can be a pain point. Touching a door handle following somebody who shared the restroom and did not wash his or her hands or not knowing if the previous occupants washed their hands can be frustrating. This is especially true at restaurants!
IKEA restroom sign “Caution: Open Door Slowly”
Solutions designed to remedy forced contact with door handles range from ad-hoc workarounds to fully engineered systems. A common practice by some people — myself included — is to exit a restroom by opening the door with paper towels (when available) and dispose the crumpled sheets in a nearby trash receptacle or the floor. My jump shot is awful; a trashcan located more than a yard away from the door usually results in my crumpled paper towel on the floor.
Innovations to handle design and/or door swing strive to improve the hygiene experience of the public restroom
SanitGrasp is a door handle resembling an oversized coat hook. Patrons use their forearm and the SanitGrasp handle together to swing the door inward. Foot Pull is a similar concept for a different appendage. Both devices address the concerns of a facility — reduce the possibility of litter and the labor required to clean it up for patrons — and those of a patron – reduced contact with a contaminated surface. But both solutions are a bit wonky.
Sign above IKEA automatic hand dryer
Some of IKEA’s restroom entrance/exit doors are bidirectional and leverage its customers’ common sense more than engineering to prevent accidents. A placard posted on the inside of a Chicago-area IKEA restroom door asks patrons to use caution and “open door slowly.” A reasonable request— assuming patrons read the card—but “caution” and “slowly” illicit subjective interpretations.
Hygiene as an Institutional Priority!
Hand sanitization unit outside of a Mariano’s restroom
Placing hand sanitizer stations outside of public restrooms seems to be an emerging trend lead by some airports, office buildings, grocery markets, restaurants, and similar establishments managing large volumes of traffic. This solution is a positive addition to public spaces because it is available for anybody to use (not just restroom patrons), neutralizes the negative effects of forced contact with dirty door handles, and requires a single unit to service multiple restrooms.
Collection of Chicago public restrooms
Public restrooms and the experiences they enable — while imperfect — are continually evolving. As you can tell from my post, I am quite enthralled with not only the history of the public restroom, but look forward to its future — and specifically how this pain point of the exit door gets resolved to sufficiently support the needs and requirements of patrons and host facility.
I know it sounds silly — and maybe even a little bit gross — but a project on public restrooms might actually be a dream project for me!
Sylver Consulting just celebrated it’s 11th anniversary, so the team decided to reflect on where we all were eleven years ago.
Brianna: September 10th, 2003, Brianna was crammed into her bedroom closet, avoiding the noise of construction on the street while she closed her first client on the phone for Sylver Consulting. An inauspicious beginning for what has been a fantastic journey for the company.
Adriano: On that day, Adriano was just outside the closet, eagerly awaiting the result of the deal. He was also beginning his PhD work in design methods at the Institute of Design at IIT.
Jean: In 2003, Jean was splitting her professional time between her marketing research consulting practice (JMD Research and Consulting) and her responsibilities as a an adjunct faculty instructor and assessment consultant to the Senior Seminar program at Columbia College Chicago. It was an exciting, busy and creative time in her life. And in retrospect, it was just about the closest she’s ever come to having the perfect work balance.
Perry: Eleven years ago, Perry was in her sophomore year of high school. Having gone from a magnet arts school with a class of 75 to a large suburban high school with over 3,500 students was a major shift, but she was very enthusiastic! She was very involved in the theatre program, as well as her work as an equestrian trainer and swim instructor.
Jen: Eleven years ago, Jen had just started her very first year of college at University of Illinois. She was very scared being away from home, had no idea what she wanted to be when she “grew up,” and was trying to find her place at a HUGE university. She was also probably going through sorority recruitment at the time, which included being judged by houses full of gir
ls every night of the week. Really fun times….She thinks 11 years later she is a much wiser, better person!
Kelly: Eleven years ago, Kelly was excited about taking her first biology and trigonometry classes for sophomore year, and finally getting used to high school.
Matthew: Eleven years ago, Matthew was living in Providence, developing the foundation of his curiosity in person-to-person interaction – primarily through computer-mediated design. People, geography, emotion, and the novel “photo phone” were all elements of his design.
There once was a small farming village nestled into the quaint countryside. Each fall, beautiful songbirds would fly over the village on their journey down south. The villagers, showing great interest in observing the birds, began setting up feeding stations all around the village in order to draw more of the beautiful songbirds to the village. However, chipmunks, craving the sweet and savory taste of the bird feed, snuck into the bird-feeding stations and stole the feed. The villagers were outraged by the number of songbirds that, instead of choosing to stop by the village, flew right over it. They started to set up traps in order to capture and poison the chipmunks and before long, the songbirds came back and the village rejoiced. The next harvest, the land was arid, lifeless, and unfruitful. The villagers wondered why the land was so arid and why none of their crops had grown. What they didn’t realize was that the chipmunks’ system of burrows aerated the soil and provided fertile land to grow crops.
This simple parable highlights an issue that can take form in many ways in our lives and in our work. The villagers did not properly identify and understand the issue and suffered the ramifications of a swift decision that satisfied a current problem. In turn, this created an entirely new and graver problem in the process. Often times, we can rely on quick fixes or what we might think is the most “practical” and “straight-forward” solution to our problem. The issue with this is that without fully understanding the problem and its causes, we lose the ability to properly conceptualize and address the “root” of the problem. When making decisions and generating solutions, we must always fully understand the problem, issue, or challenge. This way, we can always be sure that we are pursuing a sustainable and lasting solution.
Jean’s role will be developing Sylver Consulting’s client portfolio in the youth and family market. Jean’s specialization in the youth sector has enabled her to work with organizations of all kinds and across a wide range of brands, topics and categories that span nearly every facet of a young person’s life. That experience will also allow her to expand the company’s research bandwidth across other industries.
She joins Sylver Consulting from her own consulting firm, where she was responsible for strategically driven qualitative and mixed-methodology studies in a varied number of industries and companies, including Nickelodeon, J.C. Penney, McDonald’s, Wrigley, PepsiCo, Discovery, AOL Media, KFC, Kraft and more. Prior to that, she served for 5 years as Adjunct Faculty and Assessment Consultant for Columbia College Chicago, and has also held senior positions at C+R Research. Jean holds a MA in Social Sciences/Sociology from The University of Chicago and a BA in Communications/Advertising from Ohio University.
Commenting on her appointment, Jean McDonnell said, “I am delighted to be joining the Sylver Consulting team and look forward to enhancing our services to support researcher needs in the youth and family market landscape, as well as other market segments.”
Brianna Sylver, President, added, “Jean has a proven record of success in developing client partnerships and services. We look forward to leveraging for our clients the unique depth of youth experience she brings to the role.”
Contact Jean McDonnell: jean [at] sylverconsulting [dot] com
At Sylver, we’re always trying to think differently about presentations to improve the quality and accessibility of information. As a researcher, it’s easy to get excited about sharing all of your awesome results. It’s also easy to forget that presentation attendees aren’t quite as familiar with your work. This can lead to information overload, especially when you’re working with a large data set.
While cognitive psychology has much more to offer in terms of making sure an audience gets the most out of your research, I’ve included three quick examples that can improve the flow, understanding and overall quality of a presentation.
*It’s easier to build on previous knowledge than start from scratch. Try to connect new information to what’s already known. This will give the audience a starting point to build from rather than having to try to figure out exactly how this makes sense as it relates to their questions.
*Think about cognitive load. People can only work with about seven pieces of information and four connections at a time. Asking an audience to think about more than this dramatically impacts understanding of the concept you’re trying to convey. It results in information overload.
*Use schemas to frame your story. There are many different ways to describe a schema, but one of the most useful in this context is a visualization of how ideas connect. Something as simple as writing down the main ideas and using lines to symbolize how they relate can emphasize the most interesting results you have to share. This can also help to guide how a viewer codes the information into memory, making it easier to remember later and understand.
Bottom line, resist the “data dump” urge when sitting down to create your next presentation. Viewing what to share and how to share through a cognitive psychology lens will help you to avoid information overload and contribute to better understanding and retention of the information shared.
Millennials are a core focus of Sylver Consulting’s current beverage/food research. One aspect of this group is the struggle between where Millennials are now (professionally/personally) and where they aspire to be. A small observation from Walmart renders this tension visible.
Monday afternoon, on the way home from work, I stopped at the Walmart Express located on Chicago Avenue to pick up an online purchase. As I waited by register 8 for my delivery to be retrieved from the stock room, there was a millennial male checking out groceries at register 4. As the cashier finished bagging his purchase, the millennial removed a brown paper Whole Foods bag from his trendy backpack, placed his smoke gray plastic Walmart bag into it, and exited the store quickly into the crowded flow of late afternoon downtown pedestrian traffic — anonymous, but cool.
To be fair, had my box’s dimensions been a bit smaller, I would have carried it in my backpack.
Sylver Consulting’s Vice President, Adriano Braun Galvao, recently traveled to CEDIM in Monterrey, Mexico to teach the Innovation and Implementation module as part of their Certificate in Innovation and Design Thinking (CID) program. The CID is a “five-month program focused on the design implementation of innovative and sustainable solutions that meet concrete needs of world users.”
Over the course of the program, students work on a wide range of projects. Some of those projects included:
– Developing a workspace solution to foster creativity and productivity
– Creating a reward system to alleviate stress in the workplace
– Helping indigenous people share and preserve their culture
– Helping dropouts re-engage with their community and workplaces
While teaching the fifth and final module of the CID course, Adriano shared his expertise with the students in an information-rich and hands on manner, covering concepts of idea selection, value proposition development, business model generation, concept refinement, and the development of an innovation implementation plan. Adriano’s experience in helping shape implementation strategies here at Sylver and experience in leading a number of strategic, future thinking efforts for the Brazilian government made him ideally suited as the teacher for this final module.
Sylver Consulting’s President, Brianna Sylver, and Mozilla’s User Experience Research Manager, Cori Schauer, recently shared background and insights from their collaborative work together in Latin America at Northwestern’s Segal Design Institute. Their presentation, “Designing for the Next 2 Billion Users: Firefox OS Focuses on Emerging Markets,” showcased how their collaborative efforts to understand the values, aspirations and mobile behaviors of the Brazilian Middle Class consumer impacted the development of Mozilla’s new Firefox OS device.
You can view the entire presentation below or by clicking HERE