Marketing Insights from Multimedia Data: Text, Image, Audio, and Video - Grewal, Rajdeep; Gupta, Sachin; Hamilton, Rebecca (Journal of Marketing Research, 2021)
Today, both marketing practitioners and researchers have access to a wide variety of multimedia data, such as combinations of text from social media posts and online reviews, images and videos posted on Instagram or Snapchat, and audio data from customer service interactions. Despite the potential of multimedia data to provide rich insight for marketing research and marketing practice, the field has only just begun to tackle its formidable theoretical and methodological challenges and benefit from the substantive insights it allows us to uncover. This special issue intends to bring together cutting-edge research using one or more types of multimedia data to address both methodological and substantive topics related to the broad discipline of marketing. In the marketing literature, the term “multimedia” has been used to describe both marketing messages from multiple media sources, such as television, radio, and newspaper (e.g., Danaher et al. 2020; Naik and Raman 2003), and information processed using multiple modalities, such as auditory and visual processing (Tavassoli 1998; Tavassoli and Lee 2003). Although in some cases media are equivalent to modalities (e.g., radio relies only on auditory processing), in other cases the same medium uses multiple modalities (e.g., television advertisements rely on both auditory and visual processing). For analyzing multimedia data, both medium and modality are critical distinctions. If images and audio come from a single media source, such as a YouTube video, it is important to acknowledge their connection in the analysis. The visual and auditory information in the video derive their meaning, in part, by the context each modality creates for the other, making their shared source an important part of the analysis. At the same time, when analyzing multimodal data, it is important to recognize that visual and auditory data may have distinct properties, such as signal-to-noise ratio, and that data from each modality may not contribute equally to the perceived meaning of the whole (Mehta 2018). Further, because visual processing and auditory processing are distinct, the use of multiple modalities rather than a single modality can enhance learning (Mayer 2001; Tavassoli 1998), though both synergies and interference may be observed (Tavassoli and Lee 2003). In this introduction to the special issue, we first explore, from a theoretical perspective, the implications of multiple media and multiple modalities for marketing messages and data. In Table 1, we highlight the articles in this special issue and selected other works in the literature that aggregate data across multiple media, those that leverage multimodal data, and those that leverage multimodal data from multiple media. Next, we highlight opportunities offered by multimedia data for the field of marketing, including expanding the field’s methodological capabilities and addressing important substantive challenges.