Multimodality as a framework for the study of verbal and non-verbal resources in web-based science communication. The case of science crowdfunding videos (SCVs)

Abstract

In a multimodal approach to communication, language is just one of many communicative modes that can be used to deliver a message and the overall meaning of a semiotic product is conveyed by the orchestration of those modes into multimodal ensembles (i.e., combinations of semiotic modes). Multimodality studies are gaining recognition in ESP, particularly in the study of digital genres for science popularisation. These genres are web-based and rely on multiple semiotic resources afforded by digital technologies to disseminate scientific knowledge to lay audiences, such as speech, gestures, images, sound, and camera effects. This paper explores multimodal communication in science crowdfunding videos (SCVs), a digital genre intended to get the Internet audience involved in a research project. Drawing on a case study of 3 SCVs from Experiment.com and qualitative interviews to their author- scientists, the results show how the engagement of online audiences is achieved by combining language with a variety of non-verbal resources mediated through the scientist’s body language and through video edition, such as the embedding of images and video footage, and the choice of camera shot and angle. Moreover, the study reveals eight multimodal ensembles that enhance audience engagement through the scientist’s presence, text, and visual aids.