Composing multimodal texts in the academic context

Feature article by Xiao Tan, Arizona State University

Several weeks ago, I created a teaching portfolio for the purpose of applying for jobs. I started by designing a personal website on a university-sponsored platform, blindly assuming that this was the preferred format for such portfolios. I spent many hours editing and uploading course materials, curating information, embedding images, and selecting color schemes, until I finally arrived at a satisfying, although far from perfect, site. Shortly before the application deadline, I went through the materials once again and suddenly realized that the teaching portfolio was supposed to be submitted as a single PDF file. How could my digital portfolio—presented as different webpages—meet the submission requirement? In haste, I had to go back and recreate my portfolio as a 70-page document with Times New Roman font, a table of contents up front, and page numbers in the upper right corner.

 

What I got out of this experience is that although the two portfolios essentially fulfill the same rhetorical purpose of demonstrating my teaching excellence, creating them involved different processes and skills. For example, when building the website, I tried to contextualize the course materials by writing blocks of short explanations on the side. When editing the traditional portfolio, I had to think more carefully about the sequence in which the teaching documents appear.  

 

My experience of creating different types of portfolios is also illustrative of the changing communicative landscape in academia. The increasing use of digital technologies has allowed academic communication to take advantage of multimodal resources (Paltridge, 2020). This is evident in the emergence of online journals like Kairos: A Journal of Rhetoric, Technology and Pedagogy, where the content of an article is web-based and typically presented in a way that allows for multiple entries (and exits). Unlike traditional journal articles, with their IMRaD structure, Kairos articles can be explored in different orders.       

 

Newer genres, such as visual abstracts and video methods articles, are gradually receiving attention and validation (Paltridge, 2020). In other academic activities, such as conference presentations (Morell, 2015), research pitches (Ruiz-Madrid, 2021), and social media interactions (Luzón & Albero-Posac, 2020), multimodality has become an even more indispensable component, where it is now expected for people to take advantage of the affordances of online content development, including making choices about fonts, colors, layout, images, and even the inclusion of audio and video.  

 

However, this by no means suggests that multimodality is a novel construct that did not exist prior to the digital era. In fact, critical examination of composition history has revealed that even pre-printing press texts in the 15th and 16th century had rhetorically functioning multimodal features (Davis & Mueller, 2020; Palmeri, 2012). The issues of multimodality and multimodal writing were arguably brought to the spotlight by the publications of the New London Group and its members (especially Gunther Kress) in the early 2000s (Lauer, 2014). Common genres that have been studied before are now examined again through theories of multimodality. But beyond these research efforts, academic writing and multimodal composing seem to be somehow pitted against each other (Powell, 2020). Such an assumption is even more prevalent among writing teachers, who usually associate multimodal writing with something fun, entertaining, and non-academic (Tan & Matsuda, 2020). 

 

The distinctions are made with understandable reasons. To start with, discourses favoring a free market tend to conceptualize and package modality as discrete, countable entities that feature new and emerging practices (Horner, 2020). That is, modern text types are usually marketed as containing a wider variety of modes. Some modes are allegedly more noticeable and eye-catching than the others, thus creating the impression that certain texts are “more multimodal” than the others. For example, a TV commercial that combines written words, moving images, and sound is deemed “more multimodal” than an AAALGrads newsletter, although the latter also features a nice mixture of various semiotic resources. The understanding of modality as an innate textual characteristic dichotomizes multimodal and alphabetic composition. At the same time, academic writing is often conceived rather narrowly as consisting of only the production of alphabetic texts. Although undergraduate students are indeed tasked with various disciplinary-specific multimodal assignments (Lim & Polio, 2020), it is unclear how general writing courses have prepared students for these writing scenarios.

 

The false distinctions between academic writing and multimodal composing also give rise to anxiety over the extent to which multimodal assignments should be part of the writing class.

 

Proponents of multimodal composition have advocated for decades the importance of teaching students how to make meaning with a wide range of semiotic resources (e.g., Yancey, 2004). On the other hand, critics caution that doing so might deprive students of the opportunities to practice language and writing. For students who write in a second or foreign language, the opposing voices are sharper, as these students are assumed to have greater difficulties with mastering the linguistic resources needed to communicate effectively (Qu, 2017). 

 

These are, I believe, legitimate concerns. The ongoing inquiry of the pedagogical potentials of multimodal composition requires many theoretical discussions and studies based on empirical data. One way of reconciling the dispute is to avoid talking about multimodal composition as if it is a homogenous entity that is either done or not done in the classroom.  The helpfulness of a multimodal composing project essentially depends on how the activities are designed and how learning is scaffolded through these activities. In some publications, the course/project/activity design is often brushed off in simple sentences like “students are encouraged to use as many modes as possible,” which generates the illusion that multimodality itself has the magical power of helping students develop in all sorts of ways. Of course, the curtailed description might be a result of limited publication space, but more details regarding learning outcomes, the genre in focus, and learning sequences would certainly have merits. 

 

Conversely, some critiques of multimodal composing might have committed the strawman fallacy by oversimplifying what is involved in producing multimodal texts. The “multimodal vs. alphabetic” distinction lures many into thinking that multimodal composing is completely free from the use of written language. However, as my experience creating an online portfolio shows, creating a “multimodal text” demands just as much attention to the rhetorical situation, language accuracy, and academic conventions. In writing classrooms, students are usually instructed to produce various types of writing that pave the way for their final multimodal products (see Liang, 2019). These activities create space for students to interact with language in various ways. In my dissertation project, I assigned multilingual students to complete a video project. In addition to completing the required assignments, the participants have demonstrated complicated autonomous writing behaviors, creating several documents that helped them organize ideas, integrate online sources, and prepare for the creation of the videos. Throughout the project, the participants also paid constant attention to micro-level writing, such as negotiating sentence structures and word meaning. 

 

I do not intend to argue that multimodal composition is inherently better or more valuable than traditional writing. Rather, with my story and research observations, I hope to invite more discussion on what cognitive processes and skills are involved in composing different multimodal texts in academic settings, and how such experiences can be reproduced in the writing classroom to help students become critical, conscious, and responsible writers. 

     

References

  • Davis, M., & Mueller, A. (2020). The place of writing on the multimodal page. In P. R. Powell (Ed), Writing changes: Alphabetic text and multimodal composition (pp. 103-122). Modern Language Association.

  • Horner, B. (2020). Modality as social practice in written language. In P. R. Powell (Ed), Writing changes: Alphabetic text and multimodal composition (pp. 21-40). Modern Language Association. 

  • Lauer, C. (2014). Expertise with new/multi/modal/visual/digital/media technologies desired: Tracing composition’s evolving relationship with technology through the MLA JIL. Computers and Composition, 34, 60–75. https://doi.org/10.1016/j.compcom.2014.09.006

  • Liang, M.-Y. (2019). Beyond elocution: Multimodal narrative discourse analysis of L2 storytelling. ReCALL, 31(01), 56–74. https://doi.org/10.1017/S0958344018000095

  • Lim, J., & Polio, C. (2020). Multimodal assignments in higher education: Implications for multimodal writing tasks for L2 writers. Journal of Second Language Writing, 47, 100713. https://doi.org/10.1016/j.jslw.2020.100713   

  • Luzón, M. J., & Albero-Posac, S. (2020). ‘Had a lovely week at #conference2018’: An analysis of interaction through conference tweets. RELC Journal, 51(1), 33–51. https://doi.org/10.1177/0033688219896862

  • Morell, T. (2015). International conference paper presentations: A multimodal analysis to determine effectiveness. English for Specific Purposes, 37, 137–150. https://doi.org/10.1016/j.esp.2014.10.002 

  • Palmeri, J. (2012). Remixing composition: A history of multimodal writing pedagogy. Southern Illinois University Press. http://ebookcentral.proquest.com/lib/asulib-ebooks/detail.action?docID=1979892

  • Paltridge, B. (2020). Writing for Academic Journals in the Digital Era. RELC Journal, 51(1), 147–157. https://doi.org/10.1177/0033688219890359

  • Powell, P. R. (2020). Introduction: Writing changes: Beyond the binary of writing versus multimodality. In P. R. Powell (Ed), Writing changes: Alphabetic text and multimodal composition (pp. 1-18). Modern Language Association. 

  • Qu, W. (2017). For L2 writers, it is always the problem of the language. Journal of Second Language Writing, 38, 92–93. https://doi.org/10.1016/j.jslw.2017.10.007

  • Ruiz-Madrid, M. N. (2021). A multimodal discourse approach to research pitches. Journal of English for Academic Purposes, 52, 101003. https://doi.org/10.1016/j.jeap.2021.101003

  • Tan, X. & Matsuda, P. K. (2020). Teacher beliefs and pedagogical practices of integrating multimodality into first-year composition. Computers and Composition, 58. https://doi.org/10.1016/j.compcom.2020.102614.

  • Yancey, K. B. (2004). Made not only in words: Composition in a new key. College Composition and Communication, 56(2), 297–328. https://doi.org/10.2307/4140651 

 
XiaoTan.jpg

Xiao Tan is a PhD candidate in the Writing, Rhetorics, and Literacies program at Arizona State University. She also serves as the Associate Director of the Second Language Writing program at ASU. Her current research interests include multimodal writing, second language writing, and teacher education.