Abstract:
Multimodal representation is a pervasive feature characterising contemporary communication practice. Immersed in a visual culture, contemporary students and digital natives are as a result inundated with a plethora of texts that come with peculiar semiotic designs and unique affordances. This may suggest a potential textual shift with new requisite literacies in both public and academic domain.
This study aimed to explore and describe the affordances of semiotic designs as ensembles and also to evaluate the interpretative repertoires on selected multimodal texts. The study adopted semiotics, social semiotics and a multimodal critical discourse analysis approach to establish the communicative functions of semiotic modes and their semantic significances on selected multimodal texts. Relevance theory in line with Forceville (2014) was adopted to interpret and evaluate respondents’ interpretative perspectives and the aptness of inferences given in the context of a given multimodal text.
The study adopted a context-based, qualitative action research methodology with explorative and descriptive design orientation. Purposive sampling was used to identify University of Venda level 300 Media Studies students as study’s respondents. For data collection, the study used an administered test adapted in line with Chan and Choo’s (2010) Multimodal and Multiliteracies Assessment Framework. The test evaluated respondents’ interpretations of semiotic designs and the communicative functions of semiotic modes as used on a given multimodal text. This was delimitated on respondents’ interpretations of: typographic features of linguistic design, affordances of colours as a visual semiotic element; inferential meaning given to the affordances of space and position as elements of spatial semiotic design. The study also evaluated respondents’ interpretations of literary devices and their narrative effect on a given multimodal text. .
The study established that, semiotic modes can be used to convey both implicit and explicit meaning. Semiotic modes as ensembles make up compositional, ideational and relational meaning of a text and these may not be easily available to novice readers. It is for this reason that, respondents’ inferential implicatures in this study were mainly incongruent to the central meaning as evaluated in the context of a text. Decoding texts
v
with multiple semiotic designs is a nonlinear process which requires a heightened level of critical literacies with developed cognitive and perceptual skills.
The study recommends the adoption of a multiliteracies pedagogy that accounts to all semiotic designs and modes in addition to the dominant linguistic semiotic design which characterises the monomodal print based literacy offering. The study also recommends the adoption of a social semiotic perspective in the reading of both monomodal and multimodal texts as a preparatory base to requisite critical literacies associated with multimodal epistemological pedagogy.
