-->not seen as an instrument of communication

rules and patterns are not fixed but can be changed or recognized constantly

depending on how and in what context the language is used

Processing language is connected with a range of cognitive abilities and experiences

to examine the structure and organisation of people’s communication practices

considering how bodies are also used in interactions, including a focus on gaze and gesture

(Morgenstern, 2022: 2)

-->children internalize conventions in parallel to other cognitive and social skills, such as the ability to follow the others' gaze, to attract their attention, to read their intentions, the ability to make analogies, to categorize, to symbolize.

adults verbalize every possible visual, vocal, actional, tactile cue enacted by the child to co-create dialogue (Morgenstern, 2022: 5)

-->adults constantly adjust to their children according to their span of attention and their age

ChatGPT prompt: What is Prosody (linguistics)?

Prosody in linguistics refers to the rhythm, stress, and intonation of speech. It encompasses the patterns of pitch, loudness, tempo, and rhythm in spoken language, which help to convey meaning, emotion, and grammatical structure.

--> infant's selective attention can be positively affected. facilitation of children's comprehension--> symbolic gestures, especially pointing (Iverson et al., 1999)

o   a tool to expand both their lexicon and their communicative repertoire (Özçalıs¸kan & Goldin-Meadow, 2005). (in Morgenstern, 2022: 8)

-->for example, humanities produce portfolios whereas engineering and sciences create lab reports (Kessler 2022: 552)

o   Positive effects: Vandommele et al. (2017) found that learners who participated in collaborative multimodal writing assignments outperformed a control group in terms of their texts’ perceived communicative effectiveness, lexical diversity, text length, and other measures. (Kessler 2022: 552)

-->multimodal assessments can support students’ learning by situating the academic content within the lived experiences of students.

Multimodal assessments can

• provide students with diverse receptive and expressive opportunities

• position students to take on critical stances

• allow children to demonstrate diverse assets

• growth in language and literacy (Lawrence & Mathis, 2020) (Lim, Toh & Nguyen 2022:9).

o  both linguistic and non-linguistic modes in classrooms develop learners’ communicative competence (Kessler 2022: 552)

o  non-linguistic modes are of equal value, since charts, figures, graphs, and tables are an integral part of everyday communication (e.g. Lemke 1998) (Kessler 2022: 552)

--> it may actually detract from or stifle L2 development (Kessler 2022: 553)

o   Digital books

o   Videos

o   Comics

o   Digital photo stories

o   websites

o   advertisements

--> student-centered learning

o  focuses on how multimedia can facilitate learning

o combines devices (e.g., PowerPoint and lecturer's voice).

o uses words and pictures (e.g., on-screen text and animation).

o engages visual and auditory senses (e.g., slides and narration).

o Teachers role: acts as a cognitive guide to support the learner’s sense-making process. (Farias 2011: 136f.)

o  Different Processing: Words processed sequentially, images simultaneously.

o  Logogens and Imagens: Brain units for words (logogens) and images (images).

o  Learning Insights: Guides better multimedia learning design.

o  Teaching Challenge: Balance multimedia presentations to avoid cognitive overload. (Farias 2011: 138)

o   describes how textual and visual information is represented and understood.

o  understanding both text and pictures involves creating mental representations using different sign systems and principles that complement each other. (Farias 2011: 139)

• external text, internal representation of text structure, and propositional representation of its meaning. This involves symbol processing. 

o  On the right side, it shows how pictures are represented

• including external picture, internal visual perception, and mental model of the content. (Farias 2011: 139)

o 1. Descriptive Representations: Representations using symbols (e.g., texts, mathematical expressions) that have no resemblance to their referents (e.g., the word "dog").

o 2. Depictive Representations: Representations using icons (e.g., photographs, drawings) that resemble their referents (e.g., a picture of a dog). effective for conveying information that resembles object or events

o Both channels limited by how much information our working memory can handle at once (Farias 2001: 139)

o Ellie pushes away a cup of milk, indicating refusal.

o Mother interprets and verbalizes Ellie's action ("no, okay"), treating it as part of the dialogue.

o Even unintended actions by children can be integrated into communication by adults.

o Multimodal Communication: Analyzing both actions and gestures in their interactive contexts is essential for understanding how adults and children create and share meaning.

The child then makes a very unexpected gesture. Her hand goes down along her hair. She hits her head and looks at her father as she produces a short vocalization.

ELLA: eh!
The father repeats his daughter's gesture and points to her head. FATHER: baby's head.
The father then points at his own head.
FATHER: daddy's head.