Davy & MacWhinney - Spanish Sentence Production

From LearnLab
Revision as of 15:22, 9 July 2010 by Cdavy1 (talk | contribs) (Results)
Jump to: navigation, search

Spanish Sentence Production

Summary Table

Abstract

The goal of this study is to determine whether and how oral repetition can improve the fluent production of Spanish sentences of various lengths and constructions. We do this by presenting students with spoken Spanish sentences and letting them practice repeating them back. In the pilot study, students heard each sentence three times and immediately repeated it back. We measured the length of the repetition (how long it took them to repeat it back) and recorded the number and types of errors they made. We found that the practice helped students fluently repeat the sentences they heard, in terms of number of errors made and in the time needed to repeat the sentence.

Current studies train students to practice speaking sentences by describing series of pictures. During training, students see pictures and hear the sentence described by those pictures and are asked to repeat the sentence back. After the initial training phase, students should be able to respond to the pictures without hearing the spoken sentence. Future work will also look at different factors that may make a difference in training, including whether it is better to train on full sentences or on individual phrases.

Background and Significance

Levelt’s speaking model (1989) says that speaking requires three different stages of processing: conceptualization, formulation, and articulation. In the conceptualization stage, the speaker generates a pre-verbal message, activating the concepts about which they wish to speak. In the formulation stage, activation spreads to the lexical level, the lemmas, which contain the lexical form and all thematic, morphological and syntactic information that goes along with it. Finally, in the formulation stage the phonetic encoding of the lemma creates an articulatory score that the speaker uses to create the motor movements involved in speaking. This multi-modular approach suggests that for second language speakers, there may be three sources of difficulty in speaking: in conceptualizing the message, in retrieving the lemmas and the related morphological and syntactic information, and in controlling the motor movements involved in actually articulating the speech.

Yoshimura and MacWhinney (2007) implemented an oral repetition task to improve speaking in Japanese learners, having them practice reading aloud Japanese sentences containing between 0 and 3 novel words. They found that reading aloud improved fluent speech in terms of the length of utterance (how long it took them to read the sentence from start to finish) and the number of errors. A pilot study for the current line of research showed that the same pattern of results occurred when students of Spanish instead repeated sentences they heard. In this study, students heard a sentence, repeated it back, then were asked to a) translate the sentence into English and b) rate their speech in terms of fluency. They repeated this four times for each sentence. Further studies in this line of research will attempt to refine this task to achieve the greatest improvements in speech.

Glossary

Practice

Fluency

Repetition

Research Questions

1. During an oral repetition task, do students increase fluency in terms of the time it takes them to repeat back the sentence?

2. Does this task help students increase fluency in terms of the amount of errors they make?

3. Are students aware of their own speech, to the extent that they can accurately rate their own performance?

4. Will students be able to transfer their increased fluency to novel sentences?

Study One

Study one tested whether or not a repetition task could increase fluent production of the sentences.

9 third and fourth semester Spanish students at CMU participated in this study. They practiced using 40 sentences containing between four and 19 words, and between 9 and 31 syllables. During the practice phase, they heard each sentence four times and immediately repeated it back each time. After each repetition, they translated the sentence into English and rated how fluently they were able to repeat the sentence on a scale of 1 to 7.

After the practice phase, they moved on to the test phase, where they heard each sentence and repeated it back one time.

A week later, they came back for a delayed post-test, where they again heard each sentence once and repeated it back.

Hypothesis

We hypothesized that, to answer research question 1, the amount of time the student took to repeat the sentence would decrease. As to question 2, we predicted that students would produce fewer errors. We also predicted that students would be able to significantly rate their accuracy. Study 1 does not address research question 4, since it doesn't involve repeating novel sentences.

Independent Variables

The study was a within-subjects design, with the repetition number as the independent variable. So, we tracked fluency across the four repetitions of each sentence. We also varied the length of the sentences the students heard. The sentences were between four and 19 words, with an average of 8.42 words, and between 9 and 31 syllables, with an average of 15.84 words.

Dependent Variables

In this study, we use three measurements of fluency: pre-speech pause (the amount of time before the student starts speaking), articulation time (the amount of time it takes the student to say the sentence from start to finish) and the number and type of errors and corrections the students make.

Results

First, we discovered a linear relationship between the trial number (1 through 4) and the duration of the utterance (F=4.318, p=0.038). We measured this by looking both at the time between when they started speaking to when they completed the repetition, and in the initial pause, the time between when the audio stimulus ended and they started speaking. The initial pause, the amount of time before the participant started to repeat the sentence after hearing it, decreased significantly as well (F=3.204, p = 0.023).

Duration.jpg

We also discovered that across attempts, the number of correctly repeated sentences increased, and the number of incomplete sentences (ones they could not successfully repeat) decreased significantly. We also found that across attempts participants had significantly fewer missing words and different wordings (where the repetition kept the same meaning as the original but with different wording). Doing a trends analysis, we also found significant linear relationships for the number of repetitions/corrections and wrong article usages. However, contrary to what we expected, we found that in both of these cases the number of repetitions/corrections and wrong articles actually increased across attempts.

We also wanted to determine the extent to which students are aware of their own speech and whether they are able to accurately rate their own performance. To determine this, we looked at whether the time taken to repeat the sentence and a number of different errors correlated with their rating of their own speech. First we looked at the duration of the utterance, and found a significant correlation, with a rating of 3 having the longest mean duration of utterance and 7 having the lowest. Ratings of 1 and 2 had shorter durations, because ratings of 1 and 2 generally indicated that they were unable to repeat the sentence, leading to shorter, incomplete sentences. Second, we looked at whether students who rated their proficiency as being higher made fewer errors in their speech. We found that a) students who failed to complete the sentence could reliably rate their performance as a 1 or 2, and b) students with fewer errors rated their performance as higher than those who made more errors. This finding held true for all types of errors except grammatical gender errors. Students did not seem sensitive to grammatical gender errors, and were not more likely to rate their performance as lower.

Explanation

Study Two

In Study Two, in addition to hearing the sentence spoken aloud, students also see pictures that depict the sentence they hear. This way, in the training phase they both see pictures and hear the sentence they repeat, but in the testing phase they can produce the sentences without hearing them ahead of time. This ensures that their speech is not relying on echoic memory, but actually requires them to retrieve lexical and morphological information as they speak.

Students receive training on two constructions: the subjunctive (ex. "Yo dudo que tu estudies"- "I doubt that you are studying") and the preterit/imperfect contrast (ex. "Ayer/De joven tu conduciste/conducías un carro y yo saqué/sacaba fotos. - "Yesterday/As a child you drove/drove a car and I took/took pictures"). Neither of these constructions exist in English- the subjunctive case is not marked and there is no distinction between the preterit and imperfect past tense. Furthermore, both of these constructions contain two phrases, which can be trained either as one whole unit, or broken up into two separate units.

Study Two will further investigate whether it is more effective to train students using the sentence as a whole unit, or through separate phrases. For example, in the subjunctive sentences, students will either be trained on the whole thing, or on two separate phrases, "Yo dudo que-" and "que tu estudies". Doing this may potentially increase learning for two reasons: first, breaking the sentence into pieces will lower working memory constraints, increasing performance on the task; and second, using pieces may decrease cognitive load, thus freeing up more resources for learning.

Hypotheses

1. Practice: Does one practice condition lead to more improvement in fluency (in terms of correct usage and lower duration/initial pause?)

2. Test: Does one practice condition help learner to produce similar sentences more fluently when they are producing the sentences on their own?

3. Robustness: Does the practice have long-term effects on learners’ oral production?

4. Generalizeability: Is improvement limited to specific practiced sentences, or can the learners generalize to novel, similar sentences?

Independent Variables

Using two constructions will, to a certain extent, allow a within-subjects design. Each participant will receive training in one condition on one sentence construction, and in the other on the other construction.

There are two conditions: the Phrase condition and the Sentence condition. In the Sentence condition, learners will practice the sentences as a whole; in the Phrase condition, the sentences are split into two phrases which are practiced separately.

While it is possible to compare the two conditions as a within-subjects design, the two sentences are very different in nature, and lead to a very different pattern of results. So, in reporting the results we will treat each sentence construction as a separate experiment.

Dependent Variables

The dependent variables in this sentence are the same as in Study One: we are measuring fluency, in terms of pre-sentence pause, articulation time, and errors. We calculated articulation time (mean duration of utterance) as the time between when the speaker started speaking to when they finished the sentence. In cases where the speaker failed to finish the sentence, we set the duration as 15 seconds, the maximum amount of time alloted for the recording. Since during the practice phase, sentences are intrinsically longer than phrases, we normalized this duration (D) by dividing the learner's D by the native speaker's duration, allowing us to look at the duration as a ratio (D-ratio). So, the learner's production is more native-like when the value is close to 1; the greater the D-ratio, the more time the learner took compared to the native speaker, and the less native-like the repetition.

In addition to looking at the duration, we also looked at the initial pause (IP), the amount of time the learner took before he or she began speaking. This may be an indication of pre-speech planning; thus, the longer the speaker waits before he or she starts speaking, the more time he or she needed to process and formulate the sentence. Thus, more native-like performance will have a shorter IP.

Finally, we looked at the number of errors, repetitions, and corrections the learners made as they repeated the sentences. We counted a repetition as the learner repeating a phoneme, word, or phrase without correcting previous speech, and a correction as a repetition that made a correction on a previous utterance. We also coded the errors according to the type of error made. However, for the purposes of analysis, we will lump all errors together. For the purposes of this analysis, we will look at uncorrected errors per sentence, which is the total number of errors minus corrections.

Results

Practice

Our first question was whether participants improved across the practice trials, and whether one condition led to more improvement or more native-like repetition. For our measurement of temporal fluency, the D-ratio, we found significant differences for repetition (F = 39.311, p<0.01), with the third repetition taking significantly less time than the 3rd, and condition (F = 258.821, p<0.02), where the phrase condition improves less than the sentence condition. We found similar patterns of results for initial pause and uncorrected as well. Figures 1 and 2 show D-Ratios across trials for both preterit/imperfect and subjunctive sentences across conditions.


Image002.jpg

Figure 1: D-Ratio for preterit/imperfect sentences across practice trials.


Image006.jpg

Figure 2: D-Ratio for subjunctive sentences across practice trials.

Note that, while there is less improvement in the phrase condition, production is more native-like in this condition (that is, the D-Ratio is closer to 1). So, while the Phrase condition leads to less improvement, it allows for more native-like improvement.

Test

Next, we want to see whether the type of training makes a difference during the test phase, when they are producing the sentences on their own. Here, we found a different pattern of results based on the type of sentence.

For preterit/imperfect sentences, people who practiced in the Sentence condition had significantly shorter durations, shorter IPs and fewer errors than the Phrase condition. This is especially true at the delayed post-test, though there are no significant differences between immediate and delayed post-test for either condition.

Image022.jpg

Figure 3: Mean number of errors per sentence for preterit/imperfect sentences at immediate and delayed post-tests.

However, for subjunctive sentences there is a different pattern of results. For these sentences, which are shorter but more complex, while the sentence condition does better than the Phrase condition during the immediate post-test, at the delayed post-test, the Phrase condition does significantly better. In fact, the Phrase condition improves significantly by the delayed post-test, while the Sentence condition gets significantly worse.

Image024.jpg

Figure 4: Mean number of errors per sentence for subjunctive sentences at immediate and delayed post-tests.

Robustness


Generalizeability

Connections to Other Studies

References

Levelt, W. J. M. (1989). Speaking. Boston: MIT Press.

Yoshimura, Y., & MacWhinney, B. (2007). The effect of oral repetition on L2 speech fluency: an experimental tool and language tutor. Paper presented at the Speech and Language Technology in Education, The Summit Inn, Farmington, PA.

Future Plans

Below are some preliminary ideas of other questions an oral repetition task to serve to answer.


2. Interleaved vs. blocked learning

a. Have students practice one sentence over and over again, in multiple contexts. (e.g., Yo espero que tu vayas al Mercado – Yo espero que ellos vayan al Mercado, etc.)

b. Have students practice one phrase over and over again (e.g., Yo espero que tu vayas al Mercado – Yo espero que nosotros aspiremos)


3. Explicit cue focusing

a. Instruct students to explicitly focus on subjunctive, or make it more salient somehow (louder? Ask them to write down the subjunctive form they heard?)


4. Implicit Feedback

a. Have students practice sentences multiple times w/o hearing it again in between speaking trials


5. Exemplar vs. Prototype training

a. Prototype: Each verb is assigned to a subject (esperar always practiced with yo, aconsejar always practiced with tu, etc.)

b. Exemplar: mixed


6. Working Memory

a. Vary sentence length independently of phrases- prêt/imperf 1 verb vs. 2

b. Can maybe look at Study 2 for this?