EVALUATING ORAL PROFICIENCY SKILL THROUGH ANALYTICS AND HOLISTIC WAYS OF SCORING

Main Article Content

Ehsan Namaziandost
Reza Banari
Shahrzad Momtaz

Keywords

oral skill evaluation, analytic scoring, holistic scoring, teaching English as a foreign language, inclusive evaluation

Abstract

Purpose: Evaluating speaking skill is an exceedingly difficult and intricate subject. Two methods of testing oral proficiency are usually used: holistic and analytic scoring. To this end, this study aimed to compare analytic and holistic techniques for scoring in evaluating oral proficiency skills.


Methodology: The participants of this study were 70 second-grade university EFL students who were studying English Language Teaching at a university in Iran. The participants in this research were both male (n=40) and female (n=30) with the age range from24 to 25.


Main Findings: The results showed a statistically considerable diversity between analytic and holistic methods of evaluation considering that the p-value was estimated at 0.002 (P < 0.05).


Applications: The findings confirmed that employing these two scoring techniques in the procedure of evaluation may be considered proper as they seem to complement each other, one augments the other and lead to more inclusive evaluation.


Novelty/Originality: This study is new since it is the first time in an Iranian university context; oral proficiency skill was evaluated through holistic and analytic scoring.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...
Abstract 41 | PDF Downloads 52 XML Downloads 2 ePUB Downloads 1

References

1. Abedi, P., Keshmirshekan, M. H., &Namaziandost, E. (2019). The comparative effect of flipped classroom instruction versus traditional instruction on Iranian intermediate EFL learners' English composition writing. Journal of Applied Linguistics and Language Research, 6(4), 43-56. http://www.jallr.com/index.php/JALLR/article/view/1036
2. Al-Amri, M. (2010). Direct spoken English testing is still a real challenge to be worth bothering about. English Language Teaching, 3 (1), 113-117. https://pdfs.semanticscholar.org/c5a7/99 0d5eb6da7e2762e4c86717b 61766823510.pdf. https://doi.org/10.5539/elt.v3n1p113
3. Anderson, J. (2015). A guide to the practice of English language teaching for teachers and trainee teachers. Nairobi: East African Educational Publishers Ltd.
4. Azadi, G., Biria, R., &Nasri, M. (2018). Operationalising the Concept of Mediation in L2
Teacher Education. Journal of Language Teaching and Research, 9(1), 132-140.http://www.academypublicati on.com/ojs/index.php/jltr/article/view/jltr0901132140. https://doi.org/10.17507/jltr.0901.17
5. Baryla, E., Shelley, G., & Trainor, W. (2012). Transforming rubrics using factor analysis: Practical assessment. Practical Assessment Research & Evaluation, 17 (4). 1-17. https://pareonline.net/getvn.asp?v=17&n=4
6. Bilá, M. (2010). Perception and production of a second language and the concept of a foreign accent. In S. Pokrivčáková et al. (Eds.) Modernization of teaching foreign languages: CLIL, inclusive and intercultural education, pp. 123-143. Brno: Masaryk University.
7. Cambridge English: Understanding Results Guide. (2014). http://www.gml.cz/prof/zajickova/Cambri dge%20exams_information/Understanding%20results%20guide.pdf
8. Chen, G. (2016). Developing a Model of analytic rating scales to assess college students’ l2 Chinese oral performance. International Journal of Language Testing, 6 (2), 50-71.http://ijlte-1.ov2.com/content/ijlte_1_ov2_ com/wp-content_138/uploads/2019/07/443-2016-6-2.pdf
9. Chuang, Y. (2009). Foreign language speaking assessment: Taiwanese college English teachers‟ scoring performance in the holistic and analytic rating methods. The Asian EFL Journal, 11 (1), 150-173. https://www.asian-efl-journal.com/main-journals/foreign-language-speaking-assessment-chinese-taiwanese-college-english-teachers-scoring-performance-in-the-holistic-and-analytic-rating-methods/
10. Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge: Press Syndicate of the University of Cambridge.
11. Davies, A. (1999). Dictionary of language testing. Cambridge: Cambridge University Press.
12. Finson, K., Ormsbee, C., & Jensen, M. (2011). Differentiating science instruction and assessment for learners with special needs, k-8. Thousand Oaks: Corwin, A SAGE Company. https://doi.org/10.4135/9781483387529
13. Goh, C. & Burns, A. (2012). Teaching speaking: A holistic approach. New York: Cambridge University Press.
14. Göktürk, N. (2016). Examining the Effectiveness of Digital Video recordings on Oral Performance of EFL Learners. Teaching English with Technology, 16 (2), 71-96. https://eric.ed.gov/?id=EJ1135917
15. Gondová, D. (2014). Taking first steps in teaching English: Assessing learners. Žilina: EDIS.
16. Green, A. (2014). Exploring language assessment and testing: Language in action. New York: Routledge. https://doi.org/10.4324/9781315889627
17. Hashemifardnia, A., Namaziandost, E., &Rahimi Esfahani, F. (2018a). The Effect of Teaching Picture-books on ElementaryEFL Learners’ Vocabulary Learning.JELTL (Journal of English Language Teaching and Linguistics), 3(3), 247-258.https://jeltl.org/index.php/jeltl/article/view/151. https://doi.org/10.21462/jeltl.v3i3.151
18. Hashemifardnia, A., Namaziandost, E., &Sepehri, M. (2018). The effectiveness of giving grade, corrective feedback, and corrective feedback-plus-giving grade on grammatical accuracy. International Journal of Research Studies in Language Learning, 8 (1), 15-27.http://consortiacademia.org/10-5861-ijrsll-2019-3012/. https://doi.org/10.5861/ijrsll.2019.3012
19. Hashemifardnia, A., Namaziandost, E., Shafiee, S. (2018).The Effect of Implementing Flipped Classrooms on Iranian Junior High School Students' Reading Comprehension. Theory and Practice in Language Studies, 8(6), 665-673.http://www.academypublication.com/ojs/index.php/tpls/article/view/tpls0806665673. https://doi.org/10.17507/tpls.0806.17
20. Hosseini, E. Z., Nasri, M., &Afghari, A. (2017). Looking beyond teachers’ classroom behavior: novice and experienced EFL teachers’ practice of pedagogical knowledge to improve learners’ motivational strategies. Journal of Applied Linguistics and Language Research, 4(8), 183-200.http://www.jallr.com/index.php/JALLR/article/view/729
21. Jankowska, A. &Zielińska, U. (2015). Designing a self-assessment instrument for developing the speaking skill at the advanced level. In M. Pawlak and E. WaniekKlimczak (Eds.) Issues in teaching, learning and testing speaking in a second language. Heidelberg: Springer. https://doi.org/10.1007/978-3-642-38339-7_16
22. Jonnson, A. &Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2 (2), 130-144. https://ila.onlinelibrary.wiley.com/doi/abs/10.1002/jaal.525. https://doi.org/10.1016/j.edurev.2007.05.002
23. Keshmirshekan, M. H., Namaziandost, E., &Pournorouz, M. (2019). The Impacts of Creative Writing Activities on Iranian EFL Learners’ Progress in Writing, Writing Dispositions: Focus on Attitude to English Course. EPH - International Journal of Educational Research, 3(9), 12-22.https://ephjournal.com/index.php/er/article/view/1552/911
24. Khamkhien, A. (2010). Teaching English speaking and English-speaking tests in the Thai context: a reflection from Thai perspective. English Language Teaching, 3 (1), 184-190. http://www.ccsenet.org/ journal/index.p hp/elt /article/view/5253. https://doi.org/10.5539/elt.v3n1p184
25. Kondo-Brown, K. (2002). A FACETS analysis of rater bias in measuring Japanese second language writing performance. Language Testing, 19 (1), 3-31.https://journals.sagepub.c om/doi/abs/10.1191 /0265532202lt 218oa?journalCode=ltja. https://doi.org/10.1191/0265532202lt218oa
26. Llach, M. (2011). Lexical errors and accuracy in foreign language writing. New York: Multilingual Matters. https://doi.org/10.21832/9781847694188
27. Luoma, S. (2004). Assessing speaking. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511733017
28. Metruk, R. (2017). Pronunciation of English dental fricatives by Slovak University EFL Students. International Journal of English Linguistics, 7 (3), 11-16. http://www.ccsenet.org/journal/index.php/ijel/article/view/64987. https://doi.org/10.5539/ijel.v7n3p11
29. Mirshekaran, R., Namaziandost, E., & Nazari, M. (2018). The Effects of Topic Interest and L2 Proficiency on Writing Skill among Iranian EFL Learners. Journal of Language Teaching and Research, 9(6), 1270-1276.http://www.academypublication.com/ojs/index.php/jltr/article/view/jltr090612701276. https://doi.org/10.17507/jltr.0906.16
30. Murugaiah, P. (2016). Pecha Kucha style PowerPoint presentation: An innovative CALL approach to developing oral presentation skills of tertiary students. Teaching English with Technology, 16 (1), 88-104. https://eric.ed.gov/?id=EJ1135930
31. Namaziandost E., &Nasri, M. (2019a).A meticulous look at Long’s (1981) interaction hypothesis: does it have any effect on speaking skill?Journal of Applied Linguistics and Language Research, 6(2), 218-230.http://www.jallr.com/index.php/JALLR/article/view/1023
32. Namaziandost E., &Nasri, M. (2019b).The impact of social media on EFL learners’ speaking skill: A survey study involving EFL teachers and students.Journal of Applied Linguistics and Language Research, 6(3), 199-215.http://www.jallr.com/index.php/JALLR/article/view/1031
33. Namaziandost E., &Nasri, M. (2019c).Innovative practices in L2 writing materials in the EFL
classroom: Effect on writing enhancement and attitude to English course. Asian Journal of Research in Social Sciences and Humanities, 9(9), 1-12. http://www.academypublication.com/ojs/index.php/jltr/article/view/jltr09018089
34. Namaziandost, E., &Nasri, M., Rahimi Esfahani, F., &Keshmirshekan M. H. (2019). The impacts of spaced and massed distribution instruction on EFL learners’ vocabulary learning. Cogent Education, 6: 1661131. https://doi.org/10.1080/2331186X.2019.1661131.
35. Namaziandost, E., Abedi, P., &Nasri, M. (2019). The Role of Gender in the Accuracy and Fluency of Iranian Upper-intermediate EFL Learners’ L2 Oral Productions. Journal of Applied Linguistics and Language Research, 6(3),110-123. http://www.jallr.com/index.php/JALLR/article/view/1018
36. Namaziandost, E., Nasri, M., &Keshmirshekan, M. H. (2019). Cohesive conjunctions in applied linguistics research articles among Iranian and non-Iranian researchers: A comparative corpus-based study. Journal of English Language Studies, 4(2), 101 – 119.http://jurnal.untirta.ac.id/index.php/JELS/article/view/5431
37. Namaziandost, E., Nasri, M., & Rahimi Esfahani, F. (2019a). Pedagogical Efficacy of Experience-Based Learning (EBL) Strategies for Improving the Speaking Fluency of Upper-intermediate Male and Female Iranian EFL Students.International Journal of Research in English Education, 4(2), 29-41.http://ijreeonline.com/article-1-160-en.html. https://doi.org/10.29252/ijree.4.2.29
38. Namaziandost, E., Nasri, M., & Rahimi Esfahani, F. (2019b).Texts with Various Levels of Hardness, Reading Comprehension and Reading Motivation: I+1 Versus I-1. ELT Forum: Journal of English Language Teaching, 8(1), 60-77. https://journal.unnes.ac.id/sju/index.php/elt/article/view/30992
39. Namaziandost, E., Rahimi Esfahani, F., Nasri, M., &Mirshekaran, R. (2018). The Effect of Gallery Walk Technique on Pre-intermediate EFL Learners’ Speaking Skill. Language Teaching Research Quarterly, 8, 1–15.https://www.researchgate.net/publication/328449149_The_Effect_of_Gallery_Walk_Technique_on_Pre-intermediate_EFL_Learners%27_Speaking_Skill. https://doi.org/10.17507/tpls.0812.16
40. Nasri, M. &Biria, R. (2017). Integrating multiple and focused strategies for improving reading comprehension and l2 lexical development of Iranian intermediate EFL learners.International Journal of Applied Linguistics & English Literature, 6(1), 311-321.https://journals.aiac.org.au/index.php/IJALEL/article/view/2803.https://doi.org/10.757 5/aiac.ijalel.v.6n.1p.311
41. Nasri, M., Biria, R., & Karimi, M. (2018). Projecting Gender Identity in Argumentative Written Discourse. International Journal of Applied Linguistics & English Literature, 7(3), 201-205.https://journals.aiac.org.au/index.php/IJALEL/article/view/4289. https://doi.org/10.7575/aiac.ijalel.v.7n.3p.201
42. Nasri, M., Namaziandost, E., & Akbari, S. (2019). Impact of pictorial cues on speaking fluency and accuracy among Iranian pre-intermediate EF learners. International Journal of English Language and Literature Studies, 8(3), 99-109. https://ideas.repec.org/a/asi/ijells/2019p99-109.html. https://doi.org/10.18488/journal.23.2019.83.99.109
43. O’ Sullivan, B. (2012). Assessing speaking. In C. Coombe, P. Davidson, B. O’ Sullivan, and S. Stoynoff (Eds.) the Cambridge guide to second language assessment, pp. 234- 246. New York: Cambridge University Press.
44. Pan, M. (2016). Nonverbal delivery in speaking assessment. From an argument to a rating scale formulation and validation. Singapore: Springer. https://doi.org/10.1007/978-981-10-0170-3
45. Pokrivčáková, S. (2010). Modern teacher of English. Nitra: ASPA.
46. Razali, K. &Isra, M. (2016). Male and female teachers’ roles in assessment of speaking skill: Gender equality. International Journal of Child and Gender Studies, 2 (1), 1-10.https://jurnal.ar-raniry.ac.id/index.php/equality/article/view/1448/0
47. Ruammai, P. (2014). Constructing scoring instrument for writing assessment and fostering critical thinking. In H. Lee (Ed.) The international conference on language and communication innovative inquiries and emerging paradigms in language, media and communication, pp. 127-137.
48. Saritha, K. (2016). Rubric for English language teaching research. Research Journal of English Language and Literature, 4 (2), 725-731.https://scholar.google.com/citations?user=yiMY04wAAAAJ&hl=en
49. Sarwar, M., Alam, M., Hussain, S., Shah, A., &Jabeen, M. (2014). Assessing English speaking skills of prospective teachers at entry and graduation level in teacher education program. Language Testing in Asia, 4 (5),18-31. https://languagetestingasia.springeropen.com/articles/10.1186/2229-0443-4-5. https://doi.org/10.1186/2229-0443-4-5
50. Shakibaei, G., Shahamat, F., &Namaziandost, E. (2019). The effect of using authentic texts on Iranian EFL learners' incidental vocabulary learning: The case of English newspaper. International Journal of Linguistics, Literature and Translation (IJLLT), 2(5), 422-432. http://www.ijllt.org/the-effect-of-using-authentic-texts-on-iranian-efl-learners-incidental-vocabulary-learning-the-case-of-english-newspaper/
51. Taylor, L. &Galaczi, E. (2011). Scoring validity. In L. Taylor (Ed.), M. Milanovic & C. Weir (Series Eds.) Studies in language testing 30. examining speaking. research and practice in assessing second language speaking, pp. 171-233. Cambridge: Cambridge University Press.
52. Thornbury, S. (2005). How to teach speaking. Harlow: Pearson Education Limited.
53. Topkaraoğlu, M. &Dilman, H. (2014). Effects of studying vocabulary enhancement activities on students’ vocabulary production levels. Procedia - Social and Behavioral Sciences, 152, 931-936. https://doi.org/10.1016/j.sbspro.2014.09.345
54. Tuan, L. (2012). Teaching and assessing speaking performance through analytic scoring approach. Theory and Practice in Language Studies, 2 (4), 673-679. doi:10.4304/tpls.2.4.673-679. https://doi.org/10.4304/tpls.2.4.673-679
55. Ur, P. (2012). A course in English language teaching. Cambridge: Cambridge University Press.
56. Ziafar M., &Namaziandost, E. (2019). Linguistics, SLA and lexicon as the unit of language. International Journal of Linguistics, Literature and Translation (IJLLT), 2(5), 245-250. http://www.ijllt.org/linguistics-sla-and-lexicon-as-the-unit-of-language/