Qualitative evaluation questions for formative and summative assessment of multimedia projects
List of Authors
  • Abd Hadi Abd Razak , Hammuzamer Irwan Hamzah

Keyword
  • multimedia projects, formative evaluation, summative evaluation, qualitative approach

Abstract
  • This research paper focuses on utilizing the qualitative approach for both formative and summative evaluation in the context of multimedia projects. The paper aims to provide valuable insights into user experiences, preferences, and perceptions by presenting specific questions tailored to address various aspects of multimedia projects, including engagement, message delivery, target audience needs, and technical considerations. The research methodology involves conducting a comprehensive literature review to identify key dimensions and factors relevant to multimedia project evaluation. In addition, subject-matter experts are engaged in an iterative process to refine the evaluation questions, ensuring their relevance and effectiveness. The paper discusses the significance of qualitative evaluation in enhancing multimedia projects and outlines the methodology for conducting qualitative research. It also explores the benefits and limitations associated with employing a qualitative approach, addressing the implications of incorporating qualitative evaluation into multimedia project development. The paper concludes by offering practical recommendations for practitioners and researchers in effectively utilizing qualitative evaluation methods. Overall, this research contributes to advancing the knowledge and understanding of qualitative evaluation methods in enhancing the quality of multimedia projects.

Reference
  • 1. R. C. Richey and J. D. Klein, "Design and development research," in Methods of instructional design, 3rd ed., R. A. Reiser and J. V. Dempsey, Eds. New York, NY: Routledge, 2007, pp. 297-328.

    2. M. F. Shamsudin, N. Sahari, M. A. Mohamed, and S. Sulaiman, "Evaluating multimedia in engineering education: A qualitative study," Procedia-Social and Behavioral Sciences, vol. 224, pp. 44-51, 2016.

    3. S. Jamali, M. Bagheri, and M. Pormazandeh, "Evaluation of an e-learning system: A qualitative approach," Computers in Human Behavior, vol. 30, pp. 712-717, 2014.

    4. J. Scholtz, G. Marsden, and G. J. Hitch, "User-centered design: A case study on its use during the development of a multimedia application for children," Computers & Education, vol. 135, pp. 110-126, 2019.

    5. J. L. Plass and L. L. Jones, "Multimedia learning in second language acquisition," in Handbook of multimedia learning, R. E. Mayer, Ed. Cambridge, UK: Cambridge University Press, 2005, pp. 467-488.

    6. G. Veletsianos and R. Kimmons, "Assumptions and challenges of open scholarship," The International Review of Research in Open and Distributed Learning, vol. 13, no. 4, pp. 166-189, 2012.

    7. R. E. Mayer, "Multimedia learning," in Psychology of learning and motivation, vol. 51, C. L. Zedelius and J. W. Schooler, Eds. Cambridge, MA: Academic Press, 2009, pp. 85-139.

    8. M. Scriven, "Evaluation thesaurus," Sage Publications, 1991.

    9. D. L. Stufflebeam, "The CIPP model for evaluation," in Evaluation models: Viewpoints on educational and human services evaluation, 2nd ed., D. L. Stufflebeam, G. F. Madaus, and T. Kellaghan, Eds. New York, NY: Springer, 2003, pp. 279-317.

    10. J. W. Creswell, "Qualitative inquiry and research design: Choosing among five approaches," Sage Publications, 2013.

    11. J. Nielsen, "Heuristic evaluation," in Usability Inspection Methods, J. Nielsen and R. L. Mack, Eds. New York, NY: John Wiley & Sons, 1994, pp. 25-64.

    12. R. A. Krueger and M. A. Casey, "Focus groups: A practical guide for applied research," 5th ed. SAGE Publications, 2014.

    13. D. Zhang and Y. Zhu, "Research on multimedia technology and its application in education," Journal of Physics: Conference Series, vol. 1599, no. 3, p. 032074, 2020.

    14. M. Woolfson, "Research methods for digital media," in The SAGE Handbook of Digital Technology Research, S. Price and C. Jewitt, Eds. London, UK: SAGE Publications, 2019, pp. 489-504.

    15. J. A. Bargas-Avila and K. Hornbæk, "Old wine in new bottles or novel challenges: A critical analysis of empirical studies of user experience," in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, 2011, pp. 2689-2698.

    16. J. W. Creswell and C. N. Poth, "Qualitative inquiry and research design: Choosing among five approaches," 4th ed. SAGE Publications, 2017.

    17. M. Q. Patton, "Qualitative research and evaluation methods: Integrating theory and practice," 4th ed. SAGE Publications, 2015.

    18. S. N. Hesse-Biber and P. Leavy, "The practice of qualitative research," 2nd ed. SAGE Publications, 2017.

    19. M. B. Miles, A. M. Huberman, and J. Saldana, "Qualitative data analysis: A methods sourcebook," 3rd ed. SAGE Publications, 2014.

    20. G. Baxter and C. Courage, "Understanding your users: A practical guide to user requirements methods, tools, and techniques," Morgan Kaufmann, 2019.

    21. A. Cockburn, C. Gutwin, and S. Greenberg, "Designing for personal digital archiving: Understanding people's goals for their digital mementos," in Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, 2016, pp. 2735- 2746.

    22. H. I. bin Hamzah, A. bin Abdullah and R. Candrawati, "Biologically- inspired abstraction model to analyze sound signal," 2009 IEEE Student Conference on Research and Development (SCOReD), Serdang, Malaysia, 2009, pp. 180-183, doi: 10.1109/SCORED.2009.5443168.

    23. H. I. b. Hamzah and A. b. Abdullah, "A new abstraction model for biologically-inspired sound signal analyzer," 2009 IEEE Symposium on Industrial Electronics & Applications, Kuala Lumpur, Malaysia, 2009, pp. 600-605, doi: 10.1109/ISIEA.2009.5356390.

    24. H. I. Hamzah and A. Abdullah, "A proposed biologically-inspired model to analyze signal sound," 2009 IEEE International Conference on Control and Automation, Christchurch, New Zealand, 2009, pp. 1284-1289, doi: 10.1109/ICCA.2009.5410568.

    25. H. I. Hamzah and A. bin Abdullah, "A soft computing approach for the memory storage of a sound signal processor," 2010 International Symposium on Information Technology, Kuala Lumpur, Malaysia, 2010, pp. 1-5, doi: 10.1109/ITSIM.2010.5561432.

    26. T. Z. T. Muda and R. A. Salam, "Blood cell image segmentation using hybrid K-means and median-cut algorithms," 2011 IEEE International Conference on Control System, Computing and Engineering, Penang, Malaysia, 2011, pp. 237-243, doi: 10.1109/ICCSCE.2011.6190529.

    27. A. C. Omar, N. Shiratuddin, S. M. Sarif, A. A. Mutalib and S. M. Rashid, "Identification of Research Gap: T-Commerce Impulse Purchase for iTV Advertising," 2013 International Conference on Informatics and Creative Multimedia, Kuala Lumpur, Malaysia, 2013, pp. 119-122, doi: 10.1109/ICICM.2013.28.