Learning motivation is a core psychological mechanism mediating instructional design, learning environments, and academic performance, especially in cognitively demanding domains like Advanced Mathematics in higher education. This literature review synthesizes the theoretical foundations, empirical evidence, methodological trends, and limitations of research on learning motivation in AI - mediated collaborative learning. Based on Self - Determination Theory, Expectancy–Value Theory, and Achievement Goal Theory, it clarifies how AI - mediated interventions and collaborative structures shape motivation and influence academic outcomes and self - confidence. Methodologically, contemporary research is empirical - oriented, uses theory - aligned measurement tools, and relies on structural equation modeling for mediation testing, but has issues like over - reliance on cross - sectional data, poor sample representativeness, mismatched measurement tools for AI, and insufficient group - level analysis. There are critical gaps in theoretical integration, empirical exploration of longitudinal dynamics and AI effects, and methodological innovation. Future research should focus on developing unified theoretical models, expanding empirical studies, innovating measurement tools and techniques, and addressing ethical and practical issues. The review offers an overview of current research and identifies key directions for motivation research in AI - enhanced higher education.