随着科技的发展,传统钢琴教学面临着评分不够客观、反馈不及时等问题.结合当今深度学习的发展现状设计一款高移植性的设备,通过树莓派集成乐谱转录和音乐表现评分功能,为教师提供客观准确的评分结果.采用端到端方法,提出一种基于无分割整页乐谱图像识别的光学乐谱识别(Optical Music Recognition,OMR)转录系统.通过加入R_Staff_Net模块、AM-FFM模块和融入SE注意力模块对特征提取网络的改进,使其转录精度可达84.29％;将乐谱转录模型部署到树莓派中以增加系统的可移植性,将树莓派录音的用户音频和乐谱转录生成的MI-DI原曲音频进行对比,同时设计虚拟钢琴进行实时演奏以判断学生演奏的问题,根据对比后出错的数量实现自动评分功能;系统将显示演奏中的错误并通过短信通知学生.为教师辅导学生弹奏钢琴提供了新的教学方法.
Intelligent score conversion and music scoring system of Raspberry PI
With the development of science and technology,the traditional piano teaching is faced with some problems,such as not objective grading and not timely feedback.Based on the current development status of deep learning,a highly portable device is designed to provide objective and accurate scoring results for teachers through the Raspberry PI integrated score transcription and music performance scoring functions.Firstly,an Optical Music Recognition(OMR)transcription system based on unsegmented full-page music image recognition is proposed using an end-to-end approach.By adding R_Staff_Net module,AMFFM module,and integrating SE attention module and convolutional block attention module into the feature extraction network,the transcription accuracy can reach nearly 84.29％.Secondly,score transcription model was deployed in Raspberry PI to increase the portability of the system.The user audio recorded by Raspberry PI was compared with the MIDI original music audio generated by the score transcription.At the same time,a virtual piano was designed for real-time performance to judge students'performance problems,and automatic scoring function was realized according to the number of errors after comparison.Finally,the system will display the error in the performance and notify the student via SMS.It provides a new teaching method for teacher to guide the student to play the piano.
deep learningraspberry PIoptical music recognitionfeature extraction
NETL
NSTL
万方数据
