Spark is a memory based computing engine with high speed,efficiency and accuracy.MLlib is a built-in machine learning algorithm library.The library realizes parallel computing in a cluster environment,represents data in RDD form,and then calls machine learning algorithms in a dis-tributed cluster.The main core functions are feature extraction,regression classification,clustering,statistical analysis and model evaluation.This paper will cite the classic algorithm cases of spark MLlib for film data analysis,integrating the nonlinear conjugate gradient(NCG)method improves the ALS collaborative filtering recommendation algorithm,reduces the number of iterations,improves the effi-ciency of the recommendation system,and has guiding significance in the scene of real-time classifica-tion and query of massive big data.
维普
万方数据