Design and Implementation of Intelligent Jobs Analysis Big Data Platform Based on Hadoop
With the development of the Internet,online recruitment is becoming more and more popular.However,when job seekers face a large number of recruitment information,the comprehensive analysis of different dimensions of information will become extremely complicated.In view of this,the paper designs a big data platform for intelligent job analysis based on Hadoop to provide users with job data integration,archiving,retrieval,browsing and analysis mining services.The platform takes the big data analysis technology as the core,uses Python programming to crawl Web page data,uses Flume to aggregate data and collect logs,uses Kafka to buffer date,uses Spark to calculate real-time data,and uses MapReduce to compute the cleaned data.Finally,the data is visualized by using large data screen.
万方数据
维普
