摘要
以大数据时代为背景,以浙江国际海运职业技术学院船员培训系统为例,设计并实现了一套基于大数据的船员培训驾驶舱平台.通过引入Hadoop、Hive、Spark等大数据技术实现了船员培训数据的采集、分析和可视化展示.该平台的设计目标在于提升船员培训效果,通过大数据存储分析技术对培训项目、人次、计划等数据进行采集和分析,形成主题评估及数据对比,为管理层决策提供支撑.详细介绍了平台的总体架构、环境准备、数据导入、清洗、处理、持久化和可视化等关键步骤,强调了数据处理的全流程.
Abstract
the context of big data,taking the crew training system of Zhejiang International Maritime College as an example,this essay discussed the design and implementation of a crew training cockpit platform.By introducing big data technologies such as Hadoop,Hive,and Spark,the collection,analysis,and visual display of crew training data are realized.The design goal of the platform is to improve the effectiveness of crew training.Through big data storage and analysis technology,data such as training projects,person-times,and plans are collected and analyzed to form topic assessments and data comparisons,providing support for management decision-making.The key steps of the platform's overall architec-ture,environment preparation,data import,cleaning,processing,persistence,and visualiza-tion are introduced in detail,emphasizing the whole process of data processing.
NETL
NSTL
万方数据