基于动态联合熵的YARN资源质量分级算法
首发时间:2020-04-07
摘要:在云计算快速发展的背景下,大数据技术与云平台的结合也日趋紧密。大多云服务系统的资源都具有动态性、异构性、不确定性,这些特性会降低系统资源利用率、影响任务执行效率,导致系统的服务质量低于预期。Hadoop是主流的大数据处理框架,调度算法是Hadoop资源管理框架YARN的核心,但目前该资源调度框架和调度算法未能很好的应用在异构资源集群。本论文探究异构云环境中资源的动态性,分析当前Hadoop资源管理框架YARN及调度算法存在的不足,利用资源的动态联合熵设计资源质量分级算法,改进调度算法。实验结果表明,该算法能够有效提高系统任务执行效率,保证较好的稳定性。
关键词: 云计算 资源管理 调度策略 Hadoop YARN 联合熵
For information in English, please click here
YARN Resource Quality Classification Algorithm Based on Dynamic Joint Entropy
Abstract:In In the context of the rapid development of cloud computing, the combination of big data technology and cloud platforms is becoming closer. The resources of the big multi-cloud service system are dynamic, heterogeneous, and uncertain. These characteristics will reduce the system resource utilization rate, affect the task execution efficiency, and cause the service quality of the system to be lower than expected. Hadoop is the mainstream big data processing framework. The scheduling algorithm is the core of Hadoop\'s YARN framework resource management. However, the current resource scheduling framework and scheduling algorithm have not been applied well in heterogeneous resource clusters. This paper explores the dynamics of resources in a heterogeneous cloud environment, analyzes the shortcomings of the current Hadoop resource management framework YARN and scheduling algorithms, uses the dynamic joint entropy of resources to design a resource quality grading algorithm, and improves the scheduling algorithm. Experimental results show that the algorithm can effectively improve the system task execution efficiency and ensure better stability.
Keywords: Cloud Computing Resource Management Scheduling Strategy Hadoop YARN Joint Entropy
基金:
引用
No.****
动态公开评议
共计0人参与
勘误表
基于动态联合熵的YARN资源质量分级算法
评论
全部评论0/1000