A Cloud-Native Management Approach for Industry-Oriented Big Data Analytics

• 1、Computer School, Beijing University of Posts and Telecommunications, Beijing 100876

Abstract：With the development of Internet technology, digital transformation has become a necessary path for the future development of various industries, however, problems such as long data governance cycle, many work boards involved, and easy conflict of environment between marking procedures hinder the rapid generation of digital assets and are not conducive to the realization of data value on the ground. After researching the industry big data analysis business and the common data processing and analysis process technology in each scenario, a cloud-native management method for industry big data analysis is proposed, which enables pipeline processing on the data cloud by containerizing each work segment involved on the data processing process to the cloud, advancing the process of data processing so that the focus can be shifted from data processing to data usage; through the model and The automated generation and deployment of the marking program isolates the environment between different marking programs and reduces the difficulty of deployment and operation and maintenance; finally, by dividing the ClickHouse service nodes and introducing shared storage, ClickHouse elastic deployment is achieved, which can guarantee the service response time while sharing the concurrency pressure. Finally, the effectiveness of the proposed method is verified by designing comparative experiments.

Keywords： data management cloud native ClickHouse industry big data