计算机毕业设计Hadoop+Hive地震预测系统地震数据分析可视化地震爬虫大数据毕业设计 Spark 机器学习深度学习 Flink 大数据

2024 届本科毕业论文（设计）

基于Hadoop的地震预测的

分析与可视化研究

姓名：____田伟情_________

系别：____信息技术学院___

专业：数据科学与大数据技术

学号：__2011103094________

指导教师：_____王双喜________

年月日

1 绪论

2 相关技术与工具

2.1 大数据技术概述

2.2 hadoop介绍

3 地震数据分析

3.1 数据收集与处理

4 地震数据可视化

4.1 可视化技术概述

4.2 可视化设计原则

4.3 可视化实现与评估

5 系统设计与实现

5.1 登录页面

5.2

5.3 地震数据分析页面

5.4 地震数据维护页面

6 结果讨论

6.1 结果总结

6.2 结果分析

6.3 存在问题

6.4 结果展望

参考文献

致谢

基于Hadoop的地震预测的分析与可视化研究

田伟情指导教师：王双喜

(商丘师范学院信息技术学院，河南商丘 476000)

摘要：08年四川5·12汶川地震、10年青海玉树地震、13年四川雅安地震，到23年甘肃积石山地震，大大小小的地震无数次的上演，带给我们的不仅是肉体上的疼痛，还有无法治愈的心灵创伤。在睡梦中因地震而惊醒，轰隆隆的地裂声还有那些肉眼可见的消失的村庄恐怕是无数遭遇地震的人的噩梦，也是所有人共同的悲哀。即使没有亲身经历过，但新闻报道中倒坍的房屋，埋头救人的官兵，隔着屏幕也能感受到疼痛。因此，准确地预测地震的发生时间、地点和震级对于人们的生命安全和财产安全具有重要意义。然而，地震预测是一个复杂而困难的问题，受到多种因素的影响，如地质构造、地下水位变化、地壳运动等。传统的地震预测方法往往依赖于经验和专家判断，准确度和效率有限。随着大数据技术的快速发展，越来越多的地震数据被收集和存储起来。大数据分析和处理技术能够对这些海量的地震数据进行高效的分析和建模，为地震预测提供更准确的预测模型。然而，大数据在地震预测中的应用也面临着一些挑战。首先，地震数据具有高维度和复杂的特征，需要选择合适的特征参数进行分析和建模。其次，地震数据的规模庞大，传统的数据处理方法无法满足实时性要求。此外，地震数据的可视化也是一个重要问题，如何将海量的地震数据以直观的方式展示给用户，提高地震预测的可理解性和可操作性，也是一个需要解决的问题。因此，本研究基于Hadoop的大数据技术，对地震数据进行分析和可视化，提高地震预测的准确性和效率。具体而言，本研究将使用Hadoop框架对地震数据进行分析和建模，利用线性回归预测算法构建地震预测模型。同时，本研究利用echarts的可视化技术和工具，将地震数据以直观的方式展示给用户，提高地震预测的可理解性和可操作性。

关键词：地震、地震预测、Hadoop、线性回归预测算法

Analysis and visualization of earthquake prediction based on Hadoop

TIAN Weiqing Supervisor: WANG Shuangxi

（College of Information Technology, Shangqiu Normal University, Shangqiu 476000, China)

Abstract :The 5.12 Wenchuan earthquake in Sichuan in 08, the Yushu earthquake in Qinghai in 10, the Ya'an earthquake in Sichuan in 13, and the Jishishan earthquake in Gansu in 23 years, earthquakes of all sizes have been staged countless times, bringing us not only physical pain, but also incurable psychological wounds. Waking up in your sleep by the earthquake, the rumbling sound of the ground cracking, and the visible disappearance of the villages are probably the nightmare of countless people who have been hit by the earthquake, and it is also the sorrow shared by everyone. Even if you haven't experienced it yourself, you can feel the pain through the screen of the collapsed houses in the news reports and the officers and soldiers who buried their heads in saving people. Therefore, it is of great significance to accurately predict the time, place and magnitude of earthquakes for the safety of people's lives and property. However, earthquake prediction is a complex and difficult problem that is affected by a variety of factors, such as geological structure, changes in groundwater level, crustal movements, etc. Traditional earthquake prediction methods often rely on experience and expert judgment, and their accuracy and efficiency are limited. With the rapid development of big data technology, more and more seismic data is collected and stored. Big data analysis and processing technology can efficiently analyze and model these massive seismic data, and provide more accurate prediction models for earthquake prediction. However, the application of big data in earthquake prediction also faces some challenges. First of all, seismic data has high-dimensional and complex features, and it is necessary to select appropriate feature parameters for analysis and modeling. Secondly, the scale of seismic data is huge, and traditional data processing methods cannot meet the real-time requirements. In addition, the visualization of seismic data is also an important problem, and how to display massive seismic data to users in an intuitive way and improve the comprehensibility and operability of earthquake prediction is also a problem that needs to be solved. Therefore, this study analyzes and visualizes earthquake data based on Hadoop's big data technology to improve the accuracy and efficiency of earthquake prediction. Specifically, this study will use the Hadoop framework to analyze and model seismic data, and use the linear regression prediction algorithm to construct an earthquake prediction model. At the same time, this study uses the visualization technology and tools of ECHARTS to present earthquake data to users in an intuitive way, so as to improve the comprehensibility and operability of earthquake prediction.

Key words：earthquake, earthquake prediction、Hadoop、Linear regression prediction algorithm

核心算法代码分享如下：

import csv
import random
import re
import time
import datetime
import pymysql
import requests
#将数据保存到mysql方便制作表格查询
provinces = [
    '北京', '天津', '河北', '山西', '内蒙古', '辽宁', '吉林', '黑龙江', '上海', '江苏', '浙江', '安徽',
    '福建', '江西', '山东', '河南', '湖北', '湖南',
    '广东', '广西', '海南', '重庆', '四川', '贵州', '云南', '西藏', '陕西', '甘肃', '青海', '宁夏',
    '新疆', '香港', '澳门', '台湾'
]
long_provinces = [

    '北京市', '天津市', '河北省', '山西省', '内蒙古自治区', '辽宁省', '吉林省', '黑龙江省', '上海市', '江苏省', '浙江省', '安徽省',
    '福建省', '江西省', '山东省', '河南省', '湖北省', '湖南省',
    '广东省', '广西壮族自治区', '海南', '重庆市', '四川省', '贵州省', '云南省', '西藏自治区', '陕西省', '甘肃省', '青海省', '宁夏回族自治区',
    '新疆维吾尔自治区', '香港特别行政区', '澳门特别行政区', '台湾省'
]
with open('./world.csv', 'r',encoding='utf-8') as world_file :
    world_reader = csv.reader(world_file)
    errors=0
    line=0
    for row in world_reader:
        #magnitude ,stime_long, latitude, longitude, depth, addr ,stime_short,province
        magnitude = row[0]
        stime_long = row[1]
        latitude = row[2]
        longitude = row[3]
        depth = row[4]
        addr = row[5]
        stime_short = row[6]
        stime_year = row[7]
        stime_month = row[8]
        stime_clock= row[9]
        data_type='外国'
        addr=addr.replace(',','，')
        for province in provinces:
            if province in addr:
                data_type=province
        # magnitude, stime_long, latitude, longitude, depth, addr, stime_short,data_type
        print(magnitude, stime_long, latitude, longitude, depth, addr, stime_short,data_type)

        #tb_earthquake
        conn = pymysql.connect(host='bigdata', user='root', password='123456', port=3306, db='hive_earthquake')
        cursor = conn.cursor()
        data = (magnitude, stime_long, latitude, longitude, depth, addr, stime_short,data_type)
        sql = "replace into tb_earthquake (magnitude, stime_long, latitude, longitude, depth, addr, stime_short,data_type) " \
              "values (%s,%s,%s,%s,%s,%s,%s,%s)"
        cursor.execute(sql,data)
        conn.commit()
        cursor.close()
        if data_type != '外国':
            for long_province in long_provinces:
                if data_type in long_province:
                    earthquake_file = open("earthquake_hdfs.csv", mode="a+", newline='', encoding="utf-8")
                    earthquake_writer = csv.writer(earthquake_file)
                    earthquake_writer.writerow(
                        [magnitude, stime_long, latitude, longitude, depth, addr, stime_short,data_type,long_province,stime_year,stime_month,stime_clock])
                    earthquake_file.close()
        else:
            earthquake_file = open("earthquake_hdfs.csv", mode="a+", newline='', encoding="utf-8")
            earthquake_writer = csv.writer(earthquake_file)
            earthquake_writer.writerow(
                [magnitude, stime_long, latitude, longitude, depth, addr, stime_short,data_type,'无(外国)',stime_year,stime_month,stime_clock])
            earthquake_file.close()