基于Hive大数据分析springboot为后端以及vue为前端的的民宿系

news2024/7/6 19:06:36

标题基于Hive大数据分析springboot为后端以及vue为前端的的民宿系

本文介绍了如何利用Hive进行大数据分析，并结合Spring Boot和Vue构建了一个民宿管理系统。该民民宿管理系统包含用户和管理员登陆注册的功能，发布下架酒店信息，模糊搜索，酒店详情信息展示，收藏以及对收藏的酒店进行排序可视化，管理员发布，添加用户的功能。

在这里插入图片描述

1. 数据采集

通过 Python 爬虫实现数据采集，并将采集到的数据存储到数据库中。这些数据将作为我们民宿系统的基础数据，在后续的处理和分析中发挥重要作用。我们首先使用 Python 爬虫从艺龙酒店获取民宿相关信息，包括房源信息、评论信息、价格信息等。以下是一个简单的示例代码，使用 BeautifulSoup 库进行网页解析，并通过 requests 库发送 HTTP 请求获取网页内容。

import json
import time

import requests

from request_data.sync_data2mysql import request_data2mysql

hotels_data = []

class YiLongList:
    def __init__(self,inDate,outDate,city):
        self.inDate,self.outDate = inDate,outDate
        self.city = city
        self.session = requests.Session()
    def jxList(self,hotelList):
        for h in hotelList:
            hotelName = h.get("hotelName")
            starLevelDes = h.get("starLevelDes")
            commentScore = h.get("commentScore")
            price = h.get("price")
            commentScoreDes = h.get("commentScoreDes")
            commentMainTag = h.get("commentMainTag")
            commentCount = h.get("commentCount")
            hotelAddress = h.get("hotelAddress")
            trafficInfo = h.get("trafficInfo")
            areaName = h.get("areaName")
            hotelTags = h.get("hotelTags")
            if hotelTags:
                hotelTags = "|".join([i.get("tagName") for i in hotelTags])
            themeList = "|".join(h.get("themeList"))
            recallReason = h.get("recallReason")[0]
            data = {
   
                'hotelName': hotelName,
                'starLevelDes': starLevelDes,
                'commentCount': commentCount,
                'price': price,
                'commentScore': commentScore,
                'commentScoreDes': commentScoreDes,
                'areaName': areaName,
                'hotelTags': hotelTags,
                'themeList': themeList,
                'recallReason': recallReason,
                'commentMainTag': commentMainTag,
                'hotelAddress': hotelAddress,
                'trafficInfo': trafficInfo
            }
            hotels_data.append(data)

            print(hotelName,starLevelDes,commentCount,price,commentScore,commentScoreDes,\
                  areaName,hotelTags,themeList,recallReason,commentMainTag,hotelAddress,trafficInfo,)
    def getList(self,page,traceToken):
        url = "https://hotel.elong.com/tapi/v2/list"
        headers = {
   
            "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6,zh-TW;q=0.5",
            "Referer": "https://hotel.elong.com/hotel/hotellist?city=0101&inDate=2024-03-23&outDate=2022-03-24&filterList=8888_1&pageSize=20&t=1669994718934",
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36 Edg/107.0.1418.62",
            "accept": "application/json, text/plain, */*",
            "deviceid": "c2a61f65-fa3e-4360-8ccd-4093f4f99f41",
            "traceid": "4630a377-472b-4765-852f-959e2fa139cd"
        }
        if page == 0:
            params = {
   
                "city": self.city,
                "inDate": self.inDate,
                "outDate": self.outDate,
                "filterList": "8888_1",
                "pageIndex": str(page),
                "pageSize": "20",
                "sugActInfo": ""
            }
        else:
            params = {
   
                "city"