计算机毕业设计基于爬虫与文本挖掘的网络舆情监控系统的设计与实现 Python+Django+Vue 前后端分离附源码讲解文档

🍊作者：计算机编程-吉哥
🍊简介：专业从事JavaWeb程序开发，微信小程序开发，定制化项目、源码、代码讲解、文档撰写、ppt制作。做自己喜欢的事，生活就是快乐的。
🍊心愿：点赞 👍 收藏 ⭐评论 📝
🍅 文末获取源码联系

👇🏻 精彩专栏推荐订阅 👇🏻 不然下次找不到哟~
Java毕业设计项目~热门选题推荐《1000套》

Python毕业设计精品项目《1000套》

微信小程序毕业设计精品项目《1000套》

大数据/机器学习毕业设计精品项目《1000套》

1.技术选型

2.开发工具

3.功能

3.1【角色】

3.2【用户功能模块】

3.3【管理员功能模块】

4.项目演示截图

4.1 可视化报表

4.2 登录

4.3 注册

4.4 查看网络舆情

4.5 用户管理

4.6 网络舆情管理

4.7 系统简介

5.核心代码

5.1 Settings.py

5.2 Urls.py

5.3 Users_v.py

5.4 Auth.py

6.LW文档大纲参考

背景意义介绍：

基于爬虫与文本挖掘的网络舆情监控系统在当前社会中扮演着至关重要的角色。随着互联网的普及和社交媒体的兴起，网络舆情对个人、企业乃至政府机构的影响日益凸显。该系统通过自动化的数据采集技术，结合文本挖掘和自然语言处理算法，能够实时监测、分析并预警网络舆情，帮助用户洞察公众情绪、发现潜在的危机并及时做出响应。

系统的建设对于维护社会稳定、保护企业品牌声誉、指导政府决策具有重要意义。通过对海量网络数据的深入分析，该系统能够识别出热点话题、情感倾向和关键信息，为决策者提供科学的决策支持。此外，系统的可视化大屏功能，能够直观展示舆情分析结果，使得用户能够快速把握舆情动态，提高决策的时效性和准确性。

1.技术选型

Python、Django、vue、elementui、html、css、js、mysql、jdk1.8

2.开发工具

pycharm、navicat

3.功能

3.1【角色】

管理员、用户

3.2【用户功能模块】

登录
注册
主页
用户信息
网络舆情

3.3【管理员功能模块】

登录
主页
用户信息
用户管理
网络舆情管理（爬虫）
系统简介
可视化大屏

4.项目演示截图

4.1 可视化报表

4.2 登录

4.3 注册

4.4 查看网络舆情

4.5 用户管理

4.6 网络舆情管理

4.7 系统简介

5.核心代码

5.1 Settings.py

"""
Django settings for dj2 project.

Generated by 'django-admin startproject' using Django 2.0.

For more information on this file, see
https://docs.djangoproject.com/en/2.0/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/2.0/ref/settings/
"""

import os
from concurrent.futures.thread import ThreadPoolExecutor
executor = ThreadPoolExecutor(20)
from util.configread import config_read

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/2.0/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'w5yn#0gn2tt7pvu%hvwt0!lt=!$6+eqp4%m8)u3u#gknm@jm)k'

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = ["*"]

# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    "main",
    'corsheaders',
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    # 'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'threadlocals.middleware.ThreadLocalMiddleware',
    "xmiddleware.xparam.Xparam",
    "xmiddleware.xauth.Xauth",
    'corsheaders.middleware.CorsMiddleware',
    'django.middleware.common.CommonMiddleware',

]
CORS_ALLOW_CREDENTIALS = True
CORS_ORIGIN_ALLOW_ALL = True
CORS_ALLOW_HEADERS = ('*')

SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_COOKIE_NAME  = "sessionid"
SESSION_COOKIE_PATH  = "/"
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_SECURE = False
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_AGE = 1209600
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_SAVE_EVERY_REQUEST = False

ROOT_URLCONF = 'dj2.urls'
TEMPLATES_DIR = os.path.join(BASE_DIR, "templates")
TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [TEMPLATES_DIR],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = 'dj2.wsgi.application'

EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_USE_TLS = False
EMAIL_USE_SSL = True
EMAIL_HOST = 'smtp.qq.com'
EMAIL_PORT = 465
EMAIL_HOST_USER = 'yclw9@qq.com'
EMAIL_HOST_PASSWORD = 'mhbrkuayvkkgbijd'

# Database
# https://docs.djangoproject.com/en/2.0/ref/settings/#databases

# DATABASES = {
#     'default': {
#         'ENGINE': 'django.db.backends.sqlite3',
#         'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
#     }
# }

dbtype, host, port, user, passwd, dbName, charset,hasHadoop = config_read("config.ini")
dbName=dbName.replace(" ","").strip()
print(dbtype, host, port, user, passwd, dbName, charset)

if dbtype == 'mysql':
    DATABASES = {
        'default': {
            # 'ENGINE': 'django.db.backends.sqlite3',
            # 'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
            'ENGINE': 'django.db.backends.mysql',
            'OPTIONS': {
                'sql_mode': 'traditional',
                'init_command': "SET sql_mode='traditional'",  # STRICT_TRANS_TABLES
            },
            'NAME': dbName,
            'USER': user,
            'PASSWORD': passwd,
            'HOST': host,
            'PORT': port,
            'charset': charset,
            'TEST': {
                'CHARSET': charset,
                'COLLATION': 'utf8_general_ci',
            },
            'CONN_MAX_AGE':60
        },
    }
else:
    print("请使用mysql5.5数据库")
    os._exit(1)

# Password validation
# https://docs.djangoproject.com/en/2.0/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]

# Internationalization
# https://docs.djangoproject.com/en/2.0/topics/i18n/

LANGUAGE_CODE = 'zh-Hans'

# TIME_ZONE = 'UTC'
TIME_ZONE = 'Asia/Shanghai'

USE_I18N = True

USE_L10N = True

# USE_TZ = True
USE_TZ = False

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/2.0/howto/static-files/



STATIC_URL = '/assets/'
STATICFILES_DIRS =[
os.path.join(BASE_DIR, "templates/front/assets"),
]

# media
MEDIA_URL = "/media/"  # 自定义
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')  # 自定义
if os.path.isdir(MEDIA_ROOT) == False:
    os.mkdir(MEDIA_ROOT)

ALIPAY_APP_ID = '9021000132629452'
APP_PRIVATE_KEY_STRING = open('{}/util/alipay_key/app_private_2048.txt'.format(BASE_DIR)).read()
ALIPAY_PUBLIC_KEY_STRING = open('{}/util/alipay_key/alipay_public_2048.txt'.format(BASE_DIR)).read()
ALIPAY_SIGN_TYPE = 'RSA2'

5.2 Urls.py

"""dj2 URL Configuration

The `urlpatterns` list routes URLs to views. For more information please see:
    https://docs.djangoproject.com/en/2.0/topics/http/urls/
Examples:
Function views
    1. Add an import:  from my_app import views
    2. Add a URL to urlpatterns:  path('', views.home, name='home')
Class-based views
    1. Add an import:  from other_app.views import Home
    2. Add a URL to urlpatterns:  path('', Home.as_view(), name='home')
Including another URLconf
    1. Import the include() function: from django.urls import include, path
    2. Add a URL to urlpatterns:  path('blog/', include('blog.urls'))
"""
import os
from django.contrib import admin
from django.urls import path,include,re_path
from django.conf.urls import url
from django.views.static import serve
from django.views.generic import TemplateView


from . import views
from dj2.settings import dbName as schemaName

urlpatterns = [
    path('xadmin/', admin.site.urls),
    path(r'index/',views.index),
    path('{}/'.format(schemaName),include('main.urls')),#导入schemaName
    re_path(r'admin/lib/(?P<p1>.*)/(?P<p2>.*)$', views.admin_lib2),
    re_path(r'admin/lib/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)$', views.admin_lib3),
    re_path(r'admin/lib/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)/(?P<p4>.*)$', views.admin_lib4),
    re_path(r'admin/page/(?P<p1>.*)$', views.admin_page),
    re_path(r'admin/page/(?P<p1>.*)/(?P<p2>.*)$', views.admin_page2),
    re_path(r'admin/pages/(?P<p1>.*)$', views.admin_pages),
    re_path(r'admin/pages/(?P<p1>.*)/(?P<p2>.*)$', views.admin_pages2),
    
    re_path(r'front/(?P<p1>.*)$', views.schema_front1),
    re_path(r'front/(?P<p1>.*)/(?P<p2>.*)$', views.schema_front2),
    re_path(r'front/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)$', views.schema_front3),
    re_path(r'front/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)/(?P<p4>.*)$', views.schema_front4),
    re_path(r'{}/front/(?P<p1>.*)$'.format(schemaName), views.schema_front1),
    re_path(r'{}/front/(?P<p1>.*)/(?P<p2>.*)$'.format(schemaName), views.schema_front2),
    re_path(r'{}/front/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)$'.format(schemaName), views.schema_front3),
    re_path(r'{}/front/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)/(?P<p4>.*)$'.format(schemaName), views.schema_front4),
    # re_path(r'assets/(?P<p1>.*)$', views.assets1),
    # re_path(r'assets/(?P<p1>.*)/(?P<p2>.*)$',  views.assets2),
    # re_path(r'assets/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)$',  views.assets3),
    # re_path(r'assets/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)/(?P<p4>.*)$',  views.assets4),
    #re_path(r'admin/(?P<p1>.*)$', views.admin_file1),
    re_path(r'admin/(?P<p1>.*)/(?P<p2>.*)$', views.admin_file2),
    re_path(r'admin/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)$', views.admin_file3),
    re_path(r'admin/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)/(?P<p4>.*)$', views.admin_file4),
    re_path(r'layui/(?P<p1>.*)$', views.layui1),
    re_path(r'layui/(?P<p1>.*)/(?P<p2>.*)$',  views.layui2),
    re_path(r'layui/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)$',  views.layui3),
    re_path(r'layui/(?P<p1>.*)/(?P<p2>.*)/(?P<p3>.*)/(?P<p4>.*)$',  views.layui4),
    re_path(r'pages/(?P<p1>.*)$', views.front_pages),
    re_path(r'pages/(?P<p1>.*)/(?P<p2>.*)$',  views.front_pages2),
    # re_path(r'pages/(?P<p1>.*)$',  views.front_file1),
    # re_path(r'(?P<p1>css|jss|img|image|iamges|font|fonts)/(?P<p2>.*)$', views.front_file2),
    re_path(r'modules/(?P<p1>.*)$', views.front_modules),
    re_path(r'css/(?P<p1>.*)$', views.css1),
    re_path(r'js/(?P<p1>.*)$', views.js1),
    re_path(r'img/(?P<p1>.*)$', views.img1),
    path(r'test/<str:p1>/',views.test),
    path(r'null',views.null),
]

#判断admin使用vue还是jquery
if os.path.isdir(os.path.join(os.getcwd(),"templates/front/admin/dist/")):
    urlpatterns.extend([
        path(r'{}/admin/dist/index.html'.format(schemaName),
             TemplateView.as_view(template_name='front/admin/dist/index.html')),
        path(r'{}/admin/'.format(schemaName), TemplateView.as_view(template_name='front/admin/dist/index.html')),
        # 以下是后台admin的url匹配规则
        path(r'admin/dist/index.html'.format(schemaName),
             TemplateView.as_view(template_name='front/admin/dist/index.html')),
        path(r'admin/', TemplateView.as_view(template_name='front/admin/dist/index.html')),
    ])
else:
    urlpatterns.extend([
        path(r'{}/admin/index.html'.format(schemaName),
             TemplateView.as_view(template_name='front/admin/index.html')),
        path(r'{}/admin/'.format(schemaName), TemplateView.as_view(template_name='front/admin/index.html')),
        # 以下是后台admin的url匹配规则
        path(r'admin/index.html'.format(schemaName),
             TemplateView.as_view(template_name='front/admin/index.html')),
        path(r'admin/', TemplateView.as_view(template_name='front/admin/index.html')),

    ])


if os.path.isfile(os.path.join(os.getcwd(),"templates/front/index.html")):
    urlpatterns.extend([
    path(r'index.html', TemplateView.as_view(template_name='front/index.html')),
     path(r'{}/index.html'.format(schemaName), TemplateView.as_view(template_name='front/index.html')),
     path(r'{}/front/index.html'.format(schemaName), TemplateView.as_view(template_name='front/index.html')),
    path(r'', TemplateView.as_view(template_name='front/index.html')),
    ])

5.3 Users_v.py

# coding:utf-8
__author__ = "ila"

from django.http import JsonResponse

from .users_model import users
from util.codes import *
from util.auth import Auth
import util.message as mes
from dj2.settings import host,port,user,passwd,dbName,hasHadoop

def users_login(request):
    if request.method in ["POST", "GET"]:
        msg = {'code': normal_code, "msg": mes.normal_code}
        req_dict = request.session.get("req_dict")
        if req_dict.get('role')!=None:
            del req_dict['role']
        
        datas = users.getbyparams(users, users, req_dict)
        if not datas:
            msg['code'] = password_error_code
            msg['msg'] = mes.password_error_code
            return JsonResponse(msg)

        req_dict['id'] = datas[0].get('id')
        return Auth.authenticate(Auth, users, req_dict)


def users_register(request):
    if request.method in ["POST", "GET"]:
        msg = {'code': normal_code, "msg": mes.normal_code}
        req_dict = request.session.get("req_dict")

        error = users.createbyreq(users, users, req_dict)
        if error != None:
            msg['code'] = crud_error_code
            msg['msg'] = error
        return JsonResponse(msg)


def users_session(request):
    '''
    '''
    if request.method in ["POST", "GET"]:
        msg = {"code": normal_code,"msg":mes.normal_code, "data": {}}

        req_dict = {"id": request.session.get('params').get("id")}
        msg['data'] = users.getbyparams(users, users, req_dict)[0]

        return JsonResponse(msg)


def users_logout(request):
    if request.method in ["POST", "GET"]:
        msg = {
            "msg": "退出成功",
            "code": 0
        }

        return JsonResponse(msg)


def users_page(request):
    '''
    '''
    if request.method in ["POST", "GET"]:
        msg = {"code": normal_code, "msg": mes.normal_code,
               "data": {"currPage": 1, "totalPage": 1, "total": 1, "pageSize": 10, "list": []}}
        req_dict = request.session.get("req_dict")
        tablename = request.session.get("tablename")
        try:
            __hasMessage__ = users.__hasMessage__
        except:
            __hasMessage__ = None
        if __hasMessage__ and __hasMessage__ != "否":

            if tablename != "users":
                req_dict["userid"] = request.session.get("params").get("id")
        if tablename == "users":
            msg['data']['list'], msg['data']['currPage'], msg['data']['totalPage'], msg['data']['total'], \
            msg['data']['pageSize'] = users.page(users, users, req_dict)
        else:
            msg['data']['list'], msg['data']['currPage'], msg['data']['totalPage'], msg['data']['total'], \
            msg['data']['pageSize'] = [],1,0,0,10

        return JsonResponse(msg)


def users_info(request, id_):
    '''
    '''
    if request.method in ["POST", "GET"]:
        msg = {"code": normal_code, "msg": mes.normal_code, "data": {}}

        data = users.getbyid(users, users, int(id_))
        if len(data) > 0:
            msg['data'] = data[0]
        # 浏览点击次数
        try:
            __browseClick__ = users.__browseClick__
        except:
            __browseClick__ = None

        if __browseClick__ and "clicknum" in users.getallcolumn(users, users):
            click_dict = {"id": int(id_), "clicknum": str(int(data[0].get("clicknum", 0)) + 1)}
            ret = users.updatebyparams(users, users, click_dict)
            if ret != None:
                msg['code'] = crud_error_code
                msg['msg'] = ret
        return JsonResponse(msg)


def users_save(request):
    '''
    '''
    if request.method in ["POST", "GET"]:
        msg = {"code": normal_code, "msg": mes.normal_code, "data": {}}
        req_dict = request.session.get("req_dict")
        req_dict['role'] = '管理员'
        error = users.createbyreq(users, users, req_dict)
        if error != None:
            msg['code'] = crud_error_code
            msg['msg'] = error
        return JsonResponse(msg)


def users_update(request):
    '''
    '''
    if request.method in ["POST", "GET"]:
        msg = {"code": normal_code, "msg": mes.normal_code, "data": {}}
        req_dict = request.session.get("req_dict")
        if req_dict.get("mima") and req_dict.get("password"):
            if "mima" not in users.getallcolumn(users,users):
                del req_dict["mima"]
            if "password" not in users.getallcolumn(users,users):
                del req_dict["password"]
        try:
            del req_dict["clicknum"]
        except:
            pass
        error = users.updatebyparams(users, users, req_dict)
        if error != None:
            msg['code'] = crud_error_code
            msg['msg'] = error
        return JsonResponse(msg)


def users_delete(request):
    '''
    '''
    if request.method in ["POST", "GET"]:
        msg = {"code": normal_code, "msg": mes.normal_code, "data": {}}
        req_dict = request.session.get("req_dict")

        error = users.deletes(users,
            users,
            req_dict.get("ids")
        )
        if error != None:
            msg['code'] = crud_error_code
            msg['msg'] = error
        return JsonResponse(msg)

5.4 Auth.py

# coding:utf-8
# author:ila
import base64, copy
from django.http import JsonResponse
from django.apps import apps

from util.codes import *
from util import message as mes


class Auth(object):
    def authenticate(self, model, req_dict):
        """
        用户登录，登录成功返回token；登录失败返回失败原因
        :param username:账号
        :param password:密码
        :return: json
        """
        msg = {'code': normal_code, 'msg': mes.normal_code, 'data': {}}
        tablename = model.__tablename__
        encode_dict = {"tablename": tablename, "params": req_dict}

        encode_str = base64.b64encode(str(encode_dict).encode("utf-8"))
        msg['data']["id"] = req_dict.get("id")
        msg["id"] = req_dict.get("id")
        msg['token'] = encode_str.decode('utf-8')
        return JsonResponse(msg)

    def identify(self, request):
        """
        用户鉴权
        :param request:本次请求对象
        :return: list
        """

        msg = {'code': normal_code, 'msg': mes.normal_code, 'data': {}}
        # django的header被处理过了
        token = request.META.get('HTTP_TOKEN')

        if token  and token !="null":

            auth_token = copy.deepcopy(token)

            decode_str = base64.b64decode(auth_token).decode("utf8")
            decode_str=decode_str.replace('"null"','""').replace('null','""')
            decode_dict = eval(decode_str)

            tablename2 = decode_dict.get("tablename")

            params2 = decode_dict.get("params",{})

            datas=None
            allModels = apps.get_app_config('main').get_models()
            for model in allModels:
                if model.__tablename__ == tablename2:
                    datas = model.getbyparams(model, model, params2)

            if not datas:
                msg['code'] = username_error_code
                msg['msg'] = '找不到该用户信息'
                result = msg
            else:
                request.session['tablename'] = tablename2
                request.session['params'] = params2
                msg['msg'] = '身份验证通过。'
                result = msg
        else:
            msg['code'] = 401
            msg['msg'] = 'headers未包含认证信息。'
            result = msg
        return result