Docker和Dify学习笔记

文章目录

1 docker学习
- 1.1 基本命令使用
- - 1.1.1 docker ps查看当前正在运行的镜像
  - 1.1.2 docker stop停止容器
  - 1.1.3 docker compose容器编排
  - 1.1.4 docker网络
  - - [1] 进入到容器里面敲命令
    - [2] docker network ls
    - [3] brige网络模式下容器访问宿主机的方式
2 Dify的安装和基础使用
- 2.1 下载dify的工程仓库
- 2.2 创建.env配置文件
- 2.3 修改Nginx的端口
- 2.4 启动Dify
- 2.5 添加一个本地模型
- - 2.5.1 添加一个大语言模型
  - 2.5.2 添加一个Embedding模型
3 基于Dify开发功能场景
- 3.1 Echarts绘图
- 3.2 让大模型能够解析Json
4 大模型或者Dify中常见的参数
- 4.1 Temperature
4 Dify使用过程中常见问题
- 4.1 json超长了，超过80000个字符了
参考资料

1 docker学习

1.1 基本命令使用

1.1.1 docker ps查看当前正在运行的镜像

PS E:\LargeModel\dify\docker> docker ps
CONTAINER ID   IMAGE                                       COMMAND                   CREATED          STATUS                    PORTS                                         NAMES
5b2b8c1636e7   langgenius/dify-web:1.0.0                   "/bin/sh ./entrypoin…"   6 minutes ago    Up 6 minutes              3000/tcp                                      thirsty_diffie
99740eb3c659   langgenius/dify-web:1.0.0                   "/bin/sh ./entrypoin…"   6 minutes ago    Up 6 minutes              3000/tcp                                      upbeat_euclid
f8108bddcff7   nginx:latest                                "sh -c 'cp /docker-e…"   11 minutes ago   Up 11 minutes             0.0.0.0:8001->80/tcp, 0.0.0.0:8443->443/tcp   docker-nginx-1
2794a7f88a69   langgenius/dify-api:1.0.0                   "/bin/bash /entrypoi…"   11 minutes ago   Up 11 minutes             5001/tcp                                      docker-api-1
d51ee39beb2c   langgenius/dify-api:1.0.0                   "/bin/bash /entrypoi…"   11 minutes ago   Up 11 minutes             5001/tcp                                      docker-worker-1
e008dc5e6386   ubuntu/squid:latest                         "sh -c 'cp /docker-e…"   11 minutes ago   Up 11 minutes             3128/tcp                                      docker-ssrf_proxy-1
8c966517876b   langgenius/dify-sandbox:0.2.10              "/main"                   11 minutes ago   Up 11 minutes (healthy)                                                 docker-sandbox-1
a2d137e0f516   langgenius/dify-plugin-daemon:0.0.3-local   "/bin/bash -c /app/e…"   11 minutes ago   Up 11 minutes             0.0.0.0:5003->5003/tcp                        docker-plugin_daemon-1
56f97d78ab4c   langgenius/dify-web:1.0.0                   "/bin/sh ./entrypoin…"   11 minutes ago   Up 11 minutes             3000/tcp                                      docker-web-1
cfe29ccee8df   postgres:15-alpine                          "docker-entrypoint.s…"   11 minutes ago   Up 11 minutes (healthy)   0.0.0.0:5432->5432/tcp                        docker-db-1
edd0a6879ba7   semitechnologies/weaviate:1.19.0            "/bin/weaviate --hos…"   11 minutes ago   Up 11 minutes                                                           docker-weaviate-1
191a080293e4   redis:6-alpine                              "docker-entrypoint.s…"   11 minutes ago   Up 11 minutes (healthy)   6379/tcp                                      docker-redis-1
PS E:\LargeModel\dify\docker>

1.1.2 docker stop停止容器

【停止指定名称的容器】

PS E:\LargeModel\dify\docker> docker stop thirsty_diffie
thirsty_diffie

【停止所有当前在运行的容器】

PS E:\LargeModel\dify\docker> docker stop $(docker ps -aq)
5b2b8c1636e7
99740eb3c659
f8108bddcff7
2794a7f88a69
d51ee39beb2c
e008dc5e6386
8c966517876b
a2d137e0f516
56f97d78ab4c
cfe29ccee8df
edd0a6879ba7
191a080293e4
fe75be867cd5

1.1.3 docker compose容器编排

1.1.4 docker网络

[1] 进入到容器里面敲命令

docker exec -it docker-api-1 /bin/bash

案例：安装ping命令和telnet命令

root@cb7f80d95b40:/app/api# apt-get update
Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB]
Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB]
Get:4 http://deb.debian.org/debian bookworm/main amd64 Packages [8792 kB]
Get:5 http://deb.debian.org/debian bookworm-updates/main amd64 Packages [13.5 kB]
Get:6 http://deb.debian.org/debian-security bookworm-security/main amd64 Packages [246 kB]
Fetched 9306 kB in 8min 19s (18.6 kB/s)
Reading package lists... Done
root@cb7f80d95b40:/app/api# apt-get install -y iputils-ping
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libcap2-bin libpam-cap
The following NEW packages will be installed:

[2] docker network ls

PS D:\LargeModel> docker network ls
NETWORK ID     NAME                        DRIVER    SCOPE
35353aad5d22   bridge                      bridge    local
6e02ea3b25c6   docker_default              bridge    local
5973270d4a91   docker_ssrf_proxy_network   bridge    local
503869a64910   host                        host      local
a0056c2c396d   none                        null      local

[3] brige网络模式下容器访问宿主机的方式

容器内可以使用host.docker.internal来代替主机的ip

2 Dify的安装和基础使用

2.1 下载dify的工程仓库

git clone https://github.com/langgenius/dify.git
# 国内镜像站
https://gitee.com/dify_ai/dify

2.2 创建.env配置文件

我们进入dify目录下的docker目录中,

# 以示例创建一个.env的文件，执行下面命令
cp .\.env.example .env

2.3 修改Nginx的端口

默认占用的是80和443端口，如果你本机已经部署了其他的应用，占了该端口，修改.env文件中的下面两个变量

EXPOSE_NGINX_PORT=8001
EXPOSE_NGINX_SSL_PORT=8443

2.4 启动Dify

docker compose up -d

2.5 添加一个本地模型

2.5.1 添加一个大语言模型

在这里插入图片描述

这里需要注意，我的ollama是直接安装在宿主机的。但是Dify是通过docker启动起来的，这里涉及到docker和宿主机之间的通信。如果docker访问宿主机，可以使用host.docker.internal域名，Docker的DNS可以解析这个域名。

2.5.2 添加一个Embedding模型

（1）安装bge-m3模型

ollama pull bge-m3

（2）Dify中配置embedding模型
在这里插入图片描述

3 基于Dify开发功能场景

3.1 Echarts绘图

思路就是利用Dify的echarts渲染的能力(即使是Dify自带的Echarts图表生成工具也是输出了一串echarts的配置字符串，甚至还没有直接写python代码生成来的直接)。但是只能在工作流和chatflow里面使用。如果将绘图的工作流集成到Agent里面的话，会导致大模型解析不了json，输出不了内容了。

import json
import requests
from datetime import datetime, timedelta
import statistics

"""
    这是一个try参数的机制
"""
def getValidResponse(url, headers, params):
    response = None
    for resourceFlag in ['sw', 'lk', 'wlw']:
        params["resource_flag"] = resourceFlag

        response = requests.get(url= url, headers= headers, params= params, timeout= 10)
        # 检查请求是否成功
        if response.status_code == 200:
            try:
                json_data = response.json()
                print(json_data)
                if (not "data" in json_data) or json_data["data"] is None or len(json_data["data"]) <= 0:
                    continue
                else:
                    return response
            except ValueError:
                continue        
        else:
            continue        

"""
    获取某一个水库某一段时间内的的水情数据。（主要是水位）
    参数：
        startDate: start_dt=2025-03-16+10:00:00
        endDate: end_dt=2025-03-17+11:00:00
"""
def getWaterStatus(guid: str, stcd: str, startDate: str, endDate: str) -> dict:
    url = "https://sk.hubeishuiyi.cn/services/1234567890ABCDEFGHIJKLMN/res_z_detail/param"
    headers = {
        "Apikey": "F1DBECD719108635189480CF60E6553ADB3109616426BD537F25A430DFC613B491A025C4A51E77FD08C6E5B7CBE05917A461286E7B6D69F1AB1B14F946149D2065B0C675F8FEDF4B9B05C1496881BC5A"
    }
    params = {
        "is_tb": "n",
        "pageNo": 1,
        "pageSize":10000,
        "st_cd": stcd,
        "resource_flag":"sw",
        "flag": 1,
        "start_dt": startDate,
        "end_dt": endDate,
        "res_cd": guid
    }

    response = getValidResponse(url, headers, params)
    # 检查请求是否成功
    if response is not None and response.status_code == 200:
        try:
            json_data = response.json()
            # # 转为双引号的标准json字符串
            # json_data = json.dumps(json_data)
            # json_data = json.loads(json_data)
            print(json_data)
            if (not "data" in json_data) or json_data["data"] is None or len(json_data["data"]) <= 0:
                return {"result" : []} 
            
            # 以时间作为横坐标
            timeList = []
            # 水位数据
            waterLevelList = []
            # 蓄水量
            waterStorageList = []
            for dataObject in json_data["data"]:
                timeStr = dataObject["tm"]
                timeList.append(timeStr)
                waterLevel = dataObject["rz"]
                waterLevelList.append(waterLevel)
                waterStorage = dataObject["w"]
                waterStorageList.append(waterStorage)

            statisticData = {
                "time": timeList,
                "waterLevel": waterLevelList,
                "waterStorage": waterStorageList
            }

            return {"result": statisticData}
        except ValueError:
            return {"result" : "Failed to parse JSON response"}
    else:
        statisticData = {
            "failMsg": f"Request failed. url = {url}, params = {params}"
        }
        return {"result" : statisticData}

def calcYAxisMinMax(nums: list):
    minValue = min(nums) - 8
    if minValue < 0:
        minValue = 0
    maxValue = max(nums) + 8

    return minValue, maxValue

"""
    参数：
        data： tuple的列表。
"""
def create_dynamic_table_str(headers: list, data):
    table = "|"
    for header in headers:
        table += f" {header} |"
    table += "\n|"
    for _ in headers:
        table += " ------- |"
    for row in data:
        table += "\n|"
        for item in row:
            table += f" {item} |"
    return table

def constructTableString(timeList: list, waterLevelList: list) -> str:
    timeList = timeList[-10:]
    waterLevelList = waterLevelList[-10:]

    headers = ['时间', '水位(m)']
    data = list(zip(timeList, waterLevelList))
    return create_dynamic_table_str(headers, data)

def constructStatisticString(waterLevelList: list) -> str:
    headers = ['统计项', '统计值']
    data = []

    minValue = min(waterLevelList)
    data.append(("最小值", f"{minValue:.2f}"))
    maxValue = max(waterLevelList)
    data.append(("最大值", f"{maxValue:.2f}"))
    meanValue = sum(waterLevelList) / len(waterLevelList)
    data.append(("平均值", f"{meanValue:.2f}"))
    # 计算列表的中位数
    medianValue = statistics.median(waterLevelList)
    data.append(("中位数", f"{medianValue:.2f}"))
    # 标准差
    devValue = statistics.stdev(waterLevelList)
    data.append(("标准差", f"{devValue:.2f}"))

    return create_dynamic_table_str(headers, data=data)

def drawEcharts(data: dict) -> dict:
    if data is None or len(data) <= 0:
        return {"result": ""}
    if "time" not in data or "waterLevel" not in data or "waterStorage" not in data:
        return {"result": ""}
    
    waterStatusData = data
    
    # 提取时间和对应的数据
    timeList = list(reversed(waterStatusData["time"]))
    waterLevelList = waterStatusData["waterLevel"]
    waterStorageList = waterStatusData["waterStorage"]

    if timeList is None or len(timeList) < 2:
        return {"result": ""}

    # 绘图准备工作
    UTC_FORMAT = "%Y-%m-%d %H:%M:%S"
    startTime = datetime.strptime(timeList[0], UTC_FORMAT)
    endTime = datetime.strptime(timeList[-1], UTC_FORMAT)

    walterLevelMin, walterLevelMax = calcYAxisMinMax(waterLevelList)

    # 生成echarts配置
    echarts_config = {
        "color": ['#eb6877', '#0f91c4', '#46cbd4'],
        "title": {
            "subtext": f"{startTime.strftime('%m')}月{startTime.strftime('%d')}日-{endTime.strftime('%m')}月{endTime.strftime('%d')}日水位情况",
            "left": 20
        },
        "tooltip": {
            "trigger": "axis",
            "axisPointer": {
                "type": "cross"
            }
        },
        "legend": {
            # "data": ["水位", "最低气温", "降水"],
            "data": ["水位"],
            "right": 20
        },
        "xAxis": {
            "data": timeList,
            "axisLine": {
                "onZero": False
            }
        },
        "yAxis": [
            {
                "type": "value",
                "name": "水位",
                "position": "left",
                "min": walterLevelMin,
                "max": walterLevelMax,
                "axisLabel": {
                    "formatter": "{value} m"
                }
            },
            # {
            #     "type": "value",
            #     "name": "蓄水量",
            #     "position": "right",
            #     "axisLabel": {
            #         "formatter": "{value} m"
            #     }
            # }
        ],
        "series": [
            {
                "name": "水位",
                "type": "line",
                "data": waterLevelList,
                "yAxisIndex": 0,
                "itemStyle": {
                    "color": "#eae213"
                },
                "markPoint": {
                    "data": [
                        {
                            "type": 'max'
                        },
                        {
                            "type": 'min'
                        },
                    ],
                    # 设置为点
                    "symbol": 'circle', 
                    # 调整点的大小
                    "symbolSize": 8, 
                    "label": {
                        "position": 'right',
                        # 标签字体加粗
                        "fontWeight": 'bold', 
                        # 标签字体大小
                        "fontSize": 12 
                    }
                },
            },
            # {
            #     "name": "蓄水量",
            #     "type": "bar",
            #     "smooth": True,
            #     "data": waterStorageList,
            #     "yAxisIndex": 0,
            #     "itemStyle": {
            #         "color": "#4bb2fa"
            #     }
            # },
            # {
            #     "name": "降水",
            #     "type": "bar",
            #     "smooth": True,
            #     "data": rainfall,
            #     "yAxisIndex": 1,
            #     "itemStyle": {
            #         "color": "#31e84f"
            #     }
            # }
        ]
    }

    echartString = "```echarts\n" + json.dumps(echarts_config, indent=2, ensure_ascii=False) + "\n```"
    tableString = constructTableString(timeList, waterLevelList)
    statisticString = constructStatisticString(waterLevelList)

    # 生成输出文件
    output = echartString + "\n\n"  \
        + "最近十条数据展示：\n" + tableString + "\n\n" \
        + "常用统计数据展示：\n" + statisticString

    return {"result":output}

def main(guid: str, stcdList: list) -> dict:
    if guid is None or stcdList is None or len(guid) <= 0 or len(stcdList) <= 0:
        return {"result" : ""}
    
    endDate = datetime.now()
    endDateStr = endDate.strftime("%Y-%m-%d %H:%M:%S")
    startDate = endDate - timedelta(days=15)
    startDateStr = startDate.strftime("%Y-%m-%d %H:%M:%S")
    result = getWaterStatus(guid=guid, stcd=stcdList[0], startDate=startDateStr, endDate=endDateStr)
    print(f"main.result = {result['result']}")
    markdownScript = drawEcharts(result["result"])
    print(markdownScript)
    return markdownScript

if __name__ == "__main__":
    # print(main('42022220001', ['61608180']))
    # print(main('42130330004', ["61608180"]))
    # print(main('42122350024', ['90021804']))
    print(main('42058340006', [ "90006379"]))

3.2 让大模型能够解析Json

4 大模型或者Dify中常见的参数

4.1 Temperature

LLM 生成是具有随机性的，在模型的顶层通过选取不同预测概率的预测结果来生成最后的结果。我们一般可以通过控制 temperature 参数来控制 LLM 生成结果的随机性与创造性。

Temperature 一般取值在 0~1 之间，当取值较低接近 0 时，预测的随机性会较低，产生更保守、可预测的文本，不太可能生成意想不到或不寻常的词。当取值较高接近 1 时，预测的随机性会较高，所有词被选择的可能性更大，会产生更有创意、多样化的文本，更有可能生成不寻常或意想不到的词。

4 Dify使用过程中常见问题

4.1 json超长了，超过80000个字符了

这是因为Dify限制了默认的长度。本地部署的情况下可以修改.env配置文件中的相关变量数值。修改之后重启整个服务。

常见的参数和含义入下图所示：
在这里插入图片描述
重启服务所需要使用的命令：

docker compose down
docker compose up -d

参考资料

[1] https://mp.weixin.qq.com/s/n5GrGZ9hZmdhzt4avs1XSw
[2] https://wiki.eryajf.net/pages/674f53/
[3] https://zhuanlan.zhihu.com/p/20939683190
[4] https://dify.flowus.cn/haojixing/share/943044ea-005e-4e57-9e74-450700df71c2
[5] https://blog.csdn.net/luckcxy/article/details/144900399