物联网数据采集,需写入大量数据,这时就用到时序数据库来存储和快速读取
我个人使用感觉国产的开源项目TDengine,比InfluxDB操作方便很多,容易上手,支持使用SqlSugar进行数据操作,跟操作mysql数据库一样的体验
跟mysql最有特别之处,就是TDengine有个超级表=表结构模板 的概念,可以基于这个超级表的结构,建N个数量的子表,查询的时候,可以查单个子表,也可以查超级表所有子表的数据集合
超级表概念如:
建超级表A
基于超级表A的结构 建子表 : A1、A2、A3、N
然后查询的时候,可以单查A1表数据,也可以通过超级表A查所有A1、A2...集合的数据出来
用在传感器数据采集日志,一个设备一个表,最适合不过
TDengine官网技术文档 :https://docs.taosdata.com/connector/csharp/
一、window下安装TDengine
https://docs.taosdata.com/releases/tdengine/
找到同时有server和client 的版本
TDengine-server-xxx-Windows 数据库主安装文件
TDengine-client-xxx-Windows 数据库连接工具
二、服务端安装:
1、双击TDengine-server-3.0.7.1-Windows-x64.exe 安装,一直回车即可
2、配置数据库变量
打开C:\TDengine\cfg下的taos.cfg
########################################################
# #
# TDengine Configuration #
# Any questions, please email support@taosdata.com #
# #
######################################################### first fully qualified domain name (FQDN) for TDengine system
# taosd启动时,主动连接的集群中第一个dnode的end point, 默认值为localhost:6030。
firstEp vm1:6030# second fully qualified domain name (FQDN) for TDengine system, for cluster only
# taosd启动时,如果first连接不上,尝试连接集群中第二个dnode的end point, 默认值为空。
secondEp vm2:6030# local fully qualified domain name (FQDN)
# 数据节点的FQDN,缺省为操作系统配置的第一个hostname。如果习惯IP地址访问,可设置为该节点的IP地址。
fqdn vm1# first port number for the connection (12 continuous UDP/TCP port number are used)
# taosd启动后,对外服务的端口号,默认值为6030。
serverPort 6030# log file's directory
# 日志文件目录,客户端和服务器的运行日志文件将写入该目录。默认值:/var/log/taos。
logDir /home/radmin/data/tdengine/log# data file's directory
# 数据文件目录,所有的数据文件都将写入该目录。默认值:/var/lib/taos。
dataDir /home/radmin/data/tdengine/data# the arbitrator's fully qualified domain name (FQDN) for TDengine system, for cluster only
# 系统中裁决器的end point, 缺省值为空。
# arbitrator arbitrator_hostname:6042# number of threads per CPU core
# numOfThreadsPerCore 1.0# number of management nodes in the system
# 系统中管理节点个数。默认值:3。
# numOfMnodes 3# enable/disable backuping vnode directory when removing dnode
# vnodeBak 1# enable/disable load balancing
# 是否启动负载均衡。0:否,1:是。默认值:1。
# balance 1# role for dnode. 0 - any, 1 - mnode, 2 - dnode
# dnode的可选角色。0-any; 既可作为mnode,也可分配vnode;1-mgmt;只能作为mnode,不能分配vnode;2-dnode;不能作为mnode,只能分配vnode
# role 0# max timer control blocks
# maxTmrCtrl 512# time interval of system monitor, seconds
# monitorInterval 30# number of seconds allowed for a dnode to be offline, for cluster only
# dnode离线阈值,超过该时间将导致该dnode从集群中删除。单位为秒,默认值:86400*10(即10天)。
# offlineThreshold 8640000# RPC re-try timer, millisecond
# rpcTimer 300# RPC maximum time for ack, seconds.
# rpcMaxTime 600# time interval of dnode status reporting to mnode, seconds, for cluster only
# statusInterval 1# time interval of heart beat from shell to dnode, seconds
# shellActivityTimer 3# time of keeping table meta data in cache, seconds
# tableMetaKeepTimer 7200# minimum sliding window time, milli-second
# minSlidingTime 10# minimum time window, milli-second
# minIntervalTime 10# maximum delay before launching a stream compution, milli-second
# maxStreamCompDelay 20000# maximum delay before launching a stream computation for the first time, milli-second
# maxFirstStreamCompDelay 10000# retry delay when a stream computation fails, milli-second
# retryStreamCompDelay 10# the delayed time for launching a stream computation, from 0.1(default, 10% of whole computing time window) to 0.9
# streamCompDelayRatio 0.1# max number of vgroups per db, 0 means configured automatically
# 每个数据库中能够使用的最大vnode个数。
# maxVgroupsPerDb 0# max number of tables per vnode
# 每个vnode中能够创建的最大表个数。默认值:1000000。
# maxTablesPerVnode 1000000# step size of increasing table number in a vnode
# tableIncStepPerVnode 1000# cache block size (Mbyte)
# cache 16# number of cache blocks per vnode
# blocks 6# number of days per DB file
# 一个数据文件存储数据的时间跨度,单位为天,默认值:10。
# days 10# number of days to keep DB file
# 数据库中数据保留的天数,单位为天,默认值:3650。
# keep 3650# minimum rows of records in file block
# 文件块中记录的最小条数,单位为条,默认值:100。
# minRows 100# maximum rows of records in file block
# 文件块中记录的最大条数,单位为条,默认值:4096。
# maxRows 4096# enable/disable compression
# 文件压缩标志位,0:关闭,1:一阶段压缩,2:两阶段压缩。默认值:2。
# comp 2# write ahead log (WAL) level, 0: no wal; 1: write wal, but no fysnc; 2: write wal, and call fsync
# WAL级别。1:写wal, 但不执行fsync; 2:写wal, 而且执行fsync。默认值:1。
# walLevel 1# if walLevel is set to 2, the cycle of fsync being executed, if set to 0, fsync is called right away
# 当wal设置为2时,执行fsync的周期。设置为0,表示每次写入,立即执行fsync。单位为毫秒,默认值:3000。
# fsync 3000# number of replications, for cluster only
# 副本个数,取值范围:1-3。单位为个,默认值:1
replica 3# mqtt hostname
# mqttHostName test.mosquitto.org# mqtt port
# mqttPort 1883# mqtt topic
# mqttTopic /test# the compressed rpc message, option:
# -1 (no compression)
# 0 (all message compressed),
# > 0 (rpc message body which larger than this value will be compressed)
# compressMsgSize -1# max length of an SQL
# 单条SQL语句允许最长限制。默认值:65380字节。
maxSQLLength 1048576# the maximum number of records allowed for super table time sorting
# maxNumOfOrderedRes 100000# system time zone
# 默认值:从系统中动态获取当前的时区设置
# timezone Asia/Shanghai (CST, +0800)# system locale
# 默认值:系统中动态获取,如果自动获取失败,需要用户在配置文件设置或通过API设置
# locale en_US.UTF-8# default system charset
# 默认值:系统中动态获取,如果自动获取失败,需要用户在配置文件设置或通过API设置
# charset UTF-8# max number of connections allowed in dnode
# maxShellConns 5000# max numerber of connections allowed in client
# maxConnections 5000# stop writing logs when the disk size of the log folder is less than this value
# minimalLogDirGB 0.1# stop writing temporary files when the disk size of the log folder is less than this value
# minimalTmpDirGB 0.1# stop writing data when the disk size of the log folder is less than this value
# minimalDataDirGB 0.1# enbale/disable http service
# http 1# enable/disable muqq service
# mqtt 0# enable/disable system monitor
# monitor 1# enable/disable recording the SQL statements via restful interface
# httpEnableRecordSql 0# number of threads used to process http requests
# httpMaxThreads 2# maximum number of rows returned by the restful interface
# restfulRowLimit 10240# The following parameter is used to limit the maximum number of lines in log files.
# max number of rows per log filters
# 单个日志文件允许的最大行数。默认值:10,000,000行。
# numOfLogLines 10000000# time of keeping log files, days
# 日志文件的最长保存时间。大于0时,日志文件会被重命名为taosdlog.xxx,其中xxx为日志文件最后修改的时间戳,单位为秒。默认值:0天。
# logKeepDays 0# enable/disable async log
# asyncLog 1# The following parameters are used for debug purpose only.
# debugFlag 8 bits mask: FILE-SCREEN-UNUSED-HeartBeat-DUMP-TRACE_WARN-ERROR
# 131: output warning and error, 135: output debug, warning and error, 143 : output trace, debug, warning and error to log.
# 199: output debug, warning and error to both screen and file
# 207: output trace, debug, warning and error to both screen and file# debug flag for all log type, take effect when non-zero value
# debugFlag 0# debug flag for meta management messages
# mDebugFlag 135# debug flag for dnode messages
# dDebugFlag 135# debug flag for sync module
# sDebugFlag 135# debug flag for WAL
# wDebugFlag 135# debug flag for SDB
# sdbDebugFlag 135# debug flag for RPC
# rpcDebugFlag 131# debug flag for TAOS TIMER
# tmrDebugFlag 131# debug flag for TDengine client
# cDebugFlag 131# debug flag for JNI
# jniDebugflag 131# debug flag for ODBC
# odbcDebugflag 131# debug flag for storage
# uDebugflag 131# debug flag for http server
# httpDebugFlag 131# debug flag for mqtt
# mqttDebugFlag 131# debug flag for monitor
# monitorDebugFlag 131# debug flag for query
# qDebugflag 131# debug flag for vnode
# vDebugflag 131# debug flag for http server
# tsdbDebugFlag 131# enable/disable recording the SQL in taos client
# tscEnableRecordSql 0# generate core file when service crash
# enableCoreFile 1# maximum display width of binary and nchar fields in the shell. The parts exceeding this limit will be hidden
# maxBinaryDisplayWidth 30
3、在E盘创建文件夹
创建TDengine文件夹,在TDengine文件夹下创建log文件夹和data文件夹
4、启动数据库服务
进入C:\TDengine,双击taosd.exe,等待启动。
5、测试下
双击桌面的Taos Shell,输入 taos 运行
查看用户
taos> show users;
服务端安装完毕
6、开放服务器端口6030
三、客户端window为例
1、安装客户端安装包
TDengine-client-xxx-Windows 数据库连接工具
安装好后,打开目录C:\TDengine\cfg
修改配置文件taos.cfg
firstEp 值修改为服务端的IP和端口
3、测试连接
打开taos.exe,看到这样为连接成功
创建用户
create user admin pass '123456';
查看用户
show users;
创建数据库
CREATE DATABASE iot_devicedatacenter;
修改超级表标签
ALTER STABLE stable_device_valuelog ADD TAG modelid INT;
ALTER STABLE stable_device_valuelog RENAME TAG cid clientid;
修改子表标签
ALTER TABLE tb_device_valuelog_2 SET TAG modelid=0;
统计和
select sum(cast(dvalue as INT)) from tb_device_valuelog_2 where ts>="2023-12-06 00:00:00.000";
四、C#程序调用
代码例子:
using TDengineDriver;
using SqlSugar;
using Newtonsoft.Json;
public void SqlSugarTest()
{
var db = new SqlSugarClient(new ConnectionConfig()
{
DbType =SqlSugar.DbType.TDengine,
ConnectionString = "Host=localhost(服务器IP);Port=6030;Username=root;Password=taosdata;Database=power",
IsAutoCloseConnection = true
});
//建超级表
db.Ado.ExecuteCommand("CREATE STABLE IF NOT EXISTS stable_device_valuelog (ts TIMESTAMP, dkey NCHAR(20), dvalue NCHAR(20)) TAGS (did INT,cid INT)");
//建数据表并写入数据
db.Ado.ExecuteCommand(@" INSERT INTO tb_device_valuelog01 USING stable_device_valuelog TAGS(1, 2) VALUES(NOW,'status','no');");
//读数据
DataTable dt = db.Ado.GetDataTable("select * from tb_device_valuelog01 ");
}