Go语言日志库logrus
1、介绍 logrus
logrus是目前Github上star数量最多的日志包,功能强大、性能高效、高度灵活,还提供了自定义插件的功能。很
多优秀的开源项目,例如:docker、prometheus等都使用了logrus。logrus除了具有日志的基本功能外,还具有
如下特性:
-
支持常用的日志级别,logrus支持如下日志级别:Debug、Info、Warn、Error、Fatal和Panic。
-
可扩展,logrus的hook机制允许使用者通过hook的方式将日志分发到任意地方,例如:本地文件、标准输
出、elasticsearch、logstash、kafka等。
-
支持自定义日志格式,logrus内置了2种格式:JSONFormatter和TextFormatter。除此之外,logrus允许使用
者通过实现Formatter接口,来自定义日志格式。
-
结构化日志记录,logrus的Field机制可以允许使用者自定义日志字段,而不是通过冗长的消息来记录日志。
-
预设日志字段,logrus的Default Fields机制可以给一部分或者全部日志统一添加共同的日志字段,例如给某
次HTTP请求的所有日志添加X-Request-ID字段。
-
Fatal handlers:logrus允许注册一个或多个handler,当发生fatal级别的日志时调用。当我们的程序需要优
雅关闭时,该特性会非常有用。
-
它是一个结构化、插件化的日志记录库。完全兼容 golang 标准库中的日志模块。它还内置了 2 种日志输出格
式 JSONFormatter 和 TextFormatter,来定义输出的日志格式。
github地址:https://github.com/sirupsen/logrus
2、logrus 使用
2.1 安装
$ go get github.com/sirupsen/logrus
2.2 简单示例
最简单的使用logrus的示例如下:
package main
import (
log "github.com/sirupsen/logrus"
)
func main() {
log.WithFields(log.Fields{
"animal": "walrus",
}).Info("A walrus appears")
}
运行输出:
# 输出
time="2023-06-02T11:00:26+08:00" level=info msg="A walrus appears" animal=walrus
2.3 设置日志格式
2.3.1 内置日志格式
logrus内置的formatter有 2 种,logrus.TextFormatter 和 logrus.JSONFormatter。
logrus.JSONFormatter{}, 设置为 json 格式,所有设置选项在 logrus.JSONFormatter。
package main
import (
log "github.com/sirupsen/logrus"
)
func main() {
// 设置为json格式
log.SetFormatter(&log.JSONFormatter{})
log.WithFields(log.Fields{
"animal": "walrus",
}).Info("A walrus appears")
// 设置json里的日期输出格式
log.SetFormatter(&log.JSONFormatter{
TimestampFormat: "2006-01-02 15:04:05",
})
log.WithFields(log.Fields{
"animal": "walrus",
}).Info("A walrus appears")
}
# 输出
{"animal":"walrus","level":"info","msg":"A walrus appears","time":"2023-06-02T11
:44:48+08:00"}
{"animal":"walrus","level":"info","msg":"A walrus appears","time":"2023-06-02 11
:44:48"}
logrus.TextFormatter{},设置为文本格式,所有的设置选项在 logrus.TextFormatter。
package main
import (
log "github.com/sirupsen/logrus"
)
func main() {
log.SetFormatter(&log.TextFormatter{})
log.WithFields(log.Fields{
"animal": "walrus",
}).Info("A walrus appears")
log.SetFormatter(&log.TextFormatter{
TimestampFormat: "2006-01-02 15:04:05",
})
log.WithFields(log.Fields{
"animal": "walrus",
}).Info("A walrus appears")
}
# 输出
time="2023-06-02T11:47:46+08:00" level=info msg="A walrus appears" animal=walrus
time="2023-06-02 11:47:46" level=info msg="A walrus appears" animal=walrus
2.3.2 自定义日志格式
可以根据 Formatter 接口自定义日志格式,里面有一个 Format 方法,这个 Format 方法里有一个struct类型数据
*Entry, Entry.Data 是所有字段集合,Fields 类型为 map[string]interface{}。
// The Formatter interface is used to implement a custom Formatter. It takes an
// `Entry`. It exposes all the fields, including the default ones:
//
// * `entry.Data["msg"]`. The message passed from Info, Warn, Error ..
// * `entry.Data["time"]`. The timestamp.
// * `entry.Data["level"]. The level the entry was logged at.
//
// Any additional fields added with `WithField` or `WithFields` are also in
// `entry.Data`. Format is expected to return an array of bytes which are then
// logged to `logger.Out`.
type Formatter interface {
Format(*Entry) ([]byte, error)
}
// An entry is the final or intermediate Logrus logging entry. It contains all
// the fields passed with WithField{,s}. It's finally logged when Trace, Debug,
// Info, Warn, Error, Fatal or Panic is called on it. These objects can be
// reused and passed around as much as you wish to avoid field duplication.
type Entry struct {
Logger *Logger
// Contains all the fields set by the user.
Data Fields
// Time at which the log entry was created
Time time.Time
// Level the log entry was logged at: Trace, Debug, Info, Warn, Error, Fatal or Panic
// This field will be set on entry firing and the value will be equal to the one in Logger struct field.
Level Level
// Calling method, with package name
Caller *runtime.Frame
// Message passed to Trace, Debug, Info, Warn, Error, Fatal or Panic
Message string
// When formatter is called in entry.log(), a Buffer may be set to entry
Buffer *bytes.Buffer
// Contains the context set by the user. Useful for hook processing etc.
Context context.Context
// err may contain a field formatting error
err string
}
// Fields type, used to pass to `WithFields`.
type Fields map[string]interface{}
通过实现接口logrus.Formatter可以实现自己的格式。
type Formatter interface {
Format(*Entry) ([]byte, error)
}
例子:
package main
import (
"fmt"
jsoniter "github.com/json-iterator/go"
log "github.com/sirupsen/logrus"
)
type MyJSONFormatter struct {
JSONPrefix string
Otherdata string
}
func (my *MyJSONFormatter) Format(entry *log.Entry) ([]byte, error) {
entry.Data["msg"] = fmt.Sprintf("%s%s%s", my.JSONPrefix, my.Otherdata, entry.Message)
json, err := jsoniter.Marshal(&entry.Data)
if err != nil {
return nil, fmt.Errorf("failed to marshal fields to JSON , %w", err)
}
return append(json, '\n'), nil
}
func main() {
formatter := &MyJSONFormatter{
JSONPrefix: "jsonprefix-",
Otherdata: "otherdata:",
}
log.SetFormatter(formatter)
log.Info("this is customered formatter")
}
# 输出
{"msg":"jsonprefix-otherdata:this is customered formatter"}
2.3.3 第三方自定义formatter设置日志格式
-
FluentdFormatte
:Formats entries that can be parsed by Kubernetes and Google Container Engine. -
logstas
:Logs fields as Logstash Events。 -
caption-json-formatte
:logrus’s message json formatter with human-readable caption added。 -
powerful-logrus-formatte
:get fileName, log’s line number and the latest function’s name when printlog; Sava log to files。
-
nested-logrus-formatter
:Human-readable log formatter, converts logrus fields to a nestedstructure。
我们这里介绍 nested-logrus-formatter。
安装:
$ go get github.com/antonfisher/nested-logrus-formatter
package main
import (
nested "github.com/antonfisher/nested-logrus-formatter"
"github.com/sirupsen/logrus"
)
func main() {
logrus.SetFormatter(&nested.Formatter{
HideKeys: true,
FieldsOrder: []string{"component", "category"},
})
logrus.Info("info msg")
}
# 输出
Jun 2 16:59:55.116 [INFO] info msg
nested
格式提供了多个字段用来定制行为:
// Formatter - logrus formatter, implements logrus.Formatter
type Formatter struct {
// FieldsOrder - default: fields sorted alphabetically
FieldsOrder []string
// TimestampFormat - default: time.StampMilli = "Jan _2 15:04:05.000"
TimestampFormat string
// HideKeys - show [fieldValue] instead of [fieldKey:fieldValue]
HideKeys bool
// NoColors - disable colors
NoColors bool
// NoFieldsColors - apply colors only to the level, default is level + fields
NoFieldsColors bool
// NoFieldsSpace - no space between fields
NoFieldsSpace bool
// ShowFullLevel - show a full level [WARNING] instead of [WARN]
ShowFullLevel bool
// NoUppercaseLevel - no upper case for level value
NoUppercaseLevel bool
// TrimMessages - trim whitespaces on messages
TrimMessages bool
// CallerFirst - print caller info first
CallerFirst bool
// CustomCallerFormatter - set custom formatter for caller info
CustomCallerFormatter func(*runtime.Frame) string
}
-
默认,logrus 输出日志中字段是 key=value 这样的形式。使用 nested 格式,我们可以通过设置 HideKeys 为
true 隐藏键,只输出值;
-
默认,logrus 是按键的字母序输出字段,可以设置 FieldsOrder 定义输出字段顺序;
-
通过设置 TimestampFormat 设置日期格式。
package main
import (
nested "github.com/antonfisher/nested-logrus-formatter"
"github.com/sirupsen/logrus"
"time"
)
func main() {
logrus.SetFormatter(&nested.Formatter{
// HideKeys: true,
TimestampFormat: time.RFC3339,
FieldsOrder: []string{"name", "age"},
})
logrus.WithFields(logrus.Fields{
"name": "dj",
"age": 18,
}).Info("info msg")
}
# 输出
2023-06-02T17:36:03+08:00 [INFO] [name:dj] [age:18] info msg
注意到,我们将时间格式设置成time.RFC3339
,即2006-01-02T15:04:05Z07:00
这种形式。
2.4 设置日志级别
logrus 的使用非常简单,与标准库 log 类似,logrus 支持更多的日志级别。
logrus日志一共7级别,从高到低:panic,fatal,error,warn,info,debug,trace。
-
Panic
:记录日志,然后panic
。 -
Fatal
:致命错误,出现错误时程序无法正常运转。输出日志后,程序退出; -
Error
:错误日志,需要查看原因; -
Warn
:警告信息,提醒程序员注意; -
Info
:关键操作,核心流程的日志; -
Debug
:一般程序中输出的调试信息; -
Trace
:很细粒度的信息,一般用不到;
log.SetLevel(log.WarnLevel):设置输出警告级别。
package main
import (
"github.com/sirupsen/logrus"
)
func main() {
logrus.SetLevel(logrus.TraceLevel)
logrus.Trace("trace msg")
logrus.Debug("debug msg")
logrus.Info("info msg")
logrus.Warn("warn msg")
logrus.Error("error msg")
logrus.Fatal("fatal msg")
logrus.Panic("panic msg")
}
# 输出
# 另外,我们观察到输出中有三个关键信息,time、level和msg
# time:输出日志的时间
# level:日志级别
# msg:日志信息
time="2023-06-02T11:18:54+08:00" level=trace msg="trace msg"
time="2023-06-02T11:18:54+08:00" level=debug msg="debug msg"
time="2023-06-02T11:18:54+08:00" level=info msg="info msg"
time="2023-06-02T11:18:54+08:00" level=warning msg="warn msg"
time="2023-06-02T11:18:54+08:00" level=error msg="error msg"
time="2023-06-02T11:18:54+08:00" level=fatal msg="fatal msg"
logrus
有一个日志级别,高于这个级别的日志不会输出。默认的级别为InfoLevel
。所以为了能看到Trace
和
Debug
日志,我们在main
函数第一行设置日志级别为TraceLevel
。
由于logrus.Fatal
会导致程序退出,下面的logrus.Panic
不会执行到。
2.5 设置日志输出方式
log.SetOutput(os.Stdout):输出到 Stdout,默认输出到 Stderr。
输出到文件里:
logfile, _ := os.OpenFile(“./logrus.log”, os.O_CREATE|os.O_RDWR|os.O_APPEND, 0644)
logrus.SetOutput(logfile)
package main
import (
log "github.com/sirupsen/logrus"
"os"
)
func init() {
// 设置format json
log.SetFormatter(&log.JSONFormatter{})
// 设置输出警告级别
log.SetLevel(log.InfoLevel)
// 输出到stdout而不是默认的stderr
log.SetOutput(os.Stdout)
}
func main() {
log.WithFields(log.Fields{
"animal": "dog",
"size": 10,
}).Info("a group of dog emerges from the zoon")
log.WithFields(log.Fields{
"omg": true,
"number": 12,
}).Warn("the group's number increased")
// 从WithFields()返回的logrus.Entry
contextLogger := log.WithFields(log.Fields{
"common": "this is a common filed",
"other": "i also should be logged always",
})
// 共同字段输出
contextLogger.Info("I'll be logged with common and other field")
contextLogger.Info("Me too")
}
运行输出:
# 输出
{"animal":"dog","level":"info","msg":"a group of dog emerges from the zoon","siz
e":10,"time":"2023-06-02T14:11:27+08:00"}
{"level":"warning","msg":"the group's number increased","number":12,"omg":true,"
time":"2023-06-02T14:11:27+08:00"}
{"common":"this is a common filed","level":"info","msg":"I'll be logged with com
mon and other field","other":"i also should be logged always","time":"2023-06-02
T14:11:27+08:00"}
{"common":"this is a common filed","level":"info","msg":"Me too","other":"i also
should be logged always","time":"2023-06-02T14:11:27+08:00"}
// 输出到文件
package main
import (
log "github.com/sirupsen/logrus"
"os"
)
func init() {
// 设置format json
log.SetFormatter(&log.JSONFormatter{})
// 设置输出警告级别
log.SetLevel(log.InfoLevel)
logfile, _ := os.OpenFile("./logrus.log", os.O_CREATE|os.O_RDWR|os.O_APPEND, 0644)
log.SetOutput(logfile)
}
func main() {
log.WithFields(log.Fields{
"animal": "dog",
"size": 10,
}).Info("a group of dog emerges from the zoon")
log.WithFields(log.Fields{
"omg": true,
"number": 12,
}).Warn("the group's number increased")
// 从WithFields()返回的logrus.Entry
contextLogger := log.WithFields(log.Fields{
"common": "this is a common filed",
"other": "i also should be logged always",
})
// 共同字段输出
contextLogger.Info("I'll be logged with common and other field")
contextLogger.Info("Me too")
}
# logrus.log文件的内容
{"animal":"dog","level":"info","msg":"a group of dog emerges from the zoon","size":10,"time":"2023-06-02T14:20:22+08:00"}
{"level":"warning","msg":"the group's number increased","number":12,"omg":true,"time":"2023-06-02T14:20:22+08:00"}
{"common":"this is a common filed","level":"info","msg":"I'll be logged with common and other field","other":"i also should be logged always","time":"2023-06-02T14:20:22+08:00"}
{"common":"this is a common filed","level":"info","msg":"Me too","other":"i also should be logged always","time":"2023-06-02T14:20:22+08:00"}
一次可以输出到多种介质中:
package main
import (
"bytes"
"io"
"log"
"os"
"github.com/sirupsen/logrus"
)
func main() {
writer1 := &bytes.Buffer{}
writer2 := os.Stdout
writer3, err := os.OpenFile("log.txt", os.O_WRONLY|os.O_CREATE, 0755)
if err != nil {
log.Fatalf("create file log.txt failed: %v", err)
}
logrus.SetOutput(io.MultiWriter(writer1, writer2, writer3))
logrus.Info("info msg")
}
# 输出
time="2023-06-02T16:46:18+08:00" level=info msg="info msg"
2.6 logrus 的 Fatal 处理
定义了输出 Fatal 日志后,其后的日志都不能输出了,这是为什么?日志后面有个信息 exit status
。
func (logger *Logger) Fatal(args ...interface{}) {
logger.Log(FatalLevel, args...)
logger.Exit(1)
}
因为 logrus 的 Fatal 输出后,会执行 os.Exit(1)。那如果程序后面还有一些必要的程序要处理怎么办?
logrus 提供了 RegisterExitHandler 方法,在 fatal 异常时处理一些问题。
package main
import (
"fmt"
log "github.com/sirupsen/logrus"
)
func main() {
log.SetFormatter(&log.TextFormatter{
TimestampFormat: "2006-01-02 15:04:05",
})
log.RegisterExitHandler(func() {
fmt.Println("发生了fatal异常,执行一些必要的处理工作")
})
log.Warn("warn")
log.Fatal("fatal")
// 不会执行
log.Info("info")
}
运行输出:
# 输出
发生了fatal异常,执行一些必要的处理工作
time="2023-06-02 14:23:19" level=warning msg=warn
time="2023-06-02 14:23:19" level=fatal msg=fatal
和很多日志框架一样,logrus的Fatal系列函数会执行os.Exit(1)。但是logrus提供可以注册一个或多个fatal
handler函数的接口logrus.RegisterExitHandler(handler func() {} ),让logrus在执行os.Exit(1)之前进行相应的处
理。fatal handler可以在系统异常时调用一些资源释放api等,让应用正确的关闭。
2.7 切分日志文件
如果日志文件太大了,想切分成小文件,但是 logrus 没有提供这个功能。
一种是借助linux系统的 logrotate 命令来切分 logrus 生成的日志文件。
另外一种是用 logrus 的 hook 功能,做一个切分日志的插件。找到了 file-rotatelogs,但是这个库状态
已经是 archived 状态,库作者现在不接受任何修改,他也不继续维护了。所以使用还是慎重些。
在 logrus issue 里找到了这个 https://github.com/natefinch/lumberjack
切割文件的库。
package main
import (
log "github.com/sirupsen/logrus"
"gopkg.in/natefinch/lumberjack.v2"
)
func main() {
logger := &lumberjack.Logger{
Filename: "./testlogrus.log",
MaxSize: 500, // 日志文件大小,单位是 MB
MaxBackups: 3, // 最大过期日志保留个数
MaxAge: 28, // 保留过期文件最大时间,单位 天
Compress: true, // 是否压缩日志,默认是不压缩,这里设置为true,压缩日志
}
log.SetOutput(logger) // logrus设置日志的输出方式
log.SetLevel(log.TraceLevel)
log.Trace("trace msg")
log.Debug("debug msg")
log.Info("info msg")
log.Warn("warn msg")
log.Error("error msg")
log.Fatal("fatal msg")
log.Panic("panic msg")
}
# 日志文件的输出
time="2023-06-02T14:32:00+08:00" level=trace msg="trace msg"
time="2023-06-02T14:32:00+08:00" level=debug msg="debug msg"
time="2023-06-02T14:32:00+08:00" level=info msg="info msg"
time="2023-06-02T14:32:00+08:00" level=warning msg="warn msg"
time="2023-06-02T14:32:00+08:00" level=error msg="error msg"
time="2023-06-02T14:32:00+08:00" level=fatal msg="fatal msg"
将不同等级的输出保存到不同的文件中:
package main
import (
"github.com/lestrrat-go/file-rotatelogs"
"github.com/rifflock/lfshook"
log "github.com/sirupsen/logrus"
"time"
)
var infoLogName = "info"
var errorLogName = "error"
func newLfsHook(maxRemainCnt uint) log.Hook {
infoWriter, infoErr := rotatelogs.New(
infoLogName+".%Y%m%d%H",
// WithLinkName为最新的日志建立软连接,以方便随着找到当前日志文件
// rotatelogs.WithLinkName(infoLogName),
// WithRotationTime设置日志分割的时间,这里设置为一小时分割一次
rotatelogs.WithRotationTime(time.Hour),
// WithMaxAge和WithRotationCount二者只能设置一个
// WithMaxAge设置文件清理前的最长保存时间
// WithRotationCount设置文件清理前最多保存的个数
// rotatelogs.WithMaxAge(time.Hour*24),
rotatelogs.WithRotationCount(maxRemainCnt),
)
if infoErr != nil {
log.Errorf("config local file system for info logger error: %v", infoErr)
}
errorWriter, errorErr := rotatelogs.New(
errorLogName+".%Y%m%d%H",
// WithLinkName为最新的日志建立软连接,以方便随着找到当前日志文件
// rotatelogs.WithLinkName(errorLogName),
// WithRotationTime设置日志分割的时间,这里设置为一小时分割一次
rotatelogs.WithRotationTime(time.Hour),
// WithMaxAge和WithRotationCount二者只能设置一个
// WithMaxAge设置文件清理前的最长保存时间
// WithRotationCount设置文件清理前最多保存的个数
// rotatelogs.WithMaxAge(time.Hour*24),
rotatelogs.WithRotationCount(maxRemainCnt),
)
if errorErr != nil {
log.Errorf("config local file system for error logger error: %v", errorErr)
}
lfsHook := lfshook.NewHook(lfshook.WriterMap{
log.InfoLevel: infoWriter,
log.ErrorLevel: errorWriter,
}, &log.TextFormatter{DisableColors: true})
return lfsHook
}
func main(){
log.SetLevel(log.InfoLevel)
log.AddHook(newLfsHook(10))
log.Info("info!")
log.Error("error!")
}
# 输出
time="2023-06-02T16:07:54+08:00" level=info msg="info!"
time="2023-06-02T16:07:54+08:00" level=error msg="error!"
生成了 info.2023060216
和 error.2023060216
:
# info.2023060216
time="2023-06-02T16:07:54+08:00" level=info msg="info!"
# error.2023060216
time="2023-06-02T16:07:54+08:00" level=error msg="error!"
2.8 设置logrus实例
如果一个应用有多个地方使用日志,可以单独实例化一个 logrus,作为全局的日志实例。
logger是一种相对高级的用法, 对于一个大型项目, 往往需要一个全局的logrus实例,即 logger 对象来记录项目所
有的日志。
package main
import (
"github.com/sirupsen/logrus"
"os"
)
var ll = logrus.New()
func main() {
// 设置输出日志位置,可以设置日志到file里
ll.Out = os.Stdout
// 可以设置输出到文件
// logfile, _ := os.OpenFile("./logrus.log", os.O_CREATE|os.O_RDWR|os.O_APPEND, 0644)
// log.Out = logfile
ll.Formatter = &logrus.JSONFormatter{}
ll.WithFields(logrus.Fields{
"fruit": "apple",
"size": 20,
}).Info(" a lot of apples on the tree")
}
输出:
{"fruit":"apple","level":"info","msg":" a lot of apples on the tree","size":20,"
time":"2023-06-02T14:37:53+08:00"}
2.9 fields
在使用 logrus 时,鼓励用 log.WithFields(log.Fields{}).Fatal()
这种方式替代:
log.Fatalf("Failed to send event %s to topic %s with key %d", event, topic, key)
也就是不用 %s,%d 这种方式格式化,而是直接传入变量 event,topic 给 log.Fields ,这样就显得结构化日志输
出,很人性化美观。
logrus不推荐使用冗长的消息来记录运行信息,它推荐使用 Fields 来进行精细化的、结构化的信息记录。
package main
import (
log "github.com/sirupsen/logrus"
)
func main() {
log.WithFields(log.Fields{
"event": "event",
"topic": "topic",
"key": "key",
}).Info("Failed to send event")
}
# 输出
time="2023-06-02T14:52:59+08:00" level=info msg="Failed to send event" event=event key=key topic=topic
2.10 设置默认字段
比如在链路追踪里,会有一个 rquest_id ,trace_id 等,想这个 log 一直带有这 2 个字段,logrus 怎么设置?
可以用 log.WithFields(log.Fields{“request_id”: request_id, “trace_id”: trace_id})
requestLogger := log.WithFields(log.Fields{"request_id": request_id, "trace_id": trace_id})
requestLogger.Info("something happened on that request")
requestLogger.Warn("something not great happened")
例子:
package main
import (
"github.com/google/uuid"
log "github.com/sirupsen/logrus"
)
func main() {
uid := uuid.New()
request_id := uid
trace_id := uid
requestLogger := log.WithFields(log.Fields{"request_id": request_id, "trace_id": trace_id})
requestLogger.Info("something happened on that request")
requestLogger.Warn("something not great happened")
}
# 输出
time="2023-06-02T14:54:56+08:00" level=info msg="something happened on that requ
est" request_id=e16b8b4d-67e9-4600-875b-8fba6e889756 trace_id=e16b8b4d-67e9-4600
-875b-8fba6e889756
time="2023-06-02T14:54:56+08:00" level=warning msg="something not great happened
" request_id=e16b8b4d-67e9-4600-875b-8fba6e889756 trace_id=e16b8b4d-67e9-4600-87
5b-8fba6e889756
2.11 hook钩子-扩展logrus功能
hook 给 logrus 提供了强大的可扩展功能。
logrus最令人心动的功能就是其可扩展的HOOK机制了,通过在初始化时为logrus添加hook,logrus可以实现各种
扩展功能。
用户可以给 logrus 编写钩子插件,根据自己的日志需求编写 hook。
logrus 也有一些内置插件 hooks。
grus官方仅仅内置了syslog的 hook:
https://github.com/sirupsen/logrus/tree/master/hooks/syslog
此外,但Github也有很多第三方的hook可供使用。我们可以使用一些 Hook 将日志发送到 redis/mongodb 等存
储中:
-
mgorus:将日志发送到 mongodb:
https://github.com/weekface/mgorus
-
logrus-redis-hook:将日志发送到 redis:
https://github.com/rogierlommers/logrus-redis-hook
-
logrus-amqp:将日志发送到 ActiveMQ:
https://github.com/vladoatanasov/logrus_amqp
-
logrus_influxdb:发送到 InfluxDB:
https://github.com/abramovic/logrus_influxdb
-
logrus-logstash-hook:发送到 Logstash:
https://github.com/bshuster-repo/logrus-logstash-hook
-
elastic:发送到 elastcisearch:
https://github.com/olivere/elastic
https://pkg.go.dev/gopkg.in/olivere/elastic.v5
下面我们自定义 hook。
Hook接口:
logrus的hook接口定义如下,其原理是每此写入日志时拦截,修改logrus.Entry。
Levels()
方法返回感兴趣的日志级别,输出其他日志时不会触发钩子。Fire
是日志输出前调用的钩子方法。
// logrus在记录Levels()返回的日志级别的消息时会触发HOOK,
// 按照Fire方法定义的内容修改logrus.Entry。
type Hook interface {
Levels() []Level
Fire(*Entry) error
}
一个简单自定义hook如下,DefaultFieldHook
定义会在所有级别的日志消息中加入默认字段
appName="myAppName"
。
type DefaultFieldHook struct {
}
func (hook *DefaultFieldHook) Fire(entry *log.Entry) error {
entry.Data["appName"] = "MyAppName"
return nil
}
func (hook *DefaultFieldHook) Levels() []log.Level {
return log.AllLevels
}
hook的使用也很简单,在初始化前调用log.AddHook(hook)
添加相应的hook
即可。
完整案例:
package main
import (
"github.com/sirupsen/logrus"
)
type DefaultFieldHook struct {
}
func (hook *DefaultFieldHook) Fire(entry *logrus.Entry) error {
entry.Data["appName"] = "MyAppName"
return nil
}
func (hook *DefaultFieldHook) Levels() []logrus.Level {
return logrus.AllLevels
}
func main() {
logrus.SetLevel(logrus.TraceLevel)
logrus.AddHook(&DefaultFieldHook{})
logrus.Trace("trace msg")
logrus.Debug("debug msg")
logrus.Info("info msg")
logrus.Warn("warn msg")
logrus.Error("error msg")
logrus.Fatal("fatal msg")
logrus.Panic("panic msg")
}
# 输出
time="2023-06-02T15:27:36+08:00" level=trace msg="trace msg" appName=MyAppName
time="2023-06-02T15:27:36+08:00" level=debug msg="debug msg" appName=MyAppName
time="2023-06-02T15:27:36+08:00" level=info msg="info msg" appName=MyAppName
time="2023-06-02T15:27:36+08:00" level=warning msg="warn msg" appName=MyAppName
time="2023-06-02T15:27:36+08:00" level=error msg="error msg" appName=MyAppName
time="2023-06-02T15:27:36+08:00" level=fatal msg="fatal msg" appName=MyAppName
2.11.1 发送至redis
安装:
$ go get github.com/rogierlommers/logrus-redis-hook
例子:
package main
import (
logredis "github.com/rogierlommers/logrus-redis-hook"
"github.com/sirupsen/logrus"
"io/ioutil"
)
func init() {
hookConfig := logredis.HookConfig{
Host: "localhost",
Key: "mykey",
Format: "v0",
App: "aweosome",
Hostname: "localhost",
TTL: 3600,
Port: 6379,
}
hook, err := logredis.NewHook(hookConfig)
if err == nil {
logrus.AddHook(hook)
} else {
logrus.Errorf("logredis error: %q", err)
}
}
func main() {
logrus.Info("just some info logging...")
logrus.WithFields(logrus.Fields{
"animal": "walrus",
"foo": "bar",
"this": "that",
}).Info("additional fields are being logged as well")
logrus.SetOutput(ioutil.Discard)
logrus.Info("This will only be sent to Redis")
}
为了程序能正常工作,我们还需要安装redis
。
直接输入redis-server
,启动服务器。
运行程序后,我们使用redis-cli
查看。
我们看到mykey
是一个list
,每过来一条日志,就在list
后新增一项。
# 输出结果
time="2023-06-05T21:14:08+08:00" level=info msg="just some info logging..."
time="2023-06-05T21:14:08+08:00" level=info msg="additional fields are being log
ged as well" animal=walrus foo=bar this=that
redis的结果:
2.11.2 发送至elasticsearch
package main
import (
"github.com/olivere/elastic/v7"
"github.com/sirupsen/logrus"
"gopkg.in/sohlich/elogrus.v7"
)
func main() {
log := logrus.New()
client, err := elastic.NewClient(elastic.SetURL("http://localhost:9200"))
if err != nil {
log.Panic(err)
}
hook, err := elogrus.NewAsyncElasticHook(client, "localhost", logrus.DebugLevel, "mylog")
if err != nil {
log.Panic(err)
}
log.Hooks.Add(hook)
log.WithFields(logrus.Fields{
"name": "joe",
"age": 42,
}).Error("Hello world!")
}
# 输出
time="2023-06-05T21:23:41+08:00" level=error msg="Hello world!" age=42 name=joe
查看elasticsearch:
$ curl http://localhost:9200/mylog/_search
{
"took": 50,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.0,
"hits": [{
"_index": "mylog",
"_type": "log",
"_id": "rNO6i4gB1Ed8BTkaSoga",
"_score": 1.0,
"_source": {
"Host": "localhost",
"@timestamp": "2023-06-05T13:23:41.7171093Z",
"Message": "Hello world!",
"Data": {
"age": 42,
"name": "joe"
},
"Level": "ERROR"
}
}]
}
}
2.12 记录文件名和行号
调用logrus.SetReportCaller(true)
设置在输出日志中添加文件名和方法信息。
package main
import (
"github.com/sirupsen/logrus"
)
func main() {
logrus.SetReportCaller(true)
logrus.Info("info msg")
}
# 输出
time="2023-06-02T15:30:02+08:00" level=info msg="info msg" func=main.main file="
C:/Users/admin/Desktop/tt/go-logrus/015.go:9"
package main
import (
"fmt"
log "github.com/sirupsen/logrus"
"runtime"
"strings"
)
// line number hook for log the call context,
type lineHook struct {
Field string
// skip为遍历调用栈开始的索引位置
Skip int
levels []log.Level
}
// Levels implement levels
func (hook lineHook) Levels() []log.Level {
return log.AllLevels
}
// Fire implement fire
func (hook lineHook) Fire(entry *log.Entry) error {
entry.Data[hook.Field] = findCaller(hook.Skip)
return nil
}
func findCaller(skip int) string {
file := ""
line := 0
var pc uintptr
// 遍历调用栈的最大索引为第11层.
for i := 0; i < 11; i++ {
file, line, pc = getCaller(skip + i)
// 过滤掉所有logrus包,即可得到生成代码信息
if !strings.HasPrefix(file, "logrus") {
break
}
}
fullFnName := runtime.FuncForPC(pc)
fnName := ""
if fullFnName != nil {
fnNameStr := fullFnName.Name()
// 取得函数名
parts := strings.Split(fnNameStr, ".")
fnName = parts[len(parts)-1]
}
return fmt.Sprintf("%s:%d:%s()", file, line, fnName)
}
func getCaller(skip int) (string, int, uintptr) {
pc, file, line, ok := runtime.Caller(skip)
if !ok {
return "", 0, pc
}
n := 0
// 获取包名
for i := len(file) - 1; i > 0; i-- {
if file[i] == '/' {
n++
if n >= 2 {
file = file[i+1:]
break
}
}
}
return file, line, pc
}
func main() {
log.SetLevel(log.TraceLevel)
log.AddHook(&lineHook{})
log.Trace("trace msg")
log.Debug("debug msg")
log.Info("info msg")
log.Warn("warn msg")
log.Error("error msg")
log.Fatal("fatal msg")
log.Panic("panic msg")
}
# 输出
time="2023-06-02T15:43:42+08:00" level=trace msg="trace msg" ="go-logrus/016.go:
56:getCaller()"
time="2023-06-02T15:43:42+08:00" level=debug msg="debug msg" ="go-logrus/016.go:
56:getCaller()"
time="2023-06-02T15:43:42+08:00" level=info msg="info msg" ="go-logrus/016.go:56
:getCaller()"
time="2023-06-02T15:43:42+08:00" level=warning msg="warn msg" ="go-logrus/016.go
:56:getCaller()"
time="2023-06-02T15:43:42+08:00" level=error msg="error msg" ="go-logrus/016.go:
56:getCaller()"
time="2023-06-02T15:43:42+08:00" level=fatal msg="fatal msg" ="go-logrus/016.go:
56:getCaller()"
2.13 线程安全
默认情况下,logrus的api都是线程安全的,其内部通过互斥锁来保护并发写。互斥锁工作于调用hooks或者写日
志的时候,如果不需要锁,可以调用logger.SetNoLock()来关闭之。可以关闭logrus互斥锁的情形包括:
-
没有设置hook,或者所有的hook都是线程安全的实现。
-
写日志到logger.Out已经是线程安全的了,如logger.Out已经被锁保护,或者写文件时,文件是以O_APPEND
方式打开的,并且每次写操作都小于4k。