Note：文章待完结
文章目录
- 前言
- 一、整体目录结构
- 二、driver包
- 1、驱动相关driver.Driver
- 2、驱动连接：driver.Conn
- 3、预处理结构：Stmt
- 4、执行结果 driver.Result
- 5、查询结果：driver.Rows
- 6、driver.RowsAffected
- 7、driver.Value
- 8、Value定义转换相关
- 三、sql包
- 二、结语
- 三、参考
前言
在golang中，我们比较熟悉的mysql相关的库就是database/sql，这是golang的内置库，该标准库没有具体实现，只列出第三方库需要实现的具体内容。也就是说，这个库只是定义了接口，并没有具体的实现。Go语言为开发数据库驱动定义了一些标准接口，使用标准接口开发的代码，在迁移数据库时，不需要做任何修改（当然双方数据库都遵守标准接口）。下面我将基于golang1.19的源码探究这个库的实现。
源码地址：https://github.com/golang/go/tree/release-branch.go1.19/src/database/sql
一、整体目录结构
整个目录结构就是这样，包含两个包：sql和driver，这两个包必须一起配合着使用，sql包中主要包含着数据库具体实例、驱动的注册、结果集读取、转换各种定义类型结构等。driver包中主要是与数据库打交道的部分，增删改查的接口定义就在这里面。
sql包：
driver包：
二、driver包
在driver包中，主要有如下的接口定义：
Connector：抽象的数据库连接器，需要具备创建数据库连接以及返回从属的数据库驱动的能力。Driver：抽象的数据库驱动，具备创建数据库连接的能力。Conn：抽象的数据库连接，具备预处理 sql 以及开启事务的能力。Tx：抽象的事务，具备提交和回滚的能力。Statement：抽象的请求预处理状态. 具备实际执行 sql 并返回执行结果的能力。Result/Row：抽象的 sql 执行结果。
1、驱动相关driver.Driver
Driver是一个数据库驱动的接口，定义了 Open(name string) ,该方法返回一个数据库的Conn接口:
// Driver is the interface that must be implemented by a database
// driver.
//
// Database drivers may implement DriverContext for access
// to contexts and to parse the name only once for a pool of connections,
// instead of once per connection.
type Driver interface {
// Open returns a new connection to the database.
// The name is a string in a driver-specific format.
//
// Open may return a cached connection (one previously
// closed), but doing so is unnecessary; the sql package
// maintains a pool of idle connections for efficient re-use.
//
// The returned connection is only used by one goroutine at a
// time.
Open(name string) (Conn, error)
}
在上面的源码中，我们可以清晰知道，Driver接口是必须要被所有的数据库驱动程序实现的，提供而一个Open方法用于返回一个连接，这个连接可能是缓存的有效的，也可能是新建的连接。同时也提供了一个DriverContext接口，数据库驱动程序可以实现DriverContext以访问上下文，并仅为连接池解析一次名称，而不是每个连接解析一次。
DriverContext接口提供了一个OpenConnector方法用于返回一个连接器，在连接器中去获取对应的连接。连接器接口Connector提供了两个方法，Connect和Driver，其中Connect用于获取连接，并且可以附带参数ctx，Driver用于获取当前这个连接器的的驱动程序。
// If a Driver implements DriverContext, then sql.DB will call
// OpenConnector to obtain a Connector and then invoke
// that Connector's Connect method to obtain each needed connection,
// instead of invoking the Driver's Open method for each connection.
// The two-step sequence allows drivers to parse the name just once
// and also provides access to per-Conn contexts.
type DriverContext interface {
// OpenConnector must parse the name in the same format that Driver.Open
// parses the name parameter.
OpenConnector(name string) (Connector, error)
}
// A Connector represents a driver in a fixed configuration
// and can create any number of equivalent Conns for use
// by multiple goroutines.
//
// A Connector can be passed to sql.OpenDB, to allow drivers
// to implement their own sql.DB constructors, or returned by
// DriverContext's OpenConnector method, to allow drivers
// access to context and to avoid repeated parsing of driver
// configuration.
//
// If a Connector implements io.Closer, the sql package's DB.Close
// method will call Close and return error (if any).
type Connector interface {
// Connect returns a connection to the database.
// Connect may return a cached connection (one previously
// closed), but doing so is unnecessary; the sql package
// maintains a pool of idle connections for efficient re-use.
//
// The provided context.Context is for dialing purposes only
// (see net.DialContext) and should not be stored or used for
// other purposes. A default timeout should still be used
// when dialing as a connection pool may call Connect
// asynchronously to any query.
//
// The returned connection is only used by one goroutine at a
// time.
Connect(context.Context) (Conn, error)
// Driver returns the underlying Driver of the Connector,
// mainly to maintain compatibility with the Driver method
// on sql.DB.
Driver() Driver
}
接下来就是驱动的注册了，database/sql提供了一个驱动注册静态方法，驱动的具体实现中，可以调用该方法注册相关的驱动，但同时只允许注册同一类型的驱动，否则会panic。
func Register(name string, driver driver.Driver) {
driversMu.Lock()
defer driversMu.Unlock()
if driver == nil {
panic("sql: Register driver is nil")
}
if _, dup := drivers[name]; dup {
panic("sql: Register called twice for driver " + name)
}
drivers[name] = driver
}
该函数用来注册数据库驱动。当第三方开发者开发数据库驱动时，都会实现init函数，而init中会调用 Register(name string, driver driver.Driver) 完成驱动注册。如 github.com/go-sql-driver/mysql
2、驱动连接：driver.Conn
type Conn interface {
// Prepare returns a prepared statement, bound to this connection.
Prepare(query string) (Stmt, error)
// Close invalidates and potentially stops any current
// prepared statements and transactions, marking this
// connection as no longer in use.
//
// Because the sql package maintains a free pool of
// connections and only calls Close when there's a surplus of
// idle connections, it shouldn't be necessary for drivers to
// do their own connection caching.
//
// Drivers must ensure all network calls made by Close
// do not block indefinitely (e.g. apply a timeout).
Close() error
// Begin starts and returns a new transaction.
//
// Deprecated: Drivers should implement ConnBeginTx instead (or additionally).
Begin() (Tx, error)
}
Prepare：返回与当前连接相关的执行SQL语句的准备状态（Stmt），可以进行查询、删除等操作。
Close：关闭当前的链接，执行释放连接拥有的资源等清理工作。
Begin: // 返回一个代表事务处理的Tx,通过它可以进行查询、更新等操作，或者对事务进行回滚、递交。
新版本中，Begin方法已经不推荐了，被ConnBeginTx代替了, 新版本中的Begin方法多提供了入参ctx和额外的可选参数opts，便于扩展和控制。
// ConnBeginTx enhances the Conn interface with context and TxOptions.
type ConnBeginTx interface {
// BeginTx starts and returns a new transaction.
// If the context is canceled by the user the sql package will
// call Tx.Rollback before discarding and closing the connection.
//
// This must check opts.Isolation to determine if there is a set
// isolation level. If the driver does not support a non-default
// level and one is set or if there is a non-default isolation level
// that is not supported, an error must be returned.
//
// This must also check opts.ReadOnly to determine if the read-only
// value is true to either set the read-only transaction property if supported
// or return an error if it is not supported.
BeginTx(ctx context.Context, opts TxOptions) (Tx, error)
}
3、预处理结构：Stmt
// Stmt is a prepared statement. It is bound to a Conn and not
// used by multiple goroutines concurrently.
type Stmt interface {
// Close closes the statement.
//
// As of Go 1.1, a Stmt will not be closed if it's in use
// by any queries.
//
// Drivers must ensure all network calls made by Close
// do not block indefinitely (e.g. apply a timeout).
Close() error
// NumInput returns the number of placeholder parameters.
//
// If NumInput returns >= 0, the sql package will sanity check
// argument counts from callers and return errors to the caller
// before the statement's Exec or Query methods are called.
//
// NumInput may also return -1, if the driver doesn't know
// its number of placeholders. In that case, the sql package
// will not sanity check Exec or Query argument counts.
NumInput() int
// Exec executes a query that doesn't return rows, such
// as an INSERT or UPDATE.
//
// Deprecated: Drivers should implement StmtExecContext instead (or additionally).
Exec(args []Value) (Result, error)
// Query executes a query that may return rows, such as a
// SELECT.
//
// Deprecated: Drivers should implement StmtQueryContext instead (or additionally).
Query(args []Value) (Rows, error)
}
// StmtExecContext enhances the Stmt interface by providing Exec with context.
type StmtExecContext interface {
// ExecContext executes a query that doesn't return rows, such
// as an INSERT or UPDATE.
//
// ExecContext must honor the context timeout and return when it is canceled.
ExecContext(ctx context.Context, args []NamedValue) (Result, error)
}
// StmtQueryContext enhances the Stmt interface by providing Query with context.
type StmtQueryContext interface {
// QueryContext executes a query that may return rows, such as a
// SELECT.
//
// QueryContext must honor the context timeout and return when it is canceled.
QueryContext(ctx context.Context, args []NamedValue) (Rows, error)
}
Close：关闭当前的连接状态，但如果当前正在执行query,query还是会有效返回rows数据。
NumInput：返回当前预留参数的个数，当返回>=0时，数据库驱动会智能检查调用者的参数。 当数据库驱动包不知道预留参数的时候，返回-1。
Exec：执行Prepare准备好的SQL,传入参数执行Update/Insert等操作，返回Result数据，Result中包含最后插入的自增主键序号（LastInsertId）和受影响的行数（RowAffected）。
Query：执行Prepare准备好的SQL,传入需要的参数执行select操作，返回Rows结果集。
4、执行结果 driver.Result
// Result is the result of a query execution.
type Result interface {
// LastInsertId returns the database's auto-generated ID
// after, for example, an INSERT into a table with primary
// key.
LastInsertId() (int64, error)
// RowsAffected returns the number of rows affected by the
// query.
RowsAffected() (int64, error)
}
5、查询结果：driver.Rows
// Rows is an iterator over an executed query's results.
type Rows interface {
// 该函数返回查询数据库表的字段信息，这个返回的slice和SQL查询的字段一一对应，
// 而不是返回整张表的所有字段。
Columns() []string
// 用来关闭Rows迭代器
Close() error
// 该函数用来返回下一条数据，把数据赋值给dest .
// dest里面元素必须是driver.Value的值（string除外）,返回的数据里面所有的 string 都必须转换成
// []byte.如果最后没有数据了，Next 函数返回 io.EOF。
Next(dest []Value) error
}
可以看到，在新版的源码中，Exec和Query已经被单独拎出去定义了接口，方法中只是为了增加ctx参数，这也是golang为了保持向下兼容而做的，试想，如果直接在原有的接口定义的加入ctx，升级golang版本的时候这块儿肯定得花很大功夫去改造。
6、driver.RowsAffected
RowsAffected 不是别的东西，实际上只是 int64 的别名，但它实现了Result接口，用于底层实现 Result 的表示方式，构建Exec方法返回的结果集。
// RowsAffected implements Result for an INSERT or UPDATE operation
// which mutates a number of rows.
type RowsAffected int64
var _ Result = RowsAffected(0)
func (RowsAffected) LastInsertId() (int64, error) {
return 0, errors.New("LastInsertId is not supported by this driver")
}
func (v RowsAffected) RowsAffected() (int64, error) {
return int64(v), nil
}
7、driver.Value
Value 其实是一个空接口，可以容纳任何的数据。
// diver 的 Value 是驱动必须能够操作的 Value,Value要么是nil,要么是下面任意一种:
//
// int64
// float64
// bool
// []byte
// string [*] 除了Rows.Next，返回的不能是string
// time.Time
//
type Value interface{}
8、Value定义转换相关
在driver/types.go中，还定义了ValueConverter将一个普通的值（any）转换成driver.Value的接口、Valuer接口用于获取driver.Value等，就不逐个展开了。
// ValueConverter is the interface providing the ConvertValue method.
//
// Various implementations of ValueConverter are provided by the
// driver package to provide consistent implementations of conversions
// between drivers. The ValueConverters have several uses:
//
// - converting from the Value types as provided by the sql package
// into a database table's specific column type and making sure it
// fits, such as making sure a particular int64 fits in a
// table's uint16 column.
//
// - converting a value as given from the database into one of the
// driver Value types.
//
// - by the sql package, for converting from a driver's Value type
// to a user's type in a scan.
type ValueConverter interface {
// ConvertValue converts a value to a driver Value.
ConvertValue(v any) (Value, error)
}
// Valuer is the interface providing the Value method.
//
// Types implementing Valuer interface are able to convert
// themselves to a driver Value.
type Valuer interface {
// Value returns a driver Value.
// Value must not panic.
Value() (Value, error)
}
三、sql包
在sql包中，主要是sql.go中的 DB结构，对应为数据库的具象化实例。DB
// DB is a database handle representing a pool of zero or more
// underlying connections. It's safe for concurrent use by multiple
// goroutines.
//
// The sql package creates and frees connections automatically; it
// also maintains a free pool of idle connections. If the database has
// a concept of per-connection state, such state can be reliably observed
// within a transaction (Tx) or connection (Conn). Once DB.Begin is called, the
// returned Tx is bound to a single connection. Once Commit or
// Rollback is called on the transaction, that transaction's
// connection is returned to DB's idle connection pool. The pool size
// can be controlled with SetMaxIdleConns.
type DB struct {
// Atomic access only. At top of struct to prevent mis-alignment
// on 32-bit platforms. Of type time.Duration.
waitDuration int64 // Total time waited for new connections.
connector driver.Connector
// numClosed is an atomic counter which represents a total number of
// closed connections. Stmt.openStmt checks it before cleaning closed
// connections in Stmt.css.
numClosed uint64
mu sync.Mutex // protects following fields
freeConn []*driverConn // free connections ordered by returnedAt oldest to newest
connRequests map[uint64]chan connRequest
nextRequest uint64 // Next key to use in connRequests.
numOpen int // number of opened and pending open connections
// Used to signal the need for new connections
// a goroutine running connectionOpener() reads on this chan and
// maybeOpenNewConnections sends on the chan (one send per needed connection)
// It is closed during db.Close(). The close tells the connectionOpener
// goroutine to exit.
openerCh chan struct{}
closed bool
dep map[finalCloser]depSet
lastPut map[*driverConn]string // stacktrace of last conn's put; debug only
maxIdleCount int // zero means defaultMaxIdleConns; negative means 0
maxOpen int // <= 0 means unlimited
maxLifetime time.Duration // maximum amount of time a connection may be reused
maxIdleTime time.Duration // maximum amount of time a connection may be idle before being closed
cleanerCh chan struct{}
waitCount int64 // Total number of connections waited for.
maxIdleClosed int64 // Total number of connections closed due to idle count.
maxIdleTimeClosed int64 // Total number of connections closed due to idle time.
maxLifetimeClosed int64 // Total number of connections closed due to max connection lifetime limit.
stop func() // stop cancels the connection opener.
}
几个主要的字段，其他字段大部分都是和连接池参数相关的：
connector：用于创建数据库连接的抽象连接器，由第三方数据库提供具体实现。
mu：互斥锁，保证并发安全。
freeConn：数据库连接池，缓存可用的连接以供后续复用。
connRequests：唤醒通道集合，和阻塞等待连接的协程是一对一的关系。
openerCh：创建连接信号通道. 用于向连接创建协程 opener goroutine 发送信号。
stop：连接创建协程 opener goroutine 的终止器，用于停止该协程。
DB结构主要作用如下：
二、结语
本章中我们学习了，golang中database/sql的源码阅读，了解了整个database/sql最大的特点就是定义接口，不做具体实现，从而让使用方去方便使用不同的驱动实现。同时提供了DB实例，内置连接池，方便管理连接的创建和销毁。
三、参考
1、Go database/sql连接池 - 源码学习
2、Golang sql 标准库源码解析
