Postgresql源码（140）理解PG的编译流程（make、Makefile、Makefile.global.in）

PG16

PG中使用的makefile看起来代码比较多，但是实际逻辑比较简单，这里做一些抽象总结。

总结

Makefile.global.in的$(recurse)宏自动生成了target，可以方便的进入内存目录进行编译。

all: all-common-recurse
all-common-recurse:  submake-generated-headers
   MAKE -C common all

all: all-backend-recurse
all-backend-recurse:  submake-generated-headers
   MAKE -C backend all

install: install-common-recurse
install-common-recurse:  submake-generated-headers
   MAKE -C common install

install: install-backend-recurse
install-backend-recurse:  submake-generated-headers
   MAKE -C backend install

check: check-common-recurse
check-common-recurse:  submake-generated-headers  temp-install
   MAKE -C common check

check: check-backend-recurse
check-backend-recurse:  submake-generated-headers  temp-install
   MAKE -C backend check

Makefile.global.in同时定义了对.o合.bc文件的生成规则：

%.o : %.c
	@if test ! -d $(DEPDIR); then mkdir -p $(DEPDIR); fi
	$(COMPILE.c) -o $@ $< -MMD -MP -MF $(DEPDIR)/$(*F).Po

%.o : %.cpp
	@if test ! -d $(DEPDIR); then mkdir -p $(DEPDIR); fi
	$(COMPILE.cc) -o $@ $< -MMD -MP -MF $(DEPDIR)/$(*F).Po

%.bc : %.c
	$(COMPILE.c.bc) -o $@ $<

%.bc : %.cpp
	$(COMPILE.cxx.bc) -o $@ $<

在最内层目录的Makefile定义了OBJS例如：OBJS=aset.o mcxt.o ...，然后调用common.mk文件，对OBJS中的每一个元素进行隐式编译，common.mk：

简化版内层Makefile：

OBJS = aset.o mcxt.o

# 没有OBJS生成规则，make隐式生成.o文件，调用上面定义的%.o : %.c规则
objfiles.txt: $(OBJS)
	# echo or touch objfiles

如果打开了llvm，简化版内层Makefile：

OBJS = aset.o mcxt.o

# 没有aset.bc mcxt.bc生成规则，make隐式生成.bc文件，调用上面定义的%.bc : %.c规则。
objfiles.txt: $(patsubst %.o,%.bc, $(OBJS))     ## objfiles.txt: aset.bc mcxt.bc
# 没有OBJS生成规则，make隐式生成.o文件，调用上面定义的%.o : %.c规则
$(patsubst %.o,%.bc, $(OBJS)): $(OBJS)          ## aset.bc mcxt.bc: aset.o mcxt.o

1 执行make后发生了什么？

简单工程中的makefile中可以看到很多target，依赖关系一般也比较直接，例如下面例子中：

OrcV2CBindingsAddObjectFile 依赖 OrcV2CBindingsAddObjectFile.o 依赖 OrcV2CBindingsAddObjectFile.c

all: __BUILD_DIR \
		$(BUILD_DIR)/OrcV2CBindingsAddObjectFile \
		$(BUILD_DIR)/OrcV2CBindingsBasicUsage

.PHONY: __BUILD_DIR

$(BUILD_DIR)/OrcV2CBindingsAddObjectFile: $(BUILD_DIR)/OrcV2CBindingsAddObjectFile.o
	${CXX} $< ${CXXFLAGS} ${LLVM_LD_FLAGS} -o $@ 
$(BUILD_DIR)/OrcV2CBindingsAddObjectFile.o: OrcV2CBindingsAddObjectFile.c
	${CC} -c $< ${CFLAGS} ${LLVM_CC_FLAGS} -o $@

但打开PG根目录下的Makefile：

subdir = src
top_builddir = ..
include Makefile.global

SUBDIRS = \
	common \
	port \
	timezone \
	backend \
	backend/utils/mb/conversion_procs \
	backend/snowball \
	include \
	interfaces \
	backend/replication/libpqwalreceiver \
	backend/replication/pgoutput \
	fe_utils \
	bin \
	pl \
	makefiles \
	test/regress \
	test/isolation \
	test/perl

ifeq ($(with_llvm), yes)
SUBDIRS += backend/jit/llvm
endif

# There are too many interdependencies between the subdirectories, so
# don't attempt parallel make here.
.NOTPARALLEL:

$(recurse)

...
...

看不到target信息，代码的可读性比较差，因为所有target都是生成的。优点是target动态生成，代码量少，比较灵活。
所有的细节都封装在$(recurse)中，具体在Makefile.global.in中定义。

2 Makefile.global.in

截取关键代码：


standard_targets = all install installdirs uninstall distprep clean distclean maintainer-clean coverage check checkprep installcheck init-po update-po
# these targets should recurse even into subdirectories not being built:
standard_always_targets = distprep clean distclean maintainer-clean

.PHONY: $(standard_targets) install-strip html man installcheck-parallel update-unicode

# make `all' the default target
all:


##########################################################################
#
# Programs and flags

# Compilers

CPP = @CPP@
CPPFLAGS = @CPPFLAGS@
PG_SYSROOT = @PG_SYSROOT@

override CPPFLAGS := $(ICU_CFLAGS) $(CPPFLAGS)

ifdef PGXS
override CPPFLAGS := -I$(includedir_server) -I$(includedir_internal) $(CPPFLAGS)
else # not PGXS
override CPPFLAGS := -I$(top_srcdir)/src/include $(CPPFLAGS)
ifdef VPATH
override CPPFLAGS := -I$(top_builddir)/src/include $(CPPFLAGS)
endif
endif # not PGXS

CC = @CC@
GCC = @GCC@
SUN_STUDIO_CC = @SUN_STUDIO_CC@
CXX = @CXX@
CFLAGS = @CFLAGS@
CFLAGS_SL = @CFLAGS_SL@
# *_MODULE are for flags applied to extension libraries
CFLAGS_SL_MODULE = @CFLAGS_SL_MODULE@
CXXFLAGS_SL_MODULE = @CXXFLAGS_SL_MODULE@
CFLAGS_UNROLL_LOOPS = @CFLAGS_UNROLL_LOOPS@
CFLAGS_VECTORIZE = @CFLAGS_VECTORIZE@
CFLAGS_CRC = @CFLAGS_CRC@
PERMIT_DECLARATION_AFTER_STATEMENT = @PERMIT_DECLARATION_AFTER_STATEMENT@
CXXFLAGS = @CXXFLAGS@

LLVM_CPPFLAGS = @LLVM_CPPFLAGS@
LLVM_CFLAGS = @LLVM_CFLAGS@
LLVM_CXXFLAGS = @LLVM_CXXFLAGS@

# Kind-of compilers

BISON = @BISON@
BISONFLAGS = @BISONFLAGS@ $(YFLAGS)
FLEX = @FLEX@
FLEXFLAGS = @FLEXFLAGS@ $(LFLAGS)
DTRACE = @DTRACE@
DTRACEFLAGS = @DTRACEFLAGS@
ZIC = @ZIC@

# Linking

AR = @AR@
AROPT = crs
LIBS = @LIBS@
LDAP_LIBS_FE = @LDAP_LIBS_FE@
LDAP_LIBS_BE = @LDAP_LIBS_BE@
UUID_LIBS = @UUID_LIBS@
LLVM_LIBS=@LLVM_LIBS@


# Tree-wide build support

all install check installcheck: submake-generated-headers

.PHONY: submake-generated-headers

submake-generated-headers:
ifndef NO_GENERATED_HEADERS
ifeq ($(MAKELEVEL),0)
	$(MAKE) -C $(top_builddir)/src/backend generated-headers
endif
endif



##########################################################################
#
# Recursive make support

define _create_recursive_target
.PHONY: $(1)-$(2)-recurse
$(1): $(1)-$(2)-recurse
$(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install)
	$$(MAKE) -C $(2) $(3)
endef

recurse = $(foreach target,$(if $1,$1,$(standard_targets)),$(foreach subdir,$(if $2,$2,$(SUBDIRS)),$(eval $(call

在最后面我们看到了recurse的定义：
recurse = $(foreach target,$(if $1,$1,$(standard_targets)),$(foreach subdir,$(if $2,$2,$(SUBDIRS)),$(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))))
这里调用了_create_recursive_target函数来生成所有目录的target。

PG的例子太复杂，这里抽象出来一个最小用例来继续分析。

3 $(recurse)最小DEMO

Makefile


subdir = src
top_builddir = ..
include Makefile.global

SUBDIRS = \
	common \
	port \
	timezone \
	backend

$(recurse)

Makefile.global


standard_targets = all install check
.PHONY: $(standard_targets)


all:


###############################
all install check installcheck: submake-generated-headers

.PHONY: submake-generated-headers

submake-generated-headers:
ifndef NO_GENERATED_HEADERS
ifeq ($(MAKELEVEL),0)
	$(info ------------GENERATEHEADERS------------)
endif
endif

###############################
define _create_recursive_target
.PHONY: $(1)-$(2)-recurse
$(1): $(1)-$(2)-recurse
$(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install)
	$(info ------------GENERATE TATGETS------------)
	$(info $$1: $1, $$2: $2, $$3: $3)
	$(info $(1): $(1)-$(2)-recurse)
	$(info $(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install))
	$(info MAKE -C $(2) $(3))
	$$(MAKE) -C $(2) $(3)
endef

recurse = $(foreach target,$(if $1,$1,$(standard_targets)),$(foreach subdir,$(if $2,$2,$(SUBDIRS)),$(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))))



# recurse = \
#     $(foreach target,$(if $1,$1,$(standard_targets)),\
#         $(foreach subdir,$(if $2,$2,$(SUBDIRS)),\
#             $(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))\
#         )\
#     )

执行结果：
在这里插入图片描述

4 $(recurse)最小DEMO结果分析

recurse = $(foreach target,$(if $1,$1,$(standard_targets)),$(foreach subdir,$(if $2,$2,$(SUBDIRS)),$(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))))

展开后

recurse = \
    $(foreach target,$(if $1,$1,$(standard_targets)),\
        $(foreach subdir,$(if $2,$2,$(SUBDIRS)),\
            $(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))\
        )\
    )

外层循环遍历standard_targets = all install check，也就是PG定义的所有make指令，需要为每个指令定义target。
内层循环遍历SUBDIRS = common backend，这个是Makefile中定义的，也就是谁include Makefile.global，谁负责定义好SUBDIRS。表示当前目录下，哪些子目录需要进行编译。

调用_create_recursive_target函数，为makefile动态定义target：

define _create_recursive_target
.PHONY: $(1)-$(2)-recurse
$(1): $(1)-$(2)-recurse
$(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install)
	$$(MAKE) -C $(2) $(3)
endef

函数执行后，生成一系列target，两个小细节：

如果target是all install check installcheck集中情况，需要增加submake-generated-headers依赖，这个是生成头文件的，PG的一些头文件是编译时生成出来的。
如果target是check，需要增加temp-install依赖，temp-install是负责regress测试的target。

all: all-common-recurse
all-common-recurse:  submake-generated-headers
	MAKE -C common all

all: all-backend-recurse
all-backend-recurse:  submake-generated-headers
	MAKE -C backend all

install: install-common-recurse
install-common-recurse:  submake-generated-headers
	MAKE -C common install

install: install-backend-recurse
install-backend-recurse:  submake-generated-headers
	MAKE -C backend install

check: check-common-recurse
check-common-recurse:  submake-generated-headers  temp-install
	MAKE -C common check

check: check-backend-recurse
check-backend-recurse:  submake-generated-headers  temp-install
	MAKE -C backend check

5 继续完善demo

前面提到的demo只构造了target，并没有真正编译.c文件。
继续增加了.c文件编译到.o的部分。

$ tree
.
├── common
│   ├── aset.c
│   └── Makefile
├── common.mk
├── Makefile
└── Makefile.global

1 directory, 5 files

common/Makefile

subdir = src/backend/utils/mmgr
top_builddir = ../../../..
include ../Makefile.global

OBJS = \
	aset.o

include ../common.mk

common.mk

subsysfilename = objfiles.txt

ifneq ($(subdir), src/backend)
all: $(subsysfilename)
endif

objfiles.txt: $(OBJS)
	$(if $(filter-out $(OBJS),$?),( $(if $(SUBDIROBJS),cat $(SUBDIROBJS); )echo $(addprefix $(subdir)/,$(OBJS)) ) >$@,touch $@)

Makefile


subdir = src
top_builddir = ..
include Makefile.global

SUBDIRS = \
	common \
	backend

$(recurse)

Makefile.global


standard_targets = all install check
.PHONY: $(standard_targets)


all:


###############################
all install check installcheck: submake-generated-headers

.PHONY: submake-generated-headers

submake-generated-headers:
ifndef NO_GENERATED_HEADERS
ifeq ($(MAKELEVEL),0)
	$(info ------------GENERATEHEADERS------------)
endif
endif

###############################
define _create_recursive_target
.PHONY: $(1)-$(2)-recurse
$(1): $(1)-$(2)-recurse
$(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install)
	$(info ------------GENERATE TATGETS------------)
	$(info $$1: $1, $$2: $2, $$3: $3)
	$(info $(1): $(1)-$(2)-recurse)
	$(info $(1)-$(2)-recurse: $(if $(filter all install check installcheck, $(3)), submake-generated-headers) $(if $(filter check, $(3)), temp-install))
	$(info MAKE -C $(2) $(3))
	$$(MAKE) -C $(2) $(3)
endef

recurse = $(foreach target,$(if $1,$1,$(standard_targets)),$(foreach subdir,$(if $2,$2,$(SUBDIRS)),$(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))))



# recurse = \
#     $(foreach target,$(if $1,$1,$(standard_targets)),\
#         $(foreach subdir,$(if $2,$2,$(SUBDIRS)),\
#             $(eval $(call _create_recursive_target,$(target),$(subdir),$(if $3,$3,$(target))))\
#         )\
#     )

这里主要需要关注的就是common.mk。每个源码目录中都有一些文件需要编译，所以都需要配一个Makefile，这些Makefile中相同的逻辑抽取到了common.mk中。

objfiles.txt: $(OBJS)
	...
	...

注意这里OBJS = aset.o，这里makefile构造objfiles.txt依赖aset.o，而aset.o触发了Makefile的隐式规则，会自动从aset.c编译为aset.o。

除了规则外，具体的编译方法也可以在Makefile中定义，例如PG的makefile.global.in中定义了对.o文件的处理方法。

%.o : %.c
	@if test ! -d $(DEPDIR); then mkdir -p $(DEPDIR); fi
	$(COMPILE.c) -o $@ $< -MMD -MP -MF $(DEPDIR)/$(*F).Po

%.o : %.cpp
	@if test ! -d $(DEPDIR); then mkdir -p $(DEPDIR); fi
	$(COMPILE.cc) -o $@ $< -MMD -MP -MF $(DEPDIR)/$(*F).Po

llvm支持：common.mk中还有一段，如果打开了llvm后，objfiles.txt会同时依赖.o和.bc文件。

ifeq ($(with_llvm), yes)
objfiles.txt: $(patsubst %.o,%.bc, $(OBJS))
$(patsubst %.o,%.bc, $(OBJS)): $(OBJS)
endif

在makefile.global.in中也定义了对.bc文件的处理方法：

%.bc : %.c
	$(COMPILE.c.bc) -o $@ $<

%.bc : %.cpp
	$(COMPILE.cxx.bc) -o $@ $<