PostpreSQL内核学习-Chapter4 Foreign Data Wrapper（FDW）

FDW连接

preparations

两个在不同主机（满足TCP/IP连接是分属于不同IP地址下）
（如果是安装PG源码）进入源码目录下面的/contrib/postgres_fdw/，然后用make & make install编译和安装

[root@localhost /]# cd 源码存放的目录/postgresql-15.3/contrib/postgres_fdw/
[root@localhost postgres_fdw]#  make & make install

（如果是yum安装）

Connection

[userb@localhost ~]$ psql -d my_second_pgdb
psql (15.3)
Type "help" for help.

# 1.创建拓展
my_second_pgdb=# CREATE EXTENSION postgres_fdw;
CREATE EXTENSION

my_first_pgdb=# \x
Expanded display is on.

my_first_pgdb=# select * from pg_foreign_data_wrapper;
-[ RECORD 1 ]+-------------
oid          | 16392
fdwname      | postgres_fdw
fdwowner     | 10
fdwhandler   | 16390
fdwvalidator | 16391
fdwacl       | 
fdwoptions   | 

my_first_pgdb=# select * from pg_extension;
-[ RECORD 1 ]--+-------------
oid            | 13673
extname        | plpgsql
extowner       | 10
extnamespace   | 11
extrelocatable | f
extversion     | 1.0
extconfig      | 
extcondition   | 
-[ RECORD 2 ]--+-------------
oid            | 16389
extname        | postgres_fdw
extowner       | 10
extnamespace   | 2200
extrelocatable | t
extversion     | 1.1
extconfig      | 
extcondition   | 



# 2. 创建目标库连接串，包含欲访问的外部表所在主机的IP地址，欲访问的数据库进程的端口号，数据库名称
# 关系型数据库常用默认端口号
# 1.Oracle数据库默认端口号为，1521； 
# 2.MySQL数据库默认端口号为，3306； 
# 3.SQLServer数据库默认端口号为，1433;
# 4.PostgreSQL数据库默认端口号为，5432;

my_second_pgdb=# CREATE SERVER server_a_my_first_pgdb FOREIGN DATA WRAPPER postgres_fdw OPTIONS(host 'xxx.xxx.xxx.xxx', port '5432', dbname 'my_first_pgdb');
CREATE SERVER
my_second_pgdb=# select * from pg_foreign_server;
-[ RECORD 1 ]----------------------------------------------------
oid        | 16396
srvname    | server_a_my_first_pgdb
srvowner   | 10
srvfdw     | 16392
srvtype    | 
srvversion | 
srvacl     | 
srvoptions | {host=xxx.xxx.xxx.xxx,port=5432,dbname=my_first_pgdb}




# 3. 用户映射：欲访问的外部表所属用户名、密码（OPTIONS中），将其映射到当前库的某一个用户（FOR关键字后面）上来
# 这里假设当前库的被映射的用户名为usera，也就是欲访问的外部表所属的用户；映射的用户名为userb，也就是当前库的其中一个用户。
# 查看数据库中的用户和（已MS5加密后的）密码：SELECT rolname,rolpassword FROM pg_authid;
# 修改用户密码:ALTER USER usera WITH PASSWORD 'useraaa';
# 这里被映射的用户一定要有密码，不然会报错

my_second_pgdb=# CREATE USER MAPPING FOR userb SERVER server_a_my_first_pgdb OPTIONS (user 'usera', password 'useraaa');
CREATE USER MAPPING
my_secondt_pgdb=# select * from pg_user_mappings;
-[ RECORD 1 ]----------------------------
umid      | 16401
srvid     | 16396
srvname   | server_a_my_first_pgdb
umuser    | 10
usename   | userb
umoptions | {user=usera,password=useraaa}




# 4. 创建外部表（相当于外部表在本地数据库的映射），欲访问外部表或外部视图的字段（列）信息，字段可以少于目标表或视图，取自己需要的
# 这里有一个schema（模式）的概念，默认所有的表都放在名为public的模式下，如果指定了模式的就放在指定模式下
# 字段可以选取内容的意思是，例如我的外部表本来有五列字段，但可以仅访问其中两列，将欲访问的写在下方即可

my_second_pgdb=# CREATE FOREIGN TABLE fdw_a_weather(city character varying(80), temp_hi integer) SERVER server_a_my_first_pgdb OPTIONS(schema_name 'public', table_name 'weather');
CREATE FOREIGN TABLE
my_second_pgdb=# select * from pg_foreign_table;
-[ RECORD 1 ]--------------------------------------
ftrelid   | 16402
ftserver  | 16396
ftoptions | {schema_name=public,table_name=weather}

Access Foreign Data

注意查询时，外部表所在主机的数据库服务器也要开启，才能成功查询到。

my_second_pgdb=# select * from fdw_a_weather;
       city        | temp_hi 
-------------------+---------
 San Francisco     |      50
 Changsha          |      35
 Hainan Ledong     |      31
 Hainan Sanya      |      31
 Hainan Baoting    |      31
 Hainan Qiongzhong |      31
 Hainan Dongfang   |      31
 Hainan Lingshui   |      31
 Changsha          |      28
(9 rows)

连接FDW中可能遇到的问题

failed: 没有到主机的路由

my_first_pgdb=# select * from fdw_a_weather;
2023-07-05 11:10:18.061 CST [3225] ERROR:  could not connect to server "server_a_my_first_pgdb"
2023-07-05 11:10:18.061 CST [3225] DETAIL:  connection to server at "xxx.xxx.xxx.xxx", port 5432 failed: 没有到主机的路由
		Is the server running on that host and accepting TCP/IP connections?
2023-07-05 11:10:18.061 CST [3225] STATEMENT:  select * from fdw_a_weather;
ERROR:  could not connect to server "server_a_my_first_pgdb"
DETAIL:  connection to server at "xxx.xxx.xxx.xxx", port 5432 failed: 没有到主机的路由
	Is the server running on that host and accepting TCP/IP connections?

解决：可能防火墙没有放行数据库端口，这里以5432为例

# 查看已放行端口号
[root@localhost ~]# firewall-cmd --list-port
# 添加放行端口号5432
[root@localhost ~]# firewall-cmd --zone=public --add-port=5432/tcp --permanent
success
# 重新加载防火墙
[root@localhost ~]# firewall-cmd --reload
success
# 再次查看，添加成功
[root@localhost ~]# firewall-cmd --list-port
5432/tcp

failed: 拒绝连接

my_first_pgdb=# select * from fdw_a_weather;
2023-07-05 11:13:52.810 CST [3225] ERROR:  could not connect to server "server_a_my_first_pgdb"
2023-07-05 11:13:52.810 CST [3225] DETAIL:  connection to server at "xxx.xxx.xxx.xxx", port 5432 failed: 拒绝连接
		Is the server running on that host and accepting TCP/IP connections?
2023-07-05 11:13:52.810 CST [3225] STATEMENT:  select * from fdw_a_weather;
ERROR:  could not connect to server "server_a_my_first_pgdb"
DETAIL:  connection to server at "xxx.xxx.xxx.xxx", port 5432 failed: 拒绝连接
	Is the server running on that host and accepting TCP/IP connections?

解决方法：进入主机a外部表所属数据簇（Database Cluster）目录下，修改postgresql.conf文件，添加listen_addresses = '*'；修改pg_hba.conf，添加host all all 0.0.0.0/0 trust，修改完记得重启数据库的服务器（如果是别的数据库，也是找到其配置文件的监听地址项进行修改）

[usera@localhost data]$ vim postgresql.conf 
[usera@localhost data]$ vim pg_hba.conf 
[usera@localhost ~]$ pg_ctl -D /pgdata/15.3/poc/data/ restart

修改完如下图：

ERROR: relation “my_first_pgdb.weather” does not exist

my_first_pgdb=# select * from fdw_a_weather;
2023-07-05 11:42:22.321 CST [3225] ERROR:  relation "my_first_pgdb.weather" does not exist
2023-07-05 11:42:22.321 CST [3225] CONTEXT:  remote SQL command: SELECT city, temp_hi FROM my_first_pgdb.weather
2023-07-05 11:42:22.321 CST [3225] STATEMENT:  select * from fdw_a_weather;
ERROR:  relation "my_first_pgdb.weather" does not exist
CONTEXT:  remote SQL command: SELECT city, temp_hi FROM my_first_pgdb.weather

错误原因：CREATE FOREIGN TABLE时，OPTIONS后面没有正确填写外部表所属的schema模式名称，PostgreSQL中默认模式名就是public，如果没有新建过别的模式名，就填public就行了，注意：模式这一概念在db之下

Chapter 4 Foreign Data Wrappers 外部数据封装）

4.1 Foreign Data Wrapper（FDW）

What is Foreign Data?——Firstly, a table on a remote server is called a foreign table in SQL/MED.（运行在远程服务器上的表称之为外部表） PostgreSQL’s Foreign Data Wrappers are that use SQL/MES to manage foreign tables which are similar to local tables.（外部数据封装器会用SQL/MES来管理类似本地表的外部表）
After installing the necessary extension and making the appropriate settings, you can access the foreign tables on the remote servers.（安装必须的拓展和进行合适的设置后，就可以访问远程服务器的外部关系表了） For example, suppose there are two remote servers, namaly, postgresql and mysql, which have foreign_pg_tbl table and foreign_my_tbl table, respectively. In this exmaple, you can access the foreign tables from the local server by issuing the SELECT queries as shown below.（假设有两个远程服务器，名字分别为postgresql和mysql，分别有两张关系表foreign_pg_tbl和foreign_my_tbl，下例中可以用SELECT查询在本地服务器上访问外部表）
Moreover, you can execute the join operation with the foreign tables stored in different servers which are similar to the local tables.（此外，和连接本地服务器上的关系表类似，还可以对存储在不同服务器上的外部表进行连接操作）
Many FDW extensions have been developed and listed in Postgres wiki. However, almost all extensions are not properly maintained except for postgres_fdw, which is officially developed and maintained by the PostgreSQL Global Development Group as an extension to access a remote PostgreSQL server.（目前已经开发了很多外部数据封装器拓展，都列在了Postgreswiki中，除了postgres_fdw几乎所有其他的fdw拓展都没有得到良好维护，postgres_fdw 作为提供给其他服务器的访问PG的拓展程序，是由PG全球开发小组开发并维护的）
PostgreSQL’s FDW is described in detail in the following sections. Section 4.1.1 presents an overview of the FDW in PostgreSQL. Section 4.1.2 describes how the postgres_fdw extension performs.