目录
一、软件信息
二、构建fastdfs镜像
三、docker 启动fdfs服务
四、k8s部署fdfs服务
1、fdfs部署文件
五、外部服务访问
一、软件信息
-
fastdfs版本：fastdfs:V5.11
-
libfastcommon版本: V1.0.36
-
fastdfs-nginx-module版本：V1.20
-
nginx版本：1.18.0
二、构建fastdfs镜像
为了减少操作了文件数，我是直接把启动文件和dockerfile写一起了，也可以把启动脚本 start.sh 单独于dockerfile；然后在构建镜像的时候把启动文件 COPY 到镜像里。
以下是包括了启动文件为一起的dockerfile构建文件。直接使用该dockerfile即可完成镜像构建。然后按照自己的实际情况来修改配置文件和挂载，即可完成容器化运行fastdfs件存储服务。
FROM centos:7
LABEL creator=tudou date="2024-06-02"
ENV FASTDFS_PATH=/var/fastdfs \
TZ=Asia/Shanghai \
TRACKER_PORT= \
STORAGE_PORT= \
TRACKER_SERVER= \
GROUP_NAME=
#get all the dependences
RUN yum install -y git wget zip unzip gcc make \
perl openssl openssl-devel pcre pcre-devel zlib zlib-devel libevent libevent-devel \
&& yum clean all \
&& mkdir -p ${FASTDFS_PATH}/files
RUN git clone --branch V1.0.36 --depth 1 https://github.com/happyfish100/libfastcommon.git ${FASTDFS_PATH}/libfastcommon \
&& cd ${FASTDFS_PATH}/libfastcommon \
&& ./make.sh \
&& ./make.sh install \
&& rm -rf ${FASTDFS_PATH}/libfastcommon
RUN git clone --branch V5.11 --depth 1 https://github.com/happyfish100/fastdfs.git ${FASTDFS_PATH}/fastdfs \
&& cd ${FASTDFS_PATH}/fastdfs \
&& ./make.sh \
&& ./make.sh install \
&& cp -r conf/*.* /etc/fdfs/ \
&& rm -rf ${FASTDFS_PATH}/fastdfs
RUN git clone --branch V1.20 https://github.com/happyfish100/fastdfs-nginx-module.git /usr/local/fastdfs-nginx-module \
&& cp /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so \
&& sed -i "s@/usr/local/include@/usr/include/fastdfs /usr/include/fastcommon@g" /usr/local/fastdfs-nginx-module/src/config \
&& cd ${FASTDFS_PATH} \
&& wget https://nginx.org/download/nginx-1.18.0.tar.gz \
&& tar -xzvf nginx-1.18.0.tar.gz \
&& cd nginx-1.18.0 \
&& ./configure --prefix=/usr/local/nginx --add-module=/usr/local/fastdfs-nginx-module/src \
&& make \
&& make install \
&& cp /usr/local/fastdfs-nginx-module/src/mod_fastdfs.conf /etc/fdfs/ \
&& rm -rf ${FASTDFS_PATH}/nginx-1.18.0*
WORKDIR ${FASTDFS_PATH}
# 启动脚本 start.sh
RUN cat > /usr/bin/start.sh <<-EOF
#!/bin/bash
cd \$(dirname \$0)
# start.sh trackerd
# start.sh storaged
# # 自定义环境变量初始化，若无自定义环境变量，则直接跳过使用配置文件的默认值
function init_config(){
if [[ -n \${STORAGE_PORT} ]];then
sed -i "s|^port=.*$|port=\${STORAGE_PORT}|g" /etc/fdfs/storage.conf
fi
if [[ -n \${TRACKER_PORT} ]];then
sed -i "s|^port=.*$|port=\${TRACKER_PORT}|g" /etc/fdfs/tracker.conf
fi
if [[ -n \${GROUP_NAME} ]];then
sed -i "s|group_name=.*$|group_name=\${GROUP_NAME}|g" /etc/fdfs/storage.conf
sed -i "s|group_name=.*$|group_name=\${GROUP_NAME}|g" /etc/fdfs/mod_fastdfs.conf
sed -i "s|url_have_group_name =.*$|url_have_group_name = true|g" /etc/fdfs/mod_fastdfs.conf
fi
if [[ -n \${TRACKER_SERVER} ]];then
sed -i "s|tracker_server=.*$|tracker_server=\${TRACKER_SERVER}|g" /etc/fdfs/storage.conf
sed -i "s|tracker_server=.*$|tracker_server=\${TRACKER_SERVER}|g" /etc/fdfs/client.conf
sed -i "s|tracker_server=.*$|tracker_server=\${TRACKER_SERVER}|g" /etc/fdfs/mod_fastdfs.conf
fi
sed -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/tracker.conf
sed -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/storage.conf
sed -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/client.conf
sed -i "s|base_path=.*$|base_path=\${FASTDFS_PATH}|g" /etc/fdfs/mod_fastdfs.conf
}
init_config
case \$1 in
trackerd)
echo "start trackerd ..."
fdfs_trackerd /etc/fdfs/tracker.conf start && tail -f ${FASTDFS_PATH}/logs/trackerd.log
;;
storaged)
sed -i "s|store_path0.*$|store_path0=\${FASTDFS_PATH}/files|g" /etc/fdfs/storage.conf
sed -i "s|store_path0.*$|store_path0=\${FASTDFS_PATH}/files|g" /etc/fdfs/mod_fastdfs.conf
echo "start storaged ..."
fdfs_storaged /etc/fdfs/storage.conf start \
&& /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf \
&& tail -f ${FASTDFS_PATH}/logs/storaged.log
;;
*)
echo "USAGE sh $0 [ trackerd|storaged ]"
exit 1
;;
esac
EOF
# 启动脚本 start.sh
# 默认fastdfs端口
EXPOSE 22122 23000 8888 80
RUN chmod a+x /usr/bin/start.sh
ENTRYPOINT ["/usr/bin/start.sh"]
CMD ["trackerd"]
#CMD ["storaged"]# 构建 fdfs 应用镜像
# docker build -t fdfs:v1.0 -f dockerfile . --no-cache
docker build -t fdfs:v1.0 -f dockerfile . 
三、docker 启动fdfs服务
# 容器启动 trackerd 服务，并且自定义环境变量值,可不指定环境变量值
docker run -itd --name trackerd -e TRACKER_PORT=22122 -e TRACKER_SERVER=192.168.159.132:22122 -p 22122:22122 fdfs:v1.0 trackerd
# 容器启动 storaged 服务，并且自定义环境变量值,可不指定环境变量值
docker run -itd --name storage -e STORAGE_PORT=23000 -e TRACKER_SERVER=192.168.159.132:22122 -p 23000:23000 fdfs:v1.0 storaged
# 分别查看容器应用的日志
docker logs trackerd
docker logs storage
四、k8s部署fdfs服务
1、配置文件
-
tracker服务需要修改的配置文件是：tracker.conf，其他文件默认但不能删。
- storage服务需要修改的配置文件有: storage.conf、mod_fastdfs.conf 以及安装在storage端的 nginx配置文件 nginx.conf。需要给storage的数据目录做软链接，要不然，有可能无法通过nginx访问 文件，其他配置保持默认即可，但不能删。
- client.conf 配置文件不是必须的，可放在任何服务端。
# 配置文件
---
kind: ConfigMap
apiVersion: v1
metadata:
name: fdfs-config
namespace: fdfs
annotations:
kubesphere.io/creator: admin
kubesphere.io/description: client.conf 可以不挂载。
data:
client.conf: |-
connect_timeout=30
network_timeout=60
base_path=/var/fastdfs
tracker_server=tracker-svc:22122
log_level=info
use_connection_pool = false
connection_pool_max_idle_time = 3600
load_fdfs_parameters_from_tracker=false
use_storage_id = false
storage_ids_filename = storage_ids.conf
http.tracker_server_port=8080
mod_fastdfs.conf: |-
connect_timeout=2
network_timeout=30
base_path=/var/fastdfs/files
load_fdfs_parameters_from_tracker=true
storage_sync_file_max_delay = 86400
use_storage_id = false
storage_ids_filename = storage_ids.conf
tracker_server=tracker-svc:22122
storage_server_port=23000
group_name=group1
url_have_group_name = true
store_path_count=1
store_path0=/var/fastdfs/files
log_level=info
log_filename=
response_mode=proxy
if_alias_prefix=
flv_support = true
flv_extension = flv
group_count = 1
[group1]
group_name=group1
storage_server_port=23000
store_path_count=1
store_path0=/var/fastdfs/files
#[group2]
#group_name=group2
#storage_server_port=23000
#store_path_count=2
#store_path0=/var/fastdfs/files
#store_path1=/var/fastdfs/files1
nginx.conf: |
user root;
worker_processes 2;
events {
worker_connections 65535;
use epoll;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
charset utf-8;
server_tokens off;
server {
listen 80;
server_name localhost;
location / {
root html;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
server {
listen 8888;
server_name localhost;
location ~/group[1-2]/M00 {
root /data/fastdfs/store/data;
# storage 服务要创建软连接: ln -s /var/fastdfs/files/data /data/fastdfs/files/data/M00
ngx_fastdfs_module;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
storage.conf: |-
disabled=false
group_name=group1
bind_addr=
client_bind=true
port=23000
connect_timeout=30
network_timeout=60
heart_beat_interval=30
stat_report_interval=60
base_path=/var/fastdfs/files
max_connections=256
buff_size = 256KB
accept_threads=1
work_threads=4
disk_rw_separated = true
disk_reader_threads = 1
disk_writer_threads = 1
sync_wait_msec=50
sync_interval=0
sync_start_time=00:00
sync_end_time=23:59
write_mark_file_freq=500
store_path_count=1
store_path0=/var/fastdfs/files
subdir_count_per_path=256
tracker_server=tracker-svc:22122
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
file_distribute_path_mode=0
file_distribute_rotate_count=100
fsync_after_written_bytes=0
sync_log_buff_interval=10
sync_binlog_buff_interval=10
sync_stat_file_interval=300
thread_stack_size=512KB
upload_priority=10
if_alias_prefix=
check_file_duplicate=0
file_signature_method=hash
key_namespace=FastDFS
keep_alive=0
use_access_log = false
rotate_access_log = false
access_log_rotate_time=00:00
rotate_error_log = false
error_log_rotate_time=00:00
rotate_access_log_size = 0
rotate_error_log_size = 0
log_file_keep_days = 0
file_sync_skip_invalid_record=false
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.domain_name=
http.server_port=8888
tracker.conf: |-
disabled=false
bind_addr=
port=22122
connect_timeout=30
network_timeout=60
base_path=/var/fastdfs
max_connections=256
accept_threads=1
work_threads=4
min_buff_size = 8KB
max_buff_size = 128KB
store_lookup=2
store_group=group2
store_server=0
store_path=0
download_server=0
reserved_storage_space = 10%
log_level=info
run_by_group=
run_by_user=
allow_hosts=*
sync_log_buff_interval = 10
check_active_interval = 120
thread_stack_size = 64KB
storage_ip_changed_auto_adjust = true
storage_sync_file_max_delay = 86400
storage_sync_file_max_time = 300
use_trunk_file = false
slot_min_size = 256
slot_max_size = 16MB
trunk_file_size = 64MB
trunk_create_file_advance = false
trunk_create_file_time_base = 02:00
trunk_create_file_interval = 86400
trunk_create_file_space_threshold = 20G
trunk_init_check_occupying = false
trunk_init_reload_from_binlog = false
trunk_compress_binlog_min_interval = 0
use_storage_id = false
storage_ids_filename = storage_ids.conf
id_type_in_filename = ip
store_slave_file_use_link = false
rotate_error_log = false
error_log_rotate_time=00:00
rotate_error_log_size = 0
log_file_keep_days = 0
use_connection_pool = false
connection_pool_max_idle_time = 3600
http.server_port=8080
http.check_alive_interval=30
http.check_alive_type=tcp
http.check_alive_uri=/status.html
---2、fdfs部署文件
由于storage存储是各自独立的，所以要使用有状态部署。tracker是透明无状态的，可以使用无状态负载deploy部署，也可以使用有状态负载StatefulSet部署。
# kubectl apply -f fdfs-trackerd.yaml -n fdfs
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: fdfs-trackerd
name: fdfs-trackerd
namespace: fdfs
spec:
replicas: 1
selector:
matchLabels:
app: fdfs-trackerd
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
labels:
app: fdfs-trackerd
spec:
containers:
- args:
- trackerd
command:
- /usr/bin/start.sh
image: swr.cn-south-1.myhuaweicloud.com/tudou/fastdfs:latest
imagePullPolicy: IfNotPresent
name: trackerd
ports:
- containerPort: 22122
name: http-0
protocol: TCP
- containerPort: 8080
name: http-1
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/fdfs/tracker.conf
name: volume-b8e30i
readOnly: true
subPath: tracker.conf
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
name: fdfs-config
name: volume-b8e30i
---
---
apiVersion: v1
kind: Service
metadata:
labels:
app: fdfs-trackerd
name: fdfs-trackerd
namespace: fdfs
spec:
ports:
- name: port-1
port: 22122
protocol: TCP
targetPort: 22122
- name: port-2
port: 8080
protocol: TCP
targetPort: 8080
selector:
app: fdfs-trackerd
type: ClusterIP
---
# kubectl apply -f fdfs-storaged.yaml -n fdfs
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
labels:
app: fdfs-storaged
name: fdfs-storaged
namespace: fdfs
spec:
podManagementPolicy: OrderedReady
replicas: 2
revisionHistoryLimit: 10
selector:
matchLabels:
app: fdfs-storaged
serviceName: fdfs-storaged
template:
metadata:
labels:
app: fdfs-storaged
spec:
containers:
- args:
- storaged
command:
- /usr/bin/start.sh
image: swr.cn-south-1.myhuaweicloud.com/tudou/fastdfs:latest
imagePullPolicy: IfNotPresent
name: storage
ports:
- containerPort: 23000
name: http-0
protocol: TCP
- containerPort: 8888
name: http-1
protocol: TCP
- containerPort: 80
name: http-2
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/fdfs/storage.conf
name: volume-ssmj62
readOnly: true
subPath: storage.conf
- mountPath: /etc/fdfs/mod_fastdfs.conf
name: volume-zlknkz
readOnly: true
subPath: mod_fastdfs.conf
- mountPath: /usr/local/nginx/conf/nginx.conf
name: volume-q4ylwb
readOnly: true
subPath: nginx.conf
- mountPath: /var/fastdfs
name: fdfs-storage-data
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
volumes:
- configMap:
defaultMode: 420
name: fdfs-config
name: volume-ssmj62
- configMap:
defaultMode: 420
name: fdfs-config
name: volume-zlknkz
- configMap:
defaultMode: 420
name: fdfs-config
name: volume-q4ylwb
updateStrategy:
rollingUpdate:
partition: 0
type: RollingUpdate
volumeClaimTemplates:
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: fdfs-storage-data
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 120Gi
storageClassName: local
volumeMode: Filesystem
---
---
apiVersion: v1
kind: Service
metadata:
labels:
app: fdfs-storaged
name: fdfs-storaged
namespace: fdfs
spec:
clusterIP: None
clusterIPs:
- None
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- name: http-0
port: 23000
protocol: TCP
targetPort: 23000
- name: http-1
port: 8888
protocol: TCP
targetPort: 8888
- name: http-2
port: 80
protocol: TCP
targetPort: 80
selector:
app: fdfs-storaged
sessionAffinity: None
type: ClusterIP
---
五、外部服务访问
当外部服务需要访问fdfs时，需要将服务的 22122和服务的 80 端口暴露。可以将这两个端口对应的服务创建NodePort类型的服务来实现端口暴露。
