文章目录
- 1.问题现象
- 2.环境变量
- 3.实例1:交编译arm64 hello.c
- 解决方法1: 指定rootfs下的include头文件
- 解决方法2: 下载开源arm64-linux-gnu-gcc
- 小结
- 4.交叉编译依赖第3方库
- 1.前言
- 2.小知识: gcc默认搜索与支持的库
- 3.实例: 交叉编译依赖ROS的程序
- gcc/g++ 编译流程
- gcc/g++ 链接流程
- 5.遇到的问题
- 1.链接选项--rpath --rpath-link区别
- 2.scons如何添加--rpath-link
- 3.scons如何修改链接选项顺序
- 6.总结
- 7.未解决疑问, 问题
1.问题现象
- 目前使用docker + qemu +arm64 容器编译做编译.
- 之前做嵌入式开发, 使用的是传统的arm-linux-gcc 交叉编译, 使用SDK环境, 添加程序和库, 按模版添加Makefile, cmake, config, 具体原理不清.
- 最近使用scons替换Makefile, cmake; 发现gcc的 --sysroot 选项, 可以指定rootfs路径,替换默认搜索头文件, 库路径
- cc编译选项: -I /usr/include -I /usr/include/aarch64-linux-gnu
- ld链接选项: -L /usr/lib -L /usrlib
- 尝试使用传统的交叉编译方法, 编译一下项目.
- 解决遗留疑问: 如果交叉编译依赖opencv, openssl, ros等第3方库, 如何配置的问题.
2.环境变量
- 交叉编译工具路径:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/
- 交叉编译前缀:
aarch64-none-linux-gnu-
- rootfs根文件系统路径:
/home/liuj/3_work/6_rk3588_sdk/debian/binary
- ubuntu软件中心, arm64 交叉编译工具:
g++-aarch64-linux-gnu gcc-aarch64-linux-gnu
3.实例1:交编译arm64 hello.c
代码:
1_hello.c
#include <stdio.h>
int main()
{
printf("hello world\n");
return 0;
}
SConstruct
# 交叉编译工具路径
cross_Tools_Path="/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/"
# 交叉编译工具前缀
cross_Tool_Prefix="aarch64-none-linux-gnu-"
# rootfs路径
rootdir="/home/liuj/3_work/6_rk3588_sdk/debian/binary"
# scons官方交叉编译示例
env_options = {
"CC" : cross_Tool_Prefix+"gcc",
"CXX" : cross_Tool_Prefix+"g++",
"LD" : cross_Tool_Prefix+"ld",
"AR" : cross_Tool_Prefix+"ar",
"STRIP" : cross_Tool_Prefix+"strip",
}
env = Environment(**env_options)
env.AppendENVPath('PATH', cross_Tools_Path)
# 编译选项
env['CCFLAGS'] = "--sysroot=%s "%(rootdir)
# env['CCFLAGS'] += "-I %s/usr/include \
# -I %s/usr/include/aarch64-linux-gnu \
# -I %s/usr/include/freetype2 \
# -I %s//usr/local/include" \
# %(rootdir,rootdir,rootdir,rootdir)
env.Program('1_hello.c')
运行结果: scons
aarch64-none-linux-gnu-gcc -o 1_hello.o -c --sysroot=/home/liuj/3_work/6_rk3588_sdk/debian/binary 1_hello.c
In file included from 1_hello.c:1:
/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/include/stdio.h:27:10: fatal error: bits/libc-header-start.h: No such file or directory
27 | #include <bits/libc-header-start.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
编译错误原因:
- rk3588默认自带编译工具链aarch64-none-linux-gnu-gcc 不支持–sysroot选项
解决方法1: 指定rootfs下的include头文件
SConstruct 取消如下注释
env['CCFLAGS'] += "-I %s/usr/include \
-I %s/usr/include/aarch64-linux-gnu \
-I %s/usr/include/freetype2 \
-I %s//usr/local/include" \
%(rootdir,rootdir,rootdir,rootdir)
scons
编译成功
aarch64-none-linux-gnu-gcc -o 1_hello.o -c --sysroot=/home/liuj/3_work/6_rk3588_sdk/debian/binary -I /home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/include -I /home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/include/aarch64-linux-gnu -I /home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/include/freetype2 -I /home/liuj/3_work/6_rk3588_sdk/debian/binary//usr/local/include 1_hello.c
aarch64-none-linux-gnu-gcc -o 1_hello 1_hello.o
解决方法2: 下载开源arm64-linux-gnu-gcc
ubuntu软件仓库安装arm64交叉编译工具
sudo apt install g++-aarch64-linux-gnu gcc-aarch64-linux-gnu
SConstruct 修改之后如下
import os
cross_Tool_Prefix="aarch64-linux-gnu-"
rootdir="/home/liuj/3_work/6_rk3588_sdk/debian/binary"
env_options = {
"CC" : cross_Tool_Prefix+"gcc",
"CXX" : cross_Tool_Prefix+"g++",
"LD" : cross_Tool_Prefix+"ld",
"AR" : cross_Tool_Prefix+"ar",
"STRIP" : cross_Tool_Prefix+"strip",
}
env = Environment(**env_options)
env['CCFLAGS'] = "--sysroot=%s "%(rootdir)
env.Program('1_hello.c')
编译 scons
aarch64-linux-gnu-gcc -o 1_hello.o -c --sysroot=/home/liuj/3_work/6_rk3588_sdk/debian/binary 1_hello.c
aarch64-linux-gnu-gcc -o 1_hello 1_hello.o
小结
- 解决方法1 有一个问题, 编译C++文件错误, 提示找不到头文件, 暂未找到解决方法.
- 后续使用 ** 解决方法2** , 目的是验证gcc --sysroot选项做交叉编译功能
4.交叉编译依赖第3方库
1.前言
之前的文章: 搭建交叉编译环境–RK3588示范
遗留一个问题: 如果交叉编译依赖opencv, openssl, ros等第3方库, 改如何配置.
2.小知识: gcc默认搜索与支持的库
- 交叉编译工具路径:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/
查看gcc默认搜索lib库
./aarch64-none-linux-gnu-gcc --print-search-dirs
install:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/lib/gcc/aarch64-none-linux-gnu/10.3.1/
programs:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/libexec/gcc/aarch64-none-linux-gnu/10.3.1/:/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/libexec/gcc/:/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/lib/gcc/aarch64-none-linux-gnu/10.3.1/…/…/…/…/aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu/10.3.1/:/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/lib/gcc/aarch64-none-linux-gnu/10.3.1/…/…/…/…/aarch64-none-linux-gnu/bin/
libraries:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/lib/gcc/aarch64-none-linux-gnu/10.3.1/:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/lib/gcc/:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/lib/gcc/aarch64-none-linux-gnu/10.3.1/…/…/…/…/aarch64-none-linux-gnu/lib/aarch64-none-linux-gnu/10.3.1/:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/lib/gcc/aarch64-none-linux-gnu/10.3.1/…/…/…/…/aarch64-none-linux-gnu/lib/…/lib64/:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/aarch64-none-linux-gnu/libc/lib/aarch64-none-linux-gnu/10.3.1/:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/aarch64-none-linux-gnu/libc/lib/…/lib64/:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/aarch64-none-linux-gnu/libc/usr/lib/aarch64-none-linux-gnu/10.3.1/:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/aarch64-none-linux-gnu/libc/usr/lib/…/lib64/:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/lib/gcc/aarch64-none-linux-gnu/10.3.1/…/…/…/…/aarch64-none-linux-gnu/lib/:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/aarch64-none-linux-gnu/libc/lib/:
/home/liuj/3_work/6_rk3588_sdk/prebuilts/gcc/linux-x86/aarch64/gcc-arm-10.3-2021.07-x86_64-aarch64-none-linux-gnu/bin/…/aarch64-none-linux-gnu/libc/usr/lib/
大家可以查看每一个lib库路径;
通过查看, 发现主要是支持libc, libgnuc, stdc++库.
故默认aarch64-linux-gnu-gcc 只能编译: 标准c/c++, kernel, linux系统编程等API 代码
3.实例: 交叉编译依赖ROS的程序
代码
test_uint8_array.cc
#include <stdint.h>
#include <vector>
#include <ros/console.h>
#include <ros/ros.h>
#include <std_msgs/UInt8MultiArray.h>
using namespace std;
int main(int argc, char* argv[])
{
ros::init(argc, argv, "test_topic");
ros::NodeHandle n;
ros::Publisher pub_uint8_array = n.advertise<std_msgs::UInt8MultiArray>("/test/UInt8MultiArray", 5);
std_msgs::UInt8MultiArray msg_uint8_array;
// uint8_t data[4] = { 0x1, 0x2, 0x4, 0x8 };
vector<uint8_t> data = { 0x1, 0x2, 0x4, 0x8 };
while (ros::ok())
{
msg_uint8_array.data = data;
pub_uint8_array.publish(msg_uint8_array);
sleep(1);
ros::spinOnce();
}
return 0;
}
SConstruct
import os
cross_Tool_Prefix="aarch64-linux-gnu-"
rootdir="/home/liuj/3_work/6_rk3588_sdk/debian/binary"
env_options = {
"CC" : cross_Tool_Prefix+"gcc",
"CXX" : cross_Tool_Prefix+"g++",
"LD" : cross_Tool_Prefix+"ld",
"AR" : cross_Tool_Prefix+"ar",
"STRIP" : cross_Tool_Prefix+"strip",
"LINKCOM" : "$LINK -o $TARGET $SOURCES $LD_PREFLAGS $_LIBDIRFLAGS $__RPATH $LINKFLAGS $_LIBFLAGS",
}
env = Environment(**env_options)
# 编译选项
env['CCFLAGS'] = "--sysroot=%s "%(rootdir)
env['CCFLAGS'] += "-I %s/opt/ros/noetic/include "%(rootdir)
env['CPPPATH'] = ["%s/usr/include"%(rootdir),
"%s/usr/include/aarch64-linux-gnu"%(rootdir),
"%s/usr/include/freetype2"%(rootdir),
"%s/usr/local/include"%(rootdir)
]
# 链接选项
env['LD_PREFLAGS'] = "--sysroot=%s " %(rootdir)
sys_lib = [rootdir+"/lib",
rootdir+"/usr/lib",
rootdir+"/usr/lib/aarch64-linux-gnu",
rootdir+"/usr/local/lib",
rootdir+"/lib/aarch64-linux-gnu",
]
ros_lib = [rootdir+'/opt/ros/noetic/lib']
env['LIBPATH'] = sys_lib+ros_lib
dep_lib = ['roscpp','rosconsole']
env['LIBS'] = dep_lib
# env['RPATH'] = ros_lib
env['LINKFLAGS'] = "-Wl,--rpath-link={} ".format(" ".join(ros_lib))
env['LINKFLAGS'] += "-Wl,--copy-dt-needed-entries "
env.Program('test_uint8_array.cc')
scons编译成功
arch64-linux-gnu-g++ -o test_uint8_array.o -c --sysroot=/home/liuj/3_work/6_rk3588_sdk/debian/binary -I /home/liuj/3_work/6_rk3588_sdk/debian/binary/opt/ros/noetic/include -I/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/include -I/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/include/aarch64-linux-gnu -I/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/include/freetype2 -I/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/local/include test_uint8_array.cc
aarch64-linux-gnu-g++ -o test_uint8_array test_uint8_array.o --sysroot=/home/liuj/3_work/6_rk3588_sdk/debian/binary -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/lib -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/lib -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/lib/aarch64-linux-gnu -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/local/lib -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/lib/aarch64-linux-gnu -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/opt/ros/noetic/lib -Wl,--rpath-link=/home/liuj/3_work/6_rk3588_sdk/debian/binary/opt/ros/noetic/lib -Wl,--copy-dt-needed-entries -lroscpp -lrosconsole
scons: done building targets.
美化的流程显示
gcc/g++ 编译流程
gcc/g++ 链接流程
运行测试:
拷贝文件到arm64 开发板, 运行正常.
5.遇到的问题
1.链接选项–rpath --rpath-link区别
一开始使用 env[‘RPATH’], 对应的gcc链接选项为 -Wl,--rpath=xxx_lib_dir
在ld链接时提示错误:
/usr/bin/ld: warning: libxxx.so.6, needed by /a/b/c/libyyy.so, not found (try using -rpath or -rpath-link)
网上搜索到的区别: GCC 中 -L、-rpath和-rpath-link的区别
- –rpath 1.链接时指定依赖库路径 2.运行时指定链接路径
- –rpath-link 1.链接时指定依赖库路径
尝试修改为–rpath-link, 如下
-- -Wl,--rpath=/home/liuj/3_work/6_rk3588_sdk/debian/binary/opt/ros/noetic/lib
++ -Wl,--rpath-link=/home/liuj/3_work/6_rk3588_sdk/debian/binary/opt/ros/noetic/lib \
手动运行g++链接指令, 命令如下
aarch64-linux-gnu-g++ -o test_uint8_array.out test_uint8_array.o \
--sysroot=/home/liuj/3_work/6_rk3588_sdk/debian/binary \
-L/home/liuj/3_work/6_rk3588_sdk/debian/binary/lib \
-L/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/lib \
-L/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/lib/aarch64-linux-gnu \
-L/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/local/lib \
-L/home/liuj/3_work/6_rk3588_sdk/debian/binary/lib/aarch64-linux-gnu \
-L/home/liuj/3_work/6_rk3588_sdk/debian/binary/opt/ros/noetic/lib \
-Wl,--rpath-link=/home/liuj/3_work/6_rk3588_sdk/debian/binary/opt/ros/noetic/lib \
-Wl,--copy-dt-needed-entries \
-lroscpp -lrosconsole
结果:
手动链接成功
2.scons如何添加–rpath-link
查看官方文档, github下载源代码查看, 没有找到scons设置–rpath-link的选项
解决方法:
使用 LINKFKAGS
设置 --rpath-link
ros_lib = [rootdir+'/opt/ros/noetic/lib']
env['LINKFLAGS'] = "-Wl,--rpath-link={} ".format(" ".join(ros_lib))
编译结果: -->链接错误
aarch64-linux-gnu-g++ -o test_uint8_array -Wl,--rpath-link=/home/liuj/3_work/6_rk3588_sdk/debian/binary/opt/ros/noetic/lib -Wl,--copy-dt-needed-entries test_uint8_array.o -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/lib -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/lib -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/lib/aarch64-linux-gnu -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/usr/local/lib -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/lib/aarch64-linux-gnu -L/home/liuj/3_work/6_rk3588_sdk/debian/binary/opt/ros/noetic/lib -lroscpp -lrosconsole
/usr/lib/gcc-cross/aarch64-linux-gnu/9/../../../../aarch64-linux-gnu/bin/ld: cannot find /lib/aarch64-linux-gnu/libc.so.6
/usr/lib/gcc-cross/aarch64-linux-gnu/9/../../../../aarch64-linux-gnu/bin/ld: cannot find /usr/lib/aarch64-linux-gnu/libc_nonshared.a
/usr/lib/gcc-cross/aarch64-linux-gnu/9/../../../../aarch64-linux-gnu/bin/ld: cannot find /lib/aarch64-linux-gnu/ld-linux-aarch64.so.1
collect2: error: ld returned 1 exit status
scons: *** [test_uint8_array] Error 1
解决方法:
-Wl选项放在-L选项之后, -->链接通过.
新问题:
如何设置scons ld链接选项顺序
3.scons如何修改链接选项顺序
官方文档 修改编译/链接选项顺序
# cc/gcc 编译选项顺序
CCCOM = '%CC %CFLAGS %_IFLAGS -c %< -o %>',
# c++/g++ 编译选项顺序
CXXCOM = '%CXX %CXXFLAGS %_IFLAGS -c %< -o %>',
# 链接选项顺序
LINKCOM = '%LINK %LDFLAGS -o %> %< %_LDIRS %LIBS',
修改LINKCOM
即可修改scons ld链接选项顺序
我的修改如下
SConstruct
# 1.修改 ld选项顺序: 使LINKFLAGS 在 _LIBDIRFLAGS 之后
# 2.自定义 ld前缀选项 LD_PREFLAGS
env_options = {
"LINKCOM" : "$LINK -o $TARGET $SOURCES $LD_PREFLAGS $_LIBDIRFLAGS $__RPATH $LINKFLAGS $_LIBFLAGS",
}
# 3.定义LD_PREFLAGS
env['LD_PREFLAGS'] = "--sysroot=%s " %(rootdir)
运行scons, 编译链接通过,如文章 3.实例: 交叉编译依赖ROS的程序
6.总结
- make, cmake, scons本质上是调用gcc/g++ 命令进行编译, 链接. 各种参数选项, 是为了配置gcc/g++选项.
如本次使用scons, 与手动执行 gcc/g++ 交叉编译链接成功.
那cmake各种参数设置, 也是为了调整编译/链接选项, 实现手动执行gcc命令的效果. - 又一次熟悉了gcc/g++ 编译,链接流程, 交叉编译环境配置.
- 猜测–rpath-link , 是gcc检查到host x86_64, target arch64, 不能使用–rpath包含路径.
7.未解决疑问, 问题
Q1: 如何打印显示gcc默认include路径?
Q2: 为什么gcc/g++ ld链接选项顺序, 对链接结果有影响?
- 默认ld链接选项顺序
-Wl,--rpath-link -Llib_dir -llib
失败 - 而
-Llib_dir -Wl,--rpath-link -llib
成功.
知道的小伙伴留言一下.
签名
一个喜欢机器人行业的 嵌入式-内核驱动-系统-中间件-网络-开发工程师
微信: liuj1637664504