安装qemu

sudo apt-get install qemu

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace$ dpkg -l | grep qemu
ii  ipxe-qemu                                  1.21.1+git-20220113.fbbdc3926-0ubuntu1  all          PXE boot firmware - ROM images for qemu
ii  ipxe-qemu-256k-compat-efi-roms             1.0.0+git-20150424.a25a16d-0ubuntu4     all          PXE boot firmware - Compat EFI ROM images for qemu
ii  libvirt-daemon-driver-qemu                 8.0.0-1ubuntu7.3                        amd64        Virtualization daemon QEMU connection driver
ii  qemu                                       1:6.2+dfsg-2ubuntu6.6                   amd64        fast processor emulator, dummy package
ii  qemu-block-extra                           1:6.2+dfsg-2ubuntu6.5                   amd64        extra block backend modules for qemu-system and qemu-utils
ii  qemu-system-common                         1:6.2+dfsg-2ubuntu6.5                   amd64        QEMU full system emulation binaries (common files)
ii  qemu-system-data                           1:6.2+dfsg-2ubuntu6.5                   all          QEMU full system emulation (data files)
ii  qemu-system-gui                            1:6.2+dfsg-2ubuntu6.5                   amd64        QEMU full system emulation binaries (user interface and audio support)
ii  qemu-system-x86                            1:6.2+dfsg-2ubuntu6.5                   amd64        QEMU full system emulation binaries (x86)
ii  qemu-utils                                 1:6.2+dfsg-2ubuntu6.5                   amd64        QEMU utilities
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace$

创建qemu目录

运行qemu需要编译内核和busybox的源码，最好单独建立一个目录专门作为qemu运行环境目录

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment$//本文使用的是这个目录，此目录下将会存放内核源码以及busybox源码和生成的文件系统等

编译内核

1. 下载linux源码包linux-4.19.90.tar.gz（下载链接：kernel.org），并解压到qemu-environment目录：tar -xvf linux-4.19.90.tar.gz
2. 配置内核.config文件：
   （1） 先用默认配置文件创建.config文件

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/linux-4.19.90$ make defconfig
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  YACC    scripts/kconfig/zconf.tab.c
  LEX     scripts/kconfig/zconf.lex.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf
*** Default configuration is based on 'x86_64_defconfig'
#
# configuration written to .config
#
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/linux-4.19.90$ make menuconfig

（2）配置ram选项，在device drivers->block devices下，选中ram block device support，并改大default ram disk size到128MB（64MB一般也够用了）

3. 编译内核

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/linux-4.19.90$ make -j16
  ... ...
  OBJCOPY arch/x86/boot/setup.bin
  BUILD   arch/x86/boot/bzImage
Setup is 15772 bytes (padded to 15872 bytes).
System is 8153 kB
CRC 328649af
Kernel: arch/x86/boot/bzImage is ready  (#1)

4. 可能遇到的编译错误：
   （1）objtool: missing symbol table

arch/x86/entry/thunk_64.o: warning: objtool: missing symbol table
make[2]: *** [scripts/Makefile.build:403：arch/x86/entry/thunk_64.o] 错误 1

解决方法：按照对应的补丁进行修改后重新编译，https://www.spinics.net/lists/kernel/msg3797871.html

（2） multiple definition

ld: arch/x86/boot/compressed/pgtable_64.o:(.bss+0x0): multiple definition of `__force_order'; arch/x86/boot/compressed/kaslr_64.o:(.bss+0x0): first defined here
ld: arch/x86/boot/compressed/head_64.o: warning: relocation in read-only section `.head.text'
ld: warning: creating DT_TEXTREL in a PIE
make[2]: *** [arch/x86/boot/compressed/Makefile:116：arch/x86/boot/compressed/vmlinux] 错误 1
make[1]: *** [arch/x86/boot/Makefile:112：arch/x86/boot/compressed/vmlinux] 错误 2
make: *** [arch/x86/Makefile:292：bzImage] 错误 2

解决方法：将arch/x86/boot/compressed/pgtable_64.c文件里__force_order定义注释掉

编译文件系统

1. 下载busybox：https://busybox.net/

2. 解压busybox-1.35.0.tar.bz2到目录

3. busybox目录里make menuconfig：
   （1） 在settings下配置成静态编译
   （2） 在settings下配置busybox安装目录，高版本busybox默认配置了生成目录在_install里
   可以改成自己想要的目录，这里选择让busybox在上一级目录里新建一个文件夹qemu_sample安装进去
   

4. 编译busybox，并安装

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/busybox-1.35.0$ make -j16
  SPLIT   include/autoconf.h -> include/config/*
  GEN     include/bbconfigopts.h
  GEN     include/common_bufsiz.h
  GEN     include/embedded_scripts.h
  HOSTCC  applets/usage
  HOSTCC  applets/applet_tables
  ... ...
   AR      libbb/lib.a
  LINK    busybox_unstripped
Static linking against glibc, can't use --gc-sections
Trying libraries: crypt m resolv rt
 Library crypt is not needed, excluding it
 Library m is needed, can't exclude it (yet)
 Library resolv is needed, can't exclude it (yet)
 Library rt is not needed, excluding it
 Library m is needed, can't exclude it (yet)
 Library resolv is needed, can't exclude it (yet)
Final link with: m resolv
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/busybox-1.35.0$ make install

5. 安装成功后，查看下qemu-environment目录：

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/busybox-1.35.0$ ls ../
busybox-1.35.0  linux-4.19.90  qemu-sample

6. 在生成的文件系统内创建必要的目录和文件：
   （1） 创建目录

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ ls
bin  linuxrc  sbin  usr
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ mkdir dev etc lib mnt proc sys tmp var
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ ls
bin  dev  etc  lib  linuxrc  mnt  proc  sbin  sys  tmp  usr  var
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ mkdir -p etc/init.d
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$

（2） 创建文件

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ touch etc/init.d/rcS
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ vim etc/init.d/rcS 
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ chmod 777 etc/init.d/rcS
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ cat etc/init.d/rcS
echo "Welcome to linux..."
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ touch etc/passwd
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ vim etc/passwd 
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ chmod 777 etc/passwd 
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ cat etc/passwd 
root::0:0:root:/:/bin/sh
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ touch etc/inittab
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ vim etc/inittab 
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ chmod 777 etc/inittab 
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ cat etc/inittab 
::sysinit:/etc/init.d/rcS
::respawn:-/bin/sh
::askfirst:-/bin/sh
::ctrlaltdel:/bin/umount -a -r
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ touch etc/fstab
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ vim etc/fstab 
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ chmod 777 etc/fstab 
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ cat etc/fstab 
proc    /proc   proc    defaults        0       0
tmpfs   /tmp    tmpfs   defaults        0       0
sysfs   /sys    sysfs   defaults        0       0
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample$ cd dev
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample/dev$ sudo mknod console c 5 1
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample/dev$ sudo mknod null c 1 3
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment/qemu-sample/dev$  sudo mknod tty1 c 4 1

7. 创建raw ext4 image文件，用/dev/zero初始化（/dev/zero这个设备文件提供无限个’\0’字符，可用于初始化一些文件）。dd指令表示输入文件是/dev/zero，输出文件是./rootfs.ext4，输入输出的块大小均是1M Byte，拷贝32个1M块。也可以创建更大的image文件，只要小于前面编译内核时设置的ramdisk大小128MB就行。

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment$ dd if=/dev/zero of=./rootfs.ext4 bs=1M count=32
记录了32+0 的读入
记录了32+0 的写出
33554432字节（34 MB，32 MiB）已复制，0.0484468 s，693 MB/s
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment$ mkfs.ext4 rootfs.ext4
mke2fs 1.46.5 (30-Dec-2021)
丢弃设备块： 完成                            
创建含有 8192 个块（每块 4k）和 8192 个 inode 的文件系统

正在分配组表： 完成                            
正在写入 inode表： 完成                            
创建日志（1024 个块）： 完成
写入超级块和文件系统账户统计信息： 已完成

8. 创建一个目录，将image文件挂载到这个目录从而便于操作，往image文件里拷贝前面制作的根文件系统

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment$ mkdir fs
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment$ sudo mount -o loop rootfs.ext4 ./fs
rokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment$ sudo cp -rf ./qemu-sample/* ./fs
krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment$ sudo umount ./fs

9. 压缩image文件成img.gz格式

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment$ gzip --best -c rootfs.ext4 > rootfs.img.gz

启动qemu

krokodil@krokodil-SY-ZL-H110N-D3V:~/workspace/qemu-environment$ qemu-system-x86_64 -kernel linux-4.19.90/arch/x86/boot/bzImage -initrd ./rootfs.img.gz

ps：
如果是在arm64架构上，则略有不同。
make -j16后不会生成bzImage文件（因为arch/arm64/Makefile与arch/x86/Makefile不同），但是会有arch/arm64/boot/Image文件，也可以启动。
启动qemu指令略不同：

qemu-system-aarch64 -kernel linux-4.19.90/arch/arm64/boot/Image -initrd rootfs.img.gz -machine virt -cpu cortex-a57 -machine type=virt -nographic

qemu通过-drive启动：

dd if=/dev/zero of=/home/krokodil/workspace/qemu-environment/share.img bs=1M count=350
mkfs.ext4 /home/krokodil/workspace/qemu-environment/share.img
mount -o loop /home/krokodil/workspace/qemu-environment/share.img /home/krokodil/workspace/qemu-environment/fs
sudo cp -r /home/krokodil/workspace/qemu-environment/qemu-sample/* /home/krokodil/workspace/qemu-environment/fs/
cd /home/krokodil/workspace/qemu-environment/linux-4.19.90
sudo make CONFIG_PREFIX=/home/krokodil/workspace/qemu-environment/fs install
cd /home/krokodil/workspace/qemu-environment/
qemu-system-x86_64 -kernel linux-4.19.90/arch/x86/boot/bzImage -drive file=./share.img -append "init=/linuxrc root=/dev/sda"