全志T527-TP9930-Camera

一、简介

1、TP9930

TP9930 驱动模块主要实现将 4 路的 Camera 的数据转换为 BT656/BT1120 数据，从而实现在 T527 端来对数据进行处理和送显。

2、BT656/BT1120简介

BT656主要是针对PAL/NTSC等标清视频。随着高清视频的发展需要，又推出了BT1120标准，它与BT656是类似的，只不过时钟频率更高了，以适合高清视频的传输，例如1080P等。其实BT656与BT1120之间区别主要是传输数据的速率不同，例如BT656与BT1120都能传输720P的视频，但是BT656只能传输低帧率的720P，而BT1120能够传输高帧率的720P视频。

T5平台能够分别支持BT656与BT1120两种协议，只需要在驱动文件的sensor_formats[]中配置.mbus_code即可，而不需要改动其他配置参数

二、硬件信息

1、通信接口

TWI3总线

2、引脚组

3、RESET脚

4、IIC地址

5、供电

（1）摄像头电源使能

I2S2-MCLK>>>>>>>>>>>PE5

（2）DV5V0_SYS

UART2-RTS-JTAG-DO>>>>>>>>>>>>PB2

（3）DV3V3_SYS

PC7_SYS_STATUS_LED0>>>>>>>>>>>>PC7

6、BT1120引脚

配置如下：

"PK12", NCSI_PCLK

"PK14", NCSI_VSYNC

"PK15", NCSI_HSYNC

"PK16", NCS-D0

"PK17", NCS-D1

"PK18", NCS-D2

"PK19", NCS-D3

"PK20", NCS-D4

"PK21", NCS-D5

"PK22", NCS-D6

"PK23", NCS-D7

"PE6", NCS-D8

"PE7", NCS-D9

"PE8", NCS-D10

"PE9", NCS-D11

"PE10", NCS-D12

"PE11", NCS-D13

"PE12", NCS-D14

"PE15"; NCS-D15

    ncsi_bt1120_pins_a: ncsi_BT1120@0 {
            pins = "PK12", "PK14", "PK15",
            "PK16", "PK17", "PK18", "PK19",
            "PK20", "PK21", "PK22", "PK23",
            "PE6", "PE7", "PE8", "PE9",
            "PE10", "PE11", "PE12", "PE15";
            function = "ncsi";
            drive-strength = <20>;
                        };

    ncsi_bt1120_pins_b: ncsi_BT1120@1 {
            pins = "PK12", "PK14", "PK15",
            "PK16", "PK17", "PK18", "PK19",
            "PK20", "PK21", "PK22", "PK23",
            "PE6", "PE7", "PE8", "PE9",
            "PE10", "PE11", "PE12", "PE15";
             function = "gpio_in";
                        };

从上面原理图分析可知：

• 使用的是 BT1120 的模式。

• 使用的是 PE3，PE4 这一组 TWI3 的通讯接口。

• RESET 控制接口为 PK10，低电平有效，使用时拉高。

• 供电由DV3V3_SYS，DV5V0_SYS供电，I2S2-MCLK 为摄像头供电，需软件单独拉起，引脚分别为PC7、 PB2、 PE5。

三、软件配置

1、设备树

路径：device/config/chips/t527/configs/demo_linux_car/linux-5.15/board.dts

(1）使能 TWI3

（2）确认使用的是 PE3 和 PE4，然后确认 TWI3 是否正常使能。

(3)设备树：

 csi3:csi@5823000 {
                pinctrl-names = "default","sleep";
                pinctrl-0 = <&ncsi_bt1120_pins_a>;
                pinctrl-1 = <&ncsi_bt1120_pins_b>;
                status = "okay";
        };

        sensor1:sensor@5812010 {
                device_type = "sensor1";
                sensor1_mname = "tp9930";
                sensor1_twi_cci_id = <3>;//twi3
                sensor1_twi_addr = <0x8a>;
                sensor1_mclk_id = <3>;//2
                sensor1_pos = "front";
                sensor1_isp_used = <0>;
                sensor1_fmt = <0>;
                sensor1_stby_mode = <0>;
                sensor1_vflip = <0>;
                sensor1_hflip = <0>;
                sensor1_iovdd-supply = <>;
                sensor1_iovdd_vol = <>;
                sensor1_avdd-supply = <>;
                sensor1_avdd_vol = <>;
                sensor1_dvdd-supply = <>;
                sensor1_dvdd_vol = <>;
                sensor1_power_en = <>;
                sensor1_reset = <>;
                sensor1_reset = <>;
                sensor1_pwdn = <>;
                status = "okay";
        };

        vinc40:vinc@5834000 {
                vinc16_csi_sel = <3>;
                vinc16_mipi_sel = <0xff>;
                vinc16_isp_sel = <5>;
                vinc16_isp_tx_ch = <0>;
                vinc16_tdm_rx_sel = <0>;
                vinc16_rear_sensor_sel = <1>;
                vinc16_front_sensor_sel = <1>;
                vinc16_sensor_list = <0>;
                device_id = <16>;
                status = "okay";
        };

        vinc50:vinc@5835000 {
                vinc17_csi_sel = <3>;
                vinc17_mipi_sel = <0xff>;
                vinc17_isp_sel = <5>;
                vinc17_isp_tx_ch = <1>;
                vinc17_tdm_rx_sel = <0>;
                vinc17_rear_sensor_sel = <1>;
                vinc17_front_sensor_sel = <1>;
                vinc17_sensor_list = <0>;
                device_id = <17>;
                status = "okay";
        };

2、移植驱动、添加Makefile、Kconfig

驱动：tp9930.c

路径：bsp/drivers/vin/modules/sensor

（1）Makefile

（2）Kconfig

（3）./build.sh menuconfig勾选对应驱动

（4）保存./build.sh savaconfig

四、编译测试

1、video节点

配置挂载完后烧录：

2、sensor设备

cat /sys/class/video4linux/v4l-subdev1/name

3、应用程序测试

csi_test_usrptr 16 0 1280 720 /usr 4 10000 25

csi_test_usrptr 17 0 1280 720 /usr 4 10000 25

在T527 linux系统上调试TP9930双路输出时，在运行应用程序打开双路摄像头采集图像时出现程序崩溃，内容如下：

INFO   : cedarc <VeIniti[   86.954756]   FSC = 0x05: level 1 translation fault
alize:1549>: *** ic_version = 0x[   86.962996] Data abort info:
3321000012011,
01-01 00:01:52.3[   86.968993]   ISV = 0, ISS = 0x00000005
40 hwdisplay2(D) : libsdk_disp v[   86.976067]   CM = 0, WnR = 0
1.version:V2.1.20221031
open /d
ev/video17 fd = 6
[   45.439589] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000118
on:V2.1.20220906
[   86.982163] user pgtable: 4k pages, 39-bit VAs, pgdp=000000010a7f5000
[   86.992160] [0000000000000118] pgd=0000000000000000, p4d=0000000000000000, pud=0000000000000000
[   87.003563] Internal error: Oops: 96000005 [#1] PREEMPT SMP
[   87.009804] Modules linked in: vipcore 8821cs mali_kbase(O)
[   87.016056] CPU: 4 PID: 1216 Comm: csi_test_usrptr Tainted: G           O      5.15.123 #11
[   87.025418] Hardware name: sun55iw3 (DT)
[   87.029809] pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   87.037610] pc : mutex_lock+0xc/0x50
[   87.041619] lr : media_entity_setup_link+0x24/0x5c
[   87.046990] sp : ffffffc00e87baf0
[   87.050698] x29: ffffffc00e87baf0 x28: ffffff80c506a100 x27: 0000000000000000
[   87.058700] x26: 0000000000000000 x25: 0000000000000330 x24: ffffff80c0116d70
[   87.066697] x23: 0000000000000033 x22: ffffff80c3990080 x21: 0000000000000001
[   87.074698] x20: 0000000000000001 x19: ffffff80c3693830 x18: 0000000000000000
[   87.082696] x17: 0000000000000000 x16: 0000000000000000 x15: 0000007fc3770a58
[   87.090697] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
[   87.098697] x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000
[   87.106698] x8 : 0000000000000000 x7 : 0000000000000000 x6 : 0000000000000001
[   87.114698] x5 : ffffffc008c3be38 x4 : 0000000000000001 x3 : 0000000000000001
[   87.122697] x2 : ffffff80ca473fc0 x1 : 0000000000000000 x0 : 0000000000000118
[   87.130694] Call trace:
[   87.133427]  mutex_lock+0xc/0x50
[   87.137040]  __vin_sensor_setup_link+0xbc/0x130
[   87.142116]  __vin_s_input+0x74/0x390
[   87.146218]  vidioc_s_input+0x44/0x80
[   87.150319]  v4l_s_input+0x58/0x94
[   87.154127]  __video_do_ioctl+0x17c/0x3e0
[   87.158615]  video_usercopy+0x224/0x6f0
[   87.162908]  video_ioctl2+0x18/0x30
[   87.166811]  v4l2_ioctl+0x40/0x60
[   87.170521]  __arm64_sys_ioctl+0xb8/0xe0
[   87.174914]  invoke_syscall+0x54/0x124
[   87.179112]  el0_svc_common.constprop.0+0x44/0xec
[   87.184382]  do_el0_svc+0x40/0xa0
[   87.188091]  el0_svc+0x20/0x60
[   87.191508]  el0t_64_sync_handler+0xb0/0xb4
[   87.196194]  el0t_64_sync+0x1a0/0x1a4
[   87.200298] Code: d65f03c0 d2800001 d5384102 f9800011 (c85ffc03)
[   87.207126] ---[ end trace b99917f213e868dc ]---
[   87.212296] Kernel panic - not syncing: Oops: Fatal exception
[   87.218731] SMP: stopping secondary CPUs
[   87.223163] Kernel Offset: disabled
[   87.227065] CPU features: 0x3,00000201,23100e42
[   87.232138] Memory Limit: none
[   87.235558] ---[ end Kernel panic - not syncing: Oops: Fatal exception ]---

搜索找到了内核原函数，

bsp/drivers/vin/vin-video/vin_video.c:1954:

static int __vin_sensor_setup_link(struct vin_core *vinc, struct modules_config *module,

static int __vin_sensor_setup_link(struct vin_core *vinc, struct modules_config *module,
                    int i, int en)
{
    struct vin_md *vind = dev_get_drvdata(vinc->v4l2_dev->dev);
    struct v4l2_subdev *sensor = module->modules.sensor[i].sd;
    struct v4l2_subdev *subdev;
    struct media_entity *entity = NULL;
    struct media_link *link = NULL;
    __maybe_unused struct sensor_info *info = to_state(sensor);
    int ret;

    if (sensor == NULL)
        return -1;

    if (vinc->mipi_sel != 0xff)
        subdev = vind->mipi[vinc->mipi_sel].sd;
    else
        subdev = vind->csi[vinc->csi_sel].sd;

    entity = &sensor->entity;
    list_for_each_entry(link, &entity->links, list) {
        if (link->source->entity == entity && link->sink->entity == &subdev->entity)
                break;
    }
    if (link == NULL)
        return -1;
#if LINUX_VERSION_CODE >= KERNEL_VERSION(6, 1, 0)
    if (info->stream_count >= 1)
        return 0;
#else
    if (sensor->entity.stream_count >= 1)
        return 0;
#endif
    vin_log(VIN_LOG_VIDEO, "setup link: [%s] %c> [%s]\n",
        sensor->name, en ? '=' : '-', link->sink->entity->name);
    if (en)
        ret = media_entity_setup_link(link, MEDIA_LNK_FL_ENABLED);
    else
        ret = __media_entity_setup_link(link, 0);
        /* When the this function is called by the close
        function, the mutex conflicts with the close mutex,
        so the function without the mutex is used. */

    if (ret) {
        vin_warn("%s setup link %s fail!\n", sensor->name,
                link->sink->entity->name);
        return -1;
    }
    return 0;
}

在该代码中增加空指针检查和错误处理：

再次测试时出现打印

CSI subdev is NULL

设备树中配置为0xff，vinc16/17_mipi_sel = ;，执行subdev = vind->csi[vinc->csi_sel].sd;

问题出现在选择 CSI 子设备时，打印出了 "CSI subdev is NULL"。这种情况通常表示在尝试从 vind->csi[vinc->csi_sel].sd 中获取 CSI 子设备时，该指针为 NULL。

根据添加的打印信息得知，vind、vind->csi、vinc->csi_sel都不为空，检查csi设备的注册情况

搜索代码：定位到定义了一个名为 csi 的数组，每个元素的类型是 struct vin_csi_info。数组的大小是 VIN_MAX_CSI

修改为3进行测试

段错误问题解决，CSI 和 ISP 之间的链接时遇到了新的问题

定位代码：bsp/drivers/vin/vin-video/vin_video.c

添加打印，确认isp和csi 是否都非空

打印结果：

media_entity_find_link函数的实现

kernel/linux-5.15/drivers/media/mc/mc-entity.c

确认是否使用BT1120的模式，对应需要修改使用双边沿采样，具体修改位置在驱动sensor_g_mbus_config中，具体位置如下：

引脚 PE11 已经被 UART 设备占用，CSI 设备无法再次请求该引脚

找到对应的UART

关闭uart5、uart6节点

再次测试，对应的csi驱动未成功probe

定位代码：

路径：kernel/linux-5.15/drivers/base/dd.c +509

CSI驱动

路径：bsp/drivers/vin/vin-csi/sunxi_csi.c

......................

数组大小和probe有很大关系，尝试修改验证

CSI3驱动成功注册，认真看的同学应该发现，这一开始呗改为了3，现在又改回4了，是什么原因呢？

从上方我贴出的源码，VIN_MAX_CSI必须为4才能成功注册CSI3，最开始导致段错误最根本的原因不是这个，而是引脚复用导致的csi3的subdev未成功初始化。

测试：出现IIC通信问题，ACK not received

问题分析;

该i2c 报错为从设备无响应，即Sensor未正确回应，出现该问题的情况通常有如下几种。

1.VCC‑Sensor未上电，可用万用表确认。

2.初始化时序错误，导致Sensor未正常初始化，需检查Sensor驱动中初始化流程是否满足datasheet要求。

3.VCC‑Sensor有上电，初始化时序也正常，但是VCC‑Sensor上电有抖动导致Sensor异常。

4.VCC‑Sensor有上电，但Sensor初始化依赖的RST 或 INT 未正确拉高或拉低。

5.Sensor引脚未接好导致。

6.Sensor本身是坏的。

原因排查