前言

本篇博客是想介绍Android camera从application layer到camera service layer整个框架中，能够使用的所有接口方式。

主要有哪几种interface？

分为两大类：Android SDK，Android NDK。
Android NDK分为NDK和NDK vendor。对于camera module Android SDK可以分为camera API 1和camera API 2。最后有一种特殊的方式是用过HIDL interface来实现camera APP，给出总体架构图：

Android SDK

Camera API 1

• API 1作为最原始的一套camera API现在已经被Google启用，已经不在有功能上的升级，可以提供基础capture pictures and videos in your applications。API 1流程非常的特殊，他是通过JNI layer调用到libcameraservice，如果想了解JNI移步：链接: Camera Java Native Interface(JNI)介绍。
• 接下来通过camera open代码分析JNI调用过程

frameworks/base/core/java/android/hardware/Camera.java
public static Camera open() {
    int numberOfCameras = getNumberOfCameras();
    CameraInfo cameraInfo = new CameraInfo();
    for (int i = 0; i < numberOfCameras; i++) {
        getCameraInfo(i, cameraInfo);
        if (cameraInfo.facing == CameraInfo.CAMERA_FACING_BACK) {
            return new Camera(i);
        }
    }
    return null;
}

如果在camera APP中调用API 1的open function去打开默认后置camera，一开始会新建一个camera device object，直接调用camera构造函数。

frameworks/base/core/java/android/hardware/Camera.java
Camera(int cameraId) {
    if(cameraId >= getNumberOfCameras()){
            throw new RuntimeException("Unknown camera ID");
    }
    int err = cameraInit(cameraId);
	…
    initAppOps();
}

在camera 构造函数中调用cameraInit。

frameworks/base/core/java/android/hardware/Camera.java
private int cameraInit(int cameraId) {
	…
	//调用到JNI中android_hardware_Camera_native_setup
    return native_setup(new WeakReference<Camera>(this), cameraId,
            ActivityThread.currentOpPackageName());
}

在camera Init中通过native_setup interface就会调用到JNI中android_hardware_Camera_native_setup function。

frameworks/base/core/jni/android_hardware_Camera.cpp
static jint android_hardware_Camera_native_setup(JNIEnv *env, jobject thiz,
    jobject weak_this, jint cameraId, jstring clientPackageName)
{
    const char16_t *rawClientName = reinterpret_cast<const char16_t*>(
        env->GetStringChars(clientPackageName, NULL));
    jsize rawClientNameLen = env->GetStringLength(clientPackageName);
    String16 clientName(rawClientName, rawClientNameLen);
    env->ReleaseStringChars(clientPackageName,
                            reinterpret_cast<const jchar*>(rawClientName));
    int targetSdkVersion = android_get_application_target_sdk_version();
    sp<Camera> camera = Camera::connect(cameraId, clientName, Camera::USE_CALLING_UID,
                                        Camera::USE_CALLING_PID, targetSdkVersion);
	…
    sp<JNICameraContext> context = new JNICameraContext(env, weak_this, clazz, camera);
    context->incStrong((void*)android_hardware_Camera_native_setup);
    camera->setListener(context);
    return NO_ERROR;
}

• android_hardware_Camera_native_setup 中connect是Camera C/S架构的客户端，调用connect函数向服务器发送连接请求。
• JNICameraContext这个类是一个监听类，用于处理底层Camera回调函数传来的数据和消息。

frameworks/av/camera/Camera.cpp
sp<Camera> Camera::connect(int cameraId, const String16& clientPackageName,
        int clientUid, int clientPid, int targetSdkVersion)
{
    return CameraBaseT::connect(cameraId, clientPackageName, clientUid,
            clientPid, targetSdkVersion);
}

JNI调用进入C++ framework camera file connect function。

frameworks/av/camera/CameraBase.cpp
template <typename TCam, typename TCamTraits>
sp<TCam> CameraBase<TCam, TCamTraits>::connect(int cameraId,
                                               const String16& clientPackageName,
                                               int clientUid, int clientPid, int targetSdkVersion)
{
    ALOGV("%s: connect", __FUNCTION__);
	//创建一个camera对象，调用Camera和CameraBase构造函数
    sp<TCam> c = new TCam(cameraId);
    sp<TCamCallbacks> cl = c;
	//获取camera service对象
    const sp<::android::hardware::ICameraService> cs = getCameraService();
    binder::Status ret;
    if (cs != nullptr) {
        TCamConnectService fnConnectService = TCamTraits::fnConnectService;
        ret = (cs.get()->*fnConnectService)(cl, cameraId, clientPackageName, clientUid,
                                               clientPid, targetSdkVersion, /*out*/ &c->mCamera);
    }
	…
    return c;
}

CameraBase中通过获得到server端（camera service）代理对象ICameraService。
然后ICameraService通过fnConnectService链接camera service。

frameworks/av/camera/CameraBase.cpp
// establish binder interface to camera service
template <typename TCam, typename TCamTraits>
const sp<::android::hardware::ICameraService> CameraBase<TCam, TCamTraits>::getCameraService()
{
    Mutex::Autolock _l(gLock);
    if (gCameraService.get() == 0) {
        if (CameraUtils::isCameraServiceDisabled()) {
            return gCameraService;
        }
		//通过binder获取camera service
        sp<IServiceManager> sm = defaultServiceManager();
        sp<IBinder> binder;
        do {
            binder = sm->getService(String16(kCameraServiceName));
            if (binder != 0) {
                break;
            }
            ALOGW("CameraService not published, waiting...");
            usleep(kCameraServicePollDelay);
        } while(true);
        if (gDeathNotifier == NULL) {
            gDeathNotifier = new DeathNotifier();
        }
        binder->linkToDeath(gDeathNotifier);
        gCameraService = interface_cast<::android::hardware::ICameraService>(binder);
    }
    ALOGE_IF(gCameraService == 0, "no CameraService!?");
    return gCameraService;
}

getCameraService function中主要是展示通过binder的方法获取到camera service代理对象。
为什么fnConnectService可以链接camera device呢？因为fnConnectService就等于connect，如下代码：

frameworks/av/camera/Camera.cpp
CameraTraits<Camera>::TCamConnectService CameraTraits<Camera>::fnConnectService =
        &::android::hardware::ICameraService::connect;

而connect本身定义为AIDL interface，就是通过binder实现跨进程调用到camera service，在camera service中进一步处理。

frameworks/av/camera/aidl/android/hardware/ICameraService.aidl
AIDL 文件中定义的connect interface，目的为跨进程调用
ICamera connect(ICameraClient client,
        int cameraId,
        String opPackageName,
        int clientUid, int clientPid,
        int targetSdkVersion);

到此camera API 1的流程就梳理完毕，JNI在其中做一个桥梁的作用，API 1通过JNI连接到C++ framework，也展示JNI的目的就是为让java field可以调用到C++ field。

Camera API 2

• 为更好的控制camera device，Google抛弃camera API 1重新设计camera API 2接口。
• Camera2 provides in-depth controls for complex use cases, but requires you to manage device-specific configurations。
• Camera API2 框架为应用提供更接近底层的相机控件，包括高效的零复制连拍/视频流以及曝光、增益、白平衡增益、颜色转换、去噪、锐化等方面的每帧控件。
• camera API 2调用流程相对简单，直接从API 2 interface通过AIDL调用到camera service，接下来也是举例openCamera流程进行分析。

frameworks/base/core/java/android/hardware/camera2/CameraManager.java
public void openCamera(@NonNull String cameraId,
        @NonNull final CameraDevice.StateCallback callback, @Nullable Handler handler)
        throws CameraAccessException {
    openCameraForUid(cameraId, callback, CameraDeviceImpl.checkAndWrapHandler(handler),
            USE_CALLING_UID);
}

public void openCameraForUid(@NonNull String cameraId,
        @NonNull final CameraDevice.StateCallback callback, @NonNull Executor executor,
        int clientUid, int oomScoreOffset) throws CameraAccessException {
	…
    openCameraDeviceUserAsync(cameraId, callback, executor, clientUid, oomScoreOffset);
}

APP通过API 2 open camera操作可以调用到camera manager中的openCamera function，进而调用到openCameraForUid、openCameraDeviceUserAsync。

frameworks/base/core/java/android/hardware/camera2/CameraManager.java
private CameraDevice openCameraDeviceUserAsync(String cameraId,
        CameraDevice.StateCallback callback, Executor executor, final int uid,
        final int oomScoreOffset) throws CameraAccessException {
    CameraCharacteristics characteristics = getCameraCharacteristics(cameraId);
    CameraDevice device = null;
    Map<String, CameraCharacteristics> physicalIdsToChars =
            getPhysicalIdToCharsMap(characteristics);
    synchronized (mLock) {
        ICameraDeviceUser cameraUser = null;
        android.hardware.camera2.impl.CameraDeviceImpl deviceImpl =
                new android.hardware.camera2.impl.CameraDeviceImpl(
                    cameraId,
                    callback,
                    executor,
                    characteristics,
                    physicalIdsToChars,
                    mContext.getApplicationInfo().targetSdkVersion,
                    mContext);
        ICameraDeviceCallbacks callbacks = deviceImpl.getCallbacks();
        try {
            ICameraService cameraService = CameraManagerGlobal.get().getCameraService();
            if (cameraService == null) {
                throw new ServiceSpecificException(
                    ICameraService.ERROR_DISCONNECTED,
                    "Camera service is currently unavailable");
            }
            cameraUser = cameraService.connectDevice(callbacks, cameraId,
                mContext.getOpPackageName(),  mContext.getAttributionTag(), uid,
                oomScoreOffset, mContext.getApplicationInfo().targetSdkVersion);
        } catch (ServiceSpecificException e) {
		…
     }
    return device;
}

openCameraDeviceUserAsync function中通过getCameraService获取到camera service代理ICameraService，这部分操作是在class CameraManagerGlobal中实现。

frameworks/base/core/java/android/hardware/camera2/CameraManager.java
class CameraManagerGlobal {
	public ICameraService getCameraService() {
	    synchronized(mLock) {
	        connectCameraServiceLocked();
	        if (mCameraService == null && !sCameraServiceDisabled) {
	            Log.e(TAG, "Camera service is unavailable");
	        }
	        return mCameraService;
	    }
	}

	private void connectCameraServiceLocked() {
	    // Only reconnect if necessary
	    if (mCameraService != null || sCameraServiceDisabled) return;
	    Log.i(TAG, "Connecting to camera service");
	    IBinder cameraServiceBinder = ServiceManager.getService(CAMERA_SERVICE_BINDER_NAME);
	    if (cameraServiceBinder == null) {
	        // Camera service is now down, leave mCameraService as null
	        return;
	    }
	    try {
	        cameraServiceBinder.linkToDeath(this, /*flags*/ 0);
	    } catch (RemoteException e) {
	        // Camera service is now down, leave mCameraService as null
	        return;
	    }
		//直接通过AIDL 获取camera service
	    ICameraService cameraService = ICameraService.Stub.asInterface(cameraServiceBinder);
		…
	}
}

• global manger camera class主要作为保持和camera service的链接，并且distribute camera device callback notice。
• getCameraService调用到connectCameraServiceLocked，可以很清楚的看到通过binder获取到cameraService。
• 因为connectDevice interface本身定义为AIDL，跨进程的方式从java framework调用到camera service中。

frameworks/av/camera/aidl/android/hardware/ICameraService.aidl
 ICameraDeviceUser connectDevice(ICameraDeviceCallbacks callbacks,
         String cameraId,
         String opPackageName,
         @nullable String featureId,
         int clientUid, int oomScoreOffset,
         int targetSdkVersion);

以上就讲解完通过 API 2方式从APP调用到camera service。

小结

• 虽然Android SDK中的API 1和API 2都是通过AIDL调用到camera service，但是分别是在C++ code和java code中调用，这里就体现AIDL这种接口描述性语言的优势，不需要考虑binder service和client两端的编程语言，只需要按照要求编写好AIDL interface，编译之后自动生成代码，在使用的时候按接口标准就可以实现。

Android NDK

NDK Interface

• NDK的全拼是：Native Develop Kit，Android NDK 是一套允许您使用原生代码语言(例如C和C++)实现部分应用的工具集。在开发某些类型应用时，这有助于您重复使用以这些语言编写的代码库。
• 我们来分析要是使用NDK的方式来编写camera APP进程的代码，open camera的流程是怎么样的。

frameworks/av/camera/ndk/NdkCameraManager.cpp
camera_status_t ACameraManager_openCamera(
        ACameraManager* mgr, const char* cameraId,
        ACameraDevice_StateCallbacks* callback,
        /*out*/ACameraDevice** device) {
    return mgr->openCamera(cameraId, callback, device);
}

在system分区中通过使用C++代码编写的camera APP，open camera调用的是NDK中的ACameraManager_openCamera，进一步进入camera manager openCamera function。

frameworks/av/camera/ndk/NdkCameraManager.cpp
camera_status_t
ACameraManager::openCamera(
        const char* cameraId,
        ACameraDevice_StateCallbacks* callback,
        /*out*/ACameraDevice** outDevice)
{
    sp<hardware::ICameraService> cs = CameraManagerGlobal::getInstance().getCameraService();
    sp<hardware::camera2::ICameraDeviceCallbacks> callbacks = device->getServiceCallback();
    sp<hardware::camera2::ICameraDeviceUser> deviceRemote;
    int targetSdkVersion = android_get_application_target_sdk_version();
	//调用CameraService::connectDevice
    binder::Status serviceRet = cs->connectDevice(
            callbacks, String16(cameraId), String16(""), {},
            hardware::ICameraService::USE_CALLING_UID, /*oomScoreOffset*/0,
            targetSdkVersion, /*out*/&deviceRemote);
	...
}

openCamera function中先通过getCameraService获取到camera service代理对象（其中代码流程同上，就不分析了），再通过connectDevice AIDL interface 跨进程连接到camera service。

NDK Vendor Interface

• VNDK的全称是Vendor Native Development Kit，是Android 8.0引入的一种新技术。它表现一系列库的合集，用于让供应商开发自己的HALs。VNDK 包含在 system.img 中，并在运行时与供应商代码动态关联。
• 使用vndk的原因是自Android8.0以来，Google引入了Treble架构，希望对vendor和system分区进行解耦处理，期待实现：framwork进程不加载vendor共享库，vendor进程仅加载vendor共享库（和部分framework共享库），而framework进程和vendor进程之间通过HIDL和hwbinder来通信
• 这里也是通过open camera function，来分析在vendor分区中编写C++ 代码的camera APP 调用到camera service。

frameworks/av/camera/ndk/ndk_vendor/impl/ACameraManager.cpp
camera_status_t
ACameraManager::openCamera(
        const char* cameraId,
        ACameraDevice_StateCallbacks* callback,
        /*out*/ACameraDevice** outDevice) {
    sp<ACameraMetadata> rawChars;
    camera_status_t ret = getCameraCharacteristics(cameraId, &rawChars);
    Mutex::Autolock _l(mLock);
    if (ret != ACAMERA_OK) {
        ALOGE("%s: cannot get camera characteristics for camera %s. err %d",
                __FUNCTION__, cameraId, ret);
        return ACAMERA_ERROR_INVALID_PARAMETER;
    }
    ACameraDevice* device = new ACameraDevice(cameraId, callback, std::move(rawChars));
    sp<ICameraService> cs = CameraManagerGlobal::getInstance().getCameraService();
    if (cs == nullptr) {
        ALOGE("%s: Cannot reach camera service!", __FUNCTION__);
        delete device;
        return ACAMERA_ERROR_CAMERA_DISCONNECTED;
    }
    sp<ICameraDeviceCallback> callbacks = device->getServiceCallback();
    sp<ICameraDeviceUser_2_0> deviceRemote_2_0;
	//通过调用HidlCameraService中connectDevice，hidl方式
    Status status = Status::NO_ERROR;
    auto serviceRet = cs->connectDevice(
            callbacks, cameraId, [&status, &deviceRemote_2_0](auto s, auto &device) {
                                     status = s;
                                     deviceRemote_2_0 = device;
                                 });
	…
    sp<ICameraDeviceUser> deviceRemote = castResult;
    device->setRemoteDevice(deviceRemote);
    device->setDeviceMetadataQueues();
    *outDevice = device;
    return ACAMERA_OK;
}

这里直接给出的是NDK Vendor中的openCamera function，但是这个函数也是被ACameraManager_openCamera调用。
其中mgr->openCamera是调用NDK还是NDK Vendor中的interface呢？需要通过mCameraManager来确定。

ACameraManager_openCamera(mCameraManager, mCameraId, &mDeviceCb, &mDevice);
mCameraManager = ACameraManager_create();
ACameraManager* ACameraManager_create() {
    ATRACE_CALL();
    return new ACameraManager();
}

• ACameraManager_openCamera调用时会传入mCameraManager，mCameraManager是由ACameraManager_create创建出来。ACameraManager_create 创建过程直接new ACameraManager对象。

NDK ACameraManager：
struct ACameraManager {
    ACameraManager() :
            mGlobalManager(&(android::acam::CameraManagerGlobal::getInstance())) {}
    ~ACameraManager();
    camera_status_t getCameraIdList(ACameraIdList** cameraIdList);
    static void     deleteCameraIdList(ACameraIdList* cameraIdList);

    camera_status_t getCameraCharacteristics(
            const char* cameraId, android::sp<ACameraMetadata>* characteristics);
    camera_status_t openCamera(const char* cameraId,
                               ACameraDevice_StateCallbacks* callback,
                               /*out*/ACameraDevice** device);

  private:
    enum {
        kCameraIdListNotInit = -1
    };
    android::Mutex         mLock;
    android::sp<android::acam::CameraManagerGlobal> mGlobalManager;
};

NDK Vendor ACameraManager：
struct ACameraManager {
    ACameraManager() :
            mGlobalManager(&(android::acam::CameraManagerGlobal::getInstance())) {}
    ~ACameraManager();
    camera_status_t getCameraIdList(ACameraIdList** cameraIdList);
    static void     deleteCameraIdList(ACameraIdList* cameraIdList);

    camera_status_t getCameraCharacteristics(
            const char* cameraId, android::sp<ACameraMetadata>* characteristics);

    camera_status_t openCamera(const char* cameraId,
                               ACameraDevice_StateCallbacks* callback,
                               /*out*/ACameraDevice** device);
    camera_status_t getTagFromName(const char *cameraId, const char *name, uint32_t *tag);

  private:
    enum {
        kCameraIdListNotInit = -1
    };
    android::Mutex         mLock;
    android::sp<android::acam::CameraManagerGlobal> mGlobalManager;
};

• new ACameraManager需要区分是调用NDK还是NDK Vendor创建对象，不同的对象进而实现不同的调用接口逻辑。
• 再回来看openCamera function中的connectDevice interface，其中传递的参数和NDK代码中明显不同，因为NDK Vendor调用的是HIDL 定义connectDevice interface。

frameworks/hardware/interfaces/cameraservice/service/2.0/ICameraService.hal
connectDevice(ICameraDeviceCallback callback, string cameraId)
    generates (Status status, ICameraDeviceUser device);

• ICameraService.hal文件中定义的HIDL connectDevice interface。
• cs->connectDevice通过binder跨进程可以调用到HidlCameraService::connectDevice中。

小结

• 以上分析了通过NDK和NDK Vendor方式，connect device连接到camera service的代码流程，区别在于NDK使用AIDL interface而NDK Vendor使用 HIDL interface，并且在libcameraservice中对接的接口也不一样。
• System分区中new ACameraManager实例调用NDK interface，Vendor 分区中new ACameraManager实例调用NDK Vendor interface。
• HidlCameraService对上连接NDK Vendor，但是最终实现功能还是依赖于CameraService。

Camera VTS Testcase

• HIDL interface提供的接口是可以直接实现camera device所有基础的功能，但是由于没有 camera service的介入对于管理camera device是非常困难的。使用这种方式构建camera APP非常的特殊，没有经过整个Android framework，可以直接对HAL进行测试。
• Camera VTS testcase 是基于HDIL interface来实现功能的测试程序，毕竟是Google提供给HAL 厂商稳定的接口，肯定需要测试保证其稳定性。
• 对于如果是camera service process没有启动的情况下，可以使用HDIL interface来写camera APP，感兴趣可以移步: Camera HIDL接口实现camera preview功能

总结

• 这篇博客主要是对Android framework接口方式进行介绍，其中包括Android SDK和Android NDK，Android SDK可以分为API 1和API 2，其中API 1通过JNI的方式调用实现。
• JNI调用的cameraBase是和NDK、NDK vendor处于同一目录下，framework中的audio和video服务层（av目录名字含义）。
• CameraBase、API 2和NDK都是通过AIDL跨进程调用到libcameraservice，而只有NDK Vendor独特的使用HIDL跨进程调用到libcameraservice。

Reference

链接: Android JNI(一)——NDK与JNI基础