网站导航
> 文档中心 > OpenHarmony相机用户态驱动框架

OpenHarmony相机用户态驱动框架

网友: 文档中心 2022-05-09 16:10:03

作者：润和软件 郭新星
相机作为智能手机上少有的成长空间不错的，能够做出差异化的功能，每年都能成为各大Android手机厂商争相宣传的亮点。众所周知Android采用Linux 作为其内核，而Linux采用的开源协议具有传染性[1]，导致Android HAL[2]成为了手机厂商们竞争的重要战场。随着OpenHarmony 3.1[3]的发布，相机模块也逐渐完善起来，目前提供了基础预览和拍照的能力。OpenHarmony中，相机用户态驱动框架承担了和Android Camera HAL一样的角色，这部分位于OpenHarmony的HDF[4]中，对上实现相机HDI[5]接口，对下实现相机Pipeline模型，管理相机各个硬件设备。

相机用户态驱动框架（下图的CameraHost 部分）总体可以分为三层，HDI实现层，实现相机标准南向接口；框架层，对接HDI实现层的控制、流的转发，实现数据通路的搭建、管理相机各个硬件设备等功能；适配层，屏蔽底层芯片和OS差异，支持多平台适配。

模块介绍
HDI Implementation：对上实现HDI接口，向下调用框架层的接口，完成HDI接口任务的转发。
Buffer Manager ： 屏蔽不同内存管理的差异，为子系统提供统一的操作接口，同时提供buffer轮转的功能。
Pipeline Core ：解析HCS配置完成pipeline的搭建，调度pipeline中的各个node完成流的处理
Device Manager：通过调用底层硬件适配层接口，实现查询控制底层设备、枚举监听底层设备的功能
Platform Adaption ：屏蔽硬件差异，为Device Manager提供统一的操作底层硬件的能力
•目录结构

Shelldrivers/peripheral/camera|-- README_zh.md|-- bundle.json|-- figures|   `-- logic-view-of-modules-related-to-this-repository_zh.png|-- hal|   |-- BUILD.gn|   |-- adapter|   |-- buffer_manager|   |-- camera.gni|   |-- device_manager|   |-- hdi_impl|   |-- include|   |-- init|   |-- pipeline_core|   |-- test|   `-- utils|-- hal_c|   |-- BUILD.gn|   |-- camera.gni|   |-- hdi_cif|   `-- include`-- interfaces    |-- hdi_ipc    |-- hdi_passthrough    `-- include

•HDI Implementation中的预览流程

接下来我们通过已经发布的OpenHarmony 3.1开源代码，来看看预览是怎么完成的吧

drivers/peripheral/camera/hal/test/v4l2/src /preview_test.cpp存放了针对v4l2的预览测试代码，入口如下

C++TEST_F(UtestPreviewTest, camera_preview_0001){    std::cout << "==========[test log] Preview stream, expected success." <AchieveStreamOperator(); // 获取stream operator    // start stream    display_->intents = {Camera::PREVIEW}; // 预览流    display_->StartStream(display_->intents); // 起流    // Get preview    display_->StartCapture(display_->streamId_preview, display_->captureId_preview, false, true);    // release stream    display_->captureIds = {display_->captureId_preview};    display_->streamIds = {display_->streamId_preview};    display_->StopStream(display_->captureIds, display_->streamIds);}

先获取stream operator实例

C++void testdisplay::achievestreamoperator(){    // create and get streamoperator information    std::shared_ptr streamoperatorcallback = std::make_shared();    rc = cameradevice->getstreamoperator(streamoperatorcallback, streamoperator);// ........}

通过前文的streamOperator创建流

C++void TestDisplay::StartStream(std::vector intents){    // ..............................    for (auto& intent : intents) { if (intent == 0) {     std::shared_ptr producer = IBufferProducer::CreateBufferQueue();     producer->SetQueueSize(8); // 创建buffer的生产端，并和相应的流进行绑定     auto callback = [this](std::shared_ptr Prebuffer) {  BufferCallback(Prebuffer, preview_mode);  return;     };     producer->SetCallback(callback);     streamInfo->streamId_ = streamId_preview;     streamInfo->width_ = 640; // 640:picture width     streamInfo->height_ = 480; // 480:picture height     streamInfo->format_ = CAMERA_FORMAT_YUYV_422_PKG;     streamInfo->datasapce_ = 8; // 8:picture datasapce     streamInfo->intent_ = intent;     streamInfo->tunneledMode_ = 5; // 5:tunnel mode     streamInfo->bufferQueue_ = producer;     streamInfos.push_back(streamInfo); } else if (intent == 1) {      // .......................    }    rc = streamOperator->CreateStreams(streamInfos); // 创建流    // ................................    rc = streamOperator->CommitStreams(Camera::NORMAL, ability); // 提交流    // .................................}

下面我们正式进入到hal的源代码中看看是怎么创建流的吧

C++CamRetCode StreamOperator::CreateStreams(const std::vector<std::shared_ptr>& streamInfos){ // .....    for (auto it : streamInfos) {//.... std::shared_ptr stream = StreamFactory::Instance().CreateShared(     IStream::g_availableStreamType[it->intent_], it->streamId_, it->intent_, pipelineCore_, messenger_); // 创建流实例// ... StreamConfiguration scg; scg.id = it->streamId_; scg.type = it->intent_; scg.width = it->width_; scg.height = it->height_; PixelFormat pf = static_cast(it->format_); scg.format = BufferAdapter::PixelFormatToCameraFormat(pf); scg.dataspace = it->datasapce_; scg.tunnelMode = it->tunneledMode_; scg.minFrameDuration = it->minFrameDuration_; scg.encodeType = it->encodeType_; RetCode rc = stream->ConfigStream(scg); // 依据上文的流信息配置流// ... if (it->bufferQueue_ != nullptr) {  // 绑定前文的生产端     auto tunnel = std::make_shared();     CHECK_IF_PTR_NULL_RETURN_VALUE(tunnel, INSUFFICIENT_RESOURCES);     RetCode rc = tunnel->AttachBufferQueue(it->bufferQueue_);     CHECK_IF_NOT_EQUAL_RETURN_VALUE(rc, RC_OK, INVALID_ARGUMENT);     if (stream->AttachStreamTunnel(tunnel) != RC_OK) {  CAMERA_LOGE("attach buffer queue to stream [id = %{public}d] failed", it->streamId_);  return INVALID_ARGUMENT;     } } {     std::lock_guard l(streamLock_);     streamMap_[stream->GetStreamId()] = stream; // 保存流实例 }// ...}

从上面可以看出，消费端传递到了hal，那必然是由hal从bufferproducer获取buffer，并触发预览的启动流程。那看看AttachStreamTunnel 的实现吧

C++RetCode StreamBase::AttachStreamTunnel(std::shared_ptr& tunnel){    if (state_ == STREAM_STATE_BUSY || state_ == STREAM_STATE_OFFLINE) { return RC_ERROR;    }    tunnel_ = tunnel; // 绑定生产端    CHECK_IF_PTR_NULL_RETURN_VALUE(tunnel_, RC_ERROR);    tunnel_->SetBufferCount(GetBufferCount()); // 配置轮转的buffer个数    TunnelConfig config = {(uint32_t)streamConfig_.width, (uint32_t)streamConfig_.height, (uint32_t)streamConfig_.format, streamConfig_.usage};    tunnel_->Config(config);    streamConfig_.tunnelMode = true;    return RC_OK;}

CreateStream之后便是CommitStream，这里的CommitStream 做了些什么事情呢，我们接着往下看

C++CamRetCode StreamOperator::CommitStreams(OperationMode mode,      const std::shared_ptr& modeSetting){// ......    std::vector configs = {};    { std::lock_guard l(streamLock_); for (auto it : streamMap_) { // 获取流的配置，前文CreateStrea时保存的流     configs.emplace_back(it.second->GetStreamAttribute()); }    }// 检查流是否被支持    DynamicStreamSwitchMode method = streamPipeline_->CheckStreamsSupported(mode, modeSetting, configs);    if (method == DYNAMIC_STREAM_SWITCH_NOT_SUPPORT) { return INVALID_ARGUMENT;    }    if (method == DYNAMIC_STREAM_SWITCH_NEED_INNER_RESTART) { std::lock_guard l(streamLock_); for (auto it : streamMap_) {     it.second->StopStream();// 如果流被支持，但需要内部重启，这里先停流 }    }    { std::lock_guard l(streamLock_); for (auto it : streamMap_) {     if (it.second->CommitStream() != RC_OK) { // 真正的 CommitStream，下面再细说  CAMERA_LOGE("commit stream [id = %{public}d] failed.", it.first);  return DEVICE_ERROR;     } }    }    RetCode rc = streamPipeline_->PreConfig(modeSetting); // 把模式传入进行预配置    if (rc != RC_OK) { CAMERA_LOGE("prepare mode settings failed"); return DEVICE_ERROR;    }    rc = streamPipeline_->CreatePipeline(mode);// 创建pipeline    if (rc != RC_OK) { CAMERA_LOGE("create pipeline failed."); return INVALID_ARGUMENT;    }    DFX_LOCAL_HITRACE_END;    return NO_ERROR;}
C++RetCode StreamBase::CommitStream(){// ...    hostStreamMgr_ = pipelineCore_->GetHostStreamMgr(); //从pipelinecore获取hoststreamanager    CHECK_IF_PTR_NULL_RETURN_VALUE(hostStreamMgr_, RC_ERROR);// ... info.bufferPoolId_ = poolId_; info.bufferCount_ = GetBufferCount();    //  初始化 bufferpool RetCode rc = bufferPool_->Init(streamConfig_.width, streamConfig_.height, streamConfig_.usage,    streamConfig_.format, GetBufferCount(), CAMERA_BUFFER_SOURCE_TYPE_EXTERNAL); if (rc != RC_OK) {     CAMERA_LOGE("stream [id:%{public}d] initialize buffer pool failed.", streamId_);     return RC_ERROR; }    }// stream传递到pipelinecore 并进行绑定    RetCode rc = hostStreamMgr_->CreateHostStream(info, [this](std::shared_ptr buffer) { HandleResult(buffer); return;    });// ....    return RC_OK;}

CreateStream 和CommitStream结束之后便是Capture，这里包含了起流的动作，关键实现如下

C++CamRetCode StreamOperator::Capture(int captureId, const std::shared_ptr& captureInfo, bool isStreaming){// ...//  captureId 捕获请求的id; captureInfo 预览/拍照/录像的参数；isStreaming 连续捕获还是单次捕获（拍照）    CaptureSetting setting = captureInfo->captureSetting_;    auto request = std::make_shared(captureId, captureInfo->streamIds_.size(), setting,      captureInfo->enableShutterCallback_, isStreaming);    for (auto id : captureInfo->streamIds_) { // 创建捕获请求，并传递给前文创建的流 RetCode rc = streamMap_[id]->AddRequest(request); if (rc != RC_OK) {     return DEVICE_ERROR; }    }// ...}

从上面的代码可知预览、拍照、录像都是通过捕获请求触发，单次拍照则为单次捕获请求，预览和录像则是连续捕获请求。

C++RetCode StreamBase::AddRequest(std::shared_ptr& request){    CHECK_IF_PTR_NULL_RETURN_VALUE(request, RC_ERROR);    request->AddOwner(shared_from_this());    request->SetFirstRequest(false);    if (isFirstRequest) { RetCode rc = StartStream(); // 起流 if (rc != RC_OK) {     CAMERA_LOGE("start stream [id:%{public}d] failed", streamId_);     return RC_ERROR; } request->SetFirstRequest(true); isFirstRequest = false;    }    { std::unique_lock l(wtLock_); waitingList_.emplace_back(request); // 捕获请求添加到waitingList cv_.notify_one();    }    return RC_OK;}

看看StreamStream是怎么实现的吧

C++RetCode StreamBase::StartStream(){// ...    RetCode rc = pipeline_->Prepare({streamId_}); //  pipeline先完成一些准备工作// ...    state_ = STREAM_STATE_BUSY;    std::string threadName = g_availableStreamType[static_cast(streamType_)] + "#" + std::to_string(streamId_);    handler_ = std::make_unique([this, &threadName] {// 创建轮转线程 prctl(PR_SET_NAME, threadName.c_str()); while (state_ == STREAM_STATE_BUSY) {     HandleRequest(); // 处理捕获请求 }    });// ...    rc = pipeline_->Start({streamId_}); // 通知pipeline和底层硬件可以开始出帧了// ...    return RC_OK;}
C++void StreamBase::HandleRequest(){    // 如果有 捕获请求下发，则退出等待状态    if (waitingList_.empty()) { std::unique_lock l(wtLock_); if (waitingList_.empty()) {     cv_.wait(l, [this] { return !(state_ == STREAM_STATE_BUSY && waitingList_.empty()); }); }    }// ... request = waitingList_.front(); CHECK_IF_PTR_NULL_RETURN_VOID(request); if (!request->IsContinous()) { // 如果是连续捕获，则保留一份拷贝在waitinglist     waitingList_.pop_front(); }    }// 处理捕获请求    request->Process(streamId_);// 最终调用下面的Capture接口    return;}
C++RetCode StreamBase::Capture(const std::shared_ptr& request){    CHECK_IF_PTR_NULL_RETURN_VALUE(request, RC_ERROR);    CHECK_IF_PTR_NULL_RETURN_VALUE(pipeline_, RC_ERROR);    RetCode rc = RC_ERROR;    if (request->IsFirstOne() && !request->IsContinous()) { uint32_t n = GetBufferCount(); for (uint32_t i = 0; i NeedCancel()) {// 被取消的捕获则退出 CAMERA_LOGE("StreamBase::Capture stream [id:%{public}d] request->NeedCancel", streamId_); return RC_OK;    }    rc = pipeline_->Config({streamId_}, request->GetCaptureSetting());// 通知pipeline配置    if (rc != RC_OK) { CAMERA_LOGE("stream [id:%{public}d] config pipeline failed.", streamId_); return RC_ERROR;    }    rc = pipeline_->Capture({streamId_}, request->GetCaptureId());//  这里的capture指的是pipeline中的source node开始回buffer    { std::unique_lock l(tsLock_); inTransitList_.emplace_back(request);// 处理过的捕获请求存放在inTransitList    }    return RC_OK;}

到这起流的流程就结束了，pipeline回上来的帧通过OnFrame接口处理

C++RetCode StreamBase::OnFrame(const std::shared_ptr& request){// ...    bool isEnded = false;    if (!request->IsContinous()) { isEnded = true;    } else if (request->NeedCancel()) { isEnded = true;    }    { // inTransitList_ may has multiple copies of continious-capture request, we just need erase one of them. std::unique_lock l(tsLock_); for (auto it = inTransitList_.begin(); it != inTransitList_.end(); it++) {     if ((*it) == request) {  inTransitList_.erase(it);// 已经回帧的请求，从inTransitList删除  break;     } } if (isEnded) {     // if this is the last request of capture, send CaptureEndedMessage.     auto it = std::find(inTransitList_.begin(), inTransitList_.end(), request);     if (it == inTransitList_.end()) {  std::shared_ptr endMessage =      std::make_shared(streamId_, request->GetCaptureId(), request->GetEndTime(),  request->GetOwnerCount(), tunnel_->GetFrameCount());  CAMERA_LOGV("end of stream [%d], ready to send end message, capture id = %d",      streamId_, request->GetCaptureId());  messenger_->SendMessage(endMessage);  pipeline_->CancelCapture({streamId_});// 如果此次捕获结束，则取消捕获     } }    }    ReceiveBuffer(buffer);// 底层返回的buffer送还到生产端，最终帧数据送到消费端    return RC_OK;}

附录

  1. linux和Android的关系 - 知乎 (zhihu.com) ↩︎

  2. HAL Subsystem | Android Open Source Project (google.cn) ↩︎

  3. zh-cn/release-notes/OpenHarmony-v3.1-release.md · OpenHarmony/docs - Gitee.com ↩︎

  4. OpenHarmony HDF 驱动框架介绍和驱动加载过程分析-OpenHarmony技术社区-51CTO.COM ↩︎

  5. OpenHarmony HDF HDI基础能力分析与使用-51CTO.COM ↩︎

好看的书法字体下载

网络标签：

相关问题