【Yolov8部署】 VS2019 + opencv - dnn CPU环境下部署目标检测模型_vs2019 使用 opencvsharp 调用yolov8模型

文章目录

• 前言
• 一、导出yolov8模型为onnx文件
• 二、VS2019配置及opencv环境配置
• 三、opencv部署
• 总结

前言

本文主要研究场景为工业场景下，在工控机与工业相机环境中运行的视觉缺陷检测系统，因此本文主要目的为实现c++环境下，将yolov8已训练好的检测模型部署在opencv中通过cpu进行检测运算

一、导出yolov8模型为onnx文件

Yolo官方支持onnx的使用，直接使用yolo的官方库即可
配置好训练权重文件路径，输出参数按自己需求查手册配置

from ultralytics import YOLO# Load a model#model = YOLO(\"yolov8n.pt\") # Load an official modelmodel = YOLO(r\"D:\\deep_learning\\YOLOv8.2\\runs\\train\\exp9\\weights\\best.pt\") # Load a custom trained model# Export the modelsuccess = model.export(format=\"onnx\", opset=11, simplify=True)

输出成功的文件会在权重文件同目录下

这里生成的文件为11.6MB，过小的onnx文件可能是生成失败产生的

获取了onnx文件后接下来配置调用onnx文件的部分

二、VS2019配置及opencv环境配置

VS2019在微软官方网站上下载即可

opencv的版本有很多，但经过多次测试发现在cpu场景下要成功调用dnn模块至少需要4.8.0版本
opencv 4.5.4版本
opencv 4.5.5版本
opencv 4.7.0版本
经测试都会在读取onnx文件时发生致命错误

cv::dnn::readNetFromONNX(modelPath);

该条函数在调用时会直接抛出异常终止程序

0x00007FFC088CB699 处(位于 OpencvYoloYest.exe 中)有未经处理的异常: Microsoft C++ 异常: cv::Exception，位于内存位置 0x0000002AF32FB5B0 处。

opencv需要在官网下载，官网提供X64的release包可以直接下载提取使用
也可通过cmake编译cuda版opencv或是x86的动态库来使用
opencv官方releases下载地址
选取Windows版下载即可，或下载Sources包使用cmake手动编译

本文仅记录采用opencv 4.8.0版本

opencv的环境配置主要分为两部分
系统变量设置：


VS2019动态库配置：
注意C++标准需要选择为ISO C++17标准

包含目录与库目录配置

在链接器的输入附加依赖项中添加opencv动态库
注意Debug版本需要将该opencv_world480.lib改为opencv_world480d.lib

跨平台或老旧工控机需要x86版本则需通过cmake手动编译win32版本
至此环境配置就完成了，接下来是库的调用，即在opencv中的部署

三、opencv部署

本文提供代码，同时也可通过yolo官方处下载代码
链接与代码如下：
YOLOv8-CPP-Inference
inference.cpp

#include \"inference.h\"Inference::Inference(const std::string onnxModelPath, const cv::Size modelInputShape= cv::Size(640, 640), const std::string classesTxtFile= \"classes.txt\", const bool runWithCuda=false){ modelPath = onnxModelPath; modelShape = modelInputShape; classesPath = classesTxtFile; cudaEnabled = runWithCuda; loadOnnxNetwork(); //loadClassesFromFile(); The classes are hard-coded for this example}std::vector<Detection> Inference::runInference(const cv::Mat& input){ cv::Mat modelInput = input; if (letterBoxForSquare && modelShape.width == modelShape.height) modelInput = formatToSquare(modelInput); cv::Mat blob; cv::dnn::blobFromImage(modelInput, blob, 1.0 / 255.0, modelShape, cv::Scalar(), true, false); net.setInput(blob); std::vector<cv::Mat> outputs; net.forward(outputs, net.getUnconnectedOutLayersNames()); //cv::Mat cpuOutput; //outputs[0].copyTo(cpuOutput); // 将数据从 GPU 复制到 CPU 的 cv::Mat 对象中 //float* data = reinterpret_cast(outputs.data); // 将数据赋值给 float* 指针 int rows = outputs[0].size[1]; int dimensions = outputs[0].size[2]; bool yolov8 = true; // yolov5 has an output of shape (batchSize, 25200, 85) (Num classes + box[x,y,w,h] + confidence[c]) // yolov8 has an output of shape (batchSize, 84, 8400) (Num classes + box[x,y,w,h]) if (dimensions > rows) // Check if the shape[2] is more than shape[1] (yolov8) { yolov8 = true; rows = outputs[0].size[2]; dimensions = outputs[0].size[1]; outputs[0] = outputs[0].reshape(1, dimensions); cv::transpose(outputs[0], outputs[0]); } //if (cv::cuda::getCudaEnabledDeviceCount() > 0) { // 检查是否启用了GPU计算 //  // cv::cuda::GpuMat gpuData(outputs[0]); // 将 GPU 数据包装到 cv::cuda::GpuMat 中 // cv::Mat cpuData; // gpuData.download(cpuData); // 将 GPU 数据下载到 CPU 的 cv::Mat 中 // float* data = (float*)cpuData.data; // 获取 CPU 上的数据指针 //} //else { // 在没有启用GPU计算时，直接使用CPU内存中的数据指针 // data = (float*)outputs[0].data; //} //float* data = (float*)outputs[0].data; //************************GPU和CPU的数据交换 //cv::UMat umatData = outputs[0].getUMat(cv::ACCESS_READ); //cv::Mat cpuData; //umatData.copyTo(cpuData); //******************************** //float* data = (float*)cpuData.data; float* data = (float*)outputs[0].data; float x_factor = modelInput.cols / modelShape.width; float y_factor = modelInput.rows / modelShape.height; std::vector<int> class_ids; std::vector<float> confidences; std::vector<cv::Rect> boxes; for (int i = 0; i < rows; ++i) { if (yolov8) { float* classes_scores = data + 4; cv::Mat scores(1, classes.size(), CV_32FC1, classes_scores); cv::Point class_id; double maxClassScore; minMaxLoc(scores, 0, &maxClassScore, 0, &class_id); if (maxClassScore > modelScoreThreshold) { confidences.push_back(maxClassScore); class_ids.push_back(class_id.x); float x = data[0]; float y = data[1]; float w = data[2]; float h = data[3]; int left = int((x - 0.5 * w) * x_factor); int top = int((y - 0.5 * h) * y_factor); int width = int(w * x_factor); int height = int(h * y_factor); boxes.push_back(cv::Rect(left, top, width, height)); } } else // yolov5 { float confidence = data[4]; if (confidence >= modelConfidenceThreshold) { float* classes_scores = data + 5; cv::Mat scores(1, classes.size(), CV_32FC1, classes_scores); cv::Point class_id; double max_class_score; minMaxLoc(scores, 0, &max_class_score, 0, &class_id); if (max_class_score > modelScoreThreshold) {  confidences.push_back(confidence);  class_ids.push_back(class_id.x);  float x = data[0];  float y = data[1];  float w = data[2];  float h = data[3];  int left = int((x - 0.5 * w) * x_factor);  int top = int((y - 0.5 * h) * y_factor);  int width = int(w * x_factor);  int height = int(h * y_factor); } } } data += dimensions; } std::vector<int> nms_result; cv::dnn::NMSBoxes(boxes, confidences, modelScoreThreshold, modelNMSThreshold, nms_result); std::vector<Detection> detections{}; for (unsigned long i = 0; i < nms_result.size(); ++i) { int idx = nms_result[i]; Detection result; result.class_id = class_ids[idx]; result.confidence = confidences[idx]; std::random_device rd; std::mt19937 gen(rd()); std::uniform_int_distribution<int> dis(100, 255); result.color = cv::Scalar(dis(gen), dis(gen), dis(gen)); result.className = classes[result.class_id]; result.box = boxes[idx]; detections.push_back(result); } return detections;}void Inference::loadClassesFromFile(){ std::ifstream inputFile(classesPath); if (inputFile.is_open()) { std::string classLine; while (std::getline(inputFile, classLine)) classes.push_back(classLine); inputFile.close(); }}void Inference::loadOnnxNetwork(){ net = cv::dnn::readNetFromONNX(modelPath); if (cudaEnabled) { std::cout << \"\\nRunning on CUDA\" << std::endl; net.setPreferableBackend(cv::dnn::DNN_BACKEND_CUDA); net.setPreferableTarget(cv::dnn::DNN_TARGET_CUDA_FP16); } else { std::cout << \"\\nRunning on CPU\" << std::endl; net.setPreferableBackend(cv::dnn::DNN_BACKEND_OPENCV); net.setPreferableTarget(cv::dnn::DNN_TARGET_CPU); }}cv::Mat Inference::formatToSquare(const cv::Mat& source){ int col = source.cols; int row = source.rows; int _max = MAX(col, row); cv::Mat result = cv::Mat::zeros(_max, _max, CV_8UC3); source.copyTo(result(cv::Rect(0, 0, col, row))); return result;}

inference.h

#ifndef INFERENCE_H#define INFERENCE_H// Cpp native#include #include #include #include // OpenCV / DNN / Inference#include #include #include //Detection结构体用来保存目标检测的结果实例struct Detection{ int class_id{ 0 };//整形变量用来存储检测到的目标的类别，默认值为0 std::string className{};//字符串变量用来存储检测到的目标的名称，默认值为空字符串 float confidence{ 0.0 };//目标检测的置信度（即对目标存在的确定程度）。默认值为0.0。 cv::Scalar color{};//OpenCV库中的Scalar类型变量，用于存储颜色信息。它可以表示RGB、BGR或灰度颜色空间中的颜色 cv::Rect box{}; //cv::Rect 类型包含四个成员变量：x、y、width 和 height};//Infrence类用来执行目标检测class Inference{public: //构造函数（modelInputShape是指模型的大小，默认为640,640;classesTxtFile是类别名称的文本文件路径（可选参数，默认为空字符串）;runWithCuda是一个布尔值，指示是否使用CUDA加速运行（可选参数，默认为true）。 Inference(std::string onnxModelPath, cv::Size modelInputShape, std::string classesTxtFile, bool runWithCuda); //公有成员函数，用于执行目标检测推断。它接受一个cv::Mat类型的输入图像，并返回一个std::vector类型的检测结果。该函数将执行目标检测算法，将检测到的目标信息封装到Detection结构体中，并将所有检测结果存储在一个向量中。 std::vector<Detection> runInference(const cv::Mat& input); //私有成员函数，用于内部操作private: //loadClassesFromFile函数从文本文件中加载类别名称 void loadClassesFromFile(); //loadOnnxNetwork函数加载ONNX模型 void loadOnnxNetwork(); //formatToSquare函数将输入图像调整为正方形形状。 cv::Mat formatToSquare(const cv::Mat& source); //这些是私有成员变量 std::string modelPath{};//存储模型文件路径 std::string classesPath{};//类别文件路径 bool cudaEnabled{};//CUDA加速的状态 //字符串向量，用于存储目标检测的类别名称。默认情况下，它包含了一些通用的目标类别名称 std::vector<std::string> classes{ \"bright_collision\", \"dark_collision\" }; //std::vector classes{ \"person\", \"bicycle\", \"car\", \"motorcycle\", \"airplane\", \"bus\", \"train\", \"truck\", \"boat\", \"traffic light\", \"fire hydrant\", \"stop sign\", \"parking meter\", \"bench\", \"bird\", \"cat\", \"dog\", \"horse\", \"sheep\", \"cow\", \"elephant\", \"bear\", \"zebra\", \"giraffe\", \"backpack\", \"umbrella\", \"handbag\", \"tie\", \"suitcase\", \"frisbee\", \"skis\", \"snowboard\", \"sports ball\", \"kite\", \"baseball bat\", \"baseball glove\", \"skateboard\", \"surfboard\", \"tennis racket\", \"bottle\", \"wine glass\", \"cup\", \"fork\", \"knife\", \"spoon\", \"bowl\", \"banana\", \"apple\", \"sandwich\", \"orange\", \"broccoli\", \"carrot\", \"hot dog\", \"pizza\", \"donut\", \"cake\", \"chair\", \"couch\", \"potted plant\", \"bed\", \"dining table\", \"toilet\", \"tv\", \"laptop\", \"mouse\", \"remote\", \"keyboard\", \"cell phone\", \"microwave\", \"oven\", \"toaster\", \"sink\", \"refrigerator\", \"book\", \"clock\", \"vase\", \"scissors\", \"teddy bear\", \"hair drier\", \"toothbrush\" }; //这是一个OpenCV库中的Size2f类型变量，用于存储模型的输入形状（宽度和高度）。 cv::Size2f modelShape{}; //设置目标检测的阈值 float modelConfidenceThreshold{ 0.50 };//目标置信度的阈值 float modelScoreThreshold{ 0.45 };//目标得分的阈值 float modelNMSThreshold{ 0.50 };//非最大抑制的阈值 //布尔变量，指示是否使用letterbox技术将输入图像调整为正方形形状 bool letterBoxForSquare = true; //该类封装了目标检测推断的相关操作和参数，通过调用构造函数和成员函数，你可以加载模型、执行推断，并获取目标检测的结果 cv::dnn::Net net;//penCV库中的Net类型变量，用于存储加载的目标检测网络模型};#endif // INFERENCE_H

opencv_yolo.cpp

#include \"opencv_yolo.h\"OpencvYoloPt::OpencvYoloPt(Net_config config) {this->confThreshold = config.confThreshold;this->nmsThreshold = config.nmsThreshold;this->objThreshold = config.objThreshold;this->net = readNet(config.modelpath);std::ifstream ifs(\"class.names\"); //class.name中写入标签内容，当前只有\'person\'，位置与当前.cpp文件同级std::string line;while (getline(ifs, line)) this->class_names.push_back(line);this->num_class = class_names.size();if (endsWithYolo(config.modelpath, \"6.onnx\")) { //根据onnx的输入图像格式 选择分辨率 当前为1280x1280 可灵活调整anchors = (float*)anchors_1280;this->num_stride = 4; //深度this->inpHeight = 1280; //高this->inpWidth = 1280; //宽}else {  //当前为640x640 可以resize满足onnx需求 也可以调整数组或if中的onnx判断anchors = (float*)anchors_640;this->num_stride = 3;this->inpHeight = 640;this->inpWidth = 640;}}OpencvYoloPt::~OpencvYoloPt(){}Mat OpencvYoloPt::resize_image(Mat srcimg, int* newh, int* neww, int* top, int* left) {//传入图像以及需要改变的参数int srch = srcimg.rows, srcw = srcimg.cols;*newh = this->inpHeight;*neww = this->inpWidth;Mat dstimg;if (this->keep_ratio && srch != srcw) {float hw_scale = (float)srch / srcw;if (hw_scale > 1) {*newh = this->inpHeight;*neww = int(this->inpWidth / hw_scale);resize(srcimg, dstimg, Size(*neww, *newh), INTER_AREA);*left = int((this->inpWidth - *neww) * 0.5);copyMakeBorder(dstimg, dstimg, 0, 0, *left, this->inpWidth - *neww - *left, BORDER_CONSTANT, 114);}else {*newh = (int)this->inpHeight * hw_scale;*neww = this->inpWidth;resize(srcimg, dstimg, Size(*neww, *newh), INTER_AREA);*top = (int)(this->inpHeight - *newh) * 0.5;copyMakeBorder(dstimg, dstimg, *top, this->inpHeight - *newh - *top, 0, 0, BORDER_CONSTANT, 114);}}else {resize(srcimg, dstimg, Size(*neww, *newh), INTER_AREA);}return dstimg;}void OpencvYoloPt::drawPred(float conf, int left, int top, int right, int bottom, Mat& frame, int classid) {//绘制一个显示边界框的矩形rectangle(frame, Point(left, top), Point(right, bottom), Scalar(0, 0, 255), 2);//获取类名的标签及其置信度std::string label = format(\"%.2f\", conf);label = this->class_names[classid] + \":\" + label;//在边界框顶部显示标签int baseLine;Size labelSize = getTextSize(label, FONT_HERSHEY_SIMPLEX, 0.5, 1, &baseLine);top = max(top, labelSize.height);//rectangle(frame, Point(left, top - int(1.5 * labelSize.height)), Point(left + int(1.5 * labelSize.width), top + baseLine), Scalar(0, 255, 0), FILLED);putText(frame, label, Point(left, top), FONT_HERSHEY_SIMPLEX, 0.75, Scalar(0, 255, 0), 1);}void OpencvYoloPt::detect(Mat& frame) {int newh = 0, neww = 0, padh = 0, padw = 0;Mat dstimg = this->resize_image(frame, &newh, &neww, &padh, &padw);Mat blob = blobFromImage(dstimg, 1 / 255.0, Size(this->inpWidth, this->inpHeight), Scalar(0, 0, 0), true, false);this->net.setInput(blob);std::vector<Mat> outs;this->net.forward(outs, this->net.getUnconnectedOutLayersNames());int num_proposal = outs[0].size[1];int nout = outs[0].size[2];if (outs[0].dims > 2) {outs[0] = outs[0].reshape(0, num_proposal);}//生成提案std::vector<float> confidences;std::vector<Rect> boxes;std::vector<int> classIds;float ratioh = (float)frame.rows / newh, ratiow = (float)frame.cols / neww;int n = 0, q = 0, i = 0, j = 0, row_ind = 0; //xmin,ymin,xamx,ymax,box_score,class_scorefloat* pdata = (float*)outs[0].data;for (n = 0; n < this->num_stride; n++) { //特征图尺度const float stride = pow(2, n + 3);int num_grid_x = (int)ceil((this->inpWidth / stride));int num_grid_y = (int)ceil((this->inpHeight / stride));for (q = 0; q < 3; q++) {const float anchor_w = this->anchors[n * 6 + q * 2];const float anchor_h = this->anchors[n * 6 + q * 2 + 1];for (i = 0; i < num_grid_y; i++) {for (j = 0; j < num_grid_x; j++) {float box_score = pdata[4];if (box_score > this->objThreshold) {Mat scores = outs[0].row(row_ind).colRange(5, nout);Point classIdPoint;double max_class_socre;//获取最高分的值和位置minMaxLoc(scores, 0, &max_class_socre, 0, &classIdPoint);max_class_socre *= box_score;if (max_class_socre > this->confThreshold) {const int class_idx = classIdPoint.x;float cx = pdata[0]; //cxfloat cy = pdata[1]; //cyfloat w = pdata[2]; //wfloat h = pdata[3]; //hint left = int((cx - padw - 0.5 * w) * ratiow);int top = int((cy - padh - 0.5 * h) * ratioh);confidences.push_back((float)max_class_socre);boxes.push_back(Rect(left, top, (int)(w * ratiow), (int)(h * ratioh)));classIds.push_back(class_idx);}}row_ind++;pdata += nout;}}}}// 执行非最大抑制以消除冗余重叠框// 置信度较低std::vector<int> indices;dnn::NMSBoxes(boxes, confidences, this->confThreshold, this->nmsThreshold, indices);for (size_t i = 0; i < indices.size(); ++i) {int idx = indices[i];Rect box = boxes[idx];this->drawPred(confidences[idx], box.x, box.y, box.x + box.width, box.y + box.height, frame, classIds[idx]);}}

opencv_yolo.h

#pragma once#include  //文件#include  //流#include #include #include  //深度学习模块-仅提供推理功能#include  //图像处理模块#include  //媒体的输入输出/视频捕捉/图像和视频的编码解码/图形界面的接口using namespace cv;using namespace dnn;using namespace std;const float anchors_640[3][6] = { {10.0, 13.0, 16.0, 30.0, 33.0, 23.0}, {30.0, 61.0, 62.0, 45.0, 59.0, 119.0}, {116.0, 90.0, 156.0, 198.0, 373.0, 326.0} };const float anchors_1280[4][6] = { {19, 27, 44, 40, 38, 94}, {96, 68, 86, 152, 180, 137}, {140, 301, 303, 264, 238, 542}, {436, 615, 739, 380, 925, 792} };struct Net_config {float confThreshold; // 置信度阈值float nmsThreshold; // 非最大抑制阈值float objThreshold; // 对象置信度阈值std::string modelpath;};class OpencvYoloPt{public:OpencvYoloPt(Net_config config); //构造函数~OpencvYoloPt();void detect(Mat& frame); //通过图像参数，进行目标检测private:float* anchors;int num_stride;int inpWidth;int inpHeight;std::vector<String> class_names;int num_class;float confThreshold;float nmsThreshold;float objThreshold;const bool keep_ratio = true;Net net;void drawPred(float conf, int left, int top, int right, int bottom, Mat& frame, int classid);Mat resize_image(Mat srcimg, int* newh, int* neww, int* top, int* left);int endsWithYolo(std::string s, std::string sub) {return s.rfind(sub) == (s.length() - sub.length()) ? 1 : 0;}};

main.cpp

#include \"inference.h\"#include \"opencv_yolo.h\"int main(int argc, char *argv[]){ //main中调用 //执行视频检测算法和处理 bool runOnGPU = false; int deviceId = 0; // 指定要使用的GPU设备的索引 //cv::cuda::setDevice(deviceId);//本机为amd卡，无法安装cuda所以此语句屏蔽 //1. 设置你的onnx模型 //注意，在这个例子中类别是硬编码的，\'classes.txt\'只是一个占位符。 Inference inf(\"D:/VsEnvironment/dataSet/best.onnx\", cv::Size(640, 640), \"classes.txt\", runOnGPU); // classes.txt 可以缺失 string imgpath = \"D:/VsEnvironment/dataSet/deep_screw_roi/test/images/Image_20241126164807270.bmp\"; cv::Mat frame = imread(imgpath); std::vector<Detection> output = inf.runInference(frame); int detections = output.size(); std::cout << \"Number of detections: \" << detections << std::endl; for (int i = 0; i < detections; ++i) { Detection detection = output[i]; cv::Rect box = detection.box; cv::Scalar color = detection.color; //根据类别不同，显示不同的颜色 if (detection.class_id == 0) { color = cv::Scalar(0, 255, 0); // 红色 (B, G, R) } else if (detection.class_id == 1) { color = cv::Scalar(0, 0, 255); // 红色 (B, G, R) } else { color = cv::Scalar(255, 0, 0); // 红色 (B, G, R) } // Detection box cv::rectangle(frame, box, color, 2); // Detection box text std::string classString = detection.className + \' \' + std::to_string(detection.confidence).substr(0, 4); cv::Size textSize = cv::getTextSize(classString, cv::FONT_HERSHEY_DUPLEX, 1, 2, 0); cv::Rect textBox(box.x, box.y - 40, textSize.width + 10, textSize.height + 20); cv::rectangle(frame, textBox, color, cv::FILLED); cv::putText(frame, classString, cv::Point(box.x + 5, box.y - 10), cv::FONT_HERSHEY_DUPLEX, 1, cv::Scalar(0, 0, 0), 2, 0); } cv::namedWindow(\"Inference\", cv::WINDOW_NORMAL); // 创建具有可调整大小功能的窗口 cv::imshow(\"Inference\", frame); // 在窗口中显示图像 cv::destroyAllWindows(); }

总结

如下为成功运行例程，检测图像涉及科研项目，此处不便展示

以后还将完成VS2019+opencv+onnxruntime cpu与gpu的cuda加速环境下部署目标检测模型的测试，同时上传csdn文章