鸿蒙进阶——Mindspore Lite AI框架源码解读之模型加载详解(四)

文章大纲引言一 、LiteSession::LoadModelAndCompileByPath二、LiteSession::LoadModelAndCompileByPath 详细流程1、char *lite::LiteSession::LoadModelByPath(const std::string file, mindspore::ModelType model_type, size_t *size, bool use_mmap)1.1、mindspore::ModelType lite::LiteSession::LoadModelByBuff(const char *model_buf, const size_t buf_size, char **lite_buf, size_t *size, mindspore::ModelType model_type)2、PackWeightManager::InitPackWeightManager(const char *model_buf, size_t model_size, std::string *model_id, std::string *runner_id,const std::mapstd::string, std::mapstd::string, std::string *config_info)2.1、PackWeight::InitPackWeight(const void *model_buf, size_t model_size, std::string id, int numa_id, bool need_copy_buf)3、lite::ImportFromBuffer4、LiteSession::CompileGraph(Model *model)引言前一篇文章介绍了Mindspore Lite AI 的核心LiteSession 的部分Init流程这一篇接着往下介绍LiteSession处理Model对象的后续流程。一 、LiteSession::LoadModelAndCompileByPath加载指定路径的模型文件并对其进行编译。代码实现了从文件系统中读取模型数据初始化管理模型权重的工具处理模型的内存管理并最终编译为可执行的图形结构。用户可以方便地将已训练的模型加载到 LiteSession 中。从应用层传递过来的模型文件的路径会在LiteSession 被转为对应的Buffer 并封装到相应的Model对象中传入size切分Bufferintlite::LiteSession::LoadModelAndCompileByPath(conststd::stringmodel_path,mindspore::ModelType model_type){size_t model_size;booluse_mmapIsMmapEnable();//触发LoadModelByBuff函数本质上类型直接转换automodel_bufLoadModelByPath(model_path,model_type,model_size,use_mmap);...//是根据配置初始化权重管理器并在需要时支持共享模型权重的功能。autostatuslite::PackWeightManager::GetInstance()-InitPackWeightManager(model_buf,model_size,model_id_,runner_id_,config_info_);autonew_model_buflite::PackWeightManager::GetInstance()-GetSharedModelBuf(model_buf,model_id_,config_info_,is_shared_weight_);...auto*modellite::ImportFromBuffer(new_model_buf,model_size,true,model_type,model_path);...(reinterpret_castlite::LiteModel*(model))-set_keep_model_buf(true);autoretCompileGraph(model);...set_model(model);returnRET_OK;}二、LiteSession::LoadModelAndCompileByPath 详细流程1、char *lite::LiteSession::LoadModelByPath(const std::string file, mindspore::ModelType model_type, size_t *size, bool use_mmap)如果模型是MindIR的直接转换后返回貌似目前只支持kMindIR_lite,因为在std::shared_ptrmindspore::Model MSLiteModelNapi::CreateModel里传入的model_type值只有 mindspore::kMindIR , 因此本质上就是把传递过来的const char* buffer 转为char *。constchar*lite::LiteSession::LoadModelByPath(conststd::stringfile,mindspore::ModelType model_type,size_t*size,booluse_mmap){size_t buf_size;char*model_buf;if(use_mmap){model_bufreinterpret_castchar*(lite::ReadFileByMmap(file.c_str(),buf_size));}else{model_buflite::ReadFile(file.c_str(),buf_size);}...char*lite_bufnullptr;////然后传递到lite::LiteSession::LoadModelByBuff 中做了简单的转换autobuf_model_typeLoadModelByBuff(model_buf,buf_size,lite_buf,size,model_type);if(buf_model_typemindspore::ModelType::kUnknownType||lite_bufnullptr){if(use_mmap){lite::UnmapMmapBuffer(const_castvoid*(static_castconstvoid*(model_buf)),buf_size);}else{delete[]model_buf;}model_bufnullptr;returnnullptr;}returnlite_buf;}1.1、mindspore::ModelType lite::LiteSession::LoadModelByBuff(const char *model_buf, const size_t buf_size, char **lite_buf, size_t *size, mindspore::ModelType model_type)mindspore::ModelType lite::LiteSession::LoadModelByBuff(constchar*model_buf,constsize_tbuf_size,char**lite_buf,size_t*size,mindspore::ModelType model_type){if(model_typemindspore::ModelType::kMindIR_Lite){*sizebuf_size;*lite_bufconst_castchar*(model_buf);returnmindspore::ModelType::kMindIR_Lite;}#ifdefRUNTIME_CONVERT*lite_bufRuntimeConvert(model_buf,buf_size,size,ms_context_);#endifreturnmindspore::ModelType::kMindIR;}2、PackWeightManager::InitPackWeightManager(const char *model_buf, size_t model_size, std::string *model_id, std::string *runner_id,const std::mapstd::string, std::mapstd::string, std::string *config_info)初始化权重管理器PackWeightManager。它通过配置的解析、锁机制保障线程安全、动态创建权重对象并准备模型ID及其对应的运行器ID。最终利用解析出的信息调用权重对象的初始化方法完成模型权重的打包和共享准备在这一步会把上一步中模型文件的Buffer 进行处理后再传到下一步。STATUSPackWeightManager::InitPackWeightManager(constchar*model_buf,size_t model_size,std::string*model_id,std::string*runner_id,conststd::mapstd::string,std::mapstd::string,std::string*config_info){#ifdefSHARING_MODEL_WEIGHTstd::unique_lockstd::mutexl(manager_mutex_);if(pack_weight_nullptr){pack_weight_std::make_sharedPackWeight();....}//从给定的配置映射中提取NUMA ID。通过检查传入的配置信息是否为nullptr以及是否包含特定的键它能够有效地解析出NUMA ID。如果未找到相应条目则返回默认值-1。这对于在多处理器环境中管理内存和资源的分配至关重要autonuma_idstd::atoi(ParseNumaId(config_info).c_str());*model_idGenModelID();std::string idParseRunnerId(config_info);*runner_idid;if(id.empty()){MS_LOG(INFO)model use share pack weight.;id*model_id;}boolneed_copy_bufParseCopyBuf(config_info);returnpack_weight_-InitPackWeight(static_castconstvoid*(model_buf),model_size,id,numa_id,need_copy_buf);#endifreturnRET_OK;}2.1、PackWeight::InitPackWeight(const void *model_buf, size_t model_size, std::string id, int numa_id, bool need_copy_buf)初始化模型权重并将其存储在指定的缓冲区中以便在后续的计算中使用。此功能被封装在 PackWeight::InitPackWeight 方法中能够处理模型的内存管理并在多个NUMA节点之间共享相应的资源。NUMA是做核上亲和性的多核的CPU架构上的机制NUMA id当作是cpu id可以按绑核操作理解的权重buffer绑定到较近的CPU上优化访存性能STATUSPackWeight::InitPackWeight(constvoid*model_buf,size_t model_size,std::string id,intnuma_id,boolneed_copy_buf){std::lock_guardstd::mutexlock(mtx_weight_);if(model_bufnullptr||model_weights_.size()!shared_bufs_.size()){MS_LOG(ERROR)model buf is nullptr in pack weight manager.;returnRET_ERROR;}//模型缓冲区 model_buf 是否为 nullptr以及模型权重和共享缓冲区的大小是否匹配if(model_weights_.find(id)!model_weights_.end()model_weights_[id].find(numa_id)!model_weights_[id].end()){MS_LOG(INFO)same numa id, use same model buf.;returnRET_OK;}//分配内存管理器 (Allocator)std::shared_ptrAllocatorallocatornullptr;#ifdefBFC_MEMORYallocatorstd::make_sharedDynamicMemAllocator(numa_id);#elseallocatorstd::make_sharedDefaultAllocator();#endifif(allocatornullptr){MS_LOG(ERROR)allocator is nullptr in pack weight manager.;returnRET_ERROR;}auto*model_const_weightnew(std::nothrow)ModelConstWeight();if(model_const_weightnullptr){MS_LOG(ERROR)model const weight is nullptr.;returnRET_ERROR;}//需要复制缓冲区数据则为其分配新的内存并复制原有缓冲区的内容。void*new_model_bufconst_castvoid*(model_buf);if(need_copy_buf){//分配对应的内存一整个模型的buffernew_model_bufallocator-Malloc(model_size);...memcpy(new_model_buf,model_buf,model_size);model_const_weight-copy_bufneed_copy_buf;}model_const_weight-allocatorallocator;model_const_weight-numa_idnuma_id;//将模型的权重和相应的缓冲区添加到 model_weights_ 和 shared_bufs_ 中若该 id 不存在则需要创建新的映射关系。if(model_weights_.find(id)!model_weights_.end()){model_weights_[id][numa_id]model_const_weight;shared_bufs_[id][numa_id]new_model_buf;}else{std::unordered_mapint,ModelConstWeight*numa_model_weight;numa_model_weight[numa_id]model_const_weight;model_weights_[id]numa_model_weight;std::unordered_mapint,void*numa_model_buf;numa_model_buf[numa_id]new_model_buf;shared_bufs_[id]numa_model_buf;}returnRET_OK;}3、lite::ImportFromBufferlite::ImportFromBuffer 通过不同的类型的Model对应不同的ModelLoader由ModelLoader ImportModel对传入过来的Model Buffer 进行包装为LiteModel对象并把Buffer数据copy过来并设置对应的模型类型。见鸿蒙进阶——Mindspore Lite AI框架源码解读之模型加载详解(一)。4、LiteSession::CompileGraph(Model *model)在LiteSession中传入Model * 是从权重处理器的缓存区中读取出来的封封装而得到的Model *到CompileGraph 时先进行拆包转换Model-Tensor不过先会先去判断Model 的模型类型不同的模型使用不同的转换算法其中Model 的子类有一个抽象父类AbstractBaseModel目前只有MindIRModel 这个子类即用于描述MINDIR模型的。转为完成后再把处理过的Model 传入用于初始化Graph即vector mslite:Tensor 未完待续…