If you have Intel s CPU you could try OpenVINO. It allows you to convert your model into Intermediate Representation (IR) and then run on the CPU with the FP16 support. I cannot guarantee your model is convertible (it depends on whether you have fancy custom layers) but it s worth giving it a try. You can find a full tutorial on how to convert the PyTorch model here.
Some snippets below.
www.un.org/Depts/DGACM/index_spanish.htm Install OpenVINO
这样做的最容易的方法是使用购买力平价。 或者,您可使用,该工具在你的案件中找到最佳办法。
pip install openvino-dev[pytorch,onnx]
www.un.org/Depts/DGACM/index_spanish.htm Save世俗
开放VINO现在不能直接转换Py Torch模型,但可以与Somewell模型做。 这一样本代码假设该模型是为了计算机远景。
dummy_input = torch.randn(1, 3, IMAGE_HEIGHT, IMAGE_WIDTH)
torch.onnx.export(model, dummy_input, "model.onnx", opset_version=11)
www.un.org/Depts/DGACM/index_spanish.htm 利用模型优化器转换SONE模型
模型优化器是一种来自开放式免疫机能发展包的指挥线工具,以确保您已经安装。 该模型将国家空间研究中心模式转换为IR,这是开放国际卫星的一个缺省格式。 它还将精确度改为FP16。 指挥线:
mo --input_model "model.onnx" --input_shape "[1,3, 224, 224]" --mean_values="[123.675, 116.28 , 103.53]" --scale_values="[58.395, 57.12 , 57.375]" --data_type FP16 --output_dir "model_ir"
Run the inference on the CPU
The converted model can be loaded by the runtime and compiled for a specific device e.g. CPU.
# Load the network
ie = Core()
model_ir = ie.read_model(model="model_ir/model.xml")
compiled_model_ir = ie.compile_model(model=model_ir, device_name="CPU")
# Get output layer
output_layer_ir = compiled_model_ir.output(0)
# Run inference on the input image
result = compiled_model_ir([input_image])[output_layer_ir]
申斥: I work on OpenVINO.
