Question

我目前正在着手在亚马孙施肥厂建立一个管道。为此,我设立了一个Xgboost-estimator,并在我的数据集上接受了培训。培训工作按预期进行,新培训的模式可节省到特定产出桶。稍后,我要重新输入这一模式,而这一模式正是通过获得这一模式来实现的。产出桶中的沥青,从中提取模型,通过小 pick把双筒固定下来。

# download the model artifact from AWS S3
!aws s3 cp s3://my-bucket/output/sagemaker-xgboost-2021-09-06-12-19-41-306/output/model.tar.gz .

# opens the downloaded model artifact and loads it as  model  variable
model_path = "model.tar.gz"
with tarfile.open(model_path) as tar:
    tar.extractall(path=".")

model = pkl.load(open("xgboost-model", "rb"))

每当我试图打响时,我就收到一个令人不快的流入:

---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
<ipython-input-9-b88a7424f790> in <module>
     10     tar.extractall(path=".")
     11 
---> 12 model = pkl.load(open("xgboost-model", "rb"))
     13 

UnpicklingError: unpickling stack underflow

So far I retrained the model to see, if the error occurs with a different model file and it does. I also downloaded the model.tar.gz and validated it via gunzip. When extracting the binary file xgboost-model is extracted correctly, I just can t pickle it. Every occurence of the error I found on stackoverflow points at a damaged file, but this one is generated directly by SageMaker and I do note perform any transformation on it, but extracting it from the model.tar.gz. Reloading a model like this seems to be quite a common use case, referring to the documentation and different tutorials. Locally I receive the same error with the downloaded file. I tried to step directly into pickle for debugging it but couldn t make much sense of it. The complete error stack looks like this:

Exception has occurred: UnpicklingError       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
unpickling stack underflow
  File "/sagemaker_model.py", line 10, in <module>
    model = pkl.load(open( xgboost-model ,  rb ))
  File "/usr/local/Cellar/[email protected]/3.9.1_5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/Cellar/[email protected]/3.9.1_5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/local/Cellar/[email protected]/3.9.1_5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 268, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/local/Cellar/[email protected]/3.9.1_5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/Cellar/[email protected]/3.9.1_5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main (Current frame)
    return _run_code(code, main_globals, None,

造成这一问题的原因是什么,在这个过程中,我可以采取什么步骤来解决这个问题。

Answer 1

该问题根植于用于Xgboost框架的模型版本。缺省产出从1.3.0改为“json”,而制片人的文件似乎没有相应更新。因此,如果你希望通过该模式来读。

    tar.extractall(path=".")

model = pkl.load(open("xgboost-model", "rb"))

正如老板docs子所描述的那样,你需要进口带有前版本的XGBOOST框架,例如1.2.1。

Answer 2

最新XGBoost版本似乎改变了这一进程。 http://code>1.3.x:

import xgboost as xgb

model = xgb.Booster()
model.load_model( xgboost-model )

Answer 3

After reading the answer @Imoe41, I would also liked to contribute to the question. The problem is as you would see if you click on the link in the error is that (https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html), from version 1.0 of xgboost the models are being saved in json, and before version 1.0, the models were being saved in pickle. I trained the xgboost model in 2020 with sagemaker, using the xgboost version of 0.90. However, in my notebook the xgboost package version was 1.5.1.

解决办法:

Check the version of installed xgboost

进口xgboost作为Xgb印刷(xgb., 转化)

If the version is higher then 1.0, then you need to downgrade it. In order to downgrade xgboost, you also need to downgrade the other packages.

pip install scipy==1.4.1
pip install shap==0.37.0
pip install xgboost==0.90.0

Load the model as a pickle

import pickle as pkl
import tarfile
t = tarfile.open( model.tar.gz ,  r:gz )
t.extractall()
model = pkl.load(open("xgboost-model",  rb ))

友情链接