Question

Beginner on ANNs:

我正在实施一项反向传播的神经网络,以预测黄金的价格。我知道,我必须将我的数据分为培训数据、甄选数据和测试数据。

然而,我没有保证如何利用这些数据集。首先,我对数据网络进行了培训,然后在培训之后,我从测试组获得一些投入,并对产出进行比较。

我不敢肯定我是否行使这一权利,所选择的人选是否在进行?

页: 1

Answer 1

总体设想是:

Train the network for a little while on the training set.
Evaluate the network on a second set, often called the validation set. Probably what you re calling the selection set.
Train the network a little more on the training set.
Evaluate the new network on the selection set again.
Which did better, the old network or the new network? If the new network is better, we re still getting some use out of training, so goto 3. If the new network is worse, more training will probably only hurt. Use the previously version of the network, since it did better.

这样,你就可以知道何时停止培训。

一个容易的修改是,始终追踪迄今为止所看到的最佳网络,我们只是在看到一些培训尝试(例如,三个)在一行中更加恶化的时候停止培训。

第三组测试组是必要的,因为甄选组如果间接地参与培训过程。必须最后评价培训期间根本没有使用的数据。

这类事情足以进行简单的试验,但总的来说,你想要使用,以便更好地了解你的系统业绩。

Answer 2

我只想发表一点意见,即验证套是依赖模型的超参数重估的良好场所,但我在这里是新鲜的,因此没有声望。为使这一职位更值得单独张贴,我已概述了我自己的培训考试过程。实际上,我的工作流程如下:

Identify, collect, and clean data. Try to limit complaining during data munging process.
Split data into three sets: training, validation, test.
Establish two "base" models for evaluating more complex models built later on in the process. The first of these models is typically a basic linear/logistic regression using all possible features. The second models uses only the most obviously informative (initial identification of informative features depends on use case, typically involves combination of domain knowledge, basic clustering, simple correlation).
Begin more empirical feature selection (i.e. unsupervised NN, but usually random forest) and prototype a broad range of models using the training set.
Eliminate poorly performing models as well as uninformative features
Compare performance of remaining models against each other and the "base" models, using a modified version of the training set (same data, but sans uninformative features). Toss under-performing models.
Using the validation set, tune the appropriate hyper-parameters for each of the models (either by hand or gridsearch). Further reduce the number of models in consideration, ideally to just 2-3 (excluding base models).
Finally, evaluate model performance (with optimized hyper-parameters) on the test set. Again, compare models among themselves and against the base models. Make final model choice based on a problem-specific appropriate combination of computational complexity/cost, ease of interpretation/transparency/"explainability", and improvement over and/or performance vs base models.

友情链接