Question

My goal is to predict whether a shot is hit or not by the trajectory of the basketball (there are already papers that do the same, I m just reproducing them (https://arxiv.org/abs/1608.03793)). It s a very simple problem, but for some other purpose I want to try lstm s prediction. But the network was not only slow to converge but start overfitting very quickly. A time series has a length of 12 and a feature dimension of 4 (x,y,z coordinates and time)


class PolicyHead(nn.Layer):
    def __init__(self):
        super(PolicyHead, self).__init__()
        self.lstm = LSTM(4,64,2,dropout=0.3)
        self.fc = nn.Linear(64,2)

    def forward(self, past_traj):
        b,n,c = past_traj.shape
         
        return self.fc(self.lstm(past_traj, batch_id))

“entergraph

相比之下,使用了非常简单的乳房结构,预测非常好。

class PolicyHead(nn.Layer):
    def __init__(self):
        super(PolicyHead, self).__init__()
        self.fc = nn.Sequential(
            nn.Linear(12*4, 512),
            nn.ReLU(),
            nn.Linear(512,512),
            nn.ReLU(),
            nn.Linear(512,2)
        )

    def forward(self, past_traj):
        b,n,c = past_traj.shape
        return self.fc(past_traj.reshape([b,-1]))

enter image description here

Answer 1

Your graph does suggest overfitting, especially if it continues to increase if more epochs are added. I don t see where you are setting the training optimizer, but you can also use L1/L2 regularization to update the Learning rate; this will help with overfitting, and I think this will be beneficial regardless of what decision you take for your over-fitting issue.

现在,为了直接解决过度适应问题,你可以增加气球的正常化,将辍学率调整为更高的价值(即0.5),和(或)利用经常辍学(你可以共同使用)。

如果数据差异很大,而且你的批量已经很大,则按正常情况分类是非常有益的。

辍学和经常辍学只是培训过程中的行为,因此没有必要担心影响推论阶段。

class PolicyHead(nn.Layer):
    def __init__(self):
        super(PolicyHead, self).__init__()
        self.lstm = LSTM(4,64,2,dropout=0.3, recurrent_dropout=0.3)
        self.fc = nn.Linear(64,2)

    def forward(self, past_traj):
        b,n,c = past_traj.shape
         
        return self.fc(self.lstm(past_traj, batch_id))

你们可以共同利用这些价值观,但trick计是尝试错误,只使用经常辍学,尝试某些价值观,并检验其行为。

Now, if you want to apply BatchNormalization, you will need to update a little more. In your case

class PolicyHead(nn.Layer):
    def __init__(self):
        super(PolicyHead, self).__init__()
        self.lstm = LSTM(4,64,2,dropout=0.3, recurrent_dropout=0.3) #Assuming this is nn.LSTM
        self.batch_norm = nn.BatchNorm1d(64)
        self.fc = nn.Linear(64,2)

    def forward(self, past_traj):
        b,n,c = past_traj.shape
        lstm_out = self.lstm(past_traj, batch_id)[:-1:]
        return self.fc(self.batch_norm(lstm_out)

Hope this helps

友情链接