Question:
How to train a deep learning in case validation with Pytorch result NaN or high loss?

Solution:

You can try to initialize the weight and bias as shown in the following code:

In case the bias matrix variance or weight is very large, the loss will increase.

Refer to >this paper(Xavier initialization) and (He initialization) will enable you think differently:


class classificador(nn.Module):

    def __init__(self):

        super().__init__()

        self.features, self.classifiers = [], []


        self.features += [nn.Conv2d(3, 64, 3), nn.ReLU(inplace=True), nn.BatchNorm2d(64), nn.MaxPool2d(2)]

        self.features += [nn.Conv2d(64, 64, 3), nn.ReLU(inplace=True), nn.BatchNorm2d(64), nn.MaxPool2d(2)]

     

        self.flatten = nn.Flatten()


        self.classifiers += [nn.Linear(14 * 64 * 64, 256), nn.ReLU(inplace=True)]

        self.classifiers += [nn.Linear(256, 128), nn.ReLU(inplace=True)]

        self.classifiers += [nn.Linear(128, 15)]


    for m in self.features:

         if isinstance(m, nn.Conv2d):

             nn.init.kaiming_normal_(m.weight, mode="fan_out", nonlinearity="relu")

             if m.bias is not None:

                 nn.init.constant_(m.bias, 0)

    

    for m in self.classifier:

        if isinstance(m, nn.Linear):

            nn.init.normal_(m.weight, 0, 0.01)

            nn.init.constant_(m.bias, 0)

    

    self.features = nn.Sequential(*self.features)

    self.classifier = nn.Sequential(*self.classifiers)


    def forward(self, X):

        X = self.features(X)

        X = self.flatten(X)

        X = self.classifiers(X)

        return X


Suggested blogs:

>Can not run phpstan under docker with memory lack error

>Attempt to read property "data_audit" on array laravel view loop foreach

>How to use start and limit on DataTables in Laravel for mobile API?

>Login to Laravel API- Laravel

>Make Xdebug stop at breakpoints in PhpStorm using Docker

>Creating a service in Laravel using app(FQCN)


Ritu Singh

Ritu Singh

Submit
0 Answers