Question:

Why nn.Dropout change the elements values of a tensor in Python?

From what I understand, nn.Dropout can take a tensor as input and randomly makes some elements zero with a certain chance. But look at my code:

nn.Dropout(dropout) = dropout

y = torch.tensor([5.0,7.0,9.0])

y = dropout(y)

print(y)

output is tensor([ 5.5556, 7.7778, 10.0000])

I have tried many times, and there is sometimes a zero. The other parts, on the other hand, always have the same value: 5.0 -> 5.5556, 7.0 -> 7.7778, 9.0 -> 2010.

Why does this take place?

Solution:

From the >documentation, you can find this:

Furthermore, the outputs are scaled by a factor of 1/(1-p) during training. This means that during evaluation the module simply computes an identity function.

So your output is scaled by (1/(1-0.1)) in your example.

For confirmation, below code returns the same output you observed:

import torch

y = torch.tensor([5.0,7.0,9.0])

print(y*(1/(1-0.1))) # You used p=0.1

>>> tensor([ 5.5556, 7.7778, 10.0000])

Dropout layer works for training stage only. If dropout layer does not carry out this kind of scaling for inputs, the consequence would be like the network is fed with 'exaggerated' inputs compared with the inputs we used during training stage.

I think you can refer to kind of post like >this also.

>How React hooks Completely replace redux?

>How to access the state using Redux RTK in the React Native app?

>How to Build a Python GUI Application With WX Python

>How to Build a RESTful API Using Node Express and MongoDB- Part 1

>How to build an Electron application from scratch?

>How to build interactivegraphviz in Python