- Best AI Text Generators for High Quality Content Writing
- Tensorflow Error on Macbook M1 Pro - NotFoundError: Graph execution error
- How does GPT-like transformers utilize only the decoder to do sequence generation?
- How to set all tensors to cuda device?
- How should I use torch.compile properly?
- How do I check if PyTorch is using the GPU?
- WARNING:tensorflow:Using a while_loop for converting cause there is no registered converter for this op
- How to use OneCycleLR?
- Error in Python script "Expected 2D array, got 1D array instead:"?
- How to save model in .pb format and then load it for inference in Tensorflow?
- Top 6 AI Logo Generator Up Until Now- Smarter Than Midjourney
- Best 9 AI Story Generator Tools
- The Top 6 AI Voice Generator Tools
- Best AI Low Code/No Code Tools for Rapid Application Development
- YOLOV8 how does it handle different image sizes
- Best AI Tools For Email Writing & Assistants
- 8 Data Science Competition Platforms Beyond Kaggle
- Data Analysis Books that You Can Buy
- Robotics Books that You Can Buy
- Data Visualization Books that You can Buy
How to assign a name for a pytorch layer?
There are mostly two ways to assign a name for a PyTorch layer. The first way is to use the torch.nn package's naming convention, where each value in Bidirectional(...) is replaced with underscore( _ ) underscores. This way, the layered module would be called FuBLongShortTermMemory .
The second way uses prefixes or keywords to group layers by type. "Local" for example could be used as the keyword if adding redundant layers to localize receptive fields of convolutional neural networks, which can be useful in pedestrian detection applications that require high precision on pedestrian boundaries. "Generative" could serve as keyword if adding generative neural network layers that produce new content at each timestep.
How to assign a name for a pytorch layer:
Sequential
Pass an instance of collections.OrderedDict. Code below gives conv1.weights
, conv1.bias
, conv2.weight
, conv2.bias
(notice lack of torch.nn.ReLU()
, see end of this answer).
import collections
import torch
model = torch.nn.Sequential(
collections.OrderedDict(
[
("conv1", torch.nn.Conv2d(1, 20, 5)),
("relu1", torch.nn.ReLU()),
("conv2", torch.nn.Conv2d(20, 64, 5)),
("relu2", torch.nn.ReLU()),
]
)
)
for name, param in model.named_parameters():
print(name)
Dynamic
Use ModuleDict
instead of ModuleList
:
class MyModule(torch.nn.Module):
def __init__(self):
super().__init__()
self.whatever = torch.nn.ModuleDict(
{f"my_name{i}": torch.nn.Conv2d(10, 10, 3) for i in range(5)}
)
Will give us whatever.my_name{i}.weight
(or bias
) for each created module dynamically.
Direct
Just name it however you want and that's how it will be named
self.my_name_or_whatever = nn.Linear(7, 8)
You didn't think about
- If you want to plot weights, biases and their gradients you can go along this route
- You can't plot activations this way (or output from activations). Use PyTorch hooks instead (if you want per-layer gradients as they pass through network use this also)
For last task you can use third party library torchfunc (disclaimer: I'm the author) or go directly and write your own hooks.
Thank you for reading the article. If you face any problem, please comment below.