PyTorch Packages

PyTorch is a Deep Learning library that has several libraries dedicated to different purposes. The most important of them is torch.nn which helps us create neural layers. Other libraries such as torch.nn.functional, torch.optim, torch.Tensor etc are also very useful and supplement our neural networks to build a complete model. We will now take a look at some important packages of PyTorch.

PyTorch Packages

1. torch:

It is the most basic package of PyTorch which contains data structures to accommodate tensors and allows us to perform operations on them.

2. torch.nn:

The module torch.nn forms the backbone of PyTorch as it provides most of the essential libraries required for Deep Learning like the convolution layers, linear layers, different activation functions, pooling layers, loss functions and many more.

3. torch.nn.functional:

torch.nn contains object oriented libraries but torch.nn.functional contains the same features but in function form, i.e they can be invoked like other functions.

4. torch.Tensor:

Tensors are multidimensional arrays containing values of the same data types. torch.Tensor contains various methods to perform tensor operations in one line which would otherwise take some effort.

5. torch.cuda:

Cuda is a parallel computing platform that allows us to utilise GPUs for our models. And as we know PyTorch owes its fast performance to GPUs. torch.cuda enables the transformation of tensors into GPU readable form.

6. torch.distributions:

It contains the parameterized probability function of the samples allowing the construction of dynamic computation graphs.

7. torch.fft:

Contains functions related to discrete fast fourier transforms.

8. torch.fx:

Provides toolkits to transform nn.Module instances.

9. torch.jit:

Jit takes PyTorch codes and converts it to C++ script which runs faster than Python codes.

10. torch.linalg:

Enables us to perform common linear algebra operations such as finding norm, inverse, transpose of a matrix etc.

11. torch.special:

It is a scientific library inspired by the SciPy library. It enables us to perform many scientific tasks like calculating entropy, gamma function etc.

12. torch.package:

Using torch.package we can create our own PyTorch package of the code which can be used whenever required.

13. torch.profiler:

It helps us better understand the process we are executing by collecting the performance metrics during training.

14. torch.onnx:

Facilitates exporting our machine learning models to ONNX (Open Neural Network Exchange).

15. torch.optim:

One of the most important libraries which helps us optimise our model. Using torch.optim we can also set the gradient to zero before each epoch, which is very crucial for obtaining meaningful results.

16. torch.random:

Deals with initialising our tensors with random numbers along with other related operations like setting the seed, generator state etc.

17. torch.sparse:

The tensors of the datasets of deep learning models sometimes contain mostly zeros leading to wastage of storage to store them. Sparse array allows us to store only non-zero elements thereby saving storage space.

18. torch.storage:

PyTorch tensors occupy contiguous memory space irrespective of the dimensionality which can be managed by torch.tensors.

19. torch.testing:

Indicates the distance between the actual and expected outputs to determine accuracy. This module is still in prototype state.

20. torch.utils.benchmark:

Provides time measuring tools for checking the running time of our model.

21. torch.utils.checkpoint:

Instead of saving all the computational graphs we can save only some essential graphs called checkpoint.

22. torch.utils.data:

Contains libraries like datasets and dataloaders which have some pre-processed data and an iterable that wraps around the available data respectively.

23. torch.utils.dlpack:

It is used to convert a tensor from an external source to PyTorch tensor.

Summary:

Owing to its variety and quantity of packages, PyTorch becomes very powerful in designing neural networks and comfortable even for beginners.