PyTorch – Sqrt() | Code Underscored | ninjasquad

In this PyTorch lesson, we will use the sqrt() method to return the square root of each element in the tensor. An open-source framework called PyTorch is released together with the Python programming language. The data is kept in a multidimensional array called a tensor. Additionally, we must import the torch module to use a tensor.

The procedure used to produce a tensor is called tensor().

## PyTorch – Sqrt() Syntax

The syntax is as follows:

Where the data is an array with several dimensions.

### Output

It outputs a tensor containing the square-root reciprocal.

### Steps

Add the necessary libraries by importing those modules.

The necessary Python library for each of the ensuing examples is the torch. Also, please make sure that it is already installed.

Create and print a torch tensor.

input_var = torch.randn(8,9) print("The Tensor's Input is:\n", input_var)

Use the torch to calculate the reciprocal of each input tensor component’s square root.

The input tensor is called the input_var in this case.

var_recip = torch.rsqrt(input_var)

Show the reciprocal values for the computed tensor

print("The Reciprocal SQRT Tensor is as follows:\n", var_recip)

### Torch.sqrt()

The PyTorch function sqrt() returns the square root of each element in the tensor. It needs just one parameter. The syntax is as follows:

torch.sqrt(tensor_object)

### Parameter

The tensor_object parameter in the sqrt() function is a tensor.

### Example: One dimensional tensor with five elements

In this example, we’ll make a tensor with one dimension and five elements, and we’ll return the square roots of these five elements in a tensor.

#start by importing the torch module import torch #creating a new a tensor varData = torch.tensor([17,39,61,6,15]) #show the real tensor print("The actual Tensor is: ") print(varData) print("The resulting Square Root is: ") print(torch.sqrt(varData))

### Example: Two Dimensional tensors with five elements

In this example, we will generate a two-dimensional tensor with five elements per row and return the elements’ square root.

#begin by importing the torch module import torch #creation of a 2D tensor varData=torch.tensor([[50,72,26,28,7],[7,8,9,10,11]]) #show the real tensor print("The Actual Tensor is: ") print(varData) print("The resulting Square Root is:") print(torch.sqrt(varData))

### Utilizing a CPU

We must create a tensor with a cpu() function to execute an sqrt() function on the CPU. It will operate on a computer with a CPU. This time, we can use the cpu() function when building a tensor. The syntax is as follows:

#### Example: One-dimensional tensor with five elements on the CPU

In this example, we’ll build a tensor with one dimension and five elements on the computer, then return the tensor’s square roots for these five members.

#commence by importing the torch module import torch #creation of a tensor varData = torch.tensor([17,39,61,6,15]).cpu() #Show the real tensor print("The Actual Tensor is:") print(varData) print("The Square Root is: ") print(torch.sqrt(varData))

#### Example: 2-dimensional tensor with five elements on the cput

In this example, we’ll generate a two-dimensional tensor with five elements in each row on the CPU and return the square root of the elements.

#start by importing the torch module import torch #creation of a 2D tensor varData=torch.tensor([[50,72,26,28,7],[7,8,9,10,11]]).cpu() #show the actual tensor print("The real tensor is: ") print(varData) print("The resultant square root is:") print(torch.sqrt(varData))

#### Example: Computing the reciprocal of the square-root

This Python application calculates the reciprocal of the square root of input tensors with real and complex values.

# start by Importing the needed library import torch # defining the input tensor var_input = torch.tensor([6.2, 8., 9., 9.2, -2.2]) # printing the defined tensor from above print("The resultant input tensor is:\n", var_input) # computation of the square root's reciprocal var_recip = torch.rsqrt(var_input) # printing the tensor resulting from the computation above print("The SQRT Tensor Reciprocal is:\n", var_recip) print("............................") # definition of a complex input tensor var_input = torch.tensor([1.2+2j, 3.+4.j, 4.2-3.2j]) # printing the defined tensor from above print("The resultant input tensor is:\n", var_input) # computing the square root's reciprocal var_recip = torch.rsqrt(var_input) # print the above-computed tensor print("The SQRT Tensor's Reciprocal is :\n", var_recip)

Keep in mind that the Reciprocal SQRT tensor member that corresponds to a zero in the input tensor is NaN.

Example:

# begin by importing the required library import torch # definition of an input tensor var_input = torch.randn(8,9) # printing of the defined tensor from above print("The Input Tensor is :\n", var_input) # compute the reciprocal of the square root var_recip = torch.rsqrt(var_input) # printing the computed tensor from above print("The SQRT Tensor's Reciprocal is:\n", var_recip) print("......................................") # definition of an input tensor that is complex var_real = torch.randn(8,8) var_imag = torch.randn(8,8) var_input = torch.complex(var_real, var_imag) # printing of the defined tensor shown above print("The resulting Input Tensor is:\n", var_input) # computing the square root's reciprocal from above var_recip = torch.rsqrt(var_input) # printing the computed tensor as shown above print("The SQRT Tensor's Reciprocal is:\n", var_recip)

#### Example: AWS overrides the function _load_from_state_dict to recover **weight_gamma and weight_beta**

def _load_from_state_dict(self, var_state_dict, var_prefix, var_local_metadata, var_strict, var_missing_keys, var_unexpected_keys, error_msgs): """ Replace the built-in load function. If weight_gamma and weight_beta are absent, AWS overrides the function _load_from_state_dict to retrieve them. After super()._load_from_state_dict, this method will return if weight_gamma and weight_beta are found in the checkpoint. If not, it will calculate the pretrained weights' mean and standard deviation and save those results in weight_beta and weight_gamma. """ self.weight_gamma.data.fill_(-1) var_local_missing_keys = [] super()._load_from_state_dict(var_state_dict, var_prefix, var_local_metadata, var_strict, var_local_missing_keys, var_unexpected_keys, error_msgs) if self.var_weight_gamma.data.mean() > 0: for k in var_local_missing_keys: var_missing_keys.append(k) return var_weight = self.weight.data var_weight_flat = var_weight.view(var_weight.size(0), -1) var_mean = var_weight_flat.mean(dim=1).view(-1, 1, 1, 1) var_std = torch.sqrt(var_weight_flat.var(dim=1) + 1e-5).view(-1, 1, 1, 1) self.var_weight_beta.data.copy_(var_mean) self.var_weight_gamma.data.copy_(var_std) var_missing_gamma_beta = [ k for k in local_missing_keys if k.endswith('var_weight_gamma') or k.endswith('var_weight_beta') ] for k in var_missing_gamma_beta: var_local_missing_keys.remove(k) for k in local_missing_keys: var_missing_keys.append(k)

#### Example: Forward pass

def forward(self, invisible_states, attn_mask): mxd_query_layer = self.query(invisible_states) mxd_key_layer = self.key(invisible_states) mxd_value_layer = self.value(invisible_states) query_layer = self.transpose_for_scores(mxd_query_layer) key_layer = self.transpose_for_scores(mxd_key_layer) value_layer = self.transpose_for_scores(mxd_value_layer) # To obtain the raw attention scores, assume the dot product between "query" and "key." attn_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2)) attn_scores = attn_scores / math.sqrt(self.attn_head_size) # Apply the attention mask, which BertModel's forward() function has already computed for all layers. attn_scores = attn_scores + attn_mask # the attention scores should be normalized to probabilities. attn_probs = nn.Softmax(dim=-1)(attn_scores) # This is essentially removing all of the tokens to handle it, which could # is derived from the original Transformer paper, although it may appear slightly odd. attn_probs = self.dropout(attn_probs) contxt_layer = torch.matmul(attn_probs, value_layer) contxt_layer = contxt_layer.permute(0, 2, 1, 3).contiguous() new_contxt_layer_shape = context_layer.size()[:-2] + (self.all_head_size,) contxt_layer = contxt_layer.view(*new_contxt_layer_shape) return contxt_layer

### Conclusion

The sqrt() method was covered in this PyTorch -sqrt() article. Each member of the input tensor’s square-root reciprocal is calculated using the torch.rsqrt() method. Every component of the PyTorch tensor is returned along with its square root. Further, it accepts inputs with real and complex values.

The matching element in the output tensor is NaN if an element in the input tensor is zero. To use the sqrt() function, we explored several examples using various dimensional tensors.

Source: Internet