Glorot Normal

class GlorotNormal(RandomInitializer)

The Glorot normal initializer, also called Xavier normal initializer.

Reference: [GLOROT-2010]

Note

The reference actually refers to the uniform case but it’s analysis was adapted for a normal distribution

Draws samples from a truncated normal distribution centered on 0 with

  • Real case: stddev = sqrt(2 / (fan_in + fan_out))
  • Complex case: real part stddev = complex part stddev = 1 / sqrt(fan_in + fan_out)

where fan_in is the number of input units in the weight tensor and fan_out is the number of output units.

Standalone usage:

import cvnn
initializer = cvnn.initializers.GlorotNormal()
values = initializer(shape=(2, 2))                  # Returns a complex Glorot Normal tensor of shape (2, 2)

Usage in a cvnn layer:

import cvnn
initializer = cvnn.initializers.GlorotNormal()
layer = cvnn.layers.Dense(input_size=23, output_size=45, weight_initializer=initializer)
__call__(self, shape, dtype=tf.dtypes.complex64)
Returns a real-valued tensor object initialized as specified by the initializer.
The complex dtype input will only be used to know the limits to be used. This result must be used for the real and imaginary part separately.
Parameters:
  • shape – Shape of the tensor.
  • dtype – Optinal dtype. Either floating or complex. ex: tf.complex64 or tf.float32