Glorot Uniform

class GlorotUniform(RandomInitializer)

The Glorot uniform initializer, also called Xavier uniform initializer.

Reference: [GLOROT-2010]

Draws samples from a uniform distribution:

  • Real case: x ~ U[-limit, limit] where limit = sqrt(6 / (fan_in + fan_out))
  • Complex case: z / Re{z} = Im{z} ~ U[-limit, limit] where limit = sqrt(3 / (fan_in + fan_out))

where fan_in is the number of input units in the weight tensor and fan_out is the number of output units.

Standalone usage:

import cvnn
initializer = cvnn.initializers.GlorotUniform()
values = initializer(shape=(2, 2))                  # Returns a complex Glorot Uniform tensor of shape (2, 2)

Usage in a cvnn layer:

import cvnn
initializer = cvnn.initializers.GlorotUniform()
layer = cvnn.layers.Dense(input_size=23, output_size=45, weight_initializer=initializer)
__init__(self, seed=None)
Parameters:seed – Integer. An initializer created with a given seed will always produce the same random tensor for a given shape and dtype.
__call__(self, shape, dtype=tf.dtypes.complex64)
Returns a real-valued tensor object initialized as specified by the initializer.
The complex dtype input will only be used to know the limits to be used. This result must be used for the real and imaginary part separately.
Parameters:
  • shape – Shape of the tensor.
  • dtype – Optional dtype of the tensor. Either floating or complex. ex: tf.complex64 or tf.float32
[GLOROT-2010]Glorot, Xavier, and Yoshua Bengio. “Understanding the difficulty of training deep feedforward neural networks.” Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010.