Glorot Uniform¶
-
class
GlorotUniform
(RandomInitializer)¶ The Glorot uniform initializer, also called Xavier uniform initializer.
Reference: [GLOROT-2010]
Draws samples from a uniform distribution:
- Real case:
x ~ U[-limit, limit]
wherelimit = sqrt(6 / (fan_in + fan_out))
- Complex case:
z / Re{z} = Im{z} ~ U[-limit, limit]
wherelimit = sqrt(3 / (fan_in + fan_out))
where
fan_in
is the number of input units in the weight tensor andfan_out
is the number of output units.Standalone usage:
import cvnn initializer = cvnn.initializers.GlorotUniform() values = initializer(shape=(2, 2)) # Returns a complex Glorot Uniform tensor of shape (2, 2)
Usage in a cvnn layer:
import cvnn initializer = cvnn.initializers.GlorotUniform() layer = cvnn.layers.Dense(input_size=23, output_size=45, weight_initializer=initializer)
- Real case:
-
__init__
(self, seed=None)¶ Parameters: seed – Integer. An initializer created with a given seed will always produce the same random tensor for a given shape and dtype.
-
__call__
(self, shape, dtype=tf.dtypes.complex64)¶ - Returns a real-valued tensor object initialized as specified by the initializer.
- The complex dtype input will only be used to know the limits to be used. This result must be used for the real and imaginary part separately.
Parameters: - shape – Shape of the tensor.
- dtype – Optional dtype of the tensor. Either floating or complex. ex:
tf.complex64
ortf.float32
[GLOROT-2010] | Glorot, Xavier, and Yoshua Bengio. “Understanding the difficulty of training deep feedforward neural networks.” Proceedings of the thirteenth international conference on artificial intelligence and statistics. 2010. |