Complex Dense¶
-
class
ComplexDense
¶ Fully connected complex-valued layer.
Implements the operation:
\[\sigma(\textrm{input * weights + bias})\]- where data types can be either complex or real.
- activation (\(\sigma\)) is the element-wise activation function passed as the activation argument,
- weights is a matrix created by the layer
- bias is a bias vector created by the layer
-
__init__
(self, units, activation=None, use_bias=True, kernel_initializer=ComplexGlorotUniform(), bias_initializer=Zeros(), dtype=DEFAULT_COMPLEX_TYPE, init_technique: str = 'mirror', **kwargs)¶ Initializer of the Dense layer
Parameters: - units – Positive integer, dimensionality of the output space.
- activation – Activation function to use.
Either from
keras.activations
orcvnn.activations
. For complex dtype, onlycvnn.activations
module supported. If you don’t specify anything, no activation is applied (ie. “linear” activation: a(x) = x). - use_bias – Boolean, whether the layer uses a bias vector.
- kernel_initializer – Initializer for the kernel weights matrix.
Recomended to use a
ComplexInitializer
such ascvnn.initializers.ComplexGlorotUniform()
(default) - bias_initializer – Initializer for the bias vector.
Recomended to use a
ComplexInitializer
such ascvnn.initializers.Zeros()
(default) - dtype – Dtype of the input and layer.
- init_technique –
String. One of ‘mirror’ or ‘zero_imag’. Tells the initializer how to init complex number if the initializer was tensorflow’s built in initializers (not supporting complex numbers).
- ’mirror’ (default): Uses the initializer for both real and imaginary part. Note that some initializers such as Glorot or He will lose it’s property if initialized this way.
- ’zero_imag’: Initializer real part and let imaginary part to zero.
Code example
Let’s first get some data to test
img_r = np.array([[
[0, 1, 2],
[0, 2, 2],
[0, 5, 7]
], [
[0, 4, 5],
[3, 7, 9],
[4, 5, 3]
]]).astype(np.float32)
img_i = np.array([[
[0, 4, 5],
[3, 7, 9],
[4, 5, 3]
], [
[0, 4, 5],
[3, 7, 9],
[4, 5, 3]
]]).astype(np.float32)
img = img_r + 1j * img_i
Ok, we are now ready to run it through a ComplexDense
layer (with a ComplexFlatten
first of couse)
c_flat = ComplexFlatten()
c_dense = ComplexDense(units=10)
res = c_dense(c_flat(img.astype(np.complex64)))
assert res.shape == [2, 10]
assert res.dtype == tf.complex64
Now, to do this in a Sequential
model we can do it like:
model = tf.keras.models.Sequential()
model.add(ComplexInput(input_shape=(3, 3)))
model.add(ComplexFlatten())
model.add(ComplexDense(32, activation='cart_relu'))
model.add(ComplexDense(32))
model.output_shape
This will output (None, 32)
. You can run the data created previously with
res = model(img.astype(np.complex64))
assert res.dtype == tf.complex64
Doing now model.summary()
will output
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
complex_flatten_1 (ComplexFl (None, 9) 0
_________________________________________________________________
complex_dense_1 (ComplexDens (None, 32) 640
_________________________________________________________________
complex_dense_2 (ComplexDens (None, 32) 2112
=================================================================
Total params: 2,752
Trainable params: 2,752
Non-trainable params: 0
_________________________________________________________________
Note
If the input to the layer has a rank greater than 2, then Dense computes the dot product between the inputs and the kernel along the last axis of the inputs and axis 1 of the kernel (using tf.tensordot
). For example, if input has dimensions (batch_size, d0, d1)
, then we create a kernel with shape (d1, units)
, and the kernel operates along axis 2 of the input, on every sub-tensor of shape (1, 1, d1)
(there are batch_size * d0 such sub-tensors). The output in this case will have shape (batch_size, d0, units)
.