\begin{abstractEN}
Vector-valued neural learning has emerged as a promising direction in deep learning recently. Traditionally, training data for neural networks (NNs) are formulated as a vector of scalars; however, its performance may not be optimal since associations among adjacent scalars are not modeled. In this dissertation, we propose a new vector neural architecture called the Arbitrary BIlinear Product Neural Network (ABIPNN), which processes information as vectors in each neuron, and the feedforward projections are defined using arbitrary bilinear products. Such bilinear products can include circular convolution, seven-dimensional vector product, skew circular convolution, reversed-time circular convolution, or other new products not seen in previous work. Besides, we also employ vector values into convolutional neural networks (CNNs), called deep cyclic group network (DCGN) which uses the cyclic group algebra for convolutional vector-neuron learning. The input to DCGN is a three-way tensor, where the mode-$3$ dimension corresponds to the dimensionality of the input data.
% , e.g., three for RGB images. To handle vector-valued inputs, we replace scalar multiplication with circular convolution for the feedforward and backpropagation processes. As a result, every feature map and kernel map is a three-way tensor with the same mode-$3$ dimension as the input data. This way, DCGN may capture more of the relations among different data dimensions, especially for regression tasks where the target output has the same dimensionality as the input data.
As a proof-of-concept, we apply our proposed network to multispectral image denoising, singing voice separation and image inpainting. Experimental results show that ABIPNN or DCGN obtains substantial improvements when compared to conventional NNs or CNNs, suggesting that associations are learned during training. 



\flushleft\textbf{Key words: Vector neural learning, vector neural network, bilinear products, arbitrary bilinear product neural networks, backpropagation, deep cyclic group algebra, deep cyclic group networks.}
\end{abstractEN}