General regression neural network

Generalized regression neural network (GRNN) is a variation to radial basis neural networks. GRNN was suggested by D.F. Specht in 1991.[1]

GRNN can be used for regression, prediction, and classification. GRNN can also be a good solution for online dynamical systems.

GRNN represents an improved technique in the neural networks based on the nonparametric regression. The idea is that every training sample will represent a mean to a radial basis neuron.[2]

Mathematical representation

${\displaystyle Y(x)={\frac {\sum _{k=1}^{N}y_{k}K(x,x_{k})}{\sum _{k=1}^{N}K(x,x_{k})}}}$

where:

• ${\displaystyle Y(x)}$ is the prediction value of input ${\displaystyle x}$
• ${\displaystyle y_{k}}$ is the activation weight for the pattern layer neuron at ${\displaystyle k}$
• ${\displaystyle K(x,x_{k})}$ is the Radial basis function kernel (Gaussian kernel) as formulated below.

Gaussian Kernel

${\displaystyle K(x,x_{k})=e^{-d_{k}/2\sigma ^{2}},\qquad d_{k}=(x-x_{k})^{T}(x-x_{k})}$

where ${\displaystyle d_{k}}$ is the squared euclidean distance between the training samples ${\displaystyle x_{k}}$ and the input ${\displaystyle x.}$

Implementation

GRNN has been implemented in many computer languages including MATLAB,[3] R- programming language, Python (programming language) and Node.js.

Neural networks (specifically Multi-layer Perceptron) can delineate non-linear patterns in data by combining with generalized linear models by considering distribution of outcomes (sightly different from original GRNN). There have been several successful developments, including Poisson regression, ordinal logistic regression, quantile regression and multinomial logistic regression that described by Fallah in 2009. [4]

Advantages and disadvantages

Similar to RBFNN, GRNN has the following advantages:

• Single-pass learning so no backpropagation is required.
• High accuracy in the estimation since it uses Gaussian functions.
• It can handle noises in the inputs.
• It requires only less number of datasets.

The main disadvantages of GRNN are:

• Its size can be huge, which would make it computationally expensive.
• There is no optimal method to improve it.

References

1. Specht, D. F. (2002-08-06). "A general regression neural network". IEEE Transactions on Neural Networks. 2 (6): 568–576. doi:10.1109/72.97934. PMID 18282872.
2. https://minds.wisconsin.edu/bitstream/handle/1793/7779/ch2.pdf?sequence=14
3. "Generalized Regression Neural Networks - MATLAB & Simulink - MathWorks Australia".
4. Fallah, Nader; Gu, Hong; Mohammad, Kazem; Seyyedsalehi, Seyyed Ali; Nourijelyani, Keramat; Eshraghian, Mohammad Reza (2009). "Nonlinear Poisson regression using neural networks: A simulation study". Neural Computing and Applications. 18 (8): 939–943. doi:10.1007/s00521-009-0277-8. S2CID 18980875.