How can deep rectifier networks achieve linear separability and preserve distances?
This paper investigates how hidden layers of deep rectifier networks are capable of transforming two or more pattern sets to be linearly separable while preserving the distances with a guaranteed degree, and proves the universal classification power of such distance preserving rectifier networks. Th...
| Main Authors: | , , |
|---|---|
| Format: | Conference Paper |
| Published: |
2015
|
| Online Access: | http://hdl.handle.net/20.500.11937/69676 |
| _version_ | 1848762104450908160 |
|---|---|
| author | An, Senjian Boussaid, F. Bennamoun, M. |
| author_facet | An, Senjian Boussaid, F. Bennamoun, M. |
| author_sort | An, Senjian |
| building | Curtin Institutional Repository |
| collection | Online Access |
| description | This paper investigates how hidden layers of deep rectifier networks are capable of transforming two or more pattern sets to be linearly separable while preserving the distances with a guaranteed degree, and proves the universal classification power of such distance preserving rectifier networks. Through the nearly isometric nonlinear transformation in the hidden layers, the margin of the linear separating plane in the output layer and the margin of the nonlinear separating boundary in the original data space can be closely related so that the maximum margin classification in the input data space can be achieved approximately via the maximum margin linear classifiers in the output layer. The generalization performance of such distance preserving deep rectifier neural networks can be well justified by the distance-preserving properties of their hidden layers and the maximum margin property of the linear classifiers in the output layer. |
| first_indexed | 2025-11-14T10:42:16Z |
| format | Conference Paper |
| id | curtin-20.500.11937-69676 |
| institution | Curtin University Malaysia |
| institution_category | Local University |
| last_indexed | 2025-11-14T10:42:16Z |
| publishDate | 2015 |
| recordtype | eprints |
| repository_type | Digital Repository |
| spelling | curtin-20.500.11937-696762018-08-08T04:41:54Z How can deep rectifier networks achieve linear separability and preserve distances? An, Senjian Boussaid, F. Bennamoun, M. This paper investigates how hidden layers of deep rectifier networks are capable of transforming two or more pattern sets to be linearly separable while preserving the distances with a guaranteed degree, and proves the universal classification power of such distance preserving rectifier networks. Through the nearly isometric nonlinear transformation in the hidden layers, the margin of the linear separating plane in the output layer and the margin of the nonlinear separating boundary in the original data space can be closely related so that the maximum margin classification in the input data space can be achieved approximately via the maximum margin linear classifiers in the output layer. The generalization performance of such distance preserving deep rectifier neural networks can be well justified by the distance-preserving properties of their hidden layers and the maximum margin property of the linear classifiers in the output layer. 2015 Conference Paper http://hdl.handle.net/20.500.11937/69676 restricted |
| spellingShingle | An, Senjian Boussaid, F. Bennamoun, M. How can deep rectifier networks achieve linear separability and preserve distances? |
| title | How can deep rectifier networks achieve linear separability and preserve distances? |
| title_full | How can deep rectifier networks achieve linear separability and preserve distances? |
| title_fullStr | How can deep rectifier networks achieve linear separability and preserve distances? |
| title_full_unstemmed | How can deep rectifier networks achieve linear separability and preserve distances? |
| title_short | How can deep rectifier networks achieve linear separability and preserve distances? |
| title_sort | how can deep rectifier networks achieve linear separability and preserve distances? |
| url | http://hdl.handle.net/20.500.11937/69676 |