direkt zum Inhalt springen

direkt zum Hauptnavigationsmenü

Sie sind hier

TU Berlin

Page Content

Hyper-Ellipsoidal Conjugate Gradient Descent

On this page you can find a MATLAB implementation of the Hyper-Ellipsoidal Conjugate Gradient Descent algorithm, that can be used for sparse opitimization of second order kernel methods like kernel-PCA, kernel-SFA (slow feature analysis), or kernel-CCA (canonical correlation analysis). The following two files you may download, use, redistribute, and/or modify under the terms of the GNU General Public License.

hecgd.m - hyper-ellipsoidal conjugate gradient descent algorithm
errsokm.m - error function for sparse second order kernel methods

How to use these files is described here.

If you use this software in publications, please cite:

Sparse Optimization for Second Order Kernel Methods
Citation key Vollgraf2006b
Author Vollgraf, R. and Obermayer, K.
Title of Book IJCNN 2006 Conference Proceedings
Pages 145 – 152
Year 2006
ISBN 0-7803-9490-9
DOI 10.1109/IJCNN.2006.246672
Publisher IEEE
Abstract We present a new optimization procedure which is particularly suited for the solution of second-order kernel methods like e.g. Kernel-PCA. Common to these methods is that there is a cost function to be optimized, under a positive definite quadratic constraint, which bounds the solution. For example, in Kernel-PCA the constraint provides unit length and orthogonal (in feature space) principal components. The cost function is often quadratic which allows to solve the problem as a generalized eigenvalue problem. However, in contrast to Support Vector Machines, which employ box constraints, quadratic constraints usually do not lead to sparse solutions. Here we give up the structure of the generalized eigenvalue problem in favor of a non-quadratic regularization term added to the cost function, which enforces sparse solutions. To optimize this more ``complicated'' cost function, we introduce a modified conjugate gradient descent method. Starting from an admissible point, all iterations are carried out inside the subspace of admissible solutions, which is defined by the hyper-ellipsoidal constraint surface.
Link to original publication Download Bibtex entry

Zusatzinformationen / Extras

Quick Access:

Schnellnavigation zur Seite über Nummerneingabe