How can I compute kernels?

Hello

I want to calculate weighted kernels (for using in a SVM classifier) in Matlab but I'm currently compeletely confused.

I would like to implement the following weighted RBF and Sigmoid kernel:

//

x and y are vectors of size n, gamma and b are constants and w is a vector of size n with weights.

The problem now is that the fitcsvm method from Matlab need two matrices as input, i.e. K(X,Y). For example the not weighted RBF and sigmoid kernel can be computed as follows:

K_rbf = exp(-gamma .* pdist2(X,Y,'euclidean').^2)
K_sigmoid = tanh(gamma*X*Y' + b);

X and Y are matrices where the rows are the data points (vectors).

How can I compute the above weighted kernels efficiently in Matlab?

Can I just apply the weights before computing the kernel? And if yes, how can this be done?

Answer 1

[John BG]

instead of pdist2, that performs rest sqrt, and then back ^2, you could simply use the operators .^ and .* to operate element wise on W X and Y:

    K_rbf = exp(-gamma * sum(sum(W.*((X-Y).^2))))

you don't mention it, but gamma seems to be a scalar, so gamma does not need the element wise product .*

2 sum() because sum only sums along a given dimension.

Why do you write that K(x,y)=exp( .. is same as K(x,y)=tanh( .. ?

If you find this answer of any help solving your question,

please click on the thumbs-up vote link,

thanks in advance

John

[e-mail]

[Sepp]

Thanks for your answer. Yes, gamma is a scalar.

My intention was not to write that K(x,y)=exp(..) is the same as K(x,y)=tanh(..). You kan view them as K_rbf and K_sigmoid. I want to calculate for both a weighted variant.

If I execute your code, I'm getting an error. First, ".-" is not defined, it should be "-" instead.

Second, does in a kernel always hold that X is of the same size as Y? If not, then X-Y will produce an error.

[John BG]

my mistake, not '.-', just '-'

the following seems to work

    N=10;X=randi(101,1,N)/100;Y=randi(101,1,N)/100
  X =
  0.67 0.04 0.86 0.95 0.69 0.77 0.76 0.40 0.67 0.18

Y =
0.72 0.04 0.28 0.05 0.10 0.84 0.71 0.33 0.96 0.04

    W=[.1 .5 .21 .43 .1 .4 .4 .4 .4 .2]
    gamma=42e-3
    K_rbf = exp(-gamma * sum(sum(W.*((X-Y).^2))))
   =
            0.98

Is the result supposed to be scalar?

What are the units of the result, energy?

or is it just probability?

John [e-mail]

[Sepp]

Thanks again.

The result is supposed to be a matrix. The result should be of the same size as

K_rbf = exp(-gamma .* pdist2(X,Y,'euclidean').^2)

but just wheighted.

I think X-Y does not work because the number of rows is not equal for X and Y. Moreover W.*(X-Y) also does not work because w is a vector with length equal to the number of columns of X and Y.