由于我们要测试非深度方法,因此,选择SURF特征文件作为算法的输入。SURF特征文件可以从网络上下载。下载到的文件主要包含4个.mat文件:Caltech.mat, amazon.mat, webcam.mat, dslr.mat。它们恰巧对应4个不同的领域。彼此之间两两一组,就是一个迁移学习任务。每个数据文件包含两个部分:fts为800维的特征,labels为对应的标注。在测试中,我们选择由Caltech.mat作为源域,由amazon.mat作为目标域。Office+Caltech10数据集的介绍可以在本手册的附录部分找到。
我们对数据进行加载并做简单的归一化,将最后的数据存入这四个变量中。这四个变量分别对应源域的特征和标注、以及目标域的特征和标注。代码如下:
load('Caltech.mat'); % source domainfts = fts ./ repmat(sum(fts,2),1,size(fts,2));Xs = zscore(fts,1); clear ftsYs = labels; clear labelsload('amazon.mat'); % target domainfts = fts ./ repmat(sum(fts,2),1,size(fts,2));Xt = zscore(fts,1); clear ftsYt = labels; clear labels
TCA主要进行边缘分布自适应。通过整理化简,TCA最终的求解目标是:
上述表达式可以通过Matlab自带的eigs()
函数直接求解。就是我们要求解的变换矩阵。下面我们需要明确各个变量所指代的含义:
: 由源域和目标域数据共同构成的数据矩阵
: 总的类别个数。在我们的数据集中,
: MMD矩阵。当时为全MMD矩阵;当时对应为每个类别- 的矩阵。
:单位矩阵
:平衡参数,直接给出
: 中心矩阵,直接计算得出
: 拉格朗日因子,不用理会,求解用不到
我们直接给出精炼后的源码:
function [X_src_new,X_tar_new,A] = TCA(X_src,X_tar,options)% The is the implementation of Transfer Component Analysis.% Reference: Sinno Pan et al. Domain Adaptation via Transfer Component Analysis. TNN 2011.% Inputs:%%% X_src : source feature matrix, ns * n_feature%%% X_tar : target feature matrix, nt * n_feature%%% options : option struct%%%%% lambda : regularization parameter%%%%% dim : dimensionality after adaptation (dim <= n_feature)%%%%% kernel_tpye : kernel name, choose from 'primal' | 'linear' | 'rbf'%%%%% gamma : bandwidth for rbf kernel, can be missed for other kernels% Outputs:%%% X_src_new : transformed source feature matrix, ns * dim%%% X_tar_new : transformed target feature matrix, nt * dim%%% A : adaptation matrix, (ns + nt) * (ns + nt)%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Set optionslambda = options.lambda;dim = options.dim;kernel_type = options.kernel_type;gamma = options.gamma;%% CalculateX = [X_src',X_tar'];X = X*diag(sparse(1./sqrt(sum(X.^2))));[m,n] = size(X);ns = size(X_src,1);nt = size(X_tar,1);e = [1/ns*ones(ns,1);-1/nt*ones(nt,1)];M = e * e';M = M / norm(M,'fro');H = eye(n)-1/(n)*ones(n,n);if strcmp(kernel_type,'primal')[A,~] = eigs(X*M*X'+lambda*eye(m),X*H*X',dim,'SM');Z = A' * X;Z = Z * diag(sparse(1./sqrt(sum(Z.^2))));X_src_new = Z(:,1:ns)';X_tar_new = Z(:,ns+1:end)';elseK = TCA_kernel(kernel_type,X,[],gamma);[A,~] = eigs(K*M*K'+lambda*eye(n),K*H*K',dim,'SM');Z = A' * K;Z = Z*diag(sparse(1./sqrt(sum(Z.^2))));X_src_new = Z(:,1:ns)';X_tar_new = Z(:,ns+1:end)';endend% With Fast Computation of the RBF kernel matrix% To speed up the computation, we exploit a decomposition of the Euclidean distance (norm)%% Inputs:% ker: 'linear','rbf','sam'% X: data matrix (features * samples)% gamma: bandwidth of the RBF/SAM kernel% Output:% K: kernel matrix%% Gustavo Camps-Valls% 2006(c)% Jordi (jordi@uv.es), 2007% 2007-11: if/then -> switch, and fixed RBF kernel% Modified by Mingsheng Long% 2013(c)% Mingsheng Long (longmingsheng@gmail.com), 2013function K = TCA_kernel(ker,X,X2,gamma)switch kercase 'linear'if isempty(X2)K = X'*X;elseK = X'*X2;endcase 'rbf'n1sq = sum(X.^2,1);n1 = size(X,2);if isempty(X2)D = (ones(n1,1)*n1sq)' + ones(n1,1)*n1sq -2*X'*X;elsen2sq = sum(X2.^2,1);n2 = size(X2,2);D = (ones(n2,1)*n1sq)' + ones(n1,1)*n2sq -2*X'*X2;endK = exp(-gamma*D);case 'sam'if isempty(X2)D = X'*X;elseD = X'*X2;endK = exp(-gamma*acos(D).^2);otherwiseerror(['Unsupported kernel ' ker])endend
我们将TCA方法包装成函数TCA}
。注意到TCA是一个无监督迁移方法,不需要接受label作为参数。因此,函数共接受3个输入参数:
: 源域的特征,大小为
: 目标域的特征,大小为
: 参数结构体,它包含:
: 平衡参数,可以自由给出
: 算法最终选择将数据将到多少维
: 选择的核类型,可以选择RBF、线性、或无核
: 如果选择RBF核,那么它的宽度为$$\gamma$
函数的输出包含3项:
: TCA后的源域
: TCA后的目标域
: 最终的变换矩阵
我们使用如下的代码对TCA算法进行测试:
options.gamma = 2; % the parameter for kerneloptions.kernel_type = 'linear';options.lambda = 1.0;options.dim = 20;[X_src_new,X_tar_new,A] = TCA(Xs,Xt,options);% Use knn to predict the target labelknn_model = fitcknn(X_src_new,Y_src,'NumNeighbors',1);Y_tar_pseudo = knn_model.predict(X_tar_new);acc = length(find(Y_tar_pseudo==Y_tar))/length(Y_tar);fprintf('Acc=%0.4f\n',acc);
结果显示如下:
Acc=0.4499
至此,Matlab版TCA实现完成。完整代码可以在Github上找到。