Description<\/h2>\n
<\/a>b = ridge(y,X,k)<\/tt><\/tt>\u00a0returns a vector\u00a0b<\/tt>\u00a0of coefficient estimates for a multilinear ridge regression of the responses in\u00a0y<\/tt>\u00a0on the predictors in\u00a0X<\/tt>.\u00a0X<\/tt>\u00a0is an\u00a0n<\/em>-by-p<\/em>\u00a0matrix of\u00a0p<\/em>\u00a0predictors at each of\u00a0n<\/em>\u00a0observations.\u00a0y<\/tt>\u00a0is an\u00a0n<\/em>-by-1 vector of observed responses.\u00a0k<\/tt>\u00a0is a vector of ridge parameters. If\u00a0k<\/tt>\u00a0has\u00a0m<\/em>\u00a0elements,\u00a0b<\/tt>\u00a0is\u00a0p<\/em>-by-m<\/em>. By default,\u00a0b<\/tt>\u00a0is computed after centering and scaling the predictors to have mean 0 and standard deviation 1. The model does not include a constant term, and\u00a0X<\/tt>\u00a0should not contain a column of 1s.<\/p>\n b = ridge(y,X,k,scaled)<\/tt>\u00a0uses the {0<\/tt>,1<\/tt>}-valued flag\u00a0scaled<\/tt>\u00a0to determine if the coefficient estimates in\u00a0b<\/tt>\u00a0are restored to the scale of the original data.\u00a0ridge(y,X,k,0)<\/tt>\u00a0performs this additional transformation. In this case,\u00a0b<\/tt>\u00a0contains\u00a0p<\/em>+1 coefficients for each value of\u00a0k<\/tt>, with the first row corresponding to a constant term in the model.\u00a0ridge(y,X,k,1)<\/tt>\u00a0is the same as\u00a0ridge(y,X,k)<\/tt>. In this case,\u00a0b<\/tt>\u00a0contains\u00a0p<\/em>\u00a0coefficients, without a coefficient for a constant term.<\/p>\n The relationship between\u00a0b0 = ridge(y,X,k,0)<\/tt>\u00a0and\u00a0b1 = ridge(y,X,k,1)<\/tt>\u00a0is given by<\/p>\n This can be seen by replacing the\u00a0x<\/i>i<\/em><\/sub>\u00a0(i<\/i>\u00a0= \u06f1, …,\u00a0n<\/i>) in the multilinear model\u00a0y<\/i>\u00a0=\u00a0b<\/i>0<\/sub>0<\/sup>\u00a0+\u00a0b<\/i>1<\/sub>0<\/sup>x<\/i>1<\/sub>\u00a0+ … +\u00a0b<\/i>n<\/em><\/sub>0<\/sup>x<\/i>n<\/em><\/sub>\u00a0with the\u00a0z<\/em>-scores\u00a0z<\/i>i<\/em><\/sub>\u00a0= (x<\/i>i<\/em><\/sub>\u00a0\u2013\u00a0\u03bc<\/i>i<\/em><\/sub>)\/\u03c3<\/i>i\u00a0<\/em><\/sub>, and replacing\u00a0y<\/i>\u00a0with\u00a0y<\/i>\u00a0\u2013\u00a0\u03bc<\/i>y<\/em><\/sub>.<\/p>\n In general,\u00a0b1<\/tt>\u00a0is more useful for producing plots in which the coefficients are to be displayed on the same scale, such as a\u00a0<\/a>ridge trace<\/em>\u00a0(a plot of the regression coefficients as a function of the ridge parameter).\u00a0b0<\/tt>\u00a0is more useful for making predictions.<\/p>\n Coefficient estimates for multiple linear regression models rely on the independence of the model terms. When terms are correlated and the columns of the design matrix\u00a0X<\/i>\u00a0have an approximate linear dependence, the matrix (X<\/i>T<\/em><\/sup>X<\/i>)\u2013\u06f1<\/sup>\u00a0becomes close to singular. As a result, the least squares estimate<\/p>\n becomes highly sensitive to random errors in the observed response\u00a0y<\/i>, producing a large variance. This situation of\u00a0<\/a>multicollinearity<\/i>\u00a0can arise, for example, when data are collected without an experimental design.<\/p>\n Ridge regression<\/i>\u00a0addresses the problem by estimating regression coefficients using<\/p>\n where\u00a0k<\/i>\u00a0is the\u00a0<\/a>ridge parameter<\/i>\u00a0and\u00a0I<\/i>\u00a0is the identity matrix. Small positive values of\u00a0k<\/i>\u00a0improve the conditioning of the problem and reduce the variance of the estimates. While biased, the reduced variance of ridge estimates often result in a smaller mean square error when compared to least-squares estimates.<\/p>\n <\/a><\/p>\n Load the data in\u00a0acetylene.mat<\/tt>, with observations of the predictor variables\u00a0x1<\/tt>,\u00a0x2<\/tt>,\u00a0x3<\/tt>, and the response variable\u00a0y<\/tt>:<\/p>\n Plot the predictor variables against each other:<\/p>\n Note the correlation between\u00a0x1<\/tt>\u00a0and the other two predictor variables.<\/p>\n Use\u00a0ridge<\/tt>\u00a0and\u00a0x2fx<\/tt><\/a>\u00a0to compute coefficient estimates for a multilinear model with interaction terms, for a range of ridge parameters:<\/p>\n Plot the ridge trace:<\/p>\n The estimates stabilize to the right of the plot. Note that the coefficient of the\u00a0x2x3<\/tt>\u00a0interaction term changes sign at a value of the ridge parameter \u2248 \u06f5 \u00d7 \u06f1\u06f0\u2013\u06f4<\/sup>.<\/p>\n","protected":false},"excerpt":{"rendered":" Ridge regression Syntax b = ridge(y,X,k) b = ridge(y,X,k,scaled) Description b = ridge(y,X,k)\u00a0returns a vector\u00a0b\u00a0of coefficient estimates for a multilinear ridge regression of the responses in\u00a0y\u00a0on the predictors in\u00a0X.\u00a0X\u00a0is an\u00a0n-by-p\u00a0matrix of\u00a0p\u00a0predictors at each of\u00a0n\u00a0observations.\u00a0y\u00a0is an\u00a0n-by-1 vector of observed responses.\u00a0k\u00a0is a vector of ridge parameters. If\u00a0k\u00a0has\u00a0m\u00a0elements,\u00a0b\u00a0is\u00a0p-by-m. By default,\u00a0b\u00a0is computed after centering and scaling the predictors to […]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[19],"tags":[],"class_list":["post-280","post","type-post","status-publish","format-standard","hentry","category-19","category-19-id","post-seq-1","post-parity-odd","meta-position-corners","fix"],"_links":{"self":[{"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/posts\/280"}],"collection":[{"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=280"}],"version-history":[{"count":3,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/posts\/280\/revisions"}],"predecessor-version":[{"id":282,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=\/wp\/v2\/posts\/280\/revisions\/282"}],"wp:attachment":[{"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=280"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=280"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vua.nadiran.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=280"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}} m = mean(X);\r\n s = std(X,0,1)';\r\n b1_scaled = b1.\/s;\r\n b0 = [mean(y)-m*temp; b1_scaled]<\/pre>\n
![\"\"](\"http:\/\/www.kxcad.net\/cae_MATLAB\/toolbox\/stats\/normal_estimate_ug.gif\")
<\/p>\n![\"\"](\"http:\/\/www.kxcad.net\/cae_MATLAB\/toolbox\/stats\/ridge_estimate_ug.gif\")
<\/p>\nExample<\/h2>\n
load acetylene<\/pre>\n
subplot(1,3,1)\r\nplot(x1,x2,'.')\r\nxlabel('x1'); ylabel('x2'); grid on; axis square\r\n\r\nsubplot(1,3,2)\r\nplot(x1,x3,'.')\r\nxlabel('x1'); ylabel('x3'); grid on; axis square\r\n\r\nsubplot(1,3,3)\r\nplot(x2,x3,'.')\r\nxlabel('x2'); ylabel('x3'); grid on; axis square<\/pre>\n![\"\"](\"http:\/\/www.kxcad.net\/cae_MATLAB\/toolbox\/stats\/ridge1.gif\")
<\/p>\nX = [x1 x2 x3];\r\nD = x2fx(X,'interaction');\r\nD(:,1) = []; % No constant term\r\nk = 0:1e-5:5e-3;\r\nb = ridge(y,D,k);<\/pre>\n
figure\r\nplot(k,b,'LineWidth',2)\r\nylim([-100 100])\r\ngrid on \r\nxlabel('Ridge Parameter') \r\nylabel('Standardized Coefficient') \r\ntitle('{\\bf Ridge Trace}') \r\nlegend('constant','x1','x2','x3','x1x2','x1x3','x2x3')<\/pre>\n![\"\"](\"http:\/\/www.kxcad.net\/cae_MATLAB\/toolbox\/stats\/ridge2.gif\")
<\/p>\n