假设输入数据存放在矩阵 style="">X中,而回归系数存放在向量 style="">w中,那么对于给定的数据 style="">X1,预测..." />

值得收藏!60个经济大数据,让你三分钟看明白一个月!

colspan=”2″>

这是基于《机器学习实战》一书的第八章内容总结而成的知识,有一些案例和相关的代码,即可获取。

美高梅手机官方网站 1

8.1 用线性回归找到最佳拟合曲线

style=”margin:0px; padding:0px; font-size:18px”>假设输入数据存放在矩阵 style=””>X中,而回归系数存放在向量 style=””>w中,那么对于给定的数据 style=””>X1,预测结果将会通过Y1= style=””>X1.T× style=””>w给出。如何找出误差最小的W,一般采用平方误差最小,即最小二乘法。平方误差可以写做:

美高梅手机官方网站 2

style=”margin:0px; padding:0px; font-size:18px”>用矩阵表示还可以写做( style=””>y-x*w).T*( style=””>y-x*w)。如果对w求导,得到 style=””>x.T*(y- style=””>xw),令其等于零,得到:

美高梅手机官方网站 3

style=”margin:0px; padding:0px; font-size:18px”>w帽即为当前估计出的w的最优值,表示最佳估计。一般数据矩阵 style=””>x并非有逆,而公式中需要对 style=””>x.T* style=””>x求逆,一般指伪逆,python的numpy包中,linalg函数有函数pinv求伪逆,也即 style=””>x.T* style=””>x的逆。

style=”margin:0px; padding:0px; font-size:18px”>拟合数据:

[python] view
plain > class=”tracking-ad”
style=”margin:0px; padding:0px”> copy

  1. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#!/usr/bin/env python > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  2. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”># coding=utf-8 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  3. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  4. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>from > style=”margin:0px; padding:0px; border:none; background-color:inherit”> numpy  > class=”keyword”
    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>import > style=”margin:0px; padding:0px; border:none; background-color:inherit”> *  

  5. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>from > style=”margin:0px; padding:0px; border:none; background-color:inherit”> sklearn  > class=”keyword”
    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>import > style=”margin:0px; padding:0px; border:none; background-color:inherit”> linear_model  

  6. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  7. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>def > style=”margin:0px; padding:0px; border:none; background-color:inherit”> loadDataSet(fileName):  

  8. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    numFeat  = len(open(fileName).readline().split( >     class="string"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”t” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>))- > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#-1是因为最后一列为目标值,前面为特征值 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  9. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    dataMat  = []  
    
  10. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    labelMat = []  
    
  11. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    fr = open(fileName)  
    
  12. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>for > style=”margin:0px; padding:0px; border:none; background-color:inherit”> line  > class=”keyword”
    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>in > style=”margin:0px; padding:0px; border:none; background-color:inherit”> fr.readlines():  

  13. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        lineArr = []  
    
  14. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        curLine = line.strip().split( >     class="string"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”t” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  15. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">         >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>for > style=”margin:0px; padding:0px; border:none; background-color:inherit”> i  > class=”keyword”
    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>in > style=”margin:0px; padding:0px; border:none; background-color:inherit”> range(numFeat):  

  16. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">            lineArr.append(float(curLine[i]))  
    
  17. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        dataMat.append(lineArr)                  >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#准备好数据矩阵 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  18. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        labelMat.append(float(curLine[- >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>]))      > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#准备好目标向量 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  19. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>return > style=”margin:0px; padding:0px; border:none; background-color:inherit”> dataMat, labelMat  

  20. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  21. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>def > style=”margin:0px; padding:0px; border:none; background-color:inherit”> standRegres(xArr, yArr):  

  22. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    xMat = mat(xArr)  
    
  23. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    yMat = mat(yArr).T >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#原始文件为m*n的矩阵,那么特征矩阵dataMat为m*(n-1)维,labelMat为m*1维, > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  24. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    xTx  = xMat.T*xMat  
    
  25. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>if > style=”margin:0px; padding:0px; border:none; background-color:inherit”> linalg.det(xTx) ==  > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0.0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>:                       > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#计算行列式是否为0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  26. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">         >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”this matrix is singular, cannot do inverse” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  27. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        ws = linalg.pinv(xTx)*(xMat.T*yMat)        >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#用伪逆来代替逆 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  28. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">         >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>return > style=”margin:0px; padding:0px; border:none; background-color:inherit”> ws  

  29. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    ws = xTx.I*(xMat.T*yMat)  
    
  30. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>return > style=”margin:0px; padding:0px; border:none; background-color:inherit”> ws  

  31. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  32. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">xArr, yArr = loadDataSet( >     class="string"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”ex0.txt” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  33. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">ws   = standRegres(xArr, yArr) >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#回归系数 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  34. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”最小二乘法得到的回归系数:n” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,ws  

  35. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  36. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">xMat = mat(xArr)  
    
  37. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yMat = mat(yArr)  
    
  38. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yHat = xMat*ws           >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#预测值 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  39. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  40. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#===============绘制数据集散点图和最佳拟合直线图======= > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  41. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>import > style=”margin:0px; padding:0px; border:none; background-color:inherit”> matplotlib.pyplot as plt  

  42. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">fig = plt.figure()  
    
  43. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">ax  = fig.add_subplot( >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>111 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  44. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">ax.scatter(xMat[:, >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>].flatten().A[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], yMat.T[:, > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>].flatten().A[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>])  

  45. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">xCopy = xMat.copy()  
    
  46. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yHat  = xCopy*ws  
    
  47. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">ax.plot(xCopy[:, >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], yHat)  

  48. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">plt.show()  
    
  49. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  50. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#====================皮尔逊相关系数计算两个向量的相关性=== > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  51. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yHat = xMat*ws  
    
  52. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”皮尔逊相关系数计算预测值和真实值之间的相关性:n” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,corrcoef(yHat.T, yMat)  

  53. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#================sklearn里面的线性回归================ > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  54. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  55. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">clf = linear_model.LinearRegression(fit_intercept= >     class="special"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>False > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  56. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">clf.fit(xArr,yArr)  
    
  57. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”sklearn 里面线性回归训练得到的回归系数:n” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,clf.coef_  

得到的系数即预测值与真实值之间的相关性:

美高梅手机官方网站 4

Figure 8-1:
预测得到的回归系数

style=”margin:0px; padding:0px; font-size:18px”>可以看出sklearn里面预测的和计算的回归系数是一样的。

美高梅手机官方网站 5

Figure 8-2: 拟合曲线及原始数据的散点图

python
sklearn包中有计算线性回归的机器学习算法:

美高梅手机官方网站 6

Figure 8-3: sklearn 计算线性回归方程

  style=”border: 0px; margin: 0px; padding: 0px; font-size: 16px;”>今天是11月份的最后一天,回看这一月,都公布了哪些经济数据?小编为你梳理,三分钟,让你看明白一个月!

   style=”border: 0px; margin: 0px; padding: 0px; font-size: 16px;”>宏观

8.2 局部加权线性回归

style=”margin:0px; padding:0px; font-size:18px”>局部加权线性回归(Locally
Weighted Linear Regression, LWLR)

style=”margin:0px; padding:0px; font-size:18px”>给待预测点附近的每个点赋予一定的权重,然后在这个子集上基于最小均方差来进行普通的回归。算法预测每次均需要事先取出对应的数据子集,解出的回归系数w的形式如下:

美高梅手机官方网站 7

其中 > style=””>W是一个矩阵,用来给每个数据点赋予权重。

style=”margin:0px; padding:0px; font-size:18px”>LWLR使用“核”来对附近的点赋予更高的权重,该加权模型认为样本点距离越近,越可能符合同一个线性模型。核的类型通常用高斯核。权重计算如下:

美高梅手机官方网站 8

style=”margin:0px; padding:0px; font-size:18px”>权重矩阵w为对角阵,点x与x(i)越近,w(i,i)将会越大。

美高梅手机官方网站 9

coding:

[python] view
plain > class=”tracking-ad”
style=”margin:0px; padding:0px”> copy

  1. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#===============局部加权线性回归函数================== > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  2. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>def > style=”margin:0px; padding:0px; border:none; background-color:inherit”> lwlr(testPoint, xArr, yArr, k= > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1.0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>):  

  3. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    xMat    = mat(xArr)  
    
  4. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    yMat    = mat(yArr).T  
    
  5. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    m       = shape(xMat)[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>]     > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#xMat的行数 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  6. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    weights = mat(eye((m)))      >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#创建对角阵 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  7. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>for > style=”margin:0px; padding:0px; border:none; background-color:inherit”> j  > class=”keyword”
    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>in > style=”margin:0px; padding:0px; border:none; background-color:inherit”> range(m):           > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#对每个点的计算都用上了整个数据集 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  8. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        diffMat = testPoint - xMat[j,:]  
    
  9. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        weights[j, j] = exp(diffMat*diffMat.T/(- >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>2.0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>*k** > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>2 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)) > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#高斯核,对角阵,随样本点与待预测点距离的递增,权重值大小以指数级衰减 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  10. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    xTx = xMat.T *(weights*xMat)  
    
  11. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>if > style=”margin:0px; padding:0px; border:none; background-color:inherit”> linalg.det(xTx)== > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0.0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>:  

  12. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">         >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”this matrix is singular, cannot do inverse” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  13. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        ws = linalg.pinv(xTx)*(xMat.T*(weights*yMat))  
    
  14. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    ws =xTx.I*(xMat.T*(weights*yMat))                    >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#回归参数 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  15. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>return > style=”margin:0px; padding:0px; border:none; background-color:inherit”> testPoint*ws  

  16. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  17. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>def > style=”margin:0px; padding:0px; border:none; background-color:inherit”> lwlrTest(testArr, xArr, yArr, k= > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1.0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>):  > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#当对testArr中所有点的估计,testArr=xArr时,即对所以点的全部估计 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  18. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    m = shape(testArr)[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>]  

  19. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    yHat = zeros(m)  
    
  20. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>for > style=”margin:0px; padding:0px; border:none; background-color:inherit”> i  > class=”keyword”
    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>in > style=”margin:0px; padding:0px; border:none; background-color:inherit”> range(m):  

  21. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        yHat[i] = lwlr(testArr[i], xArr, yArr, k)  
    
  22. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>return > style=”margin:0px; padding:0px; border:none; background-color:inherit”> yHat  

  23. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  24. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">xArr, yArr = loadDataSet( >     class="string"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”ex0.txt” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  25. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”原始值:” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,yArr[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>]  

  26. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”估计值,k为1.0:” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,lwlr(xArr[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], xArr, yArr, > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1.0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  27. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”估计值:k为0.001″ > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,lwlr(xArr[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], xArr, yArr, > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0.001 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  28. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#=========对所有点进行估计,绘图======== > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  29. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yHat = lwlrTest(xArr, xArr, yArr,  >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0.003 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  30. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">xMat = mat(xArr)  
    
  31. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">srtInd = xMat[:, >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>].argsort( > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  32. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">xSort  = xMat[srtInd][:, >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,:]  

  33. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  34. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">fig = plt.figure()  
    
  35. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">ax  = fig.add_subplot( >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>111 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  36. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">ax.plot(xSort[:, >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], yHat[srtInd])  

  37. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">ax.scatter(xMat[:, >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>].flatten().A[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], mat(yArr).T.flatten().A[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], s =  > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>2 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>, c =  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”red” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  38. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">plt.show()  
    

 效果:

美高梅手机官方网站 10

Figure 8-5: LWLR对第一个点的估计

美高梅手机官方网站 11

美高梅手机官方网站 12

美高梅手机官方网站 13

Figure 8-6:
k分别为1.0, 0.01, 0.003时的模拟效果,上图欠拟合,下图则过拟合

style=”margin:0px; padding:0px; font-size:18px”>缺点:LWLR增加了计算量,对每个点做预测时都必需使用整个数据集。在Figure
8-4图中,k=0.01时,对预测点x=0.5估计的时候,大多数据点的权重为0,
就不必用上整个数据集,避免增加计算量。

  1、10月规模以上工业增加值同比实际增长6.1%;社会消费品零售总额31119亿元,同比名义增10.0%。

8.3 示例:预测鲍鱼的年龄

style=”margin:0px; padding:0px; font-size:18px”>使用真实的数据观察k值对模型的效果,数据来自UCI数据集,预测鲍鱼的年龄。

美高梅手机官方网站 14

Figure 8-7: 鲍鱼数据

[python] view
plain > class=”tracking-ad”
style=”margin:0px; padding:0px”> copy

  1. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#============预测鲍鱼的年龄================ > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  2. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>def > style=”margin:0px; padding:0px; border:none; background-color:inherit”> rssError(yArr, yHatArr):  

  3. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>return > style=”margin:0px; padding:0px; border:none; background-color:inherit”> ((yArr – yHatArr)** > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>2 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>).sum()  

  4. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  5. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#========训练集上的误差 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  6. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">abX, abY = loadDataSet( >     class="string"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”abalone.txt” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  7. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yHat01   = lwlrTest(abX[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abX[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>],  > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0.1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  8. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yHat1    = lwlrTest(abX[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abX[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>],  > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  9. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yHat10   = lwlrTest(abX[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abX[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>],  > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>10 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  10. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  11. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”k=0.1,训练集上的误差:” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,rssError(abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], yHat01.T)  

  12. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”k=1,  训练集上的误差:” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,rssError(abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], yHat1.T)  

  13. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”k=10, 训练集上的误差:” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,rssError(abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], yHat10.T)  

  14. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#========测试集上的误差 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  15. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yHat01   = lwlrTest(abX[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>100 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>199 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abX[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>],  > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0.1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  16. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yHat1    = lwlrTest(abX[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>100 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>199 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abX[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>],  > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  17. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yHat10   = lwlrTest(abX[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>100 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>199 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abX[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>],  > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>10 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  18. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  19. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”k=0.1,测试集上的误差:” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,rssError(abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>100 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>199 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], yHat01.T)  

  20. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”k=1,  测试集上的误差:” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,rssError(abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>100 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>199 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], yHat1.T)  

  21. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”k=10, 测试集上的误差:” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,rssError(abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>100 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>199 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], yHat10.T)  

  22. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  23. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">ws = standRegres(abX[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>], abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>99 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>])  

  24. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yHat = mat(abX[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>100 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>199 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>])*ws  

  25. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”简单线性回归上的误差和:” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,rssError(abY[ > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>100 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>: > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>199 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>],yHat.T.A)  

    style=”margin:0px; padding:0px; font-size:18px”>使用较小的核得到较低的训练误差,但会导致过拟合,也即对新的数据预测的效果比较糟糕。在简单线性回归中,达到了与局部加权线性回归类似的效果,在未知数据中,比较效果才能选取到最佳模型。

  2、10月规模以上工业企业实现利润总额6161亿元,同比增9.8%。1-10月全国规模以上工业企业实现利润总额52567.7亿元,同比增8.6%。国家统计局:企业效益向好态势进一步稳固。

8.4  缩减系数来“理解”数据

style=”margin:0px; padding:0px; font-size:18px”>当数据的特征比样本点还多时,就不能再用线性回归和LWLR,因为计算x.T*x的逆时会出错。为此引入岭回归(ridge
regression)的概念。

岭回归

style=”margin:0px; padding:0px; font-size:18px”>岭回归即在矩阵x.T*x上加一个 > style=”margin:0px; padding:0px; color:rgb(51,51,51); font-family:arial; line-height:20.02px”> > style=”margin:0px; padding:0px; font-size:18px”>λ > style=””>I从而使得矩阵非奇异,进而能对x.T*x+ > style=”margin:0px; padding:0px; line-height:20.02px”>λI求逆,此时回归系数的计算公式变为:

美高梅手机官方网站 15

style=”margin:0px; padding:0px; color:rgb(51,51,51); font-family:arial; line-height:20.02px”> > style=”margin:0px; padding:0px; font-size:18px”>岭回归最先用来处理特征数多于样本数的情况,在估计中加入偏差,从而得到更好的估计。引入 > style=”margin:0px; padding:0px; line-height:20.02px”>λ来限制了所有w之和,通过引入该乘法项,能够减少不重要的参数,即缩减(shrinkage)技术。

style=”margin:0px; padding:0px; color:rgb(51,51,51); font-family:arial; line-height:20.02px”> > style=”margin:0px; padding:0px; font-size:18px”>缩减能够去掉不重要的参数,因此能够更好的理解数据,取得更好的预测效果。

style=”margin:0px; padding:0px; color:rgb(51,51,51); font-family:arial; line-height:20.02px”> > style=”margin:0px; padding:0px; font-size:18px”>使用岭回归和缩减技术,需要对特征做标准化处理,即所有特征都减去各自的均值并除以方差。

style=”margin:0px; padding:0px; color:rgb(51,51,51); font-family:arial; line-height:20.02px”> > style=”margin:0px; padding:0px; font-size:18px”> > style=”margin:0px; padding:0px; line-height:20.02px”>

[python] view
plain > class=”tracking-ad”
style=”margin:0px; padding:0px”> copy

  1. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#=========岭回归========== > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  2. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>def > style=”margin:0px; padding:0px; border:none; background-color:inherit”> ridgeRegres(xMat, yMat, lam= > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0.2 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>):  

  3. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    xTx   = xMat.T*xMat  
    
  4. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    denom = xTx + eye(shape(xMat)[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>])*lam  > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#得到中间结果,eye(shape(xMat)[1])得到xMat列数的单位对称阵 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  5. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>if > style=”margin:0px; padding:0px; border:none; background-color:inherit”> linalg.det(denom) == > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0.0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>:  

  6. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">         >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”this matrix is singular, cannot do inverse” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  7. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        ws = linalg.pinv(denom)*(xMat.T*yMat)  
    
  8. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    ws = denom.I*(xMat.T*yMat)  
    
  9. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>return > style=”margin:0px; padding:0px; border:none; background-color:inherit”> ws  

  10. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  11. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>def > style=”margin:0px; padding:0px; border:none; background-color:inherit”> ridgeTest(xArr, yArr):  

  12. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    xMat       = mat(xArr)  
    
  13. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    yMat       = mat(yArr).T  
    
  14. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    yMean      = mean(yMat, >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)             > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#按照行取平均,每行平均 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  15. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    yMat       = yMat - yMean             >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#对y进行标准化 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  16. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    xMeans     = mean(xMat,  >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  17. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    xVar       = var(xMat,  >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)             > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#按行取方差 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  18. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    xMat       = (xMat - xMeans)/xVar     >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#对数据进行标准化 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  19. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    numTestPts =  >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>30 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  20. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    wMat       = zeros((numTestPts, shape(xMat)[ >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>]))  

  21. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>for > style=”margin:0px; padding:0px; border:none; background-color:inherit”> i  > class=”keyword”
    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>in > style=”margin:0px; padding:0px; border:none; background-color:inherit”> range(numTestPts):                  > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#进行numTestPts次计算岭回归,每次的系数向量都放到wMat的一行中。 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  22. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        ws = ridgeRegres(xMat, yMat, exp(i- >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>10 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>))  > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#参数lam每次以exp(i-10)变化 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  23. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        wMat[i,:] = ws.T  
    
  24. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>return > style=”margin:0px; padding:0px; border:none; background-color:inherit”> wMat  

  25. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  26. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">abX, abY     = loadDataSet( >     class="string"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”abalone.txt” > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  27. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">ridgeWeights = ridgeTest(abX, abY)  
    
  28. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">fig = plt.figure()     
    
  29. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">ax  = fig.add_subplot( >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>111 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  30. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">ax.plot(ridgeWeights)        >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#ridgeWeights,为30*8的矩阵,对矩阵画图,则以每列为一个根线,为纵坐标,横坐标为range(shape(ridgeWeights)[0])也即从0到29,第一行的横坐标为0,最后一行的行坐标为29 > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  31. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">plt.show()  
    

美高梅手机官方网站 16

Figure
8-8: 岭回归参数与回归系数的关系

style=”margin:0px; padding:0px; color:rgb(51,51,51); font-family:arial; line-height:20.02px”> > style=”margin:0px; padding:0px; font-size:18px”> > style=”margin:0px; padding:0px; line-height:20.02px”>上图显示回归系数与log( > style=”margin:0px; padding:0px; line-height:20.02px”>λ)的关系,最左边 > style=”margin:0px; padding:0px; line-height:20.02px”>λ最小时,8个系数的原始值与线性回归一样,最右边8个回归系数全部缩减为0,为找到最佳参数 > style=”margin:0px; padding:0px; line-height:20.02px”>λ,还需要将ws乘以测试集比较原始测试集对比误差大小。0对应lam=exp(0-10),30对应lam=exp(30-10)

style=”margin:0px; padding:0px; font-family:arial; font-size:18px; color:rgb(51,51,51)”> style=”margin:0px; padding:0px; line-height:20.02px”>lasso

style=”margin:0px; padding:0px; font-family:arial; font-size:18px; color:rgb(51,51,51)”> > style=”margin:0px; padding:0px; line-height:20.02px”>在增加约束:所有回归系数的平方和不能大于 > style=”margin:0px; padding:0px; line-height:20.02px”>λ的条件下。普通的最小二乘法回归会得到与岭回归一样的公式。

style=”margin:0px; padding:0px; font-family:arial; font-size:18px; color:rgb(51,51,51)”> > style=”margin:0px; padding:0px; line-height:20.02px”>

style=”margin:0px; padding:0px; font-family:arial; font-size:18px; color:rgb(51,51,51)”> > style=”margin:0px; padding:0px; line-height:20.02px”>使用普通的最小二乘法回归在两个或更多的特征相关时,可能会得出一个很大的正系数和一个很大的负系数。在上述约束中,可以约束避免这个问题。

style=”margin:0px; padding:0px; font-family:arial; color:rgb(51,51,51)”> > style=”margin:0px; padding:0px; font-size:18px; line-height:20.02px”>缩减方法lasso也对回归系数做了约束:

style=”margin:0px; padding:0px; font-family:arial; color:rgb(51,51,51)”> > style=”margin:0px; padding:0px; font-size:18px; line-height:20.02px”>

style=”margin:0px; padding:0px; font-family:arial; color:rgb(51,51,51)”> > style=”margin:0px; padding:0px; font-size:18px; line-height:20.02px”>在 > style=”margin:0px; padding:0px; line-height:20.02px”>λ足够小的时候,一些系数会被迫缩减到0。约束条件为平方差的时候,可通过求偏导找到最优解,为绝对值形式时,需要使用二次规划,这里使用前向逐步回归。

style=”margin:0px; padding:0px; font-family:arial; font-size:18px; color:rgb(51,51,51)”> style=”margin:0px; padding:0px; line-height:20.02px”>前向逐步回归

style=”margin:0px; padding:0px; font-family:arial; font-size:18px; color:rgb(51,51,51)”> > style=”margin:0px; padding:0px; line-height:20.02px”>每一步都尽可能减少误差,每一步所做的决策是对某个权重增加或减少一个很小的值。

style=”margin:0px; padding:0px; font-family:arial; font-size:18px; color:rgb(51,51,51)”> > style=”margin:0px; padding:0px; line-height:20.02px”>伪代码:

style=”margin:0px; padding:0px; font-family:arial; color:rgb(51,51,51)”> > > style=”margin:0px; padding:0px; font-size:18px; line-height:20.02px”>美高梅手机官方网站 17

style=”margin:0px; padding:0px; font-family:arial; color:rgb(51,51,51)”> > style=”margin:0px; padding:0px; font-size:18px; line-height:20.02px”>coding:

style=”margin:0px; padding:0px; font-family:arial; color:rgb(51,51,51)”> > > style=”margin:0px; padding:0px; font-size:18px; line-height:20.02px”>

[python] view
plain > > class=”tracking-ad”
style=”margin:0px; padding:0px”> copy

  1. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> > >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#=============前向逐步回归================= > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  2. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> > >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>def > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> regularize(xMat):            > > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#按行标准化 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  3. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    inMat = xMat.copy()  
    
  4. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    inMeans = mean(inMat,  > >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  5. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    inVar   = var(inMat,  > >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  6. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    inMat   = (inMat-inMeans)/inVar  
    
  7. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     > >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>return > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> inMat  

  8. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  9. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> > >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>def > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> stageWise(xArr, yArr, eps= > > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0.01 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>, numIt= > > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>100 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>):  

  10. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    xMat      = mat(xArr)  
    
  11. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    yMat      = mat(yArr).T  
    
  12. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    yMean     = mean(yMat,  > >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  13. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    yMat      = yMat - yMean  
    
  14. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    xMat      = regularize(xMat)         > >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#对xMat进行标准化 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  15. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    m,n       = shape(xMat)  
    
  16. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    returnMat = zeros((numIt, n))  
    
  17. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    ws        = zeros((n, > >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>))  

  18. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    wsTest    = ws.copy()  
    
  19. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">    wsMax     = ws.copy()  
    
  20. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     > >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>for > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> i  > > class=”keyword”
    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>in > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> range(numIt):  

  21. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">         > >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#print ws.T > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  22. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        lowestError = inf  
    
  23. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">         > >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>for > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> j  > > class=”keyword”
    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>in > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> range(n):  

  24. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">             > >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>for > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> sign  > > class=”keyword”
    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>in > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> [- > > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>, > > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>1 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>]:          > > class=”comment”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#分别计算增加或减少该特征对误差的影响 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  25. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">                wsTest     = ws.copy()  
    
  26. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">                wsTest[j] += eps*sign  
    
  27. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">                yTest      = xMat*wsTest  
    
  28. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">                rssE       = rssError(yMat.A, yTest.A) > >     class="comment"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,130,0); background-color:inherit”>#得到平方误差进行比较 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  

  29. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">                 > >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>if > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> rssE < lowestError:  

  30. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">                    lowestError = rssE  
    
  31. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">                    wsMax       = wsTest  
    
  32. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        ws = wsMax.copy()  
    
  33. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">        returnMat[i,:] = ws.T  
    
  34. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">     > >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>return > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> returnMat  

  35. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  36. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">xArr,yArr = loadDataSet( > >     class="string"
    

    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”abalone.txt” > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  37. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> > >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”步长为0.01,迭代次数为200:n” > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,stageWise(xArr, yArr,  > > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0.01 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,  > > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>200 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  38. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> > >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>  > > class=”string”
    style=”margin:0px; padding:0px; border:none; color:rgb(0,0,255); background-color:inherit”>”步长为0.001,迭代次数为5000:n” > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,stageWise(xArr, yArr,  > > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0.001 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>,  > > class=”number”
    style=”margin:0px; padding:0px; border:none; background-color:inherit”>5000 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  39. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">  
    
  40. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">xMat = mat(xArr)  
    
  41. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yMat = mat(yArr).T  
    
  42. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">xMat = regularize(xMat)  
    
  43. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yM   = mean(yMat,  > >     class="number"
    

    style=”margin:0px; padding:0px; border:none; background-color:inherit”>0 > > style=”margin:0px; padding:0px; border:none; background-color:inherit”>)  

  44. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">yMat = yMat- yM  
    
  45. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit">weights = standRegres(xMat, yMat.T)  
    
  46. style="margin:0px; padding:0px; border:none; color:rgb(0,0,0); background-color:inherit"> > >     class="keyword"
    

    style=”margin:0px; padding:0px; font-weight:700; background-color:inherit; color:rgb(0,102,153); font-size:14px; word-wrap:normal; border:none”>print > > style=”margin:0px; padding:0px; border:none; background-color:inherit”> weights.T  

美高梅手机官方网站 18

  3、31省份前三季GDP增速排名出炉:重庆、西藏和贵州位列前三,实现两位数增长;黑龙江、山西和辽宁增速低于全国水平。

Figure 8-9: 逐步线性回归迭代得到的回归系数

美高梅手机官方网站 19

style=”margin:0px; padding:0px; font-size:18px”>逐步线性回归算法可以帮助人们理解现有的模型并做出改进。构建了一个模型后,可以运行算法找出重要的特征,这样就有可能及时停止对那些不重要特征的收集。如上面步长0.01,迭代200次后为w1和w6都为0,表明它们对目标值没有影响,也即这些特征很可能是不需要的。

style=”margin:0px; padding:0px; font-size:18px”>当应用缩减方法(逐步线性回归或岭回归)时,模型也就增加了偏差(bias),与此同时却减少了模型的方差。

  4、1-10月,国有企业营业总收入367398.3亿元,同比增1.5%;国有企业营业总成本357969.5亿元,同比增1.6%;国有企业利润总额19196.5亿元,同比增0.4%。

8.5 权衡偏差与方差

style=”margin:0px; padding:0px”>方差:模型训练时,模型值之间的差异。

style=”margin:0px; padding:0px; font-size:18px”>偏差:模型预测值与数据之间的差异。

美高梅手机官方网站 20

style=”margin:0px; padding:0px; font-size:18px”>通过缩减法,可以将一些系数缩减成很小的值或者为0,增大模型偏差,同时模型复杂度降低

  5、10月,全社会用电量4890亿千瓦时,同比增长7.0%。1-10月,全国全社会用电量累计48776亿千瓦时,同比增长4.8%。

  6、10月,全国一般公共预算收入15359亿元,同比增5.9%。其中,个人所得税708亿元,同比增22.5%。印花税143亿元,同比降27.8%。

  7、10月末社会融资规模存量为152.41万亿元,同比增长12.7%。前10月社会融资规模增量累计为14.35万亿元,比去年同期多1.78万亿元。10月社会融资规模增量为8963亿元,比去年同期多3370亿元。

  8、10月末,M2余额151.95万亿元,同比增长11.6%;M1余额46.54万亿元,同比增长23.9%。10月人民币贷款增加6513亿元,人民币存款增加1.21万亿元。

  9、10月CPI同比涨2.1%,结束连续4个月的“1时代”,重返“2时代”;PPI环比涨0.7%,同比涨1.2%。

  10、10月我国进出口总值2.05万亿元人民币,下降0.6%。其中,出口1.19万亿元,下降3.2%;进口8606亿元,增长3.2%;贸易顺差3252.5亿元,收窄16.8%。

  美高梅手机官方网站 21

   style=”border: 0px; margin: 0px; padding: 0px; font-size: 16px;”>部委

  11、国家发改委批复京津冀城际铁路网规划,到2020年,基本实现京津石中心城区与周边城镇0.5至1小时通勤圈,京津保0.5至1小时交通圈。

美高梅手机官方网站,  12、国家发改委:1-10月,全国施工项目计划总投资1022874亿元,同比增9.1%;新开工项目计划总投资410913亿元,同比增21.8%。

  13、国家发改委等多部委印发《冰雪运动发展规划(2016—2025年)》,到2020年我国冰雪产业总规模达6000亿元,到2025年达10000亿元。

  14、财政部:2014至2016年,中央财政累计安排165亿元,支持实施新一轮退耕还林还草3010万亩,其中,2016年安排90亿元,新增任务1510万亩。

  15、财政部:截至今年9月底,全国地方已经发行新增地方政府债券11347亿元,占全年新增限额的96.2%。

  16、外汇局:中国11月末人民币合格境外机构投资者(RQFII)获批额度为5237.75亿元,上月末为5149.88亿元。

  17、外汇局:10月,我国国际收支口径的国际货物和服务贸易收入13074亿元人民币,支出11120亿元,顺差1955亿元。

  18、外汇局:三季度经常账户顺差712亿美元,环比增11%。其中,货物贸易顺差1371亿美元,环比增9%。

  19、外汇局:10月,银行结汇7274亿元人民币,售汇8258亿元人民币,结售汇逆差984亿元人民币。

  20、银监会:截至10月末,银行业金融机构总资产219.61万亿元,同比增16.5%;总负债202.47万亿元,同比增16.4%。

  美高梅手机官方网站 22

  21、交通部:1到10月,公路建设完成投资14478亿元,同比增8.8%,增速较前三季度加快1.1个百分点。前10月,全国农村公路累计完成新改建规模23.6万公里,提前完成20万公里的年度任务。

  22、农业部:10月全国农业农村经济运行稳健,农产品批发价格指数环比降6.53%、同比涨7.62%,“菜篮子”产品批发价格指数环比降7.76%、同比涨9.40%。

  23、国土资源部:前三季度,全国国有建设用地供应31.88万公顷,同比降4.6%。全国土地出让面积14.31万公顷,同比降4.8%;合同成交价款2.28万亿元,同比增24.8%。

  24、环保部:2015年,全国246个大中城市生活垃圾产生量约为18564万吨,处置率达97.3%。其中,北京城市生活垃圾产生量最大,为790.3万吨。

   style=”border: 0px; margin: 0px; padding: 0px; font-size: 16px;”>产业

  25、10月规模以上工业原油产量1605.1万吨,同比降11.3%;天然气产量108.3亿立方米,同比降1.4%。

  26、1-10月,全国粗钢产量67296万吨,同比增0.7%;钢材产量94829万吨,同比增2.4%。

  27、前9月,全国风电平均利用小时数1251小时,同比降66小时;风电弃风电量394.7亿千瓦时;平均弃风率19%。

  28、数据显示,10月我国广义乘用车销量222万辆,同比增长20%。前10月,广义乘用车销量1867万辆,同比增长15.2%。前10月,中国新能源汽车累计销量约33.7万辆,同比增长82.2%。

  29、数据显示,截至10月底,35个城市新建商品住宅库存总量为23726万平方米,环比降1.4%,同比降10.8%。合肥、南京和上海库存量同比降幅较大,分别为59.9%、55.0%和46.5%。

  30、10月,70大中城市中,一、二线城市房地产市场明显降温。与上半月比,10月下半月,深圳和成都新建商品住宅价格继续下降,北京、天津、上海、厦门和郑州等5个城市由升转降。

发表评论

电子邮件地址不会被公开。 必填项已用*标注

CopyRight © 2015-2020 美高梅网址 All Rights Reserved.
网站地图xml地图