怎么扒取网站数据

Question

数据扒取

Answer 1

ASP.NET模拟登陆，把自己伪装成浏览器

Answer 2

用httpwebrequest来抓取页面，用正则表达式解析获取需要的内容，得费点功夫的

Answer 3

1.先把网页源码下载下来处理分析.正则筛选2.转换到自己的网站上
       <table width="914" border="0" cellspacing="0" cellpadding="0">
             <tr>
               <th><a href="?f=symbol&o=asc">代码</a></th>
               <th><a href="?f=name&o=asc">名称</a></th>
               <th><a href="?f=zjj&o=asc">中间价</a></th>
   <th><a href="?f=cmj&o=asc">钞买价</a></th>
               <th><a href="?f=hmj&o=asc">汇买价</a></th>
               <th><a href="?f=chmj&o=asc">钞/汇卖价</a></th>
               <th><a href="?f=chg&o=asc">涨跌额</a></th>
               <th><a href="?f=chg_pct&o=asc">涨跌幅&darr;</a></th>
               <th>日期</th>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=AUD" target="_blank">AUD</a></td>
               <td><a href="quote.php?symbol=AUD" target="_blank">澳大利亚元</a></td>
               <td>567.5600</td>
           <td>552.7900</td>
               <td>570.3900</td>
               <td>574.3900</td>
               <td><span class="Ared">3.8500</span></td>
               <td><span class="Ared">0.6830%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=CAD" target="_blank">CAD</a></td>
               <td><a href="quote.php?symbol=CAD" target="_blank">加拿大元</a></td>
               <td>554.2100</td>
           <td>540.1400</td>
               <td>557.3500</td>
               <td>561.8300</td>
               <td><span class="Ared">3.6700</span></td>
               <td><span class="Ared">0.6666%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=JPY" target="_blank">JPY</a></td>
               <td><a href="quote.php?symbol=JPY" target="_blank">日元</a></td>
               <td>6.0472</td>
           <td>5.8814</td>
               <td>6.0687</td>
               <td>6.1113</td>
               <td><span class="Ared">0.0321</span></td>
               <td><span class="Ared">0.5337%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=KRW" target="_blank">KRW</a></td>
               <td><a href="quote.php?symbol=KRW" target="_blank">韩国元</a></td>
               <td>0.5783</td>
           <td>0.5581</td>
               <td>0.0000</td>
               <td>0.0000</td>
               <td><span class="Ared">0.0017</span></td>
               <td><span class="Ared">0.2948%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=PHP" target="_blank">PHP</a></td>
               <td><a href="quote.php?symbol=PHP" target="_blank">菲律宾比索</a></td>
               <td>13.8100</td>
           <td>13.3300</td>
               <td>13.7500</td>
               <td>13.8700</td>
               <td><span class="Ared">0.0400</span></td>
               <td><span class="Ared">0.2905%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=GBP" target="_blank">GBP</a></td>
               <td><a href="quote.php?symbol=GBP" target="_blank">英镑</a></td>
               <td>1018.9700</td>
           <td>993.1400</td>
               <td>1024.7700</td>
               <td>1033.0100</td>
               <td><span class="Ared">2.8100</span></td>
               <td><span class="Ared">0.2765%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=NZD" target="_blank">NZD</a></td>
               <td><a href="quote.php?symbol=NZD" target="_blank">新西兰元</a></td>
               <td>529.3800</td>
           <td>515.7400</td>
               <td>532.1600</td>
               <td>535.9000</td>
               <td><span class="Ared">1.4100</span></td>
               <td><span class="Ared">0.2671%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=SGD" target="_blank">SGD</a></td>
               <td><a href="quote.php?symbol=SGD" target="_blank">新加坡元</a></td>
               <td>490.2100</td>
           <td>473.1800</td>
               <td>488.2500</td>
               <td>492.1700</td>
               <td><span class="Ared">1.2500</span></td>
               <td><span class="Ared">0.2556%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=THB" target="_blank">THB</a></td>
               <td><a href="quote.php?symbol=THB" target="_blank">泰国铢</a></td>
               <td>19.0800</td>
           <td>18.4200</td>
               <td>19.0000</td>
               <td>19.1600</td>
               <td><span class="Ared">0.0400</span></td>
               <td><span class="Ared">0.2101%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=" target="_blank"></a></td>
               <td><a href="quote.php?symbol=" target="_blank">马来西亚林吉特</a></td>
               <td>187.3700</td>
           <td>0.0000</td>
               <td>187.5900</td>
               <td>188.9100</td>
               <td><span class="Ared">0.3400</span></td>
               <td><span class="Ared">0.1818%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=MOP" target="_blank">MOP</a></td>
               <td><a href="quote.php?symbol=MOP" target="_blank">澳门元</a></td>
               <td>77.6500</td>
           <td>74.9500</td>
               <td>77.5600</td>
               <td>77.8500</td>
               <td><span class="Ared">0.0900</span></td>
               <td><span class="Ared">0.1160%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=RUB" target="_blank">RUB</a></td>
               <td><a href="quote.php?symbol=RUB" target="_blank">俄罗斯卢布</a></td>
               <td>17.4300</td>
           <td>0.0000</td>
               <td>17.3900</td>
               <td>17.5300</td>
               <td><span class="Ared">0.0200</span></td>
               <td><span class="Ared">0.1149%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=USD" target="_blank">USD</a></td>
               <td><a href="quote.php?symbol=USD" target="_blank">美元</a></td>
               <td>614.6500</td>
           <td>614.5400</td>
               <td>619.5100</td>
               <td>621.9900</td>
               <td><span class="Ared">0.2500</span></td>
               <td><span class="Ared">0.0407%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=HKD" target="_blank">HKD</a></td>
               <td><a href="quote.php?symbol=HKD" target="_blank">港币</a></td>
               <td>79.2200</td>
           <td>79.2000</td>
               <td>79.8400</td>
               <td>80.1400</td>
               <td><span class="Ared">0.0090</span></td>
               <td><span class="Ared">0.0114%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=CHF" target="_blank">CHF</a></td>
               <td><a href="quote.php?symbol=CHF" target="_blank">瑞士法郎</a></td>
               <td>701.2500</td>
           <td>676.8800</td>
               <td>698.4500</td>
               <td>704.0600</td>
               <td><span class="Agreen">-0.6600</span></td>
               <td><span class="Agreen">-0.0940%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=DKK" target="_blank">DKK</a></td>
               <td><a href="quote.php?symbol=DKK" target="_blank">丹麦克朗</a></td>
               <td>114.6100</td>
           <td>110.6300</td>
               <td>114.1500</td>
               <td>115.0700</td>
               <td><span class="Agreen">-0.1600</span></td>
               <td><span class="Agreen">-0.1394%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=NOK" target="_blank">NOK</a></td>
               <td><a href="quote.php?symbol=NOK" target="_blank">挪威克朗</a></td>
               <td>102.6400</td>
           <td>99.0700</td>
               <td>102.2300</td>
               <td>103.0500</td>
               <td><span class="Agreen">-0.1500</span></td>
               <td><span class="Agreen">-0.1459%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=EUR" target="_blank">EUR</a></td>
               <td><a href="quote.php?symbol=EUR" target="_blank">欧元</a></td>
               <td>847.5700</td>
           <td>826.0900</td>
               <td>852.4100</td>
               <td>859.2500</td>
               <td><span class="Agreen">-1.5600</span></td>
               <td><span class="Agreen">-0.1837%</span></td>
               <td>2014-03-27</td>
             </tr>
             <tr>
               <td><a href="quote.php?symbol=SEK" target="_blank">SEK</a></td>
               <td><a href="quote.php?symbol=SEK" target="_blank">瑞典克朗</a></td>
               <td>96.0600</td>
           <td>92.7200</td>
               <td>95.6800</td>
               <td>96.4400</td>
               <td><span class="Agreen">-0.7500</span></td>
               <td><span class="Agreen">-0.7747%</span></td>
               <td>2014-03-27</td>
             </tr>
          </table>

Answer 4

用HttpWebRequest去访问这个网站，然后去解析返回的内容

Answer 5

http://www.cnblogs.com/yiki/archive/2007/08/28/872528.html

Answer 6

WEB开发就用WebClient或HttpWebRequest
WINFORM开发就用WebBrowser