求救C#抓取网页动态数据问题

Question

求救C#抓取网页动态数据问题

Answer 1

HttpWebRequest       public static string GetPage(string url, Encoding encoding)
        {
            HttpWebRequest request = null;
            HttpWebResponse response = null;
            StreamReader reader = null;
            try
            {
                request = (HttpWebRequest)WebRequest.Create(url);
                request.UserAgent = "www.svnhost.cn";
                request.Timeout = 20000;
                request.AllowAutoRedirect = false;                response = (HttpWebResponse)request.GetResponse();
                if (response.StatusCode == HttpStatusCode.OK && response.ContentLength < 1024 * 1024)
                {
                    reader = new StreamReader(response.GetResponseStream(), encoding);
                    string html = reader.ReadToEnd();                    return html;
                }
            }
            catch
            {
            }
            finally
            {                if (response != null)
                {
                    response.Close();
                    response = null;
                }
                if (reader != null)
                    reader.Close();                if (request != null)
                    request = null;            }            return string.Empty;
        }    例如request.UserAgent = "www.svnhost.cn?uname=hehe&tid=123" 可以在url里加参数取动态数据了

Answer 2

或者用
using( System.Net.WebClient client = new  System.Net.WebClient())
{
 byte[] buffer = client.DownloadData("http://....");
...
}

Answer 3

蜘蛛程序用C#2.0实现网络蜘蛛(WebSpider)
Asp.net 文章列表