200分求教：webRequest 如何获取和保持cookies!

Question

需求：想在内部系统加入google检索，用webrequest获取结果，然后按我自己的格式显示结果。我想让google按照http://www.google.cn/preferences?hl=zh-CN中设置，返回结果。最好能用WebRequest获取cookies然后，每次检索都带上cookies，最好不用webbrowse.请高手帮助！

Answer 1

http://www.cnblogs.com/goody9807/archive/2008/06/26/1107560.html

Answer 2

WebRequest是获取一个url即外界的信息的，cookie是你本机上的东西，检索的时候带上cookie是什么意思呢 ？

Answer 3

to wangjun8868: 这个例子我测试过，不行啊！请测试一下！
to enihs :不带上cookies，界面会变成google默认的格式，比如每页显示记录数。

Answer 4

程序想网页传递参数用post

Answer 5

 public static void Main(string[] args)
        {   
            if (args == null || args.Length != 1)
            {
                Console.WriteLine("Specify the URL to receive the request.");
                Environment.Exit(1);
            }
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create(args[0]);
            request.CookieContainer = new CookieContainer();
        
            HttpWebResponse response = (HttpWebResponse) request.GetResponse();
            
                        // Print the properties of each cookie.
            foreach (Cookie cook in response.Cookies)
            {
                Console.WriteLine("Cookie:");
                Console.WriteLine("{0} = {1}", cook.Name, cook.Value);
                Console.WriteLine("Domain: {0}", cook.Domain);
                Console.WriteLine("Path: {0}", cook.Path);
                Console.WriteLine("Port: {0}", cook.Port);
                Console.WriteLine("Secure: {0}", cook.Secure);
             
                Console.WriteLine("When issued: {0}", cook.TimeStamp);
                Console.WriteLine("Expires: {0} (expired? {1})", 
                    cook.Expires, cook.Expired);
                Console.WriteLine("Don't save: {0}", cook.Discard);    
                Console.WriteLine("Comment: {0}", cook.Comment);
                Console.WriteLine("Uri for comments: {0}", cook.CommentUri);
                Console.WriteLine("Version: RFC {0}" , cook.Version == 1 ? "2109" : "2965");                // Show the string representation of the cookie.
                Console.WriteLine ("String: {0}", cook.ToString());
            }
        }

Answer 6

哪位能针对google做个测试啊！！！网上这些资料，我测试不通过，不知道是什么缘故。

Answer 7

设置CookieContainer属性就可以，这个值默认是null，当它为null代表不接受cookie。

Answer 8

注意WebRequest不支持cookie的，但是HttpWebRequest，你可以显式地置换一下。

Answer 9

HttpWebRequest w = (HttpWebRequest)WebRequest.Create("http://www.google.cn");
CookieContainer cookie = new CookieContainer();
w.CookieContainer = cookie;
//...

Answer 10

        public string Login(string url, string usr, string pass)
        {
            string Return = null;            this.LoginUrl = url;            string loginstr = "rl=%2Fhome%2F&email=" + usr + "&password=" + pass;            loginstr = EncodePost(loginstr);
            byte[] replybyte = Encoding.UTF8.GetBytes(loginstr);            try
            {
                CookieContainer sparkc = new CookieContainer();
                SparkRequest = (HttpWebRequest)WebRequest.Create(url);
                SparkRequest.CookieContainer = sparkc;
                SparkRequest.ContentType = "application/x-www-form-urlencoded";
                SparkRequest.Method = "POST";                SparkRequest.ContentLength = replybyte.Length;
                Stream newStream = SparkRequest.GetRequestStream();
                newStream.Write(replybyte, 0, replybyte.Length);
                newStream.Close();                SparkResponse = (HttpWebResponse)SparkRequest.GetResponse();
                Stream dataStream = SparkResponse.GetResponseStream();
                StreamReader reader = new StreamReader(dataStream, Encoding.GetEncoding("UTF-8"));
                Return = reader.ReadToEnd();                foreach (Cookie temp in SparkResponse.Cookies)
                {
                    if (temp.Domain != "www.xxxxxxx.com")
                        temp.Domain = "www.xxxxxxxxxx.com";
                }                CkCollection = SparkResponse.Cookies; // 这句
            }
            catch
            {
                return null;
            }
            return Return;
        }

Answer 11

               
HttpWebRequest hreq=null;
CookieContainer cCookie = new CookieContainer();
private string SendRequest(string _url,string _requestString)
  {
   try
   {
    byte[] requestBytes  =  System.Text.Encoding.GetEncoding("GB2312").GetBytes(_requestString); 
    hreq= (HttpWebRequest)WebRequest.Create(_url);  
    hreq.ContentType  =  "application/x-www-form-urlencoded";  
    hreq.ContentLength= requestBytes.Length;
    hreq.Method  =  "POST";  
    hreq.CookieContainer  =  cCookie;  
    using(Stream writer =  hreq.GetRequestStream())  
    {  
     writer.Write(requestBytes,0,requestBytes.Length);
    }
}

Answer 12

我测试了，不行，是Google禁止了吗？！有朋友帮测试一下吗？

Answer 13

hreq.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2)";    
//或  hreq.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)";    
//或  hreq.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.2; zh-CN; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7"   
是否频繁获取，有可能被禁用

Answer 14

我的代码如下，我没有测试成功，如果测试成功的话，每条记录后面有个“导入BibTeX”       private void ttt()
        {
            string url = "";
            // 
            string indata = "";
            string outdata = "";
            CookieContainer mycookiecontainer = new CookieContainer();
            //新建一个cookiecontainer来存放cookie集合
            HttpWebRequest myhttpwebrequest = (HttpWebRequest)WebRequest.Create("http://scholar.google.com/scholar_setprefs?q=the&inststart=0&hl=zh-CN&lang=all&instq=&inst=10059428978349149543&num=10&newwindow=1&scis=yes&scisf=4&submit=%E4%BF%9D%E5%AD%98%E8%AE%BE%E7%BD%AE");
            myhttpwebrequest.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.2; zh-CN; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7";
            //新建一个httpwebrequest
            myhttpwebrequest.ContentType = "application/x-www-form-urlencoded";
            //myhttpwebrequest.ContentLength = indata.Length;
            myhttpwebrequest.Method = "post";
            myhttpwebrequest.CookieContainer = mycookiecontainer;
            //设置httpwebrequest的cookiecontainer为刚才建立的那个mycookiecontainer
            Stream myrequeststream = myhttpwebrequest.GetRequestStream();
            StreamWriter mystreamwriter = new StreamWriter(myrequeststream, Encoding.GetEncoding("UTF-8"));
            mystreamwriter.Write(indata);
            //把数据写入httpwebrequest的request流
            mystreamwriter.Close();
            myrequeststream.Close();
            //关闭打开对象
            HttpWebResponse myhttpwebresponse = (HttpWebResponse)myhttpwebrequest.GetResponse();
            //新建一个httpwebresponse
            myhttpwebresponse.Cookies = mycookiecontainer.GetCookies(myhttpwebrequest.RequestUri);//
            foreach (Cookie cook in myhttpwebresponse.Cookies)
            {
                Console.WriteLine("Cookie:");
                Console.WriteLine("{0} = {1}", cook.Name, cook.Value);
                Console.WriteLine("Domain: {0}", cook.Domain);
                Console.WriteLine("Path: {0}", cook.Path);
                Console.WriteLine("Port: {0}", cook.Port);
                Console.WriteLine("Secure: {0}", cook.Secure);                Console.WriteLine("When issued: {0}", cook.TimeStamp);
                Console.WriteLine("Expires: {0} (expired? {1})",
                    cook.Expires, cook.Expired);
                Console.WriteLine("Don't save: {0}", cook.Discard);
                Console.WriteLine("Comment: {0}", cook.Comment);
                Console.WriteLine("Uri for comments: {0}", cook.CommentUri);
                Console.WriteLine("Version: RFC {0}", cook.Version == 1 ? "2109" : "2965");                // Show the string representation of the cookie. 
                Console.WriteLine("String: {0}", cook.ToString());                Console.WriteLine("URI:" + myhttpwebrequest.RequestUri.ToString()); 
                Console.WriteLine("------------------------"); 
            }
            //获取一个包含url的cookie集合的cookiecollection
            Stream myresponsestream = myhttpwebresponse.GetResponseStream();
            StreamReader mystreamreader = new StreamReader(myresponsestream, Encoding.GetEncoding("UTF-8"));
            outdata = mystreamreader.ReadToEnd();
            //把数据从httpwebresponse的response流中读出
            mystreamreader.Close();
            myresponsestream.Close();
            Console.WriteLine(outdata);
            //显示"登录"
            Console.WriteLine("=========================================================================");            //拿到了cookie，再进行请求就能直接读取到登录后的内容了
            myhttpwebrequest = (HttpWebRequest)WebRequest.Create("http://scholar.google.com/scholar?q=the&hl=zh-CN&lr=");
            myhttpwebrequest.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.2; zh-CN; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7";
            myhttpwebrequest.CookieContainer = mycookiecontainer;//*
            //刚才那个cookiecontainer已经存有了cookie,把它附加到httpwebrequest中则能直接通过验证
            myhttpwebresponse = (HttpWebResponse)myhttpwebrequest.GetResponse();
            myhttpwebresponse.Cookies = mycookiecontainer.GetCookies(myhttpwebrequest.RequestUri);
            myresponsestream = myhttpwebresponse.GetResponseStream();
            mystreamreader = new StreamReader(myresponsestream, Encoding.GetEncoding("UTF-8"));
            outdata = mystreamreader.ReadToEnd();
            mystreamreader.Close();
            myresponsestream.Close();
            Console.WriteLine(outdata);
            //再次显示"登录"
            //如果把*行注释调，就显示"没有登录"        }

Answer 15

谢谢，初学COOKIES的小弟终于理解了！CookieContainer其实是定义一个空间，给HttpWebRequest存储cookies用的，HttpWebRequest自身没有cookies的存储区域！