自动登录获取网页源码 500错误!

Question

private void btnDownload_Click(object sender, System.EventArgs e)
{
string strResult; if (HttpContext.Current.Application["cookieheader"] != null)
{
cookieheader = (string)HttpContext.Current.Application["cookieheader"];
}
else
{
//Login into the website and keep the cookie for the session in the application variable
string strLogin = Login("https://https-sec.xfab.com/fo/logon.do", "username=username&password=password") ;
}
strResult = getPage("https://https-sec.xfab.com/fo/downloadstaticreport.do", "id=0") ;
}public static string getPage(String url, String paramList) 
{
HttpWebResponse res = null;
string strResult = "";
try 
{
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.Method = "POST";
req.KeepAlive = true;
req.ContentType = "application/x-www-form-urlencoded";
CookieContainer cookieCon = new CookieContainer();
req.CookieContainer = cookieCon;
req.CookieContainer.SetCookies(new Uri(url),cookieheader);
StringBuilder UrlEncoded = new StringBuilder();
Char[] reserved = {'?', '=', '&'};
byte[] SomeBytes = null; if (paramList != null) 
{
int i=0, j;
while(i<paramList.Length)
{
j=paramList.IndexOfAny(reserved, i);
if (j==-1)
{
UrlEncoded.Append(HttpUtility.UrlEncode(paramList.Substring(i, paramList.Length-i)));
break;
}
UrlEncoded.Append(HttpUtility.UrlEncode(paramList.Substring(i, j-i)));
UrlEncoded.Append(paramList.Substring(j,1));
i = j+1;
}
SomeBytes = Encoding.UTF8.GetBytes(UrlEncoded.ToString());
req.ContentLength = SomeBytes.Length;
Stream newStream = req.GetRequestStream();
newStream.Write(SomeBytes, 0, SomeBytes.Length);
newStream.Close();
} 
else 
{
req.ContentLength = 0;
}
res = (HttpWebResponse)req.GetResponse();
Stream ReceiveStream = res.GetResponseStream();
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
StreamReader sr = new StreamReader( ReceiveStream, encode );
Char[] read = new Char[256];
int count = sr.Read( read, 0, 256 );
while (count > 0) 
{
String str = new String(read, 0, count);
strResult += str;
count = sr.Read(read, 0, 256);
}
} 
catch(Exception e) 
{
strResult = e.ToString();
} 
finally 
{
if ( res != null ) 
{
res.Close();
}
} return strResult;
}在执行到 res = (HttpWebResponse)req.GetResponse();
时 出现System.Net.WebException: 远程服务器返回错误: (500) 内部服务器错误。 其他页面在获取的时候是没问题的,是什么原因?

Answer 1

https://https-sec.xfab.com/fo/downloadstaticreport.do
https要包进证书进去才行

Answer 2

访问普通的http没问题 https要麻烦点

Answer 3

Red_angelX(八戒) ( ) 信誉：100    Blog  2007-03-29 15:35:17  得分: 0  
 
 
   https://https-sec.xfab.com/fo/downloadstaticreport.do
https要包进证书进去才行
  
 
-----------------------------------------------------------
so fast!

Answer 4

但是在获取https://https-sec.xfab.com/fo/logon.do
这个页面的源文件时是可以的,
https://https-sec.xfab.com/fo/downloadstaticreport.do?id=0
这个页面就不行

Answer 5

private string GetUrlContext(string curl)
{  
  string[] textArray = new string[2];
    textArray = curl.Split(new char[] { '?' });
    HttpWebRequest request2 = (HttpWebRequest) WebRequest.Create(new Uri(textArray[0]));
    ServicePointManager.Expect100Continue = false;
    request2.Method = "POST";
    CookieContainer container = new CookieContainer();
    request2.CookieContainer = container;
    string s = textArray[1];
    byte[] bytes = new ASCIIEncoding().GetBytes(s);
    request2.ContentType = "application/x-www-form-urlencoded";
    request2.ContentLength = bytes.Length;
    Stream requestStream = request2.GetRequestStream();
    requestStream.Write(bytes, 0, bytes.Length);
    requestStream.Close();
     HttpWebResponse response = null;
    try
    {
        response = (HttpWebResponse) request2.GetResponse();
    }
    catch (WebException exception2)
    {
        if (exception2.Status == WebExceptionStatus.ProtocolError)
        {
            response = (HttpWebResponse) exception2.Response;
        }
        else if (exception2.Status == WebExceptionStatus.ConnectFailure)
        {
            MessageBox.Show("\u8fde\u63a5\u670d\u52a1\u5668\u5931\u8d25,\u8bf7\u68c0\u67e5\u7f51\u7edc", "\u9519\u8bef");
        }
        else
        {
            MessageBox.Show(exception2.ToString());
        }
    }
    response.Cookies = request2.CookieContainer.GetCookies(request2.RequestUri);
    Stream stream = response.GetResponseStream();
    StreamReader reader2 = new StreamReader(stream, Encoding.Default);
    text = reader2.ReadToEnd();
    stream.Close();
    response.Close();
    reader2.Close();
    return text;
}  
你写得Post太乱了~

Answer 6

"System.Net.WebException: 远程服务器返回错误: (500) 内部服务器错误。\r\n   at System.Net.HttpWebRequest.CheckFinalStatus()\r\n   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)\r\n   at System.Net.HttpWebRequest.GetResponse()\r\n   at HttpDownload.frmHttpDownload.getPage(String url, String paramList) in d:\\work\\httpdownload\\frmhttpdownload.cs:line 534"https要包进证书?如果是这个问题,该怎么做?

Answer 7

to: Red_angelX(八戒)
用你写的方法就只能得到要求登陆页面的源文件了..

Answer 8

修改了Cookies之后得到的却又是相当于输入https://https-sec.xfab.com/fo/downloadstaticreport.do?id=1这样的出错页面

Answer 9

500 Internal Server Error

Answer 10

try
        {
            response = (HttpWebResponse) request.GetResponse();
        }
        catch (WebException exception)
        {
            if (exception.Status == WebExceptionStatus.ProtocolError)
            {
                response = (HttpWebResponse) exception.Response;
            }
}
看看500错误里面具体写的是什么错误

Answer 11

string[] textArray = new string[2];
textArray = curl.Split(new char[] { '?' });
HttpWebRequest request2 = (HttpWebRequest) WebRequest.Create(new Uri(textArray[0]));
ServicePointManager.Expect100Continue = false;
request2.Method = "POST";
CookieContainer container = new CookieContainer();
request2.CookieContainer = container;request2.CookieContainer.SetCookies(new Uri(textArray[0]),cookieheader); //不加这句得到的是未登陆页面的源文件,加上则出现了500错误...string s = textArray[1];
byte[] bytes = new ASCIIEncoding().GetBytes(s);
request2.ContentType = "application/x-www-form-urlencoded";
request2.ContentLength = bytes.Length;
Stream requestStream = request2.GetRequestStream();
requestStream.Write(bytes, 0, bytes.Length);
requestStream.Close();
HttpWebResponse response = null;
try
{
response = (HttpWebResponse) request2.GetResponse();
}

Answer 12

catch (WebException exception2)
{
if (exception2.Status == WebExceptionStatus.ProtocolError)
{
response = (HttpWebResponse) exception2.Response;
}
}
catch 进入之后是exception2.Status == WebExceptionStatus.ProtocolError
从服务器接收到的响应完成了，但他指示了一个协议级错误．例如,HTTP协议错误(如401访问被拒绝)使用此状态．
是ＨＴＴＰＳ的原因吗？

Answer 13

//global myCookieContainer存放登录Cookies
CookieContainer myCookieContainer = new CookieContainer();
request2.CookieContainer=myCookieContainer;
类似这样来赋值  不用SetCookies方法试试

Answer 14

得到的是
<head><title>JRun Servlet Error</title></head><h1>500 Internal Server Error</h1><body>\r\n&lt;PRE&gt;\n&lt;B&gt;/fo/downloadstaticreport.do:&lt;/B&gt;\n\nnull\njava.lang.NullPointerException\n\tat de.taskarena.xfab.frontoffice.module.reports.DownloadStaticReportAction.performAction(DownloadStaticReportAction.java:55)\n\tat de.taskarena.xfab.servlet.XFabActionBase.perform(XFabActionBase.java:60)\n\tat org.apache.struts.action.ActionServlet.processActionPerform(ActionServlet.java:1787)\n\tat org.apache.struts.action.ActionServlet.process(ActionServlet.java:1586)\n\tat org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:510)\n\tat de.taskarena.xfab.servlet.XFabServlet.doPost(XFabServlet.java:32)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:772)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:865)\n\tat allaire.jrun.servlet.JRunSE.service(JRunSE.java:1024)\n\tat allaire.jrun.servlet.JRunSE.runServlet(JRunSE.java:936)\n\tat allaire.jrun.servlet.JRunRequestDispatcher.forward(JRunRequestDispatcher.java:88)\n\tat allaire.jrun.servlet.JRunSE.service(JRunSE.java:1163)\n\tat allaire.jrun.servlet.JRunSE.service(JRunSE.java:1153)\n\tat allaire.jrun.servlet.JvmContext.dispatch(JvmContext.java:330)\n\tat allaire.jrun.jrpp.ProxyEndpoint.run(ProxyEndpoint.java:382)\n\tat allaire.jrun.ThreadPool.run(ThreadPool.java:272)\n\tat allaire.jrun.WorkerThread.run(WorkerThread.java:75)\n\n&lt;/PRE&gt;\n</body>\r\n登陆后输入https://https-sec.xfab.com/fo/downloadstaticreport.do?id=4打开的页面也是出现这样的错误

Answer 15

登陆后输入https://https-sec.xfab.com/fo/downloadstaticreport.do?id=4打开的页面也是出现这样的错误
----------
这又是什么错误 直接从IE登录的吗?

Answer 16

是的,直接从IE登陆只有id=0是对的

Answer 17

CookieContainer myCookieContainer = new CookieContainer();
request2.CookieContainer=myCookieContainer;
类似这样来赋值  不用SetCookies方法试试结果也一样...

Answer 18

程序里是
strResult = GetUrlContext("https://https-sec.xfab.com/fo/downloadstaticreport.do?id=0");
curl=https://https-sec.xfab.com/fo/downloadstaticreport.do?id=0
是用id=0的

Answer 19

找找连https要加上哪几句吧 我这没现成的代码
另外从id!=0的情况来看得到的似乎又是正确结果-_-#

Answer 20

呵呵~~头晕了,有个页面是URL里不需加参数的也能得到正确的结果
就是这个页面有id=0就不对了

Answer 21

http://blogs.msdn.com/adarshk/archive/2005/04/22/410925.aspx

Answer 22

HTTPS包进证书怎么做?
应该是这个问题,抓取其他网页都是对的

Answer 23

错错错！如果你的Url后面有参数就很有可能失败，你是是将参数去掉再试

Answer 24

抓取其他的asp页面之类的就是有参数的也是能抓取的
只是这个网站有参数就失败,没参数是对的.

Answer 25

我以前一遇到过这个问题，后来发先是Url太长，将所有的参去掉就好了

Answer 26

HttpWebRequest.ClientCertificates
http://weblogs.asp.net/jan/archive/2003/12/04/41154.aspx  
  http://community.csdn.net/Expert/topic/3752/3752836.xml?temp=.3829767