我写了一个网页的采集程序，有些问题，请大家来帮忙解决一下

Question

我写了一个网页的采集程序，昨天晚上测试的还没什么问题，但今天运行就出问题了，采集到一定程度就出错。出错信息如下：java.io.IOException: Server returned HTTP response code: 502 for URL: http://game.bet007.com/betinfomoreimport.aspx?ballclass=1&userid=126619&username=4948138
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1149)
at java.net.URL.openStream(URL.java:1007)
at load.GetWebPage.openUrl(GetWebPage.java:175)
at load.GetWebPage.getMainPage(GetWebPage.java:104)
at load.GetWebPage.getPage(GetWebPage.java:83)
at load.GetWebPage.main(GetWebPage.java:42)
Exception in thread "main" java.lang.NullPointerException
at java.io.Reader.<init>(Reader.java:61)
at java.io.InputStreamReader.<init>(InputStreamReader.java:55)
at load.GetWebPage.openUrl(GetWebPage.java:180)
at load.GetWebPage.getMainPage(GetWebPage.java:104)
at load.GetWebPage.getPage(GetWebPage.java:83)
at load.GetWebPage.main(GetWebPage.java:42)
哪位大大看看是哪里的问题啊

Answer 1

程序如下:package load;import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.Vector;public class GetWebPage {
InputStreamReader isr;
int fd=3;//连输的天数
int ff=6;//采集的天数
String path="/1.htm";
String web="http://game.bet007.com/newbetlist.aspx?page=";
//String web="http://game.bet007.com/payoffimport.aspx?day=10";
//String weburl=new String("http://game.bet007.com/payoffimport.aspx?day=10");
/**
 * @param args
 */
public static void main(String[] args) {
// TODO 自动生成方法存根
try {
//new GetWebPage().getMainPage("181027","巡警大队");
GetWebPage web=new GetWebPage();
/*System.out.print("请输入网址:");
BufferedReader br=new BufferedReader(new InputStreamReader(System.in));
web.setWeb(br.readLine());
System.out.print("请输入文件名及路径：（例如：c:\\1.htm）");
br=new BufferedReader(new InputStreamReader(System.in));*/
System.out.println("请输入连输的天数");
BufferedReader br=new BufferedReader(new InputStreamReader(System.in));
web.setFd(br.readLine());
System.out.println("请输入采集的天数");
br=new BufferedReader(new InputStreamReader(System.in));
web.setFF(br.readLine());
System.out.println("现在开始采集请等待");
web.getPage();
//new GetWebPage().getPage();

} catch (IOException e) {
// TODO 自动生成 catch 块
System.out.println(e.getMessage());
}
} 
public void setFd(String fd){
this.fd=Integer.parseInt(fd);
//System.out.println(fd);
}
public void setFF(String ff){
this.ff=Integer.parseInt(ff);
//System.out.println(ff);
}
public void setFile(String file){
path=file;
}
public void setWeb(String web){
this.web=web;
} 
public void getPage() throws IOException{
Vector<String> in=new Vector<String>();
for(int l=1;l<=7;l++){
web=web+l;
System.out.println("采集网址："+web);
BufferedReader string=this.openUrl(web);
int i=1;
while(string.ready()){
String x=string.readLine();
if(!(x.indexOf("betinfomoreimport.aspx?ballclass=1&userid=")==-1)&&x.indexOf("查看")==-1){
x=x.substring(x.indexOf("userid="));
//System.out.println(x);
String z=x.substring(x.indexOf("=")+1,x.indexOf("&"));
String y=x.substring(x.indexOf("name=")+5,x.indexOf(" "));
//System.out.println(y);
//x="http://game.bet007.com/betinfomoreimport.aspx?ballclass=1&userid="+z+"&import=1&username="+y;
//in.add(z);
//in.add(y);
System.out.println("正在采集第"+i+"个:名字="+y+",id="+z);
if(this.getMainPage(z, y)){
//System.out.println("有");
in.add("<tr><td>"+i+"</td><td>"+y+"</td><td><div align=\"center\"><a href=\"http://game.bet007.com/betinfomoreimport.aspx?ballclass=1&userid="+z+"&import=1&username="+y+"\" target=\"_blank\">http://game.bet007.com/betinfomoreimport.aspx?ballclass=1&userid="+z+"&import=1&username="+y+"</a></div></td></tr>");
//in.add(y+"      http://game.bet007.com/betinfomoreimport.aspx?ballclass=1&userid="+z+"&import=1&username="+y);
}
i++;
}

}
}
//System.out.println(in.size());
if(this.saveFile(in)){
System.out.println("采集失败");
}else{
System.out.println("采集成功");
}
}
public boolean getMainPage(String id,String name) throws IOException {
Vector<String> in=new Vector<String>();
int f=0;
//BufferedReader string=this.openUrl("http://game.bet007.com/betinfomoreimport.aspx?ballclass=1&userid=90725&username=stgao6888");
BufferedReader string=this.openUrl("http://game.bet007.com/betinfomoreimport.aspx?ballclass=1&userid="+id+"&username="+name);
//System.out.println("http://game.bet007.com/betinfomoreimport.aspx?ballclass=1&userid="+id+"&username="+name);
//System.out.println(string.ready());
 while(string.ready()){
String x=string.readLine();
//System.out.println(x);
if(!(x.indexOf("<tr align=\"center\" bgcolor=\"#FFFFFF\">")==-1)){
f++;
//System.out.println("x");
for(int i=1;i<=15;i++){
//UserInfo user=new UserInfo();
x=string.readLine();
/*if(i==2){
//UserInfo user=new UserInfo();
x=x.substring(x.indexOf(">"));
//user.saveDate(x.substring(1,x.indexOf(" ")));
//System.out.println(x.substring(1,x.indexOf(" ")));
}*/
if(i==15){
//System.out.println(x);
if(!(x.indexOf("color=")==-1)){
x=x.substring(x.indexOf("color=")+6);
in.add(x.substring(0,3));
//user.saveMain(x.substring(0,3));
//System.out.println(x.substring(0,3));
}
}
//in.add(user);
}
}
if(f==ff){
break;
}
}
 int x=0;
 //System.out.println(in.size());
 for(int i=0;i<in.size();i++){
 String user;
 //UserInfo user=new UserInfo();
 user=in.get(i).toString();
 if(user.equals("999")){
 //System.out.println(i+":输"+user);
x++; 
 }else{
 //System.out.println(i+":正"+user);
 x=0;
 }
 if(x==fd){
 break;
 }
 }
 if(x==fd){
 //System.out.println("有");
 return true;
 }else{
 //System.out.println("无");
 return false;
 }
}
private BufferedReader openUrl(String weburl){
URL url = null;
InputStream ins=null;
InputStreamReader chars=null;
try {
url=new URL(weburl);
} catch (MalformedURLException e) {
// TODO 自动生成 catch 块
e.printStackTrace();
}
try {
//url.openConnection().getInputStream();
ins=url.openStream();
} catch (IOException e) {
// TODO 自动生成 catch 块
e.printStackTrace();
}
chars=new InputStreamReader(ins);
BufferedReader b=new BufferedReader(chars);

return b;
}
private boolean saveFile(Vector in){
File file=new File(path);
if(!file.exists()){
try {
file.createNewFile();
} catch (IOException e) {
// TODO 自动生成 catch 块
System.out.println(e.getMessage());
return true;
}
}
FileWriter page;
try {
page = new FileWriter(file);
} catch (IOException e) {
// TODO 自动生成 catch 块
System.out.println(e.getMessage());
return true;
}
BufferedWriter bufferedWriter=new BufferedWriter(page);
try {
bufferedWriter.write("<table width=\"842\" border=\"1\" align=\"center\">");
bufferedWriter.write("<tr>");
bufferedWriter.write("<td>序号</td>");
bufferedWriter.write("<td>人物帐号</td>");
bufferedWriter.write("<td>连接</td>");
bufferedWriter.write("</tr>");
} catch (IOException e1) {
// TODO 自动生成 catch 块
System.out.println(e1.getMessage());
}

for(int i=0;i<in.size();i++){
try {
bufferedWriter.write(in.get(i).toString());
} catch (IOException e) {
// TODO 自动生成 catch 块
System.out.println(e.getMessage());
return true;
}
try {
bufferedWriter.newLine();
} catch (IOException e) {
// TODO 自动生成 catch 块
System.out.println(e.getMessage());
return true;
}
//System.out.println(in.get(i).toString());
}
try {
bufferedWriter.write("</table>");
} catch (IOException e1) {
// TODO 自动生成 catch 块
e1.printStackTrace();
}
try {
bufferedWriter.close();
} catch (IOException e) {
// TODO 自动生成 catch 块
System.out.println(e.getMessage());
return true;
}
return false;
}
}

Answer 2

网页不允许你访问 换个地址试试 例如www.sohu.com 分析他的主页