用c#写了个爬虫,领导说要做个DNS缓存,请问高手如何写?
获取页面的简单代码如下: string str = string.Empty;
HttpWebRequest request;
try
{
request = (HttpWebRequest)HttpWebRequest.Create(li.strUrl);
}
catch
{
return;
} //设置http头
request.AllowAutoRedirect = true;
request.AllowWriteStreamBuffering = true;
request.Referer = "";
request.Timeout = 10 * 1000;
request.UserAgent = ""; HttpWebResponse response = null;
try
{
response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
///////
}
}
获取页面的简单代码如下: string str = string.Empty;
HttpWebRequest request;
try
{
request = (HttpWebRequest)HttpWebRequest.Create(li.strUrl);
}
catch
{
return;
} //设置http头
request.AllowAutoRedirect = true;
request.AllowWriteStreamBuffering = true;
request.Referer = "";
request.Timeout = 10 * 1000;
request.UserAgent = ""; HttpWebResponse response = null;
try
{
response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
///////
}
}
别改Hosts了,这是个文本文件,数据量大了会慢,另外一个Hosts文件仅限于当前服务器,还是存储在数据库中吧,这样不会因为服务器本地设置而出现不同的结果。
DNS 的作用本来就是域名变IP嘛。
下次访问的时候直接用IP替换域名就可以啊,但是有个问题,如果对方服务器是动态IP,就访问不了了。