这个问题困扰了很久,也问了很多人,一直没有得到解决,希望能有人帮助解决。
这是一个C#的爬虫程序,其中用到了BerkeleyDB数据库,在运行了大约三个小时之后,插入BerkeleyDB数据时发生异常,并且异常不断重复,即无法继续写入数据。这是写入BerkeleyDB数据的函数:
public void PutOneUrlToUnVisited(Url url)
{
// lock (this)
{
try
{
m_LockOfPutUnVisitedUrl.AcquireWriterLock(1000);
BdbFormatter newformatter = new BdbFormatter(2, 40960);
new UrlField(newformatter, true);
DbEntry keyEntry, dataEntry;
keyEntry = newformatter.ToDbEntry<int>(url.ImportancePriority);
dataEntry = newformatter.ToDbEntry<Url>(url);
WriteStatus status = DBS.UnVisited.Put(null, ref keyEntry, ref dataEntry);//这里插入数据时出现的异常,UnVisited是一个bdb表
DBS.UnVisited.Sync();
m_LockOfPutUnVisitedUrl_Num.AcquireWriterLock(1000);
UnVisited_Num++;
if (m_LockOfPutUnVisitedUrl_Num.IsWriterLockHeld) m_LockOfPutUnVisitedUrl_Num.ReleaseWriterLock();
if (status != WriteStatus.Success)
{
ShowEvent("写入UNVISITED表有误", WebCrawlerEventReason.MyMessage);
if (status == WriteStatus.KeyExist)
{
XBaseEvent.ShowException.Show("Existed!");
}
else
{
XBaseEvent.ShowException.Show("Not Found!");
}
}
}
catch (System.Exception ex) //异常在这里catch
{
ShowEvent("写入UNVISITED表有误", WebCrawlerEventReason.MyMessage);
}
if (m_LockOfPutUnVisitedUrl.IsWriterLockHeld) m_LockOfPutUnVisitedUrl.ReleaseWriterLock();
}出现的异常是DB_RUNRECOVERY: Fatal error, run database recovery;
在bdb存数据的文件夹下的error.txt中重复出现的条目是:
C:\Users\walikb\Desktop\database: PANIC: fatal region error detected; run recovery谢谢!C# BerkeleyDB
这是一个C#的爬虫程序,其中用到了BerkeleyDB数据库,在运行了大约三个小时之后,插入BerkeleyDB数据时发生异常,并且异常不断重复,即无法继续写入数据。这是写入BerkeleyDB数据的函数:
public void PutOneUrlToUnVisited(Url url)
{
// lock (this)
{
try
{
m_LockOfPutUnVisitedUrl.AcquireWriterLock(1000);
BdbFormatter newformatter = new BdbFormatter(2, 40960);
new UrlField(newformatter, true);
DbEntry keyEntry, dataEntry;
keyEntry = newformatter.ToDbEntry<int>(url.ImportancePriority);
dataEntry = newformatter.ToDbEntry<Url>(url);
WriteStatus status = DBS.UnVisited.Put(null, ref keyEntry, ref dataEntry);//这里插入数据时出现的异常,UnVisited是一个bdb表
DBS.UnVisited.Sync();
m_LockOfPutUnVisitedUrl_Num.AcquireWriterLock(1000);
UnVisited_Num++;
if (m_LockOfPutUnVisitedUrl_Num.IsWriterLockHeld) m_LockOfPutUnVisitedUrl_Num.ReleaseWriterLock();
if (status != WriteStatus.Success)
{
ShowEvent("写入UNVISITED表有误", WebCrawlerEventReason.MyMessage);
if (status == WriteStatus.KeyExist)
{
XBaseEvent.ShowException.Show("Existed!");
}
else
{
XBaseEvent.ShowException.Show("Not Found!");
}
}
}
catch (System.Exception ex) //异常在这里catch
{
ShowEvent("写入UNVISITED表有误", WebCrawlerEventReason.MyMessage);
}
if (m_LockOfPutUnVisitedUrl.IsWriterLockHeld) m_LockOfPutUnVisitedUrl.ReleaseWriterLock();
}出现的异常是DB_RUNRECOVERY: Fatal error, run database recovery;
在bdb存数据的文件夹下的error.txt中重复出现的条目是:
C:\Users\walikb\Desktop\database: PANIC: fatal region error detected; run recovery谢谢!C# BerkeleyDB
解决方案 »
免费领取超大流量手机卡,每月29元包185G流量+100分钟通话, 中国电信官方发货