用Dataset更新access时出现 outofmemory 错误

Question

新手提问，对.net内存机制不大了解代码功能：
在制定文件夹下读取*.mdb文件，然后逐个打开判断，满足条件的，将相应信息写入HBmdb.mdb库中，不符合则跳到下个mdb文件问题：因为mdb文件数量大，10万/量级，在读取了20万mdb，写入1万条记录后，出现outofmemory错误 代码如下， public  void HBmdb(string dirpath)
        {
            int i = 0; //统计mdb数
            int j = 0; //符合条件的mdb数            StreamWriter txtwriter = new StreamWriter(Environment.CurrentDirectory + "//log.txt", true); //记录相应信息
            OleDbConnection con = new OleDbConnection(@"Provider=Microsoft.Jet.OLEDB.4.0;Data source="+Environment.CurrentDirectory+"\\HBmdb.mdb");
            con.Open();//打开HBmdb.mdb                       DataSet ds = new DataSet();
            OleDbDataAdapter AD = new OleDbDataAdapter("SELECT 馆藏号, 序号, 题目, URL, 页码 FROM xw_shengming", con);
            AD.Fill(ds, "t");
            
            OleDbCommandBuilder dd = new OleDbCommandBuilder(AD);
            DataRow rw;            foreach (string d in Directory.GetFiles(dirpath, "*.mdb", SearchOption.AllDirectories)) 
            {
                i++;
                
                OleDbConnection cn = new OleDbConnection();
                cn.ConnectionString = @"Provider=Microsoft.Jet.OLEDB.4.0;Data source=" + d;
                cn.Open();
                              
                    OleDbCommand cmd = new OleDbCommand("SELECT 馆藏号, 序号, 题目, URL, 页码 FROM xwml", cn);
                try //判断库中是否含有满足条件的表
                {
                    cmd.ExecuteScalar();
                    
                    
                }
                catch
                {
                    cmd.Dispose();
                    cn.Close();
                    cn.Dispose();
                    continue;
                }                    OleDbDataReader reader = cmd.ExecuteReader();
                   
                while (reader.Read()) //满足条件读取
                {
                   
                   
                    if (reader["题目"].ToString() == "aa")
                    {
                        j++;
                        rw = ds.Tables["t"].NewRow();
                        rw["馆藏号"] = reader["馆藏号"];
                        rw["序号"] = reader["序号"];
                        rw["题目"] = reader["题目"];
                        rw["URL"] = reader["URL"];
                        rw["页码"] = reader["页码"];
                        ds.Tables["t"].Rows.Add(rw);
                       
                        
                        if (j % 100 == 0)
                        {
                            
                            AD.Update(ds, "t");
                            ds.AcceptChanges();                           
                            ds.Clear();
                            ds.Dispose();   
                            txtwriter.WriteLine("提出/读取" + j + "/" + i);
                            txtwriter.Write("   时间：" + DateTime.Now);
                            txtwriter.WriteLine("");
                            GC.Collect();
                            
                        }
                        break;
                       
                    }
                                   }
                reader.Close();
                reader.Dispose();
                cn.Close();
                cn.Dispose();
               
           
               
           
            }
            txtwriter.WriteLine("提出/读取" + j + "/" + i);
            txtwriter.Write("   时间：" + DateTime.Now);
            txtwriter.WriteLine("");
            txtwriter.WriteLine("结束");
            txtwriter.WriteLine("");            AD.Update(ds, "t");
            ds.AcceptChanges();
            ds.Dispose();
            AD.Dispose();
            con.Close();
            con.Dispose();
            txtwriter.Close();
           
        }

Answer 1

OH MY GOD 虚拟内存不足？

Answer 2

不是，好像是到了.net的内存限制，我机器内存肯定够，对了，除错误的代码，没有GC.Collect();

Answer 3

SELECT 馆藏号, 序号, 题目, URL, 页码 FROM xw_shengming---?分批拿

Answer 4

SELECT 馆藏号, 序号, 题目, URL, 页码 FROM xw_shengming ---? 分批拿
-----------------
不知道你具体指什么，
思路是满足一条，添加一行，然后再100条一起写入

Answer 5

OleDbCommand cmd = new OleDbCommand("SELECT 馆藏号, 序号, 题目, URL, 页码 FROM xwml", cn); 这个command, 用dataReader来读的话， 如果dataReader读完的行还是保存在内存中的话，行数很多的情况下当然会造成memory leak

Answer 6

reader.Close(); 
reader.Dispose(); 
没有释放内存资源嘛？

Answer 7

你释放资源之前memory已经不够用了呢？

Answer 8

 while (reader.Read()) //满足条件读取 
                { 
               ……             
                } 
                reader.Close(); 
                reader.Dispose(); 
                cn.Close(); 
                cn.Dispose(); 
每对一个mdb读取之后就释放了reader，而一个mdb的内容肯定不至于memory不够用

Answer 9

Are you sure?You can debug into the code to check which table will cause memory leak.

Answer 10

上面那段代码，运行通过，就因为加了GC.Collect(); 所以因该不是某个mdb文件导致内存超，而是内存没有主动释放，得强制释放