原文地址:http://www.thedelphimagazine.com/samples/1328/article.htm
Memory Lost And Found... And Released by Roy Nelson This article has been long in the making: it all started about two years ago, after a conversation with a previous colleague. The conversation centred on the fact that they were experiencing problems with their ISAPI DLLs written in Delphi. It seemed as if the DLLs were leaking 4Kb of memory every time they were loaded into memory. Eventually they switched over to CGI to fix the problem. My reaction was ‘What? My Delphi losing memory? Never!’ I did not investigate this any further at that time. However, about two months ago I saw someone mention a ‘known’ leak in the VCL on a newsgroup. I was intrigued, as the magic 4Kb number was mentioned. Everyone seemed to know about this, and even what was causing the leak. What was going on here? I then had a look at the code and I saw why it had not been plugged. This leak in Delphi DLLs exhibits itself only under specific conditions. The most important condition is that the Controls unit has to be referenced somewhere in the units used to compile the DLL. Secondly, the DLL must not make use of packages. Lastly, the DLL has to be explicitly loaded and unloaded using the LoadLibraryXX and FreeLibrary API calls. The leak is small: normally 4Kb, but it can be bigger. It occurs every time the DLL is brought into memory within a process. So, in server situations (like ISAPI, depending on how IIS is set up), where the DLL might be constantly loaded and unloaded, the leak can have a disastrous effect, depleting your system of all available virtual memory. Another situation where the leak can be dangerous is when in-process COM servers are used, as the DLL will be loaded and unloaded as the server is accessed and released. See Listing 1 for sample DLL code that causes the leak. Listing 2 shows some code that demonstrates the leak.
Memory Lost And Found... And Released by Roy Nelson This article has been long in the making: it all started about two years ago, after a conversation with a previous colleague. The conversation centred on the fact that they were experiencing problems with their ISAPI DLLs written in Delphi. It seemed as if the DLLs were leaking 4Kb of memory every time they were loaded into memory. Eventually they switched over to CGI to fix the problem. My reaction was ‘What? My Delphi losing memory? Never!’ I did not investigate this any further at that time. However, about two months ago I saw someone mention a ‘known’ leak in the VCL on a newsgroup. I was intrigued, as the magic 4Kb number was mentioned. Everyone seemed to know about this, and even what was causing the leak. What was going on here? I then had a look at the code and I saw why it had not been plugged. This leak in Delphi DLLs exhibits itself only under specific conditions. The most important condition is that the Controls unit has to be referenced somewhere in the units used to compile the DLL. Secondly, the DLL must not make use of packages. Lastly, the DLL has to be explicitly loaded and unloaded using the LoadLibraryXX and FreeLibrary API calls. The leak is small: normally 4Kb, but it can be bigger. It occurs every time the DLL is brought into memory within a process. So, in server situations (like ISAPI, depending on how IIS is set up), where the DLL might be constantly loaded and unloaded, the leak can have a disastrous effect, depleting your system of all available virtual memory. Another situation where the leak can be dangerous is when in-process COM servers are used, as the DLL will be loaded and unloaded as the server is accessed and released. See Listing 1 for sample DLL code that causes the leak. Listing 2 shows some code that demonstrates the leak.
解决方案 »
- 编程时用鼠标拖放VCL控制时画面会变花屏是怎么回事?
- 开发类似的程序，要怎么做才能提高开发效率？
- 感兴趣的近来闲聊一下Delphi跟Java的比较!!!!进者有分
- 高深的问题,如何知道图标在exe文件中的正解位置呢?
- [求救！]动态数组的下标用变量循环时遇到的问题
- 非常急！！！非常熟悉dxdbgrid的朋友请近来，请教个问题！！！
- sql server怎麼確定表裡面某一列全部是null??
- 小弟有礼了，Ehlib2.4版（中文财务帐套），安装ehlibreg.pas，上次装起来，可今天却装不起来了。请安装过的大虾进来帮帮小弟。
- 跟delphi同行们探讨一下如何自定义控件的问题，人人都有分。
- 如何监控键盘按下了那个键
- Delhpi combobox 清单中,可以做上一条,下一条吗?为什么我做的不稳定?
- SQL高手请进!
uses
Controls;
{$R *.RES}
begin
end.
? Listing 2procedure TForm1.Button1Click(Sender:TObject);
var
i : integer;
LibHandle : THandle;
begin
ShowMessage('Go');
for i := 0 to 200 do begin
LibHandle := Loadlibrary('LeakyDLL.DLL');
if LibHandle <> 0 then
FreeLibrary(LibHandle);
end;
ShowMessage('Stopped');
end; The leak can easily be observed using Task Manager on Windows NT or 2000, or by using some of the process status API (PSAPI) functions available on these operating systems. Listing 3 shows a unit which could be used to retrieve the same value as Mem Usage (the unit is also on this month’s disk). ? Listing 3unit psapiunit;
interface
uses
Windows;
type
PPROCESS_MEMORY_COUNTERS = ^PROCESS_MEMORY_COUNTERS;
PROCESS_MEMORY_COUNTERS = record
cb : DWORD;
PageFaultCount : DWORD;
PeakWorkingSetSize : DWORD;
WorkingSetSize : DWORD; //Task managers MemUsage number
QuotaPeakPagedPoolUsage : DWORD;
QuotaPagedPoolUsage : DWORD;
QuotaPeakNonPagedPoolUsage : DWORD;
QuotaNonPagedPoolUsage : DWORD;
PagefileUsage : DWORD; //TaskMan's VM Size number
PeakPagefileUsage : DWORD;
end;
TProcessMemoryCounters = PROCESS_MEMORY_COUNTERS;
function GetProcessMemoryInfo(Process : THandle;
var MemoryCounters : ProcessMemoryCounters;
cb : DWORD) : BOOL; stdcall;
function ProcessMemoryUsage(ProcessID : DWORD): DWORD;
implementation
uses Sysutils;
function GetProcessMemoryInfo; external 'psapi.dll';
function ProcessMemoryUsage(ProcessID : DWORD): DWORD;
var
ProcessHandle : THandle;
MemCounters : TProcessMemoryCounters;
begin
Result := 0;
ProcessHandle := OpenProcess(PROCESS_QUERY_INFORMATION or
PROCESS_VM_READ, false, ProcessID );
try
if GetProcessMemoryInfo( ProcessHandle, MemCounters,
sizeof(MemCounters)) then
Result := MemCounters.WorkingSetSize;
finally
CloseHandle( ProcessHandle );
end;
end;
end. The code that shows the leak (see Listing 4) is slightly different to the code in Listing 2, as we need to get all the pages mapped into memory, to get a baseline from which to start monitoring any leaked pages. This prevents us getting small, false positive leaks. However, executing the checking code twice will generally force all of the used pages into memory. The companion disk includes a small test application which calls the various DLLs (in the project group), that is: both leaking and non-leaking DLLs. ? Listing 4procedure TForm1.RadioGroup1Click(Sender: TObject);
var
i : integer;
LibHandle : THandle;
MemUsageStart : integer;
DllName : string;
ProcessID : DWord;
begin
...
ProcessID := GetCurrentProcessID;
// Dummy call to force PSAPI.DLL to be loaded and added to
// the workingset
MemUsageStart := ProcessMemoryUsage(ProcessID);
// Dummy Load and unload the DLL once to get a more
// accurate picture of the full workingset
LibHandle := Loadlibrary(PChar(DllName));
if LibHandle <> 0 then
FreeLibrary(LibHandle)
else
RaiseLastWin32Error;
MemUsageStart := ProcessMemoryUsage(ProcessID);
for i := 1 to 200 do begin
LibHandle := Loadlibrary(PChar(DllName));
if LibHandle <> 0 then
FreeLibrary(LibHandle)
else
RaiseLastWin32Error;
Caption := 'Memory Lost '+ IntToStr(
(ProcessMemoryUsage(ProcessID) - MemUsageStart)
div 1024) +' kbytes Loaded '+IntToStr(i);
Application.ProcessMessages;
end;
...
end; Figure 1 shows the memory leak being displayed. Note the same leakage will be seen using the Windows Task Manager.
The reason for this leak is that, deep in the bowels of the VCL, memory is allocated for a very specific purpose. Unfortunately, this memory cannot be released safely, for a variety of reasons which we will look at later. The allocated memory is used as a linked list of thunks, or stubs; each stub is used to splice a VCL windowed component into the Windows messaging mechanism. The code generated for one of these stubs enables Windows to call what it thinks is a normal window message procedure (WNDPROC for the old-timers), where in fact it is calling a method of your object. To create such a stub you need to pass the method (TWndMethod) of your object to a function called MakeObjectInstance (from Classes.pas in Delphi 6 and Forms.pas in Delphi 5 and earlier). MakeObjectInstance will return a pointer to a stub that can be called by the Windows OS. All of this happens without you needing to be aware of internal workings, when you create a TWinControl derived class, when you call AllocateHWnd or when you use MakeObject- Instance to subclass a window. How Does MakeObjectInstance Work? MakeObjectInstance allocates 4Kb blocks of memory using Virtual- Alloc (with special flags: we will get back to these as they are rather important). Each block (see Listing 5) contains a link to the next allocated block, a few other fields needed for the stubs to work, and an array of 314 entries (used in a linked list). We are mainly interested in this array, as each entry contains the small code stub which enables your method to act like a normal WNDPROC function (see Listing 6). Each empty stub in the stub array points to the next empty stub. ? Listing 5const
InstanceCount = 313;
PInstanceBlock = ^TInstanceBlock;
TInstanceBlock = packed record
Next: PInstanceBlock;
Code: array[1..2] of Byte;
WndProcPtr: Pointer;
Instances: array[0..InstanceCount] of TObjectInstance;
end;
? Listing 6PObjectInstance = ^TObjectInstance;
TObjectInstance = packed record
Code: Byte;
Offset: Integer;
case Integer of
0: (Next: PObjectInstance);
1: (Method: TMethod);
end; So for each VCL window that is created in an application, a call is made to MakeObjectInstance and one slot in the array will be used. Once the limit for the stub array is reached, a new block is allocated (using VirtualAlloc) and added to a linked list. As windows are destroyed, the used stub is ed as empty by being added back to the empty stub linked list. These blocks are the reason for the leak, as they are never freed. The reasons why the VCL code does not free these memory blocks seem to be as follows. First, there is no need to do so in a standalone EXE, as the process memory will collapse when the EXE shuts down. Second, if you are using packages this is not a problem, as the memory blocks will be shared amongst the packages. Once again the memory will be vaporised as soon as the EXE shuts down. Lastly, in a DLL you can never be sure if all windows/objects for which a stub was created have been destroyed or not, hence one cannot safely release the memory. I’ll show you a way to be 98% sure. After having read all of this you might well say this leak will not occur if no window is used in the DLL, and you would be right. But if the Controls unit is referenced then the TApplication object is created (in the Forms unit). During the initialisation of the Controls unit the InitControls procedure is called, where the Application object’s ShowHint is set to True. This has the effect that a THintWindow object is instantiated, MakeObject- Instance is called, and hence a memory block and stub is allocated. If this was not set to True here the leak would never actually appear! Note that the TApplication window will not be created in a DLL. At first glance it seems as if it is nearly impossible to plug this leak without major surgery to Forms.pas (or Classes.pas in Delphi 6), as one cannot directly get access to the address of the allocated blocks. To release the memory one needs to know the allocation address of each block of memory that contains the stub array. The implementation of Make- ObjectInstance returns a pointer to somewhere inside the stub array, but that does not allow you to identify the starting address of the memory block occupied by the array, making it apparently impossible to free it. However, there is a way of finding out what the allocation address is for a block of memory pointed to by a pointer, if the memory was allocated using VirtualAlloc, by calling the VirtualQuery API function. We will then have the allocation base for the block and we can release the memory. There is a danger, though: what if there are windowed controls still making use of the stubs in that memory block? If we were to release the memory before the object is released, which will call FreeObj- ectInstance to the stub as free, non-existent memory will be accessed and an Access Violation will occur. The best strategy seems to be to check each stub to see whether it is free or not. If all the stubs in a block are free, then that block can be released. This raises another issue: how does one determine if a stub is free or not? All free stubs will point to the next free stub; this means that a free stub will either point to another stub in the same block of memory or another stub in another block of memory. So all we need to do is to find the stubs which do not point inward into a memory block. By looking at the special flags of the memory pointed to by the outward-pointing stubs, we can find that all VCL created stubs will be ed as executable and read/ write, the reason being that the VCL needs to write code into the stubs. This differs from the normal flags used for normal application code, which are executable and copy-on-write. The application code will also have a flag indicating that it has been mapped into memory as part of a section. Hence we have a way to determine if a stub is pointing to an object or another block, in which case it means the stub is empty. We can now start to look at a first iteration of getting and releasing a stub memory block where no windowed controls are allocated in our code. We do this as follows: we call MakeObjectInstance passing in a pointer to dummy function that will then return a pointer to a stub. Once we have this pointer we can then immediately free the stub again by calling FreeObjectIns- tance on the pointer. Next we pass the pointer to the VirtualQuery function, which provides us with an allocation base, which is the start of the main block of memory containing all the stubs and ancillary code. Because we know the layout of the block in memory we can now scan the stub array to make sure all stubs are free. To test whether a stub is free we can check if the location the stub points to falls within the allocated memory block, as the block will always be 4Kb in size. If we find that a stub points outside the allocated block of memory, we can assume the stub is pointing to ‘live’ memory and is not free (Listing 7).
var
i : integer;
tmpObjPtr : PObjectInstance;
LowerBound, UpperBound : DWORD;
begin
Result := false;
if tmpBlockPtr <> nil then begin
LowerBound := DWORD(tmpBlockPtr);
UpperBound := LowerBound + 4096;
Result := true;
for i := 0 to InstanceCount do begin
tmpObjPtr := tmpBlockPtr^.Instances[i].Next;
if (tmpObjPtr = nil) or (((DWORD(tmpObjPtr) >= LowerBound) and
(DWORD(tmpObjPtr) <= UpperBound))) then
continue;
Result := false;
Exit;
end;
end; For a more generalised case where windowed controls might be created in the DLL, the situation changes dramatically, because we can never be sure in which block our dummy stub is located. If, for instance, the stub is in the instance block which was allocated first, we cannot get to the second instance block. However, at least one of the stubs will have to point to another stub in one of the other instance blocks, so this enables us to get to all of the allocated blocks, although via a roundabout route. Thus to find all allocated instance blocks we need to scan each stub in each instance block, while recording which block each stub points to. Once we have a list of all the blocks we can check whether the blocks are free or not, and only release the free instance blocks (see Listing 8). ? Listing 8var
ReleaseList : TList;
procedure SweepForBlocks(BlockPtr : PInstanceBlock);
procedure ScanStubs(tmpBlockPtr : PInstanceBlock);
var
i : integer;
tmpObjPtr : PObjectInstance;
InstMemInfo : TMemoryBasicInformation;
LowerBound, UpperBound : DWORD;
begin
if tmpBlockPtr <> nil then begin
LowerBound := DWORD(tmpBlockPtr);
UpperBound := LowerBound + 4096;
for i := 0 to InstanceCount do begin
tmpObjPtr := tmpBlockPtr^.Instances[i].Next;
if (tmpObjPtr = nil) or (((DWORD(tmpObjPtr) >= LowerBound) and
(DWORD(tmpObjPtr) <= UpperBound))) then
continue;
SizeofBuff := sizeof(TMemoryBasicInformation);
if VirtualQuery(tmpObjPtr ,InstMemInfo, SizeofBuff) = SizeofBuff then
with InstMemInfo do begin
if InstMemInfo.AllocationBase = tmpBlockPtr then
continue;
if ((AllocationProtect and
PAGE_EXECUTE_READWRITE) = PAGE_EXECUTE_READWRITE) and
((Type_9 and MEM_PRIVATE) = MEM_PRIVATE) then begin
SweepForBlocks(InstMemInfo.AllocationBase);
continue;
end;
end;
end;
end;
end;
begin
while BlockPtr <> nil do begin
with ReleaseList do
if IndexOf(BlockPtr) = -1 then begin
Add(BlockPtr);
ScanStubs(BlockPtr);
end;
BlockPtr := BlockPtr^.Next;
end;
end; The disk includes two units: one for the simple case where no windows are allocated in user code, and the other where windows have been allocated. The code in the simplified unit simply finds the only allocated instance block and releases the memory in the finalisation section of the unit. To use this unit you simply include it in your DLL project. The same goes for the other unit, which implements the more generalised solution to the problem. However, the generalised solution jumps through several fiery hoops to make sure all allocated instance blocks are found and released. It would be safest to make sure that whichever unit you use is the very first unit in your project file uses clause, so that its finalization section is called at the last possible moment. Well, there you have it: a rare archaeological find in the VCL. I hope this will help those who have experienced this issue.
Var
Block : PInstanceBlock;
Begin
While InstBlockList <> NIL do
Begin
Block := InstBlockList^.Next;
VirtualFree(InstBlockList, 4096, MEM_DECOMMIT);
InstBlockList := Block;
end;
end;
在finalization段最后调用它
DeleteObjectInstance;