win32 programming lesson 17: memory mapped files (finally, cool stuff again, all this work is...
TRANSCRIPT
Win32 ProgrammingLesson 17: Memory Mapped Files(Finally, cool stuff again, all this work is getting tedious!)
Where are we? We’ve gotten pretty familiar with memory But now it’s time to look at some of the clever
tricks you can play with memory
Memory mapped files Three primary purposes
Loading and executing .exe and DLL files Accessing files on disk quickly without need to
buffer Allow multiple processes to share the same data
in memory
CreateProcess Revisited Five steps:
Find the .exe file on disk Create a new Process kernel object Create a private address space for the process Reserve space in memory at 0x00400000 (by
default) and load the .exe image Physical storage is from the .exe image, not the
paging file
Next… System uses LoadLibrary to load the needed
DLLs from disk Once again, physical storage underlying the
DLLs is on disk, not the system paging file (clever eh?)
If everything can’t be mapped CreateProcess fails and returns FALSE
Multiple Instances Very clever – remember
COPY_ON_WRITE? DLLs share the same image in memory – but
get new pages when they modify their memory image
However, what if we want to share variables that change?
Background To work out how to do this, we need to understand a
little more of the foundations Every .exe or DLL has multiple sections Each section has one or more of the following
attributes: READ WRITE EXECUTE SHARED (effectively turns off COPY_ON_WRITE)
Can see this using DumpBin
Using a Shared Data segment See example
Memory mapped data files How to reverse the order of all bytes in a file?
Four ways… One file, one buffer (load it all!) Two files, one buffer (read it in in chunks, reverse
each chunk) One file, two buffers (read in 1024 bytes from the
start and the end, and swap them) One file, no buffers!
Whassat? Yep, one file, zero buffers Map the file into memory Work on the memory image directly, letting
the OS handle the buffering
Six step process You must:
Create or open the kernel object that references the file
Create a file-mapping kernel object Tell the system to map all or part of the file Tell the system when mapping is no longer
needed Close the file-mapping object Close the file
Step 1: Opening the file Easy: Createfile
HANDLE CreateFile( PCSTR pszFileName, DWORD dwDesiredAccess, DWORD dwShareMode, PSECURITY_ATTRIBUTES psa, DWORD dwCreationDisposition, DWORD dwFlagsAndAttributes, HANDLE hTemplateFile);
Parms dwDesiredAccess
0 – just get the attributes GENERIC_READ GENERIC_WRITE GENERIC_READ | GENERIC_WRITE
dwShareMode 0 – not shared FILE_SHARE_READ FILE_SHARE_WRITE FILE_SHARE_READ | FILE_SHARE_WRITE
Step 2: Creating the file-mapping HANDLE CreateFileMapping(
HANDLE hFile, PSECURITY_ATTRIBUTES psa, DWORD fdwProtect, DWORD dwMaximumSizeHigh, DWORD dwMaximumSizeLow, PCTSTR pszName);
See MSDN
Parms dwMaximumSizeHigh/Low are important 64-bit representation of the file length so
there is enough disk space if you want to make writes
See example for what happens as we step through…
Step 3: Map the file in memory Using:
PVOID MapViewOfFile( HANDLE hFileMappingObject, DWORD dwDesiredAccess, DWORD dwFileOffsetHigh, DWORD dwFileOffsetLow, SIZE_T dwNumberOfBytesToMap);
Parms FILE_MAP_WRITE: You can read and write file data. CreateFileMapping had to be called by passing PAGE_READWRITE.
FILE_MAP_READ: You can read file data. CreateFileMapping could be called with any of the protection attributes: PAGE_READONLY, PAGE_READWRITE, or PAGE_WRITECOPY.
FILE_MAP_ALL_ACCESS: Same as FILE_MAP_WRITE. FILE_MAP_COPY: You can read and write file data. Writing
causes a private copy of the page to be created.
Step 4: Unmap the file BOOL UnmapViewOfFile(PVOID pvBaseAd
dress); And flush, if you need to…
BOOL FlushViewOfFile( PVOID pvAddress, SIZE_T dwNumberOfBytesToFlush);
pvAddress is what you want to flush…
Step 5/6 Cleaning up CloseHandle(hFileMapping); CloseHandle(hFile); Easy But remember kernel usage counts – means
that we can close the file much earlier for neater code if we want to
Example Let’s look at the one in the book – file
reverse…
Processing a really big file __int64 qwFileOffset = 0, qwNumOf0s = 0;
while (qwFileSize > 0) { // Determine the number of bytes to be mapped in this view DWORD dwBytesInBlock = sinf.dwAllocationGranularity; if (qwFileSize < sinf.dwAllocationGranularity) dwBytesInBlock = (DWORD) qwFileSize; PBYTE pbFile = (PBYTE) MapViewOfFile(
hFileMapping, FILE_MAP_READ, (DWORD) (qwFileOffset >> 32), // Starting byte (DWORD) (qwFileOffset & 0xFFFFFFFF), // in file dwBytesInBlock); // # of bytes to map
// Count the number of Js in this block. for (DWORD dwByte = 0; dwByte < dwBytesInBlock; dwByte++) { if (pbFile[dwByte] == 0) qwNumOf0s++; } // Unmap the view; we don't want multiple views // in our address space. UnmapViewOfFile(pbFile); // Skip to the next set of bytes in the file. qwFileOffset += dwBytesInBlock; qwFileSize -= dwBytesInBlock; }
Specifying the Base Address Say you want to share a memory-mapped file
which contains a linked-list Need to have the same base address in both
processes Can call MapViewOfFileEx
Assignment Recreate my little shared-memory form
program EASY!