一 windows系统的内存管理涉及哪些
1.1 虚拟内存管理机制
- windows操作系统使用虚拟内存技术,将磁盘文件,通过映射对象(存储在物理内存)关联,映射到虚拟内存作为文件试图。即用户操作"虚拟内存中File View Object"-> 物理内存中"File Mapping" -> 磁盘文件。
1.2 分页和分段机制
- windows采用分页机制,将内存划分为固定大写的页面,通过页表来映射虚拟地址和物理地址
- 分段机制是将内存划分为不同大小的段,通过段表来管理。
1.3 内存保护
windows通过硬件机制保护系统内存资源,防止程序对内存的非法访问,确保系统的稳定性和安全性
1.4 高级内存管理
windows还提供内存压缩、内存回收、内存优化等功能
内存压缩可以节省生存空间,内存回收能释放不再使用的内存资源,而内存优化则旨在提高系统性能和响应速度。
二 虚拟内存的实现技术 - 文件映射
2.1 磁盘文件,文件映射对象,文件视图的关系
MSDN - 文件映射
注:
文件映射: 将文件内存(File on Disk)于进程的一部分虚拟地址空间关联。
文件映射对象:调用CreateFileMapping创建文件映射对象,维护磁盘文件和虚拟地址关联。
文件视图: 进程用来访问磁盘文件的虚拟地址空间部分。如果使用指针从文件视图读取和写入文件视图的过程,就像使用动态分配的内存一样。使用文件映射可提高可提高效率,因为文件驻留在磁盘上,但文件视图驻留在内存中。
2.2 应用场景(为什么要用文件映射?)
1)解决大文件频繁读写效率问题
直接加载文件到内存(传统方式)
C++读取txt文件
1 逐行读取
void readTxt(string file)
{
ifstream infile;
infile.open(file.data()); //将文件流对象与文件连接起来
assert(infile.is_open()); //若失败,则输出错误消息,并终止程序运行
string s;
while(getline(infile,s))
{
cout<<s<<endl;
}
infile.close(); //关闭文件输入流
}
传统方式,将磁盘文件,加载到内存中,频繁操作磁盘文件。
文件映射方式,将磁盘文件,通过关联到文件映射对象中,读取IO效率更高。减少磁盘文件访问次数。
2) 解决多个进程访问共享磁盘文件场景
多个进程,都可以访问文件映射对象。解决IPC通信,数据复制的开销。
2.3 原理
通过文件映射机制,进程可以通过访问内存的方式直接读写磁盘文件,修改内容后由操作系统自动同步的文件进行更新。使用FlushViewOfFile可刷新缓存区,将映射在虚拟内存当中的数据立即回写到磁盘文件。
过程如下所示:
1. CreateFileMapping创建文件映射对象(内核对象,多进程可访问),关联磁盘文件(文件句柄hFile)
2. MapViewOfFile将文件映射对象,映射到进程虚拟地址
2.4 代码
/*
This program demonstrates file mapping, especially how to align a
view with the system file allocation granularity.
*/
#include <windows.h>
#include <stdio.h>
#include <tchar.h>
#define BUFFSIZE 1024 // size of the memory to examine at any one time
#define FILE_MAP_START 138240 // starting point within the file of
// the data to examine (135K)
/* The test file. The code below creates the file and populates it,
so there is no need to supply it in advance. */
TCHAR * lpcTheFile = TEXT("fmtest.txt"); // the file to be manipulated
int main(void)
{
HANDLE hMapFile; // handle for the file's memory-mapped region
HANDLE hFile; // the file handle
BOOL bFlag; // a result holder
DWORD dBytesWritten; // number of bytes written
DWORD dwFileSize; // temporary storage for file sizes
DWORD dwFileMapSize; // size of the file mapping
DWORD dwMapViewSize; // the size of the view
DWORD dwFileMapStart; // where to start the file map view
DWORD dwSysGran; // system allocation granularity
SYSTEM_INFO SysInfo; // system information; used to get granularity
LPVOID lpMapAddress; // pointer to the base address of the
// memory-mapped region
char * pData; // pointer to the data
int i; // loop counter
int iData; // on success contains the first int of data
int iViewDelta; // the offset into the view where the data
//shows up
// Create the test file. Open it "Create Always" to overwrite any
// existing file. The data is re-created below
hFile = CreateFile(lpcTheFile,
GENERIC_READ | GENERIC_WRITE,
0,
NULL,
CREATE_ALWAYS,
FILE_ATTRIBUTE_NORMAL,
NULL);
if (hFile == INVALID_HANDLE_VALUE)
{
_tprintf(TEXT("hFile is NULL\n"));
_tprintf(TEXT("Target file is %s\n"),
lpcTheFile);
return 4;
}
// Get the system allocation granularity.
GetSystemInfo(&SysInfo);
dwSysGran = SysInfo.dwAllocationGranularity;
// Now calculate a few variables. Calculate the file offsets as
// 64-bit values, and then get the low-order 32 bits for the
// function calls.
// To calculate where to start the file mapping, round down the
// offset of the data into the file to the nearest multiple of the
// system allocation granularity.
dwFileMapStart = (FILE_MAP_START / dwSysGran) * dwSysGran;
_tprintf (TEXT("The file map view starts at %ld bytes into the file.\n"),
dwFileMapStart);
// Calculate the size of the file mapping view.
dwMapViewSize = (FILE_MAP_START % dwSysGran) + BUFFSIZE;
_tprintf (TEXT("The file map view is %ld bytes large.\n"),
dwMapViewSize);
// How large will the file mapping object be?
dwFileMapSize = FILE_MAP_START + BUFFSIZE;
_tprintf (TEXT("The file mapping object is %ld bytes large.\n"),
dwFileMapSize);
// The data of interest isn't at the beginning of the
// view, so determine how far into the view to set the pointer.
iViewDelta = FILE_MAP_START - dwFileMapStart;
_tprintf (TEXT("The data is %d bytes into the view.\n"),
iViewDelta);
// Now write a file with data suitable for experimentation. This
// provides unique int (4-byte) offsets in the file for easy visual
// inspection. Note that this code does not check for storage
// medium overflow or other errors, which production code should
// do. Because an int is 4 bytes, the value at the pointer to the
// data should be one quarter of the desired offset into the file
for (i=0; i<(int)dwSysGran; i++)
{
WriteFile (hFile, &i, sizeof (i), &dBytesWritten, NULL);
}
// Verify that the correct file size was written.
dwFileSize = GetFileSize(hFile, NULL);
_tprintf(TEXT("hFile size: %10d\n"), dwFileSize);
// Create a file mapping object for the file
// Note that it is a good idea to ensure the file size is not zero
hMapFile = CreateFileMapping( hFile, // current file handle
NULL, // default security
PAGE_READWRITE, // read/write permission
0, // size of mapping object, high
dwFileMapSize, // size of mapping object, low
NULL); // name of mapping object
if (hMapFile == NULL)
{
_tprintf(TEXT("hMapFile is NULL: last error: %d\n"), GetLastError() );
return (2);
}
// Map the view and test the results.
lpMapAddress = MapViewOfFile(hMapFile, // handle to
// mapping object
FILE_MAP_ALL_ACCESS, // read/write
0, // high-order 32
// bits of file
// offset
dwFileMapStart, // low-order 32
// bits of file
// offset
dwMapViewSize); // number of bytes
// to map
if (lpMapAddress == NULL)
{
_tprintf(TEXT("lpMapAddress is NULL: last error: %d\n"), GetLastError());
return 3;
}
// Calculate the pointer to the data.
pData = (char *) lpMapAddress + iViewDelta;
// Extract the data, an int. Cast the pointer pData from a "pointer
// to char" to a "pointer to int" to get the whole thing
iData = *(int *)pData;
_tprintf (TEXT("The value at the pointer is %d,\nwhich %s one quarter of the desired file offset.\n"),
iData,
iData*4 == FILE_MAP_START ? TEXT("is") : TEXT("is not"));
// Close the file mapping object and the open file
bFlag = UnmapViewOfFile(lpMapAddress);
bFlag = CloseHandle(hMapFile); // close the file mapping object
if(!bFlag)
{
_tprintf(TEXT("\nError %ld occurred closing the mapping object!"),
GetLastError());
}
bFlag = CloseHandle(hFile); // close the file itself
if(!bFlag)
{
_tprintf(TEXT("\nError %ld occurred closing the file!"),
GetLastError());
}
return 0;
}