WarlockFish/linux-insides-zh
1
0
Fork 0
mirror of https://github.com/MintCN/linux-insides-zh.git synced 2026-04-25 19:20:28 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #21 from mudongliang/master

Browse Source 
Paging Translated!
This commit is contained in:
mudongliang
2016-02-26 19:29:27 +08:00
parent d97ecd0c8b 897c15a832
commit 6d10b231f0
1 changed files with 81 additions and 83 deletions
Download Patch File Download Diff File Expand all files Collapse all files

164
Theory/Paging.md
View File

@@ -1,39 +1,39 @@
分页
================================================================================
Introduction
简介
--------------------------------------------------------------------------------
In the fifth [part](http://0xax.gitbooks.io/linux-insides/content/Booting/linux-bootstrap-5.html) of the series `Linux kernel booting process` we learned about what the kernel does in its earliest stage. In the next step the kernel will initialize different things like `initrd` mounting, lockdep initialization, and many many others things, before we can see how the kernel runs the first init process.
在 Linux 内核启动过程中的[第五部分](http://0xax.gitbooks.io/linux-insides/content/Booting/linux-bootstrap-5.html)，我们学到了内核在最早的阶段做的工作。在接下来的步骤中，在我们明白内核如何运行第一个 init 进程，内核初始化不同的事，比如 `initrd` 加载， lockdep 初始化，以及许多许多其他的事。
Yeah, there will be many different things, but many many and once again many work with **memory**.
是的，那将有很多不同的事，但是还有更多更多更多关于**内存**的工作。
In my view, memory management is one of the most complex part of the linux kernel and in system programming in general. This is why before we proceed with the kernel initialization stuff, we need to get acquainted with paging.
在我看来，一般而言，内存管理是 Linux 内核和系统编程最复杂的部分之一。这就是为什么在我们处理内核初始化事情之前，我们需要了解分页。
`Paging` is a mechanism that translates a linear memory address to a physical address. If you have read the previous parts of this book, you may remember that we saw segmentation in real mode when physical addresses are calculated by shifting a segment register by four and adding an offset. We also saw segmentation in protected mode, where we used the descriptor tables and base addresses from descriptors with offsets to calculate the physical addresses. Now that we are in 64-bit mode, will see paging.
分页是将线性地址转换为物理地址的机制。如果我们已经读过了这本书之前的部分，你可能记得我们在实模式下有分段机制，当时物理地址是由左移四位段寄存器加上偏移算出来的。我们也看了保护模式下的分段机制，其中我们使用描述符表和从描述符得到的基地址加上偏移得到物理地址。由于我们在 64 位模式，我们将看分页机制。
As the Intel manual says:
正如 Intel 手册中说的：
> Paging provides a mechanism for implementing a conventional demand-paged, virtual-memory system where sections of a program’s execution environment are mapped into physical memory as needed.
> 分页机制提供一种机制，为了实现一种常见的页需求，虚拟内存系统，其中一个程序执行环境中的段将要按照需求被映射到物理地址。
So... In this post I will try to explain the theory behind paging. Of course it will be closely related to the `x86_64` version of the linux kernel for, but we will not go into too much details (at least in this post).
所以... 在这个帖子中我将尝试解释分页背后的理论。当然它将与64位版本的 Linux 内核关系密切，但是我们将不会深入太多细节（至少在这个帖子里面）。
Enabling paging
开启分页
--------------------------------------------------------------------------------
There are three paging modes:
有三种分页模式：
* 32-bit paging;
* PAE paging;
* IA-32e paging.
* 32 位分页模式；
* PAE 分页；
* IA-32e 分页。
We will only explain the last mode here. To enable the `IA-32e paging` paging mode we need to do following things:
我们这里将只解释最后一种模式。为了开启 `IA-32e 分页模式`，我们需要做如下事情：
* set the `CR0.PG` bit;
* set the `CR4.PAE` bit;
* set the `IA32_EFER.LME` bit.
* 设置 `CR0.PG` 位；
* 设置 `CR4.PAE` 位；
* 设置 `IA32_EFER.LME` 位。
We already saw where those this bits were set in [arch/x86/boot/compressed/head_64.S](https://github.com/torvalds/linux/blob/master/arch/x86/boot/compressed/head_64.S):
我们已经在 [arch/x86/boot/compressed/head_64.S](https://github.com/torvalds/linux/blob/master/arch/x86/boot/compressed/head_64.S) 中看见了这些位被设置了：
```assembly
movl $(X86_CR0_PG | X86_CR0_PE), %eax
@@ -49,17 +49,17 @@ btsl $_EFER_LME, %eax
wrmsr
```
Paging structures
分页数据结构
--------------------------------------------------------------------------------
Paging divides the linear address space into fixed-size pages. Pages can be mapped into the physical address space or even external storage. This fixed size is `4096` bytes for the `x86_64` linux kernel. To perform the linear address translation to a physical address special structures are used. Every structure is `4096` bytes size and contains `512` entries (this only for `PAE` and `IA32_EFER.LME` modes). Paging structures are hierarchical and the linux kernel uses 4 level of paging in the `x86_64` architecture. The CPU uses a part of the linear address to identify the entry in another paging structure which is at the lower level or physical memory region (`page frame`) or physical address in this region (`page offset`). The address of the top level paging structure located in the `cr3` register. We already saw this in [arch/x86/boot/compressed/head_64.S](https://github.com/torvalds/linux/blob/master/arch/x86/boot/compressed/head_64.S):
分页将线性地址分为固定尺寸的页。页会被映射进入物理地址空间或外部存储设备。这个固定尺寸在 `x86_64` 内核中是 `4096` 字节。为了将线性地址转换位物理地址，特殊的结构会被用到。每个结构都是 `4096` 字节并包含 `512` 项（这只为 `PAE` 和 `IA32_EFER.LME` 模式）。分页结构是层次级的， Linux 内核在 `x86_64` 框架中使用4层的分层机制。CPU使用一部分线性地址去确定另一个分页结构中的项，这个分页结构可能在最低层，物理内存区域（页框），在这个区域的物理地址（页偏移）。最高层的分页结构的地址存储在 `cr3` 寄存器中。我们已经从 [arch/x86/boot/compressed/head_64.S](https://github.com/torvalds/linux/blob/master/arch/x86/boot/compressed/head_64.S) 这个文件中已经看到了。
```assembly
leal pgtable(%ebx), %eax
movl %eax, %cr3
```
We built the page table structures and put the address of the top-level structure in the `cr3` register. Here `cr3` is used to store the address of the top-level structure, the `PML4` or `Page Global Directory` as it is called in the linux kernel. `cr3` is 64-bit register and has the following structure:
我们构建页表结构并且将这个最高层结构的地址存放在 `cr3` 寄存器中。这里 `cr3` 用于存储最高层结构的地址，在 Linux 内核中被称为 `PML4` 或 `Page Global Directory` 。 `cr3` 是一个64位的寄存器，并且有着如下的结构：
```
63 52 51 32
@@ -76,26 +76,26 @@ We built the page table structures and put the address of the top-level structur
--------------------------------------------------------------------------------
```
These fields have the following meanings:
这些字段有着如下的意义：
* Bits 2:0 - ignored;
* Bits 51:12 - stores the address of the top level paging structure;
* Bit 3 and 4 - PWT or Page-Level Writethrough and PCD or Page-level cache disable indicate. These bits control the way the page or Page Table is handled by the hardware cache;
* Reserved - reserved must be 0;
* Bits 63:52 - reserved must be 0.
* 第 0 到第 2 位 - 忽略；
* 第 12 位到第 51 位 - 存储最高层分页结构的地址；
* 第 3 位 到第 4 位 - PWT 或 Page-Level Writethrough 和 PCD 或 Page-level Cache Disable 显示。这些位控制页或者页表被硬件缓存处理的方式；
* 保留位 - 保留，但必须为 0 ；
* 第 52 到第 63 位 - 保留，但必须为 0 ；
The linear address translation address is following:
线性地址转换过程如下所示：
* A given linear address arrives to the [MMU](http://en.wikipedia.org/wiki/Memory_management_unit) instead of memory bus.
* 64-bit linear address splits on some parts. Only low 48 bits are significant, it means that `2^48` or 256 TBytes of linear-address space may be accessed at any given time.
* `cr3` register stores the address of the 4 top-level paging structure.
* `47:39` bits of the given linear address stores an index into the paging structure level-4, `38:30` bits stores index into the paging structure level-3, `29:21` bits stores an index into the paging structure level-2, `20:12` bits stores an index into the paging structure level-1 and `11:0` bits provide the byte offset into the physical page.
* 一个给定的线性地址到达 [MMU](http://en.wikipedia.org/wiki/Memory_management_unit) 而不是存储器总线；
* 64位线性地址分为很多部分。只有低 48 位是有意义的，它意味着 `2^48` 或 256TB 的线性地址空间在任意给定时间内都可以被访问；
* `cr3` 寄存器存储这个最高层分页数据结构的地址；
* 给定的线性地址中的第 39 位到第 47 位存储一个第 4 级分页结构的索引，第 30 位到第 38 位存储一个第3级分页结构的索引，第 29 位到第 21 位存储一个第 2 级分页结构的索引，第 12 位到第 20 位存储一个第 1 级分页结构的索引，第 0 位到第 11 位提供物理页的字节偏移；
schematically, we can imagine it like this:
按照图示，我们可以这样想象它：
![4-level paging](http://oi58.tinypic.com/207mb0x.jpg)
![四层分页](http://oi58.tinypic.com/207mb0x.jpg)
Every access to a linear address is either a supervisor-mode access or a user-mode access. This access is determined by the `CPL` (current privilege level). If `CPL < 3` it is a supervisor mode access level otherwise, otherwise it is a user mode access level. For example, the top level page table entry contains access bits and has the following structure:
每一个对线性地址的访问不是一个管态访问就是用户态访问。这个访问是被 `CPL (Current Privilege Level)` 所决定。如果 `CPL < 3` ，那么它是管态访问级，否则，它就是用户态访问级。比如，最高级页表项包含访问位和如下的结构：
```
63 62 52 51 32
@@ -112,33 +112,33 @@ Every access to a linear address is either a supervisor-mode access or a user-mo
--------------------------------------------------------------------------------
```
Where:
其中：
* 63 bit - N/X bit (No Execute Bit) - presents ability to execute the code from physical pages mapped by the table entry;
* 62:52 bits - ignored by CPU, used by system software;
* 51:12 bits - stores physical address of the lower level paging structure;
* 12:9 bits - ignored by CPU;
* MBZ - must be zero bits;
* Ignored bits;
* A - accessed bit indicates was physical page or page structure accessed;
* PWT and PCD used for cache;
* U/S - user/supervisor bit controls user access to the all physical pages mapped by this table entry;
* R/W - read/write bit controls read/write access to the all physical pages mapped by this table entry;
* P - present bit. Current bit indicates was page table or physical page loaded into primary memory or not.
* 第 63 位 - N/X 位（不可执行位）显示被这个页表项映射的所有物理页执行代码的能力；
* 第 52 位到第 62 位 - 被CPU忽略，被系统软件使用；
* 第 12 位到第 51 位 - 存储低级分页结构的物理地址；
* 第 9 位到第 11 位 - 被 CPU 忽略；
* MBZ - 必须为 0 ；
* 忽略位；
* A - 访问位暗示物理页或者页结构被访问；
* PWT 和 PCD 用于缓存；
* U/S - 用户/管理位控制对被这个页表项映射的所有物理页用户访问；
* R/W - 读写位控制着被这个页表项映射的所有物理页的读写权限
* P - 存在位。当前位表示页表或物理页是否被加载进内存；
Ok, we know about the paging structures and their entries. Now let's see some details about 4-level paging in the linux kernel.
好的，我们知道了分页结构和它们的表项。现在我们来看一下 Linux 内核中的 4 级分页机制的一些细节。
Paging structures in the linux kernel
Linux 内核中的分页结构
--------------------------------------------------------------------------------
As we've seen, the linux kernel in `x86_64` uses 4-level page tables. Their names are:
就如我们已经看到的那样， `x86_64`Linux 内核使用4级页表。它们的名字是：
* Page Global Directory
* Page Upper Directory
* Page Middle Directory
* Page Table Entry
* 页全局目录
* 页上层目录
* 页中间目录
* 页表项
After you've compiled and installed the linux kernel, you can see the `System.map` file which stores the virtual addresses of the functions that are used by the kernel. For example:
在你已经编译和安装 Linux 内核，你可以看到存储内核中使用的函数的虚拟地址的 `System.map` 文件。例如：
```
$ grep "start_kernel" System.map
@@ -146,7 +146,7 @@ ffffffff81efe497 T x86_64_start_kernel
ffffffff81efeaa2 T start_kernel
```
We can see `0xffffffff81efe497` here. I doubt you really have that much RAM installed. But anyway, `start_kernel` and `x86_64_start_kernel` will be executed. The address space in `x86_64` is `2^64` size, but it's too large, that's why a smaller address space is used, only 48-bits wide. So we have a situation where the physical address space is limited to 48 bits, but addressing still performed with 64 bit pointers. How is this problem solved? Look at this diagram:
这里我们可以看见 `0xffffffff81efe497` 。我怀疑你是否真的有安装这么多内存。但是无论如何， `start_kernel` 和 `x86_64_start_kernel` 将会被执行。在 `x86_64` 中，地址空间的大小是 `2^64` ，但是它太大了，这就是为什么我们使用一个较小的地址空间，只是 48 位的宽度。所以一个情况出现，虽然物理地址空间限制到 48 位，但是寻址仍然使用 64 位指针。 这个问题是如何解决的？看下面的这个表。
```
0xffffffffffffffff +-----------+
@@ -166,12 +166,12 @@ We can see `0xffffffff81efe497` here. I doubt you really have that much RAM inst
0x0000000000000000  +-----------+
```
This solution is `sign extension`. Here we can see that the lower 48 bits of a virtual address can be used for addressing. Bits `63:48` can be either only zeroes or only ones. Note that the virtual address space is split in 2 parts:
这个解决方案是 `sign extension` 。这里我们可以看到一个虚拟地址的低 48 位可以被用于寻址。第 48 位到第 63 位全是 0 或 1 。注意这个虚拟地址空间被分为两部分：
* Kernel space
* Userspace
* 内核空间
* 用户空间
Userspace occupies the lower part of the virtual address space, from `0x000000000000000` to `0x00007fffffffffff` and kernel space occupies the highest part from `0xffff8000000000` to `0xffffffffffffffff`. Note that bits `63:48` is 0 for userspace and 1 for kernel space. All addresses which are in kernel space and in userspace or in other words which higher `63:48` bits are zeroes or ones are called `canonical` addresses. There is a `non-canonical` area between these memory regions. Together these two memory regions (kernel space and user space) are exactly `2^48` bits wide. We can find the virtual memory map with 4 level page tables in the [Documentation/x86/x86_64/mm.txt](https://github.com/torvalds/linux/blob/master/Documentation/x86/x86_64/mm.txt):
用户空间占用虚拟地址空间的低部分，从 `0x000000000000000` 到 `0x00007fffffffffff` ，而内核空间占据从 `0xffff8000000000` 到 `0xffffffffffffffff` 的高部分。注意，第 48 位到第 63 位是对于用户空间是 0 ，对于内核空间是 1 。内核空间和用户空间中的所有地址是标准地址，而在这些内存区域中间有非标准区域。这两块内存区域（内核空间和用户空间）合起来是 48 位宽度。我们可以在 [Documentation/x86/x86_64/mm.txt](https://github.com/torvalds/linux/blob/master/Documentation/x86/x86_64/mm.txt) 找到 4 级页表下的虚拟内存映射:
```
0000000000000000 - 00007fffffffffff (=47 bits) user space, different per mm
@@ -193,21 +193,21 @@ ffffffffff600000 - ffffffffffdfffff (=8 MB) vsyscalls
ffffffffffe00000 - ffffffffffffffff (=2 MB) unused hole
```
We can see here the memory map for user space, kernel space and the non-canonical area in-between them. The user space memory map is simple. Let's take a closer look at the kernel space. We can see that it starts from the guard hole which is reserved for the hypervisor. We can find the definition of this guard hole in [arch/x86/include/asm/page_64_types.h](https://github.com/torvalds/linux/blob/master/arch/x86/include/asm/page_64_types.h):
这里我们可以看到用户空间，内核空间和非标准空间的内存映射。用户空间的内存映射很简单。让我们来更近地查看内核空间。我们可以看到它始于为管理程序 (hypervisor) 保留的防卫空洞 (guard hole) 。我们可以在 [arch/x86/include/asm/page_64_types.h](https://github.com/torvalds/linux/blob/master/arch/x86/include/asm/page_64_types.h) 这个文件中看到防卫空洞的概念！
```C
#define __PAGE_OFFSET _AC(0xffff880000000000, UL)
```
Previously this guard hole and `__PAGE_OFFSET` was from `0xffff800000000000` to `0xffff80ffffffffff` to prevent access to non-canonical area, but was later extended by 3 bits for the hypervisor.
以前防卫空洞和 `__PAGE_OFFSET` 是从 `0xffff800000000000` 到 `0xffff80ffffffffff` ，来防止对非标准区域的访问，但是后来为了管理程序扩展了 3 位。
Next is the lowest usable address in kernel space - `ffff880000000000`. This virtual memory region is for direct mapping of the all physical memory. After the memory space which maps all physical addresses, the guard hole. It needs to be between the direct mapping of all the physical memory and the vmalloc area. After the virtual memory map for the first terabyte and the unused hole after it, we can see the `kasan` shadow memory. It was added by [commit](https://github.com/torvalds/linux/commit/ef7f0d6a6ca8c9e4b27d78895af86c2fbfaeedb2) and provides the kernel address sanitizer. After the next unused hole we can see the `esp` fixup stacks (we will talk about it in other parts of this book) and the start of the kernel text mapping from the physical address - `0`. We can find the definition of this address in the same file as the `__PAGE_OFFSET`:
紧接着是内核空间中最低的可用空间 - `ffff880000000000` 。这个虚拟地址空间是为了所有的物理内存的直接映射。在这块空间之后，还是防卫空洞。它位于所有物理内存的直接映射地址和被 vmalloc 分配的地址之间。在第一个 1TB 的虚拟内存映射和无用的空洞之后，我们可以看到 `ksan` 影子内存 (shadow memory) 。它是通过 [commit](https://github.com/torvalds/linux/commit/ef7f0d6a6ca8c9e4b27d78895af86c2fbfaeedb2) 提交到内核中，并且保持内核空间无害。在紧接着的无用空洞之后，我们可以看到 `esp` 固定栈（我们会在本书其他部分讨论它）。内核代码段的开始从物理地址 - `0` 映射。我们可以在相同的文件中找到将这个地址定义为 `__PAGE_OFFSET` 。
```C
#define __START_KERNEL_map _AC(0xffffffff80000000, UL)
```
Usually kernel's `.text` start here with the `CONFIG_PHYSICAL_START` offset. We saw it in the post about [ELF64](https://github.com/0xAX/linux-insides/blob/master/Theory/ELF.md):
通常内核的 `.text` 段开始于 `CONFIG_PHYSICAL_START` 偏移。我们已经在 [ELF64](https://github.com/0xAX/linux-insides/blob/master/Theory/ELF.md) 相关帖子中看见。
```
readelf -s vmlinux | grep ffffffff81000000
@@ -216,43 +216,41 @@ readelf -s vmlinux | grep ffffffff81000000
90766: ffffffff81000000 0 NOTYPE GLOBAL DEFAULT 1 startup_64
```
Here i checked `vmlinux` with the `CONFIG_PHYSICAL_START` is `0x1000000`. So we have the start point of the kernel `.text` - `0xffffffff80000000` and offset - `0x1000000`, the resulted virtual address will be `0xffffffff80000000 + 1000000 = 0xffffffff81000000`.
这里我将 `CONFIG_PHYSICAL_START` 设置为 `0x1000000` 来检查 `vmlinux` 。所以我们有内核代码段的起始点 - `0xffffffff80000000` 和 偏移 - `0x1000000` ，计算出来的虚拟地址将会是 `0xffffffff80000000 + 1000000 = 0xffffffff81000000` 。
After the kernel `.text` region there is the virtual memory region for kernel modules, `vsyscalls` and an unused hole of 2 megabytes.
在内核代码段之后有一个为内核模块 `vsyscalls` 准备的虚拟内存区域和 2M 无用的空洞。
We've seen how the kernel's virtual memory map is laid out and how a virtual address is translated into a physical one. Let's take for example following address:
我们已经看见内核虚拟内存映射是如何布局的以及虚拟地址是如何转换位物理地址。让我们以下面的地址为例：
```
0xffffffff81000000
```
In binary it will be:
在二进制内它将是：
```
1111111111111111 111111111 111111110 000001000 000000000 000000000000
63:48 47:39 38:30 29:21 20:12 11:0
```
This virtual address is split in parts as described above:
这个虚拟地址将被分为如下描述的几部分：
* `63:48` - bits not used;
* `47:39` - bits of the given linear address stores an index into the paging structure level-4;
* `38:30` - bits stores index into the paging structure level-3;
* `29:21` - bits stores an index into the paging structure level-2;
* `20:12` - bits stores an index into the paging structure level-1;
* `11:0` - bits provide the byte offset into the physical page.
* `48-63` - 不使用的位；
* `37-49` - 给定线性地址的这些位描述一个 4 级分页结构的索引；
* `30-38` - 这些位存储一个 3 级分页结构的索引；
* `21-29` - 这些位存储一个 2 级分页结构的索引；
* `12-20` - 这些位存储一个 1 级分页结构的索引；
* `0-11` - 这些位提供物理页的偏移；
That is all. Now you know a little about theory of `paging` and we can go ahead in the kernel source code and see the first initialization steps.
Conclusion
就这样了。现在你知道了一些关于分页理论，而且我们可以在内核源码上更近一步，查看那些最先的初始化步骤。
总结
--------------------------------------------------------------------------------
It's the end of this short part about paging theory. Of course this post doesn't cover every detail of paging, but soon we'll see in practice how the linux kernel builds paging structures and works with them.
这简短的关于分页理论的部分至此已经结束了。当然，这个帖子不可能包含分页的所有细节，但是我们很快会看到在实践中 Linux 内核如何构建分页结构以及使用它们工作。
**Please note that English is not my first language and I am really sorry for any inconvenience. If you've found any mistakes please send me PR to [linux-internals](https://github.com/0xAX/linux-internals).**
Links
链接
--------------------------------------------------------------------------------
* [Paging on Wikipedia](http://en.wikipedia.org/wiki/Paging)