Performance Zone is brought to you in partnership with:

Sasha Goldshtein is a Senior Consultant for Sela Group, an Israeli company specializing in training, consulting and outsourcing to local and international customers.Sasha's work is divided across these three primary disciplines. He consults for clients on architecture, development, debugging and performance issues; he actively develops code using the latest bits of technology from Microsoft; and he conducts training classes on a variety of topics, from Windows Internals to .NET Performance. You can read more about Sasha's work and his latest ventures at his blog: http://blogs.microsoft.co.il/blogs/sasha. Sasha writes from Herzliya, Israel. Sasha is a DZone MVB and is not an employee of DZone and has posted 204 posts at DZone. You can read more from them at their website. View Full User Profile

An Exercise in Virtual to Physical Memory Translation

09.24.2013
| 3613 views |
  • submit to reddit

In this post, we will explore virtual address translation through a real-world example. If you’d like, the instructions here should be sufficiently detailed enough for you to perform this experiment on your own with a kernel debugger (a local kernel debugger session or even LiveKD is sufficient).

Let’s start with the basics. For a great walkthrough of how memory translation works on x86-64 and x86 systems, you should read Luke Hutchinson’s blog post. Alternatively, Windows Internals (6th edition) contains an even more detailed description of address translation in the Memory Manager chapter (Volume 2).

Now let’s go ahead and perform the exercise to see how memory translation actually works. (This is an adapted version of a real exercise we give in our Windows Internals course.)

Begin by making sure which flavor of hardware and software combination you are working with. Specifically, determine whether you are using x86 without PAE (very unlikely), x86 with PAE, or x64. If you are really paranoid and want to make sure you’re using PAE, check out this great answer on SuperUser.

Next, run Windows Calculator (calc.exe) and a kernel debugger. Find the calc.exe process in the kernel debugger using the !process 0 0 calc.exe command. Make note of the Peb virtual address – this is the virtual address we will be translating. Also make note of the DirBase physical address – this is the starting point for the address translation process.

From this point on, the process depends on the hardware and software you’re running (x86 without PAE, x86 with PAE, or x64). Generally, you need to know the structure of the page table entries in all levels of the page tables, and you need to know the structure of the virtual address – how the virtual address is split into page table indices. Armed with this knowledge, you can translate addresses just like the CPU can!

On x64 systems, the 64-bit virtual address is divided into five subsections, as follows:

Page map level-four selector

Page directory pointer selector

Page table selector

Page table entry selector

Byte within page

9 bits

9 bits

9 bits

9 bits

12 bits

The page table entry on x64 systems contains the page frame number in bits 12 to 39 (where the least significant bit is bit 0 and the most significant bit is bit 63). The page size for small pages is four KB. Each entry in the page table is eight bytes long.

On x86 systems with PAE, the 32-bit virtual address is divided into four subsections, as follows:

Page directory pointer index

Page directory index

Page table index

Byte within page

2 bits

9 bits

9 bits

12 bits

The page table entry on x86 systems with PAE contains the page frame number in bits 12 to 31. The page size for small pages is four KB. Each entry in the page table is eight bytes long.

On x86 systems without PAE, the 32-bit virtual address is divided into three subsections, as follows:

Page directory index

Page table index

Byte within page

10 bits

10 bits

12 bits

The page table entry on x86 systems without PAE contains the page frame number in bits 12 to 31. The page size for small pages is four KB. Each entry in the page table is four bytes long.

To view the contents of physical memory in the debugger, use the !dd and !dq commands. Their parameters are identical to the dd and dq commands, but they operate directly on physical memory.

We will walk through an example on a real x64 system. The virtual address we would like to translate is 7fffffdf000, and the directory base (CR3 register) is pointing to the physical address 26994000.

First, we break the virtual address into its constituents, as described above:

Page map level-four selector

Page directory pointer selector

Page table selector

Page table entry selector

Byte within page

9 bits

9 bits

9 bits

9 bits

12 bits

00f

1ff

1ff

1df

000

Next, we start the translation from the page map level-four selector, which is 00f. The directory base is at the physical address 26994000, and we need to offset it (recall that each entry in the page directory is eight bytes):

lkd> !dq 26994000+f*8 L1
#26994078 00800000`05b48867

Now we have the first-level page table entry. Bits 12 to 39 correspond to the page frame number, which tells us where the next level page table resides. The page frame number in this case is 5b48. The page frame number must be multiplied by the page size (4KB = 0x1000) to obtain the actual physical address of the next-level page table.

The index in the next-level page table is 1ff, so we actually need the value at the physical address 5b48000 + 1ff*8, which is where we find the next page table entry to decipher:

lkd> !dq 5b48000 + 1ff*8 L1
# 5b48ff8 00900000`05c49867

In a similar fashion, we can determine that the page frame number in this case is 5c49. The index in the next level page table is again 1ff, so we turn to the next level page table entry:

lkd> !dq 5c49000+1ff*8 L1
# 5c49ff8 00a00000`05bca867

We are almost done. The next level page table index is 1df, and the page frame number in the last output is 5bca. We turn to the last level page table entry:

lkd> !dq 5bca000+1df*8 L1
# 5bcaef8 82a00000`056cb847

The page frame number in this case is 56cb, and our data is at byte offset zero within the respective page. In other words, the virtual address we started with maps to the physical address 56cb000.

To verify this, we can ask the debugger to display the data at this physical address as well as the data at the virtual address we had in the first place. Fortunately, the data are identical.

lkd> !db 56cb000 L40
# 56cb000 00 00 00 08 00 00 00 00-ff ff ff ff ff ff ff ff ................
# 56cb010 00 00 06 ff 00 00 00 00-40 26 f3 76 00 00 00 00 ........@&.v....
# 56cb020 e0 1d 38 00 00 00 00 00-00 00 00 00 00 00 00 00 ..8.............
# 56cb030 00 00 38 00 00 00 00 00-00 a9 f3 76 00 00 00 00 ..8........v....

lkd> db 7fffffdf000 L40
000007ff`fffdf000 00 00 00 08 00 00 00 00-ff ff ff ff ff ff ff ff ................
000007ff`fffdf010 00 00 06 ff 00 00 00 00-40 26 f3 76 00 00 00 00 ........@&.v....
000007ff`fffdf020 e0 1d 38 00 00 00 00 00-00 00 00 00 00 00 00 00 ..8.............
000007ff`fffdf030 00 00 38 00 00 00 00 00-00 a9 f3 76 00 00 00 00 ..8........v....
Published at DZone with permission of Sasha Goldshtein, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)