Talk:3 GB barrier/Archives/2017/November

Notes on the rewrite and on the previous version

I just completed a rather sweeping rewrite to include the more subtle causes of the barrier. In particular, there was previously no mention at all of the artificial 4 GB physical address limit imposed by Windows x86 client editions. As this causes the "3 GB barrier" to remain in effect even in platforms that can remap the conflicting RAM, and as Windows client editions are by far the most popular personal computer operating systems, this seemed a significant omission. It could only be covered properly by correctly explaining all of the issues that lead up to needing to go beyond the 4 GB address point even for RAM configurations of only 4 GB.

Some notes on the previous version:

• 32-bit x86 CPUs are in no way limited to 4 GB of physical address space and have not been so limited since the Pentium Pro. References are in the main article. How do you think the upper levels of Windows Server 2000 (yes, 2000, which was x86 only) got up to 32 GB RAM? (the only reason they didn't say "64 GB" was lack of hardware for testing.)

• There is no such thing as "separate physical address spaces for memory vs. I/O devices" or "not using memory-mapped I/O" on anything resembling a standard x86 platform. I invite anyone to show me separate physical address spaces could be implemented using any CPU and chipset that has ever been used on an x86-based, Windows-compatible PC. Yes, it is a theoretical possibility, but that's all it is. As for "no MMIO", even if you somehow managed to find a complete set of PCI devices with no functional MMIO locations, PCI devices will always present their configuration spaces within the same physical address space used for RAM, and existing bridge chips always map PCI address space into the common physical address space as defined by the address lines coming out of the CPU. They just don't have a way to do anything else, and the CPU doesn't have an architected way to select a separate "I/O physical address space."

• x64 processors do not implement 64-bit physical addresses. The architectural limit is 52 bits, first implementations implemented 40 bits, and current implementations only implement 48 bits. See the x86-64 article for references: Bit 63 in the PTE is the XD bit. Bits 52 through 62 in the page table entries are reserved for operating system use, and Windows x64 does use them for a "working set index." The physical addresses therefore have to stop at bit 51. It is of course possible that a sort of a "PAE squared" mode might be created, widening the PTEs yet again, but 52 bits is already so far beyond current practical use that x64 itself will probably not last long enough to need it.

• PAE is not in any way a "hack". It is simply a modified page table lookup scheme involving three tables instead of two, and the 32-bit-wide page table entries widened to 64 bits. The latter allows for more bits of physical address (as well as the NX bit). Anyone who thinks PAE is a hack, and then recommends going to x64, needs to consider that x64 takes the PAE address translation scheme and adds a fourth level of page table lookup... so it must be a hack on top of a hack? No, PAE is just fine.

• AWE has utterly nothing to do with causing or breaking the 3 GB barrier and should not be mentioned here.

Jeh (talk) 19:42, 8 June 2010 (UTC)

The Distinction between Windows 'Client' and 'Server' editions is false. Server editions, with the specific exception of large memory editions, were also limited to 4GB by service pack.

All 8086-baseed Windows-compatible CPU's and Chipsets had a separate I/O space. Using the I/O commands, the I/O line was asserted, and the data moved differently. The Address lines were shared, but only the low address lines, since the I/O space was not as big as the memory space. You should be able to download a PDF of an 80386 CPU to see how it worked. — Preceding unsigned comment added by 203.206.162.148 (talk) 06:02, 19 June 2012 (UTC)

The DataCentre Editon of 2003 retained the capacity to address >4GB, after it was removed from XP and other versions of 2003. As is documented in the existing references to this article. — Preceding unsigned comment added by 203.206.162.148 (talk) 06:20, 19 June 2012 (UTC)

Sorry but while your statements here are correct as written, they do not affect the validity of the statements in the article.

Although there are some 32-bit Server editions that are limited to 4 GB RAM (see Comparison of Microsoft Windows versions#Physical_memory_limits), they can nevertheless access RAM above the 4 GB address point. Therefore, RAM that conflicted with devices' PCI address ranges and was therefore relocated above the 4 GB point is still usable by them, up to 4 GB maximum. (Perhaps from 0 to 3.5 GB, and then from 4.0 to 4.5 GB, to make room for 0.5 GB of PCI memory-mapped BARs). The "3.5 GB barrier" is caused by a RAM address limit that is only present in the non-server or "client" products. As opposed to the 4 GB limit on the amount of RAM. The smaller Server editions are indeed limited to 4 GB RAM, but not all of that RAM has to be below the 4 GB address.

As for separate I/O space... I assure you, I am well familiar with I/O port space on x86. Again, what you say is correct, but it is not relevant to this article. You are talking about I/O port space (e.g. 0x3F8:3FF for the first serial port). It is true that I/O port space is disjoint with RAM addresses, so they do not conflict with each other. No amount of I/O port space defined by devices would ever conflict with RAM regardless of the RAM's addresses.

But what we are talking about here, what DOES conflict with and cause the relocation of RAM, are ranges of physical addresses (the same sorts of addresses used by RAM) that are decoded by I/O devices. A PCI or PCI-E device can define up to five BARs (Bus Address Ranges) and any of these can refer to physical memory addresses rather than I/O port space. We typically refer to this as "memory-mapped I/O". ISA devices could use memory mapped I/O as well, it was just harder to configure, and there was typically just one range of such addresses per card. The CPU uses regular old memory-addressing instructions, not IN and OUT, to refer to such locations, and the devices respond according to their host-controller interface spec. Any of the standard references on PCI, such as PCI System Architecture by Mindshare, Inc., will explain this further.

The trend in the last decade or so, by the way, is to get away from I/O port space and use memory-mapped IO exclusively, for several different good reasons. Yes, if all devices used only I/O port space then there would be very little to conflict with RAM addresses and the "3 GB barrier" would be more like the 3.9 GB barrier... but that's not how high performance PCI devices are being built. If you look at Device Manager in Windows, and choose the "resources by type" display, you will see how many addresses below 4 GB are being used by your devices - your video card will almost certainly be the largest consumer. Jeh (talk) 06:34, 19 June 2012 (UTC)

I hope these paragraphs clarify the issues. If not, please ask. Jeh (talk) 06:41, 19 June 2012 (UTC)

The first section of this article is misleading and has no substance whatsoever. It also reflects a grave misunderstanding of system architecture and memory management. I have deleted it. Before editing please research address translation and virtual addressing.

32bit processors can address 4GB RAM maximum and the CPU is the only part of the operating system which can access RAM. The OS has no relevance to how much RAM the CPU can handle, Windows uses virtual addresses translated to physical addresses by the CPU TLB. 32bit chipset, 32bit processor, 4GB RAM limit.

Windows Internals 6th Edition, p187

Second, the only Intel 32 CPU with PAE capability and support for more than 4GB of RAM was Xeon. Every other Intel chipset up to Core 2 Duo generation has a 4GB physical limit. https://en.wikipedia.org/wiki/List_of_Intel_chipsets

PAE is just another level of paging.

" Architecture Overview"

To understand PAE, it is useful to understand the derivation of the sizes of the various structures and bit fields. Recall that the goal of PAE is to allow addressing of more than 4 GB of RAM. The 4-GB limit for RAM addresses without PAE comes from the 12-bit byte offset and the 20-bit page frame number fields of physical addresses: 12 + 20 = 32 bits of physical address, and 232 bytes = 4 GB. (Note that this is due to a limit of the physical address format and the number of bits allocated for the PFN within a page table entry. The fact that virtual addresses are 32 bits wide on x86, with or without PAE, does not limit the physical address space.)" --Windows Internals, 6th Edition 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

Above clearly states 32bit CPU = 4GB of physical memory addressable by the CPU. Also note virtual addressing in Windows is in no way related to the physical RAM available for the processor. 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

On Virtual Memory:

Windows implements a virtual memory system based on a flat (linear) address space that provides each process with the illusion of having its own large, private address space. Virtual memory provides a logical view of memory that might not correspond to its physical layout. At run time, the memory manager, with assistance from hardware, translates, or maps, the virtual addresses into physical addresses,where the data is actually stored. By controlling the protection and mapping,the operating system can ensure that individual processes don’t bump into one another or over write operating system data. --Internals 6th Edition p15 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

Regarding earlier statement on Windows not being able to address RAM above FFFFFF:

"The size of the virtual address space varies for each hardware platform. On 32-bit x86 systems, the total virtual address space has a theoretical maximum of 4 GB. By default, Windows allocates half this address space (the lower half of the 4-GB virtual address space, from 0x00000000 through 0x7FFFFFFF) to processes for their unique private storage and uses the other half (the upper half, addresses 0x80000000 through 0xFFFFFFFF) for its own protected operating system memory utilization. The mappings of the lower half change to reflect the virtual address space of the currently executing process, but the mappings of the upper half always consist of the operating system’s virtual memory" --Internals 6th, p15 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

^See where it says "Virtual Address", not "Physical Address". That's because 0xF is not RAM, but a virtual address inside Windows.

Also, 20 processes could run simultaneously on Windows XP, each using 4GB address space but as a whole the system would eat 8GB of RAM. If that sounds like fiction, it's not. This is how OS address translation and virtualization works..... 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

On the so called "3GB barrier":

"Now that we’ve described the basic components of the virtual address space in Windows, let’s examine the specific layout on the x86, IA64, and x64 platforms."

x86 Address Space Layouts

By default, each user process on 32-bit versions of Windows has a 2-GB private address space; the operating system takes the remaining 2 GB. However, the system can be configured with the increaseuserva BCD boot option to permit user address spaces up to 3 GB. Two possible address space layouts are shown in Figure 10-8. The ability for a 32-bit process to grow beyond 2 GB was added to accommodate the need for 32-bit applications to keep more data in memory than could be done with a 2-GB address space. Of course, 64-bit systems provide a much larger address space. --Internals 6th, page239

Microsoft are specifically referring to X86, not x86-64, and not IA64.115.188.27.154 (talk) 16:49, 16 July 2016 (UTC)

x64 Virtual Addressing Limitations As discussed previously, 64 bits of virtual address space allow for a possible maximum of 16 exabytes (EB) of virtual memory, a notable improvement over the 4 GB offered by 32-bit addressing. With such a copious amount of memory, it is obvious that today’s computers, as well as tomorrow’s foreseeable machines, are not even close to requiring support for that much memory. "115.188.27.154 (talk) 16:49, 16 July 2016 (UTC)

"Unlike on x86 systems, where the default address space is divided in two parts (half for a process and half for the system), the 64-bit address is divided into a number of different size regions whose components match conceptually the portions of user, system, and session space. The various sizes of these regions, listed in Table 10-8, represent current implementation limits that could easily be extended in future releases " --Internals 6th Edition, p237

^ AMD x64/IA64 platforms do not have the 3GB usermode restriction because the CPU can address as much ram as it likes (for arguments sake). Any x64 CPU can count from 0-32GB if the machine has 32GB installed. This means usermode process running in Windows 32 have 4GB of address space available because the x64 CPU can map the process address space anywhere into RAM.

On x86 platforms this is impossible, because with the exception of Xeon the machines are not physically capable of addressing above 4GB of RAM. 2GB kernel slice leaves only 2GB left for usermode apps.. This may have been what Guy was taking about regarding AWE.115.188.27.154 (talk) 16:49, 16 July 2016 (UTC)

Merge with PCI hole ?

Hmm, interesting to see this article. I recall getting bitched out by someone years ago mentioning the similarity of this with the limits of conventional memory, and that "there is no such 3 gb barrier and the real problem is the PCI hole, you moron"... and yet here this article is, with many references, not written by me.

Well anyway, I went ahead a while back and documented the PCI hole thing in an article of its own, and now I'm wondering if perhaps this article and that one should be merged. I've decided to pretty much ignore the topic and let someone else make up their mind on it.

Do whatever. DMahalko (talk) 20:52, 8 February 2011 (UTC)

After thinking about this for a while... my take is that the PCI hole is an interesting topic in its own right. The PCI hole is necessary to, but not sufficient to be the sole cause of, the 3 GB barrier as described here -- but the PCI hole exists even where the 3 GB barrier does not, and programmers at some low levels must allow for it even in environments where there is no 3 GB barrier. Therefore the PCI hole article should describe the PCI hole in as much technical detail as is available, and then refer here: "The PCI hole is one of the contributing causes to the so-called 3 GB barrier experienced by many, but not all, computer users." As for this article, the tech details on the PCI hole could then be kept to a minimum here. (I don't like seeing such details repeated in several articles.) In other words, I don't think either article should absorb the other, and there is ample reason to have both. What do you think? Jeh (talk) 10:25, 21 February 2011 (UTC)

they are separate topics, cause and affect. Keep separate unless PCI hole is too small a topic. Widefox; talk 13:32, 12 October 2012 (UTC)

Rename Rewrite

Without checking all the refs, is 3 GB barrier the common term for the 32-bit x86 memory limit? Any support for a more descriptive title like x86 memory limit or better term? Widefox; talk 13:32, 12 October 2012 (UTC)

Maybe you should actually look at the references before asking, and before applying an "OR" tag to the whole article? Just a suggestion. A Google search on "3 GB barrier" will answer your question - yes, it's the common term. "x86 memory limit" is not suitable as there are several "x86 memory limits", and this particular limit isn't merely a metter of x86 architecture.

While I'm here... re section heads: Yes, blank lines after sections heads are optional. However it is longstanding practice here that you should not change the established format of an article simply to match your personal preference or experience.

And re binary prefixes: We use IEC binary prefixes only in a few circumstances, none of which apply here. See WP:MOSNUM. Jeh (talk) 20:39, 12 October 2012 (UTC)

Thanks for the reply, and agree my edit wasn't great. Saying that...I did actually check some of the refs and didn't see it in the first six. Which refs use the term? There's not a single ref with it in the title. My understanding is that WP:TITLE says it's normal to base the article name on a WP:RS, so together with verifiability WP:V it is a OK to question this (both on the talk page and in the article). Tagging with something like OR (or ref improve) would seem roughly right to me if sourcing isn't obvious for the title of an article. Would you agree?

My initial reaction to the title is surprise. Yes Google gives many hits - a top one linking back to here but to question the "3"!. Because there is not one barrier, much confusion, and several limit issues here - some of which are, and some not discussed. Completely overlooked is the real underlying issue of kernel-mode vs user-mode space split e.g. 2GB/2GB vs 3GB/1GB splits. My understanding was that they are dynamic on MS (4/0-0/4) (although the MS ref [1] indicates they are fixed, not dynamic), and fixed (commonly 3GB/1GB but fully adjustable at compile time) on the Linux kernel, Solaris, BSD and others may vary! Ironically linux commonly has a 3GB user-space barrier, MS not (commonly 2/2). (Distros like Ubuntu have been auto selecting a PAE kernel at install time when more than 3GB virtual is detected). These limits, and others such as MS Vista Starter edition having a 1GB limit, are possibly best all summarised at (disclosure - my wiki) [2]. So...MS is overrepresented in the examples, and for instance there's no attribution to who claims in the first sentence, and which OSs - I will tag in order to progress the article.

All of the above mean, this is not a problem per se, but the title is the tip of the iceberg of the wider scope "common memory limits on x86".

blank lines - that sounds sensible but I haven't come across that wisdom - is there a link for that practice?

IEC binary prefixes - yes you're right. WP:BRD seems to cover this as OK, wouldn't you agree?

Can you explain why you don't consider PSE-36 a valid see also? They don't have to be directly related topics you know, and I will remove the redundant one I inserted. Widefox; talk 12:03, 15 October 2012 (UTC)

What you call the "real underlying issue" of the user mode / lernel mode address space split has absolutely nothing to do with the 3GB barrier. (And your claim that it does causes me to question the validity of all of your opinions here; please see WP:CIR.) That is a division in virtual address space. The 3 GB barrier has to do with physical address space. They have very little to do with each other, and in particular the location split in virtual space (which might as easily be 1 GB user / 3 GB kernel) does not influence the amount of RAM you get to use when the 3GB barrier gets in your way.

PSE isn't related either. PSE causes some of the RAM to be managed as large pages, 2MB or 4MB at a time instead of 4K. But it doesn't cause the 3 GB barrier nor help in working around it.

Microsoft is "overrepresented" because this issue is most widely reported on Microsoft operating systems. It doesn't occur at all on Linux to my knowledge, so why should we mention it? It doesn't occur on VMS either... Should we mention that too?

And this article is not about the rest of "the iceberg" of x86 addressing limits. It most certainly is not about issues in virtual address space. It is intended to thoroughly cover the topic named by the article title. That is the "3GB barrier". By the way, if you'll read WP:TITLE a little more, you'll find that WP practice is to use popular names in articles, regardless of what is exactly correct. More: I simply can't think of a better title. ("Lack of availability of RAM above about 3 GB on x86 processors using certain operating systems" is more precisely descriptive, but is not better.) If you can, let's discuss your idea, but simply tagging the article because you're "surprised" at the title is not constructive. Jeh (talk) 10:02, 16 October 2012 (UTC)

Just to clarify above - I was talking about related limit issues, which affect people that have 4GB RAM in a system and don't get what they expect (I am not suggesting we widen the scope, just provide context via see also etc). I agree with you that "real underlying issue" is incorrect in the topic of this article (a physical memory addressing issue), but a competing limit for every process in such as system. If you let me rephrase "a completely separate, but related barrier issue affecting such systems", and I mention it not because I'm conflating virtual memory with physical memory, but because the relation (however small) affects users of such systems. Despite my misleading sentence above, its clear from my reference that the issues are separate. (As an aside: for instance earlier Linux didn't have a 3GB barrier - it could only address <1GB physical RAM due to using a 1GB virtual kernel space mapping to physical, so was a "1GB barrier").

Coming back to the topic, it does affect Linux - how could it not - its physical memory addresses as we both agree. So I disagree with you, we should mention it. (Feel free to mention VMS if you find a reference, as it would also affect it.) Just because you don't know that fact (or deduce from the underlying principle that RAM hardware addresses that aren't addressable at the hardware level will not be addressable by anything higher up) doesn't mean you should be discouraged from editing this page, like the impression I'm still getting from you, OK? Some of your comments might be seen as WP:OWN, but despite the hostility (questioning competence, suggesting I read TITLE more (yes I know its the common term), through to whitespace ownership), the title remains unreferenced (although it sounds like it is correct), and my questions unanswered - I'm still waiting for that whitespace link I asked for BTW. As this is a hardware issue, so OS independent, why is it written as an OS issue in the lead?! This is factually incorrect / OR, as tagged. Extensions such as PAE being used by an OS is a workaround issue which is why it is such an odd article, portraying it as an OS issue, when it is hardware, with workarounds (and their related problems), commercial limits (e.g. 1GB Vista Starter, 2GB 7 Starter) and competing limits by different OSs/issues. The PAE article does a much better job specifying which limit with which OS edition, which is why I linked the article so prominently. I changed this section name to rewrite, not rename (which was a question, not a suggestion). In your first reply here, you said "and this particular limit isn't merely a metter of x86 architecture.", it is, with workarounds. Widefox; talk 14:17, 16 October 2012 (UTC)

I'm on an intense work project through the end of the week. I'll get back to you after that. In the meantime, please do not assume consensus.

Might I suggest an experiment? Both of you, please try conversing without any personal pronouns. No you, he, they or usernames, and no convoluted language layering used to follow the letter but not the spirit. Talk about article content and not about user behavior. --Guy Macon (talk) 00:48, 18 October 2012 (UTC)

Agree.

OK, the PSE-36 article is now referenced as being an alternative to PAE for addressing up to 64GB RAM. Implementations are different. Widefox; talk 16:27, 19 October 2012 (UTC)

There was never contention over whether PSE36 allowed access to more than 4 GB RAM; of course it does. But the topic of this article is the 3 GB barrier. Does PSE36 cause the 3 GB barrier? No. Are there any operating systems that use PSE36 to break the 3 GB barrier? Not that I know of; Windows certainly does not. (In fact I don't know of any OSs that use PSE36 for "normal" memory management. Rather MM starts with the usual 4K pages, and large pages are used for a few special jobs, like mapping all of the kernel or mapping special buffers as requested by an app.) That's why I don't think it's relevant here. Granted they're both MM topics, but they're both leaves on the tree of MM topics; their relationship to each other is pretty thin, and including it as a "See also" could even lead to misunderstanding. Jeh (talk) 17:16, 19 October 2012 (UTC)

I agree 100%. I have actually used PSE36 to break the 3 GB barrier, but I wasn't using an OS. This was a headless embedded system that was designed to load and run a tight assembly language loop until someone pulls the plug. I needed to store and manipulate exactly 4GB of data in RAM plus few kB for the program, and PSE36 was just what I needed. I don't know of an OS that does that. Perhaps QNX might be able to -- it is pretty good at doing unusual things with the hardware -- but using Using PSE36 to break the 3 GB barrier on an OS that isn't a RTOS? Those dilithium crystals canna' take the strain! She's going warp nine as it is!! You canna' change the laws of physics Cap'n... --Guy Macon (talk) 21:47, 19 October 2012 (UTC)

Popping PSE-36 off the see also stack into the article might solve it, sorry I mean "energize"! ... (Leaving the scope and howto issues aside)...make this so! ...?:

"PSE-36 can be used to break the 3GB barrier at the OS level, but is not used by Linux, Microsoft Windows and OS X for general memory management due to performance reasons compared with the newer PAE. It was used at the driver level by Microsoft Windows RAM disk, and up to the application level on Windows NT and Windows 2000 servers, for example SAP,[3] and Oracle with the Intel Extended Server Memory Architecture (ESMA) PSE36 driver.[4]"

More than that might be undue weight, causing transporter error - she cannae take it! Widefox; talk 14:04, 20 October 2012 (UTC)

JEH are u a member at NexusMods? because I had this same discussion with a user Fomod who quoted exactly what you've written here in the talk page: :32-bit x86 CPUs are in no way limited to 4 GB of physical address space and have not been so limited since the Pentium Pro

Guy is correct on every point made. The so called 3GB limit exists only when hardware does not support over 4GB of RAM. This would include nearly all chipsets made by Intel up to the 965. Look at Intel chipset memory support page here on Wiki.

IA32 and IA64 are not Windows OS versions, IA32 is an Intel 32bit processor without PAE capabilty. Therefore it cannot access over 4GB, and therefore neither can Windows.

ALL 32bit processes are capable of 4GB virtual addressing, ALL. It's the Intel x86 compilers which limit process address space, not the OS. 32bit apps complied for AMD64 have pointers allowing addressing over 3GB. x86 compiled apps have truncated pointers which above 3.2GB which point to null. It depends on the process.

@Guy I can give you undocumented info published by Microsoft confirming both artificial OS restriction and the real x86 IA-32 hardware limitation. Please let me know I will gladly supply it for you. :) 115.188.27.154 (talk) 23:12, 14 July 2016 (UTC)

No, Im' not a member of "NexusMods". I've written approx. that same text in a few different places, though, so it's possible "Fomod" picked it up from one of those. It's hardly an original thought, though; I can't claim it.

"Guy is correct on every point made." The last time Guy made any points here was nearly two years ago. I countered and he did not reply.

"I can give you undocumented info published by Microsoft confirming both artificial OS restriction" - how can it be both "undocumented" and "published"? In any case, why not just post it here if you think it's relevant? Why does Guy have to ask you for it?

"IA32 and IA64 are not Windows OS versions," -- Technically true. I believe in the current editions MS is using the "32-bit" and "64-bit" qualifiers. Nor does MS have versions specific to, say, Intel64 vs. ia32. But if one talks about "Windows for IA32" we can be reasonably certain they're talking about 32-bit Windows, yes? What's your point?

"The so called 3GB limit exists only when hardware does not support over 4GB of RAM." Not so. 32-bit Windows enforces the 4 GB maximum physical address, and therefore has the 3GB limit, on all 32-bit client versions, XP SP2 and later, even when hardware (all of it, CPU and chipset) supports over 4GB of RAM. Even if there's more than 4 GB of RAM present. But Windows server editions don't limit themselves to 4 GB physical address (i.e. they can use RAM that has been "remapped" to above that point) and therefore don't have the 3GB issue. So, yes, although hardware contributes to the problem, Windows chooses to use high physical addresses or not, and if not, the 3 GB limit applies. And yes, this article's scope does include Windows' contribution to the issue.

" IA32 is an Intel 32bit processor without PAE capabilty." So you're saying that when they added PAE, they dropped the "IA32" designation? What do they call them, then, given that "x86" is not an Intel-specific term? You're aware that there were ia32 (i.e. not Intel64) Xeons, and all of them had PAE? Today, I find nothing in the Intel® 64 and IA-32 Architectures Software Developer’s Manual"[5] to support your claim that if the processor has PAE, it can't be an IA32 processor. Specifically, the manual covers both IA32 and Intel 64; volume 3, section 4.4 covers PAE paging on IA32 platforms; paging on Intel64 isn't discussed until section 4.5. So please give a reference.

"[a 32-bit processor without PAE] cannot access over 4GB, and therefore neither can Windows." True, but there are 32-bit processors (and entire platforms) that can address over 4GB via PAE, and 32-bit Windows nearly always enables PAE these days (so they can use the NX feature). Yes, we have the 3 GB barrier because 32-bit Windows clients refuse to look at RAM above 0xFFFFFFFF. (But the corresponding Server editions do not so refuse, so they don't have the barrier.) Since memory-mapped IO devices have to fit in that same 4 GB, this leaves somewhat less than 4 GB for RAM. And wham, there's the Windows-caused "3 GB barrier" on the client versions. It's often a bit more; it can be less if you have devices chewing up a lot of address space. I have one system here that shows up with just 2 GB RAM! The same will happen on any other OS that refuses to look at physical addresses above 0xFFFFFFFF. (Note that it is not the amount of RAM that matters to Windows. It's where the RAM shows up in the physical address space that matters. If for some reason you had 4 GB of RAM but 3 GB of it was at addresses above 0xFFFFFFFF, these versions of Windows would let you use 1 GB of the RAM.)

"ALL 32bit processes are capable of 4GB virtual addressing, ALL. It's the Intel x86 compilers which limit process address space, not the OS. 32bit apps complied for AMD64 have pointers allowing addressing over 3GB. x86 compiled apps have truncated pointers which above 3.2GB which point to null." I don't know what virtual addressing or v.a.s. has to do with this article, but:

1. It's the "split address space" design of some OSs, including 32-bit Windows, that limits per-process address space to 2 GB (or 3 GB with the /3GB option). However, a process running in that environment can still issue addresses all the way to 0xFFFFFFFF virtual! It's just that the OS doesn't make any pages from 0x80000000 through 0xFFFFFFFF (or, with the /3GB option, from 0xC0000000) user-mode accessible, so these attempts will fail with a memory access violation.

2. Sorry but your claim about "truncated pointers above 3.2GB which point to null" is just plain wrong. Evidence: Compile a program that calculates a pointer value of, oh, 0xFE000000 and dereferences it, under an exception handler. Look at the code under the debugger, you'll find no truncation; the address is there. Let it try to do the reference and look at your exception info - you'll find the address it failed to reference was whatever you told it to, not NULL. Also, if you build a 32-bit app with that very same compiler, but build the exe with the "large address aware" flag, and run it on x64 Windows, you can actually use all the way to 0xFFFFFFFF virtual. Run it on x86 Windows with the /3GB switch and it can use up to 0xBFFFFFFF. With the same binary. There is no need for a "compiled for AMD64" switch for a 32-bit compiler. btw, since we use the same compilers for kernel mode code, it'd be pretty funny if they refused to emit certain addresses!

And please sign your posts. Thank you. And if a bot comes along and signs for you, please don't move the resulting sig to the top of the article. Thank you. Jeh (talk) 23:41, 14 July 2016 (UTC)

Scope and merge with old version

I just looked at the history, and the old version [6] lead is more correct. This depends on the scope of this article (the meaning of 3GB barrier - references of it are currently evading the article), so in the mean time, I've tagged the issues. Widefox; talk 14:48, 16 October 2012 (UTC)

I agree. Please put the old lead back in. Note that this does not imply that we have consensus for leaving it that way forever -- I want to see what Jeh has to say when he gets some time, but in my opinion the old lead is superior. --Guy Macon (talk) 00:38, 18 October 2012 (UTC)

I'm cautious about editing further until we find what the topic/scope of the "3GB barrier" is... the first 50 or so Google hits didn't find a WP:RS, but plenty of common usage (tip "3GB memory barrier" also helps weed out other issues such as /3GB). The best so far is the non-RS [7]. Right now, I'm leaning more towards the topic (as titled) being an OS user frustration issue (with the largest user base being Windows users), so I'd rather hold off changing back to the underlying x86 architecture hardware issue until we have some RS, else it is OR. I'm happy for anyone to remove all my changes, switch versions etc in the mean time. There's also quite a lot of good coverage on the hardware issue x86 memory address (overlap) topic on other articles, so I've linked the different articles to help for now, and I've finished fixing up issues elsewhere. Widefox; talk 09:27, 18 October 2012 (UTC)

I consider Jeff Atwood (author of The ASP.NET 2.0 Anthology and Effective Programming: More Than Writing Code) to be a reliable source. Blogs are OK if they are written by recognized experts. I would also consider Daniel Rutter to be a RS. See http://www.dansdata.com/askdan00015.htm

If you really want to avoid blogs, here is a paper by Sun Microsystems: http://techfiles.de/dmelanchthon/files/memory_hole.pdf VMWare is also a good source (the VM has to emulate x86 hardware so they have the same issues even though the "physical" addressing is virtual). http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1014006&sliceId=1&docTypeID=DT_KB_1_1&dialogID=461356092&stateId=1%200%20461354836

Of course we need sources, but I know what the answer will be: the limit is in hardware, it varies by motherboard, Microsoft products tend to pick a fixed arbitrary limit that is compatible with every motherboard, and BSD and Linux both tend to use whatever is available on the motherboard they are running on. --Guy Macon (talk) 11:03, 18 October 2012 (UTC)

OK, good. Agree those are all RS. Lead can be "3 GB barrier (or 3GB memory barrier http://www.dansdata.com/askdan00015.htm)" (although he uses "3Gb" but let's overlook that), and it is a hardware issue (of course) so the old lead would fit better. That Sun paper is good, the same topic but is commonly titled PCI hole (formal title Memory Hole in Large Memory X86 Based Systems) and doesn't mention this title which is what I'm after. That sways me towards proposing the merge suggested above (reversing my opinion above now we have sources). Jeff Atwood yes - [8] and the non-RS [9] which gives us a reference for the range "~2.75GB to 3.5GB " (which we should proviso that there's no lower limit - there's 1GB graphics cards etc... also this is good [10] (Vista max is 3.12 GB, min 3GB or less). All of these (apart from the Sun) are software perspective. PCI hole is the main cause of the 3 GB barrier effect. PCI hole has those two references already. As the Sun ref links the common terms, a merge proposal would be my next step. Widefox; talk 14:26, 18 October 2012 (UTC)

A blog by Jeff Atwood (author of The ASP.NET 2.0 Anthology and Effective Programming: More Than Writing Code) is a reliable source. Blogs are OK if they are written by recognized experts. --Guy Macon (talk) 07:19, 16 July 2016 (UTC)

Windows x86 4GB+ kernel patches

It'd be nice if we could work the existence of kernel patches for client versions of 32-bit Windows in here somehow, both because it's likely to be interesting to readers and because there are potential dangers to using them that are rarely mentioned by providers (blue screen loops, insecure/outdated kernels, etc.). The trouble is, the various patches and descriptions of how to make them are self-published, and the secondary sources aren't better - the first link may be closest to a "reliable source", but it's also one of the worst explanations, with a fundamental misunderstanding of the relevance of PAE. Geoff provides a great technical description, and he has the history to be an expert, but (perhaps understandably, given that history) his article is pervaded with suggestions of ulterior motives. Is anyone aware of resources that I'm missing? GreenReaper (talk) 12:07, 16 June 2014 (UTC)

I think this fails under, or at least is severely damaged by, WP:NOTHOWTO - Wikipedia is not a "tweak guide." And I agree re the tone of Geoff Chappel's writing. He even says he has no evidence for ulterior motives but goes on to ascribe them to Microsoft anyway. Jeh (talk) 18:50, 16 June 2014 (UTC)

Recent edits

Re. recent edits by Widefox: Taking things a step at a time... how about moving the article to "3 GB memory limit", with a comment in the lede "sometimes colloquially referred to as the "3GB barrier"? There is a RS for "limit" here and certainly we can find ample evidence for the "sometimes referred to as" claim; we don't need a reliable source for the latter, just a few examples. Jeh (talk) 19:48, 7 November 2014 (UTC)

The problem may be either in the scope of the article, or just the name. It isn't a fixed size e.g. 3GB (it may be 0GB with a large graphics card), and it's not a barrier. Without properly setting this out with WP:RS, the article is meandering. Looking back, there was a suggestion to merge with PCI hole by User:DMahalko. That has merit, but isn't perfect either. As is, it has a danger of being a howto (be able to address all your memory). Countering that opinion is 640 kB barrier. Ideas? Widefox; talk 19:57, 7 November 2014 (UTC)

We can take care of "it isn't a fixed size" with references (there is a direct quote we can use in MarkR's article). There is no "how to" aspect because in no case does the article tell you how to work around the limit (without changing OSs). And let's not let a quest for perfection stand in the way of "good enough". Jeh (talk) 20:15, 7 November 2014 (UTC)

I do not think the article is "meandering". It explains the limit, starting with the principles and proceeding through an explanation of how the various facts combine together to cause the apparent limit. On the contrary, I frankly think your criticism is "meandering". You're all over the place.

I especially object to your concern that "it has a danger of being a howto". That could be said about a great many articles. Your phrasing makes it clear that you don't think it's a howto now. Why don't we worry about fixing that problem when and if it occurs? Clearly articles' current form should not run counter to anything in WP:NOT; but must they also be edited so as to head off any possible future WP:NOT issues? I think that's a very unreasonable criterion! (And anyway, it's ok if an article happens to provide or suggest a solution to a problem. NOTHOWTO just says that "...instructing the reader in the imperative mood about how to use or do something..." is not acceptable.)

Let's start with the title - which is defined in the lede, with a reference. You justifiably added a template saying that the title-term is not mentioned in the reference. That to me would indicate a problem with the title, if we can't find a RS that uses this term in this way. (Whether or not this fixes any other of the article's problems as you see them is a separate issue.) I have proposed a fix: Move the article to a name that IS defined in a RS, which I gave above. This is not intended to address what you see as the overall "problem" with the article, but it is intended to address the concern you raised by adding that particular template. What do you think about that specific suggestion? Jeh (talk) 21:43, 7 November 2014 (UTC)

Scope: 1. RAM addressing limits on OSes (possibly best as a list article) or 2. hardware limit due to 4GB address space (ala 640 kB barrier). The lede now says this is a software limit, when it is a hardware limit. We agreed to reset the article before, hence the meandering (like it or not). Widefox; talk 11:38, 8 November 2014 (UTC)

1. "List" articles are discouraged in general. In fact, lists within articles are discouraged in favor of prose. An encyclopedia is supposed to present information in context, not just a series of facts.

1a. Most addressing limits on OSs are hard limits, e.g. licensing ("you have to pay for a higher-priced product to use >4 GB RAM") or implementation details ("we didn't implement PAE or PSE, so we're stuck at 4 GB RAM for now"). The "3 GB limit" phenomenon is not. It really is a different sort of thing.

2. This is not solely a hardware limit, and it is not very much related to the 640KB barrier. Hardware on the systems in question is perfectly capable of making 4 GB of RAM available to the OS out of 4 GB installed. If it were a hardware limit then it would appear on all operating systems, not just client versions of x86 Windows. It is the result of details of the hardware implementation (memory hole remapping) combined with an OS restriction (x86 Windows clients since XP SP2 are coded to refuse to use RAM at physical addresses above 4 GB), even though they implement an addressing mode that is required to do so (PAE)). If your complaints are based, or significantly based, on your assumption that it's solely a hardware limit then your complaints are ill-founded.

3. What you see as "meandering" in the article is a step-by-step explanation of each of the factors: PCI address space is constrained to <4GB; RAM and PCI devices can't be at the same address, so where PCI devices are present, RAM cannot be; memory controllers relocate conflicting RAM above the 4 GB point; x86 Windows clients refuse to use RAM above the 4 GB point.

3. When did we "agree to reset the article before", and to what? And so what?

4. I don't understand why you won't comment yay or nay on a specific suggestion to address one of your specific complaints. Jeh (talk) 17:35, 8 November 2014 (UTC)

1. prose is preferred in an article, but where does it say lists articles are discouraged? They're a valid option if it fits this better (being able to be quantitative about the limits) - e.g. there's a related Physical Address Extension#Microsoft_Windows table.

1a depends how you define "hard limit" - it's the other way around for me - hardware limits all systems, software can only operate inside limits of the hardware, and software can be modified. (Also to disambiguate, some OSes have "hard" and "soft" process limits (for resources such as memory etc)). It is a (32-bit) hardware address space limit (and agree it would be good to have an article rather than a list). It is the same for all OSes - 32-bit Linux same as Windows. 32-bit PAE same. 64-bit same. Of course, MS sometimes sets lower than technical limits for other reasons (licensing or support).

2. It is. Again it depends on definition of "perfectly capable". Address space ran out, there's workarounds/extensions. Some chipsets / OSes didn't support those. Mess. Use an extension like PAE to extend that to 36-bit or (obscurely PSE-36), or workarounds like remapping. Hardware limit. Software may impose a lower limit or not, but fundamentally hardware yes (but of course it depends what the scope is - we define it by a "barrier size" which seems to be the variable effect not the cause but may be handy if the COMMONNAME). If I remember correctly, MS chose not to support PAE on client versions due to needing PAE drivers for all hardware, a lesser issue on servers which would also benefit first.

3. To clarify, I meant over time, not down the article, hence the reset. Wasn't it the lede wording? 4. and 2. well I don't have much more to add. Widefox; talk 03:32, 9 November 2014 (UTC)

1. If prose is preferred to a list in an article then I do not think it is a stretch to conclude that an article with no prose, just a big list, is to be discouraged unless the information to be presented needs nothing more than what a list can provide.

1a. What I mean by "soft limit" is that neither the hardware nor the operating system has anything that says exactly what this limit will be. On some platforms it's 3.6 GB. On others it's 3.2. I have one dual-CPU-socket rig on which I intentionally loaded up PCI space, just to see how bad it would get - the OS saw only 2 GB of RAM.

I do not see how you can say "32-bit Linux same as Windows" when Linux does not always show the problem on platforms where Windows 32-bit clients do. To my knowledge, only 32-bit Windows clients refuse to use RAM above the 4 GB address boundary even on systems where a) memory hole remapping is present and enabled, b) PAE support is present, and c) PAE is enabled in the OS.

2. But, but... Microsoft does implement PAE on 32-bit client versions, at least if you enable DEP.. which btw is enabled by default in recent versions; therefore, so is PAE. (PAE is required for DEP because without PAE there simply is no NX bit in the PTEs.) So lack of PAE support is not why the "3.5 GB limit" exists on these operating systems. It exists because there is code that does approximately this in 32-bit Windows client versions:

if (physical_memory_address > 0xFFFFFFFF) ignore_the_memory();

The issue was never that "PAE drivers" had to be written. Drivers in Windows are unaware of PAE per se, in that drivers do not manipulate page table entries directly (and actually would have a tough time doing so within the exposed DDIs). A properly written driver—one that uses the PHYSICAL_ADDRESS type where appropriate—will run without so much as a rebuild with or without PAE. The issue was that some drivers, particularly those for devices that were commonly used in client systems but not so much in servers (i.e. high-end graphics cards that had been optimized for gaming performance), had been written with an "optimization" to use only 32-bit types to hold physical addresses. With PAE turned on this of course was a disaster, as the high bits of the addresses were simply being ignored. So Microsoft added something like the above pseudocode in their 32-bit client editions, XP SP2 and later.

One might say this is a form of "lack of PAE support", but since drivers do not directly interface with PTEs, and the PHYSICAL_ADDRESS type has always been 64 bits wide (as far back as the October '92 NT 3.1 pre-release DDK, long before PAE support was added), I would disagree. It's just "lack of support for physical addresses wider than 32 bits."

(If you're wondering why a PHYSICAL_ADDRESS was that wide that long ago, it had to do with some of the non-x86 platforms, on which some of the "address bits" were not really "addresses" but rather a sort of "instruction" to a memory controller, bus bridge, or etc.)

And this is why I say that it is, at least in the final step, an OS issue. Even where the platform offers memory hole remapping (which implies greater-than-32-bit physical addresses, which implies PAE support), even if Windows runs with PAE enabled (as it does by default in recent versions, because otherwise you don't have DEP/NX, and they want that), the 32-bit Windows clients say "that's all very well, but I'm just going to pretend that that RAM above 0xFFFFFFFF physical doesn't exist." 32-bit Linux does not do that. 32-bit Windows servers do not do that.

Are you familiar with the "last opportunity to avoid the accident" principle? 32-bit Windows clients do not simply not take the opportunity to avoid the problem. They actively cause the problem, even where the rest of the platform allows the solution, even with PAE enabled, via the code represented above.

3. I'm trying, but I really don't understand what you're saying here. I can't see any "agreement to reset" in the talk page history, unless you refer to you and Guy Macon agreeing to change the lede to say that it's purely a hardware issue... and I most strenuously disagree with that, for reasons just given.

Seeing no objection (I guess), I'm going to go ahead and move the article and fix the reference so the article title uses a term that is defined in a RS... unless I can find a RS for the existing one. Probably won't happen tonight. Jeh (talk) 07:54, 9 November 2014 (UTC)

I would advise against just "going ahead" and changing the article while things are still being discussed on the talk page. There is no reason to rush. I would have no issue with you reverting Widefox's recent changes. That's what WP:BRD advises.

I believe that your statement above ("neither the hardware nor the operating system has anything that says exactly what this limit will be. On some platforms it's 3.6 GB. On others it's 3.2.") is incorrect. 32-bit hardware does indeed say exactly what the upper limit will be. It varies with the hardware, but for a particular hardware instance it is fixed. The operating system may set an upper limit that is lower than the hardware upper limit. Any OS that runs on a 32-bit processor has the same limitation -- if it has instructions that access the extra features in x86-64 it will not run unmodified on x86. If it runs unmodified on x86 it is subject to the hardware limit.

Jeff Atwood (author of The ASP.NET 2.0 Anthology and Effective Programming: More Than Writing Code) wrote this: "To be perfectly clear, this isn't a Windows problem-- it's an x86 hardware problem." [11]

Microsoft says "address space used for hardware and ROMs causes that much memory to get shoved upwards, and it ends up above the 4GB boundary. Without PAE, the processor is capable only of addressing memory below the 4GB boundary, which means that the memory above that boundary is inaccessible."[12]

Sun Microsystems says "The PCI hole exists below 4GB to ensure that all 32-bit software can reach those addresses. The physical memory hidden by this hole is not usable by the operating system software and therefore is “missing” from the system. The size of the PCI hole is the total amount of PCI/AGP address space consumed for all devices as configured by the BIOS." [13]

Note that I am not saying what I think the wording of the article should be, nor am I agreeing with any other comment (including my own comments from two years ago when hardware and OSs were quite different). --Guy Macon (talk) 21:15, 9 November 2014 (UTC)

Even though I don't completely comprehend what you argued with, "Any OS that runs on a 32-bit processor has the same limitation" might not always be correct, for some PNP Operating System with some specific Graphical drivers might re-map addressing space above firmware level, on OS level directly, that would varies the "same limitation". In other words, platform like this, on Windows and on Linux it would have different physical addressing limits rather than the same. Janagewen (talk) 23:16, 9 November 2014 (UTC)

replies to Guy Macon:

" I would have no issue with you reverting Widefox's recent changes. " Thanks, but I agree with the "not cited in reference" template on the article title. I haven't found a RS for the article title. I have found one for "3 GB limit". The only way to fix that is to change the "definition sentence in the lede" - and that means changing the article title. In the meantime I'm not going to revert the template, because it's a completely valid criticism.

"Microsoft says: Without PAE, the processor is capable only of addressing memory below the 4GB boundary, which means that the memory above that boundary is inaccessible." True, but (I really don't know why this has to be said over and over) all the versions of Windows we're talking about can use PAE. So, if it were not for the intentional blockage I mentioned above, they can address RAM above the 4 GB boundary. At some point recently (certainly by 7) they even enabled PAE by default. They had to because they wanted hardware DEP enabled by default.

"It varies with the hardware, but for a particular hardware instance it is fixed." That's not true. Due to the weird granularities with which PCI address space is allocated, it depends in part on the order in which PCI resources are assigned.

Although certainly an authority on .NET programming, I'm not aware of Jeff Atwood's credentials as far as the details of Windows kernel mode memory management are concerned. The pseudocode I gave above does exist as real code in 32-bit Windows client editions. It does not exist in 32-bit Windows Server editions, nor in Linux, etc. 32-bit Windows Server editions, and Linux, running with PAE enabled on platforms that implement memory hole remapping, will not see the "3 GB barrier". Windows 32-bit client editions will. That makes it a Windows issue. Jeh (talk) 01:20, 10 November 2014 (UTC)

From your comment elsewhere on this page (" I countered and he did not reply") you appear to think that if someone chooses to not respond, then your argument must be valid, so I am replying now. Regarding my claim that "It varies with the hardware, but for a particular hardware instance it is fixed" you asserted "That's not true. Due to the weird granularities with which PCI address space is allocated, it depends in part on the order in which PCI resources are assigned". The problem with that argument is that, just because a system can be configured in such a way so as to have different amounts of memory available, that does not change the upper limit, which I define as the largest amount of memory available no matter how you configure the system. Looking at the larger picture, what we have on this article are several editors who have a deep knowledge of this issue (in my case mostly a deep knowledge of what the actual hardware can and cannot do, based upon my work on embedded realtime robotic control software that runs on the bare metal without bothering to support keyboards, mice, displays or (other than when loading the software) disks -- I consider myself to only have a basic understanding of how Linux or Windows interacts with that hardware). As often happens in situations like that, what seem to be disagreements about facts are often really just two people talking about closely related but subtly different things, such as "the amount of memory accessible with a specific PCI configuration" and "the amount of memory available no matter what the PCI configuration, including with all PCI devices disabled." --Guy Macon (talk) 04:50, 15 July 2016 (UTC)

I would also point out that 115.188.27.154 saying "Guy is correct on every point made" followed by a paraphrase of what he thinks I meant is a sure way to trigger the sort of misunderstanding I mentioned above. Again, I would strongly encourage everyone involved to try conversing without any personal pronouns. No "you", "he", "they" or "usernames", and no convoluted language used to follow the letter but not the spirit of the "no personal pronouns" rule. In other words, talk about article content and not about user behavior. --Guy Macon (talk) 05:02, 15 July 2016 (UTC)

115.188.27.154 (talk) 17:53, 15 July 2016 (UTC) I agree I was generalising on you being %100 correct, however the points I made to Jeh were my own and directed at JEH not you.115.188.27.154 (talk) 17:53, 15 July 2016 (UTC)

I take your point about the two subtly different things. But the subject of this article, the "3 GB limit", does vary, even with the same hardware but a different enumeration order (e.g. enum by firmware vs. OS, or by different versions of the same OS. So it is the former case, not the latter. The latter would simply be 4 GB for 32-bit Windows clients (XP SP2 and later) or for any other OS that will not let itself use physical addresses higher than 0xFFFFFFFF. But that isn't the interesting case, not the case being discussed here. "Everybody knows" that with 32 bits you can address 4 GB. The reasons the "3 GB barrier" merits an article at all, the reason it has been discussed on just about every IT-oriented forum on the planet since XP SP2 showed up, are that, one, it surprised people who were expecting 4 GB - and why not, they have 4 GB, and Windows claims to support 4 GB; two, it's a "strange" number, not anything that has an obvious connection to binary numbers; and three, it does vary markedly from one system to another, for no obvious reason. Jeh (talk) 05:24, 15 July 2016 (UTC)

Oh - I also take your point about WP:FOC. However I see nothing wrong with addressing points made by someone by noting that they said them. Just as I said "I take your point" just above. But I don't see that using e.g. "you said..." as part of a reply to what they said is an out-of-bounds comment about "user behavior". That they said it is already evident from the page. Jeh (talk) 06:39, 15 July 2016 (UTC)

"There is no 3GB barrier"

On no, I'm not going through a repeat of Nexus again. I don't care if it's been discussed on every site on the internet, it doesn't change anything. Like I said to you I've been working for Fujitsu Deployment for 12 years and surprisingly we too discuss these matters. MSDN is not even a reliable source because half the articles are inaccurate or outdated. There is no 3GB barrier, there are 32bit motherboards without remapping and 4GB limits. If you had been around in 2004 with an AMD64 CPU and XP with 8GB of RAM installed you'd already know already..... And 0xFFFFFF? Do u know what 0xFFFFFFF is? You should look that one up. ;)

How about I make a statement, if you disagree that's fine I will provide documentation to back that one point up. Then u edit OK? We'll move on to the next one and the next one....and so on...  :) Just let me what u want to see and I'll supply it but please be specific. Btw how do I even do that...can we post screenshots or anything here or....? I have no idea how this works... :P 115.188.27.154 (talk) 17:53, 15 July 2016 (UTC)

You can try it that way if you like.

"Like I said to you I've been working for Fujitsu Deployment for 12 years" - where did you say that? The word "Fujitsu" does not occur on this page nor on any of the other talk pages you've edited.... unless it was under a different IP.

I told you on Nexus via pm before blocking you. Not that it really matters. PS, I know full well you're not a developer, so u can drop that facade too.

I was going to comment on your "processes are not fooled into thinking they are the only process because the memory addresses don't start at 0:" or something. But waste of time.. Every single process in Windows has a virtual address starting with 0. That's how virtualization works.

"There is no 3GB barrier, there are 32bit motherboards without remapping and 4GB limits." And the result of those is a limit on widely-reported usable RAM of around 3 GB. This is commonly called the "3GB barrier". The limit definitely results from the scenario you describe, so what you mean by "there is no 3GB barrier", I can't figure out. At one point you recommended that people read Windows Internals; Mark Russinovich has also written this:

"The result is that, if you have a system with 3GB or more of memory and you are running a 32-bit Windows client, you may not be getting the benefit of all of the RAM. "[14]

Yes...and what?

I'm aware of what Mark says, Technet blogs are not Winternals publications, they are just blogs.115.188.27.154 (talk) 01:26, 16 July 2016 (UTC)

A blog by Jeff Atwood (author of The ASP.NET 2.0 Anthology and Effective Programming: More Than Writing Code) is a reliable source. Blogs are OK if they are written by recognized experts. --Guy Macon (talk) 07:19, 16 July 2016 (UTC)

Winternals editions are around 650 pages of architectural breakdown with undocumented info not available on MS. And have u got something specific to ask me because I don't see any questions yet. .115.188.27.154 (talk) 01:26, 16 July 2016 (UTC)

...i.e. the memory beyond about 3 GB just isn't used by Windows. So the claim "there is no 3GB barrier" just doesn't make a lot of sense to me. I've seen it, and if you have the experience you claim, you've seen it. Maybe you want to rename the article to "RAM usability limits of substantially less than 4 GB in operating systems with a claimed upper limit of 4 GB"? Or "... in operating systems that refuse to access RAM at addresses higher than 0xFFFFFFFF"?

No, but what I have seen is XP using 46GB of RAM on a database server with Opteron installed. 32bit OS use 64GB of RAM...amazing I know. And 0xFFFFF is a virtual address range ASSIGNED by Windows to onboard devices, that's how drivers access hardware. FFFFF isn't real nor does it relate to RAM. The OS kernel accesses RAM through real memory addresses starting with 0x1.

"If you had been around in 2004 with an AMD64 CPU and XP with 8GB of RAM installed you'd already know already....." What makes you think I wasn't? I've been around for far longer than that. I've also had Itaniums and Alphas and VAXes through the "lab" here. I've written quite a bit of code that deals directly with page tables, etc... if there's anything I know in this field, it's how Windows (and VMS) and x86 and x64 do memory management.

You developed code for page tables? So you work for Microsoft? That's impressive because the whole system uses pagetables. Page tables allow a 32bit OS access over 4GB, PAE uses page tables. You knew that already though right? Also the amount of RAM a CPU can access is not related to the OS.115.188.27.154 (talk) 01:26, 16 July 2016 (UTC)

You're not a developer, so you can just drop that facade already.... 115.188.27.154 (talk) 01:26, 16 July 2016 (UTC) I honestly don't know why you're writing this page mate as you have a pretty limited understanding of the architecture... .. 115.188.27.154 (talk) 01:26, 16 July 2016 (UTC)

" And 0xFFFFFF? Do u know what 0xFFFFFFF is? " Uh, I typed neither of those. I typed 0xFFFFFFFF, eight F's, which is 32 bits of 1's, which is the highest possible unsigned number in 32 bits. i.e. the address of the highest-addressed location with 32-bit addresses. It's easier than writing out 4,294,967,295, and far more precise than saying "4 GB". Jeh (talk) 18:41, 15 July 2016 (UTC)

As I said above nothing at all to do with RAM115.188.27.154 (talk) 01:26, 16 July 2016 (UTC)

@Jeh there is one fundamental law of Operating Systems which you don't understand, and that is Windows uses virtual addressing. The CPU and kernel is the only part of the system which has direct access to RAM. The physical to virtual translation is done by the CPU TLB.

Somebody deleted all my references so I will add again.

Memory Limits for Windows and Windows Server Releases

Limit on X86

Windows Server 2008 Datacenter 64 GB

Windows Server 2008 Enterprise 64 GB

Windows Server 2003 R2 Datacenter Edition 64 GB 16 GB with 4GT)

Windows Server 2003 R2 Enterprise Edition 64 GB (16 GB with 4GT)

Windows Server 2003 with Service Pack 2 (SP2), Datacenter Edition 64 GB

Windows Server 2003 with Service Pack 2 (SP2), Enterprise Edition 64 GB [1]

https://msdn.microsoft.com/en-nz/library/windows/desktop/aa366778(v=vs.85).aspx 115.188.27.154 (talk) 12:51, 16 July 2016 (UTC)

External links modified

Hello fellow Wikipedians,

I have just added archive links to one external link on 3 GB barrier. Please take a moment to review my edit. You may add {{cbignore}} after the link to keep me from modifying it, if I keep adding bad data, but formatting bugs should be reported instead. Alternatively, you can add {{nobots|deny=InternetArchiveBot}} to keep me off the page altogether, but should be used as a last resort. I made the following changes:

• Attempted to fix sourcing for http://www.wired.com/science/discoveries/news/2002/07/53998

When you have finished reviewing my changes, please set the checked parameter below to true or failed to let others know (documentation at {{Sourcecheck}}).

Y An editor has reviewed this edit and fixed any errors that were found.

• If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
• If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—^{cyberbot II}_{Talk to my owner:Online} 08:02, 29 March 2016 (UTC)

Someone please tell me how I can upload some evidential documentation and put an end to this discussion.115.188.27.154 (talk) 01:26, 16 July 2016 (UTC) :)

Please read WP:OR. Also, I am guessing that if you pick ten-words at random fom out of your "undocumented" material and google that phrase, you will find that it is available on many sites that you can link to in a talk page discussion. --Guy Macon (talk) 07:19, 16 July 2016 (UTC)

Ok I'll read it.. Btw yes this really is undocumented info which is why you won't find it on MSDN. Windows Internals 6th Edition, part 2. Published by Microsoft.

It's essentially a bible for those in deployment and testing sectors of the industry. Or anyone wanting to get under the hood of Windows really.....:)

From the introduction:

Windows Internals, Sixth Edition is intended for advanced computer professionals (both developers and system administrators) who want to understand how the core components of the Microsoft Windows 7 and Windows Server 2008 R2 operating systems work internally. With this knowledge, developers can better comprehend the rationale behind design choices when building applications specific to the Windows platform. Such knowledge can also help developers debug complex problems

Because this book describes undocumented behavior of the internal architecture and the operation of the Windows operating system (such as internal kernel structures and functions), this content is subject to change between releases. (External interfaces, such as the Windows API, are not subject to incompatible changes.) 115.188.27.154 (talk) 13:41, 16 July 2016 (UTC)

115.188.27.154 (talk) 12:54, 16 July 2016 (UTC)

@Guy: Ok I tried google books but unsurprisingly page 321 is missing from the Internals preview. p321 contains the undocumented info on licensing restrictions. What to do?

115.188.27.154 (talk) 13:20, 16 July 2016 (UTC)

Vandalism.....

What is going here? all my posts have been rearranged..115.188.27.154 (talk) 14:36, 18 July 2016 (UTC)

Replies to previous discussion

Many claims and comments have been made on each of a number of distinct sub-topics; replying to each in place would result in even more of a cluttered mess than the above. (I'd have to make essentially the same reply in several places.) Accordingly I have copied the existing comments and grouped them here by subtopic. In the below, the comments to which I'm replying are at zero-level indent, italicized, and with the original signatures for the blocks in which they appeared. My replies are indented by one level. Jeh (talk) 03:43, 9 August 2016 (UTC)

I don't think I missed anything that is both substantive and not already covered, but if I have, or if it appears I have quoted anything out of context so as to alter its intended meaning, please let me know.

I would appreciate that replies either be at the end of each subsection here (and be confined to the subtopic of that subsection), or in a whole new section. Line-by-line replies will not, I think, be conducive to discussion (as well as being discouraged by WP:TALK). Jeh (talk) 18:16, 10 August 2016 (UTC)

About FFFFFF, etc.

(We need to get this out of the way first, because we need this notation for later...)

115.188.27.154 wrote:

"Regarding earlier statement on Windows not being able to address RAM above FFFFFF: [...] 0xF is not RAM, but a virtual address inside Windows." 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

and in an apparent attempt at defense of that, .154 quoted:

[...] By default, Windows allocates half this address space (the lower half of the 4-GB virtual address space, from 0x00000000 through 0x7FFFFFFF) to processes for their unique private storage and uses the other half (the upper half, addresses 0x80000000 through 0xFFFFFFFF) for its own protected operating system memory utilization. The mappings of the lower half change to reflect the virtual address space of the currently executing process, but the mappings of the upper half always consist of the operating system’s virtual memory"

— Windows Internals, 6th Edition, page 15

and then .154 wrote:

"See where it says "Virtual Address", not "Physical Address". That's because 0xF is not RAM, but a virtual address inside Windows." 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

No sir. You have drawn a universal conclusion from a specific case. Your conclusion is incorrect. (It's as if someone saw a clock with Roman numerals on it and concluded that any use of Roman numerals must therefore be referring to a time of day.)

In that discussion, hex notation is indeed used for virtual addresses. And in many other discussions too.

But hex notation can be, and often is, used for RAM addresses. Or, more generally: Addresses in physical address space.

Heck, it's also used in contexts where it doesn't refer to addresses at all. Hexadecimial—i.e. base 16 notation—is just another way of writing a number! Do you deny that physical addresses are numbers? What notation do you think we should write them in?

(By the way, the "0x" is just C syntax (later adopted by many others) for "the following number is expressed in hex". We often leave it off where it's clear from the context. The C "x" format specifier doesn't emit the "0x", for example. The presence or absence of the "0x" doesn't say anything about what the number means.)

Here are some counterexamples to your claim. First, run the sysinternals tool RAMmap and look at the "physical pages" tab. Look at the left column. See the column heading? "Physical address". See the numbers in that column? "0x1000", etc. There is one line in that output per physical page. The numbers in the left column are the physical addresses of the first byte in each page. (Each page runs from the address shown to that address plus 0xFFF. 0x1000 bytes = 4,096 bytes = one page in x86, unless we're using large pages.)

For a summary, just look at the "physical ranges" tab.

More: Look at MarkR's post, Pushing the limits of Windows: Physical memory. Scroll down to the section "32-bit Client Effective Memory Limits". Look at the screen snapshots from the "meminfo" tool, and from Device Manager. And read what he wrote. See? Addresses in physical address space, written in hex.

Essentially the same material is in Russinovich, Solomon, and Ionescu: Windows Internals, 6th Edition, Part 2, section "32-bit Client Effective Memory Limits", page 321-323.

Here's another example from the same book:

The DirBase field [in the debugger output shown] shows that the page directory pointer table is at physical address 0xced25440. (emph. added - jeh)

— Russinovich, Solomon, and Ionescu: Windows Internals, 6th Edition, Part 2, page 263.

There you go. Physical addresses expressed in hex. You will find more examples throughout the same book, and throughout many other books. The Intel and AMD Processor Architecture references, for example.

So, your claim that any hex number (whether it has a 0x prefix or not) must be a virtual address—or any kind of address at all, for that matter—is wrong.

And every argument you've made here that's predicated on that claim is therefore invalid too.

Btw, aside from not proving anything about what a hex number can and can't mean, the passage you quoted isn't relevant to the issue of the 3 GB problem. (I'll call it a "problem" rather than a "barrier"; the actual "barrier" imposed by the OS that causes the approximately-3 GB-RAM-limit is at physical address 0x100000000 and above, which is 4 GB, but we can't exactly call it the "4 GB problem"...) That text is about virtual address space. The 3 GB problem is purely about physical addresses.

But that text does refute another claim of yours, as follows:

.154 also claimed:

"And 0xFFFFF is a virtual address range ASSIGNED by Windows to onboard devices, that's how drivers access hardware. FFFFF isn't real nor does it relate to RAM. " 115.188.27.154 (talk) 01:26, 16 July 2016 (UTC)

Incorrect on several points, I'm afraid. We already disproved your notion that any number written in hex must be a virtual address, but there are further notions here that need to be corrected.

Yes, 0xFFFFF could be a virtual address. (You could try running the sysinternals tool VMmap on several processes and see if it is in any of them. Have you done that?) And, windows does assign virtual addresses to the physical addresses decoded by hardware, and that is how drivers access hardware.

But 0xFFFFF would absolutely never be one of those virtual addresses.

I guess you're thinking that the hex numbers you see in the "memory" resources in Device Manager are virtual addresses? No. Those are physical addresses. They're in the same address space as RAM.

In fact... if you run RAMmap, as I suggested above, you'll notice that everywhere there are apparently-missing pages in what RAMmap shows you, those will correspond to address ranges of I/O devices shown in device manager's list. Why? Because they're in the same address space. Physical address space. There are "holes" in the ranges of addresses assigned to RAM because there are I/O devices in those places; the same address can't be used for both RAM and for an I/O device's "memory space". (Which is why "memory hole remapping" happens.)

That Device Manager is showing you physical addresses is stated explicitly in the Windows Internals book:

[...] the physical address map includes not only RAM but device memory, and x86 and x64 systems typically map all device memory below the 4 GB address boundary [...]. You can see the physical memory layout with the MemInfo tool from Winsider Seminars & Solutions. Figure 10-44 shows the output of MemInfo when run on a 32-bit system, using the –r switch to dump physical memory ranges. Note the gap in the memory address range from page 9F0000 to page 100000, and another gap from DFE6D000 to FFFFFFFF (4 GB). [...] You can use Device Manager on your machine to see what is occupying the various reserved memory regions that can’t be used by Windows (and that will show up as holes in MemInfo’s output). The reason is that the physical address map includes not only RAM but device memory...

— Russinovich, Solomon, and Ionescu: Windows Internals, 6th Edition, Part 2, section ""32-bit Client Effective Memory Limits", page 321-323

Now, 0xFFFFF could also be a virtual address. But it would never be a v.a. that Windows maps to an I/O device! Why not? Because 0xFFFFF, if it's defined at all, would be in the per-process, user-mode virtual address range. (That's according to the very text you quoted from Windows Internals, Book 1, p.15, and which I copied above.) The virtual addresses that Windows assigns to I/O devices' memory regions are always kernel space addresses, from 0x80000000 and up (0xFFFF8000`00000000 and up on x64).

If Windows assigned a virtual address like 0xFFFFF to an I/O device, then that I/O hardware would be directly exposed to access from user mode! And that's never allowed. (It would also only be usable from a single process, which would be pointless.) Jeh (talk) 03:43, 9 August 2016 (UTC)

And this is really key to understanding the "3 GB problem". The device addresses shown by Device Manager are physical addresses. i.e. they're in the same address space as RAM, and they're at or below 0xFFFFFFFF, i.e. within the first 4 GB of physical address space. That's why the PCI hole exists: RAM that "would be" at those same addresses has to yield to the I/O devices. (Which will be shown by tools like RAMmap and MemInfo.) That's why "memory hole remapping" exists: RAM at addresses that conflict with memory-mapped IO devices gets "remapped" into the address range above 0xFFFFFFFF. (Which will also be shown by tools like RAMmap and MemInfo.) Thereby inaccessible unless your hardware can access RAM at such addresses (x86 can, with PAE), and your operating system can run with PAE enabled (x86 Windows can; later versions even do so by default), and your operating system is willing to look at RAM at addresses above 0xFFFFFFFF (Windows x86 XP SP2 and later "client" editions are not). (All of the claims in this paragraph are well-referenced in the next sections, so I won't repeat the refs here.) Jeh (talk) 18:16, 10 August 2016 (UTC)

32 bit CPUs -> 4GB RAM maximum?

"32bit processors can address 4GB RAM maximum " 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

Incorrect, as clearly stated by text regarding PAE from Windows Internals from which you quoted (and the paragraphs following, which you somehow missed). 32-bit Xeons are 32-bit processors. So, of course, was the Pentium Pro, the first Intel processor that implemented PAE. These are 32-bit processors, and they can address more than 4GB RAM. Therefore, your claim is wrong.

The fact that some 32-bit processors are limited to 4 GB RAM does not mean that that limit applies to all 32-bit processors and their platforms. Jeh (talk) 03:43, 9 August 2016 (UTC)

"32bit chipset, 32bit processor, 4GB RAM limit." 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

Yes, the chipset does get in the act. But so what? 32-bit Xeons and chipsets for them that can address more than 4 GB RAM certainly do exist. They're 32-bit CPUs—that's the width of their GPRs, and of their virtual addresses—but they can address up to 64 GB RAM. So your original claim that 32-but processors are limited to 4 GB RAM is still false. Now, you can say that some 32-bit x86 CPUs + chipsets have a 4 GB RAM address limit. You might even say "most". But you absolutely can't say "all".

By the way, this case is hardly unique: The industry history is full of machines whose "bit width" didn't match their physical address size. HP 1000: 16-bit processor; I don't remember how much RAM it supported, but it was definitely more than 64KB—I think the last ones handled well over 1 MB. (For further complication, even though this was a 16-bit machine, it was word- and not byte-addressable. So without the MMU it only had 32,768 addressable locations, not 64K as one might expect from 16 bits.) PDP-11: 16-bit machine, but with the MMU it implemented 22-bit physical addresses, hence up to 4 MB RAM. VAX: 32-bit machine, 30-bit PAs—but they dedicated half of that range to I/O space so it only supported 512 MB RAM. x64: 64-bit machine (but currently only implements 48 bits of virtual address out of the 64), and physical addresses are limited to 52 bits. There are many, many other examples.

On x86 it is really only coincidence (from the design of the page table entry format) that on x86 without PAE, the PA size matches the VA size and the general purpose register size. There was no absolute requirement that the non-PAE PTE had to supply 20 bits of physical page number, aka page frame number, the same number of bits as the virtual page number. Had Intel originally laid out the PTE differently we might have had 8 or 16 GB—or only 1 or 2 GB!—of RAM addressability from the very first x86 that supported paging.

And as above... any of your arguments that depend on this claim, are thereby also invalid. Jeh (talk) 03:43, 9 August 2016 (UTC)

About PAE and hardware DEP

"PAE is just another level of paging." 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

PAE is a processor mode on x86 that adds a third level (the page directory pointer table) in the page table hierarchy, plus it makes the page table entries 8 bytes wide instead of 4. The wider PTE format makes room for more bits of page frame number, allowing 36-bit physical addresses—i.e. RAM addressing of up to 64 GB. Reference:

"In addition, IA-32 architecture’s paging mechanism includes extensions that support: Physical Address Extensions (PAE) to address physical address space greater than 4 GBytes" (emph. added)

— Intel 64 and ia32 Architectures Software Developer's Manual, Volume 1, page 3-8

PAE also allows hardware Data Execution Protection (no-execute protection on pages). That is, PAE is required for hardware DEP. On x86 if you're not in PAE mode, you can't have hardware DEP. That's because hardware DEP involves an XD bit (Execute-Disabled; NX, No Execute, in AMD terms). The XD bit is bit 63 in each PTE.[15] --Intel 64 and ia32 Architectures Software Developer's Manual], Volume 3A, page 4-17 and -18, Table 4-11 Without PAE the page table entries' bits only go up to bit 31 and there is no XD bit,[16] --Intel 64 and ia32 Architectures Software Developer's Manual, Volume 3A, page 4-12, Table 4-6. i.e. without PAE, you simply don't—can't—have XD bits. This is why Windows started enabling PAE by default on systems that support DEP, even if the installed RAM size didn't require it: They want to enable hardware DEP, and hardware DEP requires PAE mode. Reference:

support for hardware DEP on 32-bit systems requires loading the PAE kernel (%SystemRoot%\System32\Ntkrnlpa.exe), even if that system does not require extended physical addressing (for example, physical addresses greater than 4 GB). The operating system loader automatically loads the PAE kernel on 32-bit systems that support hardware DEP. To force the non-PAE kernel to load on a system that supports hardware DEP, the BCD option nx must be set to AlwaysOff, and the pae option must be set to ForceDisable.

— Russinovich, Solomon, and Ionescu: Windows Internals, 6th Edition, Part 2, page 205.

.154 quoted this:

To understand PAE, it is useful to understand the derivation of the sizes of the various structures and bit fields. Recall that the goal of PAE is to allow addressing of more than 4 GB of RAM. The 4-GB limit for RAM addresses without PAE (emph. added - jeh) comes from the 12-bit byte offset and the 20-bit page frame number fields of physical addresses: 12 + 20 = 32 bits of physical address, and 2^32 bytes = 4 GB. (Note that this is due to a limit of the physical address format and the number of bits allocated for the PFN within a page table entry. The fact that virtual addresses are 32 bits wide on x86, with or without PAE, does not limit the physical address space.)

— Russinovich, Solomon, and Ionescu: Windows Internals, 6th Edition, Part 2, page 261.

I agree with all of that. But then .154 wrote:

"Above clearly states 32bit CPU = 4GB of physical memory addressable by the CPU." 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

No. No, it does not. Quite the opposite. "Above clearly states" that the 4 GB limit for RAM addresses without PAE is 4 GB, but that this limit comes NOT from the "32 bit CPU", but from the PTE format—the fact that the PTE provides 20 bits of PFN. And, I say again, that's without PAE! You ignored completely the last sentence that you quoted. And you also missed the very next paragraph. Here they are:

The fact that virtual addresses are 32 bits wide on x86, with or without PAE, does not limit the physical address space. (emph. added - jeh)

Under PAE, the PFN is expanded to 24 bits. Combined with the 12-bit byte offset, this allows addressing of 2^(24 + 12) bytes, or 64 GB, of memory.

— Russinovich, Solomon, and Ionescu: Windows Internals, 6th Edition, Part 2, page 261.

They couldn't have been more clear: PAE allows 32-bit processors to address more than 4 GB of RAM. 64 GB of RAM, in fact.

And, of course... any of your arguments that depend on your "32 bit CPU, 4 GB RAM limit" claim are therefore voided.

Matter of fact, if no 32 bit CPU could address RAM of address greater than 0xFFFFFFFF, then every 32-bit system, regardless of operating system, would show the "3 GB problem". But, they don't.

Once again: the "32-bit" nature of a CPU means that its virtual addresses are 32 bits wide. But that says nothing about the size of its physical addresses. And it's the size of the physical addresses that determines its physical address space (the range of possible addresses). With PAE, x86's physical addresses are 36 bits wide. That's enough to address 64 GB of RAM. Jeh (talk) 03:43, 9 August 2016 (UTC)

"IA32 is an Intel 32bit processor without PAE capabilty." 115.188.27.154 (talk) 23:12, 14 July 2016 (UTC)

Incorrect. Intel calls all of their x86 processors, with and without PAE capability , "IA32".[17] (Of course, darn few of them don't support PAE these days.) Jeh (talk) 03:43, 9 August 2016 (UTC)

"Prior to SP2 all releases booted with PAE. After SP2 PAE kernel loaded on Server editions only." (from [18] edit summary of 2016-08-08T11:49:52])

Incorrect. As clearly stated in Windows Internals:

Thirty-two-bit systems that have hardware support for nonexecutable memory (described earlier, in the section “No Execute Page Protection”) are booted by default using this PAE kernel, because PAE mode is required to implement the no-execute feature.

— Windows Internals, 6th Edition, book 2, page 260

Similar text is on page 391 of the fourth edition (covering XP), and page 714 of the fifth edition (covering Vista). These versions all support no-execute page protection—hardware DEP. To support that, they have to run with the PAE kernel. Therefore the PAE kernel is not only available, but is loaded by default, on all 32-bit versions, client and server (if DEP is supported).

I believe you are confusing the issue of "the PAE kernel" (which is necessary on x86 CPUs to address RAM above physical address 0xFFFFFFFF) with Windows' wired-in physical address limit. The latter is present even on Windows 32-bit clients running with the PAE kernel, and is not an "architectural" limit, any more than the license-enforced 4 GB capacity limit is. Jeh (talk) 03:43, 9 August 2016 (UTC)

Later add: I have seen the phrase "full support of PAE" here and there. There is no such concept in the Intel or AMD architecture: The PAE-enable bit in CR4 is either off or it's on; if it's on, address translation uses PAE's three-level page table scheme and the OS must set up the page tables accordingly. What this phrase may be referring to is the concept of running with PAE (i.e. with the PAE kernel) but with RAM size and/or addressability artificially limited to less than what PAE can support. This is a valid concept, but I wouldn't call it "incomplete support of PAE". Think of it rather as "running with the PAE kernel, but with quantity and/or address limits on what RAM the OS will use, even though PAE supports more." Or just "...but with artificial RAM limits" for short. Which is, of course, what 32-bit Windows clients do. Even before (as well as after) XP SP2 all 32-bit Windows clients were limited to 4 GB RAM or less, even with PAE enabled. It's just that with XP SP1 and earlier some of that 4 GB RAM could be located at addresses above the 4 GB address limit (0x7FFFFFFF). This may be what you're thinking of when you claim that XP SP2 and later didn't or couldn't boot with the "PAE kernel". Absolutely they do. I can show you how to prove it to yourself, if you want. Jeh (talk) 09:23, 9 August 2016 (UTC)

Effect of OS on RAM limits

" Also the amount of RAM a CPU can access is not related to the OS." 115.188.27.154 (talk) 01:26, 16 July 2016 (UTC)

Correct but irrelevant. It doesn't matter how much the CPU can access if the OS actively refuses to access some of it. x86 Windows XP SP2 and later 32-bit client editions refuse to use any RAM at physical addresses above 0xFFFFFFFF. That's why they show the "3 GB problem" even though other OSs don't show the same problem on the same hardware.

There is an important difference, which I think many people miss here, between a 4 GB RAM size limit—i.e. the maximum supported amount of RAM (which all Windows 32-bit client editions have, if not less) and "4 GB physical addressing limit" (which was added to XP SP2 and later 32-bit client editions of Windows). Look at Figure 10-43 on page 322 of Windows Internals, Sixth Ed., Book 2. The same diagram is at this web page: "Pushing the limits of Windows: Physical memory", right under the heading "32-bit Client Effective Memory Limits". The total RAM illustrated is 4 GB. But see the piece from "4 GB" to "4.5 GB" that's labelled "Inaccessible RAM"? Windows XP SP2 and later client editions refuse to access that RAM, even though the CPU + platform can do it, and even though the total RAM would thereby be not greater than 4 GB. They refuse because that RAM (after memory hole remapping) lies at addresses above 0xFFFFFFFF (in decimal: 4294967295), i.e. the 4 GB address limit. They so refuse because Microsoft found various third-party drivers that were not properly handling physical addresses above 0xFFFFFFFF—in other words, addresses that wouldn't fit in 32 bits. This address limit is the source of the "3 GB problem" on platforms that support both memory hole remapping and physical addresses above 0xFFFFFFFF, i.e. above the 4 GB address limit. Reference:

Although client SKUs support PAE addressing modes on x86 systems in order to provide hardware no-execute protection (which would also enable access to more than 4 GB of physical memory), [...] the problematic client driver ecosystem led to the decision for client editions to ignore physical memory that resides above 4 GB, even though they can theoretically address it. (emph. added)

— Russinovich, Solomon, and Ionescu: Windows Internals, 6th Edition, Part 2, section "Windows Client Memory Limits", page 321

Do you see? "Even though they can theoretically address it". If it was an "architectural limit" then they couldn't theoretically address it. It is not an "architectural limit" any more than are the license-imposed RAM or CPU count limits. Jeh (talk) 03:43, 9 August 2016 (UTC)

"The OS has no relevance to how much RAM the CPU can handle" 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC)

That is true, but the OS can most certainly choose to not access RAM above a certain amount... or above a certain address... even though the CPU could "handle" more. Which is what Windows does. That is, 32-bit Windows XP SP2 and later client versions choose to not access RAM at addresses above 0xFFFFFFFF, nor RAM of a quantity in excess of 4 GB. Jeh (talk) 07:20, 9 August 2016 (UTC)

Other references - isn't it really just hardware?

"Jeff Atwood (author of The ASP.NET 2.0 Anthology and Effective Programming: More Than Writing Code) wrote this:"

To be perfectly clear, this isn't a Windows problem -- it's an x86 hardware problem.

— [http://blog.codinghorror.com/dude-wheres-my-4-gigabytes-of-ram/

While he certainly has expertise in e.g. ASP.NET, I am unaware of anything Jeff Atwood has written that shows him to be a RS in the area of Windows memory management. Certainly he made a number of fumbles on that page:

Addressing more than 4 GB of memory is possible in a 32-bit operating system, but it takes nasty hardware hacks like 36-bit PAE extensions in the CPU, together with nasty software hacks like the AWE API.

— [http://blog.codinghorror.com/dude-wheres-my-4-gigabytes-of-ram/

PAE is not a "nasty hardware hack", it's just another level of page table hierarchy with wider page table entries. (And if he considers that a "nasty hardware hack" then he must hate x64, which adds yet another level of page table hierarchy.) Whether it's a "nasty hack" or not, x86 Windows has supported PAE since Windows Server 2000 (up to 32GB RAM on x86, e.g. on the Unisys ES7000, which also supported up to 32 Xeons) and even the Windows client versions have been running the PAE kernel by default ever since XP started supporting hardware DEP. As I documented previously (from the Intel architecture book as well as Windows Internals) you can't have hardware DEP without running in PAE mode. And Windows does enable hardware DEP by default (again, see previous references). Yeah, it's such a "nasty hardware hack" that Windows enables it by default.

As for the AWE API, again, no. For the system as a whole to take advantage of more than 4 GB RAM does not require any use of the AWE API whatsoever.

The AWE API is something like the EMS mechanism on DOS. It allows an app to create a "window" within its normal v.a.s. (by using VirtualAlloc with the MEM_PHYSICAL option) and then map that window (with MapUserPhysicalPages) into various allocations of physical memory. The app creates those allocations with the AllocateUserPhysicalPages API. (This API does not allow the app to specify which physical pages it wants to access, and the OS will only hand it physical pages that are not currently in use and have been cleared of previous content, so there is no security hole here.) If there's a lot of free RAM on the machine then the app can create enough of these allocations that the total RAM allocated this way could exceed 2, 3, or even 4 GB. And it is true that this is the only way an app on x86 Windows could use more than 3 GB quantity of RAM.

But that has nothing to do with whether or not the OS can assign, in response to ordinary paging, RAM to the process that happens to lie above the 4 GB address boundary. Or for the system as a whole to use such RAM. Or for the system as a whole to use more than 4 GB RAM total. You absolutely do not need AWE for any of that. Reference: Windows Internals, 6th ed., Book 2, page 211. See also the documentation of the Windows APIs mentioned.

Heck, I don't even know how an app could learn of a physical address for a given virtual address, not without e.g. a pseudodriver to help out. There's no standard API for that. Even if there were, the information could well be obsolete before it was actually returned to the app that requested it.

By the way, AWE also does not affect the v.a.s. available to the app! The 32-bit app is still limited to 32-bit virtual addresses and (on x86) to the usual total of 2 or 3 GB maximum v.a.s. (4 GB if running on x64.) Yes, with these APIs the original VirtualAlloc'd region can be used to access a total of much more... but not all of the total at once. There are interesting restrictions on what you can do in an AWE area. You can use it as the source or destination for I/O operations. But it's nonpageable, so you can't map it to a file... which also means you can't run code out of it. Jeh (talk) 09:33, 11 August 2016 (UTC)

Unless the application is specifically coded to be take advantage of these hacks, it's confined to 4 GB. Well, actually, it's stuck with even less-- 2 GB or 3 GB of virtual address space, at least on Windows.

— [http://blog.codinghorror.com/dude-wheres-my-4-gigabytes-of-ram/

Well, yes, unless an app is specifically coded to use the AWE API, it's confined to 2 or 3 GB of virtual address space. (And with the AWE API, it still is.) But that says nothing about what addresses of RAM might be "underneath" that v.a.s. On an x86 Windows system running with PAE, the physical memory used by each process and by the OS can be scattered anywhere within the total RAM that is on the machine and recognized by Windows - including RAM above the 4 GB address boundary, if you're running a Windows version that lets itself use that. Remember, with PAE, the 4 GB boundary isn't really a boundary... except that x86 Windows clients, XP SP2 and later, decided to make it one even though they run with PAE.

In any case, a Windows app uses only virtual addresses; it never sees or cares about physical addresses at all, couldn't obtain a physical address from the OS if it wanted to, never provides any storage space for them... so why would the app need to change? It doesn't. It doesn't need to use the AWE API, and no other changes are necessary either. The awareness of wider-than-32-bit physical addresses is only in operating system data structures like page table entries, working set lists, the PFN array, etc. These are managed by the OS and the app has no way of looking them. Jeh (talk) 09:33, 11 August 2016 (UTC)

"Microsoft says... "

Address space used for hardware and ROMs causes that much memory to get shoved upwards, and it ends up above the 4GB boundary. Without PAE (emph. added - jeh), the processor is capable only of addressing memory below the 4GB boundary, which means that the memory above that boundary is inaccessible

— Raymond Chen's blog

Sure, but (I tell you three times) as he says right there in the text, that's without PAE! And (as I've previously documented), x86 Windows is perfectly capable of running with PAE, and of using it to access RAM above that boundary, as far back as Windows 2000 Server! (As the person who asked the question at the top of that page said. In fact the example thereby provided proves that Windows is running with PAE, since it can see all of the customer's 4 GB RAM. The OS must be an x86 server, or XP SP1 or earlier client. It can't be x64 because on x64 the PAE boot option would have no effect at all; there is just one address translation mode on x64. Called ia32e paging by Intel, ) Jeh (talk) 09:33, 11 August 2016 (UTC)

"Sun Microsystems says"

The PCI hole exists below 4GB to ensure that all 32-bit software can reach those addresses. The physical memory hidden by this hole is not usable by the operating system software and therefore is “missing” from the system. The size of the PCI hole is the total amount of PCI/AGP address space consumed for all devices as configured by the BIOS.

— Memory Hole in Large Memory x86 Systems XES by Sun Microsystems

This claim completely ignores memory hole remapping, which we know happens. In fact, the "Conclusion" of that paper states "Work is being done by the BIOS and/or chip manufacturers that will either remap physical memory or move device address space in order to eliminate the hole." Guess what, it's already been done. We already know that a platform that doesn't implement memory hole remapping will always show the 3 GB problem, regardless of OS; the question is, what happens on platforms that do implement it? This paper says nothing about that case, was apparently written before memory hole remapping even existed (or perhaps the author hadn't heard of it yet), so it cannot be said to address that question at all.

By the way, the rest of the paper doesn't ignore PAE, but contains some of the same old errors, like the claim that "all software must explicitly program for and use the PAE extensions. This will affect applications, operating systems, hardware device drivers and hardware itself. " Well, maybe in Sun's OS that is true. I wouldn't know. But in Windows, applications are no more aware of PAE than they are of physical addresses in general. Jeh (talk) 09:33, 11 August 2016 (UTC)

Reviewing the 3 GB problem in light of the above

"The so called 3GB limit exists only when hardware does not support over 4GB of RAM. " 115.188.27.154 (talk) 23:12, 14 July 2016 (UTC)

Incorrect. Not to say it doesn't exist on such hardware; of course it does. On a platform (processor + chipset) that can't address RAM above address 0xFFFFFFFF, memory hole remapping with 4 GB RAM installed is impossible; there are no addresses to move the RAM in the "memory hole" to. So you just end up with the memory hole, and you have the 3 GB problem regardless of what the OS does; as far as the OS is concerned the RAM that conflicts with the I/O devices is just invisible. And in that case the "3 GB problem" is not at all due to anything the OS does.

But it does not only exist on such hardware! On platforms that do support memory hole remapping as well as more than 4 GB RAM, Windows 32-bit client editions still show the "3 GB problem". Because the BIOS moves the RAM that conflicts with the I/O devices to addresses above 0xFFFFFFFF, but then Windows refuses to look at that RAM. Reference:

[...] the problematic client driver ecosystem led to the decision for client editions to ignore physical memory that resides above 4 GB, even though they can theoretically address it.

— Russinovich, Solomon, and Ionescu: Windows Internals, 6th Edition, Part 2, section "Windows Client Memory Limits", page 321

So, because of 32-bit Windows clients' decision to not look for RAM at addresses above 0xFFFFFFFF, you've "lost" the RAM that conflicts with I/O device space, even though the platform remapped it.

You see, the ability to use RAM above address 0xFFFFFFFF is an "and" condition: The processor has to be able to do it (with PAE, it can); AND the chipset has to be able to do it (and some chipsets for 32-bit CPUs can); AND the operating system has to be willing to do it. Windows client editions, XP SP2 and later, are not willing. So it doesn't matter what the platform can do.

btw, there's another category of system where the 3 GB problem occurs: x64 processors that are running with a 32-bit OS—like XP—and with 4 GB RAM. This was actually quite a common setup in the latter years of XP. People wanted a newer processor, and the better ones were the x64 processors. But a LOT of people ran 32-bit XP on them (and no, I don't mean XP for x64, which was really Server 2003 SP1 and had very limited driver support). These machines were capable of addressing RAM above address 0xFFFFFFFF, and they could do memory hole remapping. Many other OSs could see all 4 GB of RAM (thanks to the above two abilities). But XP SP2 refused to look at addresses above that point, so memory hole remapping didn't help. So—"3 GB problem", and in the end, it was because of XP SP2's RAM address limit. (Not the same as "RAM capacity limit". Remember, the problem occurs even if total RAM is just 4 GB.)

So it definitely is a Windows issue. This is why "install a 64 GB OS" was always a suggested fix, even though a 32 vs 64 bit processor was never the issue. The issue was that all 64-bit Windows editions allow themselves to look at RAM addresses above 0xFFFFFFFF, while XP SP2 and later Windows 32-bit client editions did not, even though they do use the PAE kernel by default. Jeh (talk) 03:43, 9 August 2016 (UTC)

Within this box is discussion of claims that are either true but irrelevant, or else not true but irrelevant, to the cause and cure of the 3 GB problem. Nevertheless, all have been answered.

True, or mostly true... but irrelevant "Windows uses virtual addresses translated to physical addresses by the CPU TLB." 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC) Well, the "TLB" is just a cache of page table entries. The CPU could omit the TLB and things would still work, albeit slowly (since every address translation would then have to read all levels of page tables). Like the TLB, the PTs just contain data; they don't actually "do" the translation. The CPU component that actually does the translation is the MMU. Jeh (talk) 03:43, 9 August 2016 (UTC) .154 quoted this: Windows implements a virtual memory system based on a flat (linear) address space that provides each process with the illusion of having its own large, private address space. Virtual memory provides a logical view of memory that might not correspond to its physical layout. At run time, the memory manager, with assistance from hardware, translates, or maps, the virtual addresses into physical addresses,where the data is actually stored. By controlling the protection and mapping,the operating system can ensure that individual processes don’t bump into one another or over write operating system data. — Windows Internals, 6th Edition, page 15 ...all correct, but none of this is either in dispute or relevant to the issue. Jeh (talk) 03:43, 9 August 2016 (UTC) "Also note virtual addressing in Windows is in no way related to the physical RAM available for the processor." 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC) Again, correct. MarkR even said so; you quoted him (and I quoted your quote above). But again, this is neither in dispute nor relevant to the 3 GB problem. Jeh (talk) 03:43, 9 August 2016 (UTC) "Also, 20 processes could run simultaneously on Windows XP, each using 4GB address space but as a whole the system would eat 8GB of RAM. If that sounds like fiction, it's not. This is how OS address translation and virtualization works..... " 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC) "20 processes"? Piker! Heck, I have a screen snap from Windows NT 4.0 that shows about 20 processes immediately after login. (These days Windows creates more than that before you ever log in.) Remembering what systems I had when I was running NT 4.0... it couldn't have had more than 64 MB (yes, megabytes) of RAM. So, yeah, I know about this. But... sorry but yes, your specific example is fiction, because XP will never use more than 4 GB of RAM—unless you're talking about XP 64-bit edition (which was really Server 2003 SP1 with an XP skin). Jeh (talk) 03:43, 9 August 2016 (UTC) Compilers, processes, and addressing: Wrong and irrelevant "ALL 32bit processes are capable of 4GB virtual addressing, ALL. " 115.188.27.154 (talk) 23:12, 14 July 2016 (UTC) True, but irrelevant here. This claim is about virtual addressing, and the 3 GB problem is all about physical address space. Jeh (talk) 03:43, 9 August 2016 (UTC) "It's the Intel x86 compilers which limit process address space, not the OS." 115.188.27.154 (talk) 23:12, 14 July 2016 (UTC) Incorrect. The x86 compilers are perfectly happy to produce code that references virtual addresses all the way to 0xFFFFFFFF—if they weren't, then how could we use them to build e.g. kernel mode drivers? It's x86 Windows, not the compiler, that says that addresses above 0x7FFFFFFF are inaccessible from user mode. (Or up to 0xBFFFFFFF if you're running with /3GB with an LAA executable. Or up to 0xFFFFFFFF if you're running an x86 LAA exe on x64 Windows.) Jeh (talk) 03:43, 9 August 2016 (UTC) "32bit apps complied for AMD64 have pointers allowing addressing over 3GB. " 115.188.27.154 (talk) 23:12, 14 July 2016 (UTC) Also incorrect. There is no such concept or option as 32 bit apps "compiled for AMD64" or "compiled for Intel64" or "compiled for x86". You don't compile the apps any differently. You compile them for 32-bit ("x86" in the Visual Studio settings) or for 64-bit ("x64"). And you either use the "large address aware" flag, or not. (If you disagree, please tell me what compiler switch you're thinking of?) The "large address aware" option does not change the code the compiler outputs by one iota; it just sets a bit in the exe or dll header that tells the OS "it's ok to let the process running this code see virtual addresses above 0x7FFFFFFF". The OS is what implements the limit. But the mistake here is worse than that. Very little code ever uses addresses that are determined at compile time! If your exe and all the DLLs you use are linked "large address aware", then in a /3GB environment on an x86 processor + OS—or in a 32 bit app running on x64 Windows—then you are likely to use virtual addresses above 0x7FFFFFFF simply because those addresses are available in user space in your process... and they are therefore among the addresses the OS can hand to you when you allocate memory. Say, with malloc or VirtualAlloc or MapViewOfFile. You see, it's the OS, not the compiler and linker, that are responsible for managing the v.a.s. of the process and figuring out where the exe, each DLL, each thread's stack, etc., should go. Without the "large address aware" flag on your exe the OS just never puts any of your code or data at an address above 0x7FFFFFFF. It's that simple. Jeh (talk) 03:43, 9 August 2016 (UTC) "x86 compiled apps have truncated pointers which above 3.2GB which point to null." 115.188.27.154 (talk) 23:12, 14 July 2016 (UTC) No, they don't. If that was true then we could never use those same compilers for building kernel mode code, like drivers, for x86. Or for building x86 programs to run in the 4 GB user mode address space that's available under Windows on x64. But, we do. Jeh (talk) 03:43, 9 August 2016 (UTC) " It depends on the process." 115.188.27.154 (talk) 23:12, 14 July 2016 (UTC) Actually it depends on the presence or absence of the LAA flag in the exe and the DLLs. (All DLLs, along with the exe, have to be LAA for it to work.) Jeh (talk) 03:43, 9 August 2016 (UTC) " Every single process in Windows has a virtual address starting with 0. That's how virtualization works." 115.188.27.154 (talk) 01:26, 16 July 2016 (UTC) I think you meant "that's how virtual memory works"; the term "virtualization" means something else these days. That aside, be careful about generalizing. That's not how virtual memory works on every OS. There are virtual memory operating systems where every process doesn't start at the same address, where they all share the same address space. But on Windows your statement here is true... ...well, almost. Although the process-space address range on Windows does start at 0, Windows never actually lets any process use virtual addresses 0 through 0xFFFF. (Except DOS applications running on 32-bit Windows.) So... in terms of the virtual addresses Windows will let you actually use, process space starts at 0x10000, not 0! Anyway, all of the above is about virtual addresses. And the "3 GB problem" is about physical addresses. So the above is irrelevant to the issue. Jeh (talk) 03:43, 9 August 2016 (UTC) Direct access to RAM " The CPU and kernel is the only part of the system which has direct access to RAM." 115.188.27.154 (talk) 12:51, 16 July 2016 (UTC) Actually, not even the kernel has "direct access to RAM." Every address referenced by Windows' kernel mode code (as well as by user mode code, of course) is a virtual address (or as the Intel and AMD docs call it, a "linear address") and is translated to a physical address. Once paging is enabled (bit 31 in CR0), the MMU translates all addresses. no exceptions. Reference: Note Even kernel-mode code (such as device drivers) cannot reference physical memory addresses directly, but it may do so indirectly by first creating virtual addresses mapped to them. — Russinovich, Solomon, and Ionescu: Windows Internals, 6th Edition, Part 2, page 252. There is no way to tell the MMU that addresses asserted by Windows' kernel mode code shouldn't get translated, i.e. should be interpreted as "direct access" to RAM. Jeh (talk) 07:14, 9 August 2016 (UTC) "The OS kernel accesses RAM through real memory addresses starting with 0x1." 115.188.27.154 (talk) 01:26, 16 July 2016 (UTC) I'm not sure what you mean by "real memory addresses", but you're mistaken either way. If you mean virtual addresses, v.a.'s 0 through 0xFFFF are never assigned to anything in Windows (unless you're in a 16-bit app on 32-bit Windows... haven't even thought about that case for years), so "starting with 0x1" simply wouldn't be done. If you mean physical addresses, then that can't be right either, because not even kernel mode can reference physical addresses once the OS boots. So, no: there are no "real memory addresses starting at 0x1" that can be accessed by the OS kernel to access RAM. If you want to know how the kernel accesses specific physical pages, look up "Hyperspace" in Windows Internals. Windows dynamically creates virtual addresses whose corresponding page table entries contain the PFNs of the physical pages to be accessed. The resulting virtual addresses are always in kernel space. In the past, Windows has used a scheme of mapping all of RAM into a region of kernel space. NT4 did this, if RAM was under 512 MB, using "large pages". This would of course be mapped into the kernel-space virtual address range (0x80000000 and above). However it still wasn't using "real memory addresses" (that is, the virtual addresses so created were not the same as the physical addresses they were mapped to). And Windows hasn't done that for a long time. (It still, of course, uses "large pages" for other things.) Nor will any such scheme ever start at 0x1. Ever. Any virtual-to-physical mapping has to be page-aligned. Besides, 0x1 would be in user mode virtual address space. Yeah, let's open up access to RAM to user mode... sure, that's a great idea. Jeh (talk) 03:43, 9 August 2016 (UTC) "and the CPU is the only part of the operating system which can access RAM." 115.188.27.154 (talk) 15:32, 16 July 2016 (UTC) What did you really mean here? The CPU is not part of the operating system at all! (Also just FYI: other things on the platform besides the CPU can access RAM.) Jeh (talk) 03:43, 9 August 2016 (UTC) Completely off topic, but needs to be corrected "AMD x64/IA64 platforms do not have the 3GB usermode restriction because the CPU can address as much ram as it likes (for arguments sake). Any x64 CPU can count from 0-32GB if the machine has 32GB installed. This means usermode process running in Windows 32 have 4GB of address space available because the x64 CPU can map the process address space anywhere into RAM. On x86 platforms this is impossible, because with the exception of Xeon the machines are not physically capable of addressing above 4GB of RAM. 2GB kernel slice leaves only 2GB left for usermode apps.. " 115.188.27.154 (talk) 16:49, 16 July 2016 (UTC) Sorry, no: That's not how process address space is implemented; that's not why the 64-bit systems don't have the 3 GB user mode restriction; and that's not relevant to the "3 GB problem" anyway. (The quotes you provided don't back up your claims here at all, so I didn't copy them.) It's not relevant because the "3 GB usermode restriction" you're talking about is about virtual addressing, but the "3 GB problem" is about physical addresses. Note that the "3 GB usermode restriction" is a constant limit; it isn't 3.2 GB on one system, 2.8 on another, etc. It's at exactly 0xBFFFFFFF. The real reason 64-bit Windows OSs can let a 32-bit exe with large-address-aware use 4 GB of v.a.s. is that their kernel mode components don't have to fit, along with the user mode code and data, in a total of 4 GiB of virtual address space (as they do on x86). The kernel mode v.a.s. on 64 bit Windows is up at 0xFFFF8000`00000000 through 0xFFFFFFFF`FFFFFFFF (even when a 32-bit user mode program is executing). So the 32-bit user mode code and data has from 0x1000 through 0xFFFFFFFF virtual. This has utterly nothing to do with how much RAM you have, or can have. And that's not how process address space is implemented. The entire address space of a process does not have to be mapped into RAM at once. And whatever portion is mapped into RAM, does not have to occupy physically contiguous RAM or physically contiguous address space. The relationship you think exists is simply not there. Note that x64 systems don't have the "3 GB user mode restriction" even on platforms with only, say, 2 GiB of RAM. Consider that for a very long time we've been running 32-bit Windows, with many 2 GB processes, even on machines with much smaller amounts of RAM, like 512 MB or even 64 MB. Heck, the prerelease version of Windows NT 3.1 would boot in 8 MB! (But you needed 12 if you wanted to use the SDK and DDK, and 16 was recommended.) But anyway, this has absolutely, positively, NOTHING to do with the "3 GB problem", nor with the ability to address above 4 GB of RAM. Jeh (talk) 03:43, 9 August 2016 (UTC) "But, I've seen it" "No, but what I have seen is XP using 46GB of RAM on a database server with Opteron installed. 32bit OS use 64GB of RAM...amazing I know. " 115.188.27.154 (talk) 01:26, 16 July 2016 (UTC) Well, a 32 bit OS can certainly use 64 GB of RAM. x86 Server 2003 Datacenter edition, for example. But no unpatched version of 32-bit Windows XP would ever use more than 4 GB (quite aside from the 0xFFFFFFFF physical address limit in XP SP2 and later). It might report that the 64 GB is present, but it won't actually use it as RAM for the OS. So, sorry, but I think something must have been... different from what you describe. I strongly suspect it was XP x64 edition, i.e. not a 32-bit OS. XP x64 edition. was actually Server 2003 x64 SP1 with an XP skin. It would use up to 128 GB RAM and looked pretty much exactly like standard 32-bit XP, unless you looked in the right places. It's also possible that that machine was running a hacked version of 32-bit XP that let it bypass both the address limit and the RAM size limit. (Fujitsu-Seimens did have an OEM source license in the 2000-XP time frame, so they could have built it that way, no binary patching needed.) Jeh (talk) 03:43, 9 August 2016 (UTC)

In conclusion

The "3 GB problem" will happen with Windows XP SP2 and later 32-bit client editions, even on platforms that do memory hole remapping and support > 4 GB RAM. The reason, on such systems, is not "architectural"; it was introduced with a very minor change to XP SP2, which forbade it from using RAM at addresses above 0xFFFFFFFF.

And 32-bit processors and their platforms are not necessarily limited to 4 GB RAM. (Some are, but some are not.)

And hex numbers are used for physical addresses.

And 64-bit processors with 32-bit Windows also show the "3 GB problem", even though they do memory hole remapping and support RAM addresses above 0xFFFFFFFF, again because of 32-bit Windows' physical addressing limit.

Most of the objections to the above depend on at least one of the following claims, which are not supported by references, while their refutations (shown in parentheses) are:

• The 3 GB problem is due to 32-bit CPUs' inability to use more than 4 GB RAM. (But with PAE, they can do that.)
• PAE isn't an answer because to support RAM addresses above 4 GB requires all applications to be recoded ... (It doesn't)
• ... or to use AWE. (It doesn't.)
• But Windows doesn't run with PAE enabled. (It does.)
• The 3 GB problem only exists on platforms that don't do memory hole remapping or can only support up to 4 GB RAM. (It does exist on Windows x86 client editions, XP SP2 and later, even if they do support memory hole remapping and physical addresses above 0xFFFFFFFF, because these Windows editions refuse to address RAM at addresses above 0xFFFFFFFF, even though they can "see" it). Jeh (talk) 03:43, 9 August 2016 (UTC)