Bug #1687

Kernel only detection 2 GB of memory on AMD

Added by vince.dragonfly about 4 years ago. Updated about 4 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:-
Target version:-

Description

Hi.

Just wondering if there are there any known issues with detecting the
memory. We have an AMD Opteron dual processor machine, and Dragonfly
only sees 2 GB with 4 GB in the machine. This is Dragonfly
v2.5.1.901.g48d80-DEVELOPMENT. We tried with both a regular default
kernel that installed from a snapshot ISO image and an SMP enabled
kernel that I compiled.

It also only sees 2GB on Dragonfly 2.4.1-release.

We tested with 4GB and 5GB of RAM.
The BIOS sees the full amount of memory.

The motherboard is a Iwill DK8S2-SATA with two single core AMD
Opteron(tm) processors.

dmesg shows

DragonFly v2.5.1.901.g48d80-DEVELOPMENT #0: Thu Feb 25 00:16:40 CST 2010
:/usr/obj/u1/dragonfly/dfsrc/src/sys/GENERIC
TSC clock: 2004881030 Hz, i8254 clock: 1193382 Hz
CPU: AMD Opteron(tm) Processor 246 (2004.56-MHz 686-class CPU)
Origin = "AuthenticAMD" Id = 0xf5a Stepping = 10
Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
AMD Features=0xe0500800<SYSCALL,NX,MMX+,LM,3DNow!+,3DNow!>
real memory = 2147418112 (2047 MB)
avail memory = 2065563648 (1969 MB)
...
DragonFly/MP: Multiprocessor motherboard
cpu0 (BSP): apic id: 0, version: 0x00040010
cpu1 (AP): apic id: 1, version: 0x00040010
...

Top shows
Memory: 6020K Active, 4384K Inact, 89M Wired, 44M Buf, 1898M Free

sysctl also shows 2GB.

History

#1 Updated by dillon about 4 years ago

Hmm. Do a verbose boot and post the dmesg output. Lets see what
memory list the kernel gets from the BIOS.

-Matt

#2 Updated by vince.dragonfly about 4 years ago

On Mon, Mar 01, 2010 at 08:50:47PM -0800, Matthew Dillon wrote:
> Hmm. Do a verbose boot and post the dmesg output. Lets see what
> memory list the kernel gets from the BIOS.
>
> -Matt

OK. Here are the parts in /var/log/messages that seem to have anything
to do with memory. I assume you didn't want the whole output (it's
long) but I could provide it if you need.

kernel: CPU: AMD Opteron(tm) Processor 246 (2004.56-MHz 686-class CPU)
kernel: Origin = "AuthenticAMD" Id = 0xf5a Stepping = 10
kernel:
Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
kernel: AMD Features=0xe0500800<SYSCALL,NX,MMX+,LM,3DNow!+,3DNow!>
kernel: Data TLB: 32 entries, fully associative
kernel: Instruction TLB: 32 entries, fully associative
kernel: L1 data cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative
kernel: L1 instruction cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative
kernel: L2 internal cache: 1024 kbytes, 64 bytes/line, 1 lines/tag, 16-way associative
kernel: real memory = 2147418112 (2047 MB)
kernel: Physical memory chunk(s):
kernel: 0x00001000 - 0x0009efff, 647168 bytes (158 pages)
kernel: 0x008da000 - 0x7ffdffff, 2138071040 bytes (521990 pages)
kernel: avail memory = 2065563648 (1969 MB)
kernel: ACPI MADT: RSDP not in EBDA
kernel: ACPI MADT: RSDP in BIOS mem
kernel: ACPI MADT: MADT in XSDT
kernel: ACPI MADT: LAPIC address 0xfee00000, flags 0x1
...

Also,

New information. We tested various memory configurations.
Dragonfly correctly detected 4 GB of memory with only one of them.

Total RAM
---------
5G 4 x 1G in each slot on one cpu and 4 x 256M in each slot on the
other cpu.
bios reported 5G, dragonfly reported 4G (correct: 32 bit kernel)

* This is the only combination where DF sees 4 GB *

4G 4 x 1G all slots on one cpu
bios reported 4G, dragonfly reported 2G

This is weird. Same configuration as above but remove the 256M
RAM's from the slots for the other cpu and it no longer
detects all the RAM on the first cpu.

4G 1G in slots 0 and 2 on each cpu
bios reported 4G, dragonfly reported 3G

4G 2 x 1G in slots 0 and 1 on each cpu.
bios reported 4G, dragonfly reported 2G

5G 2 x 1G slots 0 and 1 + 2 x 256M in slots 2 and 3 on each cpu
bios reported 5G, dragonfly reported 2G

#3 Updated by vince.dragonfly about 4 years ago

One more thing. There is a brief message we didn't catch before right
at the start, when booting, that says X amount of memory above 4 GB is
ignored. I don't know how to capture it. It doesn't go into the
syslog. That seems to indicate that Dragonfly is apparently seeing all
the memory initially.

- Vince

#4 Updated by TGEN about 4 years ago

Vincent Stemen wrote:
> One more thing. There is a brief message we didn't catch before right
> at the start, when booting, that says X amount of memory above 4 GB is
> ignored. I don't know how to capture it. It doesn't go into the
> syslog. That seems to indicate that Dragonfly is apparently seeing all
> the memory initially.

Looks like your BIOS is remapping memory above 4Gibyte in some cases
automatically, to make room for PCI memio mappings etc. If you want to
make use of the additional memory, you have to run pc64.
--
Thomas E. Spanjaard

#5 Updated by dillon about 4 years ago

I think the only way the kernel will be able to see all the memory
on that box is if you run a 64-bit DFly kernel.

-Matt
Matthew Dillon
<>

#6 Updated by vince.dragonfly about 4 years ago

On Tue, Mar 02, 2010 at 10:57:58PM -0800, Matthew Dillon wrote:
> I think the only way the kernel will be able to see all the memory
> on that box is if you run a 64-bit DFly kernel.

So it is normal to only be able to use 2GB of memory in 32-bit mode??

- Vince

#7 Updated by TGEN about 4 years ago

Vincent Stemen wrote:
> On Tue, Mar 02, 2010 at 10:57:58PM -0800, Matthew Dillon wrote:
>> I think the only way the kernel will be able to see all the memory
>> on that box is if you run a 64-bit DFly kernel.
>
> So it is normal to only be able to use 2GB of memory in 32-bit mode??

It is if the BIOS only maps 2Gibyte worth of RAM below the 4Gibyte mark.
Most BIOSes allow you to disable memory remapping entirely though, in
which case you can use as much memory as is left when subtracting all
other physical address space consumers (most importantly memio).

Cheers,
--
Thomas E. Spanjaard

#8 Updated by vince.dragonfly about 4 years ago

On Wed, Mar 03, 2010 at 07:20:28PM +0000, Thomas E. Spanjaard wrote:
> Vincent Stemen wrote:
> > On Tue, Mar 02, 2010 at 10:57:58PM -0800, Matthew Dillon wrote:
> >> I think the only way the kernel will be able to see all the memory
> >> on that box is if you run a 64-bit DFly kernel.
> >
> > So it is normal to only be able to use 2GB of memory in 32-bit mode??
>
> It is if the BIOS only maps 2Gibyte worth of RAM below the 4Gibyte mark.
> Most BIOSes allow you to disable memory remapping entirely though, in
> which case you can use as much memory as is left when subtracting all
> other physical address space consumers (most importantly memio).

BTW, I've been meaning to follow up on this and say thanks for the
information. You were right. I didn't realize that some of these
server boards mapped the memory so that only 2 or 3 GB is visible below
the 4GB mark.

We tested FreeBSD, just to confirm, and it had the same problem.
I could not find any option to disable memory remapping in the BIOS
though. Apparently, these boards are pretty much meant to only be run
with 64bit kernels.

It does seem to work fine with a 64bit Dragonfly kernel :-).

Regards,
Vince

Also available in: Atom PDF