DragonFlyBSD

:Hi guys,
:
:I installed the 1.8.0 release from the liveCD and it screwed up my
:partition table. How can I set the drive geometry so that the installer
:will use my c/h/s values instead of the (possibly incorrect) values said
:to be obtained from BIOS?
:
:I noticed this problem in FreeBSD but it seems to have been mitigated by
:the installer offering a "G" option to set drive geometry around 6.1/6.2
:release, not sure exactly when.
:
:I posted a question on the users list on Feb. 26 but hearing nothing
:(and wanting to install Dfly 1.8.0 release) I am posting here on bugs.
:Someone with a multiboot setup who doesn't track changes to his
:partition table can suffer loss of data because of this error. It could
:be a relatively serious problem, especially for newbies to multibooting
:as symptoms probably won't appear immediately, it will be difficult to
:diagnose.
:
:Thanks,
:Rand

Hmm. usually that sort of problem is due to the disk mode in the
    BIOS setup not being set to Large or logical block mode.  Nobody
    has used CHS in a long time, and BIOSes still get confused by faked
    CHS numbers.

-Matt
                    Matthew Dillon 
                    <dillon@backplane.com>

Hi Matt,

There needs to be an option such as FreeBSD (and NetBSD and iirc
OpenBSD) are providing, to set c/h/s for use by the DFly installer. I
don't understand the comment that nobody is using it- all the *BSD show
their view of it and if it doesn't match the partition table (which is
laid out to this day in terms of c/h/s whether it's actual or virtual)
it causes problems.

For example, my machines are set up with 255 heads 63 sectors and how
ever many cylinders the drive maps to. I fdisk the drive and I create
my partitions aligned on cylinder boundaries. When DFly installs it uses
16 heads and some other factor for sectors. He changed the units on the
partition table and this means that the partitions I had created before
and after (locations, not times) the DFly target slice are no longer on
integral boundaries, and that writing on the DFly filesystems can leak
into my other partitions and corrupt data.

There should be a simple way to set the c/h/s in the installer.
Otherwise it does not play nice in a multiboot scenario.

Cheers,
Rand

:Hi Matt,
:
:There needs to be an option such as FreeBSD (and NetBSD and iirc
:OpenBSD) are providing, to set c/h/s for use by the DFly installer. I
:don't understand the comment that nobody is using it- all the *BSD show
:their view of it and if it doesn't match the partition table (which is
:laid out to this day in terms of c/h/s whether it's actual or virtual)
:it causes problems.
:
:For example, my machines are set up with 255 heads 63 sectors and how
:ever many cylinders the drive maps to. I fdisk the drive and I create
:my partitions aligned on cylinder boundaries. When DFly installs it uses
:16 heads and some other factor for sectors. He changed the units on the
:partition table and this means that the partitions I had created before
:and after (locations, not times) the DFly target slice are no longer on
:integral boundaries, and that writing on the DFly filesystems can leak
:into my other partitions and corrupt data.
:
:There should be a simple way to set the c/h/s in the installer.
:Otherwise it does not play nice in a multiboot scenario.
:
:Cheers,
:Rand

Well, my understanding is that tha partition table (the DOS parition
    table) stores linear block values in addition to CHS values.  If you
    tell the BIOS that the disk is in LBS or LARGE mode, the BIOS should 
    use the linear block values.  CHS values are undependable no matter what
    you do.  There is simply no proper way to set them that works with
    all BIOSes.

-Matt
                    Matthew Dillon 
                    <dillon@backplane.com>

I'm curious how old your machine is, and how your partition table was
created in the first place.

This is a good explanation of why modern machines 'don't use' CHS:
http://www.dewassoc.com/kbase/hard_drives/lba.htm

I don't doubt that your partition table got screwed up, but based on
many years of installing/multibooting OS's of many flavors, I'm a
bit skeptical about the CHS thing being responsible for it. Perhaps
there is a different problem which needs attention.

How did you recover from this problem, BTW? Did you edit the bad
partition table by hand, or restore a backup copy, or.....?

I have two machines booting about 12 OS between the two of them. This
one is a pIII server box on a Dell motherboard. So far I run OpenBSD,
NetBSD, FreeBSD, and four Slackwares on this box and I'm hoping to add
DFly and Solaris, I haven't been able to install DFly on my newer
(2.2GHz P4) because it (and 1.6.0) fails in init and we weren't able to
get the issue straightened out.

I created the partition table using Linux fdisk, then I preallocate all
the partitions according to which OS I plan to install. I leave three
primaries for *BSD and Solaris flavours and then I allocate extended
partitions as necessary for my Linux systems.

It's tangential that modern machines 'don't use' c/h/s; the issue is
that the partition table is designed around c/h/s values and that's how
space gets allocated. When you use fdisk you can certainly allocate
things in terms of tracks or cyls or megs or what have you, but the
partition table contains c/h/s values and the map is built around this
geometry. There's no other common point of control for multiple OS
coexistence; how else can partition boundaries be respected?

All the other BSDs installers make provisions for preventing this...that
being the case, I don't understand the skepticism or the surprise! It
hardly seems reasonable that all these other variants make a provision
to deal with an issue that is obsolete or irrelevant. Rather, it seems
extremely critical. If I understand correctly that the DFly installer
was taken from the FreeBSD installer, then I am hoping it will be a very
simple matter to add the G option to DFly's installer.

I keep records of the partition table, because I multiboot a lot of
systems. Before every installation I check the partition table against
my expectations, and I do the same thing afterwards as a sanity check. I
saw that the DFly installer noted that he was using different values
than I set for c/h/s. I would have stopped him but the opportunity I was
expecting to change the geometry never arrived. Another message was
issued that said that he adjusted the allocation to end on an integral
cylinder boundary- and that's the source of the problem.

After DFly built the filesystems I halted the system, booted my trusty
Slackware system and did an fdisk -l to see what had happened.

The numbers of cylinders, heads, and sectors per track had been changed
(and therefore the sizes of cylinders, which is the crucial part) and
the slice I had preallocated for DFly had changed in size. Luckily, the
next DOS partition was my Linux swap file, which provided enough buffer
that the intrusion by the DFly filesystems wasn't able to reach my other
filesystems on the same drive.

To fix it, I reset the cyl/head/sectors values to what they should be (I
understand these are arbitrary, but all the OS on the drive have to
agree to the same view) and changed the limits on the DFly slice back to
my original even cylinder allocation, and all was well.

It is possible than in many cases the partition table gets built with
geometry that corresponds to the BIOS geometry so if you multiboot on a
system like that you never observe this problem. It could be that Linux
fdisk wants to use 255 heads with 63 sectors/track. (I say this because
I noted in many newsgroup postings the c/h/s on Linux distros had this
geometry.) I don't know how that got decided. Maybe if you fdisk with
one of the BSD variants it's different. The bottom line is that
everybody has to sing the same tune and it doesn't matter what the c/h/s
values are as long as all the OS agree to use those values.

It seems to me that for DFly to be able to peaceably exist with other OS
on the same drive in all circumstances, there needs to be a provision in
the installer, such as is part of the OpenBSD, NetBSD, and FreeBSD
installers, to be able to change DFly's view of the disk geometry to
match the existing partition table.

This BIOS doesn't seem to have any options, and at any rate I'm already
running 7 other OS on the box without problems until now. The drives are
much larger than the old limits so the BIOS and other kernels are
clearly using large drive mappings sucessfully here.

The c/h/s values can be undependable in the abstract; they just have to
work with the one BIOS that the partition table is used by. If I have x
bytes to map and I arbitrarily decide that I'll call a cylinder 8225280
bytes, and that there are 255 tracks in a cylinder, and 63 sectors in a
track, then those c/h/s values tell me exactly where I am on the drive.
LBA is just another way to express the same mapping. It doesn't matter
what c/h/s I use as long as everybody who references the drive uses the
same values.

Rand

There is simply no way to address more than 8 GB with CHS. After
that you're simply out of bits. You have two options after that:
either fill all bits with ones to mark that you reached this limit, or
you simply don't care about the additional bits and wrap around the
values.

What is the result of this? You have to use logical block
addressing. And really, once you're there, you can't care less about
CHS values and alignment anymore.

However, I do agress that changing existing table values is a really
bad thing, so the installer shouldn't do this in any case.

No, you are mistaken. The DragonFly installer is really the
BSDinstaller, but a rather oldish version. Possibly you mistake the
source of scepticism. We're just too few people to do everything
people could possibly need. If you want to fix this issue, please go
ahead! We appreciate every helping hand!

cheers
simon

Hi Simon,

Thanks for your note.

I'm not suggesting that anyone is using c/h/s for purposes of
addressing. I'm saying that it's used in terms of allocating space.
Whether the fields are big enough internally to contain cylinders of
sufficient number to map the drive or not is not critical; what is
critical is that the fdisk interface and all the installers work on
allocating space based on their view of cylinder size. The problem is
that if everyone doesn't agree on the size of this thing called a
"cylinder" then the start and end points of the DOS partitions (BSD
slices) will not be equal in everyone's view and encroachment on
neighboring filesystems is inevitable.

Let's consider an example scenario where we use the common Linux fdisk
value of a cylinder arbitrarily valued at 8,225,280 bytes. Further,
we'll agree that there are 255 heads (tracks per cylinder) which would
then necessarily be understood to be 32,256 bytes in size. Lastly, we'll
say that our disk is divided further into a unit called a sector, that
we'll decide that there are 63 such sectors per track, which would then
work out to a nice round value of 512 bytes in size.

We'll work on a fictitious 120GB drive, which is really not 120GB but
rather 120x10**9 bytes in size. I note that Linux fdisk seems to fiddle
with the actual size a bit to get round numbers. (During the process of
working this example out I see that BSD does similarly but not
identically. The main thing to be aware of is that some rounding occurs
for convenience, and will affect the actual space available in the last
cylinder as it will not really be x bytes in size.)

I'll allocate three DOS partitions for my BSD systems using the view of
a cylinder containing 8225280 bytes, and I'll be careful to allocate
things in even units of cylinders.

I install OpenBSD in partition #1 and FreeBSD (using the G option) in
partition #3.

I've been wanting to install DragonFly since 1.6.0 so here's my chance
with the newly-announced 1.8.0_release! I'll install him in partition #2.

I run the DragonFly installer and I receive a message that he is using
values of 232,581 cylinders, 16 heads (16 tracks per cylinder) and 63
sectors per track. He also establishes normative values for sector and
track size which means cylinder size is a derived value. We work
backwards to find the cylinder size from the known dimensions of sectors
per track (63), and tracks per cylinder (16). With this view of the
geometry, the unit values work out as follows:

cylinder: based on a geometry of 16 tracks per cylinder and 32,256 bytes
per track. It works out to be 516,096 bytes per cylinder. Cross
checking, 516,096 x 232581 is more or less 120GB. Everything is fine.

What's the problem then?

The partition table was set up with a cylinder size of 8,225,280 bytes.
DragonFly uses a different view of cylinder size as 516,096 bytes.

These cylinder values are not integrally related, that is, 516,096 does
not divide 8,225,280 evenly.

When DragonFly calculates the partion cylinder ranges based on his view
of the geometry, he will almost certainly never find that the LBA value
which should be the end of an integral cylinder boundary is the one that
the partition table says is the end of his slice. Therefore, he
automagically (and I'll say improperly) adjusts the LBA up or down (this
part can only speculate- he makes an adjustment according to the message
posted but it could be that he only increases or decreases, I don't know
what he does) to be on an integral boundary from his view of the
cylinder size. What is certain is that he does adjust the partition
range to correspond with his view of a cylinder containing 516,096
bytes. I'll guess that he rounds up, which is quite a bit worse; if he
rounded down it would be wasteful and unnecessary but wouldn't expose
the data on the mext partition to loss/corruption.

Depending on how he rounds, and what the preexisting view of cylinder
size in the DOS partition table is, you can readily see that his view of
his slice may well encroach on another partition.

I should have given a better explanation at the start, I apologize for
anything that was unclear. The main idea here is that regardless of how
data is being addressed, it's being allocated by every OS in terms of
his view of how big a cylinder is. And if all OS on the drive don't
share the same view, and therefore agree on the start/end addresses of
partitions, there can certainly be loss of data.

This is why it seems to be essential to provide a way to change
DragonFly's view of the disk geometry as necessary to match the geometry
the partition table is already using.

Yes, it would be sufficient, if we're not able to easily provide a way
to specify geometry. At any rate, we agree that he should not touch the
partition table. After all, we've already gone through fdisk and
disklabel by this point. If anyone wanted to change things he could have
done so at the proper time.

I should point out that the installer did two bad things:

1. adjusted the geometry settings in the partition table (from c/h/s of
14593/255/63 and to a c/h/s of 232581/16/63)

2. changed the ending point of the DFly slice (DOS partition) to match
the DFly view of what it means to be on an integral cylinder boundary

It's certainly improper to modify the partition table units and it's an
imminent risk for the data on the drive in a multiboot scenario to
change partition boundaries.

I don't understand why he doesn't simply use the unit values in the
partition table.

I stand corrected, thanks for the clarification.

I'm sorry if it seemed as though I were saying "I know how to fix this,
why don't you take care of it?" Really I meant to say, I know what the
problem is, and I don't know how to fix it or even more importantly, how
to circumvent it. The few responses I got seemed to range from "yawn" to
"no, it's not happening as you think" which certainly didn't sound like
what you just said! I would be delighted to help but alas, I'm not
expert in any of the areas one would need to be knowledgeable in to do
anything about this other than to recognize the problem and ask for help.

I was hoping when I posted the question in the user list that someone
would say "yeah, you simply do such and such a thing in fdisk and this
problem won't happen" but since I didn't get such a reply, I decided to
post here.

Cheers,
Rand

[your email address is broken]

I doubt any fdisk does rounding to get nice human numbers.

So you're actually destroying the slice and then re-creating it? This
is important information. Until now I was under the impression that
the installer changes the slice table without being asked to change
slices.

So the slice table shouldn't be changed at all, I guess.

Actually the OS do not use any CHS view. It is only fdisk we're
talking here about. After having set up the slice table, nobody cares
about CHS.

fdisk can already do this, so this is "simply" a matter of adding this
to the installer. However, I disagree that this feature should be
exposed to common users. Setting up CHS values is a rather advanced
topic, so I simply suggest doing a manual install instead, where you
have everything under control. A clean user interface is an important
thing. Maybe it could be integrated into expert mode, however.

Yet, I suggest not to care about CHS at all. Simply ONLY care about
LBA. Don't care to put slices on cylinder boundaries. Where is the
point in this anyways?

Yah. It shouldn't even re-create the slice.

Well, nobody really is for the BSDinstaller. That's part of the reason
that we still are not using the new installer.

"yea, you simply update the installer to 2.0 and then we'll check
again" :)

cheers
simon

A 120GB disk really is 120GB, which is 120*10^9. G is for giga, which is
10^9, not 2^30. That's Gi: gibi, or giga-binary. See
http://physics.nist.gov/cuu/Units/prefixes.html and
http://physics.nist.gov/cuu/Units/binary.html .

Cheers,
--
Thomas E. Spanjaard
tgen@netphreax.net

that's intentional, as the spam is out of hand.
However, I check here frequently. If anyone would like to mail me,
please say so in the thread and I'll get back to you.

Yet, it happens. It has to happen, because the manufacturers don't build
disks with integral 1024 units.

No, where did you get the impression I'm destroying a slice and then
recreating it? The issue is the two things I stated (which weren't
included in the reply).

Here's what I wrote earlier:

> I should point out that the installer did two bad things:
 >
 > 1. adjusted the geometry settings in the partition table (from c/h/s 
  > of 14593/255/63 and to a c/h/s of 232581/16/63)
 >
 > 2. changed the ending point of the DFly slice (DOS partition) to 
match > the DFly view of what it means to be on an integral cylinder 
boundary

> It's certainly improper to modify the partition table units and it's 
  > an imminent risk for the data on the drive in a multiboot scenario 
to > change partition boundaries.

That's right.

Agreed, I don't mean to split hairs, to me fdisk is part of the OS.
Nobody cares but fdisk and disklabel. Disklabel also thinks in sectors
and cyls. Have a look and see.

The point is what I said: cylinder size is important because DFly
changes the partition table to match his view of what an integral
cylinder boundary means. If he doesn't change the partition table, and
doesn't exceed the slice allocated to him, then we can reach the level
of "not to care about CHS at all." Until then, it's a significant data
integrity exposure.

Right.

Not only should it not even recreate the slice, it shouldn't touch the
partition table unless I ask him to in fdisk.

you are (implicitly) asking the installer. i guess. again, nobody
knows the installer, please go ahead and try updating/fixing it.

cheers
simon

I'm not implicitly asking the installer! The installer is using a
preallocated slice (I don't use the fdisk step in the installer). He's
smart enough to see that I've set the partition type to FreeBSD and
gives me the option to install there.

I will look and see what I can find out, but again, this is not my area
of expertise. I don't even know where to begin looking. This may take a
while!

Cheers,
Rand

Well, in anycase it is invoking fdisk, otherwise the slice table
wouldn't change. fdisk is the only part of DragonFly changing the
slice table.

cheers
simon