OSDev.org

Posted: **Fri Jul 12, 2013 2:58 pm**

Hi,

I'm rewriting my build chain. Previously, I could build to CD .iso files or to a USB drive, using separate procedures. I am trying now to install to a flat disk binary image, which could be loaded directly by a VM or dd-ed onto a USB drive (possibly leaving an unallocated partition at the end?).

I'm having no end of trouble. I have found many examples that seem promising, but they tend to fail around the same step.

I wrote a shell script. Part of it reads:

Code: Select all

export CYLINDERS=10
export HEADS=16
export SECTORS_PER_TRACK=63
export SECTOR_SIZE=512

#Make HDD image
dd if=/dev/zero of=build/moss-disk.bin bs=$((HEADS*SECTORS_PER_TRACK*SECTOR_SIZE)) count=$CYLINDERS

#Make partition on disk file (re-enable as necessary)
cat <<EOF | fdisk -u -C$((CYLINDERS)) -S$((SECTORS_PER_TRACK)) -H$((HEADS)) build/moss-disk.bin
o
n
p



a
1
p
w
EOF

#Mount the disk image in a loop device
#	Note: the number is the start field of the about partition * SECTOR_SIZE bytes/sector
#	See the previous fdisk command, which should have printed it (prints as 2048, instead of 63)
sudo losetup -o$((2048*SECTOR_SIZE)) /dev/loop0 build/moss-disk.bin

#Make a file system on it (re-enable as necessary)
#	Note: the 4016 is the number of blocks (see fdisk command above, which should have printed it.
#	Note: source also explains how to set up FAT32
sudo mke2fs /dev/loop0 4016

#Mount partition
sudo mount -t ext2 /dev/loop0 /mnt

#Unmount the disk image from the loop device
sudo losetup -d /dev/loop0

All seems well until the mount step, which fails with:

Code: Select all

mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

In case you're wondering, the output of the fdisk call ends with:

Code: Select all

Disk build/moss-disk.bin: 5 MB, 5160960 bytes
16 heads, 63 sectors/track, 10 cylinders, total 10080 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x7daf4112

              Device Boot      Start         End      Blocks   Id  System
build/moss-disk.bin1   *        2048       10079        4016   83  Linux

The sources that I read said that the loop device must be offset to the start of the partition of else this problem can occur. From fdisk, above, that number is 2048*SECTOR_SIZE=1048576, which I am offsetting by.

I don't really know what to do here. The offset is different from the usual 63, but that oughtn't to matter as I am offsetting by the correct amount? I also tried hardcoding the correct values into the shell script, but to no avail.

Thanks,

Posted: **Fri Jul 12, 2013 3:42 pm**

Why do you hardcode number of blocks (4016) ? Just command mkfs to use full device.
What does mkfs output? Did it tell you about some errors?
What messages appear in your dmesg when you are trying to mount /dev/loop0?

Posted: **Fri Jul 12, 2013 4:42 pm**

Nable wrote:Why do you hardcode number of blocks (4016) ? Just command mkfs to use full device.
What does mkfs output? Did it tell you about some errors?

I wanted to be sure I was following directions. Removing the argument produces the exact same output:

Code: Select all

mke2fs 1.42 (29-Nov-2011)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=0 blocks, Stripe width=0 blocks
1008 inodes, 4016 blocks
200 blocks (4.98%) reserved for the super user
First data block=1
Maximum filesystem blocks=4194304
1 block group
8192 blocks per group, 8192 fragments per group
1008 inodes per group

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done

Nable wrote:What messages appear in your dmesg when you are trying to mount /dev/loop0?

Code: Select all

[36383.852268] EXT2-fs (loop0): error: can't find an ext2 filesystem on dev loop0.

Posted: **Fri Jul 12, 2013 5:44 pm**

Is seems really strange (your script worked almost successfully at my test PC, except commands for fdisk but that shouldn't make difference).
What does "file -s /dev/loop0" say?
In my case:

Code: Select all

# file moss-disk.bin
moss-disk.bin: x86 boot sector; partition 1: ID=0x83, active, starthead 1, startsector 63, 10017 sectors, code offset 0x0
# file -s /dev/loop0
/dev/loop0: Linux rev 1.0 ext2 filesystem data

Posted: **Fri Jul 12, 2013 6:00 pm**

Nable wrote:Is seems really strange (your script worked almost successfully at my test PC, except commands for fdisk but that shouldn't make difference).
What does "file -s /dev/loop0" say?
In my case:
Code: Select all
# file moss-disk.bin
moss-disk.bin: x86 boot sector; partition 1: ID=0x83, active, starthead 1, startsector 63, 10017 sectors, code offset 0x0
# file -s /dev/loop0
/dev/loop0: Linux rev 1.0 ext2 filesystem data

Hmmmm. At all stages, the /dev/loop0 reads as "/dev/loop0: data", while build/disk-moss.bin reads as:

#Right after dd
build/moss-disk.bin: data
#Right after fdisk
build/moss-disk.bin: x86 boot sector; partition 1: ID=0x83, active, starthead 0, startsector 2048, 8032 sectors, code offset 0x0
#Right after losetup and mke2fs
build/moss-disk.bin: x86 boot sector; partition 1: ID=0x83, active, starthead 0, startsector 2048, 8032 sectors, code offset 0x0

Posted: **Sat Jul 13, 2013 5:41 am**

So, mkfs didn't do its work.
'mkfs.ext2 /dev/loop0' ( strange ideas come to my mind, e.g. 'try other FSs because smth related to ext2 may be broken in your system' ) and then 'hexedit /dev/loop0' to check that smth changed.

Posted: **Sat Jul 13, 2013 3:43 pm**

Nable wrote:So, mkfs didn't do its work.
'mkfs.ext2 /dev/loop0' ( strange ideas come to my mind, e.g. 'try other FSs because smth related to ext2 may be broken in your system' ) and then 'hexedit /dev/loop0' to check that smth changed.

Also tried "mkdosfs -F32 /dev/loop0 4016". In any case, the hex dumps reveal all zeros both before and after the file system.

EDIT: By looking at build/moss-disk.bin at offset 0x100000 (=2048*512) bytes in, I see that the file system seems to have been created--something is there at least. /dev/loop0 still appears to be all zeros though.
EDIT2: By using losetup without any offset, I can see that both build/moss-disk.bin and /dev/loop0 have a bootsector. At offset 0x100000, build/moss-disk.bin has its file system, but /dev/loop0 does not. This is after a fresh losetup too.
EDIT3: At the implication of this page, I tried "sudo partprobe /dev/loop0" which didn't help and rebooted, which did. After losetup, /dev/loop0 reads the same as build/moss-disk.bin, looking as I would expect with various offets into losetup. With the correct offset, it mounts fine! The magic seems to have been the restart; is there a way to make it work without having to do that?

Posted: **Sat Jul 13, 2013 6:13 pm**

Since the restart, I can't reproduce the broken /dev/loop0 before.

Well, on to installing GRUB on this thing. I refined the preceding process a little bit after reading a few more questions. /dev/loop0 now contains the entire disk image, while /dev/loop1 contains only the first partition:

Code: Select all

#See: http://wiki.osdev.org/Loopback_Device
#See: http://superuser.com/questions/130955/how-to-install-grub-into-an-img-file

export CYLINDERS=100
export HEADS=16
export SECTORS_PER_TRACK=63
export SECTOR_SIZE=512

#Make HDD image
dd if=/dev/zero of=build/moss-disk.bin bs=$((HEADS*SECTORS_PER_TRACK*SECTOR_SIZE)) count=$CYLINDERS

#Make partition on disk file (re-enable as necessary)
cat <<EOF | fdisk -u -C$((CYLINDERS)) -S$((SECTORS_PER_TRACK)) -H$((HEADS)) build/moss-disk.bin
o
n
p



a
1
p
w
EOF

#Mount the disk image in a loop device
sudo losetup /dev/loop0 build/moss-disk.bin
#Make the disk image's partitions into devices
sudo kpartx -v -a /dev/loop0
#Mount the disk image's first partition in a loop device
#	Note: normally you now would mount /dev/loop0p1 directly.  But, GrUB doesn't like that.
sudo losetup /dev/loop1 /dev/mapper/loop0p1

#Make a file system on first partition
#	Note: source also explains how to set up FAT32
sudo mke2fs /dev/loop1

#Note: may have to unmount loops, restart, and then remount loops here.  See http://forum.osdev.org/viewtopic.php?f=1&t=26907

#Mount the first partition
sudo mount /dev/loop1 /mnt
#Setup a device map, as per S.U. question (link above)
#	Note: IDK why, but I had to create it in /tmp first.
#	Note: I haven't tested this part in the script; I typed it in by hand using CTRL+V/CTRL+I to
#	        get the TAB. I based this off of the device map in the host.
sudo mkdir -p /mnt/boot/grub
cat > /tmp/device.map <<EOF
(hd0)\t/dev/loop0
(hd0,1)\t/dev/loop1

EOF
sudo cp /tmp/device.map /mnt/boot/grub/device.map

#Install grub
sudo grub-install --root-directory=/mnt /dev/loop0

#Unmount the disk image from the loop devices
sudo losetup -d /dev/loop0
sudo losetup -d /dev/loop1

The problem is that now the grub-install command is giving:

Code: Select all

expr: non-integer argument
expr: non-integer argument
/mnt/boot/grub/device.map:2: error: No close parenthesis found
cmp: EOF on /tmp/grubYiC7i9
/mnt/boot/grub/device.map:2: error: No close parenthesis found
cmp: EOF on /tmp/grubYiC7i9
/mnt/boot/grub/device.map:2: error: No close parenthesis found
cmp: EOF on /tmp/grubYiC7i9
/mnt/boot/grub/device.map:2: error: No close parenthesis found
cmp: EOF on /tmp/grubYiC7i9
/mnt/boot/grub/device.map:2: error: No close parenthesis found
cmp: EOF on /tmp/grubYiC7i9
The file /mnt/boot/grub/stage1 not read correctly.

This is bizarre. Using gedit, the entire source of /mnt/boot/grub/device.map is:

Code: Select all

(hd0)	/dev/loop0
(hd0,1)	/dev/loop1

Posted: **Tue Jul 16, 2013 3:49 pm**

I give up on grub-install. It can DIAF, since it doesn't work at this. Wrote a script; it just copies the files over.
People of the future: see here for my crufty workaround.

Posted: **Mon Aug 12, 2013 4:04 pm**

Hi,

This is my first post here, but I've been lurking for a while...

I've spent most of today trying to get something similar, as I too was using grub-mkrescue. The script posted earlier in this thread looks like it's based off the same example I started from. I finally got it working (it still kicks up a fuss about ioctls on the mapper, but it does work nevertheless) and I blogged the script I ended up with here.

(Caveat: I'm on Fedora 18, I've no idea how this will play on other systems).

Posted: **Fri Dec 13, 2013 9:53 am**

In the device.map, the character between the "(device)" and the "file" should be a whitespace, but not a tab. It is very confusing because the device.map generated by --recheck uses tab instead of whitespace.

Posted: **Fri Dec 13, 2013 12:56 pm**

As an alternative approach, may I humbly propose my disk image tool - DITO?
http://forum.osdev.org/viewtopic.php?f=2&t=27349

OSDev.org

Setting up a HDD Image File / Installing GRUB

Setting up a HDD Image File / Installing GRUB

Re: Setting up a HDD Image File / Installing GRUB

Re: Setting up a HDD Image File / Installing GRUB

Re: Setting up a HDD Image File / Installing GRUB

Re: Setting up a HDD Image File / Installing GRUB

Re: Setting up a HDD Image File / Installing GRUB

Re: Setting up a HDD Image File / Installing GRUB

Re: Setting up a HDD Image File / Installing GRUB

Re: Setting up a HDD Image File / Installing GRUB

Re: Setting up a HDD Image File / Installing GRUB

Re: Setting up a HDD Image File / Installing GRUB

Re: Setting up a HDD Image File / Installing GRUB