Hello,
I’m formatting a USB disk for backup using the backup application: rear
The formatting is really slow and I’m wondering if there is a quickformat option within Ubuntu that I might be able to use instead of rear to prepare the disk.
The disk is being formatted with the ext3 filesystem.
In Windows there is a quickformat option for NTFS partitions and I’m wondering if there is something similar for mkfs (not that I can see it though)
Thanks
VW
Are you looking for a GUI or CLI option?
Preferably CLI, because I may be able to patch Rear, and then use the existing process.
But if there is a GUI fix and that’s the only option, then I’ll prepare the disks myself via the GUI.
The current process is painful and has to be changed if at all possible.
Thanks
VW
@VeeDub I just looked into manual page. There is no such thing as a ‘quick’ format for the ext4/ext3 file system. Such option is only available for ntfs and fat32 filesystem under Linux. The mkfs.ext4 is much faster tho. May I know your USB disk size? Is it bigger than 4TB?
I’ve just noticed, that according to GParted the disk has been formatted with ext4, despite the fact that rear reported that the partition was formatted with ext3.
Not sure if this matters, but reporting it, in case it is relevant.
The disk is 4TB, per marketing, which means approx 3.64TB in reality.
There are some optimization option in the /etc/mke2fs.conf. For example:
cat /etc/mke2fs.conf
[defaults]
base_features = sparse_super,large_file,filetype,resize_inode,dir_index,ext_attr
default_mntopts = acl,user_xattr
enable_periodic_fsck = 0
blocksize = 4096
inode_size = 256
inode_ratio = 16384
[fs_types]
ext3 = {
features = has_journal
}
ext4 = {
features = has_journal,extent,huge_file,flex_bg,metadata_csum,64bit,dir_nlink,extra_isize
inode_size = 256
}
small = {
inode_size = 128
inode_ratio = 4096
}
floppy = {
inode_size = 128
inode_ratio = 8192
}
big = {
inode_ratio = 32768
}
huge = {
inode_ratio = 65536
}
news = {
inode_ratio = 4096
}
largefile = {
inode_ratio = 1048576
blocksize = -1
}
largefile4 = {
inode_ratio = 4194304
blocksize = -1
}
hurd = {
blocksize = 4096
inode_size = 128
}
[options]
fname_encoding = utf8
Your disk is 4TB, so if I were you I would try something as follows:
sudo mkfs.ext4 -T largefile4 /dev/diskNameHere
Another option is to pass the -E lazy_itable_init=0. For example:
sudo mkfs.ext4 -E lazy_itable_init=0 -T largefile4 /dev/diskNameHere
Here is what you get:
lazy_itable_init[= <0 to disable, 1 to enable>]
If enabled and the uninit_bg feature is enabled, the inode table will not be fully initialized by mke2fs. This speeds up filesystem initialization noticeably, but it requires the kernel to finish initializing the filesystem in the background when
the filesystem is first mounted. If the option value is omitted, it defaults to 1 to enable lazy inode table zeroing.
So when you mount the disk first time it will initialize creating those inodes. That is all I can think. Try it out.
I’ve since realised that I didn’t give you the full picture.
The disk needs to be bootable as well, so there are two partitions.
I think I might try setting up the next disk via GParted and see how long that takes.
I’ve had a look both at disks and gparted, and I’m having trouble working out how I can setup another bootable USB disk with the same properties as the one prepared by rear.
Here are the two partitions on the existing disk that were created by the rear setup
What I want to try and do is work out a way that I can setup a 2nd disk with the same properties where the data volume is setup quickly. I don’t know how exactly rear is formatting the data partition, but it is taking hours!
I want to see if I can create and format the data partition in some other way that doesn’t check all the sectors, write 0’s or whatever it is doing.
Thanks
You can copy the partition table layout of a whole disk using standard tools named sfdisk. For example, if device is /dev/sda and want to copy that to /dev/sdb, then. Make a copy:
# *******************************************
# !!! WARNING !!!
# BE CAREFUL WITH DEVICE NAMEs
# *******************************************
sudo sfdisk -d /dev/sda > sda.part.copy.txt
There here is how to restore keeping the same disk and partition IDs to the /dev/sdb:
# *******************************************
# !!! WARNING !!!
# BE CAREFUL WITH DEVICE NAMEs
# *******************************************
sudo sfdisk /dev/sdb < sda.part.copy.txt
## OR ##
sudo sfdisk -f /dev/sdb < sda.part.copy.txt
Of course, partition table replication is possible with one command as the root user when you wish to keep the same partitions and IDs:
sfdisk -d /dev/sda | sfdisk -f /dev/sdb
Here is how to restore generating a new disk and partition IDs. First use the grep and sed command to remove older IDS:
grep -v "^label-id" sda.part.copy.txt | sed -e 's/, *uuid=[0-9A-F-]*//' > without-IDs-sda.part.copy.txt
diff sda.part.copy.txt without-IDs-sda.part.copy.txt
sudo -f /dev/sdb < without-IDs-sda.part.copy.txt
More options and info here https://www.cyberciti.biz/faq/linux-backup-restore-a-partition-table-with-sfdisk-command/
Thanks, this is what I want to do.
I want to restore generating a new disk and partition IDs
Unfortunately, I don’t follow your comments where you say
First use the grep and sed command
I just don’t understand how the grep command fits in with the earlier examples.
I think I understand the earlier examples, I just need clarity on how I include the grep / sed commands as part of the procedure.
Basically, I want to replicate the partition / file system structure but with new partition IDs
Thanks
VW
The grep and sed will remove uuid. This is useful when using two disks on the same machine. Typically uuid is used to mount and do other stuff. Hence, having different uuid helps. I hope this helps.
I have executed the following commands:
sudo sfdisk -d /dev/sda > sda.part.copy.txt
grep -v “^label-id” sda.part.copy.txt | sed -e ‘s/, uuid=[0-9A-F-]//’ > without-IDs-sda.part.copy.txt
diff sda.part.copy.txt without-IDs-sda.part.copy.txt
sudo sfdisk -f /dev/sdb < without-IDs-sda.part.copy.txt
This appears to have created, but not formatted the partitions as per disks. If you compare the screenshots from disks below with the previous screenshots
When I try:
sudo e2label /dev/sdb2 REAR-002
The response is:
e2label: Bad magic number in super-block while trying to open /dev/sdb2
sudo mkfs.ext3 /dev/sdb2
took around 2.5 hours
That’s the issue, is there any alternative to provision a disk quicker?
Does it need to be ext3? Or in other words: Is there a reason why you don’t use ext4?
Also this is wrong:
mkfs.ext4 -E lazy_itable_init=0 /dev/sdX
You want it to be enabled so you need to set it to “1”:
mkfs.ext4 -E lazy_itable_init=1 /dev/sdX
To make a disk bootable you can simpy use GParted and set the “Boot”-Flag on your Disk.
I would have to test that, for whatever reason when rear prepares the disks it uses ext3
So, if I prepare a disk myself, and make it ext4, then I’ll need to confirm that all other rear operations work.
But my initial (admittedly un-informed) feeling is that ext4 ought to be fine.
The key question from my perspective is that assuming an ext4 partition is compatible with rear; will an ext4 format complete significantly quicker?
It is crazy how long it takes to format a disk.
Well, that’s the question. We would have to find out.
rear format /dev/sdX does not seem to do more than just creating a partition and formatting it so it shouldn’t be a problem to just format the disk yourself.
After formatting you just need to set the following in your /etc/rear/local.conf:
OUTPUT=USB
BACKUP_URL=usb:///dev/sdX
I haven’t tried it out but this is how it should work.
mkfs.ext4 -E lazy_itable_init=1 /dev/sdb
worked really well, it formatted the partition in around 1 minute. So that was perfect.
Rear has been able to backup to the data partition, but there were some errors during the backup related to the EFI partition (see below).
For some reason the boot partition was resized from 1GB to 419MB and this has created some issues. So I need to be able to setup the EFI partition as 1GB.
You could simply clone the partition table to the destination drive like mentioned above. Just be careful with the device names.