Situation:
libc6 got damaged for reasons unknown on a server. I pinpointed the problem to that accurately so it is nothing else. Obviously with libc6 damaged, I could not use terminals etc as I will just get a segmentation fault error if I type any command in the running server. Obviously I cannot reboot as it wont boot with corrupt libc, which unfortunately I tried. Unfortunately my system backup is corrupt for that server.
QUESTION: Given the above situation and the fact that I can boot into another Linux Admin OS on the same server and disk. Is it possible to force a reinstall libc6 from the Admin Linux distro to the other distro partition where the damaged libc6 resides.?
I saw during a websearch for a similar problem the solution, that is too cryptic.
Blockquote Use apt-get download libc6 to get the package, then
dpkg -x libc6*.dpkg /partition_path
to “install” it.
That looks easy but the problem for me is how do I reinstall a package from an Active Admin Linux distro to another inactive distro residing on another partition (as that wont boot anymore) and I cannot make out if the above solution actually addresses it.? It is something I never did before. It is a nasty problem that can easily be solved if I can somehow reinstall libc6 on an inactive distro on a partition from an active distro on another partition.
All my backups were corrupt (I will have to investigate that as it is peculiar) and I managed to get out of this one without any data loss whatsoever and could install libc6 while the server was running a completely different distro on another partition. I never knew this is possible, but it opens up a huge tool for correcting broken libraries that cannot be done while the OS is running.
Nixcraft,
I will do so in detail. I will have to set up a distro on a test machine and break libc6 and then demonstrate how to do it from another distro so it will take a bit of time till I am there. I will then possibly rename this thread and take out the post which are not relevant anymore such as this one so everybody can use the info.
Nixcraft, I cannot edit my original post, so here is the procedure: You can clean up the thread and post this as the first post and remove the rest if you wish to keep things clean. Thanks.
I found this method to be the most powerful way to fix a server as it can be done completely remotely and completely prevents annoying and time-consuming reinstalls or where backup recovery is not the option due to valuable recent data… Especially on very valuable installations.
Let’s say you have two linux distributions to dualboot on a system and one of them bricks because of a library that is corrupt or bad upgrade or installation.
In this case we will use libc6 upgrade as an example as it bricks a system the fastest. It does also work with any other corrupted package install and not only libc6.
) During a libc6 upgrade, that failed, the new version of libc6 unconfigures the existing libc6 and then fails to install.
) Your system then has a non working unconfigured libc6 and the system bricks as in the attached image, where you do not even get terminal output and everything basically segfaults.
) See attached image of a system I intentionally corrupted libc6 by a botched upgrade in order to show the solution.
) As you can see, the system is royally bricked and the only solution would be to reinstall or recover from backup.
) If you try to boot into this system, you will immediately get a kernel dump and relevant crash.
There is however another method by which you neither have to chroot (never works and is very dubious) or reinstall at all.
Here is how it works.
Find the version of libc6 that is installed on your system (preferably in dpkg form or create a dpkg) from the distribution sources or google search. It is crucial you get the version that came with this distro and note what the device number is of the corrupted installation (Let’s say /dev/sda1)
Once you have that, dual boot into the other working linux distro (let’s say on /dev/sda2) on the computer and su to root. Just forget SUDO it is a wolf in sheepsclothes and makes nothing more secure.
While at the root prompt, do the following:
3.1 Use apt-get or other means to download libc6 from your distro’s source disks or online location into your root directory.
3.2 Start gparted and make sure you identify which device number the distro that is not working is located on. For our purposes you have e.g. the corrupted installation on /dev/sda1 which you verified with gparted.
3.3 mkdir /temp (This creates a directory to mount the corrupted distro on)
3.4 mount /dev/sda1 /temp (You now mounted the corrupted directory on /temp directory)
3.5 dpkg -x libc6.dpkg /temp* (First create dpkg version of your libc6 sources . This is easily found in howtos on the web and a trivial issue.)
3.6 You now have installed the original libc6 to /dev/sda1 from the distro on /dev/'sda2 … sort of a remote passive installation.
3.7 Reboot, and boot into the corrupted distro. You will now see that everything works perfectly again. The problem is now solved.
CLEANUP: Now that your system is running again on /dev/sda1, you must not repeat the same procedure that caused the problem. If it was a botched libc6 install it will corrupt the system again… unless you remove the upgrade package that is partially installed. This is an involved process, buit you have to do it as an apt-get update will brick your system again if you dont remove the offending package.
This is a completely different problem and I refer you to another of my threads or you can use the web to solve that.
If the package just self-corrupted (without any apt-get or package install), then the cleanup is not necessary.
Regarding (4) I refer you to my thread “Apt & Unmet Dependencies” on Nixcraft on how to remove the offending package. See here to follow the instructions. Apt & unmet dependencies.
Dont you just love Unix/Linux. I find it amazing that I was able to do this, and it now saved my bacon on 3 occassions.
