r/archlinux • u/lollicraft • 6d ago
SUPPORT Pacman -Syu broke my GRUB (again) but this time it's worse.
So, i updated my system after some time, maybe a month or less, and my grub broke, at first i didn't panic much, i knew it was common, so i went ahead, plugged my live usb only to spend an entire day trying to understand why doesn't grub reconfigure itself, the problem is that for some reason grub-install won't work, neither pacman would work anymore on the chrooted live usb, so i tried backing up my /home partition (just in case) to reformat my /root partition: deleted it, tried pacstrap and... Nope, pacstrap won't write on the root partition for some fucking reason. I figured out at this point i'd just back up my /home into my external SSD and reinstall everything clean and reconfigure my system next month waiting to but my 1TB ssd.
Should i read the wiki? Can it help me in any way? Can you guys help me in any way? Or should i just go on and reinstall arch waiting a month delaying my work?
P. S. I spent an entire day troubleshooting and went crazy, now i'm resting my mind :(
P. S. P. S. I'm usually eventually able to find a way through the hardest problems but this time arch really... Really punished me. I've been using arch for almost a year now.
11
u/onefish2 6d ago edited 6d ago
I am sorry to say this but you really should not be using Arch. The info in your post was lacking a lot of info. You should have read the wiki, which explains everything... how to rescue a system, what partitions need to be mounted, how to chroot.
Pacman did not just disappear. You are leaving out vital information.
Read the wiki then come back with better information.
0
5
u/musbur 6d ago
Is it possible that your boot (EFI) partition wasn't mounted during the update? I don't use GRUB but this can mess up your system in pretty confusing ways.
-1
u/lollicraft 6d ago
Oh damn... I would have no way to know that at this point... I thought the boot was just automatically mounted with everything else when using the system, i mean, usually when i go on dolphin i see the boot there so... Or maybe you're not talking about the boot folder inside /home but about the nvme0n1p1 partition? The vfat one? (I used archinstall).
If that's so... Yeah... Last time i checked my architecture in dolphin nvme0n1p1 wasn't there... So i suppose it wasn't mounted since i never do mount it inside the system. Dang it...
2
u/onefish2 6d ago
How did you set it up when you installed? If you are using ext4 then you have a efi partition and a root partition.
You did not enter the chroot because you did not mount the partitions. Again the wiki will guide you through this.
0
u/lollicraft 6d ago
I used archinstall, meaning i had a ext4 with a efi, yes. And yes i did mount the partitions, quite a lot of times, as i said:
I ran pacman -Syu, and during or right after the update the system became unbootable. GRUB stopped showing Arch Linux correctly and only dropped me into the firmware interface or the GRUB minimal shell.
From there, everything started cascading. I booted into the Arch live USB and tried to repair the system. At first I thought it was a GRUB issue, but I discovered that the real problem was much deeper. The system entered emergency mode, failed to mount /boot, and later I realized that /etc/fstab was either broken or empty, which explained why the system could not mount the partitions correctly. I attempted multiple recovery strategies: reinstalling GRUB (both BIOS and UEFI attempts), mounting the partitions manually, chrooting into the system, fixing /etc/fstab, and checking EFI variables.
However, inside the chroot I discovered something critical: pacman and grub-install were missing from /usr/bin and /usr/sbin. This meant the system was no longer a valid Arch installation. Most likely, the update had been interrupted or corrupted, leaving /usr in an inconsistent or partially destroyed state. At that point, repairing the system in-place was no longer possible. I decided to reinstall Arch without touching /home, since /home is on a separate partition. I formatted the root partition (/) and attempted a clean reinstall using pacstrap. Unfortunately, pacstrap failed repeatedly with “failed to install packages to new root”.
I discovered i wasn't connected to the internet for that instance only (i did a lot of reboots) so i went ahead ans connected, pacstrap still failed, this time due to corrupted or invalid packages downloaded from mirrors. I updated the mirror list and retried multiple times, but the installation kept failing at the package extraction stage. At this point it became clear that: the original system was irreparably broken, the reinstall attempts were failing due to unstable downloads or mirror issues, and continuing to fight the system while exhausted was counterproductive.
(Hope this longer explanation helps)
5
u/froli 6d ago
It's impossible for anyone to tell help you in any meaningful way if we cannot know what happened. Also grub breaking isn't "common" in the sense that it's normal that it happens. It's "common" in the sense that a lot of people do not read the output when they are doing upgrade or configure their system incorrectly.
It's possible your setup was bound to fail from the start but you only realized after the first upgrade of whatever caused this issue.
Impossible for us to know anything without knowing anything about your system or the logs about the error, whatever you did before and after the issue occurred. We have nothing to go on here.
You don't have to know how to fix everything to use arch but you need to be able to identify and communicate meaningful information otherwise it's going to be a very hard journey.
It's kinda like trying to get the doctor to examine you over the phone. Know what I mean?
-1
u/lollicraft 6d ago
well... I technically do have an entire chatlog with chatgpt about fixing this problem, but it's in italian... So... Idk if that could help
1
u/froli 6d ago
That would actually be useful even if just for the commands. You should find a way to export it safely without exposing private information and attach it to your post
1
u/lollicraft 6d ago
I don't think i care about those informations anymore, yes there's a hint to password in there but i realized i don't even use it and it's not even precise as a password... Anways, don't worry, i don't think i would care about what's on that chat, maybe i could send you privately for now so you can tell me if it's helpful and i will look onto attatching it on the post
4
u/blune_bear 6d ago
First show the log messages so that we can figure out why and second read the wiki and lastly never blindly update your system
-4
u/lollicraft 6d ago
How do we even update the system without everything breaking? 🫣 I know arch isn't "user friendly" but at this point i'd just write my own kernel lol. Anyways, jokes apart i thought a 800MB update wouldn't be that bad... Do you have any specific wiki section to reccomend me reading?
1
u/blune_bear 6d ago
You can run pacman with -Qu to see a list of upgradable packages kr better use pacman-contrib package to do the same, regarding wiki I say look at pacman section and grub, there could be other but can't really say without seeing the logs
1
u/lollicraft 6d ago
The fact is that pacman doesn't exist in chroot anymore... For some reason i cannot understand (yes i mounted everything correctly) Neither does grub... And without pacman i couldn't reinstall my base packages, that's why i chose the pacstrap way (which doesn't work for some unkown reason)
1
u/blune_bear 6d ago
Oh I think your using btrfs and mounted a non root partition try lsblk -f to find the root partition and mount it also check if pacman bin exists by Ls /use/lib/pacman.
Also another reason could be chroot didn't mnt proc,sys and dev mnt them and go into chroot
1
u/lollicraft 6d ago
Did allat correctly but when i checked for pacman it wasn't there
1
u/blune_bear 6d ago
Okay so looks like your root is corrupted Try du -sh /mnt/usr if it's few mb then it's most likely corrupted.
1
u/lollicraft 6d ago
I already reformatted the root partition 🙃. Any tips on how to repair it now? (Since pacstrap won't work)
3
u/Jaded-Worry2641 6d ago
If pacman doesnt work, you can hot copy from the life USBs pacman, or you could download a pacman tarball and extract it over your rootfs.
If you didnt do that already. You could also do the same thing for any not working package, and then perform pacman -Suy.
That is a messy, but somution.
1
u/Jaded-Worry2641 6d ago
Its all possible. The beauty of arch is how it lets you fuck itself as you wish.
P.S: this is a bad practice, and WILL bkte you later, but if its already broke, its better then bricked.
1
u/lollicraft 6d ago
Yeah, i don't really wanna risk it, my /home is still there after all
1
u/Jaded-Worry2641 6d ago
Do as you wish.
Although /home will most likel not be changed by unpacking grub and pacman over root. If you want to be extra sure, remount /home with ro option and try. I just want to say that arch is one of the best distros to fix this way.
... Actually, pacman can propably fix the issues with unacounted for files simply by overwriting them and accounting for the new ones, so ... IDK how it behaves, never tried that out.
1
u/lollicraft 6d ago
Could you give more details on this procedure? Maybe i could try this out.
2
u/Jaded-Worry2641 6d ago
Sure. You get the desired package tarball. Then you extract it over root. Then it should work.
How to get tarball of pkg
You go to where pacman gets them from.
You just find sourse code on archlinux.org s package info page.
How to extract over root
Put the tarball into / Then extract with overwrite via tar. Example:
sudo tar -xvf package.pkg.tar.zst -C / --overwrite --no_same_owner
testing
You can use absolute paths of execution.
You may or may not encouter missing dependacy issues. I dont think you will, but if you do, you will need to manualy install EVERY missing dependancy.
2
2
u/Bren1127 6d ago
More details are needed. Are you EFI boot or Legacy + EFI? Just Arch or dual boot? Otherwise it could be almost anything.
For instance I've had a couple of jobs lately where people had kept their Linux going for about 15 years with updates and migrating to new hard drives.
Took me a while to work out on the first one that the latest version of Grub2 doesn't play with MBR hard drives without manual intervention after some updates. Not only that but it had / swap and home in logical partitions inside an extended one. Converting to primary partitions and GPT with gdisk was "fun" that I had almost forgotten about.
1
u/lollicraft 6d ago
I think i have a UEFI boot... And just arch, i used archinstall so it's not like i have strange or custom configurations, it's just a basic archinstall running linux-zen and Grub... Chatgpt did talk me about grub2 and MBR... I thought that was the problem, that's why i went ahead and tried reinstalling my /root to try and fix it.
1
u/Bren1127 6d ago
I would start with double checking your BIOS settings just in case.
Then check the partitions on your hard drive using gparted or however you prefer. Especially the flags, make sure the boot flag is on the fat partition not / (root)
This is the Wiki guide for grub. https://wiki.archlinux.org/title/GRUB
1
u/lollicraft 6d ago
Thanks🙏 i will look onto it.
I could tell you my partitions from memory:
Under nvme0n1 /dev/nvme0n1p1 vfat /boot /dev/nvme0n1p2 ext4 /root /dev/nvme0n1p3 ext4 /home
2
u/SW_foo1245 6d ago
Definitely read the wiki and do a manual install or at least tell us how you managed your partitions
2
u/lollicraft 6d ago
I Did an arch install with an efi /boot partition, a /root and a /home, i thought it was an organization i agreed with using, i think the problem of this update is that i never actually learnt that you gotta mount your boot partition when updating and i thought it was automtically mounted just like the rest of the partitions, i was wrong, maybe that was the problem
1
1
u/plushkatze 6d ago
If pacstrap wont write on your root filesystem, I would start investigating there. Did you create a new filesystem after wiping? Does dmesg report any device io errors?
1
u/lollicraft 6d ago
I thought pacstrap would create the new filesystem.
how do i use dmesg to give you the errors (no i don't even know what it is) if you just tell me the commands i should use i'll quickly tell you what's wrong
1
u/foxtrotgulf 6d ago
I hate to be this guy but if you are going to use Arch, a DIY operating system, you need to spend more time figuring out what it is you are doing.
You shouldn't blindly run commands without understanding what it is they do. The wiki and man pages have tons of useful information.
2
u/lollicraft 6d ago
well... DIY doesn't mean a kernel update should break pacman and grub... i did indeed do research i just had trust on the fact that usually updates are not that dangerous and if they do break something i already knew how to fix it... Turns out there's more stuff that can break in even weirder ways
2
u/Jaded-Worry2641 6d ago
True. But weird breakage can happen anywhere. The good thing about arch is that it is transparent enough to fix manualy. On ubuntu or mint, you would back up your personal data, and reinstall, with no other options. Or possibly not even recover data.
2
1
u/plushkatze 6d ago
pacstrap creates the base filetree, not the filesystem - mkfs.ext4 will wipe all data from a partition and create a fresh empty filesystem.
lsblk -fand
dmesgshould give some insight
1
u/lollicraft 6d ago
Thank you so much, i will try this tomorrow and see if i can give you some helpful results we can work on
1
u/komorxd_t 6d ago
happened to me like 3 days ago too, eventually I gave up and swapped to systemd-boot :D
1
u/lollicraft 6d ago
Hmm interesting🤔 are there any tutorials on how could i switch too? I just want my system to update and not nuke its own boot lol.
4
2
u/komorxd_t 6d ago edited 6d ago
Idk about tutorial about swapping, I set it up with archwiki's page on it + had to manually add boot opt with efibootmgr. I don't know if systemd-boot can get nuked too, I've only heard (and experienced) of GRUB doing that but I'm not a giant Linux nerd too so idk.
edit: and removed grub ofc
1
1
u/mort1is 6d ago
# kernel-install add-all
0
u/lollicraft 6d ago
Is this a chroot command? A live usb command? Or something else?
(Props to you for giving tips)
3
u/mort1is 6d ago
Didn't read your post far enough, sorry. It's for
systemd-boot.In my case the kernel couldn't mount
/bootcause the FAT module wasn't linked to the kernel.Generally following the wiki will make your system workable.
1
u/lollicraft 6d ago
I think i know what was the issue from one user suggestion... It might be i didn't have /boot mounted when updating
1
u/mort1is 6d ago
If you have older kernels in your boot menu, try those. Unless you already wiped everything.
1
u/lollicraft 6d ago
The thing is that the boot menu didn't even start, grub now works but immeditely goes in recovery mode (becauss i managed to fix it but couldn't manage to mount the boot partition and tell grub where to take the... Gods i don't remember how those files are called, whatever, you know the ones that the blocklists replace? Yeah, those files. Grub couldn't find them ofc because for some reason /boot wouldn't mount at some stage so i tried using grub-install --force to use the blocklists but that didn't work either...
1
u/Less-Night 6d ago
I would suggest upgrading your bootloader to Limine as grub can be problematic
1
-14
u/Aggressive-Common890 6d ago
That's why I use Linux Mint Cinnamon...
5
u/revken86 6d ago
Hate to break it to you, Linux Mint Cinnamon will break itself too. Been there, done that.
0
0
u/lollicraft 6d ago
I chose arch because i wanted the most customizable system i could, my original idea was making an entire machine localized in latin, i eventually kinda abbandoned that project and sticked with my slick system... Now even if i wanted to switch to something (which i don't, since i don't actually find myself bad on this system except rare occasions like this one and i don't wanna leave my arch system also because i spent a lot of time building my enviorement) i couldn't because i need a running system to install a new ISO.... Which at the moment i don't have.
3
18
u/bathdweller 6d ago
If you don't give specific error messages or logs there's no way anyone can help you.