I have done a fuck up again (mkfs.fat on an ext4 partition)

foxonefoxone OG
in Help

Hello,
while setting up my backup system (oh, the irony), I accidentally launched a mkfs.fat on my ext4 hard drive.

Of course, since i was setting up the backup system, I didn't have any backup.

I desperately ^C but it was too late and the filesystem was gone, so now i'm trying to recover the data.
Photorec of course can see everything, but with random file names. I'd be nice to be able to recover it with the correct file names, so that the files would actually have context (most are source codes and stuff referenced via file name).

I already made an image of the disk, and I'm working on that.
Using testdisk i can see partitions with the original partition name:

However, whatever partition i try to open gives me no files:

I also found online some info on how to recover the partition using backup superblocks, but all the offset i tried don't seem to work (doing the dry-run fsck gives me a superblock error).

At this point i have no idea what to do, besides using photorec (which i would like to not resort to).

«1

Comments

  • InceptionHostingInceptionHosting Hosting ProviderOG

    A simple method would be to boot sysrescuecd, spin up gparted and use that to do recovery.

    Don't forget to burn $7 before you start for the lowend gods.

    Side note: When you up fuck you really up fuck dont you :)

    Thanked by (8)AlwaysSkint skorous bdl vimalware ouvoun Unixfy poisson yoursunny

    https://inceptionhosting.com
    Please do not use the PM system here for Inception Hosting support issues.

  • foxonefoxone OG

    @AnthonySmith said:
    A simple method would be to boot sysrescuecd, spin up gparted and use that to do recovery.

    Why gparted? How would i use it? Last time i used it, that was just a partitioning tool.
    Or does sysrescuecd have something extra?

    Side note: When you up fuck you really up fuck dont you :)

    Fuck me

    Thanked by (1)bdl
  • InceptionHostingInceptionHosting Hosting ProviderOG

    @foxone said: Why gparted? How would i use it? Last time i used it, that was just a partitioning tool.

    Or does sysrescuecd have something extra?

    gparted has whole partition recovery tools, sysrescuecd because its easy to find and you can get additional functionality to check stuff.

    https://inceptionhosting.com
    Please do not use the PM system here for Inception Hosting support issues.

  • nullnotherenullnothere OG

    @foxone said: I accidentally launched a mkfs.fat on my ext4 hard drive.

    Depending on how quickly you killed the mkfs.fat you may still be able to use some of the (hopefully) not-overwritten superblocks of the (original) ext4 FS and recover your files metadata.

    See: https://help.ubuntu.com/community/DataRecovery (look at TestDisk and see if you can quickly undo some of the damage).

    Specifically look at the -S option of mkfs.ext4 and see if that also helps you along somewhat. Now that you are working off an (disposable) image file you can boldly experiment.

  • foxonefoxone OG

    I have tried to recover using a superblock backup, but the filesystem looks so corrupted that at the end i just get an empty partition.

    I used this software: https://gist.github.com/countingpine/0fd04b99058ebc910fefc910606ae6d0 to find for superblocks on the partition, then i used fsck.ext4 -n -v -b xxxx -B 4096 /dev/sda to try to recover the filesystem. It gave me a ton of errors and corrected the filesystem.

    What's strange is that when i try to mount the disk, this shows:

    mount: sda: more filesystems detected on /dev/sda; use -t <type> or wipefs(8).

    Specifying to mount the ext4 partition -t ext4 makes it work, but i get a completely empty partition with the name of the original partition.

    At this point i have no idea what i should do :(

  • MaxKVMMaxKVM
    edited July 2020

    @foxone

    The original photo shows the error on /dev/sdb. Were the commands run on the proper device? :#

    AMD EPYC powered Performance NVMe VPS - Los Angeles, Dallas, New York, Amsterdam, Singapore | Support | Status

  • foxonefoxone OG
    edited July 2020

    @MaxKVM said:
    @foxone

    The original photo shows the error on /dev/sdb. Were the commands run on the proper device? :#

    Yes! It's changing between sda and sdb since when i disconnect and reconnect it quickly (it's a clone of the original hdd in a usb caddy) it shows up as /dev/sdb.

  • foxonefoxone OG

    So, i've tried to recover from a backup superblock after remaking the fs table using testdisk, and after a shitton of errors, the partition is completely empty. :(

    FOXO1TB: ***** FILE SYSTEM WAS MODIFIED *****
FOXO1TB: 297/61054976 files (12.5% non-contiguous), 3937795/244190208 blocks

    Should i just lose hope?

  • ehabehab

    i recommend you continue to study and gain experience. Maybe i'll ask you when i run into disk crap.

    Thanked by (3)Ympker imok yoursunny
  • AlwaysSkintAlwaysSkint OG

    Reclone and do a testdisc deep scan?

    It wisnae me! A big boy done it and ran away.
    NVMe2G for life! until death (the end is nigh)

  • foxonefoxone OG

    @AlwaysSkint said:
    Reclone and do a testdisc deep scan?

    Testdisk deep scan finds the partitions but they are unrecoverable

  • InceptionHostingInceptionHosting Hosting ProviderOG
    edited July 2020

    Well if all else fails, this probably will too but might be worth at least trying extundelete again it is available on the sysrescuecd.

    It won't recover the partition but maybe some of the files.

    https://inceptionhosting.com
    Please do not use the PM system here for Inception Hosting support issues.

  • comicomi OG

    Superblock recovery in theory should work, other than that you might want to check what fsck says about the journal, that may yield some hints.

    In theory you have everything you need:
    filenames are basically data on disc, they are maps ("filename", inode_number)
    knowing inode_number you can get the inode itself as long as you have the superblock
    if you find inode, you find the data.

    You can see the details here:
    https://www.kernel.org/doc/html/latest/filesystems/ext4/dynamic.html#finding-an-inode

    Automatic tools do that already, so if they fail there must be something else going on, like a problem with journal...

    Thanked by (1)vimalware
  • foxonefoxone OG

    @comi said:
    Superblock recovery in theory should work, other than that you might want to check what fsck says about the journal, that may yield some hints.

    In theory you have everything you need:
    filenames are basically data on disc, they are maps ("filename", inode_number)
    knowing inode_number you can get the inode itself as long as you have the superblock
    if you find inode, you find the data.

    You can see the details here:
    https://www.kernel.org/doc/html/latest/filesystems/ext4/dynamic.html#finding-an-inode

    Automatic tools do that already, so if they fail there must be something else going on, like a problem with journal...

    If the journal is broken, what does that cause? There might be a problem with it indeed since the hard driwe was turned off suddently.

  • comicomi OG

    @foxone said:

    @comi said:
    Superblock recovery in theory should work, other than that you might want to check what fsck says about the journal, that may yield some hints.

    In theory you have everything you need:
    filenames are basically data on disc, they are maps ("filename", inode_number)
    knowing inode_number you can get the inode itself as long as you have the superblock
    if you find inode, you find the data.

    You can see the details here:
    https://www.kernel.org/doc/html/latest/filesystems/ext4/dynamic.html#finding-an-inode

    Automatic tools do that already, so if they fail there must be something else going on, like a problem with journal...

    If the journal is broken, what does that cause? There might be a problem with it indeed since the hard driwe was turned off suddently.

    if you mounted it read/write there is new journal already, so the question is whether the journal was broken. If it was and some of the metadata was corrupted it is as good as lost.

    Did fsck or dmesg sad anything about it?
    Did fsck or dmesg said anything else useful?
    Did you make a dd copy before trying to restore?
    Is it "completely" empty? No lost+found?
    Did you try the -S?

  • foxonefoxone OG

    @comi said:

    if you mounted it read/write there is new journal already, so the question is whether the journal was broken. If it was and some of the metadata was corrupted it is as good as lost.

    Did fsck or dmesg sad anything about it?
    Did fsck or dmesg said anything else useful?

    Just that the superblock is not valid

    Did you make a dd copy before trying to restore?

    Of course, i'm working on that

    Is it "completely" empty? No lost+found?

    Yes.

    Did you try the -S?

    Yes

    Anyway, i'm now scanning it with R-Linux. It looks like pretty much every software is fooled by the fact that it was a GPT reformatted as MBR with vfat - so they all find the fat partition and say "hey that's working, what's the issue then?". I'll try later to zero a couple of megabytes at the beginning to destroy the partition table and fat partition definitely.

    It could find some superblocks and directories (along with real file names) correctly. So i guess the data is still there - i just need a way to find it.

  • nullnotherenullnothere OG

    @foxone said: I'll try later to zero a couple of megabytes at the beginning to destroy the partition table and fat partition definitely.

    If you have any options try to guide/force to look for an EXT4 FS (and related metadata). Depending on how much got written by the mkfs.fat, you still have the data blocks and possibly a fair number of the super blocks from which you can try to recover the data.

    And of course you're working with a disposable image - so that's really comforting to try multiple ways without loosing anything (more).

    There will be a fair bit of trial and error required (unless you get lucky) and even if you do get the actual file contents, figuring out the file name/metadata will likely be tricky. If the above diagram seems reasonably accurate, then you probably can recover most of the data even if painfully/manually.

  • foxonefoxone OG

    @nullnothere said:

    @foxone said: I'll try later to zero a couple of megabytes at the beginning to destroy the partition table and fat partition definitely.

    If you have any options try to guide/force to look for an EXT4 FS (and related metadata). Depending on how much got written by the mkfs.fat, you still have the data blocks and possibly a fair number of the super blocks from which you can try to recover the data.

    And of course you're working with a disposable image - so that's really comforting to try multiple ways without loosing anything (more).

    There will be a fair bit of trial and error required (unless you get lucky) and even if you do get the actual file contents, figuring out the file name/metadata will likely be tricky. If the above diagram seems reasonably accurate, then you probably can recover most of the data even if painfully/manually.

    The scan is still ongoing. I will report once it's finished. But if it can recover the files I might as well just buy the license for 79$ and call it a day. (free version has a 256kb limit per file)

  • comicomi OG

    @foxone said: the fact that it was a GPT reformatted as MBR with vfat

    mkfs does not touch partition table, how do you know it's a fact?

    I have recreated your situation:

    # fallocate -l 10G wdtfs.img
# losetup /dev/loop1 wdtfs.img 
# fdisk /dev/loop1
# mkfs.ext4 /dev/loop1p1
# mount /dev/loop1p1 /mnt/
# mkdir -p {a..c}/{a..c}
# touch {a..c}/{a..c}/{a..c}
# umount /mnt
# cp wdtfs.img wdtfs1.img
# mkfs.fat /dev/loop1p1
# mke2fs -n /dev/loop1p1
# e2fsck -b 819200 /dev/loop1p1
# mount -t ext4 /dev/loop1p1 /mnt
# ll /mnt

  total 12K
  drwxr-xr-x  3 root root 4.0K Jul 14 13:35 .
  drwxr-xr-x 19 root root 4.0K Apr  8 01:37 ..
  drwx------  5 root root 4.0K Jul 14 13:35 lost+found

# ll /mnt/lost+found/

  total 20K
  drwxr-xr-x 5 root root 4.0K Jul 14 13:15 '#131073'
  drwxr-xr-x 5 root root 4.0K Jul 14 13:15 '#393217'
  drwxr-xr-x 5 root root 4.0K Jul 14 13:15 '#524289'
  drwx------ 5 root root 4.0K Jul 14 13:35  .
  drwxr-xr-x 3 root root 4.0K Jul 14 13:35  ..

# ll /mnt/lost+found/#131073/

  total 20K
  drwxr-xr-x 5 root root 4.0K Jul 14 13:15 .
  drwx------ 5 root root 4.0K Jul 14 13:35 ..
  drwxr-xr-x 2 root root 4.0K Jul 14 13:16 a
  drwxr-xr-x 2 root root 4.0K Jul 14 13:16 b
  drwxr-xr-x 2 root root 4.0K Jul 14 13:16 c

    I have lost filesystem root directory but everything else is intact.
    Maybe even it is still there somewhere, but at this point it would be easier to manually identify it's contents unless you don't have hundreds of entries right at the top level.
    If you are not getting the same then either there is something else going on or you are not executing it right.

  • foxonefoxone OG
    edited July 2020

    Then i guess something else is going on and i have no idea what. The exact command i used (got it from my history) was:

    526 sudo mkfs.fat /dev/sda

    This is a sector that looks like an ext4 superblock.

    However, when trying to use it...

    Am i doing something wrong? Am i not supposed to use sector numbers?

  • comicomi OG

    first, no, you are not supposed to use sector numbers

    mke2fs -n /dev/loop1p1
mke2fs 1.45.6 (20-Mar-2020)
/dev/loop1p1 contains a vfat file system
Proceed anyway? (y,N) y
Creating filesystem with 2621179 4k blocks and 655360 inodes
Filesystem UUID: a815893f-b3f2-4491-b826-75d426613cc2
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

    you need to use one of these

    second, why e2fsck -n? By default you should agree with fsck unless you know better

  • foxonefoxone OG

    @comi said:
    first, no, you are not supposed to use sector numbers

    mke2fs -n /dev/loop1p1
mke2fs 1.45.6 (20-Mar-2020)
/dev/loop1p1 contains a vfat file system
Proceed anyway? (y,N) y
Creating filesystem with 2621179 4k blocks and 655360 inodes
Filesystem UUID: a815893f-b3f2-4491-b826-75d426613cc2
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

    you need to use one of these

    second, why e2fsck -n? By default you should agree with fsck unless you know better

    Because i don't want to touch the image yet, i just want to check if it recognizes the superblock or not.

  • comicomi OG

    @foxone said:

    @comi said:
    first, no, you are not supposed to use sector numbers

    mke2fs -n /dev/loop1p1
mke2fs 1.45.6 (20-Mar-2020)
/dev/loop1p1 contains a vfat file system
Proceed anyway? (y,N) y
Creating filesystem with 2621179 4k blocks and 655360 inodes
Filesystem UUID: a815893f-b3f2-4491-b826-75d426613cc2
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

    you need to use one of these

    second, why e2fsck -n? By default you should agree with fsck unless you know better

    Because i don't want to touch the image yet, i just want to check if it recognizes the superblock or not.

    ah, oki oki

  • foxonefoxone OG
    edited July 2020

    Halfway through the disk these are the browsable partitions

  • comicomi OG
    edited July 2020

    526 sudo mkfs.fat /dev/sda

    maan, I'm so sorry, I completely missed that you created filesystem over the entire disk not the partition - I completely ignored that

    yup, you destroyed or damaged the partition table.

    In theory if you can create new partition table exactly the same as it was before you can recover every partition one by one.

    More specificaly same number of partitions in the same order of the exact same size.

  • foxonefoxone OG

    @comi said:

    526 sudo mkfs.fat /dev/sda

    maan, I'm so sorry, I completely missed that you created filesystem over the entire disk not the partition - I completely ignored that

    yup, you destroyed or damaged the partition table.

    In theory if you can create new partition table exactly the same as it was before you can recover every partition one by one.

    More specificaly same number of partitions in the same order of the exact same size.

    Sadly i have no idea how the old partition was made. This disk was my main disk (thus had /, /boot, swap and /home partitions), then i moved it to be a data disk (deleting /boot and /, but keeping /home and enlarging it to occupy the residual space), then i finally got rid of the swap partition and thus enlarged the partition to the full disk size. However, i have no idea if it was actually full disk or a couple of megs in front of it or what.

    R-Linux is finding a lot of inodes and master blocks, but they all seem from android images i had stored on the drive, or the old / and /boot partitions. It looks like the original 1tb partition disappeared out of nowhere for some reason...

  • YmpkerYmpker OGContent Writer
    edited July 2020

    @foxone How many furries were lost?

    Thanked by (1)vimalware

    Ympker's VPN LTD Comparison, Uptime.is, Ympker's GitHub.

  • foxonefoxone OG

    @Ympker said:
    @foxone How many furries were lost?

    None. I funnily keep those on a couple 12tb drives in my server

    Thanked by (1)Ympker
  • yoursunnyyoursunny OG

    @comi said:
    In theory if you can create new partition table exactly the same as it was before you can recover every partition one by one.

    More specificaly same number of partitions in the same order of the exact same size.

    So the lesson is that we should email ourselves the partition table, so that we can recover data even if some files are not yet in the backup.

    HostBrr aff best VPS; VirmAche aff worst VPS.
    Unable to push-up due to shoulder injury 😣

  • foxonefoxone OG

    @yoursunny said:

    @comi said:
    In theory if you can create new partition table exactly the same as it was before you can recover every partition one by one.

    More specificaly same number of partitions in the same order of the exact same size.

    So the lesson is that we should email ourselves the partition table, so that we can recover data even if some files are not yet in the backup.

    This so much. I've just done a dd of 1gb of all my drives to keep safe just in case.

    And btw, i still haven't recovered my data. I just removed the drive and i'll keep it there until i find time and will to try more.

Sign In or Register to comment.

© LowEndSpirit 2024

ipv6 ready