Register FAQ SearchLogin
Tuxera Home
View unanswered posts | View active topics It is currently Thu Oct 02, 2014 16:28



Post new topic Reply to topic  [ 10 posts ] 
Data loss / corruption with ntfs-3g 4.4 for osx 
Author Message

Joined: Fri Aug 14, 2009 10:31
Posts: 5
Post Data loss / corruption with ntfs-3g 4.4 for osx
Hello everybody,

I posted this problem yesterday in the comments of macntfs-3g.blogspot.com, but now found that this is probably the more appropriate place.

I had some very disturbing experiences with ntfs-3g for osx (version 4.4) last week, and I was wondering if anybody has an idea how to get to the source of the problem. Basically a folder on my ntfs partition got completely corrupted (twice, on two seperate days) and I have not dared to use it since.

How the problem occured:
I was id3-tagging my mp3 collection (on an ntfs-partition, in osx, using musicbrainz picard), when my macbook pro went into sleep because it ran out of battery. After that (I might have rebooted, not sure anymore), the folder in which I was working, somehow "lost" the majority of contents. Using chkdisk in windows brought up a lot of chkdsk.00X files in that directory. It was not a big deal, I just copied the stuff back from a backup. I thought the "forced sleep mode" had caused the problem.

However, two days later, I continued to work in that directory (id3-tagging). I shut down the computer properly, and the day after, one of the folders again completely disappeard, and another folder had become a zero-KB file. I was quite shocked, since I was sure that I haven't done anything "wrong".

If anybody has any idea how to find out what the problem is (and how to prevent it in the future...), I would really appreciate it.

My partition setup, in case that helps anything (macbook pro 13", about 2 months old, 500GB WD HD, osx 10.5.7):

1st partition: efi
2nd partition: HFS+, OSX system
3rd partition: NTFS, data
4th partition: NTFS, WinXP system

all primary partitions.

It happened about a week ago, I just avoided using it since, are there any specific log files I can look at to check what had gone wrong? Or do I maybe have a bad drive (although the HFS+ partition works flawless so far)? Or can I use some kind of monitoring software to get more information when it happens the next time?

I'd be happy to dig a bit deeper, if you have any pointers for me...


Fri Aug 14, 2009 10:47
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1148
Post Re: Data loss / corruption with ntfs-3g 4.4 for osx
Hi,

Quote:
I'd be happy to dig a bit deeper, if you have any pointers for me...

Before you try any fixing, can you build a metadata image, as described in viewtopic.php?f=4&t=1263 and make it available, so that I can search for clues about what went wrong.
Just note I will probably not provide you with a fix, my only aim is to determine the root cause and help making this not happen again.

Regards

Jean-Pierre


Fri Aug 14, 2009 11:50
Profile

Joined: Fri Aug 14, 2009 10:31
Posts: 5
Post Re: Data loss / corruption with ntfs-3g 4.4 for osx
Thanks for the reply. I'll try to get that metadata image tomorrow when I get home. But note that after the problem has occured the second time, the partition has been mounted several times (although I think I have not written any files on it since, I might have deleted the zero-kb file), so I'm not sure how relevant the information still is. If it's not useful I'll try to reproduce the problem again (i.e. by doing similar work in that folder).

I'm not trying to fix or recover that data, I'd be more than happy to just find out what had gone wrong (and if a switch to FAT32 or some other outdated filesystem would prevent this in the future...).

Btw, what's the recommended method to check a partition for bad sectors (or other drive hardware failures)? Chkdsk in windows? I guess I should do that first before looking for software problems.


Fri Aug 14, 2009 13:03
Profile

Joined: Fri Aug 14, 2009 10:31
Posts: 5
Post Re: Data loss / corruption with ntfs-3g 4.4 for osx
Hi,

I tried to get that metadata image, not quite successful though. I did the following:

First I tried to use that tool, which gave me the following error:
Quote:
Executing command: 'sudo' '-S' '/bin/bash' '-c' '(./ntfs-3g.clone -mso - "/dev/disk0s3" | bzip2 > "/sergej_ntfs_metadata.img.bz2"; test "${PIPESTATUS[*]}" == "0 0" && (exit 0)) || (exit 1)'...
Password:
ntfs-3g.clone 2009.4.4
Error opening '/dev/disk0s3': Resource busy


Then I noticed that the partition was mounted with the standard OSX NTFS driver (however, the WinXP system partition was mounted with NTFS-3G). So I unmounted and tried to mount it manually:
Quote:
sudo /usr/local/bin/ntfs-3g /dev/disk0s3 /Volumes/DATA

$MFTMirr does not match $MFT (record 0).
Failed to mount '/dev/disk0s3': Input/output error
NTFS is either inconsistent, or there is a hardware fault, or it's a
SoftRAID/FakeRAID hardware. In the first case run chkdsk /f on Windows
then reboot into Windows twice. The usage of the /f parameter is very
important! If the device is a SoftRAID/FakeRAID then first activate
it and mount a different device under the /dev/mapper/ directory, (e.g.
/dev/mapper/nvidia_eahaabcc1). Please see the 'dmraid' documentation
for more details.


So I booted into windows and ran chkdsk /f
Quote:
C:\>chkdsk D: /F
The type of the file system is NTFS.
Volume label is DATA.

CHKDSK is verifying files (stage 1 of 3)...
Deleted corrupt attribute list entry
with type code 144 in file 11133.
File verification completed.
CHKDSK is verifying indexes (stage 2 of 3)...
Correcting error in index $I30 for file 11133.
Correcting a minor error in file 11133.
Deleting index entry SoCalled Chaos in index $I30 of file 8107.
Deleting index entry Fight for Your Mind in index $I30 of file 8110.
Deleting index entry cover.jpg in index $I30 of file 9836.
Deleting index entry Alchemy_ Dire Straits Live in index $I30 of file 10083.
Deleting index entry Making Movies in index $I30 of file 10083.
Deleting index entry Money for Nothing in index $I30 of file 10083.
Deleting index entry cover.jpg in index $I30 of file 10314.
Index verification completed.
CHKDSK is recovering lost files.
Recovering orphaned file Alanis Morissette (8107) into directory file 8058.
Recovering orphaned file Ben Harper (8110) into directory file 8058.
Recovering orphaned file Emir Kusturica No Smoking Orchestra (9845) into direct
ory file 8092.
Recovering orphaned file Gotan Project (9855) into directory file 8092.
Recovering orphaned file Mark Knopfler (9882) into directory file 8092.
Recovering orphaned file Moby (9888) into directory file 8092.
Recovering orphaned file Mr. Bungle (9891) into directory file 8092.
Recovering orphaned file Muse (9892) into directory file 8092.
Recovering orphaned file US3 (9913) into directory file 8092.
Recovering orphaned file .DS_Store (10545) into directory file 8092.
Recovering orphaned file unknown (10739) into directory file 8108.
Recovering orphaned file Various Artists (10754) into directory file 8092.
CHKDSK is verifying security descriptors (stage 3 of 3)...
Security descriptor verification completed.
Inserting data attribute into file 11133.
Correcting errors in the Master File Table (MFT) mirror.
Correcting errors in the master file table's (MFT) BITMAP attribute.
Correcting errors in the Volume Bitmap.
Windows has made corrections to the file system.

278803435 KB total disk space.
10087976 KB in 9358 files.
3836 KB in 1706 indexes.
0 KB in bad sectors.
85695 KB in use by the system.
65536 KB occupied by the log file.
268625928 KB available on disk.

4096 bytes in each allocation unit.
69700858 total allocation units on disk.
67156482 allocation units available on disk.

C:\>



then back to OSX and now the partition was mounted with NTFS-3G. So I tried again to create a meta-image:
Quote:
Executing command: 'sudo' '-S' '/bin/bash' '-c' '(./ntfs-3g.clone -mso - "/dev/disk0s3" | bzip2 > "/sergej_ntfs_metadata.img.bz2"; test "${PIPESTATUS[*]}" == "0 0" && (exit 0)) || (exit 1)'...
Password:
ntfs-3g.clone 2009.4.4
Failed to read lock '/dev/disk0s3': Resource temporarily unavailable
Error opening '/dev/disk0s3': Resource temporarily unavailable
ERROR(35): Opening '/dev/disk0s3' as NTFS failed: Resource temporarily unavailable


Tried to unmount and remount, that did not change anything. Any ideas what I'm doing wrong?

regards,
sergej


Sat Aug 15, 2009 11:47
Profile
Tuxera Developer

Joined: Sat Oct 06, 2007 09:24
Posts: 530
Post Re: Data loss / corruption with ntfs-3g 4.4 for osx
sergej wrote:
Hi,

I tried to get that metadata image, not quite successful though. I did the following:

First I tried to use that tool, which gave me the following error:
Quote:
Executing command: 'sudo' '-S' '/bin/bash' '-c' '(./ntfs-3g.clone -mso - "/dev/disk0s3" | bzip2 > "/sergej_ntfs_metadata.img.bz2"; test "${PIPESTATUS[*]}" == "0 0" && (exit 0)) || (exit 1)'...
Password:
ntfs-3g.clone 2009.4.4
Error opening '/dev/disk0s3': Resource busy

My bad, I omitted one thing in the FAQ, that you have to unmount the partition before trying to extract metadata from it. You're getting "resource busy" because it is mounted.
(I added it to the FAQ just now, as point number 4.)

- Erik


Sat Aug 15, 2009 11:52
Profile

Joined: Fri Aug 14, 2009 10:31
Posts: 5
Post Re: Data loss / corruption with ntfs-3g 4.4 for osx
Ok, now it worked. I hope that chkdsk /f in windows didn't destroy all the useful information. But if I remember correctly, when extracting metadata didn't work using the standard osx ntfs-driver, I also tried it with the unmounted device, and that didn't work either (I think I got something similar as "$MFTMirr does not match $MFT").

Anyways, you can find the file here (6.9 MB):
http://rapidshare.com/files/267602034/s ... g.bz2.html

If I should upload the file somewhere else just let me know.

regards,
sergej


Sat Aug 15, 2009 12:07
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1148
Post Re: Data loss / corruption with ntfs-3g 4.4 for osx
Hi,

Quote:
Ok, now it worked. I hope that chkdsk /f in windows didn't destroy all the useful information.

It probably has, judging from your previous post.
Quote:
Anyways, you can find the file here (6.9 MB)

I got it, but it is probably not very useful.

Regards

Jean-Pierre


Sat Aug 15, 2009 13:09
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1148
Post Re: Data loss / corruption with ntfs-3g 4.4 for osx
Hi,

It would be useful to know what you were doing. I see mostly actions on directories in your recent actions, were you renaming directories or moving your files from a directory to another ?

The latest actions, on Aug 6th at 22:06 UTC, were on directories "Oasis" and "Oasis/The Masterplan", now moved to dir00000.chk Do you remember what these actions were ? Was it a shutdown, and if so, did you umount properly ?

Most fixes by chkdsk were made on directories, which were last changed on Aug 5th by 23:00 UTC (was it a shutdown ?) or on Aug 6th (at 19:50, 20:14, 21:44 and 22:07). Did you shutdown several times ?

Do you remember when you shut down or unmount ?

Regards

Jean-Pierre


Sat Aug 15, 2009 15:14
Profile

Joined: Fri Aug 14, 2009 10:31
Posts: 5
Post Re: Data loss / corruption with ntfs-3g 4.4 for osx
I'll try to answer your questions:

jpa wrote:
It would be useful to know what you were doing. I see mostly actions on directories in your recent actions, were you renaming directories or moving your files from a directory to another ?

The latest actions, on Aug 6th at 22:06 UTC, were on directories "Oasis" and "Oasis/The Masterplan", now moved to dir00000.chk Do you remember what these actions were ? Was it a shutdown, and if so, did you umount properly ?


I was using a tool to clean up the id3-tags of some mp3 albums (musicbrainz picard). It also renames the files according to a pattern (Track Artist - Title.mp3), creates a directory structure like "Artist/Album" and when done moves them to a different dir. I did this on several albums (around 20 maybe, roughly guessed). So most of the actions were probably small updates in the id3-tags and moving/renaming quite a lot of files.

jpa wrote:
Most fixes by chkdsk were made on directories, which were last changed on Aug 5th by 23:00 UTC (was it a shutdown ?) or on Aug 6th (at 19:50, 20:14, 21:44 and 22:07). Did you shutdown several times ?

Do you remember when you shut down or unmount ?


I almost always shutdown before I go to bed (between 23.00 and 1.00 CEST, which translates to 21 to 23 UTC). I never unmounted manually. I don't think I've shutdown four times, it is much more likely that I just closed the lid and the laptop went to sleep (I guess it does not unmount then?), but I cannot tell you for sure unfortunately, it's too long ago.

I hope this helps a bit. If this is not enough information I'll try to reproduce the problem, as it has happened twice already (doing the same kind of work) there is a good chance it will happen again. Then I would write down exactly what I was doing.

regards,
sergej


Sat Aug 15, 2009 19:10
Profile
NTFS-3G Lead Developer

Joined: Tue Sep 04, 2007 17:22
Posts: 1148
Post Re: Data loss / corruption with ntfs-3g 4.4 for osx
Hi,

Quote:
it is much more likely that I just closed the lid and the laptop went to sleep

So, until this is clarified, you should wait a couple of minutes before closing the lid or shutting down.

Thank you for you cooperation.
Regards

Jean-Pierre


Sun Aug 16, 2009 09:58
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 


Who is online

Users browsing this forum: Google [Bot], Yahoo [Bot] and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Original forum style by Vjacheslav Trushkin.