[WinMac] UNSOLVED MYSTERY: Defragmenting Services for Macintosh volumes


Daniel L. Schwartz(expresso[at]snip.net)
Sun, 11 Apr 1999 00:18:51 -0500


UNSOLVED MYSTERY: Defragmenting Services for Macintosh


        Good afternoon!

        While perusing the "Win NT SFM Unsolved Mysteries" page at:
 <http://www.macwindows.com/NTunsolv.html#defrag>, I came across the solution by looking at the possible causes. Before continuing, it's important to note that the issues of defragmentation and file corruption are closely interrelated.

        With that in mind, let's start out with the basics as to the 3 general ways dual Mac files can be stored on a non-MacOS server:

        1) Via native NTFS (Native Transactional File System) and SFM, splitting the forks into two streams of data. As long as the files stay on NTFS volumes, the streams will be preserved; so then everything will remain all hunky-dory... Diskeeper will transparently defragment the files per its normal operation [More below];


        2) Via combining the two file forks into a single fork file, such as the way MacBinary works. Defragmenters will work as they would on any DOS or Unix file; but this method involves extra CPU overhead and disk space on the Mac to split the 2 forks apart again so they can be used by the MacOS;


        3) Via the PC Exchange Control Panel’s method, of storing the Resource Forks in a separate hidden (to the MacOS) files. The Desktop files are also stored in hidden (to the MacOS) files. According to the support at Thursby, DAVE also works in this way. [Lots more below].


>>>>
Defragmenting Services for Macintosh volumes
December 15, 1998 -- A reader told us that Executive Software (makers of Diskeeper) told him that Diskeeper can't defragment NT Services for Macintosh volumes, and that NT itself doesn't support it. Another reader had partial success with Diskeeper:

December 17, 1998
Gary Peterson

"We are using Diskeeper 3.0 on NTS4 with SFM. We have 2 SCSI cards with 4 SFM drives attached to each card. On one card, Diskeeper crashes every time we try to defrag a drive. I have to come in on a weekend and turn off SFM to defrag drives. On the other card, no problems. The autoscheduler kicks in every 12 hours and works fine on all 4 drives. No problems in 6 months."


<<<<


        More on Method 1: Native SFM/NTFS support. First, there was a fundamental shift in NTFS between NT4 and earlier versions: Prior to NT4, Executive Software licensed the NTFS source code from Microsoft and created special API’s to perform defragmenting. However, starting with NT4, Microsoft incorporated these API’s directly into the NTFS system... Now, all any app needs to do is call these API's and voìla - Instant NTFS defragmenter. And, since NTFS considers the two streams as a single file, then everything reads, writes, and defragments simultaneously. It is important to note that NTFS Cluster size is critical in 2 ways:

        A) The defragmenting API’s only work with cluster sizes of 512, 1024, 2048, and 4096 bytes;

        B) 512 byte cluster sizes are more easily fragmented due to the 1024 byte file record size. This limits the effective cluster size range to 1024, 2048, and 4096 bytes. This is important because the CONVERT utility (FAT -> NTFS conversion utility) will only create 512 byte clusters... Pay attention to this when performing an NT installation on a fresh drive!

        Note: AFP over IP (Apple Filing Protocol over IP) "husbands" Method 1 but can extend it to FAT partitions as well: Essentially, it is a hybrid of Methods 1 and 2; and normal defragmentation methods and corruption warnings apply.


        More on Method 3: Splitting the Mac files into two discrete files. The advantage to this method is that one can store Mac files on any remote mountable volume - FAT12, FAT16,§ FAT32, as well as NTFS, as well as on 95/98 and NT/Workstation as well as NT/Server. The downside is that NT will see these hidden files as separate files, as opposed to a single dual stream file as in SFM/NTFS. An additional downside is that the 2 files required to make up a dual fork Mac file can become widely separated when defragmenting a partition, partially defeating the very purpose of defragmenting. An additional issue arises when these file pairs are stored on a  Native Transactional File System  (emphasis added) volume: NTFS will not say a file has been written until it has been actually written... But what happens if your server crashes when one file fork has been written to its' file, but not the other? The two file forks now all of a sudden are corrupted due to "forking," i.e the versions are out of sync.


§ NOTE: FAT12, FAT16, and *I think* FAT32 drives can be directly mounted on a Mac via PC Exchange as well.

§§ BEWARE: Both FAT12 and FAT16 are used in DOS floppies!

 -------------

REFERENCES:
 <http://www.be.com> explains how the BeOS "journaling" (but not streaming) file system works, and is an effective primer in that it is similar to how NTFS handles its transactions;

 <www.diskeeper.com> and <http://www.sysinternals.com> explain how defragmenting works in NT; and also is the source for various free tools;

 <http://www.cyan.de> and <http://www.teamasa.com/cyan2.htm> explain how AFP over IP works.


 -------------



        This note may be a bit long; but hopefully this will not only explain the way NT effectively defragments files but also explain how these same Mac files can become corrupted.




        Yours truly,
        Daniel L. Schwartz,
        Electrical Engineer.

        Dan's Macintosh Consulting
        239 Great Road
        Maple Shade, NJ 08052

        609-642-7666

 -----------------------------------------------------------------

        <mailto:expresso@snip.net, Dan@Hemnet.com>
       
        ALTERNATE: <mailto:expresso@workmail.com>

                Webmaster for   <http://www.Faulknerstudios.com>,
                                        <http://www.BrakeAndGo.com>

        **Your Corel Solution Partner**

                **Your UltraBac Solution Source**

 -----------------------------------------------------------------
* Windows-MacOS Cooperation List *


This archive was generated by hypermail 2.0b2 on Sat Apr 10 1999 - 22:25:23 PDT