Removing Invisible Files on Mac OS X

Typical usage scenarios:

Preparing files for upload to a non-Mac system
Preparing files for upload to a server
Preparing files for burning onto multi-platform CDs
Preparing files for use with a tool that does not know about Mac OS X hidden files

What are invisible files?

Invisible files on Mac OS X come in two different flavors: Unix-style invisible files and Mac OS 9-style invisible files.

On Unix systems every file whose name starts with a full stop (e.g., ".secret") is considered to be invisible. Mac OS 9-style invisible files rely on a meta-data flag that is part of the HFS/HFS+/Mac OS X Extended (Journaled) file system used by Mac OS X; these files can have any name.

Invisible files are simply files that are not usually displayed by the Finder and in the various file dialogs; in all other respects they are perfectly "normal" files.

Invisible files exist because they usually contain "operating system stuff" that it is a bad idea for the user to mess around with.

Mac OS 9 used invisible files for a number of purposes, but the major one that still affect Mac OS X installations today is its use of "Icon\r" files. These files contain the custom icon for the folder in which they are placed. In other words, whenever you see a folder that has a non-standard icon, chances are that there will be an invisible "Icon\r" file inside of it.

Unix systems make heavy use of invisible files for all kinds of purposes. A lot of configuration information is kept in invisible plain text files, such as ".profile", ".cshrc", etc. Removing these configuration files can have disastrous effects and may even prevent your system from re-booting properly.

Many third-party unix applications, such as version control systems (git, mercurial, subversion, cvs, etc.) use invisible folders and files to maintain state information.

The most commonly used hidden files in Mac OS X are ".DS_Store" and ".localized" files.

Pretty much every folder on your hard disk is likely to contain a ".DS_Store" file. Its use by Apple is largely undocumented, but it is commonly believed to contain Finder-related information about the other files in the folder that contains it. This appears to include at least positioning information (the location at which the icon appears in the Finder's "as Icons" view), what type of document each file represents, file comments, file color, etc. The main purpose of this file appears to be to boost the Finder's performance when displaying the folder contents to the user by acting as a "cache".

The ".localized" files appear to simply contain user language dependent data. This allows the Finder to display localized folder (and file?) names. A good example of this is the "Applications" folder, which on the file system is called "Applications" but magically appears to the user with a localized name (e.g. "Anwendungen").

Why would I want to remove invisible files?

There are a number of reasons why you might want to get rid of some or all invisible files within a folder hierarchy:

export files to a non-Macintosh system
export files to another Macintosh system
prevent multi-platform tools that are not aware of the Mac's invisible files from "chocking" on them
save disk space
...

All those hidden files can cause a mayhem when used in a context other than with a Mac-only application on your own Mac. For one thing if they make their way onto a non-Macintosh system, they may no longer be hidden and all those ".DS_Store" files will pollute your folder hierarchy. The Finder is prone to putting a ".DS_Store" file into every directory that you look at, even if you only have a quick browse on a Windows NTFS volume. The multi-platform CDs that you burn are also likely to have those files appear in them..

Worse than just polluting but not otherwise impairing non-Macintosh systems, invisible files may actually prevent you from transferring files onto those systems in the first place. The so-called SMB remote file sharing with Windows machines for instance is prone to "chocking" on hidden "Icon\r" files in particular. After transferring 10GB of data over the connection, you'll suddenly be presented with an error message "..\..\..\Icon could not be found. Aborting.".

Many professional tools that have been migrated from other platforms (Unix, Windows, etc.) also simply do not know about Mac OS X's invisible files and this can produce all kinds of problems. Version control systems (such as Subversion or CVS), for instance, will not ignore the hidden ".DS_Store" files and will consider that the folder has changed each time those files are updated (potentially each time you point the Finder at them!). Once they are under version control, they can cause further mayhem especially in multi-user environments.

As mentioned before, the ".DS_Store" files act as a cache for the Finder and if for any reason (there appear to be plenty!) they get out of sync with the real contents of the folder, they can lead to stale information about your files being displayed. Removing the ".DS_Store" file will force the Finder to update its information (you will however lose file comments, colors, positioning, etc information if you do this!).

Precautions

It pays to be cautious when using this feature. We recommend that you:

Make a backup of your folder hierarchy and remove the invisible files on the copy and not the original
NEVER mess around with the hidden files in the system folder
NEVER mess around with the hidden files at the root of your home folder