intro
links in general
links on windows
some link-making programs
web links
Intro
Some time ago I came across Gobolinux. Gobolinux is a Linux distribution which attempts to make the Linux folder structure more like Windows', by organizing the crazy *nix folder hierarchy into three groups, for user, program, and system files.
Gobolinux is able to do this with "symbolic links". In a nutshell, symbolic links are files that contain a path to another file or folder (or maybe something else) somewhere else on your hard drive; and anything trying to read or write the link will automatically read or write the resource to which the link points. The OS handles it all and the user program doesn't need to worry about whether a file is a plain file or a link to another file.
Symbolic links - and others - have been on *nixes for, well, since the Stone Age I think. And they're damn useful. Gobolinux is a perfect example of the amazing system hacks that are possible using links. (Redirection is King.)
On Windows it's another story.
If you're like my friend Kurt, you might say: "Yeah, it's another story all right. 'Cause Windows doesn't have symbolic links!" Now Kurt, and those of you like him - well-meaning, but a bit hasty (in other words, a Mac person) - actually don't have the real story. The real story is that Windows has had symbolic links since Windows 2000, or more specifically, NTFS 5.0. This used to be called "junction point" functionality, and was renamed "reparse point" functionality by the time Windows XP came out.
To some of you, the age of this functionality, or even its existence, may come as a surprise. This is itself not a surprise. More than four years after this functionality became available, it seems Microsoft still has barely documented it, and hasn't exposed it in Windows except in the most cursory way. It is still a far bigger chore than necessary finding information about this functionality in Microsoft's knowledge base. Anyone who want to learn about it can only resort to scrounging across the web. Links on Windows exist, but they are useable only due to the efforts of a handful of third party tools developers. That's the Windows story on symlinks.
This article is an attempt to present a comprehensive and authoritative user's view of Windows links, particularly symlinks. Of course, this is something that only the folks at Microsoft can really provide. But I think I can say one thing conclusively: when a functionality isn't really documented, even the most widely accepted wisdom about it can turn out to be wrong.
And now..
Re-organizing Windows' File System to Mimic UNIX'
...just kidding.
Links, Generally
There are two general types of link: hard links and soft or symbolic links.
To understand what a hard link really is, you have to know a small something about filesystems.
We're used to browsing through our files with a file manager, like Explorer. We typically understand this process as looking "in" containers - folders - for contents - files usually.
In fact this is a completely inappropriate metaphor/model for the organization of your hard disk. In reality, the folders that you see in the manager are tables, lists of resources such as files, other "folders" (more lists of resources), or what have you. When you click on a folder in your file manager, the OS presents you with the list of items from the table, as its "contents".
A hard link is any entry in any of these tables. Every item you're looking at in your file manager is not the resource itself, as we typically think; not the folder, not the file, but a statement as to where the OS can find that folder or file on disk. Oh, and some throwaway info, like the file name, attributes, you know. Non-essentials.
Now it's only a short step from knowing this to wondering: what's to stop me from putting a _second_ filesystem reference to some file in a table completely different from the first - that is, in another folder? Answer: Nothing. Entries in a table are not actually restricted according to the "container/contents" metaphor; all the folders on your hard drive can "contain" all the contents on your entire drive (or entire volume) simultaneously. Now you're starting to appreciate Cyberspace for real. (Understanding that all the representations of files and folders, and other things, in your file manager - really in the entire user interface - are completely separate entities from the actual computer resources or data themselves, and treated completely differently by the OS, goes a long way to making sense of strange little things about how your computer works. I personally still get a little excited thinking about it. :) )
A symbolic link is a bit less technical. It's a file or directory (or some other filesystem entity, I suppose) that contains a path to the linked resource, in a format that the OS understands so it can automagically pass IO operations from the link to the linked resource, invisibly to the program Iing or Oing to the symbolic link.
These differences result in different behavior for hard and soft links. Hard links are unbreakable, for example; because the information you see in your file browser, such as the file name, is stored in the filesystem table, and not at the disk address referenced in the filesystem table, this information can be changed at will without risk of affecting that link or any other link. It has to be this way, or filesystems would never work to begin with! Meanwhile, symbolic links, which rely on paths, break when you move or rename the target. As files and not filesystem record entries they are more independent of the target, which is good for operations like delete.
Links On Windows
Windows hard links, from what I've read, behave pretty much like hard links in UNIX and Linux: they can only be used for files, and they cannot cross disk volume or file system boundaries; all hard links to a disk address must reside in the same partition as the disk address itself.
Windows "junctions" or directory reparse points, while some (such as the Winbolic site) may call them "hard links" or even hard-link-like, are not hard links, because: you can create a Windows symlink to a non-existent folder; you can break a symlink by renaming a symlink's target folder; and you can "fix" a symlink by creating a folder having the name targeted by a symlink. In other words, these links are clearly path-based, and not dependent on disk address or anything on that level. This actually is evident in that the symlinks work across partition boundaries, unlike hard links.
A Windows shortcut is like a symbolic link, and sometimes people speaking of symlinks call it a symlink, but it isn't strictly a symlink. A crucial characteristic of a symbolic link is that the OS will manage any calls to it as if they were calls to the link's target. Windows doesn't do that for shortcuts, so they're not real symlinks.
However, symlink-like behavior can result from wrapping a shortcut in a shell extension. This is what's going on in your "My Network Places" folder. This type of link works across volume boundaries and across networks. Because it is a shell extension of a directory, you can delete these without fear that you will lose anything at the other end of the link.
Let's quickly look at this type of link "under the hood". We must do this with the command line, which ignores the shell extension active in these folders. Here's what happens when you ls a folder in your nethood folder:
There are files in there, a Desktop.ini and a target.lnk. Here's the inside of that Desktop.ini file:
Here's the inside of that target.lnk file, like any other shortcut file. Note the path.
It's interesting to ask whether these links could emulate symlinks. The answer is that they can. All that's required is substituting the target.lnk with one that points somewhere else. I can create a shortcut to some folder on my hard drive, rename it "target" (not "target.lnk", that will create a file called "target.lnk.lnk"), and cp it over the one you want to replace. Enter that folder again; now you see the contents of the folder targeted by the shortcut you just created. Re-name the parent folder from "LIBERRY on Penny" to something like "Thisfolder" and the icon will change to a normal folder icon, further masking its underlying link-nature. So this way of doing things has an advantage over normal symlinks, which can only be local. I haven't had any success in using this type of link to open a file.
Back to Windows symlinks. There are a few limitations to Windows symlinks, particularly when compared to their *nix brethren:
Quick And Normal Restrictions:
Only an empty folder can serve as a link, just as you can only mount a partition to an empty folder. Windows symlinks don't work across networks, but are strictly local. And Windows symlinks work only on NTFS 5.0 or later. They don't work on [variations of FAT], though NTFS symlinks can point to paths on a volume of [variations of FAT]. No biggie. We should all watch our FAT intake anyway, har har, okay cut me some slack here people. I'm the author dammit.
Windows Symlinks Are For Directories
According to many vociferous and mournful webpages I've read, Windows symlinks are for directories, not for files. And certainly there isn't any program I've found that lets you create symlinks for files. However, I'm not convinced this is the last word. The Windows SDK states clearly that a file or a directory can store a reparse point; and, file symlinks appear to be part of some kind of Windows backup or storage scheme; see the SisRestoredLink function in the SDK, for example.
(Alternately, hard links are only good for files, and that appears to be all there is to say about that.)
Moving Partitions Breaks Mounted Drive Links, So Using Symlinks Won't Necessarily Reduce The Number Of Drives You Have
There's a problem that exists with mounted drives. Moving your partitions will break mounted drive links. Why? Windows points a drive link using a volume's GUID. When you move a partition, Windows recognizes it as a new device and assigns it a new GUID. Now all of your directories mounting that volume are using an invalid GUID, so they won't work.
Now you can only create a symlink to a folder that's already accessible through a lettered volume, or one mounted to a directory. If you connect a symlink across a mounted drive, however, it'll break when you move the partition, because the mount link will break. So to have stable cross-volume symlinks, you must mount the drive to a letter. This means that if you're a partition monkey - and who isn't, honestly? - you won't break the 26-drive barrier any time soon using symlinks.
Deleting Symlinks With Normal Windows Filesystem Tools Is Not Only Dangerous But Also Bizarre
First there was bad news. Then, it looked like there was good news. But then it turned out to be ugly news.
The Bad: According to every source I've read on the web, you don't want to delete your symlinks except with a program specially designed to work with them. The reason invariably given is that if you try to delete a symlink with, for example, Explorer, you will delete all the files under the link.
The Good: For some mysterious reason I decided to test some delete operations using Explorer and rmdir. And lo! the directories and files under the links didn't delete! Could it be that people writing about symlinks all this time have just been parroting the First Symlink Hacker's warning about deleting, and it's wrong?
The Ugly: Actually, no; it's worse than you've heard. Explorer's behavior on deleting a link depends on the amount of data in the target directory. There appears to be a threshold (in my tests, between 406 and 449 megabytes, on a partition with 3.42G capacity, with 1.73G used, after deleting the 449 meg), above which, Explorer will delete the contents under your symlink when you delete the symlink. Restoring the symlink from the Recycle Bin does not restore the deleted data. But below the volume threshold, Explorer does not delete the target's data, but flags it invisibly for final deletion! This means you can delete a symlink, and then still use the data formerly under it, until you empty the Recycle Bin. Then the contents of the targeted folder will vanish. There is no warning about this behavior.
Also unexpected is that the actual target directory appears never to delete; only its contents.
Lastly, a couple tips: when creating symlinks, add an extension to the folder name, like ".sym". Believe me, until someone makes icon overlays for them, you'll be glad you did. (Note that I haven't done this in my examples, but that's because the links were temporary and not pointed at important data. In this case, do as I recommend, not as I do.) And be careful you don't create a circular hierarchy with linked directories. Unless you want to start a new spectator sport, The Never-Ending Virus Scan. Then by all means, go ahead.
Command line and Sysinternals' Junction.exe
The fundamental way to create links in Windows is via the command line. It's pretty darn simple.
There are three command line tools I know of that you can use to make a Windows symbolic link. All of them will, as we say in polite society, fit inside your nose. (For those of you not fit to be polite, that means "no bigger than a booger".) Linkd.exe (weighing in at 12K) comes with the Windows 2000 Resource Kit. In other words, it costs money. Windows XP includes a multipurpose filesystem tool called fsutil, so if you have XP, you can use that to create Windows symbolic links. Unfortunately, if you don't already have Windows XP, it too costs money. Also unfortunately, an fsutil install will take up at least 1.5 gigabytes on your hard drive (you may not have any space left over to make links).
I prefer to use Mark Russinovich's Sysinternals tool, Junction (weight, 36K). You can use it to find, create, and delete Windows symbolic links. It's easy to use, the source is available to study, it works on any Windows that supports NTFS 5.0 or greater. And it still fits inside your nose!
The command line syntax, reproduced from the junction command, is:
The -s switch causes Junction to search subdirectories of symlink directories as well as normal ones; it would be nice if there were a switch that told Junction not to search subdirectories of symlinks. (I'm personally planning on getting on this, just as soon as I learn C++.)
The dir command will tell you whether a directory is a symlink or not:
When creating a symlink, Junction will create the link folder itself, so you don't have to do this separately before using Junction.
When deleting a symlink with Junction, it will not turn the link back into a normal directory, but remove it entirely.
It's possible to create an invalid link with Junction by submitting an incorrect mount path. Suspect you've done this if you create a symlink with Junction and then immediately get errors when trying to cd to (not just read) the link directory, such as "can't find file specified" using the command line, or "not accessible, folder moved or removed" via Explorer. Basically you'll be told the directory isn't there (because it isn't). You can overwrite symlinks without deleting them first, so all you have to do to correct an invalid path is re-execute your Junction command, with the correct path this time.
Rekenwonder's Junction Link Magic
If you like GUIs better than the command line, Junction Link Magic is for you. It's as easy to use as Junction, so whichever you prefer is likely to be a matter of taste.
When you first open JLM, it scans your hard drive for junction points.
Waiting for this search process to finish can be more than a little annoying, and thankfully there's a cancel button. However, if you want to work with a previously-created link, you'll just have to suffer through it while the program finds them all, because there doesn't appear to be any way to target the search or otherwise tell JLM where to find a link you already know exists.
Once the search is over, you'll see a list of links:
Selecting a link enables the Edit and Remove buttons.
Clicking on Create opens this panel:
In the left pane, you select the folder in which you'll create a link for the folder you select in the right pane. The folder on the left has to be in an NTFS 5.0 volume and must be empty.
The contents of the new link; note the path, and the service pack uninstall folders which reside in the WINNT directory:
In contrast with Junction, deleting a link with JLM changes the directory back to a normal directory, and does not remove it.
Anton Popov's Hard Link Magic
Hard Link Magic is a double-barreled link editor: it offers a panel-based GUI; and it also integrates with Explorer, so you can create a hard link by right-clicking a file. First, the GUI.
On the left you locate the file you want to link - or if you are a stickler for exactness (like some I might mention), you locate the hard link whose disk reference you want to duplicate in another hard link. You are restricted by disk volume, shown in the drop-down box at the top. On the right you choose the location for the new hard link and the new hard link name.
Note that Hard Link Magic, just like other programs, will browse right through Windows symlinks as if they were plain directories.
On the left, I've browsed to D:usr.symsparkaneDocuments. Usr.sym is a symlink which replaces the awkwardly named "Documents and Settings". The displayed documents are on my K drive, but HLM thinks I'm on my D drive. I try to create a link arclink2.mdb to my arclink.mdb file and:
Oops. Well at least HLM will raise an error here and not corrupt the filesystem or fail mysteriously. What happens though if we step through another symlink back to the original disk volume? Here I've stepped through the example directory I've been using back to D:WINNT.
And I try to create a new link:
And here's my new hard link:
So HLM is not fooled by misdirection; it knows when its old link and new link are in the same disk volume.
Now the HLM Explorer extension. It's easy: right-click the file you want to hard link, select "Create Hard Link", navigate to a directory in the dialog that opens and enter a file name for the link.
See how easy?
Explorer Integration with Michael Elsdörfer's NTFS Link
All these tools are nice, but they're (mostly) there to provide a fix for the lack of integration of links into Explorer. Almost at the last second while writing this article, I've come across another program that adds support for both hard and soft links into Explorer, called NTFS Link. It adds a shell extension that allows you to create a soft link for a directory, or a hard link for a file, when doing a drag and drop operation using the right mouse button.
Although the menu option says "Create Hardlink" (not to be confused with HLM's menu option "Create Hard Link"), it will create a symlink when you drag and drop a directory.
NTFS Link's source is available, and it is in active development. Michael hopes to have a much enhanced version out "soon". Here's a short list of improvements copied from an email Michael sent me:
Among minor stuff:
#) Internationalization: I'm from Germany, and thus I am using a German
version of windows. IMHO it's a bit annoying having an English item in the
menu, if all the other items a German.
#) Support for deleting junctions: Currently, if cannot delete junctions
(i.e., the reparse point of a folder). Pressing DEL in Explorer will cause
the whole content of the junction target to be deleted. This will be fixed,
by
a) Another shell extension overriding the DEL behavior of Windows Explorer
b) Adding a new "Unlink" menu item to the context menu of junction
directories.
#) A lot of additions giving the user feedback which objects in the Windows
Explorer are links, and providing information about the link (such as the
junction target or the number of links pointing the a file):
a) Overlaying the icons of links in the Explorer by a small glyph,
indicating that the object is a link
b) Providing a little tool tip for link objects in Explorer
c) Adding custom columns to the "Details" view in Explorer
And that's just "minor stuff"! You heard it here first: if this is all the new version of NTFS Link does, it will surpass all other Windows link tools I've found.
Lastly, Mr Elsdörfer has expressed his opinion that symlinking should be possible not just for directories, but also for files, and has expressed interest in looking into this. If he creates a tool for symlinking files, some major filesystem hacks may be just around the corner...
The link to download NTFS Link is in the hyperlinks section.
Link Security, Briefly
Security for symlinks appears to be independent of the targets. I can, for example, change a symlink's inherited permissions to native, and remove any delete permission, and the target directory's permissions will be unchanged. So symlink security isn't transparent.
Note that with appropriate security settings on a symlink or target directory, you can prevent users from deleting sensitive data. The delete limitation is bad but this is an easy way to work around it.
Security for hard links appears to be shared. If I remove all permissions for normal users to "Coffee Bean2.bmp", and then check the permissions for "Coffee Bean.bmp", it too will have no normal user permissions. Thus permissions appear to be stored with the file data, not at the filesystem level.
The Innovation! The Innovation!
The almost non-existant integration of reparse point management functionality into common Windows filesystem tools is strange and frustrating. The bizarre delete behavior of Explorer exposes this integration as not only ignored, but ignored in a bad way.
It should be a simple right-click in Explorer to create a symlink or hard link. Links should have icon overlays to visually alert users to their nature. And deleting the links should delete just the links, not the targeted data.
Mounted volumes shouldn't be dependent on IDs that change when managing the volumes - that's the idea behind separate volumes, after all, is being able to move them around.
Symlinks should work across networks. It's just a path, isn't it? It works for shortcuts and shell extensions, doesn't it? So why not for symlinks?
I have a hard time believing that much time would have been necessary for one of the MS whiz-kids to throw this simple, barely-scratching-the-surface functionality together.
Why is it this way? According to JLM's help file, Microsoft is "scared" of giving users access to such a powerful functionality. But that doesn't make any sense. If Microsoft worried about this kind of thing, how is it I can type "del *.*" at a command prompt?. And the fact is, using links is dangerous only because Microsoft has ignored them. It would be easy to make the Never-Ending Virus Scan the worst thing that could happen with symlinks. I wouldn't be surprise if it were easy to do something about that too.
My speculations on why Microsoft has ignored Windows links run along two paths.
One, I wonder if they have avoided integrating Windows links because they would create an environment where filesystem hacking would become possible, and if this would have made things too complicated for developers (IMicrosoft'sO). There are several things that could be vastly improved in Windows with filesystem hacks possible, such as advanced package management (or even unadvanced package management), improved library management (which might mitigate DLL Hell), and user directories organization. But perhaps this would have caused problems for those making software to run on Windows, who would have had to start taking all these configuration possibilities into account. And I think they would have had to, as I think that if these kinds of hacks became possible - through the GUI, in particular -, demand for its development and for software that took advantage of it would appear and would never go away.
Two, other things on Windows are more or less crippled as well; the DOS prompt commands are a great example. Windows' roots being decidedly non-poweruser, it has never adopted the whole UNIX mindset of simple, small tools combined with user creativity, and we could certainly see links as falling into that category of tool. So maybe this is just part of the whole command-line-squeamishness from which Windows seems to suffer as a whole.
It's interesting to think about. It is strange though that Microsoft goes on about Innovation, while ignoring an innovative functionality that could have a big impact on the usability of its filesystem.
Hyper Links
Hyperlinks are neither soft nor hard links, but rather noted for their excitability.
Definitions of terms:
Programs to create filesystem links:
Info/Code:
Microsoft Website:
ravi