Derek Cicerone Installing...

WiX, Setup, and trying to make it all easy, not just easier

Monday, February 13, 2006

Auto-generation of short file and directory names

Auto-generation of short file and directory names has finally been checked into WiX 3.0. This feature should be available in the very next release!

Will this feature be ported back to WiX 2.0?

Unfortunately, this feature will not be ported back to WiX 2.0. It is simply too disruptive (it modifies the schema for and elements) and dangerous for the stable 2.0 codebase.

Should I upgrade to WiX 3.0 to take advantage of this feature?

This is a decision that each individual team should make for themselves, but we do have some recommendations based on what features your group is using and when you expect to ship.

First off, if your product is shipping in 2006, please do not use WiX 3.0. We simply cannot guarantee with any certainty that WiX will be stable and bug-free enough in time for you to ship your product. We also reserve the right to continue deprecating and replacing certain functionality in WiX 3.0 as well as create incompatibilities between versions of WiX 3.0. For example, so far we’ve deprecated FragmentRef, deprecated all src attributes (in favor of more descriptive attributes like SourceFile), broken backwards compatibility for wixout files, completely re-vamped the WiX extensions and how they are packaged, etc…

If you are shipping in 2007 or later, then WiX 3.0 may be appropriate for your group. We hope to have WiX 3.0 entering lockdown sometime near the middle to end of this year (more likely at the end). For comparison, WiX 2.0 entered lockdown in the fall of 2005, so we like to spend about a year on stabilization. Some of the things to look at for your group are whether you can tolerate the need to infrequently update your source code whenever features are deprecated or re-named. We have a tool called WixCop which will automatically update most wix source code the latest schemas, however, it currently also re-formats code to a very strict standard of 2-space indentation and double-quotes for attribute values. For groups willing to adopt this standard, wixcop makes it trivial to keep up-to-date and also police the style of source code in your organization. Sometimes features are deprecated with no replacement (like FragmentRef). In these cases, significant re-design by hand may be needed (currently we only anticipate one more breaking change like this: we will no longer support nested elements, instead, this functionality will be replaced by , , … elements with much more explicit usage).

I’m using WiX 3.0, how do I migrate my authoring to the new schema?

If your group can use WixCop, that is definitely the preferred route since it will automatically update all your sources (hopefully) without mistakes. As I mentioned before, the downside is that it also re-formats code to 2 space indenting.

To migrate elements manually, do the following:

Change this:

To this:

Change:

Name -> ShortName

LongName -> Name

LongName is now deprecated.

Migration of elements is a bit more complicated.

Change:

Name -> ShortName

LongName -> Name

SourceName -> ShortSourceName

LongSource -> SourceName (note that this is a bit weird because the old name was missing “Name”)

LongName and LongSource are now deprecated.

I’m using WiX 3.0 how do I take advantage of this feature and stop specifying short names?

For existing authoring, after migrating to the new schema as outlined above, remove the Short* attributes. WiX will then automatically generate the short names for you.

For new authoring, just stop specifying the Short* attributes. So for a file, just specify File/@Name. For a directory, just specify Directory/@Name (and the optional Directory/@SourceName if necessary).

How do I specify a file name that is already short?

If you have a file name that is already short and 8.3-compliant, like file.txt or example.doc, you just specify this file name in the Name attribute (not the ShortName attribute). WiX will automatically determine that you’ve specified a valid short file name and suppress generating a short name. The idea here is that you should never have to specify anything other than the Name attribute, regardless of whether the desired name is long or short.

How does WiX create the short file and directory names?

The short file names are created using the following algorithm:

  1. Create a string by concatenating “File” + ‘|’ + <parent> + ‘|’ + <lowercase filename>. The pipe characters (‘|’) are necessary for canonicalization. The “File” at the front allows us to expand this feature in the future to cover or other short file names we’d like to generate under components. The parent component identifier is used instead of the file identifier because someday I’d like to add the feature of automatically generating the file identifier as well. The long file name is lowercased for canonicalization purposes. Canonicalization for short file names basically means that we’d like to see a collision of generated short file names whenever we can expect to see a collision of long file names within a single component.
  2. MD5 hash the string into a 128-bit hash.
  3. Convert the hash into a base64 string and replace illegal file name characters with appropriate alternatives. Specifically, we replace ‘/’ with underscore (‘_’) and ‘+’ with dash (‘-‘).
  4. Lowercase the entire base64 string and truncate it to 8 characters in length.
  5. If an extension existed for the original long file name, append up to 3 characters from that extension onto the end of the new string from the last step. For example, if the long file name was “sources”, then no extension is added, if the long file name was “text.txt”, append “.txt”, and if the long file name was “my project.csproj”, append “.csp”.

The algorithm for creating short directory names is very similar. The only differences are that we use the directory’s identifier instead of the parent file identifier in step 1 and there is no extension handling for step 5.

How will using generated short file and directory names affect patching?

Since the generated short file and directory names are actually hashes of the user-specified long file and directory names, these names should remain consistent as long as the file names and relevant identifiers included in the hash (see previous question for more info about which identifiers are included in the hash) do not change.

What happens if I move a file containing a generated short file name to another component?

The generated short file name will likely change since the parent component identifier is part of the hash which defines the generated short file name. This is expected behavior and by design.

What if I use the same long file names in many components; will the generated short file names collide?

We’ve tried very hard not to prevent short file names for colliding when appropriate. The only time that we’d like generated short file names to collide is when the same long file name is duplicated in a single component (since this makes the installer attempt to install multiple files to the same file name on the target machine). Since the hash used to generate the short file names includes the parent component identifier of the file, theoretically, re-using the same long file name in multiple components should not be a problem.

What happens when a generated short file name collides with another generated short file name(s)?

Light.exe has now been updated to detect collisions of generated short file names across all files in a single msi/msm file. It does not take into account directories since it’s possible to re-map almost any directory to any path on the target machine.

If a collision is detected before you ship your product for the first time, you can update any or all of the colliding files by manually specifying the ShortName attribute. Unfortunately, it’s not possible for WiX to automatically resolve collisions since that would not result in 100% stable generated short file names (depending upon whatever else you compiled with, names may differ across products or merge modules). So these collisions must always be resolved by hand. In testing, we found collisions to be quite rare (in fact we were never able to create one).

If a collision is detected while patching an older product or while creating a minor product update, it is very important to only specify ShortName on the newly added files for the patch or update. This will keep the previously shipped file names consistent. Please remember that although you may not anticipate your customer ever installing to short file names, it is almost always possible that they did.

What happens when a generated short directory name collides with another generated short directory name(s)?

Currently: nothing. If this is a problem, we can attempt to detect this scenario in the future.

When will the File/@LongName and other deprecated attributes be removed from WiX?

We usually keep deprecated features in for about a year or so to allow everyone time to update. The deprecated attributes will likely work until at least 2007.

I’m using WiX 3.0, do I have to update my schema? Do I have to use the auto-generated short names?

No and no. As long as support for the deprecated attributes remains (until 2007), you don’t need to update. However, we strongly encourage updating since the deprecated attributes will eventually be removed entirely.

You never need to use the new auto-generated short names feature if you don't want to use it. Simply begin specifying the new ShortName attributes to continue manually specifying short file names.

4 Comments:

At Friday, June 09, 2006 10:07:00 PM, Anonymous Danilo said...

In the first sentence of step 1 of the algorithm under "How does WiX create the short file and directory names?", some of the characters are not properly escaped.

That sentence should read: Create a string by concatenating "File" + "|" + <parent> + "|" + <lowercase>.

I hope this comes out ok when I post as the preview feature changes the escaped characters in the edit control back to un-escaped ones.

 
At Friday, June 09, 2006 10:48:00 PM, Blogger Derek Cicerone said...

Thank you for the correction danilo - it should be fixed now.

 
At Thursday, August 31, 2006 5:36:00 AM, Anonymous Graeme said...

Any update on 3.0 stability?

 
At Thursday, August 31, 2006 10:16:00 AM, Blogger Derek Cicerone said...

WiX 3.0 is still very much in development. We suggest you avoid using it in production unless you will be shipping your product in 2007 or later.

 

Post a Comment

<< Home