MSI Validation in WiX

One of the most frequently overlooked steps in building a Windows Installer setup is validation. Validation is very important; it finds all sorts of problems with per-user installations, collisions between installed files, and other common authoring mistakes that are hard to catch until the complete MSI or MSM file is built. Often, when debugging a setup problem, I've found that just running validation would quickly find the mistake(s).

Ok, so if validation is so important, why doesn't everyone run it? There's a few reasons: some people simply aren't aware of MSI validation, others are lazy, and the grand majority of people simply just forget because its an optional step.

With WiX 3.0, running validation is now easier than ever because its integrated directly into Light.exe. Just keep using the WiX tools like you always have in the past. If Light finds any validation problems while linking your MSI or MSM file, it will simply display a warning or error message. It's really that simple. Best of all, Light will even display source line information about where an ICE error or warning came from when it's available.

With the addition of the validation infrastructure to Light in WiX 3.0, we thought it would also be useful for the users of WiX 2.0 and other installation creation programs to have a simple way to run validation on their MSI or MSM files from the command line. So, we're also releasing a new tool called Smoke which can run validation on any MSI or MSM file and reports errors and warnings in the same convenient command-line format as the other WiX tools.

What if I don't want to run validation?
If you would like to suppress validation, simply supply the option -sval to light.exe. However, this is strongly discouraged since it means that you won't automatically get validation coverage on your setup.

Can I suppress an individual Internal Consistency Evaluator (ICE)?
Absolutely, just pass -sice: on the command line. For instance, to suppress ICE33, you would pass -sice:ICE33.

Are there any downsides to running validation in Light?
Yes, running validation will make your builds a bit slower. However, due to the benefits, the time spent validating is a very good trade-off in comparision to time wasted debugging issues that could have been caught simply by keeping validation on.

Do I really need to run validation?
Running validation is a very important step in insuring that you are creating a quality setup experience for your users. With this new validation feature, we were finally able to test our own MSI packages and we were shocked at the number of issues it found (since we'd like to consider ourselves somewhat experts in the setup area). This resulted in several improvements to Candle and some of the extensions, but even with these improvements, we'll never be able to guarantee the coverage provided by simply running validation.

Is there any documention on validation from the Windows Installer team?
There are several good articles on MSDN about using validation:

• Internal Consisteny Evaluators
• ICE Reference - very useful for finding out what a particular ICE does
• Merge Module ICE Reference - for finding out about merge module ICEs

Where do ICEs come from?
ICEs are nothing more than custom actions which run against a database and display warnings and errors based on what they find. Since they are merely custom actions, they are not surprisingly packaged into special Windows Installer database files with a .cub extension. The extension may seem a bit arbitrary until you learn one vital piece of information: the true extension of these files is "cube" - "cub" is merely an 8.3 compliant name. It's a play on words because the tests are called ICEs (ice cubes - get it?).

The Windows Installer team distributes several cube files. The two most notable files are darice.cub and mergemod.cub. These cube files contain, respectively, the ICEs for MSI and MSM files.

Which ICEs will WiX run against my MSI or MSM file?
For MSI files, WiX will run all the ICE tests found in darice.cub. For MSM files, WiX runs mergemod.cub. Please note that ICE33 has been disabled by default because it forces authors to use the advertised COM tables instead of normal registry keys whenever possible. Due to problems with using advertised COM registration in certain scenarios (like repair), the WiX team does not suggest this practice and thus has disabled the ICE by default.

How does WiX run validation?
WiX uses the method outlined here. Basically, Light takes your MSI file, merges a .cub file into it, registers an external UI handler (like its going to run an installation), and then runs the custom action sequence found in the _ICESequence table (ignoring the ICEs you chose to suppress of course).

So, if an ICE is nothing more than a custom action, can I make my own?
You can create your own ICE custom action and package it into a cube file, just like the standard cube files. However, at this time, Light and Smoke do not support passing custom cube files into the validation engine. If you would like this feature to be added, please open a request on SourceForge.

Why is Smoke called "Smoke"?
It's another play on words. Because the smoke program essentially tests an MSI or MSM file for problems, its like a "smoke test".

heat.exe - making setup easier

Today is April 5, 2006 – the 2 year anniversary of the WiX Open Source Project!

What better way to celebrate the anniversary of the WiX Toolset than by releasing a new tool? Today, I'm proud to announce the availability of Heat in WiX 3.0. Heat is very similar in functionality to Tallow or the engine inside of ClickThrough: it has the capability to quickly capture files and directories from a computer and turn them into WiX authoring. However, the functionality of heat goes well beyond what either of these tools could provide for capturing WiX authoring. A big thanks goes to Reid for helping me make Heat a reality - he came up with a lot of the original ideas for this project and was a huge help with the harder design and naming issues.

Heat is our attempt to create an entire ecosystem of what we're calling "harvesters" and "mutators" (more on the mutators later). Harvesters are essentially WiX extensions which enable a developer to "harvest" any information into WiX authoring. For example, we've just released a new WiX extension called the WixUtilExtension. This extension contains a harvester that enables you to very quickly harvest an entire directory of files and any self-registration inside of those files. So if you have an entire directory of DLL files which each use self-reg for their COM registration, you no longer need to run Tallow on each individual file. You can now just run "heat.exe dir C:\directory -out sourceFile.wxs" and do this all automatically.

Unlike Tallow, Heat is optimized for working with WiX 3.0. This means that it does not create Id attributes on Registry elements and doesn't set short file names. Instead it relies on the auto-generation capabilities of Candle wherever possible. When Heat cannot rely on Candle to generate an identifier, it does its best to create a unique one based on the name of the file or directory that is being harvested. So if you harvest a file called "foo.dll", unless there is a collision with other files captured at the same time, its File/@Id will be "foo.dll" and the name of the component containing that file will be "foo.dll". This should make it much, much easier to capture individual parts of your setup and paste them together into your product with minimal identifier collisions.

Heat makes creating simple setup authoring faster than ever before with a templating capability. In the past, each time you create a new setup, you also had to create new Product, Package, etc... elements. Or if you're like me, I'm guessing you just copied another WiX source file and tweaked it to make your setup. With Heat, this is no longer necessary. Just run Heat with the "template" command line option like this: "heat.exe dir C:\directory -out sourceFile.wxs -template:product". This will do the normal harvesting, but then wrap the output up in a Product element. Additionally, it will hook up your root directories under TARGETDIR and put your components under a default feature. All you need to do is replace a few placeholders like "PUT-PRODUCT-NAME-HERE" with the actual values for your product and compiler/link. It's literally that easy. Just in case you are wondering, we still support auto-generation of guids - just use the -gg option. It will even generate the ProductCode. Of course, if you'd like to create a merge module quickly, you can just run with the -template:module option.

If you've read this far, there's a very good chance that you're either excited by all the new functionality in Heat or a bit disappointed because nothing in there seems to be ground-breaking. Well of course, I've saved the best part for last. Heat is not a super-tallow or command line version of ClickThrough. It's an engine for harvesting information into WiX authoring. Although all my examples so far have dealt with files and directories, there's nothing about how Heat works that prevents it from capturing more interesting things. Although its a bit under construction, Heat has the capability, today, to capture IIS web sites. Simply run 'heat.exe website "My WebSite Name" -out sourceFile.wxs' to harvest a web site and all its files. You can then compile and link with the WixIIsExtension to create an entire web site setup in minutes! Please note: there are currently a few missing bits for this feature, like it doesn't set all the Id attributes yet (I'll try to get this done for next week). Once this feature is completely finished, since Heat can set all the identifiers for you uniquely, generate all the guids, and wrap everything in a Product element with the nececessary ComponentRef elements, all you have to do is fill in a few placeholders like "PUT-PRODUCT-NAME-HERE"

I mentioned the idea of mutators earlier but didn't really go into what it meant. Mutators are a new class of functionality within WiX similar to the compiler or linker. A mutator is simply a class which takes a WiX CodeDOM object as its input and mutates it. Mutating is essentially like running a transform on XML, but since its a WiX CodeDOM, its strongly-typed and much easier to work with versus XML transforms. Mutators are the secret behind why Heat is so simple and powerful. In the past, if a harvesting tool (like Tallow) had options for tweaking its output by using fragments or wrapping the output in a Product element, the code doing the actual harvesting needed to be aware of those settings. This made adding new functionality very cumbersome because each new extension had to be aware of all the options it should be support. Even in Tallow, you may notice that depending upon how registry keys are harvested (from a .reg file or a self-reg capture), the output may differ. In Heat this is not a problem - harvesters just capture the very simplest information possible, then rely upon the mutators to mutate the output to reflect the specific options the user wants.

In later posts, I'll go into the details about how Heat works and how you can very easily write your own extensions to harvest whatever you find interesting. As you might imagine, Heat can do much, much more than I'd want to talk about in this introductory post. I hope to more fully explain its capabilities in later posts.

How does this affect ClickThrough?

ClickThrough will be refactored to use the harvesters and mutators of Heat as its new engine for capturing files from your machine. There are currently a few issues with the internal WiX authoring created by ClickThrough - this will address those issues and ensure that future versions of ClickThrough remain more in-sync with any changes made to the WiX source schema.

How does this affect Tallow?

Tallow will no longer exist in WiX 3.0. Tallow will continue to be released as part of WiX 2.0. The code for Tallow in WiX 2.0 and 3.0 is identical, so if you prefer to use Tallow instead of Heat, you can simply grab it from WiX 2.0.

Why doesn't Heat allow me to specify the text for the placeholders on the command line so that I can generate my setup every time?

Heat is designed to allow a setup author to very quickly generate their setup authoring the first time. From then onward, the authoring should be maintained manually to ensure that guids and identifiers remain stable and component rules are not broken. In order to use Heat to generate setup every time, it would be necessary to create a database which tracks the components, their guids & identifiers, and their files for every release of your product. We call this a component catalog. Heat does not provide a component cataloging capability - however - it does expose the necessary functionality so that anyone could write a component catalog, hook it into heat as an extension, and generate their setup automatically without breaking component rules.

Where should I mail Heat bug reports or feature requests?

Please submit bug reports or feature requests to the WiX site on SourceForge. I've created new categories for Heat/harvester - please use them to ensure the bugs or feature requests are properly assigned.

Are there any other planned features for Heat right now?

Sure, I think it would be nice to harvest a SQL database. There have also been a lot of ideas floating around about working with Visual Studio projects. I'm sure people will come up with lots of cool new things to do with Heat.