Mike Volodarsky's blog

Formerly the core server PM for IIS 7.0 and ASP.NET, now I run LeanSentry.
UPDATES: New troubleshooting guide released! Fix IIS website hangs →

IconHandler 2.0: file icons in your ASP.NET applications

Since its release, IconHandler has been a pretty popular module (on its own and with the custom DirectoryListingModule).  Today, I am releasing v2.0 of IconHandler, which contains some much-requested functionality and fixes a few issues that people have reported with the original version.

(I don’t RTFM, take me to download)

Here are the notable changes in v2.0:

1.       Icon pre-generation. The IconHandler now includes IconGen.exe, a tool you can use to export all of the icons registered on your workstation machine, and use those icons on a production server instead of retrieving them on demand from the OS. This way you get the real icons instead of the generic icon for all file types not registered on the server. for More on this later …

2.       Kernel caching. When using pre-generated icons, you can now enable kernel caching for maximum speed. If getting icons dynamically, you can use either response caching or icon caching.

3.       Fixes for several issues. This includes fixes for the “Win32 handle that was passed to Icon is not valid or is the wrong type” and “Object is currently in use elsewhere” exceptions.

I've also been asked for more information on how to deploy and use IconHandler. I’ll describe this below, as well as some general best practices and techniques you should consider when using it.

First, to deploy and use IconHandler:

1)      Place the assemblies in the provided sample application into your application’s /BIN directory. These include ShellIconHandler.dll, and ShellIcons.dll.

2)      Register the <iconHandler> configuration section in your application’s root web.config

3)      Add the handler entry for IIS 7.0 Integrated mode to the <handlers> section, and IIS 6.0/ IIS 7.0 Classic mode to the <httpHandlers> section.

4)      Specify the desired configuration settings for the handler using the <iconHandler> configuration section

Here is a sample web.config file in the root of your application that accomplishes steps 2-4:

<configuration>

  <!-- ShellIconHandler configuration section declaration -->

  <configSections>

    <section name="iconHandler" type="Mvolo.ShellIcons.Web.ShellIconHandlerConfigurationSection" />

  </configSections>

  <system.webServer>

    <!-- Add IconHandler for IIS 7.0 Integrated mode -->

    <handlers>

      <add name="iconhandler" path="geticon.axd" verb="GET" type="Mvolo.ShellIcons.Web.ShellIconHandler" />

    </handlers>

    <validation validateIntegratedModeConfiguration="false" />

  </system.webServer>

  <system.web>

    <!-- Add IconHandler for IIS 6.0 / IIS 7.0 Classic mode -->

    <httpHandlers>

      <add path="geticon.axd" verb="GET" type="Mvolo.ShellIcons.Web.ShellIconHandler" />

    </httpHandlers>

  </system.web>

  <!--

  Icon Handler by Mike Volodarsky

  Retrieves the shell icon for the specified file name.

  -->

  <iconHandler enabled="true"

              alwaysUseExtension="true"

              enableClientCaching="true"

              enableServerCaching="true" />

</configuration>

This configuration works on IIS 7.0 for both Integrated and Classic mode apps, as well as on IIS 6.0 and IIS 5.1 (Remove the system.webServer stuff for ASP.NET 1.1).

One cool way to use the IconHandler is with the DirectoryListingModule. Or write your own control.

To test IconHandler:

1)      Make a request to geticon.axd?file=FILENAME

2)      Get back the icon as a PNG file that can be displayed in the browser

IconHandler showing a file icon 

The file parameter can contain a filename or an extension. If the filename is provided, the handler can extract the icon for that specific file if one exists and specifies a custom icon. Generally, it’s preferred to use an extension because it avoids a file system access, AND, makes server caching of icons more efficient. The handler also provides a alwaysUseExtension config setting which instructs it to always use the extension part of the specified path to facilitate this usage.

You can also specify the optional size querystring parameter to select between small and large icons, for example geticon.axd?file=.ppt&size=small will give you a small version of the icon above.

Tweak configuration:

IconHandler provides some configuration settings you can tweak for your app:

enabled

You guessed it. If set to false, requests to IconHandler will be rejected.

enableClientCaching

If set to true, enables caching in the browser for 30 minutes at a time.

enableServerCaching

If set to true, the icons returned from the OS will be cached and re-used for subsequent requests for the same extension. This only works when asking for icons for an extension (not a file path), or if alwaysUseExtension is true.

enableResponseCaching

If set to true, will cache the response using ASP.NET output cache. Use this as an alternative to enableServerCaching to allow for greater performance or when not using extensions, especially when used together with the kernel cache.

alwaysUseExtension

If set to true, the handler always uses the extension part of the file parameter even if it contains a file name or a path. In v2.0, I set this to true by default, because it makes caching more effective and helps avoid the “Win32 handle that was passed to Icon is not valid or is the wrong type” issue.

useSavedIcons

If set to true, the handler will use the icons generated with IconGen.exe in the /App_Resources/Icons directory of the application instead of retrieving them from the OS. This mode can only be used with extensions, it does not support retrieving icons for specific files. Also, if the extension does not have a pre-generated icon, 404 is returned instead of a generic icon.

Pre-generating icons:

By default, IconHandler retrieves the icons on demand using the shell API SHGetFileInfo. This works well on a Windows Vista or Windows XP workstation that has all the right file types registered to installed applications, but on production servers (typically Windows Server 2003 or 2008) these applications are not installed so you would get generic icons for most file types.

To help with this, IconHandler provides the useSavedIcons config setting that instructs it to find the pre-generated icon for the requested extension in the /App_Resources/Icons directory.

How do these icons get there? Simple, you use the provided IconGen.exe tool to export the icons from your workstation to a directory, and then deploy them to the /App_Resources/Icons directory in your app. Here is an example:

> IconGen.exe c:Icons large

This will export all of the registered icons on your machine to the c:Icons directory as large icons. You can repeat the process using “small” to generate the small icons too.

To export only a specific set of icons that you want to use, provide the path to a text file containing the list of extensions, one per line. This file can look like this:

.txt
.exe
.docx

Here is a cheesy trick I baked into the tool to do this. First, run it for all extensions, and pipe the standard out to a text file:

>IconGen.exe c:icons large > extensions.txt

You’ll end up with extensions.txt containing all of the known extensions on your machine (836 for my Vista machine). Then, delete the lines you don’t want in the file, and re-run the tool with the file provided:

>rmdir c:icons /s/q
>IconGen.exe c:icons large extensions.txt

Voila.

Caching tips:

The IconHandler supports 3 types of caching: client caching (browser/proxies), server caching (caches the icons obtained from the OS), and response caching (caches the entire response containing the icon).

The recommended way to use this is as follows:

1)      enableClientCaching = true. Each browser will cache each icon it gets for 30 minutes. Why 30 mins, when icons are not likely to ever change? If you are still reading this far, you can ask me to change it J

2)      enableServerCaching = true if using extensions, and fetching icons dynamically, OR

3)      enableResponseCaching = true, if you are not using extensions, OR if you are using saved icons where #2 doesn’t apply. With this, you can also configure kernel caching for max performance.

To enable kernel caching, you need to add the following to your application’s root web.config:

  <system.web>

    <caching>

      <outputCache enableKernelCacheForVaryByStar="true" />

    </caching>

  </system.web>

This allows ASP.NET to enable kernel caching for all urls that vary by all parameters. Note that this may start kernel caching other pages that weren’t previously, so be aware of that when enabling. Also be aware of the standard cache bloat implications if you request a lot of unique urls.

Here is a obligatory “theoretical” perf comparison of 200 clients making sets of 7 requests for same 7 extensions (theoretical because there is a 100% hit ratio):

Performance test of IconHandler caching modes

1)      No caching: 350-ish RPS

2)      Server bitmap caching (enableServerCaching=true): 1100 RPS

3)      Response caching (useSavedIcons=true, enableServerCaching=false, enableResponseCaching=true): 4400 RPS

4)      Response caching + kernel cache: 9300 RPS

As always, kernel caching has superior performance to other forms of caching. NOTE that at the 9300 RPS, my CPU usage drops from the fully utilized 100% to about 35%. This means that the client is not able to drive the server fast enough, so we can project about 26000 RPS on fully utilized server.

This also effectively makes IconHandler performance the same as native IIS static file handler on cache hits, and since there is a finite number of extensions, real performance on a busy site should approach it as well.

Using the API:

IconHandler comes with ShellIcons.dll, which provides a function that you can use to retrieve icons from the OS by p/invoking into SHGetFileInfo.

You can reference this DLL in your projects and call into the API to get icons (full trust required due to p/invoke). Here is a snippet:

using Mvolo.ShellIcons;


using (IconContainer icon = ShellIcons.GetIconForFile(".ppt", true, true))

{

    Bitmap b = icon.Icon.ToBitmap();

    // Do stuff with the icon …

}

The ShellIcons.GetIconForFile method allows you to get icons for a specific file or extension, optionally getting either a large or small icon.

The method returns the icon wrapped in the IconContainer class, which implements CriticalHandle. That means that if you hang on to the icon and forget to dispose it, the CLR will do it even under extreme conditions like thread abort and appdomain unload. But, you can only have so many of the icon handles open before the OS will fail to create any more, so it’s a good idea to dispose of the handle immediately like I show in the snippet above. You can than hang on to the Bitmap after the icon is disposed.

Possible issues to be aware of:

1)      My icons all show up as the generic icon! That means you don’t have file associations for the icons – try pre-generating on a machine that does, and then use saved icons.

2)      IconHandler requires full trust when fetching icons on demand. If you use saved icons, it should run fine under Medium trust.

3)      If  you are fetching icons on demand, and don’t use caching, or are making requests with file paths and not extensions, under heavy load you may cause the ShGetFileInfo API to exceed the number of allowed handles and start returning bogus handles. In this case, IconHandler will throw an exception indicating “You have exceeded the maximum number of open GDI handles. Close some of the existing icon handles to avoid this condition”.

4)      If you are fetching icons on demand, ShGetFileInfo API may end up loading all kinds of DLLs into your process to get icons for various file extensions.  If this is not acceptable, use saved icons instead.

 That’s it. Oh, and here is the download J:

Download: IconHandler 2.0 (DLLs and sample application)

Download: IconGen.exe (tool to pre-generate icons)

Enjoy! And let me know if you hit any issues ...

Thanks,

Mike

 

IIS 7.0 Bit-rate throttling module

Last week, the IIS team released bit-rate throttling module to the web.

As the self-proclaimed daddy of the project (I designed and wrote the initial prototype in early 2007), I am very thrilled to see it out. The new IIS media team folks have done a great job getting it production ready and rounding out the feature-set, which you can review in its full glory in Vishal's post.

Download links:
- 32 bit - http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1640
- 64 bit - http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1641

So what is the bit-rate throttlng module about? In essense, it is an effective solution for saving a LOT of bandwidth for web sites serving media

Imagine this scenario - a client connects to your video site, clicks on your featured video, watches 5 seconds of it to realize they have no interest in watching further, and move on to the next video.

In those 5 seconds, the server could have sent out 5 minutes worth of the video, and you paid for 5 minutes worth of bandwidth. With the bit-rate throttler + media bitrate detection, the server would only end up sending a little over 5 seconds worth, and you would end up paying only for what was used.

This translates to VERY major savings for most video sites.

The way I originally wrote the module, it was actually comprised of two separate modules: a throttler, and the media bit-rate detector. Eventually these got combined into a single module, but it is still useful to think about it from the point of view of two separate features.

  • The throttler component is a mechanism to control the rate at which IIS sends out the response for any request to the server.  It works in conjuction with another module or configuration that instructs it to do two things for any response: pre-send any configured amount of data without restricting the response rate, and then send the rest of the response at the configured rate.
  • The bit-rate detector is a separate module that can parse common media files, and determine their encoded rate (many video file formats, including ASF, include the maximum encoded rate in the header of the media file). It can then configure the throttler to send that media file at a rate that is just over the encoded rate.

It's also worth noting that the throttler uses a high-performance asyncronous loop to push the data out, without tying up server threads for what can be a very long operation. For responses coming from files (like most large video files), it also does not need to read the content's of the file being sent into memory, instead just instructing http.sys to send portions of the file out to the client at a time.  Because of this, it won't significantly affect your memory usage. While this mechanism is not as efficient as http.sys's own site-wide bandwidth throttling (which cannot be used to do what we are trying to accomplish here), it is pretty much as lean as it can be.

While these two components play different roles, the current bit-rate module release combines them into a single module.  Still, thanks to the flexibility of configuration, you can set both static rules for throttling abitrary content on your site, as well as rules that can automatically detect the bit-rate of media files.

In addition to controlling the response rates, the bit-rate throttling provides the following features:

  • Fast Start - the ability to send the first part of the media file without rate limiting, to seed the playback buffer in the player and make sure that playback can begin as soon as possible (most players try to prebuffer a certain amount of the video, often 5 seconds, before starting playback). This also insures that if the connection suffers a hickup, the playback can continue uninterrupted
  • Disconnect detection - when the client stops watching the video, goes to another page, or closes the video, the bitrate throttler detects the connection closure and stops sending the file.
  • Built-in support for detecting the playback rate for common media formats, including .asf, .avi, .flv, .m4v, .mov, .mp3, .mp4, .rm, .rmvb, .wma, and .wmv.
  • Ability to configure static throttling rates, and media auto-detection rates at any configuration level.

It is also possible to add support for additional media formats by using the configuration that tells the module how to find the bitrate in the mediate file. Finally (as I intended from the very start), the module retains the ability to configure the throttling rate programatically.  This means that you can write your own module that automatically determines the desired throttling rate for any response - media or not - which opens up the possibilities for any custom throttling scheme.

The bit-rate throttling support is a key part of the media story comprising Silverlight, Expression encoder, and future media features coming out of the IIS 7.0 team. And, it directly affects your bottom line by saving cold hard cash on bandwidth costs, which can get fairly large for media-intensive sites.

So, go download the bit-rate throttler and start saving money today (and let me know if you are - I'd love to hear your success stories).

Thanks,

Mike