After the last update, the elapsed time numbers as reported in the IIS logs look much better.
While reviewing the IIS logs, I noticed that the elapsed time for some requests occasionally were unusually long: a very simple page could take well over a second. Some of this is just "Internet weather," with slow links between the server and the client, but that's not the whole story. It turns out that minimizing initial request latency on a low-traffic site can requires a few tricks that aren't needed with high-traffic sites. Part of the problem is that this server is shared with a couple of other applications at the moment (mainly SQL Server for another site). Since the site isn't very busy yet, if it's idle for long enough, the IIS worker process can get paged out or the data pages in SQL Server can be released—then things are slow for the first requests after that.
A key design goal of mine has been to make sure that most pages and images on the site are cacheable by http.sys and proxies. To meet that goal, I take care to make sure that both the HTML and the HTTP response headers are the same for all users. For example, I don't set cookies from most of the pages themselves. The only exceptions at the moment are the registration and login pages (which aren't enabled yet).
Since most pages can be cached in the kernel, I decided to increase the UriScavengerTime for http.sys from the default of 120 seconds to 43200 seconds (12 hours). This will allow cacheable items to stay in kernel memory longer, which will reduce access latency after a long idle period. The site is still pretty small, so the increase in kernel memory use should be minimal.
I also disabled compression on id.ashx from web.config. That reduced the payload size from 36 bytes to 3.
Cleaned up the content and layout on a few pages, and added navigation menus on the left and right, using a three-column fluid layout. Since the CSS file uses a far-future cache expiration date, I just changed the filename. I'm also using ASP.NET themes, so that's all I need to do: the new file is picked up automatically by the theme.
Also: modified web.config to set a cookie on robots.txt, as part of a long-term effort to automate bot detection without relying on the User-Agent string. I'm also forcing Cache-Control: public, to help encourage caching by proxies, in spite of the cookie.
Also: made a few changes to the AppPool and IIS settings. Since the overall traffic is still light, I decided to disable recycling for now, which also requires disabling timeouts. I also tweaked the IIS log settings from the default, to include cookie values and the HTTP referrer. That will allow me to run some interesting reports later on. I double-checked to make sure that changing the settings didn't disturb http.sys caching; everything is OK there.
As part of my regular routine, I ran logparser against the site's IIS logs, to check for HTTP 500 and 404 errors. To my surprise, I found a 500 error; it was in the ajax1.aspx code sample for the book (called from file28.htm). It turns out that I introduced a bug when it was integrated into the site from the standalone Visual Studio solution. Since I use ASP.NET themes, the code required StyleSheetTheme to be set to the empty string, to avoid the runtime insisting on the presence of a <head> element.