Avoiding duplicate content due to case-sensitive URLs

A point that's been raised a fair bit recently by SEO specialists I work with is duplicate page content. This happens when you're using URL rewriting to point a URL to a page, and there is more than one way to get to that content.

For example:

http://www.website.com/news
and
http://www.website.com/news/home

Could both lead to the same place. Google crawls both of these URLs, and distinguishes them as separate. This can allegedly affect your ranking, and as a result, it's good to have your duplicate URL 301 redirect to the other.

This is also the case with URL rewriting case-sensitive URLs:

http://www.website.com/ThisURL
and
http://www.website.com/thisurl

A lot of regular expression matching URL rewriters such as URLRewriter.NET will happily handle both URLs and point you to the same page content. To get around this, I've implemented some code in the global.asax file to convert URLs to all lower-case.

http://www.website.com/ThisURL

will be checked in the global.asax file, and a 301 redirect will send you to:

http://www.website.com/thisurl

There's one other thing to bare in mind; ASP.NET's built-in AJAX functionality requires you to have some references to AXD files such as scriptresource.axd and webresource.axd. These files have case-sensitive querystrings, and if you don't add an exception for AXD files, these files won't be reachable, and your AJAX controls will no longer work.

Finally, you also have to ensure that you don't get in the way of postbacks. If you don't distinguish between a postback and a standard URL request, your postbacks will no longer work, and all your forms will break.

If you check out the code sample below, you'll see that I've ensured that only GET requests are checked, and that AXD files are excluded.

	protected void Application_BeginRequest(object sender, EventArgs e)
	{
	//If upper case letters are found in the URL, redirect to lower case URL.
	if (Regex.IsMatch(HttpContext.Current.Request.RawUrl.ToString(), @"[A-Z]") == true)
	{
	if (Request.HttpMethod == "GET" && !(Request.RawUrl.Contains(".axd")))
	{
	string LowercaseURL = HttpContext.Current.Request.RawUrl.ToString().ToLower();
	Response.Clear();
	Response.Status = "301 Moved Permanently";
	Response.AddHeader("Location", LowercaseURL);
	Response.End();
	}
	}
	}

view raw global-snippet.asax.cs hosted with ❤ by GitHub

The Script Manager enableHistory attribute

Overview
Geeks with blogs have produced a lovely blog that'll show you how to set up your page's script manager to allow your back and forward browser buttons to trigger an event.

It runs through how to enable it using the enableHistory attribute on the script manager and how to wire up the event so that when you use the back buttons it'll fire.

Something that's worth pointing out though, is that you MUST add a history point during a postback event before leaving the page, or your event won't be fired. This may seem obvious, but I didn't realise!

So you'll definitely need a line like this in a postback event:

ScriptManager1.AddHistoryPoint("Key", "Value");

Without that happening at least once before leaving the page, your navigate event won't fire.

A .NET tab list control for content managed sites

Overview

Most websites these days seem to contain tabbed content; it's a way for more information to be displayed on a page whilst still technically being above the fold, and it's historically been a very good way of organising and pooling content together. However, having a tabbed layout on your website can create a few problems too, such as:

• How search spiders crawl your content
• How you URL-rewrite your content so that you can drop users onto a specific page with a specific tab pre-selected for them to look at.

Possible solutions

How tabs are handled from an ASP.NET technical perspective depends largely on the content that you're displaying.

In some cases, tabbed content is minimal in size, and you can get away with creating a very fast user interface by pulling all of the content for each tab in on the preload, and then showing and hiding the containers holding the content.

In this case, if you have five tabs, you could create five hidden divs and show or hide them depending on which tab has been clicked. This approach also has the benefit of allowing search spiders to crawl all of your tabbed information in one hit, as it's held in one page.

In other cases, sites will need several tabs, each containing a LOT of content. In cases like these, it's better to call this information when it's required, ie when a user has clicked on the tab they want to see. In cases like these, it's better to pull the content in using AJAX (be it via JQuery, or via .NET update panels). I'm choosing update panels.

Pre-selecting your tab

A good idea to begin with, is that when you enter a URL such as

/this-page?tab=your-tab-name-here

Your tab should be automatically selected and loaded in. This allows users a method of getting straight into a particular part of your content, and the tabs aren't an obstruction. You can do this by checking the URL you came in on in the page_load event, and popping your active tab in the update panel.

if (!Page.IsPostBack)
{
  hidValue.Value = (0).ToString();
  if (Request.Params["Tab"] != null)
  {
      TabName = Request.Params["Tab"].ToString();
      //ContentPageId is a global integer to hold the ID of my tab
         hidValue.Value = ContentPageId.ToString();
  }
}
else
{
  //on postback set content type to hidden field value
     ContentPageId = Int16.Parse(hidValue.Value);
}
Repeater1.DataBind();

We call the databind method for the Repeater1 object because that's what's binding our tabs to our update panel. The repeater will grab the ContentPageId, determine what tab you want to show, and set that as the active tab accordingly.

Binding our clickable tabs

Each tab within the repeater has a href with a link of /?tab=your-tab-name-here, and an onclick attribute that fires a javascript function that updates a hidden input value which will hold the ID that you can use to reference your content. This attribute then returns false so as not to get the href to fire.

So your javascript function might look something like this:

function SetHidValue(value) {
document.getElementById("hiddenInput").value = value;
document.getElementById("SubmitButton").click();
}

Each anchor tag in the repeater will look something like this:

<a href="/?tab=your-tab-here" onclick="javascript:setHidValue(14)">Tab title</a>

Where the argument passed into setHidValue() is the dynamically populated ID of the tab you're after. The methodology of using tab IDs is required in my case because I'm using a content management solution (It could be EPiServer or Umbraco for example), which will require a page ID to grab the relevant page object when I know which tab has been asked for.

Our click() event in the javascript function fires a submit button that'll trigger a postback within the scope of the update panel. Your page_load should then check to see if you've got a hidden input value, and if so, you know which tab to show. If you don't have one, the second choice should be your tab= querystring parameter.

What this means, is if you don't have javascript (or you're a search spider and you want to crawl individual tabs without the help of javascript), you can use the querystring URLs instead.

This post is going to end up being quite complicated, so if you have any suggestions or questions, please pop them in a comment, and I'll try to address them.

Upgrading an EPiServer CMS 5 site to EPiServer CMS 6

Overview

I've built a few websites in the EPiServer CMS now, and because I've developed those websites in quick succession, I have rarely needed to use the deployment center's upgrade wizard.

I've recently had to upgrade an EPiServer site from version 5R2SP1 to version 6, and had to initally provide a scope for any areas that might go wrong.

The upgrade wizard's upgrade process makes changes to the EPiServer database, adds some extra files to the site's /bin/ directory, and also makes some fairly significant changes to the web.config.

Changes to the file system are quite easy to spot (the upgrade wizard produces a log file for your perusal), but the major issue that I encountered with upgrading was changes made to the database.

The problem

When hosting an EPiServer website on one server and the database on a separate server, the upgrade wizard must make modifications to the data on both servers. The upgrade wizard is part of the deployment center, and so changes to the filesystem were done with no problem. However, the upgrade wizard kept throwing an error complaining that it couldn't connect to the database, and that authentication failed. This is because unless your servers are hosted on the same domain, your windows account is unlikely to be authenticated. Regardless of whether you specify windows authentication or sql authentication with the SA password, you'll get the same problem.

There are two main methods for working around this problem, namely a manual upgrade using SQL scripts, or the method I chose.

For the manual upgrade, you can see Bjorn Isaksen's blog on it here:
Upgrade EPiServer database manually from 5 to 6

The solution

The method I chose was to do the following:

1) Ensure a code and CMS freeze to ensure that no data is updated and no content is updated during the upgrade.

2) Take a copy of the live database, and restore it to our development server.

3) Take a copy of the latest development code, happy that this is a codeset you're okay to go live with.

4) Set this copy of the code up in IIS as a separate EPiServer site.

5) Point the connection strings in that copy to point to your newly restored EPiServer database.

6) Run the upgrade wizard, following the prompts and ensuring that the upgrade's successful.

7) Publish the newly modified code (whether you export that from your source control directory, or build it manually, it's up to your current development process model).

8) Attach the new database from your development site to the user acceptance testing database server (essentially your backup copy of the live site). I restored this to a fresh database so that I could have a clean swap in the connectionstrings.config file.

9) Upload the code to the user acceptance testing development server.

10) Change the connectionstrings.config file to point to your newly restored database.

I then tested the functionality of the site to ensure that everything was behaving as it should, and hey presto! One upgraded site!

Reference

For further information on the upgrade process and what's involved in the filesystem changes, you can see Fredrik Haglund's blog here:

Configuration management and EPiServer CMS 6

Shrinking the transaction logs for a SQL Server database

Overview

Bare in mind that shrinking log files is not the same as compressing SQL Backups. It's important to note this, because if you Google for each, you'll return different results.

Compressing SQL Backups is the act of compressing *.bak files that are backups of the database itself.

Shrinking SQL Log files is the act of removing transaction logs that you don't want anymore; essentially, you are removing unwanted transactions from the database's .LDF file.

I've worked on a few projects that pull in several million visitors per week, where the database server has hardly any disk space left due to large transaction log files that don't need to be there in this business case.

FYI, I gathered the information to do this from this Microsoft article: http://support.microsoft.com/kb/907511

The process

There are two commands that have to be executed, which are explained below with an example database.

1) backup log dbTestDatabase to disk = 'C:\SQL\Backups\dbTestDatabaseLogs.bak'

This command is backing up your transaction log to a .bak file. In the process of doing this, the virtual logs that are in use by the DMBS are freed up after they're backed up. This then allows you to do step 2.

2) DBCC SHRINKFILE ('dbTestDatabase_Log', 1) WITH NO_INFOMSGS

This means that you're going to shrink the logical name for the dbTestDatabase.ldf file to 1mb and you don't want messages back. To get this name, right-click on your database, and choose properties. Then click "Files", and your logical name is shown on the row with the file type of "log".

This command removes the virtual logs that you've just backed up. If you run this command without running the backup first, you won't get the LDF file down to your desired size, as the DBMS will still be using a lot of the transactions you're trying to remove.

If you still can't get the log file down to your desired file size, a few virtual transactions may have piled up in the meantime, so run step one again, and then re-run step 2 quickly afterwards.