First step is to include the javascript needed to run the spell checking. You'll need two includes form.js and SpellCheckEntirePage.js.

<script type="text/javascript" language="javascript" 
  src="/_layouts/1033/form.js?rev=df60y6YolDjUVbi91%2BZw%2Fg%3D%3D"></script>
<script type="text/javascript" language="javascript"
  src="/_layouts/1033/SpellCheckEntirePage.js?rev=zYQ05cOj5Dk74UkTZzEIRw%3D%3D"></script>

When I saw SpellCheckEntirePage.js for the first time I had to laugh because my initial estimate of the task was 3-4 days, I ended up doing it in less than 4 hours.

Next step is to add the button to actually check the spelling.

<input type="button" value="Spell Check"
  onclick="javascript:SpellCheckEntirePage('<%= SPContext.Current.Web.Url %>/_vti_bin/SpellCheck.asmx', '<%= SPContext.Current.Web.Url %>/_layouts/SpellChecker.aspx');" />

Done!

Well almost. There were a couple of fields on the form which didn't make sense to spell check. But looking at the source of SpellCheckEntirePage.js you can quickly find the solution. Just add excludeFromSpellCheck="true" to the fields you don't want to check.

OK, now I'm done.

Well not quite yet. The "excludeFromSpellCheck" doesn't work on People pickers. But SharePoint has this problem too. If you edit a list item with a people picker and run the spell checker it will try to spell check people's login names which is never going to work. I went ahead and added a method to my master page which turns spell check off for people picker fields. It fixed the edit list item spell checking problem too :). I do have to warn you I suck at javascript so if anyone can send me a better way of doing this I would appreciate it.

function disableSpellCheckOnPeoplePickers() {
  var elements = document.body.getElementsByTagName("*");
  for (index = 0; index < elements.length; index++) {
    if (elements[index].tagName == "INPUT"
        && elements[index].parentNode
        && elements[index].parentNode.tagName == "SPAN") {
      var elem = elements[index];
      if (elem.parentNode.getAttribute("NoMatchesText") != "") {
        disableSpellCheckOnPeoplePickersAllChildren(elem.parentNode);
      }
    }
  }
}

function disableSpellCheckOnPeoplePickersAllChildren(elem) {
  try {
    elem.setAttribute("excludeFromSpellCheck", "true");
    for (var i = 0; i < elem.childNodes.length; i++) {
      disableSpellCheckOnPeoplePickersAllChildren(elem.childNodes[i]);
    }
  } catch (e) {
  }
}

Here is a list of why I don't like it, in no particular order:

• CodePlex -- First there was the SVN bridge (yes, someone hated TFS enough to make a product to make it look like something else) then Microsoft just caved and supported SVN directly. Not really a problem with TFS, it's just an indication that others feel the same way as I do.
• Size -- I guess Microsoft doesn't really like it either since they don't even ship it with Visual Studios, if you do need it, its a separate download and a big one at that -- 200MB+. Subversion, Tortoise SVN, and Ankh on the other hand combined are less than 15MB. How they filled up 200MB I have no idea, maybe they installed a client that doesn't suck somewhere I don't know about.
• Read-only Files -- Every file on your system is read-only. Why does everything need to be read-only. If I'm in another editor and I want to make a change to a file, I need to switch over to VS and check out a file for edit. This is rediculous.
• Identical Files -- I just want to see what changed. Why does TFS insist upon showing me all the files even if they are identical. Call me crazy but I like to see what files I actually changed when I check in. Someone told me there is some command line tool to see this but that's just silly.
• Code reviews/update -- On my current project I'm the project lead and I like to review the junior developers code when I update my code. Yeah, I can't do that, or I haven't found the button yet. Nor can I find the button to view a particular revision without finding a file that was part of that change and viewing it's history.
• Deleted files -- Team Explorer doesn't show them by default. This took me a while to figure this one out. You need to go into VS options then find TFS and then click show deleted files. Why is this not on by default and why isn't there a button to toggle this setting in Team Explorer.
• Project file modification -- We have some people working inside a VPN and some out, the TFS server name is different depending on where you sit. Since the solution file stores the connection string to TFS it's constantly getting checked in.
• Everything needs to be in the solution -- If you have a bunch of support files or non-.NET files, all of them need to be added to the solution or they don't get checked in or out. VS doesn't help you with this either because you can't have a solution folder map to a directory.
• Integrating with non-.NET developers -- If you have a mixed development environment (Java, Ruby, etc) like we do. You need to run TFS and something else because all the non-.NET developers can't use TFS especially if they are on a Mac. Usually the something else is much better anyway so you might as well use that instead.
• Working offline/Speed -- If you ever travel relax and take a nap because you won't be developing. Everything you do talks to the server and that in turn makes it slow. Open up a file and start making changes, to the server you go. Move a file, to the server you go. Diff a change you made, to the server you go.
• Workspaces -- Who ever came up with the concept of workspaces at Microsoft should be ashamed of him/herself. They are just a stupid idea. If you want two copies of the same project checked out (I do it with SVN if I'm working on a quick bug fix and a big feature at the same time) good luck because I can't figure it out.
• Server isn't free -- 'Nuff said.

I'm sure there are other problems but this is all I could think of while writing this.

Sure there are some nice things about TFS, but honestly the cons far outweigh the pros. So if you have to decide which SCM to use and you run across this blog I think it's pretty obvious that I don't recommend it. I love SVN and as GIT (tools) mature I'm sure I will switch to that.

Let me start by showing an overview of the workflow we are trying to create.

This is about as simple as it gets. Start -> Manager Approve -> Done.

If you look at "ManagerReview" you see we have three things.

• ManagerReviewInit - This activity is responsible for creating the task
  
  Gotcha #1 - If we create the task here we encapsulate the ability that later in the workflow we can transition back to the ManagerReview state and it will know how to create a new task.
• OnManagerReviewTaskCreatedEvent - This activity is going to initialize the task
• OnManagerReviewTaskChangedEvent - This activity is going to perform the logic of when the task is completed

ManagerReviewInit

OnManagerReviewTaskCreated

OnManagerReviewTaskChanged

public bool TaskComplete { get; set; }

private void OnManagerReviewTaskChanged_Invoked(object sender, ExternalDataEventArgs e) {
  TaskComplete = AfterTaskProperties.ExtendedProperties[SPBuiltInFieldId.TaskStatus].ToString() == "Completed";
}

My method takes a different approach. I started my journey by looking at the ListViewWebPart code directly in Reflector, because it obviously needs to know how to do this because that's where the user changes it from the web site. From there I looked at "ListViewWebPart.GetToolParts" method which is the method SharePoint calls to get additional web part setting controls from the web part. In the case of the ListViewWebPart it returned a ListViewToolPart. The ListViewToolPart had a method called "ApplyChanges", this looks promising. Damn, obfuscated. No worry I can decipher IL for the most part. Bingo, there was the call I was looking for "ApplyViewToListWebPart". It took all the parameters I would expect, a reference to the ListViewWebPart and a toolbar type.

Here is the resulting code to wrap all the reflection nasty bits...

public static class ListViewWebPartHelper {
  public static void SetToolbarType(
    this ListViewWebPart lvwp,
    SPLimitedWebPartManager limitedWebPartManager,
    ListViewWebPartToolbarType type) {
#pragma warning disable 618
    string storageKey = lvwp.StorageKey.ToString("B").ToUpper();
    string listId = lvwp.ListName;
    string viewId = lvwp.ViewGuid;
    uint toolbarType = (uint)type;

    // get the SPWebPartManager from SPLimitedWebPartManager
    PropertyInfo webPartManagerProp = typeof(SPLimitedWebPartManager).GetProperty(
      "WebPartManager",
      BindingFlags.NonPublic | BindingFlags.Instance);
    SPWebPartManager webPartManager = (SPWebPartManager)webPartManagerProp.GetValue(limitedWebPartManager, null);

    // get the SPWebPartConnection from SPWebPartManager
    PropertyInfo spWebPartsProp = typeof(SPWebPartManager).GetProperty(
      "SPWebParts",
      BindingFlags.NonPublic | BindingFlags.Instance);
    SPWebPartCollection webParts = (SPWebPartCollection)spWebPartsProp.GetValue(webPartManager, null);

    // Call the ApplyViewToListWebPart method on the SPWebPartConnection
    // internal void ApplyViewToListWebPart(
    //    string storageKey,
    //    string listID,
    //    string viewID,
    //    uint toolbarType,
    //    out uint flags)
    MethodInfo applyViewToListWebPart = typeof(SPWebPartCollection).GetMethod(
      "ApplyViewToListWebPart",
      BindingFlags.NonPublic | BindingFlags.Instance,
      null,
      new[]{
        typeof(string),
        typeof(string),
        typeof(string),
        typeof(uint),
        typeof(uint).MakeByRefType()
      },
      null);
    object[] parameters = new object[5];
    parameters[0] = storageKey;
    parameters[1] = listId;
    parameters[2] = viewId;
    parameters[3] = toolbarType;
    applyViewToListWebPart.Invoke(webParts, parameters);
#pragma warning restore 618
  }
}

public enum ListViewWebPartToolbarType {
  Full = 0,
  Summary = 1,
  NoToolbar = 2
}

And here is how to use the code...

SPFile homePage = _ctx.Site.Files["default.aspx"];
SPLimitedWebPartManager limitedWebPartManager = homePage.GetLimitedWebPartManager(PersonalizationScope.Shared);
ListViewWebPart webPart = (ListViewWebPart)limitedWebPartManager.WebParts[0];
webPart.SetToolbarType(limitedWebPartManager, ListViewWebPartToolbarType.NoToolbar);
limitedWebPartManager.SaveChanges(webPart);

And here is my rant to Microsoft that each and everyone of these articles about this issue must include... Really, Microsoft did you not think that people would want to do this. You have had 2 service packs come out for SharePoint and you couldn't have fixed this yet. Please fix this in SharePoint 2010.

If you have a class like this and need to call "PrivateMethod"

public class PrivateMethodClass {
  private int PrivateMethod(int arg1, ref int arg2, out int arg3) {
    arg2 = arg2 + 10;
    arg3 = 999;
    return arg1;
  }
}

This is the code you need to call it

PrivateMethodClass target = new PrivateMethodClass();
MethodInfo methodInfo = typeof(PrivateMethodClass).GetMethod(
  "PrivateMethod",
  BindingFlags.NonPublic | BindingFlags.Instance,
  null,
  new[] {
    typeof(int), 
    typeof(int).MakeByRefType(), 
    typeof(int).MakeByRefType()
  },
  null);
object[] parameters = new object[3];
parameters[0] = 1;
parameters[1] = 2;
parameters[2] = 3;
int result = (int)methodInfo.Invoke(target, parameters);
Console.WriteLine("parameter[0]: " + parameters[0]);
Console.WriteLine("parameter[1]: " + parameters[1]);
Console.WriteLine("parameter[2]: " + parameters[2]);
Console.WriteLine("result: " + result);

There are two things that are different from a typical reflection call. The first is the call to "MakeByRefType" which as the name suggests makes a reference type from a type. This is required for a "ref" or a "out" parameter. The second is the creation of the parameters array before invoking the method.

First off in your element manifest file you need to use the "ControlAssembly" and "ControlClass" attributes off of CustomAction. MSDN doesn't say a whole lot about these attributes but essentially they allow you to specify a web control class in your assembly which can render the action. (If you need to create hierarchical menus check out this article.) Here is the XML to get you started:

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
  <CustomAction
    Id="MyCustomAction"
    RegistrationType="List"
    GroupId="ActionsMenu"
    Location="Microsoft.SharePoint.StandardMenu"
    Sequence="1000"
    ControlAssembly="[Fully qualified assembly name]"
    ControlClass="MyNamespace.MyCustomAction">
  </CustomAction>
</Elements>

Next you will need to create a web control class which renders the menu item, as seen below:

public class MyCustomAction : WebControl {
  private MenuItemTemplate _action;

  protected override void CreateChildControls() {
    SPWeb site = SPContext.Current.Web;

    _action = new MenuItemTemplate {
      Text = "My Action",
      Description = "My Action",
      ImageUrl = "/_layouts/images/NEWITEM.GIF",
      ClientOnClickNavigateUrl = "http://www.nearinfinity.com"
    };

    Controls.Add(_action);
  }
}

Compile and deploy your solution. Oh and make sure to add your assembly and namespace to the safe control list or you will end up pulling your hair out because once again SharePoint will silently not render your menu item. You should now see the "My Action" on every list in the site. We have now reproduced what a standard CustomAction element in your element manifest would do normally. Now we need to find a way to determine which list our control is being added to. The best way I found for doing that is to traverse the control hierarchy up the parent chain until I found the containing ListViewWebPart. The code is quite simple and looks like this:

private ListViewWebPart GetParentListViewWebPart() {
  Control parent = Parent;
  while (parent != null) {
    if (parent is ListViewWebPart) {
      return (ListViewWebPart)parent;
    }
    parent = parent.Parent;
  }
  return null;
}

You should know where I'm going with this now, but for completness I will show you the last piece of code below which restricts it to a single list instance:

private Guid TARGET_LIST_ID = new Guid ("0D9B9302-8599-4CE5-8695-1B95FE7378F1");

protected override void OnLoad(EventArgs e) {
  base.OnLoad(e);
  if (!Page.IsPostBack) {
    EnsureChildControls();
    _action.Visible = false;
    ListViewWebPart listView = GetParentListViewWebPart();
    if (listView != null) {
      Guid listGuid = new Guid(listView.ListName);
      if (TARGET_LIST_ID == listGuid) {
        _action.Visible = true;
      }
    }
  }
}

Now that we have the list Guid that our control is being rendered for I will leave it as an exercise to the reader to restrict it to a given content type.

Why SharePoint doesn't provide this kind of functionality out of the box amazes me, but at least they provide a powerful API that you can get in there and work around it in a not so horribly kludgy way. Maybe SharePoint 2010 will include this :)

I’m going to ignore samples 2 (String.Format) and 4 (String.Replace) because those are obviously going to be slow. They require parsing the string to find tokens.

Sample 5 (StringBuilder) is a little more interesting. Looking at the StringBuilder in Reflector you’ll see it’s doing some tricks under the cover to handle multi-threaded access to the StringBuilder object. Which from what I can deduce causes performance degradations. Not to mention the need to make 10 method calls (Append) and one object allocation (StringBuilder) from my code.

Sample 1 (Simple Concatenation) and 3 (string.Concat) are actually identical when viewed in Reflector (other than the minor difference of new lines, but that is caused by the difference in the code).

1. Simple Concatenation .maxstack 3 .locals init ( [0] string s, [1] string CS$1$0000, [2] string[] CS$0$0001) L_0000: nop L_0001: ldc.i4.s 10 L_0003: newarr string L_0008: stloc.2 L_0009: ldloc.2 L_000a: ldc.i4.0 L_000b: ldstr "<div class=\"user-action-time\">" L_0010: stelem.ref L_0011: ldloc.2 L_0012: ldc.i4.1 L_0013: call string perftest.Program::st() L_0018: stelem.ref L_0019: ldloc.2 L_001a: ldc.i4.2 L_001b: call string perftest.Program::st() L_0020: stelem.ref L_0021: ldloc.2 L_0022: ldc.i4.3 L_0023: ldstr "</div>\r\n<div class=\"user-gravatar32\">" L_0028: stelem.ref L_0029: ldloc.2 L_002a: ldc.i4.4 L_002b: call string perftest.Program::st() L_0030: stelem.ref L_0031: ldloc.2 L_0032: ldc.i4.5 L_0033: ldstr "</div>\r\n<div class=\"user-details\">" L_0038: stelem.ref L_0039: ldloc.2 L_003a: ldc.i4.6 L_003b: call string perftest.Program::st() L_0040: stelem.ref L_0041: ldloc.2 L_0042: ldc.i4.7 L_0043: ldstr "<br/>" L_0048: stelem.ref L_0049: ldloc.2 L_004a: ldc.i4.8 L_004b: call string perftest.Program::st() L_0050: stelem.ref L_0051: ldloc.2 L_0052: ldc.i4.s 9 L_0054: ldstr "</div>" L_0059: stelem.ref L_005a: ldloc.2 L_005b: call string [mscorlib]System.String::Concat(string[]) L_0060: stloc.0 L_0061: ldloc.0 L_0062: stloc.1 L_0063: br.s L_0065 L_0065: ldloc.1 L_0066: ret

3. string.Concat .maxstack 3 .locals init ( [0] string s, [1] string CS$1$0000, [2] string[] CS$0$0001) L_0000: nop L_0001: ldc.i4.s 10 L_0003: newarr string L_0008: stloc.2 L_0009: ldloc.2 L_000a: ldc.i4.0 L_000b: ldstr "<div class=\"user-action-time\">" L_0010: stelem.ref L_0011: ldloc.2 L_0012: ldc.i4.1 L_0013: call string perftest.Program::st() L_0018: stelem.ref L_0019: ldloc.2 L_001a: ldc.i4.2 L_001b: call string perftest.Program::st() L_0020: stelem.ref L_0021: ldloc.2 L_0022: ldc.i4.3 L_0023: ldstr "</div><div class=\"user-gravatar32\">" L_0028: stelem.ref L_0029: ldloc.2 L_002a: ldc.i4.4 L_002b: call string perftest.Program::st() L_0030: stelem.ref L_0031: ldloc.2 L_0032: ldc.i4.5 L_0033: ldstr "</div><div class=\"user-details\">" L_0038: stelem.ref L_0039: ldloc.2 L_003a: ldc.i4.6 L_003b: call string perftest.Program::st() L_0040: stelem.ref L_0041: ldloc.2 L_0042: ldc.i4.7 L_0043: ldstr "<br/>" L_0048: stelem.ref L_0049: ldloc.2 L_004a: ldc.i4.8 L_004b: call string perftest.Program::st() L_0050: stelem.ref L_0051: ldloc.2 L_0052: ldc.i4.s 9 L_0054: ldstr "</div>" L_0059: stelem.ref L_005a: ldloc.2 L_005b: call string [mscorlib]System.String::Concat(string[]) L_0060: stloc.0 L_0061: ldloc.0 L_0062: stloc.1 L_0063: br.s L_0065 L_0065: ldloc.1 L_0066: ret

So now lets get to why I’m writing this post. Trying to micro-optimize code that is already micro-optimized. So lets start by looking at what string.Concat is doing inside.

public static string Concat(params string[] values)
{
    int totalLength = 0;
    if (values == null)
    {
        throw new ArgumentNullException("values");
    }
    string[] strArray = new string[values.Length];
    for (int i = 0; i < values.Length; i++)
    {
        string str = values[i];
        strArray[i] = (str == null) ? Empty : str;
        totalLength += strArray[i].Length;
        if (totalLength < 0)
        {
            throw new OutOfMemoryException();
        }
    }
    return ConcatArray(strArray, totalLength);
}

string s =
string.Concat(
    string.Concat(@"<div class=""user-action-time"">", st(), st()),
    string.Concat(@"</div><div class=”“user-gravatar32”“>”, st(), @”</div><div class=”“user-details”“>”, st()),
    string.Concat(“<br/>”, st(), “</div>”));
return s;

I had a bet with a co-worker that C# would outperform F# for a simple counting exercise. This turned out to be quite a surprise for many reasons. In this blog post I will try to explain why.

Problem

Sum the numbers from 1 to n using brute force (can't use the (1 + N)*(N/2) equation). Do this in both C# and F# and compare the speed. Since F# needs to do this recursively I thought this should be a no contest win for C#.

Using F#

We initially wrote the code without using tail recursion which immediately ended with a stack overflow. So we went back and wrote it doing it the functional way, with tail recursion. Here is what we came up with.

module FSharpLib

	let sumNum fromNum toNum =
		let rec sumNumTail fromNum toNum result =
			if fromNum > toNum then 
				result 
			else 
				sumNumTail (fromNum+1L) toNum (result+fromNum)
		sumNumTail fromNum toNum 0L

Using C# - Attempt 1

You'll see later why I say attempt 1. But, this is the simplest C# code I could think of.

public static long CSharpForLoop(long fromNumber, long toNumber) {
	long result = 0;
	for (long i = fromNumber; i <= toNumber; i++) {
		result += i;
	}
	return result;
}

The Results - Attempt 1

This is the results of running the above code using a fromNumber of 1 and a toNumber of 10000000 and running it 10 times alternating between C# and F# to filter out start-up times, etc.

Reflector is an amazing tool

I turned to my good friend Reflector to see what was going on. Here is what the F# code looked like when viewing it as C# code.

public static long sumNum(long fromNum, long toNum)
{
    FastFunc<long, FastFunc<long, FastFunc<long, long>>> sumNumTail = new clo@7();
    return FastFunc<long, long>.InvokeFast3<long, long>(sumNumTail, fromNum, toNum, 0L);
}

public override long Invoke(long fromNum, long toNum, long result)
{
	while (fromNum <= toNum)
	{
		result += fromNum;
		toNum = toNum;
		fromNum += 1L;
	}
	return result;
}

C# - Attempt 2

public static long CSharpWhileLoop(long fromNumber, long toNumber) {
	long result = 0;
	while (fromNumber <= toNumber)
	{
		result += fromNumber;
		// I left out the toNum = toNum because it didn't do anything.
		fromNumber += 1L;
	}
	return result;
}

To the IL code Batman

This is the IL for the F# code.

	L_0000: nop              // while loop
	L_0001: nop 
	L_0002: ldarg.1          // fromNum <= toNum
	L_0003: ldarg.2 
	L_0004: ble.s L_0009     
	L_0006: nop              // return result
	L_0007: ldarg.3 
	L_0008: ret              
	L_0009: nop              // result += fromNum
	L_000a: ldarg.1 
	L_000b: ldc.i8 1
	L_0014: add              
	L_0015: ldarg.2          // fromNum += 1L
	L_0016: ldarg.3 
	L_0017: ldarg.1 
	L_0018: add              
	L_0019: starg.s result   // loop
	L_001b: starg.s toNum
	L_001d: starg.s fromNum
	L_001f: br.s L_0000      
	

	L_0000: nop                   // long result = 0L
	L_0001: ldc.i4.0
	L_0002: conv.i8 
	L_0003: stloc.0               
	L_0004: br.s L_0012           // while loop
	L_0006: nop                   // result += fromNumber
	L_0007: ldloc.0 
	L_0008: ldarg.0 
	L_0009: add                   
	L_000a: stloc.0               // fromNumber += 1L
	L_000b: ldarg.0 
	L_000c: ldc.i4.1 
	L_000d: conv.i8 
	L_000e: add                   
	L_000f: starg.s fromNumber    
	L_0011: nop 
	L_0012: ldarg.0               // fromNumber <= toNumber
	L_0013: ldarg.1 
	L_0014: cgt 
	L_0016: ldc.i4.0 
	L_0017: ceq 
	L_0019: stloc.2 
	L_001a: ldloc.2 
	L_001b: brtrue.s L_0006       // loop
	L_001d: ldloc.0 
	L_001e: stloc.1 
	L_001f: br.s L_0021
	L_0021: ldloc.1 
	L_0022: ret 
	

What just happened.

Wow, the C# compiler is quite a bit more verbose. First of all, C# makes the while loop into a do/while loop. Secondly, in the loop condition C# creates a temporary variable to store the comparison and then checks for 'true' ("brtrue.s"). If you look at the F# IL you'll see the use of the "ble.s" instructions (branch if less or equal) instead. Finally we see that C# uses a few "conv.i8" instructions (convert to 8 byte/64-bit int) which I can only guess adds to it's defeat.

Conclusion

So is F# just a more performant language or is C# just compiling to more robust IL code which as F# matures will end up suffering the same fate? I personally think (hope actually) the later is true. I'm not an expert in compilers but I would have to assume the C# compiler team is using every trick in the book to make highly performant IL code without sacrificing robustness and I can only assume that F# will get to a point where it needs to make the same concessions to handle some obscure cases.

There are two drivers for windows, one when the badge is in USB bootloader mode and one while the badge is in normal mode. To get to bootloader mode you need to hold down the mode select button while inserting the battery. The driver provided on the CD is for normal mode which took me a while to figure out. To get to the normal mode you need to plug the USB cord in BEFORE you insert the battery. Looking at the firmware I noticed the check for a USB connection is done as one of the very first things. Windows will then recognize the device and you can install the driver from the CD. The driver for bootloader mode is actually part of the Freescale JM60 bootloader program which can also be found on the CD but must be installed before you try bootloader mode.

Also, I couldn't get the full version of Code Warrior so the next best thing was the demo from Freescale. But, the demo has a 32k firmware limit which means removing some of the original firmware code. I removed fat.c from the project and recompiled and fixed until I got it working. This will of course remove your ability to read or write to the SD card using FAT.

Next step is to make the final hack which will hopefully be done by tomorrow so we can show it off to Kingpin. I'll post the final hack information and code tomorrow if it gets done.

Code You Write To Make It Work

   1:  IEnumerable<int> numbers = Enumerable.Range(0, 500);
   2:  RemoteContext remoteContext = new RemoteContext(
   3:     new[] { this.GetType().Assembly }, 
   4:     new[] { "localhost", "someremotehost" },
   5:     7776);
   6:  RemoteOptions remoteOptions = new RemoteOptions();
   7:  IEnumerable<int> results = numbers.AsRemotable(remoteContext, remoteOptions).Select(i => DoSomeWork(i));

Let's walk through the code.

• Line 1, gives us an enumerable of numbers from 0 to 500 for processing.
• Line 2, creates the RemoteContext. This is really just a client which has a list of server connections to which we can send work.
• Line 3, is a list of assemblies that contain the necessary code to run your LINQ statement. The client will actually send a copy of the dll to the RemoteLINQ server so that the server will have a copy of the "DoSomeWork" (line 7) method so that it can execute it.
• Line 4, is the list of servers to divvy the work across.
• Line 5, is just the port that the client will try to connect to the server on.
• Line 6, contains any options to direct how the work gets split across the servers, etc.
• Line 7, is your LINQ statement. Notice the "AsRemotable". This is where all the magic starts. This will actually return a IRemoteEnumerable which knows how to take anything to the right of it and run it remotely.

How it works

Lets start out with what lambda expression look like in compiled code. When we compile lambda expressions they are moved into a new method with a compiler generated name. If you reflect on a class with lambdas and get all the methods on that type you will see a bunch of method names which you didn't create. Those are your lambdas. The fact that they are first class methods allows us from the outside, with reflection of course, to call those methods. So when we call "Select" on an IRemoteEnumerable, RemoteLINQ is taking the lambda expression passed in, and storing information about it into a serializable object (assembly, declaring type, method name, etc.). This will allow us to send that information to the server, who can then call the method without all the other stuff around it.

On the server side of things, when we start up a RemoteLINQ server, it will create a worker thread for each processor in the machine. The worker thread will wait quietly until new work arrives on the socket connection. When work arrives we take the serialized object I mentioned before and find the assembly, type, and method and execute that method with the work given to us. In this case the work will be one of the numbers from 0 to 500. The client will handle load balancing the servers by choosing the server with the least amount of pending work per processor. The amount of work queued up on the server is sent back after each unit of work is completed so that the client will have the most up to date picture of what the servers are doing to make the best guess as to where to send the next piece of work.

That's about it. You now have a way to split complex LINQ tasks across multiple machine with just a single call on your LINQ statement.

Download The Code

The Test Data

<root>
  <child id='123'/>
  <child id='234'/>
  ...
</root>

The Test

XmlDocument.Load

private static void XmlDocumentReader(string fileName) {
    XmlDocument doc = new XmlDocument();
    doc.Load(fileName);
    XmlNodeList nodes = doc.SelectNodes("//child");
    if (nodes == null) {
        throw new ApplicationException("invalid data");
    }
    foreach (XmlNode node in nodes) {
        string id = node.Attributes["id"].Value;
        ProcessId(id);
    }
}

LINQ to XML

private static void XDocumentReader(string fileName) {
    XDocument doc = XDocument.Load(fileName);
    if (doc == null | doc.Root == null) {
        throw new ApplicationException("invalid data");
    }
    foreach (XElement child in doc.Root.Elements("child")) {
        XAttribute attr = child.Attribute("id");
        if (attr == null) {
            throw new ApplicationException("invalid data");
        }
        string id = attr.Value;
        ProcessId(id);
    }
}

XmlReader

private static void XmlReaderReader(string fileName) {
    using (XmlReader reader = new XmlTextReader(fileName)) {
        while (reader.Read()) {
            if (reader.NodeType == XmlNodeType.Element) {
                if (reader.Name == "child") {
                    reader.MoveToAttribute("id");
                    string id = reader.Value;
                    ProcessId(id);
                }
            }
        }
    }
}

The Results

UTF8Encoding

ASCIIEncoding

UTF32Encoding

Conclusion