About Bryan Matthews

I am a software developer for VC3, Inc., an IT consulting company in Columbia, SC. In addition to my professional software work I also dabble in python/django web development as well as jQuery and other JavaScript goodness. In my free time I play guitar and enjoy outdoor activities as much as possible.

Learning F# with TDD: Part 2

Last time I talked about setting up F# testing using NUnit, TestDriven.Net, and NaturalSpec.  This time around I’ll elaborate a little bit on the testing aspects, and also talk about active patterns.

Running tests

First, I noticed a little problem with running the tests.  If I right-clicked on a test method and selected “Run Test(s)”, then the test would run as expected.  However, if I selected “Run Test(s)” for a project or solution, the tests would not be found as described in the getting started post, so I would see something like “0 passed, 0 failed, 0 skipped, …”.

The solution for me was to transition from using local spec methods to defining a “Tests” class, with the “TestFixtureAttribute”, and define my specs as static methods (they could be instance methods if needed of course).

[<TestFixture>]
type Tests = class
    new() = {}

    [<Scenario>]
    static member When_... () =
        // test body

end

 

Voila, tests run as expected!

Now, a little explanation of how you write the specs…

Writing specs

From  Introducing NaturalSpec – A Domain-specific language (DSL) for testing – Part I

With the Keyword “Given” I can create a test context (the objects I want to test). In this sample I created a list with 5 elements. With the keyword “When” I call a function which does something with my test context. In this case I want to remove the value 3. In the Assert section (keywords “It should” or “It shouldn’t”) I can give some observations, which should hold for my manipulated test context.

  • Create the object(s) to be tested using “Given”
  • Pipe into “When” to call functions (or do other things)
  • Pipe into “It should/shouldn’t” to set up assertions
  • Finally, pipe into the “Verify” method to run the tests and print the output

Here’s an example:

[<TestFixture>]
type Tests = class
    new() = {}

    [<Scenario>]
    static member When_getting_the_state_of_a_new_connection_it_should_be_closed () =
        Given NewConnection()
            |> When getting DbConnection.State
            |> It should equal "Closed"
            |> Verify

end

 

The NewConnection() method call is entirely fabricated. It serves as the test context in the example.

“When getting” is followed by a method that takes as input the test context object type (presumably a connection object) and returns some value (string).

The “equal” method, based on the previous information, takes in two strings and asserts that they are equals.

And here’s the output:

Scenario: When getting the state of a new connection it should be closed
  – Given <fun:When_getting_the_state_of_a_new_connection_it_should_be_closed…
     - When getting 
      => It should equal “Closed”
  ==> Result is: “Closed”
  ==> OK
  ==> Time: 0.2060s

1 passed, 0 failed, 0 skipped, took 0.61 seconds (NUnit 2.5.10).

 

There’s room for improvement, but it’s good enough for now, so on to active patterns!

Active Patterns

From MSDN:

Active patterns enable you to define named partitions that subdivide input data, so that you can use these names in a pattern matching expression just as you would for a discriminated union. You can use active patterns to decompose data in a customized manner for each partition.

 

One example they give is even and odd.  You can define a pattern for “Even” or “Odd” and use that in pattern matching like so:

match input with
    | Even -&gt; // do something if even...
    | Odd -&gt; // do something if odd...

 

Compare that with:

if (input % 2 === 0) {
    // do something if even...
}
else {
    // do something if odd...
}

 

Of course, you could do something like this:

if (isEven(input)) {

 

…but that still doesn’t read quite as well in my opinion.

Even more interesting, you can define partial active patterns that might match the given input, and if so, can transform the result in some way.  The example they gave was number parsing.

let (|Integer|_|) (str: string) =
    let mutable i = 0
    if System.Int32.TryParse(str, &amp;i) then Some(i)
    else None

let (|Float|_|) (str: string) =
    let mutable f = 0.0
    if System.Double.TryParse(str, &amp;f) then Some(f)
    else None

// Consuming code somewhere...
match str with
    | Integer i -&gt; printfn "%d : Integer" i
    | Float f -&gt; printfn "%f : Floating point" f
    | _ -&gt; printfn "%s : Not matched." str

 

This illustrates what makes pattern matching different from traditional control structures. It’s actually a reusable control structure plus potential augmentation of the input.

Here’s a post where someone who really digs pattern matching attempts to explain why, if you’re interested.

ExoWeb type loading changes

I recently posted a message to the ExoSuite Google Group in regards to type loading in ExoWeb.

Batch loading

Types are now batch loaded. This means that prior to type caching your pages should only send a single type request (or very few depending on your usage). This was implemented to cut down on the large number of requests sent from the browser, since browsers have limits on the number of concurrent request.

You don’t have to do anything to enable this functionality.

Type loading and rules

We noticed that a surprisingly large number of types were being requested, and some of them didn’t seem like types that should be needed. I was able to track this down to the “AllowedValues” rule, which was force loading types when instantiated. Since this rule is very common, and often spans types, this would result in a cascading type loading effect. I changed the rule’s active type loading behavior to a passive approach: it becomes enabled when the relevant types are loaded.

There are two possible side-effects of this change that come to mind.

First, if you happen to have an allowed values property on a type that is neither the containing type of the property or the property type, then the rule may never be enabled, even if the data is all loaded and editable. This is an odd scenario that shouldn’t happen in practice. To illustrate, if type Person has a property Org of type Organization, and its allowed values are World.AllOrganizations, and type World is not used, then the rule will never be activated. The only way that this example makes sense is if the type World is a required type for the entire app (or feature area), and so will always be available.

Second, if you are not actually referencing a type in your query paths but subsequently used that type in templates/scripts or expected a $extend to fire when the type was loaded, then in the past you may gotten away with this because of the overly aggressive nature of type loading. Now, types are only loaded because they are referenced in your queries or are otherwise required for object loading (i.e. due to lazy loading). One caveat: several other rules (e.g. CompareRule) may still force type loading to occur. They are less likely to actually change the nature of type loading, but still this behavior may also be changed in the future for other rules.

The moral of the story: don’t depend on rules to perform type loading.

Learning F# with TDD: Part 1

I decided to dive into a few languages I’ve been meaning to learn by way of the calculator kata.  First up: F#.

First, some prerequisites.

  • F#, obviously.  This comes with VS 2010 so you probably already have it.
  • We’re doing TDD, so we need some sort of testing framework.  I decided to go with NaturalSpec.  This means we’ll need to download the package from github, build it, and also download and install NUnit and TestDriven.NET.  The author has a good blog post on setting up and running the test project.

Keep in mind that I’m extremely new to the language, so the code I ended up with probably doesn’t reflect what a seasoned F# programmer would write.  I tried to follow conventions that I could pick out as much as possible.  The first part of the kata is an add function.  Pretty simple.  In the interest of not getting bogged down in one language I didn’t take if very far, so I will only be able to make very broad observations at this point.

Two positive things that I took away from it:  pattern matching and pipe.

Pattern matching is something that I knew was popular in functional programming circle but had never experienced first hand.  Now that I have a taste of it I think it’s a handy abstraction, but I have a feeling that I haven’t even begun to scratch the surface of what it could mean.

let addNumbers (numbers : List) =
    match numbers.Length with
    | 0 -> 0
    | 1 -> numbers.[0]
    | _ -> numbers |> Seq.reduce (fun a b -> a + b)

This is simply an alternative form of the familiar if-else branching logic, nothing special really, but it is pleasantly terse.

Pipe is even more exciting.  Take the following for example.

numbers.Split ',' |> Array.toList |> List.map int |> addNumbers

Using the pipe operator (that’s “|>”) you can stream a value through a series of transformations, in this case splitting the string, converting the array to a list, converting the items in the list to integers, and finally passing the list to a method called “addNumbers”.

In any other language this would probably read similarly if your transformations are all instance methods, however, that breaks down as soon as you have to use global or static/class methods.  The pipe form always reads nicely from the beginning to the end.  Of course, this example could be a little simpler since our function can technically accept an array as its argument.

To be continued…

Disable the audible beep in Mac Terminal

This was starting to get on my nerves.  I really don’t need a fancy sound clip to keep reminding me that I deleted as far as I can go.  I always do that, since I’m a bit OCD.  This one took me too long to find, mainly becuase of tabs.

Anyway, to disable the sounds, go to Terminal -> Preferences -> Settings (top tab group) -> Advanced (inner tab group), look for the “Bell” group label and uncheck the “Audible Bell” option.  Note that this is specific to a particular settings configuration/theme, so if you switch themes the behavior will change.

Tfs Rollback

Using the tf command line tool you can rollback all or part of a tfs checkin.  This is useful for two reasons: 1) If you at least want to roll back all changes in one or more file (not some changes in a file) then it saves you from having to manually revert changes (I’m a firm believer in not giving yourself the opportunity to goof up) , and 2) the check is actually marked as a rollback, so you don’t have to rely on checkin notes to identify it as such.

As a side note, based on what I’ve seen in the way that tfs represents changes, this rollback changeset is simply an add, edit or delete, with the additional information that it was a rollback of a prior change.  The original checkin is still in the tfs history.

To do a rollback, first open up the visual studio command prompt.  On my machine the command is: %comspec% /k “”c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat”" x86.

Here’s the help for tf rollback (using the command tf rollback /?):

TF – Team Foundation Version Control Tool, Version 10.0.30319.1

Copyright (c) Microsoft Corporation.  All rights reserved.

Rolls back the changes in a single or a range of changesets:

tf rollback /changeset:changesetfrom~changesetto [itemspec] [/recursive]

            [/lock:none|checkin|checkout] [/version:versionspec]

            [/keepmergehistory] [/noprompt] [/login:username,[password]]

tf rollback /toversion:versionspec itemspec [/recursive]

            [/lock:none|checkin|checkout] [/version:versionspec]

            [/keepmergehistory] [/noprompt] [/login:username,[password]]

Versionspec:

    Date/Time         D”any .Net Framework-supported format”

                      or any of the date formats of the local machine

    Changeset number  Cnnnnnn

    Label             Llabelname

    Latest version    T

    Workspace         Wworkspacename;workspaceowner

First, we need to make sure we are working within our workspace.

cd path\to\workspace\root

So, for example, we could rollback an entire changeset:

tf rollback /changeset:12345

Or, we can rollback a single item in the changeset:

tf rollback /changeset:12345 $/Path/To/Item

There is also a login param that you can pass if needed:

tf rollback /changeset:12345 /login:domain\username,password

Enjoy!

 

Visual Studio External Tools

Here’s a neat visual studio tip. You can run any external command from within visual studio, and add a button to the command bar or map it to a keyboard shortcut. This is really important to me because I rely heavily on keyboard shortcuts. I love that I can hit a combination of keys and perform a repetitive task automatically, plus the result can be incorporated into visual studio by way of the output window.

Some examples of what you might use this for:

  • Get latest on several team projects to start your day.
  • Check out a specific set of files.
  • Automate various git commands from within visual studio.
  • Run a batch file with a set of arguments that you define when you run it.
  • Launch another application (i.e.: web browser).

What follows is inspired by Rob Conery’s Git series on tekpub. You’ll have to watch the full video to see how he uses external tools to incorporate git in Visual Studio.


To get started from the menu go to Tools –> External Tools.

Then you should see a dialog that looks like this:

Some of the options are pretty self-explanatory.

  • Title is obvious.
  • Command is the path to the command to execute, also obvious.
  • Arguments is pretty simple on the surface, but we have some interesting options at our disposal, which you can use to make the command contextual.

  • We have similar options for the initial directory.

In this example I’m using the file name and extension of the current item (open document) to run using node, assuming it’s a JavaScript file. I selected “Use Output window” so that the output will show up in visual studio, rather than launching a separate window. This is handy if you don’t need to perform any additional tasks after the fact.

So, now that you have your external command, you can run it from the “Tools” menu.

You can also add buttons to the toolbar, map keyboard shortcuts (search for “ExternalCommand1, “ExternalCommand2”, etc.), and maybe even add context menu items (though I haven’t figured that one out completely).

Intro to Client-Server Mapping in ExoWeb

As a follow up to my Brief Introduction to ExoWeb, I would like to go into more detail about the communications between client and server.

The client interacts with the server in two basic ways.

  1. Pulling information: type and instance data.
  2. Pushing information: changes to the model and/or instructions to do something.

The first type, pulling information, is pretty straightforward. Type information doesn’t change (frequently) so it is actually cached on the client if possible. Querying for instance data is similar to querying a database. You request an object of a particular type with a particular id, but you can also include a set of paths that tell the services what additional related data you are interested in. The primary component of a query might look something like this.

{ from: "Person", id: "1", and: ["this.Additional.Properties.To.Load"] }

The second type, pushing information, gets a little more interesting. If you’re going to sync data between two disconnected systems there are two primary approaches that you might take. First, you could send simple data from point A to point B, manipulate it, then send it back from point B to point A in the same form. As you could imagine this can get pretty expensive and tedious. Of course, you might just send the portions of data that have changed to cut down on overhead. The bigger problem, I think, is that you don’t really know what has happened unless you do full graph comparisons. Another approach is to keep track of changes as they occur and send those back and forth. This is what ExoWeb does.

changes: [ { type: "ValueChange", instance: { id: "1", type: "Person" }, property: "FirstName", oldValue: "Rob", newValue: "Robert" }, ... } ]

The client can also raise events, for example “save”, a built-in event. In response, the server can send back additional changes to the model.

changes: [ { type: "Save", idChanges: [ { type: "Person", from: "?4", to: "644" } ] }, ... ]

The payload isn’t limited to data. It can also include conditions, for example, permissions, or invalid data conditions. More on that later.

This has been a very basic introduction to the way client-server communication works in ExoWeb. Look out for more updates in the future.

A Brief Introduction to ExoWeb

A while back I talked about the ExoWeb project and scripts that are now hosted on github. That post was a little vague because not much has been documented about what you can actually do with ExoWeb. That’s what I hope to do here.

First off, let me restate the purpose of ExoWeb as simply as I can. ExoWeb exists to bridge the gap between the client and server. You can think of it as a combination client-server mapper and model-view mapper.

Client Server Mapping

When I say that ExoWeb is a client-server mapper what I mean is that it handles mapping from an object model on the server to the client (and vice versa) in much the same way that an ORM maps from a database schema to an object model. The general idea is that you’re persisting objects and that you would like to be able to easily manipulate those objects in JavaScript code, without any extra work.

For example, say you have a C# class that is persisted to a database using your choice of ORM.

public class Person
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string FullName
    {
        get
        {
            return FirstName + " " + LastName;
        }
    }
}

You would want to be able to write JavaScript code to modify this object like so (ignoring the details of the syntax [1]).

person.set_FirstName("Bob");

ExoWeb aims to let you do this with very little effort.

What are some of the things that you might want to do with the model?

  • Have the full name automatically reflect the fact that the first name is now “Bob”
  • Save these changes by simply calling a method
  • “Round trip” to the server to perform complex or sensitive operations
  • Display the person’s information in a view

…which leads us to the next primary area of functionality.

View Model Mapping

You have a model on the client, so now you want to display some information to the user and possibly accept input. What are some of the tools that you might want to have at your disposal?

  • Create regions in your form that correspond to lists in the model
  • Two-way binding of properties to inputs elements and form regions
  • Respond to changes and redraw parts of the form as needed
  • Attach behavior to the form such as clicking links, toggling regions, etc
  • Take advantage of model metadata like formats, allowed values, validation rules, etc
  • Allow for lazy loading of certain information when it is needed on the form

That’s a glance at the basic features that we want from ExoWeb. So, what does it actually look like?

Disclaimer: ExoWeb is under heavy development, so this stuff is always changing. Feedback is welcome.

Setting up a Page

First off we have to include the ExoWeb script(s). Take a look at the github repo to see what’s available. The “dist” folder contains 3 scripts: exoweb-msajax.js, exoweb-msajax-nojquery.js, and jquery.exoweb-msajax.js. For now we’ll just include exoweb-msajax.js. ExoWeb currently also requires jQuery (1.3 or 1.4) and the MsAjax client libraries (specifically, MicrosoftAjax.js and MicrosoftAjaxTemplates.js). These will be included in the repo in the near future.

Side note: ExoWeb currently uses the MsAjax client libraries, primarily for two purposes: observer and integrated template rendering. In the near future these dependencies should be isolated so that other implementations could be used. For example, better integration with jQuery is a personal goal of mine.

Now that we have the right scripts included we need to set up the page’s context. This tells ExoWeb what data to load and what to do before and after it is loaded. We do this by calling the $exoweb function. It accepts a JavaScript object with any of the following properties.

  • model: A JavaScript object that defines the data to load from the server.
  • types: An array of types to load. This is usually not needed since types are automatically loaded based on the “model” parameter.
  • init: A function that is invoked as soon as model or type data has been requested.
  • contextReady: A function that is invoked as soon as the model or type data has been loaded, but before the UI is rendered.
  • domReady: A function that is invoked after the UI has been rendered.

You can call $exoweb any number of times, and as an example our query might look something like this (please forgive the contrived example).

$exoweb({
    init: function() {
        ExoWeb.UI.Template.load("/path/to/template");
    },
    types: [
        {
            from: "CustomWidget",
            and: ["this.CustomInformation"]
        }
    ],
    model: {
        widget: {
            from: "Widget",
            id: "1",
            and: ["this.Type.DefaultPrice"]
        }
    },
    contextReady: function() {
        context.model.widget.doSomething();
    },
    domReady: function() {
        $(".widget input[type=button].save").click(function() {
            context.server.save(context.model.widget);
        });
    }
});

During init we want to load an external template file. I will talk more about this in the future, but for now I will say that ExoWeb allows you to bind parts of your form to a dynamic template that is selected based on the HTML element and the data that will be rendered. This allows you to break out reusable templates for common display scenarios, but it also allows your form to be dynamic in the sense that the markup can change automatically based on user input.

The types and model portions are what trigger requests to the server. This will load up all of the metadata that is needed, as well as the instance data for the instances and paths that you specify. When the context is ready it will have a model property that contains a property for each thing that you requested (in this case, the widget).

Context ready allows you to perform custom operations after all of your data is loaded. For example, if you have an add/edit page you can perform setup logic that may be needed only in the add scenario.

DOM ready is a good place to put any DOM manipulation code that is specific to your page and it’s data and templates. For general UI behavior I would recommend using jQuery to attach behavior as needed based on css selectors (a topic for another day). Notice here the use of the “server” object, which is found on the context. The server object allows you to save, round trip, raise events, and inspect the changes that have occurred.

Now, to display this on the form we might write something like this.

<div class="sys-template widget" sys:attach="dataview"
    dataview:data="{~ context.model.widget, source={{ window }} }">
    <h3>Editing <span>{binding Name}</span></h3>
    <div class="sys-template" sys:attach="dataview" dataview:data="{@ Type }">
        <label>Type:</label>
        <select class="sys-template" sys:value="{binding systemValue}"
            sys:attach="dataview" dataview:data="{binding options}">
            <option value="{binding systemValue}">{binding displayValue}
                </option>
        </select>
    </div>
    <div class="sys-template" sys:attach="dataview" dataview:data="{@ Price }">
        <label>Price:</label>
        <input type="text" sys:value="{binding displayValue}" />
    </div>
    <input type="button" class="save" value="Save" />
</div>

The form is not rendered until the “widget” object has been loaded. Also, notice the “@” extension that is used to incorporate metadata in the form. For a pick list it will give you the list of options for a particular property. Also, for other properties it allows you to automatically convert from and to a “system” or “display” format. In a nutshell, the “display” format is a human-readable form of the underlying data that also includes format conditions for things like email, phone number, etc, while the “system” format is not intended to be shown to a user but instead can be used to uniquely identify an object.

This has been a very brief introduction to ExoWeb. There is much more to talk about and you can expect follow up posts in the near future.

Getting started with virtualenvwrapper

A while back I toyed with using virtualenv and virtualenvwrapper to manage isolated python packages for development, but I didn’t get very far at the time. Recently I’ve been dabbling in ruby and have used RVM to manage ruby environments and gems. This experience has been a positive one, so I decided to revisit virtualenvwrapper to see how it would compare.

Installing

Note that I have tested this process on Ubuntu 10.04 and 11.04. I have not tested on other environments. Windows setup may follow in a later post.

The basic steps that I used are…

  • Install python
  • Install virtualenv
  • Install virtualenvwrapper
  • Configure virtualenvwrapper
  • Create a new virtual environment
  • Install packages

A script to accomplish most of this can be found here @/linux/virtualenvwrapper.sh.

1) Make sure that the script can be executed:

sudo chmod +x virtualenvwrapper.sh

2) Execute the script:

./virtualenvwrapper.sh

3) Initialize virtualenvwrapper:

The script mentioned above adds the following to ~/.bashrc, but will not take effect until you open a new terminal window. I’m sure there’s a way to avoid having to do this. In the code below, ~/.virtualenvs can be changed to any path you wish. Also, the path may vary (link).

export WORKON_HOME=~/.virtualenvs
source /usr/local/bin/virtualenvwrapper.sh

4) Create a new environment:

mkvirtualenv env

5) Begin working on your new environment:

workon env

6) Install packages

This can be done using pip (or easy_install, I believe) and the packages are installed in $WORKON_HOME/env/…

You can list all of your required packages in a single pip requirements file and install at once.

Example:

Django==1.1.2
django-tagging==0.3.1
South==0.7.1

Then, use pip to install the packages:

pip install -r /path/to/requirements

Update 12/29/2010: Using a specific version of python would be nice, especially since it’s something that can be done easy enough with RVM. I will be investigating this so look out for a future post on the topic.