Fitting Acceptance Tests with DSLs

Reading Ayende's post about Fit prompts me to say that I, too, haven't been overly pleased with the Fit/FitNesse experience in the .NET world; Java maybe a different story...The Fit book was/is excellent at demonstrating what can be done with Fit and why it's valuable to have customer driven/written acceptance tests, but the framework just doesn't seem to have matured enough on the .NET framework.

I've started writing a DSL for acceptance testing based on the ideas presented by Dan North on his blog which I found while looking at Behave#. I believe the Behave# project has good intentions, but the idea is too low-level; customers, business analysts, etc. aren't going to understand what's going on there, especially when asked to use anonymous methods.

Dan North's ideas seem to provide a good context for establishing boundaries around what constitutes a given function of the application, how it is expected to be used/work, and how to verify the application behaved as expected given a number of situations (or scenarios, test cases, what-have-you). And, perhaps most importantly, the language he and his team uses as a template for story writing provides a solid foundation as the basis for an extensible, customer-friendly, acceptance testing DSL. Let's take a look at one of the examples Dan provides:

Story: Account Holder withdraws cash

As an Account Holder

I want to withdraw cash from an ATM

So that I can get money when the bank is closed


Scenario 1: Account has sufficient funds

Given the account balance is $100

And the card is valid

And the machine contains enough money

When the Account Holder requests $20

Then the ATM should dispense $20

And the account balance should be $80

And the card should be returned


Scenario 2: Account has insufficient funds

Given the account balance is $10

And the card is valid

And the machine contains enough money

When the Account Holder requests $20

Then the ATM should not dispense any money

And the ATM should say there are insufficient funds

And the account balance should be $20

And the card should be returned


Scenario 3: Card has been disabled

Given the card is disabled

When the Account Holder requests $20

Then the ATM should retain the card

And the ATM should say the card has been retained


Scenario 4: The ATM has insufficient funds

…

The Story heading presents the title of the story/functionality, followed by three important pieces: the role, desired functionality, and the reason why the functionality is important. These are represented by the As a[n], I want, So that prefixed statements. While it is arguable whether these statements represent any valuable testing information for developers and/or testers, it does provide them with important contextual and business-need information about the desired functionality. From there, each scenario describes the background/"setup" context (the Givens), the action or functionality to execute (the When), and the desired outcome (the Thens).

Given these ideas, here's a DSL that could accomplish this and also yield us (the developers) automated test fixtures for verification during continuous integration cycles:

Story "Account Holder withdraws cash":
    Role "Account Holder"
    Feature "Withdraw cash from ATM"
    Benefit "Access money when the bank is closed"

    Scenario "Account has sufficient funds":
       Given:
          That "The account balance is $100":
            CreateAccount "012324", 100
          And "The card is valid":
            AddATMCardToAccount "012324", "32123456879", Valid
          And "The machine contains enough money":
            SetupAtm 10000
       When "The Account Holder Requests $20":
          UseATM "32123456789", Withdrawal, 20
       Verify:
          That "The ATM should dispense $20":
            VerifyATM LastTransaction, Withdrawal 20
          And "The account balance should be $80":
            VerifyAccount "012324", Balance 80
          And "The card should be returned":
            VerifyATM State NoCardPresent

What if the DSL processor produced the following:

[TestFixture]
public class StoryFixture_Account_Holder_withdraws_cash
{
[Test]
public void Scenario_Account_has_sufficient_funds()
{
  Account a = new Account("012324", 100);
  a.AddAtmCard(new AtmCard("32123456879", AtmCardStatus.Valid));
  AutomatedTeller m = new AutomatedTeller(10000);

  m.Withdrawl(a.AtmCard, 20);

  Assert.AreEqual(Operation.Withdrawl, m.LastTransaction.Operation);
  Assert.AreEqual(20, m.LastTransaction.Amount);
  Assert.AreEqual(80, a.Balance);
  Assert.AreEqual(AtmState.NoCardPresent, m.State);
}
}

Not all of the syntactic sugar has been worked out here, but I think the idea of the foundation DSL extended with the application specific language is clear. While this is better than having to write lower-level C# with anonymous methods, I think users - as in customers, business analysts, etc. - would be better served with a UI that provided them the necessary "guidance", if you will, to create their scenarios as necessary. Something like:

Would then output the DSL file. Now note that none of that UI works, it's all just show right now and very little of the DSL parsing is functional as of yet. With more time, I hope to have this publicly available for contributions, etc. if there's interest.

Finally, Somebody, Got it Right!

My wife and I were heading out of town this weekend but, before we could begin our trip, she had to take a PRAXIS test for her masters so she could student teach this fall and, thus, graduate in Spring of 2008. She's enrolled at Shippensburg University and the test just so happened to be offered there today which, semi-conveinently, wasn't too far out of the way of our destination. So, I tagged along, naturally, as why double back home just to turn around and head in the same general direction. The trick? Find something to occupy myself for 2-3 hours during the test...

Naturally, I took my work-issued laptop and reading material. When I first sat down, I figured I'd crack-open GMail and send a few necessarsy replies before studying. I figured, as most modern-day universities do, Ship would have wireless and I'd be online in no time.

After finding and connecting to the campus WAP, a sinking feeling fell upon me; I suddenly started having flashbacks from college when NIMDA and CODE-RED crippled my alma mater's network. While I was protected by a Linksys router in my dorm then, I did not have such the "luxury" today. This time, it was only me, my laptop, and the Ship campus network with a direct pipe to the Internet. I became very, very afraid.

My first course of action was to double check and see that at least, by some strange occurence of good fortune, that work had, in-fact, enabled Windows Firewall and that I at least had that protection.

Negative.

Four-letter words abound (silently as there was a gentleman sleeping in the chair next to me) and five minutes later, I disconnected, and turned on the firewall. Thank God I run as a non-admin; who knows what could've happened. Right?

Maybe.

Turns out once I felt "secure" about the connection was about to plug into, somebody actually thought out network security at good ol' Shippensburg University. When I fired up my browser to send those e-mails, I was greeted by a solitary web page saying "we don't recognize your computer on our network. Please login using your student ID and password to connect to the network". WOW!!! How 'bout that?! The network admins here had enough sense and forethought to say "with all the laptops, hand-helds and who knows what else roaming around these days, we need to be extra cautious. Anything new/strange that suddenly shows up on our network, isolate it on a separate LAN and require them to authenticate before bringing them in."

Needless to say, I was very, very, very impressed. It only took 4, almost 5, years for academia to "catch-up" on network safety/security, but it looks like it happened anyway. I'm thankful that my wife gave me her login credentials (in case I had to use a lab), or else, it could've turned out to be a worse afternoon -- in many, many regards.

And for those killjoys out there, yeah, I do realize that being summer, it's highly likely that the campus isn't that flooded with too many God-knows how nastily infected student machines; always better safe than sorry.

DSL Support in Brail, Part II

So continuing from Part I, here is our example view:

<?brail
dsl Html:
  html:
    head:
      title:
        text "Untitled page"
      end
    end
    body:
      p:
        text "Hello World"
      end
    end
  end
end
?>

As covered in part one, the initial expansion of the "dsl" macro yields us the following code in Boo:

dsl = Castle.MonoRail.Views.Brail.DslProvider(self)
dsl.Register(Castle.MonoRail.Views.Brail.HtmlExtension(OutputStream))

or, in C#:

DslProvider dsl = new Castle.MonoRail.Views.Brail.DslProvider(this);
dsl.Register(new Castle.MonoRail.Views.Brail.HtmlExtension(OutputStream));

The "this" and "OutputStream" references are what we're going tackle now. The Brail view engine for MonoRail functions loosely similar to traditional ASP.NET applications in that all ASPX inherit from System.Web.UI.Page; in Brail, all views inherit Castle.MonoRail.Views.Brail.BrailBase. Similarly to ASP.NET, whenever a view is rendered by MonoRail, the Brail view engine parses the view and generates a new assembly that contains a new object that derives from BrailBase. So, if our view were named "default.brail", our runtime-generated assembly would contain a class called "default_brail" that inherits BrailBase. When an MonoRail request is made and once MonoRail has determined what view(s) to render, the Run() method on the BrailBase, which is an abstract/must-override method and that is where the code in your View files is placed for execution. This is where the "dsl" macro is expanded.

Back to the "this" and "OutputStream" references...DSL support was designed to work similarly to VB's with block, allowing short-hand to invoke many methods on an object but only reference object by name once (for a laugh, read the documentation on the with...end with statement. Performance my butt...). This turns out to be quite a trick to accomplish, from the perspective of the compiler/parser; take this for example:

<?brail
dsl Html:
  for name in names:
    p:
      text "Hello ${name}"
    end
  end
end
?>

What's the context of the name and names variables? Remember, this code gets stuffed into the view's Run() method. Here's the resulting code created by the compiler within the Run() method:

dsl = Castle.MonoRail.Views.Brail.DslProvider(self)
dsl.Register(Castle.MonoRail.Views.Brail.HtmlExtension(OutputStream))
for dsl.name in dsl.names:
  dsl.p( { dsl.text("Hello ${name}") } )
end

Done double-taking? You probably said, "why the hell does it look like its trying to get the value of properties called "name" and "names" off the dsl object?! That's gotta be a bug..." And you'd be right.

Partially. "dsl.name" is a bug (found by Ayende) and has since been fixed, as you're actually declaring and referencing a new variable there all at one time. But, as you can also see, the DSL macro went thru the code and replaced all the other tag macros (like "p" to indicate a paragraph tag) as being methods on the dsl instance. The reason for that is duck-typing.

Under the covers, DslProvider implements Boo's IQuackFu interface allowing other .NET languages to work with the duck-typing semantics of the Boo language. Here, when a method is invoked on the DslProvider instance, Boo sees that DslProvider implements IQuackFu and invokes QuackInvoke with the method name and parameters that was called; in the case of "dsl.p(...)", "p" and the block of code are the parameters. The DslProvider keeps an IDictionary of all the methods that the referenced language implementation (HtmlExtension, in our case). When QuackInvoke is called on DslProvider, it tries to find a match give then the method name and number of parameters on the HtmlExtension object and then forwards the call onto it.

So what about the "dsl.names" reference? Simple answer: more duck-typing. Let's say your controller is written as so:

public class HomeController : SmartDispatcherController
{
  public void Default()
  {
    PropertyBag["names"] = new string[] { "Ayende", "Harris" };
  }
}

Without going into crazy detail, the Brail view engine essentially makes the items in the PropertyBag, Request, and other common and useful System.Web objects and collections class-level instances on the dynamically-generated BrailBase instance (that is a much-simplified explanation, but it does the job here). Duck-typing comes into play here because these "variables" are being treated as properties on the DslProvider instance. This time, QuackGet is called on DslProvider and a similar lookup is done, except this time against the BrailBase instance instead of the the language implementation.

So what's "OutputStream"? OutputStream is a property on BrailBase. Go ahead. Ask. Why isn't the generated code:

dsl = Castle.MonoRail.Views.Brail.DslProvider(self)
dsl.Register(Castle.MonoRail.Views.Brail.HtmlExtension(dsl.OutputStream))

Because, at this point, the macro is generating code just before the "with...end with", so we don't need to resolve the "OutputStream" reference using duck-typing. The "OutputStream" referenced is passed to the HtmlExtension object because the HtmlExtension object is what's actually responsible for "Response.Write" calls to give you your precious markup.

The idea for using the "with...end with" style seemed appropriate here as it provides, IMHO, more readable code instead of a lot of nested dsl.somethings( { dsl.moresomethings( { dsl.even_more_somethings( {...} ) } ) } ) when needing to produce an extensive amount of markup. The implementation of the "with...end with" design was taken by an example on the Boo project of how to do just that and can be found in the project's repository.

All that for pretty syntax.

Next time, we'll cover current ideas, status, and any other features that may/not be on the horizon.

DSL Support in Brail, Part I

Ayende mentioned on his blog about the work I've done to incorporate support for DSLs in Brail views within MonoRail. While he did a great job giving an overview of how to make use of the functionality, I figured I'd dive into the actual implementation and give some details of how this is achieved and where this can be taken -- at least my "vision".

The Beginning

First, none of this is even possible without mentioning what the Boo project team has done and giving a bit of a background - to my understanding - of what's at the core. With the 0.7.8 release of Boo, the team enabled the ability to invoke methods of an object via Boo's macro syntax. What's that mean; and what's a macro? C# developers have been using -- no, well, ok, pun intended -- since C# was introduced. In C#, using() is or can be thought of as a macro as when the C# compiler finds a using() block, it expands it into a try/catch/finally block calling dispose on the instance referenced in the using statement. Here's an example; this code:

using(SqlConnection c = new SqlConnection())
{
// do stuff
}

is rewritten/transformed by the compiler to:

SqlConnection c = null;
try
{
   c= new SqlConnection()
}
finally
{
   if(c != null)
   {
      c.Dispose()
   }
}

which is why the using statement only accepts types that implement IDisposable, as it ensures Dispose() is always called. So, basically, macros are shorthand for developers.

Back to Boo. The 0.7.8 release allows developers to invoke methods using the macro syntax. So if I had a method called text that took a string as a parameter:

def text(value as string):
   print value

in Boo 0.7.8, I can call text like this:

text "hello world"

Well, that's all well and good, isn't it? Nothing really "magical" there; how does this enable DSLs in Brail?

The Awakening

One of the major advantages that Boo offers developers is the ability to extend the language in numerous ways and macros is just one, "simple" option developers have at their disposal. The only caviaet is that your macros, obviously, need to produce valid, compilable code. So, from the example above, when the compiler parses:

text "hello world"

it rewrites/transforms the source to:

text("hello world")

So, how, or rather what, does a MonoRail view in Brail that's something like this:

<?brail
dsl Html:
  html:
    head:
       title:
          text "Untitled page"
       end
    end
    body:
       p:
          text "Hello World"
       end
    end
  end
end
?>

Give you the output you expect?

The Implementation

As you might guess, "dsl" is a macro that takes one parameter: the language you intend to use. So what does "dsl Html" generate? This, in Boo:

dsl = Castle.MonoRail.Views.Brail.DslProvider(self)
dsl.Register(Castle.MonoRail.Views.Brail.HtmlExtension(OutputStream))

or, in C#:

DslProvider dsl = new Castle.MonoRail.Views.Brail.DslProvider(this);
dsl.Register(new Castle.MonoRail.Views.Brail.HtmlExtension(OutputStream));

Okay, that might be a lot to swallow. "dsl Html" starts off by creating a new instance of a DslProvider object. The DslProvider object acts as a registry of languages to use. In this case, the second line tells the DslProvider that we're specifically interested in working with the HtmlExtension.

What's up with "this" and "OutputStream"? Here's where we get into the real nitty gritty, and for that, I leave you to Part II in this series. Hope this is as interesting for you as it is for me!!

And Now for Something Completely Different

Yes, I just quoted Monty Python...And speaking of odd, fad programming languages...

A friend of mine got me reading Scott Hannselman's blog and one of his latest entries was on why Ruby is cool and worth learning. At the end of the post, he provides a link for an opportunity to try Ruby now.

I was skeptical. I figured it was a post on how to setup Linux and getting Apache with Ruby and Ruby on Rails up and running. Not quite so.

The site is an interactive console. Yes. You read that right. An interactive Ruby console. With a tutorial. All AJAXified and spiffy; 'cause that's how those Ruby folks roll. It guides you through interesting Ruby constructs and takes approximately 15 minutes. Really, really interesting and innovative stuff.

Go. Play. Learn Ruby. Do and experience something completely different without "ruining" your machine.

Bit Shifting: What is it and Why

OK, so maybe I'm way behind the times, but I finally figured out why in the world one would ever want to shift bits. I came by this discovery after reading one of Ron Jefferies' recent posts. Pay particular attention to the color method:


private int color(int count) {
// int gray = (255 - count*4 + count/2) & 0xFF;
int gray = (255 - count*6) & 0xFF;
int rgb = (gray << 16) | (gray << 8) | (gray << 0);
return rgb;
}

The return value is then fed into a Color object (in .NET, you would use this value in the Color.FromArgb method), which expects the value of the parameter to have the individual color values (e.g., alpha/opacity value, red, green, and blue values, respectively) in specific parts of the 32-bit value. So...let's digress a bit to CSC242 - Assembly...

1 byte = 8 bits, which, in memory would be:
00: 00 00 00 00 // 00: represents a memory address, each pair representing 2 bits

2 bytes = 16 bits, in memory
00: 00 00 00 00 00 00 00 00

3 bytes = 24 bits, in memory
00: 00 00 00 00 00 00 00 00 00 00 00 00

4 bytes = 32 bits, in memory
00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Now, an integer, on a 32-bit machine, occupies (appropriately so), 4 bytes of memory. So, when you assign an integer a value of, say, 135, it looks like this in memory (values in binary):

int val = 135; //c# assignment
00: 00 00 00 00 00 00 00 00 00 00 00 10 00 01 11 // 10000111 in binary is 135

This is exactly how the computer stores the value in memory. However, this won't "work" for the Color.FromArgb(int argb) method because as the documentation states:

"The byte-ordering of the 32-bit ARGB value is AARRGGBB. The most significant byte (MSB), represented by AA, is the alpha component value. The second, third, and fourth bytes, represented by RR, GG, and BB, respectively, are the color components red, green, and blue, respectively."

Most significant bit translates to the first pair of zeros you'd read in the value of (in bold below):

00: 00 00 00 00 00 00 00 00 00 00 00 10 00 01 11 // 1000111 in binary is 135

So, if we wanted 135 to be the red value, we need move - or shift - the bits that represent 135 up to the proper position. This is where the left-shift (<< in C/C++/C#/Java) operation comes in. When you apply the left-shift operator, you must specify the number of bits you want to move. In order to move the values of 135 into the proper position for the Color.FromArgb(int) method to interpret the value as the value we want for red, we must shift the value 16 bits.

int red = (135 << 16);
00: 00 00 00 10 00 01 11 00 00 00 00 00 00 00 00

Shifting the bits left 16 positions, is effectively multiplying 135 by 2^16, or 65536. To get the green value of the color, we need to shift the bits left 8 positions:

int green = (135 << 8);
00: 00 00 00 00 00 00 00 10 00 01 11 00 00 00 00

We apply the same to the value for blue, but shifting the bits really has no effect as the blue component stays in the last byte:

int blue = (135 << 0);
00: 00 00 00 00 00 00 00 00 00 00 00 10 00 01 11 // 10000111 in binary is 135

Now, to make the composite value - that is, combining the red, green and blue values into 1 value - we add the values together, bitwise. This is accomplished using the bitwise OR operator:

int rgb = (red | green | blue);
00: 00 00 00 00 10 00 01 11 10 00 01 11 10 00 01 11

The rgb variable now has all three color components in the proper positions. Having zeros in the most significant bits translates to a value of 0 for the alpha/transparency component that the FromArgb(int) method is expecting - giving us a completely opaque color.

This may not be the most clear description; I found out that I by showing the disassembly while running a program shows the effect (however, in hex) that shifting has on values. Hopefully, I'll be able to post some shots to help explain.

"Everyone See Palmer Now"

And now for something completely different...

When I heard that ESPN was awarded the contract to carry Monday Night Football, I rejoiced. Madden is/was/always will be a Hall of Fame coach; and that's where he belongs.

I hope he wasn't watching Brett Farve tonight. If so, I sure hope the EMTs were on stand-by
to revive him after every down Farve played. Cincinnati's defense just obliterated Green Bay's offense.

And then there was Cincinnati's offence. The title of this post came from a banner I caught while watching the game. I can think of another acronym for ESPN other, having more to do with the commentator's profound oral fixation with Palmer's genitalia. It made me sick (excusing the fact that I am a huge fan of the Men of Steel and bleed Black 'n' Gold). Is it phenomenal that Palmer was able to recover in less than half the amount of time any normal human being would require. But was he just as good as ESPN seem to inflate his performance?

Nope. Nay. Um, I'm thinking no.

Why? Green Bay sucks. Farve should've retired while he was semi-ahead (even though last year was absolutely putrid). The have a rookie offensive line. Farve even said himself, paraphrased from the commentators they "are extremely talented...but with absolutely no real world experience". Did the Bengals tear apart their defense? Yeah, but the seam was frayed to begin with.

Now, to switch gears a bit, it wasn't just this game that I think the commentating was one sided. This past Friday (yes, the PIT @ PHI game), the commentators were more impressed with how the Eagles are managing without T.O., how great it must be to be without all those distractions, and how the team is gelling together. Did they look good? Sure. I'm sure it's nice to have all that behind you. But they didn't really do anything until the third quarter, and that was without McNabb. The first two quarters, Pittsburgh held the Eagles to three mile-long field goals. Sure, Akers has a leg and then some, but the wind was in his favor. Put him in Heinz Field and I'm sure he makes the first, but the last two I'm not so sure about.

So what's the point to all this? I am glad that we're out of the Michaels-Madden duo for Monday Night, but I think the only guy that had things together on both these games was the PTI guy - Tony Cornhiser (or something like that). Every body needs preseason. So let's hope that Monday Night Football on ESPN only gets better as the season goes on.