Software Development With Scott

WebTestingExplorer Documentation Moving Forward

2012-01-11T09:13:00.001-08:00

I started writing some documentation for WebTestingExplorer in the site Wiki. Because I would like people to start trying it out, I'm hoping that it will help them get started without necessarily having to pour over all the code and Javadocs. The core API documentation is (and always will be) in the Javadocs, but I will use the Wiki to explain some of the higher-level concepts and design philosophies. So if you get the chance, download, build, try it out, and, of course, let me know how it goes.

Suppressing Firefox favicon Requests from WebDriver

2012-01-01T16:07:00.000-08:00

Right now, WebTestingExplorer only works with Firefox. There are a couple of reasons for that which I won't go into here. Anyway, we decided to wire our HTTP requests to go through a proxy by default. This allows us to look for specific status codes, presumably "undesirable" ones like the 500-series, or possibly 404's.

When we did this and pointed the tool at a specific URL, the first thing we noticed was Firefox firing off a bunch of requests to various locations on the target site fishing around for the favicon, most of which (kind of by definition) ended with a 404. I'm surprised that for as long as I have used Firefox attached to a proxy for testing and debugging, I had never taken note of this before. In this case, it's really annoying -- after all, one of the points of using the proxy in the first place was to verify that all requests came back with a valid response.

The first thing I considered was adding special-case code for all favicon requests in the handlers for the proxy. That seemed kind of lame, so the second thing I thought about was looking for a way to suppress the requests in the first place. It turns out that there are Firefox preferences for this -- browser.chrome.favicons and browser.chrome.site_icons -- as described here.

I have two main points left for this post. First, I wanted to show the code for how set these preferences in Selenium WebDriver when you build your FirefoxProfile:


final FirefoxProfile profile = new FirefoxProfile();



// The following preferences control Firefox's default behavior of sending

// requests for favicon.ico, which results in lots of bogus 404's for sites

// that don't have favicons. 

profile.setPreference("browser.chrome.favicons", false);

profile.setPreference("browser.chrome.site_icons", false);

    

FirefoxDriver driver = new FirefoxDriver(profile);

Second, I will pitch my opinion that in most automated testing scenarios, you probably ought to be suppressing favicon requests from Firefox, even if your site actually has one. A good way to do this is to save the preferences into a custom Firefox profile used for automated testing. If you don't already have one of those, you should. I will try to provide some tips on how to maintain and use one of those in a future post.

Introducing WebTestingExplorer

2012-01-01T15:09:00.000-08:00

My friend and I recently started working on a new open-source project, WebTestingExplorer (hosted on code.google.com). We are setting out to do a high-quality, industrial-strength implementation of some of the ideas we worked on in graduate school at the University of Maryland, specifically applied to web application testing. The super-high-level idea is to incrementally "explore" the web user interface, simultaneously looking for bugs and generating test cases. Although the project is in a very early state, the current implementation can do a few useful things, such as:

Detecting unexpected HTTP status codes and Javascript errors during exploration.
Generating and replaying regression test cases, looking for differences in state.

A few tenets we have adopted are:

Most everything regarding the exploration and test case replay process is configurable.
Configuration is done primarily via code, not command-line arguments or XML files.
The application-under-test is inherently "testable" (or we can make it so), such as having unique id's for important HTML elements, and hooks that enable reliable timing of actions.

If you are interested in automated test case generation and/or web application testing, feel free to start playing with it and sending feedback through the Google Code project. Although the primary vehicle for project communication will be the Project Wiki, I hope to blog here about some of the interesting issues we encounter. So until next time...

EasyMock Issue #2: Bad Argument equals()

2011-05-29T15:35:00.001-07:00

One of the two canonical EasyMock failure messages (along with "Expectation failure on verify" is "Unexpected method call". This failure happens between replay() and verify() whenever EasyMock sees the class-under-test making a method call on a mock object for which the test did not set up an expectation. Generally, there are two possible problems to immediately look for:

A deficiency in the test, i.e. the behavior of the class-under-test is not correctly modeled during the expectation-setting phase of the test case.
A bug in the class-under-test, i.e. the test is expecting the right thing but the code isn't doing it.

(Incidentally, when it turns out to be #1, programmers tend to complain about how difficult it is to write and maintain mock-object tests, and when it is instead #2, they often fail to give mock-object-testing enough credit for detecting their bug.)

But there is a third possibility, which I would argue is neither a bug in the test nor in the class-under-test, but rather a bug in a completely separate, possibly innocuous-looking class that they both use. As usual, let's look at an example.

Here is the class-under-test, CustomerController for this post (slightly modified from the one I used in the previous two posts:


public class CustomerController {

  private OrderService orderService;



  public CustomerController(OrderService orderService) {

    this.orderService = orderService;

  }

  

  public void postOrders(Order...orders) {

    orderService.postOrders(Arrays.asList(orders));

  }

}

And the test for the postOrders() method, which uses a mock OrderService:


public class TestCustomerController {

  @Test

  public void testPostOrders() {

    OrderService mockOrderService = EasyMock.createMock(OrderService.class);

    CustomerController controller = new CustomerController(mockOrderService);

    List<Order> expectedOrders = new ArrayList<Order>();

    expectedOrders.add(new Order(1));

    

    EasyMock.expect(mockOrderService.postOrders(expectedOrders)).andReturn(1);

    EasyMock.replay(mockOrderService);

    controller.postOrders(new Order(1));

  }

}

It is also important to note that we are posting Order instances:


public class Order {

  private int orderId;



  public Order(int orderId) {

    this.orderId = orderId;

  }

}

When you run this, alas, you get an error:

java.lang.AssertionError:

Unexpected method call postOrders([Order@1e4cbc4]):

postOrders([Order@b2fd8f]): expected: 1, actual: 0

at org.easymock.internal.MockInvocationHandler.invoke(MockInvocationHandler.java:45)

at org.easymock.internal.ObjectMethodsFilter.invoke(ObjectMethodsFilter.java:73)

at org.easymock.internal.ClassProxyFactory$MockMethodInterceptor.intercept(ClassProxyFactory.java:92)

at OrderService$$EnhancerByCGLIB$$6d00fba4.postOrders()

at CustomerController.postOrders(CustomerController.java:11)

at TestCustomerController.testPostOrders(TestCustomerController.java:19)

The CustomerController code is quite trivial and correct in this case. The test is less trivial but also, well, at least ought to work. So what is going on here?

When EasyMock compares a mock method call to the expectation, it checks equality on each argument. In this case, that fails because the Order instance in the expectation does not equal the Order instance in the invocation. Note that Order is not overriding equals(), and since I set the test up to use different instances, the default instance-equality check fails. This example is pretty contrived. In reality, the more common case I've seen is that there actually is an equals() override, but somebody changes the class and breaks it, in which case the test is finding a real bug (albeit not in the actual class-under-test!).

There are two ways to fix this problem. First, I could override equals() (and hashCode(), of course) if it doesn't already exist on the class in question (the case with Order) or fix equals() if it were already there. That solution is probably more robust. The quick-and-dirty fix is to make the expectation and the invocation use the exact same instance(s). I will actually show the code for that:


public class TestCustomerController {

  @Test

  public void testPostOrders() {

    OrderService mockOrderService = EasyMock.createMock(OrderService.class);

    CustomerController controller = new CustomerController(mockOrderService);

    List<Order> expectedOrders = new ArrayList<Order>();

    

    // I will use this here AND below...

    Order order = new Order(1);

    expectedOrders.add(order);

    

    EasyMock.expect(mockOrderService.postOrders(expectedOrders)).andReturn(1);

    EasyMock.replay(mockOrderService);

    controller.postOrders(order);

  }

}

Sorry if I've rambled on for too long about something that is actually quite obvious. But I have made and seen others make this type of error enough times that I can hopefully save someone some time. Next time, I will discuss an EasyMock limitation that might impact your interface design.

EasyMock Issue #1: Missing Behavior Definition

2011-04-30T21:27:00.000-07:00

I am planning to do a series of posts (including examples) on issues that people (myself included) run into while using EasyMock. The first of these involves EasyMock throwing an IllegalStateException "missing behavior definition for the preceding method call". The executive summary is that this is caused by the omission of a return value specification (andReturn(...)). If that is all you need to know, thanks for coming. If you would like to see a specific example, keep reading.

Let's say we are writing unit tests for a CustomerController class that uses an OrderService class to post a Customer's Orders:


// OrderService.java

public class OrderService {

  public int postOrders(List<Order> orders) {

    return orders.size();

  }

}



// CustomerController.java

public class CustomerController {

  private OrderService orderService;



  public CustomerController(OrderService orderService) {

    this.orderService = orderService;

  }

  

  public void postOrders(Integer...orderIds) {

    List<Order> orderList = new ArrayList<Order>();

    for (int orderId : orderIds) {

      orderList.add(new Order(orderId));

    }

    orderService.postOrders(orderList);

  }

}

In the real world, OrderService probably connects to external resources and therefore we want to mock it when creating a unit test for CustomerController. The interaction between CustomerController and OrderService consists of a single call to the OrderService.postOrders(List) method. Simple enough, here is our first attempt at writing this test:


public class TestCustomerController {

  @Test

  public void testPostOrders() {

    OrderService mockOrderService = EasyMock.createMock(OrderService.class);

    CustomerController controller = new CustomerController(mockOrderService);

    List<Order> expectedOrders = new ArrayList<Order>();

    expectedOrders.add(new Order(1));

    

    // You may do the following assuming postOrders returns void:

    mockOrderService.postOrders(expectedOrders);

    EasyMock.expectLastCall();

    

    // But unfortunately it doesn't.

    EasyMock.replay(mockOrderService);

    controller.postOrders(1);

  }

}

When you run this, it blows up with the following exception:

java.lang.IllegalStateException: missing behavior definition for the preceding method call postOrders([Order@82c01f])
 at org.easymock.internal.MocksControl.replay(MocksControl.java:174)
 at org.easymock.EasyMock.replay(EasyMock.java:1970)
 at TestCustomerController.testPostOrders(TestCustomerController.java:22)

Since I gave away the punch line in the opening paragraph of this post, you know that this happens because OrderService.postOrders() is not a void method; rather, it returns an int. I would argue that this is a pretty easy mistake to make, though, because the implementation of CustomerController does not care about or capture this return value. So unless we're paying pretty close attention to the OrderService interface, we might assume it returns void. And unfortunately, EasyMock requires that ALL non-void method calls on mocks have return values specified, even if the class under test will ignore it.

So the correct way to set up the OrderService.postOrders() expectation is as follows:


EasyMock.expect(mockOrderService.postOrders(expectedOrders)).andReturn(1);

In my next "EasyMock Issues" post, I will talk about a couple of things to look for when your expectations are mysteriously failing to match.

The Law of Demeter Makes You Create Mock-Friendly Objects

2011-04-16T21:01:00.000-07:00

One of the reasons why I haven't been blogging is that much of the time, I have been busy with teaching -- specifically, as an adjunct in the Master of Software Engineering program at Seattle University (of which I am also an alumnus).

The last two winters, I have taught software design and modeling. Something that I touch on with my students is the Law of Demeter. The LoD states (among a couple of other things) that a software object in an object-oriented program should only talk to its "immediate friends". That rules out long method call chains like this:

customer.getOrders().get(0).getLineItems().get(0).getProduct().getQuantity()

This is bad for at least two obvious reasons. First, the objects in this program are tightly-coupled, and changes to the object model will ripple through the system in annoying ways. Second, if you write this in a Java or C#-like language, rest assured that eventually you will run into a situation where something in this chain is null, and worst of all, the ensuing NullPointerException will not pinpoint exactly what it is. So don't write code like this.

But the reason I'm writing this post is to draw attention to a third disadvantage of long method call chains: They complicate mock object testing.

Let's look at a slightly less pathological example based on the following classes:

public class CustomerController {

private Customer customer;

public CustomerController(Customer customer) {

this.customer = customer;

}

public void postOrders() {

customer.getOrdersList().postOrders();

}

public class Customer {

private OrdersList ordersList = new OrdersList();

public OrdersList getOrdersList() {

return ordersList;

}

public void postOrders() {

ordersList.postOrders();

}

public class OrdersList {

private List orders = new ArrayList();

public void postOrders() {

for (Order order : orders) {

order.post();

}

public class Order {

public void post() {

System.out.println("Posting order");

}

We have a CustomerController that controls (what else?) a Customer and triggers the posting of the customer's Orders. Forget the unlikeliness of this particular design. (In reality, the Customer, OrderList, and Order classes would probably be pure-data classes, and CustomerController would use a service class to post orders.) Let's think about how we would test CustomerController.

Presuming that the Order.post() method ultimately makes external connections, we want to mock the Customer.postOrders() method and simply verify that CustomerController calls it. If we try to write the test this way:

@Test

public void testBadPostOrders() {

Customer customer = EasyMock.createMock(Customer.class);

// Next line: NullPointerException!

customer.getOrdersList().postOrders();

EasyMock.expectLastCall();

EasyMock.replay(customer);

CustomerController controller = new CustomerController(customer);

controller.postOrders();

EasyMock.verify(customer);

}

We should have known better than to try to call customer.getOrdersList().postOrders() on an EasyMock mock. To fix this, we could add a mock OrderList, return it from an expected call to customer.getOrderList() on the Customer mock, and create an orderList.postOrders() expectation. But doing that type of thing will rapidly become annoying. And this is a very simple example.

We can move toward a better solution by noting that in this case, the test itself (which you can think of as client code of the Customer class) is not following the Law of Demeter: It is talking to the OrderList, which is not an "immediate friend". Yet, from the perspective of the test, the Customer class doesn't leave it a lot of choice.

Fortunately, we can refactor the Customer and CustomerController classes to make them more mock-testable. We can add a postOrders() helper method to the Customer class:

public class Customer {

private OrdersList ordersList = new OrdersList();

public void postOrders() {

ordersList.postOrders();

}

And we can change CustomerController to call it:

public class CustomerController {

private Customer customer;

public CustomerController(Customer customer) {

this.customer = customer;

}

public void postOrders() {

customer.postOrders();

}

Then we can easily write a working test that uses the mock framework to verify that the controller does, in fact, post the orders:

@Test

public void testGoodPostOrders() {

Customer customer = EasyMock.createMock(Customer.class);

customer.postOrders();

EasyMock.expectLastCall();

EasyMock.replay(customer);

CustomerController controller = new CustomerController(customer);

controller.postOrders();

EasyMock.verify(customer);

}

And again, as many others have pointed out, good designs and good tests feed back into one another.

Back to Blogging

2011-04-16T20:43:00.001-07:00

Wow, it has been over a year since my last post. Clearly I have been very busy. But still, I really do want to be here sharing my thoughts about development and testing. So I'm going to try and get back into the swing. I have a couple of ideas for posts queued up. I am planning to get them done over the next couple of weeks, and we'll go from there.

One thing that I will do differently from now on is that whenever I have a useful link, recommendation, or brief thought to share, I will do it on Twitter. If you're interested, follow me there -- I am @sdmcmaster.

Until next time...

Using Java to Download Links to Picasa Web Albums

2010-01-18T14:58:00.000-08:00

I mentioned a couple of weeks ago that I moved all of my digital photos to Picasa Web Albums. It turns out that my wife would like access to these photos. I uploaded the albums as "Unlisted", so the web interface provides no way to get links for sharing in bulk. I have somewhere around 150 albums, so going into each one to grab a sharing URL would be a really tedious way to spend the afternoon on my vacation day. I would rather write a program. Picasa Web Albums Data API to the rescue.

Since this was my first program using Google Data APIs, it took me longer to set up a working environment than it did to actually write the Java code. The getting-started guide is here. For my case, I needed as external dependencies 1) JavaMail and 2) the JavaBeansActivation Framework, both of which most open-source-Java developers already have sitting around somewhere convenient. Then I grabbed the latest Google Data Java Client libraries, unzipped, and put all of the jars in gdata/java/lib and gdata/java/deps on the classpath of my project.

From there, the code was very straightforward. Note the placeholders for Google username and password.

import java.net.URL;
import java.util.List;

import com.google.gdata.client.photos.PicasawebService;
import com.google.gdata.data.photos.GphotoEntry;
import com.google.gdata.data.photos.UserFeed;

/**
 * Short script to download links to all of my Picasa
 * Web Albums.  This includes private/public/protected
 * access levels.
 * 
 * @author Scott McMaster (smcmaster@acm.org)
 *
 */
public class DownloadPicasaWebAlbumLinks {
 private static final String USERNAME = "NOTMYREALUSERNAME";
 private static final String PASSWORD = "NOTMYREALPASSWORD";

 @SuppressWarnings("unchecked")
 public static void main(String[] args) throws Exception {
  PicasawebService myService = new PicasawebService(
    "Download Album Links");
  myService.setUserCredentials(USERNAME, PASSWORD);

  URL albumFeedUrl = new URL(
    "http://picasaweb.google.com/data/feed/api/user/" + USERNAME
      + "?kind=album");
  UserFeed albumFeed = myService.getFeed(albumFeedUrl, UserFeed.class);
  List albums = albumFeed.getEntries();
  for (GphotoEntry album : albums) {
   System.out.println(album.getHtmlLink().getHref());
  }
 }
}

For "unlisted" albums, this will print out, for each album, the same link you get if you click "Link to this album" in the UI.

Moving to the Cloud Part 2: Email

2009-12-31T13:05:00.000-08:00

In my last post, I mentioned my goal of moving all of my important data out of local storage and into the cloud. First, I dealt with my photos. My next order of business was email.

Traditionally, I used Outlook to organize my email and contacts. So I had years' worth of messages organized into personal folders on a local hard drive. Whenever I wanted to move to a new machine, I would export all of these as .pst files and re-import them in a new location. That was annoying, risky, and error-prone.

Now that I am paying for all of this Google storage (as discussed in my last post), it seemed obvious that Gmail was the place to move all of my old email. I wasn't sure off the top of my head exactly how to do that, but a quick search turned up many variations on a simple trick -- enable IMAP in Gmail settings, bring the Gmail account into Outlook via IMAP, and drag-and-drop folders and messages. Perfect!

As for my contacts, I imported them into the Google space (via Google Voice) a long time ago. And I never did a lot of personal calendaring with Outlook. (Online solutions are far better for that now anyway.)

Overall, I was very pleased at how easy it was to migrate away from Outlook. There are two more things that I care about enough to want to move to the cloud -- my music and my schoolwork/teaching/research files. I will cover those in future posts.

Moving to the Cloud Part 1: Photos

2009-12-30T21:17:00.000-08:00

Well, it's been a long time since I've posted. Without going into a lot of details, I've had a number of life changes in the last few months. Hopefully I can get back into blogging a bit.

One goal I've made for 2010 is to care a lot less about what, exactly, is on the fleet of machines and disk drives sitting around my house. That means moving content into the cloud. I decided to start with what is most important to me, namely, my family photos.

I was a pretty early adopter of digital photography, getting my first camera in 1997. So I have a lot of pictures. I've used Picasa for a long time (well before Google bought it), so it seemed like a natural fit to upload everything into Picasa Web Albums. And of course, there is a web API, so in the future, I can manage my photos with custom scripts and applications.

The Picasa fat client application makes the upload process incredibly easy. (I used the beta version for Linux, which now runs via the WINE emulator.) One thing I had to make sure to do was force it to upload the full-size images for each album, lest I lose some JPEG quality for more recent years' photos.

Another thing I had to do was upgrade my Google Account storage, because the free space wasn't going to be nearly enough. Conveniently, there is currently a deal to get a free Eye-Fi card with a year's worth of 200G of storage. I went for that. I don't really need anywhere near 200G just for my photos, but I will use some more of that space in future blog posts. And I can't wait for my new Eye-Fi card to arrive in the mail.

Testing Exception Messages, JUnit 4.7 Edition

2009-10-24T14:22:00.000-07:00

Historically, JUnit has not elegantly supported verification of expected-exception messages in unit tests. Some time ago, I wrote a post about techniques for handling this situation. Brian Brooks (clearly a diligent web searcher) found that post and was kind enough to send me a message pointing out that the latest version of JUnit, 4.7, finally includes support for this incredibly important and common testing scenario.

In honor of this exciting occasion, I wanted to post an example. Here is the class I'm going to test:

public class PositiveInteger {
 private int num;

 public PositiveInteger(int num) {
  if(num <= 0) {
   throw new IllegalArgumentException("Number must be positive");
  }
  this.num = num;
 }
}

Let's test -- hmmm -- the constructor, I guess. I created two test cases -- one which passes, and one which intentionally fails (just to show the resulting output):

import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;


public class TestPositiveNumber {

 @Rule
 public ExpectedException thrown = ExpectedException.none();

 @Test
 public void testConstructorInvalid_PASS() {
  thrown.expect(IllegalArgumentException.class);
  thrown.expectMessage("Number must be positive");
  new PositiveInteger(-5);
 }

 /**
  * Intentionally-failing test to demonstrate the results
  * when checking an exception message.
  */
 @Test
 public void testConstructorInvalid_FAIL() {
  thrown.expect(IllegalArgumentException.class);
  thrown.expectMessage("Not gonna pass");
  new PositiveInteger(-5);
 }
}

JUnit now supports a style embodied in "rules", which get marked with the @Rule annotation and provide a means by which the test execution behavior can be modified. You can see that here with the "thrown" field, of type "ExpectedException". It would be great if a) that field didn't have to be public, and b) the functionality could be implemented using a local variable, but unfortunately the Java language won't currently support such an implementation. So a public field it is. In a test, you can configure the rule. Here I do that with calls to the "expect" and "expectMessage" methods. When JUnit executes the test case, it checks the result against the rule and fails if the rule is not matched. In the test case that does not pass, the failure takes this form:

java.lang.AssertionError: 
Expected: (exception with message a string containing "Not gonna pass" and an instance of java.lang.IllegalArgumentException)
     got: 

 at org.junit.Assert.assertThat(Assert.java:778)
 at org.junit.Assert.assertThat(Assert.java:736)

Very useful. Thanks to Kent Beck for adding this support.

URLEncoder Demonstrates API Design Hazards

2009-09-26T19:53:00.000-07:00

I had to use java.net.URLEncoder yesterday. This experience was a great reminder of how annoying this API is. It demonstrates a couple of issues, so I thought I would comment on it here.

The first thing you can see here is how important it is to get the API right the first time, lest things go downhill from there. In its original form, URLEncoder provided only one static method which encoded a string as "application/x-www-form-urlencoded" (suitable for issuing an HTTP request). This method has the signature encode(String), and it used the platform's default encoding. Unfortunately, this is technically hazardous, because URL encodings are supposed to always be UTF-8. Therefore, wiser heads built and used replacement classes (such as URLCodec from Apache Commons).

Java 1.4 finally "fixed" this deficiency in the platform with a new encode(String,String) overload, the second parameter being the name of a character encoding to use. The old encode(String) method is marked as deprecated. The documentation itself states that: Note: The World Wide Web Consortium Recommendation states that UTF-8 should be used. Not doing so may introduce incompatibilites.

I don't know, but I'm guessing that the powers-that-be would have liked to make encode(String) do UTF-8 encoding. But they felt constrained by the legacy decision to use the platform default, in the name of not breaking any existing code. Never mind the substantial odds that any such existing code is already broken in this case wherever the default encoding is not UTF-8.

But wait, it gets worse. The encode(String,String) method also throws UnsupportedEncodingException whenever (surprise!) the second parameter does not refer to a valid encoding. I'm no rocket scientist, but it seems to me that if you're following the W3C recommendation and always passing UTF-8, you don't have to worry about this exception being thrown. And yet because the exception is checked, you have no choice but to handle it and clutter up your code, perhaps with a catch block that just does assert(false).

This all begs the question, if the second parameter should always be "UTF-8" for some definition of "always", why should the API require a second parameter at all? I submit that a better approach would have been to create, instead of or in addition to encode(String,String), an encodeUTF8(String) method that always did UTF-8 encoding without the added hassle of passing a (constant) parameter and dealing with a checked exception...

"How to Think About Algorithms" Book

2009-09-13T15:59:00.000-07:00

I have been gradually making my way through "How to Think About Algorithms" by Jeff Edmonds.

I have recently recommended this book to several people, and I wanted to mention it here, too. Edmonds uses what I'll call a more intuitive approach to algorithms that I think would be appealing to those with an engineering mindset. Here I'm constrasting it with the texts that I've used as a student in algorithms courses -- the classic Sedgewick and CLRS books.
(Maybe more importantly in this economy, though, the subject matter and exercises in this book are a rich source of review for those preparing for tech interviews!)

Tailing Inside Eclipse

2009-09-02T20:23:00.000-07:00

I'm currently working on a project that writes to a log file inside my workspace when I test-execute it. Obviously it's useful to actually view that log file once in awhile, but to do so inside Eclipse requires regularly F5-refreshing it in the Project Explorer. I started to wish that Eclipse would prompt me to automatically reload the file when it detected modification, like some text editors do. But then it occurred to me how convenient it would be if I could actually tail the log from inside Eclipse itself. After all,

The less I leave the IDE, The more productive I can be.

(That rhymes.)

This seemed like an obvious enough desire that somebody would have built a plugin for it already, and sure enough, a search turned up a couple of different examples. The one that I went with is called NTail. It works like a champ. It also has a few extra nice features, like the ability to highlight lines in the file based on a regular expression. If this sounds useful to you, I'd encourage you to check it out.

Changing Return Values to Exceptions, Java/AspectJ Edition

2009-08-08T17:28:00.000-07:00

In the last couple of posts, I've been talking about APIs that communicate errors via return values, and how to map those to exceptions. As we saw most recently, one reasonably effective approach in C++ makes use of macros, but it requires a degree of development discipline, and the resulting code remains somewhat unsatisfactory.

In this post, I will present an approach for Java that mitigates these issues. Specifically, you can neatly map return values to exceptions using AspectJ.

(Note that I am using Eclipse for this work. It's great that AspectJ is now an Eclipse project, because it can be incredibly frustrating to use without solid IDE support.)

I'll start with some Java math utility routines that are roughly equivalent to what I was working with in C++. One difference is that in Java, I don't need a custom sqrt() wrapper because the Math.sqrt() method already uses a return value, Double.NaN, to indicate error.

package com.scottmcmaster365.math;

public class MyMathUtils
{
  public static double radiusFromArea(double area)
  {
    return Math.sqrt(area / 3.14);
  }

  public static double circumferenceFromRadius(double radius)
  {
    return 2 * 3.14 * radius;
  }
}

I want to wire up an aspect that runs when these functions are returning, checks for NaN, and throws an exception. With a bit of understanding of AspectJ, this is quite simple:

package com.scottmcmaster365.aspects;

import com.scottmcmaster365.math.MyMathUtils;

/**
 * This aspect maps double NaN return values to runtime exceptions.
 * (For instructional purposes only.)
 * @author Scott McMaster
 *
 */
public aspect MapNaNResultToExceptionAspect
{
  pointcut mathClass() : within(MyMathUtils);
 
  pointcut mathMethods() : mathClass() && execution(* *(..));
 
  after () returning (double d) : mathMethods()
  {
    if( Double.isNaN(d) )
    {
      throw new RuntimeException("Result is NaN");
    }
  }
}

Hopefully even if you're not familiar with AspectJ, that makes at least a little sense to you. Otherwise, I suggest checking out a good tutorial. But in a nutshell, the pointcuts pick out the methods in the MyMathUtils class, and the after advice grabs the return value and allows us to inject code at compile-time.

With this aspect in effect, the client for finding the circumference given the area of a circle will throw a RuntimeException during the call to radiusFromArea():

final double TESTAREA = -1.0;
double radius = MyMathUtils.radiusFromArea(TESTAREA);
double circumference = MyMathUtils.circumferenceFromRadius(radius);
System.out.println(circumference);

That's exactly what I was after: Clean code that doesn't have to check return values explicitly or via macros. Score one for AspectJ. In fact, by making a very minor change to the pointcuts and advice spec, I can even fire the aspect at the Math.sqrt() call site, which (in a more complicated example) narrow the defect down even further. I'll leave this as an exercise for the reader unless someone comments that they want to see it.

Changing Return Values to Exceptions, C++ Edition

2009-08-06T19:44:00.000-07:00

In my last post, I discussed one reason why exceptions are superior to return values from the perspective of an API client. Despite this, you might still run into APIs (albeit hopefully legacy ones) that heavily use status codes and magic return values to indicate errors, and it may not be within your sphere of influence to change. A friend of mine asked me how to deal with this situation.

Ideally we would like to map error results to exceptions. One approach that works in C++ is to leverage the preprocessor via a "failure-checking" macro.

Consider my example from the last post which uses a couple of simple functions to calculate the circumference of a circle given the area. Here is the original client code:

float area = TEST_AREA;
float radius = radiusFromArea(area);
float circumference = circumferenceFromRadius(radius);
cout << circumference << endl;

Let's say that "radiusFromArea" uses the original version of the "mysqrt" function which returns -1 when it receives invalid input, and let's also stipulate that we cannot change the API for any of these functions. We'd like to present an exception to our callers as soon as we detect a bad value returned from one of the functions. We could do that with some spaghetti-ish code:

float area = TEST_AREA;
float radius = radiusFromArea(area);
if( radius < 0 )
{
  throw runtime_error("cannot be negative");
}
float circumference = circumferenceFromRadius(radius);
if( circumference < 0 )
{
  throw runtime_error("cannot be negative");
}
cout << circumference << endl;

(Ideally we would get the exception from "mysqrt" like I demonstrated last time, but recall that for the purposes of this post, we're pretending that we cannot change that function. And yes, in this contrived situation, we could easily check the negativeness of "area", but that would defeat the instructional value of the example.)

As far as isolating where a negative value might be coming from, that works. But it's annoying and ugly and it takes too much typing. That's why C/C++ programmers traditionally tend to be pretty bad about this type of error-checking. We can clean it up considerably with a macro:

#define IfNegativeThrow(x) { \
    if(x < 0 ) \
      throw runtime_error("cannot be negative"); \
    }

The new client code becomes:

float area = TEST_AREA;
float radius;
IfNegativeThrow(radius = radiusFromArea(area));
float circumference;
IfNegativeThrow(circumference = circumferenceFromRadius(radius));
cout << circumference << endl;

This is functionally equivalent to the immediately-preceding example, but it is more concise and (in my opinion) cleaner. You can potentially improve on it by adding an extra macro parameter for the exception message. Macros similar to IfNegativeThrow() can be found in many industrial-strength C++ codebases. This pattern is especially prevalent in old-school COM programming, where it is often used to deal with the infamous HRESULT.

(Shout-out to anybody who still does COM programming like that on a regular basis. Last time I did, I think Bill Clinton was still President.)

There remain some significant drawbacks here. Doing those assignments inside macros still leaves you with less-than-aesthetic code. You still have to maintain discipline about using the macros. And if you are in a language that doesn't support the preprocessor, things get more difficult and/or annoying. (You may have to implement IfNegativeThrow() as a full-fledged method, for example, and incur the requisite overhead.) In a later post, I'll address these issues.

Exceptions Versus Return Values

2009-08-04T21:06:00.000-07:00

I'm alarmed by the number of programmers I've been talking to lately -- not just C/C++ programmers, but also Java and C# programmers -- who are inclined to forgo the throwing of exceptions and instead use magic return values and/or status codes to indicate an API failure.

Consequenty I sat down to write a blog post about this. As a general rule, before I write something, I check to see if someone else has done it already, and sure enough, a number of people have taken on this issue competently. So instead of imitating their work, I wanted to point you to the best example, namely Ned Batchelder's.

One slight addition I wanted to make was to point out that Ned's article is primarily focused on the use of return values as status codes. I'm actually more concerned about overloading the return value with a sentinel value that is intended to indicate failure (a practice that Ned covers in the section on "Valuable channels"). In my opinion, this is far more insidious than the correct application of the status code idiom because it can time- and space-shift errors far from where they originate, which can unnecessarily complicate debugging.

Take the following square-root function as an example. Let's use C++.

float mysqrt(float n)
{
  if( n < 0 )
  {
    return -1.0;
  }
  return sqrt(n);
}

Here, -1 is being used as a sentinel value to indicate what any high school math student knows, namely, that you can't take the square root of a negative number. Consider what a client of this function must go through. Say I'm starting with the area of a circle, and I want to calculate the circumference using a couple of convenient helper methods:

float radiusFromArea(float area)
{
  return mysqrt(area / 3.14);
}

float circumferenceFromRadius(float radius)
{
  return 2 * 3.14 * radius;
}

Here is the client code:

float area = TEST_AREA;
float radius = radiusFromArea(area);
float circumference = circumferenceFromRadius(radius);
cout << circumference << endl;

If TEST_AREA is negative, we notice that the circumference is whacked (-6.28) when we are all done. That is, hopefully we notice and don't propagate the bad value forward, but I'll give the author of this code the benefit of the doubt for now. Even so, we still have no idea if the error is in radiusFromArea, circumferenceFromRadius, or some unknown function that one of them calls. Now condider what happens if mysqrt is written to throw an exception rather than return a "magic" error value:

float mysqrt(float n)
{
  if( n < 0 )
  {
    throw invalid_argument("can't take sqrt of negative number");
  }
  return sqrt(n);
}

When this code runs on a negative area for input, you see the error immediately in the form of an unhandled exception, with a stack trace pointing right to where mysqrt is called with the bogus value. This is much preferable for debugging, even in this simplistic example. Hopefully this will help convince you that exceptions are superior to the alternatives.

Download Eclipse 3.5 Galileo With Amazon S3 and CloudFront

2009-06-24T19:56:00.000-07:00

One piece of exciting news today is that Eclipse 3.5 (Galileo) is available. But the best part is that you can download it via Amazon S3 and CloudFront. As of earlier today at least, the "old-fashioned" mirrors were dragging, but the S3 download was extremely snappy.

Unfortunately, the S3 link isn't very well-advertised on the download page, so I wanted to mention it here. When you get to the download page for the distro you want, look down the page for "Download from Amazon Web Services". I bet you'll be impressed.

Mock Responses with Optional Elements in soapUI

2009-06-20T10:23:00.000-07:00

SoapUI will probably always be one of my favorite development and testing tools. Recently I had the opportunity to explore its web service simulation and mocking capabilities.

I needed to create some fairly-detailed mock responses for a service where pretty much every element in schema for the response was optional. By default, soapUI does not add optional elements to the mock response XML. This was causing me a significant amount of pain. I noticed in the Preferences, WSDL Settings, that there was an "Include Optional" option that says "always include optional schema elements when creating requests": I discovered that this option not only applies to creating requests, but also to creating mock responses. Problem solved. But I couldn't find this documented anywhere, so I wanted to point it out.

An Example of Why You Shouldn't Test in Production

2009-06-17T21:42:00.000-07:00

If you ever need a concrete example of why it is worthwhile to set up a proper testing environment for your online or data-based applications, consider this case. According to the Chicago Tribune, Illinois resident Tom Feddor, pround owner of license plate number of "0" has received some 170 bogus parking tickets because

A glitch occurred at the Chicago Department of Revenue involving Feddor's 0 plates being used during tests of ticketing equipment.

The article continues:

It turned out that some city parking-enforcement aides punched in 0 when testing their electronic ticket-issuing devices, Revenue Department spokesman Ed Walsh said. Officials weren't aware there was a 0 plate or that Feddor was receiving tickets, Walsh said in response to the Tribune inquiry.

But thank goodness:

"But we are taking steps to rectify the situation so in the future an actual registered plate number will not be used to do the testing," Walsh said.

Before you laugh too hard, be honest: How many of you have run "test" transactions through your live/production systems exactly in this manner? And what precautions did you have in place?

Why We Automate Tests

2009-06-11T11:59:00.000-07:00

James Whittaker, who is now blogging at the Google Testing Blog, talked a bit about manual versus automated testing. First he implied that testers are unjustifiably infatuated with their test automation frameworks:

Too often we are solution focused and marry ourselves to our automation rather than the product we are testing. Whenever I see testers that seem more vested in their automation than in the product they are shipping I worry the pendulum has swung too far.

He also tried to relate this question back to good test design.

Let's shift the debate to good test design. Whether those tests ultimately end up being executed manually or via a tool is a moot point, let's just make them good ones that solve real testing problems.

I'm all for good test design and solving real testing problems, but how tests are run is not a moot point, and I'd like to give all of those "solution-focused" testers a little more credit than that. Let me start with some simple logic:

Any series of steps that you perform more than once on a computer should be automated if at all possible.
A test case is (at least in some sense) a series of steps that you perform on a computer.
You want to run your test cases more than once (i.e. to perform regression testing).
Therefore, you should automate your test cases.

Occasionally, I'll meet an engineer that seems satisfied to autonomically do the same things over and over, day after day, but for the most part, testers understand my line of reasoning better than most people.

If automating a test case was as simple as running that test case in the first place, there would be nothing more to say, and this post would be finished. Of course, that's not reality, and therein lies the rub. But I would argue that you're still better off automating far more often than not. Automation yields significant benefits to both cost and quality.

Imagine that you design a test case and execute it the first time for a cost of x. In my experience, if a test case is ever going to detect a bug, there is a >90% chance that will occur the first time you run it. But if it really is a good test case like James is arguing for, you want to run it again and again, ideally with each new build of the software to look for regressions and related bugs.

Let's say that the cost to automate a test case is 10x what it takes to run it manually. If you only expect to run that test case five times over the life of the software, you should stick with manual testing. But if you expect to run it more than ten times over the life of the software, it is in your long-term financial interest to automate it.

Now let's say that you do expect to run the test case ten times, and you decide to forgo automation. If you want achieve the same quality assurances with your testing, you are committing to nine additional manual executions of that test. This is where quality risks start to creep into the process. How likely is it that you will really get all nine of those executions in given your busy schedule? How timely will they be? Will you perform them consistently and correctly, or will you make some mistakes and miss some bugs? Automation solves those problems.

Finally, what if you can build a tool that drives the cost differential between manual and automated testing from 10x down to 5x? And you can leverage that tool over the long-term not for one but rather n test cases? Granted, I am ignoring a couple of important factors, like the cost of automated-test-case maintenance and the magnitude of x relative to the release schedule. But I submit that the gist of the reasoning still holds. In many cases, we can build tools that work this well or better, maybe even for less than the 5x cost in this example. That is why testers work so hard at building test automation tools.

Streaming Office 2007 Documents Requires New Content Types

2009-05-24T15:43:00.000-07:00

Streaming out content that opens in Microsoft Office applications is a fairly common feature in web applications. This involves setting a Content-Type header to the appropriate MIME type. The new XML/ZIP-based file formats in Office 2007 bring with them new MIME types. There is a complete list of those here.

Another common technique is to stream XML or HTML and basically lie about the content type to get data presented in Word or (especially) Excel. After all, generating true native Office-formatted files on a web server generally involves third-party components, and depending on your platform and specific needs, those can get expensive. But Excel 2007 creates a gotcha for this use case called "extension hardening", where Excel actually verifies that a file format matches the extension it claims to be before opening it, and if not, gives an annoying, scary prompt to the user.

I first experienced this last week, with Excel 2007 trying to open an Excel 2007 document from a web site which set the content type to "application/vnd.ms-excel" -- i.e. an Excel 2007 document claiming to be an Excel 2003 document. This is bothersome, but at least it's easy to work around by changing the content type to "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet". (That's a mouthful.) It doesn't do much for the HTML case, however, as described in the above link. No good workaround...

Latest Google Toolbar Globally Hijacks Ctrl-Space, Causes Massive Eclipse Irritation

2009-04-12T21:28:00.000-07:00

I installed the latest version of the Google Toolbar a couple of days ago. Shortly thereafter, I did a bit of work in Eclipse and noticed that Ctrl-Space no longer invoked Eclipse Content Assist, but rather brought up a "helpful" Google search window. I have since learned that Google calls this the "quick search box", and the behavior is system-wide, at least on Windows. More importantly, I discovered how to turn it off: There is an option for it in the Google Toolbar Options dialog as shown below. Certainly they could have come up with something better than Ctrl-Space, couldn't they?

Perils of (Ab)using the java.ext.dirs Property

2009-03-23T20:34:00.000-07:00

I frequently use the java.ext.dirs system property as a shortcut for setting the classpath for a command-line Java program, i.e.:

java -Djava.ext.dirs=lib com.scottmcmaster365.MyMainProgram

No, this isn't great for production use, but for quick-and-dirty testing, it's much more convenient than creating a launcher batch file / shell script or manually typing a ginormous claspath.

Well, this practice jumped up and bit me the other day. After the requisite amount of pain and suffering, I observed that setting java.ext.dirs does not extend but rather overrides the real optional-extension location (at least with Sun's JVM), as sort-of-alluded-to here. As a consequence in this situation, Java will, at runtime, lose track of your optional packages. In my case, this manifested itself as an inability to load the JCA crypto providers (i.e. ClassNotFoundException).

If you're not using any of the standard optional packages, setting your own extensions path doesn't seem to matter. But I guess the moral of the story is that you shouldn't count on the extension mechanism to stand in for the classpath mechanism, even in all expedient situations.

Generating Strongly-Typed Properties Classes With openArchitectureWare, Part 4: Generating Java Classes

2009-03-11T21:03:00.000-07:00

In the fourth part of this series, I'm finally read to generate strongly-typed wrappers for Java .properties files. If you would like to review how I got to this point, check out the earlier installments here, here, and here.

First, a little bit about my code generation philosophy. I prefer to only generate specializations -- the things that will actually vary from model to model -- when possible. Inheritance works well for this: I can put common functionality in base classes and generate subclasses. This cuts down on the amount of code that I have to work with in the template. As an added bonus, the base class can be tested (and perhaps even used) independently of the code generation process.

With that in mind, I will start by creating base classes. Recall that my PropertySet is defined as supporting two different "load types" -- resource bundle and classloader resource. That means two base classes, which I will call ResourceBundleProperties and ClassLoaderResourceProperties. Since my goal is to demonstrate openArchitectureWare, model-driven development, and code generation, and not to write the be-all-and-end-all of .properties file loaders, this code won't support everything you can imagine. Generally speaking, I'm following the tried-and-true approaches described here.

ResourceBundleProperties.java

package com.scottmcmaster365.properties;

import java.util.Properties;

/**
 * Load a properties file from a resource bundle
 * following the approach described here:
 *  http://www.javaworld.com/javaworld/javaqa/2003-08/01-qa-0808-property.html?page=2
 * @author scottmcm
 *
 */
public class ResourceBundleProperties
{
 protected java.util.Properties properties;
 
 /**
  * Load the properties from the resource bundle with the given name.
  * @param name
  */
 public ResourceBundleProperties(String name)
 {
  ClassLoader loader = Thread.currentThread ().getContextClassLoader();

  final java.util.ResourceBundle rb = java.util.ResourceBundle
    .getBundle(name, java.util.Locale.getDefault(), loader);

  properties = new java.util.Properties();
  for (java.util.Enumeration<String> keys = rb.getKeys(); keys
    .hasMoreElements();) {
   final String key = (String) keys.nextElement();
   final String value = rb.getString(key);

   properties.put(key, value);
  } 
  
  if (properties == null) {
   throw new IllegalArgumentException("Could not load " + name + " as resource bundle");
  }
 }
 
 public Properties getProperties() {
  return properties;
 }
}

ClassLoaderResourceProperties.java

package com.scottmcmaster365.properties;

import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;

/**
 * Load a properties file from a resource bundle
 * following the approach described here:
 *  http://www.javaworld.com/javaworld/javaqa/2003-08/01-qa-0808-property.html?page=2
 * @author scottmcm
 *
 */
public class ClassLoaderResourceProperties
{
 protected java.util.Properties properties;
 
 private static final String SUFFIX = ".properties";

 /**
  * Load the properties from the resource bundle with the given name.
  * @param name
  * @throws IOException 
  */
 public ClassLoaderResourceProperties(String name) throws IOException
 {
  ClassLoader loader = Thread.currentThread ().getContextClassLoader();

  InputStream in = null;
  try {
   if (!name.endsWith(SUFFIX)) {
    name = name.concat(SUFFIX);
   }

   in = loader.getResourceAsStream(name);
   if (in != null) {
    properties = new java.util.Properties();
    properties.load(in);
   }
  } finally {
   if (in != null)
    try {
     in.close();
    } catch (Throwable ignore) {
    }
  }
  
  if (properties == null) {
   throw new IllegalArgumentException("Could not load " + name + " as classloader resource");
  }
 }
 
 public Properties getProperties() {
  return properties;
 }
}

Next I create a code generation template (an .xpt file, written in the oAW Xpand template language) to create a subclass based on the information in PropertySet models.

JavaWrapperTemplate.xpt

«IMPORT metamodel»

«DEFINE javaClass FOR properties::PropertySet»
 «FILE Package.replaceAll("\\.", "/") + "/" + Name + ".java"»
  package «Package»;
  
  /**
   * Load a properties file based on the approach described here:
   *   http://www.javaworld.com/javaworld/javaqa/2003-08/01-qa-0808-property.html?page=2
   */
  public class «Name-»
  «IF LoadAs == properties::LoadType::CLASSLOADER_RESOURCE-»
   extends com.scottmcmaster365.properties.ClassLoaderResourceProperties
  «ELSE-»
   extends com.scottmcmaster365.properties.ResourceBundleProperties
  «ENDIF-»
  {
   public «Name»() 
    «IF LoadAs == properties::LoadType::CLASSLOADER_RESOURCE»
     throws java.io.IOException
    «ENDIF»
   {
    super("«ResourceName»");
   }
   
   «EXPAND property FOREACH Properties»
   «EXPAND propertyWithDefault FOREACH Properties»
  }
 «ENDFILE»
«ENDDEFINE»

«DEFINE property FOR properties::Property»
 public «Type» get«Name»() {
  «IF Type == properties::PropertyType::Integer»
   return Integer.parseInt(properties.getProperty("«Name»"));
  «ELSE»
   return properties.getProperty("«Name»");
  «ENDIF»
 }
«ENDDEFINE»

«DEFINE propertyWithDefault FOR properties::Property»
 public «Type» get«Name»(String defaultValue) {
  «IF Type == properties::PropertyType::Integer»
   return Integer.parseInt(properties.getProperty("«Name»", defaultValue));
  «ELSE»
   return properties.getProperty("«Name»", defaultValue);
  «ENDIF»
 }
«ENDDEFINE»

While not quite as simple as the template I presented last time to generate sample .properties file, this is still pretty straightforward. Notice how I determine which base class to extend using the value of LoadAs defined in the input model. The last two Xpand templates for Property generate overloaded getProperty() methods, with and without the default value.

Before I can use this template, I need to wire it into my oAW workflow. So I make the following addition to generator.oaw, right after checking the model with the .chk file:

<!--  generate code -->
<component class="org.openarchitectureware.xpand2.Generator">
 <metaModel idRef="mm"/>
 <expand
  value="template::JavaWrapperTemplate::javaClass FOR model" />
 <outlet path="${src-gen}" >
  <postprocessor class="org.openarchitectureware.xpand2.output.JavaBeautifier" />
 </outlet>
</component>

Now when I invoke generator.oaw as an oAW workflow using my sample PropertySet model, I get the following code generated into my project's src-gen directory:

MyProperties.java

package com.scottmcmaster365.propertiessample;

/**
 * Load a properties file based on the approach described here:
 *   http://www.javaworld.com/javaworld/javaqa/2003-08/01-qa-0808-property.html?page=2
 */
public class MyProperties
  extends
   com.scottmcmaster365.properties.ClassLoaderResourceProperties {
 public MyProperties()

 throws java.io.IOException

 {
  super("MyProperties.properties");
 }

 public Integer getMyIntegerProperty() {

  return Integer.parseInt(properties.getProperty("MyIntegerProperty"));

 }

 public String getMyStringProperty() {

  return properties.getProperty("MyStringProperty");

 }

 public Integer getMyIntegerProperty(String defaultValue) {

  return Integer.parseInt(properties.getProperty("MyIntegerProperty",
    defaultValue));

 }

 public String getMyStringProperty(String defaultValue) {

  return properties.getProperty("MyStringProperty", defaultValue);

 }

}

The benefit of having a strongly-typed wrapper can be seen in this unit test example:

@Test
public void testLoadMyProperties() throws IOException
{
 MyProperties properties = new MyProperties();
 assertEquals(2, properties.getMyIntegerProperty());
 assertEquals("hello world", properties.getMyStringProperty());
}

And that's just about it. If I want to use my new properties wrapper in a real project, I will need to pull in all of the files from this project and add some sort of "generate" target to my Ant build which uses the oAW Ant task to invoke generator.oaw.

I should point out that there is a notable overhead involved in spinning up the oAW code generation pipeline, and if you have to do it multiple times per build, your build process can slow down considerably. It is therefore beneficial to consolidate into as few model files (.xmi's) as possible. So a possible enhancement to what I have done here would be to metamodel a class which is a set of PropertySets and then enhance my Xpand templates to generate multiple files, so that I can cram more than one PropertySet into a model.

I hope that after reading this series, you're ready to run out and EMF, oAW, modeling, and code generation. If you do, please share your experiences with me, because I'm always looking out for new techniques and best practices.

Generating Strongly-Typed Properties Classes With openArchitectureWare, Part 3: Checks, Workflow, and a Template

2009-03-10T19:21:00.000-07:00

Last time I created a sample model using the Properties metamodel that I built in the first post of this series. Now I'm going to put it all together and actually generate something with openArchitectureWare -- not Java code just yet, but rather, a sample .properties file which can be customized and loaded using the Java code that I'll create in the next entry.

Checks
openArchitectureWare provides Check constraints for validating models. For the Properties scheme that I have designed, it makes sense to require models to conform to three constraints:

PropertySet must have a name. I will use this to generate the Java class name.
PropertySet must have a resource name. This will tell me what file to load at runtime.
Properties must have a name. This one is pretty obvious.

Here is how to encode these rules in the Check language (file Checks.chk):

import Properties;

context properties::PropertySet ERROR "PropertySet must have a name" :
Name != null && Name.length > 0;

context properties::PropertySet ERROR "PropertySet must have a resource name" :
ResourceName != null && ResourceName.length > 0;



context properties::Property ERROR "Properties must have a name" :
Name != null && Name.length > 0;

The "import" line pulls in the Properties metamodel. Marking these constraints as "ERROR" stops oAW processing with the specified message when such a condition is encountered. The rest of the code is pretty intuitive. Later, when I talk about oAW Workflows, you'll see how this gets wired into the generation process.

Simple Template
It's time to write an oAW template to generate a sample .properties file. This is done with the oAW Xpand language, whose most visible feature is the cute little «chevrons» that mark off the directives. I won't attempt to detail the features of Xpand, but rather point you to the documentation. Suffice to say that it's very easy to get started. Here is a template (PropertiesFileTemplate.xpt) that generates a .properties file from a model:

«IMPORT metamodel»

«DEFINE propertiesFile FOR properties::PropertySet»
«FILE ResourceName + ".SAMPLE"»
«EXPAND property FOREACH Properties-»
«ENDFILE»
«ENDDEFINE»

«DEFINE property FOR properties::Property»
«Name» = «DefaultValue-»
«ENDDEFINE»

When I finally run this in the next section, it will generate a file in .properties format with each property in the model. The file is named based on the value of the ResourceName attribute of the PropertySet. It is suffixed with ".SAMPLE" because presumably it would be customized in a real application. In my next post, I'll use Xpand to achieve the ultimate goal of this project, namely, to generate some Java code to load and wrap these properties. First, though, I need to tie everything together so that the oAW generator can run.

Workflow
"Workflows" are essentially build scripts for openArchitectureWare-based artifacts. They generally end with an extention of ".oaw", and they can be launched directly from Eclipse in an openArchitectureWare project using a "Run As | openArchitectureWare Workflow" context menu item. There are a handful of things that you can do in a workflow, but in my experience, they tend to be pretty boilerplate. Here is one (generator.oaw) that runs Checks and invokes the generator on the PropertiesFileTemplate for our model:

<?xml version="1.0"?>
<workflow>
 <property name="model" value="PropertiesGenerator/src/SamplePropertySet.xmi" />
 <property name="src-gen" value="src-gen" />
 
 <!-- set up EMF for standalone execution -->
 <bean class="org.eclipse.mwe.emf.StandaloneSetup" >
  <platformUri value=".."/>
 </bean>

 <!-- load model and store it in slot 'model' -->
 <component class="org.eclipse.mwe.emf.Reader">
  <uri value="platform:/resource/${model}" />
  <modelSlot value="model" />
 </component>

 <!-- check model -->
 <component class="oaw.check.CheckComponent">
  <metaModel id="mm"
   class="org.eclipse.m2t.type.emf.EmfRegistryMetaModel"/>
  <checkFile value="metamodel::Checks" />
  <emfAllChildrenSlot value="model" />
 </component>

 <!--  generate sample .properties file -->
 <component class="org.openarchitectureware.xpand2.Generator">
  <metaModel idRef="mm"/>
  <expand
   value="template::PropertiesFileTemplate::propertiesFile FOR model" />
  <outlet path="." />
 </component>
</workflow>

In a more serious implementation, the execution of the .oaw file would be integrated in an Ant build using oAW's custom Ant task. For this project, I'll just use the oAW Eclipse runner.

The model to work with is declared as a property at the top of the file. The key sections are the last two -- "check model" and "generate sample .properties file". The only potentially-tricky part is getting the package references right. So here is a snapshot of my project for you to compare to:

Now I actually have a working code generation pipeline set up. To conclude in my next post, I'll write a slightly more sophisticated template to generate Java code, and maybe talk a little bit about code generation philosophy.

Generating Strongly-Typed Properties Classes With openArchitectureWare, Part 2: Model

2009-03-01T13:31:00.000-08:00

Previously I demonstrated how to create an Ecore metamodel describing Java .properties files, with the ultimate goal of using openArchitectureWare to build strongly-typed accessor classes for my properties. Before I start writing oAW templates, though, I want to have a sample model that I can work with which is relatively simple but still exercises most of the breadth of my metamodel.

The oAW sample project contains a model already called "Model.xmi". The "xmi" is "XML Metadata Interchange", an OMG specification concerned with interoperable models, and the default format for models in Ecore/EMF. I will delete the existing Model.xmi file and create a new model called "SamplePropertySet.xmi". One way to do this is to open the Properties.ecore metamodel, right-click on the class that I want at the "root" of my model (namely, PropertySet), and select "Create Dynamic Instance". Choosing a location in my workspace, away I go.

The .xmi file can be edited with the "Sample Reflective Ecore Model Editor", which is essentially the same as the tree-node-properties editor I used on the .ecore file, except now the entities that I edit on my model are instances of the classes and attributes defined in my metamodel. In other words, the "root" of my model is a PropertySet, and to that I can add children which are Property instances. When I select one of these and open the Eclipse Properties grid, the properties are the class attributes defined in the metamodel. If you have been following along and are still a little fuzzy on the relationship between metamodel and model, the following illustration might be helpful:

I'm going to test my openArchitectureWare code generation templates with a couple of properties -- one of type "Integer" and another of type "String". My "completed" model looks like this:

There are a number of other things that I could do with EMF at this point, such as generating a Java object model and "starter" Eclipse plugins for working with my models. But I don't need to do any of that for this project: Starting next time, everything going forward will be in the Land of openArchitectureWare.

Generating Strongly-Typed Properties Classes With openArchitectureWare, Part 1 Addendum: Graphical Ecore Editor

2009-03-01T11:55:00.000-08:00

Last time I showed how to build an Ecore metamodel in Eclipse using the Sample Ecore Model Editor. Ed Merks from the EMF team (whose EMF book is out in a new edition, by the way) commented that I could be using the Graphical Ecore Editor, which is a part of the Ecore Tools subproject. He is absolutely correct. (Thanks Ed.) There are a couple of steps to go through to do that, so I'll talk about those here.

First, you need to install the Ecore Tools package, which you can find on the Ganymede Update Site (assuming you're using Eclipse 3.4) under "Models and Model Development". After you do that, you can select File | New | Other in Eclipse, and there will be an "Ecore Tools" folder available. Underneath that, choose "Ecore Diagram". A wizard appears, and you can select your .ecore file as the domain model.

This will add a file to your project which has the extension ".ecorediag", and when you open it up, you'll get a Rose-like boxes-lines-and-properties editor which is far more user-friendly than the sample editor. Here is what my Properties Generator metamodel looks like in diagram form after some slight rearrangement:

Generating Strongly-Typed Properties Classes With openArchitectureWare, Part 1: Metamodel

2009-02-28T17:16:00.000-08:00

I do a lot of model-driven software engineering and code generation. But you might not know that, because I've never written about it at all in my blog. In the next couple of posts, I'll go about changing that.

First, a bit about the application that I'm going to build. I have long believed in having wrapper classes for resources and properties. Visual Studio and .NET now do a lot of this for you automatically. In Java, we're still pretty much on our own. What I would like to do is automatically generate a Java class which exposes a strongly-typed getter for each property in a .properties file.

To accomplish this, I'm going to use one of my favorite model-driven development tools, openArchitectureWare (oAW). oAW is comprised of a pretty extensive toolchain for model transformation, validation, and code generation, and of course, it's open-source. As you will see, it integrates well with the Eclipse Modeling Framework (EMF). If you would like to follow along, install the openArchitecture Eclipse plugin. While you're at it, check out the oAW documentation, particularly the oAW-EMF tutorial (but note that we're going to take a simpler approach that what is described there).

In Eclipse, I start by creating a new "openArchitectureWare Project", which is available courtesy of the oAW plugin. I took the option to "Create a sample", because it creates a number of EMF and oAW files that I need. The sample project looks like this:

Right now, it's possible to right-click the "generator.oaw" file and select "Run As...oAW Workflow" and see some sample Java code get generated into the src-gen folder from Model.xmi.

Rather than stop right now and discuss what each of these files does, I'll proceed through the development of my application and hopefully you'll pick things up intuitively. For starters, I'm not going to use extensions, so I can go ahead and delete Extensions.ext and GeneratorExtensions.ext. This will break the generator.oaw build for now, but I'll put it back together as I proceed.

One of the most important files, and the focus of this post (finally!) is the .ecore file, which I will rename to "Properties.ecore". This is where the metamodel is defined. In this example, I want to create models for properties, so my metamodel needs to describe what a "property" is. In the Java .properties file sense, it is basically a "name", a "type", and a "value". Additionally, I want to bundle individual properties into "property sets" which are stored in .properties files. The property sets themselves need a "name" and, optionally, a Java package to live in. Finally, for illustrative purposes, I'll support loading them either as Java classloader resources or resource bundles.

When I double-click to edit the .ecore file, it opens in the "Sample Ecore Model Editor" by default. This editor is nothing to write home about, and in fact I suspect they call it "Sample" because the idea behind EMF is that tool builders will write (and sell you) their own higher-end Ecore editors, probably in the form of Rational-Rose-like graphical modeling environments. No matter -- this editor will work fine for my purposes here. The upside of the simple Ecore editor is that it is quite intuitive to use -- just right-click nodes to add children (classes, enums, attributes, references to other classes, etc.), and use the Properties view to configure them. My metamodel ends up looking like this:

The only trick I want to point out here is how to configure the "Properties" EReference on the PropertySet EClass. First, I want to have multiple Property instances in a PropertySet, which I achieve by setting the "Upper Bound" on Properties to be "-1". Second, a PropertySet logically "contains" its Properties, so I set the "Containment" property to "true". (If you don't do this last step, you will experience perhaps-confusing difficulties when you actually go to create a Property while modeling in the Eclipse EMF model editor.)

Now I need to use Eclipse to create a sample model based on this metamodel to use for testing while I develop code generation templates. I'll demonstrate that in my next post.

Getting Integer and Long System Properties in Java

2009-02-14T12:36:00.000-08:00

In the learn-something-new-every-day department, in a code review this week for one of the developers on my team, I noticed that he was calling Integer.getInteger(). I had never seen this before. Turns out that this method and its overloads get system properties as Integers, handling the numeric conversion for you and returning null if it fails. There is also a Long.getLong() that does the same thing for (you guessed it) Longs. So all of those times I called System.getProperty() followed by Integer.parseInt() with a catch of NumberFormatException, I was doing WAY too much work. Good to know.

Programming Culture

2009-01-24T15:04:00.000-08:00

I was asked recently how long I thought it should take someone to become "proficient in a new programming language", assuming said individual is already proficient in a different language. In various contexts, people will throw out numbers like six weeks, six months, a year, or more. Obviously the answer depends greatly on what the questioner means by "proficient", as well as what languages are involved. But I think it's the wrong question.

There are several good analogies between spoken languages and programming languages. I've heard it said that a good (spoken) language learner can become conversational new languages very quickly, but the hard part (and the part that takes the most time) is figuring out how to apply that language within its native culture.

I have picked up a (very) little bit of Chinese here and there, and I know that "xiao jie" translates into English as "miss" and is used as a term of address for women. But I'm afraid to use it in practice (at least until I discuss the matter further with one of my Chinese friends), because I have heard that its appropriateness might vary depending on the specific context, as well as from where in China the listener hails, and first and foremost I do not want to offend anyone.

Similarly, a competent software engineer can certainly become comfortable with the syntax and mechanics of using a programming language in a relatively short time. Learning how to apply those concepts in such a way that experienced practitioners of the language feel comfortable reading and working with the code, though, will take more time.

Back in the .com heydey, I was working on desktop applications at Microsoft and looking for an entry into the web world in my spare time. A friend of mine was working at a web startup in DC, and they were doing Perl. I won't name names to protect the innocent, but my friend was and is a True Perl Expert. So while going through the process of learning Perl using (among other things) the Larry Wall book, I would periodically send him my code, usually quite impressed with myself, and ask him how I was doing. The reply always went something like:

"Well, yes, that would work, but it looks too much like C++. A Perl programmer would never do it that way."

Perl, it turns out, has a very distinct culture. Although I had learned the Perl language, I had not learned how to apply it within the context of that culture. I wasn't correctly using the language's idioms, shortcuts, and libraries, and the code I was producing was "turning off" a more experienced Perl hand. Framing the situation in terms of Webster's definition of "culture", I was not respecting the "shared attitudes, values, goals, and practices" of the Perl programmers' community.

I know how my friend felt. Since that time, I've had the opportunity to look at a lot of Java and C# code written by unreformed C/C++ hands, and it is sometimes quite amusing. My personal favorite -- and the deadest giveaway -- is consistent use of:

String s = new String("Hello");

But there are many more subtle signs, ranging from code formatting and indentation to how one writes a simple conditional expression, and, perhaps most importantly in this age of large platforms, not knowing which libraries and APIs are available. Finally, there is the issue of context: Using a language and its libraries to write toy programs to reverse strings is one thing, but using it to put together a scalable web application is an entirely different matter.

Before a programmer can become truly "proficient" in a team environment, (s)he must internalize this knowledge. There is really no substitute for time spent reading code and observing more experienced developers. To borrow again from the world of spoken-language learning, immersion will yield the best results.

Joel Spolsky has an interesting essay on the cultural differences between Unix and Windows. If we wanted to, we could do similar analyses of Java vs. C++ vs. .NET, C# vs. VB, and others. To take one hypothetical, a C# programmer can start writing syntatically (and even stylistically) correct Java code in short order. But to ask that programmer to immediately and independently stand up a web application using Spring, Hibernate, and Struts, with an automated Maven build, unit tests, and one-touch deployment, and make it follow all of the best practices of the Java culture so that others can effectively carry the application forward -- well, that doesn't strike me as reasonable regardless of how experienced that person was in C# and .NET.

In this time where many software developers are reevaluating their careers and techology portfolios (often not by choice), I'm not saying that folks should avoid putting themselves or other developers in positions where they have to adapt to new programming languages and cultures. Quite the contrary -- throughout my career, I have observed that there is much to be gained from a "cross-polination" of ideas between different parts of the software development world. I would suggest, however, that when we do go this route, we need to set realistic expectations for people making these changes.

.NET FileTime Struct Revisited

2009-01-18T09:53:00.000-08:00

On my old blog, I wrote about the problems with the .NET FileTime struct and gave an approach to working around them. Since then, a couple of people have sent me Spaces messages suggesting a way to simplify at least a portion of my solution. Unfortunately, because of all of the recent changes on Spaces, and because I do not check my old space on anything like a regular basis, I have "misplaced" those messages. If anyone having problems with FileTime lands here with improvements to share, please do so and I will follow up ASAP. Thanks!

Library Thing

2009-01-11T14:38:00.000-08:00

Last fall I wrote about using an Amazon Widget to track my library of technical books. Well, I just got turned on to Library Thing, which seems to be a FAR better approach to recording and sharing my interest in books. I won't go through all of Library Thing's features here, but in a nutshell it provides a means to "socially catalog" your books, in the same sense that del.icio.us provides for bookmarking. My first pass at entering my library is here. Perhaps the best part is that after I entered these books, I was able to get book recommendations based on the catalogs of others with similar interests, and I can attest that they are amazingly good. In fact, of the top seven recommendations that it gave me, two were books that I already owned but had forgotten to enter. Finally, with the Library Thing blog widget, you can now see random books from my library in the sidebar of my blog.

Why to Consider Using an Application Server Tier

2008-12-31T21:00:00.000-08:00

Toward the end of .NET Rocks!' latest interview with Oren Eini (which I enjoyed), Oren mentioned that he was thinking about "how to kill three-tier architecture", going on to say that the one reason you wanted to have an application server was "connection pooling". Richard Cambell went on to say that you might use an application server if you have "application resources or some kind of a set of objects that are processor intensive or long running that are independent, the execution of that is somewhat independent with what the web server has to do".

(It's great that .NET Rocks! has transcripts, which makes it extremely easy to quote this stuff. Thanks Carl.)

I think the discussion above is an incomplete view which sells an independent application server tier far short. No, you don't always need application servers, but they do have practical uses beyond the scenarios mentioned above, and they are often borderline-essential for a robust web application architecture.

The rest of this post is primarily intended for my .NET brethren. Three-tier server architecture discussions get rather confusing in the Microsoft world because a) Microsoft's tooling doesn't really support it well out of the box (and in fact much of it pushes you very hard in the two-tier direction), and b) both the web server and application server typically end up being IIS. Java and LAMP developers are quite familiar with using Apache to front for Tomcat, JBoss, or their company's favorite expensive commercial JEE server.

Perhaps most importantly, having separate application servers is critical to the proper implementation of a DMZ, which you most likely want if you are running on the capital-I Internet. In a nutshell, this architecture allows you to better protect your database if your externally-facing web server tier is compromised. See this link into the Wikipedia article for a better explanation.

Additionally, separate web and application tiers allow for limitless options in independent provisioning, scaling, and tuning. The power here should not be underestimated. You can tune your web servers for I/O throughput and your application servers for raw processing horsepower if that is what your application demands. If you are serving a lot of static content, you can move it forward to the web server tier and take the load off your more expensive application servers. You can use different load-balancing and encryption strategies. You can also choose to use a cheaper computing platform on one tier (typically, the web tier) to save on hardware costs and licensing fees.

I have a book that I like to refer people to when this subject comes up -- Architecting Enterprise Solutions: Patterns for High-Capability Internet-based Systems, by Paul Dyson and Andrew Longshaw. It's a patterns book that discusses the tradeoffs involved in robust Internet architectures. I'm not aware of anything else quite like it, so if you're interested (or if you would simply like to read something that will probably expand your engineering horizons), get it.

I've been known to use the title of this post, "Why to Consider Using an Application Server Tier", as an interview question. I guess I might not be able to do that anymore. But it's a fun blog discussion topic, which is worth it in my opinion.

Integrating Log4Net and WF

2008-12-23T20:10:00.000-08:00

Like many people, I've been trying to take greater advantage of the public library lately. Of course, the supply of technical books is fairly limited, so I basically get whatever they happen to have on the shelf on a given day that looks relevant. (That's saying nothing about the great digital offerings that most libraries now have, but they're a little hard to read on the bus.)

One of the books that I have now is Pro WF. In the process of working through this book and improving my core Workflow Foundation skills, I've been trying to build some useful tools, such as the NUnit test fixture class that I wrote about recently.

I've grown a little tired of writing CodeActivities here and there just to print out the current value of some variable, so I started thinking about something more general and with a wider applicability. Put that together with the fact that I've never been particularly satisfied with Microsoft-supplied logging solutions (which would be the subject of another post), and what I decided to do was create a custom WF activity that logs using Log4Net and takes advantage of dependency properties to configure the variables to log. The source code is on my Code Gallery page here.

Using the Log4NetActivity is very straightforward. It does assume that you configure Log4Net prior to use -- that should be handled in the startup of your host application. Then you just need to give it the name of a logger along with a message.

The message can -- and probably will -- use the syntax of a format string, which can pull from up to ten dependency properties (Arg0 to Arg9) configured on the activity. That's the only remotely-interesting thing about the design itself. Originally I wanted to allow a dynamic set of dependency properties, but in the end I didn't see any way to make that work. The closest thing I could find was this excellent post and sample code by Guy Burstein, but had I gone that route, I believe the best I would have been able to do was force you to declare each argument to log before binding, and that would have been annoying.

The activity can also use the Exception-logging capability of the Log4Net APIs, but the Exception itself is not configurable via a dependency property, limiting its usefulness. (I have no better ideas here.)

Certainly nothing revolutionary, but hopefully somewhat useful.

Dozer Custom Converter for Joda DateTime

2008-12-19T15:47:00.000-08:00

I mentioned in my last post how excited I was to try out Dozer. Today I got the chance to play around with it a little bit. As a proof-of-concept for myself, I wrote a custom converter to handle mapping of Joda-Time DateTime's to and from Java Calendars. I thought that this should be very easy, and I was happy to find out that it was. The first step was to actually implement the CustomConverter interface:

package com.scottmcmaster365.dozerdemo;

import java.util.Calendar;

import net.sf.dozer.util.mapping.MappingException;
import net.sf.dozer.util.mapping.converters.CustomConverter;

import org.joda.time.DateTime;

/**
* Custom converter for Joda DateTime's.
* @author Scott McMaster
* http://www.scotmcmaster365.com/
*
*/
public class DateTimeConverter implements CustomConverter {

 @SuppressWarnings("unchecked")
 @Override
 public Object convert(Object existingDestinationFieldValue,
   Object sourceFieldValue, Class destinationClass, Class sourceClass)
 {
  if( sourceFieldValue == null )
  {
   return null;
  }
  
  if( sourceFieldValue instanceof Calendar )
  {
   // Note that DateTime is immutable, so
   // we can't do much with the existingDestinationFieldValue.
   return new DateTime(sourceFieldValue);
  }
  else if( sourceFieldValue instanceof DateTime )
  {
   Calendar result;
   if( existingDestinationFieldValue == null )
   {
    result = Calendar.getInstance();
   }
   else
   {
    result = (Calendar) existingDestinationFieldValue;
   }
   result.setTimeInMillis(((DateTime) sourceFieldValue).getMillis());
   return result;
  }
  
  throw new MappingException("Misconfigured/unsupported mapping");
 }

}

The logic here is very straightforward, although you could imagine creating many more Dozer converters for DateTime's. (One that accepts a formatted String comes immediately to mind.) You can wire this up with a little bit of XML as so: Here are a couple of unit tests that demonstrate the capability. Not shown are a couple of little classes, TypeWithCalendar and TypeWithDateTime, which have bean properties called "value" of type java.util.Calendar and org.joda.DateTime, respectively.

/**
 * Test mapping from Calendar <-> DateTime
 * using Dozer.
 * @author Scott McMaster
 *
 */
public class TestDozerDateTimeConverter
{
 private DozerBeanMapper mapper;

 @Before
 public void setUp()
 {
  List mappingFiles = new ArrayList();
  mappingFiles.add("dozermappings.xml");
  mapper = new DozerBeanMapper();
  mapper.setMappingFiles(mappingFiles);
 }
 
 @Test
 public void testToCalendarType()
 {
  DateTime then = new DateTime(1995, 11, 27, 0, 0, 0, 0);
  TypeWithDateTime joda = new TypeWithDateTime();
  joda.setValue(then);
  
  TypeWithCalendar calendar = 
    (TypeWithCalendar) mapper.map(joda, TypeWithCalendar.class);
  
  assertEquals((Long) then.getMillis(), (Long) calendar.getValue().getTimeInMillis());
 }
 
 @Test
 public void toJodaType()
 {
  Calendar then = Calendar.getInstance();
  then.set(1995, 10, 27);
  TypeWithCalendar calendar = new TypeWithCalendar();
  calendar.setValue(then);
  
  TypeWithDateTime joda =
   (TypeWithDateTime) mapper.map(calendar, TypeWithDateTime.class);
  
  assertEquals((Long) then.getTimeInMillis(), (Long) joda.getValue().getMillis());
 }
}

Dozer Can Help Me Manage Parallel Object Hierarchies!

2008-12-16T20:15:00.000-08:00

Some time ago, I wrote a post called "Robust Architecture for Object-Relational Mapping", in which I discussed and diagrammed a separation of "Business/Domain" and "Persistence" objects into separate, parallel object hierarchies. The idea is to isolate lazy loading and other ORM features (as well as any database schema oddities) away from the actual objects that implement the business logic. The main drawback to this approach is that it requires you to implement potentially-tedious and error-prone logic to transfer data back and forth between the layers. Taking this further, I think it's perfectly reasonable for a modern enterprise application to have up to four parallel object hierarchies:

Persistence objects, which can closely mirror the database schema if necessary and guarantee that "transparent" ORM stays that way.
Business/Domain objects, which reflect the application's core entities and the business logic that manipulates them.
View objects, which are data objects optimized to serve the View layer.
Message objects, which are exposed to the outside world via services and need to remain stable in the face of changes to the business layer.

In anything other than a trivial application and given current technologies, it is very difficult to use a single object model to serve all of the needs described above. (Trust me, I've tried.) So in reality, we spend a lot of time in the transfer implementation, moving values back and forth between the different flavors of objects, either with tedious set/get code, a hand-rolled reflection-based framework, or (more likely) a combination of both.

Clearly this is annoying. So I was excited today when one of my friends told me about Dozer, which purports to be a "powerful, yet simple Java Bean to Java Bean mapper". I haven't tried it yet -- just spent a lot of time going through the documentation. But it appears that Dozer will allow me to wire up hierarchies like those I describe above in a minimal-code, declarative fashion. I'm looking forward to trying it out and reporting my experiences.

Unit Testing WF Workflows with NUnit

2008-12-07T16:56:00.000-08:00

When I looked around for approaches to testing Windows Workflow Foundation workflows, I was surprised at how few examples I found. There is a fair amount of buzz around testing custom activities, but not much on entire workflows. Maybe that's because folks consider testing a complete workflow to be an "integration testing" task rather than a "unit testing" one. As an eternal pragmatist, that is a distinction that I could not possibly care less about if I tried. I mean, you really should be testing your complete workflows to the extent possible, and if your unit testing tools work for that, why not use them? So I set out to create a base class using NUnit which could easily be extended by test fixtures that want to run workflows.

My first cut at this is on my Code Gallery page. The base class is called WorkflowTestFixture. It takes a generic type parameter which is the (sequential) workflow that you want to execute. The resulting test fixture code comes out looking like this:

[TestFixture]
public class TestMyWorkflow : ScottMcMaster365.WorkflowTestFixture<MyWorkflow>
{

    [Test]
   public void TestNormalExecution()
   {
      AddArgument("AccountNumber", 60005);

Run();

Assert.AreEqual(20000.00M, GetOutput("AvailableCredit"));

}

AddArgument, Run, and GetOutput are base class methods which set up the workflow's inputs, synchronously execute the workflow, and retrieve output parameters, respectively. The actual implementation of WorkflowTestFixtue<> is nothing special -- in fact, it comes out looking a lot like some of the code that K. Scott Allen posted for testing custom activities. (I'm not sure why Scott didn't create a base class for the stuff that he finds overly verbose. Maybe he did that in a later post that doesn't Google as well.)

Clearly there are some workflows and styles of workflow for which this approach will not work well. Right now I see it as primarily useful for sequential workflows which can execute directly to completion without a lot of external integrations with databases, services, etc. That said, I believe that still constitutes an important class of workflow. As for the limitations, I hope to remove some them in the future. Suggestions are welcome.

Looking for Tips on Formatting Code in Blogger

2008-11-25T20:52:00.001-08:00

OK, if you read my last post, you probably noticed that the formatting of the Java and (especially) XML code was, well, not so great. And it took me some time just to get it that good. I have some code-to-highlighted-HTML tools that I like to use, but when I push it into Blogger, it gets mangled in various ways. For all its many faults, Spaces at least didn't give me a lot of trouble in this department. I searched around a bit for techniques and tools to improve this situation. Offhand, I didn't find anything that really impressed me as a good solution. I'll keep looking and trying new things, but in the meantime, if you have some good tips here, please share them.

dbUnit and HSQLDB Databases with Schema Names

2008-11-25T19:59:00.000-08:00

There are several ways to manage unit tests that involve a data layer. Someday I'll enumerate them in another post. But when I'm doing Java, one of my favorites is using HSQLDB as a fast, in-memory stand-in for a more heavyweight database. I also like dbUnit to manage setting and resetting the data in the in-memory database. I ran into an interesting problem when trying to use dbUnit with an HSQLDB database that employed named schemas (like, say, Oracle). In case you've never had to create your own schemas in DDL before, you can create the schema "SCOTTSCHEMA" like this:

Statement ddl = conn.createStatement();
ddl.executeUpdate("CREATE SCHEMA SCOTTSCHEMA AUTHORIZATION DBA");

Now, when you get ready to use dbUnit and create your IDatabaseConnection, you need to pass the schema name:

IDatabaseConnection dbUnitConn = new DatabaseConnection(conn, "SCOTTSCHEMA");

Now I should be able to load up XML files using dbUnit that look like this for a hypothetical "CONTACTS" table:

<?xml version="1.0" encoding="UTF-8"?> <dataset> <SCOTTSCHEMA.CONTACTS FIRST_NAME="Scott" LAST_NAME="McMaster"/> <SCOTTSCHEMA.CONTACTS FIRST_NAME="Tracy" LAST_NAME="McMaster"/> </dataset>

It turns out that passing the schema name in the second parameter is necessary, but not sufficient: If you try to run a database operation against dbUnitConn at this point, you'll get the following exception:

org.dbunit.dataset.NoSuchTableException: SCOTTSCHEMA.CONTACTS at org.dbunit.database.DatabaseDataSet.getTableMetaData(DatabaseDataSet.java:222) at org.dbunit.operation.DeleteAllOperation.execute(DeleteAllOperation.java:109) at org.dbunit.operation.CompositeOperation.execute(CompositeOperation.java:79) at ...

When I first hit this, it was easier to figure out what was going on from the dbUnit code rather than the documentation. I discovered rather quickly that you also need to set FEATURE_QUALIFIED_TABLE_NAMES in the IDatabaseConnection's config:

dbUnitConn.getConfig().setFeature(DatabaseConfig.FEATURE_QUALIFIED_TABLE_NAMES, true);

Now dbUnit will build its internal map of database tables prefixed with the schema name and will be able to find the table metadata when you execute an operation. It seems like it would make sense to default this feature to "true" when you create a database connection with a schema name...

My New Blog Location

2008-11-22T12:18:00.000-08:00

Well, I finally did it. My old buddy Chad has been on me about moving my blog to a different platform. I was reluctant, because I feel like I've built up a pretty good body of content on my old blog, and I didn't want to risk losing regular readers or anyone who found me while looking for solutions to their technical problems on Google. But some time ago, I turned off comments due to an endless stream of spam appearing on ancient posts. I'm tired of not having comments. I want comments. And I want a whole lot of other features that Windows Live Spaces does not provide, even after all these years. So here I am on Blogger. I plan to keep the content the same -- book and article reviews and suggestions, discussion of the software development tools that I use, sample code in Java, .NET, Ruby, and anything else that I find interesting. So I'm glad you found this and hope you'll stick with me while I get this blog going. Of course, you can always find my previous work here.