Category Archives: Software Development

The users are in control

I really enjoyed and appreciated this essay from Raganwald about the user experience at work versus that of their home PC environments (among other topics).

I particularly liked the point:

And meanwhile, the very same users could walk across the street and buy themselves a much better PC for less money than we pay and take it home the same day.

Ain’t that the truth. I put together my Core 2 Duo system for the same price as my crappy Pentium 4 hyperthreaded number at work. The time frames were not that far apart. The Core 2 runs circles around this sick puppy.

A company’s philosophy should be to get users (especially developers like me!) whatever hardware/software they need immediately. Within minutes or hours, not days or weeks. Of course, then you have to trust your employees to make good requests. But if you don’t trust them to know what they need, why trust them to do their job at all?

The essay goes on to talk about writing applications that take advantage of modern PC horsepower. I think I’m doing an ok job of this at work now. For example, we have a database of assets that is continually growing. It used to be we could view all of the assets on a single page that took about 30 seconds to load off-site.

Now that list will take several minutes to bring up. Yeah, we’re growing. So we need tools to help manage all of that information. One thing I’m building right now (as soon as I’m done writing this, as a matter of fact) is a quick filtering functionality on a desktop app that talks to the database. The list of assets is filtered as you type, taking advantage of the fast PCs we have these days.

That’s just one example. I can think of others that are immediately useful in business apps:

  • better visualization – it takes time and thought to develop good data visualization, but the results are usually worth it
  • drag & drop support – make sense to drag assets from a customer to another? I don’t know, maybe.
  • dynamic views – use all that processing power to show something more interesting than fields on a scrolling form. Graphics views that change in response to context
  • track history, undo/redo – might make sense in some contexts
  • attach more meaningful information – pictures, videos, documents, whatever. – with stuff like WPF, it’s easier than ever to display varied content

Technorati Tags: , , ,

Don’t ignore naive or "stupid" algorithms — hardware is cheap and fast

I just had a nice reality check. Sort of pleasant in that I realized I could save a LOT of memory usage (like from 35MB down to 9 MB), but also aggravating because I have spent probably 10-20 hours developing a clever algorithm designed for speed.

Lesson learned. I should have built the naive version first. Instead, I wrote up two successively more “brilliant” versions that went through all sorts of hoops to get the most speed out of it. Of course, to do this, they took up all sorts of memory with indexes, and the index creation was starting to take about 10 seconds or longer. I should have just built the naive version.

I just wrote the naive version and realized I could have done that in about 5 minutes and saved many hours of tweaking. The component is a type of indexing component, so there were three metrics: index creation time, lookup time, index size. Here’s a rough comparison just to give an idea:

  Clever Algorithm Naive Algorithm
Index Creation Time 10s 0.3s
Lookup Time 0.0001s 0.005s
Index Size 35MB 9 MB
# items

~27,000

Pretty impressive speed numbers aren’t they! That clever algorithm really rocks. And it would be awesome to use if I was doing a lot of searching consecutively, but the searching in my app is tied to the UI, thus to the user, so in reality 0.005 seconds is not that much different than 0.0001 seconds. <sigh>

The numbers above are from my main machine, which is a Core 2 Duo. Just to be safe I tested the naive algorithm on my 4-year-old Pentium 4 laptop to validate that it still has acceptable performance on an older machine. The creation takes 0.05 seconds, but lookup time isn’t much slower, if at all.

And 9MB index is MUCH better than a 35MB index.

In summary, lessons learned:

  1. Hardware is cheap and fast. Don’t waste time optimizing for speed if you don’t have to. While there are signs the raw speed of a processor is plateauing as multiple cores become more important, in general, speed is always increasing.
  2. If you’re running something when a user inputs something, speed isn’t critical (as long as you have it faster than human response time)
  3. Every application is different, so measure and think critically. If my app needed to run the search 100 times per second, the clever algorithm would definitely be better.
  4. There is almost always a tradeoff between speed and size. Which is more important depends on the app.
  5. Write the dumb algorithm first. It might be good enough and you’ll save yourself hours of development and debugging time.

Technorati Tags: , , , , ,

Thoughts on Process: Automation (and examples)

If a process, or part of a process, can be automated it should be. For example, in a project at work, part of our process is to make sure that every dialog present in the English resources of an MFC application is also present in all the other dialogs. We do this manually by loading each language into the app and triggering every dialog. This is tedious, time-consuming, and error-prone. Did I mention we have 12 languages?

A better way would be to have a utility that quickly parses a resource file and extracts the dialog names and compares that list to all the other languages. It could even by expanded to do a control-by-control comparison to make sure all of those were present.

At some point, of course, the dialogs have to be visually inspected for spacing issues in other languages, but for an automated check, it can find the most common errors pretty quickly.

I am a huge fan of automating processes like this. There’s no reason to waste brainpower on highly-repetitive tasks. Even if it takes me a whole day to write a tool, it’s usually worth it.

Examples of automated processes:

  • Build process – if you’re building and packaging your product manually, you’re doing it wrong. All modern environments include command-line, scriptable components. When we build our flagship product, I type “make” and ten minutes later there’s a setup.exe in a distribution folder for our team.
  • Unit testing – Unit testing suites are usually automated to some degree (you hit a button, all tests run), but it can be taken further by running them during builds or as part of a continuous integration server.
  • Documentation/Change-logs – whenever we produce a new build, we send out an e-mail to internal staff about the changes. These are usually culled from the check-in comments in subversion. It’s a manual process. It might be nice to automatically dump them to a file, which can then be edited instead of written from scratch.
  • Code checks – this can encompass almost anything, but having static analysis tools is invaluable. Like the utility that I mentioned earlier to compare resource files across all the languages we support, it can save tons of manual, “stupid” labor.
  • Loading source onto a new machine – How long does it take to get up and running on a new machine? Admittedly, you shouldn’t be switching machines all THAT often, but when you do how easy is it to grab the source and start debugging? Are all the required libraries and tools in source control and automatically configured by build scripts?
  • E-mail – If you’re like me, you get scores if not hundreds of e-mails a day. How are you organizing, sorting, responding, ignoring, deleting them? Setup filters to put them into different folders, highlight or tag them when certain keywords appear. Also, get Google Desktop Search or Windows Desktop Search. I like both of them, but I’m currently using Google’s version. I may switch back in a while.
  • Bug reporting – While not strictly about automation, I think it’s close enough. Reporting bugs and code changes in a text file is good for a while–if you’re the only one working, and you have a small number of them to deal with. Once you start involving more programmers, and perhaps a manager who wants to see some basic reports, the text file doesn’t cut it. Get a simple bug reporting tool. I use BugTracker.Net because it’s easy, simple and does exactly what we need with minimum fuss. How do I know what to work on? I open up a web page and it tells me. I’ve automated not only some manual labor, but also some needless thought processes.
  • Calendaring – Do you need to write a weekly report for your manager? Keep track of employee’s vacation schedules? Use Outlook’s (or whatever PIM you choose) task list and calendar for anything you need to remember about a specific date. Set reminders for when you need to think about them, and then forget about them.
  • Data production – if you’re in a production environment generating data that needs to be analyzed, create tools to do as much of it as possible. Of course, the tools need to be checked for correctness, but once you’re confident, do it and don’t look back.

There are many, many ways you can optimize, reduce, and automated the work you’re doing. Remember, the whole point is to get rid of the “dumb” work and let yourself concentrate on the important, creative things.

Technorati Tags: , , , , , , ,

Thoughts on Process: How much is enough

I’ve been thinking a lot about process lately, specifically how much process is enough, and how to automate things I do regularly. First of all, I think I’m lucky to work in a place that does not have a burdensome process in order to get anything done.

At its core, process is necessary because we as humans make mistakes quite easily. A process is a set of well-known, written guidelines to follow when doing a task.

I say written, though quite a few processes are mental, because if a process isn’t formalized, it’s more “This-is-how-we-do-things” than a real process. Process has to be repeatable by not only yourself, but by others.

I’m not the kind of person who really seeks after process formalization. I avoid paperwork like the plague, but even I admit that at some point you need to formalize at least something. The real question is how much process is enough. The answer is: exactly as much as you need and no more. The problem is, that amount is very difficult to find. I almost always err on the side of not adding overhead to anything until I see a clear need.

Process should not be added unless there is a clear and quantifiable benefit to the project. And I think the people who carry about the process should be the ones to judge if there is a benefit. Of course, there are exceptions, but I think for standard software projects, less is definitely more.

As an example, suppose I need bug tracking software. I find one that I like, and install it and we use it — it’s perfect for our situation. Along comes somebody else and insists that we switch to the corporate policy, even though my group does nothing like the rest of the company. It’s also a larger, more complicated program, and my group loses control of it. (If this sounds like a real situation, it almost is).

When confronted with this hypothetical, you should be asking yourselves, but what do I benefit? What does my team benefit? What does the product benefit? I do NOT ask what does the company benefit–why does that matter? If it doesn’t help our product and makes our jobs slightly more difficult, then the detriment is worse than any supposed benefit to the company as a whole. I’m sure you could argue this example both ways (for example, if support is an issue, maybe the company is right), but the point is to avoid processes and standards for the sake of themselves.

Here’s the key: Another word for process should be simplification. Having a process should simplify life, even if the actual steps are complicated. It’s the fact they’re written down, or we have automated tools, or that’s it formalized into a printed sheet of paper on a binder at our desk that simplifies it–it takes away our ability to inject mistakes. Does this make us mindless drones? For this process, yes!, but the point is that we shouldn’t be thinking about this anyway! Our minds should be hard at work creating something NEW. The process exists to make sure things go exactly as they need to, for the express purpose of freeing our minds to work on more interesting things.

A good process that I recently had to formalize was our final release build of our flagship product. After too many false starts, we realized that there were too many people, too many resources from various sources, and it was impossible to coordinate it all in someone’s mind. We needed a simple process to be signed off on. So a written checklist was created. We won’t use this for every build–just those that go to the customer. It’s simple, it works, it lets us worry about things worth worrying about.

I’ll talk about automation in another post…

Technorati Tags: , , ,

What I’m doing to become a better developer

There was a meme going around about 5 steps people will take to become a better developer. I’m not a famous enough blogger to get tagged, but I’ll share my 2 cents anyway. There’s nothing like publicizing something to make you committed: 

  • Read more source code from other projects (especially .Net projects)

Now that I’ve started my own large, non-trivial application, more problems quickly become apparent–how do you structure an application, handle communication? The overall infrastructure becomes vitally important. Unlike DocView in MFC, .Net does not have a standard MVC-style architecture to build on top of.

It makes no sense to iterate over and over to come up with patterns that others have already solved well.

Projects I want to look at: Paint.Net, RSS Bandit, SharpDevelop, Rotor, Mono

  • Become a refactoring fiend

It takes a lot of work, but refactoring code whenever you can/should really pays dividends. I’m doing this in my current personal project, and even though it’s taken a few hours to do some of the major refactoring, it was worth it because implementing new features becomes nearly painless once the structure of the code allowed it. It’s a little harder to do at work, but I have done a little. I want to take this to the next level by always taking the time to restructure the existing code whenever the opportunity presents itself. Thankfully, I’ve spent a lot of time developing unit tests. This will help me use design patterns more thoughtfully, as well as become familiar with some of the more esoteric refactoring patterns in Martin Fowler’s Refactoring.

There are too many good books to name, but those are two of the best ones. I go through books quickly. Some books are worth reading over and over until the concepts become part of you. Of course, for that to happen, you have to actually implement the ideas into your projects. I definitely learned things in Code Complete my first time through that I have made part of me–so much so, that I probably couldn’t tell you what they were. But I know I could do better.

  • Exercise

Huh? That’s not related to programming! au contraire! Self-improvement is a lot more than learning more about your field–it’s training your body and mind to be in better shape. If I feel better, I think better. Now that I have my iPod nano and all the podcasts I could want, I have no excuse not to exercise more than I do. I’ve been getting better, but I have a ways to improve. A healthy body directly correlates to a healthy mind.

  • Build more original Lego models

Another one that isn’t really about programming?! Not so fast. I’ve built Legos for a while, but rarely have designed and built my own models. That needs to change. I have 12,000 Legos–I need to be exercising creativity, design processes, and hard work to achieve some original results. And I want to have fun.

Some other things I want to do:

These are other things I’m interested in trying out this year, but don’t have specific goals in mind yet:

  • Look into WPF — maybe good for UI for my BrickBuilder project?
  • learn more about COM. This has been slowly fading on the priority list. Is there a good reason to learn about COM these days, other than legacy support?
  • Continuous integration — I want to know more about it, but not sure how it would fit into our scenario at work. Once we get our staging server up, it might be a better fit.

UnitTest++

I’ve got a great process using NUnit in all my .Net projects. However, at work we have a large MFC/C++ application that has NO automated testing. I’ve been planing on getting unit testing into it for quite a while, but finally decided this is the week to do it.

It turns out that C++ unit testing is nowhere near as standardized or easy as .Net or Java. Partly, it’s the lack of reflection/introspection, but I think it’s also because the language itself just doesn’t lend itself to it.

There are a few very good resources for C++ unit testing. I first came across Noel Llopis’ Exploring the C++ Unit Testing Framework Jungle. I was just about set on using CxxTest, just as he had decided, but realizing that the post was written at the end of 2004, I continued my research. Sure, something must have improved since then! I came across UnitTest++: The New choice for C++ Unit Testing. It turns out to be partially written by Noel Llopis, so I thought this must be good since he gave such good reviews of the earlier solutions. You can read more about it here.

UnitTest++ seems to be the best of all possible worlds. It’s extremely simple to add test cases. I won’t include examples here, because the previous links have plenty.

The only thing I wish I could find is an open source code coverage C++ tool. I couldn’t survive without NUnit/NCover in C#.

Deep Computing Philosophy from Steve Yegge

If you haven’t read Steve Yegge, you owe it to yourself to do so. He only writes about once a month, but every single article is worth reading, whether you agree with him on everything or not. His latest is fascinating and incites some interesting pondering about the future of software…I’m going to have to think about a lot of it…

Why you shouldn’t catch System.Exception

There are probably many reasons, but my favorite is that, if you are resorting to catching anything at all, you probably don’t understand the code well.

For example, in complicated parsing of text, it is often easier to just put a huge try {…} catch (Exception ex) {…} around the entire thing, rather than take the time necessary to understand how your code works, and where exactly things could go wrong.

It is much better to go through the code line-by-line and prove to yourself that that there are only a few places of possible unknown exception behavior. Obviously, exceptions are meant to catch things you can’t reliably predict, but in most code, there are large areas of things like string manipulation, arithmetic, or other simple procedures, that will not throw an exception. It makes sense to delineate these areas, and surround only the trouble spots with try-catch.

For example, look at this fragment:

string strDelimiters = ":, ;"; 
try 
{ 
    string[] tempStrings = message.Split(strDelimiters.ToCharArray()); 
    string[] subStrings = new string[tempStrings.Length]; 
    int numSubStrings = 0; 
    //remove empty substrings 
    for (int i=0;i<tempStrings.Length;i++) 
    { 
        if (tempStrings[i] != null && tempStrings[i].Length > 0) 
        { 
            subStrings[numSubStrings] = tempStrings[i]; 
            numSubStrings++; 
        } 
    } 
...

 There is no reason to surround this with a try {} catch{} block. Doing so lets you off the hook of digging into this and realizing how bad it is, especially with .Net 2.0. 🙂

Simple Command Mapping Infrastructure in .Net

Unlike MFC, .Net does not offer a built-in framework for message handling in WinForm applications. MFC developers, for better or worse, have a rather large mechanism that automatically maps command messages from menus, toolbars, and windows into the classes that want them.

.Net has no such all-encompassing framework, though some 3rd-party frameworks exist (perhaps I’ll go over some in a a future entry).

The lack of a built-in application framework gives developers more freedom by not locking them into a certain mind set, but it also means that you have to develop your own framework or do everything from scratch in every application.

Short of developing an entire framework, there are some practices you can follow in .Net (and any GUI application, for that matter) to ease management of commands.

I’ll talk about one possibility that I recently chose to implement in a small WinForms application. First of all, let’s look at the overall architecture I’m aiming for:

It’s roughly a Model-View-Controller (MVC) architecture. What I want to do is almost completely isolate the GUI layer from knowledge of how the program actually works. The breakdown of responsibility is as follows:

  1. GUI – contains menus, status bar, views, panels, toolbars, etc. and routes messages from them to the controller via command objects.
  2. Controller – coordinates views, reports, and GUI.
  3. ReportGenerator – the “model” in this architecture. Generates and stores the actual data that will be presented.
  4. Views – Does the drawing for the report in different ways.

With this architecture, the GUI is restricted to doing only a few things, all of them related to the state of the GUI:

  1. Receiving events from toolbar buttons, etc.
  2. Setting menu/button states based on general program state (i.e., report running, idle)
  3. Changing the current view based on a request from controller.
Command Objects

A popular way to handle commands from toolbars and menus is to use the Command design pattern. The advantages to this pattern are numerous, but as far as isolating the GUI from implementation details, they allow us to put all the work required to get things done into command objects and the controller.

Here’s a sample design for commands. As you can see, it’s exceedingly simple.

An interface defines the basic functionality: Execute(). The concrete base classes are created with parameters that tell it the information it needs to know. When the user selects a command in the GUI, the GUI must then run the associated command.

For example, here’s a command that merely changes the current view:

class ChangeViewCommand : ICommand
{
private ViewType viewType;

public ChangeViewCommand(ViewType view)
{
this.viewType = view;
}

#region ICommand Members

public void Execute()
{
DiskViewController.Controller.SetCurrentView(viewType);
}

#endregion
}

When the command is created, it is initialized with the view type. When it’s executed, it merely calls the controller to do the work. The amount of work that is done in the controller versus command objects really depends on the application and individual commands. In this specific case, it might not seem like it’s doing a lot, but it would be easy to add Undo support, logging, coordination among multiple classes, or any other requirement for a command. In addition, the benefit of using command objects even in simple cases like this will become clear shortly.

Associating GUI objects with Command objects

Many applications use menus as well as toolbars with duplicate functionality. There are also keyboard shortcuts that can invoke the same command. An easy way in .Net to associate the GUI with command objects is to use a hash table. The keys consist of the menu and toolbar objects, and the values are the command objects.

//declaration:
private Dictionary<object, ICommand> commandMap = new Dictionary<object, ICommand>();

//toolbar
commandMap[toolStripButtonPie] = new ChangeViewCommand(ViewType.Pie);
commandMap[toolStripButton3DPie] = new ChangeViewCommand(ViewType.Pie3D);
commandMap[toolStripButtonStart] = new StartCommand(controller);
commandMap[toolStripButtonStop] = new StopCommand(controller);

//menu
commandMap[pieToolStripMenuItem] = commandMap[toolStripButtonPie];
commandMap[Pie3DToolStripMenuItem] = commandMap[toolStripButton3DPie];
commandMap[startAnalysisDriveToolStripMenuItem] = commandMap[toolStripButtonStart];
commandMap[stopAnalysisToolStripMenuItem] = commandMap[toolStripButtonStop];

 You can see that some commands take the controller as argument–an option to discovering it from a static class variable. Also notice that commands from menu objects can reuse the same object that is associated with the toolbar button.

In order to run the commands, the GUI needs only respond to OnClick events from menus and toolbars and in many cases, the same delegate can be assigned to many of the menu items:

private void OnCommand(object sender, EventArgs e)
{
    if (commandMap.ContainsKey(sender))
    {
        ICommand cmd = commandMap[sender];
        if (cmd != null)
        {
              cmd.Execute();
              //could also do controller.RunCommand(cmd);
        }
    }
    else
    {
        throw new NotImplementedException();
    }
}

In conclusion, using the techniques presented here you can easily split your program into manageable chunks, with each layer doing only what it needs to.