Tag Archives: unit testing

Adapting to Changes

Change is a fact of life. Nowhere is this more obvious in a medium where the very thing we make is completely intangible and malleable: software development. There is almost nothing that is impossible in software–there are only limited resources.

This malleability has been one of the reasons for the enormous pace of change we’ve seen in computers in the last half century, but at the same time it’s been a stumbling block to true engineering practices.

Regardless of philosophy, change will happen in all software projects to some degree. There are two extremes of development that most organizations fall between:

  • A single coder hacks together whatever the boss needs changed today. (Also fixes bugs from yesterday’s fixes.)
  • NASA’s awesomely rigid development practices for the Space Shuttle software (see the Manager’s Handbook for Software Development on the Book list) resulting in extremely high-quality, virtually bug-free software (at a very high price).

For most of us, neither extreme is appropriate. Deciding the right balance of change control is a hard problem, but it’s not impossible. Still, some organizations manage to do so badly at it that it deserves to be studied and dissected.

The Census

As an example, consider the 2010 Census, a project that has been covered in the press and on tech blogs over the last few months. The following is from the second paragraph of the report on the GAO’s study of this massive program:

In October 2007, GAO reported that changes to requirements had been a contributing factor to both cost increases and schedule delays experienced by the FDCA [Field Data Collection Automation] program. Increases in the number of requirements led to the need for additional work and staffing. In addition, an initial underestimate of the contract costs caused both cost and schedule revisions. In response to the cost and schedule changes, the Bureau decided to delay certain system functionality, which increased the likelihood that the systems testing…would not be as comprehensive as planned.[…] Without effective management of these and other key risks, the FDCA program faced an increased probability that the system would not be delivered on schedule and within budget or perform as expected. Accordingly, GAO recommended that the FDCA project team strengthen its risk management activities, including risk identification and oversight.

The Bureau has recently made efforts to further define the requirements for the FDCA program, and it has estimated that the revised requirements will result in significant cost increases.

As with many government projects, there is enough failure to go around. The key idea I want to focus on is a failure on the part of the government agency and on the part of the contractor implementing the technology to adequately plan for change.

Before that, though, I want to say that I have no knowledge of their specific management or technical practices, other than what is publicly reported. For all I know, all parties performed extremely well and still failed. (However, from friends and acquaintances in government and contractor positions around D.C., I think it’s fair to say that most organizations here are behind the curve in software best practices, to put it politely.)

With that (admittedly harsh) assumption, it’s apparent that the agency failed to adequately plan, think through requirements, and communicate effectively to the contracted company. On the developer’s side, it’s apparent they didn’t have effective mechanisms for handling changes.

Before a project is undertaken, a set of fundamental principles must be agreed upon:

  1. The client will change their minds. There is no such thing as “written in stone.”
  2. Making changes is hard or impossible.

These two principles are fundamentally in conflict. On the one hand, the developer must know that changes are going to occur: sometimes large changes. They can plan for that from the code level up to their organization and processes.

On the other hand, the client, must realize that they can’t just suggest a change at the end of the project and expect it to be thrown in and work correctly.

The tension between these two principles implies certain practices and expectations on all parties, which in an ideal world would lead to a successful outcome more often.

Developer Responsibilities

The developer, knowing that the client does not understand the full set of requirements up front, designs their system in a way that can easily handle change. This means modularity, this means testability. From details as small as designing methods to operate on Streams instead of filenames, to as large as easily reusable components.

The first step is a change in mind set: an acceptance that change will occur, and that it’s ok. When I first started development, frequent changes really frustrated me. Sometimes they still do, but I’m getting better (I hope). Once the mind is malleable, practices can be implemented to handle it effectively.

A few coding practices that encourage changes:

  • Test-driven development (or at least full unit testing). In this day and age, with how easy it is to accomplish this, is there a good reason not to do it? I’m not convinced by many arguments against unit testing. A full suite of unit tests gives you confidence in your code, and the ability to safely refactor even large portions of your program. Full TDD I could go either way.
  • Constant refactoring to fit the best design possible (within reason. Obviously, there is no “best” design). Software, like everything else in the universe, obeys the law of entropy. Without constant maintenance, it will degrade. Designs degrade as you add on to them. Classes degrade as you stuff them full of things. Don’t stand for it–take the time to make things neat. Get some good books on refactoring if you don’t know how to start.
  • Highly Modular. Apply all your OO-design skills here. It matters. Cohesion and coupling really do matter.
  • Small, frequent iterations. Most software systems these days are too big and too complex to understand at once. Iterative development is the key to success.
  • Strong source-control practices. Does this even need to be discussed? If your organization doesn’t have good source control, you’re doing software wrong.

Other practices on the developer’s part read like a list of Extreme programming fundamentals:

  • Communication – The key to any relationship, whether personal or corporate, is communication. The right amount should be discovered and adhered to. It should probably be slightly more than you’re comfortable with (since many developers would rather not talk to anyone ;).
  • Simplicity – I believe that the single hardest thing to do in developing software is to manage complexity. It’s requires more brainpower and creativity than all other activities. Coding is easy. Coding so that it is easy to understand a year from now is difficult. If the design is needlessly complex up front, then change is that much harder to implement.  Always remember that software complexity increases exponentially.
  • Courage – Stand up for correct principles. Don’t allow politics to interfere with what needs to be done. Don’t be afraid to change the design if it needs to happen. Change coding practices as needed. Never be afraid of the truth–just deal with it.
  • Respect – Both sides need to remember that the other is expert in their domain. Humility is key.
  • Responsibility – this is something I fear is lacking much in government. Everybody needs to take responsibility for their part. This implies accountability. This can be shared responsibility at some levels, or the-buck-stops-here responsibility at higher levels. If no one is responsible, nobody cares if it fails. Courage is a prerequisite to this.

This mind set needs to encompass not just pure software development, but also processes at the organizational level. This can take many forms, but a few ideas:

  • Personal relationships between clients and developers
  • Flexible team structure and members
  • An easy way to submit and discuss change requests
  • A culture that accepts changes
  • A good understanding of the problem domain

Jeremy Miller (The Shade Tree Developer) talks a lot about processes and practices and it is a very good read.

Unfortunately, there exists an attitude at many companies that it’s better to milk the clients for as much money as possible rather than do the hard work of getting a good process.

Client Responsibilities

I think my introduction and the discussion about developer responsibilities may imply that because software is so ephemeral, it is therefore easy to change it. After all, it’s not a physical object that has to be broken down and rebuilt–what’s so hard? To anyone who has worked on non-trivial projects, this is laughable. Let’s clarify:

It is easy to program. It’s painfully, mind-bendingly, insanely difficult to design software. So difficult, in fact, that no one can do it well. The sooner everybody understand that, the sooner we can get started with real work. Good design is thought-work, iterated over and over, added to experience and analysis.

Some details are easy to change. Others are fundamental to the structure of the building. By way of metaphor, changing your mind about a house you’re building to suggest a stone facade instead of brick is relatively easy. Deciding you would like to have ten stories added onto your house changes the game significantly. Now you have to build a completely new foundation.

I believe the customer’s responsibility is to gain knowledge and insight into the development practice so they understand why things are difficult. Computers are so fundamental to our culture, we can no longer afford to have companies and agencies run by people who don’t understand them to some degree. The exact degree is debatable, but it’s certainly higher than where we’re at now. In my own situation, I think that when I started my boss did not realize how seemingly-trivial changes could be phenomenally difficult to implement, and therefore bug-prone–all because of design decisions made earlier. I now have him understanding that there is no such thing as a simple change.  (Of course, I still occasionally underpromise and overdeliver–I have to maintain my miracle-worker image after all. 🙂

A client who wants software done and doesn’t want to end up paying an extra $3,000,000,000 for software should understand that critical requirements can’t be left until the product is being delivered. “We didn’t think of it before” is not a good excuse.

A client who understands this will realize how critical initial requirements are and ensure they’re communicated clearly. Most of all, they don’t wait around twirling their thumbs, only to reject the system when it’s done because it doesn’t meet their requirements. They will insist on periodic feedback and demos to make sure things are going the right way.

The responsibility for huge failures is on everybody’s shoulders. I believe that it can often be traced back to an unwillingness to face up to the truth, lack of responsibility, and misunderstanding the nature of the project.

Large projects aren’t doomed to fail

There is a fundamental truth, however, of all projects: there exists at least one change that is too large to do. Maybe the Census required that change–I don’t know. I do know that if both sides follow the best practices they can, the number of must-fail changes can be minimized.

That said, here’s why I refuse to believe the census project wasn’t sorely mismanaged from the outset by all sides: FedEx, UPS, Amazon, Wal-Mart, Microsoft, and dozens (hundreds?) of other companies have built enormous and complex systems for managing large databases and mobile platforms involving thousands of partners. It is ridiculous to me that they could not have built a system for our government quicker, cheaper, and less epic failure than the one they got. If the Census is fundamentally more complex than the problems they deal with, someone please let me know.

Change is part of everything now

The ability to adapt to change is critical, not just for software developers, but for everyone in our society. Technology, events, trends, and money all flow so quickly now that those who can adapt quickly will succeed.

It’s not just software companies that need to adopt sound development practices. There will always be companies that build out our infrastructure (Microsoft, Google, Cisco, etc.), but in the end every company will be a software development company. Disregarding good practices because “we’re not a software” does not work anymore. It’s not true. You are a software company. We all will be.

Idealism or…

If a lot of this seems idealistic, that’s ok. I wish the world were like Star Trek as much as the next geek, but I realize that there are a lot of motivation$ out there. The world is what it is, but if we aren’t trying to improve it by working towards a standard, then what’s the point? We can improve it. There is no excuse for such catastrophic wastes of time, money, and effort.

Review: Pragmatic Unit Testing in C# with NUnit, 2nd Ed.

I saw this book when I bought Programming WPF a few weeks ago and it looked promising enough to buy. I’ve been doing unit testing in C# for a few years now, but I thought there were always things to learn and maybe I’d pick up a few new ideas.

It is easy to contrast this book with Beck’s Test Driven Development: By Example, and the two books definitely have a very different feel.

Beck’s book has a very evangelical feel to it, and it’s main purpose is to teach a mind set more than technical details. I believe this is important–maybe of first importance–but once you understand that, the rest of the book is a little simplistic for more experienced developers.

Pragmatic Unit Testing, on the other hand, focuses much more on the practical aspects (hmmm….I highly suspect that’s where the title comes from….) of unit testing. I liked the ideas on how to use categories and attributes to segregate tests that take too long to run on a regular basis. I also liked the section on singletons and getting around time-dependencies. The DateTime.Now problem is something I’ve had to deal with quite a bit in our server-side software that has a lot of time-dependant behavior. (In most cases, the problems were solved with refactoring the time into a function parameter.)

There are also good discussions of more mundane issues like how to deploy NUnit, where to put tests in a project, team practices, GUIs, threading, and C#-specific issues.

The discussion about mock objects (a very basic introduction) is also quite clear and understandable–more so than many resources I’ve seen on the web, which often assume you already know all about them.

Something I don’t like: the acronyms (BICEP, CORRECT, A TRIP). They kind of bug me. I like the ideas behind the acronyms and I think it’s more important and effective (for me, anyway) to internalize the principles of testing rather than remembering specific acronyms and the words they go with. YMMV.

Last Word…

I will probably only read Test Drive Development: By Example once,  but I will definitely come back to Pragmatic Unit Testing occasionally to refresh my ideas.

6 Ways to Increase Your Confidence As You Code

One of the key requirements for being able to reliably update software is the confidence that the changes you are making are safe. The amount of confidence required increases with the complexity of the system.

In my day job I work on a real-time messaging system that can have very, very little downtime. As the service grows and sees more traffic, the amount of downtime shrinks. We start to worry now if upgrades take longer than 5 minutes. (It’s almost to the point where we’ll need redundant systems in order to do maintenance).

To upgrade this software, I have to have an awful lot of confidence in the code changes made. Sometimes that confidence varies.

What to do to gain confidence:

1. Consistent process

Having a set of rules to follow is a fundamental requirement of good software engineering. I’m not going to discuss what the process should be, but you should have one that works well.

Why is this important?

Programmers like order. We like well-defined problems where we can see the end from the beginning. We don’t like haziness, indeterminism, or too many choices.

A process nails down the unknowns–it tells you very specifically what the next thing to do is. A good process leaves no room for doubt.

A good development, testing, and deployment process is the first step to building confidence in what you’re doing.

For my messaging system example, here’s a short summary of what our upgrade process is. We didn’t just come up with it–it evolved as our business grew and the requirements grew with it.

  1. All unit and integration tests pass (we have about 2200 automated tests for this that can run in about 4 minutes)
  2. We’ve run it on staging server for a few weeks with no issues.
  3. All new features have been tested on the staging server
  4. Formal change document has been submitted and approved.
  5. Formal test results documented.
  6. Previous version backed up.
  7. Perform upgrade
  8. Monitor system (including custom automated monitoring tools)

At any point, I know where we are in the process and what needs to be done next. Sure, there may be details within these steps that require thought and creativity, but the process guides it all and makes us more confident that we’re not performing ad-hoc operations.

2. Unit Testing

There are other types of testing, but it all starts at the unit level, with simple tests that exercise your code line by line, function by function, feature by feature. I recently wrote a few thoughts about unit testing. Unit testing is where you can see the overall wellness of your code–you want that green bar!

Without unit testing, how do you know the code you’re writing is doing what it should? do you just run it and push it through its paces? This is highly inefficient for most types of code. You’ll run out of steam before you start getting close to edge cases.

The fact is that automated unit tests are a baseline for confidence in your code. You need to be able demonstrate time and again that your code performs well.

This all presupposes that you are writing good unit tests. If you’re not sure, start studying. I don’t buy the arguments about lulling developers into a false sense of security–sure, that can happen, but having good developers who understand this is a prerequisite.

If you’re not unit testing–what is your basis for confidence in your code?

3. Code Coverage

Code coverage goes hand-in-hand with unit testing as a good way to automatically discover what areas of your program are in need of more testing. I’ve found that one of the biggest barriers to unit testing a large C++ application we have is that we have no way of easily measuring test coverage. If we had time, we could definitely to the analysis ourselves, or we could spend a lot of money to get a C++ instrumentation profiler, but these are slow and very tedious to use in my experience.

In .Net, use the tools to your advantage.

The psychological benefits of seeing 75-, 90-, 95-, even 100-percent coverage are immense. You know that every line of the program has at least been touched.

Of course, most code coverage tools analyze line coverage, not path coverage. Combine  complexity analysis with code coverage to determine which functionality should probably have better testing. There are plenty of free and commercial tools that will give you cyclomatic complexity, among other metrics.

Use other analysis tools like FxCop to make sure your other ducks are in a row. It can find easy-to-overlook problems like not validating arguments of public methods, which can then lead to more unit tests and more coverage to achieve.

4. Automation

Take yourself out of the equation as much as possible. The point of a process is to be repeatable–it’s like automating yourself. Not only should unit testing be automated (thankfully, most testing frameworks handle this easily), but so should coverage and quality analyses.

What about deployment? Automate it. Documentation generation? CD master creation? Web upload? E-mail notification? Automate them all. Production builds should be invoked with a single command.

Working on boring, repeatable code? Automate it with code-gen.

The bottom line is: Don’t waste your brain cells on stuff that is highly repeatable, especially when it is prone to mistakes.

5. Code Review

Last week, a rather serious bug was discovered in some of our software (not released yet, thankfully, but close). The bug was mine, and I knew exactly what the problem was, but instead of designing a solution by myself, I brought a co-worker into the discussion just to bounce ideas off of. He had great suggestions, and made me think of things I might not necessarily have thought of on my own. We both went over the code and came to a solution that was simple and acceptable to both of us. The confidence level was much higher with this than it would have been otherwise.

This story is repeated daily by programmers throughout the world. Code review is a practice based on the simple notion that there is no one person smart enough to get it correct the first time.

Even if you’re working alone, which I often do, it pays huge dividends to regularly review your code with an eye for finding trouble. If you see any weakness at all, don’t ignore it–fix it. If you’re reviewing your own code, it’s a good idea to wait a bit after the time you wrote it. This gives your brain a chance to forget a little bit about it. Then, if you find you can’t understand it anymore, it’s either too complicated, or (if it fundamentally really is complicated) you need better comments.

Reviewing with other people has more benefit, however. Not everybody thinks the same way about problems. People have different experience, different expertise and focus, and you can’t take advantage of that if you don’t let them teach you. Even if the other people have less expertise than you, it is still beneficial (assuming they have some basic competency that they can bring to the discussion).

Once you let other people tear into your code (nicely, I hope), your confidence can be higher because you can add the confidence other people have in it (once your problems are corrected, of course!)

6. Repeatable Experiences

In the end, one of the best ways to increase your confidence in yourself, your code, and your practices is to have the evidence of repeated experiences behind you. You’re always learning, and that learning contributes to improvements in processes, testing, and your personal coding practices. Once you learn what works, especially during tricky upgrades, you can go into the next trial with increased confidence that you’re doing something right.

Have any other ideas on increasing confidence? Leave them in the comments!

Technorati Tags: ,,

Unit testing benefits programmers who are already good

In order to kick my unit testing skills up a notch, I’ve been reading a lot about it lately. Today I had the thought: “Unit testing only helps already-good programmers.”

My reasoning is that bad programmers are going to write bad tests, or not enough test cases, or bad test cases, or won’t take the effort to refactor their code when necessary, or won’t realize their code reeks, and so on and so forth.

The message from unit testing, and the XP camp in general, is that well-factored, object-oriented, testable code is key. But don’t those criteria presuppose some fairly intense skills on the part of the programmers? Merely introducing unit-testing into your processes won’t automatically improve the quality of code if someone has no clue.

This is a depressing line of thought. It’s not entirely true, however.

It IS possible that enforcing a rule on unit tests will “inspire” inexperienced developers to improve their code. Once they see the beauty of automated unit testing, hopefully some will realize that their code was NOT testable, NOT pretty, NOT well-factored, and start taking steps to change that. The knowledge to do that (why and how), however, will have to come from somewhere else. And since they’re not a good programmer, will they do this?

For these people, then, unit testing is of little benefit–chances are the code is of such low quality that the tests will just conform instead of try to break it.

You can’t teach someone the vision of unit testing without teaching the vision of a lot of other things as well. It doesn’t make sense otherwise.

A programmer who already understands how to build well-factored code is going to use unit tests in an entirely different way than someone who doesn’t understand them. To these people, it’s a way of verifying that it works to spec, and that it’s safe to change the implementation details without destroying the system.

I got into unit testing because I’m a good developer. It didn’t make me a good developer–it made me a better developer. (I said good developer, not great. I’m good because I realize that I always need to improve, and I take steps to do so, not because I’m a genius.)

That last paragraph and parentheses deserves more attention. How do you define a good programmer? Are they innately good, or are they good because they do certain things? Do you unit test because you’re a good programmer, or does the act of unit testing make you a good programmer? Is there a paradox here? Are both true? Neither?

Obviously, there aren’t just two kinds of programmers: 1) good and 2) bad. There is a spectrum. Obviously, again, just doing something doesn’t mean you’re automatically good or better, either. So I think you have to unit test because you’re a good programmer (or someone is forcing you, which is a different topic altogether) already and not the other way around.

I believe being a good programmer must come from within you. Becoming a better programmer can be done with the help of education, tools, and processes.

(Unit testing is just the topic I’m thinking about lately–you could replace it with any practice and the ideas are still the same.)

Technorati Tags: ,,

Top 5 Attributes of Highly Effective Programmers

What attributes can contribute to a highly successful software developer versus the ordinary run-of-the-mill kind? I don’t believe the attributes listed here are the end-all, be-all list, nor do I believe you have to be born with them. Nearly all things in life can be learned, and these attributes are no exception.

Like this article? Check out my new book, Writing High-Performance .NET Code.

Humility

Humility is first because it implies all the other attributes, or at least enables them. There are a lot of misunderstandings of what humility is and sometimes it’s easier to explain it by describing what humility isn’t:

  • humility isn’t letting people walk all over you
  • humility isn’t suppressing your opinions
  • humility isn’t thinking you’re a crappy programmer

C.S. Lewis said it best, in the literary guise of a devil trying to subvert humanity:

Let him think of [humility] not as self-forgetfulness but as a certain kind of opinion (namely, a low opinion) of his own talents and character. . . Fix in his mind the idea that humility consists in trying to believe those talents to be less valuable than he believes them to be. . . By this method thousands of humans have been brought to think that humility means pretty women trying to believe they are ugly and clever men trying to believe they are fools.*

Ok, so we realize humility isn’t pretending to be worse than you are and it’s not timidity. So what is it?

Simply put, humility is an understanding that the world doesn’t begin and end with you. It’s accepting that you don’t know everything there is to know about WPF, or Perl, or Linux. It’s an acknowledgment of the fact that, even if you’re an expert in some particular area, there is still much to learn. In fact, there is far more to learn than you could possibly do in a lifetime, and that’s ok.

Once you start assuming you’re the expert and final word on something, you’ve stopped growing, stopped learning, and stopped progressing. Pride can make you obsolete faster than you can say “Java”.

The fact is that even if you’re humble, you’re probably pretty smart. If you work in a small organization with few programmers (or any organization with few good programmers), chances are you’re more intelligent than the majority of them when it comes to computers. (If you are smarter than all of them about everything, then either you failed the humility test or you need to get out of that company fast). Since you happen to know more about computers and how software works than most people, and since everybody’s life increasingly revolves around using computers, this will give you the illusion that you are smarter than others about everything. This is usually a mistake.

Take Sales and Marketing for example. I have about 50 Dilbert strips hung in my office. I would guess half of them make fun of Sales or Marketing in some way. It’s easy, it’s fun, and it’s often richly deserved!

But if they didn’t sell your software, you wouldn’t get paid. You need them as much as they need you. If someone asked you to go sell your software, how would you do it? Do you even like talking to people? As clueless as they are about the realities of software development, they have skills you don’t.

There are some industries where extreme ego will get you places. I do not believe this to be the usual case in software, at least in companies run by people who understand software. Ego is not enough–results matter. If you have a big ego, you’d better be able to back it up. Unfortunately, the problem with egos is that they grow–eventually you won’t be able to keep up with it, and people will see through you.

The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague. – Dijkstra

Without humility, you will make mistakes. Actually, even with humility you’ll make mistakes, but you’ll realize it sooner. By assuming that you know how to solve a problem immediately, you may take steps to short-circuit the development process. You may think you understand the software so well that you can easily fix a bug with just a few tweaks…and yet, you didn’t realize that this other function over here is now broken. A humble programmer will first say “I don’t know the right way to solve this yet” and take the time to do the analysis.

Finally, humble people are a lot more pleasant to work with. They don’t make their superiority an issue. They don’t always have the “right answer” (meaning everybody else is wrong, of course). You can do pair programming with a humble person, you can do code reviews with a humble person, you can instruct a humble person.

Leave your ego out of programming.

It’s not at all important to get it right the first time. It’s vitally important to get it right the last time. – Andrew Hunt and David Thomas

Perhaps most importantly, a humble programmer can instruct him/herself…

Love of Learning

If you’re new to this whole programming thing, I hate to break it to you: school has just begun. Whatever you thought of your BS/MS in Computer Science you worked so hard at–it was just the beginning. It will get you your first job or two, but that’s it. If you can’t learn as you go, from now until you retire, you’re dead weight. Sure, you’ll be able to find a job working somewhere with your pet language for the rest of your life–COBOL and Fortran are still out there after all, but if you really want to progress you’re going to have to learn.

Learning is not compulsory. Neither is survival. – W. Edwards Deming

This means reading. A lot. If you don’t like reading, I suggest you start–get into Harry Potter, fantasy, science fiction, historical fiction, whatever. Something. Just read. Then get some technical books. Start with my list of essential developer books. They’re not as exciting as Harry Potter, but they’re not bad either.

A lot material is online, but for high quality, authoritative prose on fundamental subjects, a good book beats all.

Reading isn’t enough, though. You have to practice. You have to write your own test projects. You have to force yourself to push your boundaries. You can start by typing in code sample, but then you need to change them in ways unique to you. You should have personal projects and hobbies that expand your skills. Be writing your own tools, or “fun” programs. Write a game. Do what it takes to learn new things!

The type of programs you write have a big bearing on how well you learn new material. It has to be something that interests you, or you won’t keep it up. In my case, I’m developing software related to my LEGO hobby. In the past, I’ve written tools for word puzzles, system utilities, multimedia plug-ins, and more. They all started out of a need I had, or a desire to learn something new and useful to me personally.

Another aspect of this love of learning is related to debugging software. An effective developer is not satisfied with a problem until they understand how it works, why it happens, and the details of how to fix it. The details matter–understanding why a bug occurs is just as important as knowing what to do to fix it.

Learn from mistakes. I have seen programmers who make a mistake, have the correct solution pointed out to them, say, “huh, wonder why it didn’t work for me”, and go on their merry way, none-the-wiser. Once their code is working “the way it should” they’re done. They don’t care why or how–it works and that’s enough. It’s not enough! Understand the mistake,what fixed it, and why.

Good judgement comes from experience, and experience comes from bad judgement. – Fred Brooks

Obviously, some balance has to be struck here. You cannot learn everything–it simply isn’t possible. Our profession is becoming increasingly specialized because there is simply too much out there. I also think that in some respects, you need to love learning just for the sake of learning.

Detail-orientedness

Developing software with today’s technologies is all about the details. Maybe in 100 years, software will progress to the point where it can write itself, be fully component-pluggable, self-documenting, self-testing, and then…there won’t be any programmers. But until that comes along (if ever), get used to paying attention to a lot of details.

To illustrate: pick a feature of any software product, and try to think of all the work that would have to be done to change it in some way. For anything non-trivial, you could probably come up with a list of a hundred discrete tasks: modifying the UI (which includes graphics, text, localization, events, customization, etc.), unit tests, algorithms, interaction with related components, and on and on, each discrete step being broken into sub-steps.

I have always found that plans are useless, but planning is indispensable. – Dwight Eisenhower

Here’s a problem, though: few humans can keep every single task in their head, especially over time. Thankfully, detail-orientedness does not necessarily mean being able to mentally track each and every detail. It means that you develop a mental pattern to deal with them. For example, the steps of changing a piece of software could be:

  1. Thoroughly understand what the code is doing and why
  2. Look for any and all dependencies and interactions with this code
  3. Have a well-thought-out mental picture of how it fits together.
  4. Examine the consequences of changing the feature.
  5. Update all related code that needs to (and repeat this cycle for those components)
  6. Update auxiliary pieces that might depend on this code (build system, installer, tests, documentation, etc.)
  7. Test and repeat.

An example: I find that as I’m working on a chunk of code, I realize there are several things I need to do after I’m finished with my immediate task. If I don’t do them, the software will break. If I try to remember all of them, one will surely slip by the wayside. I have a few choices here:

  1. Defer until later, while trying to remember them all
  2. Do them immediately
  3. Defer until later, after writing  them down

Each of these might be useful in different circumstances. Well…maybe not #1. I think that’s doomed to failure from the start and creates bad habits. If the secondary tasks  are short, easy, and well-understood, just do them immediately and get back to your primary task.

However, if you know they’ll require a lot of work, write them down. I prefer a sturdy engineering notebookin nearly all cases, but text files, Outlook tasks, notes, OneNote, bug tracking systems, and other methods can all work together to enable this.

The more experience you have, the easier you’ll be able to track the details you need to worry about. You’ll also analyze them quicker, but you will always need some way of keeping track of what you need to do next. There are simply too many details. Effective organization is a key ability of any good software engineer.

Another aspect of paying attention to details is critical thinking. Critical thinking implies a healthy skepticism about everything you do. It is particularly important as you examine the details of your implementation, designs, or plans. It’s the ability to pull out of those details what is important, what is correct, or on the other hand, what is garbage and should be thrown out. It also guides when you should use well-known methods of development, and when you need to come up with a novel solution to a hard problem.

Adaptability

“Enjoying success requires the ability to adapt. Only by being open to change will you have a true opportunity to get the most from your talent.” – Nolan Ryan

Change happens. Get used to it. This is a hard one for me, to tell the truth. I really, really like having a plan and following it, adapting it to my needs, not those of others.

The fact is, in software development, the project you end up writing will not be the one you started. This can be frustrating if you don’t know how to handle it.

To become adaptive first requires a change in mind set. This mind set says that change is inevitable, it’s ok, and you’re ready for it. How do you become ready for it? This is a whole other topic in itself, and I will probably devote a separate essay for it.

Other than the shift in mind set, start using techniques and technologies that enable easy change. Things like unit testing, code coverage, and refactoring all enable easier modification of code.

In war as in life, it is often necessary when some cherished scheme has failed, to take up the best alternative open, and if so, it is folly not to work for it with all your might. – Winston Churchill

For me, the first step in changing my mind set is to not get frustrated every time things change (“But you specifically said we were NOT going to implement the feature to work this way!”).

Passion

I think passion is up there with humility in importance. It’s so fundamental, that without it, the others don’t matter.

Anyone can dabble, but once you’ve made that commitment, your blood has that particular thing in it, and it’s very hard for people to stop you. – Bill Cosby

It’s also the hardest to develop. I’m not sure if it’s innate or not. In my own case, I think my passion developed at a very early age. It’s been there as long as I remember, even if I had periods of not doing much with it.

I’ve interviewed dozens of prospective developers at my current job, and this is the one thing I see consistently lacking. So many of them are in it just for another job. If that’s all you want, just a job to pay the bills, so be it. (Of course, I have to ask, if that’s the case, why are you reading this article?)

One person with passion is better than forty people merely interested. – E. M. Forster

There’s a world of difference between someone who just programs and someone who loves to program. Someone who just programs will probably not be familiar with the latest tools, practices, techniques, or technologies making their way down the pipeline. They won’t think about programming outside of business hours. On the weekends, they do their best to forget about computers. They have no personal projects, no favorite technologies, no blogs they like to read, and no drive to excel. They have a hard time learning new things and can be a large burden on an effective development team.

Ok, that’s maybe a bit of exaggeration, but by listing the counterpoints, it’s easier to see symptoms of someone who does have passion:

  • Thinks and breaths technology
  • Reads blogs about programming
  • Reads books about programming
  • Writes a blog about programming
  • Has personal projects
  • These personal projects are more important than the boring stuff at work
  • Keeps up with latest technologies for their interests
  • Pushes for implementation of the latest technologies (not blindly, of course)
  • Goes deep in technical problems.
  • Not content with merely coding to spec.
  • Needs an outlet of creativity, whether it be professional (software design) or personal (music, model building, LEGO building, art, etc.)
  • Thinks of the world in terms of Star Trek

Just kidding on the last one…

…(maybe)

That’s my list. It’s taken a few months to write this, and I hope it’s genuinely useful to someone, especially new, young software engineers just getting started. This is a hard industry, but it should also be fun. Learning these attributes, changing your mind set, and consciously deciding to become the engineer and programmer you want to be are the first steps. And also part of every step thereafter.

Nobody is born with any of these–they are developed, practiced, and honed to perfection over a lifetime. There is no better time to start than now.

* The Screwtape Letters, C.S. Lewis

Easily Unit Testing Event Handlers

In C#, If you need to unit test a class that fires an event in certain circumstances (perhaps even asynchronously), you need to handle a little more than just running some code and doing the assertion. You have to make sure your unit test waits for the event to be fired. Here’s one naive way of doing it, a WRONG way:

   1: private bool statsUpdated = false;
   2: private ManualResetEvent statsUpdatedEvent = new ManualResetEvent(false);
   3:
   4: [Test]
   5: public void CheckStats()
   6: {
   7:     BrickDatabase db = new BrickDatabase(tempFolder, maxCacheAge);
   8:
   9:     statsUpdated = false;
  10:     statsUpdatedEvent.Reset();
  11:
  12:     db.InventoryStatsUpdated += new EventHandler(db_InventoryStatsUpdated);
  13:     db.DoSomethingThatFiresEvent();
  14:
  15:     statsUpdatedEvent.WaitOne();
  16:
  17:     Assert.IsTrue(statsUpdated);
  18: }
  19:
  20: void db_InventoryStatsUpdated(object sender, EventArgs e)
  21: {
  22:     statsUpdated = true;
  23:     statsUpdatedEvent.Set();
  24: }

There are a number of things wrong with this:

  1. The class variables. More complex unit test class. Have to coordinate these variables across multiple functions.
  2. Since they are class variables, you will want to reuse them, but you’d better remember to reset the event and the boolean every time!
  3. Have to have two functions to do something really, really simple.
  4. The WaitOne() does not have a timeout, so if the wait is ever satisfied then statsUpdated is guaranteed to be true.

Here’s a better way of doing it, using anonymous methods in C# 2.0:

   1: [Test]
   2: public void CheckStats()
   3: {
   4:     BrickDatabase db = new BrickDatabase(tempFolder, maxCacheAge);
   5:     bool statsUpdated = false;
   6:     ManualResetEvent statsUpdatedEvent = new ManualResetEvent(false);
   7:
   8:     db.InventoryStatsUpdated += delegate
   9:     {
  10:         statsUpdated = true;
  11:         statsUpdatedEvent.Set();
  12:     };
  13:
  14:     db.DoSomethingThatFiresEvent();
  15:
  16:     statsUpdatedEvent.WaitOne(5000,false);
  17:
  18:     Assert.IsTrue(statsUpdated);
  19: }

Improvements?

  1. The event is just part of the method. Since the event handler is an anonymous delegate, it can access the enclosing method’s local variables.
  2. Added 5,000ms timeout to the WaitOne() function to prevent hanging of unit tests.

Thoughts on Process: Automation (and examples)

If a process, or part of a process, can be automated it should be. For example, in a project at work, part of our process is to make sure that every dialog present in the English resources of an MFC application is also present in all the other dialogs. We do this manually by loading each language into the app and triggering every dialog. This is tedious, time-consuming, and error-prone. Did I mention we have 12 languages?

A better way would be to have a utility that quickly parses a resource file and extracts the dialog names and compares that list to all the other languages. It could even by expanded to do a control-by-control comparison to make sure all of those were present.

At some point, of course, the dialogs have to be visually inspected for spacing issues in other languages, but for an automated check, it can find the most common errors pretty quickly.

I am a huge fan of automating processes like this. There’s no reason to waste brainpower on highly-repetitive tasks. Even if it takes me a whole day to write a tool, it’s usually worth it.

Examples of automated processes:

  • Build process – if you’re building and packaging your product manually, you’re doing it wrong. All modern environments include command-line, scriptable components. When we build our flagship product, I type “make” and ten minutes later there’s a setup.exe in a distribution folder for our team.
  • Unit testing – Unit testing suites are usually automated to some degree (you hit a button, all tests run), but it can be taken further by running them during builds or as part of a continuous integration server.
  • Documentation/Change-logs – whenever we produce a new build, we send out an e-mail to internal staff about the changes. These are usually culled from the check-in comments in subversion. It’s a manual process. It might be nice to automatically dump them to a file, which can then be edited instead of written from scratch.
  • Code checks – this can encompass almost anything, but having static analysis tools is invaluable. Like the utility that I mentioned earlier to compare resource files across all the languages we support, it can save tons of manual, “stupid” labor.
  • Loading source onto a new machine – How long does it take to get up and running on a new machine? Admittedly, you shouldn’t be switching machines all THAT often, but when you do how easy is it to grab the source and start debugging? Are all the required libraries and tools in source control and automatically configured by build scripts?
  • E-mail – If you’re like me, you get scores if not hundreds of e-mails a day. How are you organizing, sorting, responding, ignoring, deleting them? Setup filters to put them into different folders, highlight or tag them when certain keywords appear. Also, get Google Desktop Search or Windows Desktop Search. I like both of them, but I’m currently using Google’s version. I may switch back in a while.
  • Bug reporting – While not strictly about automation, I think it’s close enough. Reporting bugs and code changes in a text file is good for a while–if you’re the only one working, and you have a small number of them to deal with. Once you start involving more programmers, and perhaps a manager who wants to see some basic reports, the text file doesn’t cut it. Get a simple bug reporting tool. I use BugTracker.Net because it’s easy, simple and does exactly what we need with minimum fuss. How do I know what to work on? I open up a web page and it tells me. I’ve automated not only some manual labor, but also some needless thought processes.
  • Calendaring – Do you need to write a weekly report for your manager? Keep track of employee’s vacation schedules? Use Outlook’s (or whatever PIM you choose) task list and calendar for anything you need to remember about a specific date. Set reminders for when you need to think about them, and then forget about them.
  • Data production – if you’re in a production environment generating data that needs to be analyzed, create tools to do as much of it as possible. Of course, the tools need to be checked for correctness, but once you’re confident, do it and don’t look back.

There are many, many ways you can optimize, reduce, and automated the work you’re doing. Remember, the whole point is to get rid of the “dumb” work and let yourself concentrate on the important, creative things.

Technorati Tags: , , , , , , ,

UnitTest++

I’ve got a great process using NUnit in all my .Net projects. However, at work we have a large MFC/C++ application that has NO automated testing. I’ve been planing on getting unit testing into it for quite a while, but finally decided this is the week to do it.

It turns out that C++ unit testing is nowhere near as standardized or easy as .Net or Java. Partly, it’s the lack of reflection/introspection, but I think it’s also because the language itself just doesn’t lend itself to it.

There are a few very good resources for C++ unit testing. I first came across Noel Llopis’ Exploring the C++ Unit Testing Framework Jungle. I was just about set on using CxxTest, just as he had decided, but realizing that the post was written at the end of 2004, I continued my research. Sure, something must have improved since then! I came across UnitTest++: The New choice for C++ Unit Testing. It turns out to be partially written by Noel Llopis, so I thought this must be good since he gave such good reviews of the earlier solutions. You can read more about it here.

UnitTest++ seems to be the best of all possible worlds. It’s extremely simple to add test cases. I won’t include examples here, because the previous links have plenty.

The only thing I wish I could find is an open source code coverage C++ tool. I couldn’t survive without NUnit/NCover in C#.

Simulating a Server in .Net (C#)

I recently had a need to simulate an FTP server connection in some unit tests. Rather than developing a full test-server, for this purpose, I could easily simulate an FTP stream by creating it in memory, and then passing that stream to the functions  that expect it:

public Stream CreateTestStream(string text) { byte[] buff = Encoding.ASCII.GetBytes(text); return new MemoryStream(buff, false); }

This function merely accepts text (FTP is text-based after all), and returns a memory stream. I can then use it in a unit test:

[Test] [ExpectedException(typeof(FtpException),"InvalidResponse")] public void FtpResponse_Constructor_BadStream_InvalidCode() { using (Stream s = CreateTestStream("NoErrorCode")) { FtpResponse response = new FtpResponse(s); } }

Checking all values of an enumeration in C#

I have a utility function that takes in a status enumeration and returns a string description associated with the given status code.

It looks something like this:

internal static string MailProcessCodeToString(MailProcessCodes eCode)
{
switch (eCode)
{
case MailProcessCodes.mpcGoodData:
    return “No errors detected in the order mail”;case MailProcessCodes.mpcNoHeader:
    return “Could not find header in order mail”;

//etc….
I wanted to create a unit test that ensure there was an error message for every possible value of the eCode. Rather than write a separate unit test for each value (there are over 30–don’t ask).It’s fairly easy to do in .Net (using NUnit):

[Test]
public void MailProcessCodeToString_AllSuccess()
{
   
int[] vals = (int[])Enum.GetValues(typeof(MailProcessCodes));
   
foreach (int val in vals)
    {
       
string errorString = Utils.MailProcessCodeToString((MailProcessCodes)val);       

        string msg = string.Format(“{0} has no error string defined”,
                      
Enum.GetName(typeof(MailProcessCodes), val));        Assert.IsTrue(errorString.Length > 0, msg);
    }
}

Enum.GetValues() returns an array of all defined values for an enum, and Enum.GetName() translates a value into the name of the constant.

Now I have a unit  test which will tell me which error code does not have a corresponding string.