Tag Archives: .net

Performance and Other Issues with System.Enum

I’ve been doing a lot of very intense .Net coding at work for the last few months and it’s very gratifying to see that .Net can handle some pretty serious high-load scalable server work. We are doing some pretty amazing things. However, there are all sorts of gotchas—you do have to learn how to do it correctly (but that’s true of any technology).

One of the things we’re most concerned about right now is performance, and where performance is concerned, one of the big gotchas is the .Net Framework Class Library. Don’t get me wrong, for 99.99% of all .Net projects, the performance is perfectly adequate and you don’t have to care (much). On the other hand, if you’re dealing with highly-scalable systems, you should definitely validate the performance of any general-purpose library you use.

As with all performance question, the only 100% solid advice in all situations is: measure, measure, measure.

General Purpose Methods May Not Be Fast

I did a profiler run on one of our components the other day and a few methods showed up that I would never expect to see in a profiler trace:

We were calling these methods a lot. In my mind, they both basically boiled down to logical AND (&) statements. Unfortunately that’s not the case.

If you use an IL decompiler like ILSpy to examine the code of these methods, you’ll see they do quite a bit more. For example, HasFlag also checks whether the types are equal—useful, but that’s a lot more expensive than a simple & check.

IsDefined is even worse. There are half a dozen checks for specific types, followed by an allocation of an array containing all the enum values, and finally a binary search. If your enums are string types, then it’s just a linear search with string comparisons.

It’s easy to understand why these methods are so heavy—they’re being extra cautious for correctness-sake. That’s probably what you want in a general-purpose framework.

However, if your goal is performance and you have a closed system, complete control of your code, and adequate tests, then you can make better decisions.

Rather than do this:

if (bookFlags.HasFlag(BookTypes.Fiction))

Do this instead:

if ((bookFlags & BookTypes.Fiction) != 0)


As for IsDefined, just don’t use it. Or use it in debug-only assertions to validate functionality.

Another Enum method to avoid if you can is ToString(), especially on enumerations that have [Flags].

HasFlag Gotcha

Suppose you have this code:

[Flags]
enum NodeFlags
{
    Exception = 1,
    Failure = 2,
    Canceled = 4,
    TimedOut = 8,
 
    AnyFailure = Exception | Canceled | TimedOut
}
 
if (node.Flags.HasFlag(NodeFlags.Failure)
    || node.Flags.HasFlag(NodeFlags.TimedOut)
    || node.Flags.HasFlag(NodeFlags.Exception))

You may be tempted to do this:

if (node.Flags.HasFlag(NodeFlags.AnyFailure))

This will not work. It will check that ALL of the bits are set, not just one of them.

You will need to do this:

if ((node.Flags & NodeFlags.AnyFailure) != 0)

We saw that around 5% of our CPU time was being spent just in Enum.IsDefined and Enum.HasFlag. With these changes, we reduce that to approximately 0.

If you find tips like these helpful, you’ll love my book, C# 4.0 How-To.

Measure Amount of Data to Serialize with a Null Stream

If you’ve got to serialize some data, especially in a binary format, it’s common to output the length of the data. This is useful for versioning, random access, knowing when you’re done reading the records, among other reasons.

Therefore, you need to know the size of the data you’re going to serialize. There are a few ways to do this:

  1. Measure the position you’re at, write the data, measure the new position, subtract, and that’s your length.
  2. If you want to write the length first (which is usually better), you can write a dummy value, such as 0, then writing the data, then backing up in the stream, and writing the real value.
  3. If you can’t back up the stream (very possible in some situations, or undesirable in others), you can measure the amount of data before you write. However, now you have to maintain that code in addition to the actual serialization.
  4. My solution presented here, avoids having to maintain separate code by writing the data to a null stream which does not write any data, but keeps track of how much data was “written.”
class NullStream : System.IO.Stream
{
    public override bool CanRead { get { return false; } }
 
    public override bool CanSeek { get { return false; } }
 
    public override bool CanWrite {get { return true; } }
 
    public override void Flush() { /*do nothing*/ }
 
    public override long Length { get { return Position; } }
 
    private long _position = 0;
    public override long Position 
    { 
        get 
        {
            return _position;
        }
        set
        {
            _position = value;
        }
    }
 
    public override int Read(byte[] buffer, int offset, int count)
    {
        throw new InvalidOperationException();
    }
 
    public override void Write(byte[] buffer, int offset, int count)
    {
        Position += count;
    }
 
    public override long Seek(long offset, System.IO.SeekOrigin origin)
    {
        throw new InvalidOperationException();
    }
 
    public override void SetLength(long value)
    {
        throw new InvalidOperationException();
    }
}

You can use it like this:

long GetDataSize()
{
    using (NullStream stream = new NullStream())
    {
        if (SaveData(stream))
        {
            return stream.Position;
        }
    }
    return 0;
}

There is a downside to something like this: you’re still essentially doing a lot of the work of serialization. Sure, you’re not writing out the bytes anywhere, but if, say, you need to encode a string as bytes before writing to the stream, that’s still going to happen.

Still, this technique made sense in my case, maybe it will work for you.

C# 4.0 How-To Available Now!

Well, it’s finally out! Amazon no longer lists the book as available for pre-sale, and it should be shipping to purchasers today or tomorrow. If you’re a B&N shopper, you can also order it there, or grab it in stores within a few days.

From the product description:

Real Solutions for C# 4.0 Programmers

Need fast, robust, efficient code solutions for Microsoft C# 4.0? This book delivers exactly what you’re looking for. You’ll find more than 200 solutions, best-practice techniques, and tested code samples for everything from classes to exceptions, networking to XML, LINQ to Silverlight. Completely up-to-date, this book fully reflects major language enhancements introduced with the new C# 4.0 and .NET 4.0. When time is of the essence, turn here first: Get answers you can trust and code you can use, right now!

Beginning with the language essentials and moving on to solving common problems using the .NET Framework, C# 4.0 How-To addresses a wide range of general programming problems and algorithms. Along the way is clear, concise coverage of a broad spectrum of C# techniques that will help developers of all levels become more proficient with C# and the most popular .NET tools.

Fast, Reliable, and Easy to Use!

  • Write more elegant, efficient, and reusable code
  • Take advantage of real-world tips and best-practices advice
  • Create more effective classes, interfaces, and types
  • Master powerful data handling techniques using collections, serialization, databases, and XML
  • Implement more effective user interfaces with both WPF and WinForms
  • Construct Web-based and media-rich applications with ASP.NET and Silverlight
  • Make the most of delegates, events, and anonymous methods
  • Leverage advanced C# features ranging from reflection to asynchronous programming
  • Harness the power of regular expressions
  • Interact effectively with Windows and underlying hardware
  • Master the best reusable patterns for designing complex programs

I’ll be doing a book giveaway at some point as well, once I get my own shipment. Stay tuned!

Get it from Amazon

Get it from Barnes and Noble

C# 4.0 How-To now available for pre-sale!

csharp_howto_ben_watson For the last year, aside from starting a great job with Bing, I’ve also been working on a book about C# 4.0 and the upcoming .Net framework. The news: it is finally available for presale! This book is not your typical C# reference. It’s designed to be an easy guide to how to accomplish specific tasks, using a problem/solution approach. Some examples:
  • How to use P/LINQ (new in .Net 4!)
  • Override Equals and implement IEquatable<T> correctly
  • Enforce coding contracts (new in .Net 4!)
  • Convert numbers to strings in arbitrary bases
  • Various ways of rounding, including “snapping” to specific intervals.
  • Dynamic discovery of WCF services
  • Make your Silverlight 3 application run out-of-the-browser
  • Speed up array access
  • Easily split work among multiple processors
  • Localize WinForms, WPF, ASP.Net, and Silverlight apps

…and hundreds of other topics, covering everything from the basics of C# to WPF, ASP.Net, interaction with the operating system, common application patterns and more. I cover all the new stuff that’s in both the C# language and the .Net 4 framework classes, as well as existing functionality.

Each topic begins with a brief description of when/where/why you would need to use the technique, followed by a brief explanation and source code.

I often just want a reference I can quickly dive into to remind me of how something is done. This book is my attempt to put in writing what I find valuable, both when I was learning C# and now when I just need to locate a sample quickly.

Over the next few months I’ll talk more about what’s in the book, and hopefully get back into blogging more programming topics.

An easy stack layout panel for WinForms

This is a simple, but useful tip. Users of WPF are spoiled. They have all sorts of layout options. Those of us still working in WinForms have FlowLayoutPanel and TableLayoutPanel. That’s it. WPF has those and more.

For my current project, I needed a panel to layout controls vertically. The TableLayoutPanel can be awkward to work with, at least for what I need it to. At first glance, the FlowLayoutPanel looks it won’t work, since it produces something like this:

FlowLayoutTable_1

That’s with changing the FlowDirection to TopDown and putting AutoScroll to true.

But what I want is this:

image

To achieve this layout, merely set all the following properties on a FlowLayoutPanel:

  • AutoScroll = True
  • FlowDirection = TopDown
  • WrapContents = False

et voilà, Instant stack panel.

Converting OLE_COLOR to System.Drawing.Color

I’ve been working on a project using Visual Studio Tools for Office 2008 (VSTO) and at one point I needed to get the colors for categories in Outlook 2007. There are actually 3 colors, and they are returned as uint’s–why the .Net wrappers don’t convert to colors for you, I don’t know (to avoid linking to System.Drawing?).

The important thing is to convert them into the friendly System.Drawing.Color objects I know and love. For this task, there exists a handy ColorTranslator class. There is a FromOle method that does the exact chore you need. Here’s a sample of my code:

 

   1: private void GetCategoryColors()
   2: {
   3:     foreach (OutlookLib.Category category in
   4:         Application.ActiveExplorer().Session.Categories)
   5:     {
   6:         CategoryColor color = new CategoryColor(
   7:             ColorTranslator.FromOle((Int32)category.CategoryGradientTopColor),
   8:             ColorTranslator.FromOle((Int32)category.CategoryGradientBottomColor),
   9:             ColorTranslator.FromOle((Int32)category.CategoryBorderColor));
  10:         _categoryCollection[category.Name] = color;
  11:     }
  12: }

Google Chrome – the Good, Meh, Ugly, and This Means War

I just read all about Google Chrome, their new open-source browser, in the comic they put out. No download link at this time, but I’m sure it’s coming. My initial thoughts:

The Good

  • The JavaScript changes seems to make sense. The better garbage collector and speed improvements can’t hurt.
  • The free API to download lists of malware or phishing sites is pretty nice.

The Meh

  • UI changes. Making each tab its own browser entity and putting controls in each tab? That’s it? So what!
  • Some of the search enhancements are interesting, but I don’t think that anyone will care that much in the end.
  • Showing most popular pages…meh
  • Unclear on the plugin model. Will they have their own? Will they run ActiveX (they imply yes). How about Firefox plugin compatibility? All we need is yet another API for writing plugins.

The Ugly

  • A new process for each tab? Are you serious? I understand that it’s (maybe)the only way to completely isolate web pages from each other, but given how many pages some people have running, that means an extra 50 processes on the system. That’s a lot of resources. I know their idea was to consider each web page an application, and of course each desktop application is its own process, but I don’t think we actually treat most web pages like applications. We create new browser tabs and switch pages with wild abandon. Most web sites are NOT applications–they’re reference. They’re just books open to 50 pages at once. (Was process isolation really a problem that needed solving? I almost NEVER have runaway tabs in IE7)
  • Proprietary JavaScript hooks. Sure, it’s open source, but they’re still building things into their version of JavaScript that only work with their browser.

This Means War

  • First front: SilverLight. Gears seems to be a direct assault on the concept of .Net and SilverLight. The technology and scope are different now, but I think ultimately they’re going after the same target: having the rich-client experience in your browser on multiple OSes/browsers.
  • Second front: Firefox: the only people who are going to download Chrome or even understand what it is are the people who use Firefox. If Chrome succeeds, it will be at Firefox’s detriment. Thanks for playing.

Overall, I felt a big “meh” after reading the comic. While many of the ideas are interesting, overall, I don’t see a compelling reason to switch. I’ll try it out when it becomes available, and my opinions will probably change on some things, but Google is going to have to do a lot more to overthrow IE. Maybe their purpose really is to just throw ideas out there and see what sticks, what gets integrated into competing products, etc. We’ll just have to see what happens next. It’s going to be a fun couple of years!

(P.S. Also, please everyone, especially media, start mocking Chrome for it’s “p%%n mode” just liked you mocked IE.)

Review: Pragmatic Unit Testing in C# with NUnit, 2nd Ed.

I saw this book when I bought Programming WPF a few weeks ago and it looked promising enough to buy. I’ve been doing unit testing in C# for a few years now, but I thought there were always things to learn and maybe I’d pick up a few new ideas.

It is easy to contrast this book with Beck’s Test Driven Development: By Example, and the two books definitely have a very different feel.

Beck’s book has a very evangelical feel to it, and it’s main purpose is to teach a mind set more than technical details. I believe this is important–maybe of first importance–but once you understand that, the rest of the book is a little simplistic for more experienced developers.

Pragmatic Unit Testing, on the other hand, focuses much more on the practical aspects (hmmm….I highly suspect that’s where the title comes from….) of unit testing. I liked the ideas on how to use categories and attributes to segregate tests that take too long to run on a regular basis. I also liked the section on singletons and getting around time-dependencies. The DateTime.Now problem is something I’ve had to deal with quite a bit in our server-side software that has a lot of time-dependant behavior. (In most cases, the problems were solved with refactoring the time into a function parameter.)

There are also good discussions of more mundane issues like how to deploy NUnit, where to put tests in a project, team practices, GUIs, threading, and C#-specific issues.

The discussion about mock objects (a very basic introduction) is also quite clear and understandable–more so than many resources I’ve seen on the web, which often assume you already know all about them.

Something I don’t like: the acronyms (BICEP, CORRECT, A TRIP). They kind of bug me. I like the ideas behind the acronyms and I think it’s more important and effective (for me, anyway) to internalize the principles of testing rather than remembering specific acronyms and the words they go with. YMMV.

Last Word…

I will probably only read Test Drive Development: By Example once,  but I will definitely come back to Pragmatic Unit Testing occasionally to refresh my ideas.

10 Ways to Learn New Things in Development

Expanding upon one of the topics in my post about 5 Attributes of Highly Effective Developers, I’ve been thinking of various ways to kick-start learning opportunities in my career and hobbies.

1. Read books. There are tons of books about programming–probably most of them are useless, but there are many, many gems that can greatly influence your abilities.

I still find that it’s easier and faster to find information about many topics in familiar books than to find similarly valuable information online. Read all your books to get to this point.

Books are also valuable from theory, architecture, design point of view. There just aren’t that many places on the web to get high-quality, authoritative instruction in this.

Like this? Please check out my latest book, Writing High-Performance .NET Code.

2. Read Code. This is something I was late to. I didn’t start reading a lot of significant code until after I had a few years of professional programming experience. I would be a better programmer if I had started earlier. I try to read some source code every week (not related to work, not my own, etc.) from an open source project. Start with programs that you use and are interested in. I started with Paint.Net and it solidified a lot of .Net program design technique for me.

Reading other people’s code shows you different ways of doing things than you might have thought of on your own.

3. Write Code – Lots of it. Fundamentally, the best way to learn something is to do it. You can’t fully internalize something until you’ve written it. This starts with something as simple as copying the code examples from tutorials and books. That’s copying by hand, not cut&paste. There’s a difference. The idea is internalize and think, not blindly copy. Look up new API calls as you go. Tweak things.

Most importantly, develop your own projects–whether they’re simple games, participation in an open source project, or a simple plug-in to a program you use.

Try to use new technologies, new techniques, new designs–do things differently. Do things better in this project than in previous ones.

This is really the core point–if you want to be a better developer than develop.

4. Talk to other developers – about specific problems you have, as well as the latest tech news from [Apple|Microsoft|Google|Other]. This not only helps you feel part of a team or a community, but exposes you to a wide variety of different ideas.

Different types of projects require different designs, coding techniques, processes and thinking.

If you work in a small team (like I do) and you don’t have access to many other people, go find some at a local user group meeting. If nothing else, participate in online forums (you’ll have to look harder for an intelligent discussion).

5. Teach others. Similar to just reading code versus writing it, teaching other people can do wonders for forcing you to learn a topic in depth.

The very idea that you’re going to have to teach a topic to someone else should force you to learn something with a far better understanding than you might otherwise. You can face questions.

If you can’t explain a concept to a 6 year-old, you don’t fully understand it. – Albert Einstein

Teaching situations are myriad: one-on-one with your office-mate, water-cooler meetings, informal weekly gatherings, learning lunches, classrooms, seminars, and more.

How about setting up a once-a-week 30 minute informal discussion among like-minded developers? Each week, someone picks a topic they want to know more about and teaches it to the others, instigating a conversation. If you knew were going to teach the group about synchronization objects, don’t you think you’d want to understand the ins and outs of critical section implementation?

6. Listen to podcasts

If you’ve got time where your brain isn’t otherwise occupied, subscribe to podcasts. My current favorite programming-related one is .Net Rocks. They also do a video screen cast called dnrTV.

These will help you keep up on the latest and greatest technologies. You can’t learn everything and podcasts are a good way to get shallow, broad knowledge about a variety of topics, from which you can do your own deep investigations.

If there are other, high-quality developer podcasts, I’d love to hear about them.

7. Read blogs

There are more blogs than people to read them, but some are extremely well-done. I’m not even going to post links to any–there are plenty of other resources out there for that. This is one of the best ways to connect to people who actually develop the software you love and use.

8. Learn a new language

If all you’ve ever done is C(++,#)/Java there are a LOT of other ways to think about computer problems. Learning a new language will change the way you think. It’s not just a different syntax–it’s fundamentally rewiring the brain. Sure, all languages get compiled down to assembler in the end, but that doesn’t mean a high level abstraction isn’t valuable.

Functional, query, and aspect-oriented languages are starting to merge with C-based languages–are you ready?

9. Learn the anti-patterns

Aside from knowing what to do, learn what not to do. Read Dailywtf.com often and take the lessons to heart if you don’t already know.

It’s all well and good to understand proper OO design, coding style, and what you should be writing, but it’s easy to get into bad habits if you’re not careful. Learning to recognize bad ideas is vital when taking charge of a project.

Wikipedia has a thorough breakdown of many common anti-patterns,

10. Be Humble

Learning means:

  • Replacing faulty knowledge with better knowledge
  • Adding knowledge that you do not already have

There’s no way to learn until you admit you have some deficiencies. It all comes back to humility, doesn’t it? If you ever start thinking you know everything you need to, you’re in trouble. True learning is about hungrily seeking after knowledge and internalizing it. It takes lots effort. We all know this in theory, but we have to be constantly reminded.

6 Ways to Increase Your Confidence As You Code

One of the key requirements for being able to reliably update software is the confidence that the changes you are making are safe. The amount of confidence required increases with the complexity of the system.

In my day job I work on a real-time messaging system that can have very, very little downtime. As the service grows and sees more traffic, the amount of downtime shrinks. We start to worry now if upgrades take longer than 5 minutes. (It’s almost to the point where we’ll need redundant systems in order to do maintenance).

To upgrade this software, I have to have an awful lot of confidence in the code changes made. Sometimes that confidence varies.

What to do to gain confidence:

1. Consistent process

Having a set of rules to follow is a fundamental requirement of good software engineering. I’m not going to discuss what the process should be, but you should have one that works well.

Why is this important?

Programmers like order. We like well-defined problems where we can see the end from the beginning. We don’t like haziness, indeterminism, or too many choices.

A process nails down the unknowns–it tells you very specifically what the next thing to do is. A good process leaves no room for doubt.

A good development, testing, and deployment process is the first step to building confidence in what you’re doing.

For my messaging system example, here’s a short summary of what our upgrade process is. We didn’t just come up with it–it evolved as our business grew and the requirements grew with it.

  1. All unit and integration tests pass (we have about 2200 automated tests for this that can run in about 4 minutes)
  2. We’ve run it on staging server for a few weeks with no issues.
  3. All new features have been tested on the staging server
  4. Formal change document has been submitted and approved.
  5. Formal test results documented.
  6. Previous version backed up.
  7. Perform upgrade
  8. Monitor system (including custom automated monitoring tools)

At any point, I know where we are in the process and what needs to be done next. Sure, there may be details within these steps that require thought and creativity, but the process guides it all and makes us more confident that we’re not performing ad-hoc operations.

2. Unit Testing

There are other types of testing, but it all starts at the unit level, with simple tests that exercise your code line by line, function by function, feature by feature. I recently wrote a few thoughts about unit testing. Unit testing is where you can see the overall wellness of your code–you want that green bar!

Without unit testing, how do you know the code you’re writing is doing what it should? do you just run it and push it through its paces? This is highly inefficient for most types of code. You’ll run out of steam before you start getting close to edge cases.

The fact is that automated unit tests are a baseline for confidence in your code. You need to be able demonstrate time and again that your code performs well.

This all presupposes that you are writing good unit tests. If you’re not sure, start studying. I don’t buy the arguments about lulling developers into a false sense of security–sure, that can happen, but having good developers who understand this is a prerequisite.

If you’re not unit testing–what is your basis for confidence in your code?

3. Code Coverage

Code coverage goes hand-in-hand with unit testing as a good way to automatically discover what areas of your program are in need of more testing. I’ve found that one of the biggest barriers to unit testing a large C++ application we have is that we have no way of easily measuring test coverage. If we had time, we could definitely to the analysis ourselves, or we could spend a lot of money to get a C++ instrumentation profiler, but these are slow and very tedious to use in my experience.

In .Net, use the tools to your advantage.

The psychological benefits of seeing 75-, 90-, 95-, even 100-percent coverage are immense. You know that every line of the program has at least been touched.

Of course, most code coverage tools analyze line coverage, not path coverage. Combine  complexity analysis with code coverage to determine which functionality should probably have better testing. There are plenty of free and commercial tools that will give you cyclomatic complexity, among other metrics.

Use other analysis tools like FxCop to make sure your other ducks are in a row. It can find easy-to-overlook problems like not validating arguments of public methods, which can then lead to more unit tests and more coverage to achieve.

4. Automation

Take yourself out of the equation as much as possible. The point of a process is to be repeatable–it’s like automating yourself. Not only should unit testing be automated (thankfully, most testing frameworks handle this easily), but so should coverage and quality analyses.

What about deployment? Automate it. Documentation generation? CD master creation? Web upload? E-mail notification? Automate them all. Production builds should be invoked with a single command.

Working on boring, repeatable code? Automate it with code-gen.

The bottom line is: Don’t waste your brain cells on stuff that is highly repeatable, especially when it is prone to mistakes.

5. Code Review

Last week, a rather serious bug was discovered in some of our software (not released yet, thankfully, but close). The bug was mine, and I knew exactly what the problem was, but instead of designing a solution by myself, I brought a co-worker into the discussion just to bounce ideas off of. He had great suggestions, and made me think of things I might not necessarily have thought of on my own. We both went over the code and came to a solution that was simple and acceptable to both of us. The confidence level was much higher with this than it would have been otherwise.

This story is repeated daily by programmers throughout the world. Code review is a practice based on the simple notion that there is no one person smart enough to get it correct the first time.

Even if you’re working alone, which I often do, it pays huge dividends to regularly review your code with an eye for finding trouble. If you see any weakness at all, don’t ignore it–fix it. If you’re reviewing your own code, it’s a good idea to wait a bit after the time you wrote it. This gives your brain a chance to forget a little bit about it. Then, if you find you can’t understand it anymore, it’s either too complicated, or (if it fundamentally really is complicated) you need better comments.

Reviewing with other people has more benefit, however. Not everybody thinks the same way about problems. People have different experience, different expertise and focus, and you can’t take advantage of that if you don’t let them teach you. Even if the other people have less expertise than you, it is still beneficial (assuming they have some basic competency that they can bring to the discussion).

Once you let other people tear into your code (nicely, I hope), your confidence can be higher because you can add the confidence other people have in it (once your problems are corrected, of course!)

6. Repeatable Experiences

In the end, one of the best ways to increase your confidence in yourself, your code, and your practices is to have the evidence of repeated experiences behind you. You’re always learning, and that learning contributes to improvements in processes, testing, and your personal coding practices. Once you learn what works, especially during tricky upgrades, you can go into the next trial with increased confidence that you’re doing something right.

Have any other ideas on increasing confidence? Leave them in the comments!

Technorati Tags: ,,