Short vs. Long Weak References and Object Resurrection

Last time, I talked about the basics of using WeakReference, what they meant and how the CLR treats them. Today, I’ll discuss some important subtleties.

Short vs. Long Weak References

First, there are two types of weak references in the CLR:

  • Short – Once the object is reclaimed by garbage collection, the reference is set to null. All of the examples in the previous article, with WeakReference and WeakReference<T>, were examples of short weak references.
  • Long – If the object has a finalizer AND the reference is created with the correct options, then the reference will point to the object until the finalizer completes.

Short weak references are fairly easy to understand. Once the garbage collection happens and the object has been collected, the reference gets set to null, the end. A short weak reference can only be in one of two states: alive or collected.

Using long weak references is more complicated because the object can be in one of three states:

  1. Object is still fully alive (has not been promoted or garbage collected).
  2. Object has been promoted and the finalizer has been queued to run, but has not yet run.
  3. The object has been cleaned up fully and collected.

With long weak references, you can retrieve a reference to the object during stages 1 and 2. Stage 1 is the same as with short weak references, but stage 2 is tricky. Now the object is in a possibly undefined state. Garbage collection has started, and as soon as the finalizer thread starts running pending finalizers, the object will be cleaned up. This can happen at any time, so using the object is very tricky. The weak reference to the target object remains non-null until the target object’s finalizer completes.

To create a long weak reference, use this constructor:

WeakReference<MyObject> myRefWeakLong 
    = new WeakReference<MyObject>(new MyObject(), true);

The true argument specifies that you want to track resurrection. That’s a new term and it is the whole point of long weak references.

Aside: Resurrection

First, let me say this up front: Don’t do this. You don’t need it. Don’t try it. You’ll see why. I don’t know if there is a special reason why resurrection is allowed in .NET, or it’s just a natural consequence of how garbage collection works, but there is no good reason to do something like this.

So here’s what not to do:

class Program
{
    class MyObject
    {
        ~MyObject()
        {
        myObj = this;
        }
    }

    static MyObject myObj = new MyObject();

    static void Main(string[] args)
    {
        myObj = null;
        GC.Collect();
        GC.WaitForPendingFinalizers();
    }
}

By setting the myObj reference back to an object, you are resurrecting that object. This is bad for a number of reasons:

  • You can only resurrect an object once. Because the object has already been promoted to gen 1 by the garbage collector, it has a guaranteed limited lifetime.
  • The finalizer will not run again, unless you call GC.ReRegisterForFinalize() on the object.
  • The state of the object can be indeterminate. Objects with native resources will have released those resources and they will need to be reinitialized. It can be tricky picking this apart.
  • Any objects that the resurrected object refers to will also be resurrected. If those objects have finalizers they will also have run, leaving you in a questionable state.

So why is this even possible? Some languages consider this a bug, and you should to. Some people use this technique for object pooling, but this is a particularly complex way of doing it, and there are many better ways. You should probably consider object resurrection a bug as well. If you do happen upon a legitimate use case for this, you should be able to fully justify it enough to override all of the objections here.

Weak vs. Strong vs. Finalizer Behavior

There are two dimensions for specifying a WeakReference<T>: the weak reference’s creation parameters and whether the object has a finalizer. The WeakReference’s behavior based on these is described in this table:

  No finalizer Has finalizer
trackResurrection = false short short
trackResurrection = true short long

An interesting case that isn’t explicitly specified in the documentation is when trackResurrection is false, but the object does have a finalizer. When does the WeakReference get set to null? Well, it follows the rules for short weak references and is set to null when the garbage collection happens. Yes, the object does get promoted to gen 1 and the finalizer gets put into the queue. The finalizer can even resurrect the object if it wants, but the point is that the WeakReference isn’t tracking it–because that’s what you said when you created it. WeakReference’s creation parameters do not affect how the garbage collector treats the target object, only what happens to the WeakReference.

You can see this in practice with the following code:

class MyObjectWithFinalizer 
{ 
    ~MyObjectWithFinalizer() 
    { 
        var target = myRefLong.Target as MyObjectWithFinalizer; 
        Console.WriteLine("In finalizer. target == {0}", 
            target == null ? "null" : "non-null"); 
        Console.WriteLine("~MyObjectWithFinalizer"); 
    } 
} 

static WeakReference myRefLong = 
    new WeakReference(new MyObjectWithFinalizer(), true); 

static void Main(string[] args) 
{ 
    GC.Collect(); 
    MyObjectWithFinalizer myObj2 = myRefLong.Target 
          as MyObjectWithFinalizer; 
    
    Console.WriteLine("myObj2 == {0}", 
          myObj2 == null ? "null" : "non-null"); 
    
    GC.Collect(); 
    GC.WaitForPendingFinalizers(); 
    
    myObj2 = myRefLong.Target as MyObjectWithFinalizer; 
    Console.WriteLine("myObj2 == {0}", 
         myObj2 == null ? "null" : "non-null"); 
}

The output is:

myObj2 == non-null 
In finalizer. target == non-null 
~MyObjectWithFinalizer 
myObj2 == null 

Finding Weak References in a Debugger

Windbg can show you how to find where your weak references, both short and long.

Here is some sample code to show you what’s going on:

using System; 
using System.Diagnostics; 

namespace WeakReferenceTest 
{ 
    class Program 
    { 
        class MyObject 
        { 
            ~MyObject() 
            { 
            } 
        } 

        static void Main(string[] args) 
        { 
            var strongRef = new MyObject(); 
            WeakReference<MyObject> weakRef = 
                new WeakReference<MyObject>(strongRef, trackResurrection: false); 
            strongRef = null; 

            Debugger.Break(); 

            GC.Collect(); 

            MyObject retrievedRef; 

            // Following exists to prevent the weak references themselves 
            // from being collected before the debugger breaks 
            if (weakRef.TryGetTarget(out retrievedRef)) 
            { 
                Console.WriteLine(retrievedRef); 
            } 
        } 
    } 
} 

Compile this program in Release mode.

In Windbg, do the following:

  1. Ctrl+E to execute. Browse to the compiled program and open it.
  2. Run command: sxe ld clrjit (this tells the debugger to break when the clrjit.dll file is loaded, which you need before you can execute .loadby)
  3. Run command: g
  4. Run command .loadby sos clr
  5. Run command: g
  6. The program should now break at the Debugger.Break() method.
  7. Run command !gchandles

You should output similar to this:

0:000> !gchandles
  Handle Type          Object     Size     Data Type
011112f4 WeakShort   02d324b4       12          WeakReferenceTest.Program+MyObject
011111d4 Strong      02d31d70       36          System.Security.PermissionSet
011111d8 Strong      02d31238       28          System.SharedStatics
011111dc Strong      02d311c8       84          System.Threading.ThreadAbortException
011111e0 Strong      02d31174       84          System.Threading.ThreadAbortException
011111e4 Strong      02d31120       84          System.ExecutionEngineException
011111e8 Strong      02d310cc       84          System.StackOverflowException
011111ec Strong      02d31078       84          System.OutOfMemoryException
011111f0 Strong      02d31024       84          System.Exception
011111fc Strong      02d3142c      112          System.AppDomain
011113ec Pinned      03d333a8     8176          System.Object[]
011113f0 Pinned      03d32398     4096          System.Object[]
011113f4 Pinned      03d32178      528          System.Object[]
011113f8 Pinned      02d3121c       12          System.Object
011113fc Pinned      03d31020     4424          System.Object[]

Statistics:
      MT    Count    TotalSize Class Name
70e72554        1           12 System.Object
01143814        1           12 WeakReferenceTest.Program+MyObject
70e725a8        1           28 System.SharedStatics
70e72f0c        1           36 System.Security.PermissionSet
70e724d8        1           84 System.ExecutionEngineException
70e72494        1           84 System.StackOverflowException
70e72450        1           84 System.OutOfMemoryException
70e722fc        1           84 System.Exception
70e72624        1          112 System.AppDomain
70e7251c        2          168 System.Threading.ThreadAbortException
70e35738        4        17224 System.Object[]
Total 15 objects

Handles:
    Strong Handles:       9
    Pinned Handles:       5
    Weak Short Handles:   1

The weak short reference is called a “Weak Short Handle” in this output.

Next Time

The first article explained how WeakReference works, and this one explained a few of the subtleties, including some behavior you probably don’t want to use. Next time, I’ll go into why you would want to use WeakReference in the first place, and provide a sample application.


Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code by Ben Watson. Available now in print and as an eBook at:

Prefer WeakReference<T> to WeakReference

In Writing High-Performance .NET Code, I mention the WeakReference type briefly in Chapter 2 (Garbage Collection), but I don’t go into it too much. However, for the blog, I want to start a small series of posts going into some more detail about WeakReference, how it works, and when to use it, with some example implementations. In this first post, I’ll just cover what it is, what options there are, and how to use it.

A WeakReference is a reference to an object, but one that still allows the garbage collector to destroy the object and reclaim its memory. This is in contrast to a strong (i.e., normal) reference that does prevent the GC from cleaning up the object.

There are two versions of WeakReference:

First, let’s take a look a WeakReference, which has been around since .NET 1.1.

You allocate a weak reference like this:

var weakRef = new WeakReference(myObj);
myObj = null;

myObj is an existing object of your choice. Once you assign it to the weakRef variable, you should set the original strong reference to null (otherwise, what’s the point?). Now, whenever there is a garbage collection the object weakRef is referring to may be collected. To access this object, you may be tempted to make use WeakReference’s IsAlive property, as in this example:

WeakReference ref1 = new WeakReference(new MyObject());
if (ref1.IsAlive)
{
    // wrong!
    DoSomething(ref1.Target as MyObject);
}

IsAlive is a rather silly property. If it returns false, it’s fine–you know the object has been collected. However, if it returns true, then you don’t actually know anything useful because the object could still be garbage collected before you get a chance to make a strong reference to it!

The correct way to use this is to ignore the IsAlive property completely and just attempt to make a strong reference from Target:

MyObject obj = ref1.Target as MyObject;
if (obj != null)
{
    // correct
    DoSomething(obj);
}

Now there is no race. If obj ends up being non-null, then you’ve got a strong reference that is guaranteed to not be garbage collected (until your own strong reference goes out of scope).

Recognizing some of the silliness and umm…weakness of WeakReference, WeakReference<T> was introduced in .NET 4.5 and it formalizes the above procedure by removing both the Target and IsAlive properties from the class and providing you with these two methods:

  • SetTarget – Set a new target object
  • TryGetTarget – Attempt to retrieve the object and assign it to a strong reference

This example demonstrates the usage, which is essentially the same as the correct way to use WeakReference from above:

WeakReference<MyObject> ref2 = new WeakReference<MyObject>(new MyObject());
MyObject obj2;
if (ref2.TryGetTarget(out obj2))
{
    DoSomething(obj2);
}

You could also ask yourself: Why is there a SetTarget method at all? After all, you could just allocate a new WeakReference<T>.

If you are using WeakReference<T> at all, it likely means you are somewhat memory conscious . In this case, allocating new WeakReference<T> objects will contribute extra, unnecessary memory pressure, potentially worsening the overall performance. Why do this, when you can just reuse the WeakReference<T> object for new targets as needed?

Next time, more details on weak references, particularly the differences between short and long weak references, and taking a peek at them in the debugger. We’ll also cover when you should actually use WeakReference<T> at all.


Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code by Ben Watson. Available now in print and as an eBook at:

Using Windbg to answer implementation questions for yourself (Can a Delegate Invocation be Inlined?)

The other day, a colleague of mine asked me: Can a generated delegate be inlined? Or something similar to this. My answer was that the generated code is going to be JITted and optimized like any other code, but later I started thinking…. “Wait a sec, can the actual call to the delegate be inlined?”

I’m going to give you the answer before I even start this article: no.

I cover the rules of method inlining that the JITter uses in my book, Writing High-Performance .NET Code, but I don’t discuss this specific situation. You could logically make the leap, however, that there are two other rules that imply this:

  • Virtual methods will not be inlined
  • Interface calls with multiple concrete implementations in a single call site will not be inlined.

While neither of those rules are delegate-specific, you can infer that a delegate call might have similar constraints. You could ask around on the Internet. Somebody on stackoverflow.com will surely answer you, but I want to show you how to find out the answer to this for yourself, which is an invaluable skill for harder questions, where you might not be able to find out the answer unless you know people on the CLR team (which I do, but I *still* try to find out answers before I bother them).

First, let’s see a test program that will exercise various types of function calls, starting with a simple method call that we would expect to be inlined.

using System;
using System.Runtime.CompilerServices;

namespace DelegateInlining
{
    class Program
    {
        static void Main(string[] args)
        {
            TestNormalFunction();
        }
        
        private static int Add(int x, int y) { return x + y; }

        [MethodImpl(MethodImplOptions.NoInlining)]
        private static void TestNormalFunction()
        {
            int z = Add(1, 2);
            Console.WriteLine(z);
        }
    }
}

The code we’re interested in inlining is the Add method. Don’t confuse that with the NoInlining option on TestNormalFunction, which is there to prevent the test method itself from being inlined The test method is there to allow breakpoint setting and debugging.

Build this code in Release mode for x86. Then open Windbg.

If you’re not used to using Windbg, I highly encourage you to start. It is far more powerful than Visual Studio’s debugger, especially when it comes to debugging the details of .NET. It is not strictly necessary for this particular exercise, but it is what I recommend.

To get, Windbg, install the Windows SDK—there is the option to install only the debugger if you wish.

In Windbg:

  1. Ctrl-E to open an executable program. Navigate to and open the Release build of the above program. It will start executing and immediately break
  2. Type the command: sxe ld clr. What we want to do is set a breakpoint inside the TestNormalFunction. To do that, we need to use the SOS debugger extension, which relies on clrjit.dll, which hasn’t been loaded in the process yet. So the first thing to do is set a breakpoint on loading clrjit.dll: sxe ld clrjit
  3. Enter the command g for “go” (or hit F5). The program will then break on the load of clrjit.dll.
  4. Enter the command .loadby sos clr – this will load the SOS debugging helper.
  5. Enter the command !bpmd DelegateInlining Program.TestNormalFunction – this will set a managed breakpoint on this method.
  6. Enter the command g to continue execution. Execution will break when it enters TestNormalFunction.
  7. Now you can see the disassembly for this method (menu View | Dissassembly).

00b80068 55              push    ebp
00b80069 8bec            mov     ebp,esp
00b8006b e8e8011b70      call    mscorlib_ni+0x340258 (70d30258)
00b80070 8bc8            mov     ecx,eax
00b80072 ba03000000      mov     edx,3
00b80077 8b01            mov     eax,dword ptr [ecx]
00b80079 8b4038          mov     eax,dword ptr [eax+38h]
00b8007c ff5014          call    dword ptr [eax+14h]
00b8007f 5d              pop     ebp
00b80080 c3              ret

There are some calls there, but none of them are to Add—they are all functions inside of mscorlib. The call to the dword ptr is virtual function call. These are all related to calling Console.WriteLine.

The key is the instruction at address 00b80072, which moves the value 3 directly into register edx. This is the inlined Add call. The compiler inlined not only the function call, but the trivial math as well (an easy optimization the compiler will do for constants).

So far so good. Now let’s look at the same type of thing through a delegate.

delegate int DoOp(int x, int y);

[MethodImpl(MethodImplOptions.NoInlining)]
private static void TestDelegate()
{
    DoOp op = Add;
    int z = op(1, 2);
    Console.WriteLine(z);
}

Change the Main method above to call TestDelegate instead. Follow the same steps given previously for Windbg, but this time set a breakpoint on TestDelegate.

00610077 42              inc     edx
00610078 00e8            add     al,ch
0061007a 8220d0          and     byte ptr [eax],0D0h
0061007d ff8bc88d5104    dec     dword ptr [ebx+4518DC8h]
00610083 e8481b5671      call    clr!JIT_WriteBarrierECX (71b71bd0)
00610088 c7410cc4053304  mov     dword ptr [ecx+0Ch],43305C4h
0061008f b870c04200      mov     eax,42C070h
00610094 894110          mov     dword ptr [ecx+10h],eax
00610097 6a02            push    2
00610099 ba01000000      mov     edx,1
0061009e 8b410c          mov     eax,dword ptr [ecx+0Ch]
006100a1 8b4904          mov     ecx,dword ptr [ecx+4]
006100a4 ffd0            call    eax
006100a6 8bf0            mov     esi,eax
006100a8 e8ab017270      call    mscorlib_ni+0x340258 (70d30258)
006100ad 8bc8            mov     ecx,eax
006100af 8bd6            mov     edx,esi
006100b1 8b01            mov     eax,dword ptr [ecx]
006100b3 8b4038          mov     eax,dword ptr [eax+38h]
006100b6 ff5014          call    dword ptr [eax+14h]
006100b9 5e              pop     esi
006100ba 5d              pop     ebp
006100bb c3              ret

Things got a bit more complicated. As you’ll read in Writing High-Performance .NET Code, assigning a method to a delegate actually results in a memory allocation. That’s fine as long that operation is cached and reused. What we’re really interested in here starts at address 00610097, where you can see the value 2 being pushed onto the stack. The next line moves the value 1 to the edx register. There are our two function arguments. Finally, at address 006100a4, we’ve got another function call, which is the call to Add, and the key to this whole thing becomes clear. The address of that function had to be retrieved via pointer, which means it’s essentially like a virtual method call for the purposes of inlining.

You can also do the same exercise with a lambda expression (it will look similar to the delegate disassembly above).

So there’s the simple answer.

There is one more interesting case: a delegate that calls into method A that calls method B. We already know that method A won’t be inlined, but can method B be inlined into method A?

[MethodImpl(MethodImplOptions.NoInlining)]
private static void TestDelegateWithFunctionCall()
{
    DoOp op = (x, y) => Add(x, y);
    int z = op(1, 2);
    Console.WriteLine(z);
} 

You can do the same analysis as above. You will see the call into the delegate/lambda will not be inlined, but there is no further function call, so yes, Method B can be inlined.

There you have it. Even though, the answer was pretty clear from the start, you at least have the tools to answer it or yourself. Don’t be afraid of the debugger, or of looking at assembly code, even for .NET programs.


Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code by Ben Watson. Available now in print and as an eBook at:

Almost 6 years at Microsoft…

I will have been at Microsoft for 6 years this fall. What an incredible journey I’m on. When I first joined, Bing was Live Search and was a totally different organization. It was a lot smaller. I think back then we could not even be called underdogs—Live Search just was not relevant. The progress we have made in the last few years has been nothing short of amazing. We have overtaken Google in some key areas, and the future is nothing but positive for us.

I have been working on the same team almost since I joined (I had two months on a team that was reorganized) and I could not have been luckier to land where I did, right in the heart of the query serving pipeline.

Since then, we’ve gone through some major redesign efforts and pushed out some amazing pieces of software to drive Bing (and many other parts of Microsoft). I became extremely familiar with .NET performance, and that has been key to my own personal success and that of my team’s. The challenges remain, and always will, which is a good nothing.

I work on a great team with some really smart people. I enjoy going to work almost every day. A few things bother me, but I love most of it. I really could not ask for anything better…


Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code by Ben Watson. Available now in print and as an eBook at:

Announcing Writing High-Performance .NET Code

This blog has been silent for far too long. That’s because I’ve been heads-down on a side project for the last 10 months. I’d like to announce my latest technical book:

Writing High-Performance .NET Code

Cover-Tall-2000x2828

If you write managed code, you want this book. If you have friends who write managed code, they want this, even if they don’t know it yet.

Do you want your .NET code to have the absolute best performance it can? This book demystifies the CLR, teaching you how and why to write code with optimum performance. Learn critical lessons from a person who helped design and build one of the largest high-performance .NET systems in the world.

This book does not just teach you how the CLR works—it teaches you exactly what you need to do now to obtain the best performance today. It will expertly guide you through the nuts and bolts of extreme performance optimization in .NET, complete with in-depth examinations of CLR functionality, free tool recommendations and tutorials, useful anecdotes, and step-by-step guides to measure and improve performance.

Among the topics you will learn are how to:

  • Choose what to measure and why
  • Use many amazing tools, freely available, to solve problems quickly
  • Understand the .NET garbage collector and its effect on your application
  • Use effective coding patterns that lead to optimal garbage collection performance
  • Diagnose common GC-related issues
  • Reduce costs of JITting
  • Use multiple threads sanely and effectively, avoiding synchronization problems
  • Know which .NET features and APIs to use and which to avoid
  • Use code generation to avoid performance problems
  • Measure everything and expose hidden performance issues
  • Instrument your program with performance counters and ETW events
  • Use the latest and greatest .NET features
  • Ensure your code can run on mobile devices without problems
  • Build a performance-minded team

…and much more.

See http://www.writinghighperf.net for up-to-date information about the book. You can also like the Facebook page or subscribe to this blog to see updates.

The book is currently available via Amazon and Kobo. Barnes and Noble is pending. More retailers and formats will follow. See the Buy page to check for current availability.

I will also be posting some blog entries with topics inspired by the book, but weren’t quite a good fit.


Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code by Ben Watson. Available now in print and as an eBook at:

Goodbye, Wireless!

I’m going to talk about a product today, and no, this post is not sponsored. This is just something I recently started using and it really worked out for me.

When we moved into our current home, I knew I wanted to setup a media center on our TV. It would incorporate Windows Media Center with the MediaBrowser plugin, an array of disks storing all of our house’s media (all legal of course!), a remote control, the NetFlix plugin, etc. The only weakness in the whole system was the network. I went through a couple of wireless card and antenna solutions, even upgrading our router to one that could do 5 ghz and dedicated that to just the Media Center.

It worked ok, but there were some downsides:

  • The wireless speed never quite lived up to what was advertised. Partly this is because of the configuration and placement of the TV/computer—5 ghz is far more directional than the standard 2.4 ghz–but I was also probably expecting too much.
  • Because of this, the disk array had to be physically plugged into the media center computer to be reliable. This increased the clutter of the living room–one more ugly box that wouldn’t fit inside the media cabinet—and made managing that storage space a pain.

I was starting to consider wiring our house for Ethernet when a friend mentioned this: the D-Link PowerLine AV 500. Power-line networking. The last I had heard about this was probably 10 years ago when it was first being introduced, and I haven’t given it a thought since. These adaptors are the epitome of ease. One of them goes into a socket near my router, with a network cable going from router to adapter. The other adapter goes next to my media center, with another connecting ethernet cable. There’s a simple software utility (or just push-button) to secure them to prevent others from leaching your Internet (if you have shared wiring).

It has made all of the difference. No more unsightly wireless antenna, the disk array has been moved upstairs to the office, and the speeds are MUCH faster than before. There is no issues whatsoever streaming high-def over this network. If you’ve got a media center that needs to be far away from your router, then this is the thing to get.


Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code by Ben Watson. Available now in print and as an eBook at:

How To Debug GC Issues Using PerfView

Update: If you find this article useful, you can find a lot more information about garbage collection, debugging, PerfView, and .NET performance in my book Writing High-Performance .NET Code.

In my previous artlcle, I discussed 4 ways to optimize your server application for good garbage collection performance. An essential part of that process is being able to analyze your GC performance to know where to focus your efforts. One of the first tools I always turn to is a little utility that has been publically released by Microsoft.

PerfView Overview

PerfView is a stand-alone, no-install utility that can help you debug CPU and memory problems. It’s so light-weight and non-intrusive that it can be used to diagnose production applications with minimal impact.

I’ve never used it for CPU performance, so I can’t comment on that aspect of it, but that is the primary use for it (which is helpful to keep in mind when trying to grok the “quirky” UI).

PerfView collects data in two ways (as far as memory analysis is concerned):

  1. ETW tracing – This is the heart and soul of PerfView. It’s primarily an event analyzer with advanced grouping abilities to show you only the important things. If you want to know more about ETW, see this series at the ntdebugging blog.
  2. Heap dump – PerfView can dump the heap of your process and apply the same analysis and views that it does for events.

The basic view of the utility is a spreadsheet-like UI with function names and associated inclusive/exclusive costs – just like you would expect to see in a typical CPU profiler. The same paradigm is useful for memory analysis as well.

There are other views that summarize the collected events for you in easy-to-understand reports. We’ll take a quick look at all of this.

In this article, I’ll use PerfView to show you how to see the following:

  • How frequently garbage collections occur and how long they take.
  • The cause for Gen2 collections.
  • The source of large-object allocations.
  • The roots of all the memory in the heap to see who’s holding on to it.
  • A diff of the heap to see what’s changing most frequently.

Test Program

When using a new utility like this, it’s often extremely helpful to create your own test programs with known behavior to ensure that you can use the utility to see what you expect. I’ve created a very simple one, here:

class Program
{
    private static List<int[]> arrays = new List<int[]>();
    private static Random rand = new Random();

    static void Main(string[] args)
    {            
        Console.WriteLine("Press any key to exit...");
        while (!Console.KeyAvailable)
        {
            int size = rand.Next(1024, 100000);
            int[] newArray = new int[size];
            arrays.Add(newArray);
            System.Threading.Thread.Sleep(10);
        }
        Console.WriteLine("Done, exiting");
    }
}

This program “leaks” memory by continually creating arrays and storing them in a list that never gets cleared.

I also make it use server GC, to match what I discussed in the first article.

You can download the sample solution here.

Taking a Trace

When you startup PerfView, you’ll see a window like this:

image

The manual is completely integrated into the program and can be accessed using the links in the menu bar. It’s a fairly dense information dump, but you can learn quite a bit about how to really get the most of out this utility.

First, start the test program and let it run in the background until we’re done taking the trace.

In PerfView, open the Collect menu and select the Collect command. A collection dialog will appear. Don’t change any setting for the moment and just hit Start Collection.You’ll see some status indicating the size and duration of the data collected. Let it go for at least 30 seconds. Note that you don’t specify which process you’re interested in at this stage – PerfView collects events for the machine as a whole.

image

When you’re done click Stop Collection. PerfView will process the collected events for a few seconds or minutes, and then a window will pop up asking you to select a process. Just cancel this (it wants to show you a CPU profile, which we’re not interested in right now) to get back to main screen.

You’ll now see a file show up: PerfViewData.etl (unmerged). Click on the little arrow next to this and you’ll see:

image

From this, we’ll find all the data we’re interested in.

Get GC Stats (pause times and more)

The first place to start is just to get an overall picture of GC performance for your app. There is a view for just that. Double-click the GCStats report, and that will bring up a window with tables for each app. Find MemoryLeak.exe

My test run yields this summary table:

image

Every garbage collection was a generation 2 collection (that’s generally a bad thing), but at least they were fast (to be expected in such a simple program).

Reason for Gen 2 Collection

Gen 2 GCs can happen for two reasons—surviving a gen 1 collection, or allocating on the large object heap. This view will also tell us, further down, which of these is the reason:

image

The collections happened because of large object allocation. You can also see that the second GC happened about 14 seconds after the first, and the next about 32 seconds after that. There are tons of other stats in this view, so look around and see what you can divine about the program’s behavior from this.

Get Source of Large Allocations

From the main PerfView screen, open the GC Heap Alloc Stacks view and find the correct process. This shows you a list of objects which represent the tops of allocation stacks.

image

PerfView has helpfully organized all large-object allocations under the LargeObject entry. Double click this to see all such all allocations:

image

Important: If you see entries like this:

OTHER <<clr?>>

Then right-click on the list and click on Lookup Symbols. Follow the instructions to get the symbol server setup so you can see CLR and Windows function names.

From the above entry view, it’s apparent that the vast majority of large objects are arrays being allocated in Main()—exactly what we expect given our predictable leaky program.

A note on the strange column names: remember how I said this program is designed for CPU profiling? These are typical columns for showing% of CPU time in various parts of a stack, repurposed for memory analysis. Inc % is the percent of bytes allocated on this object compared to all recorded allocations, Inc is the number of bytes allocated, and Inc Ct is the number of objects allocated.

In the above example, this reads: Allocated 6589 arrays for a total of 3.9 GB, accounting for 98% of the memory allocated in the process.

By the way, these are not 100% accurate numbers. There is some sampling going on because of how the events work, but it should be fairly close in most applications.

Who’s Referencing Leaking Memory?

One of the few ways to “leak” memory in C# is to hold onto it unknowingly. By taking a heap dump, we can see the path of object references for who’s holding onto memory.

We’ll need to do a different type of collection. In the main PerfView window. Go to the Memory menu and click Take Heap Snapshot.

image

Find your process and click Dump GC Heap. This performs a live heap walk (that is, the application continues running, so it’s possible the view is slightly inconsistent—not usually an issue), sampling what it finds, and presenting the results in the same type of view as before:

image

Right away you can see that the static variable MemoryLeak.Program.arrays is holding onto 100% of memory in our application. The stack to the root isn’t that interesting in this case because all static variables are rooted directly, but if this were a member field, you would see the hierarchy of objects that are holding onto these references.

Use Two Heap Dumps to see What’s Increasing Fastest

While the test program is still running, take another heap dump, ensuring you save it to a different file. Open both dump views and in the second one, go to the Diff menu and there will be an option to use the other file as a baseline for the diff. This will bring up another window showing you the changes between the two dump files—extremely helpful for narrowing down the most likely areas for leaks.

Important: If you want to analyze the perf trace on a different computer than the one you took it on, you must tell PerfView to merge the file—this will cause all the different files it generated to be combined and symbols reconciled. Just right-click on the ETL file and select Merge. You can also optionally Zip the file (which implies a Merge).

Next Time

Next time, we’ll look at some more drastic measures for protecting yourself against expensive GCs—for when all else fails.

Resources

  • Download the sample test program here.
  • Get PerfView here.

Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code by Ben Watson. Available now in print and as an eBook at:

4 Essential Tips for High-Performance Garbage Collection on Servers

Garbage CollectionUpdate: If you find this useful, you can read a much more complete treatment of garbage collection and performance in my book Writing High-Performance .NET Code.

Update: Part 2 – How to Debug GC Issues with PerfView is now available.

On this blog, I’ve alluded to the fact that I work on high-performance server applications, most recently in .Net. Writing these in .Net is just as possible as it is in native code, but it does come with its own set of challenges. In particular, one of the biggest things you need to learn how to deal with is garbage collection.

There is a lot out there already written about the CLR’s garbage collector, so I’m not going to go over many of the details. If you need a primer on it, MSDN has some documentation:

Read that first. For the rest of this article series, I will assume that you understand how the GC basically works.

In this and future articles, I’ll cover a lot of the stuff I’ve learned to improve application performance in the face of garbage collection.

Tip 1: Use Server GC

There are two modes of garbage collection (GC): workstation and server. As long as you’re running multiple processors, you almost certainly want server mode collection. With workstation mode, a GC happens on the thread that makes the allocation that causes the GC. The collection happens at normal priority.

With server GC, a thread for every core is created just for doing GC. There is also a small object heap and a large object heap created for each GC thread. All of the program’s allocations are spread among these heaps (more on large object heaps later). When no GC is happening, these threads are blocked and do nothing. When a GC is triggered, all of the user threads get paused, and all the GC threads wake up at highest priority and do collection in parallel. All of these optimizations lead to server GC usually being much faster than workstation GC.

A word about concurrent collections: In workstation GC, concurrent collections are enabled by default. However, this only applies to generation 2 collections. Generations 0 and 1 are always blocking. However, given that it’s concurrent, that means that it will compete with your own threads that are trying to get actual work done. In a high-performance server scenario, that may not be acceptable. A better strategy is to ensure that generation 2 collections never (or extremely rarely) happen.

You enable server GC by putting this in app.config:

<configuration>
   <runtime>
      <gcServer enabled="true"/>
   </runtime>
</configuration>

 

Tip 2: Objects Live Briefly or Forever

A histogram of object lifetimes in your app should look essentially like this:

image

Object last either a vanishingly brief amount of time, or they last forever – it’s the stuff in the middle that will kill your performance.

This has everything to do with the generations of garbage collection and object survivorship. There are three generations: 0, 1, and 2. Generation 0 happens most often and is the fastest—ideally lasting only a couple of milliseconds, if that. Objects that didn’t get cleaned up in generation 0 are put into generation 1. Generation 1 collections are also very fast, usually as fast as generation 0. The problem, though, is that objects that make it to generation 1 have a fair chance of surviving this generation, and being put into generation 2.

Generation 2 is the problem. A generation 2 collection is much slower than 0 or 1—often on the order of hundreds of milliseconds or even seconds—that means your process is paused completely for that time. You do not want objects to survive to generation 2.

So how often do collections happen? There is no hard-and-fast rule: it all depends on your allocation rate, memory pressure, and patterns that you’ve trained into the GC. The GC will adapt over time, training itself on your memory usage patterns. All of this completely depends on your application and I’ll look at ways to measure all of this in a future article.

Tip 3: All Long-Lived Objects Must Be Pooled

It may be that you can’t ensure all objects for a given request are cleaned up in the first generation 0 collection that occurs. If requests are in memory longer than the time between collections, then you’re guaranteed to have survivorship.

For these types of objects, first see if you can factor them so that not all parts them have to live that long. Control object lifetime very closely and null out references once you’re done.

Once you’ve done that, hopefully there are only a handful of objects that really must last the entire length of a request. For those, create a pool of them with reinitialization semantics—effectively move them to the far end of that histogram above, where they live forever.

This works because of the adaptive nature of the garbage collector – it learns over time that if it does a collection and doesn’t free up much memory, it will schedule that generation of collection to happen less frequently. In my own case, at one point, our server had trained the GC to do a generation 2 collection less often than once per day, under a constant load. With enough work, we could probably get that to essentially never.

You may be able to get quite far without the need to implement object pooling. Or you may need to pool only a small number of objects, and the survivorship of the remaining objects is not enough to cause problematic garbage collections—only measurement and observation will tell you for sure.

Tip 4: All Large Objects Must Be Pooled

There is a way to cause an object to automatically be in generation 2: make it at least 85000 bytes in size. Anything at least that size gets put into a Large Object Heap. Only generation 2 collections service that type of heap.

Want to cause a generation 2 collection? Do this:

byte[] buffer = new byte[85000];

If you want high-performance, you absolutely cannot do this per request on a server. These types of buffers, or other large objects, must be pooled. There is no built-in pooling mechanism in .Net—you must write your own. There are usually not too many large objects you’ll need to pool: strings and byte buffers are the usual suspects, if you need to do much serialization/deserialization, but also look out for collections of any type.

If you want to know more about the Large Object Heap and why 85000 bytes is the threshold, read this great article: Large Object Heap Uncovered.

Pooling collection objects comes with its own set of challenges:

  • You can’t assume the full collection is valid (the difference between length and capacity). If you use pooled arrays, for example, you have to track the length separately, since only a small portion of the array may be valid. This can drastically affect the interfaces between components.
  • Pooled collections that can grow over time will cause your memory to rise indefinitely unless you put limits on the size of the pool and/or the size of collections within the pool.
  • Large Object Heaps are not compacted during collection, which means that you can fragment the heap such that it’s wasting a lot of memory. It all depends on your allocation and collection pattern. I may talk about heap fragmentation in another article.

Once you solve those, you’re good to go… no more generation 2 collections!

Next Time…

In my next article, I’ll cover tools you can use to measure garbage collection statistics, and how you can use that knowledge to improve your performance.


Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code by Ben Watson. Available now in print and as an eBook at:

Alternative to Double-Checked Locking: Lazy<T>

A common pattern for lazy-initialization of a possibly-expensive type is double-checked locking.

The pattern is essentially this:

// member field
private volatile MyType myObject = null;
private object myLock = new object();

public MyType MyObject
{
    get
    {
        if (myObject == null)
        {
            lock(myLock)
            {
                if (myObject == null)
                {
                    myObject = new MyType();
                }
            }
        }
    }   
}

It’s quite common, and it’s also common to get it subtly wrong (not using volatile, for example).

However, in .Net 4, there’s a simpler way to do this: use Lazy<T>. This type uses a delegate to initialize the value with double-checked locking (there are other possible thread safety modes as well).

private Lazy<MyType> myObject = new Lazy<MyType>( () => new MyType );

public MyType MyObject
{
    get
    {
        return myObject.Value;
    }   
}

With Lazy<T>, the delegate you pass in will not be called until the Value property is accessed. Depending on the options you set, only a single thread will call your delegate and create the real type. I’d much rather do this than have to look at double-checked locking patterns scattered throughout a code base.

Like this tip? Check out my book, C # 4.0 How-To, where you’ll finds hundreds of great tips like this one.


Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code by Ben Watson. Available now in print and as an eBook at:

Use Appropriate Collections

.Net makes using fancy collections very easy. In fact, it can be almost too easy. It is so simple to just throw in a List<T> or a ConcurrentDictionary<K, T> that it’s tempting to do it at every opportunity.

Today’s tip is to stop and think critically about the type of collections you need.

Some examples:

  • I was doing a code review recently and saw that this person was using Dictionary<string, bool> where every value was true. This is not a dictionary—it’s a set with complicated accessors. So use HashSet<T> – it has a simpler API, which will lead to your own code being simpler and more correct semantically. Dictionary<K, T> has a meaning, and if you’re abusing that meaning, then your code is likely incorrect, or in the best case, misleading (and thus a maintenance problem, and thus incorrect!).
  • Performance characteristics are often non-intuitive. For example, if you’re doing a lot of searching for values, which would you use? Dictionary<K, T> or List<T>? Most people would say the Dictionary, and perhaps in 95% of cases, they’re right. On the other hand, if you only have a few values, you may get better performance because of cache locality with a binary search over a List (or even a linear search), which is usually implemented as an array.
  • Speaking of arrays, if you have a read-only vector of a known size, use arrays instead of Lists. It’s semantically more correct, simpler, and usually more performant.
  • Another code review I was doing had usage of ConcurrentDictionary<K, T>. This sounds like a great type to use when you need to modify and/or read a dictionary with multiple threads, but the usage of this type is not that straightforward, and in fact the official documentation is unclear on some things. In this case, it was better to redesign at a higher level to avoid use of this type.
  • I’ve seen this type of code in reviews a few times:
var list = new List<MyObject>();
for (int i =0;i < source.Count; ++i)
{
    list.Add(source.Get(i));
}

        In most apps maybe this doesn’t matter. If you care about performance, then you should care that there are going to be an unknown number of pointless memory allocations, plus multiple copies of the old data into the new arrays. In a high-performance system, this matters. Use the constructor of the collection that takes an initial size. (Guess how many items are in the default List<T> internal item array: None.)

Some questions you can ask yourself:

  • Is the collection semantically correct? Or am I abusing it because it can do what I want? Do I have restrictions on the use of the collection that are not obeyed by the API of the collection? Is there a more appropriate data structure I can use? If not, should I wrap this collection into an API with suitable restrictions?
  • Am I using the simplest collection possible?
  • Is this collection type performant for how it will be used? How can I measure to make sure? Am I making assumptions that may be unfounded because of usage patterns or hardware optimizations? Am I initializing the data structure correctly, to avoid unnecessary memory reallocations?
  • Is the collection type I’m considering too complex to use effectively? Would it be better to redesign something at a higher level to avoid needing to use this collection type?

Collections are usually the core of any application (if you don’t have data, what you are acting upon?). Getting these right means simpler code, better performance, higher readability, and long-term maintainability. It doesn’t take any more work (usually) – it just takes a few moments to think about what you’re really doing.

Like this tip? Check out my book, C # 4.0 How-To, where you’ll finds hundreds of great tips like this one.


Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code by Ben Watson. Available now in print and as an eBook at: