Tag Archives: debugging

Short vs. Long Weak References and Object Resurrection

Last time, I talked about the basics of using WeakReference, what they meant and how the CLR treats them. Today in part 2, I’ll discuss some important subtleties. Part 3 of this series can be found here.

Short vs. Long Weak References

First, there are two types of weak references in the CLR:

  • Short – Once the object is reclaimed by garbage collection, the reference is set to null. All of the examples in the previous article, with WeakReference and WeakReference<T>, were examples of short weak references.
  • Long – If the object has a finalizer AND the reference is created with the correct options, then the reference will point to the object until the finalizer completes.

Short weak references are fairly easy to understand. Once the garbage collection happens and the object has been collected, the reference gets set to null, the end. A short weak reference can only be in one of two states: alive or collected.

Using long weak references is more complicated because the object can be in one of three states:

  1. Object is still fully alive (has not been promoted or garbage collected).
  2. Object has been promoted and the finalizer has been queued to run, but has not yet run.
  3. The object has been cleaned up fully and collected.

With long weak references, you can retrieve a reference to the object during stages 1 and 2. Stage 1 is the same as with short weak references, but stage 2 is tricky. Now the object is in a possibly undefined state. Garbage collection has started, and as soon as the finalizer thread starts running pending finalizers, the object will be cleaned up. This can happen at any time, so using the object is very tricky. The weak reference to the target object remains non-null until the target object’s finalizer completes.

To create a long weak reference, use this constructor:

WeakReference<MyObject> myRefWeakLong 
    = new WeakReference<MyObject>(new MyObject(), true);

The true argument specifies that you want to track resurrection. That’s a new term and it is the whole point of long weak references.

Aside: Resurrection

First, let me say this up front: Don’t do this. You don’t need it. Don’t try it. You’ll see why. I don’t know if there is a special reason why resurrection is allowed in .NET, or it’s just a natural consequence of how garbage collection works, but there is no good reason to do something like this.

So here’s what not to do:

class Program
    class MyObject
        myObj = this;

    static MyObject myObj = new MyObject();

    static void Main(string[] args)
        myObj = null;

By setting the myObj reference back to an object, you are resurrecting that object. This is bad for a number of reasons:

  • You can only resurrect an object once. Because the object has already been promoted to gen 1 by the garbage collector, it has a guaranteed limited lifetime.
  • The finalizer will not run again, unless you call GC.ReRegisterForFinalize() on the object.
  • The state of the object can be indeterminate. Objects with native resources will have released those resources and they will need to be reinitialized. It can be tricky picking this apart.
  • Any objects that the resurrected object refers to will also be resurrected. If those objects have finalizers they will also have run, leaving you in a questionable state.

So why is this even possible? Some languages consider this a bug, and you should to. Some people use this technique for object pooling, but this is a particularly complex way of doing it, and there are many better ways. You should probably consider object resurrection a bug as well. If you do happen upon a legitimate use case for this, you should be able to fully justify it enough to override all of the objections here.

Weak vs. Strong vs. Finalizer Behavior

There are two dimensions for specifying a WeakReference<T>: the weak reference’s creation parameters and whether the object has a finalizer. The WeakReference’s behavior based on these is described in this table:

  No finalizer Has finalizer
trackResurrection = false short short
trackResurrection = true short long

An interesting case that isn’t explicitly specified in the documentation is when trackResurrection is false, but the object does have a finalizer. When does the WeakReference get set to null? Well, it follows the rules for short weak references and is set to null when the garbage collection happens. Yes, the object does get promoted to gen 1 and the finalizer gets put into the queue. The finalizer can even resurrect the object if it wants, but the point is that the WeakReference isn’t tracking it–because that’s what you said when you created it. WeakReference’s creation parameters do not affect how the garbage collector treats the target object, only what happens to the WeakReference.

You can see this in practice with the following code:

class MyObjectWithFinalizer 
        var target = myRefLong.Target as MyObjectWithFinalizer; 
        Console.WriteLine("In finalizer. target == {0}", 
            target == null ? "null" : "non-null"); 

static WeakReference myRefLong = 
    new WeakReference(new MyObjectWithFinalizer(), true); 

static void Main(string[] args) 
    MyObjectWithFinalizer myObj2 = myRefLong.Target 
          as MyObjectWithFinalizer; 
    Console.WriteLine("myObj2 == {0}", 
          myObj2 == null ? "null" : "non-null"); 
    myObj2 = myRefLong.Target as MyObjectWithFinalizer; 
    Console.WriteLine("myObj2 == {0}", 
         myObj2 == null ? "null" : "non-null"); 

The output is:

myObj2 == non-null 
In finalizer. target == non-null 
myObj2 == null 

Finding Weak References in a Debugger

Windbg can show you how to find where your weak references, both short and long.

Here is some sample code to show you what’s going on:

using System; 
using System.Diagnostics; 

namespace WeakReferenceTest 
    class Program 
        class MyObject 

        static void Main(string[] args) 
            var strongRef = new MyObject(); 
            WeakReference<MyObject> weakRef = 
                new WeakReference<MyObject>(strongRef, trackResurrection: false); 
            strongRef = null; 



            MyObject retrievedRef; 

            // Following exists to prevent the weak references themselves 
            // from being collected before the debugger breaks 
            if (weakRef.TryGetTarget(out retrievedRef)) 

Compile this program in Release mode.

In Windbg, do the following:

  1. Ctrl+E to execute. Browse to the compiled program and open it.
  2. Run command: sxe ld clrjit (this tells the debugger to break when the clrjit.dll file is loaded, which you need before you can execute .loadby)
  3. Run command: g
  4. Run command .loadby sos clr
  5. Run command: g
  6. The program should now break at the Debugger.Break() method.
  7. Run command !gchandles

You should output similar to this:

0:000> !gchandles
  Handle Type          Object     Size     Data Type
011112f4 WeakShort   02d324b4       12          WeakReferenceTest.Program+MyObject
011111d4 Strong      02d31d70       36          System.Security.PermissionSet
011111d8 Strong      02d31238       28          System.SharedStatics
011111dc Strong      02d311c8       84          System.Threading.ThreadAbortException
011111e0 Strong      02d31174       84          System.Threading.ThreadAbortException
011111e4 Strong      02d31120       84          System.ExecutionEngineException
011111e8 Strong      02d310cc       84          System.StackOverflowException
011111ec Strong      02d31078       84          System.OutOfMemoryException
011111f0 Strong      02d31024       84          System.Exception
011111fc Strong      02d3142c      112          System.AppDomain
011113ec Pinned      03d333a8     8176          System.Object[]
011113f0 Pinned      03d32398     4096          System.Object[]
011113f4 Pinned      03d32178      528          System.Object[]
011113f8 Pinned      02d3121c       12          System.Object
011113fc Pinned      03d31020     4424          System.Object[]

      MT    Count    TotalSize Class Name
70e72554        1           12 System.Object
01143814        1           12 WeakReferenceTest.Program+MyObject
70e725a8        1           28 System.SharedStatics
70e72f0c        1           36 System.Security.PermissionSet
70e724d8        1           84 System.ExecutionEngineException
70e72494        1           84 System.StackOverflowException
70e72450        1           84 System.OutOfMemoryException
70e722fc        1           84 System.Exception
70e72624        1          112 System.AppDomain
70e7251c        2          168 System.Threading.ThreadAbortException
70e35738        4        17224 System.Object[]
Total 15 objects

    Strong Handles:       9
    Pinned Handles:       5
    Weak Short Handles:   1

The weak short reference is called a “Weak Short Handle” in this output.

Next Time

The first article explained how WeakReference works, and this one explained a few of the subtleties, including some behavior you probably don’t want to use. Next time, I’ll go into why you would want to use WeakReference in the first place, and provide a sample application.

Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code, 2nd Edition by Ben Watson. Available for pre-order:

Using Windbg to answer implementation questions for yourself (Can a Delegate Invocation be Inlined?)

The other day, a colleague of mine asked me: Can a generated delegate be inlined? Or something similar to this. My answer was that the generated code is going to be JITted and optimized like any other code, but later I started thinking…. “Wait a sec, can the actual call to the delegate be inlined?”

I’m going to give you the answer before I even start this article: no.

I cover the rules of method inlining that the JITter uses in my book, Writing High-Performance .NET Code, but I don’t discuss this specific situation. You could logically make the leap, however, that there are two other rules that imply this:

  • Virtual methods will not be inlined
  • Interface calls with multiple concrete implementations in a single call site will not be inlined.

While neither of those rules are delegate-specific, you can infer that a delegate call might have similar constraints. You could ask around on the Internet. Somebody on stackoverflow.com will surely answer you, but I want to show you how to find out the answer to this for yourself, which is an invaluable skill for harder questions, where you might not be able to find out the answer unless you know people on the CLR team (which I do, but I *still* try to find out answers before I bother them).

First, let’s see a test program that will exercise various types of function calls, starting with a simple method call that we would expect to be inlined.

using System;
using System.Runtime.CompilerServices;

namespace DelegateInlining
    class Program
        static void Main(string[] args)
        private static int Add(int x, int y) { return x + y; }

        private static void TestNormalFunction()
            int z = Add(1, 2);

The code we’re interested in inlining is the Add method. Don’t confuse that with the NoInlining option on TestNormalFunction, which is there to prevent the test method itself from being inlined The test method is there to allow breakpoint setting and debugging.

Build this code in Release mode for x86. Then open Windbg.

If you’re not used to using Windbg, I highly encourage you to start. It is far more powerful than Visual Studio’s debugger, especially when it comes to debugging the details of .NET. It is not strictly necessary for this particular exercise, but it is what I recommend.

To get, Windbg, install the Windows SDK—there is the option to install only the debugger if you wish.

In Windbg:

  1. Ctrl-E to open an executable program. Navigate to and open the Release build of the above program. It will start executing and immediately break
  2. Type the command: sxe ld clr. What we want to do is set a breakpoint inside the TestNormalFunction. To do that, we need to use the SOS debugger extension, which relies on clrjit.dll, which hasn’t been loaded in the process yet. So the first thing to do is set a breakpoint on loading clrjit.dll: sxe ld clrjit
  3. Enter the command g for “go” (or hit F5). The program will then break on the load of clrjit.dll.
  4. Enter the command .loadby sos clr – this will load the SOS debugging helper.
  5. Enter the command !bpmd DelegateInlining Program.TestNormalFunction – this will set a managed breakpoint on this method.
  6. Enter the command g to continue execution. Execution will break when it enters TestNormalFunction.
  7. Now you can see the disassembly for this method (menu View | Dissassembly).
00b80068 55              push    ebp
00b80069 8bec            mov     ebp,esp
00b8006b e8e8011b70      call    mscorlib_ni+0x340258 (70d30258)
00b80070 8bc8            mov     ecx,eax
00b80072 ba03000000      mov     edx,3
00b80077 8b01            mov     eax,dword ptr [ecx]
00b80079 8b4038          mov     eax,dword ptr [eax+38h]
00b8007c ff5014          call    dword ptr [eax+14h]
00b8007f 5d              pop     ebp
00b80080 c3              ret

There are some calls there, but none of them are to Add—they are all functions inside of mscorlib. The call to the dword ptr is virtual function call. These are all related to calling Console.WriteLine.

The key is the instruction at address 00b80072, which moves the value 3 directly into register edx. This is the inlined Add call. The compiler inlined not only the function call, but the trivial math as well (an easy optimization the compiler will do for constants).

So far so good. Now let’s look at the same type of thing through a delegate.

delegate int DoOp(int x, int y);

private static void TestDelegate()
    DoOp op = Add;
    int z = op(1, 2);

Change the Main method above to call TestDelegate instead. Follow the same steps given previously for Windbg, but this time set a breakpoint on TestDelegate.

00610077 42              inc     edx
00610078 00e8            add     al,ch
0061007a 8220d0          and     byte ptr [eax],0D0h
0061007d ff8bc88d5104    dec     dword ptr [ebx+4518DC8h]
00610083 e8481b5671      call    clr!JIT_WriteBarrierECX (71b71bd0)
00610088 c7410cc4053304  mov     dword ptr [ecx+0Ch],43305C4h
0061008f b870c04200      mov     eax,42C070h
00610094 894110          mov     dword ptr [ecx+10h],eax
00610097 6a02            push    2
00610099 ba01000000      mov     edx,1
0061009e 8b410c          mov     eax,dword ptr [ecx+0Ch]
006100a1 8b4904          mov     ecx,dword ptr [ecx+4]
006100a4 ffd0            call    eax
006100a6 8bf0            mov     esi,eax
006100a8 e8ab017270      call    mscorlib_ni+0x340258 (70d30258)
006100ad 8bc8            mov     ecx,eax
006100af 8bd6            mov     edx,esi
006100b1 8b01            mov     eax,dword ptr [ecx]
006100b3 8b4038          mov     eax,dword ptr [eax+38h]
006100b6 ff5014          call    dword ptr [eax+14h]
006100b9 5e              pop     esi
006100ba 5d              pop     ebp
006100bb c3              ret

Things got a bit more complicated. As you’ll read in Writing High-Performance .NET Code, assigning a method to a delegate actually results in a memory allocation. That’s fine as long that operation is cached and reused. What we’re really interested in here starts at address 00610097, where you can see the value 2 being pushed onto the stack. The next line moves the value 1 to the edx register. There are our two function arguments. Finally, at address 006100a4, we’ve got another function call, which is the call to Add, and the key to this whole thing becomes clear. The address of that function had to be retrieved via pointer, which means it’s essentially like a virtual method call for the purposes of inlining.

You can also do the same exercise with a lambda expression (it will look similar to the delegate disassembly above).

So there’s the simple answer.

There is one more interesting case: a delegate that calls into method A that calls method B. We already know that method A won’t be inlined, but can method B be inlined into method A?

private static void TestDelegateWithFunctionCall()
    DoOp op = (x, y) => Add(x, y);
    int z = op(1, 2);

You can do the same analysis as above. You will see the call into the delegate/lambda will not be inlined, but there is no further function call, so yes, Method B can be inlined.

There you have it. Even though, the answer was pretty clear from the start, you at least have the tools to answer it or yourself. Don’t be afraid of the debugger, or of looking at assembly code, even for .NET programs.

Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code, 2nd Edition by Ben Watson. Available for pre-order:

A Debugging Exercise

A few weeks ago I had an interesting debugging problem. A program we develop had a memory leak in it that Visual Studio was catching when it ended. The trace text was something like this:

Detected memory leaks!
Dumping objects ->
{292300} normal block at 0x05590040, 2771928 bytes long.
Data: <> FF FF FF 00 FF FF FF 00 FF FF FF 00 FF FF FF 00
{292294} normal block at 0x05110040, 2613312 bytes long.
Data: <> FF FF FF 00 FF FF FF 00 FF FF FF 00 FF FF FF 00

Visual Studio 2003 is often pretty good at telling you where a memory leak occured, giving you a source code file and line number of where the memory allocation originated. In the case above, I don’t have that. There are two relatively easy ways of solving this.

The first method is to examine the data that isn’t being freed. In this case, I have two large objects of about 2.5 MB each. That’s pretty big–there aren’t too many (if any) individual objects in the software that are that large. So it could be an array. The data begins with a repeated pattern {FF FF FF 00}. Definitely looks like an array of 4-byte segments. Applying a little knowledge of the application itself–it displays large bitmaps–and we realize that the memory comes from an array of RGB values.

Another way to investigate this is to set a breakpoint on the memory allocation that isn’t being freed. The number in braces {292300} is the memory allocation number in debug mode in Visual C++. In order for this technique to work, you must first ensure that the number is the same each time. In my case, the memory leak would happen if I just opened and closed the program (because an image is always being drawn) without doing anything else.

First, set a breakpoint at the beginning of the program (or any place definitely before the memory leak). Start the program and when it breaks, enter the following into the watch window:


And for the value enter 292300.

If you continue the execution, the program will break on the 292,300th memory allocation. It will stop in the memory allocator, and you can then look at your stack frame and see exactly where the memory allocation is taking place.

MSDN has a great article about this.

In my problem it was for a set of memory blocks being used as working space for resizing and smoothing bitmaps. They were allocated once during execution and thus did not really provide a long-term threat. This leads to a possible third solution:

Ignore. All memory is reclaimed once the program exits. Nobody would ever do that, though…. would they?

…of course not…

Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code, 2nd Edition by Ben Watson. Available for pre-order:

On Fixing Computers

As the family’s computer guy, I often get asked to fix, maintain, look at, improve, or otherwise modify my family’s computers over vacations. This Christmas vacation is no exception. While I was at my brother’s house he asked me to look at his computer–remove spyware, make sure everything was running smoothly. I updated Spybot to 1.4, ran Ad Aware, and removed a bogus web search toolbar. And that’s about it. All-in-all, the computer was running smoothly as-is.

A few hours later, I got called to look at it because a new error message was being displayed–by Norton Internet Security. The anti-virus e-mail checker was disabled. I think this is what happened: It turns out that running Spybot had turned on the Windows Firewall, even though Norton had its own.

On the one hand, I probably could have read the descriptions of the problems Spybot had found and realized what it would do. On the other hand, why did Spybot find a problem when there really wasn’t one?

Computer software, including operating systems and the applications we run on them, has gotten far too complex for us to understand the implications of even simple operations.

Everytime we install an application on Windows, it makes changes to the file system, the registry, the start menu, and more. If it’s a system utility, it can modify much more.

And don’t tell me it’s any better on Linux. You have to deal with the potential of mismatched shared libraries every time an application is upgraded.

There a couple of technologies that I see as easing some of these problems:

  • .Net – xcopy deployment (for the most part), easy to use XML config files. Drop in and play for simple apps. Many Linux apps work well like this, why not Windows?
  • System change monitoring – like MS Anti-spyware, it notifies you whenever a system change occurs and lets you stop it before damage is done. What if we had a system that allows you to set what kind of changes you want to be notified about? What if it monitored a much broader set of features than today? This method presupposes a fairly good knowledge of computers. I like MS anti-spyware, but I wonder what people thing when it brings up a window asking whether to continue with a registry change, or run a script. Most people simply dismiss things like that, which is unfortunate.

Check out my latest book, the essential, in-depth guide to performance for all .NET developers:

Writing High-Performance.NET Code, 2nd Edition by Ben Watson. Available for pre-order: