Posted by: Brian de Alwis | November 16, 2009

Creating proxies for round-tripping unknown objects in C# server apps

I continue to do some support work for some C#-based groupware infrastructure called GroupLab.Networking.  This infrastructure provides a shared dictionary abstraction.  Each participant (a client) in an application creates its own local shared dictionary instance and connects it to a master dictionary (the server); updates to the dictionary are transparently sent to and from the master.  It’s fantastic for distributed data-intensive applications.

A usual deployment is to use the shipped dictionary monitor (dictmon.exe) as the master dictionary.  This monitor is a simple GUI that shows the state of the dictionary, and allows some manipulation of the dictionary.

One of the users brought up a problem recently: he wanted to use the dictionary monitor as his master dictionary, but his clients were using custom objects that were not bound into dictmon.exe. Using these objects with dictmon.exe led to serialization errors.  Since having to recompile dictmon for different applications is pretty distasteful, and providing some way to specify and dynamically load required assemblies is only slightly better, I wanted to come up with a better solution. My solution constructs proxy objects to stand in the place of these unknown objects; what’s particularly cool is that the proxy objects can themselves be serialized to restore the original object’s definition, allowing round-tripping (e.g., the server can send back a proxied object, which will be properly deserialized to the original object type).

Since this issue wasn’t particularly easy to solve, and I wasn’t able to find any previous solutions, I thought I’d take a couple of minutes to document my solution.

Serialization Binders

Since these objects are part of a larger object graph, we need some way to intercept the deserialization of each unserializable object to substitute a replacement object.

The .NET serialization framework doesn’t quite provide this level of support.  But it does provide a way to do type substitutions by providing your own SerializationBinder. These binders are generally intended for migrating older object definitions to newer versions. Jeff Richter has a set of columns in MSDN Magazine that provided a good overview of the serialization framework (1, 2, 3).

The binder seems to be called for every object instance, and is provided the object’s full type name and defining assembly name. I created my own binder that first checks to see if the assembly could be found; if so, then we defer to the normal type resolving process.  If the assembly cannot be found, then it spoofs up a proxy for that type.

internal class ProxyingBinder : SerializationBinder
{
    public override Type BindToType(string assemblyName, string typeName)
    {
        try
        {
            // if the assembly cannot be found, then Assembly.Load throws
            // an exception
            Assembly asmble = Assembly.Load(assemblyName);
            // continue the normal resolution process: returning null leads
            // to a SerializationException
            return asmble.GetType(typeName);
        }
        catch (Exception e) {/*ignore*/}
        return BuildProxyType(assemblyName, typeName);
    }
}

Unfortunately we cannot simply return single special type as the original assembly and type information is tossed by the serialization framework once the binder has been consulted. I tried several solutions to record this information, but the only workable solution was to dynamically create a class for each type that required proxying to hold on to the required information.

Dynamically Creating Proxying Classes

Thus, for each unknown class, we spoof up a proxy class. This proxy class records the unknown class’ defining assembly and full type name, as provided to the binder. The proxy class is serializable: on deserialization, it simply records the serialization information defining the unknown object; on serialization, it simply regurgitates the unknown object’s serialization information.

Dynamically creating classes and methods is a bit painful. I was fortunate to find some good tutorials. To simplify my work, I created an abstract class to provide the plumbing required for the proxied objects.

/// <summary>
/// This class provides serialization plumbing for proxied objects.
/// </summary>
[Serializable]
public abstract class ProxiedObject : ISerializable
{
    private List<string> entryNames;
    private List<object> entryValues;
    private List<Type> entryTypes;

    protected ProxiedObject(SerializationInfo info, StreamingContext context)
    {
	entryNames = new List<string>(info.MemberCount);
	entryValues = new List<object>(info.MemberCount);
	entryTypes = new List<Type>(info.MemberCount);

	SerializationInfoEnumerator enumerator = info.GetEnumerator();
	while(enumerator.MoveNext())
	{
	    entryNames.Add(enumerator.Current.Name);
	    entryValues.Add(enumerator.Current.Value);
	    entryTypes.Add(enumerator.Current.ObjectType);
	}
    }

    public void GetObjectData(SerializationInfo info, StreamingContext context)
    {
	info.FullTypeName = GetFullTypeName();
	info.AssemblyName = GetAssemblyName();
	for(int i = 0; i < entryNames.Count; i++)
	{
	    info.AddValue(entryNames[i], entryValues[i], entryTypes[i]);
	}
    }

This ProxiedObject class defines two abstract methods which are to be over-ridden by the various dynamically created subclasses to provide the unknown type’s assembly and type names.

    /// <summary>
    /// Provide the full type name of the proxied object's class.
    /// </summary>
    /// <returns>the full type name</returns>
    protected abstract String GetFullTypeName();

    /// <summary>
    /// Provide the assembly name defining the proxied object.
    /// </summary>
    /// <remarks>Will be overridden in subclasses representing the
    ///   various proxied classes.</remarks>
    /// <returns>the assembly name</returns>
    protected abstract String GetAssemblyName();

And finally provides a nice ToString():

    public override string ToString()
    {
        return String.Format("Proxy: {0}, {1}", GetFullTypeName(), GetAssemblyName()) ;
    }
}

The actual dynamic class create for the proxying types is as follows:

public class ProxyingBinder : SerializationBinder
{
    private AssemblyBuilder assemblyBuilder;
    private ModuleBuilder moduleBuilder;

    // reuse previously-created proxy types where possible
    private IDictionary proxyTypes = new Dictionary();

    private Type BuildProxyType(string assemblyName, string typeName)
    {
	String fullTypeName = String.Format("{0}, {1}", typeName, assemblyName);
	Type proxyType;
	if (proxyTypes.TryGetValue(fullTypeName, out proxyType))
	{
	    return proxyType;
	}
	if (assemblyBuilder == null)
	{
	    AssemblyName asmbleName = new AssemblyName("SDProxyAssembly");
	    assemblyBuilder = Thread.GetDomain().
		DefineDynamicAssembly(asmbleName, AssemblyBuilderAccess.Run);
	    moduleBuilder = assemblyBuilder.DefineDynamicModule("Proxies");
	}

	// provide a distinctive name for the proxy type; use the count to
	// ensure uniqueness in case different assemblies provide types with
	// the same name
	String proxyTypeName = String.Format("Proxy{0}_{1}", proxyTypes.Count.ToString(),
	    typeName.Replace('.', '_'));
	TypeBuilder typeBuilder =
	    moduleBuilder.DefineType(proxyTypeName,
		TypeAttributes.Class | TypeAttributes.AutoClass | TypeAttributes.AutoLayout
		| TypeAttributes.AnsiClass | TypeAttributes.Public
		| TypeAttributes.Serializable | TypeAttributes.BeforeFieldInit,
		typeof(ProxiedObject));

	CreateSerializationConstructor(typeBuilder);
	CreateGetAssemblyNameMethod(typeBuilder, assemblyName);
	CreateGetFullTypeNameMethod(typeBuilder, typeName);

	return proxyTypes[fullTypeName] = typeBuilder.CreateType();
	// for debugging:
	// Type t = proxyTypes[fullTypeName] = typeBuilder.CreateType();
	// asmble.Save("debug-proxy-assembly");
	// return t;
    }

    private void CreateSerializationConstructor(TypeBuilder typeBuilder)
    {
	Type[] constructorArgTypes = new Type[] { typeof(SerializationInfo), typeof(StreamingContext) };
	ConstructorInfo superConstructor = typeof(ProxiedObject).GetConstructor(
	    BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance,
	    null, CallingConventions.Standard, constructorArgTypes, new ParameterModifier[0]);

	ConstructorBuilder constructor = typeBuilder.DefineConstructor(
	    MethodAttributes.Family | MethodAttributes.HideBySig
		| MethodAttributes.SpecialName | MethodAttributes.RTSpecialName,
	    CallingConventions.Standard,
	    constructorArgTypes);
	ILGenerator il = constructor.GetILGenerator();
	il.Emit(OpCodes.Ldarg_0);
	il.Emit(OpCodes.Ldarg_1);
	il.Emit(OpCodes.Ldarg_2);
	il.Emit(OpCodes.Call, superConstructor);
	il.Emit(OpCodes.Nop);
	il.Emit(OpCodes.Nop);
	il.Emit(OpCodes.Nop);
	il.Emit(OpCodes.Ret);
    }

    private void CreateGetAssemblyNameMethod(TypeBuilder typeBuilder, string assemblyName)
    {
	MethodBuilder method = typeBuilder.DefineMethod("GetAssemblyName",
	    MethodAttributes.Family | MethodAttributes.HideBySig
		| MethodAttributes.Virtual, CallingConventions.Standard,
	    typeof(string), Type.EmptyTypes);
	ILGenerator il = method.GetILGenerator();
	LocalBuilder myLB0 = il.DeclareLocal(typeof(string));
	il.Emit(OpCodes.Nop);
	il.Emit(OpCodes.Ldstr, assemblyName);
	il.Emit(OpCodes.Stloc_0);
	Label label = il.DefineLabel();
	il.Emit(OpCodes.Br_S, label);
	il.MarkLabel(label);
	il.Emit(OpCodes.Ldloc_0);
	il.Emit(OpCodes.Ret);
    }

    private void CreateGetFullTypeNameMethod(TypeBuilder typeBuilder, string typeName)
    {
	MethodBuilder method = typeBuilder.DefineMethod("GetFullTypeName",
	    MethodAttributes.Family | MethodAttributes.HideBySig
		| MethodAttributes.Virtual, CallingConventions.Standard,
	    typeof(string), Type.EmptyTypes);
	ILGenerator il = method.GetILGenerator();
	LocalBuilder myLB0 = il.DeclareLocal(typeof(string));
	il.Emit(OpCodes.Nop);
	il.Emit(OpCodes.Ldstr, typeName);
	il.Emit(OpCodes.Stloc_0);
	Label label = il.DefineLabel();
	il.Emit(OpCodes.Br_S, label);
	il.MarkLabel(label);
	il.Emit(OpCodes.Ldloc_0);
	il.Emit(OpCodes.Ret);
    }
}

Following jconwell‘s advice, I figured out the IL by creating a sample class and then using ildasm.exe to decompile them. Unfortunately I did not find peverify.exe very useful: it missed two earliers mistakes wher I neglected to create the local string vars in the GetFullTypeName() and GetAssemblyName() methods, resulting in invalid stores. It took a bit of time for the InvalidProgramExceptions to clue in.  More useful was to use the commented-out debugging code to save the resulting assembly and examine and compare the generated methods with the sample methods using ildasm.exe.

Summary

In this post, I outlined a solution for handling unknown serialized objects. This solution is notable in that the proxied objects can be round-tripped back to their originator, or sent on to other clients that have the appropriate type definitions.

There are two outstanding problems to this solution:

  1. Garbage: These proxied types will not be garbage collected. Even if we tried to place objects in separate assemblies and used weak references, assemblies are not GC’d, though I did see some references describing the possibility of dynamic assemblies being collected in .NET 4.0.
  2. Type safety: As the binder is not provided any typing information on the unknown objects, we cannot spoof up type-compliant proxies. My solution works fine providing the proxies are held in generic object references. A SerializationException will be thrown any proxies are held in a type reference.
Advertisements

Responses

  1. Hello!

    This is a great article and it’s proving to be very useful in a de-serialization scenario I’m currently facing. However, I am getting a “Could not load file or assembly ‘SDProxyAssembly’…” in the end of the deserialization binder loop.

    Any ideas?

  2. On a sidenote, I was able to fully save and validate the dynamic assembly (peverify.exe), and Modules panel showed the assembly correctly all the way through.

    Kind regards
    http://Blog.VascoOliveira.com

    • Unfortunately it’s 2 years later and I don’t remember anything beyond what I’ve written here. :-/

      I wonder if the SDProxyAssembly is being “saved” to somewhere that doesn’t have execute permissions? Are you running from a network drive?

      • I saved just for validation purposes, to check with peverify.exe if IL generation was ok. My goal is use it virtually upon deserialization.

        So I fixed this issue by adding a handler to AppDomain.CurrentDomain.AssemblyResolve, which gets called whenever there’s a resolution of an assembly fails, and provide the SDProxyAssembly manually.

        Now I am getting another issue “Object cannot be stored in an array of this type.”… This is a painfull path.

        Either way, I’ll keep you posted on this in case you have any ideas.

  3. Brian, do you remember how you managed to get the proxied object replacement to work in case of interface arrays?

    For instance if we have an array of an unknown interface type, if we return a ProxyObject for that unknown interface, the deserialization engine fails to set the proxy object on the array since it doesn’t implement the interface.

    • The code above is what’s in use in GroupLab.Networking. I’m not sure if your situation ever came up in testing — or in practice.

      • Thanks. Sorry for digging this up again 🙂


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories