Consider this code:

BinaryFormatter bf = new BinaryFormatter();
bf.AssemblyFormat = FormatterAssemblyStyle.Simple;
ret = bf.Deserialize(memoryStream);

Setting AssemblyFormat to FormatterAssemblyStyle.Simple should mean this:
“In simple mode, the assembly used during deserialization need not match exactly the assembly used during serialization. Specifically, the version numbers need not match as the LoadWithPartialName method is used to load the assembly.”
But this is not true if the assembly is signed!

My current solution:

using (MemoryStream ms = new MemoryStream(...))
{
    var versionMismatchAssemblyResolveHandler = new ResolveEventHandler((s,ea)=>
    {
        Assembly asm = Assembly.Load(new AssemblyName(ea.Name).Name);
        return asm;
    });
 
    try
    {
        AppDomain.CurrentDomain.AssemblyResolve += versionMismatchAssemblyResolveHandler;
 
        BinaryFormatter bf = new BinaryFormatter();
        bf.AssemblyFormat = FormatterAssemblyStyle.Simple;
        ret = bf.Deserialize(ms);
    }
    finally
    {
        AppDomain.CurrentDomain.TypeResolve -= versionMismatchAssemblyResolveHandler;
    }
}

This will workaround the situation emulate the behaviour.