Sunday, October 2, 2011

Stupid BOOL return value between Win 32 (unmanaged) DLL and .NET managed application

If you are developing a C# application and you have to call a function inside a Win 32 DLL which returns a BOOL value. You will encounter an unusual problem. Like when your application works in Debug mode but dies hard in Release mode.

For example you have something like this in C++ (the unmanaged DLL)

extern "C" __cdecl bool DoWork();

Very normal stuff, eh? And you think you can do this in C#

            EntryPoint = DllConstants.ENTRY_POINT_DO_WORK,
            CallingConvention = CallingConvention.Cdecl
        private static extern bool DoWorkCpp();

Noooo, DO NOT DO IT! You know why? Because in C++, bool value is stored in 1 byte but in C# (and in ANSI C), bool is 4 bytes. You have to marshal manually using [returnMarshalAs(UnmanagedType.I1)], and it will work. Hope this helps many out there. Peace out!