As you should already understood by now: rule #1 when working with exceptions in DLLs is "never let exception escape DLL". That because caller may not know how to work with exception object generated by different programming language. For example, a Delphi .exe file have no idea to to read exception message from Microsoft C++ exception; it doesn't know how to properly release exception object after exception is handled. Therefore, all exceptions in DLL functions must be captured and handled by translating them to error code or other error signature as required by DLL API.
How this should be done? That highly depends on what your DLL API is. This also depends on what framework you do use. There are 3 possible cases:
See also: Creating bug reports for DLL exceptions.
Note: if you develop DLL that will only be used in executable compiled in exactly the same version of the compiler, and you want to pass exceptions between modules for simplicity - see this article.