Window- and Console-Friendly Win32 Applications

[This entry was updated on May 2, 2007 with additional information about limitations of this technique when used with the DOS shell, as noted in the comments. –Oliver]

It’s sometimes handy to create an application that can run either with a GUI or, if started from the command line, a command line UI. For example, you might want an installer to work this way so that it can support both GUI-driven and “silent” (command-line) installations. Windows provides a simple but not necessarily obvious way to create these dual-mode applications.

Most Windows applications run on the windows subsystem. These applications can try to write to the console, but they aren’t attached to one by default. Using this method your GUI will work just fine but any command line output will be discarded.

An application can be run on a different subsystem, of which “console” is one choice, by setting the /subsystem flag of the linker. When these applications are run they are attached to the console, and they can also open new windows. The catch is that Windows will make sure there is always a console present. If the program is run by, say, double-clicking it in Explorer then Windows will first create a new black-and-white console window, attach the application to it, and then launch the application. The application can close the console window, but not before it’s been seen by the user.

Here’s an approach that does work: Create a windows subsystem application and call AttachConsole(). If run from the Explorer there’s no console to attach to, but that’s ok–you weren’t going to use it, anyway. If your application is run from the command line then this call will attach your application to that same console–which is exactly what you want.

This technique works when running the application from bash and similar shells, from Ant scripts, and so on. However, it does not work with the DOS shell because the DOS shell will not wait for any GUI (Windows subsytem) application. You can work around this in the DOS shell using the /wait option to the start command, like so:

c:> start /wait dualmode.exe

6 Responses to Window- and Console-Friendly Win32 Applications

  1. UKCodeMonkey says:

    I’ve tried this but I found that when I run the from the console, the console isn’t blocked while the app runs as in normal console apps. Therefore even though my app can write to the console, I can type in the console and start other processes.

  2. Oliver Goldman says:

    Here’s a sample application:#define _WIN32_WINNT 0x0500#includeint __stdcall WinMain( HINSTANCE, HINSTANCE, LPSTR, int ) {AttachConsole( ATTACH_PARENT_PROCESS );// Demonstrate writing to the console::WriteConsoleW( ::GetStdHandle( STD_OUTPUT_HANDLE), L”Hello, World!\n”, 14, NULL, NULL );// Demonstrate UIMessageBoxW( NULL, L”Hello, World!”, L”Hello, World!”, MB_OK );return 0;}

  3. UKCodeMonkey says:

    Using the sample code above I see:C:\Test>HelloWorld2.exeC:\Test>Hello, World!When I should see:C:\Test>HelloWorld2.exeHello, World!C:\Test>Control is returned to the command line while the app is still running. The goal I’m after is having control returned to the console after the exe has exited, the way the program would behave had it been built as a console application rather than a windows subsystem.

  4. Oliver Goldman says:

    The DOS shell explicitly detaches itself from GUI applications. To get this technique to work with the DOS shell, you need to use the /wait option to the start command:c:\> start /wait HelloWorld.exeThis limitation does not apply to other environments, like the bash shell, running from ant build scripts, and so on.Clearly this is a significant limitation. I’ve updated the original entry to note this issue.

  5. UKCodeMonkey says:

    Hi Oliver, “start /wait …” works for me. Thanks for the post!

  6. OC says:

    start /wait is what I was looking for. Awesome.