Building Cygwin GCC 4.5.3 with --enable-plugin

Many people have tried building GCC 4.5.3 on Cygwin with the configure flag of --enable-plugin only to find that make subsequently fails with the following errors.
checking for -rdynamic... objdump: conftest: not a dynamic object no
checking for library containing dlopen... none required
checking for -fPIC -shared... no
configure: error:
Building GCC with plugin support requires a host that supports
-fPIC, -shared, -ldl and -rdynamic.
make[1]: *** [configure-gcc] Error 1

It appears that GCC would not build with --enable-plugin on Cygwin as -rdynamic and -fPIC are not applicable for a Windows PE.

Why would it need -rdynamic and -fPIC? I honestly thought those requirements are purely artificial as it would be absolutely possible to create a DLL under Windows and there is no reason why GCC can't use it.

So I set about finding out how this could be done and eventually found two articles [1][2] that allowed me to get to a working solution. I've combined the two articles to create these procedures.

Note: Start the following steps from a directory where you will have all your GCC files location in it.

1) Download and extract helper files I've put together which you'll be using later.

# obtain Cygwin GCC Plugin helper files
wget http://www.zachsaw.com/downloads/cygwin_gcc_plugin/cygwin_gcc_plugin_files.tar.bz2
tar -jxvf cygwin_gcc_plugin_files.tar.bz2

2) Run the following commands

# obtain GCC 4.5.3 (see http://gcc.gnu.org/mirrors.html for alternate mirrors)
wget http://www.netgull.com/gcc/releases/gcc-4.5.3/gcc-4.5.3.tar.bz2
tar -jxvf gcc-4.5.3.tar.bz2

# apply patches
cd gcc-4.5.3
patch -p0 < ../cygwin_gcc_plugin/finish_decl.diff
patch -p0 < ../cygwin_gcc_plugin/cygwin_enable_plugin.diff
patch -p0 < ../cygwin_gcc_plugin/cygwin_enable_plugin_cc1.diff
patch -p0 < ../cygwin_gcc_plugin/cygwin_enable_plugin_cc1plus.diff

# build GCC
cd ..
mkdir gcc-objdir
mkdir gcc-dist

cd gcc-objdir

../gcc-4.5.3/configure --disable-bootstrap --enable-version-specific-runtime-libs --enable-static --enable-shared --enable-shared-libgcc --disable-__cxa_atexit --with-dwarf2 --disable-sjlj-exceptions --enable-languages=c,c++,lto --enable-lto --enable-libssp --enable-plugin --enable-threads=posix --prefix=$PWD/../gcc-dist

make
make install
cd ..

*** make took 2 hours on my machine!

3) Create symlink for g++-plugin and gcc-plugin in /usr/bin to point to the newly built g++ and gcc respectively.

4) Set environment vars for $GCC_PLUGIN_DIR and $builtgcc.

export GCC_PLUGIN_DIR=`g++-plugin -print-file-name=plugin`
export builtgcc=<path>/gcc-objdir

5) Generate import libraries.

mkdir $GCC_PLUGIN_DIR/lib
cygwin_gcc_plugin/lazyimp.pl $GCC_PLUGIN_DIR/lib/libcc1.a $builtgcc/gcc/cc1.def
cygwin_gcc_plugin/lazyimp.pl $GCC_PLUGIN_DIR/lib/libcc1plus.a $builtgcc/gcc/cc1plus.def  

This will take some time.

6) Compile dll.c and archive it into libcc1.a and libcc1plus.a

gcc-plugin -O3 -Wall cygwin_gcc_plugin/dll.c -c 
ar rcs $GCC_PLUGIN_DIR/lib/libcc1.a dll.o
ar rcs $GCC_PLUGIN_DIR/lib/libcc1plus.a dll.o 

7) You now have both libcc1.a (for C) and libcc1plus.a (for C++) that you can link to when you build your gcc plugin.


To build a plugin:
g++-plugin -c -I`g++-plugin -print-file-name=plugin`/include main.cpp
g++-plugin -fPIC -shared -Wl,-e,_lazymain@12 -L`g++-plugin -print-file-name=plugin`/lib -o plugin.dll main.o -lcc1plus

Or one step build and link:
g++-plugin -shared -Wl,-e,_lazymain@12 -I`g++-plugin -print-file-name=plugin`/include -L`g++-plugin -print-file-name=plugin`/lib -o plugin.dll main.cpp -lcc1plus

Test the plugin:
g++-plugin -S -fplugin=./plugin.dll test.cpp

GCC Linker - Library Flag Order

My recent ventures into the Linux world has brought about some experiences I have not encountered before in the Microsoft world.

One in particular is the behavior of GCC linker in treating the library flag parameters. It appears that the order of which is of high importance. Get it wrong and it would be as if the flag was never specified!

For example,
    g++ -shared -lfoo -o plugin.dll main.o

In the above example, main.o depends on libfoo.a. However, as main.o is specified *after* -lfoo, g++ will fail to find the necessary dependencies. Rather than keeping a list of all exported symbols as it encounters them, the linker throws the symbols away as soon as it finishes processing the module if nothing before it requires them. In the case of the example, libfoo.a does not depend on main.o -- it is the other way around. So, all of libfoo.a's export symbols get thrown away!

I do not even want to start postulating the reason behind this design, but I would imagine it had something to do with memory usage. Imagine if you'd done it the other way around (which MSVC and Borland both do), you'd end up with a huge export table consisting of all the export symbols from every single module. More than likely the linker would simply fail with an out of memory error (prime example: the notoriously problematic ilink32.exe).

So, some things are upside down in the Linux world (or you could argue vice versa for the Microsoft world depending on your POV) and it does take a little getting used to.

The correct linker command line for the above example? Here it is.
    g++ -shared -o plugin.dll main.o -lfoo

Multithreading under Cygwin

A while ago, I found a nasty Cygwin bug that prevents all multithreaded applications from working properly under multi-core/thread CPUs. This includes any CPUs with more than one logical core as seen by Windows (e.g. Dual / Quad core CPUs with or without Hyperthreading as well as Single core CPUs with Hyperthreading). It's such a fundamental flaw with Cygwin that virtually renders all multithreaded applications useless since most PCs these days have at least 2 logical cores (8 is common).

Multi logical core CPUs are nothing new. The first Hyperthreaded CPU was a Pentium 4 Northwood (all Northwood CPUs had Hyperthreading although it was only the later steppings that Intel marketting decided to enable the feature for the consumer market) and that dates back to January 2002. That's more than 10 years the Cygwin bug went unnoticed!

How did a Cygwin bug as enormous as that survive 10 years in the wild, still without a fix in sight?

I see a few possibilities.

1) No one does anything of production quality under Cygwin (at least not multithreaded ones).
2) Someone noticed it but just thought Cygwin's naturally slow due to its "emulation" layer.
3) Someone noticed it and reported it but no one cared.
4) Someone noticed it and did not have the technical know-how to root cause it to a Cygwin issue.

Then again, it's not really surprising. Microsoft Windows 64-bit OSes have a WOW64 bug that plagued them since XP and is still not fixed.

The workaround

While there's no way to get a multithreaded app to fully utilize a multicore processor, you could at least get it to run as fast as on a single core processor. To do this, you would have to set your CPU affinity for the Cygwin process to a single logical CPU.

You could do this via the Task Manager - right-click on an Image Name under the Processes tab and select Set Affinity. If you need to start the Cygwin process with it bound to a single logical core from the very beginning, you could use the Start command. It has an affinity option (/AFFINITY 1). However, this only works in Windows Vista and later. You would need to use Microsoft Sysinternal's psexec on WinXP.