[Xquartz-dev] Newbie question about building XQuartz: autoreconf fails

[Tom Lane]

So I'm trying to follow the server build directions at
https://www.xquartz.org/Developer-Info.html

I've checked out the git repo (since I can't find a source tarball
anywhere), but I can't get past the "autoreconf -fvi" step:

$ autoreconf -fvi
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal -I /opt/X11/share/aclocal -I /usr/local/share/aclocal --force -I m4
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/local/autoconf-2.69/bin/autoconf --force
autoreconf: running: /usr/local/autoconf-2.69/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
configure.ac:35: installing './compile'
configure.ac:41: installing './config.guess'
configure.ac:41: installing './config.sub'
configure.ac:34: installing './install-sh'
configure.ac:34: installing './missing'
Xext/Makefile.am:1: error: Libtool library used but 'LIBTOOL' is undefined
Xext/Makefile.am:1: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
Xext/Makefile.am:1: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
Xext/Makefile.am:1: If 'LT_INIT' is in 'configure.ac', make sure
Xext/Makefile.am:1: its definition is in aclocal's search path.
... lots more repetitions of that ...
autoreconf: automake failed with exit status: 1

I installed libtool but that didn't change the results.

Seems like the top-level configure.ac doesn't think it should use libtool,
but all the sub-makefiles do think that. What am I doing wrong?

regards, tom lane

[Tom Lane]

Post by Tom Lane
So I'm trying to follow the server build directions at
https://www.xquartz.org/Developer-Info.html
...
I installed libtool but that didn't change the results.

Ah: self-inflicted problem. I'd stuck libtool in a nonstandard directory
(--prefix=/usr/local/libtool-2.4.6), which meant that the recipe's
value for ACLOCAL wasn't good enough: I needed to add
"-I /usr/local/libtool-2.4.6/share/aclocal" to it.

However, that didn't get me a whole lot further. I get compile errors
that look like maybe the wrong compiler is being used: in particular
xf86Opt.h's declaration "Bool bool" fails because bool is a macro for
_bool, coming from stdbool.h. There are a lot of other scary-looking
warnings as well. This is with current Xcode (8.3.3) on up-to-date
Sierra.

Any advice would be appreciated ...

regards, tom lane

[Tom Lane]

Post by Tom Lane
Any advice would be appreciated ...

... a few hours later, I have what seems to be a working Xquartz build.
Thought I'd document what I found out. Some of this might just help
the next newbie, but some of it says Xquartz needs some work.

* The Developer-Info.html recipe implies you can just use the default
branch in the git repo, ie "master". That one doesn't build, even though
Jeremy is evidently maintaining it in parallel with the active branch,
which seems to be server-1.18-branch. So, check out the latter before
building.

* That recipe also suggests using --enable-maintainer-mode as a configure
argument, but configure doesn't recognize it. Don't know if this is a
leftover or if there's some significant version skew here.

* I recommend adding -Wno-deprecated-declarations to the CFLAGS,
else you'll get so many of those you can't see the actual warnings.
(Of which there are not many, I'm glad to see.)

* You need autoconf, automake, and libtools in addition to the tools
mentioned on Developer-Info.html. Best to let them install into
/usr/local as per default, else you'll need to adjust ACLOCAL.

* With a modern automake (I'm using 1.15.1), the autoreconf process
produces a boatload of warnings:

hw/dmx/Makefile.am:25: warning: source file '$(top_srcdir)/mi/miinitext.c' is in a subdirectory,
hw/dmx/Makefile.am:25: but option 'subdir-objects' is disabled
automake: warning: possible forward-incompatibility.
automake: At least a source file is in a subdirectory, but the 'subdir-objects'
automake: automake option hasn't been enabled. For now, the corresponding output
automake: object file(s) will be placed in the top-level directory. However,
automake: this behaviour will change in future Automake versions: they will
automake: unconditionally cause object files to be placed in the same subdirectory
automake: of the corresponding sources.
automake: You are advised to start using 'subdir-objects' option throughout your
automake: project, to avoid future incompatibilities.

Might be a good idea to fix that sometime soon.

* Building with current Xcode produces an X server that crashes at startup,
with a trace like this:

Thread 5 Crashed:
0 libsystem_kernel.dylib 0x00007fff99cb5d42 __pthread_kill + 10
1 libsystem_pthread.dylib 0x00007fff99da3457 pthread_kill + 90
2 libsystem_c.dylib 0x00007fff99c1b420 abort + 129
3 libsystem_c.dylib 0x00007fff99c1b592 abort_report_np + 181
4 libsystem_c.dylib 0x00007fff99c41f28 __chk_fail + 48
5 libsystem_c.dylib 0x00007fff99c41ef8 __chk_fail_overflow + 16
6 libsystem_c.dylib 0x00007fff99c41fe3 __strlcpy_chk + 85
7 X11.bin 0x00000001007aa74b _XSERVTransSocketReopen + 555 (Xtranssock.c:537)
8 X11.bin 0x00000001007a68d0 _XSERVTransSocketReopenCOTSServer + 128 (Xtranssock.c:777)
9 X11.bin 0x00000001007a84b5 _XSERVTransReopen + 341 (Xtrans.c:532)
10 X11.bin 0x00000001007a834e _XSERVTransReopenCOTSServer + 78 (Xtrans.c:623)
11 X11.bin 0x000000010079e045 ListenOnOpenFD + 405 (connection.c:1307)
12 X11.bin 0x00000001005c28af DarwinEventHandler + 767 (darwinEvents.c:293)
13 X11.bin 0x0000000100777198 mieqP<rocessDeviceEvent + 472 (mieq.c:559)
14 X11.bin 0x0000000100777579 mieqProcessInputEvents + 665 (mieq.c:643)
15 X11.bin 0x00000001005c2b24 ProcessInputEvents + 20 (darwinEvents.c:422)
16 X11.bin 0x000000010071022b Dispatch + 155 (dispatch.c:356)
17 X11.bin 0x000000010071b958 dix_main + 1512 (main.c:300)
18 X11.bin 0x00000001005c6c70 server_thread + 64 (quartzStartup.c:66)
19 libsystem_pthread.dylib 0x00007fff99da093b _pthread_body + 180
20 libsystem_pthread.dylib 0x00007fff99da0887 _pthread_start + 286
21 libsystem_pthread.dylib 0x00007fff99da008d thread_start + 13

I dug into it and determined that even though _XSERVTransSocketReopen is
being very careful to allocate the correct amount of memory, gcc is
deciding to generate a __strlcpy_chk() call that abort()s if the string
length is >= 14 bytes, because it thinks the declared size of struct
sockaddr.sa_data is a hard limit that it ought to enforce.

I worked around that by the expedient of adding -D_FORTIFY_SOURCE=0 to
CFLAGS, and that gave me an X server that seems to be working so far.
But that's a pretty crummy substitute for a proper fix. I wonder why
other people are not seeing this. I also wonder how the released
version of XQuartz is actually being built.

regards, tom lane

[Jeremy Huddleston Sequoia]

Post by Tom Lane

Post by Tom Lane
Any advice would be appreciated ...

... a few hours later, I have what seems to be a working Xquartz build.
Thought I'd document what I found out. Some of this might just help
the next newbie, but some of it says Xquartz needs some work.

The source for https://www.xquartz.org/Developer-Info.html is in https://github.com/XQuartz/xquartz.github.io. If you want to send a PR with some updates / modernization, I'd be happy to merge it in. I'm hoping to have some cycles next month to do an update that will roll up the latest upstream changes and pull in the focus loss patch that you tested (thanks).

Post by Tom Lane
* The Developer-Info.html recipe implies you can just use the default
branch in the git repo, ie "master". That one doesn't build, even though
Jeremy is evidently maintaining it in parallel with the active branch,
which seems to be server-1.18-branch. So, check out the latter before
building.
* That recipe also suggests using --enable-maintainer-mode as a configure
argument, but configure doesn't recognize it. Don't know if this is a
leftover or if there's some significant version skew here.

Yeah, recent versions of autotools removed that flag.

Post by Tom Lane
* I recommend adding -Wno-deprecated-declarations to the CFLAGS,
else you'll get so many of those you can't see the actual warnings.
(Of which there are not many, I'm glad to see.)
* You need autoconf, automake, and libtools in addition to the tools
mentioned on Developer-Info.html. Best to let them install into
/usr/local as per default, else you'll need to adjust ACLOCAL.
* With a modern automake (I'm using 1.15.1), the autoreconf process
hw/dmx/Makefile.am:25: warning: source file '$(top_srcdir)/mi/miinitext.c' is in a subdirectory,
hw/dmx/Makefile.am:25: but option 'subdir-objects' is disabled
automake: warning: possible forward-incompatibility.
automake: At least a source file is in a subdirectory, but the 'subdir-objects'
automake: automake option hasn't been enabled. For now, the corresponding output
automake: object file(s) will be placed in the top-level directory. However,
automake: this behaviour will change in future Automake versions: they will
automake: unconditionally cause object files to be placed in the same subdirectory
automake: of the corresponding sources.
automake: You are advised to start using 'subdir-objects' option throughout your
automake: project, to avoid future incompatibilities.
Might be a good idea to fix that sometime soon.
* Building with current Xcode produces an X server that crashes at startup,
0 libsystem_kernel.dylib 0x00007fff99cb5d42 __pthread_kill + 10
1 libsystem_pthread.dylib 0x00007fff99da3457 pthread_kill + 90
2 libsystem_c.dylib 0x00007fff99c1b420 abort + 129
3 libsystem_c.dylib 0x00007fff99c1b592 abort_report_np + 181
4 libsystem_c.dylib 0x00007fff99c41f28 __chk_fail + 48
5 libsystem_c.dylib 0x00007fff99c41ef8 __chk_fail_overflow + 16
6 libsystem_c.dylib 0x00007fff99c41fe3 __strlcpy_chk + 85
7 X11.bin 0x00000001007aa74b _XSERVTransSocketReopen + 555 (Xtranssock.c:537)
8 X11.bin 0x00000001007a68d0 _XSERVTransSocketReopenCOTSServer + 128 (Xtranssock.c:777)
9 X11.bin 0x00000001007a84b5 _XSERVTransReopen + 341 (Xtrans.c:532)
10 X11.bin 0x00000001007a834e _XSERVTransReopenCOTSServer + 78 (Xtrans.c:623)
11 X11.bin 0x000000010079e045 ListenOnOpenFD + 405 (connection.c:1307)
12 X11.bin 0x00000001005c28af DarwinEventHandler + 767 (darwinEvents.c:293)
13 X11.bin 0x0000000100777198 mieqP<rocessDeviceEvent + 472 (mieq.c:559)
14 X11.bin 0x0000000100777579 mieqProcessInputEvents + 665 (mieq.c:643)
15 X11.bin 0x00000001005c2b24 ProcessInputEvents + 20 (darwinEvents.c:422)
16 X11.bin 0x000000010071022b Dispatch + 155 (dispatch.c:356)
17 X11.bin 0x000000010071b958 dix_main + 1512 (main.c:300)
18 X11.bin 0x00000001005c6c70 server_thread + 64 (quartzStartup.c:66)
19 libsystem_pthread.dylib 0x00007fff99da093b _pthread_body + 180
20 libsystem_pthread.dylib 0x00007fff99da0887 _pthread_start + 286
21 libsystem_pthread.dylib 0x00007fff99da008d thread_start + 13
I dug into it and determined that even though _XSERVTransSocketReopen is
being very careful to allocate the correct amount of memory, gcc is
deciding to generate a __strlcpy_chk() call that abort()s if the string
length is >= 14 bytes, because it thinks the declared size of struct
sockaddr.sa_data is a hard limit that it ought to enforce.

Yeah. That's unfortunate misunderstanding of POSIX on the part of the compiler team there. Was it really FSF gcc that you're using? If so, please file a bug at https://gcc.gnu.org/bugzilla

If you're seeing that with clang, pleas let me know what version of clang. I thought we fixed clang's over-zealous __builtin_object_size before it was released.

Post by Tom Lane
I worked around that by the expedient of adding -D_FORTIFY_SOURCE=0 to
CFLAGS, and that gave me an X server that seems to be working so far.
But that's a pretty crummy substitute for a proper fix. I wonder why
other people are not seeing this. I also wonder how the released
version of XQuartz is actually being built.

It was being built with a version of clang that didn't have that bug. ;)

--Jeremy

[Tom Lane]

Post by Jeremy Huddleston Sequoia

Post by Tom Lane
I dug into it and determined that even though _XSERVTransSocketReopen is
being very careful to allocate the correct amount of memory, gcc is
deciding to generate a __strlcpy_chk() call that abort()s if the string
length is >= 14 bytes, because it thinks the declared size of struct
sockaddr.sa_data is a hard limit that it ought to enforce.

Yeah. That's unfortunate misunderstanding of POSIX on the part of the compiler team there. Was it really FSF gcc that you're using? If so, please file a bug at https://gcc.gnu.org/bugzilla

Nope ...

Post by Jeremy Huddleston Sequoia
If you're seeing that with clang, pleas let me know what version of clang. I thought we fixed clang's over-zealous __builtin_object_size before it was released.

This is current Xcode, as I said before.

$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

regards, tom lane

[Jeremy Huddleston Sequoia]

Post by Tom Lane

Post by Jeremy Huddleston Sequoia

Post by Tom Lane
I dug into it and determined that even though _XSERVTransSocketReopen is
being very careful to allocate the correct amount of memory, gcc is
deciding to generate a __strlcpy_chk() call that abort()s if the string
length is >= 14 bytes, because it thinks the declared size of struct
sockaddr.sa_data is a hard limit that it ought to enforce.

Yeah. That's unfortunate misunderstanding of POSIX on the part of the compiler team there. Was it really FSF gcc that you're using? If so, please file a bug at https://gcc.gnu.org/bugzilla

Nope ...

Post by Jeremy Huddleston Sequoia
If you're seeing that with clang, pleas let me know what version of clang. I thought we fixed clang's over-zealous __builtin_object_size before it was released.

This is current Xcode, as I said before.
$ gcc -v
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Ok, then that is clang, not gcc ;) ... gcc is just an alias for clang for folks with gcc muscle memory ;)

Here's the background on the bug: https://bugs.llvm.org/show_bug.cgi?id=30346

I know it was an issue in the OSS releases of 3.8 and 3.9, but I thought it was never in an Xcode release. Unfortunately, it looks like the bug did indeed ship in Xcode 8.3. Luckily, it's not present in Xcode 9.

--Jeremy

[Tom Lane]

Post by Jeremy Huddleston Sequoia
Here's the background on the bug: https://bugs.llvm.org/show_bug.cgi?id=30346
I know it was an issue in the OSS releases of 3.8 and 3.9, but I thought it was never in an Xcode release. Unfortunately, it looks like the bug did indeed ship in Xcode 8.3. Luckily, it's not present in Xcode 9.

Ah. Can you say when that's going to be available to the great unwashed?

regards, tom lane

[Jeremy Huddleston Sequoia]

Post by Tom Lane

Post by Jeremy Huddleston Sequoia
Here's the background on the bug: https://bugs.llvm.org/show_bug.cgi?id=30346
I know it was an issue in the OSS releases of 3.8 and 3.9, but I thought it was never in an Xcode release. Unfortunately, it looks like the bug did indeed ship in Xcode 8.3. Luckily, it's not present in Xcode 9.

Ah. Can you say when that's going to be available to the great unwashed?

Xcode 9 Beta 3 is available for download on developer.apple.com right now. I don't know when the final release will be.

Post by Tom Lane
regards, tom lane