Adding a simple unit test to a Rails application to verify that a generated XML file conforms to the schema it declares should be simple enough. Sure, Ruby and XML used to be an odd couple, but with recent versions of libxml-ruby, the language has gained a nice set of bindings to one of the more popular native XML libraries, libxml2. This library provides access to fast XML parsing, manipulation, and validation. For my test, all I needed was a to read the generated XML file and then validate it against a schema. It really should be just a quick two-liner. And it was. Only — it didn’t quite work as I had hoped for.

I installed the libxml-ruby gem version 0.9.8 on my Mac, wrote the test … and was horrified to see that seemingly any validation error detected by the schema validator resulted in a Bus Error that brought down the entire Ruby VM, and not the nice error message I had hoped for.

To isolate the problem from the Rails environment, I wrote a small validation script, along with a toy schema and XML document to replicate the problem in a simpler setting. First the script, validate.rb:

require 'rubygems'
require 'libxml'
 
include LibXML
 
if ARGV.length < 2
 puts 'Wrong number of arguments'
 puts 'ruby validate.rb path/to/schema/file path/to/xml/file...'
 exit 1
end
 
puts "Using libxml version #{XML::LIBXML_VERSION}"
 
schema = XML::Schema.document(XML::Document.file(ARGV.shift))
 
ARGV.each do | xml_file_name |
 instance = XML::Document.file(xml_file_name)
 result = instance.validate_schema(schema)
 puts "#{xml_file_name} is #{result ? 'VALID' : 'INVALID'}"
end

And then the schema (which, obviously, turns out to be a great deal simpler than the one used by the application “back in reality”):

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema
 targetNamespace="http://not.important/status"
 xmlns:tns="http://not.important/status"
 xmlns:xs="http://www.w3.org/2001/XMLSchema"
 elementFormDefault="qualified"
 attributeFormDefault="unqualified">
 
 <xs:complexType name="Release">
   <xs:sequence>
     <xs:element name="name" type="xs:string"/>
     <xs:element name="state" type="xs:string"/>
   </xs:sequence>
 </xs:complexType>
 
 <xs:element name="release" type="tns:Release"/>
</xs:schema>

The small test document which triggers a validation error looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<release xmlns="http://not.important/status">
<Name>release</Name>
<state>Ready</state>
</release>

Note the wrong case on the <name> element. Running validate.rb against the schema and sample document replicated the problem nicely:

$ruby validate.rb schema.xsd sample1.xml
Using libxml version 2.6.16
validate.rb:18: [BUG] Segmentation fault
ruby 1.8.6 (2008-03-03) [universal-darwin9.0]
 
Abort trap

Not pretty. Not pretty at all. I repeated the test on an Ubuntu machine with no problems whatsoever, so the problem looked as though it was confined to my Mac, or maybe OS X in general. I posted a message on the libxml-ruby mailing list and got a response from Norhitio Yamakawa stating that the test ran without problems on his Mac running Mac OS X 10.4.11 and libxml2 version 2.7.2. That last piece of information got my attention. My Ubuntu machine was also at 2.7.x, so the problem was very likely caused by the older 2.6.16 my Mac was using.

What confused me, though, was that I had originally installed libxml2 using MacPorts, and port insisted that I was using 2.7.3:

$port list libxml2
libxml2                        @2.7.3          textproc/libxml2

2.6.16 is the version installed along with Leopard 10.5.6, so the problem seemed to be that the native build step of the libxml-ruby gem picked up the wrong libxml2 library. This should be easily fixed with a –build-flag telling Gem to look elsewhere:

gem install -r libxml-ruby -- --build-flags --with-opt-lib=/opt/local/lib
Building native extensions.  This could take a while...
ERROR:  Error installing libxml-ruby:
	ERROR: Failed to build gem native extension.
 
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby extconf.rb install -r libxml-ruby -- --build-flags --with-opt-lib=/opt/local/lib
checking for socket() in -lsocket... no
checking for gethostbyname() in -lnsl... no
checking for atan() in -lm... no
checking for atan() in -lm... yes
checking for inflate() in -lz... no
checking for inflate() in -lzlib... no
checking for inflate() in -lzlib1... no
 extconf failure: need zlib
 
Gem files will remain installed in /Library/Ruby/Gems/1.8/gems/libxml-ruby-0.9.8 for inspection.
Results logged to /Library/Ruby/Gems/1.8/gems/libxml-ruby-0.9.8/ext/libxml/gem_make.out

Not what I had hoped for, but maybe I was just short on zlib:

$port list zlib
zlib                           @1.2.3          archivers/zlib

Not so. Why was the build step failing when the library it reported missing was clearly already installed? gem_make.out didn’t provide many clues, but mkmf.log inside the gem folder (/Library/Ruby/Gems/1.8/gems/libxml-ruby-0.9.8/ext/libxml/mkmf.log on my system, if you must know) did. There I found:

"gcc -o conftest -I. -I/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/universal-darwin9.0 -I.  -arch ppc -arch i386 -Os -pipe -fno-common conftest.c  -L"." -L"/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib" -L"/opt/local/lib" -L"/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib" -L. -arch ppc -arch i386    -lm  -lruby -lz -lm  -lpthread -ldl -lm  "
ld warning: in /opt/local/lib/libz.dylib, file is not of required architecture
Undefined symbols for architecture ppc:
  "_inflate", referenced from:
      _t in ccgCKe7W.o
ld: symbol(s) not found for architecture ppc
collect2: ld returned 1 exit status
lipo: can't open input file: /var/tmp//ccfKVgnE.out (No such file or directory)
checked program was:
/* begin */
1: /*top*/
2: int main() { return 0; }
3: int t() { inflate(); return 0; }
/* end */

So, the reason that the compilation failed was that the installed MacPort library wasn’t a Universal Binary and so couldn’t be used to built one either — that’s essentially what the linker is telling us:

ld: symbol(s) not found for architecture ppc

What is perhaps slightly baffling to me is why it would try to create a Universal Binary in the first place — after all, the Gem is only going to be used on the machine it is compiled and installed on. And short of any quantum mechanics weirdness, I just don’t see that machine change gender from Intel to PCC all of a sudden.

Update (2009.03.07) See Charlies comment for an explanation of why rbconfig tries to build the gem as a Universal binary.

I decided I could live with an Intel-only version of the gem and used the ARCHFLAGS environment variable to tell the C/C++ compiler which architectures to compile for:

sudo bash
ARCHFLAGS='-arch i386' gem install -r libxml-ruby -- --build-flags --with-opt-lib=/opt/local/lib

Success! The Gem built and installed as expected and a test run of the script showed that the Bus Error had left the building:

$ruby validate.rb schema.xsd sample1.xml
Using libxml version 2.6.16
Error: Element '{http://not.important/status}Name': This element is not expected. Expected is ( {http://not.important/status}name ). at sample1.xml:3.
validate.rb:18:in `validate_schema': Error: Element '{http://not.important/status}Name': This element is not expected. Expected is ( {http://not.important/status}name ). at sample1.xml:3. (LibXML::XML::Error)
	from validate.rb:18
	from validate.rb:16:in `each'
	from validate.rb:16

Much better, but still a little strange — why was the script still reporting libxml to be at version 2.6.16. Well, it turned out that getting libxml-ruby to link its native parts against the right version of libxml2 was just the beginning. The build process was still picking up the header files from the OS version.

In principle, getting the build system to use the right version of the header files should be simple enough — you just have to make sure the location you want it to read header files from is placed first on the compiler command line. But that little issue turned out to be harder to solve than I had hoped for — I never quite succeeded in finding the Right Way(tm) to accomplish this and instead ended up abusing the ARCHFLAGS variable a little:

sudo bash
ARCHFLAGS='-arch i386 -I /opt/local/include/libxml2' gem install -r libxml-ruby -- --build-flags --with-opt-lib=/opt/local/lib

It just so happens that the ARCHFLAGS variable is expanded as the very first thing on the compiler command line by rbconfig.rb (the script that actually generates the Makefile used by Gem), effectively allowing me to place my preferred header include directory ahead of the ones added by the build system (and yes, dear Watson – they point to the older OS version).

And the result? Success!

$ruby validate.rb schema.xsd sample1.xml
Using libxml version 2.7.2
Error: Element '{http://not.important/status}Name': This element is not expected. Expected is ( {http://not.important/status}name ). at sample1.xml:3.
validate.rb:18:in `validate_schema': Error: Element '{http://not.important/status}Name': This element is not expected. Expected is ( {http://not.important/status}name ). at sample1.xml:3. (LibXML::XML::Error)
	from validate.rb:18
	from validate.rb:16:in `each'
	from validate.rb:16