Shell Scripts Produce Corrupt PDFs under Bash for Windows

[tajmone]

The .sh script to build the PDF docs via asciidoctor-fopub don't work as expected under Bash for Windows, for they seem unable to locate the default template images, thus creating a PDF with no admonitions icons:

SEVERE: Image not found. URI: D:/local/path/to/asciidoctor-fopub/build/fopub/docbook/images/note.svg. (No context info available)

whereas the image mentioned in the error is actually there, with the correct path and filename!

It must be something related to how paths are handled in Bash for Windows vs real *nix bash, and how these interact with FOP.

We need to:

• Find a TEMPORARY FIX:
  • Make the .sh scripts for PDF build detect if running under Git for Windows' Bash (via MinGW env var) and abort execution if so.
• Find a REAL SOLUTION to make the .sh scripts work.
  • Delete all .bat scripts and keep only Bash scripts.

[tajmone]

This is a rather annoying problem because if asciidoctor-fopub could run without problems in Bash for Windows we could get rid of the batch scripts and use only Bash scripts — the assumption being that all contributors to the project should have Git, and therefore a Bash also on Windows.

Having to maintain all scripts in two versions (Bash + batch) is not only a burden, but it could easily lead to having scripts out of synch, therefore I'm not happy about it. Also, Bash offers many useful tools which are not available on the CMD, many of which are useful for working on cross-platform repos (e.g. unix2dos, dos2unix, iconv, etc.).

The current solution of Bash scripts aborting when detecting that they are running under MinGW* is far from ideal — just a safeguard.

@thoni56, any idea why asciidoctor-fopub fails under Bash for Win?

[thoni56]

I've done some experimentation with this in Cygwin and Msys2 and get the same problem in both. Once I've run pdf_build.sh once and have a manual.xml I can explore fopub problems. I'm dumping some experiments and observations here.

Running fopub from asciidoctor-fopub directory

When I run the following command from the directory of asciidoctor-fopub

$ ./fopub ../Alan/alan-docs/manual/manual.xml

I get some missing images, but there is also

 Cannot read configuration file:///home/Thomas/Utveckling/asciidoctor-fopub/build/fopub/docbook-xsl/xslthl-config.xml: \home\Thomas\Utveckling\asciidoctor-fopub\build\fopub\docbook-xsl\xslthl-config.xml (Path cannot be found)
java.io.FileNotFoundException: \home\Thomas\Utveckling\asciidoctor-fopub\build\fopub\docbook-xsl\xslthl-config.xml (Path cannot be found)
...

My guess is that that means that fopub uses the Windows Java (whatever you have) with Cygwin/Msys2 (and I presume Git Bash for Windows) paths. (\home\Thomas\Utveckling\asciidoctor-fopub\build\fopub\docbook-xsl\xslthl-config.xml has been re-formatted as a Windows path but not remapped.)

But I can't understand who/where that reference to the config file is. If I knew that I could at least investigate what happens if that config file was read.

You have probably already seen that there is actually a manual.pdf generated, but without images.

Running fopub from directory of manual

Running

../../../asciidoctor-fopub/fopub manual.xml

I get the same Java exception, but the images are included (which is no big surprise).

Running pdf_build.sh on Cygwin

When running

./pdf_build.sh

from the manual directory on Cygwin creates a completely different error:

USAGE
fop [options] [-fo|-xml] infile [-xsl file] [-awt|-pdf|-mif|-rtf|-tiff|-png|-pcl|-ps|-txt|-at [mime]|-print] <outfile>

[OPTIONS]
...
java.io.FileNotFoundException: Error: xml file C:\home\Thomas\Utveckling\Alan\alan-docs\manual\manual.xml not found

This led me to try to change the invocation of fopub in the script to not move into the assets directory but instead reference the xsl config with a path, thus

 fopub -t ../_assets/alan-xsl-fopub/xsl-fopub manual.xml

which kind-of improved things. Instead of the "USAGE" message I got another "File not found" exception:

org.apache.fop.apps.FOPException: javax.xml.transform.TransformerException: java.io.FileNotFoundException: C:\Users\Thomas\Utveckling\Alan\alan-docs\manual\db5.ent

for which the path looks ok (Windowsy enough) but the db5.ent file is nowhere to be found. Ideas where to look?

(For Msys2 I get the same behaviour, provided I comment out the Bash for Windows check. I have not tried with actual Git Bash for Windows)

Summary of findings

1. fopub does seem to work under Cygwin/Msys2
2. There is a path problem when the fopub\docbook-xsl\xslthl-config.xml is to be read (by some Java code/program)
3. The pdf_build.sh seems to have a problem setting up for execution of fopub in Cygwin/Msys2 environments

[tajmone]

Mhhhh. I suspected this was the problem. I've experienced something similar in the StdLib repo, with other dependencies. The best approach IMO is to:

1. Extract all assets' absolute paths and store into Shell env-variables.
2. Convert them to correct Bash or Windows paths as required.
3. Invoke fopub with all parameters as absolute paths.

But, from what I remember, the main problem are the DocBook settings files, which don't work well with relative paths across multiple OSs. I remember having experienced some problems locating the fonts, and that it could only be done using Windows paths for some reasons.

So, we'll have to look at the settings too, because images and fonts are controlled by the template settings.

This is the reason why I was considering looking into Asciidoctor's native PDF backend, and check if the previous issues have been solved in the meantime (there were some problems with footnotes at the time, but they might be solved now). Because it's a Gem, it's going to be much easier to use it in the toolchain without headaches. Also, Java has given lot's of problems with fopub so far, with Gradle incompatibility issues (which were finally solved) and more.