use filename recorder for better TeX & LaTeX dependency scanning

[bdbaddog]

This issue was originally created at: 2007-09-12 13:50:52.
This issue was reported by: liblit.
liblit said at 2007-09-12 13:50:52

TeX and LaTeX are so fiendishly complex that scanning for dependencies can be very difficult to get right. Fortunately, TeX and LaTeX can actually record the names of files they processed. This should make it easier to build highly precise TeX and LaTeX dependency scanners.

The key is the often-overlooked "-recorder" flag to tex, pdftex, latex, pdflatex, etc. If the source document being processed is "master.tex", and the "-recorder" flag is given on the command line, then a file "master.fls" will be created listing all files read or written. The file is extremely easy to parse and is the most accurate record you will ever get of what files were involved in rendering a given document.

The "-recorder" functionality does not replace document rendering: it happens at the same time. So this is sort of the TeX analogue of gcc's "-MD" flag. Taking advantage of this probably requires that SCons be able to deal with builders that update dependencies as a side effect of the build itself. I remember reading somewhere that SCons is supposed to be gaining this ability, but I cannot remember where I saw that now. If it is on the SCons wiki somewhere, please edit the appropriate wiki page to link to this report. That way, once the general builder-that-reports-dependencies feature is ready, it will be easy to find this feature request that describes how to use it with TeX and LaTeX.

liblit said at 2007-09-12 13:53:09

How portable is "-recorder"? I don't know. It is available in the tetex builds of TeX and LaTeX that I used under Fedora Linux. I do not have easy access to MiKTeX, TeX Live, etc., so I cannot check whether they also support this. If anyone knows, or can check, please add that information to this report so that we know whether or not this will work on many platforms.

gregnoel said at 2008-06-05 17:53:21

Bug party triage: Rob to work with Steven to develop a general facility that can be used with other compilers that provide this sort of information as a part of the compile (such as 'gcc -MD').

managan said at 2008-09-08 08:43:11

*** Issue 1663 has been marked as a duplicate of this issue. ***

managan said at 2008-09-17 10:14:40

I tracked down some web page copies of man pages that claim the -recorder option existed back in 2001. I suspect it is much older than that but I don't know how much.

It seems reasonable that we can expect it to exist on any system with tex or latex.

gregnoel said at 2008-12-26 13:29:17

Adjust triage of issues.

managan said this issue is duplicated by #1663 at 2008-09-08 08:43:11.

[luhk]

I don't think that using --recorder will work for dependency scanning. If LaTeX encounters a missing file (i.e., a dependency that has not been generated yet) it will stop at this point without recording the missing file in the .fls file. So, the .fls could only be used to check dependencies after a successful build. Is there a point in this?

The only way I can think of to use LaTeX itself to get the dependencies (which would also address #1663) is to run it, parse the log file and look for ! LaTeX Error: File `dependency.tex' not found., build the dependency, repeat for the next dependency. However, this does not seem to be very feasible.

[Grinzold]

Just run with

-recorder

and

-interaction=batchmode

Then latex will not stop and you can find all dependencies in the .fls- and in the .log-file.