- 1 Introduction
- 2 Emacs Setup
- 3 Links and Systems
- 4 Background
- 5 Basics
- 6 Parsing
- 7 PAX Locatives
- 8 Navigating Sources in Emacs
- 9 Generating Documentation
- 10 Transcripts
- 11 Writing Extensions
What if documentation really lived in the code?
Docstrings are already there. If some narrative glued them together,
we'd be able develop and explore the code along with the
documentation due to their physical proximity. The main tool that
PAX provides for this is DEFSECTION
:
(defsection @foo-random-manual (:title "Foo Random manual")
"Foo Random is a random number generator library."
(foo-random-state class)
(uniform-random function)
(@foo-random-examples section))
Like this one, sections can have docstrings and
references to
definitions (e.g. (UNIFORM-RANDOM FUNCTION)
). These docstrings and
references are the glue. To support interactive development, PAX
See Emacs Setup.
Beyond interactive workflows, Generating Documentation from sections and all the referenced items in Markdown or HTML format is also implemented.
With the simplistic tools provided, one may emphasize the narrative as with Literate Programming, but documentation is generated from code, not vice versa, and there is no support for chunking.
Code is first, code must look pretty, documentation is code.
PAX automatically recognizes and marks up code with
backticks and links names in code to their definitions.
Take, for instance, SBCL's ABORT
function, whose docstring is
written in the usual style, uppercasing names of symbols:
(docstring #'abort)
=> "Transfer control to a restart named ABORT, signalling a CONTROL-ERROR if
none exists."
Note how in the generated documentation, ABORT
is set with a
monospace font, while CONTROL-ERROR
is Autolinked:
-
[function] ABORT &OPTIONAL CONDITION
Transfer control to a restart named
ABORT
, signalling aCONTROL-ERROR
if none exists.
The following transcript shows the raw Markdown for the previous example.
(document #'abort :format :markdown)
.. - [function] **ABORT** *&OPTIONAL CONDITION*
..
.. Transfer control to a restart named `ABORT`, signalling a [`CONTROL-ERROR`][7c2c] if
.. none exists.
..
.. [7c2c]: http://www.lispworks.com/documentation/HyperSpec/Body/e_contro.htm "CONTROL-ERROR (MGL-PAX:CLHS CONDITION)"
..
Here is an example of how it all works together:
(mgl-pax:define-package :foo-random
(:documentation "This package provides various utilities for random.
See FOO-RANDOM:@FOO-RANDOM-MANUAL.")
(:use #:common-lisp #:mgl-pax))
(in-package :foo-random)
(defsection @foo-random-manual (:title "Foo Random manual")
"FOO-RANDOM is a random number generator library inspired by CL:RANDOM.
Functions such as UNIFORM-RANDOM use *FOO-STATE* and have a
:RANDOM-STATE keyword arg."
(foo-random-state class)
(state (reader foo-random-state))
"Hey we can also print states!"
(print-object (method () (foo-random-state t)))
(*foo-state* variable)
(gaussian-random function)
(uniform-random function)
;; This is a subsection.
(@foo-random-examples section))
(defclass foo-random-state ()
((state :reader state)))
(defmethod print-object ((object foo-random-state) stream)
(print-unreadable-object (object stream :type t)))
(defvar *foo-state* (make-instance 'foo-random-state)
"Much like *RANDOM-STATE* but uses the FOO algorithm.")
(defun uniform-random (limit &key (random-state *foo-state*))
"Return a random number from the between 0 and LIMIT (exclusive)
uniform distribution."
nil)
(defun gaussian-random (stddev &key (random-state *foo-state*))
"Return a random number from a zero mean normal distribution with
STDDEV."
nil)
(defsection @foo-random-examples (:title "Examples")
"Let's see the transcript of a real session of someone working
with FOO:
```cl-transcript
(values (princ :hello) (list 1 2))
.. HELLO
=> :HELLO
=> (1 2)
(make-instance 'foo-random-state)
==> #<FOO-RANDOM-STATE >
```")
Note how (VARIABLE *FOO-STATE*)
in the DEFSECTION
form both
exports *FOO-STATE*
and includes its documentation in
@FOO-RANDOM-MANUAL
. The symbols VARIABLE
and
FUNCTION
are just two instances of locatives,
which are used in DEFSECTION
to refer to definitions tied to
symbols.
(DOCUMENT @FOO-RANDOM-MANUAL)
generates fancy Markdown or HTML
output with automatic markup and Autolinks uppercase words found in docstrings,
numbers sections, and creates a table of contents.
One can even generate documentation for different but related libraries at the same time with the output going to different files but with cross-page links being automatically added for symbols mentioned in docstrings. In fact, this is what PAX World does. See Generating Documentation for some convenience functions to cover the most common cases.
The transcript in the code block tagged with
cl-transcript
is automatically checked for up-to-dateness when
documentation is generated.
Here is a quick recipe for setting up PAX for use via SLIME to
take advantage of the conveniences on offer.
Conversely, there is no need to do any of this just to use
DEFSECTION
, write docstrings and for Generating Documentation.
If PAX was installed from Quicklisp, then evaluate this in CL to copy the Elisp code to a stable location:
(mgl-pax:install-pax-elisp "~/quicklisp/")
Assuming the Elisp file is in the ~/quicklisp/
directory, add
something like this to your .emacs
:
(add-to-list 'load-path "~/quicklisp/")
(require 'mgl-pax)
(global-set-key (kbd "C-.") 'mgl-pax-document)
(global-set-key (kbd "s-x t") 'mgl-pax-transcribe-last-expression)
(global-set-key (kbd "s-x r") 'mgl-pax-retranscribe-region)
-
For Navigating Sources in Emacs, loading
mgl-pax
extendsslime-edit-definitions
(M-.
) by addingmgl-pax-edit-definitions
toslime-edit-definition-hooks
. There are no related variables to customize. -
For Browsing Live Documentation,
mgl-pax-browser-function
andmgl-pax-web-server-port
can be customized in Elisp. To browse within Emacs, choosew3m-browse-url
(see w3m), and make sure both the w3m binary and the w3m Emacs package are installed. On Debian, simply install thew3m-el
package. With other browser functions, a HUNCHENTOOT web server is started. -
See Transcribing with Emacs for how to use the transcription features. There are no related variables to customize.
If you installed PAX with Quicklisp, the location of mgl-pax.el
may change with updates, and you may want to copy the current
version of mgl-pax.el
to a stable location by evaluating this in
CL:
(mgl-pax:install-pax-elisp "~/quicklisp/")
If working from, say, a git checkout, there is no need for this step.
-
[function] INSTALL-PAX-ELISP TARGET-DIR
Copy
mgl-pax.el
distributed with this package toTARGET-DIR
.
Assuming the Elisp file is in the ~/quicklisp/
directory, add
something like this to your .emacs
:
(add-to-list 'load-path "~/quicklisp/")
(require 'mgl-pax)
If the Lisp variable mgl-pax-autoload
is true (the default), then
MGL-PAX will be loaded in the connected Lisp on-demand via SLIME.
If loading fails, mgl-pax
will be unloaded from Emacs and any
overridden Slime key bindings restored.
The recommended key bindings are this:
(global-set-key (kbd "C-.") 'mgl-pax-document)
(global-set-key (kbd "s-x t") 'mgl-pax-transcribe-last-expression)
(global-set-key (kbd "s-x r") 'mgl-pax-retranscribe-region)
The global key bindings above are global because their commands work
in any mode. If that's not desired, one may bind C-.
locally in
all Slime related modes like this:
(slime-bind-keys slime-parent-map nil '(("C-." mgl-pax-document)))
If the customizable variable mgl-pax-hijack-slime-doc-keys
is
true, then upon loading mgl-pax', the following changes are made to
slime-doc-map(assuming it's bound to
C-c C-d`):
-
C-c C-d a
: replacesslime-apropos
withmgl-pax-apropos
-
C-c C-d z
: replacesslime-apropos-all
withmgl-pax-apropos-all
-
C-c C-d p
: replacesslime-apropos-package
withmgl-pax-apropos-package
-
C-c C-d d
: replacesslime-describe-symbol
withmgl-pax-document
-
C-c C-d f
: replacesslime-describe-function
withmgl-pax-document
-
C-c C-d c
: installsmgl-pax-current-definition-toggle-view
-
C-c C-d u
: installsmgl-pax-edit-parent-section
Calling mgl-pax-unhijack-slime-doc-keys
reverts these changes.
Here is the official repository and the HTML documentation for the latest version.
PAX is built on top of the DRef library (bundled in the same repository). See DRef's HTML documentation
- [system] "mgl-pax"
- Version: 0.4.0
- Description: Documentation system, browser, generator. See the PAX Manual.
- Long Description: The set of dependencies of the
mgl-pax
system is kept light, and its heavier dependencies are autoloaded via ASDF when the relevant functionality is accessed. See themgl-pax/navigate
,mgl-pax/document
,mgl-pax/transcribe
andmgl-pax/full
systems. To keep deployed code small, client systems should declare an ASDF dependency on this system, never on the others, which are intended for autoloading and interactive use. - Licence: MIT, see COPYING.
- Author: Gábor Melis
- Mailto: mega@retes.hu
- Homepage: http://melisgl.github.io/mgl-pax
- Bug tracker: https://github.com/melisgl/mgl-pax/issues
- Source control: GIT
- Depends on: dref, mgl-pax-bootstrap, named-readtables, pythonic-string-reader
- Defsystem depends on: mgl-pax.asdf
- [system] "mgl-pax/full"
- Description: The
mgl-pax
system with all features preloaded exceptmgl-pax/web
. - Long Description: Do not declare a dependency on this system. It is for interactive use.
- Licence: MIT, see COPYING.
- Author: Gábor Melis
- Mailto: mega@retes.hu
- Depends on: mgl-pax/document, mgl-pax/navigate, mgl-pax/transcribe
- Description: The
As a user, I frequently run into documentation that's incomplete
and out of date, so I tend to stay in the editor and explore the
code by jumping around with SLIME's M-.
(slime-edit-definition
).
As a library author, I spend a great deal of time polishing code but
precious little writing documentation.
In fact, I rarely write anything more comprehensive than docstrings for exported stuff. Writing docstrings feels easier than writing a separate user manual, and they are always close at hand during development. The drawback of this style is that users of the library have to piece the big picture together themselves.
That's easy to solve, I thought, let's just put all the narrative that holds docstrings together in the code and be a bit like a Literate Programmer turned inside out. The original prototype, which did almost everything I wanted, was this:
(defmacro defsection (name docstring)
`(defun ,name () ,docstring))
Armed with this DEFSECTION
, I soon found myself
organizing code following the flow of user-level documentation and
relegated comments to implementation details entirely. However, some
parts of DEFSECTION
docstrings were just listings of
all the functions, macros and variables related to the narrative,
and this list was repeated in the DEFPACKAGE
form complete with
little comments that were like section names. A clear violation of
OAOO, one of them had to go, so DEFSECTION
got a list
of symbols to export.
That was great, but soon I found that the listing of symbols is
ambiguous if, for example, a function, a compiler macro and a class
were named by the same symbol. This did not concern exporting, of
course, but it didn't help readability. Distractingly, on such
symbols, M-.
was popping up selection dialogs. There were two
birds to kill, and the symbol got accompanied by a type, which was
later generalized into the concept of locatives:
(defsection @introduction ()
"A single line for one man ..."
(foo class)
(bar function))
After a bit of elisp hacking, M-.
was smart enough to
disambiguate based on the locative found in the vicinity of the
symbol, and everything was good for a while.
Then, I realized that sections could refer to other sections if
there were a SECTION
locative. Going down that path, I soon began to
feel the urge to generate pretty documentation as all the necessary
information was available in the DEFSECTION
forms. The design
constraint imposed on documentation generation was that following
the typical style of upcasing symbols in docstrings, there should be
no need to explicitly mark up links: if M-.
works, then the
documentation generator shall also be able figure out what's being
referred to.
I settled on Markdown as a reasonably non-intrusive format, and a
few thousand lines later PAX was born. Since then, locatives and
references were factored out into the DRef
library to let PAX focus on M-.
and documentation.
Now let's examine the most important pieces.
-
[macro] DEFSECTION NAME (&KEY (PACKAGE '*PACKAGE*) (READTABLE '*READTABLE*) (EXPORT T) TITLE LINK-TITLE-TO (DISCARD-DOCUMENTATION-P *DISCARD-DOCUMENTATION-P*)) &BODY ENTRIES
Define a documentation section and maybe export referenced symbols. A bit behind the scenes, a global variable with
NAME
is defined and is bound to aSECTION
object. By convention, section names start with the character@
. See Introduction for an example.Entries
ENTRIES
consists of docstrings and references in any order. Docstrings are arbitrary strings in markdown format.References are
XREF
s given in the form(NAME LOCATIVE)
. For example,(FOO FUNCTION)
refers to the functionFOO
,(@BAR SECTION)
says that@BAR
is a subsection of this one.(BAZ (METHOD () (T T T)))
refers to the default method of the three argument generic functionBAZ
.(FOO FUNCTION)
is equivalent to(FOO (FUNCTION))
. See the DRef Introduction for more.The same name may occur in multiple references, typically with different locatives, but this is not required.
The references are not
LOCATE
d until documentation is generated, so they may refer to things yet to be defined.Exporting
If
EXPORT
is true (the default),NAME
and the names of references amongENTRIES
which areSYMBOL
s are candidates for exporting. A candidate symbol is exported if-
it is accessible in
PACKAGE
, and -
there is a reference to it in the section being defined which is approved by
EXPORTABLE-REFERENCE-P
.
See
DEFINE-PACKAGE
if you use the export feature. The idea with confounding documentation and exporting is to force documentation of all exported symbols.Misc
TITLE
is a string containing markdown orNIL
. If non-NIL
, it determines the text of the heading in the generated output.LINK-TITLE-TO
is a reference given as an(NAME LOCATIVE)
pair orNIL
, to which the heading will link when generating HTML. If not specified, the heading will link to its own anchor.When
DISCARD-DOCUMENTATION-P
(defaults to*DISCARD-DOCUMENTATION-P*
) is true,ENTRIES
will not be recorded to save memory. -
-
[variable] *DISCARD-DOCUMENTATION-P* NIL
The default value of
DEFSECTION
'sDISCARD-DOCUMENTATION-P
argument. One may want to set*DISCARD-DOCUMENTATION-P*
to true before building a binary application.
-
[macro] DEFINE-PACKAGE PACKAGE &REST OPTIONS
This is like
CL:DEFPACKAGE
but silences warnings and errors signalled when the redefined package is at variance with the current state of the package. Typically this situation occurs when symbols are exported by callingEXPORT
(as is the case withDEFSECTION
) as opposed to adding:EXPORT
forms to theDEFPACKAGE
form and the package definition is subsequently reevaluated. See the section on package variance in the SBCL manual.The bottom line is that if you rely on
DEFSECTION
to do the exporting, then you'd better useDEFINE-PACKAGE
.
-
[macro] DEFINE-GLOSSARY-TERM NAME (&KEY TITLE URL (DISCARD-DOCUMENTATION-P *DISCARD-DOCUMENTATION-P*)) &BODY DOCSTRING
Define a global variable with
NAME
, and set it to aGLOSSARY-TERM
object.TITLE
,URL
andDOCSTRING
are markdown strings orNIL
. Glossary terms areDOCUMENT
ed in the lightweight bullet + locative + name/title style. See the glossary entry name for an example.When a glossary term is linked to in documentation, its
TITLE
will be the link text instead of the name of the symbol (as withSECTION
s).Glossary entries with a non-
NIL
URL
are like external links: they are linked to theirURL
in the generated documentation. These offer a more reliable alternative to using markdown reference links and are usually not included inSECTION
s.When
DISCARD-DOCUMENTATION-P
(defaults to*DISCARD-DOCUMENTATION-P*
) is true,DOCSTRING
will not be recorded to save memory.
When encountering a word such as CLASSes
, PAX needs to find the
name in it that makes sense in the context. Codification, for
example, looks for interesting names, Navigating Sources in Emacs for
names with Lisp DEFINITIONS
, and
Linking for names with any kind of definition.
This is not as straightforward as it sounds because it needs to
handle cases like nonREAD
able, CLASS
es, all the various forms of
Linking in docstrings as well as in comments, and the (NAME LOCATIVE)
syntax in DEFSECTION
.
-
[glossary-term] word
A word is a string from which we want to extract a name. When Navigating, the word is
slime-sexp-at-point
or the label of a Markdown reference link if point is over one. Similarly, when Generating Documentation, it is a non-empty string between whitespace characters in a docstring or the label of a Markdown reference link.
-
[glossary-term] raw name
A raw name is a string from which a name may be read. Raw names correspond to an intermediate parsing step between words an names. See Names in Raw Names.
-
[glossary-term] name
A name is a DRef name. That is, a symbol or a string associated with a definition, whose kind is given by a locative.
Depending on the context, trimming and depluralization may be enabled (see Raw Names in Words), while the possible names may be restricted to symbols (see Names in Raw Names).
-
Trimming: Enabled for Navigating Sources in Emacs and Codification.
-
Depluralization: Enabled when the word is part of the normal flow of text (i.e. not for Specific Reflink with Text, Unspecific Reflink with Text and various Elisp functions such as
mgl-pax-apropos
unless they determine their argument from buffer contents). -
Symbols only: This is the case for Codification and Unspecific Autolink to prevent string-based definitions from littering the documentation with links without the control provided by explicitly
IMPORT
ing symbols.
For a word, a number of raw names is generated by trimming delimiter characters and plural markers, and for each raw name a number of names are considered until one is found suitable in the context. The following subsections describe the details of the parsing algorithm.
From words, raw names are parsed by trimming some prefixes and suffixes. For a given word, multiple raw names are considered in the following order.
-
The entire word.
-
Trimming the characters `#<{;"'`` from the left of the word.
-
Trimming the characters `,;:.>}"'`` from the right of the word.
-
Trimming both of the previous two at the same time.
-
From the result of 4., If a word ends with what looks like a plural marker (case-insensitive), then a name is created by removing it. For example, from the word
BUSES
the plural markerES
is removed to produce the nameBUS
. The list of plural markers considered isSES
(e.g.GASSES
),ES
(e.g.BUSES
),S
(e.g.CARS
),ZES
(e.g.FEZZES
), andREN
(e.g.CHILDREN
). -
From the result of 4., removing the prefix before the first, and the suffix after the last uppercase character if it contains at least one lowercase character.
For each raw name from Raw Names in Words, various names may be considered until one is found suitable in the context.
The following examples list the names considered for a given raw
name, assuming that READTABLE-CASE
is :UPCASE
as well as that FOO
and |Foo|
are interned.
-
"foo"
:FOO
,"foo"
,"FOO"
(rules 1, 2, 3) -
"FOO"
:FOO
,"FOO"
(rules 1, 2) -
"Foo"
:"Foo"
,"FOO"
(rules 2, 3) -
"|Foo|"
:|Foo|
(rule 4) -
"\"foo\""
:"foo"
(rule 5)
The rules are:
-
If the raw name is not mixed case (i.e. it doesn't have both upper- and lowercase characters) and it names an interned symbol (subject to the current Package and Readtable), then that symbol is considered as a name.
-
The raw name itself (a string) is considered a name.
-
The raw name upcased or downcased according to
READTABLE-CASE
(subject to the current readtable) but still as a string. This is to allow[dref][package]
to refer to the"DREF"
package regardless of whether the symbolDREF
is interned in the current package. -
If the raw name is explicitly a symbol (it starts with
#\|
), and it names an interned symbol (subject to the current Package and Readtable), then that symbol is considered as a name and nothing else. -
If the raw name has an embedded string (it starts with
#\"
) andREAD-FROM-STRING
can read the embedded string from it, then that string is considered as a name and nothing else.
For example, when `M-.` is pressed while point is over `nonREADable.`, the last word of the sentence `It may be nonREADable.`, the following [raw name][f5af]s are considered until one is found with a definition:
-
The entire word,
"nonREADable."
. -
Trimming left does not produce a new raw name.
-
Trimming right removes the dot and gives
"nonREADable"
. -
Trimming both is the same as trimming right.
-
No plural markers are found.
-
The lowercase prefix and suffix is removed around the uppercase core, giving
"READ"
. This names an interned symbol which has a definition, soM-.
will visit it.
When Generating Documentation, Autolinking behaves similarly.
To the Basic Locative Types defined by DRef, PAX adds a few of its own.
-
[locative] SECTION
- Direct locative supertypes:
VARIABLE
Refers to a
SECTION
defined byDEFSECTION
.SECTION
isEXPORTABLE-LOCATIVE-TYPE-P
but not exported by default (seeEXPORTABLE-REFERENCE-P
). - Direct locative supertypes:
-
[locative] GLOSSARY-TERM
- Direct locative supertypes:
VARIABLE
Refers to a
GLOSSARY-TERM
defined byDEFINE-GLOSSARY-TERM
.GLOSSARY-TERM
isEXPORTABLE-LOCATIVE-TYPE-P
but not exported by default (seeEXPORTABLE-REFERENCE-P
). - Direct locative supertypes:
-
[locative] DISLOCATED
Refers to a symbol in a non-specific context. Useful for suppressing Unspecific Autolinking. For example, if there is a function called
FOO
then`FOO`
will be linked (if
*DOCUMENT-LINK-CODE*
) to its definition. However,[`FOO`][dislocated]
will not be. With a dislocated locative,
LOCATE
always fails with aLOCATE-ERROR
condition. Also see Escaping Autolinking.DISLOCATED
references do notRESOLVE
.
-
[locative] ARGUMENT
An alias for
DISLOCATED
, so that one can refer to an argument of a macro without accidentally linking to a class that has the same name as that argument. In the following example,FORMAT
may link toCL:FORMAT
(if we generated documentation for it):"See FORMAT in DOCUMENT."
Since
ARGUMENT
is a locative, we can prevent that linking by writing:"See the FORMAT argument of DOCUMENT."
ARGUMENT
references do notRESOLVE
.
-
[locative] INCLUDE SOURCE &KEY LINE-PREFIX HEADER FOOTER HEADER-NL FOOTER-NL
This pseudolocative refers to a region of a file.
SOURCE
can be aSTRING
or aPATHNAME
, in which case the whole file is being pointed to, or it can explicitly supplySTART
,END
locatives.INCLUDE
is typically used to include non-lisp files in the documentation (say markdown or Elisp as in the next example) or regions of Lisp source files. This can reduce clutter and duplication.(defsection @example-section () (mgl-pax.el (include #.(asdf:system-relative-pathname :mgl-pax "src/mgl-pax.el") :header-nl "```elisp" :footer-nl "```")) (foo-example (include (:start (dref-ext:make-source-location function) :end (dref-ext:source-location-p function)) :header-nl "```" :footer-nl "```")))
In the above example, when documentation is generated, the entire
src/mgl-pax.el
file is included in the markdown output surrounded by the strings given asHEADER-NL
andFOOTER-NL
. The documentation ofFOO-EXAMPLE
will be the region of the file from theSOURCE-LOCATION
of theSTART
reference (inclusive) to theSOURCE-LOCATION
of theEND
reference (exclusive). If only one ofSTART
andEND
is specified, then they default to the beginning and end of the file, respectively.Since
START
andEND
are literal references, pressingM-.
onPAX.EL
will open thesrc/mgl-pax.el
file and put the cursor on its first character.M-.
onFOO-EXAMPLE
will go to the source location of theFOO
function.With the
LAMBDA
locative, one can specify positions in arbitrary files.-
SOURCE
is either an absolute pathname designator or a list matching the destructuring lambda list(&KEY START END)
, whereSTART
andEND
must beNIL
or(<NAME> <LOCATIVE>)
lists (not evaluated) like aDEFSECTION
entry. TheirSOURCE-LOCATION
s constitute the bounds of the region of the file to be included. Note that the file of the source location ofSTART
andEND
must be the same. IfSOURCE
is a pathname designator, then it must be absolute so that the locative is context independent. -
If specified,
LINE-PREFIX
is a string that's prepended to each line included in the documentation. For example, a string of four spaces makes markdown think it's a code block. -
HEADER
andFOOTER
, if non-NIL
, are printed before the included string. -
HEADER-NL
andFOOTER-NL
, if non-NIL
, are printed between twoFRESH-LINE
calls.
INCLUDE
is notEXPORTABLE-LOCATIVE-TYPE-P
, andINCLUDE
references do notRESOLVE
. -
-
[locative] DOCSTRING
DOCSTRING
is a pseudolocative for including the parse tree of the markdownDOCSTRING
of a definition in the parse tree of a docstring when generating documentation. It has no source location information and only works as an explicit link. This construct is intended to allow docstrings to live closer to their implementation, which typically involves a non-exported definition.(defun div2 (x) "X must be [even* type][docstring]." (/ x 2)) (deftype even* () "an even integer" '(satisfies evenp)) (document #'div2) .. - [function] DIV2 X .. .. X must be an even integer. ..
There is no way to
LOCATE
DOCSTRING
s, so nothing toRESOLVE
either.
-
[locative] GO (NAME LOCATIVE)
Redirect to a definition in the context of the reference designated by
NAME
andLOCATIVE
. This pseudolocative is intended for things that have no explicit global definition.As an example, consider this part of a hypothetical documentation of CLOS:
(defsection @clos () (defmethod macro) (call-next-method (go (defmethod macro))))
The
GO
reference exports the symbolCALL-NEXT-METHOD
and also produces a terse redirection message in the documentation.GO
behaves as described below.-
A
GO
referenceRESOLVE
s to whatNAME
withLOCATIVE
resolves to:(resolve (dref 'xxx '(go (print function)))) ==> #<FUNCTION PRINT> => T
-
The
DOCSTRING
of aGO
reference isNIL
. -
SOURCE-LOCATION
(thusM-.
) returns the source location of the embedded reference:(equal (source-location (dref 'xxx '(go (print function)))) (source-location (dref 'print 'function))) => T
-
-
[locative] CLHS &OPTIONAL NESTED-LOCATIVE
Refers to definitions, glossary entries, sections, issues and issue summaries in the Common Lisp HyperSpec. These have no source location so
M-.
will not work. What works is linking in documentation, including Browsing Live Documentation. The generated links are relative to*DOCUMENT-HYPERSPEC-ROOT*
and work even if*DOCUMENT-LINK-TO-HYPERSPEC*
isNIL
. All matching is case-insensitive.-
definitions: These are typically unnecessary as
DOCUMENT
will produce the same link for e.g.PPRINT
,[PPRINT][function]
, or[PPRINT][]
if*DOCUMENT-LINK-TO-HYPERSPEC*
is non-NIL
and thePPRINT
function in the running Lisp is not linkable. When Browsing Live Documentation, a slight difference is that everything is linkable, so using theCLHS
link bypasses the page with the definition in the running Lisp. -
glossary terms:
[lambda list][(clhs glossary-term)]
(lambda list)
-
issues:
-
[ISSUE:AREF-1D][clhs]
(ISSUE:AREF-1D) -
[ISSUE:AREF-1D][(clhs section)]
(ISSUE:AREF-1D)
-
-
issue summaries: These render as (SUMMARY:CHARACTER-PROPOSAL:2-6-5):
-
[SUMMARY:CHARACTER-PROPOSAL:2-6-5][clhs]
-
[SUMMARY:CHARACTER-PROPOSAL:2-6-5][(clhs section)]
Since these summary ids are not particularly reader friendly, the anchor text a Specific Reflink with Text may be used:
[see this][SUMMARY:CHARACTER-PROPOSAL:2-6-5 (clhs section)]
(see this).
-
-
sections:
-
by section number:
[3.4][clhs]
or[3.4][(clhs section)]
(3.4) -
by section title (substring match):
[lambda lists][clhs]
or[lambda lists][(clhs section)]
(lambda lists) -
by filename:
[03_d][clhs]
or[03_d][(clhs section)]
(03_d) -
by alias: Format directives are aliases of the sections describing them. Thus,
[~c][clhs]
is equivalent to[22.3.1.1][clhs]
and[Tilde C: Character][clhs]
. The full list is ~C ~% ~& ~| ~~ ~R ~D ~B ~O ~X ~F ~E ~G ~$ ~A ~S ~W ~_ ~< ~:> ~I ~/ ~T ~< Justification ~> ~* ~[ ~] ~{ ~} ~? ~( ~) ~P ~; ~^ ~Newline.Similarly, reader macro characters are aliases of the sections describing them. The full list is ( ) ' ; " ` , # #\ #' #( #* #: #. #B #O #X #R #C #A #S #P #= ## #+ #- #| #< #).
Finally, loop keywords have aliases to the sections describing them. For example, the strings
loop:for
,for
and:for
are aliases ofCLHS
6.1.2.1
. Theloop:*
aliases are convenient for completion at the prompt when Browsing Live Documentation, while the other aliases are for defaulting to buffer contents.
-
As the above examples show, the
NESTED-LOCATIVE
argument of theCLHS
locative may be omitted. In that case, definitions, glossary terms, issues, issue summaries, and sections are considered in that order. Sections are considered last because a substring of a section title can be matched by chance easily.All examples so far used Reflinks. Autolinking also works if the name is marked up as code or is codified (e.g. in
COS clhs
(COS
clhs).As mentioned above,
M-.
does not do anything overCLHS
references. Slightly more usefully, the live documentation browser understandsCLHS
links so one can enter inputs like3.4 clhs
,"lambda list" clhs
orerror (clhs function)
.CLHS
references do notRESOLVE
. -
Integration into SLIME's M-.
(slime-edit-definition
) allows
one to visit the SOURCE-LOCATION
of a definition. PAX extends
standard Slime functionality by
-
adding support for all kinds of definitions (see e.g.
ASDF:SYSTEM
,READTABLE
in Basic Locative Types), not just the ones Slime knows about, -
providing a portable way to refer to even standard definitions,
-
disambiguating the definition based on buffer content, and
-
adding more powerful completions.
The definition is either determined from the buffer content at point
or is prompted for. At the prompt, TAB-completion is available for
both names and locatives. With a prefix argument (C-u M-.
), the
buffer contents are not consulted, and M-.
always prompts.
The M-.
extensions can be enabled by loading src/mgl-pax.el
. See
Emacs Setup. In addition, the Elisp command
mgl-pax-edit-parent-section
visits the source location of the
section containing the definition with point in it.
A close relative of M-.
is C-.
for Browsing Live Documentation.
- [system] "mgl-pax/navigate"
- Description: Support for Navigating Sources in Emacs via Slime's
M-.
in MGL-PAX. - Long Description: Do not declare a dependency on this system. It is
autoloaded by Elisp or by accessing the functionality provided if the
mgl-pax
system is loaded. - Licence: MIT, see COPYING.
- Author: Gábor Melis
- Mailto: mega@retes.hu
- Depends on: alexandria, dref/full, mgl-pax, swank(?)
- Defsystem depends on: mgl-pax.asdf
- Description: Support for Navigating Sources in Emacs via Slime's
When M-.
is invoked, it first tries to find a name in the
current buffer at point. If no name is found, then it
prompts.
First, (slime-sexp-at-point)
is taken as a word, from which the
name will be parsed. Then, candidate locatives are
looked for before and after the word. Thus, if a locative is the
previous or the next expression, then M-.
will go straight to the
definition which corresponds to the locative. If that fails, M-.
will try to find the definitions in the normal way, which may
involve popping up an xref buffer and letting the user interactively
select one of possible definitions.
M-.
works on parenthesized references, such as those in
DEFSECTION
:
(defsection @foo ()
(cos function))
Here, when the cursor is on one of the characters of COS
or just
after COS
, pressing M-.
will visit the definition of the
function COS
.
To play nice with Generating Documentation, forms suitable for Autolinking are recognized:
function cos
cos function
... as well as Reflinks:
[cos][function]
[see this][cos function]
... and Markdown inline code:
cos `function`
`cos` function
`cos` `function`
Everything works the same way in comments and docstrings as in code.
In the next example, pressing M-.
on RESOLVE*
will visit its
denoted method:
;;; See RESOLVE* (method () (function-dref)) for how this all works.
At the minibuffer prompt, the definitions to edit can be specified as follows.
-
NAME
: Refers to allDREF:DEFINITIONS
ofNAME
with a Lisp locative type. See theseNAME -> DEFINITIONS
examples:print -> PRINT FUNCTION PRINT -> PRINT FUNCTION MGL-PAX -> "mgl-pax" ASDF:SYSTEM, "MGL-PAX" package pax -> "PAX" PACKAGE "PAX" -> "PAX" PACKAGE
Note that depending on the Lisp implementation there may be more definitions. For example, SBCL has an
UNKNOWN
:DEFOPTIMIZER
definition forPRINT
. -
NAME
LOCATIVE
: Refers to a single definition (as in(DREF:DREF NAME LOCATIVE)
). Example inputs of this form:print function dref-ext:docstring* (method nil (t))
-
LOCATIVE
NAME
: This has the same form as the previous: two sexps, but here the first one is the locative. If ambiguous, this is considered in addition to the previous one. Example inputs:function print (method nil (t)) dref-ext:docstring*
In all of the above NAME
is a raw name, meaning that print
will be recognized as PRINT
and pax
as "PAX"
.
The package in which symbols are read is the Elisp
slime-current-package
. In Lisp buffers, this is the buffer's
package, else it's the package of the Slime repl buffer.
When M-.
prompts for the definition to edit, TAB-completion is
available in the minibuffer for both names and locatives. To reduce
clutter, string names are completed only if they are typed
explicitly with an opening quotation mark, and they are
case-sensitive. Examples:
-
pri<TAB>
invokes the usual Slime completion. -
print <TAB>
(note the space) listsFUNCTION
(0
1
) and (PAX:CLHS
FUNCTION
) as locatives. -
class dref:<TAB>
listsDREF:XREF
(0
1
) andDREF:DREF
(0
1
) (all the classes in the packageDREF
). -
pax:locative <TAB>
lists all locative types (see the CL functionDREF:LOCATIVE-TYPES
). -
package "MGL<TAB>
lists the names of packages that start with"MGL"
. -
package <TAB>
lists the names of all packages as strings and alsoCLASS
,MGL-PAX:LOCATIVE
becausePACKAGE
denotes a class and also a locative.
For more powerful search, see Apropos.
-
[function] DOCUMENT DOCUMENTABLE &KEY (STREAM T) PAGES (FORMAT :PLAIN)
Write
DOCUMENTABLE
inFORMAT
toSTREAM
diverting some output toPAGES
.FORMAT
is a 3BMD output format (currently one of:MARKDOWN
,:HTML
and:PLAIN
).STREAM
may be aSTREAM
object,T
orNIL
as withCL:FORMAT
.To look up the documentation of the
DOCUMENT
function itself:(document #'document)
The same with fancy markup:
(document #'document :format :markdown)
To document a
SECTION
:(document pax::@pax-manual)
To generate the documentation for separate libraries with automatic cross-links:
(document (list pax::@pax-manual dref::@dref-manual) :format :markdown)
See Utilities for Generating Documentation for more.
Definitions that do not define a first-class object are supported via DRef:
(document (dref:locate 'foo 'type))
There are quite a few special variables that affect how output is generated, see Codification, Linking to the HyperSpec, Linking to Sections, Link Format and Output Details.
For the details, see the following sections, starting with
DOCUMENTABLE
. Also see Writing Extensions andDOCUMENT-OBJECT*
.
The DOCUMENTABLE
argument of DOCUMENT
may be a single object (e.g.
#'PRINT
'), a definition such as (DREF 'PRINT 'FUNCTION)
,
a string, or a nested list of these. More precisely, DOCUMENTABLE
is
one of the following:
-
single definition designator: A
DREF
or anything else that isLOCATE
able. This includes non-DREF
XREF
s and first-class objects such asFUNCTION
s. The generated documentation typically includes the definition'sDOCSTRING
. See Output Details for more. -
docstring: A string, in which case it is processed like a docstring in
DEFSECTION
. That is, with docstring sanitization, Codification, and Linking. -
list of documentables: A nested list of
LOCATE
able objects and docstrings. The objects in it are documented in depth-first order. The structure of the list is otherwise unimportant.
If PAGES
are NIL
, then DOCUMENT
- like CL:FORMAT
- returns a
string (when STREAM
is NIL
) else NIL
.
If PAGES
, then a list of output designators are returned, one for
each non-empty page (to which some output has been written), which
are determined as follows.
-
The string itself if the output was to a string.
-
The stream if the output was to a stream.
-
The pathname of the file if the output was to a file.
If the default page given by the STREAM
argument of DOCUMENT
was
written to, then its output designator is the first element of the
returned list. The rest of the designators correspond to the
non-empty pages in the PAGES
argument of DOCUMENT
in that order.
The PAGES
argument of DOCUMENT
is to create multi-page documents
by routing some of the generated output to files, strings or
streams. PAGES
is a list of page specification elements. A page spec
is a property list with keys :OBJECTS
, :OUTPUT
,
:URI-FRAGMENT
, :SOURCE-URI-FN
, :HEADER-FN
and :FOOTER-FN
. OBJECTS
is
a list of objects (references are allowed but not required) whose
documentation is to be sent to :OUTPUT
.
PAGES
may look something like this:
`((;; The section about SECTIONs and everything below it ...
:objects (, @sections)
;; ... is so boring that it's not worth the disk space, so
;; send it to a string.
:output (nil)
;; Explicitly tell other pages not to link to these guys.
:uri-fragment nil)
;; Send the @EXTENSION-API section and everything reachable
;; from it ...
(:objects (, @extension-api)
;; ... to build/tmp/pax-extension-api.html.
:output "build/tmp/pax-extension-api.html"
;; However, on the web server html files will be at this
;; location relative to some common root, so override the
;; default:
:uri-fragment "doc/dev/pax-extension-api.html"
;; Set html page title, stylesheet, charset.
:header-fn 'write-html-header
;; Just close the body.
:footer-fn 'write-html-footer)
;; Catch references that were not reachable from the above. It
;; is important for this page spec to be last.
(:objects (, @pax-manual)
:output "build/tmp/manual.html"
;; Links from the extension api page to the manual page will
;; be to ../user/pax-manual#<anchor>, while links going to
;; the opposite direction will be to
;; ../dev/pax-extension-api.html#<anchor>.
:uri-fragment "doc/user/pax-manual.html"
:header-fn 'write-html-header
:footer-fn 'write-html-footer))
Documentation is initially sent to a default stream (the STREAM
argument of DOCUMENT
), but output is redirected if the thing being
currently documented is the :OBJECT
of a PAGE-SPEC
.
-
:OUTPUT
can be a number things:-
If it's
NIL
, then output will be collected in a string. -
If it's
T
, then output will be sent to*STANDARD-OUTPUT*
. -
If it's a stream, then output will be sent to that stream.
-
If it's a list whose first element is a string or a pathname, then output will be sent to the file denoted by that and the rest of the elements of the list are passed on to
CL:OPEN
. One extra keyword argument is:ENSURE-DIRECTORIES-EXIST
. If it's true,ENSURE-DIRECTORIES-EXIST
will be called on the pathname before it's opened.
Note that even if
PAGES
is specified,STREAM
acts as a catch all, absorbing the generated documentation for references not claimed by any pages. -
-
:HEADER-FN
, if notNIL
, is a function of a single stream argument, which is called just before the first write to the page. Since:FORMAT
:HTML
only generates HTML fragments, this makes it possible to print arbitrary headers, typically setting the title, CSS stylesheet, or charset. -
:FOOTER-FN
is similar to:HEADER-FN
, but it's called after the last write to the page. For HTML, it typically just closes the body. -
:URI-FRAGMENT
is a string such as"doc/manual.html"
that specifies where the page will be deployed on a webserver. It defines how links between pages will look. If it's not specified and:OUTPUT
refers to a file, then it defaults to the name of the file. If:URI-FRAGMENT
isNIL
, then no links will be made to or from that page. -
:SOURCE-URI-FN
is a function of a single,DREF
argument. If it returns a value other thanNIL
, then it must be a string representing an URI. This affects*DOCUMENT-MARK-UP-SIGNATURES*
and*DOCUMENT-FANCY-HTML-NAVIGATION*
. Also seeMAKE-GIT-SOURCE-URI-FN
.
While generating documentation, symbols may be read (e.g. from
docstrings) and printed. What values of *PACKAGE*
and *READTABLE*
are used is determined separately for each definition being
documented.
-
If the values of
*PACKAGE*
and*READTABLE*
in effect at the time of definition were captured (e.g. byDEFINE-LOCATIVE-TYPE
andDEFSECTION
), then they are used. -
Else, if the definition has a Home Section (see below), then the home section's
SECTION-PACKAGE
andSECTION-READTABLE
are used. -
Else, if the definition has an argument list, then the package of the first argument that's not external in any package is used.
-
Else, if the definition is named by a symbol, then its
SYMBOL-PACKAGE
is used, and*READTABLE*
is set to the standard readtable(NAMED-READTABLES:FIND-READTABLE :COMMON-LISP)
. -
Else,
*PACKAGE*
is set to theCL-USER
package and*READTABLE*
to the standard readtable.
The values thus determined come into effect after the name itself is printed, for printing of the arglist and the docstring.
CL-USER> (pax:document #'foo)
- [function] FOO <!> X Y &KEY (ERRORP T)
Do something with X and Y.
In the above, the <!>
marks the place where *PACKAGE*
and
*READTABLE*
are bound.
The home section of an object is a SECTION
that contains the
object's definition in its SECTION-ENTRIES
or NIL
. In the
overwhelming majority of cases there should be at most one
containing section.
If there are multiple containing sections, the following apply.
-
If the name of the definition is a non-keyword symbol, only those containing sections are considered whose package is closest to the
SYMBOL-PACKAGE
of the name, where closest is defined as having the longest common prefix between the twoPACKAGE-NAME
s. -
If there are multiple sections with equally long matches or the name is not a non-keyword symbol, then it's undefined which one is the home section.
For example, (MGL-PAX:DOCUMENT FUNCTION)
is an entry in the
MGL-PAX::@BASICS
section. Unless another section that contains
it is defined in the MGL-PAX package, the home section is guaranteed
to be MGL-PAX::@BASICS
because the SYMBOL-PACKAGE
s of
MGL-PAX:DOCUMENT
and MGL-PAX::@BASICS
are the same (hence their
common prefix is maximally long).
This scheme would also work, for example, if the home package
of DOCUMENT
were MGL-PAX/IMPL
, and it were reexported from
MGL-PAX
because the only way to externally change the home package
would be to define a containing section in a package like
MGL-PAX/IMP
.
Thus, relying on the package system makes it possible to find the intended home section of a definition among multiple containing sections with high probability. However, for names which are not symbols, there is no package system to advantage of.
-
[variable] *DOCUMENT-NORMALIZE-PACKAGES* T
Whether to print
[in package <package-name>]
in the documentation when the package changes.
- [system] "mgl-pax/document"
- Description: Support for Generating Documentation in MGL-PAX.
- Long Description: Do not declare a dependency on this system. It is
autoloaded by Elisp or by accessing the functionality provided if the
mgl-pax
system is loaded. - Licence: MIT, see COPYING.
- Author: Gábor Melis
- Mailto: mega@retes.hu
- Depends on: 3bmd, 3bmd-ext-code-blocks, alexandria, colorize, md5, mgl-pax/navigate, mgl-pax/transcribe, trivial-utf-8
- Defsystem depends on: mgl-pax.asdf
Documentation for definitions in the running Lisp can be browsed
directly without generating documentation in the offline manner.
HTML documentation, complete with Codification and Linking, is
generated from docstrings of all kinds of definitions and PAX
SECTION
s in the running Lisp on the fly. This allows ad-hoc
exploration of the Lisp, much like describe-function
,
apropos-command
and other online help commands in Emacs, for which
direct parallels are provided.
Still, even without Emacs and SLIME, limited functionality can be accessed through PAX Live Home Page by starting the live documentation web server manually.
If Emacs Setup has been done, the Elisp function
mgl-pax-document
(maybe bound to C-.
) generates and displays
documentation as a single HTML page. If necessary, a disambiguation
page is generated with the documentation of all matching
definitions. For example, to view the documentation of this very
SECTION
, one can do:
M-x mgl-pax-document
View Documentation of: pax::@browsing-live-documentation
Alternatively, pressing C-.
with point over the text
pax::@browsing-live-documentation
in a buffer achieves the same
effect.
In interactive use, mgl-pax-document
behaves similarly to
M-.
except:
-
It shows the
DOCUMENT
ation of some definition and does not visit itsSOURCE-LOCATION
. -
It considers definitions with all
LOCATIVE-TYPES
not justLISP-LOCATIVE-TYPES
because it doesn't needSOURCE-LOCATION
.This also means that completion works for
CLHS
definitions:-
"lambda list<TAB>
lists"lambda list"
and"lambda list keywords"
, both HyperSpec glossary entries. This is similar tocommon-lisp-hyperspec-glossary-term
in Elisp but also works for HyperSpec section titles. -
"#<TAB>
lists all sharpsign reader macros (similar tocommon-lisp-hyperspec-lookup-reader-macro
in Elisp). -
"~<TAB>
lists allCL:FORMAT
directives (similar tocommon-lisp-hyperspec-format
in Elisp). -
"loop:~<TAB>
lists all loop keywords.
-
-
It works in non-
lisp-mode
buffers by reinterpreting a few lines of text surrounding point as lisp code (hence the suggested global binding). -
It supports fragment syntax at the prompt:
NAME LOCATIVE FRAGMENT-NAME FRAGMENT-LOCATIVE
This is like
NAME LOCATIVE
, but the browser scrolls to the definition ofFRAGMENT-NAME FRAGMENT-LOCATIVE
within that page.For example, entering this at the prompt will generate the entire PAX manual as a single page and scroll to the very section you are reading within it:
pax::@pax-manual pax:section pax::@browsing-live-documentation pax:section
-
If the empty string is entered at the prompt, and there is no existing w3m buffer or w3m is not used, then PAX Live Home Page is visited. If there is a w3m buffer, then entering the empty string displays that buffer.
The convenience function
mgl-pax-current-definition-toggle-view
(C-c C-d c
) documents the
definition with point in it.
When the value of the Elisp variable mgl-pax-browser-function
is w3m-browse-url
(see Emacs Setup), the Emacs w3m browser is
used without the need for a web server, and also offering somewhat
tighter integration than Browsing with Other Browsers.
With w3m's default key bindings, moving the cursor between links involves
TAB
and S-TAB
(or <up>
and <down>
). RET
and <right>
follow a link, while B
and <left>
go back in history.
In addition, the following PAX-specific key bindings are available:
-
M-.
visits the source location of the definition corresponding to the link under the point. -
Invoking
mgl-pax-document
on a section title link will show the documentation of that section on its own page. -
n
moves to the next PAX definition on the page. -
p
moves to the previous PAX definition on the page. -
u
follows the firstUp:
link (to the first containingSECTION
) if any. -
U
is likeu
but positions the cursor at the top of the page. -
v
visits the source location of the current definition (the one under the cursor or the first one above it). -
V
visits the source location of the first definition on the page.
When the value of the Elisp variable mgl-pax-browser-function
is not w3m-browse-url
(see Emacs Setup), requests are served via
a web server started in the running Lisp, and documentation is most
likely displayed in a separate browser window.
By default, mgl-pax-browser-function
is nil
, which makes PAX use
browse-url-browser-function
. You may want to customize the related
browse-url-new-window-flag
or, for Chrome, set
browse-url-chrome-arguments
to ("--new-window")
.
By default, mgl-pax-web-server-port
is nil
, and PAX will pick a
free port automatically.
In the browser, clicking on the locative on the left of the
name (e.g. in - [function] PRINT
) will raise and focus the Emacs
window (if Emacs is not in text mode, and also subject to window
manager focus stealing settings), then go to the corresponding
source location. For sections, clicking on the lambda link will do
the same (see *DOCUMENT-FANCY-HTML-NAVIGATION*
).
Finally, note that the URL
s exposed by the web server are subject to
change, and even the port used may vary by session if the Elisp
variable mgl-pax-web-server-port
is nil.
-
[variable] *BROWSE-HTML-STYLE* :CHARTER
The HTML style to use for browsing live documentation. Affects only non-w3m browsers. See
*DOCUMENT-HTML-DEFAULT-STYLE*
for the possible values.If you change this variable, you may need to do a hard refresh in the browser (often
C-<f5>
).
The Elisp functions mgl-pax-apropos
, mgl-pax-apropos-all
, and
mgl-pax-apropos-package
can display the results of DREF-APROPOS
in
the live documentation browser.
These extend the functionality of slime-apropos
,
slime-apropos-all
and slime-apropos-package
to support more
kinds of definitions in an extensible way. The correspondence
is so close that the PAX versions might take over the Slime key
bindings.
Note that apropos functionality is also exposed via the PAX Live Home Page.
More concretely, the PAX versions supports the following extensions:
-
Definitions with string names. One can search for
ASDF:SYSTEM
s,PACKAGE
s andCLHS
sections, glossary entries, format directives, reader macro characters, loop keywords. -
Exact or substring matching of the name and the package.
-
Matching only symbol or string names.
On the PAX Live Home Page, one may Browse by Locative Types, which gives access to some of the apropos functionality via the browser without involving Emacs.
On the result page:
-
A
DREF-APROPOS
form to reproduce the results at the REPL is shown. -
One may toggle the
EXTERNAL-ONLY
andCASE-SENSITIVE
boolean arguments. -
One may switch between list, and detailed view. The list view only shows the first, bulleted line for each definition, while the detailed view includes the full documentation of definitions with the exception of
SECTION
s. -
The returned references are presented in two groups: those with non-symbol and those with symbol names. The non-symbol group is sorted by locative type then by name. The symbol group is sorted by name then by locative type.
With mgl-pax-apropos-all
and mgl-pax-apropos-package
being
simple convenience functions on top of mgl-pax-apropos
, we only
discuss the latter in detail here. For the others, see the Elisp
docstrings.
The STRING
argument consists of a name pattern and a DTYPE
.
The name pattern has the following forms.
-
:print
matches definitions whose names are the stringprint
or a symbol withSYMBOL-NAME
print
. Vertical bar form as in:|prInt|
is also also supported and is useful in whenCASE-SENSITIVE
is true. -
"print"
matches definitions whose names containprint
as a substring. -
print
is like the previous, substring matching case. Use this form to save typing if the pattern does not contain spaces and does not start with a colon. -
The empty string matches everything.
After the name pattern, STRING
may contain a
DTYPE
that the definitions must match.
-
print t
matches definitions withLISP-LOCATIVE-TYPES
, which is the default (equivalent toprint
). -
print function
matches functions whose names containprint
(e.g.CL:PRINT
andCL:PPRINT
). -
:print function
is like the previous example but with exact name match (so it matchesCL:PRINT
but notCL:PPRINT
). -
print variable
matches for example*PRINT-ESCAPE*
. -
print (or variable function)
matches all variables and functions withprint
in their names. -
array (or type (not class))
matchesDEFTYPE
s and but notCLASS
es with the stringarray
in their names. -
Â
pax:section
(note the leading space) matches all PAX sections (EXTERNAL-ONLY
NIL
is necessary to see many of them). -
print dref:pseudo
matches definitions withPSEUDO-LOCATIVE-TYPES
such asMGL-PAX:CLHS
. -
print dref:top
matches definitions with all locative types (LOCATIVE-TYPES
).
When mgl-pax-apropos
is invoked with a prefix argument, it
prompts for a package pattern among other things. The pattern may be
like the following examples.
-
:none
restricts matches to non-symbol names. -
:any
restricts matches to symbol names. -
:cl
restricts matches to symbols in the CL package. -
:|X Y|
is similar to the previous, but the vertical bar syntax allows for spaces in names. -
mgl
restricts matches to packages whose name containsmgl
as a substring. -
"x y"
is the same as the previous, but the explicit quotes allow for spaces in names.
The above examples assume case-insensitive matching.
When Browsing Live Documentation, the home page provides
quick access to documentation of the definitions in the system. In
Emacs, when mgl-pax-document
is invoked with the empty string, it
visits the home page.
The home page may also be accessed directly by going to the root page of the web server (if one is started). Here, unless the home page is viewed with w3m, one may directly look up documentation and access Apropos via the input boxes provided.
-
[function] ENSURE-WEB-SERVER &KEY PORT HYPERSPEC-ROOT
Start or update a web server on
PORT
for Browsing Live Documentation. Returns the baseURL
of the server (e.g.http://localhost:32790
), which goes to the PAX Live Home Page. If the web server is running already(ENSURE-WEB-SERVER)
simply returns its baseURL
.Note that even when using Emacs but Browsing with Other Browsers, the web server is started automatically. When Browsing with w3m, no web server is involved at all. Calling this function explicitly is only needed if the Emacs integration is not used, or to override
PORT
andHYPERSPEC-ROOT
.-
If
PORT
isNIL
or 0, then the server will use any free port. -
If there is a server already running and
PORT
is notNIL
or 0, then the server is restarted onPORT
. -
If
HYPERSPEC-ROOT
isNIL
, the HyperSpec pages will be served from any previously providedHYPERSPEC-ROOT
or, failing that, from*DOCUMENT-HYPERSPEC-ROOT*
. -
If
HYPERSPEC-ROOT
is non-NIL
, then pages in the HyperSpec will be served fromHYPERSPEC-ROOT
. The following command changes the root without affecting the server in any other way:(ensure-web-server :hyperspec-root "/usr/share/doc/hyperspec/")
-
The PAX Live Home Page lists the top-level PAX sections: those
that have no other SECTION
s referencing them (see DEFSECTION
).
The PAX Live Home Page lists all ASDF:SYSTEM
s and PACKAGE
s in the Lisp.
For easier overview, the they are grouped based on their
SOURCE-LOCATION
s. Two systems are in the same group if the directory
of one (i.e. the directory of the .asd
file in which it was
defined) is the same or is below the other's.
A PACKAGE
presented under a group of systems, if the SOURCE-LOCATION
of the package is below the the top-most directory among the systems
in the group.
The PAX Live Home Page lists PACKAGE
s
unrelated to any ASDF:SYSTEM
as systemless.
The PAX Live Home Page provides quick links to Apropos result pages for all Basic Locative Types which may have definitions.
-
[glossary-term] related
Two definitions are related if the directory of one's
SOURCE-LOCATION
s contains the directory of the other's.
The Markdown in docstrings is processed with the 3BMD library.
-
Docstrings can be indented in any of the usual styles. PAX normalizes indentation by stripping the longest run of leading spaces common to all non-blank lines except the first. The following two docstrings are equivalent:
(defun foo () "This is indented differently") (defun foo () "This is indented differently")
-
When Browsing Live Documentation, the page displayed can be of, say, a single function within what would constitute the offline documentation of a library. Because markdown reference link definitions, for example
[Daring Fireball]: http://daringfireball.net/
can be defined anywhere, they wouldn't be resolvable in that case, their use is discouraged. Currently, only reflink definitions in the vicinity of their uses are resolvable. This is left intentionally vague because the specifics are subject to change.
See
DEFINE-GLOSSARY-TERM
for a better alternative to markdown reference links.
Docstrings of definitions which do not have a Home Section and are
not SECTION
s themselves are assumed to have been written with no
knowledge of PAX and to conform to markdown only by accident. These
docstrings are thus sanitized more aggressively.
-
Indentation of what looks like blocks of Lisp code is rounded up to a multiple of 4. More precisely, non-zero indented lines between blank lines or the docstring boundaries are reindented if the first non-space character of the first line is an
(
or a;
character. -
Special HTML characters
<&
are escaped. -
Furthermore, to reduce the chance of inadvertently introducing a markdown heading, if a line starts with a string of
#
characters, then the first one is automatically escaped. Thus, the following two docstrings are equivalent:The characters #\Space, #\Tab and #Return are in the whitespace group. The characters #\Space, #\Tab and \#Return are in the whitespace group.
For syntax highlighting, GitHub's fenced code blocks markdown extension to mark up code blocks with triple backticks is enabled so all you need to do is write:
```elisp
(defun foo ())
```
to get syntactically marked up HTML output. Copy src/style.css
from PAX and you are set. The language tag, elisp
in this example,
is optional and defaults to common-lisp
.
See the documentation of 3BMD and Colorize for the details.
Displaying pretty mathematics in TeX format is supported via
MathJax. It can be done inline with $
like this:
$\int_0^\infty e^{-x^2} dx=\frac{\sqrt{\pi}}{2}$
which is displayed as $$
like this:
$$\int_0^\infty e^{-x^2} dx=\frac{\sqrt{\pi}}{2}$$
to get:
MathJax will leave code blocks (including those inline with
backticks) alone. Outside code blocks, escape $
by prefixing it
with a backslash to scare MathJax off.
Escaping all those backslashes in TeX fragments embedded in Lisp strings can be a pain. Pythonic String Reader can help with that.
-
[variable] *DOCUMENT-UPPERCASE-IS-CODE* T
When true, interesting names extracted from codifiable words marked up as code with backticks. For example, this docstring
"T PRINT CLASSes SECTION *PACKAGE* MGL-PAX ASDF CaMeL Capital"
is equivalent to this:
"`T` `PRINT` `CLASS`es `SECTION` `*PACKAGE*` `MGL-PAX` `ASDF` CaMel Capital"
and renders as
T
PRINT
CLASS
esSECTION
MGL-PAX
ASDF
CaMel Capitalwhere the links are added due to
*DOCUMENT-LINK-CODE*
.To suppress codification, add a backslash to the beginning of the a codifiable word or right after the leading
*
if it would otherwise be parsed as markdown emphasis:"\\SECTION *\\PACKAGE*"
The number of backslashes is doubled above because that's how the example looks in a docstring. Note that the backslash is discarded even if
*DOCUMENT-UPPERCASE-IS-CODE*
is false.
-
[glossary-term] codifiable
A word is codifiable if
-
it has a single uppercase character (e.g. it's
T
) and no lowercase characters at all, or -
there is more than one uppercase character and no lowercase characters between them (e.g.
CLASS
es, nonREAD
able,CLASS-NAME
s but notClasses
oraTe
.
-
-
[glossary-term] interesting
A name is interesting if
-
it names a symbol external to its package, or
-
it is at least 3 characters long and names an interned symbol, or
-
it names a Local Definition.
-
-
[variable] *DOCUMENT-DOWNCASE-UPPERCASE-CODE* NIL
If true, then all Markdown inline code (e.g. `code`, which renders as
code
) – including Codification – which has no lowercase characters is downcased in the output. Characters of literal strings in the code may be of any case. If this variable is:ONLY-IN-MARKUP
and the output format does not support markup (e.g. it's:PLAIN
), then no downcasing is performed. For example,`(PRINT "Hello")`
is downcased to
`(print "Hello")`
because it only contains uppercase characters outside the string. However,
`MiXed "RESULTS"`
is not altered because it has lowercase characters.
If the first two characters are backslashes, then no downcasing is performed, in addition to Escaping Autolinking. Use this to mark inline code that's not Lisp.
Press `\\M-.` in Emacs.
PAX supports linking to definitions either with explicit Reflinks or with Autolinks.
When generating offline documentation, only the definitions in
DOCUMENTABLE
may be linkable, but when
Browsing Live Documentation, everything is linkable as
documentation is generated on-demand.
Many examples in this section link to standard Common Lisp definitions. In the offline case, these will link to external URLs, while in the live case to disambiguation pages that list the definition in the running Lisp and in the HyperSpec.
Invoking M-.
on WORD
or NAME
in
any of the following examples will disambiguate based on the textual
context, determining the locative. This is because navigation and
linking use the same Parsing algorithm, although linking is a bit
more strict about trimming, depluralization, and it performs
Filtering Links. On the other hand, M-.
cannot visit the
CLHS
references because there are no associated source
locations.
The Markdown reference link syntax [label][id]
is
repurposed for linking to definitions. In the following, we
discuss the various forms of reflinks.
The first name in WORD
(with depluralization) that forms a valid
DREF
with LOCATIVE
is determined, and that definition is
linked to. If there is no such DREF
, then an UNRESOLVABLE-REFLINK
warning is signalled.
Examples:
The Markdown link definition (i.e. type
above) needs no backticks
to mark it as code, but here and below, the second example relies on
*DOCUMENT-UPPERCASE-IS-CODE*
being true.
Format: [LINK TEXT][
NAME
LOCATIVE
]
If NAME
and LOCATIVE
form a valid DREF
, then that
definition is linked to with link text LINK TEXT
. If there is no
such DREF
, then an UNRESOLVABLE-REFLINK
warning is signalled.
In this form, if NAME
starts with #\"
, then it's read as a string,
else as a symbol.
Examples:
-
[see this][eql type]
renders as see this. -
[see this]["MGL-PAX" package]
renders as see this.
Format: [
WORD
][]
The first name in WORD
(with depluralization, symbols only) that
has some DEFINITIONS
is determined, and those definitions are linked
to. If no name with any definition is found, then an
UNRESOLVABLE-REFLINK
warning is signalled.
Examples:
-
single link:
[PRINT][]
renders asPRINT
. -
no definitions:
[BAD-NAME][]
renders as BAD-NAME.
Format: [LINK TEXT][
NAME
]
The DEFINITIONS
of NAME
are determined, and those definitions are
linked to. If NAME
has no definitions, then an UNRESOLVABLE-REFLINK
warning is signalled.
In this form, if NAME
starts with #\"
, then it's read as a string,
else as as symbol.
Examples:
Format: [label][id]
This is a normal Markdown reference link if id
is not a valid locative.
-
[see this][user-defined]
renders unchanged.(dref:dref 'user-defined 'locative) .. debugger invoked on LOCATE-ERROR: .. Could not locate USER-DEFINED LOCATIVE. .. USER-DEFINED is not a valid locative type or locative alias.
(document "[see this][user-defined]" :format :markdown) .. [see this][user-defined] ..
Use URL
s with DEFINE-GLOSSARY-TERM
as a better alternative to
Markdown reference links (see Markdown in Docstrings).
-
[condition] UNRESOLVABLE-REFLINK WARNING
When
DOCUMENT
encounters a Reflink that looks like a PAX construct but has no matching definition, it signals anUNRESOLVABLE-REFLINK
warning.-
If the
OUTPUT-REFLINK
restart is invoked, then no warning is printed and the markdown link is left unchanged.MUFFLE-WARNING
(0
1
) is equivalent toOUTPUT-REFLINK
. -
If the
OUTPUT-LABEL
restart is invoked, then no warning is printed and the markdown link is replaced by its label. For example,[NONEXISTENT][function]
becomesNONEXISTENT
. -
If the warning is not handled, then it is printed to
*ERROR-OUTPUT*
, and it behaves as ifOUTPUT-LABEL
was invoked.
-
-
[function] OUTPUT-REFLINK &OPTIONAL CONDITION
Invoke the
OUTPUT-REFLINK
restart. SeeUNRESOLVABLE-REFLINK
.
-
[function] OUTPUT-LABEL &OPTIONAL CONDITION
Invoke the
OUTPUT-LABEL
restart. SeeUNRESOLVABLE-REFLINK
.
Markdown inline code automatically links to the corresponding
definitions without having to use Reflinks. This works especially
well in conjunction with Codification. The following examples
assume that *DOCUMENT-UPPERCASE-IS-CODE*
is true. If that's not the
case, explicit backticks are required on WORD
(but not on
LOCATIVE
).
Format: WORD
LOCATIVE
or
LOCATIVE
WORD
The first name in WORD
(with depluralization) that forms a valid
DREF
with LOCATIVE
is determined, and that definition is
linked to. If no such name is found, then Unspecific Autolink is
attempted.
Examples:
-
PRINT function
renders asPRINT
function. -
type EQL
renders as typeEQL
. -
type EQL function
renders as typeEQL
function.
If LOCATIVE
has spaces, then it needs to be marked up as code, too.
For example,
DREF-NAME `(reader dref)`
renders as DREF-NAME
(reader dref)
.
Format: WORD
The first name in WORD
(with depluralization, symbols only) that
has some DEFINITIONS
is determined, and those definitions are linked
to. If no such name is found or the autolink to this name is
suppressed (see below), then WORD
is left unchanged. If a locative
is found before or after WORD
, then Specific Autolink is tried
first.
Examples:
Unspecific Autolinking is suppressed if the name found has a Local Definition or was linked to before in the same docstring:
-
"
My other CAR is also a CAR
" renders as "My otherCAR
is also aCAR
". -
"
[COS][] and COS
" renders as "COS
andCOS
". -
"
[EQL][type] and EQL
" renders as "EQL
andEQL
". -
"
EQ and the EQ function
" renders as "EQ
and theEQ
function".
Unspecific Autolinking to T
and NIL
is also suppressed (see
*DOCUMENT-LINK-TO-HYPERSPEC*
):
- "
T and NIL
" renders as "T
andNIL
".
As an exception, a single link (be it either a Specific Link or an
unambiguous Unspecific Link) to a SECTION
or GLOSSARY-TERM
is not
suppressed to allow their titles to be displayed properly:
In the common case, when *DOCUMENT-UPPERCASE-IS-CODE*
is true,
prefixing an uppercase word with a backslash prevents it from being
codified and thus also prevents Autolinking form kicking in. For
example,
\DOCUMENT
renders as DOCUMENT. If it should be marked up as code but not autolinked, the backslash must be within backticks like this:
`\DOCUMENT`
This renders as DOCUMENT
. Alternatively, the
DISLOCATED
or the ARGUMENT
locative may be used as in
[DOCUMENT][dislocated]
.
-
[variable] *DOCUMENT-LINK-TO-HYPERSPEC* T
If true, consider definitions found in the Common Lisp HyperSpec for linking. For example,
PRINT
renders asPRINT
.
In offline documentation, this would be a link to the hyperspec unless
#'PRINT
in the running Lisp isDOCUMENTABLE
.When Browsing Live Documentation, everything is linkable, so the generated link will go to a disambiguation page that lists the definition in the Lisp and in the HyperSpec.
Locatives work as expected (see
*DOCUMENT-LINK-CODE*
):FIND-IF
links toFIND-IF
,FUNCTION
links toFUNCTION
(0
1
), and[FUNCTION][type]
links toFUNCTION
.Unspecific Autolinking to
T
andNIL
is suppressed. If desired, use Reflinks such as[T][]
(that links toT
(0
1
)) or[T][constant]
(that links toT
).Note that linking explicitly with the
CLHS
locative is not subject to the value of this variable.
-
[variable] *DOCUMENT-HYPERSPEC-ROOT* "http://www.lispworks.com/documentation/HyperSpec/"
A URL of the Common Lisp HyperSpec. The default value is the canonical location. When invoked from Emacs, the Elisp variable
common-lisp-hyperspec-root
is in effect.
The following variables control how to generate section numbering, table of contents and navigation links.
-
[variable] *DOCUMENT-LINK-SECTIONS* T
When true, HTML anchors are generated before the headings (e.g. of sections), which allows the table of contents to contain links and also code-like references to sections (like
@FOO-MANUAL
) to be translated to links with theTITLE
being the link text.
-
[variable] *DOCUMENT-MAX-NUMBERING-LEVEL* 3
A non-negative integer. In their hierarchy, sections on levels less than this value get numbered in the format of
3.1.2
. Setting it to 0 turns numbering off.
-
[variable] *DOCUMENT-MAX-TABLE-OF-CONTENTS-LEVEL* 3
An integer that determines the depth of the table of contents.
-
If negative, then no table of contents is generated.
-
If non-negative, and there are multiple top-level sections on a page, then they are listed at the top of the page.
-
If positive, then for each top-level section a table of contents is printed after its heading, which includes a nested tree of section titles whose depth is limited by this value.
If
*DOCUMENT-LINK-SECTIONS*
is true, then the tables will link to the sections. -
-
[variable] *DOCUMENT-TEXT-NAVIGATION* NIL
If true, then before each heading a line is printed with links to the previous, parent and next section. Needs
*DOCUMENT-LINK-SECTIONS*
to be on to work.
-
[variable] *DOCUMENT-FANCY-HTML-NAVIGATION* T
If true and the output format is HTML, then headings get a navigation component that consists of links to the previous, parent, next section, a self-link, and a link to the definition in the source code if available (see
:SOURCE-URI-FN
inDOCUMENT
). This component is normally hidden, it is visible only when the mouse is over the heading. Has no effect if*DOCUMENT-LINK-SECTIONS*
is false.
-
[variable] *DOCUMENT-LINK-CODE* T
Whether definitions of things other than
SECTION
s are allowed to be linkable.
-
[glossary-term] linkable
When a reference is encountered to definition D while processing documentation for some page C, we say that definition D is linkable (from C) if
-
D denotes a
SECTION
and*DOCUMENT-LINK-SECTIONS*
is true, or -
D does not denote a
SECTION
and*DOCUMENT-LINK-CODE*
is true
... and
-
We are Browsing Live Documentation, or
-
D is an external definition (
CLHS
or denotes aGLOSSARY-TERM
with aURL
), or -
D's page is C, or
-
D's page is relativizable to C.
In the above,_D's page_ is the last of the pages in the
DOCUMENTABLE
to which D's documentation is written (see:OBJECTS
inPAGES
), and we say that a page is relativizable to another if it is possible to construct a relative link between their:URI-FRAGMENT
s. -
Specific links are those Reflinks and Autolinks that have a single locative and therefore at most a single matching definition. These are Specific Reflink, Specific Reflink with Text and Specific Autolink.
A specific link to a linkable definition produces a link in the output. If the definition is not linkable, then the output will contain only what would otherwise be the link text.
Unspecific links are those Reflinks and Autolinks that do not specify a locative and match all definitions with a name. These are Unspecific Reflink, Unspecific Reflink with Text and Unspecific Autolink.
To make the links predictable and manageable in number, the following steps are taken.
-
Definitions that are not symbol-based (i.e. whose
DREF-NAME
is not a symbol) are filtered out to prevent unrelatedPACKAGE
s,ASDF:SYSTEM
s andCLHS
sections from cluttering the documentation without the control provided by importing symbols. -
All references with
LOCATIVE-TYPE
LOCATIVE
are filtered out. -
Non-linkable definitions are removed.
-
If the definitions include a
GENERIC-FUNCTION
, then all definitions withLOCATIVE-TYPE
METHOD
,ACCESSOR
,READER
andWRITER
are removed to avoid linking to a possibly large number of methods.
If at most a single definition remains, then the output is the same as with a Specific Link. If multiple definitions remain, then the link text is output followed by a number of numbered links, one to each definition.
The following variables control various aspects of links and URL
s.
-
[variable] *DOCUMENT-URL-VERSIONS* (2 1)
A list of versions of PAX URL formats to support in the generated documentation. The first in the list is used to generate links.
PAX emits HTML anchors before the documentation of
SECTION
s (see Linking to Sections) and other things (see Linking). For the functionFOO
, in the current version (version 2), the anchor is<a id="MGL-PAX:FOO%20FUNCTION">
, and its URL will end with#MGL-PAX:FOO%20FUNCTION
.Note that to make the URL independent of whether a symbol is internal or external to their
SYMBOL-PACKAGE
, single colon is printed where a double colon would be expected. Package and symbol names are both printed verbatim except for escaping colons and spaces with a backslash. For exported symbols with no funny characters, this coincides with howPRIN1
would print the symbol, while having the benefit of making the URL independent of the Lisp printer's escaping strategy and producing human-readable output for mixed-case symbols. No such promises are made for non-ASCII characters, and their URLs may change in future versions. Locatives are printed withPRIN1
.Version 1 is based on the more strict HTML4 standard and the id of
FOO
is"x-28MGL-PAX-3A-3AFOO-20FUNCTION-29"
. This is supported by GitHub-flavoured Markdown. Version 2 has minimal clutter and is obviously preferred. However, in order not to break external links, by default, both anchors are generated.Let's understand the generated Markdown.
(defun foo (x)) (document #'foo :format :markdown) => ("<a id=\"x-28MGL-PAX-3AFOO-20FUNCTION-29\"></a> <a id=\"MGL-PAX:FOO%20FUNCTION\"></a> - [function] **FOO** *X* ") (let ((*document-url-versions* '(1))) (document #'foo :format :markdown)) => ("<a id=\"x-28MGL-PAX-3AFOO-20FUNCTION-29\"></a> - [function] **FOO** *X* ")
-
[variable] *DOCUMENT-MIN-LINK-HASH-LENGTH* 4
Recall that Markdown reference links (like
[label][id]
) are used for Linking. It is desirable to have ids that are short to maintain legibility of the generated markdown, but also stable to reduce the spurious diffs in the generated documentation, which can be a pain in a version control system.Clearly, there is a tradeoff here. This variable controls how many characters of the MD5 sum of the full link id (the reference as a string) are retained. If collisions are found due to the low number of characters, then the length of the hash of the colliding reference is increased.
This variable has no effect on the HTML generated from markdown, but it can make markdown output more readable.
-
[variable] *DOCUMENT-BASE-URL* NIL
When
*DOCUMENT-BASE-URL*
is non-NIL
, this is prepended to all Markdown relativeURL
s. It must be a validURL
without no query and fragment parts (that is, http://lisp.org/doc/ but not http://lisp.org/doc?a=1 or http://lisp.org/doc#fragment). Note that intra-page links using onlyURL
fragments (e.g. and explicit HTML links (e.g.<a href="...">
) in Markdown are not affected.
While documentation is generated for a definition, that definition is considered local. Other local definitions may also be established. Local definitions inform Codification through interesting names and affect Unspecific Autolinking.
(defun foo (x)
"FOO adds one to X."
(1+ x)
In this example, while the docstring of FOO
is being processed, the
global definition (DREF 'FOO 'FUNCTION)
is also considered local,
which suppresses linking FOO
in the FOO
's docstring back to its
definition. If FOO
has other definitions, Unspecific Autolinking to
those is also suppressed.
Furthermore, the purely local definition (DREF 'X 'ARGUMENT)
is
established, causing the argument name X
to be
codified because X
is now interesting.
See DOCUMENTING-REFERENCE
and WITH-DISLOCATED-NAMES
in
Extending DOCUMENT
.
Let's recap how escaping Codification, downcasing, and Linking works.
-
One backslash in front of a word turns codification off. Use this to prevent codification words such as DOCUMENT, which is all uppercase hence codifiable, and it names an exported symbol hence it is interesting.
-
One backslash right after an opening backtick turns autolinking off.
-
Two backslashes right after an opening backtick turns autolinking and downcasing off. Use this for things that are not Lisp code but which need to be in a monospace font.
In the following examples capital C/D/A letters mark the presence,
and a/b/c the absence of codification, downcasing, and autolinking
assuming all these features are enabled by
*DOCUMENT-UPPERCASE-IS-CODE*
, *DOCUMENT-DOWNCASE-UPPERCASE-CODE*
,
and *DOCUMENT-LINK-CODE*
.
DOCUMENT => [`document`][1234] (CDA)
\DOCUMENT => DOCUMENT (cda)
`\DOCUMENT` => `document` (CDa)
`\\DOCUMENT` => `DOCUMENT` (CdA)
[DOCUMENT][] => [`document`][1234] (CDA)
[\DOCUMENT][] => [DOCUMENT][1234] (cdA)
[`\DOCUMENT`][] => [`document`][1234] (CDA) *
[`\\DOCUMENT`][] => [`DOCUMENT`][1234] (CdA)
[DOCUMENT][dislocated] => `document` (CDa)
Note that in the example marked with *
, the single backslash,
that would normally turn autolinking off, is ignored because it is
in an explicit link.
By default, DREF
s are documented in the following format.
- [<locative-type>] <name> <arglist>
<docstring>
The line with the bullet is printed with DOCUMENTING-REFERENCE
. The
docstring is processed with DOCUMENT-DOCSTRING
while
Local Definitions established with WITH-DISLOCATED-NAMES
are in
effect for all variables locally bound in a definition with ARGLIST
,
and *PACKAGE*
is bound to the second return value of DOCSTRING
.
With this default format, PAX supports all locative types, but for some Basic Locative Types defined in DRef and the PAX Locatives, special provisions have been made.
- For definitions with a
VARIABLE
orCONSTANT
locative, their initform is printed as their arglist. The initform is theINITFORM
argument of the locative if provided, or the global symbol value of their name. If noINITFORM
is provided, and the symbol is globally unbound, then no arglist is printed.
When the printed initform is too long, it is truncated.
-
Depending of what the
SETF
locative refers to, theARGLIST
of the setf expander, setf function, or the method signature is printed as with theMETHOD
locative. -
For definitions with a
METHOD
locative, the arglist printed is the method signature, which consists of the locative'sQUALIFIERS
andSPECIALIZERS
appended. -
For definitions with an
ACCESSOR
,READER
orWRITER
locative, the class on which they are specialized is printed as their arglist. -
For definitions with a
STRUCTURE-ACCESSOR
locative, the arglist printed is the locative'sCLASS-NAME
argument if provided. -
For definitions with a
CLASS
locative, the arglist printed is the list of immediate superclasses withSTANDARD-OBJECT
,CONDITION
and non-exported symbols omitted. -
For definitions with a
CONDITION
locative, the arglist printed is the list of immediate superclasses withSTANDARD-OBJECT
,CONDITION
and non-exported symbols omitted. -
For definitions with a
ASDF:SYSTEM
locative, their most important slots are printed as an unnumbered list. -
For definitions with the
LOCATIVE
locative type, theirLOCATIVE-TYPE-DIRECT-SUPERS
andLOCATIVE-TYPE-DIRECT-SUBS
are printed. -
When documentation is being generated for a definition with the
SECTION
locative, a new (sub)section is opened (seeWITH-HEADING
), within which documentation for its each of itsSECTION-ENTRIES
is generated. A fresh line is printed after all entries except the last. -
For definitions with a
GLOSSARY-TERM
locative, no arglist is printed, and if non-NIL
,GLOSSARY-TERM-TITLE
is printed as name. -
For definitions with a
GO
locative, itsLOCATIVE-ARGS
are printed as its arglist, along with a redirection message. -
See the
INCLUDE
locative. -
For definitions with a
CLHS
locative, theLOCATIVE-ARGS
are printed as the arglist. ForCLHS
SECTION
s, the title is included in the arglist. -
For definitions with an
UNKNOWN
locative, theLOCATIVE-ARGS
are printed as the arglist. There is no docstring.
-
[variable] *DOCUMENT-MARK-UP-SIGNATURES* T
When true, some things such as function names and arglists are rendered as bold and italic. In
:HTML
output, locative types become links to sources (if:SOURCE-URI-FN
is provided, seePAGES
), and the symbol becomes a self-link for your permalinking pleasure.For example, a reference is rendered in markdown roughly as:
- [function] foo x y
With this option on, the above becomes:
- [function] **foo** *x y*
Also, in HTML
**foo**
will be a link to that very entry and[function]
may turn into a link to sources.
Documentation Generation is supported on ABCL, AllegroCL, CLISP,
CCL, CMUCL, ECL and SBCL, but their outputs may differ due to the
lack of some introspective capability. SBCL generates complete
output. see ARGLIST
, DOCSTRING
and SOURCE-LOCATION
for
implementation notes.
In addition, CLISP does not support the ambiguous case of
Browsing Live Documentation because the current implementation
relies on Swank to list definitions of symbols (as
VARIABLE
, FUNCTION
, etc), and that simply
doesn't work.
Two convenience functions are provided to serve the common case of having an ASDF system with some readmes and a directory with for the HTML documentation and the default CSS stylesheet.
-
[function] UPDATE-ASDF-SYSTEM-READMES OBJECT ASDF-SYSTEM &KEY (URL-VERSIONS '(1)) (FORMATS '(:MARKDOWN))
Convenience function to generate up to two readme files in the directory holding the
ASDF-SYSTEM
definition.OBJECT
is passed on toDOCUMENT
.If
:MARKDOWN
is inFORMATS
, thenREADME.md
is generated with anchors, links, inline code, and other markup added. Not necessarily the easiest on the eye in an editor but looks good on GitHub.If
:PLAIN
is inFORMATS
, thenREADME
is generated, which is optimized for reading in text format. It has less cluttery markup and no Autolinking.Example usage:
(update-asdf-system-readmes @pax-manual :mgl-pax :formats '(:markdown :plain))
Note that
*DOCUMENT-URL-VERSIONS*
is bound toURL-VERSIONS
, which defaults to using the uglier, version 1 style ofURL
for the sake of GitHub.
-
[function] UPDATE-ASDF-SYSTEM-HTML-DOCS SECTIONS ASDF-SYSTEM &KEY PAGES (TARGET-DIR (ASDF/SYSTEM:SYSTEM-RELATIVE-PATHNAME ASDF-SYSTEM "doc/")) (UPDATE-CSS-P T) (STYLE *DOCUMENT-HTML-DEFAULT-STYLE*)
Generate pretty HTML documentation for a single ASDF system, possibly linking to GitHub. If
UPDATE-CSS-P
, copy theSTYLE
files toTARGET-DIR
(see*DOCUMENT-HTML-DEFAULT-STYLE*
).Example usage:
(update-asdf-system-html-docs @pax-manual :mgl-pax)
The same, linking to the sources on GitHub:
(update-asdf-system-html-docs @pax-manual :mgl-pax :pages `((:objects (,mgl-pax::@pax-manual) :source-uri-fn ,(make-git-source-uri-fn :mgl-pax "https://github.com/melisgl/mgl-pax"))))
See the following variables, which control HTML generation.
-
[variable] *DOCUMENT-HTML-DEFAULT-STYLE* :DEFAULT
The HTML style to use. It's either
STYLE
is either:DEFAULT
or:CHARTER
. The:DEFAULT
CSS stylesheet relies on the default fonts (sans-serif, serif, monospace), while:CHARTER
bundles some fonts for a more controlled look.The value of this variable affects the default style of
UPDATE-ASDF-SYSTEM-HTML-DOCS
.
-
[variable] *DOCUMENT-HTML-MAX-NAVIGATION-TABLE-OF-CONTENTS-LEVEL* NIL
NIL
or a non-negative integer. If non-NIL
, it overrides*DOCUMENT-MAX-NUMBERING-LEVEL*
in the dynamic HTML table of contents on the left of the page.
-
[variable] *DOCUMENT-HTML-LANG* "en"
The value for the
html
element'sxml:lang
andlang
attributes in the generated HTML.
-
[variable] *DOCUMENT-HTML-CHARSET* "UTF-8"
The value for
charset
attribute of the<meta http-equiv='Content-Type' content='text/html'>
element in the generated HTML.
-
[variable] *DOCUMENT-HTML-HEAD* NIL
Stuff to be included in the
<head>
of the generated HTML.
-
[variable] *DOCUMENT-HTML-SIDEBAR* NIL
Stuff to be included in the HTML sidebar.
-
If
NIL
, a default sidebar is generated, with*DOCUMENT-HTML-TOP-BLOCKS-OF-LINKS*
, followed by the dynamic table of contents, and*DOCUMENT-HTML-BOTTOM-BLOCKS-OF-LINKS*
. -
If a
STRING
(0
1
), then it is written to the HTML output as is without any escaping. -
If a function designator, then it is called with a single argument, the HTML stream, where it must write the output.
-
-
[variable] *DOCUMENT-HTML-TOP-BLOCKS-OF-LINKS* NIL
A list of blocks of links to be displayed on the sidebar on the left, above the table of contents. A block is of the form
(&KEY TITLE ID LINKS)
, whereTITLE
will be displayed at the top of the block in a HTMLDIV
withID
followed by the links.LINKS
is a list of(URI LABEL)
elements, whereURI
maybe a string or an object beingDOCUMENT
ed or aREFERENCE
thereof.
-
[variable] *DOCUMENT-HTML-BOTTOM-BLOCKS-OF-LINKS* NIL
Like
*DOCUMENT-HTML-TOP-BLOCKS-OF-LINKS*
, only it is displayed below the table of contents.
It is generally recommended to commit generated readmes (see
UPDATE-ASDF-SYSTEM-READMES
), so that users have something to read
without reading the code and sites like GitHub can display them.
HTML documentation can also be committed, but there is an issue with
that: when linking to the sources (see MAKE-GIT-SOURCE-URI-FN
), the
commit id is in the link. This means that code changes need to be
committed first, and only then can HTML documentation be regenerated
and committed in a followup commit.
The second issue is that GitHub is not very good at serving HTML files from the repository itself (and http://htmlpreview.github.io chokes on links to the sources).
The recommended workflow is to use
gh-pages, which can be made relatively
painless with the git worktree
command. The gist of it is to make
the doc/
directory a checkout of the branch named gh-pages
.
There is a good
description
of this general process. Two commits are needed still, but it is
somewhat less painful.
This way the HTML documentation will be available at
http://<username>.github.io/<repo-name>
It is probably a good idea to add sections like the Links and Systems section to allow jumping between the repository and the gh-pages site.
-
[function] MAKE-GITHUB-SOURCE-URI-FN ASDF-SYSTEM GITHUB-URI &KEY GIT-VERSION
This function is a backward-compatibility wrapper around
MAKE-GIT-SOURCE-URI-FN
, which supersedesMAKE-GITHUB-SOURCE-URI-FN
. All arguments are passed on toMAKE-GIT-SOURCE-URI-FN
, leavingURI-FORMAT-STRING
at its default, which is suitable for GitHub.
-
[function] MAKE-GIT-SOURCE-URI-FN ASDF-SYSTEM GIT-FORGE-URI &KEY GIT-VERSION (URI-FORMAT-STRING "~A/blob/~A/
A#LS")Return an object suitable as
:SOURCE-URI-FN
of a page spec (see thePAGES
argument ofDOCUMENT
). The function looks at the source location of the object passed to it, and if the location is found, the path is made relative to the toplevel directory of the git checkout containing the file of theASDF-SYSTEM
and finally an URI pointing to your git forge (such as GitHub) is returned. A warning is signalled whenever the source location lookup fails or if the source location points to a directory not below the directory ofASDF-SYSTEM
.If
GIT-FORGE-URI
is"https://github.com/melisgl/mgl-pax/"
andGIT-VERSION
is"master"
, then the returned URI may look like this:https://github.com/melisgl/mgl-pax/blob/master/src/pax-early.lisp#L12
If
GIT-VERSION
isNIL
, then an attempt is made to determine to current commit id from the.git
in the directory holdingASDF-SYSTEM
. If no.git
directory is found, then no links to the git forge will be generated.URI-FORMAT-STRING
is aCL:FORMAT
control string for four arguments:-
GIT-FORGE-URI
, -
GIT-VERSION
, -
the relative path to the file of the source location of the reference,
-
and the line number.
The default value of
URI-FORMAT-STRING
is for GitHub. If using a non-standard git forge, such as Sourcehut or GitLab, simply pass a suitableURI-FORMAT-STRING
matching the URI scheme of your forge. -
PAX World is a registry of documents, which can generate cross-linked HTML documentation pages for all the registered documents. There is an official PAX World.
-
[function] REGISTER-DOC-IN-PAX-WORLD NAME SECTIONS PAGE-SPECS
Register
SECTIONS
andPAGE-SPECS
underNAME
(a symbol) in PAX World. By default,UPDATE-PAX-WORLD
generates documentation for all of these.SECTIONS
andPAGE-SPECS
must be lists ofSECTION
s andPAGE-SPEC
s (SEEDOCUMENT
) or designators of function of no arguments that return such lists.
For example, this is how PAX registers itself:
(defun pax-sections ()
(list @pax-manual))
(defun pax-pages ()
`((:objects ,(pax-sections)
:source-uri-fn ,(make-git-source-uri-fn
:mgl-pax
"https://github.com/melisgl/mgl-pax"))))
(register-doc-in-pax-world :pax 'pax-sections 'pax-pages)
-
[function] UPDATE-PAX-WORLD &KEY (DOCS *REGISTERED-PAX-WORLD-DOCS*) DIR UPDATE-CSS-P (STYLE *DOCUMENT-HTML-DEFAULT-STYLE*)
Generate HTML documentation for all
DOCS
. Files are created inDIR
((asdf:system-relative-pathname :mgl-pax "world/")
by default ifDIR
isNIL
).DOCS
is a list of entries of the form (NAME
SECTIONS
PAGE-SPECS
). The default forDOCS
is all the sections and pages registered withREGISTER-DOC-IN-PAX-WORLD
.In the absence of
:HEADER-FN
:FOOTER-FN
,:OUTPUT
, every spec inPAGE-SPECS
is augmented with HTML headers, footers and output location specifications (based on the name of the section).If necessary a default page spec is created for every section.
What are transcripts for? When writing a tutorial, one often wants to include a REPL session with maybe a few defuns and a couple of forms whose output or return values are shown. Also, in a function's docstring an example call with concrete arguments and return values speaks volumes. A transcript is a text that looks like a REPL session, but which has a light markup for printed output and return values, while no markup (i.e. prompt) for Lisp forms. PAX transcripts may include output and return values of all forms, or only selected ones. In either case, the transcript itself can be easily generated from the source code.
The main worry associated with including examples in the
documentation is that they tend to get out-of-sync with the code.
This is solved by being able to parse back and update transcripts.
In fact, this is exactly what happens during documentation
generation with PAX. Code sections tagged with cl-transcript
are
retranscribed and checked for consistency (that is, no difference in
output or return values). If the consistency check fails, an error
is signalled that includes a reference to the object being
documented.
Going beyond documentation, transcript consistency checks can be used for writing simple tests in a very readable form. For example:
(+ 1 2)
=> 3
(values (princ :hello) (list 1 2))
.. HELLO
=> :HELLO
=> (1 2)
All in all, transcripts are a handy tool especially when combined with the Emacs support to regenerate them and with PYTHONIC-STRING-READER's triple-quoted strings, that allow one to work with nested strings with less noise. The triple-quote syntax can be enabled with:
(in-readtable pythonic-string-syntax)
- [system] "mgl-pax/transcribe"
- Description: Support for Transcripts in MGL-PAX.
- Long Description: Do not declare a dependency on this system. It is
autoloaded by Elisp or by accessing the functionality provided if
the
mgl-pax
system is loaded. - Licence: MIT, see COPYING.
- Author: Gábor Melis
- Mailto: mega@retes.hu
- Depends on: alexandria, mgl-pax/navigate
- Defsystem depends on: mgl-pax.asdf
Typical transcript usage from within Emacs is simple: add a Lisp
form to a docstring or comment at any indentation level. Move the
cursor right after the end of the form as if you were to evaluate it
with C-x C-e
. The cursor is marked by #\^
:
This is part of a docstring.
```cl-transcript
(values (princ :hello) (list 1 2))^
```
Note that the use of fenced code blocks with the language tag
cl-transcript
is only to tell PAX to perform consistency checks at
documentation generation time.
Now invoke the Elisp function mgl-pax-transcribe
where the cursor
is, and the fenced code block from the docstring becomes:
(values (princ :hello) (list 1 2))
.. HELLO
=> :HELLO
=> (1 2)
^
Then you change the printed message and add a comment to the second return value:
(values (princ :hello-world) (list 1 2))
.. HELLO
=> :HELLO
=> (1
;; This value is arbitrary.
2)
When generating the documentation you get a
TRANSCRIPTION-CONSISTENCY-ERROR
because the printed output and the
first return value changed, so you regenerate the documentation by
marking the region bounded by #\|
and the cursor at #\^
in the
example:
|(values (princ :hello-world) (list 1 2))
.. HELLO
=> :HELLO
=> (1
;; This value is arbitrary.
2)
^
then invoke the Elisp function mgl-pax-retranscribe-region
to get:
(values (princ :hello-world) (list 1 2))
.. HELLO-WORLD
=> :HELLO-WORLD
=> (1
;; This value is arbitrary.
2)
^
Note how the indentation and the comment of (1 2)
were left alone,
but the output and the first return value got updated.
Alternatively, C-u 1 mgl-pax-transcribe
will emit commented markup:
(values (princ :hello) (list 1 2))
;.. HELLO
;=> :HELLO
;=> (1 2)
C-u 0 mgl-pax-retranscribe-region
will turn commented into
non-commented markup. In general, the numeric prefix argument is the
index of the syntax to be used in *TRANSCRIBE-SYNTAXES*
. Without a
prefix argument, mgl-pax-retranscribe-region
will not change the
markup style.
Finally, not only do both functions work at any indentation level but in comments too:
;;;; (values (princ :hello) (list 1 2))
;;;; .. HELLO
;;;; => :HELLO
;;;; => (1 2)
The dynamic environment of the transcription is determined by the
:DYNENV
argument of the enclosing cl-transcript
code block (see
Controlling the Dynamic Environment).
Transcription support in Emacs can be enabled by loading
src/mgl-pax.el
. See Emacs Setup.
-
[function] TRANSCRIBE INPUT OUTPUT &KEY UPDATE-ONLY (INCLUDE-NO-OUTPUT UPDATE-ONLY) (INCLUDE-NO-VALUE UPDATE-ONLY) (ECHO T) (CHECK-CONSISTENCY *TRANSCRIBE-CHECK-CONSISTENCY*) DEFAULT-SYNTAX (INPUT-SYNTAXES *TRANSCRIBE-SYNTAXES*) (OUTPUT-SYNTAXES *TRANSCRIBE-SYNTAXES*) DYNENV
Read forms from
INPUT
and write them (iffECHO
) toOUTPUT
followed by any output and return values produced by callingEVAL
on the form. The variables *, **, ***, /, //, ///, -, +, ++, +++ are locally bound and updated as in a REPL. SinceTRANSCRIBE
EVAL
uates arbitrary code anyway, forms are read with*READ-EVAL*
T
.INPUT
can be a stream or a string, whileOUTPUT
can be a stream orNIL
, in which case output goes into a string. The return value is theOUTPUT
stream or the string that was constructed.Go up to Transcribing with Emacs for nice examples. A more mind-bending one is this:
(transcribe "(princ 42) " nil) => "(princ 42) .. 42 => 42 "
However, the above may be a bit confusing since this documentation uses
TRANSCRIBE
markup syntax in this very example, so let's do it differently. If we have a file with these contents:(values (princ 42) (list 1 2))
it is transcribed to:
(values (princ 42) (list 1 2)) .. 42 => 42 => (1 2)
Output to all standard streams is captured and printed with the
:OUTPUT
prefix (".."
). The return values above are printed with the:READABLE
prefix ("=>"
). Note how these prefixes are always printed on a new line to facilitate parsing.Updating
TRANSCRIBE
is able to parse its own output. If we transcribe the previous output above, we get it back exactly. However, if we remove all output markers, leave only a placeholder value marker and pass:UPDATE-ONLY
T
with source:(values (princ 42) (list 1 2)) =>
we get this:
(values (princ 42) (list 1 2)) => 42 => (1 2)
With
UPDATE-ONLY
, the printed output of a form is transcribed only if there were output markers in the source. Similarly, withUPDATE-ONLY
, return values are transcribed only if there were value markers in the source.No Output/Values
If the form produces no output or returns no values, then whether or not output and values are transcribed is controlled by
INCLUDE-NO-OUTPUT
andINCLUDE-NO-VALUE
, respectively. By default, neither is on so:(values) .. =>
is transcribed to
(values)
With
UPDATE-ONLY
true, we probably wouldn't like to lose those markers since they were put there for a reason. Hence, withUPDATE-ONLY
,INCLUDE-NO-OUTPUT
andINCLUDE-NO-VALUE
default to true. So, withUPDATE-ONLY
the above example is transcribed to:(values) .. => ; No value
where the last line is the
:NO-VALUE
prefix.Consistency Checks
If
CHECK-CONSISTENCY
is true, thenTRANSCRIBE
signals a continuableTRANSCRIPTION-OUTPUT-CONSISTENCY-ERROR
whenever a form's output as a string is different from what was inINPUT
, provided thatINPUT
contained the output. Similarly, for values, a continuableTRANSCRIPTION-VALUES-CONSISTENCY-ERROR
is signalled if a value read from the source does not print as the as the value returned byEVAL
. This allows readable values to be hand-indented without failing consistency checks:(list 1 2) => ;; This is commented, too. (1 ;; Funny indent. 2)
See Transcript Consistency Checking for the full picture.
Unreadable Values
The above scheme involves
READ
, so consistency of unreadable values cannot be treated the same. In fact, unreadable values must even be printed differently for transcribe to be able to read them back:(defclass some-class () ()) (defmethod print-object ((obj some-class) stream) (print-unreadable-object (obj stream :type t) (format stream "~%~%end"))) (make-instance 'some-class) ==> #<SOME-CLASS --> --> end>
where
"==>"
is the:UNREADABLE
prefix and"-->"
is the:UNREADABLE-CONTINUATION
prefix. As with outputs, a consistency check between an unreadable value from the source and the value fromEVAL
is performed withSTRING=
by default. That is, the value fromEVAL
is printed to a string and compared to the source value. Hence, any change to unreadable values will break consistency checks. This is most troublesome with instances of classes with the defaultPRINT-OBJECT
method printing the memory address. See Finer-Grained Consistency Checks.Errors
If an
ERROR
condition is signalled, the error is printed to the output and no values are returned.(progn (print "hello") (error "no greeting")) .. .. "hello" .. debugger invoked on SIMPLE-ERROR: .. no greeting
To keep the textual representation somewhat likely to be portable, the printing is done with
(FORMAT T "#<~S ~S>" (TYPE-OF ERROR) (PRINC-TO-STRING ERROR))
.SIMPLE-CONDITION
s are formatted to strings withSIMPLE-CONDITION-FORMAT-CONTROL
andSIMPLE-CONDITION-FORMAT-ARGUMENTS
.Syntaxes
Finally, a transcript may employ different syntaxes for the output and values of different forms. When
INPUT
is read, the syntax for each form is determined by trying to match all prefixes from all syntaxes inINPUT-SYNTAXES
against a line. If there are no output or values for a form inINPUT
, then the syntax remains undetermined.When
OUTPUT
is written, the prefixes to be used are looked up inDEFAULT-SYNTAX
ofOUTPUT-SYNTAXES
, ifDEFAULT-SYNTAX
is notNIL
. IfDEFAULT-SYNTAX
isNIL
, then the syntax used by the same form in theINPUT
is used or (if that could not be determined) the syntax of the previous form. If there was no previous form, then the first syntax ifOUTPUT-SYNTAXES
is used.To produce a transcript that's executable Lisp code, use
:DEFAULT-SYNTAX
:COMMENTED-1
:(make-instance 'some-class) ;==> #<SOME-CLASS ;--> ;--> end> (list 1 2) ;=> (1 ;-> 2)
To translate the above to uncommented syntax, use
:DEFAULT-SYNTAX
:DEFAULT
. IfDEFAULT-SYNTAX
isNIL
(the default), the same syntax will be used in the output as in the input as much as possible.Dynamic Environment
If
DYNENV
is non-NIL
, then it must be a function that establishes the dynamic environment in which transcription shall take place. It is called with a single argument: a thunk (a function of no arguments). See Controlling the Dynamic Environment for an example.
-
[variable] *TRANSCRIBE-CHECK-CONSISTENCY* NIL
The default value of
TRANSCRIBE
'sCHECK-CONSISTENCY
argument.
-
[variable] *TRANSCRIBE-SYNTAXES* ((:DEFAULT (:OUTPUT "..") (:NO-VALUE "=> ; No value") (:READABLE "=>") (:UNREADABLE "==>") (:UNREADABLE-CONTINUATION "-->")) (:COMMENTED-1 (:OUTPUT ";..") (:NO-VALUE ";=> ; No value") (:READABLE ";=>") (:READABLE-CONTINUATION ";->") (:UNREADABLE ";==>") (:UNREADABLE-CONTINUATION ";-->")) (:COMMENTED-2 (:OUTPUT ";;..") (:NO-VALUE ";;=> ; No value") (:READABLE ";;=>") (:READABLE-CONTINUATION ";;->") (:UNREADABLE ";;==>") (:UNREADABLE-CONTINUATION ";;-->")))
The default syntaxes used by
TRANSCRIBE
for reading and writing lines containing output and values of an evaluated form.A syntax is a list of of the form
(SYNTAX-ID &REST PREFIXES)
wherePREFIXES
is a list of(PREFIX-ID PREFIX-STRING)
elements. For example the syntax:COMMENTED-1
looks like this:(:commented-1 (:output ";..") (:no-value ";=> No value") (:readable ";=>") (:readable-continuation ";->") (:unreadable ";==>") (:unreadable-continuation ";-->"))
All of the above prefixes must be defined for every syntax except for
:READABLE-CONTINUATION
. If that's missing (as in the:DEFAULT
syntax), then the following value is read withREAD
and printed withPRIN1
(hence no need to mark up the following lines).When writing, an extra space is added automatically if the line to be prefixed is not empty. Similarly, the first space following the prefix is discarded when reading.
See
TRANSCRIBE
for how the actual syntax to be used is selected.
-
[condition] TRANSCRIPTION-ERROR ERROR
Represents syntactic errors in the
SOURCE
argument ofTRANSCRIBE
and also serves as the superclass ofTRANSCRIPTION-CONSISTENCY-ERROR
.
-
[condition] TRANSCRIPTION-CONSISTENCY-ERROR TRANSCRIPTION-ERROR
A common superclass for
TRANSCRIPTION-OUTPUT-CONSISTENCY-ERROR
andTRANSCRIPTION-VALUES-CONSISTENCY-ERROR
.
-
[condition] TRANSCRIPTION-OUTPUT-CONSISTENCY-ERROR TRANSCRIPTION-CONSISTENCY-ERROR
Signalled (with
CERROR
) byTRANSCRIBE
when invoked with:CHECK-CONSISTENCY
and the output of a form is not the same as what was parsed.
-
[condition] TRANSCRIPTION-VALUES-CONSISTENCY-ERROR TRANSCRIPTION-CONSISTENCY-ERROR
Signalled (with
CERROR
) byTRANSCRIBE
when invoked with:CHECK-CONSISTENCY
and the values of a form are inconsistent with their parsed representation.
The main use case for consistency checking is detecting out-of-date examples in documentation, although using it for writing tests is also a possibility. Here, we focus on the former.
When a markdown code block tagged cl-transcript
is processed
during Generating Documentation, the code in it is replaced with
the output of with (TRANSCRIBE <CODE> NIL :UPDATE-ONLY T :CHECK-CONSISTENCY T)
. Suppose we have the following example of the
function GREET
, that prints hello
and returns 7.
```cl-transcript
(greet)
.. hello
=> 7
```
Now, if we change GREET
to print or return something else, a
TRANSCRIPTION-CONSISTENCY-ERROR
will be signalled during
documentation generation. Then we may fix the documentation or
CONTINUE
from the error.
By default, comparisons of previous to current output, readable and
unreadable return values are performed with STRING=
, EQUAL
, and
STRING=
, respectively, which is great in the simple case.
Non-determinism aside, exact matching becomes brittle as soon as the
notoriously unportable pretty printer is used or when unreadable
objects are printed with their #<>
syntax, especially when
PRINT-UNREADABLE-OBJECT
is used with :IDENTITY T
.
To get around this problem, consistency checking of output,
readable and unreadable values can be customized individually by
supplying TRANSCRIBE
with a CHECK-CONSISTENCY
argument
like ((:OUTPUT <OUTPUT-CHECK>) (:READABLE <READABLE-CHECK>) (:UNREADABLE <UNREADABLE-CHECK>))
. In this case,
<OUTPUT-CHECK>
may be NIL
, T
, or a function designator.
-
If it's
NIL
or there is no:OUTPUT
entry in the list, then the output is not checked for consistency. -
If it's
T
, then the outputs are compared with the default,STRING=
. -
If it's a function designator, then it's called with two strings and must return whether they are consistent with each other.
The case of <READABLE-CHECK>
and <UNREADABLE-CHECK>
is similar.
Code blocks tagged cl-transcript
can take arguments, which they
pass on to TRANSCRIBE
. The following shows how to check only the
output.
```cl-transcript (:check-consistency ((:output t)))
(error "Oh, no.")
.. debugger invoked on SIMPLE-ERROR:
.. Oh, no.
(make-condition 'simple-error)
==> #<SIMPLE-ERROR {1008A81533}>
The dynamic environment in which forms in the transcript are
evaluated can be controlled via the :DYNENV
argument of
cl-transcript
.
```cl-transcript (:dynenv my-transcript)
...
```
In this case, instead of calling TRANSCRIBE
directly, the call will
be wrapped in a function of no arguments and passed to the function
MY-TRANSCRIPT
, which establishes the desired dynamic environment
and calls its argument. The following definition of MY-TRANSCRIPT
simply packages up oft-used settings to TRANSCRIBE
.
(defun my-transcript (fn)
(let ((*transcribe-check-consistency*
'((:output my-transcript-output=)
(:readable equal)
(:unreadable nil))))
(funcall fn)))
(defun my-transcript-output= (string1 string2)
(string= (my-transcript-normalize-output string1)
(my-transcript-normalize-output string2)))
(defun my-transcript-normalize-output (string)
(squeeze-whitespace (delete-trailing-whitespace (delete-comments string))))
A more involved solution could rebind global variables set in transcripts, unintern symbols created or even create a temporary package for evaluation.
-
[function] SQUEEZE-WHITESPACE STRING
Replace consecutive whitespace characters with a single space in
STRING
. This is useful to undo the effects of pretty printing when building comparison functions forTRANSCRIBE
.
-
[function] DELETE-TRAILING-WHITESPACE STRING
Delete whitespace characters after the last non-whitespace character in each line in
STRING
.
-
[function] DELETE-COMMENTS STRING &KEY (PATTERN ";")
For each line in
STRING
delete the rest of the line after and including the first occurrence ofPATTERN
. On changed lines, delete trailing whitespace too. This function does not parseSTRING
as Lisp forms, hence all occurrences ofPATTERN
(even those seemingly in string literals) are recognized as comments.Let's define a comparison function:
(defun string=/no-comments (string1 string2) (string= (delete-comments string1) (delete-comments string2)))
And use it to check consistency of output:
```cl-transcript (:check-consistency ((:output string=/no-comments))) (format t "hello~%world") .. hello ; This is the first line. .. world ; This is the second line. ```
Just to make sure the above example works, here it is without being quoted.
(format t "hello~%world") .. hello ; This is the first line. .. world ; This is the second line.
Once everything in Extending DRef has been done, there are only a couple of PAX generic functions left to extend.
-
[generic-function] DOCUMENT-OBJECT* OBJECT STREAM
Write
OBJECT
in*FORMAT*
toSTREAM
. Specialize this on a subclass ofDREF
if that subclass is notRESOLVE
able, else on the type of object it resolves to. This function is for extension only. Don't call it directly.
-
[generic-function] EXPORTABLE-REFERENCE-P PACKAGE SYMBOL LOCATIVE-TYPE LOCATIVE-ARGS
Return true if
SYMBOL
is to be exported fromPACKAGE
when it occurs in aDEFSECTION
in a reference withLOCATIVE-TYPE
andLOCATIVE-ARGS
.SYMBOL
is accessible inPACKAGE
.The default method calls
EXPORTABLE-LOCATIVE-TYPE-P
withLOCATIVE-TYPE
and ignores the other arguments.By default,
SECTION
s andGLOSSARY-TERM
s are not exported although they areEXPORTABLE-LOCATIVE-TYPE-P
. To export symbols naming sections from MGL-PAX, the following method could be added:(defmethod exportable-reference-p ((package (eql (find-package 'mgl-pax))) symbol (locative-type (eql 'section)) locative-args) t)
-
[generic-function] EXPORTABLE-LOCATIVE-TYPE-P LOCATIVE-TYPE
Return true if symbols in references with
LOCATIVE-TYPE
are to be exported by default when they occur in aDEFSECTION
. The default method returnsT
, while the methods for locative typesSECTION
,GLOSSARY-TERM
,PACKAGE
,ASDF:SYSTEM
,METHOD
andINCLUDE
returnNIL
.This function is called by the default method of
EXPORTABLE-REFERENCE-P
to decide what symbolsDEFSECTION
shall export when itsEXPORT
argument is true.
Also note that due to the Home Section logic, especially for
locative types with string names, DREF-EXT:DOCSTRING*
should
probably return a non-NIL
package.
DEFINE-LOCATIVE-ALIAS
can be used to help M-.
and Specific Autolinks disambiguate
references based on the context of a name as described on Parsing.
The following example shows how to make docstrings read more naturally by defining an alias.
(defclass my-string ()
())
(defgeneric my-string (obj)
(:documentation "Convert OBJ to MY-STRING."))
;;; This version of FOO has a harder to read docstring because
;;; it needs to disambiguate the MY-STRING reference.
(defun foo (x)
"FOO takes and argument X, a [MY-STRING][class] object.")
;;; Define OBJECT as an alias for the CLASS locative.
(define-locative-alias object class)
;;; Note how no explicit link is needed anymore.
(defun foo (x)
"FOO takes an argument X, a MY-CLASS object.")
Similarly, defining the indefinite articles as aliases of the CLASS
locative can reduce the need for explicit linking.
(define-locative-alias a class)
(define-locative-alias an class)
Since these are unlikely to be universally helpful, make sure not to
export the symbols A
and AN
.
For all definitions that it encounters, DOCUMENT
calls
DOCUMENT-OBJECT*
to generate documentation. The following utilities
are for writing new DOCUMENT-OBJECT*
methods, which emit markdown.
-
[variable] *FORMAT*
Bound by
DOCUMENT
to itsFORMAT
argument, this allows markdown output to depend on the output format.
-
[macro] WITH-HEADING (STREAM OBJECT TITLE &KEY LINK-TITLE-TO) &BODY BODY
Write a markdown heading with
TITLE
toSTREAM
. NestedWITH-HEADING
s produce nested headings. If*DOCUMENT-LINK-SECTIONS*
, generate anchors based on the definition ofOBJECT
.LINK-TITLE-TO
behaves like theLINK-TITLE-TO
argument ofDEFSECTION
.
-
[macro] DOCUMENTING-REFERENCE (STREAM &KEY REFERENCE NAME PACKAGE READTABLE (ARGLIST NIL)) &BODY BODY
Write
REFERENCE
toSTREAM
as described in*DOCUMENT-MARK-UP-SIGNATURES*
, and establishREFERENCE
as a Local Definition for the processing ofBODY
.-
REFERENCE
defaults to the reference for which documentation is currently being generated. -
NAME
defaults to(XREF-NAME REFERENCE)
and is printed after theLOCATIVE-TYPE
. -
*PACKAGE*
and*READTABLE*
are bound toPACKAGE
andREADTABLE
for the duration of printing theARGLIST
and the processing ofBODY
. If either isNIL
, then a default value is computed as described in Package and Readtable. -
ARGLIST
:-
If it is not provided, then it defaults to (
ARGLIST
REFERENCE
). -
If
NIL
, then it is not printed. -
If it is a list, then it is must be a lambda list and is printed without the outermost parens and with the package names removed from the argument names.
-
If its is a string, then it must be valid markdown.
-
-
It is not allowed to have
WITH-HEADING
within the dynamic extent ofBODY
.
-
-
[macro] WITH-DISLOCATED-NAMES NAMES &BODY BODY
For each name in
NAMES
, establish a Local Definition.
-
[function] DOCUMENT-DOCSTRING DOCSTRING STREAM &KEY (INDENTATION " ") EXCLUDE-FIRST-LINE-P (PARAGRAPHP T)
Write
DOCSTRING
toSTREAM
, sanitizing the markdown from it, performing Codification and Linking, finally prefixing each line withINDENTATION
. The prefix is not added to the first line ifEXCLUDE-FIRST-LINE-P
. IfPARAGRAPHP
, then add a newline before and after the output.
-
[function] ESCAPE-MARKDOWN STRING &KEY (ESCAPE-INLINE T) (ESCAPE-HTML T) (ESCAPE-BLOCK T)
Backslash escape markdown constructs in
STRING
.-
If
ESCAPE-INLINE
, then escape*_`[]\\
characters. -
If
ESCAPE-HTML
, then escape<&
characters. -
If
ESCAPE-BLOCK
, then escape whatever is necessary to avoid starting a new markdown block (e.g. a paragraph, heading, etc).
-
-
[function] PRIN1-TO-MARKDOWN OBJECT &KEY (ESCAPE-INLINE T) (ESCAPE-HTML T) (ESCAPE-BLOCK T)
Like
PRIN1-TO-STRING
, but bind*PRINT-CASE*
depending on*DOCUMENT-DOWNCASE-UPPERCASE-CODE*
and*FORMAT*
, andESCAPE-MARKDOWN
.
SECTION
objects rarely need to be dissected since
DEFSECTION
and DOCUMENT
cover most needs. However, it is plausible
that one wants to subclass them and maybe redefine how they are
presented.
-
[class] SECTION
DEFSECTION
stores itsNAME
,TITLE
,PACKAGE
,READTABLE
andENTRIES
arguments inSECTION
objects.
-
[reader] SECTION-NAME SECTION (:NAME)
The name of the global variable whose value is this
SECTION
object.
-
[reader] SECTION-PACKAGE SECTION (:PACKAGE)
*PACKAGE*
will be bound to this package when generating documentation for this section.
-
[reader] SECTION-READTABLE SECTION (:READTABLE)
*READTABLE*
will be bound to this when generating documentation for this section.
-
[reader] SECTION-TITLE SECTION (:TITLE)
A markdown string or
NIL
. Used in generated documentation.
- [function] SECTION-LINK-TITLE-TO SECTION
-
[function] SECTION-ENTRIES SECTION
A list of markdown docstrings and
XREF
s in the order they occurred inDEFSECTION
.
GLOSSARY-TERM
objects rarely need to be dissected since
DEFINE-GLOSSARY-TERM
and DOCUMENT
cover most needs. However, it is
plausible that one wants to subclass them and maybe redefine how
they are presented.
-
[class] GLOSSARY-TERM
See
DEFINE-GLOSSARY-TERM
.
-
[reader] GLOSSARY-TERM-NAME GLOSSARY-TERM (:NAME)
The name of the global variable whose value is this
GLOSSARY-TERM
object.
-
[reader] GLOSSARY-TERM-TITLE GLOSSARY-TERM (:TITLE)
A markdown string or
NIL
. Used in generated documentation (see Output Details).
-
[reader] GLOSSARY-TERM-URL GLOSSARY-TERM (:URL)
A string or
NIL
.