mirror-llvm-project

History

Reid Kleckner 97f1c7c185 [docs] Register MLIR lexer for Sphinx docs (#206194 ) Fixes building `docs-clang-html` when `cmake -DCLANG_ENABLE_CIR=ON`. Fragments of generated documentation contain code blocks tagged with the `mlir` language, and that generates Pygments warnings, which fails the build. I chose to register a sphinx extension, which in turn registers the MLIR pygments dialect from utils/docs/llvm_sphinx. The idea here is that `mlir` syntax highlighting should work in any llvm-project documentation anywhere (openmp, or anything else). I'm a little less confident about the change to the mlir lexer itself, which seems to be necessary in order to get pygments to run without error. Here is an example error message of what happens if you don't match whitespace tokens as Text: ``` Warning, treated as error: /path/to/index.md:3:Lexing literal_block ' %0 = cir.foo : !cir.int<s, 32>\n' as "mlir" resulted in an error at token: ' '. Retrying in relaxed mode. ``` Authored with LLM assistance	2026-06-30 11:46:53 -07:00
..
mlir_lexer.py	[docs] Register MLIR lexer for Sphinx docs (#206194 )	2026-06-30 11:46:53 -07:00
README.md

[docs] Register MLIR lexer for Sphinx docs (#206194 )

Fixes building `docs-clang-html` when `cmake -DCLANG_ENABLE_CIR=ON`.

Fragments of generated documentation contain code blocks tagged with the
`mlir` language, and that generates Pygments warnings, which fails the
build.

I chose to register a sphinx extension, which in turn registers the MLIR
pygments dialect from utils/docs/llvm_sphinx. The idea here is that
`mlir` syntax highlighting should work in any llvm-project documentation
anywhere (openmp, or anything else).

I'm a little less confident about the change to the mlir lexer itself,
which seems to be necessary in order to get pygments to run without
error. Here is an example error message of what happens if you don't
match whitespace tokens as Text:

```
Warning, treated as error:
/path/to/index.md:3:Lexing literal_block '   %0 = cir.foo : !cir.int<s, 32>\n' as "mlir" resulted in an error at token: ' '. Retrying in relaxed mode.
```

Authored with LLM assistance

2026-06-30 11:46:53 -07:00

mlir_lexer.py

[docs] Register MLIR lexer for Sphinx docs (#206194 )

2026-06-30 11:46:53 -07:00

README.md

Pygments Lexer for MLIR

This file contains a simple Pygments lexer configuration for MLIR, derived from the version used in the original CGO paper. Pygments allows for advanced configurable syntax highlighting of any code. This lexer is known to be incomplete and support mostly core IR with a subset of built-in types. Additions and customizations are welcome.

Standalone Usage

Install Pygments, e.g., by running pip install Pygments or a Python package manager of your choosing. Use the standalone pygmentize command by instructing it to load the custom lexer:

pygmentize -l /path/to/mlir_lexer.py:MlirLexer -x myfile.mlir

This will produce highlighted output in the terminal. Other output formats are available, see Pygments documentation for more information.

MkDocs / Python-Markdown

Create a Markdown extension that registers the lexer via Pygments' LEXERS mapping:

# e.g., docs/pygments/mlir.py
from markdown import Extension
import pygments.lexers._mapping as _mapping

def _register_mlir_lexer():
    if "MlirLexer" not in _mapping.LEXERS:
        _mapping.LEXERS["MlirLexer"] = (
            "your.module.path.mlir_lexer",  # adjust to your project
            "MLIR",
            ("mlir",),
            ("*.mlir",),
            ("text/x-mlir",),
        )

class MlirHighlightExtension(Extension):
    def extendMarkdown(self, md):
        _register_mlir_lexer()

def makeExtension(**kwargs):
    return MlirHighlightExtension(**kwargs)

Add to mkdocs.yml (before pymdownx.highlight):

markdown_extensions:
  - your.module.path.mlir
  - pymdownx.highlight

LaTeX Usage

First, make sure your distribution includes the minted package and list in the preamble.

\usepackage{minted}

Place the mlir_lexer.py in a place where the latex binary can find it, typically in the working directory next to the main .tex file. Note that you will have to invoke latex with the -shell-escape flag. See the minted package documentation for more information.

Leverage the custom lexer facility of minted to use this lexer in your document as:

\begin{minted}{mlir_lexer.py:MlirLexer -x}
   ... your code here ...
\end{minted}