README.bigb
= OurBigBook
{c}
{title2=previously Cirodown}
https://ourbigbook.com source code + a compatible local CLI static wiki generator and markup language to write complex structured wikis/books/blogs with reference implementation in JavaScript.
This repository (The OurBigBook Project) contains two main things:
* \x[ourbigbook-cli]: a static Wiki generator that can be invoked from the command line with the \x[ourbigbook-executable]. The OurBigBook CLI calls the \x[ourbigbook-library] to convert each input file.
The markup language used is called \x[ourbigbook-markup] (file extension: `.bigb`), and it is compatible with https://ourbigbook.com[], TODO support import/export to/from https://ourbigbook.com[] to local filesystem.
* \x[ourbigbook-web]: the source code of https://ourbigbook.com[], which is a regular database-backed dynamic website. This is basically separate package that depends on the OurBigBook Library:
* information about OurBigBook Web is present at: \x[ourbigbook-web]{full}
* the source code of the OurBigBook Web is present under `web/`
Everything else in this repository outside of that section/directory is about the OurBigBook Library and the OurBigBook CLI, not the dynamic website.
Key links:
* https://cirosantilli.com[]: showcase demo document with interesting content. Primary inspiration for OurBigBook development.
* https://cirosantilli.com/oxford-nanopore-river-bacteria[]: a self-contained tutorial style part of the above. Note how internal links integrate seamlessly into the more global topic of biology, e.g. when talking about DNA we link to the global topic https://cirosantilli.com/dna[].
* https://github.com/cirosantilli/cirosantilli.github.io[] and https://github.com/cirosantilli/cirosantilli.github.io/blob/dev/oxford-nanopore-river-bacteria.bigb[]: source of the above showcase documents
* \x[design-goals]{full}: feature overview
* https://github.com/cirosantilli/ourbigbook[]: OurBigBook source code
* https://github.com/cirosantilli/ourbigbook/blob/master/README.bigb[]: source for this document
* https://cirosantilli.com/ourbigbook[]: rendered version of this document
* https://cirosantilli.com/ourbigbook/editor[]: live in-browser demo
* https://cirosantilli.com/ourbigbook-template[]: good template to get started, see \x[quick-start]{full}
= Features
{parent=ourbigbook}
* \x[internal-cross-reference][references] to any \x[header]{p} (including e.g. h2, h3, etc. in other files), \x[image]{p}, etc. with amazing \x[error-reporting][error checking and reporting]: never break internal links without knoing again, and quickly find out what broke when you do. E.g.:
README.bigb
``
= My website
See: \x[not-readme], or \x[not-readme-h2].
``
not-readme.bigb
``
= Not readme
== Not readme h2
``
`README.bigb` would render something like:
``
See: not readme, or not readme.
``
The following would fail and point you out the file and line of the failure:
* nonexistent id:
``
\x[id-that-does-not-exist]
``
* duplicate IDs:
``
= My h1
== My h2
== My h2
``
* https://katex.org/[KaTeX] server side \x[mathematics], works on browsers with JavaScript disabled:
``
I like $\sqrt{2}$, but I adore this \x[equation-quadratic-equation]:
$$
x^2 + 2x + 1
$$
{title=Quadratic equation}
``
* multi-file features out of the box so you don't need a separate wrapper like Jekyll to make a multi-page website:
* \x[internal-cross-file-reference]{p}
* single-source multi-format output based on \x[include]{p} and build options:
* by default, one HTML per source with includes rendered as links between pages, e.g.:
README.bigb
``
= My website
== h2
\Include[not-readme]
``
not-readme.bigb
``
= Not readme
== Not readme h2
``
produces `index.html` and `not-readme.html`
* with the \x[split-headers] option, you can output each header of an input file into a separate output file. The previous filesystem would produce:
* `index.html`: which contains the full `README.bigb` output
* `split.html`: split version of the above containing only the `= My website` header and not `h2`
* `h2.html`: only contains the `h2` header
* `not-readme.html` contains the full output of `not-readme.bigb`
* `not-readme-split.html`: only contains the `= Not readme` header
* `not-readme-h2.html`: only contains the `= Not readme h2` header
Each of those pages automatically gets a \x[table-of-contents]
* \x[embed-includes] single file output from multiple input files. Includes are parsed smartly, not just source copy pasted, e.g. included headers are shifted from `h1` to `h2` correctly.
On the previous sample filesystem, it would produce a single output file `index.html` which would contain a header structure like:
``
= My website
== h2
=== Not readme
==== Not readme h2
``
* supports both local serverless rendering to HTML files for local viewing, and server oriented rendering such as GitHub pages, e.g. \x[internal-cross-reference]{p} automatically get `.html` extension and or not. E.g.:
* locally, a link `\x[not-readme]` would render as `` and `not-readme.bigb` produces `not-readme.html`
* when publishing, `\x[not-readme]` would render as `` and `not-readme.bigb` also produces `not-readme.html`, which the server converts to just `http://my-website.com/not-readme`
* cross file configuration files to factor out common page parts like headers, footers and other metadata, e.g.:
* `main.liquid.html`: https://github.com/Shopify/liquid[Liquid template] used for all pages, see example at: \x[play-with-the-template]{full}
* `main.scss`: CSS stylesheet generated from https://sass-lang.com/[SASS] input, see example at: \x[play-with-the-template]{full}
* `ourbigbook.tex`: global LaTeX math definitions, e.g.:
``
\newcommand{\abs}[1]{\left|#1\right|}
``
and then you can use:
``
$\abs{x}$
``
in any .bigb file of the project.
* \x[ourbigbook-json]: per repository configuration options
* \x[table-of-contents] that crosses input files via includes. E.g. in:
README.bigb
``
= My website
== h2
\Include[not-readme]
``
not-readme.bigb
``
= Not readme
== Not readme h2
``
the table of contents for `index.html` also contains the headers for `not-readme.bigb` producing:
* My website
* h2
* Not readme
* Not readme h2
This means that you can split large \x[h-splitdefault-argument][splitDefault] input files if rendering starts to slow you down, and things will still render exactly the same.
* check that local files and images linked to actually exist: \x[a-check-argument]. E.g.:
``
\a[i-don-exist.txt]
``
would lead to a build error.
* associate headers to files with the \x[h-file-argument] e.g.:
``
Here's an example of a nice image: \x[file/path/to/my/image.png].
= path/to/my/image.png
{file}
This image was taken when I was on vacation!
``
would automatically add a preview of the image on the output.
* advanced header/ID related features:
* \x[id-based-header-levels]:
``
= Furry animal
I like \x[furry-animal]{p}, especially my cat, here is his photo: \x[image-my-cat].
== Cat
\Image[My_cat.jpg]
{title=My cat.}
``
* \x[scope]{p} either with directories or with within a single file:
``
See the important conclusion of my experiment: \x[report-of-my-experiment/conclusion]
= Report of my experiment
{scope}
== Introduction
== Middle
== Conclusion
``
* \x[cross-reference-title-inflection] for capitalization and pluralization, e.g.;
``
= Dog
== Snoopy
{c}
\x[dog]{c}{p} are fun. But the \x[dog] I like the most is \x[snoopy]!
``
would render:
* `\x[dog]{c}{p}` as `Dogs`: capitalized because of `{c}` and pluralized because of `{p}`
* `\x[dog]` as `dogs`: auto lowercased because its header `= Dog` does not have `{c}`
* `\x[snoopy]` as `Snoopy`: title capitalization kept to upper case due to `{c}` on the header `== Snoopy`
* \x[synonym]{p}, e.g.:
``
= User interface
= UI
{c}
{synonym}
{title2}
\x[user-interface]{c} is too long, I just say \x[ui].
``
would render something like:
``
User interface is too long, I just say UI
``
Furthermore, this also generates a output file:
``
ui.html
``
which redirects to the ain `user-interface.html`, so it serves as a way to have backward compatibility on page renames.
And the `title2` makes it appears on the main title under parenthesis, something like:
``
``
* \x[h-disambiguate-argument][header disambiguation], e.g.:
``
My favorite fruits are \x[apple-fruit]{p}!
My favorite least favorite brand is is \x[apple-company]! \x[apple] computers are too expensive.
== Apple
{disambiguate=fruit}
== Apple
{c}
{disambiguate=company}
= Apple
{c}
{synonym}
``
which renders something like:
* `\x[apple-fruit]{p}`: `apples`
* `\x[apple-company]`: `Apple`
* `\x[apple]`: also `Apple` because of the synonym
* `== Apple\n{disambiguate=fruit}`: `
* `== Apple\n{disambiguate=company}`: `
* tags are regular headers: \x[h-child-argument], \x[x-child-argument]
``
= Animal
== Dog
{tag=domestic}
{tag=cute}
== Cat
{tag=domestic}
{tag=cute}
== Bat
{tag=flying}
= Flying
= Cute
= Domestic
``
* \x[unlimited-header-levels], levels higher than 6 are rendered in HTML as an appropriately styled `div`s with an ID:
``
= h1
== h2
=== h3
==== h4
===== h5
====== h6
======= h7
======== h8
``
* generate lists of \x[incoming-links] between internal headers: it shows every internal link coming into the current page
* is written in JavaScript and therefore runs natively on the browser to allow live previews as shown at: https://cirosantilli.com/ourbigbook/editor[]
* helps you with the publishing:
* `ourbigbook --publish` publishes in a single command to the configured target (default GitHub pages): \x[publish]{full}
* OurBigBook tries to deal with media such as images and video intelligently for you, e.g.: \x[where-to-store-images]{full}. E.g. you can keep media in a separate media repository, `my-media-repository`, and then by configuring on \x[ourbigbook-json]:
``
"media-providers": {
"github": {
"default-for": ["image", "video"],
"path": "media",
"remote": "yourname/myproject-media"
}
}
``
you can use images in that repository with:
``
\Image[My_image_basename.jpg]
``
instead of:
``
\Image[https://raw.githubusercontent.com/cirosantilli/myproject--media/master/My_image_basename.jpg]
``
* `inotifywait` watch and automatically rebuild with \x[watch]:
``
ourbigbook --watch input-file.bigb
``
= Quick start
{parent=ourbigbook}
= Play with the template
{parent=quick-start}
Learn the syntax basics in 5 minutes: https://cirosantilli.com/ourbigbook/editor[].
Play with https://github.com/cirosantilli/ourbigbook-template[a OurBigBook template] locally:
``
git clone https://github.com/cirosantilli/ourbigbook-template
cd ourbigbook-template
npm install
npx ourbigbook .
firefox index.html
``
That template can be seen rendered live at: http://cirosantilli.com/ourbigbook-generate-multifile/ Other templates are documented at: \x[generate].
To \x[publish-to-github-pages] on your repository you can just fork the repository https://github.com/cirosantilli/ourbigbook-template to your own https://github.com/johndoe/ourbigbook-template and then:
``
git remote set-url origin git@github.com:johndoe/ourbigbook-template.git
npx ourbigbook --publish
``
and it should now be visible at: https://johndoe.github.io/ourbigbook-template
Then, every time you make a change you can publish the new version with:
``
git add .
git commit --message 'hacked stuff'
ourbigbook --publish .
``
or equivalently with the \x[publish-commit] shortcut:
``
ourbigbook --publish-commit 'hacked stuff'
``
If you want to publish to your root page https://johndoe.github.io instead of https://johndoe.github.io/ourbigbook-template you need to rename the `master` branch to `dev` as mentioned at \x[publish-to-github-pages-root-page]:
``
git remote set-url origin git@github.com:johndoe/johndoe.github.io.git
# Rename master to dev, and delete the old master.
git checkout -b dev
git push origin dev:dev
git branch -D master
git push --delete origin master
npx ourbigbook --publish
``
The following files of the template control the global style of the output, and you are free to edit them:
* `main.liquid.html`: global HTML template in https://shopify.github.io/liquid/[Liquid format]. Available variables are documented at \x[template], and it is being selected in that repository with :
``
"template": "main.liquid.html"
``
in the \x[ourbigbook-json] configuration file.
* `main.scss`: https://sass-lang.com/[Sass] file that gets converted to raw CSS `main.css` by `npx ourbigbook .`.
Sass is just much more convenient to write than raw CSS.
That file gets included into the global HTML template inside `main.liquid.html` at:
``
``
= Important command line options
{parent=quick-start}
When you run:
``
npx ourbigbook .
``
it converts all files in the current directory separately, e.g.:
* `README.bigb` to `index.html`, since `README` is a magic name that we want to show on the root URL
* `not-readme.bigb` to `not-readme.html`, as this one is a regular name unlike `README`
* `main.scss` to `main.css`
If one of the input files starts getting too large, usually the toplevel `README.bigb` in which you dump everything by default like Ciro does, you can speed up development and just compile files individually with either:
``
npx ourbigbook README.bigb
npx ourbigbook not-readme.bigb
``
Note however that when those individual files have a \x[internal-cross-file-reference] to something defined in `not-readme.bigb`, e.g. via `\x[h2-in-not-the-readme]`, then you must first do a first pass once with:
``
npx ourbigbook .
``
to parse all files and extract all necessary IDs to the \x[id-database]. That would be optimized slightly wit the \x[no-render] command line option:
``
npx ourbigbook --no-render .
``
to only extract the IDs but not render, which speeds things up considerably
When dealing with large files, you might also be interested in the following amazing options:
* \x[split-headers]
* \x[h-splitdefault-argument]
To produce a single standalone output file that contains everything in a directory run:
``
npx ourbigbook --embed-resources --embed-includes README.bigb
xdg-open index.html
``
You can now just give `index.html` to any reader and they should be able to view it offline without installing anything. The flags are:
* \x[embed-includes]: without this, `\Include[not-readme]` shows as a link to the file `not-readme.html` which comes from `not-readme.bigb` With the flag, `not-readme.bigb` output gets embedded into the output `index.html` directly
* \x[embed-resources]: by default, we link to CSS and JavaScript that lives inside `node_modules`. With this flag, that CSS and JavaScript is copied inline into the document instead. One day we will try to handle \x[image]{p} that way as well
= Useless knowledge
{parent=quick-start}
Install the NPM package globally and use it from the command line for a quick conversion:
``
npm install -g ourbigbook
printf 'ab\ncd\n' | ourbigbook --body-only
``
or to a file:
``
printf 'ab\ncd\n' | ourbigbook > tmp.html
``
You almost never want to do this except when \x[developing-ourbigbook], as it won't be clear what version of `ourbigbook` the document should be compiled with. Just be a good infant and use OurBigBook \x[play-with-the-template][with the template] that contains a `package.json` via `npx`, OK?
Furthermore, the default install of Chromium on Ubuntu 21.04 uses Snap and blocks access to dotfiles. For example, in a sane NVM install, our global CSS would live under `/home/ciro/.nvm/versions/node/v14.17.0/lib/node_modules/ourbigbook/dist/ourbigbook.css`, which gets blocked because of the `.nvm` part:
* https://forum.snapcraft.io/t/dot-files/7062
* https://bugs.launchpad.net/snapd/+bug/1607067
* https://superuser.com/questions/1546550/chromium-81-wont-display-dotfiles-anymore
* https://askubuntu.com/questions/1184357/why-cant-chromium-suddenly-access-any-partition-except-for-home
* https://askubuntu.com/questions/1214346/as-a-user-is-there-any-way-to-change-the-confinement-of-a-snap-package
One workaround is to use \x[embed-resources], but this of course generates larger outputs.
To run master globally from source for development see: \x[run-ourbigbook-master]{full}. This one actually works despite the dotfile thing since your development path is normally outside of dotfiles.
Try out the JavaScript API with \a[lib_hello.js]:
``
npm install ourbigbook
./lib_hello.js
``
= Design goals
{parent=ourbigbook}
OurBigBook is designed entirely to allow writing complex professional HTML and PDF scientific books, blogs, articles and encyclopedias.
OurBigBook aims to be the ultimate \x[latex-output-format][LaTeX] "killer", allowing books to be finally published as either HTML or PDF painlessly (LaTeX being only a backend to PDF generation).
It aims to be \x[features][more powerful] and \x[saner] and than Markdown and Asciidoctor.
= Saner
{parent=design-goals}
Originally, OurBigBook was is meant to be both saner and more powerful than Markdown and Asciidoctor.
But alas, as Ciro started implementing and using it, he started to bring some Markdown \x[insane-macro-shortcuts][insanity he missed back in].
And so this "degraded" slightly into a language slightly saner than Asciidoctor but with an amazing Node.js implementation that makes it better for book writing and website publishing.
Notably, we hope that our escaping will be a bit saner backslash escapes everything instead of Asciidoctor's "different escapes for every case" approach: https://github.com/asciidoctor/asciidoctor/issues/901
But hopefully, having starting from a saner point will still produce a saner end result, e.g. there are sane constructs for every insane one.
It is intended that this will be an acceptable downside as OurBigBook will be used primarily large complex content such as books rather than forum posts, and will therefore primarily written either:
* in text editors locally, where users have more features than in random browser textareas
* in a dedicated website that will revolutionize education, and therefore have a good JavaScript editing interface: https://github.com/cirosantilli/write-free-science-books-to-get-famous-website
For example, originally OurBigBook had exactly five magic characters, with similar functions as in LaTeX:
* `\` backslash to start a macro, like LaTeX
* `{` and `}`: left and right square brackets to delimit \x[positional-vs-named-arguments][optional macro arguments]
* `[` and `]`: left and right curly braces bracket to start an optional arguments
and double blank newlines for \x[paragraph]{p} if you are pedantic, but this later degenerated into many more with \x[insane-macro-shortcuts].
We would like to have only square brackets for both optional and mandatory to have even less magic characters, but that would make the language difficult to parse for computer and humans. LaTeX was right for once!
This produces a very regular syntax that is easy to learn, including doing:
* arbitrary nesting of elements
* adding arbitrary properties to elements
This sanity also makes the end tail learning curve of the endless edge cases found in Markdown and Asciidoctor disappear.
The language is designed to be philosophically isomorphic to HTML to:
* further reduce the learning curve
* ensure that most of HTML constructs can be reached, including arbitrary nesting
More precisely:
* macro names map to tag names, e.g.: `\\a` to `* one of the arguments of macros, maps to the content of the HTML element, and the others map to attributes.
E.g., in a link:
``\a[http://example.com][Link text\]``
the first macro argument:
``http://example.com``
maps to the `href` of ` ``Link text``
maps to the internal content of `Link text<>`.
= More powerful
{parent=design-goals}
The \x[saner][high sanity of OurBigBook], also makes creating new macro extensions extremely easy and intuitive.
All built-in language features use the exact same API as new extensions, which ensures that the extension API is sane forever.
Markdown is clearly missing many key features such as block attributes and \x[internal-cross-reference]{p}, and has no standardized extension mechanism.
The "more powerful than Asciidoctor" part is only partially true, since Asciidoctor is very featureful can do basically anything through extensions.
The difference is mostly that OurBigBook is completely and entirely focused on making amazing scientific books, and so will have key features for that application out-of-the box, notably:
* amazing header/ToC/ID features including proper error reports: never have a internal broken link or duplicate ID again
* \x[mathematics][server side pre-rendered maths with KaTeX]: all divs and spans are ready, browser only applies CSS, no JavaScript gets executed
* \x[publish]: we take care of website publishing for you out-of-the-box, no need to integrate into an external project like Jekyll
* \x[split-headers]:
* https://github.com/asciidoctor/asciidoctor/issues/626 feature request
* https://github.com/owenh000/asciidoctor-multipage third party plugin that does it
and we feel that some of those features have required specialized code that could not be easily implemented as a standalone macro.
Another advantage over Asciidoctor is that the reference implementation of OurBigBook is in JavaScript, and can therefore be used on browser live preview out of the box. Asciidoctor does Transpile to JS with https://github.com/opal/opal[Opal], but who wants to deal with that layer of complexity?
= Related projects
{parent=design-goals}
Static wiki generators: this is perhaps the best way of classifying this project :-)
* https://github.com/gollum/gollum[]: already has a local server editor! But no WYSIWYG nor live preview. Git integration by default, so when you save on the UI already generates a Git commit. We could achieve that with: https://github.com/isomorphic-git/isomorphic-git[], would be really nice. Does not appear to have built-in static generation:
* https://stackoverflow.com/questions/7210391/have-anyone-use-gollum-site-to-generete-markdown-wikis-and-host-it-on-heroku
* https://github.com/dreverri/gollum-site
Does not appear to check that any links are correct.
* https://github.com/wcchin/markypydia
* https://obsidian.md/ closed source, Markdown with \x[internal-cross-file-reference] + a SaaS. Appears to require payment for any publishing. 28k followers 2021: https://twitter.com/obsdmd[]. Founders are likely Canadians of Asian descent from Waterloo University: https://www.linkedin.com/in/lishid/ | https://www.linkedin.com/in/ericaxu/ also working in parallel on https://dynalist.io/ 2020 review at: https://www.youtube.com/watch?v=aK2fOQRNSxc Has offline editor with side-by-side preview. Compares with https://roamresearch.com/[Roam] and https://roamresearch.com/[Notion], but can't find any public publishing on those, seem to be enterprise only things.
Static book generators:
* https://github.com/rstudio/bookdown[], https://bookdown.org/[]. Very similar feature set to what we want!!! Transpiles to markdown, and then goes through Pandoc: https://bookdown.org/yihui/bookdown/pandoc.html[], thus will never run on browser without huge translation layers. But does have an obscene amount of output formats however.
* https://gohugo.io/[Hugo]. Pretty good, similar feature set to ours. But Go based, so hard on browser, and adds adhoc features on top of markdown once again
* https://en.wikipedia.org/wiki/Personal_wiki
* https://github.com/vimwiki/vimwiki
* https://github.com/hplgit/doconce
* https://www.gwern.net/About#source is pretty interesting, uses https://github.com/jaspervdj/Hakyll/ + some custom stuff.
* https://github.com/JerrySievert/bookmarkdown
* https://www.gitbook.com/
* https://github.com/rust-lang/mdBook[]. Impressive integrated search feature. Like Gitbook but implemented in Rust.
* https://github.com/facebook/docusaurus React + markdown based, written in TypeScript. So how can it be build fast? Gotta benchmark.
Less related but of interest, similar philosophy to what Ciro wants, but no explicitly reusable system:
* http://www.uprtcl.io/
* https://libretexts.org
* https://physics.info/
* https://hypertextbook.com/
* https://tutorial.math.lamar.edu/
= Motivation
{parent=design-goals}
Ciro Santilli developed OurBigBook to perfectly satisfy his writing style, which is basically "create one humongous document where you document everything you know about a subject so everyone can understand it, and just keep adding to it".
https://cirosantilli.com[] is the first major document that he has created in OurBigBook.
He decided to finally create this new system after having repeatedly facing limitations of Asciidoctor which were ignored/wontfixed upstream, because Ciro's writing style is not as common/targeted by Asciidoctor.
Following large documents Ciro worked extensively on:
* https://github.com/cirosantilli/china-dictatorship
* https://github.com/cirosantilli/linux-kernel-module-cheat
made the limitations of Asciidoctor clear to Ciro, and were major motivation in this work.
The key limitations have repeatedly annoyed Ciro were:
* cannot go over header level 6, addressed at: \x[unlimited-header-levels]
* the need for \x[split-headers] to avoid one too large HTML output that will never get indexed properly by search engines, and takes a few seconds to load on any browser, which is unacceptable user experience
= OurBigBook Markup
{c}
{parent=ourbigbook}
OurBigBook Markup is the markup language used in the \x[ourbigbook] project.
It works both on the \x[ourbigbook-web] dynamic website, and on \x[ourbigbook-cli] static websites from the command line.
= Macro
{parent=ourbigbook-markup}
This section documents all OurBigBook macros.
Macros are magic commands that do cool stuff, e.g. `\Image` to create an image.
The most common macros also have \x[insane-macro-shortcuts] to keep the syntax shorter.
The general macro syntax is described at \x[ourbigbook-markup-syntax]{full}.
= Link
{parent=macro}
= `\a`
{synonym}
{title2}
\x[insane-macro-shortcuts][Insane] autolink (link text is the same as the link):
\OurbigbookExample[[
The website http://example.com is cool. See also:
\Q[http://example.com/2]
]]
Exact parsing rules described at: \x[insane-link-parsing-rules]{full}.
Equivalent sane version:
\OurbigbookExample[[[
The website \a[http://example.com] is cool.
\Q[\a[http://example.com/2]]
]]]
Insane link with custom text:
\OurbigbookExample[[
The website http://example.com[example.com] is cool.
]]
Equivalent sane version:
\OurbigbookExample[[
The website \a[http://example.com][example.com] is cool.
]]
If the custom text is empty, an autolink is generated. This is often useful if you want your link to be followed by punctuation:
\OurbigbookExample[[
The website is really cool: http://example.com[].
]]
This could also be achieved with the sane syntax of course, but this pattern saves a tiny bit of typing.
Link with multiple paragraphs inside it:
\OurbigbookExample[[
\a[http://example.com][Multiple
paragraphs]
]]
= `\a` `check` argument
{parent=link}
If `{check=0}` is given, this disables the local file existence check that is done by default for \x[relative-link]{p}.
If `check` is not given explicitly, OurBigBook does the check by default if the link is not a \x[url-with-protocol] For example, the following are not checked:
* `http://cirosantilli.com`
* `https://cirosantilli.com`
* `file:///etc/fstab`
* `ftp://cirosantilli.com`
and the following are:
* `index.js`
* `../index.js`
* `path/to/index.js`
* `/path/to/index.js`. Since it stats with a `/`, this path has to exist relative to \x[project-toplevel-directory], not relative to the current `.bigb` source.
* `//example.com/path/to/index.js`. This gets treated the same as `/example.com/path/to/index.js`.
For example, it is often the case in computer programming tutorials that we want to refer to source files in the current directory, so you could have in your `README.bigb`:
``
See this awesome source file: \a[index.js]
``
and then if `index.js` does not exist in the project, compilation leads to an error.
This check is important because as you start documenting several source files, it is almost inevitable getting wrong paths due to renames or typos without this type of error checking
The most common use case for disabling checks is as follows.
https://github.com/cirosantilli/cirosantilli.github.io/blob/da296c3c933704484d5cd1a42754f60ec00f672b/README.bigb was rendered at https://cirosantilli.com[], and contains links to https://cirosantilli.com/markdown-style-guide[], whose source lives in a separate non-OurBigBook repository: https://github.com/cirosantilli/markdown-style-guide/
Therefore, if we did from `README.bigb`:
``
See this awesome style guide: \a[markdown-style-guide]
``
we do not want OurBigBook do check that the file `markdown-style-guide` exists in the local filesystem.
For this reason, we have instead to write:
``
See this awesome style guide: \a[markdown-style-guide]{check=0}
``
The general lesson is clear: in a server, paths could be re-routed to anything, including content that lies outside of a OurBigBook project.
= `\a` `href` argument
{parent=link}
The link target, e.g. in:
``
\a[http://example.com]
``
`href` equals `http://example.com`.
Important behaviours associated with this property for local links:
* they are checked for existence in the local filesystem: \x[a-check-argument]
* they are modified as per \x[relative-link]{full} to account for \x[scope]{p} with \x[split-headers]
= `\a` `ref` argument
{parent=link}
Analogous to the \x[x-ref-argument], e.g.:
\OurbigbookExample[[
Trump said this and that.https://en.wikipedia.org/wiki/Donald_Trump_Access_Hollywood_tape#Trump's_responses{ref}https://web.archive.org/web/20161007210105/https://www.donaldjtrump.com/press-releases/statement-from-donald-j.-trump{ref} Then he said that and this.https://en.wikipedia.org/wiki/Donald_Trump_Access_Hollywood_tape#Trump's_responses{ref}https://web.archive.org/web/20161007210105/https://www.donaldjtrump.com/press-releases/statement-from-donald-j.-trump{ref}
]]
= `\a` `relative` argument
{parent=link}
If given, this \x[boolean-argument] forces a given link to be a \x[relative-link] or not.
Otherwise, the link is automatically guessed based on the address given as explained at \x[relative-link]{full}.
= Relative link
{parent=a-relative-argument}
A relative link is a link that points to a resource that will be present a final URL relative to the input `.bigb` file.
For example, it is often the case in computer programming tutorials that we want to refer to source files in the current directory.
So from our `README.bigb`, we could want to write something like:
\OurbigbookExample[[
Have a look at this amazing source file: \a[index.js].
]]
and here `\a[ourbigbook]` is a relative link.
A non-relative link is something like:
\OurbigbookExample[[
This is great website: https://cirosantilli.com
]]
which points to an absolute URL.
A link being relative has the following effects
* the correct relative path to the file is used when using nested \x[scope]{p} with \x[split-headers]. For example, if we have:
``
= h1
== h2
{scope}
=== h3
\a[index.js]
``
then in split header mode, `h3` will be rendered to `h2/h3.html`.
Therefore, if we didn't do anything about it, the link to `index.js` would render as `href="index.js"` and thus point to `h2/index.js` instead of the correct `index.js`.
Instead, OurBigBook automatically converts it to the correct `href="../index.js"`
OurBigBook considers a link relative by default if:
* it is not a \x[url-with-protocol]
* it does not start with a slash `/`
Therefore, the following are not relative links by default:
* `http://cirosantilli.com`
* `https://cirosantilli.com`
* `file:///etc/fstab`
* `ftp://cirosantilli.com`
* `/path/to/index.js`
* `//example.com/path/to/index.js`
and the following are:
* `index.js`
* `../index.js`
* `path/to/index.js`
Implemented at: https://github.com/cirosantilli/ourbigbook/issues/87[].
= URL with protocol
{c}
{parent=relative-link}
A URL with protocol is a URL that matches the regular expression `^[a-zA-Z]+://`. The following are examples of URLs with protocol:
* `http://cirosantilli.com`
* `https://cirosantilli.com`
* `file:///etc/fstab`
* `ftp://cirosantilli.com`
The following aren't:
* `index.js`
* `../index.js`
* `path/to/index.js`
* `/path/to/index.js`
* `//example.com/path/to/index.js`. This one is a bit tricky. Web browsers would consider this as a https://stackoverflow.com/questions/28446314/why-use-protocol-relative-urls-at-all[protocol-relative URL], which technically implies a protocol, although that protocol would be different depending how you are viewing the file, e.g. locally through `file://` vs on a with website `https://`.
For simplicity's sake, we just consider it as a URL without protocol.
= Insane link parsing rules
{parent=link}
Insane start at any of the recognized protocols are the ones shown at: \x[known-url-protocols]{full}.
* `http://`
* `https://`
absolutely anywhere if not escaped, e.g.:
``
ahttp://example.com
``
renders something like:
``
a
``
To prevent expansion, you have to escape the protocol with a backslash `\\`, e.g.:
``
\http://example.com
``
Empty domains like:
``
http://
``
don't becomes links however. But this one does:
``
http://a
``
Insane links end when either the end of the document or one of the following characters is found:
* space ` `
* newline `\n`
* open or close square bracket `[` or `]`
* open or close curly braces `{` or `}`
As a consequence, to have an insane link followed immediately by a punctuation like a period you should use an empty argument as in:
\OurbigbookExample[[
Check out this website: http://example.com[].
]]
otherwise the punctuation will go in it. Another common use case is:
\OurbigbookExample[[
As mentioned on the tutorial (http://example.com[see this link]).
]]
If you want your link to include one of the terminating characters, e.g. `]`, all characters can be escaped with a backslash, e.g.:
\OurbigbookExample[[
Hello http://example.com/\]a\}b\\c\ d world.
]]
Note that the `http://example.com` inside `\a[http://example.com]` only works because we do some post-processing magic that prevents its expansion, otherwise the link would expand twice:
\OurbigbookExample[[
\P[http://example.com]
\a[http://example.com]
]]
This magic can be observed with \x[help-macros] by seeing that the `href` argument of the `a` macro has the property:
``
"elide_link_only": true,
``
= Bold
{parent=macro}
{title2=`\b`}
\OurbigbookExample[[
Some \b[bold] text.
]]
= Line break
{parent=macro}
{title2=`\br`}
There is basically one application for this: poetry, which would be too ugly with \x[code-block] due to fixed width font:
\OurbigbookExample[[
Paragraph 1 Line 1\br
Paragraph 1 Line 2\br
Paragraph 2 Line 1\br
Paragraph 2 Line 2\br
]]
= Code block
{parent=macro}
{title2=\c[[``]], \c[[`]], `\C`, `\c`}
Inline code (code that should appear in the middle of a paragraph rather than on its own line) is done with a single backtick (\c[[`]]) \x[insane-macro-shortcuts][insane macro shortcut]:
\OurbigbookExample[[
My inline `x = 'hello\n'` is awesome.
]]
and block code (code that should appear on their own line) is done with two or more backticks (\c[[``]]):
\OurbigbookExample[[
``
f() {
return 'hello\n';
}
``
]]
The sane version of inline code is a lower case `c`:
\OurbigbookExample[[[
My inline \c[[x = 'hello\n']] is awesome.
]]]
and the sane version of block math is with an upper case `C`:
\OurbigbookExample[[[
\C[[
f() {
return 'hello\n';
}
]]
]]]
The capital vs lower case theme is also used in other elements, see: \x[block-vs-inline-macros].
If the content of the sane code block has many characters that you would need to \x[escape-characters][escape], you will often want to use \x[literal-arguments], which work just like the do for any other argument. For example:
\OurbigbookExample[[[[
\C[[[
A paragraph.
\C[[
And now, some long, long code, with lots
of chars that you would need to escape:
\ [ ] { }
]]
A paragraph.
]]]
]]]]
Note that the initial newline is skipped automatically in code blocks, just as for any other element, due to: \x[argument-leading-newline-removal], so you don't have to worry about it.
The distinction between inline `\c` and block `\C` code blocks is needed because in HTML, https://stackoverflow.com/questions/5371787/can-i-have-a-pre-tag-inside-a-p-tag-in-tumblr/58603596#58603596[`pre` cannot go inside `P`].
We could have chosen to do some magic to differentiate between them, e.g. checking if the block is the only element in a paragraph, but we decided not to do that to keep the language saner.
And now a code block outside of \x[ourbigbookexample] to test how it looks directly under \x[toplevel]:
``
Hello
Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello
HelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHello
Hello
``
And nos a very long inline code: `Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello`
\Comment[[[[
TODO implement.
We can have cross references to code blocks as for other elements such as \x[image]{p}:
\OurbigbookExample[[[
See this awesome code \x[my-code]:
``
ab
cd
``
{id=my-code}
]]]
]]]]
Block code can have a title, e.g. see this one: \x[code-my-nice-code]{full}:
\OurbigbookExample[[
``
ab
cd
``
{id=code-my-nice-code}
{title=My nice code block.}
]]
= `\OurbigbookExample`
{parent=macro}
Shows both the OurBigBook code and its rendered output, e.g.:
\OurbigbookExample[[[
\OurbigbookExample[[
Some `ineline` code.
]]
]]]
Its input should be thought of as a literal code string, and it then injects the rendered output in the document.
This macro is used extensively in the OurBigBook documentation.
= Comment
{parent=macro}
{title2=`\Comment`}
The `Comment` and `comment` macros are regular macros that does not produce any output. Capitalization is explained at: \x[block-vs-inline-macros]{full}.
You will therefore mostly want to use it with a \x[literal-arguments][literal argument], which will, as for any other macro, ignore any macros inside of it.
\OurbigbookExample[[[
Before comment.
\Comment[[
Inside comment.
]]
After comment.
]]]
And an inline one:
\OurbigbookExample[[[
My inline \comment[[inside comment]] is awesome.
\comment[[inside comment]] inline at the start.
]]]
= Header
{parent=macro}
{title2=`\H`}
\x[insane-macro-shortcuts][Insane] with `= ` (equal sign space):
``
= My h1
== My h2
=== My h3
``
Insane headers end at the first newline found. They cannot therefore contain raw newline tokens.
Equivalent sane:
``
\H[1][My h1]
\H[2][My h2]
\H[3][My h3]
``
Custom ID for \x[internal-cross-reference]{p} on insane headers:
``
= My h1
{id=h1}
== My h2
{id=h2}
=== My h3
{id=h3}
``
Sane equivalent:
``
\H[1][My h1]{id=h1}
\H[2][My h2]{id=h2}
\H[3][My h3]{id=h3}
``
= Unlimited header levels
{parent=header}
There is no limit to how many levels we can have, for either sane or insane headers!
HTML is randomly limited to `h6`, so OurBigBook just renders higher levels as an `h6` with a `data-level` attribute to indicate the actual level for possible CSS styling:
``
``
The recommended style is to use insane headers up to `h6`, and then move to sane one for higher levels though, otherwise it becomes very hard to count the `=` signs.
To avoid this, we considered making the insane syntax be instead:
``
= 1 My h1
= 2 My h2
= 3 My h3
``
but it just didn't feel as good, and is a bit harder to type than just smashing `=` n times for lower levels, which is the most common use case. So we just copied markdown.
= My h3
{parent=unlimited-header-levels}
= My h4
{parent=my-h3}
= My h5
{parent=my-h4}
= My h6
{parent=my-h5}
= My h7
{parent=my-h6}
= My h8
{parent=my-h7}
= My h9
{parent=my-h8}
= My h10
{parent=my-h9}
= My h11
{parent=my-h10}
= My h12
{parent=my-h11}
= My h13
{parent=my-h12}
= Skipping header levels
{parent=header}
The very first header of a document can be of any level, although we highly recommend your document to start with a `\H[1]`, and to contain exactly just one `\H[1]`, as this has implications such as:
* `\H[1]` is used for the document title: \x[html-document-title]
* `\H[1]` does not show on the \x[table-of-contents]
After the initial header however, you must not skip a header level, e.g. the following would give an error because it skips level 3:
``
= my 1
== my 1
==== my 4
``
= The toplevel header
{parent=header}
If the document has only a single header of the highest level, e.g. like the following has only a single `h2`:
``
== My 2
=== My 3 1
=== My 3 2
``
then this has some magical effects.
= The toplevel header IDs don't show
{parent=the-toplevel-header}
Header IDs won't show for the toplevel level. For example, the headers would render like:
``
My 2
1. My 3 1
2. My 3 2
``
rather than:
``
1. My 2
1.2. My 3 1
1.2. My 3 2
``
This is because in this case, we guess that the `h2` is the toplevel.
= The ID of the first header is derived from the filename
{parent=the-toplevel-header}
TODO: we kind of wanted this to be the ID of the toplevel header instead of the first header, but this would require an extra postprocessing pass (to determine if the first header is toplevel or not), which might affect performance, so we are not doing it right now.
When the OurBigBook input comes from a file (and not e.g. stdin), the default ID of the first header in the document is derived from the basename of the OurBigBook input source file rather than from its title.
This is specially relevant when \x[include][including] other files.
For example, in file named `my-file.bigb` which contains:
``
= Awesome ourbigbook file
]]
``
the ID of the header is `my-file` rather than `awesome-ourbigbook-file`. See also: \x[automatic-id-from-title].
If the file is an \x[index-file] other than \x[the-toplevel-index-file], then the basename of the parent directory is used instead, e.g. the toplevel ID of a file:
``my-subdir/README.bigb``
would be:
``#my-subdir``
rather than:
``#README.bigb``
For the toplevel index file however, the ID is just taken from the header itself as usual. This is done because you often can't general control the directory name of a project.
For example, a \x[publish-to-github-pages][GitHub pages] root directory must be named as `.github.io`. And users may need to rename directories to avoid naming conflicts.
As a consequence of this, the toplevel index file cannot \x[include][be included in other files].
= `\H` arguments
{parent=header}
= `\H` `c` argument
{parent=h-arguments}
If given, makes the header capitalized by default on \x[internal-cross-file-reference]{p}.
More details at: \x[cross-reference-title-inflection]{full}.
= `\H` `child` argument
{parent=h-arguments}
This \x[multiple-argument] marks given IDs as being children of the current page.
The effect is the same as adding the \x[x-child-argument] argument to an under the header. Notably, such marked target IDs will show up on the \x[tagged] autogenerated \x[header-metadata-section].
Example:
``
= Animal
== Mammal
=== Bat
=== Cat
== Wasp
== Flying animal
{child=bat}
{child=wasp}
\x[bat]
\x[wasp]
``
renders exactly as:
``
= Animal
== Mammal
=== Bat
=== Cat
== Wasp
== Flying animal
\x[bat]{child}
\x[wasp]{child}
``
The header `child` syntax is generally preferred because at some point while editing the content of the header, you might accidentally remove mentions to e.g. `\x[bat]{child}`, and then the relationship would be lost.
The \x[h-tag-argument] does the same as the \x[x-child-argument] but in the opposite direction.
= `\H` `disambiguate` argument
{parent=h-arguments}
Sometimes the short version of a name is ambiguous, and you need to add some extra text to make both its title and ID unique.
For example, the word "Python" could either refer to:
* the programming language: https://en.wikipedia.org/wiki/Python_(programming_language)
* the genus of snakes: https://en.wikipedia.org/wiki/Python_(genus)
The `disambiguate` \x[positional-vs-named-arguments][name argument], which is automatically defined for all macros, helps you deal more neatly with such problems.
Have a look at this example:
``
My favorite snakes are \x[python-genus]{p}!
My favorite programming language is \x[python-programming-language]!
\x[python-genus]{full}
\x[python-programming-language]{full}
== Python
{disambiguate=genus}
== Python
{c}
{disambiguate=programming language}
{title2=.py}
{wiki}
``
which renders as:
\Q[
My favorite snakes are \x[python-genus]{p}!
My favorite programming language is \x[python-programming-language]!
\x[python-genus]{full}
\x[python-programming-language]{full}
= Python
{disambiguate=genus}
{parent=h-disambiguate-argument}
= Python
{c}
{disambiguate=programming language}
{parent=h-disambiguate-argument}
{title2=.py}
{wiki}
]
from which we observe how `disambiguate`:
* gets added to the ID after conversion following the same rules as \x[automatic-id-from-title]
* shows up on the header between parenthesis, much like Wikipedia, as well as in \x[x-full-argument][`full` cross references]
* does not show up on non-`full` references. This makes it much more likely that you will be able to reuse the title automatically on a cross reference without `content`: we wouldn't want to say "My favorite programming language is Python (programming language)" all the time, would we?
* gets added to the default \x[h-wiki-argument] inside parenthesis, following Wikipedia convention, therefore increasing the likelihood that you will be able to go with the default Wikipedia value
Besides disambiguating headers, the `disambiguate` argument has a second related application: disambiguating IDs of images. For example:
\OurbigbookExample[[
\x[image-the-title-of-my-disambiguate-image]{full=0}
\x[image-the-title-of-my-disambiguate-image-2]{full=0}
\x[image-the-title-of-my-disambiguate-image]{full}
\x[image-the-title-of-my-disambiguate-image-2]{full}
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=The title of my disambiguate image.}
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=The title of my disambiguate image.}
{disambiguate=2}
]]
Note that unlike for headers, `disambiguate` does not appear on the title of images at all. It serves only to create an unique ID that can be later referred to. Headers are actually the only case where `disambiguate` shows up on the visible rendered output. We intend on making this application obsolete however with:
This use case is even more useful when `title-from-src` is enable by default for the \x[ourbigbook-json/media-providers] entry, so you don't have to repeat titles several times over and over.
= `\H` `file` argument
{parent=h-arguments}
If given, the current section contains metadata about file or other resource with the given URL.
If empty, the URl of the file is extracted directly from the header. Otherwise, the given URL is used.
for example:
``
The file \x[file/path/to/myfile.c] is very useful.
= path/to/myfile.c
{file}
An explanation of what this file is about.
``
renders the same as:
``
The file \x[file/path/to/myfile.c] is very useful.
= path/to/myfile.c
{file}
{id=file/path/to/myfile.c}
An explanation of what this file is about.
\a[path/to/myfile.c][View file]
``
so note how:
* \x[automatic-id-from-title] does not normalize the path, e.g. it does not convert `/` to `-`.
Also, a `file/` prefix is automatically added to the ID. This is needed with \x[split-headers] to avoid a collision between:
* `path/to/myfile.c`: the actual file
* `file/path/to/myfile.c`: the metadata about that file. Note that locally the `.html` extension is added as in `file/path/to/myfile.c.html` which avoids the collision. But on a server deployment, the `.html` is not present, and there would be a conflict if we didn't add that `file/` prefix.
* a link to the is added automatically, since users won't be able to click it from the header, as clicking on the header will just link to the header itself
* a preview is added. The type of preview is chosen as follows:
* if the URL has an image extension, do an \x[image] preview
* otherwise if the URL has a video extension, or is a YouTube URL, do a \x[video] preview
* otherwise, don't show a preview, as we don't know anything sensible to show
In some cases however, especially when dealing with external URLs, we might want to have a more human readable title with a `file` specified as in:
``
The video \x[tank-man-by-cnn-1989] is very useful.
= Tank Man by CNN (1989)
{file=https://www.youtube.com/watch?v=YeFzeNAHEhU}
An explanation of what this video is about.
``
which renders something like:
``
The video \x[tank-man-by-cnn-1989] is very useful.
= Tank Man by CNN (1989)
\a[https://www.youtube.com/watch?v=YeFzeNAHEhU][View file]
An explanation of what this video is about.
\Video[https://www.youtube.com/watch?v=YeFzeNAHEhU]
``
= `\H` `file` argument demo
{parent=h-file-argument}
= Tank_man_standing_in_front_of_some_tanks.jpg
{file}
{parent=h-file-argument-demo}
An explanation of what this video is about.
= Tank Man by CNN (1989)
{file=https://www.youtube.com/watch?v=YeFzeNAHEhU}
{parent=h-file-argument-demo}
An explanation of what this video is about.
= `\H` `numbered` argument
{c}
{parent=h-arguments}
This \x[boolean-argument] determines whether renderings of a header will have section numbers or not. This affects all of:
* \x[header]{p} themselves
* \x[table-of-contents] links
* \x[internal-cross-reference]{p} with the \x[x-full-argument]
This option can be set by default for all files with:
By default, headers are numbered as in a book, e.g.:
``
= h1
== h2
=== h3
==== h4
``
renders something like:
``
= h1
Table of contents
* 1. h2
* 1.1. h3
* 1.1.1. h4
== 1. h2
=== 1.1. h3
==== 1.1.1. h4
``
However, for documents with a very large number of sections, or \x[unlimited-header-levels][deeply nested headers] those numbers start to be more noise than anything else, especially in the table of contents and you are better off just referring to IDs. E.g. imagine:
``
1.3.1.4.5.1345.3.2.1. Some deep level
``
When documents reach this type of scope, you can disable numbering with the `numbered` option.
This option can be set on any header, and it is inherited by all descendants.
The option only affects descendants.
E.g., if in the above example turn numbering off at `h2`:
``
= h1
== h2
{numbered=0}
=== h3
==== h4
``
then it renders something like:
``
= h1
Table of contents
* 1. h2
* h3
* h4
== 1. h2
=== h3
==== h4
``
The more common usage pattern to disable it on toplevel and enable it only for specific "tutorial-like sections". An example can be seen at:
* https://cirosantilli.com/[]: huge toplevel wiki, for which we don't want numbers
* https://cirosantilli.com/x86-paging[]: a specific tutorial, for which we want numbers
which is something like:
``
= Huge toplevel wiki
{numbered=0}
== h2
=== A specific tutorial
{numbered}
{scope}
==== h4
===== h5
``
then it renders something like:
``
= Huge toplevel wiki
Table of contents
* h2
* A specific tutorial
* 1. h4
* 1.1. h5
== h2
=== A specific tutorial
==== 1. h4
===== 1.1. h5
``
Note how in this case the number for `h4` is just `1.` rather than `1.1.1.`. We only show numberings relative to the first non-numbered header, because the `1.1.` wouldn't be very meaningful otherwise.
= `\H` `parent` argument
{c}
{parent=h-arguments}
= ID-based header levels
{c}
{synonym}
{title2}
In addition to the basic way of specifying header levels with an explicit level number as mentioned at \x[header]{full}, OurBigBook also supports a more indirect ID-based mechanism with the `parent` argument of the `\H` element.
We hightly recommend using `parent` for all but the most trivial documents.
For example, the following fixed level syntax:
``
= My h1
== My h2 1
== My h2 2
=== My h3 2 1
``
is equivalent to the following ID-based version:
``
= My h1
= My h2 1
{parent=my-h1}
= My h2 2
{parent=my-h1}
= My h3 2 1
{parent=my-h2-h}
``
The main advantages of this syntax are felt when you have a huge document with \x[unlimited-header-levels][very large header depths]. In that case:
* it becomes easy to get levels wrong with so many large level numbers to deal with. It is much harder to get an ID wrong.
* when you want to move headers around to improve organization, things are quite painful without a refactoring tool (which we intend to provide in the \x[browser-editor-with-preview]), as you need to fix up the levels of every single header.
If you are using the ID-based syntax however, you only have to move the chunk of headers, and change the `parent` argument of a single top-level header being moved.
Note that when the `parent=` argument is given, the header level must be `1`, otherwise OurBigBook assumes that something is weird and gives an error. E.g. the following gives an error:
``
= My h1
== My h2
{parent=my-h1}
``
because the second header has level `2` instead of the required `= My h2`.
When \x[scope]{p} are involved, the rules are the same as those of internal reference resolution, including the leading `/` to break out of the scope in case of conflicts.
See also: \x[header-explicit-levels-vs-nesting-design-choice]{full} for further rationale.
= ID-based header levels and scope resolution
{c}
{parent=h-parent-argument}
When mixing both \x[h-parent-argument] and \x[scope]{p}, things get a bit complicated, because when writing or parsing, we have to first determine the parent header before resolving scopes.
As a result, the follow simple rules are used:
* start from the last header of the highest level
* check if the `{parent=XXX}` is a suffix of its ID
* if not, proceed to the next smaller level, and so on, until a suffix is found
Following those rules for example, a file `tmp.bigb`:
``
= h1
{scope}
= h1 1
{parent=h1}
{scope}
= h1 1 1
{parent=h1-1}
= h1 1 2
{parent=h1-1}
= h1 1 3
{parent=h1/h1-1}
= h1 2
{parent=h1}
{scope}
= h1 2 1
{parent=h1-2}
{scope}
= h1 2 1 1
{parent=h1-2/h1-2-1}
``
will lead to the following header tree with \x[log-headers]:
``
= h1 tmp
== h2 1 tmp/h1-1
=== h3 1.1 tmp/h1-1/h1-1-1
=== h3 1.2 tmp/h1-1/h1-1-2
=== h3 1.3 tmp/h1-1/h1-1-3
== h2 2 tmp/h1-2
=== h3 2.1 tmp/h1-2/h1-2-1
==== h4 2.1.1 tmp/h1-2/h1-2-1/h1-2-1-1
``
= Header explicit levels vs nesting design choice
{parent=h-parent-argument}
Arguably, the language would be even saner if we did:
``
\H[My h1][
Paragraph.
\H[My h2][]
]
``
rather than having explicit levels as in `\H[1][My h1]` and so on.
But we chose not to do it like most markups available because it leads to too many nesting levels, and hard to determine where you are without tooling.
Ciro later "invented" (?) \x[h-parent-argument], which he feels reaches the perfect balance between the advantages of those two options.
= `\H` `scope` argument
{parent=h-arguments}
= Scope
{synonym}
In some use cases, the sections under a section describe inseparable parts of something.
For example, when documenting an experiment you executed, you will generally want an "Introduction", then a "Materials" section, and then a "Results" section for every experiment.
On their own, those sections don't make much sense: they are always referred to in the context of the given experiment.
The problem is then how to get unique IDs for those sections.
One solution, would be to manually add the experiment ID as prefix to every subsection, as in:
``
= Experiments
See: \x[full-and-unique-experiment-name/materials]
== Introduction
== Full and unique experiment name
=== Introduction
{id=full-and-unique-experiment-name/introduction}
See our awesome results: \x[full-and-unique-experiment-name/results]
For a more general introduction to all experiments, see: \x[introduction].
=== Materials
{id=full-and-unique-experiment-name/materials}
=== Results
{id=full-and-unique-experiment-name/results}
``
but this would be very tedious.
To keep those IDs shorter, OurBigBook provides the `scope` \x[boolean-argument] property of \x[header]{p}, which works analogously to C++ namespaces with the header IDs.
Using `scope`, the previous example could be written more succinctly as:
``
= Experiments
See: \x[full-and-unique-experiment-name/materials]
== Introduction
== Full and unique experiment name
{scope}
=== Introduction
See our awesome results: \x[results]
For a more general introduction to all experiments, see: \x[/introduction].
=== Materials
=== Results
``
Note how:
* full IDs are automatically prefixed by the parent scopes prefixed and joined with a slash `/`
* we can refer to other IDs withing the current scope without duplicating the scope. E.g. `\x[results]` in the example already refers to the ID `full-and-unique-experiment-name/materials`
* to refer to an ID outside of the scope and avoid name conflicts with IDs inside of the current scope, we start a reference with a slash `/`
So in the example above, `\x[/introduction]` refers to the ID `introduction`, and not `full-and-unique-experiment-name/introduction`.
= `scope` resolution
{parent=h-scope-argument}
When nested scopes are involved, \x[internal-cross-reference]{p} resolution peels off the scopes one by one trying to find the closes match, e.g. the following works as expected:
``
= h1
{scope}
== h2
{scope}
=== h3
{scope}
\x[h2]
``
Here OurBigBook:
* first tries to loop for an `h1/h2/h3/h2`, since `h1/h2/h3` is the current scope, but that ID does not exist
* so it removes the `h3` from the current scope, and looks for `h1/h2/h2`, which is still not found
* then it removes the `h2`, leading to `h1/h2`, and that one is found, and therefore is taken
= Directory-based `scope`
{parent=h-scope-argument}
Putting files in subdirectories of the build has the same effect as adding a \x[scope] to their top level header.
Notably, all headers inside that directory get the directory prepended to their IDs.
The toplevel directory is determined as described at: \x[the-toplevel-index-file].
= Test scope 1
{parent=h-scope-argument}
{scope}
For fun and profit.
= Test scope 2
{parent=test-scope-1}
{scope}
Let's break this local link: \a[ourbigbook].
= Not scoped
{parent=test-scope-2}
= `\H` `scope` argument of toplevel headers
{parent=h-scope-argument}
When \x[the-toplevel-header] is given the `scope` property OurBigBook automatically uses the file path for the scope and heaves fragments untouched.
For example, suppose that file `full-and-unique-experiment-name` contains:
``
= Full and unique experiment name
{scope}
== Introduction
== Materials
``
In this case, multi-file output will generate a file called `full-and-unique-experiment-name.html`, and the URL of the subsections will be just:
* `full-and-unique-experiment-name.html#introduction`
* `full-and-unique-experiment-name.html#materials`
instead of
* `full-and-unique-experiment-name.html#full-and-unique-experiment-name/introduction`
* `full-and-unique-experiment-name.html#full-and-unique-experiment-name/materials`
Some quick interactive cross file link tests:
* \x[not-readme-with-scope]
* \x[not-readme-with-scope/h2]
* \x[not-readme-with-scope/image-my-image]
= `\H` `splitDefault` argument
{parent=h-arguments}
When using \x[split-headers], \x[internal-cross-reference]{p} always point to non-split pages as mentioned at \x[internal-cross-reference-targets-in-split-headers].
If the `splitDefault` \x[boolean-argument] is given however:
* the split header becomes the default, e.g. `index.html` is now the split one, and `nosplit.html` is the non-split one
* the header it is given for, and all of its descendant headers will use the split header as the default internal cross target, unless the header is already rendered in the current page. This does not propagate across \x[include]{p} however.
For example, consider `README.bigb`:
``
= Toplevel
{splitDefault}
\x[h2][toplevel to h2]
\x[notreadme][toplevel to notreadme]
\Include[notreadme]
== h2
``
and `notreadme.bigb`:
``
= Notreadme
\x[h2][notreadme to h2]
\x[notreadme][notreadme to notreadme h2]
== Notreadme h2
``
Then the following links would be generated:
* `index.html`: split version of `README.bigb`, i.e. does not contain `h2`
* `toplevel to h2`: `h2.html`. Links to the split version of `h2`, since `h2` is also affected by the `splitDefault` of its parent, and therefore links to it use the split version by default
* `toplevel to notreadme`: `notreadme.html`. Links to non-split version of `notreadme.html` since that header is not `splitDefault`, because `splitDefault` does not propagate across includes
* `nosplit.html` non-split version of `README.bigb`, i.e. contains `h2`
* `toplevel to h2`: `#h2`, because even though `h2` is `splitDefault`, that header is already present in the current page, so it would be pointless to reload the split one
* `toplevel to notreadme`: `notreadme.html`
* `h2.html` split version of `h2` from `README.bigb`
* `notreadme.html`: non-split version of `notreadme.bigb`
* `notreadme to h2`: `h2.html`, because `h2` is `splitDefault`
* `notreadme to notreadme h2`: `#notreadme-h2`
* `notreadme-split.html`: split version of `notreadme.bigb`
* `notreadme to h2`: `h2.html`, because `h2` is `splitDefault`
* `notreadme to notreadme h2`: `notreadme.html#notreadme-h2`, because `notreadme-h2` is not `splitDefault`
The major application of this is that Ciro likes to work with a huge `README.bigb` containing thousands of random small topics. This is the case for example for: https://cirosantilli.com
And splitting those into separate source files would be quite laborious, as it would require duplicating IDs on the filename, and setting up \x[include]{p}.
However, after this README reaches a certain size, page loads start becoming annoyingly slow, even despite already loading large assets like \x[image]{p} video \x[video]{p} only on hover or click: the annoying slowness comes from the loading of the HTML itself before the browser can jump to the ID.
And even worse: this README corresponds to the main index page of the website, which will make what a large number of users will see be that slowness.
Therefore, once this README reaches a certain size, you can add the `splitDefault` attribute to it, to make things smoother for readers.
And if you have a smaller, more self-contained, and highly valuable tutorial such as https://cirosantilli.com/x86-paging[], you can just split that into a separate `.bigb` source file.
This way, any links into the smaller tutorial will show the entire page as generally desired.
And any links from the tutorial, back to the main massive README will link back to split versions, leading to fast loads.
This feature was implemented at: https://github.com/cirosantilli/ourbigbook/issues/131
= `\H` `splitSuffix` argument
{parent=h-arguments}
If given, add a custom suffix to the output filename of the header when using \x[split-headers].
If the given suffix is empty, it defaults to `-split`.
For example, given:
``
= my h1
== my h2
``
a `--split-headers` conversion would normally place `my h2` into a file called:
``
my-h2.html
``
However, if we instead wrote:
``
== my h2
{splitSuffix}
``
it would not be placed under:
``
my-h2-split.html
``
and if we set a custom one as:
``
== my h2
{splitSuffix=asdf}
``
it would go instead to:
``
my-h2-asdf.html
``
This option is useful if the root of your website is written in OurBigBook, and you want to both:
* have a section that talks about some other project
* host the documentation of that project inside the project source tree
For example, https://cirosantilli.com with source at https://github.com/cirosantilli/cirosantilli.github.io has a quick section about OurBigBook: https://cirosantilli.com#ourbigbook[].
Therefore, without a custom suffix, the split header version of that header would go to https://cirosantilli.com/ourbigbook[], which would collide with this documentation, that is present in a separate repository: https://github.com/cirosantilli/ourbigbook[].
Therefore a `splitSuffix` property is used, making the split header version fall under `/ourbigbook-split`, and leaving the nicer `/ourbigbook` for the more important project toplevel.
If given on the \x[the-toplevel-header]{p}, which normally gets a suffix by default to differentiate from the non-split version, it replaces the default `-split` suffix with a custom one.
For example if you had `notindex.bigb` as:
``
= Not index
``
then it would render to:
``
notindex-split.bigb
``
but if you used instead:
``
= Not index
{splitSuffix=asdf}
``
then it would instead be:
``
notindex-asdf.bigb
``
= `\H` `synonym` argument
{parent=h-arguments}
= Synonym
{synonym}
This option is similar to \x[h-title2-argument] but it additionally:
* creates a new ID that you can refer to, and renders it with the alternate chosen title
* the rendered ID on \x[internal-cross-reference]{p} is the same as what it is a synonym for
* the synonym header is not rendered at all, including in the \x[table-of-contents]
* when using \x[split-headers], a redirect output file is generated from the synonym to the main ID
Example:
``
= Parent
== GNU Debugger
{c}
= GDB
{c}
{synonym}
I like to say \x[gdb] because it is shorter than \x[gnu-debugger].
``
renders something like:
``
= GNU Debugger
I like to say \a[#gnu-debugger][GDB] because it is shorter than \x[#gnu-debugger][GNU Debugger].
``
Furthermore, if \x[split-headers] is used, another file is generated:
``
gdb.html
``
which contains a redirection from `gdb.html` to `gnu-debugger.html`.
Implemented at: https://github.com/cirosantilli/ourbigbook/issues/114
= `\H` `title` argument
{parent=h-synonym-argument}
Contains the main content of the header. The \x[insane-syntax]:
``
= My title
``
is equivalent to the sane:
``
\H[1][My title]
``
and in both cases `My title` is the title argument.
The title argument is also notably used for \x[automatic-id-from-title].
= Automatic ID from title
{parent=h-title-argument}
If a \x[the-toplevel-header][non-toplevel] macro has the `title` property is present but no explicit `id`, an ID is created automatically from the `title`, by applying the following transformations:
* do a \x[id-output-format] conversion on the title to remove for example any HTML tags that would be present in the conversion output
* convert all characters to lowercase. This uses \x[javascript-case-conversion]. Note that this does convert non-ASCII characters to lowercase, e.g. `É` to `é`.
* if \x[ourbigbook-json/id-normalize-latin] is `true` (the default) do \x[ourbigbook-json/latin-normalization]. This converts e.g. `é` to `e`.
* if \x[ourbigbook-json/id-normalize-punctuation] is `true` (the default) do \x[ourbigbook-json/punctuation-normalization]. This converts e.g. `+` to `plus`.
* convert consecutive sequences of all non `a-z0-9` ASCII characters to a single hyphen `-`. Note that this leaves non-ASCII characters untouched.
* strip leading or trailing hyphens
Note how those rules leave non-ASCII Unicode characters untouched, except for:
* capitalization changes wher applicable, e.g. `É` to `é`
as capitalization and determining if something "is a letter or not" in those cases can be tricky.
For toplevel headers, see: \x[the-id-of-the-first-header-is-derived-from-the-filename].
So for example, the following automatic IDs would be generated: \x[table-examples-of-automatically-generated-ids].
\Table{title=Examples of automatically generated IDs.}
[
|| title
|| id
|| latin normalization
|| punctuation normalization
|| comments
| My favorite title
| my-favorite-title
|
|
|
| Ciro's markdown is awesome
| ciro-s-markdown-is-awesome
|
|
| `'` is an ASCII character, but it is not in `a-z0-9`, therefore it gets converted to a hyphen `-`
| É你
| e你
| true
|
| The Latin https://en.wikipedia.org/wiki/Acute_accent[acute accented] `e`, `É`, is converted to its lower case form `é` as per the \x[javascript-case-conversion].
Then, due to \x[ourbigbook-json/latin-normalization], `é` is converted to `e`.
The Chinese character `你` is left untouched as Chinese characters have no case, and no ASCII analogue.
| É你
| é你
| false
|
| Same as the previous, but `é` is not converted to `e` since \x[ourbigbook-json/latin-normalization] is turned off.
| C++ is great
| c-plus-plus-is-great
|
| true
| This is the effect of \x[ourbigbook-json/punctuation-normalization].
| I \i[love] dogs.
| i-love-dogs
|
|
| `love` is extracted from the italic tags `love` with \x[id-output-format] conversion.
]
For \x[the-toplevel-header][the toplevel header], its ID is derived from the basename of the OurBigBook file without extension instead of from the `title` argument.
= `\H` `title2` argument of a synonym header
{parent=h-synonym-argument}
Unlike \x[h-title2-argument], the synonym does not show up by default next to the title. This is because we sometimes want that, and sometimes not. To make the title appear, you can simply add an empty `title2` argument to the synonym header as in:
``
= GNU Debugger
{c}
= GDB
{c}
{synonym}
{title2}
= Quantum computing
= Quantum computer
{synonym}
``
which renders something like:
``
= GNU Debugger (GDB)
= Quantum computing
``
Note how we added the synonym to the title only when it is not just a simple flexion variant, since `Quantum computing (Quantum computer)` would be kind of useless would be kind of useless.
= `\H` `tag` argument
{c}
{parent=h-arguments}
{tag=multiple-argument}
Same as \x[x-child-argument] but in the opposite direction, e.g.:
``
== Mammal
=== Bat
{tag=flying-animal}
=== Cat
== Flying animal
``
is equivalent in every way to:
``
== Mammal
=== Bat
=== Cat
== Flying animal
{child=bat}
``
Naming rationale:
* `parent` as the opposite of child is already taken to be then "main parent" via the "\x[h-parent-argument]"
* we could have renamed the \x[h-child-argument] to `tags` as in "this header tags that one", but it would be a bit confusing `tags` vs `tag`
So `child` vs `tag` it is for now.
You generally want to use `tag` instead of the \x[h-child-argument] because otherwise some very large header categories are going to contain Huge lists of children, which is not very nice when editing.
It is possible to enforce the \x[h-child-argument] or the \x[h-tag-argument] in a given project with the \x[ourbigbook-json/lint-h-tag] option.
= `\H` `title2` argument
{parent=h-arguments}
The `title2` argument can be given to any element that has the `title` argument.
Its usage is a bit like the `description=` argument of \x[image]{p}, allowing you to add some extra content to the header without affecting its ID.
Unlike `description=` however, `title2` shows up on all \x[x-full-argument][`full`] references, including appearances in the \x[table-of-contents], which make it more searchable.
Its primary use cases are:
* give acronyms, or other short names names of fuller titles such as mathematical/programming notation
One primary reason to not use the acronyms as the main section name is to avoid possible ID ambiguities with other acronyms.
* give the header in different languages
For example, given the OurBigBook input:
``
= Toplevel
The Toc follows:
== North Atlantic Treaty Organization
{c}
{title2=NATO}
\x[north-atlantic-treaty-organization]
\x[north-atlantic-treaty-organization]{full}
``
the rendered output looks like:
``
= Toplevel
The ToC follows:
* North Atlantic Treaty Organization (NATO)
== North Atlantic Treaty Organization (NATO)
North Atlantic Treaty Organization
Section 1. "North Atlantic Treaty Organization (NATO)"
``
Related alternatives to `title2` include:
* \x[h-disambiguate-argument] when you do want to affect the ID to remove ambiguities
* \x[h-synonym-argument]
Parenthesis are added automatically around all rendered `title2`.
The `title2` argument has a special meaning when applied to a \x[header] with the \x[h-synonym-argument], see \x[h-title2-argument-of-a-synonym-header].
\Comment[[[[
= `\H` `tutorial` argument
{parent=h-arguments}
{tag=boolean-argument}
This option is a convenience helper for the common use case of "writting a tutorial". The same would also apply to a report, or an article.
]]]]
= `\H` `wiki` argument
{parent=h-arguments}
If given, show a link to the Wikipedia article that corresponds to the header.
If a value is not given, automatically link to the Wiki page that matches the header exactly with spaces converted to underscores.
Here is an example with an explicit wiki argument:
``
==== Tiananmen Square
{wiki=Tiananmen_Square}
``
which looks like:
= Tiananmen Square
{id=wiki-explicit}
{parent=h-wiki-argument}
{wiki=Tiananmen_Square}
or equivalently with the value deduced from the title:
``
= Tiananmen Square
{wiki}
``
which looks like:
= Tiananmen Square
{id=wiki-implicit}
{parent=h-wiki-argument}
{wiki}
You can only link to subsections of wiki pages with explicit links as in:
``
= History of Tiananmen Square
{wiki=Tiananmen_Square#History}
``
which looks like:
= History of Tiananmen Square
{id=wiki-explicit-subsection}
{parent=h-wiki-argument}
{wiki=Tiananmen_Square#History}
= Header metadata section
{parent=header}
OurBigBook adds some header metadata to the toplevel header at the bottom of each page. this section describes this metadata.
Although the \x[table-of-contents]{child} has a macro to specify its placement, it is also automatically placed at the bottom of the page, and could be considered a header metadata section.
= Incoming links
{parent=header-metadata-section}
Lists other sections that link to the current section.
E.g. in:
``
= tmp
== tmp 1
=== tmp 1 1
=== tmp 1 2
\x[tmp-1]
== tmp 2
\x[tmp-1]
``
the page `tmp-1.html` would contain a list of incoming links as:
* `tmp-1-2`
* `tmp-2`
since those pages link to the `tmp-1` ID.
= Tagged
{parent=header-metadata-section}
Lists sections that are \x[secondary-children] of the current section, i.e. tagged under the current section.
The main header tree hierarchy descendants already show under the \x[table-of-contents] instead.
E.g. in:
``
= tmp
== Mammal
== Flying
== Animal
=== Bat
{tag=mammal}
{tag=flying}
=== Bee
{tag=flying}
=== Dog
{tag=mammal}
``
the tagged sections for:
* Mammal will contain Bat and Dog
* Flying will contain Bat and Bee
= Ancestors
{parent=header-metadata-section}
Shows a list of ancestors of the page. E.g. in:
``
= Asia
== China
=== Beijing
==== Tiananmen Square
=== Hong Kong
``
the ancestor lists would be for:
* Hong Kong: China, Asia
* Tiananmen Square: Beijing, China, Asia
* Beijing: China, Asia
* China: Asia
so we see that this basically provides a type of breadcrumb navigation.
= Image
{parent=macro}
{title2=`\Image` and `\image`}
A block image with \x[block-vs-inline-macros][capital] 'i' `Image` showcasing most of the image properties \x[image-my-test-image].
\OurbigbookExample[[
Have a look at this amazing image: \x[image-my-test-image].
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=The title of my image.}
{id=image-my-test-image}
{width=600}
{height=200}
{source=https://en.wikipedia.org/wiki/File:Tianasquare.jpg}
{description=The description of my image.}
]]
This exemplifies the following parameters:
* `title`: analogous to the \x[h-title-argument]. Shows up preeminently, and sets a default ID if one is not given.
* \x[image-description-argument][`description`]: similar to `title`, but allow for further explanations without them appearing in \x[internal-cross-reference][cross references] to the image
* `source`: a standardized way to credit an image by linking to a URL that contains further image metadata
For further discussion on the effects of ID see: \x[image-id]{full}.
And this is how you make an inline image inline one with lower case `i`:
\OurbigbookExample[[
My inline \image[Tank_man_standing_in_front_of_some_tanks.jpg][test image] is awesome.
]]
Inline images can't have captions.
And now for an image outside of \x[ourbigbookexample] to test how it looks directly under \x[toplevel]: \x[image-my-test-image-toplevel]{full}.
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{id=image-my-test-image-toplevel}
= Image height
{parent=image}
By default, we fix image heights to `height=315`, and let the `width` be calculated proportionally once the image loads. We therefore ignore the actual image size. This is done to:
* prevent reflows as the page loads images and can determine their actual sizes, especially is the user opens the page at a given ID in the middle of the page
* create a more uniform media experience by default, unless a custom image size is actually needed e.g. if the image needs to be larger
= Image ID
{parent=image}
Here is an image without a \x[image-description-argument][description] but with an ID so we can link to it: \x[image-my-test-image-2].
\OurbigbookExample[[
Have a look at this amazing image: \x[image-my-test-image-2].
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{id=image-my-test-image-2}
]]
This works because \x[x-full-argument][`full` is the default cross reference style for `Image`], otherwise the link text would be empty since there is no `title`, and OurBigBook would raise an error.
OurBigBook can optionally deduce the title from the basename of the `src` argument if the `titleFromSrc` \x[boolean-argument] is given, or if `title-from-src` is set as the default \x[ourbigbook-json/media-providers][media provider] for the media type:
\OurbigbookExample[[
Have a look at this amazing image: \x[image-tank-man-standing-in-front-of-some-tanks].
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{titleFromSrc}
]]
= Image caption
{parent=image-id}
If the image has neither \x[image-id][ID] nor title nor \x[image-description-argument][description] nor `source`, then it does not get a caption at all:
\OurbigbookExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
]]
If the image does not have an ID nor title, then it gets an automatically generated ID, just like every other OurBigBook output HTML element, and it is possible for readers to link to that ID on the rendered version, e.g. as:
``
#_123
``
Note that the `123` is not linked to the `Figure .`, but just a sequential ID that runs over all elements.
This type of ID is of course not stable across document revisions however, since if an image is added before that one, the link will break. So give an ID or title for anything that you expect users to link to.
Also, it is not possible to link to such images with an \x[internal-cross-reference], like any other OurBigBook element with autogenerated temporary IDs.
Another issue to consider is that in paged output formats like PDF, the image could float away from the text that refers to the image, so you basically always want to refer to image by ID, and not just by saying "the following image".
We can also see that such an image does not increment the Figure count:
\OurbigbookExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{id=image-my-test-image-count-before}
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{id=image-my-test-image-count-after}
]]
If the image has any visible metadata such as `source` or `description` however, then the caption does show and the Figure count gets incremented:
\OurbigbookExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{source=https://en.wikipedia.org/wiki/File:Tianasquare.jpg}
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{description=This is the description of my image.}
]]
= Where to store images
{parent=image}
= Store images inside the repository itself
{parent=where-to-store-images}
If you are making a limited repository that will not have a ton of images, then you can get away with simply git tracking your images in the main repository.
With this setup, no further action is needed. For example, with a file structure of:
``
./README.bigb
./Tank_man_standing_in_front_of_some_tanks.jpg
``
just use the image from \C[README.bigb] as:
\OurbigbookExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
]]
However, if you are making a huge tutorial, which can have a huge undefined number of images (i.e. any scientific book), then you likely don't want to git track your images in the git repository.
A generally better alternative is to \x[store-images-in-a-separate-media-repository], and especially \x[store-images-in-a-separate-media-repository-and-track-it-as-a-git-submodule].
= Store images in a separate media repository
{parent=where-to-store-images}
In this approach, you create a separate GitHub repository in addition to the main one containing the text to contain only media such as images.
This approach is more suitable than \x[store-images-inside-the-repository-itself] if you are going to have a lot of images.
When using this approach, you could of course just point directly to the final image URL, e.g. as in:
\OurbigbookExample[[
\Image[https://raw.githubusercontent.com/cirosantilli/media/master/Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg]
]]
but OurBigBook allows you use configurations that allow you to enter just the image basename: `Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg` which we will cover next.
In order to get this to work, the recommended repository setup is:
* `./main-repo/.git`: main repository at https://github.com/username/main-repo
* `./main-repo/data/media/.git/`: media repository at https://github.com/username/main-repo-media[], and where `data/` is gitignored.
The directory and repository names are not mandatory, but if you place media in `data/media` and name its repository by adding the `*-media` suffix, then `ourbigbook` will handle everything for you without any further configuration in \x[ourbigbook-json/media-providers].
This particular documentation repository does have a different setup as can be seen from its \a[ourbigbook.json]. Then, when everything is setup correctly, we can refer to images simply as:
\OurbigbookExample[[
\Image[Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg]{provider=github}
]]
In this example, we also needed to set `{provider=github}` explicitly since it was not set as the default image provider in our `ourbigbook.json`. In most projects however, all of your images will be in the default repository, so this won't be needed.
`provider` must not be given when a full URL is given because we automatically detect providers from URLs, e.g.:
``
\Image[https://raw.githubusercontent.com/cirosantilli/media/master/Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg]{provider=github}
``
is an error.
TODO implement: `ourbigbook` will even automatically add and push used images in the `my-tutorial-media` repository for you \x[publish][during publishing]!
You should then use the following rules inside `my-tutorial-media`:
* give every file a very descriptive and unique name as a full English sentence
* never ever delete any files, nor change their content, unless it is an improvement in format that does change the information contained of the image TODO link to nice Wikimedia Commons guideline page
This way, even though the repositories are not fully in sync, anyone who clones the latest version of the `*-media` directory will be able to view any version of the main repository.
Then, if one day the media repository ever blows up GitHub's limit, you can just migrate the images to another image server that allows arbitrary basenames, e.g. AWS, and just configure your project to use that new media base URL with the \x[ourbigbook-json/media-providers] option.
The reason why images should be kept in a separate repository is that images are hundreds or thousands of times larger than hand written text.
Therefore, images could easily fill up the maximum repository size you are allowed: https://webapps.stackexchange.com/questions/45254/file-size-and-storage-limits-on-github#84746 and then what will you do when GitHub comes asking you to reduce the repository size?
https://git-lfs.github.com/[Git LFS] is one approach to deal with this, but we feel that it adds too much development overhead.
= Store images in a separate media repository and track it as a git submodule
{parent=store-images-in-a-separate-media-repository}
This is likely the sanest approach possible, as it clearly specifies which media version matches which repository version through the submodule link.
Furthermore, it is possible to make the submodule clone completely optional by setting things up as follows. For your OurBigBook project `yourname/myproject` create a `yourname/myproject-media` with the media, and track it as a submodule under `yourname/myproject/media`.
Then, add to \x[ourbigbook-json/media-providers]:
```
"media-providers": {
"github": {
"default-for": ["image", "video"],
"path": "media",
"remote": "yourname/myproject-media"
}
}
```
Now, as mentioned at \x[ourbigbook-json/media-providers], everything will work beautifully:
* `ourbigbook .` local conversion will use images from `media/` if it exists, e.g.:
```
\Image[myimage.jpg]
```
will render `media/myimage.jpg`. So after cloning the submodule, you will be able to see the images on the rendered pages without an internet connection.
But if the submodule is not cloned, not problem, renders will detect that and automatically use GitHub images.
Then, when you do:
```
ourbigbook --publish
```
the following happen:
* `\Image[myimage.jpg]` uses the GitHub URL
* automatically push `media/` to GitHub in case there were any updates
* also, that directory is automatically gitignore, so it won't be pushed as part of the main render and thus duplicate things
= Store images in Wikimedia Commons
{parent=where-to-store-images}
Wikimedia Commons is another great possibility to upload your images to:
\OurbigbookExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/thumb/5/5b/Gel_electrophoresis_insert_comb.jpg/450px-Gel_electrophoresis_insert_comb.jpg]
{source=https://commons.wikimedia.org/wiki/File:Gel_electrophoresis_insert_comb.jpg}
]]
OurBigBook likes Wikimedia Commons so much that we automatically parse the image URL and if it is from Wikimedia Commons, automatically deduce the `source` for you. So the above image renders the same without the `source` argument:
\OurbigbookExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/5/5b/Gel_electrophoresis_insert_comb.jpg]
]]
And like for non-Wikimedia images, you can automatically generate a `title` from the `src` by setting the `titleFromSrc` \x[boolean-argument] or if `title-from-src` is set as the default \x[ourbigbook-json/media-providers][media provider] for the media type:
\OurbigbookExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/5/5b/Gel_electrophoresis_insert_comb.jpg]
{titleFromSrc}
]]
And a quick test for a more complex thumb resized URL:
\OurbigbookExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/thumb/5/5b/Gel_electrophoresis_insert_comb.jpg/450px-Gel_electrophoresis_insert_comb.jpg]
]]
If you really absolutely want to turn off the `source`, you can explicitly set:
\OurbigbookExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/5/5b/Gel_electrophoresis_insert_comb.jpg]
{source=}
]]
but you don't want to do that for the most commonly Wikimedia Commons used license of CC BY+, do you? :-)
Upsides of using Wikimedia Commons for your images:
* makes it easier for other writers to find and reuse your images
* automatically generates resized versions of the uploaded images into several common dimensions so you can pick the smallest one that fits your desired \x[image-height] to reduce bandwidth usage
* if you have so many images that they would blow even the size of a \x[store-images-in-a-separate-media-repository][separate media repository], this will still work
Downsides:
* forces you to use the Creative Commons license
* requires the content to be educational in nature
* uploading a bunch of images to Wikimedia Commons does feel a bit more laborious than it should because you have to write down so much repeated metadata for them
= Image lazy loading
{parent=image}
We do this by default because OurBigBook is meant to allow producing huge single page documents like Ciro likes it, and in this way:
* images that the user is looking at will load first
* we save a lot of bandwidth for the user who only wants to browse one section
TODO: maybe create a mechanism to disable this for the entire build with \x[ourbigbook-json].
= Background color of transparent images
{parent=image}
For the love of God, there is no standardized for SVG to set its background color without a rectangle? https://stackoverflow.com/questions/11293026/default-background-color-of-svg-root-element `viewport-fill` was just left in limbo?
And as a result, many many many SVG online images that you might want to reuse just rely on white pages and don't add that background rectangle.
Therefore for now we just force white background on \x[overview-of-files-in-this-repository][our default CSS], which is what most SVGs will work with. Otherwise, you can lose the entire image to our default black background.
Then if someone ever has an SVG that needs another background color, we can add an image attribute to set that color as a local style.
= Image generators
{parent=image}
TODO implement: mechanism where you enter a textual description of the image inside the code body, and it then converts to an image, adds to the `-media` repo and pushes all automatically. Start with dot.
https://github.com/cirosantilli/ourbigbook/issues/40
= Image argument
{parent=image}
= Image `check` argument
{parent=image-argument}
{tag=named-argument}
{tag=boolean-argument}
Analogous to the \x[a-check-argument] when checking if the \x[image-src-argument] exists or not.
= Image `description` argument
{parent=image-argument}
{tag=named-argument}
The `description` argument similar to the \x[image-title-argument] argument, but allows allowing longer explanations without them appearing in \x[internal-cross-reference][cross references] to the image.
For example, consider:
\OurbigbookExample[[
See this image: \x[image-description-argument-test-1].
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=Tank man standing in front of some tanks.}
{id=image-description-argument-test-1}
{description=Note how the tanks are green.}
{source=https://en.wikipedia.org/wiki/File:Tianasquare.jpg}
]]
In this example, the reference `\x[image-description-argument-test-1]` expands just to
\Q[Tank man standing in front of some tanks]
and does not include the description, which only shows on the image.
The description can be as large as you like. If it gets really large however, you might want to consider moving the image to its own header to keep things slightly saner. This will be especially true after we eventually do: https://github.com/cirosantilli/ourbigbook/issues/180[].
If the description contains any element that would take its own separate line, like multiple paragraphs or a list, we automatically add a line grouping the description with the corresponding image to make that clearer, otherwise it can be hard to know which title corresponds to a far away image. Example with multiple paragraphs:
\OurbigbookExample[[
Stuff before the image.
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=Tank man standing in front of some tanks.}
{id=image-description-argument-test-2}
{source=https://en.wikipedia.org/wiki/File:Tianasquare.jpg}
{description=Note how the tanks are green.
But the shirt is white.}
Stuff after the image description.
]]
= Image `src` argument
{parent=image-argument}
{tag=positional-argument}
The address of the image, e.g. in:
``
\Image[image.png]
``
the `src` is `image.png`.
Analogous to the \x[a-href-argument].
= Image `title` argument
{parent=image-argument}
Analogous to the \x[h-title-argument].
= Include
{title2=`\Include`}
{parent=macro}
The `\Include` macro allows including an external OurBigBook headers under the current header.
It exists to allow optional single page HTML output while still retaining the ability to:
* split up large input files into multiple files to make renders faster during document development
* suggest an optional custom output split with one HTML output per OurBigBook input, in order to avoid extremely large HTML pages which could be slow to load
`\Include` takes one mandatory argument: the ID of the section to be included, much like \x[internal-cross-reference]{p}.
There is however one restriction: only \x[the-toplevel-header]{p} can be pointed to. This restriction allows us to easily find the included file in the filesystem, and dispenses the need to do a first `./ourbigbook` run to generate the \x[internal-cross-file-reference-internals][ID database]. This works because \x[the-id-of-the-first-header-is-derived-from-the-filename].
Headers of the included document are automatically shifted to match the level of the child of the level where they are being included.
If \x[embed-includes] is given, the external document is rendered embedded into the current document directly, essentially as if the source had been copy pasted (except for small corrections such as the header offsets).
Otherwise, the following effects happen:
* The headers of the included tree appear in the \x[table-of-contents] of the document as links to the corresponding external files.
This is implemented simply by reading a previously generated database file much like \x[internal-cross-file-reference-internals], which avoids the slowdown of parsing all included files every time.
As a result, you have to do an initial parse of all files in the project to extract their headers however, just as you would need to do when linking to those headers.
* the include itself renders as a link to the included document
* \x[embed-includes]
Here is an example of inclusion of the files `not-readme.bigb` and `not-readme-2.bigb`:
``
\Include[not-readme]
\Include[not-readme-2]
\Include[not-readme-with-scope]
``
The above is the recommended and slightly \x[insane-macro-shortcuts][insaner] version of:
``
\Include[not-readme]
\Include[not-readme-2]
\Include[not-readme-with-scope]
``
The insaner version is a bit insaner because the `\Include` magically discards the following newline node that follows it if it just a plaintext node containing exactly a newline. With a double newline, the newline would already have been previously taken out on the lexing stage as part of a paragraph.
\x[include-example]{full} shows what those actually render like.
= `\Include` from subdirectories
{parent=include}
When you are in a subdirectory, include resolution just is simply relative to the subdirectory. E.g. we could do:
subdir/index.bigb
``
= Subdir
\Include[notindex]
\Include[subdir2/notindex]
``
subdir/notindex.bigb
``
= Notindex
``
subdir/subdir2/notindex.bigb
``
= Notindex
``
It is not currently possible to include from ancestor directories: https://github.com/cirosantilli/ourbigbook/issues/214[].
= `\Include` `parent` argument
{parent=include}
This option is analogous to \x[h-parent-argument], but for \x[include]{p}.
For example, consider you have:
``
= Animal
== Dog
== Cat
== Bat
``
and now you want to split `Cat` to `cat.bigb`.
If you wrote:
``
= Animal
== Dog
\Include[cat]
== Bat
``
Cat would be a child of Dog, since that is the previous header, which is not what we want.
Instead, we want to write:
``
= Animal
== Dog
\Include[cat]{parent=animal}
== Bat
``
and now Cat will be a child of Animal as desired.
Implemented at: https://github.com/cirosantilli/ourbigbook/issues/127
= `\Include` example
{parent=include}
This shows what includes render as.
\Include[not-readme]
\Include[not-readme-2]
\Include[not-readme-with-scope]
\Include[subdir]
\Include[subdir/notindex]
= Italic
{parent=macro}
{title2=`\i`}
\OurbigbookExample[[
Some \i[italic] text.
]]
= `\JsCanvasDemo`
{parent=macro}
The `JsCanvasDemo` macro allows you to create interactive HTML/JavaScript https://en.wikipedia.org/wiki/Canvas_element[canvas] demos easily.
These demos:
* only start running when the user scrolls over them for the first time
* stop automatically when they leave the viewport
so you can stuff as many of them as you want on a page, and they won't cause the reader's CPU to fry an egg.
\OurbigbookExample[[[
\JsCanvasDemo[[
new class extends OurbigbookCanvasDemo {
init() {
super.init('hello');
this.pixel_size_input = this.addInputAfterEnable(
'Pixel size',
{
'min': 1,
'type': 'number',
'value': 1,
}
);
}
draw() {
var pixel_size = parseInt(this.pixel_size_input.value);
for (var x = 0; x < this.width; x += pixel_size) {
for (var y = 0; y < this.height; y += pixel_size) {
var b = ((1.0 + Math.sin(this.time * Math.PI / 16)) / 2.0);
this.ctx.fillStyle =
'rgba(' +
(x / this.width) * 255 + ',' +
(y / this.height) * 255 + ',' +
b * 255 +
',255)'
;
this.ctx.fillRect(x, y, pixel_size, pixel_size);
}
}
}
}
]]
]]]
And another one showing off some https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API[WebGL]:
\JsCanvasDemo[[
new class extends OurbigbookCanvasDemo {
init() {
super.init('webgl', {context_type: 'webgl'});
this.ctx.viewport(0, 0, this.ctx.drawingBufferWidth, this.ctx.drawingBufferHeight);
this.ctx.clearColor(0.0, 0.0, 0.0, 1.0);
this.vertexShaderSource = `
#version 100
precision highp float;
attribute float position;
void main() {
gl_Position = vec4(position, 0.0, 0.0, 1.0);
gl_PointSize = 64.0;
}
`;
this.fragmentShaderSource = `
#version 100
precision mediump float;
void main() {
gl_FragColor = vec4(0.18, 0.0, 0.34, 1.0);
}
`;
this.vertexShader = this.ctx.createShader(this.ctx.VERTEX_SHADER);
this.ctx.shaderSource(this.vertexShader, this.vertexShaderSource);
this.ctx.compileShader(this.vertexShader);
this.fragmentShader = this.ctx.createShader(this.ctx.FRAGMENT_SHADER);
this.ctx.shaderSource(this.fragmentShader, this.fragmentShaderSource);
this.ctx.compileShader(this.fragmentShader);
this.program = this.ctx.createProgram();
this.ctx.attachShader(this.program, this.vertexShader);
this.ctx.attachShader(this.program, this.fragmentShader);
this.ctx.linkProgram(this.program);
this.ctx.detachShader(this.program, this.vertexShader);
this.ctx.detachShader(this.program, this.fragmentShader);
this.ctx.deleteShader(this.vertexShader);
this.ctx.deleteShader(this.fragmentShader);
if (!this.ctx.getProgramParameter(this.program, this.ctx.LINK_STATUS)) {
console.log('error ' + this.ctx.getProgramInfoLog(this.program));
return;
}
this.ctx.enableVertexAttribArray(0);
var buffer = this.ctx.createBuffer();
this.ctx.bindBuffer(this.ctx.ARRAY_BUFFER, buffer);
this.ctx.vertexAttribPointer(0, 1, this.ctx.FLOAT, false, 0, 0);
this.ctx.useProgram(this.program);
}
draw() {
this.ctx.clear(this.ctx.COLOR_BUFFER_BIT);
this.ctx.bufferData(this.ctx.ARRAY_BUFFER, new Float32Array([Math.sin(this.time / 60.0)]), this.ctx.STATIC_DRAW);
this.ctx.drawArrays(this.ctx.POINTS, 0, 1);
}
}
]]
= List
{parent=macro}
{title2=`* `, `\L`, `\Ul`, `\Ol`}
\x[insane-macro-shortcuts][Insane] with `* ` (asterisk space):
\OurbigbookExample[[
* a
* b
* c
]]
Equivalent saner with \x[auto-parent][implicit `ul` container]:
\OurbigbookExample[[
\L[a]
\L[b]
\L[c]
]]
Equivalent fully sane with explicit container:
\OurbigbookExample[[
\Ul[
\L[a]
\L[b]
\L[c]
]
]]
The explicit container is required if you want to pass extra arguments properties to the `ul` list macro, e.g. a title and an ID: \x[list-my-id]{full}:
\OurbigbookExample[[
\Ul
{id=list-my-id}
[
\L[a]
\L[b]
\L[c]
]
]]
This is the case because without the explicit container in an implicit `ul` list, the arguments would stick to the last list item instead of the list itself.
It is also required if you want ordered lists:
\OurbigbookExample[[
\Ol[
\L[first]
\L[second]
\L[third]
]
]]
Insane nested list with two space indentation:
\OurbigbookExample[[
* a
* a1
* a2
* a2
* b
* c
]]
The indentation must always be exactly equal to two spaces, anything else leads to errors or unintended output.
Equivalent saner nested lists with implicit containers:
\OurbigbookExample[[
\L[
a
\L[a1]
\L[a2]
\L[a2]
]
\L[b]
\L[c]
]]
Insane list item with a paragraph inside of it:
\OurbigbookExample[[
* a
* I have
Multiple paragraphs.
* And
* also
* a
* list
* c
]]
Equivalent sane version:
\OurbigbookExample[[
\L[a]
\L[
I have
Multiple paragraphs.
\L[And]
\L[also]
\L[a]
\L[list]
]
\L[c]
]]
Insane lists may be escaped with a backslash as usual:
\OurbigbookExample[[
\* paragraph starting with an asterisk.
]]
You can also start insane lists immediately at the start of a \x[positional-vs-named-arguments][positional or named argument], e.g.:
\OurbigbookExample[[
\P[* a
* b
* c
]
]]
And now a list outside of \x[ourbigbookexample] to test how it looks directly under \x[toplevel]:
\L[a]
\L[b]
\L[c]
= Mathematics
{parent=macro}
{title2=`$$`, `$`, `\M`, `\m`}
Via https://katex.org/[KaTeX] server side, oh yes!
Inline math is done with the dollar sign (`$`) \x[insane-macro-shortcuts][insane macro shortcut]:
\OurbigbookExample[[
My inline $\sqrt{1 + 1}$ is awesome.
]]
and block math is done with two or more dollar signs (`$$`):
\OurbigbookExample[[
$$
\sqrt{1 + 1} \\
\sqrt{1 + 1}
$$
]]
The sane version of inline math is a lower case `m`:
\OurbigbookExample[[[
My inline \m[[\sqrt{1 + 1}]] is awesome.
]]]
and the sane version of block math is with an upper case `M`:
\OurbigbookExample[[[
\M[[
\sqrt{1 + 1} \\
\sqrt{1 + 1}
]]
]]]
The capital vs lower case theme is also used in other elements, see: \x[block-vs-inline-macros].
In the sane syntax, \x[escape-characters][as with any other argument], you have to either escape any closing square brackets `]` with a backslash `\`:
\OurbigbookExample[[
My inline \m[1 - \[1 + 1\] = -1] is awesome.
]]
or with the equivalent double open and close:
``
My inline \m[[1 - [1 + 1] = -1]] is awesome.
``
HTML escaping happens as you would expect, e.g. < shows fine in:
\OurbigbookExample[[
$$
1 < 2
$$
]]
Equation IDs and titles and linking to equations works identically to \x[image]{p}, see that section for full details. Here is one equation reference example that links to the following insane syntax equation: \x[equation-my-first-insane-equation]:
\OurbigbookExample[[
$$
\sqrt{1 + 1}
$$
{title=My first insane equation.}
]]
and the sane equivalent \x[equation-my-first-sane-equation]:
\OurbigbookExample[[[
\M{title=My first sane equation.}[[
\sqrt{1 + 1}
]]
]]]
Here is a raw one just to test the formatting outside of a `ourbigbook_comment`:
$$\sqrt{1 + 1}$$
= Math defines across blocks
{parent=mathematics}
First here is an invisible block (with `{show=0}`) defining with a `\newcommand` definition after this paragraph:
\OurbigbookExample[[
$$
\newcommand{\foo}[0]{bar}
$${show=0}
]]
We make it invisible because this block only contains KaTeX definitions, and should not render to anything.
Then the second math block uses those definitions:
\OurbigbookExample[[
$$
\foo
$$
]]
Analogously with `\def`, definition:
\OurbigbookExample[[
$$
\gdef\foogdef{bar}
$${show=0}
]]
and the second block using it:
\OurbigbookExample[[
$$
\foogdef
$$
]]
And just to test that `{show=1}` actually shows, although it is useless, and that `{show=0}` skips incrementing the equation count:
\OurbigbookExample[[
$$1 + 1$${show=1}
$$2 + 2$${show=0}
$$3 + 3$${show=1}
]]
= `ourbigbook.tex`
{parent=mathematics}
If your project has multiple `.bigb` input files, you can share Mathematics definitions across all files by adding them to the `ourbigbook.tex` file on the toplevel directory.
For example, if `ourbigbook.tex` contains:
``
\newcommand{\foo}[0]{bar}
``
then from any `.bigb` file we in the project can use:
``
$$
\foo
$$
``
= Paragraph
{title2=`\P`}
{parent=macro}
OK, this is too common, so we opted for some \x[insane-macro-shortcuts][insanity] here: double newline is a paragraph!
\OurbigbookExample[[
Paragraph 1.
Paragraph 2.
]]
Equivalently however, you can use an explicit `\P` macros as well, which is required for example to add properties to a paragraph, e.g.:
\OurbigbookExample[[
\P{id=paragraph-1}[Paragraph 1]
\P{id=paragraph-2}[Paragraph 2]
]]
Paragraphs are created automatically inside \x[macro-argument-syntax][macro argument] whenever a double newline appears.
Note that OurBigBook paragraphs render in HTML as `div` with `class="p"` and not as `p`. This means that you can add basically anything inside them, e.g. a list:
``
My favorite list is:
\Ul[
\li[aa]
\li[bb]
]
because it is simple.
``
which renders as a single paragraph.
One major advantage of this, is that when writing documentation, you often want to keep lists or code blocks inside a given paragraph, so that it is easy to reference the entire paragraph with an ID. Think for example of paragraphs in the C++ standard.
= Passthrough
{parent=macro}
{title2=`\passthrough`}
Dumps its contents directly into the rendered output.
This construct is not XSS safe, see: \x[unsafe-xss]{full}.
Here for example we define a paragraph in raw HTML:
\OurbigbookExample[[[
\passthrough[[
]]
]]]
And for an inline passthrough:
\OurbigbookExample[[[
Hello \passthrough[[raw]] world!
]]]
= Quotation
{parent=macro}
{title2=`\Q`}
With `q`:
\OurbigbookExample[[
And so he said:
\Q[
Something very smart
And with multiple paragraphs.
]
and it was great.
]]
= Table
{parent=macro}
{title2=`|| `, `| `, `\Table`, `\Tr`, `\Th` and `\Td`}
The \x[insane-code-and-math-shortcuts][insane] syntax marks:
* headers with `|| ` (pipe, pipe space) at the start of a line
* regular cells with `| ` (pipe, space) at the start of a line
* separates rows with double newline
For example:
\OurbigbookExample[[
|| Header 1
|| Header 2
| 1 1
| 1 2
| 2 1
| 2 2
]]
Empty cells are allowed without the trailing space however:
\OurbigbookExample[[
| 1 1
|
| 1 3
| 2 1
|
| 2 3
]]
Equivalent fully explicit version:
\OurbigbookExample[[
\Table[
\Tr[
\Th[Header 1]
\Th[Header 2]
]
\Tr[
\Td[1 1]
\Td[1 2]
]
\Tr[
\Td[2 1]
\Td[2 2]
]
]
]]
Any white space indentation inside an explicit `\Tr` can make the code more readable, and is automatically removed from final output due to \x[remove-whitespace-children] which is set for `\Table`.
To pass further arguments to an implicit table such as `title` or `id`, you need to use an explicit `table` macro as in: \x[table-my-table].
\OurbigbookExample[[
\Table
{title=My table title.}
{id=table-my-table}
[
|| Header 1
|| Header 2
| 1 1
| 1 2
| 2 1
| 2 2
]
]]
We would like to remove that explicit toplevel requirement as per: https://github.com/cirosantilli/ourbigbook/issues/186[] The rules of when the caption shows up or not similar to those of \x[image]{p} as mentioned at \x[image-caption]{full}.
Multiple source lines, including paragraphs, can be added to a single cell with insane syntax by indenting the cell with exactly two spaces just as for \x[list]{p}, e.g.:
\OurbigbookExample[[
|| h1
|| h2
|| h3
h3 2
| 11
| 12
12 2
| 13
| 21
| 22
| 23
]]
Arbitrarily complex nested constructs may be used, e.g. a table inside a list inside table:
\OurbigbookExample[[
| 00
| 01
* l1
* l2
| 20
| 21
| 30
| 31
| 10
| 11
]]
And now a table outside of \x[ourbigbookexample] to test how it looks directly under \x[toplevel]:
\Table{title=My table title.}
[
\Tr[
\Th[Header 1]
\Th[Header 2]
]
\Tr[
\Td[1 1]
\Td[1 2]
]
\Tr[
\Td[2 1]
\Td[2 2]
]
]
And a fully insane one:
|| Header 1
|| Header 2
| 1 1
| 1 2
| 2 1
| 2 2
= Table sorting
{parent=table}
JavaScript interactive on-click table sorting is enabled by default, try it out by clicking on the header row:
\OurbigbookExample[[
|| String col
|| Integer col
|| Float col
| ab
| 2
| 10.1
| a
| 10
| 10.2
| c
| 2
| 3.4
]]
Powered by: https://github.com/tristen/tablesort
= Table of contents
{parent=macro}
{title2=`\Toc`}
OurBigBook automatically adds a ToC at the end of the first \x[the-toplevel-header][non-toplevel header] of every document.
For example, on a standard document with a single toplevel header:
``
= Animal
Animals are cute!
== Dog
== Cat
``
the ToC is rendered like:
``
= Animal
Animals are cute!
Table of Contents
* Dog
* Cat
== Dog
== Cat
``
You may customize ToC placement with `\Toc` macro, but just don't do it, it will just go against common convention and confuse readers.
This is especially important when considering \x[split-headers], where you almost always want an automatically generated ToC for every split header, otherwise you would need to add them all manually.
Only one ToC is rendered per document. Any ToC besides the first one is ignored. In particular, this means that \x[include]{p} with \x[embed-includes] work seamlessly and render a single table of contents, even if multiple `\Toc` macros are present in the included pages.
The \x[internal-cross-reference][ID] of the ToC is always fixed to `#toc`. If you try to use that for another element, you will get the following error:
``
error: tmp.bigb:3:1: reserved ID "toc"
``
The ToC ignores \x[the-toplevel-header] if you have one.
For when you want a quick outline of the header tree on the terminal, also consider the \x[log-headers] option.
= Table of contents JavaScript open close interaction
{parent=table-of-contents}
To the left of table of content entries you can click on an open/close icon to toggle the visibility of different levels of the table of contents.
The main use case covered by the expansion algorithm is as follows:
* the page starts with all nodes open to facilitate Ctrl + F queries
* if you click on a node in that sate, you close all its children, to get a summarized overview of the contents
* if you click one of those children, it opens only its own children, so you can interactively continue exploring the tree
The exact behaviour is:
* the clicked node is open:
* state 1 all children are closed. Action: open all children recursively, which puts us in state 2
* state 2: not all children are closed. Action close all children, which puts us in state 1. This gives a good overview of the children, without any children of children getting in the way.
* state 3: the clicked node is closed (not showing any children). Action: open it to show all direct children, but not further descendants (i.e. close those children). This puts us in state 1.
Note that those rules make it impossible to close a node by clicking on it, the only way to close a node os to click on its parent, the state transitions are:
* 3 -> 1
* 1 -> 2
* 2 -> 1
but we feel that it is worth it to do things like this to cover the main use case described above without having to add two buttons per entry.
Clicking on the link from a \x[header] up to the table of contents also automatically opens up the node for you in case it had been previously closed manually.
= Video
{parent=macro}
{title2=`\Video` and `\video`}
Very analogous to \x[image]{p}, only differences will be documented here.
In the case of videos, \x[where-to-store-images] becomes even more critical since videos are even larger than images, such that the following storage approaches are impractical off the bat:
* \x[store-images-inside-the-repository-itself]
* \x[store-images-in-a-separate-media-repository]
As a result, then https://commons.wikimedia.org[Wikimedia Commons] is one of the best options \x[store-images-in-wikimedia-commons][much like for images]:
\OurbigbookExample[[
\Video[https://upload.wikimedia.org/wikipedia/commons/8/85/Vacuum_pump_filter_cut_and_place_in_eppendorf.webm]
{id=sample-video-in-wikimedia-commons}
{title=Nice sample video stored in Wikimedia Commons.}
{start=5}
]]
We also handle more complex transcoded video URLs just fine:
\OurbigbookExample[[
\Video[https://upload.wikimedia.org/wikipedia/commons/transcoded/1/19/Scientific_Industries_Inc_Vortex-Genie_2_running.ogv/Scientific_Industries_Inc_Vortex-Genie_2_running.ogv.480p.vp9.webm]
{id=sample-video-in-wikimedia-commons-transcoded}
{title=Nice sample video stored in Wikimedia Commons transcoded.}
]]
Commons is better than YouTube if your content is on-topic there because:
* they have no ads
* it allows download of the videos: https://www.quora.com/Can-I-download-Creative-Commons-licensed-YouTube-videos-to-edit-them-and-use-them[].
* it makes it easier for other users to find and re-use your videos
If your video does not fit the above Wikimedia Commons requirements, YouTube could be a good bet. OurBigBook https://github.com/cirosantilli/ourbigbook/issues/50[automatically detects YouTube URLs] for you, so the following should just work:
\OurbigbookExample[[
\Video[https://youtube.com/watch?v=YeFzeNAHEhU&t=38]
{id=sample-video-from-youtube-implicit-youtube}
{title=Nice sample video embedded from YouTube implicit from `youtube.com` URL.}
]]
The `youtu.be` domain hack URLs also work;
\OurbigbookExample[[
\Video[https://youtu.be/YeFzeNAHEhU?t=38]
{id=sample-video-from-youtube-implicit-youtu-be}
{title=Nice sample video embedded from YouTube implicit from `youtu.be` URL.}
]]
Alternatively, you can reach the same result in a more explicit and minimal way by setting `{provider=youtube}` and the \x[video-start-argument][`start`] arguments:
\OurbigbookExample[[
\Video[YeFzeNAHEhU]{provider=youtube}
{id=sample-video-from-youtube-explicit}
{title=Nice sample video embedded from YouTube with explicit `youtube` argument.}
{start=38}
]]
When the `youtube` provider is selected, the Video address should only to contain the YouTube video ID, which shows in the YouTube URL for the video as:
``
https://www.youtube.com/watch?v=
``
Remember that you can also enable the `youtube` provider by default on your \x[ourbigbook-json] with:
``
"media-provider" {
"youtube": {"default-for": "video"}
}
``
But you can also use raw video files from any location that can serve them of course, e.g. here is one stored in this repository: \x[sample-video-in-repository].
\OurbigbookExample[[
\Video[Tank_man_side_hopping_in_front_of_some_tanks.mp4]
{id=sample-video-in-repository}
{title=Nice sample video stored in this repository.}
{source=https://www.youtube.com/watch?v=YeFzeNAHEhU}
{start=3}
]]
And as for images, setting `titleFromSrc` automatically calculates a title for you:
\OurbigbookExample[[
\Video[Tank_man_side_hopping_in_front_of_some_tanks.mp4]
{titleFromSrc}
{source=https://www.youtube.com/watch?v=YeFzeNAHEhU}
]]
= Video lazy loading
{parent=video}
Unlike \x[image-lazy-loading], we don't support video lazy loading yet because:
* non-`youtube` videos use the `video` tag which has no `loading` property yet
* `youtube` videos are embedded with `iframe` and `iframe` has no `loading` property yet
Both of this cases could be worked around with JavaScript:
* non-`youtube`: set `src` from JavaScript as shown for images: https://stackoverflow.com/questions/2321907/how-do-you-make-images-load-lazily-only-when-they-are-in-the-viewport/57389607#57389607[].
But this breaks page semantics however, we don't know how to work around that
* `youtube` videos: same as above for the `iframe`, but this should be less problematic since YouTube videos are not viewable without JavaScript anyways, and who cares about `iframe` semantics?
= `\Video` `start` argument
{parent=video}
The time to start playing the video at in seconds. Works for both `youtube` and non-YouTube videos.
= Internal cross reference
{parent=macro}
= `\x` macro
{title2}
{synonym}
Every macro in OurBigBook can have an optional `id` and many also have a reserved `title` property.
When a macro in the document has a `title` argument but no `id` argument given, get an auto-generated ID from the title: \x[automatic-id-from-title].
For macros that do have an ID, derived from `title` or not, you can write a cross reference to it, e.g.:
\OurbigbookExample[[
See this \x[internal-cross-reference]{p} awesome section.
]]
An explicit link body can be given just as for regular HTTP \x[link]{p} as:
\OurbigbookExample[[
See this \x[internal-cross-reference]{p}[awesome section].
]]
= Cross reference title inflection
{parent=internal-cross-reference}
A common usage pattern is that we want to use \x[header] titles in \x[x-full-argument][non-full] \x[internal-cross-reference]{p} as the definition of a concept without repeating the title, for example:
``
== Dog
Cute animal.
\x[cats][Cats] are its natural enemies.
== Cats
This is the natural enemy of a \x[dog][dog].
\x[dog][Dogs] are cute, but they are still the enemy.
One example of a cat is \x[felix-the-cat].
=== Felix the Cat
Felix is not really a \x[cats][cat], just a carton character.
``
However, word https://en.wikipedia.org/wiki/Inflection[inflection] makes it much harder to avoid retyping the definition again.
For example, in the previous example, without any further intelligent behaviour we would be forced to re-type `\x[dog][dog]` instead of the desired `\x[dog]`.
OurBigBook can take care of some inflection cases for you.
For capitalization, both headers and cross reference macros have the `c` \x[boolean-argument] which stands for "capitalized":
* for headers, `c` means that the header title has fixed capitalization as given in the title, i.e.
* if the title has a capital first character, it will always show as a capital, as is the case for most https://en.wikipedia.org/wiki/Proper_noun[proper noun]
* if it is lower case, it will also always remain lower case, as is the case for some rare proper nouns, notably https://en.wikipedia.org/wiki/Arm_Holdings[the name of certain companies]
This means that for such headers, `c` in the `x` has no effect. Maybe we should give an error in that case. But lazy now, send PR.
* for cross reference macros, `c` means that the first letter of the title should be capitalized.
Using this option is required when you are starting a sentence with a non-proper noun.
Capitalization is handled by a \x[javascript-case-conversion].
For pluralization, cross reference macros have the `p` \x[boolean-argument] which stands for "pluralize":
* if given and true, this automatically pluralizes the last word of the target title by using the https://github.com/blakeembrey/pluralize library
* if given and false, automatically singularize
* if not given, don't change the number of elements
If your desired pluralization is any more complex than modifying the last word of the title, you must do it manually however.
With those rules in mind, the previous OurBigBook example can be written with less repetition as:
``
== Dog
Cute animal.
\x[cats]{c} are its natural enemies.
== Cats
This is the natural enemy of a \x[dog].
\x[dog]{p} are cute, but they are still the enemy.
One example of a cat is \x[Felix the Cat].
=== Felix the Cat
{c}
Felix is not really a \x[cats][cat], just a carton character.
``
If plural and capitalization don't handle your common desired inflections, you can also just create custom ones with the \x[h-synonym-argument].
Now for a live example for quick and dirty interactive testing.
= Cross reference title inflection example
{parent=cross-reference-title-inflection}
\OurbigbookExample[[
\x[inflection-example-not-proper]
]]
\OurbigbookExample[[
\x[inflection-example-not-proper]{c}
]]
\OurbigbookExample[[
\x[inflection-example-not-proper]{full}
]]
\OurbigbookExample[[
\x[inflection-example-proper]
]]
\OurbigbookExample[[
\x[inflection-example-proper]{c}
]]
\OurbigbookExample[[
\x[inflection-example-not-proper-lower]
]]
\OurbigbookExample[[
\x[inflection-example-not-proper-lower]{c}
]]
\OurbigbookExample[[
\x[inflection-example-proper-lower]
]]
\OurbigbookExample[[
\x[not-readme]
]]
\OurbigbookExample[[
\x[not-readme]{c}
]]
\OurbigbookExample[[
\x[inflection-example-not-proper]{p}
]]
\OurbigbookExample[[
\x[inflection-plural-examples]
]]
\OurbigbookExample[[
\x[inflection-plural-examples]{p}
]]
\OurbigbookExample[[
\x[inflection-plural-examples]{p=0}
]]
\OurbigbookExample[[
\x[inflection-plural-examples]{p=1}
]]
\OurbigbookExample[[
\x[not-the-readme-header-with-fixed-case]
]]
= Inflection example not-proper
{parent=cross-reference-title-inflection-example}
= Inflection example proper
{c}
{parent=cross-reference-title-inflection-example}
= inflection example not-proper lower
{parent=cross-reference-title-inflection-example}
= inflection example proper lower
{c}
{parent=cross-reference-title-inflection-example}
= Inflection plural examples
{parent=cross-reference-title-inflection-example}
= `\x` within `title` restrictions
{parent=internal-cross-reference}
If you use `\x` within a `title`, which most commonly happens for \x[image][image titles], that can generate complex dependencies between IDs, which would either be harder to implement, or lead to infinite recursion.
To prevent such problems, OurBigBook emits an error if you use an `\x` without content in the `title` of one of the following elements:
* any \x[header]. For example, the following gives an error:
``
= h1
{id=myh1}
== \x[myh1]
``
This could be solved by either adding a content to the reference:
``
= h1
{id=myh1}
== \x[myh1][mycontent]
``
or by adding an explicit ID to the header:
``
= h1
{id=myh1}
== \x[myh1]
{id=myh2}
``
* non-header (e.g. an \x[image]) that links to the title of another non-header
For non-headers, things are a bit more relaxed, and we can link to headers, e.g.:
``
= h1
\Image[myimg.jpg]
{title=my \x[h1].}
``
This is allowed because OurBigBook calculates IDs in two stages: first for all headers, and only later non non-headers.
What you cannot do is link to another image e.g.:
``
\Image[myimg.jpg]
{id=myimage1}
{title=My image 1.}
\Image[myimg.jpg]
{title=my \x[h1].}
``
and there the workaround are much the same as for headers: either explicitly set the cross reference content:
``
\Image[myimg.jpg]
{id=myimage1}
{title=My image 1.}
\Image[myimg.jpg]
{title=my \x[h1][My image 1].}
``
or explicitly set an ID:
``
\Image[myimg.jpg]
{id=myimage1}
{title=My image 1.}
\Image[myimg.jpg]
{id=myimage2}
{title=my \x[h1].}
``
While it is technically possible relax the above limitations and give an error only in case of loops, it would require a bit of extra work which we don't want to put in right now: https://github.com/cirosantilli/ourbigbook/issues/95[].
Furthermore, the above rules do not exclude infinite rendering loops, but OurBigBook detects such loops and gives a nice error message, this has been fixed at: https://github.com/cirosantilli/ourbigbook/issues/34
For example this would contain an infinite loop:
``
\Image[myimg.jpg]
{id=myimage1}
{title=\x[myimage2].}
\Image[myimg.jpg]
{id=myimage2}
{title=\x[myimage1].}
``
This infinite recursion is fundamentally not technically solved: the user has to manually break the loop by providing an `x` content explicitly, e.g. in either:
``
\Image[myimg.jpg]
{id=myimage1}
{title=\x[myimage2][my content 2].}
\Image[myimg.jpg]
{id=myimage2}
{title=\x[myimage1].}
``
or:
``
\Image[myimg.jpg]
{id=myimage1}
{title=\x[myimage2].}
\Image[myimg.jpg]
{id=myimage2}
{title=\x[myimage1][my content 1].}
``
A closely related limitation is the simplistic approach to \x[x-id-output-format].
= Internal cross file reference
{parent=internal-cross-reference}
Reference to the first header of another file:
\OurbigbookExample[[
\x[not-readme]
]]
Reference to a non-first header of another file:
\OurbigbookExample[[
\x[h2-in-not-the-readme]
]]
To make toplevel links cleaner,if target header is the very first element of the other page, then the link does not get a fragment, e.g.: `\x[not-readme]` rendered as:
``
``
and not:
``
``
while `\x[h2-in-not-the-readme]` is rendered with the fragment:
``
``
Reference to the first header of another file that is a second inclusion:
\OurbigbookExample[[
\x[included-by-not-readme]
]]
Reference to another header of another file, with \x[x-full-argument][`full`]:
\OurbigbookExample[[
\x[h2-in-not-the-readme]{full}.
]]
Note that when `full` is used with references in another file in \x[embed-includes][multi page mode], the number is not rendered as explained at: \x[x-full-argument-in-internal-cross-file-references]{full}.
Reference to an image in another file:
\OurbigbookExample[[
\x[image-not-readme-xi]{full}.
]]
Reference to an image in another file:
\OurbigbookExample[[
\x[image-figure-in-not-the-readme-without-explicit-id]{full}.
]]
Remember that the \x[the-toplevel-header][ID of the toplevel header] is automatically derived from its file name, that's why we have to use:
\OurbigbookExample[[
\x[not-readme]
]]
instead of:
``
\x[not-the-readme]
``
Reference to a subdirectory:
\OurbigbookExample[[
\x[subdir]
\x[subdir/h2]
\x[subdir/notindex]
\x[subdir/notindex-h2]
]]
Implemented at: https://github.com/cirosantilli/ourbigbook/issues/116
Reference to an internal header of another file: \x[h2-in-not-the-readme]. By default, That header ID gets prefixed by the ID of the top header.
When using \x[embed-includes] mode, the cross file references end up pointing to an ID inside the current HTML element, e.g.:
``
``
rather than:
``
``
This is why IDs must be unique for elements across all pages.
= Internal cross file reference internals
{parent=internal-cross-file-reference}
{title2=`db.sqlite3`}
= ID database
{c}
{synonym}
When running in Node.js, OurBigBook dumps the IDs of all processed files to a `out/db.sqlite3` file in \x[the-out-directory], and then reads from that file when IDs are needed.
When converting under a directory that contains \x[ourbigbook-json], `out/db.sqlite3` is placed inside the same directory as the `ourbigbook.json` file.
If there is no `ourbigbook.json` in parent directories, then `out/db.sqlite3` is placed in the current working directory.
These follows the principles described at: \x[the-current-working-directory-does-not-matter-when-there-is-a-ourbigbook-json].
`db.sqlite3` is not created or used when handling input from stdin.
When running in the browser, the same JavaScript API will send queries to the server instead of a local SQLite database.
To inspect the ID database to debug it, you can use:
``
sqlite3 out/db.sqlite3 .dump
``
It is often useful to dump a single table, e.g. to dump the `ids` table:
``
sqlite3 out/db.sqlite3 '.dump ids'
``
and one particularly important query is to dump a list of all known IDs:
``
sqlite3 out/db.sqlite3 'select id from ids'
``
You can force `ourbigbook` to not use the ID database with the \x[no-db] command line option
= Internal cross reference title link removal
{parent=internal-cross-file-reference}
If the target `title` argument contains a link from either another \x[internal-cross-reference]{p} or a regular \x[link][external hyperlink], OurBigBook automatically prevents that link from rendering as a link when no explicit body is given.
This is done because https://stackoverflow.com/questions/9882916/are-you-allowed-to-nest-a-link-inside-of-a-link[nested links are illegal in HTML], and the result would be confusing.
This use case is most common when dealing with media such as \x[image]{p}. For example in:
``
= afds
\x[image-aa-zxcv-lolol-bb]
== qwer
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=aa \x[zxcv][zxcv] \a[http://example.com][lolol] bb}
== zxcv
``
the `\x[image-aa-zxcv-lolol-bb]` renders something like:
``
aa zxcv lolol bb
``
and not:
``
aa zxcv lolol bb
``
Live example:
\OurbigbookExample[[
This is a nice image: \x[image-aa-zxcv-lolol-bb].
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=aa \x[internal-cross-reference-title-link-removal][zxcv] \a[http://example.com][lolol] bb}
]]
= `\x` arguments
{parent=internal-cross-reference}
= `\x` `c` argument
{parent=x-arguments}
Capitalizes the first letter of the target title.
For more details, see: \x[cross-reference-title-inflection]{full}.
= `\x` `child` argument
{parent=x-arguments}
Setting the `child` \x[boolean-argument] on a cross reference to a header as in:
``
\x[my-header]{child}
``
makes that header show up on the list of extra parents of the child.
This allows a section to have multiple parents, e.g. to include it into multiple categories. For example:
``
= Animal
== Mammal
=== Bat
=== Cat
== Flying animal
These animals fly:
* \x[bat]{child}
These animals don't fly:
* \x[cat]
``
would render something like:
``
= Animal
== Mammal
=== Bat (Parent section: Mammal)
(Tags: Flying animal)
=== Cat (Parent section: Mammal)
== Flying animal (Parent section: Animal)
These animals fly:
* \x[bat]
These animals don't fly:
* \x[cat]
``
so note how "Bat" has a list of tags including "Flying animal", but Cat does not, due to the `child`.
This property does not affect how the \x[table-of-contents][`\Toc`] is rendered. We could insert elements sections there multiple times, but it has the downside that browser Ctrl + F searches would hit the same thing multiple times on the table of contents, which might make finding things harder.
``
== My title{id=my-id}
Read this \x[my-id][amazing section].
``
If the second argument, the `content`, is not present, it expand to the header title, e.g.:
``
== My title{id=my-id}
Read this \x[my-id].
``
is the same as:
``
== My title{id=my-id}
Read this \x[my-id][My title].
``
A live demo can be seen at: \x[x-child-argument-demo].
Generally, a better alternative to this argument is to use \x[h-child-argument].
= Secondary children
{parent=x-child-argument}
The term refers to sections that have a parent/child relationship via either of the:
* \x[x-child-argument]
* \x[x-parent-argument]
* \x[h-child-argument]
rather than via the usual \x[header] hierarchy.
Secondary children show up for example on the \x[tagged] metadata section, but not on the \x[table-of-contents], which is what the header hierarchy already shows.
Secondary children are normally basically used as "tags": a header such as `Bat` can be a direct child of `Mammal`, and a secondary child of `Flying animal`, or vice versa. Both `Mammal` and `Flying animal` are then basically ancestors. But we have to chose one main ancestor as "the parent", and other secondary ancestors will be seen as tags.
= `\x` `child` argument demo
{parent=x-child-argument}
{scope}
= Animal
{parent=x-child-argument-demo}
= Ant
{parent=animal}
= Cow
{parent=animal}
Oh, and cows are also \x[mammal]{parent}{p}.
= Mammal
{parent=animal}
= Bat
{parent=mammal}
= Cat
{parent=mammal}
= Flying animal
{parent=animal}
\x[bat]{child}{c}{p} can fly.
But \x[cat]{p} can't.
= `\x` `parent` argument
{parent=x-child-argument}
The `parent` argument is exactly like the \x[x-child-argument], but it reverses the direction of the parent/child relation.
= `\x` `full` argument
{parent=x-arguments}
To also show the section auto-generated number as in "Section X.Y My title" we add the optional `{full}` \x[boolean-argument] to the cross reference, for example:
\OurbigbookExample[[
\x[x-full-argument]{full}.
]]
`{full}` is not needed for cross references to most macros besides \x[header]{p}, which use `full` by default as seen by the `default_x_style_full` macro property in \x[help-macros]. This is for example the case for \x[image]{p}. You can force this to be disabled with `{full=0}`:
\OurbigbookExample[[
Compare \x[image-my-test-image]{full=0} vs \x[image-my-test-image]{full=1}.
]]
= `\x` `full` argument in internal cross file references
{parent=x-full-argument}
For example in the following \x[internal-cross-file-reference]:
\OurbigbookExample[[
\x[h2-in-not-the-readme]{full}.
]]
we get just something like:
``
Section "h2 in not the readme"
``
instead of:
``
Section 1.2 "h2 in not the readme"
``
This is because the number "Section 1.2" might already have been used in the current page, leading to confusion.
= `\x` `p` argument
{parent=x-arguments}
Pluralizes or singularizes the last word of the target title.
For more details, see: \x[cross-reference-title-inflection]{full}.
= `\x` `ref` argument
{parent=x-arguments}
The `ref` argument of `\x` marks the link as reference, e.g.:
``
Trump said this and that.\x[donald-trump-access-hollywood-tape]{ref}
= Donald Trump Access Hollywood tape
``
renders something like:
``
Trump said this and that.*
``
This could currently be replicated without `ref` by just using:
``
Trump said this and that.\x[donald-trump-access-hollywood-tape][*]
``
but later on we might add more precise reference fields like the page of a book or date fetched as Wikipedia supports.
Implemented at: https://github.com/cirosantilli/ourbigbook/issues/137
= OurBigBook Markup syntax
{c}
{parent=ourbigbook-markup}
= Insane macro shortcuts
{parent=ourbigbook-markup-syntax}
= Insane syntax
{synonym}
= Insane
{synonym}
Certain commonly used macros have insane macro shortcuts that do not start with backslash (`\`).
Originally, \x[design-goals][Ciro wanted to avoid those], but they just feel too good to avoid.
Every insane syntax does however have an equivalent sane syntax.
The style recommendation is: use the insane version which is shorter, unless you have a specific reason to use the sane version.
= Macros with insane shortcut
{parent=insane-macro-shortcuts}
* \x[paragraph]{child}{full}: `\n\n` (double newline)
* \x[link]{child}{full}: `a http://example.com b` (space followed by `http://`)
* \x[mathematics]{child}{full}: `$`, described at: \x[insane-code-and-math-shortcuts]
* \x[code-block]{child}{full}: \c[[`]], described at: \x[insane-code-and-math-shortcuts]
* \x[list]{child}{full}: `* ` and indentation
* \x[table]{child}{full}: `|| `, `| ` and indentation
= Insane code and math shortcuts
{parent=macros-with-insane-shortcut}
The insane code and math shortcuts work very analogously and are therefore described together in this section.
The insane inline code syntax:
\OurbigbookExample[[
a `b c` d
]]
and is equivalent to the sane:
``
a \c[[b c]] d
``
The insane block code:
\OurbigbookExample[[
a
``
b c
``
d
]]
and is equivalent to the sane:
``
a
\C[[
b c
]]
d
``
= Insane macro shortcut extra arguments
{parent=macros-with-insane-shortcut}
Insane arguments always work by abbreviating:
* the macro name
* one or more of its positional arguments, which are fixed as either \x[literal-arguments][literal or non-literal] for a given insane construct
This means that you can add further arguments as usual.
For example, an insane code block with an id can be written as:
``
a `b c`{id=ef} g
``
because that is the same as:
\OurbigbookExample[[
a \c[b c]{id=ef} g
]]
So we see that the `b c` argument is the very first argument of `\c`.
Extra arguments must come after the insane opening, e.g. the following does not work:
``
a {id=ef}`b c` g
``
This restriction things easy to parse for humans and machines alike.
= Escapes in insane macro shortcuts
{parent=macros-with-insane-shortcut}
Literal backticks and dollar signs can be produced witha backslash escape as in:
\OurbigbookExample[[
a \` \$ b
]]
It is not possible to escape backticks (\c[[`]]) inside an insane inline code, or dollar signs (`$`) in insane math.
The design reason for that is because multiple backticks produce block code.
The upside is that then you don't have to escape anything else, e.g. backslashes (`\`) are rendered literally.
The only way to do it is to use the sane syntax instead:
\OurbigbookExample[[[
a \c[[b ` c]] d
a \m[[\sqrt{\$4}]] d
]]]
Within block code and math, you can just add more separators:
\OurbigbookExample[[
```
code with two backticks
``
nice
```
]]
= Macro argument syntax
{parent=ourbigbook-markup-syntax}
= Positional vs named arguments
{parent=macro-argument-syntax}
{title2=`[...]` vs `{key=...}`}
Every argument in OurBigBook is either positional or named.
For example, in a \x[header] definition with an ID:
``
= My asdf
{id=asdf qwer}
{scope}
``
which is equivalent to the \x[insane-macro-shortcuts][sane] version:
``
\H[1][My asdf]
{id=asdf qwer}
{scope}
``
we have:
* two positional argument: `[1]` and `[My asdf]`. Those are surrounded by square brackets `[]` and have no name
* two named arguments: `{id=asdf qwer}` and `{scope}`.
The first one has name `id`, followed by the separator `=`, followed by the value `asdf qwer`.
The separator `=` always is optional. If not given, it is equivalent to an empty value, e.g.:
``
{id=}
``
is the same as:
``
{id}
``
You can determine if a macro is positional or named by using \x[help-macros]. Its output contains something like:
``
"h": {
"name": "h",
"positional_args": [
{
"name": "level"
},
{
"name": "content"
}
],
"named_args": {
"id": {
"name": "id"
}
"scope": {
"name": "scope"
}
},
``
and so we see that `level` and `content` are positional arguments, and `id` and `scope` are named arguments.
Generally, positional arguments are few (otherwise it would be hard to know which is which is which), and are almost always used for a given element so that they save us from typing the name too many times.
The order of positional arguments must of course be fixed, but named arguments can go anywhere. We can even mix positional and named arguments however we want, although this is not advised for clarity.
The following are therefore all equivalent:
``
\H[1][My asdf]{id=asdf qwer}{scope}
\H[1][My asdf]{scope}{id=asdf qwer}
\H{id=asdf qwer}{scope}[1][My asdf]
\H{scope}[1]{id=asdf qwer}[My asdf]
``
Just like named arguments, positional arguments are never mandatory.
= Positional argument
{parent=positional-vs-named-arguments}
See: \x[positional-vs-named-arguments]{full}.
= Positional argument default values
{parent=positional-argument}
Most positional arguments will default to an empty string if not given.
However, some positional arguments can have special effects if not given.
For example, an anchor with the first positional argument present (the URL), but not the second positional argument (the link text) as in:
\OurbigbookExample[[
\a[http://example.com]
]]
has the special effect of generating automatic links as in:
``
\a[http://example.com][http://example.com]
``
This can be contrasted with named arguments, for which there is always a default value, notably for \x[boolean-argument]{p}.
See also: \x[link]{full}.
= Mandatory positional arguments
{parent=positional-argument}
Some positional arguments are required, and if not given OurBigBook reports an error and does not render the node.
This is for example the `level` of a \x[header].
These arguments marked with the `mandatory: true` \x[help-macros] argument property.
= Named argument
{parent=positional-vs-named-arguments}
See: \x[positional-vs-named-arguments]{full}.
= Boolean argument
{parent=macro-argument-syntax}
{child=a-relative-argument}
{child=h-child-argument}
{child=h-numbered-argument}
{child=h-splitdefault-argument}
{child=x-c-argument}
{child=x-p-argument}
{child=x-ref-argument}
To also show the section auto-generated number as in "Section X.Y My title" we add the optional `{full}` \x[boolean-argument] to the cross reference, for example:
Name arguments marked in \x[help-macros] as `boolean: true` must either:
* take no value and no `=` sign, in which case the value is implicitly set to `1`
* take value exactly `0` or `1`
* not be given, in which case a custom per-macro default is used. Tha value is the `default` from \x[help-macros], or `0` if such default is not given
For example, \x[x-full-argument][the `full` argument] of \x[internal-cross-reference]{p} is correctly written as:
\OurbigbookExample[[
\x[boolean-argument]{full}
]]
without the `=` sign, or equivalently:
\OurbigbookExample[[
\x[boolean-argument]{full=1}
]]
The `full=0` version is useful in the case of reference targets that unlike \x[header]{p} expand the title on the cross reference by default, e.g. \x[image]{p}:
\OurbigbookExample[[
\x[boolean-argument]{full=1}
]]
The name "boolean argument" is given by analogy to the https://stackoverflow.com/questions/16109358/what-is-the-correct-readonly-attribute-syntax-for-input-text-elements/24588427#24588427["boolean attribute" concept in HTML5].
= JavaScript interface for arguments
{c}
{parent=macro-argument-syntax}
The JavaScript interface sees arguments as follows:
``
function macro_name(args)
``
where args is a dict such that:
* optional arguments have the key/value pairs explicitly given on the call
* mandatory arguments have a key documented by the API, and the value on the call.
For example, the link API names its arguments `href` and `text`.
= Literal arguments
{title2=`[[...]]` and `{{key=...}}`}
{parent=macro-argument-syntax}
Arguments that are opened with more than one square brackets `[` or curly braces `{` are literal arguments.
In literal arguments, OurBigBook is not parsed, and the entire argument is considered as text until a corresponding close with the same number of characters.
Therefore, you cannot have nested content, but it makes it extremely convenient to write \x[code-block]{p} or \x[mathematics].
For example, a multiline code block with double open and double close square brackets inside can be enclosed in triple square brackets:
\OurbigbookExample[[[
A literal argument looks like this in OurBigBook:
\C[[
\C[
A multiline
code block.
]
]]
And another paragraph.
]]]
The same works for inline code:
\OurbigbookExample[[[
The program \c[[puts("]");]] is very complex.
]]]
Within literal blocks, only one thing can be escaped with backslashes are:
* leading open square bracket `[`
* trailing close square bracket `]`
The rule is that:
* if the first character of a literal argument is a sequence of backslashes (`\`), and it is followed by another argument open character (e.g. `[`, remove the first `\` and treat the other characters as regular text
* if the last character of a literal argument is a `\`, ignore it and treat the following closing character (e.g. `]`) as regular text
See the following open input/output pairs:
``
\c[[\ b]]
\c[[\a b]]
\c[[\[ b]]
\c[[\\[ b]]
\c[[\\\[ b]]
``
and close examples:
``
\c[[a \]]
\c[[a \]]]
\c[[a \\]]]
``
= Argument leading newline removal
{parent=macro-argument-syntax}
If the very first character of an argument is a newline, then that character is ignored if it would be part of a regular plaintext node.
For example:
``
\C[[
a
b
]]
``
generates something like:
``
``
instead of:
``
``
This is extremely convenient to improve the readability of code blocks and similar constructs.
The newline is however considered if it would be part of some \x[insane-macro-shortcuts]{p=0}. For example, we can start an \x[list][insane list] inside a \x[quotation]{p} as in:
\OurbigbookExample[[
\Q[
* a
* b
]
]]
where the insane list requires a leading newline `\n* ` to work. That newline is not ignored, even though it comes immediately after the `\Q[` opening.
= Argument newlines between arguments removal
{parent=macro-argument-syntax}
The macro name and the first argument, and two consecutive arguments, can be optionally separated by exactly one newline character, e.g.:
``
\H[2]
{scope}
[Design goals]
``
is equivalent to:
``
\H[2]{scope}[Design goals]
``
and this non-recommended mixed style:
``
\H[2]{scope}
[Design goals]
``
This allows to greatly improve the readability of long argument lists by having them one per line.
There is one exception to this however: inside an \x[header][insane header], any newline is interpreted as the end of the insane header. This is why the following works as expected:
``
== My header 2 `some code`
{id=asdf}
``
and the `id` gets assigned to the header rather than the trailing code element.
= Escape characters
{parent=macro-argument-syntax}
For \x[literal-arguments][non-literal macro arguments], you have to use a backslash to escape:
* `\`: backslashes start macros
* `\` and `\`: open and close \x[positional-vs-named-arguments][positional macro arguments]
* `\` and `\`: open and close \x[positional-vs-named-arguments][optional macro arguments]
* `$` (dollar sign): \x[insane-macro-shortcuts][insane macro shortcut] for \x[mathematics]
* \c[[`]] (backtick): \x[insane-macro-shortcuts][insane macro shortcut] for \x[code-block]{p}
The escape rules for literal arguments are described at: \x[literal-arguments]{full}.
This is good for short arguments of regular text, but for longer blocks like \x[code-block]{p} or \x[mathematics], you may want to use \x[literal-arguments]
= Macro argument property
{parent=macro-argument-syntax}
= `remove_whitespace_children`
{parent=macro-argument-property}
In HTML, certain elements such as `
{c}
{title2=previously Cirodown}
https://ourbigbook.com source code + a compatible local CLI static wiki generator and markup language to write complex structured wikis/books/blogs with reference implementation in JavaScript.
This repository (The OurBigBook Project) contains two main things:
* \x[ourbigbook-cli]: a static Wiki generator that can be invoked from the command line with the \x[ourbigbook-executable]. The OurBigBook CLI calls the \x[ourbigbook-library] to convert each input file.
The markup language used is called \x[ourbigbook-markup] (file extension: `.bigb`), and it is compatible with https://ourbigbook.com[], TODO support import/export to/from https://ourbigbook.com[] to local filesystem.
* \x[ourbigbook-web]: the source code of https://ourbigbook.com[], which is a regular database-backed dynamic website. This is basically separate package that depends on the OurBigBook Library:
* information about OurBigBook Web is present at: \x[ourbigbook-web]{full}
* the source code of the OurBigBook Web is present under `web/`
Everything else in this repository outside of that section/directory is about the OurBigBook Library and the OurBigBook CLI, not the dynamic website.
Key links:
* https://cirosantilli.com[]: showcase demo document with interesting content. Primary inspiration for OurBigBook development.
* https://cirosantilli.com/oxford-nanopore-river-bacteria[]: a self-contained tutorial style part of the above. Note how internal links integrate seamlessly into the more global topic of biology, e.g. when talking about DNA we link to the global topic https://cirosantilli.com/dna[].
* https://github.com/cirosantilli/cirosantilli.github.io[] and https://github.com/cirosantilli/cirosantilli.github.io/blob/dev/oxford-nanopore-river-bacteria.bigb[]: source of the above showcase documents
* \x[design-goals]{full}: feature overview
* https://github.com/cirosantilli/ourbigbook[]: OurBigBook source code
* https://github.com/cirosantilli/ourbigbook/blob/master/README.bigb[]: source for this document
* https://cirosantilli.com/ourbigbook[]: rendered version of this document
* https://cirosantilli.com/ourbigbook/editor[]: live in-browser demo
* https://cirosantilli.com/ourbigbook-template[]: good template to get started, see \x[quick-start]{full}
= Features
{parent=ourbigbook}
* \x[internal-cross-reference][references] to any \x[header]{p} (including e.g. h2, h3, etc. in other files), \x[image]{p}, etc. with amazing \x[error-reporting][error checking and reporting]: never break internal links without knoing again, and quickly find out what broke when you do. E.g.:
README.bigb
``
= My website
See: \x[not-readme], or \x[not-readme-h2].
``
not-readme.bigb
``
= Not readme
== Not readme h2
``
`README.bigb` would render something like:
``
See: not readme, or not readme.
``
The following would fail and point you out the file and line of the failure:
* nonexistent id:
``
\x[id-that-does-not-exist]
``
* duplicate IDs:
``
= My h1
== My h2
== My h2
``
* https://katex.org/[KaTeX] server side \x[mathematics], works on browsers with JavaScript disabled:
``
I like $\sqrt{2}$, but I adore this \x[equation-quadratic-equation]:
$$
x^2 + 2x + 1
$$
{title=Quadratic equation}
``
* multi-file features out of the box so you don't need a separate wrapper like Jekyll to make a multi-page website:
* \x[internal-cross-file-reference]{p}
* single-source multi-format output based on \x[include]{p} and build options:
* by default, one HTML per source with includes rendered as links between pages, e.g.:
README.bigb
``
= My website
== h2
\Include[not-readme]
``
not-readme.bigb
``
= Not readme
== Not readme h2
``
produces `index.html` and `not-readme.html`
* with the \x[split-headers] option, you can output each header of an input file into a separate output file. The previous filesystem would produce:
* `index.html`: which contains the full `README.bigb` output
* `split.html`: split version of the above containing only the `= My website` header and not `h2`
* `h2.html`: only contains the `h2` header
* `not-readme.html` contains the full output of `not-readme.bigb`
* `not-readme-split.html`: only contains the `= Not readme` header
* `not-readme-h2.html`: only contains the `= Not readme h2` header
Each of those pages automatically gets a \x[table-of-contents]
* \x[embed-includes] single file output from multiple input files. Includes are parsed smartly, not just source copy pasted, e.g. included headers are shifted from `h1` to `h2` correctly.
On the previous sample filesystem, it would produce a single output file `index.html` which would contain a header structure like:
``
= My website
== h2
=== Not readme
==== Not readme h2
``
* supports both local serverless rendering to HTML files for local viewing, and server oriented rendering such as GitHub pages, e.g. \x[internal-cross-reference]{p} automatically get `.html` extension and or not. E.g.:
* locally, a link `\x[not-readme]` would render as `` and `not-readme.bigb` produces `not-readme.html`
* when publishing, `\x[not-readme]` would render as `` and `not-readme.bigb` also produces `not-readme.html`, which the server converts to just `http://my-website.com/not-readme`
* cross file configuration files to factor out common page parts like headers, footers and other metadata, e.g.:
* `main.liquid.html`: https://github.com/Shopify/liquid[Liquid template] used for all pages, see example at: \x[play-with-the-template]{full}
* `main.scss`: CSS stylesheet generated from https://sass-lang.com/[SASS] input, see example at: \x[play-with-the-template]{full}
* `ourbigbook.tex`: global LaTeX math definitions, e.g.:
``
\newcommand{\abs}[1]{\left|#1\right|}
``
and then you can use:
``
$\abs{x}$
``
in any .bigb file of the project.
* \x[ourbigbook-json]: per repository configuration options
* \x[table-of-contents] that crosses input files via includes. E.g. in:
README.bigb
``
= My website
== h2
\Include[not-readme]
``
not-readme.bigb
``
= Not readme
== Not readme h2
``
the table of contents for `index.html` also contains the headers for `not-readme.bigb` producing:
* My website
* h2
* Not readme
* Not readme h2
This means that you can split large \x[h-splitdefault-argument][splitDefault] input files if rendering starts to slow you down, and things will still render exactly the same.
* check that local files and images linked to actually exist: \x[a-check-argument]. E.g.:
``
\a[i-don-exist.txt]
``
would lead to a build error.
* associate headers to files with the \x[h-file-argument] e.g.:
``
Here's an example of a nice image: \x[file/path/to/my/image.png].
= path/to/my/image.png
{file}
This image was taken when I was on vacation!
``
would automatically add a preview of the image on the output.
* advanced header/ID related features:
* \x[id-based-header-levels]:
``
= Furry animal
I like \x[furry-animal]{p}, especially my cat, here is his photo: \x[image-my-cat].
== Cat
\Image[My_cat.jpg]
{title=My cat.}
``
* \x[scope]{p} either with directories or with within a single file:
``
See the important conclusion of my experiment: \x[report-of-my-experiment/conclusion]
= Report of my experiment
{scope}
== Introduction
== Middle
== Conclusion
``
* \x[cross-reference-title-inflection] for capitalization and pluralization, e.g.;
``
= Dog
== Snoopy
{c}
\x[dog]{c}{p} are fun. But the \x[dog] I like the most is \x[snoopy]!
``
would render:
* `\x[dog]{c}{p}` as `Dogs`: capitalized because of `{c}` and pluralized because of `{p}`
* `\x[dog]` as `dogs`: auto lowercased because its header `= Dog` does not have `{c}`
* `\x[snoopy]` as `Snoopy`: title capitalization kept to upper case due to `{c}` on the header `== Snoopy`
* \x[synonym]{p}, e.g.:
``
= User interface
= UI
{c}
{synonym}
{title2}
\x[user-interface]{c} is too long, I just say \x[ui].
``
would render something like:
``
User interface is too long, I just say UI
``
Furthermore, this also generates a output file:
``
ui.html
``
which redirects to the ain `user-interface.html`, so it serves as a way to have backward compatibility on page renames.
And the `title2` makes it appears on the main title under parenthesis, something like:
``
User interface (UI)
``
* \x[h-disambiguate-argument][header disambiguation], e.g.:
``
My favorite fruits are \x[apple-fruit]{p}!
My favorite least favorite brand is is \x[apple-company]! \x[apple] computers are too expensive.
== Apple
{disambiguate=fruit}
== Apple
{c}
{disambiguate=company}
= Apple
{c}
{synonym}
``
which renders something like:
* `\x[apple-fruit]{p}`: `apples`
* `\x[apple-company]`: `Apple`
* `\x[apple]`: also `Apple` because of the synonym
* `== Apple\n{disambiguate=fruit}`: `
Apple (fruit)
`* `== Apple\n{disambiguate=company}`: `
Apple (company)
`* tags are regular headers: \x[h-child-argument], \x[x-child-argument]
``
= Animal
== Dog
{tag=domestic}
{tag=cute}
== Cat
{tag=domestic}
{tag=cute}
== Bat
{tag=flying}
= Flying
= Cute
= Domestic
``
* \x[unlimited-header-levels], levels higher than 6 are rendered in HTML as an appropriately styled `div`s with an ID:
``
= h1
== h2
=== h3
==== h4
===== h5
====== h6
======= h7
======== h8
``
* generate lists of \x[incoming-links] between internal headers: it shows every internal link coming into the current page
* is written in JavaScript and therefore runs natively on the browser to allow live previews as shown at: https://cirosantilli.com/ourbigbook/editor[]
* helps you with the publishing:
* `ourbigbook --publish` publishes in a single command to the configured target (default GitHub pages): \x[publish]{full}
* OurBigBook tries to deal with media such as images and video intelligently for you, e.g.: \x[where-to-store-images]{full}. E.g. you can keep media in a separate media repository, `my-media-repository`, and then by configuring on \x[ourbigbook-json]:
``
"media-providers": {
"github": {
"default-for": ["image", "video"],
"path": "media",
"remote": "yourname/myproject-media"
}
}
``
you can use images in that repository with:
``
\Image[My_image_basename.jpg]
``
instead of:
``
\Image[https://raw.githubusercontent.com/cirosantilli/myproject--media/master/My_image_basename.jpg]
``
* `inotifywait` watch and automatically rebuild with \x[watch]:
``
ourbigbook --watch input-file.bigb
``
= Quick start
{parent=ourbigbook}
= Play with the template
{parent=quick-start}
Learn the syntax basics in 5 minutes: https://cirosantilli.com/ourbigbook/editor[].
Play with https://github.com/cirosantilli/ourbigbook-template[a OurBigBook template] locally:
``
git clone https://github.com/cirosantilli/ourbigbook-template
cd ourbigbook-template
npm install
npx ourbigbook .
firefox index.html
``
That template can be seen rendered live at: http://cirosantilli.com/ourbigbook-generate-multifile/ Other templates are documented at: \x[generate].
To \x[publish-to-github-pages] on your repository you can just fork the repository https://github.com/cirosantilli/ourbigbook-template to your own https://github.com/johndoe/ourbigbook-template and then:
``
git remote set-url origin git@github.com:johndoe/ourbigbook-template.git
npx ourbigbook --publish
``
and it should now be visible at: https://johndoe.github.io/ourbigbook-template
Then, every time you make a change you can publish the new version with:
``
git add .
git commit --message 'hacked stuff'
ourbigbook --publish .
``
or equivalently with the \x[publish-commit] shortcut:
``
ourbigbook --publish-commit 'hacked stuff'
``
If you want to publish to your root page https://johndoe.github.io instead of https://johndoe.github.io/ourbigbook-template you need to rename the `master` branch to `dev` as mentioned at \x[publish-to-github-pages-root-page]:
``
git remote set-url origin git@github.com:johndoe/johndoe.github.io.git
# Rename master to dev, and delete the old master.
git checkout -b dev
git push origin dev:dev
git branch -D master
git push --delete origin master
npx ourbigbook --publish
``
The following files of the template control the global style of the output, and you are free to edit them:
* `main.liquid.html`: global HTML template in https://shopify.github.io/liquid/[Liquid format]. Available variables are documented at \x[template], and it is being selected in that repository with :
``
"template": "main.liquid.html"
``
in the \x[ourbigbook-json] configuration file.
* `main.scss`: https://sass-lang.com/[Sass] file that gets converted to raw CSS `main.css` by `npx ourbigbook .`.
Sass is just much more convenient to write than raw CSS.
That file gets included into the global HTML template inside `main.liquid.html` at:
``
``
= Important command line options
{parent=quick-start}
When you run:
``
npx ourbigbook .
``
it converts all files in the current directory separately, e.g.:
* `README.bigb` to `index.html`, since `README` is a magic name that we want to show on the root URL
* `not-readme.bigb` to `not-readme.html`, as this one is a regular name unlike `README`
* `main.scss` to `main.css`
If one of the input files starts getting too large, usually the toplevel `README.bigb` in which you dump everything by default like Ciro does, you can speed up development and just compile files individually with either:
``
npx ourbigbook README.bigb
npx ourbigbook not-readme.bigb
``
Note however that when those individual files have a \x[internal-cross-file-reference] to something defined in `not-readme.bigb`, e.g. via `\x[h2-in-not-the-readme]`, then you must first do a first pass once with:
``
npx ourbigbook .
``
to parse all files and extract all necessary IDs to the \x[id-database]. That would be optimized slightly wit the \x[no-render] command line option:
``
npx ourbigbook --no-render .
``
to only extract the IDs but not render, which speeds things up considerably
When dealing with large files, you might also be interested in the following amazing options:
* \x[split-headers]
* \x[h-splitdefault-argument]
To produce a single standalone output file that contains everything in a directory run:
``
npx ourbigbook --embed-resources --embed-includes README.bigb
xdg-open index.html
``
You can now just give `index.html` to any reader and they should be able to view it offline without installing anything. The flags are:
* \x[embed-includes]: without this, `\Include[not-readme]` shows as a link to the file `not-readme.html` which comes from `not-readme.bigb` With the flag, `not-readme.bigb` output gets embedded into the output `index.html` directly
* \x[embed-resources]: by default, we link to CSS and JavaScript that lives inside `node_modules`. With this flag, that CSS and JavaScript is copied inline into the document instead. One day we will try to handle \x[image]{p} that way as well
= Useless knowledge
{parent=quick-start}
Install the NPM package globally and use it from the command line for a quick conversion:
``
npm install -g ourbigbook
printf 'ab\ncd\n' | ourbigbook --body-only
``
or to a file:
``
printf 'ab\ncd\n' | ourbigbook > tmp.html
``
You almost never want to do this except when \x[developing-ourbigbook], as it won't be clear what version of `ourbigbook` the document should be compiled with. Just be a good infant and use OurBigBook \x[play-with-the-template][with the template] that contains a `package.json` via `npx`, OK?
Furthermore, the default install of Chromium on Ubuntu 21.04 uses Snap and blocks access to dotfiles. For example, in a sane NVM install, our global CSS would live under `/home/ciro/.nvm/versions/node/v14.17.0/lib/node_modules/ourbigbook/dist/ourbigbook.css`, which gets blocked because of the `.nvm` part:
* https://forum.snapcraft.io/t/dot-files/7062
* https://bugs.launchpad.net/snapd/+bug/1607067
* https://superuser.com/questions/1546550/chromium-81-wont-display-dotfiles-anymore
* https://askubuntu.com/questions/1184357/why-cant-chromium-suddenly-access-any-partition-except-for-home
* https://askubuntu.com/questions/1214346/as-a-user-is-there-any-way-to-change-the-confinement-of-a-snap-package
One workaround is to use \x[embed-resources], but this of course generates larger outputs.
To run master globally from source for development see: \x[run-ourbigbook-master]{full}. This one actually works despite the dotfile thing since your development path is normally outside of dotfiles.
Try out the JavaScript API with \a[lib_hello.js]:
``
npm install ourbigbook
./lib_hello.js
``
= Design goals
{parent=ourbigbook}
OurBigBook is designed entirely to allow writing complex professional HTML and PDF scientific books, blogs, articles and encyclopedias.
OurBigBook aims to be the ultimate \x[latex-output-format][LaTeX] "killer", allowing books to be finally published as either HTML or PDF painlessly (LaTeX being only a backend to PDF generation).
It aims to be \x[features][more powerful] and \x[saner] and than Markdown and Asciidoctor.
= Saner
{parent=design-goals}
Originally, OurBigBook was is meant to be both saner and more powerful than Markdown and Asciidoctor.
But alas, as Ciro started implementing and using it, he started to bring some Markdown \x[insane-macro-shortcuts][insanity he missed back in].
And so this "degraded" slightly into a language slightly saner than Asciidoctor but with an amazing Node.js implementation that makes it better for book writing and website publishing.
Notably, we hope that our escaping will be a bit saner backslash escapes everything instead of Asciidoctor's "different escapes for every case" approach: https://github.com/asciidoctor/asciidoctor/issues/901
But hopefully, having starting from a saner point will still produce a saner end result, e.g. there are sane constructs for every insane one.
It is intended that this will be an acceptable downside as OurBigBook will be used primarily large complex content such as books rather than forum posts, and will therefore primarily written either:
* in text editors locally, where users have more features than in random browser textareas
* in a dedicated website that will revolutionize education, and therefore have a good JavaScript editing interface: https://github.com/cirosantilli/write-free-science-books-to-get-famous-website
For example, originally OurBigBook had exactly five magic characters, with similar functions as in LaTeX:
* `\` backslash to start a macro, like LaTeX
* `{` and `}`: left and right square brackets to delimit \x[positional-vs-named-arguments][optional macro arguments]
* `[` and `]`: left and right curly braces bracket to start an optional arguments
and double blank newlines for \x[paragraph]{p} if you are pedantic, but this later degenerated into many more with \x[insane-macro-shortcuts].
We would like to have only square brackets for both optional and mandatory to have even less magic characters, but that would make the language difficult to parse for computer and humans. LaTeX was right for once!
This produces a very regular syntax that is easy to learn, including doing:
* arbitrary nesting of elements
* adding arbitrary properties to elements
This sanity also makes the end tail learning curve of the endless edge cases found in Markdown and Asciidoctor disappear.
The language is designed to be philosophically isomorphic to HTML to:
* further reduce the learning curve
* ensure that most of HTML constructs can be reached, including arbitrary nesting
More precisely:
* macro names map to tag names, e.g.: `\\a` to `* one of the arguments of macros, maps to the content of the HTML element, and the others map to attributes.
E.g., in a link:
``\a[http://example.com][Link text\]``
the first macro argument:
``http://example.com``
maps to the `href` of ` ``Link text``
maps to the internal content of `Link text<>`.
= More powerful
{parent=design-goals}
The \x[saner][high sanity of OurBigBook], also makes creating new macro extensions extremely easy and intuitive.
All built-in language features use the exact same API as new extensions, which ensures that the extension API is sane forever.
Markdown is clearly missing many key features such as block attributes and \x[internal-cross-reference]{p}, and has no standardized extension mechanism.
The "more powerful than Asciidoctor" part is only partially true, since Asciidoctor is very featureful can do basically anything through extensions.
The difference is mostly that OurBigBook is completely and entirely focused on making amazing scientific books, and so will have key features for that application out-of-the box, notably:
* amazing header/ToC/ID features including proper error reports: never have a internal broken link or duplicate ID again
* \x[mathematics][server side pre-rendered maths with KaTeX]: all divs and spans are ready, browser only applies CSS, no JavaScript gets executed
* \x[publish]: we take care of website publishing for you out-of-the-box, no need to integrate into an external project like Jekyll
* \x[split-headers]:
* https://github.com/asciidoctor/asciidoctor/issues/626 feature request
* https://github.com/owenh000/asciidoctor-multipage third party plugin that does it
and we feel that some of those features have required specialized code that could not be easily implemented as a standalone macro.
Another advantage over Asciidoctor is that the reference implementation of OurBigBook is in JavaScript, and can therefore be used on browser live preview out of the box. Asciidoctor does Transpile to JS with https://github.com/opal/opal[Opal], but who wants to deal with that layer of complexity?
= Related projects
{parent=design-goals}
Static wiki generators: this is perhaps the best way of classifying this project :-)
* https://github.com/gollum/gollum[]: already has a local server editor! But no WYSIWYG nor live preview. Git integration by default, so when you save on the UI already generates a Git commit. We could achieve that with: https://github.com/isomorphic-git/isomorphic-git[], would be really nice. Does not appear to have built-in static generation:
* https://stackoverflow.com/questions/7210391/have-anyone-use-gollum-site-to-generete-markdown-wikis-and-host-it-on-heroku
* https://github.com/dreverri/gollum-site
Does not appear to check that any links are correct.
* https://github.com/wcchin/markypydia
* https://obsidian.md/ closed source, Markdown with \x[internal-cross-file-reference] + a SaaS. Appears to require payment for any publishing. 28k followers 2021: https://twitter.com/obsdmd[]. Founders are likely Canadians of Asian descent from Waterloo University: https://www.linkedin.com/in/lishid/ | https://www.linkedin.com/in/ericaxu/ also working in parallel on https://dynalist.io/ 2020 review at: https://www.youtube.com/watch?v=aK2fOQRNSxc Has offline editor with side-by-side preview. Compares with https://roamresearch.com/[Roam] and https://roamresearch.com/[Notion], but can't find any public publishing on those, seem to be enterprise only things.
Static book generators:
* https://github.com/rstudio/bookdown[], https://bookdown.org/[]. Very similar feature set to what we want!!! Transpiles to markdown, and then goes through Pandoc: https://bookdown.org/yihui/bookdown/pandoc.html[], thus will never run on browser without huge translation layers. But does have an obscene amount of output formats however.
* https://gohugo.io/[Hugo]. Pretty good, similar feature set to ours. But Go based, so hard on browser, and adds adhoc features on top of markdown once again
* https://en.wikipedia.org/wiki/Personal_wiki
* https://github.com/vimwiki/vimwiki
* https://github.com/hplgit/doconce
* https://www.gwern.net/About#source is pretty interesting, uses https://github.com/jaspervdj/Hakyll/ + some custom stuff.
* https://github.com/JerrySievert/bookmarkdown
* https://www.gitbook.com/
* https://github.com/rust-lang/mdBook[]. Impressive integrated search feature. Like Gitbook but implemented in Rust.
* https://github.com/facebook/docusaurus React + markdown based, written in TypeScript. So how can it be build fast? Gotta benchmark.
Less related but of interest, similar philosophy to what Ciro wants, but no explicitly reusable system:
* http://www.uprtcl.io/
* https://libretexts.org
* https://physics.info/
* https://hypertextbook.com/
* https://tutorial.math.lamar.edu/
= Motivation
{parent=design-goals}
Ciro Santilli developed OurBigBook to perfectly satisfy his writing style, which is basically "create one humongous document where you document everything you know about a subject so everyone can understand it, and just keep adding to it".
https://cirosantilli.com[] is the first major document that he has created in OurBigBook.
He decided to finally create this new system after having repeatedly facing limitations of Asciidoctor which were ignored/wontfixed upstream, because Ciro's writing style is not as common/targeted by Asciidoctor.
Following large documents Ciro worked extensively on:
* https://github.com/cirosantilli/china-dictatorship
* https://github.com/cirosantilli/linux-kernel-module-cheat
made the limitations of Asciidoctor clear to Ciro, and were major motivation in this work.
The key limitations have repeatedly annoyed Ciro were:
* cannot go over header level 6, addressed at: \x[unlimited-header-levels]
* the need for \x[split-headers] to avoid one too large HTML output that will never get indexed properly by search engines, and takes a few seconds to load on any browser, which is unacceptable user experience
= OurBigBook Markup
{c}
{parent=ourbigbook}
OurBigBook Markup is the markup language used in the \x[ourbigbook] project.
It works both on the \x[ourbigbook-web] dynamic website, and on \x[ourbigbook-cli] static websites from the command line.
= Macro
{parent=ourbigbook-markup}
This section documents all OurBigBook macros.
Macros are magic commands that do cool stuff, e.g. `\Image` to create an image.
The most common macros also have \x[insane-macro-shortcuts] to keep the syntax shorter.
The general macro syntax is described at \x[ourbigbook-markup-syntax]{full}.
= Link
{parent=macro}
= `\a`
{synonym}
{title2}
\x[insane-macro-shortcuts][Insane] autolink (link text is the same as the link):
\OurbigbookExample[[
The website http://example.com is cool. See also:
\Q[http://example.com/2]
]]
Exact parsing rules described at: \x[insane-link-parsing-rules]{full}.
Equivalent sane version:
\OurbigbookExample[[[
The website \a[http://example.com] is cool.
\Q[\a[http://example.com/2]]
]]]
Insane link with custom text:
\OurbigbookExample[[
The website http://example.com[example.com] is cool.
]]
Equivalent sane version:
\OurbigbookExample[[
The website \a[http://example.com][example.com] is cool.
]]
If the custom text is empty, an autolink is generated. This is often useful if you want your link to be followed by punctuation:
\OurbigbookExample[[
The website is really cool: http://example.com[].
]]
This could also be achieved with the sane syntax of course, but this pattern saves a tiny bit of typing.
Link with multiple paragraphs inside it:
\OurbigbookExample[[
\a[http://example.com][Multiple
paragraphs]
]]
= `\a` `check` argument
{parent=link}
If `{check=0}` is given, this disables the local file existence check that is done by default for \x[relative-link]{p}.
If `check` is not given explicitly, OurBigBook does the check by default if the link is not a \x[url-with-protocol] For example, the following are not checked:
* `http://cirosantilli.com`
* `https://cirosantilli.com`
* `file:///etc/fstab`
* `ftp://cirosantilli.com`
and the following are:
* `index.js`
* `../index.js`
* `path/to/index.js`
* `/path/to/index.js`. Since it stats with a `/`, this path has to exist relative to \x[project-toplevel-directory], not relative to the current `.bigb` source.
* `//example.com/path/to/index.js`. This gets treated the same as `/example.com/path/to/index.js`.
For example, it is often the case in computer programming tutorials that we want to refer to source files in the current directory, so you could have in your `README.bigb`:
``
See this awesome source file: \a[index.js]
``
and then if `index.js` does not exist in the project, compilation leads to an error.
This check is important because as you start documenting several source files, it is almost inevitable getting wrong paths due to renames or typos without this type of error checking
The most common use case for disabling checks is as follows.
https://github.com/cirosantilli/cirosantilli.github.io/blob/da296c3c933704484d5cd1a42754f60ec00f672b/README.bigb was rendered at https://cirosantilli.com[], and contains links to https://cirosantilli.com/markdown-style-guide[], whose source lives in a separate non-OurBigBook repository: https://github.com/cirosantilli/markdown-style-guide/
Therefore, if we did from `README.bigb`:
``
See this awesome style guide: \a[markdown-style-guide]
``
we do not want OurBigBook do check that the file `markdown-style-guide` exists in the local filesystem.
For this reason, we have instead to write:
``
See this awesome style guide: \a[markdown-style-guide]{check=0}
``
The general lesson is clear: in a server, paths could be re-routed to anything, including content that lies outside of a OurBigBook project.
= `\a` `href` argument
{parent=link}
The link target, e.g. in:
``
\a[http://example.com]
``
`href` equals `http://example.com`.
Important behaviours associated with this property for local links:
* they are checked for existence in the local filesystem: \x[a-check-argument]
* they are modified as per \x[relative-link]{full} to account for \x[scope]{p} with \x[split-headers]
= `\a` `ref` argument
{parent=link}
Analogous to the \x[x-ref-argument], e.g.:
\OurbigbookExample[[
Trump said this and that.https://en.wikipedia.org/wiki/Donald_Trump_Access_Hollywood_tape#Trump's_responses{ref}https://web.archive.org/web/20161007210105/https://www.donaldjtrump.com/press-releases/statement-from-donald-j.-trump{ref} Then he said that and this.https://en.wikipedia.org/wiki/Donald_Trump_Access_Hollywood_tape#Trump's_responses{ref}https://web.archive.org/web/20161007210105/https://www.donaldjtrump.com/press-releases/statement-from-donald-j.-trump{ref}
]]
= `\a` `relative` argument
{parent=link}
If given, this \x[boolean-argument] forces a given link to be a \x[relative-link] or not.
Otherwise, the link is automatically guessed based on the address given as explained at \x[relative-link]{full}.
= Relative link
{parent=a-relative-argument}
A relative link is a link that points to a resource that will be present a final URL relative to the input `.bigb` file.
For example, it is often the case in computer programming tutorials that we want to refer to source files in the current directory.
So from our `README.bigb`, we could want to write something like:
\OurbigbookExample[[
Have a look at this amazing source file: \a[index.js].
]]
and here `\a[ourbigbook]` is a relative link.
A non-relative link is something like:
\OurbigbookExample[[
This is great website: https://cirosantilli.com
]]
which points to an absolute URL.
A link being relative has the following effects
* the correct relative path to the file is used when using nested \x[scope]{p} with \x[split-headers]. For example, if we have:
``
= h1
== h2
{scope}
=== h3
\a[index.js]
``
then in split header mode, `h3` will be rendered to `h2/h3.html`.
Therefore, if we didn't do anything about it, the link to `index.js` would render as `href="index.js"` and thus point to `h2/index.js` instead of the correct `index.js`.
Instead, OurBigBook automatically converts it to the correct `href="../index.js"`
OurBigBook considers a link relative by default if:
* it is not a \x[url-with-protocol]
* it does not start with a slash `/`
Therefore, the following are not relative links by default:
* `http://cirosantilli.com`
* `https://cirosantilli.com`
* `file:///etc/fstab`
* `ftp://cirosantilli.com`
* `/path/to/index.js`
* `//example.com/path/to/index.js`
and the following are:
* `index.js`
* `../index.js`
* `path/to/index.js`
Implemented at: https://github.com/cirosantilli/ourbigbook/issues/87[].
= URL with protocol
{c}
{parent=relative-link}
A URL with protocol is a URL that matches the regular expression `^[a-zA-Z]+://`. The following are examples of URLs with protocol:
* `http://cirosantilli.com`
* `https://cirosantilli.com`
* `file:///etc/fstab`
* `ftp://cirosantilli.com`
The following aren't:
* `index.js`
* `../index.js`
* `path/to/index.js`
* `/path/to/index.js`
* `//example.com/path/to/index.js`. This one is a bit tricky. Web browsers would consider this as a https://stackoverflow.com/questions/28446314/why-use-protocol-relative-urls-at-all[protocol-relative URL], which technically implies a protocol, although that protocol would be different depending how you are viewing the file, e.g. locally through `file://` vs on a with website `https://`.
For simplicity's sake, we just consider it as a URL without protocol.
= Insane link parsing rules
{parent=link}
Insane start at any of the recognized protocols are the ones shown at: \x[known-url-protocols]{full}.
* `http://`
* `https://`
absolutely anywhere if not escaped, e.g.:
``
ahttp://example.com
``
renders something like:
``
a
``
To prevent expansion, you have to escape the protocol with a backslash `\\`, e.g.:
``
\http://example.com
``
Empty domains like:
``
http://
``
don't becomes links however. But this one does:
``
http://a
``
Insane links end when either the end of the document or one of the following characters is found:
* space ` `
* newline `\n`
* open or close square bracket `[` or `]`
* open or close curly braces `{` or `}`
As a consequence, to have an insane link followed immediately by a punctuation like a period you should use an empty argument as in:
\OurbigbookExample[[
Check out this website: http://example.com[].
]]
otherwise the punctuation will go in it. Another common use case is:
\OurbigbookExample[[
As mentioned on the tutorial (http://example.com[see this link]).
]]
If you want your link to include one of the terminating characters, e.g. `]`, all characters can be escaped with a backslash, e.g.:
\OurbigbookExample[[
Hello http://example.com/\]a\}b\\c\ d world.
]]
Note that the `http://example.com` inside `\a[http://example.com]` only works because we do some post-processing magic that prevents its expansion, otherwise the link would expand twice:
\OurbigbookExample[[
\P[http://example.com]
\a[http://example.com]
]]
This magic can be observed with \x[help-macros] by seeing that the `href` argument of the `a` macro has the property:
``
"elide_link_only": true,
``
= Bold
{parent=macro}
{title2=`\b`}
\OurbigbookExample[[
Some \b[bold] text.
]]
= Line break
{parent=macro}
{title2=`\br`}
There is basically one application for this: poetry, which would be too ugly with \x[code-block] due to fixed width font:
\OurbigbookExample[[
Paragraph 1 Line 1\br
Paragraph 1 Line 2\br
Paragraph 2 Line 1\br
Paragraph 2 Line 2\br
]]
= Code block
{parent=macro}
{title2=\c[[``]], \c[[`]], `\C`, `\c`}
Inline code (code that should appear in the middle of a paragraph rather than on its own line) is done with a single backtick (\c[[`]]) \x[insane-macro-shortcuts][insane macro shortcut]:
\OurbigbookExample[[
My inline `x = 'hello\n'` is awesome.
]]
and block code (code that should appear on their own line) is done with two or more backticks (\c[[``]]):
\OurbigbookExample[[
``
f() {
return 'hello\n';
}
``
]]
The sane version of inline code is a lower case `c`:
\OurbigbookExample[[[
My inline \c[[x = 'hello\n']] is awesome.
]]]
and the sane version of block math is with an upper case `C`:
\OurbigbookExample[[[
\C[[
f() {
return 'hello\n';
}
]]
]]]
The capital vs lower case theme is also used in other elements, see: \x[block-vs-inline-macros].
If the content of the sane code block has many characters that you would need to \x[escape-characters][escape], you will often want to use \x[literal-arguments], which work just like the do for any other argument. For example:
\OurbigbookExample[[[[
\C[[[
A paragraph.
\C[[
And now, some long, long code, with lots
of chars that you would need to escape:
\ [ ] { }
]]
A paragraph.
]]]
]]]]
Note that the initial newline is skipped automatically in code blocks, just as for any other element, due to: \x[argument-leading-newline-removal], so you don't have to worry about it.
The distinction between inline `\c` and block `\C` code blocks is needed because in HTML, https://stackoverflow.com/questions/5371787/can-i-have-a-pre-tag-inside-a-p-tag-in-tumblr/58603596#58603596[`pre` cannot go inside `P`].
We could have chosen to do some magic to differentiate between them, e.g. checking if the block is the only element in a paragraph, but we decided not to do that to keep the language saner.
And now a code block outside of \x[ourbigbookexample] to test how it looks directly under \x[toplevel]:
``
Hello
Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello
HelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHelloHello
Hello
``
And nos a very long inline code: `Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello Hello`
\Comment[[[[
TODO implement.
We can have cross references to code blocks as for other elements such as \x[image]{p}:
\OurbigbookExample[[[
See this awesome code \x[my-code]:
``
ab
cd
``
{id=my-code}
]]]
]]]]
Block code can have a title, e.g. see this one: \x[code-my-nice-code]{full}:
\OurbigbookExample[[
``
ab
cd
``
{id=code-my-nice-code}
{title=My nice code block.}
]]
= `\OurbigbookExample`
{parent=macro}
Shows both the OurBigBook code and its rendered output, e.g.:
\OurbigbookExample[[[
\OurbigbookExample[[
Some `ineline` code.
]]
]]]
Its input should be thought of as a literal code string, and it then injects the rendered output in the document.
This macro is used extensively in the OurBigBook documentation.
= Comment
{parent=macro}
{title2=`\Comment`}
The `Comment` and `comment` macros are regular macros that does not produce any output. Capitalization is explained at: \x[block-vs-inline-macros]{full}.
You will therefore mostly want to use it with a \x[literal-arguments][literal argument], which will, as for any other macro, ignore any macros inside of it.
\OurbigbookExample[[[
Before comment.
\Comment[[
Inside comment.
]]
After comment.
]]]
And an inline one:
\OurbigbookExample[[[
My inline \comment[[inside comment]] is awesome.
\comment[[inside comment]] inline at the start.
]]]
= Header
{parent=macro}
{title2=`\H`}
\x[insane-macro-shortcuts][Insane] with `= ` (equal sign space):
``
= My h1
== My h2
=== My h3
``
Insane headers end at the first newline found. They cannot therefore contain raw newline tokens.
Equivalent sane:
``
\H[1][My h1]
\H[2][My h2]
\H[3][My h3]
``
Custom ID for \x[internal-cross-reference]{p} on insane headers:
``
= My h1
{id=h1}
== My h2
{id=h2}
=== My h3
{id=h3}
``
Sane equivalent:
``
\H[1][My h1]{id=h1}
\H[2][My h2]{id=h2}
\H[3][My h3]{id=h3}
``
= Unlimited header levels
{parent=header}
There is no limit to how many levels we can have, for either sane or insane headers!
HTML is randomly limited to `h6`, so OurBigBook just renders higher levels as an `h6` with a `data-level` attribute to indicate the actual level for possible CSS styling:
``
My title
``
The recommended style is to use insane headers up to `h6`, and then move to sane one for higher levels though, otherwise it becomes very hard to count the `=` signs.
To avoid this, we considered making the insane syntax be instead:
``
= 1 My h1
= 2 My h2
= 3 My h3
``
but it just didn't feel as good, and is a bit harder to type than just smashing `=` n times for lower levels, which is the most common use case. So we just copied markdown.
= My h3
{parent=unlimited-header-levels}
= My h4
{parent=my-h3}
= My h5
{parent=my-h4}
= My h6
{parent=my-h5}
= My h7
{parent=my-h6}
= My h8
{parent=my-h7}
= My h9
{parent=my-h8}
= My h10
{parent=my-h9}
= My h11
{parent=my-h10}
= My h12
{parent=my-h11}
= My h13
{parent=my-h12}
= Skipping header levels
{parent=header}
The very first header of a document can be of any level, although we highly recommend your document to start with a `\H[1]`, and to contain exactly just one `\H[1]`, as this has implications such as:
* `\H[1]` is used for the document title: \x[html-document-title]
* `\H[1]` does not show on the \x[table-of-contents]
After the initial header however, you must not skip a header level, e.g. the following would give an error because it skips level 3:
``
= my 1
== my 1
==== my 4
``
= The toplevel header
{parent=header}
If the document has only a single header of the highest level, e.g. like the following has only a single `h2`:
``
== My 2
=== My 3 1
=== My 3 2
``
then this has some magical effects.
= The toplevel header IDs don't show
{parent=the-toplevel-header}
Header IDs won't show for the toplevel level. For example, the headers would render like:
``
My 2
1. My 3 1
2. My 3 2
``
rather than:
``
1. My 2
1.2. My 3 1
1.2. My 3 2
``
This is because in this case, we guess that the `h2` is the toplevel.
= The ID of the first header is derived from the filename
{parent=the-toplevel-header}
TODO: we kind of wanted this to be the ID of the toplevel header instead of the first header, but this would require an extra postprocessing pass (to determine if the first header is toplevel or not), which might affect performance, so we are not doing it right now.
When the OurBigBook input comes from a file (and not e.g. stdin), the default ID of the first header in the document is derived from the basename of the OurBigBook input source file rather than from its title.
This is specially relevant when \x[include][including] other files.
For example, in file named `my-file.bigb` which contains:
``
= Awesome ourbigbook file
]]
``
the ID of the header is `my-file` rather than `awesome-ourbigbook-file`. See also: \x[automatic-id-from-title].
If the file is an \x[index-file] other than \x[the-toplevel-index-file], then the basename of the parent directory is used instead, e.g. the toplevel ID of a file:
``my-subdir/README.bigb``
would be:
``#my-subdir``
rather than:
``#README.bigb``
For the toplevel index file however, the ID is just taken from the header itself as usual. This is done because you often can't general control the directory name of a project.
For example, a \x[publish-to-github-pages][GitHub pages] root directory must be named as `.github.io`. And users may need to rename directories to avoid naming conflicts.
As a consequence of this, the toplevel index file cannot \x[include][be included in other files].
= `\H` arguments
{parent=header}
= `\H` `c` argument
{parent=h-arguments}
If given, makes the header capitalized by default on \x[internal-cross-file-reference]{p}.
More details at: \x[cross-reference-title-inflection]{full}.
= `\H` `child` argument
{parent=h-arguments}
This \x[multiple-argument] marks given IDs as being children of the current page.
The effect is the same as adding the \x[x-child-argument] argument to an under the header. Notably, such marked target IDs will show up on the \x[tagged] autogenerated \x[header-metadata-section].
Example:
``
= Animal
== Mammal
=== Bat
=== Cat
== Wasp
== Flying animal
{child=bat}
{child=wasp}
\x[bat]
\x[wasp]
``
renders exactly as:
``
= Animal
== Mammal
=== Bat
=== Cat
== Wasp
== Flying animal
\x[bat]{child}
\x[wasp]{child}
``
The header `child` syntax is generally preferred because at some point while editing the content of the header, you might accidentally remove mentions to e.g. `\x[bat]{child}`, and then the relationship would be lost.
The \x[h-tag-argument] does the same as the \x[x-child-argument] but in the opposite direction.
= `\H` `disambiguate` argument
{parent=h-arguments}
Sometimes the short version of a name is ambiguous, and you need to add some extra text to make both its title and ID unique.
For example, the word "Python" could either refer to:
* the programming language: https://en.wikipedia.org/wiki/Python_(programming_language)
* the genus of snakes: https://en.wikipedia.org/wiki/Python_(genus)
The `disambiguate` \x[positional-vs-named-arguments][name argument], which is automatically defined for all macros, helps you deal more neatly with such problems.
Have a look at this example:
``
My favorite snakes are \x[python-genus]{p}!
My favorite programming language is \x[python-programming-language]!
\x[python-genus]{full}
\x[python-programming-language]{full}
== Python
{disambiguate=genus}
== Python
{c}
{disambiguate=programming language}
{title2=.py}
{wiki}
``
which renders as:
\Q[
My favorite snakes are \x[python-genus]{p}!
My favorite programming language is \x[python-programming-language]!
\x[python-genus]{full}
\x[python-programming-language]{full}
= Python
{disambiguate=genus}
{parent=h-disambiguate-argument}
= Python
{c}
{disambiguate=programming language}
{parent=h-disambiguate-argument}
{title2=.py}
{wiki}
]
from which we observe how `disambiguate`:
* gets added to the ID after conversion following the same rules as \x[automatic-id-from-title]
* shows up on the header between parenthesis, much like Wikipedia, as well as in \x[x-full-argument][`full` cross references]
* does not show up on non-`full` references. This makes it much more likely that you will be able to reuse the title automatically on a cross reference without `content`: we wouldn't want to say "My favorite programming language is Python (programming language)" all the time, would we?
* gets added to the default \x[h-wiki-argument] inside parenthesis, following Wikipedia convention, therefore increasing the likelihood that you will be able to go with the default Wikipedia value
Besides disambiguating headers, the `disambiguate` argument has a second related application: disambiguating IDs of images. For example:
\OurbigbookExample[[
\x[image-the-title-of-my-disambiguate-image]{full=0}
\x[image-the-title-of-my-disambiguate-image-2]{full=0}
\x[image-the-title-of-my-disambiguate-image]{full}
\x[image-the-title-of-my-disambiguate-image-2]{full}
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=The title of my disambiguate image.}
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=The title of my disambiguate image.}
{disambiguate=2}
]]
Note that unlike for headers, `disambiguate` does not appear on the title of images at all. It serves only to create an unique ID that can be later referred to. Headers are actually the only case where `disambiguate` shows up on the visible rendered output. We intend on making this application obsolete however with:
This use case is even more useful when `title-from-src` is enable by default for the \x[ourbigbook-json/media-providers] entry, so you don't have to repeat titles several times over and over.
= `\H` `file` argument
{parent=h-arguments}
If given, the current section contains metadata about file or other resource with the given URL.
If empty, the URl of the file is extracted directly from the header. Otherwise, the given URL is used.
for example:
``
The file \x[file/path/to/myfile.c] is very useful.
= path/to/myfile.c
{file}
An explanation of what this file is about.
``
renders the same as:
``
The file \x[file/path/to/myfile.c] is very useful.
= path/to/myfile.c
{file}
{id=file/path/to/myfile.c}
An explanation of what this file is about.
\a[path/to/myfile.c][View file]
``
so note how:
* \x[automatic-id-from-title] does not normalize the path, e.g. it does not convert `/` to `-`.
Also, a `file/` prefix is automatically added to the ID. This is needed with \x[split-headers] to avoid a collision between:
* `path/to/myfile.c`: the actual file
* `file/path/to/myfile.c`: the metadata about that file. Note that locally the `.html` extension is added as in `file/path/to/myfile.c.html` which avoids the collision. But on a server deployment, the `.html` is not present, and there would be a conflict if we didn't add that `file/` prefix.
* a link to the is added automatically, since users won't be able to click it from the header, as clicking on the header will just link to the header itself
* a preview is added. The type of preview is chosen as follows:
* if the URL has an image extension, do an \x[image] preview
* otherwise if the URL has a video extension, or is a YouTube URL, do a \x[video] preview
* otherwise, don't show a preview, as we don't know anything sensible to show
In some cases however, especially when dealing with external URLs, we might want to have a more human readable title with a `file` specified as in:
``
The video \x[tank-man-by-cnn-1989] is very useful.
= Tank Man by CNN (1989)
{file=https://www.youtube.com/watch?v=YeFzeNAHEhU}
An explanation of what this video is about.
``
which renders something like:
``
The video \x[tank-man-by-cnn-1989] is very useful.
= Tank Man by CNN (1989)
\a[https://www.youtube.com/watch?v=YeFzeNAHEhU][View file]
An explanation of what this video is about.
\Video[https://www.youtube.com/watch?v=YeFzeNAHEhU]
``
= `\H` `file` argument demo
{parent=h-file-argument}
= Tank_man_standing_in_front_of_some_tanks.jpg
{file}
{parent=h-file-argument-demo}
An explanation of what this video is about.
= Tank Man by CNN (1989)
{file=https://www.youtube.com/watch?v=YeFzeNAHEhU}
{parent=h-file-argument-demo}
An explanation of what this video is about.
= `\H` `numbered` argument
{c}
{parent=h-arguments}
This \x[boolean-argument] determines whether renderings of a header will have section numbers or not. This affects all of:
* \x[header]{p} themselves
* \x[table-of-contents] links
* \x[internal-cross-reference]{p} with the \x[x-full-argument]
This option can be set by default for all files with:
By default, headers are numbered as in a book, e.g.:
``
= h1
== h2
=== h3
==== h4
``
renders something like:
``
= h1
Table of contents
* 1. h2
* 1.1. h3
* 1.1.1. h4
== 1. h2
=== 1.1. h3
==== 1.1.1. h4
``
However, for documents with a very large number of sections, or \x[unlimited-header-levels][deeply nested headers] those numbers start to be more noise than anything else, especially in the table of contents and you are better off just referring to IDs. E.g. imagine:
``
1.3.1.4.5.1345.3.2.1. Some deep level
``
When documents reach this type of scope, you can disable numbering with the `numbered` option.
This option can be set on any header, and it is inherited by all descendants.
The option only affects descendants.
E.g., if in the above example turn numbering off at `h2`:
``
= h1
== h2
{numbered=0}
=== h3
==== h4
``
then it renders something like:
``
= h1
Table of contents
* 1. h2
* h3
* h4
== 1. h2
=== h3
==== h4
``
The more common usage pattern to disable it on toplevel and enable it only for specific "tutorial-like sections". An example can be seen at:
* https://cirosantilli.com/[]: huge toplevel wiki, for which we don't want numbers
* https://cirosantilli.com/x86-paging[]: a specific tutorial, for which we want numbers
which is something like:
``
= Huge toplevel wiki
{numbered=0}
== h2
=== A specific tutorial
{numbered}
{scope}
==== h4
===== h5
``
then it renders something like:
``
= Huge toplevel wiki
Table of contents
* h2
* A specific tutorial
* 1. h4
* 1.1. h5
== h2
=== A specific tutorial
==== 1. h4
===== 1.1. h5
``
Note how in this case the number for `h4` is just `1.` rather than `1.1.1.`. We only show numberings relative to the first non-numbered header, because the `1.1.` wouldn't be very meaningful otherwise.
= `\H` `parent` argument
{c}
{parent=h-arguments}
= ID-based header levels
{c}
{synonym}
{title2}
In addition to the basic way of specifying header levels with an explicit level number as mentioned at \x[header]{full}, OurBigBook also supports a more indirect ID-based mechanism with the `parent` argument of the `\H` element.
We hightly recommend using `parent` for all but the most trivial documents.
For example, the following fixed level syntax:
``
= My h1
== My h2 1
== My h2 2
=== My h3 2 1
``
is equivalent to the following ID-based version:
``
= My h1
= My h2 1
{parent=my-h1}
= My h2 2
{parent=my-h1}
= My h3 2 1
{parent=my-h2-h}
``
The main advantages of this syntax are felt when you have a huge document with \x[unlimited-header-levels][very large header depths]. In that case:
* it becomes easy to get levels wrong with so many large level numbers to deal with. It is much harder to get an ID wrong.
* when you want to move headers around to improve organization, things are quite painful without a refactoring tool (which we intend to provide in the \x[browser-editor-with-preview]), as you need to fix up the levels of every single header.
If you are using the ID-based syntax however, you only have to move the chunk of headers, and change the `parent` argument of a single top-level header being moved.
Note that when the `parent=` argument is given, the header level must be `1`, otherwise OurBigBook assumes that something is weird and gives an error. E.g. the following gives an error:
``
= My h1
== My h2
{parent=my-h1}
``
because the second header has level `2` instead of the required `= My h2`.
When \x[scope]{p} are involved, the rules are the same as those of internal reference resolution, including the leading `/` to break out of the scope in case of conflicts.
See also: \x[header-explicit-levels-vs-nesting-design-choice]{full} for further rationale.
= ID-based header levels and scope resolution
{c}
{parent=h-parent-argument}
When mixing both \x[h-parent-argument] and \x[scope]{p}, things get a bit complicated, because when writing or parsing, we have to first determine the parent header before resolving scopes.
As a result, the follow simple rules are used:
* start from the last header of the highest level
* check if the `{parent=XXX}` is a suffix of its ID
* if not, proceed to the next smaller level, and so on, until a suffix is found
Following those rules for example, a file `tmp.bigb`:
``
= h1
{scope}
= h1 1
{parent=h1}
{scope}
= h1 1 1
{parent=h1-1}
= h1 1 2
{parent=h1-1}
= h1 1 3
{parent=h1/h1-1}
= h1 2
{parent=h1}
{scope}
= h1 2 1
{parent=h1-2}
{scope}
= h1 2 1 1
{parent=h1-2/h1-2-1}
``
will lead to the following header tree with \x[log-headers]:
``
= h1 tmp
== h2 1 tmp/h1-1
=== h3 1.1 tmp/h1-1/h1-1-1
=== h3 1.2 tmp/h1-1/h1-1-2
=== h3 1.3 tmp/h1-1/h1-1-3
== h2 2 tmp/h1-2
=== h3 2.1 tmp/h1-2/h1-2-1
==== h4 2.1.1 tmp/h1-2/h1-2-1/h1-2-1-1
``
= Header explicit levels vs nesting design choice
{parent=h-parent-argument}
Arguably, the language would be even saner if we did:
``
\H[My h1][
Paragraph.
\H[My h2][]
]
``
rather than having explicit levels as in `\H[1][My h1]` and so on.
But we chose not to do it like most markups available because it leads to too many nesting levels, and hard to determine where you are without tooling.
Ciro later "invented" (?) \x[h-parent-argument], which he feels reaches the perfect balance between the advantages of those two options.
= `\H` `scope` argument
{parent=h-arguments}
= Scope
{synonym}
In some use cases, the sections under a section describe inseparable parts of something.
For example, when documenting an experiment you executed, you will generally want an "Introduction", then a "Materials" section, and then a "Results" section for every experiment.
On their own, those sections don't make much sense: they are always referred to in the context of the given experiment.
The problem is then how to get unique IDs for those sections.
One solution, would be to manually add the experiment ID as prefix to every subsection, as in:
``
= Experiments
See: \x[full-and-unique-experiment-name/materials]
== Introduction
== Full and unique experiment name
=== Introduction
{id=full-and-unique-experiment-name/introduction}
See our awesome results: \x[full-and-unique-experiment-name/results]
For a more general introduction to all experiments, see: \x[introduction].
=== Materials
{id=full-and-unique-experiment-name/materials}
=== Results
{id=full-and-unique-experiment-name/results}
``
but this would be very tedious.
To keep those IDs shorter, OurBigBook provides the `scope` \x[boolean-argument] property of \x[header]{p}, which works analogously to C++ namespaces with the header IDs.
Using `scope`, the previous example could be written more succinctly as:
``
= Experiments
See: \x[full-and-unique-experiment-name/materials]
== Introduction
== Full and unique experiment name
{scope}
=== Introduction
See our awesome results: \x[results]
For a more general introduction to all experiments, see: \x[/introduction].
=== Materials
=== Results
``
Note how:
* full IDs are automatically prefixed by the parent scopes prefixed and joined with a slash `/`
* we can refer to other IDs withing the current scope without duplicating the scope. E.g. `\x[results]` in the example already refers to the ID `full-and-unique-experiment-name/materials`
* to refer to an ID outside of the scope and avoid name conflicts with IDs inside of the current scope, we start a reference with a slash `/`
So in the example above, `\x[/introduction]` refers to the ID `introduction`, and not `full-and-unique-experiment-name/introduction`.
= `scope` resolution
{parent=h-scope-argument}
When nested scopes are involved, \x[internal-cross-reference]{p} resolution peels off the scopes one by one trying to find the closes match, e.g. the following works as expected:
``
= h1
{scope}
== h2
{scope}
=== h3
{scope}
\x[h2]
``
Here OurBigBook:
* first tries to loop for an `h1/h2/h3/h2`, since `h1/h2/h3` is the current scope, but that ID does not exist
* so it removes the `h3` from the current scope, and looks for `h1/h2/h2`, which is still not found
* then it removes the `h2`, leading to `h1/h2`, and that one is found, and therefore is taken
= Directory-based `scope`
{parent=h-scope-argument}
Putting files in subdirectories of the build has the same effect as adding a \x[scope] to their top level header.
Notably, all headers inside that directory get the directory prepended to their IDs.
The toplevel directory is determined as described at: \x[the-toplevel-index-file].
= Test scope 1
{parent=h-scope-argument}
{scope}
For fun and profit.
= Test scope 2
{parent=test-scope-1}
{scope}
Let's break this local link: \a[ourbigbook].
= Not scoped
{parent=test-scope-2}
= `\H` `scope` argument of toplevel headers
{parent=h-scope-argument}
When \x[the-toplevel-header] is given the `scope` property OurBigBook automatically uses the file path for the scope and heaves fragments untouched.
For example, suppose that file `full-and-unique-experiment-name` contains:
``
= Full and unique experiment name
{scope}
== Introduction
== Materials
``
In this case, multi-file output will generate a file called `full-and-unique-experiment-name.html`, and the URL of the subsections will be just:
* `full-and-unique-experiment-name.html#introduction`
* `full-and-unique-experiment-name.html#materials`
instead of
* `full-and-unique-experiment-name.html#full-and-unique-experiment-name/introduction`
* `full-and-unique-experiment-name.html#full-and-unique-experiment-name/materials`
Some quick interactive cross file link tests:
* \x[not-readme-with-scope]
* \x[not-readme-with-scope/h2]
* \x[not-readme-with-scope/image-my-image]
= `\H` `splitDefault` argument
{parent=h-arguments}
When using \x[split-headers], \x[internal-cross-reference]{p} always point to non-split pages as mentioned at \x[internal-cross-reference-targets-in-split-headers].
If the `splitDefault` \x[boolean-argument] is given however:
* the split header becomes the default, e.g. `index.html` is now the split one, and `nosplit.html` is the non-split one
* the header it is given for, and all of its descendant headers will use the split header as the default internal cross target, unless the header is already rendered in the current page. This does not propagate across \x[include]{p} however.
For example, consider `README.bigb`:
``
= Toplevel
{splitDefault}
\x[h2][toplevel to h2]
\x[notreadme][toplevel to notreadme]
\Include[notreadme]
== h2
``
and `notreadme.bigb`:
``
= Notreadme
\x[h2][notreadme to h2]
\x[notreadme][notreadme to notreadme h2]
== Notreadme h2
``
Then the following links would be generated:
* `index.html`: split version of `README.bigb`, i.e. does not contain `h2`
* `toplevel to h2`: `h2.html`. Links to the split version of `h2`, since `h2` is also affected by the `splitDefault` of its parent, and therefore links to it use the split version by default
* `toplevel to notreadme`: `notreadme.html`. Links to non-split version of `notreadme.html` since that header is not `splitDefault`, because `splitDefault` does not propagate across includes
* `nosplit.html` non-split version of `README.bigb`, i.e. contains `h2`
* `toplevel to h2`: `#h2`, because even though `h2` is `splitDefault`, that header is already present in the current page, so it would be pointless to reload the split one
* `toplevel to notreadme`: `notreadme.html`
* `h2.html` split version of `h2` from `README.bigb`
* `notreadme.html`: non-split version of `notreadme.bigb`
* `notreadme to h2`: `h2.html`, because `h2` is `splitDefault`
* `notreadme to notreadme h2`: `#notreadme-h2`
* `notreadme-split.html`: split version of `notreadme.bigb`
* `notreadme to h2`: `h2.html`, because `h2` is `splitDefault`
* `notreadme to notreadme h2`: `notreadme.html#notreadme-h2`, because `notreadme-h2` is not `splitDefault`
The major application of this is that Ciro likes to work with a huge `README.bigb` containing thousands of random small topics. This is the case for example for: https://cirosantilli.com
And splitting those into separate source files would be quite laborious, as it would require duplicating IDs on the filename, and setting up \x[include]{p}.
However, after this README reaches a certain size, page loads start becoming annoyingly slow, even despite already loading large assets like \x[image]{p} video \x[video]{p} only on hover or click: the annoying slowness comes from the loading of the HTML itself before the browser can jump to the ID.
And even worse: this README corresponds to the main index page of the website, which will make what a large number of users will see be that slowness.
Therefore, once this README reaches a certain size, you can add the `splitDefault` attribute to it, to make things smoother for readers.
And if you have a smaller, more self-contained, and highly valuable tutorial such as https://cirosantilli.com/x86-paging[], you can just split that into a separate `.bigb` source file.
This way, any links into the smaller tutorial will show the entire page as generally desired.
And any links from the tutorial, back to the main massive README will link back to split versions, leading to fast loads.
This feature was implemented at: https://github.com/cirosantilli/ourbigbook/issues/131
= `\H` `splitSuffix` argument
{parent=h-arguments}
If given, add a custom suffix to the output filename of the header when using \x[split-headers].
If the given suffix is empty, it defaults to `-split`.
For example, given:
``
= my h1
== my h2
``
a `--split-headers` conversion would normally place `my h2` into a file called:
``
my-h2.html
``
However, if we instead wrote:
``
== my h2
{splitSuffix}
``
it would not be placed under:
``
my-h2-split.html
``
and if we set a custom one as:
``
== my h2
{splitSuffix=asdf}
``
it would go instead to:
``
my-h2-asdf.html
``
This option is useful if the root of your website is written in OurBigBook, and you want to both:
* have a section that talks about some other project
* host the documentation of that project inside the project source tree
For example, https://cirosantilli.com with source at https://github.com/cirosantilli/cirosantilli.github.io has a quick section about OurBigBook: https://cirosantilli.com#ourbigbook[].
Therefore, without a custom suffix, the split header version of that header would go to https://cirosantilli.com/ourbigbook[], which would collide with this documentation, that is present in a separate repository: https://github.com/cirosantilli/ourbigbook[].
Therefore a `splitSuffix` property is used, making the split header version fall under `/ourbigbook-split`, and leaving the nicer `/ourbigbook` for the more important project toplevel.
If given on the \x[the-toplevel-header]{p}, which normally gets a suffix by default to differentiate from the non-split version, it replaces the default `-split` suffix with a custom one.
For example if you had `notindex.bigb` as:
``
= Not index
``
then it would render to:
``
notindex-split.bigb
``
but if you used instead:
``
= Not index
{splitSuffix=asdf}
``
then it would instead be:
``
notindex-asdf.bigb
``
= `\H` `synonym` argument
{parent=h-arguments}
= Synonym
{synonym}
This option is similar to \x[h-title2-argument] but it additionally:
* creates a new ID that you can refer to, and renders it with the alternate chosen title
* the rendered ID on \x[internal-cross-reference]{p} is the same as what it is a synonym for
* the synonym header is not rendered at all, including in the \x[table-of-contents]
* when using \x[split-headers], a redirect output file is generated from the synonym to the main ID
Example:
``
= Parent
== GNU Debugger
{c}
= GDB
{c}
{synonym}
I like to say \x[gdb] because it is shorter than \x[gnu-debugger].
``
renders something like:
``
= GNU Debugger
I like to say \a[#gnu-debugger][GDB] because it is shorter than \x[#gnu-debugger][GNU Debugger].
``
Furthermore, if \x[split-headers] is used, another file is generated:
``
gdb.html
``
which contains a redirection from `gdb.html` to `gnu-debugger.html`.
Implemented at: https://github.com/cirosantilli/ourbigbook/issues/114
= `\H` `title` argument
{parent=h-synonym-argument}
Contains the main content of the header. The \x[insane-syntax]:
``
= My title
``
is equivalent to the sane:
``
\H[1][My title]
``
and in both cases `My title` is the title argument.
The title argument is also notably used for \x[automatic-id-from-title].
= Automatic ID from title
{parent=h-title-argument}
If a \x[the-toplevel-header][non-toplevel] macro has the `title` property is present but no explicit `id`, an ID is created automatically from the `title`, by applying the following transformations:
* do a \x[id-output-format] conversion on the title to remove for example any HTML tags that would be present in the conversion output
* convert all characters to lowercase. This uses \x[javascript-case-conversion]. Note that this does convert non-ASCII characters to lowercase, e.g. `É` to `é`.
* if \x[ourbigbook-json/id-normalize-latin] is `true` (the default) do \x[ourbigbook-json/latin-normalization]. This converts e.g. `é` to `e`.
* if \x[ourbigbook-json/id-normalize-punctuation] is `true` (the default) do \x[ourbigbook-json/punctuation-normalization]. This converts e.g. `+` to `plus`.
* convert consecutive sequences of all non `a-z0-9` ASCII characters to a single hyphen `-`. Note that this leaves non-ASCII characters untouched.
* strip leading or trailing hyphens
Note how those rules leave non-ASCII Unicode characters untouched, except for:
* capitalization changes wher applicable, e.g. `É` to `é`
as capitalization and determining if something "is a letter or not" in those cases can be tricky.
For toplevel headers, see: \x[the-id-of-the-first-header-is-derived-from-the-filename].
So for example, the following automatic IDs would be generated: \x[table-examples-of-automatically-generated-ids].
\Table{title=Examples of automatically generated IDs.}
[
|| title
|| id
|| latin normalization
|| punctuation normalization
|| comments
| My favorite title
| my-favorite-title
|
|
|
| Ciro's markdown is awesome
| ciro-s-markdown-is-awesome
|
|
| `'` is an ASCII character, but it is not in `a-z0-9`, therefore it gets converted to a hyphen `-`
| É你
| e你
| true
|
| The Latin https://en.wikipedia.org/wiki/Acute_accent[acute accented] `e`, `É`, is converted to its lower case form `é` as per the \x[javascript-case-conversion].
Then, due to \x[ourbigbook-json/latin-normalization], `é` is converted to `e`.
The Chinese character `你` is left untouched as Chinese characters have no case, and no ASCII analogue.
| É你
| é你
| false
|
| Same as the previous, but `é` is not converted to `e` since \x[ourbigbook-json/latin-normalization] is turned off.
| C++ is great
| c-plus-plus-is-great
|
| true
| This is the effect of \x[ourbigbook-json/punctuation-normalization].
| I \i[love] dogs.
| i-love-dogs
|
|
| `love` is extracted from the italic tags `love` with \x[id-output-format] conversion.
]
For \x[the-toplevel-header][the toplevel header], its ID is derived from the basename of the OurBigBook file without extension instead of from the `title` argument.
= `\H` `title2` argument of a synonym header
{parent=h-synonym-argument}
Unlike \x[h-title2-argument], the synonym does not show up by default next to the title. This is because we sometimes want that, and sometimes not. To make the title appear, you can simply add an empty `title2` argument to the synonym header as in:
``
= GNU Debugger
{c}
= GDB
{c}
{synonym}
{title2}
= Quantum computing
= Quantum computer
{synonym}
``
which renders something like:
``
= GNU Debugger (GDB)
= Quantum computing
``
Note how we added the synonym to the title only when it is not just a simple flexion variant, since `Quantum computing (Quantum computer)` would be kind of useless would be kind of useless.
= `\H` `tag` argument
{c}
{parent=h-arguments}
{tag=multiple-argument}
Same as \x[x-child-argument] but in the opposite direction, e.g.:
``
== Mammal
=== Bat
{tag=flying-animal}
=== Cat
== Flying animal
``
is equivalent in every way to:
``
== Mammal
=== Bat
=== Cat
== Flying animal
{child=bat}
``
Naming rationale:
* `parent` as the opposite of child is already taken to be then "main parent" via the "\x[h-parent-argument]"
* we could have renamed the \x[h-child-argument] to `tags` as in "this header tags that one", but it would be a bit confusing `tags` vs `tag`
So `child` vs `tag` it is for now.
You generally want to use `tag` instead of the \x[h-child-argument] because otherwise some very large header categories are going to contain Huge lists of children, which is not very nice when editing.
It is possible to enforce the \x[h-child-argument] or the \x[h-tag-argument] in a given project with the \x[ourbigbook-json/lint-h-tag] option.
= `\H` `title2` argument
{parent=h-arguments}
The `title2` argument can be given to any element that has the `title` argument.
Its usage is a bit like the `description=` argument of \x[image]{p}, allowing you to add some extra content to the header without affecting its ID.
Unlike `description=` however, `title2` shows up on all \x[x-full-argument][`full`] references, including appearances in the \x[table-of-contents], which make it more searchable.
Its primary use cases are:
* give acronyms, or other short names names of fuller titles such as mathematical/programming notation
One primary reason to not use the acronyms as the main section name is to avoid possible ID ambiguities with other acronyms.
* give the header in different languages
For example, given the OurBigBook input:
``
= Toplevel
The Toc follows:
== North Atlantic Treaty Organization
{c}
{title2=NATO}
\x[north-atlantic-treaty-organization]
\x[north-atlantic-treaty-organization]{full}
``
the rendered output looks like:
``
= Toplevel
The ToC follows:
* North Atlantic Treaty Organization (NATO)
== North Atlantic Treaty Organization (NATO)
North Atlantic Treaty Organization
Section 1. "North Atlantic Treaty Organization (NATO)"
``
Related alternatives to `title2` include:
* \x[h-disambiguate-argument] when you do want to affect the ID to remove ambiguities
* \x[h-synonym-argument]
Parenthesis are added automatically around all rendered `title2`.
The `title2` argument has a special meaning when applied to a \x[header] with the \x[h-synonym-argument], see \x[h-title2-argument-of-a-synonym-header].
\Comment[[[[
= `\H` `tutorial` argument
{parent=h-arguments}
{tag=boolean-argument}
This option is a convenience helper for the common use case of "writting a tutorial". The same would also apply to a report, or an article.
]]]]
= `\H` `wiki` argument
{parent=h-arguments}
If given, show a link to the Wikipedia article that corresponds to the header.
If a value is not given, automatically link to the Wiki page that matches the header exactly with spaces converted to underscores.
Here is an example with an explicit wiki argument:
``
==== Tiananmen Square
{wiki=Tiananmen_Square}
``
which looks like:
= Tiananmen Square
{id=wiki-explicit}
{parent=h-wiki-argument}
{wiki=Tiananmen_Square}
or equivalently with the value deduced from the title:
``
= Tiananmen Square
{wiki}
``
which looks like:
= Tiananmen Square
{id=wiki-implicit}
{parent=h-wiki-argument}
{wiki}
You can only link to subsections of wiki pages with explicit links as in:
``
= History of Tiananmen Square
{wiki=Tiananmen_Square#History}
``
which looks like:
= History of Tiananmen Square
{id=wiki-explicit-subsection}
{parent=h-wiki-argument}
{wiki=Tiananmen_Square#History}
= Header metadata section
{parent=header}
OurBigBook adds some header metadata to the toplevel header at the bottom of each page. this section describes this metadata.
Although the \x[table-of-contents]{child} has a macro to specify its placement, it is also automatically placed at the bottom of the page, and could be considered a header metadata section.
= Incoming links
{parent=header-metadata-section}
Lists other sections that link to the current section.
E.g. in:
``
= tmp
== tmp 1
=== tmp 1 1
=== tmp 1 2
\x[tmp-1]
== tmp 2
\x[tmp-1]
``
the page `tmp-1.html` would contain a list of incoming links as:
* `tmp-1-2`
* `tmp-2`
since those pages link to the `tmp-1` ID.
= Tagged
{parent=header-metadata-section}
Lists sections that are \x[secondary-children] of the current section, i.e. tagged under the current section.
The main header tree hierarchy descendants already show under the \x[table-of-contents] instead.
E.g. in:
``
= tmp
== Mammal
== Flying
== Animal
=== Bat
{tag=mammal}
{tag=flying}
=== Bee
{tag=flying}
=== Dog
{tag=mammal}
``
the tagged sections for:
* Mammal will contain Bat and Dog
* Flying will contain Bat and Bee
= Ancestors
{parent=header-metadata-section}
Shows a list of ancestors of the page. E.g. in:
``
= Asia
== China
=== Beijing
==== Tiananmen Square
=== Hong Kong
``
the ancestor lists would be for:
* Hong Kong: China, Asia
* Tiananmen Square: Beijing, China, Asia
* Beijing: China, Asia
* China: Asia
so we see that this basically provides a type of breadcrumb navigation.
= Image
{parent=macro}
{title2=`\Image` and `\image`}
A block image with \x[block-vs-inline-macros][capital] 'i' `Image` showcasing most of the image properties \x[image-my-test-image].
\OurbigbookExample[[
Have a look at this amazing image: \x[image-my-test-image].
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=The title of my image.}
{id=image-my-test-image}
{width=600}
{height=200}
{source=https://en.wikipedia.org/wiki/File:Tianasquare.jpg}
{description=The description of my image.}
]]
This exemplifies the following parameters:
* `title`: analogous to the \x[h-title-argument]. Shows up preeminently, and sets a default ID if one is not given.
* \x[image-description-argument][`description`]: similar to `title`, but allow for further explanations without them appearing in \x[internal-cross-reference][cross references] to the image
* `source`: a standardized way to credit an image by linking to a URL that contains further image metadata
For further discussion on the effects of ID see: \x[image-id]{full}.
And this is how you make an inline image inline one with lower case `i`:
\OurbigbookExample[[
My inline \image[Tank_man_standing_in_front_of_some_tanks.jpg][test image] is awesome.
]]
Inline images can't have captions.
And now for an image outside of \x[ourbigbookexample] to test how it looks directly under \x[toplevel]: \x[image-my-test-image-toplevel]{full}.
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{id=image-my-test-image-toplevel}
= Image height
{parent=image}
By default, we fix image heights to `height=315`, and let the `width` be calculated proportionally once the image loads. We therefore ignore the actual image size. This is done to:
* prevent reflows as the page loads images and can determine their actual sizes, especially is the user opens the page at a given ID in the middle of the page
* create a more uniform media experience by default, unless a custom image size is actually needed e.g. if the image needs to be larger
= Image ID
{parent=image}
Here is an image without a \x[image-description-argument][description] but with an ID so we can link to it: \x[image-my-test-image-2].
\OurbigbookExample[[
Have a look at this amazing image: \x[image-my-test-image-2].
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{id=image-my-test-image-2}
]]
This works because \x[x-full-argument][`full` is the default cross reference style for `Image`], otherwise the link text would be empty since there is no `title`, and OurBigBook would raise an error.
OurBigBook can optionally deduce the title from the basename of the `src` argument if the `titleFromSrc` \x[boolean-argument] is given, or if `title-from-src` is set as the default \x[ourbigbook-json/media-providers][media provider] for the media type:
\OurbigbookExample[[
Have a look at this amazing image: \x[image-tank-man-standing-in-front-of-some-tanks].
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{titleFromSrc}
]]
= Image caption
{parent=image-id}
If the image has neither \x[image-id][ID] nor title nor \x[image-description-argument][description] nor `source`, then it does not get a caption at all:
\OurbigbookExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
]]
If the image does not have an ID nor title, then it gets an automatically generated ID, just like every other OurBigBook output HTML element, and it is possible for readers to link to that ID on the rendered version, e.g. as:
``
#_123
``
Note that the `123` is not linked to the `Figure .`, but just a sequential ID that runs over all elements.
This type of ID is of course not stable across document revisions however, since if an image is added before that one, the link will break. So give an ID or title for anything that you expect users to link to.
Also, it is not possible to link to such images with an \x[internal-cross-reference], like any other OurBigBook element with autogenerated temporary IDs.
Another issue to consider is that in paged output formats like PDF, the image could float away from the text that refers to the image, so you basically always want to refer to image by ID, and not just by saying "the following image".
We can also see that such an image does not increment the Figure count:
\OurbigbookExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{id=image-my-test-image-count-before}
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{id=image-my-test-image-count-after}
]]
If the image has any visible metadata such as `source` or `description` however, then the caption does show and the Figure count gets incremented:
\OurbigbookExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{source=https://en.wikipedia.org/wiki/File:Tianasquare.jpg}
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]{description=This is the description of my image.}
]]
= Where to store images
{parent=image}
= Store images inside the repository itself
{parent=where-to-store-images}
If you are making a limited repository that will not have a ton of images, then you can get away with simply git tracking your images in the main repository.
With this setup, no further action is needed. For example, with a file structure of:
``
./README.bigb
./Tank_man_standing_in_front_of_some_tanks.jpg
``
just use the image from \C[README.bigb] as:
\OurbigbookExample[[
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
]]
However, if you are making a huge tutorial, which can have a huge undefined number of images (i.e. any scientific book), then you likely don't want to git track your images in the git repository.
A generally better alternative is to \x[store-images-in-a-separate-media-repository], and especially \x[store-images-in-a-separate-media-repository-and-track-it-as-a-git-submodule].
= Store images in a separate media repository
{parent=where-to-store-images}
In this approach, you create a separate GitHub repository in addition to the main one containing the text to contain only media such as images.
This approach is more suitable than \x[store-images-inside-the-repository-itself] if you are going to have a lot of images.
When using this approach, you could of course just point directly to the final image URL, e.g. as in:
\OurbigbookExample[[
\Image[https://raw.githubusercontent.com/cirosantilli/media/master/Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg]
]]
but OurBigBook allows you use configurations that allow you to enter just the image basename: `Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg` which we will cover next.
In order to get this to work, the recommended repository setup is:
* `./main-repo/.git`: main repository at https://github.com/username/main-repo
* `./main-repo/data/media/.git/`: media repository at https://github.com/username/main-repo-media[], and where `data/` is gitignored.
The directory and repository names are not mandatory, but if you place media in `data/media` and name its repository by adding the `*-media` suffix, then `ourbigbook` will handle everything for you without any further configuration in \x[ourbigbook-json/media-providers].
This particular documentation repository does have a different setup as can be seen from its \a[ourbigbook.json]. Then, when everything is setup correctly, we can refer to images simply as:
\OurbigbookExample[[
\Image[Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg]{provider=github}
]]
In this example, we also needed to set `{provider=github}` explicitly since it was not set as the default image provider in our `ourbigbook.json`. In most projects however, all of your images will be in the default repository, so this won't be needed.
`provider` must not be given when a full URL is given because we automatically detect providers from URLs, e.g.:
``
\Image[https://raw.githubusercontent.com/cirosantilli/media/master/Chrysanthemum_Xi_Jinping_with_black_red_liusi_added_by_Ciro_Santilli.jpg]{provider=github}
``
is an error.
TODO implement: `ourbigbook` will even automatically add and push used images in the `my-tutorial-media` repository for you \x[publish][during publishing]!
You should then use the following rules inside `my-tutorial-media`:
* give every file a very descriptive and unique name as a full English sentence
* never ever delete any files, nor change their content, unless it is an improvement in format that does change the information contained of the image TODO link to nice Wikimedia Commons guideline page
This way, even though the repositories are not fully in sync, anyone who clones the latest version of the `*-media` directory will be able to view any version of the main repository.
Then, if one day the media repository ever blows up GitHub's limit, you can just migrate the images to another image server that allows arbitrary basenames, e.g. AWS, and just configure your project to use that new media base URL with the \x[ourbigbook-json/media-providers] option.
The reason why images should be kept in a separate repository is that images are hundreds or thousands of times larger than hand written text.
Therefore, images could easily fill up the maximum repository size you are allowed: https://webapps.stackexchange.com/questions/45254/file-size-and-storage-limits-on-github#84746 and then what will you do when GitHub comes asking you to reduce the repository size?
https://git-lfs.github.com/[Git LFS] is one approach to deal with this, but we feel that it adds too much development overhead.
= Store images in a separate media repository and track it as a git submodule
{parent=store-images-in-a-separate-media-repository}
This is likely the sanest approach possible, as it clearly specifies which media version matches which repository version through the submodule link.
Furthermore, it is possible to make the submodule clone completely optional by setting things up as follows. For your OurBigBook project `yourname/myproject` create a `yourname/myproject-media` with the media, and track it as a submodule under `yourname/myproject/media`.
Then, add to \x[ourbigbook-json/media-providers]:
```
"media-providers": {
"github": {
"default-for": ["image", "video"],
"path": "media",
"remote": "yourname/myproject-media"
}
}
```
Now, as mentioned at \x[ourbigbook-json/media-providers], everything will work beautifully:
* `ourbigbook .` local conversion will use images from `media/` if it exists, e.g.:
```
\Image[myimage.jpg]
```
will render `media/myimage.jpg`. So after cloning the submodule, you will be able to see the images on the rendered pages without an internet connection.
But if the submodule is not cloned, not problem, renders will detect that and automatically use GitHub images.
Then, when you do:
```
ourbigbook --publish
```
the following happen:
* `\Image[myimage.jpg]` uses the GitHub URL
* automatically push `media/` to GitHub in case there were any updates
* also, that directory is automatically gitignore, so it won't be pushed as part of the main render and thus duplicate things
= Store images in Wikimedia Commons
{parent=where-to-store-images}
Wikimedia Commons is another great possibility to upload your images to:
\OurbigbookExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/thumb/5/5b/Gel_electrophoresis_insert_comb.jpg/450px-Gel_electrophoresis_insert_comb.jpg]
{source=https://commons.wikimedia.org/wiki/File:Gel_electrophoresis_insert_comb.jpg}
]]
OurBigBook likes Wikimedia Commons so much that we automatically parse the image URL and if it is from Wikimedia Commons, automatically deduce the `source` for you. So the above image renders the same without the `source` argument:
\OurbigbookExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/5/5b/Gel_electrophoresis_insert_comb.jpg]
]]
And like for non-Wikimedia images, you can automatically generate a `title` from the `src` by setting the `titleFromSrc` \x[boolean-argument] or if `title-from-src` is set as the default \x[ourbigbook-json/media-providers][media provider] for the media type:
\OurbigbookExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/5/5b/Gel_electrophoresis_insert_comb.jpg]
{titleFromSrc}
]]
And a quick test for a more complex thumb resized URL:
\OurbigbookExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/thumb/5/5b/Gel_electrophoresis_insert_comb.jpg/450px-Gel_electrophoresis_insert_comb.jpg]
]]
If you really absolutely want to turn off the `source`, you can explicitly set:
\OurbigbookExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/5/5b/Gel_electrophoresis_insert_comb.jpg]
{source=}
]]
but you don't want to do that for the most commonly Wikimedia Commons used license of CC BY+, do you? :-)
Upsides of using Wikimedia Commons for your images:
* makes it easier for other writers to find and reuse your images
* automatically generates resized versions of the uploaded images into several common dimensions so you can pick the smallest one that fits your desired \x[image-height] to reduce bandwidth usage
* if you have so many images that they would blow even the size of a \x[store-images-in-a-separate-media-repository][separate media repository], this will still work
Downsides:
* forces you to use the Creative Commons license
* requires the content to be educational in nature
* uploading a bunch of images to Wikimedia Commons does feel a bit more laborious than it should because you have to write down so much repeated metadata for them
= Image lazy loading
{parent=image}
We do this by default because OurBigBook is meant to allow producing huge single page documents like Ciro likes it, and in this way:
* images that the user is looking at will load first
* we save a lot of bandwidth for the user who only wants to browse one section
TODO: maybe create a mechanism to disable this for the entire build with \x[ourbigbook-json].
= Background color of transparent images
{parent=image}
For the love of God, there is no standardized for SVG to set its background color without a rectangle? https://stackoverflow.com/questions/11293026/default-background-color-of-svg-root-element `viewport-fill` was just left in limbo?
And as a result, many many many SVG online images that you might want to reuse just rely on white pages and don't add that background rectangle.
Therefore for now we just force white background on \x[overview-of-files-in-this-repository][our default CSS], which is what most SVGs will work with. Otherwise, you can lose the entire image to our default black background.
Then if someone ever has an SVG that needs another background color, we can add an image attribute to set that color as a local style.
= Image generators
{parent=image}
TODO implement: mechanism where you enter a textual description of the image inside the code body, and it then converts to an image, adds to the `-media` repo and pushes all automatically. Start with dot.
https://github.com/cirosantilli/ourbigbook/issues/40
= Image argument
{parent=image}
= Image `check` argument
{parent=image-argument}
{tag=named-argument}
{tag=boolean-argument}
Analogous to the \x[a-check-argument] when checking if the \x[image-src-argument] exists or not.
= Image `description` argument
{parent=image-argument}
{tag=named-argument}
The `description` argument similar to the \x[image-title-argument] argument, but allows allowing longer explanations without them appearing in \x[internal-cross-reference][cross references] to the image.
For example, consider:
\OurbigbookExample[[
See this image: \x[image-description-argument-test-1].
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=Tank man standing in front of some tanks.}
{id=image-description-argument-test-1}
{description=Note how the tanks are green.}
{source=https://en.wikipedia.org/wiki/File:Tianasquare.jpg}
]]
In this example, the reference `\x[image-description-argument-test-1]` expands just to
\Q[Tank man standing in front of some tanks]
and does not include the description, which only shows on the image.
The description can be as large as you like. If it gets really large however, you might want to consider moving the image to its own header to keep things slightly saner. This will be especially true after we eventually do: https://github.com/cirosantilli/ourbigbook/issues/180[].
If the description contains any element that would take its own separate line, like multiple paragraphs or a list, we automatically add a line grouping the description with the corresponding image to make that clearer, otherwise it can be hard to know which title corresponds to a far away image. Example with multiple paragraphs:
\OurbigbookExample[[
Stuff before the image.
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=Tank man standing in front of some tanks.}
{id=image-description-argument-test-2}
{source=https://en.wikipedia.org/wiki/File:Tianasquare.jpg}
{description=Note how the tanks are green.
But the shirt is white.}
Stuff after the image description.
]]
= Image `src` argument
{parent=image-argument}
{tag=positional-argument}
The address of the image, e.g. in:
``
\Image[image.png]
``
the `src` is `image.png`.
Analogous to the \x[a-href-argument].
= Image `title` argument
{parent=image-argument}
Analogous to the \x[h-title-argument].
= Include
{title2=`\Include`}
{parent=macro}
The `\Include` macro allows including an external OurBigBook headers under the current header.
It exists to allow optional single page HTML output while still retaining the ability to:
* split up large input files into multiple files to make renders faster during document development
* suggest an optional custom output split with one HTML output per OurBigBook input, in order to avoid extremely large HTML pages which could be slow to load
`\Include` takes one mandatory argument: the ID of the section to be included, much like \x[internal-cross-reference]{p}.
There is however one restriction: only \x[the-toplevel-header]{p} can be pointed to. This restriction allows us to easily find the included file in the filesystem, and dispenses the need to do a first `./ourbigbook` run to generate the \x[internal-cross-file-reference-internals][ID database]. This works because \x[the-id-of-the-first-header-is-derived-from-the-filename].
Headers of the included document are automatically shifted to match the level of the child of the level where they are being included.
If \x[embed-includes] is given, the external document is rendered embedded into the current document directly, essentially as if the source had been copy pasted (except for small corrections such as the header offsets).
Otherwise, the following effects happen:
* The headers of the included tree appear in the \x[table-of-contents] of the document as links to the corresponding external files.
This is implemented simply by reading a previously generated database file much like \x[internal-cross-file-reference-internals], which avoids the slowdown of parsing all included files every time.
As a result, you have to do an initial parse of all files in the project to extract their headers however, just as you would need to do when linking to those headers.
* the include itself renders as a link to the included document
* \x[embed-includes]
Here is an example of inclusion of the files `not-readme.bigb` and `not-readme-2.bigb`:
``
\Include[not-readme]
\Include[not-readme-2]
\Include[not-readme-with-scope]
``
The above is the recommended and slightly \x[insane-macro-shortcuts][insaner] version of:
``
\Include[not-readme]
\Include[not-readme-2]
\Include[not-readme-with-scope]
``
The insaner version is a bit insaner because the `\Include` magically discards the following newline node that follows it if it just a plaintext node containing exactly a newline. With a double newline, the newline would already have been previously taken out on the lexing stage as part of a paragraph.
\x[include-example]{full} shows what those actually render like.
= `\Include` from subdirectories
{parent=include}
When you are in a subdirectory, include resolution just is simply relative to the subdirectory. E.g. we could do:
subdir/index.bigb
``
= Subdir
\Include[notindex]
\Include[subdir2/notindex]
``
subdir/notindex.bigb
``
= Notindex
``
subdir/subdir2/notindex.bigb
``
= Notindex
``
It is not currently possible to include from ancestor directories: https://github.com/cirosantilli/ourbigbook/issues/214[].
= `\Include` `parent` argument
{parent=include}
This option is analogous to \x[h-parent-argument], but for \x[include]{p}.
For example, consider you have:
``
= Animal
== Dog
== Cat
== Bat
``
and now you want to split `Cat` to `cat.bigb`.
If you wrote:
``
= Animal
== Dog
\Include[cat]
== Bat
``
Cat would be a child of Dog, since that is the previous header, which is not what we want.
Instead, we want to write:
``
= Animal
== Dog
\Include[cat]{parent=animal}
== Bat
``
and now Cat will be a child of Animal as desired.
Implemented at: https://github.com/cirosantilli/ourbigbook/issues/127
= `\Include` example
{parent=include}
This shows what includes render as.
\Include[not-readme]
\Include[not-readme-2]
\Include[not-readme-with-scope]
\Include[subdir]
\Include[subdir/notindex]
= Italic
{parent=macro}
{title2=`\i`}
\OurbigbookExample[[
Some \i[italic] text.
]]
= `\JsCanvasDemo`
{parent=macro}
The `JsCanvasDemo` macro allows you to create interactive HTML/JavaScript https://en.wikipedia.org/wiki/Canvas_element[canvas] demos easily.
These demos:
* only start running when the user scrolls over them for the first time
* stop automatically when they leave the viewport
so you can stuff as many of them as you want on a page, and they won't cause the reader's CPU to fry an egg.
\OurbigbookExample[[[
\JsCanvasDemo[[
new class extends OurbigbookCanvasDemo {
init() {
super.init('hello');
this.pixel_size_input = this.addInputAfterEnable(
'Pixel size',
{
'min': 1,
'type': 'number',
'value': 1,
}
);
}
draw() {
var pixel_size = parseInt(this.pixel_size_input.value);
for (var x = 0; x < this.width; x += pixel_size) {
for (var y = 0; y < this.height; y += pixel_size) {
var b = ((1.0 + Math.sin(this.time * Math.PI / 16)) / 2.0);
this.ctx.fillStyle =
'rgba(' +
(x / this.width) * 255 + ',' +
(y / this.height) * 255 + ',' +
b * 255 +
',255)'
;
this.ctx.fillRect(x, y, pixel_size, pixel_size);
}
}
}
}
]]
]]]
And another one showing off some https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API[WebGL]:
\JsCanvasDemo[[
new class extends OurbigbookCanvasDemo {
init() {
super.init('webgl', {context_type: 'webgl'});
this.ctx.viewport(0, 0, this.ctx.drawingBufferWidth, this.ctx.drawingBufferHeight);
this.ctx.clearColor(0.0, 0.0, 0.0, 1.0);
this.vertexShaderSource = `
#version 100
precision highp float;
attribute float position;
void main() {
gl_Position = vec4(position, 0.0, 0.0, 1.0);
gl_PointSize = 64.0;
}
`;
this.fragmentShaderSource = `
#version 100
precision mediump float;
void main() {
gl_FragColor = vec4(0.18, 0.0, 0.34, 1.0);
}
`;
this.vertexShader = this.ctx.createShader(this.ctx.VERTEX_SHADER);
this.ctx.shaderSource(this.vertexShader, this.vertexShaderSource);
this.ctx.compileShader(this.vertexShader);
this.fragmentShader = this.ctx.createShader(this.ctx.FRAGMENT_SHADER);
this.ctx.shaderSource(this.fragmentShader, this.fragmentShaderSource);
this.ctx.compileShader(this.fragmentShader);
this.program = this.ctx.createProgram();
this.ctx.attachShader(this.program, this.vertexShader);
this.ctx.attachShader(this.program, this.fragmentShader);
this.ctx.linkProgram(this.program);
this.ctx.detachShader(this.program, this.vertexShader);
this.ctx.detachShader(this.program, this.fragmentShader);
this.ctx.deleteShader(this.vertexShader);
this.ctx.deleteShader(this.fragmentShader);
if (!this.ctx.getProgramParameter(this.program, this.ctx.LINK_STATUS)) {
console.log('error ' + this.ctx.getProgramInfoLog(this.program));
return;
}
this.ctx.enableVertexAttribArray(0);
var buffer = this.ctx.createBuffer();
this.ctx.bindBuffer(this.ctx.ARRAY_BUFFER, buffer);
this.ctx.vertexAttribPointer(0, 1, this.ctx.FLOAT, false, 0, 0);
this.ctx.useProgram(this.program);
}
draw() {
this.ctx.clear(this.ctx.COLOR_BUFFER_BIT);
this.ctx.bufferData(this.ctx.ARRAY_BUFFER, new Float32Array([Math.sin(this.time / 60.0)]), this.ctx.STATIC_DRAW);
this.ctx.drawArrays(this.ctx.POINTS, 0, 1);
}
}
]]
= List
{parent=macro}
{title2=`* `, `\L`, `\Ul`, `\Ol`}
\x[insane-macro-shortcuts][Insane] with `* ` (asterisk space):
\OurbigbookExample[[
* a
* b
* c
]]
Equivalent saner with \x[auto-parent][implicit `ul` container]:
\OurbigbookExample[[
\L[a]
\L[b]
\L[c]
]]
Equivalent fully sane with explicit container:
\OurbigbookExample[[
\Ul[
\L[a]
\L[b]
\L[c]
]
]]
The explicit container is required if you want to pass extra arguments properties to the `ul` list macro, e.g. a title and an ID: \x[list-my-id]{full}:
\OurbigbookExample[[
\Ul
{id=list-my-id}
[
\L[a]
\L[b]
\L[c]
]
]]
This is the case because without the explicit container in an implicit `ul` list, the arguments would stick to the last list item instead of the list itself.
It is also required if you want ordered lists:
\OurbigbookExample[[
\Ol[
\L[first]
\L[second]
\L[third]
]
]]
Insane nested list with two space indentation:
\OurbigbookExample[[
* a
* a1
* a2
* a2
* b
* c
]]
The indentation must always be exactly equal to two spaces, anything else leads to errors or unintended output.
Equivalent saner nested lists with implicit containers:
\OurbigbookExample[[
\L[
a
\L[a1]
\L[a2]
\L[a2]
]
\L[b]
\L[c]
]]
Insane list item with a paragraph inside of it:
\OurbigbookExample[[
* a
* I have
Multiple paragraphs.
* And
* also
* a
* list
* c
]]
Equivalent sane version:
\OurbigbookExample[[
\L[a]
\L[
I have
Multiple paragraphs.
\L[And]
\L[also]
\L[a]
\L[list]
]
\L[c]
]]
Insane lists may be escaped with a backslash as usual:
\OurbigbookExample[[
\* paragraph starting with an asterisk.
]]
You can also start insane lists immediately at the start of a \x[positional-vs-named-arguments][positional or named argument], e.g.:
\OurbigbookExample[[
\P[* a
* b
* c
]
]]
And now a list outside of \x[ourbigbookexample] to test how it looks directly under \x[toplevel]:
\L[a]
\L[b]
\L[c]
= Mathematics
{parent=macro}
{title2=`$$`, `$`, `\M`, `\m`}
Via https://katex.org/[KaTeX] server side, oh yes!
Inline math is done with the dollar sign (`$`) \x[insane-macro-shortcuts][insane macro shortcut]:
\OurbigbookExample[[
My inline $\sqrt{1 + 1}$ is awesome.
]]
and block math is done with two or more dollar signs (`$$`):
\OurbigbookExample[[
$$
\sqrt{1 + 1} \\
\sqrt{1 + 1}
$$
]]
The sane version of inline math is a lower case `m`:
\OurbigbookExample[[[
My inline \m[[\sqrt{1 + 1}]] is awesome.
]]]
and the sane version of block math is with an upper case `M`:
\OurbigbookExample[[[
\M[[
\sqrt{1 + 1} \\
\sqrt{1 + 1}
]]
]]]
The capital vs lower case theme is also used in other elements, see: \x[block-vs-inline-macros].
In the sane syntax, \x[escape-characters][as with any other argument], you have to either escape any closing square brackets `]` with a backslash `\`:
\OurbigbookExample[[
My inline \m[1 - \[1 + 1\] = -1] is awesome.
]]
or with the equivalent double open and close:
``
My inline \m[[1 - [1 + 1] = -1]] is awesome.
``
HTML escaping happens as you would expect, e.g. < shows fine in:
\OurbigbookExample[[
$$
1 < 2
$$
]]
Equation IDs and titles and linking to equations works identically to \x[image]{p}, see that section for full details. Here is one equation reference example that links to the following insane syntax equation: \x[equation-my-first-insane-equation]:
\OurbigbookExample[[
$$
\sqrt{1 + 1}
$$
{title=My first insane equation.}
]]
and the sane equivalent \x[equation-my-first-sane-equation]:
\OurbigbookExample[[[
\M{title=My first sane equation.}[[
\sqrt{1 + 1}
]]
]]]
Here is a raw one just to test the formatting outside of a `ourbigbook_comment`:
$$\sqrt{1 + 1}$$
= Math defines across blocks
{parent=mathematics}
First here is an invisible block (with `{show=0}`) defining with a `\newcommand` definition after this paragraph:
\OurbigbookExample[[
$$
\newcommand{\foo}[0]{bar}
$${show=0}
]]
We make it invisible because this block only contains KaTeX definitions, and should not render to anything.
Then the second math block uses those definitions:
\OurbigbookExample[[
$$
\foo
$$
]]
Analogously with `\def`, definition:
\OurbigbookExample[[
$$
\gdef\foogdef{bar}
$${show=0}
]]
and the second block using it:
\OurbigbookExample[[
$$
\foogdef
$$
]]
And just to test that `{show=1}` actually shows, although it is useless, and that `{show=0}` skips incrementing the equation count:
\OurbigbookExample[[
$$1 + 1$${show=1}
$$2 + 2$${show=0}
$$3 + 3$${show=1}
]]
= `ourbigbook.tex`
{parent=mathematics}
If your project has multiple `.bigb` input files, you can share Mathematics definitions across all files by adding them to the `ourbigbook.tex` file on the toplevel directory.
For example, if `ourbigbook.tex` contains:
``
\newcommand{\foo}[0]{bar}
``
then from any `.bigb` file we in the project can use:
``
$$
\foo
$$
``
= Paragraph
{title2=`\P`}
{parent=macro}
OK, this is too common, so we opted for some \x[insane-macro-shortcuts][insanity] here: double newline is a paragraph!
\OurbigbookExample[[
Paragraph 1.
Paragraph 2.
]]
Equivalently however, you can use an explicit `\P` macros as well, which is required for example to add properties to a paragraph, e.g.:
\OurbigbookExample[[
\P{id=paragraph-1}[Paragraph 1]
\P{id=paragraph-2}[Paragraph 2]
]]
Paragraphs are created automatically inside \x[macro-argument-syntax][macro argument] whenever a double newline appears.
Note that OurBigBook paragraphs render in HTML as `div` with `class="p"` and not as `p`. This means that you can add basically anything inside them, e.g. a list:
``
My favorite list is:
\Ul[
\li[aa]
\li[bb]
]
because it is simple.
``
which renders as a single paragraph.
One major advantage of this, is that when writing documentation, you often want to keep lists or code blocks inside a given paragraph, so that it is easy to reference the entire paragraph with an ID. Think for example of paragraphs in the C++ standard.
= Passthrough
{parent=macro}
{title2=`\passthrough`}
Dumps its contents directly into the rendered output.
This construct is not XSS safe, see: \x[unsafe-xss]{full}.
Here for example we define a paragraph in raw HTML:
\OurbigbookExample[[[
\passthrough[[
Hello raw HTML!
]]
]]]
And for an inline passthrough:
\OurbigbookExample[[[
Hello \passthrough[[raw]] world!
]]]
= Quotation
{parent=macro}
{title2=`\Q`}
With `q`:
\OurbigbookExample[[
And so he said:
\Q[
Something very smart
And with multiple paragraphs.
]
and it was great.
]]
= Table
{parent=macro}
{title2=`|| `, `| `, `\Table`, `\Tr`, `\Th` and `\Td`}
The \x[insane-code-and-math-shortcuts][insane] syntax marks:
* headers with `|| ` (pipe, pipe space) at the start of a line
* regular cells with `| ` (pipe, space) at the start of a line
* separates rows with double newline
For example:
\OurbigbookExample[[
|| Header 1
|| Header 2
| 1 1
| 1 2
| 2 1
| 2 2
]]
Empty cells are allowed without the trailing space however:
\OurbigbookExample[[
| 1 1
|
| 1 3
| 2 1
|
| 2 3
]]
Equivalent fully explicit version:
\OurbigbookExample[[
\Table[
\Tr[
\Th[Header 1]
\Th[Header 2]
]
\Tr[
\Td[1 1]
\Td[1 2]
]
\Tr[
\Td[2 1]
\Td[2 2]
]
]
]]
Any white space indentation inside an explicit `\Tr` can make the code more readable, and is automatically removed from final output due to \x[remove-whitespace-children] which is set for `\Table`.
To pass further arguments to an implicit table such as `title` or `id`, you need to use an explicit `table` macro as in: \x[table-my-table].
\OurbigbookExample[[
\Table
{title=My table title.}
{id=table-my-table}
[
|| Header 1
|| Header 2
| 1 1
| 1 2
| 2 1
| 2 2
]
]]
We would like to remove that explicit toplevel requirement as per: https://github.com/cirosantilli/ourbigbook/issues/186[] The rules of when the caption shows up or not similar to those of \x[image]{p} as mentioned at \x[image-caption]{full}.
Multiple source lines, including paragraphs, can be added to a single cell with insane syntax by indenting the cell with exactly two spaces just as for \x[list]{p}, e.g.:
\OurbigbookExample[[
|| h1
|| h2
|| h3
h3 2
| 11
| 12
12 2
| 13
| 21
| 22
| 23
]]
Arbitrarily complex nested constructs may be used, e.g. a table inside a list inside table:
\OurbigbookExample[[
| 00
| 01
* l1
* l2
| 20
| 21
| 30
| 31
| 10
| 11
]]
And now a table outside of \x[ourbigbookexample] to test how it looks directly under \x[toplevel]:
\Table{title=My table title.}
[
\Tr[
\Th[Header 1]
\Th[Header 2]
]
\Tr[
\Td[1 1]
\Td[1 2]
]
\Tr[
\Td[2 1]
\Td[2 2]
]
]
And a fully insane one:
|| Header 1
|| Header 2
| 1 1
| 1 2
| 2 1
| 2 2
= Table sorting
{parent=table}
JavaScript interactive on-click table sorting is enabled by default, try it out by clicking on the header row:
\OurbigbookExample[[
|| String col
|| Integer col
|| Float col
| ab
| 2
| 10.1
| a
| 10
| 10.2
| c
| 2
| 3.4
]]
Powered by: https://github.com/tristen/tablesort
= Table of contents
{parent=macro}
{title2=`\Toc`}
OurBigBook automatically adds a ToC at the end of the first \x[the-toplevel-header][non-toplevel header] of every document.
For example, on a standard document with a single toplevel header:
``
= Animal
Animals are cute!
== Dog
== Cat
``
the ToC is rendered like:
``
= Animal
Animals are cute!
Table of Contents
* Dog
* Cat
== Dog
== Cat
``
You may customize ToC placement with `\Toc` macro, but just don't do it, it will just go against common convention and confuse readers.
This is especially important when considering \x[split-headers], where you almost always want an automatically generated ToC for every split header, otherwise you would need to add them all manually.
Only one ToC is rendered per document. Any ToC besides the first one is ignored. In particular, this means that \x[include]{p} with \x[embed-includes] work seamlessly and render a single table of contents, even if multiple `\Toc` macros are present in the included pages.
The \x[internal-cross-reference][ID] of the ToC is always fixed to `#toc`. If you try to use that for another element, you will get the following error:
``
error: tmp.bigb:3:1: reserved ID "toc"
``
The ToC ignores \x[the-toplevel-header] if you have one.
For when you want a quick outline of the header tree on the terminal, also consider the \x[log-headers] option.
= Table of contents JavaScript open close interaction
{parent=table-of-contents}
To the left of table of content entries you can click on an open/close icon to toggle the visibility of different levels of the table of contents.
The main use case covered by the expansion algorithm is as follows:
* the page starts with all nodes open to facilitate Ctrl + F queries
* if you click on a node in that sate, you close all its children, to get a summarized overview of the contents
* if you click one of those children, it opens only its own children, so you can interactively continue exploring the tree
The exact behaviour is:
* the clicked node is open:
* state 1 all children are closed. Action: open all children recursively, which puts us in state 2
* state 2: not all children are closed. Action close all children, which puts us in state 1. This gives a good overview of the children, without any children of children getting in the way.
* state 3: the clicked node is closed (not showing any children). Action: open it to show all direct children, but not further descendants (i.e. close those children). This puts us in state 1.
Note that those rules make it impossible to close a node by clicking on it, the only way to close a node os to click on its parent, the state transitions are:
* 3 -> 1
* 1 -> 2
* 2 -> 1
but we feel that it is worth it to do things like this to cover the main use case described above without having to add two buttons per entry.
Clicking on the link from a \x[header] up to the table of contents also automatically opens up the node for you in case it had been previously closed manually.
= Video
{parent=macro}
{title2=`\Video` and `\video`}
Very analogous to \x[image]{p}, only differences will be documented here.
In the case of videos, \x[where-to-store-images] becomes even more critical since videos are even larger than images, such that the following storage approaches are impractical off the bat:
* \x[store-images-inside-the-repository-itself]
* \x[store-images-in-a-separate-media-repository]
As a result, then https://commons.wikimedia.org[Wikimedia Commons] is one of the best options \x[store-images-in-wikimedia-commons][much like for images]:
\OurbigbookExample[[
\Video[https://upload.wikimedia.org/wikipedia/commons/8/85/Vacuum_pump_filter_cut_and_place_in_eppendorf.webm]
{id=sample-video-in-wikimedia-commons}
{title=Nice sample video stored in Wikimedia Commons.}
{start=5}
]]
We also handle more complex transcoded video URLs just fine:
\OurbigbookExample[[
\Video[https://upload.wikimedia.org/wikipedia/commons/transcoded/1/19/Scientific_Industries_Inc_Vortex-Genie_2_running.ogv/Scientific_Industries_Inc_Vortex-Genie_2_running.ogv.480p.vp9.webm]
{id=sample-video-in-wikimedia-commons-transcoded}
{title=Nice sample video stored in Wikimedia Commons transcoded.}
]]
Commons is better than YouTube if your content is on-topic there because:
* they have no ads
* it allows download of the videos: https://www.quora.com/Can-I-download-Creative-Commons-licensed-YouTube-videos-to-edit-them-and-use-them[].
* it makes it easier for other users to find and re-use your videos
If your video does not fit the above Wikimedia Commons requirements, YouTube could be a good bet. OurBigBook https://github.com/cirosantilli/ourbigbook/issues/50[automatically detects YouTube URLs] for you, so the following should just work:
\OurbigbookExample[[
\Video[https://youtube.com/watch?v=YeFzeNAHEhU&t=38]
{id=sample-video-from-youtube-implicit-youtube}
{title=Nice sample video embedded from YouTube implicit from `youtube.com` URL.}
]]
The `youtu.be` domain hack URLs also work;
\OurbigbookExample[[
\Video[https://youtu.be/YeFzeNAHEhU?t=38]
{id=sample-video-from-youtube-implicit-youtu-be}
{title=Nice sample video embedded from YouTube implicit from `youtu.be` URL.}
]]
Alternatively, you can reach the same result in a more explicit and minimal way by setting `{provider=youtube}` and the \x[video-start-argument][`start`] arguments:
\OurbigbookExample[[
\Video[YeFzeNAHEhU]{provider=youtube}
{id=sample-video-from-youtube-explicit}
{title=Nice sample video embedded from YouTube with explicit `youtube` argument.}
{start=38}
]]
When the `youtube` provider is selected, the Video address should only to contain the YouTube video ID, which shows in the YouTube URL for the video as:
``
https://www.youtube.com/watch?v=
``
Remember that you can also enable the `youtube` provider by default on your \x[ourbigbook-json] with:
``
"media-provider" {
"youtube": {"default-for": "video"}
}
``
But you can also use raw video files from any location that can serve them of course, e.g. here is one stored in this repository: \x[sample-video-in-repository].
\OurbigbookExample[[
\Video[Tank_man_side_hopping_in_front_of_some_tanks.mp4]
{id=sample-video-in-repository}
{title=Nice sample video stored in this repository.}
{source=https://www.youtube.com/watch?v=YeFzeNAHEhU}
{start=3}
]]
And as for images, setting `titleFromSrc` automatically calculates a title for you:
\OurbigbookExample[[
\Video[Tank_man_side_hopping_in_front_of_some_tanks.mp4]
{titleFromSrc}
{source=https://www.youtube.com/watch?v=YeFzeNAHEhU}
]]
= Video lazy loading
{parent=video}
Unlike \x[image-lazy-loading], we don't support video lazy loading yet because:
* non-`youtube` videos use the `video` tag which has no `loading` property yet
* `youtube` videos are embedded with `iframe` and `iframe` has no `loading` property yet
Both of this cases could be worked around with JavaScript:
* non-`youtube`: set `src` from JavaScript as shown for images: https://stackoverflow.com/questions/2321907/how-do-you-make-images-load-lazily-only-when-they-are-in-the-viewport/57389607#57389607[].
But this breaks page semantics however, we don't know how to work around that
* `youtube` videos: same as above for the `iframe`, but this should be less problematic since YouTube videos are not viewable without JavaScript anyways, and who cares about `iframe` semantics?
= `\Video` `start` argument
{parent=video}
The time to start playing the video at in seconds. Works for both `youtube` and non-YouTube videos.
= Internal cross reference
{parent=macro}
= `\x` macro
{title2}
{synonym}
Every macro in OurBigBook can have an optional `id` and many also have a reserved `title` property.
When a macro in the document has a `title` argument but no `id` argument given, get an auto-generated ID from the title: \x[automatic-id-from-title].
For macros that do have an ID, derived from `title` or not, you can write a cross reference to it, e.g.:
\OurbigbookExample[[
See this \x[internal-cross-reference]{p} awesome section.
]]
An explicit link body can be given just as for regular HTTP \x[link]{p} as:
\OurbigbookExample[[
See this \x[internal-cross-reference]{p}[awesome section].
]]
= Cross reference title inflection
{parent=internal-cross-reference}
A common usage pattern is that we want to use \x[header] titles in \x[x-full-argument][non-full] \x[internal-cross-reference]{p} as the definition of a concept without repeating the title, for example:
``
== Dog
Cute animal.
\x[cats][Cats] are its natural enemies.
== Cats
This is the natural enemy of a \x[dog][dog].
\x[dog][Dogs] are cute, but they are still the enemy.
One example of a cat is \x[felix-the-cat].
=== Felix the Cat
Felix is not really a \x[cats][cat], just a carton character.
``
However, word https://en.wikipedia.org/wiki/Inflection[inflection] makes it much harder to avoid retyping the definition again.
For example, in the previous example, without any further intelligent behaviour we would be forced to re-type `\x[dog][dog]` instead of the desired `\x[dog]`.
OurBigBook can take care of some inflection cases for you.
For capitalization, both headers and cross reference macros have the `c` \x[boolean-argument] which stands for "capitalized":
* for headers, `c` means that the header title has fixed capitalization as given in the title, i.e.
* if the title has a capital first character, it will always show as a capital, as is the case for most https://en.wikipedia.org/wiki/Proper_noun[proper noun]
* if it is lower case, it will also always remain lower case, as is the case for some rare proper nouns, notably https://en.wikipedia.org/wiki/Arm_Holdings[the name of certain companies]
This means that for such headers, `c` in the `x` has no effect. Maybe we should give an error in that case. But lazy now, send PR.
* for cross reference macros, `c` means that the first letter of the title should be capitalized.
Using this option is required when you are starting a sentence with a non-proper noun.
Capitalization is handled by a \x[javascript-case-conversion].
For pluralization, cross reference macros have the `p` \x[boolean-argument] which stands for "pluralize":
* if given and true, this automatically pluralizes the last word of the target title by using the https://github.com/blakeembrey/pluralize library
* if given and false, automatically singularize
* if not given, don't change the number of elements
If your desired pluralization is any more complex than modifying the last word of the title, you must do it manually however.
With those rules in mind, the previous OurBigBook example can be written with less repetition as:
``
== Dog
Cute animal.
\x[cats]{c} are its natural enemies.
== Cats
This is the natural enemy of a \x[dog].
\x[dog]{p} are cute, but they are still the enemy.
One example of a cat is \x[Felix the Cat].
=== Felix the Cat
{c}
Felix is not really a \x[cats][cat], just a carton character.
``
If plural and capitalization don't handle your common desired inflections, you can also just create custom ones with the \x[h-synonym-argument].
Now for a live example for quick and dirty interactive testing.
= Cross reference title inflection example
{parent=cross-reference-title-inflection}
\OurbigbookExample[[
\x[inflection-example-not-proper]
]]
\OurbigbookExample[[
\x[inflection-example-not-proper]{c}
]]
\OurbigbookExample[[
\x[inflection-example-not-proper]{full}
]]
\OurbigbookExample[[
\x[inflection-example-proper]
]]
\OurbigbookExample[[
\x[inflection-example-proper]{c}
]]
\OurbigbookExample[[
\x[inflection-example-not-proper-lower]
]]
\OurbigbookExample[[
\x[inflection-example-not-proper-lower]{c}
]]
\OurbigbookExample[[
\x[inflection-example-proper-lower]
]]
\OurbigbookExample[[
\x[not-readme]
]]
\OurbigbookExample[[
\x[not-readme]{c}
]]
\OurbigbookExample[[
\x[inflection-example-not-proper]{p}
]]
\OurbigbookExample[[
\x[inflection-plural-examples]
]]
\OurbigbookExample[[
\x[inflection-plural-examples]{p}
]]
\OurbigbookExample[[
\x[inflection-plural-examples]{p=0}
]]
\OurbigbookExample[[
\x[inflection-plural-examples]{p=1}
]]
\OurbigbookExample[[
\x[not-the-readme-header-with-fixed-case]
]]
= Inflection example not-proper
{parent=cross-reference-title-inflection-example}
= Inflection example proper
{c}
{parent=cross-reference-title-inflection-example}
= inflection example not-proper lower
{parent=cross-reference-title-inflection-example}
= inflection example proper lower
{c}
{parent=cross-reference-title-inflection-example}
= Inflection plural examples
{parent=cross-reference-title-inflection-example}
= `\x` within `title` restrictions
{parent=internal-cross-reference}
If you use `\x` within a `title`, which most commonly happens for \x[image][image titles], that can generate complex dependencies between IDs, which would either be harder to implement, or lead to infinite recursion.
To prevent such problems, OurBigBook emits an error if you use an `\x` without content in the `title` of one of the following elements:
* any \x[header]. For example, the following gives an error:
``
= h1
{id=myh1}
== \x[myh1]
``
This could be solved by either adding a content to the reference:
``
= h1
{id=myh1}
== \x[myh1][mycontent]
``
or by adding an explicit ID to the header:
``
= h1
{id=myh1}
== \x[myh1]
{id=myh2}
``
* non-header (e.g. an \x[image]) that links to the title of another non-header
For non-headers, things are a bit more relaxed, and we can link to headers, e.g.:
``
= h1
\Image[myimg.jpg]
{title=my \x[h1].}
``
This is allowed because OurBigBook calculates IDs in two stages: first for all headers, and only later non non-headers.
What you cannot do is link to another image e.g.:
``
\Image[myimg.jpg]
{id=myimage1}
{title=My image 1.}
\Image[myimg.jpg]
{title=my \x[h1].}
``
and there the workaround are much the same as for headers: either explicitly set the cross reference content:
``
\Image[myimg.jpg]
{id=myimage1}
{title=My image 1.}
\Image[myimg.jpg]
{title=my \x[h1][My image 1].}
``
or explicitly set an ID:
``
\Image[myimg.jpg]
{id=myimage1}
{title=My image 1.}
\Image[myimg.jpg]
{id=myimage2}
{title=my \x[h1].}
``
While it is technically possible relax the above limitations and give an error only in case of loops, it would require a bit of extra work which we don't want to put in right now: https://github.com/cirosantilli/ourbigbook/issues/95[].
Furthermore, the above rules do not exclude infinite rendering loops, but OurBigBook detects such loops and gives a nice error message, this has been fixed at: https://github.com/cirosantilli/ourbigbook/issues/34
For example this would contain an infinite loop:
``
\Image[myimg.jpg]
{id=myimage1}
{title=\x[myimage2].}
\Image[myimg.jpg]
{id=myimage2}
{title=\x[myimage1].}
``
This infinite recursion is fundamentally not technically solved: the user has to manually break the loop by providing an `x` content explicitly, e.g. in either:
``
\Image[myimg.jpg]
{id=myimage1}
{title=\x[myimage2][my content 2].}
\Image[myimg.jpg]
{id=myimage2}
{title=\x[myimage1].}
``
or:
``
\Image[myimg.jpg]
{id=myimage1}
{title=\x[myimage2].}
\Image[myimg.jpg]
{id=myimage2}
{title=\x[myimage1][my content 1].}
``
A closely related limitation is the simplistic approach to \x[x-id-output-format].
= Internal cross file reference
{parent=internal-cross-reference}
Reference to the first header of another file:
\OurbigbookExample[[
\x[not-readme]
]]
Reference to a non-first header of another file:
\OurbigbookExample[[
\x[h2-in-not-the-readme]
]]
To make toplevel links cleaner,if target header is the very first element of the other page, then the link does not get a fragment, e.g.: `\x[not-readme]` rendered as:
``
``
and not:
``
``
while `\x[h2-in-not-the-readme]` is rendered with the fragment:
``
``
Reference to the first header of another file that is a second inclusion:
\OurbigbookExample[[
\x[included-by-not-readme]
]]
Reference to another header of another file, with \x[x-full-argument][`full`]:
\OurbigbookExample[[
\x[h2-in-not-the-readme]{full}.
]]
Note that when `full` is used with references in another file in \x[embed-includes][multi page mode], the number is not rendered as explained at: \x[x-full-argument-in-internal-cross-file-references]{full}.
Reference to an image in another file:
\OurbigbookExample[[
\x[image-not-readme-xi]{full}.
]]
Reference to an image in another file:
\OurbigbookExample[[
\x[image-figure-in-not-the-readme-without-explicit-id]{full}.
]]
Remember that the \x[the-toplevel-header][ID of the toplevel header] is automatically derived from its file name, that's why we have to use:
\OurbigbookExample[[
\x[not-readme]
]]
instead of:
``
\x[not-the-readme]
``
Reference to a subdirectory:
\OurbigbookExample[[
\x[subdir]
\x[subdir/h2]
\x[subdir/notindex]
\x[subdir/notindex-h2]
]]
Implemented at: https://github.com/cirosantilli/ourbigbook/issues/116
Reference to an internal header of another file: \x[h2-in-not-the-readme]. By default, That header ID gets prefixed by the ID of the top header.
When using \x[embed-includes] mode, the cross file references end up pointing to an ID inside the current HTML element, e.g.:
``
``
rather than:
``
``
This is why IDs must be unique for elements across all pages.
= Internal cross file reference internals
{parent=internal-cross-file-reference}
{title2=`db.sqlite3`}
= ID database
{c}
{synonym}
When running in Node.js, OurBigBook dumps the IDs of all processed files to a `out/db.sqlite3` file in \x[the-out-directory], and then reads from that file when IDs are needed.
When converting under a directory that contains \x[ourbigbook-json], `out/db.sqlite3` is placed inside the same directory as the `ourbigbook.json` file.
If there is no `ourbigbook.json` in parent directories, then `out/db.sqlite3` is placed in the current working directory.
These follows the principles described at: \x[the-current-working-directory-does-not-matter-when-there-is-a-ourbigbook-json].
`db.sqlite3` is not created or used when handling input from stdin.
When running in the browser, the same JavaScript API will send queries to the server instead of a local SQLite database.
To inspect the ID database to debug it, you can use:
``
sqlite3 out/db.sqlite3 .dump
``
It is often useful to dump a single table, e.g. to dump the `ids` table:
``
sqlite3 out/db.sqlite3 '.dump ids'
``
and one particularly important query is to dump a list of all known IDs:
``
sqlite3 out/db.sqlite3 'select id from ids'
``
You can force `ourbigbook` to not use the ID database with the \x[no-db] command line option
= Internal cross reference title link removal
{parent=internal-cross-file-reference}
If the target `title` argument contains a link from either another \x[internal-cross-reference]{p} or a regular \x[link][external hyperlink], OurBigBook automatically prevents that link from rendering as a link when no explicit body is given.
This is done because https://stackoverflow.com/questions/9882916/are-you-allowed-to-nest-a-link-inside-of-a-link[nested links are illegal in HTML], and the result would be confusing.
This use case is most common when dealing with media such as \x[image]{p}. For example in:
``
= afds
\x[image-aa-zxcv-lolol-bb]
== qwer
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=aa \x[zxcv][zxcv] \a[http://example.com][lolol] bb}
== zxcv
``
the `\x[image-aa-zxcv-lolol-bb]` renders something like:
``
aa zxcv lolol bb
``
and not:
``
aa zxcv lolol bb
``
Live example:
\OurbigbookExample[[
This is a nice image: \x[image-aa-zxcv-lolol-bb].
\Image[Tank_man_standing_in_front_of_some_tanks.jpg]
{title=aa \x[internal-cross-reference-title-link-removal][zxcv] \a[http://example.com][lolol] bb}
]]
= `\x` arguments
{parent=internal-cross-reference}
= `\x` `c` argument
{parent=x-arguments}
Capitalizes the first letter of the target title.
For more details, see: \x[cross-reference-title-inflection]{full}.
= `\x` `child` argument
{parent=x-arguments}
Setting the `child` \x[boolean-argument] on a cross reference to a header as in:
``
\x[my-header]{child}
``
makes that header show up on the list of extra parents of the child.
This allows a section to have multiple parents, e.g. to include it into multiple categories. For example:
``
= Animal
== Mammal
=== Bat
=== Cat
== Flying animal
These animals fly:
* \x[bat]{child}
These animals don't fly:
* \x[cat]
``
would render something like:
``
= Animal
== Mammal
=== Bat (Parent section: Mammal)
(Tags: Flying animal)
=== Cat (Parent section: Mammal)
== Flying animal (Parent section: Animal)
These animals fly:
* \x[bat]
These animals don't fly:
* \x[cat]
``
so note how "Bat" has a list of tags including "Flying animal", but Cat does not, due to the `child`.
This property does not affect how the \x[table-of-contents][`\Toc`] is rendered. We could insert elements sections there multiple times, but it has the downside that browser Ctrl + F searches would hit the same thing multiple times on the table of contents, which might make finding things harder.
``
== My title{id=my-id}
Read this \x[my-id][amazing section].
``
If the second argument, the `content`, is not present, it expand to the header title, e.g.:
``
== My title{id=my-id}
Read this \x[my-id].
``
is the same as:
``
== My title{id=my-id}
Read this \x[my-id][My title].
``
A live demo can be seen at: \x[x-child-argument-demo].
Generally, a better alternative to this argument is to use \x[h-child-argument].
= Secondary children
{parent=x-child-argument}
The term refers to sections that have a parent/child relationship via either of the:
* \x[x-child-argument]
* \x[x-parent-argument]
* \x[h-child-argument]
rather than via the usual \x[header] hierarchy.
Secondary children show up for example on the \x[tagged] metadata section, but not on the \x[table-of-contents], which is what the header hierarchy already shows.
Secondary children are normally basically used as "tags": a header such as `Bat` can be a direct child of `Mammal`, and a secondary child of `Flying animal`, or vice versa. Both `Mammal` and `Flying animal` are then basically ancestors. But we have to chose one main ancestor as "the parent", and other secondary ancestors will be seen as tags.
= `\x` `child` argument demo
{parent=x-child-argument}
{scope}
= Animal
{parent=x-child-argument-demo}
= Ant
{parent=animal}
= Cow
{parent=animal}
Oh, and cows are also \x[mammal]{parent}{p}.
= Mammal
{parent=animal}
= Bat
{parent=mammal}
= Cat
{parent=mammal}
= Flying animal
{parent=animal}
\x[bat]{child}{c}{p} can fly.
But \x[cat]{p} can't.
= `\x` `parent` argument
{parent=x-child-argument}
The `parent` argument is exactly like the \x[x-child-argument], but it reverses the direction of the parent/child relation.
= `\x` `full` argument
{parent=x-arguments}
To also show the section auto-generated number as in "Section X.Y My title" we add the optional `{full}` \x[boolean-argument] to the cross reference, for example:
\OurbigbookExample[[
\x[x-full-argument]{full}.
]]
`{full}` is not needed for cross references to most macros besides \x[header]{p}, which use `full` by default as seen by the `default_x_style_full` macro property in \x[help-macros]. This is for example the case for \x[image]{p}. You can force this to be disabled with `{full=0}`:
\OurbigbookExample[[
Compare \x[image-my-test-image]{full=0} vs \x[image-my-test-image]{full=1}.
]]
= `\x` `full` argument in internal cross file references
{parent=x-full-argument}
For example in the following \x[internal-cross-file-reference]:
\OurbigbookExample[[
\x[h2-in-not-the-readme]{full}.
]]
we get just something like:
``
Section "h2 in not the readme"
``
instead of:
``
Section 1.2 "h2 in not the readme"
``
This is because the number "Section 1.2" might already have been used in the current page, leading to confusion.
= `\x` `p` argument
{parent=x-arguments}
Pluralizes or singularizes the last word of the target title.
For more details, see: \x[cross-reference-title-inflection]{full}.
= `\x` `ref` argument
{parent=x-arguments}
The `ref` argument of `\x` marks the link as reference, e.g.:
``
Trump said this and that.\x[donald-trump-access-hollywood-tape]{ref}
= Donald Trump Access Hollywood tape
``
renders something like:
``
Trump said this and that.*
``
This could currently be replicated without `ref` by just using:
``
Trump said this and that.\x[donald-trump-access-hollywood-tape][*]
``
but later on we might add more precise reference fields like the page of a book or date fetched as Wikipedia supports.
Implemented at: https://github.com/cirosantilli/ourbigbook/issues/137
= OurBigBook Markup syntax
{c}
{parent=ourbigbook-markup}
= Insane macro shortcuts
{parent=ourbigbook-markup-syntax}
= Insane syntax
{synonym}
= Insane
{synonym}
Certain commonly used macros have insane macro shortcuts that do not start with backslash (`\`).
Originally, \x[design-goals][Ciro wanted to avoid those], but they just feel too good to avoid.
Every insane syntax does however have an equivalent sane syntax.
The style recommendation is: use the insane version which is shorter, unless you have a specific reason to use the sane version.
= Macros with insane shortcut
{parent=insane-macro-shortcuts}
* \x[paragraph]{child}{full}: `\n\n` (double newline)
* \x[link]{child}{full}: `a http://example.com b` (space followed by `http://`)
* \x[mathematics]{child}{full}: `$`, described at: \x[insane-code-and-math-shortcuts]
* \x[code-block]{child}{full}: \c[[`]], described at: \x[insane-code-and-math-shortcuts]
* \x[list]{child}{full}: `* ` and indentation
* \x[table]{child}{full}: `|| `, `| ` and indentation
= Insane code and math shortcuts
{parent=macros-with-insane-shortcut}
The insane code and math shortcuts work very analogously and are therefore described together in this section.
The insane inline code syntax:
\OurbigbookExample[[
a `b c` d
]]
and is equivalent to the sane:
``
a \c[[b c]] d
``
The insane block code:
\OurbigbookExample[[
a
``
b c
``
d
]]
and is equivalent to the sane:
``
a
\C[[
b c
]]
d
``
= Insane macro shortcut extra arguments
{parent=macros-with-insane-shortcut}
Insane arguments always work by abbreviating:
* the macro name
* one or more of its positional arguments, which are fixed as either \x[literal-arguments][literal or non-literal] for a given insane construct
This means that you can add further arguments as usual.
For example, an insane code block with an id can be written as:
``
a `b c`{id=ef} g
``
because that is the same as:
\OurbigbookExample[[
a \c[b c]{id=ef} g
]]
So we see that the `b c` argument is the very first argument of `\c`.
Extra arguments must come after the insane opening, e.g. the following does not work:
``
a {id=ef}`b c` g
``
This restriction things easy to parse for humans and machines alike.
= Escapes in insane macro shortcuts
{parent=macros-with-insane-shortcut}
Literal backticks and dollar signs can be produced witha backslash escape as in:
\OurbigbookExample[[
a \` \$ b
]]
It is not possible to escape backticks (\c[[`]]) inside an insane inline code, or dollar signs (`$`) in insane math.
The design reason for that is because multiple backticks produce block code.
The upside is that then you don't have to escape anything else, e.g. backslashes (`\`) are rendered literally.
The only way to do it is to use the sane syntax instead:
\OurbigbookExample[[[
a \c[[b ` c]] d
a \m[[\sqrt{\$4}]] d
]]]
Within block code and math, you can just add more separators:
\OurbigbookExample[[
```
code with two backticks
``
nice
```
]]
= Macro argument syntax
{parent=ourbigbook-markup-syntax}
= Positional vs named arguments
{parent=macro-argument-syntax}
{title2=`[...]` vs `{key=...}`}
Every argument in OurBigBook is either positional or named.
For example, in a \x[header] definition with an ID:
``
= My asdf
{id=asdf qwer}
{scope}
``
which is equivalent to the \x[insane-macro-shortcuts][sane] version:
``
\H[1][My asdf]
{id=asdf qwer}
{scope}
``
we have:
* two positional argument: `[1]` and `[My asdf]`. Those are surrounded by square brackets `[]` and have no name
* two named arguments: `{id=asdf qwer}` and `{scope}`.
The first one has name `id`, followed by the separator `=`, followed by the value `asdf qwer`.
The separator `=` always is optional. If not given, it is equivalent to an empty value, e.g.:
``
{id=}
``
is the same as:
``
{id}
``
You can determine if a macro is positional or named by using \x[help-macros]. Its output contains something like:
``
"h": {
"name": "h",
"positional_args": [
{
"name": "level"
},
{
"name": "content"
}
],
"named_args": {
"id": {
"name": "id"
}
"scope": {
"name": "scope"
}
},
``
and so we see that `level` and `content` are positional arguments, and `id` and `scope` are named arguments.
Generally, positional arguments are few (otherwise it would be hard to know which is which is which), and are almost always used for a given element so that they save us from typing the name too many times.
The order of positional arguments must of course be fixed, but named arguments can go anywhere. We can even mix positional and named arguments however we want, although this is not advised for clarity.
The following are therefore all equivalent:
``
\H[1][My asdf]{id=asdf qwer}{scope}
\H[1][My asdf]{scope}{id=asdf qwer}
\H{id=asdf qwer}{scope}[1][My asdf]
\H{scope}[1]{id=asdf qwer}[My asdf]
``
Just like named arguments, positional arguments are never mandatory.
= Positional argument
{parent=positional-vs-named-arguments}
See: \x[positional-vs-named-arguments]{full}.
= Positional argument default values
{parent=positional-argument}
Most positional arguments will default to an empty string if not given.
However, some positional arguments can have special effects if not given.
For example, an anchor with the first positional argument present (the URL), but not the second positional argument (the link text) as in:
\OurbigbookExample[[
\a[http://example.com]
]]
has the special effect of generating automatic links as in:
``
\a[http://example.com][http://example.com]
``
This can be contrasted with named arguments, for which there is always a default value, notably for \x[boolean-argument]{p}.
See also: \x[link]{full}.
= Mandatory positional arguments
{parent=positional-argument}
Some positional arguments are required, and if not given OurBigBook reports an error and does not render the node.
This is for example the `level` of a \x[header].
These arguments marked with the `mandatory: true` \x[help-macros] argument property.
= Named argument
{parent=positional-vs-named-arguments}
See: \x[positional-vs-named-arguments]{full}.
= Boolean argument
{parent=macro-argument-syntax}
{child=a-relative-argument}
{child=h-child-argument}
{child=h-numbered-argument}
{child=h-splitdefault-argument}
{child=x-c-argument}
{child=x-p-argument}
{child=x-ref-argument}
To also show the section auto-generated number as in "Section X.Y My title" we add the optional `{full}` \x[boolean-argument] to the cross reference, for example:
Name arguments marked in \x[help-macros] as `boolean: true` must either:
* take no value and no `=` sign, in which case the value is implicitly set to `1`
* take value exactly `0` or `1`
* not be given, in which case a custom per-macro default is used. Tha value is the `default` from \x[help-macros], or `0` if such default is not given
For example, \x[x-full-argument][the `full` argument] of \x[internal-cross-reference]{p} is correctly written as:
\OurbigbookExample[[
\x[boolean-argument]{full}
]]
without the `=` sign, or equivalently:
\OurbigbookExample[[
\x[boolean-argument]{full=1}
]]
The `full=0` version is useful in the case of reference targets that unlike \x[header]{p} expand the title on the cross reference by default, e.g. \x[image]{p}:
\OurbigbookExample[[
\x[boolean-argument]{full=1}
]]
The name "boolean argument" is given by analogy to the https://stackoverflow.com/questions/16109358/what-is-the-correct-readonly-attribute-syntax-for-input-text-elements/24588427#24588427["boolean attribute" concept in HTML5].
= JavaScript interface for arguments
{c}
{parent=macro-argument-syntax}
The JavaScript interface sees arguments as follows:
``
function macro_name(args)
``
where args is a dict such that:
* optional arguments have the key/value pairs explicitly given on the call
* mandatory arguments have a key documented by the API, and the value on the call.
For example, the link API names its arguments `href` and `text`.
= Literal arguments
{title2=`[[...]]` and `{{key=...}}`}
{parent=macro-argument-syntax}
Arguments that are opened with more than one square brackets `[` or curly braces `{` are literal arguments.
In literal arguments, OurBigBook is not parsed, and the entire argument is considered as text until a corresponding close with the same number of characters.
Therefore, you cannot have nested content, but it makes it extremely convenient to write \x[code-block]{p} or \x[mathematics].
For example, a multiline code block with double open and double close square brackets inside can be enclosed in triple square brackets:
\OurbigbookExample[[[
A literal argument looks like this in OurBigBook:
\C[[
\C[
A multiline
code block.
]
]]
And another paragraph.
]]]
The same works for inline code:
\OurbigbookExample[[[
The program \c[[puts("]");]] is very complex.
]]]
Within literal blocks, only one thing can be escaped with backslashes are:
* leading open square bracket `[`
* trailing close square bracket `]`
The rule is that:
* if the first character of a literal argument is a sequence of backslashes (`\`), and it is followed by another argument open character (e.g. `[`, remove the first `\` and treat the other characters as regular text
* if the last character of a literal argument is a `\`, ignore it and treat the following closing character (e.g. `]`) as regular text
See the following open input/output pairs:
``
\c[[\ b]]
\ b\c[[\a b]]
\a b\c[[\[ b]]
[ b\c[[\\[ b]]
\[ b\c[[\\\[ b]]
\\[ b``
and close examples:
``
\c[[a \]]
a \\c[[a \]]]
a ]\c[[a \\]]]
a \]``
= Argument leading newline removal
{parent=macro-argument-syntax}
If the very first character of an argument is a newline, then that character is ignored if it would be part of a regular plaintext node.
For example:
``
\C[[
a
b
]]
``
generates something like:
``
a
b
``
instead of:
``
a
b
``
This is extremely convenient to improve the readability of code blocks and similar constructs.
The newline is however considered if it would be part of some \x[insane-macro-shortcuts]{p=0}. For example, we can start an \x[list][insane list] inside a \x[quotation]{p} as in:
\OurbigbookExample[[
\Q[
* a
* b
]
]]
where the insane list requires a leading newline `\n* ` to work. That newline is not ignored, even though it comes immediately after the `\Q[` opening.
= Argument newlines between arguments removal
{parent=macro-argument-syntax}
The macro name and the first argument, and two consecutive arguments, can be optionally separated by exactly one newline character, e.g.:
``
\H[2]
{scope}
[Design goals]
``
is equivalent to:
``
\H[2]{scope}[Design goals]
``
and this non-recommended mixed style:
``
\H[2]{scope}
[Design goals]
``
This allows to greatly improve the readability of long argument lists by having them one per line.
There is one exception to this however: inside an \x[header][insane header], any newline is interpreted as the end of the insane header. This is why the following works as expected:
``
== My header 2 `some code`
{id=asdf}
``
and the `id` gets assigned to the header rather than the trailing code element.
= Escape characters
{parent=macro-argument-syntax}
For \x[literal-arguments][non-literal macro arguments], you have to use a backslash to escape:
* `\`: backslashes start macros
* `\` and `\`: open and close \x[positional-vs-named-arguments][positional macro arguments]
* `\` and `\`: open and close \x[positional-vs-named-arguments][optional macro arguments]
* `$` (dollar sign): \x[insane-macro-shortcuts][insane macro shortcut] for \x[mathematics]
* \c[[`]] (backtick): \x[insane-macro-shortcuts][insane macro shortcut] for \x[code-block]{p}
The escape rules for literal arguments are described at: \x[literal-arguments]{full}.
This is good for short arguments of regular text, but for longer blocks like \x[code-block]{p} or \x[mathematics], you may want to use \x[literal-arguments]
= Macro argument property
{parent=macro-argument-syntax}
= `remove_whitespace_children`
{parent=macro-argument-property}
In HTML, certain elements such as `
- ` cannot have any `text` nodes in them, and any whitespace is ignored, see https://stackoverflow.com/questions/2161337/can-we-use-any-other-tag-inside-ul-along-with-li/60885802#60885802[].
A similar concept applies to OurBigBook, e.g.:
``
\Ul[
\L[aa]
\L[bb]
]
``
does not parse as:
``
\Ul[\L[aa]\L[bb]]
``
but rather as:
``
\Ul[\L[aa]\L[bb]]
``
because the `content` argument of `ul` is marked with `remove_whitespace_children` and automatically removes any whitespace children (such as a newline) as a result.
This also applies to consecutive sequences of \x[auto-parent] macros, e.g.:
``
\L[aa]
\L[bb]
``
also does not include the newline between the list items.
The definition of whitespace is the same as the ASCII whitespace definition of HTML5: ` \r\n\f\t`.
= `multiple` argument
{child=h-child-argument}
{parent=macro-argument-property}
By default, arguments can be given only once.
However, arguments with the `multiple` property set to `true` can be given multiple times, and each time the argument is given, the new value is appended to a list containing all the values.
An example is the \x[h-child-argument]{child}.
\x[internals-api][Internally], multiple is implemented by creating a new level in the \x[abstract-syntax-tree], and storing each argument separately under a newly generated dummy nodes as in:
``
AstNode: H
AstArgument: child
AstNode: Comment
AstArgument: content
AstNode: plaintext
AstNode: x
AstNode: Comment
AstArgument: content
AstNode: plaintext
AstNode: x
``
= OurBigBook Markup concepts
{parent=ourbigbook-markup}
These are shared concepts that are used across other sections.
= `auto_parent` macro property
{id=auto-parent}
{parent=ourbigbook-markup-concepts}
Some sequences of macros such as `l` from \x[list]{p} and `tr` from \x[table]{p} automatically generate implicit parents, e.g.:
``
\Ul[
\L[aa]
\L[bb]
]
``
parses exactly like:
``
\L[aa]
\L[bb]
``
The children are always added as arguments of the `content` argument of the implicit parent.
If present, the `auto_parent` macro property determines which auto-parent gets added to those macros.
= Block vs inline macros
{parent=ourbigbook-markup-concepts}
Every OurBigBook macro is either block or inline:
* a block macro is one that takes up the entire line when rendered
All block macros start with a capital letter, e.g. `\H` for \x[header]{p}.
* and an inline macro is one that goes inside of a line.
Every inline macro starts with a lowercase letter e.g. `\a` for \x[link]{p}.
Some macros have both a block and an inline version, and like any other macro, those are differentiated by capitalization:
* \x[mathematics]
* \x[code-block]{p}
* \x[comment]{p}
= Known URL protocols
{parent=ourbigbook-markup-concepts}
Certain common URL protocols are treated as "known" by OurBigBook, and when found they have special effects in some parts of the conversion.
The currently known protocols are:
* `http://`
* `https://`
Effects of known protocols include:
* \x[insane-link-parsing-rules]: mark the start of insane links
* \x[store-images-in-a-separate-media-repository]: mark an image `src` to ignore `provider`
= JavaScript case conversion
{c}
{parent=ourbigbook-markup-concepts}
Some parts of OurBigBook use "JavaScript case conversion".
This means that the conversion is done as if by the `toLowerCase`/`toUpperCase` functions.
The most important fact about those functions is that they do convert non-ASCII Unicode capitalization, e.g. between `É` and `é`:
* https://stackoverflow.com/questions/3590833/does-javascript-string-tolowercase-follow-unicode-standards-in-case-conversion
* https://stackoverflow.com/questions/929079/unicode-lowercase-characters
These conversions are also specified in the Unicode standard.
= OurBigBook CLI
{c}
{parent=ourbigbook}
= `ourbigbook` executable
{title2}
{synonym}
Convert a `.bigb` file to HTML and output the HTML to a file with the same basename without extension, e.g.:
``
ourbigbook hello.bigb
firefox hello.html
``
Files named `README.bigb` are automatically converted to `index.html` so that they will show on both GitHub READMEs and at the website's base address:
``
ourbigbook README.bigb
firefox hello.html
``
Convert all `.bigb` files in a directory to HTML files next to each corresponding `.bigb` file, e.g. `somefile.bigb` to `somefile.html`:
``
ourbigbook .
``
The HTML output files are placed right next to each corresponding `.bigb`.
The output file can be selected explicitly with: \x[outfile].
Output to stdout instead of saving it to a file:
``
ourbigbook --stdout README.bigb
``
In order to resolve \x[internal-cross-file-reference]{p}, this actually does two passes:
* first an ID extraction pass, which parses all inputs and dumps their IDs to the ID database
* then a second render pass, which uses the IDs in the ID database
Convert a `.bigb` file from stdin to HTML and output the contents of `` to stdout:
``
printf 'ab\ncd\n' | ourbigbook --body-only
``
= Index file
{parent=ourbigbook-cli}
The following basenames are considered "index files":
* `README.bigb`
* `index.bigb`
Those basenames have the following magic properties:
* the default output file name for an index file in HTML output is either:
* `index.html` when in the \x[project-toplevel-directory]. E.g. `README.bigb` renders to `index.html`. Note that GitHub and many other static website hosts then automatically hide the `index.html` part from the URL, so that your `README.bigb` hosted at `http://example.com` will be accessible simply under `http://example.com` and not `http://example.com/index.html`
* the name of the subdirectory in which it is located when not in the \x[project-toplevel-directory]. E.g. `mysubdir/index.bigb` outputs to `mysubdir.html`
Previously, we had placed the output in `mysubdir/index.html`, but this is not as nice as it makes GitHub pages produce URLs with a trailing slash as `mysubdir/`, which is ugly, see also: https://stackoverflow.com/questions/5948659/when-should-i-use-a-trailing-slash-in-my-url
* the default \x[the-toplevel-header][toplevel header] ID of an index files is derived from the parent directory basename rather than from the source file basename
= Project toplevel directory
{parent=index-file}
This directory is determined by first checking the presence of a \x[ourbigbook-json] file.
If a \x[ourbigbook-json] is found, then the project toplevel directory is the directory that contains that file.
* otherwise, if the input path is a descendant of the current working directory, then the current working directory is used, see also: \x[the-current-working-directory-does-not-matter-when-there-is-a-ourbigbook-json]
* otherwise, if the input path is a directory, it is used
* otherwise, the directory containing the input file is used
For example, consider the file following file structure relative to the current working directory:
``
path/to/notindex.bigb
``
In this case:
* if there is no `ourbigbook.json` file:
* if we run `ourbigbook .`: the toplevel directory is the current directory `.`, and so `notindex.bigb` has ID `path/to/notindex`
* if we run `ourbigbook path`: same
* if we run `ourbigbook path/to`: same
* if we run `ourbigbook path/to/notindex.bigb`: same
* if there is a `path/ourbigbook.json` file:
* if we run `ourbigbook .`: the toplevel directory is the current directory `.` because the `ourbigbook.json` is below the entry point and is not seen, and so `notindex.bigb` has ID `path/to/notindex`
* if we run `ourbigbook path`: the toplevel directory is the directory with the `ourbigbook.json`, `path`, and so `notindex.bigb` has ID `to/notindex`
* if we run `ourbigbook path/to`: same
* if we run `ourbigbook path/to/notindex.bigb`: same
= The toplevel index file
{parent=project-toplevel-directory}
This is the \x[index-file] present in the \x[project-toplevel-directory].
Being the toplevel index file has the following implications compared to other \x[index-file]{p}:
* its ID is derived from the header itself, not the directory, see also: \x[the-id-of-the-first-header-is-derived-from-the-filename]
= The current working directory does not matter when there is a `ourbigbook.json`
{parent=the-toplevel-index-file}
When the file or directory being converted has an ancestor directory with a `ourbigbook.json` file, then your current working directory does not have any effect on OurBigBook output. For example if we have:
``
/project/ourbigbook.json
/project/README.bigb
/project/subdir/README.bigb
``
then all of the following conversions produce the same output:
* directory conversion:
* `cd /project && ourbigbook .`
* `cd / && ourbigbook project`
* `cd project/subdir && ourbigbook ..`
* file conversion:
* `cd /project && ourbigbook README.bigb`
* `cd / && ourbigbook project/README.bigb`
* `cd project/subdir && ourbigbook ../README.bigb`
When there isn't a `ourbigbook.json`, everything happens as though there were an empty `ourbigbook.json` file in the current working directory. So for example:
* outputs that would be placed relative to inputs are still placed in that place, e.g. `README.bigb -> index.html` always stay together
* outputs that would be placed next to the `ourbigbook.json` are put in the current working directory, e.g. \x[the-out-directory]
Internally, the general philosophy is that the JavaScript API in \a[index.js] works exclusively with paths relative to the \x[project-toplevel-directory]. It is then up to callers such as \a[ourbigbook] to ensure that filesystem specifics handle the relative paths correctly.
= OurBigBook CLI options
{c}
{parent=ourbigbook-cli}
= `ourbigbook` executable options
{synonym}
= `--china`
{parent=ourbigbook-cli-options}
This is the most important option of the software.
It produces a copy of the HTML of https://cirosantilli.com/china-dictatorship to stdout.
The data is stored inside an NPM package, making it hard to censor that information, see also: https://cirosantilli.com/china-dictatorship#mirrors
Usage:
``
ourbigbook --china > china.html
firefox china.html
``
= `--dry-run`
{parent=ourbigbook-cli-options}
The `--dry-run` option is a good way to debug \x[publish][`--publish` option], as it builds the publish output files without doing any git commands that would be annoying to revert. So after doing:
``
./ourbigbook --dry-run --publish .
``
you can just go and inspect the generated HTML to see what would get pushed at:
``
cd out/publish/out/publish/
``
see also: \x[the-out-directory].
Inspiration: https://github.com/cirosantilli/linux-kernel-module-cheat/tree/6d0a900f4c3c15e65d850f9d29d63315a6f976bf#dry-run-to-get-commands-for-your-project
= `--dry-run-push`
{parent=dry-run}
Similar to \x[dry-run], but it runs all git commands except for `git push`, which gives a clearer idea of what `--publish` would actually do including the git operations, but without publishing anything:
``
./ourbigbook --dry-run --publish .
``
= `--embed-includes`
{parent=ourbigbook-cli-options}
Makes \x[include]{p} render the included content in the same output file as the include is located, instead of the default behaviour of creating links.
In addition to this:
* \x[internal-cross-file-reference]{p} are disabled, and the cross file ID database does not get updated.
It should be possible to work around this, but we are starting with the simplest implementation that forbids it.
The problem those cause is that the IDs of included headers show as duplicate IDs of those in the ID database.
This should be OK to start with because the more common use case with `--html-sinle-page` is that of including all headers in a single document.
Otherwise, `include` only adds the headers of the other file to the table of contents of the current one, but not the body of the other file. The ToC entries then point to the headers of the included external files.
You may want to use this option together with \x[embed-resources] to produce fully self-contained individual HTML files for your project.
= `--embed-resources`
{parent=ourbigbook-cli-options}
Embed as many external resources such as images and CSS as possible into the HTML output files, rather than linking to external resources.
The use case for this option is to produce a single HTML file for an entire build that is fully self contained, and can therefore be given to consumers and viewed offline, much like a PDF.
Examples of embeddings done:
* CSS and JavaScript are copy pasted in place into the HTML.
The default built-in CSS and JavaScript files used by OurBigBook (e.g. the KaTeX CSS \x[mathematics][used for mathematics]) are currently all automatically downloaded as NPM package dependencies to ourbigbook
Without `--embed-resources`, those CSS and JavaScript use their main cloud CDN URLs, and therefore require Internet connection to view the generated documents.
The embedded version of the document can be viewed offline however.
There is however a known bug: KaTeX fonts are not currently embedded, so math won't work properly. The situation is similar as for images, but a bit harder because we also need to fetch the blobs from the CSS, which is likely doable from Webpack:
* https://github.com/cirosantilli/ourbigbook/issues/157
* https://stackoverflow.com/questions/41657087/webpack-inline-font-with-url-loader
* https://stackoverflow.com/questions/35369419/how-to-use-images-in-css-with-webpack
Examples of embedding that could be implemented in the future:
* \x[image]{p} are downloaded if needed and embedded as `data:` URLs.
Doing this however has a downside: it would slow the page loading down. The root problem is that HTML was not designed to contain assets, and notably it doesn't have byte position indices that can tell it to skip blobs while parsing, and how to refer to them later on when they show up on the screen. This is kind of why EPUB exists: https://github.com/cirosantilli/ourbigbook/issues/158
Images that are managed by the project itself and already locally present, such as those inside the project itself or due to \x[ourbigbook-json/media-providers] usually don't require download.
For images linked directly from the web, we maintain a local download cache, and skip downloads if the image is already in the cache.
To re-download due to image updates, use either:
* `--asset-cache-update`: download all images such that the local disk timestamp is older than the HTTP modification date with https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/If-Modified-Since[`If-Modified-Since`]
* `--asset-cache-update-force`: forcefully redownload all assets
Keep in mind that certain things can never be embedded, e.g.:
* YouTube videos, since YouTube does not offer any download API
= `--generate`
{parent=ourbigbook-cli-options}
Generate one of the template repositories locally:
* `ourbigbook --generate default`: a good starter template that illustrates many key OurBigBook features
* https://github.com/cirosantilli/ourbigbook-template
* https://cirosantilli.com/ourbigbook-template
* `ourbigbook --generate min`: a minimal template that is still sane
* https://github.com/cirosantilli/ourbigbook-template-min
* https://cirosantilli.com/ourbigbook-template-min
* `ourbigbook --generate subdir`: a template in which OurBigBook source is located a subdirectory `docs/`:
* https://github.com/cirosantilli/ourbigbook-template-subdir
* https://cirosantilli.com/ourbigbook-template-subdir
This template illustrates that everything works exactly as if OurBigBook source were in the git repository toplevel.
This is a convenient setup for programming projects that want to use OurBigBook for their documentation without polluting their toplevel.
End users almost never want this, because it means that to have a sane setup you need to:
* install OurBigBook globally with `npm install -g ourbigbook`
* generate the template
* then install OurBigBook locally again with `npm install`
so maybe we should just get rid of that option and just ensure that we can provide an up-to-date working template for the latest relase.
For now we are keeping this as it is useful to automate the updating of templates during the \x[release-procedure].
= `--help-macros`
{parent=ourbigbook-cli-options}
You can get an overview of all macros in JSON format with:
``
ourbigbook --help-macros
``
= `--log`
{parent=ourbigbook-cli-options}
Give multiple times to enable a list of certain types of logs to stderr help debugging, e.g.:
``
./ourbigbook --log ast tokens -- README.bigb
``
Note that this follows https://github.com/tj/commander.js/tree/e0e723810357e915210af38ccf5098ffe1fb8e65#variadic-option[commander.js' insane variadic argumentso syntax], and thus the `--` is required above. If you want to omit it for a single value you have to add the `=` sign as in:
```
./ourbigbook --log=ast README.bigb
```
Values not documented in other sections:
* `ast`: the full final parsed \x[abstract-syntax-tree] as JSON
* `ast-simple`: a simplified view of the \x[abstract-syntax-tree] with one AstNode or AstArgument per line and showing only the most important fields
* `ast-pp-simple`: view snapshots of the various \x[abstract-syntax-tree] post process stages, more info at: \x[conversion-process-overview]
* `ast-inside`: print the AST from inside the `ourbigbook.convert` call before it returns.
This is useful to debug the program if `ourbigbook.convert` blows up on the next stages before returning.
* `db`: show database transactions done by OurBigBook, to help debug stuff like \x[internal-cross-file-reference]{p}
* `parse`: parsing steps
* `tokenize`: tokenization steps
* `tokens`: final parsed token stream
* `tokens-inside`: like `ast-inside` but for tokens.
Also adds token index to the output, which makes debugging the parser way easier.
= `--log headers`
{parent=log}
This nifty little option outputs to stderr what the header graph looks like!
It is a bit like a \x[table-of-contents] in your terminal, for when you need to have a look at the outline of the document to decide where to place a new header, but are not in the mood to open a browser or use the \x[browser-editor-with-preview].
Sample output excerpt for this document:
``
= h1 ourbigbook
== h2 1 quick-start
== h2 2 design-goals
=== h3 2.1 saner
=== h3 2.2 more-powerful
== h2 3 paragraphs
== h2 4 links
``
This option can also serve as a debug tool for header tree related features (confession: that was its original motivation!).
TODO
= `--log perf`
{parent=log}
print \x[performance] statistics to stderr, for example
``
./ourbigbook --log=perf README.bigb
``
could output:
``
perf start: 181.33060800284147
perf tokenize_pre: 181.4424349963665
perf tokenize_post: 318.333980999887
perf parse_start: 319.1866770014167
perf post_process_start: 353.5477180033922
perf post_process_end: 514.1527540013194
perf render_pre: 514.1708239987493
perf render_post: 562.834307000041
perf end: 564.0349840000272
perf convert_input_end 566.1234430000186
perf convert_path_pre_sqlite 566.1564619988203
perf convert_path_pre_sqlite_transaction 566.2528780028224
perf convert_path_post_sqlite_transaction 582.256645001471
perf convert_path_end 582.3469280004501
``
which shows how long different parts of the \x[conversion-process-overview][conversion process] took to help identify bottlenecks.
This option can also be useful to mark phases of the conversion to identify from which phase other logs are coming from, e.g. if we wanted to know which part of the conversion is making a ton of database requests we could run:
``
ourbigbook --log db perf -- README.bigb
``
and we would see the database requests made at each conversion phase.
Note that `--log perf` currently does not take sub-converts into account, e.g. \x[include] and \x[ourbigbookexample] both call the toplevel conversion function `convert`, and therefore go through all the conversion intervals, but we do not take those it account, and just dump them all into the same toplevel interval that they happen in, currently between `post_process_start` and `post_process_end`.
= `--no-db`
{parent=ourbigbook-cli-options}
Don't use the \x[id-database] during this run. This implies that the on-disk database is not read, and also not written to. Instead, a temporary clean in-memory database is used.
= `--no-html-x-extension`
{parent=ourbigbook-cli-options}
If not given, \x[internal-cross-reference]{p} render with the `.html` extension as in:
``
``
This way, those links will work when rendering locally to `.html` files which is the default behaviour of:
``
ourbigbook .
``
If given however, the links render without the `.html` as in:
``
``
which is what is needed for servers such as GitHub Pages, which automatically remove the `.html` extension from paths.
This option is automatically implied when publishing to targets that remove the `.html` extension such as GitHub pages.
= `--no-render`
{parent=ourbigbook-cli-options}
Only extract IDs to fill the \x[id-database], don't render. This saves time if you only want to render a single file which has references to other files without getting any errors.
= `--no-render-timestamp`
{parent=ourbigbook-cli-options}
OurBigBook stores the timestamp of the last sucessful:
* \x[conversion-process-overview][ID extraction]
* \x[conversion-process-overview][render]
for each file.
For ID extraction, we always skip the extraction if the filesystem timestamp of a source file is older than the last sucessful extraction.
For render:
* we also skip rendering by default when you invoke ourbigbook on a directory, e.g. `ourbigbook .`, as this greatly speeds up the interactive error fixing turnaround time
* we always re-render fully when you specify a single file, e.g. `ourbigbook path/to/README.bigb`
However, note that skipping renders, unlike for ID extraction, can lead to some outdated pages.
This option disables the timestamp skip for rendering, so ensure that you will get a fully clean updated render.
E.g. consider if you had two files:
file1.bigb
``
= File 1
== File 1 1
``
file2.bigb
``
= File 2
== File 2 1
\x[file-1-1]
``
We then do the initial convertion:
``
ourbigbook .
``
we see output like:
``
extract_ids file1.bigb
extract_ids file1.bigb finished in 45.61287499964237 ms
extract_ids file2.bigb
extract_ids file2.bigb finished in 15.163879998028278 ms
render file1.bigb
render file1.bigb finished in 23.21016100049019 ms
render file2.bigb
render file2.bigb finished in 25.92908499762416 ms
``
indicating full conversion without skips.
But then if we just modify fil1.bigb as:
``
= File 1
== File 1 1 hacked
{id=file-1-1}
``
the following conversion with `ourbigbook .` would look like:
``
extract_ids file1.bigb
extract_ids file1.bigb finished in 45.61287499964237 ms
extract_ids file2.bigb
extract_ids file2.bigb skipped by timestamp
render file1.bigb
render file1.bigb finished in 41.026930000633 ms
render file2.bigb
render file2.bigb skipped by timestamp
``
and because we skipped `file2.bigb` render, it will still have the outdated "File 1 1" instead of "File 1 1 hacked".
We could in principle solve this problem by figuring out exactly which files need to be changed when a given ID changes, and we already \x[conversion-process-overview][have to solve a similar problem due to query bundling]. Also, this will need to be done sonner or later for the \x[ourbigbook-web]. But lazy now: https://github.com/cirosantilli/ourbigbook/issues/207[], this is hard stuff.
= `--outdir `
{id=outdir}
{parent=ourbigbook-cli-options}
Set a custom output directory for the conversion
If not given, the \x[project-toplevel-directory] is used.
Suppose we have an input file `./test.bigb`. Then:
``
ourbigbook --outdir my_outdir test.bigb
``
places its output at:
``
my_outdir/test.html
``
The same would happen if we instead did a full directory conversion as in:
``
ourbigbook --outdir my_outdir .
``
The output would also be placed in `my_outdir/test.html`.
This option also relocates \x[the-out-directory] to the target destination, e.g.:
``
ourbigbook --outdir my_outdir test.bigb
``
would generate:
``
my_outdir/out
``
This means that the source tree remains completely clean, and every output and temporary cache is put strictly under the selected `--outdir`.
= `--outfile `
{id=outfile}
{parent=ourbigbook-cli-options}
Save the output to a given file instead of outputting to stdout:
``
./ourbigbook --outfile not-readme.html not-readme.bigb
``
The generated output is slightly different than that of:
``
./ourbigbook not-readme.bigb > not-readme.html
``
because with `--outfile` we know where the output is going, and so we can generate relative includes to default CSS/JavaScript files.
= `-O --output-format `
{id=output-format}
{parent=ourbigbook-cli-options}
Default: `html`.
= `id` output format
{parent=output-format}
This output format is mostly a joke.
It is an intermediate step in \x[automatic-id-from-title], that unlike HTML output does not have any tags.
So for example, converting:
``
\i[asdf]
``
with the `id` output format produces simply:
``
asdf
``
instead of the HTML output:
``
asdf
``
This conversion type is useful in situations that users don't expect conversion to produce any HTML tags. For example, you could create a header:
``
= My \i[asdf]
``
and then following the \x[automatic-id-from-title] algorithm, that header would have the more commonly desired ID `my-asdf`, and not `my-asdf`.
Besides being more intuitive, this also guarantees greater format portability, in case we ever decide to support other output formats!
We decided to expose this format from the CLI just for fun, as it posed no extra work at all as it is treated internally exactly like any other conversion format.
And it also was a good start generalizing OurBigBook to multiple outputs, as this is a simple format.
The conversion is very simplistic, it basically just pastes the `content` of most macros. Important exceptions to that include:
* `\x`: see \x[x-id-output-format]
So for example if we had:
``
= Title with more
{id=title}
\Image[my.png]{title=My \i[nice] \x[title]
``
then the image ID would be `image-my-nice-title`:
* `image-` automatically added prefix
* `my `: literal text
* `nice`: extract the content of the \x[italic]
* `title`: note how we use the `href` `title` instead of what is actually rendered, `Title with more`, because `\x` is treated somewhat specially for performance reasons
Stuff that is primarily non-textual like \x[image]{p} is just completely removed. We could put effort in outputting their title correctly, but meh, not worth the effort.
= `\x` `id` output format
{parent=id-output-format}
`\x` uses `href` if the content is not given explicitly.
Previously, if `\x` didn't have a content, we were actually rendering the `\x` to calculate the ID. But then we noticed that doing so would require another parse pass, so we just went for this simpler approach. This is closely linked to \x[x-within-title-restrictions].
For example in:
``
= Animal
\x[image-i-like-dog]
\Image[dog.jpg]
{title=I like \x[dog]{p}}
== Dog hacked
``
note that the ID of the image is `image-i-like-dog`, i.e. `\x[dog]` is not expanded to `Dog hacked`, the `href` is used directly.
If you wanted `image-i-like-dog-hacked` instaead, you would need to explicitly give it as in:
``
= Animal
\x[image-i-like-dog-hacked]
\Image[dog.jpg]
{title=I like \x[dog][dog hacked]}
== Dog hacked
``
= Unimplemented output formats
{parent=id-output-format}
= `ourbigbook` output format
{parent=unimplemented-output-formats}
TODO: https://github.com/cirosantilli/ourbigbook/issues/83
= `latex` output format
{parent=unimplemented-output-formats}
TODO: https://github.com/cirosantilli/ourbigbook/issues/38
One day, one day. Maybe.
= `-p, --publish`
{id=publish}
{parent=ourbigbook-cli-options}
OurBigBook tooling is so amazing that we also take care of the HTML publishing for you!
Once publish target is properly setup, all you have to do is run:
``
git add README.bigb
git commit -m 'more content!'
ourbigbook --publish
``
and your changes will be published to the default target specified in \x[ourbigbook-json].
If not specified, the default target is \x[publish-to-github-pages].
Only changes committed to Git are pushed.
Files that `ourbigbook` knows how to process get processed and only their outputs are added to the published repo, those file types are:
* `.bigb` files are converted to `.html`
* `.scss` files are converted to `.css`
Every other Git-tracked file is pushed as is.
When `--publish` is given, stdin input is not accepted, and so the current directory is built by default, i.e. the following two are equivalent:
``
./ourbigbook --publish
./ourbigbook --publish .
``
Publishing only happens if the build has no errors.
= Publish to GitHub Pages
{parent=publish}
https://pages.github.com/[GitHub pages] is the default OurBigBook publish target.
Since that procedure is so important, it is documented directly at: \x[play-with-the-template].
= Publish to GitHub pages root page
{parent=publish-to-github-pages}
If you want to publish your root user page, which appears at `/` (e.g. https://github.com/cirosantilli/cirosantilli.github.io for the user `cirosantilli`), GitHub annoyingly forces you to use the `master` branch for the HTML output:
* https://github.com/isaacs/github/issues/212
* https://stackoverflow.com/questions/31439951/how-can-i-use-a-branch-other-than-master-for-user-github-pages
This means that you must place your `.bigb` input files in a branch other than `master` to clear up `master` for the generated HTML.
`ourbigbook` automatically detects if your repository is a root repository or not by parsing `git remote` output, but you must setup the branches correctly yourself.
So on a new repository, you must https://stackoverflow.com/questions/42871542/how-to-create-a-git-repository-with-the-default-branch-name-other-than-master[first checkout to a different branch] as in:
``
git init
git checkout -b dev
``
or to move an existing repository to a non-master branch:
``
git checkout -b dev
git push origin dev:dev
git branch -D master
git push --delete origin master
``
You then will also want to set your default repository branch to `dev` in the settings for that repository: https://help.github.com/en/github/administering-a-repository/setting-the-default-branch
= `-P, --publish-commit `
{id=publish-commit}
{parent=ourbigbook-cli-options}
Like \x[publish], but also automatically:
* `git add -u` to automatically add change to any files that have been previously git tracked
* `git commit -m ` to create a new commit with those changes
This allows you to publish your changes live in a single command such as:
``
ourbigbook --publish-commit 'my amazing change' .
``
With great power comes great responsibility of course, but who cares!
= `-S`, `--split-headers`
{parent=ourbigbook-cli-options}
{id=split-headers}
Split each header into its own separate HTML output file.
This option allows you to keep all headers in a single file, which is much more convenient than working with a billion separate source files, and let them grow naturally as new information is added, but still be able to get a small output page on the rendered website that contains just the content of the given header. Such split pages load faster on the browser, and might get better Google PageRank.
For example given an input file called `hello.bigb` and containing:
``
= h1
h1 content.
A link to another section: \x[h1-1].
== h1 1
h1-1 content.
== h1 1 1
h1-1-1 content.
== h1 1 2
h1-1-2 content.
``
a conversion command:
``
ourbigbook --split-headers hello.bigb
``
would produce the following output files:
* `hello.html`: contains the entire rendered document as usual.
Remember that this is called `hello.html` instead of `h1.html` because \x[the-toplevel-header][the toplevel header ID is automatically derived from its filename].
Each header contains a on-hover link to the single-file split version of the header.
* `hello-split.html`: contains only the contents directly under `= h1`, but not under any of the subheaders, e.g.:
* `h1 content.` appears in this rendered output
* `h1-1-1` does not appear in this rendered output
The `-split` suffix can be customized with the \x[h-splitsuffix-argument] option.
The `-split` suffix is appended in order to differentiate the output path from `hello.html`
* `h1-1.html`, `h1-1-1.html`, `h1-1-2.html`: contain only the contents direcly under their headers, analogously to `hello-split.html`, but now we don't need to worry about the input filename and collisiont, and just directly use the ID of each header
`--split-headers` is implied by \x[publish]: the published website will automatically get the split pages. There is no way to turn it off currently. A pull request would be accepted, especially if it offers a \x[ourbigbook-json] way to do it. Maybe it would be nice to have a more generalized way of setting any CLI option equivalent from the `ourbigbook.json`, and an option `cli` vs `cli-publish` so that `cli-publish` is publish only. Just lazy for now/not enough pressing use case met.
= Internal cross reference targets in split headers
{parent=split-headers}
By defeault, all internal cross references point to the non-split version of headers, including those found in split headers.
The rationale for this is that it gives readers the most context around the header by simply scrolling.
For example, considering the example document at \x[split-headers], \x[internal-cross-reference]{p} such as `\x[h1-1]` would point:
* from the non-split `hello.html` to the section in the current non-split file `#h1-1`
* from split `hello-split.html` to the same section in non-split file with `hello.html#h1-1`
The same applies to \x[internal-cross-file-reference]{p} when there are multiple input files.
In order to make the split version be the default for some headers, you can use the \x[h-splitdefault-argument].
= `--template`
{parent=ourbigbook-cli-options}
Select a custom https://github.com/harttle/liquidjs[Liquid template] file for the output.
The recommended file name for this file is:
``
main.liquid.html
``
which is already ignored by default from the published output.
This repository has a sample `main.liquid.html` at: \a[main.liquid.html].
If not given, the default template at one point was:
``
{{ body }}
``
This will get out of sync sooner or later with the code, but this should still serve as a good base example for this documentation.
Defined variables:
* `body`: the rendered body
* `git_sha`: SHA of the latest git commit of the source code if in a git repository
* `html_ext`: `.html` for local renders, empty for server renders.
So e.g. to link to an ID `myid` you can use:
``
``
This will ideally be replaced with a more generic link to arbitrary ID mechnism at some point: https://github.com/cirosantilli/ourbigbook/issues/135
* `root_page`: relative path to the toplevel page, e.g. either `index.html`, `../index.html` locally or `./`, `../` on server oriented rendereing
* `root_replath`: relative path from the rendered output to the toplevel directory.
This allows for toplevel resources like CSS to be found seamlessly form inside subdirectories, specially when rendering locally.
For example, for the toplevel \x[css] `main.css` which is generated from \a[main.scss], we can use:
``
``
Then, when a file is locally, for example under a subdirectory `mysubdir/myfile.html`, OurBigBook will set:
``
root_relpath=../
``
giving the desired:
``
``
And if the output path were instead just `myohterfile.html`, `root_replath` expands to an empty string, giving again the correct:
``
``
This will ideally be replaced with a more generic link to arbitrary ID mechnism at some point: https://github.com/cirosantilli/ourbigbook/issues/135
* `style`: default OurBigBook stylesheets
* `title`
We pick Liquid because it is server-side safe: if we ever some day offer a compilation service, Liquid is designed to prevent arbitrary code execution and infinite loops in templates.
= `--unsafe-ace`
{parent=ourbigbook-cli-options}
{tag=security}
This option enables actions that would allow \x[arbitrary-code-execution], so you should only pass it if you trust the repository author. Enabled functionality includes:
* \x[ourbigbook-json/prepublish]
= `-w`, `--watch`
{parent=ourbigbook-cli-options}
{id=watch}
Don't quit `ourbigbook` immediately.
Instead, watch the selected file or directory for changes, and rebuild individual files when changes are detected.
Watch every `.bigb` file in an entire directory:
``
ourbigbook --watch .
``
When a directory is given as the input path, this automatically first does an ID extraction pass on all files to support \x[internal-cross-file-reference]{p}.
Now you can just edit any OurBigBook file such has `README.bigb`, save the file in your editor, and refresh the webpage and your change should be visible, no need to run a `ourbigbook` command explicitly every time.
Exit by entering Ctrl + C on the terminal.
Watch a single file:
``
ourbigbook --watch README.bigb
``
When a single file is watched, the reference database is not automatically updated. If it is not already up-to-date, you should first update it with:
``
ourbigbook .
``
otherwise you will just get a bunch of undefined ID errors every time the input file is saved.
TODO: integrate Live Preview: https://asciidoctor.org/docs/editing-asciidoc-with-live-preview/ to also dispense the browser refresh.
= `-W`, `--web`
{parent=ourbigbook-cli-options}
{id=web}
Sync local directory to \x[ourbigbook-web] instead of doing anything else.
The source code is uploaded, and conversion to HTML happens on the server, no conversion is done locally.
= `--web-email`
{parent=ourbigbook-cli-options}
Set the email for \x[web] from the command line. If not given, you will be prompted for it from the command line.
= `--web-url`
{parent=ourbigbook-cli-options}
Set a custom URL for \x[web] from the command line. If not given, the canonical https://ourbigbook.com is used. This option is used e.g. for testing locally e.g. with:
``
ourbigbook --web-url http://localhost:3000
``
= `ourbigbook.json`
{parent=ourbigbook-cli}
{scope}
OurBigBook configuration file that affects the behaviour of ourbigbook for all files in the directory.
`ourbigbook.json` not used for input from stdin, since we are mostly doing quick tests in that case.
While `ourbigbook.json` is optional, it is used to determine the toplevel directory of a OurBigBook project, which has some effects such as those mentioned at \x[the-toplevel-index-file].
Therefore, it is recommended that you always have a `ourbigbook.json` in your project's toplevel directory, even if it is going to be an empty JSON containing just:
``
{}
``
For example, if you convert a file in a subdirectory such as:
``
ourbigbook subdir/notindex.bigb
``
then `ourbigbook` walks up the filesystem tree looking for `ourbigbook.json`, e.g.:
* is there a `./subdir/ourbigbook.json`?
* otherwise, is there a `./ourbigbook.json`?
* otherwise, is there a `../ourbigbook.json`?
* otherwise, is there a `../../ourbigbook.json`?
and so on.
If we reach the root path `/` and no `ourbigbook.json` is found, then we understand that there is no `ourbigbook.json` file present.
= `ignore`
{parent=ourbigbook-json}
List of paths relative to the \x[project-toplevel-directory] that \x[ourbigbook-executable] will ignore.
Useful if your project has a large directory that does not contain OurBigBook sources, and you don't want OurBigBook to mess with it.
Only ignores recursive conversions, e.g. given:
``
"ignore": [
"web"
]
``
doing:
``
ourbigbook .
``
skips that directory, but
``
ourbigbook web/myfile.bigb
``
converts it because it was explicitly requested.
TODO: also ignore during \x[watch].
= `ourbigbook.json` `id`
{id=id}
{parent=ourbigbook-json}
Dictionary of options that control \x[automatic-id-from-title] generation.
= `id` `normalize` `latin`
{parent=id}
If `true`, does \x[ourbigbook-json/latin-normalization] on the title.
Default: `true`.
= Latin normalization
{c}
{parent=id-normalize-latin}
ASCII normalization is a custom OurBigBook defined normalization that converts many characters that look like latin characters into latin characters.
For now, we are using the `deburr` method of Lodash: https://lodash.com/docs/4.17.15#deburr[], which only affects latin-like characters.
One notable effect is that it converts variants of ASCII letters to ASCII letters. E.g. `é` to `e` removing the accent.
This operation is kind of a superset of Unicode normalization acting only on Latin-like characters. where unicode basically only removes things like diactricts.
OurBigBook normalization also does other natural transformations that Unicode does not do, e.g. `æ` to `ae`.
Bibliography:
* https://stackoverflow.com/questions/990904/remove-accents-diacritics-in-a-string-in-javascript
* https://github.com/cirosantilli/ourbigbook/issues/162
= `id` `normalize` `punctuation`
{parent=id}
If `true`, does \x[punctuation-normalization] on the title.
Default: `true`.
= Punctuation normalization
{c}
{parent=id-normalize-punctuation}
Some selected punctuation marks are automatically converted into their dominant corresponding pronunciations. These are:
* `+`: `plus`
* `&`: `and`
Dashes are added around the signs if needed, e.g.:
* `C++`: `c-plus-plus`
* `Q&A`: `q-and-a`
= `lint`
{parent=ourbigbook-json}
Dictionary of lint options to enable. OurBigBook tries to be strict about forcing specific styles by default, e.g. forbids triple newline paragraph. But sometimes we just can't bear it :-)
= `lint` `h-parent`
{parent=lint}
Possible values:
* `parent`: forces headers to use \x[h-parent-argument] to specify their level
* `number`: forces headers to not use \x[h-parent-argument] to specify their level, i.e. to use a number or a number of `=`
You should basically always set either one of those on any serious project. Forgetting a `parent=` in a project that uses `parent=` everywhere else is a common cause of build bugs, and can be hard to debug without this type of linting enabled.
= `lint` `h-tag`
{parent=lint}
Possible values:
* `child`: forbids \x[header]{p} from using the \x[h-tag-argument]. They should instead use the \x[h-child-argument].
* `tag`: forbids \x[header]{p} from using the \x[h-child-argument]. They should instead use the \x[h-tag-argument].
= `h`
{parent=ourbigbook-json}
{scope}
This dictionnary stores options related to headers.
= `numbered`
{parent=h}
Sets the default \x[h-numbered-argument] argument of the toplevel headers of each source file.
Note that since the option is inherited by descendants, this can also affect the rendering of ancestors.
https://github.com/cirosantilli/ourbigbook/issues/188 contains a proposal to instead inherit this property across \x[include]{p}.
= `splitDefault`
{parent=h}
This options is exactly analogous to the \x[numbered] option, but it affects the \x[h-splitdefault-argument] instead of the \x[h-numbered-argument].
= `splitDefaultNotToplevel`
{parent=h}
If given, the the toplevel output of each input source is always non-split, and a split version if not generated at all.
This overrides the \x[h-splitdefault-argument] for the toplevel header.
E.g.:
ourbigbook.json
``
{
"h": {
"splitDefault": true,
"splitDefaultNoToplevel": true,
}
}
``
my-first-header.bigb
``
= My first header
== My second header
``
When converted with:
``
ourbigbook --split-headers my-first-header.bigb
``
woule lead only to two output files:
* my-first-header: not split
* my-second-header: split
Without `splitDefaultNoToplevel` we would instead have:
* my-first-header: split
* my-first-header-nosplit: not split
* my-second-header: split
The initial use case for this was in \x[ourbigbook-web]. If we didn't do this, then there would be two versions of every article at the toplevel of a file: split and nosplit.
This would be confusing for users, who would e.g. see two new articles on the article index every time they create a new one.
It would also mean that metadata such as comments would be visible in two separate locations.
So instead of filtering the duplicate articles on every index, we just don't generate them in the first place.
= `media-providers`
{parent=ourbigbook-json}
The `media-providers` entry of `ourbigbook.json` specifies properties of how media such as \x[image]{p} and \x[video]{p} are retrieved and rendered.
The general format of `media-providers` looks like:
``
"media-providers": {
"github": {
"default-for": ["image"], // "all" to default for both image, video and anything else
"path": "data/media/", // data is gitignored, but should not be nuked like out/
"remote": "cirosantilli/ourbigbook-media"
},
"local": {
"default-for": ["video"],
"path": "media/",
},
"youtube": {}
}
``
Properties that are valid for every provider:
* `default-for`: use this provider as the default for the given types of listed macros.
The first character of the macros are case insensitive and must be given as lower case. Therefore e.g.:
* `image` applies to both `image` and `Image`
* giving `Image` is an error because that starts with an upper case character
* `title-from-src` (`bool`): extract the `title` argument from the `src` by default for media such as \x[image]{p} and \x[video]{p} as if the `titleFromSrc` macro argument had been given, see also: \x[image-id]{full}
Direct children of media-providers and subproperties that are valid only for them specifically:
* `local`: tracked in the current Git repository as mentioned at \x[store-images-inside-the-repository-itself]{full}
* `path`: location of the cloned local repository relative to the root the main repository
* `github`: tracked in a separate Git repository as mentioned at \x[store-images-in-a-separate-media-repository]{full}
* `path`: analogous to `path` for `local`: a local location for this GitHub provider, where the repository can optionally be cloned.
TODO implement the amazing behaviour described below:
If this directory exists then local renders automatically use it instead of linking to the GitHub resources, therefore allowing you to develop locally without the internet and Still see images.
Then, during deployment, the GitHub version is used on the render regardless. Furthermore, we also automatically `git push` this repository during deployment to ensure that any asset changes will be available.
This path is ignored from OurBigBook conversion as if added to \x[ourbigbook-json/ignore], and is not added to the final output, because you are already going to have a copy of it.
The sanest approach is generally to track this directory as a Git submodule as mentioned at: \x[store-images-in-a-separate-media-repository-and-track-it-as-a-git-submodule]
* `remote`: `/`
* `youtube`: YouTube \x[video]{p}
See also: https://github.com/cirosantilli/ourbigbook/issues/40
= `outputOutOfTree`
{parent=ourbigbook-json}
Default: `false`
If `true` place the HTML output under \x[the-out-directory] at `out/html`.
For example:
``
ourbigbook hello.bigb
``
would be placed under:
``
out/html/hello.html
``
instead of `./hello.html`.
Advantages:
* the source tree becomes cleaner, especially when using \x[split-headers] which can produce hundreds of output files from a single input file
* if you want to track several `.html` source files in-tree, you don't need to add an exception to each of of them on the `.gitignore` as:
``
*.html
!/main.liquid.html
``
Disadvantages:
* you have to type more to open each output file on the terminal
Generally we recommend starting out without this option, and enabling it as the project grows and you feel like the `.html` floating around in-tree are starting to annoy you too much.
This option is always forced to `false` when \x[outdir] is given.
Implemented at: https://github.com/cirosantilli/ourbigbook/issues/163
= `prepublish`
{parent=ourbigbook-json}
Path of a script that gets executed before running \x[publish][`ourbigbook --publish`].
The script arguments are:
* the publish output directory.
That directory is guaranteed to exist when `prepublish` is called.
For `git`-based publish targets, all files are almost ready in there, just waiting for a `git add .` that follows `prepublish`.
This means that you can use this script to place or remove files from the final publish output.
If the `prepublish` script returns with a non-zero exit value, the publish is aborted.
= `redirects`
{parent=ourbigbook-json}
Generate custom redirects.
When dealing with regular \x[header]{p}, you generally don't want to use this option, because the \x[h-synonym-argument] does that for you.
This option can be useful however for dealing with things that are outside of your OurBigBook project.
For example, at one point, this project renamed the repository https://github.com/cirosantilli/cirodown[] to https://github.com/cirosantilli/ourbigbook[].
Unfortunatly, GitHub Pages does not generate redirects like github.com itself.
So in this case, we've added to the \x[ourbigbook-json] of the toplevel user repository https://github.com/cirosantilli/cirosantilli.github.io the lines:
``
"redirects": [
["cirodown", "ourbigbook"]
],
``
which produces a file in the output called `cirodown.html` that redirects to `ourbigbook.html`.
In this case, `cirodown` and `ourbigbook` don't have to be any regular IDs present in the database, those strings are just used directly.
TODO ideally we should check for conflicts with regular output from split headers IDs or their synonyms. But lazy.
= OurBigBook Library
{c}
{parent=ourbigbook}
= JavaScript API
{c}
{title2}
{synonym}
The main entry point for the JavaScript API is the `ourbigbook.convert` function.
An example can be seen under \a[lib_hello.js].
Note that while doing a simple conversion is easy, things get harder if you want to take multifile features in consideration, notably \x[internal-cross-file-reference-internals].
This is because these features require interacting with the \x[id-database], and we don't do that from the default `ourbigbook.convert` API because different deployments will have very different implementations, notably:
* local Node.js run uses SQLite, an implementation can be seen in the \a[ourbigbook] file class `SqliteIdProvider` and `SqliteFileProvider`
* the in-browser version that runs in the browser editor of the \x[ourbigbook-web] makes API calls to the server
= OurBigBook Web
{c}
{parent=ourbigbook}
{title2=OurBigBook.com}
= Dynamic website
{title2}
{synonym}
OurBigBook Web is the project that powers https://OurBigBook.com[]. Some more motivation is available at: https://cirosantilli.com/ourbigbook-com[].
OurBigBook Web is a regular databased backed dynamic website. This is unlike the static websites generated by \x[ourbigbook-cli]. Static websites are simpler and cheaper to run, but they are harder to setup for non-programmers, and they cannot have multiuser features such as likes, comments, and "view versions of this article by other users", which is are core functionality of the OurBigBook Project.
The source for OurBigBook Web, source code is located under \a[web/]. OurBigBook Web can be seen as a separate Node.js package which uses the \x[ourbigbook-library] as a dependency.
OurBigBook Web was originally forked from the following starter boilerplate: https://github.com/cirosantilli/node-express-sequelize-realworld-example-app[]. We are trying to keep tech synced as much as possible between both projects, since the boilerplate is useful as a tech demo to quickly try out new technologies in a more minimal setup.
= Local development server
{parent=ourbigbook-web}
It is highly recommended that you use the exact same Node.js and NPM versions as given under \a[package.json] `engines.js` entry. The best way to do that is likely to use NVM as explained at: https://stackoverflow.com/questions/16898001/how-to-install-a-specific-version-of-node-on-ubuntu/47376491#47376491
First time setup:
``
cd ourbigbook &&
npm run link &&
npm run build-assets &&
cd web/ &&
npm install &&
./bin/generate-demo-data.js --users 2 --articles-per-user 10
# Or short version:
#./bin/generate-demo-data.js -u 2 -a 10
``
where:
* `npm run build-assets` needs to be re-run if any assets (e.g. CSS or Js file mentioned at \x[overview-of-files-in-this-repository]) on the `./ourbigbook/` toplevel are modified. No need to re-run it for changes under `web/`.
* \a[web/bin/generate-demo-data.js] also creates the database and is not optional. If you want to start with an empty database instead of the demo one, you can run instead \a[web/bin/sync-db.js]:
``
./bin/sync-db
``
We also provide a shortcut for that setup as:
``
npm run web-setup
./bin/generate-demo-data.js --users 2 --articles-per-user 10
``
After this initial setup, run the development server:
``
npm run dev
``
And the website is now running at http://localhost:3000[]. If you created the \x[demo-data], you can login with:
* email: `user0@mail.com`, `user1@mail.com`, etc.
* password: `asdf`
Custom demo user passwords can be set by exporting the `OURBIGBOOK_DEMO_USER_PASSWORD` variable, e.g.:
``
OURBIGBOOK_DEMO_USER_PASSWORD=qwer ./bin/generate-demo-data.js -u 2 -a 10
``
this is useful for production.
To run on a different port use:
``
PORT=3001 npm run dev
``
We also offer shortcuts on toplevel for the `npm install` and `npm run dev` commands so you can skip the `cd web` for those:
``
npm install
npm run dev
``
Whenever you save any changes to the backend server, we listen to this and automatically restart the server, so after a few seconds or less, you can refresh the web page to obtain the backend update.
For frontend, changes are automatically recompiled by the webpack development server, so you can basically just refresh pages and they will be updated straightaway.
= Demo data
{parent=ourbigbook-web}
You can generate demo data for \x[ourbigbook-web] with \a[web/bin/generate-demo-data.js], e.g.:
``
cd web
./bin/generate-demo-data --users 2 --articles-per-user 10
``
Every time this is run, it tries to update existing entities such as users and articles first, and only creates them if they don't exist. This allows us to update all demo data on a live website that also has users without deleting any user data.
Note however that if you ever increase the ammount of demo users, you might overwrite real user data. E.g. if you first do:
``
./bin/generate-demo-data --users 2 --articles-per-user 10
``
and then some time later:
``
./bin/generate-demo-data --users 4 --articles-per-user 10
``
it is possible that some real user will have taken up the username that we use for the third user, which did not exist previously, and then hacks their articles away. So never ever do that! Just stick to the default values in production.
As a safeguard, to be able to run this in production you have to also pass the `--force-production` flag;
``
./bin/generate-demo-data --users 2 --articles-per-user 10 --force-production
``
To first fully clear the database, including any real user data, before doing anything else, use `--clear`, e.g.:
``
./bin/generate-demo-data --users 4 --articles-per-user 10 --clear
``
To clear the database and start with an empty database use `--empty`:
``
./bin/generate-demo-data --empty
``
= Log database queries done
{parent=ourbigbook-web}
``
DEBUG='sequelize:sql:*' npm run dev
``
Shortcut:
``
npm run devs
``
= Local optimized frontend
{parent=ourbigbook-web}
First run the first time setup from \x[local-development-server].
Then, when running for the first time, or whenever frontend changes are made, you need to create optimized frontend assets with:
``
npm run build-dev
``
before you finally start the server each time with:
``
npm start
``
This setup runs the Next.js server in production mode locally. Running this setup locally might help debug some front-end deployment issues.
Building like this notably runs full typescript type checking, which is a good way to find bugs early.
But otherwise you will just normally use the \x[local-run-as-identical-to-deployment-as-possible] setup instead for development, as that makes iterations quicker are you don't have to re-run the slow `npm run build-dev` command after every frontend change.
`build-dev` is needed instead of `build` because it uses `NODE_ENV_OVERRIDE` which is needed because Next.js forces `NODE_ENV=production` and wontfixed changing it: https://github.com/vercel/next.js/issues/4022#issuecomment-374010365[], and that would lead to the PostgreSQL database being used, instead of the SQLite one we want.
`build` runs `npm run build-assets` on toplevel which repacks ourbigbook itself and is a bit slow. To speed things up during the development loop, you can also use:
``
npm run build-dev-nodeps
``
instead, which builds only the stuff under `web/`.
TypeScript type checking an also be run in isolation as mentioned at \x[ourbigbook-web-typescript-type-checking]{full} with:
``
npm run tsc
``
= Local run as identical to deployment as possible
{parent=ourbigbook-web}
Here we use PostgreSQL instead of SQLite with the prebuilt static frontend.
For when you really need to debug some deployment stuff locally.
Setup:
``
createdb ourbigbook
psql -c "CREATE ROLE ourbigbook_user with login password 'a'"
psql -c 'GRANT ALL PRIVILEGES ON DATABASE ourbigbook TO ourbigbook_user'
echo "SECRET=$(tr -dc A-Za-z0-9 > .env
``
Run after every modification
``
npm run build-prod
npm run start-prod
``
and then visit the running website at: http://localhost:3000/
To optionally nuke the database and create the demo data:
``
npm run seed-prod
``
or alternatively to start from a clean database:
``
psql -c "DROP DATABASE ourbigbook"
createdb ourbigbook
psql -c 'GRANT ALL PRIVILEGES ON DATABASE ourbigbook TO ourbigbook_user'
``
You can inspect the database interactively with:
``
psql ourbigbook
``
and then running SQL commands.
= Local development run with PostgreSQL
{parent=local-run-as-identical-to-deployment-as-possible}
If you have determined that a bug is PostgreSQL specific, and it is easier to debug it interactively, first create the database as mentioned at \x[local-run-as-identical-to-deployment-as-possible] and then:
``
OURBIGBOOK_POSTGRES=true ./bin/generate-demo-data.js
OURBIGBOOK_POSTGRES=true npm run dev
``
or shortcut for the run:
``
npm run dev-pg
``
Note that doing `sync-db` also requires `NODE_ENV=production` as in:
``
NODE_ENV=production OURBIGBOOK_POSTGRES=true ./bin/sync-db.js
``
because we have to shell out to the ugly migration CLI, and that only understands `NODE_ENV`.
= Heroku deployment
{c}
{parent=ourbigbook-web}
Got it running perfectly at as of April 2021 https://ourbigbook.com with the following steps.
Initial setup for a Heroku project called `ourbigbook`:
``
sudo snap install --classic heroku
heroku login
heroku git:remote -a ourbigbook
git remote rename heroku prod
# Automatically sets DATABASE_URL.
heroku addons:create -a ourbigbook heroku-postgresql:hobby-dev
# We need this to be able to require("ourbigbook")
heroku config:set -a ourbigbook SECRET="$(tr -dc A-Za-z0-9 # Password of users generated with ./web/bin/generate-demo-data
heroku config:set -a ourbigbook OURBIGBOOK_DEMO_USER_PASSWORD="$(tr -dc A-Za-z0-9 # You can get it later to login with the demo users from the Heroku web interface
``
Additionally, you also need to setup the PostgreSQL test database for both OurBigBook CLI and OurBigBook Web as documented at: \x[test-system] and \x[local-run-as-identical-to-deployment-as-possible], because we will be running tests on PostgreSQL before pushing:
``
createdb ourbigbook
createdb ourbigbook_cli
psql -c "CREATE ROLE ourbigbook_user with login password 'a'"
psql -c 'GRANT ALL PRIVILEGES ON DATABASE ourbigbook TO ourbigbook_user'
psql -c 'GRANT ALL PRIVILEGES ON DATABASE ourbigbook_cli TO ourbigbook_user'
``
Then deploy with:
``
cd web
npm run deploy-prod
``
Get an interactive shell on the production server:
``
heroku run -a ourbigbook bash
``
From there you could then for example update the \x[demo-data] with:
``
cd web
bin/generate-demo-data.js --force-production
``
This should in theory not affect any real user data, only the demo articles and users, so it might be safe. In theory!
Alternatively, we could do this at once with;
``
heroku run -a ourbigbook web/bin/generate-demo-data.js --force-production
``
Drop into a PostgreSQL shell on production:
``
heroku psql -a ourbigbook
``
Of course, any writes could mean loss of user data!
Run a query directly from your terminal:
``
heroku psql -a ourbigbook -c 'SELECT username,email FROM "User" ORDER BY "createdAt" DESC LIMIT 50'
``
If some spurious bugs crashes the server, you might want to restart it with:
``
heroku restart -a ourbigbook
``
Download a dump of the database as per https://devcenter.heroku.com/articles/heroku-postgres-import-export[]:
``
heroku pg:backups:capture -a ourbigbook
heroku pg:backups:download -a ourbigbook
``
Restore it to a local database:
``
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U ourbigbook_user -d ourbigbook latest.dump
``
The local password is `a` (single letter). TODO automate login.
= Custom domain name setup
{parent=heroku-deployment}
The domain `ourbigbook.com` was leased from: https://porkbun.com/[]. Unfortunatly, HTTPS on Heroku with a custom domain requires using a paying tier, so we upgraded from the free tier to the cheapest paid tier, "Hobby Project", to start with: https://stackoverflow.com/questions/52185560/heroku-set-ssl-certificates-on-free-plan
On the Porkbun web UI, we added a DNS record of type :
``
ALIAS ourbigbook.com .herokudns.com
``
where `heroku-id` was obtained from:
``
heroku domains:add ourbigbook.com
heroku domains
``
and we removed all other `ALIAS`/`CNAME` records from Porkbun.
= Staging deployment
{parent=heroku-deployment}
Before pushing any new changes, and especially ones that seem dangerous, it is a good idea to first deploy to a staging server.
We have a staging server running at: https://ourbigbook-staging.herokuapp.com/
To set it up, we just follow the exact same steps as for \x[heroku-deployment] but with a different app ID. E.g. using the `ourbigbook-staging` heroku project ID:
``
git remote add staging https://git.heroku.com/ourbigbook-staging.git
heroku addons:create -a ourbigbook-staging --confirm ourbigbook-staging heroku-postgresql:hobby-dev
heroku config:set -a ourbigbook-staging SECRET="$(tr -dc A-Za-z0-9 npm run deploy-staging
``
To copy the main database in staging we can follow the instructions at: https://stackoverflow.com/questions/10673630/how-do-i-transfer-production-database-to-staging-on-heroku-using-pgbackups-gett Considering a production Heroku app ID of `ourbigbook`:
``
heroku maintenance:on -a ourbigbook-staging &&
heroku pg:copy ourbigbook::DATABASE_URL DATABASE_URL -a ourbigbook-staging &&
heroku maintenance:off -a ourbigbook-staging
``
To get a shell on the stating server you can run:
``
heroku run -a ourbigbook-staging bash
``
= Dependency organization
{parent=heroku-deployment}
On the toplevel we have:
* `.`: OurBigBook package
* `web/`: \x[ourbigbook-web] package that depends on the local OurBigBook package through relative path `..`
Every require outside of `web/` must be relative, except for executables such as \a[ourbigbook] or demos such as \a[lib_hello.js], or else the deployment will break.
This is because we don't know of a super clean way of adding the toplevel `ourbigbook` package to the search path as `npm run link` does not work well on Heroku.
A known workaround to allow `npm run build-assets` is done at: \a[web/build.sh].
Currently, Heroku deployment does the following:
* install both `dependencies` and `devDependencies`
* `npm run build`
* remove `devDependencies` from the final output to save space and speed some things up
The `devDependencies` should therefore only contain things which are needed for the build, typically asset compressors like Webpack, but not components that are required at runtime.
This setup creates some conflict between what we want for OurBigBook command line users, and Heroku deployment.
Notably, OurBigBook command line users will want SQLite, and Heroku never, and SQLite installation is quite slow.
Since we were unable to find any way to make things more flexible on the `package.json` with some kind of optional depenency, for now we are just hacking out any dependencies that we don't want Heroku to install at all from \a[package.json] and \a[web/package.json] with `sed` rom \a[heroku-prebuild].
Further discussion at: https://github.com/cirosantilli/ourbigbook/issues/156
= OurBigBook Web tech stack
{parent=ourbigbook-web}
* Frontend
* Next.js
* React
* Frontend backend communication
* JSON API
* Next.js makes its prerender server-side queries directly to the database without going through the API
* Backend
* Express.js
* Sequelize
* SQLite for local development, PostgreSQL for deployment
= OurBigBook Web database migration setup
{parent=ourbigbook-web}
Any pending migrations are done automatically during deployment as part of `npm run build`, more precisely they are run from \a[web/bin/sync-db.js].
We also have a custom setup where, if the database is not initialized, we first:
* just creates the database from the latest model descriptions
* manually fill in the `SequelizeMeta` migration tracking table with all available migrations to tell Sequelize that all migrations have been done up to this point
This is something that should be merged into Sequelize itself, or at least asked on Stack Overflow, but lazy now.
= OurBigBook Web directory structure
{c}
{parent=ourbigbook-web}
* \a[web/app.js]: server executable entry point, can also be used programmatically from Node.js
* \a[web/models]: Sequelize database models. Most changes in this folder require the creation of a corresponding \a[web/migrations]
* \a[web/migrations]: database migrations
* \a[web/pages]: Next.js URL entry points
* \a[web/front] and \a[web/front.tsx]: files that can be imported from either front-end of backend. See: https://stackoverflow.com/questions/64926174/module-not-found-cant-resolve-fs-in-next-js-application/70363153#70363153
* \a[web/back] and \a[web/back.ts]: files that can be imported only from backend. See: https://stackoverflow.com/questions/64926174/module-not-found-cant-resolve-fs-in-next-js-application/70363153#70363153
* \a[web_api.js]: helpers to access the OurBigBook HTTP REST API. These have to be outside of `web/` because \x[ourbigbook-cli] uses them e.g. for syncing local files to the server, and OurBigBook CLI cannot depend on OurBigBook Web components, only the other way around, otherwise we could create circular dependencies. That exact same JavaScript code is also used from the front-end! The infinite joys of homomorphic JS.
= web/bin/rerender-articles.js
{file}
{parent=ourbigbook-web-directory-structure}
Rerender all articles.
This has to be done to see updates on OurBigBook changes that change the render output.
Notably, this would be mandatory in case of CSS changes that require corresponding HTML changes.
As the website grows, we will likely need to do a lazy version of this that marks pages as outdated, and then renders on the fly, plus a background thread that always updates outdated pages.
The functionality of this script should be called from a migration whenever such HTML changes are required. TODO link to an example. We had one at `web/migrations/20220321000000-output-update-ancestor.js` that seemed to work, but lost it. It was simple though. Just you have to instantiate your own
= OurBigBook Web help
{c}
{parent=ourbigbook-web}
This section will eventually be moved into the website itself. We'll just likely paste it into the database under an account that is always created or something like that.
= OurBigBook Web keyboard shortcuts
{parent=ourbigbook-web-help}
* Global:
* Ctrl + Enter: submit the current form, e.g. save/create artiles or comments, login, register
* TODO N: create a new article. This requires making sure that all input fields and textareas don't propagate N key events. We did that as a one off for E in comment textareas.
* Page specific:
* Article page (and for Index page on user page):
* E: edit the page
* L: like or unlike the page. This would require moving like state out of the Like button, which is a bit annoying.
= Step debug the dynamic website
{parent=ourbigbook-web}
Put a `debugger` statment where you want to break and run:
``
npm run devi
``
where `i` stands for `inspect` as in `node inspect`.
This pauses at the start of execution. So just run `c` and normal execution resumes until the `debugger;` statement is reached.
= OurBigBook Web unit tests
{parent=ourbigbook-web}
All our tests are all located inside \a[test.js].
They can be run with:
``
cd web
npm test
``
The dynamic website tests also uses Mocha just like \x[test-system][the tests for OurBigBook CLI and OurBigBook Library], so similar usage patterns apply, e.g. to run just a single test:
``
npm test -- -g 'substring of test title'
``
or to show database queries being done in the tests:
``
DEBUG='*:sql:*' npm test
``
To run the tests on PostgreSQL instead of SQLite, first setup as in \x[local-run-as-identical-to-deployment-as-possible], and then:
``
npm run test-pg
``
``
npm run deploy-prod
``
Note that this will erase all data present in the database used. In order to point to a custom database use:
``
DATABASE_URL_TEST=postgres://realworld_next_user:a@localhost:5432/realworld_next_test npm run test-pg
``
We don't use `DATABASE_URL` when running tests as a safeguard to reduce the likelihood of accidentally nuking the production database.
The tests include two broad classes of tests:
* API tests: launch the server on a random port, and run API commands, thus testing the entire backend
* smaller unit tests that only call certain functions directly
* TODO: create frontend tests: https://github.com/cirosantilli/node-express-sequelize-nextjs-realworld-example-app/issues/11
= OurBigBook Web Next.js unit tests
{parent=ourbigbook-web-unit-tests}
By default, we don't make any requests to Next.js, because starting up Next.js is extremely slow for regular test usage and would drive us crazy.
In regular OurBigBook Web usage through a browser, Next.js handles all GET requests for us, and the API only handles the other modifying methods like POST.
However, we are trying to keep the API working equally well for GET, and also factored out with Next.js, and that's what we use for the testing by default.
But testing Next.js requests before deployment is a must, and is already done by default by `npm run deploy-prod` from \x[heroku-deployment], and can be done manually with:
``
npm run test-next
``
or e.g. to run just a single test:
``
npm run test-next -- -g 'api: create an article and see it on global feed'
``
for for Postgres:
``
npm run test-pg-next
``
If you are not making any changes to the website itself, e.g. only to the test system, then you can skip the slow rebuild with:
``
test-next-nobuild
``
Note that Next.js tests are just present inside other tests, e.g. `api: create an article and see it on global feed` also tests some stuff when not testing Next.js. Running `npm run test-next` simply enables the Next.js tests on top of the non Next.js ones that get run by default.
These tests can only be run in production mode, and so our scripts automatically rebuild every time before running the tests, which makes things quite slow. This required because in development mode, Next.js is extremelly soft, and e.g. does not raise 500 instead returning a 200 page with error messages. Bad default.
= OurBigBook Web TypeScript type checking
{parent=ourbigbook-web}
TypeScript type checking of \x[ourbigbook-web] is run automatically during build, e.g. by:
``
npm run build-dev
``
as mentioned at \x[local-optimized-frontend].
To speed up the development loop further, you can run just the TypeScript type checking with:
``
cd web
nmp run typecheck
``
The output format is also a bit nicer that what is shown in `npm run build-dev`.
= Tooling
{parent=ourbigbook}
Unlike all languages which rely on ad-hoc tooling, we will support every single tool that is required and feasible to be in this repository in this repository, in a centralized manner.
= Conversion to/from other formats
{parent=tooling}
The only thing we have for now is the quick and dirty \a[adoc-to-ciro].
The better approach would be to implement a converter in Haskell from anything to OurBigBook.
And from OurBigBook to anything, create new output formats inside OurBigBook to those other formats.
= Editor support
{parent=tooling}
= Vim
{parent=editor-support}
You can install the support with https://github.com/VundleVim/Vundle.vim[Vundle] with:
``
set nocompatible
filetype off
set rtp+=~/.vim/bundle/Vundle.vim
call vundle#begin()
Plugin 'gmarik/vundle'
let g:snipMate = {}
let g:snipMate.snippet_version = 1
Plugin 'MarcWeber/vim-addon-mw-utils'
Plugin 'tomtom/tlib_vim'
Plugin 'garbas/vim-snipmate'
Plugin 'cirosantilli/ourbigbook', {'rtp': 'vim'}
``
or by directly dropping the files named below under your `~/.vim/`, e.g. `vim/syntax/ourbigbook.vim`
The following support is provided:
* \a[vim/syntax/ourbigbook.vim]: syntax highlighting.
As for other programming languages, this cannot be perfect without actually parsing, which would be slow for larger files.
But even the imperfect approximation already covers a lot of the most cases.
Notably it turns off spelling from parts of the document like URLs and code which would otherwise contain many false positive spelling errors.
* \a[vim/snippets/ourbigbook.snippets]: snippets for https://github.com/honza/vim-snippets[], which you also have to install first for them to work.
For example, with those snippets installed, you can easily create links to headers. Suppose you have:
``
= My long header
``
To create an internal cross reference to it you can:
* copy `My long header` to the clipboard, see copy to clipboard shortcuts at: https://stackoverflow.com/questions/3961859/how-to-copy-to-clipboard-in-vim/67890119#67890119
* type `\x` and then hit tab
and it will automatically expand to:
``
\x[my-long-header]
``
This provides a reasonable alternative for ID calculation, until a ctags-like setup gets implemented (never/\x[browser-editor-with-preview]-only? ;-))
Similarly for \x[h-parent-argument] you can do:
``
{p
``
would expand to:
``
{parent=my-long-header}
``
Syntax highlighting can likely never be perfect without a full parser (which is slow), but even the imperfect approximate setup (as provided for most other languages) is already a huge usability improvement.
We will attempt to err on the side of "misses some stuff but does not destroy the entire page below" whenever possible.
= Browser editor with preview
{parent=editor-support}
There are two versions of this editor:
* \a[editor.html] is a toy/demo with no backing database.
That editor can be viewed directly locally with:
``
git clone https://github.com/cirosantilli/ourbigbook
cd ourbigbook
npm install
npm run build-assets
chrome editor.html
``
It also appears at https://cirosantilli.com/ourbigbook/editor[] hosted simply under GitHub pages.
* a similar looking editor will also appear on the \x[ourbigbook-web], but this time linked to the database.
That more advanced editor will actually save results back to the database, and show allow preview of features such as linking to headers of other pages.
Issues for the editor are being tracked under: https://github.com/cirosantilli/ourbigbook/labels/editor
We must achieve an editor setup with synchronized live side-by-side preview.
Likely, we will first do a non WYSIWYG editor with side by side preview with scroll sync.
Then, if the project picks up steam, we can start considering a full WYSIWYG.
It would be amazing to have a WebKit interface that works both on browser for the and locally.
Possibilities we could reuse:
* Editor.js
Returns JSON AST!
* website: https://editorjs.io/
* source: https://github.com/codex-team/editor.js
* WYSIWYG: yes
* preview scroll sync: yes
* StackEdit
* markup implementation: PageDown
* website: https://stackedit.io
* source: https://github.com/benweet/stackedit
* demo: https://stackedit.io/app
* WYSIWYG: no
* preview scroll sync: yes
* Editor.md
* website: https://github.com/pandao/editor.md
* source: https://github.com/pandao/editor.md
* demo: https://pandao.github.io/editor.md
* WYSIWYG: no
* preview scroll sync: yes but buggy when tested 2019-12-12 on live website
* https://github.com/markdown-it/markdown-it[markdown-it]
Custom editor and highlight via https://github.com/highlightjs/highlight.js/[highlight.js].
* markup implementation: custom
* website: https://markdown-it.github.io
* source: https://github.com/markdown-it/markdown-it
* WYSIWYG: no
* preview scroll sync: yes
* editor hangs on large input: yes
* Quill.md
* website: https://quilljs.com
* source: https://github.com/quilljs/quill/
* demo: https://pandao.github.io/editor.md
* WYSIWYG: yes
* markdown output: no https://github.com/quilljs/quill/issues/74
* https://ui.toast.com/tui-editor/
* https://www.froala.com/wysiwyg-editor
= Error reporting
{parent=tooling}
A lot of effort has been put into making error reporting as good as possible in OurBigBook, to allow authors to quickly find what is wrong with their source code.
Error reporting is for example tested with `assert_error` tests in \a[test.js].
Please report any error reporting bug you find, as it will be seriously tracked under the: https://github.com/cirosantilli/ourbigbook/issues?q=label%3Aerror-reporting+[`error-reporting` label].
Notably, OurBigBook should never throw an exception due to a syntax error, as that prevents error messages from being output at all.
= Order of reported errors
{parent=error-reporting}
One important philosophy of the error reporting is that the very first message should be the root cause of the problem whenever possible: users should not be forced to search a hundred messages to find the root cause. In this way, the procedure:
* solve the first error
* reconvert
* solving the new first error
* reconvert
* etc.
should always deterministically lead to a resolution of all problems.
Error messages are normally sorted by file, line and column, regardless of which conversion stage they happened (e.g. a tokeniser error first gets reported before a parser error).
There is however one important exception to that: broken internal cross references are always reported last.
For example, consider the following syntactically wrong document:
```
= a
\x[b]
``
== b
```
Here we have an unterminated code block at line 5.
However, this unterminated code block leads the header `b` not to be seen, and therefore the reference `\x[b]` on line 3 to fail.
Therefore, if we sorted naively by line, the broken reference would shoe up first:
``
error: tmp.bigb:3:3: cross reference to unknown id: "b"
error: tmp.bigb:5:1: unterminated literal argument
``
But in a big document, this case could lead to hundreds of undefined references to show up before the actual root unterminated literal problem.:
``
error: tmp.bigb:3:3: cross reference to unknown id: "b"
error: tmp.bigb:4:3: cross reference to unknown id: "b"
error: tmp.bigb:5:3: cross reference to unknown id: "b"
...
error: tmp.bigb:1000:1: unterminated literal argument
``
Therefore, we force undefined references to show up last to prevent this common problem:
``
error: tmp.bigb:1000:1: unterminated literal argument
error: tmp.bigb:3:3: cross reference to unknown id: "b"
error: tmp.bigb:4:3: cross reference to unknown id: "b"
error: tmp.bigb:5:3: cross reference to unknown id: "b"
...
``
= Security
{parent=ourbigbook}
= Arbitrary code execution
{parent=security}
OurBigBook is designed to not allow arbitrary code execution by default on any \x[ourbigbook-cli] command.
This means that it it should be safe to just download any untrusted OurBigBook repository, and convert it with the \x[ourbigbook-executable], even if you don't trust its author.
In order to allow code execution for pre/post processing tasks e.g. from \x[ourbigbook-json/prepublish], use the \x[unsafe-ace] option.
Note however that you have to be careful in general, since e.g. a malicious author could create a package with their own malicious version of the `ourbigbook` executable, that you could unknowingly run with with the standard `npx ourbigbook` execution.
= unsafe-xss
{title2=`--unsafe-xss`}
{parent=security}
OurBigBook HTML output is designed to be XSS safe by default, any non-XSS safe constructs must be enabled with a non-default flag or setting, see: \x[unsafe-xss].
Of course, we are walking on eggs, and this is hard to assert, so the best thing to do later on will be to parse the output e.g. with https://developer.mozilla.org/en-US/docs/Web/API/DOMParser[`DOMParser`] to ensure that it is valid and does not contain any `script` tags, but it is not as simple as that: https://stackoverflow.com/questions/37435077/execute-javascript-for-xss-without-script-tags/61588322#61588322
XSS unsafe constructs lead to errors by default. XSS unsafe constructs can be allowed \x[ourbigbook-executable][from the command] line with:
``
./ourbigbook --unsafe-xss
``
or from the \x[ourbigbook-json] file with an entry of form:
``
"unsafe-xss": true
``
= Developing OurBigBook
{parent=ourbigbook}
= Run OurBigBook master
{parent=developing-ourbigbook}
Install master globally on your machine:
``
git clone https://github.com/cirosantilli/ourbigbook
cd ourbigbook
npm link
npm link ourbigbook
npm run build-assets
``
so you can now run the `ourbigbook` command from any directory in your computer, for example to convert the ourbigbook documentation itself:
``
ourbigbook .
``
Note that this repository uses \x[ourbigbook-json/outputoutoftree], and so the output will be present at `out/html/index.html` rather than `index.html`.
We also have a shortcut for `npm link` and `npm link ourbigbook`:
``
npm run link
``
`npm run link` produces symlinks so that any changes made to the Git source tree will automatically be visible globally, see also: https://stackoverflow.com/questions/28440893/install-a-locally-developed-npm-package-globally The symlink structure looks like:
``
/home/ciro/ourbigbook/node_modules/ourbigbook -> /home/ciro/.nvm/versions/node/v14.17.0/lib/node_modules/ourbigbook -> /home/ciro/ourbigbook
``
As mentioned at \x[useless-knowledge], most users don't want global installations of OurBigBook. But this can be handy during development, as you can immediately see how your changes to OurBigBook source code affect your complex example of interest. For example, Ciro developed a lot of OurBigBook by hacking https://github.com/cirosantilli/cirosantilli.github.io directly with OurBigBook `master`.
Just remember that if you add a new dependency, you must redo the symlinking business:
``
npm install
npm run link
``
Asked if there is a better way at: https://stackoverflow.com/questions/59389027/how-to-interactively-test-the-executable-of-an-npm-node-js-package-during-develo[]. The symlink business can be undone with:
``
npm unlink
rm node_modules/ourbigbook
``
= Run an isolated OurBigBook master
{parent=run-ourbigbook-master}
\x[run-ourbigbook-master] mentions how to install and then run OurBigBook master globally, which is useful build some projects locally on maste.
To instead install locally in the current directory only instead, which can be useful for bisection:
``
npm install
ln -s .. node_modules/ourbigbook
npm run build-assets
./ourbigbook .
``
= Test system
{parent=developing-ourbigbook}
Run all tests:
``
npm test
``
To run all tests on PostgreSQL as in the \x[ourbigbook-web], first setup the PostgreSQL database similarly to \x[local-run-as-identical-to-deployment-as-possible]:
``
createdb ourbigbook_cli
psql -c "CREATE ROLE ourbigbook_user with login password 'a'"
psql -c 'GRANT ALL PRIVILEGES ON DATABASE ourbigbook_cli TO ourbigbook_user'
``
and then run with:
``
npm run test-pg
``
List all tests:
``
node node_modules/mocha-list-tests/mocha-list-tests.js main.js
``
as per: https://stackoverflow.com/questions/41380137/list-all-mocha-tests-without-executing-them/58573986#58573986[].
Run just one test by name:
``
npm test -- -g 'one paragraph'
``
as per: https://stackoverflow.com/questions/10832031/how-to-run-a-single-test-with-mocha todo: what if the test name is a substring? You will want this Bash alias:
``
npmtg() ( npm test -- -g "$*" )
``
which allos you to just:
``
npmtg one paragraph
``
Run all tests that don't start with `executable:`:
``
npm test -- -g '^(?!executable:)'
``
This works because `-g` takes JavaScript regular expressions, so we can use negative lookahead, see also: https://stackoverflow.com/questions/26908288/with-mocha-how-do-i-run-all-tests-that-dont-have-slow-in-the-name
There are two types of test in our test suite:
* tests that call the `ourbigbook.convert` \x[javascript-api] directly.
These tests don't actually create files in the filesystem, and just mock the filesystem instead with a dictionary.
Database access is not mocked however, we just use Sqlite's fantastic in-memory mode.
Whenever possible, these tests check their results just from the \x[abstract-syntax-tree] tree returned by the API, which is cleaner than parsing the HTML. But sometimes HTML parsing is inevitable.
* tests that call the \x[ourbigbook-executable] itself:
* their titles are prefixed with `executable: `
* they tend to be a lot slower than the API test
* can test functionality that is done outside of the `ourbigbook.convert` JavaScript API, notably stuff prevent in ourbigbook, so they are more end to end
* don't do any mocking, and could therefore be more representative.
However, as of 2022, we have basically eliminated all the hard database access mocking and are using the main database methods directly.
So all that has to be mocked is basically stuff done in the ourbigbook executable itself.
This means that except for more specific options, the key functionality of ourbigbook, which is to convert multiple paths, can be done very well in a non executable test.
The only major difference is that instead of passing command line arguments like in `ourbigbook .` to convert multiple files in a directory, you have to use `convert_before` and `convert_before_norender` and specify conversion order one by one.
This test robustness is new as of 2022, and many tests were previously written with executable that would now also work as unit tests, and we generally want that to be the case to make the tests go faster.
* work by creating an actual physical filesystem under `out/test/` with the OurBigBook files and other files like `ourbigbook.json`, and then running the executable on that directory.
`npm test` first deletes the `out/test` directory before running the tests. After running, the generated files are kept so you can inspect them to help debug any issues.
* all these tests check their results by parsing the HTML and searching for elements, since here we don't have access to the \x[abstract-syntax-tree]. It wouldn't be impossible to obtain it however, as it is likely already JSON serializable.
Step debug during a test run. Add the statement:
``
debugger;
``
to where you want to break in the code, and then run:
``
npm run testi -- -g 'p with id before'
``
where `i` in `testi` stands for `inspect` from `node inspect`. Also consider the alias:
``
npmtgi() ( npm run testi -- -g "$*" )
``
Note however that this does not work for tests that run the `ourbigbook` executable itself, since those spawn a separate process. TODO how to do it? Tried along:
``
const out = child_process.spawnSync('node', ['inspect', 'ourbigbook'].concat(options.args), {
cwd: tmpdir,
input: options.stdin,
stdio: 'inherit',
});
``
but not working, related: https://stackoverflow.com/questions/23612087/gulp-target-to-debug-mocha-tests So for now, we are just printing the command being run as in:
``
cmd: cd out/test/executable-ourbigbook.json-outputOutOfTree && ourbigbook --split-headers .
``
so you can just re-run it manually with `node inspect` as in:
``
cd out/test/executable-ourbigbook.json-outputoutoftree && node inspect "../../../ourbigbook" --split-headers .
``
This works since the `tmp` directory is not deleted in case of failure.
= Overview of files in this repository
{parent=developing-ourbigbook}
Source files:
* \a[index.js]: main code. Must be able to run in the browser, so no Node.js specifics. Exposes the central `convert` function. You should normally use the packaged `dist/ourbigbook.js` when using ourbigbook as an external dependency.
* \a[ourbigbook]: CLI executable. Is basically just a CLI interface frontend to `convert`
* \a[test.js]: contains all the Mocha tests, see also: \x[test-system]
* \a[README.md]: minimal Markdown README until GitHub / NPM support OurBigBook :-)
* \a[ourbigbook_runtime.js]: JavaScript functionality to be included in the final documents to enable interactive document features such as the \x[table-of-contents]. You should use the packaged `dist/ourbigbook_runtime.js` instead of this file directly however.
* \a[main.scss] this file simply contains the customized CSS for https://cirosantilli.com/ourbigbook/[] and does not get otherwise distributed with OurBigBook, see: \x[css]
Generated files:
* `dist/` contains fully embedded packaged versions that work on browsers as per common JavaScript package naming convention. All the following files are generated with Webpack with:
``
npm run webpack
``
, which is called from `npm run build-assets`. If you are developing just one of those files interactively, e.g. just the CSS or just the JavaScript;, you could speed thing up a bit by using watch mode, which only recompiles what is needed:
``
npm run webpack -- -w
``
* `dist/ourbigbook.js`: OurBigBook \x[javascript-api] converter for browser usage. The source entrypoint for it is located at \a[index.js]. Contains the code of every single dependency used from `node_modules` used by `index.js`. This is notably used for the live-preview of a \x[browser-editor-with-preview].
* `dist/ourbigbook_runtime.js`: similar `dist/ourbigbook.js`, but contains the converted output of \a[ourbigbook_runtime.js]. You should include this in every OurBigBook HTML output.
* `dist/ourbigbook.css`: minimized CSS needed to view OurBigBook output as intended. Embeds all OurBigBook CSS dependencies, notably the KaTeX CSS without which \x[mathematics] displays as garbage. The Sass entry point for it is: \a[ourbigbook.scss].
* `dist/editor.css`: the CSS of the editor, rendered from \a[editor.scss].
TODO it would be good to get a dev server running somehow, the following fails because it serves under `localhost`: When developing those output files under `dist`, you may want to run the webpack development server with:
``
npm run webpack-dev
``
This automatically reconverts when the input source files are modified, and does not optimize assets, leading to a faster conversion and easier debugging.
= The `out` directory
{parent=overview-of-files-in-this-repository}
OurBigBook stores some metadata and outputs it generates/needs inside the `./out/` directory that it creates inside the \x[outdir].
Overview of files it contains:
* `db.sqlite3`: \x[internal-cross-file-reference-internals]
* `publish`: a git clone of the source of the main repository to ensure that untracked files won't accidentally go into the output
* `publish/out/db.sqlite3`: like `out/db.sqlite3` but from the clean clone of `out/publish`
* `publish/out/publish`: the final generated output directory that gets published, e.g. as in \x[publish-to-github-pages]
= Conversion process overview
{parent=developing-ourbigbook}
A conversion follows the following steps done for each file to be converted:
* tokenizer. Reads the input and converts it to a linear list of tokens.
* parser. Reads the list of tokens and converts it into an \x[abstract-syntax-tree]. Parse can be called multiple times recursively when doing things like.
* ast post process pass 1.
An ast post process pass takes abstract syntax tree that comes out of a previous step, e.g. the original parser output, and modifies the it tree to achieve various different functionalities.
We may need iterate the tree multiple times to achieve all desired effects, at the time of writing it was done twice. Each iteration is called pass.
You can view snapshots of the tree after each pass with the \x[log] option:
``
ourbigbook --log=ast-pp-simple input.bigb
``
This first pass basically does few but very wide reacing operations that will determine what data we will have to fetch from the database during the followng DB queries step.
It might also do some operations that are required for pass 2 but that don't necessarily fetch data, not sure anymore.
E.g. this is where the following functionality are implemented:
* \x[synonym] and \x[scope]
* \x[ourbigbookexample] and \x[embed-includes]: they are stitched into the main AST
* ast post process pass 2: we now do every other post process operation that was not done in pass 1, e.g.:
* \x[insane] paragraphs, lists and tables
* ast post process pass 3: this does some minimal tree hierarchy linking between parents and children. TODO could it be merged into 2? Feels likely
* render, which converts our AST tree into a output string. This is run once for the toplevel, and once for every header of the document if \x[split-headers] are enabled. We need to do this because header renders are different from their toplevel counterparts, e.g. their first paragraph has id `p-1` and not `p-283`. All of those renders are done from the same parsed tree however, parsing happens only once.
This step is skipped when using the \x[no-render] option, or during ID extraction.
TODO it is intended that it should not be possible for there to be rendering errors once the previous steps have concluded successfully. This is currently not the case for at least one known scenario however: \x[internal-cross-reference]{p} that are not defined.
Sub-steps include:
* DB queries: this is the first thing we do during the rendering step.
Every single database query must be done at this point, in one go.
Database queries are only done while rendering, never when parsing. The database is nothing but a cache for source file state, and this separation means that we can always cache input source state into the database during parsing without relying on the database itself, and thus preventing any circular dependencies from parsing to parsing.https://cirosantilli.com/china-dictatorship/reddit{ref}
Keeping all queries together is fundamental for performance reasons, especially of \x[browser-editor-with-preview] in the \x[ourbigbook-web]: imagine doing 100 scattered server queries:
``
SELECT * from Ids WHERE id = '0'
SELECT * from Ids WHERE id = '1'
...
SELECT * from Ids WHERE id = '100'
``
vs grouping them together:
``
SELECT * from Ids WHERE id IN ('0', '1', ..., '100')
``
It also has the benefit of allowing us to remove `async`/`await` from almost every single function in the code, which considerably slows down the CPU-bound execution path.
As an added bonus, it also allows us to clearly see the impact of database queries when using \x[log-perf].
We call this joining up of small queries into big ones "query bundling".
* at the every end of the conversion, we then save the database changes calculated during parsing and post processing back to the DB so that the conversion of other files will pick them up.
Just like for the SELECT, we do a single large INSERT/UPDATE query per database to reduce the round trips.
Conversion of a directory with multiple input files works as follows:
* do one conversion pass without render
* do a global database check for all files that have been parsed which checks in one go for:
* duplicate IDs
* references from one non-header title to another non-header title as mentioned at \x[x-within-title-restrictions]
Ideally, failure of any of the above checks should lead to the database not being updated with new values, but that is not the case as of writing.
* do one conversion pass with render. To speed up conversion, we might at some point start storing a parsed JSON after the first conversion pass, and then just deserialize it and convert the deserialized output directly without re-parsing.
The two pass approach is required to resolve \x[internal-cross-reference]{p}
= Abstract syntax tree
{parent=conversion-process-overview}
{title2=AST}
The implementation of much of the functionality of OurBigBook involves manipulating the abstract syntax tree.
The structure of the AST is as follows:
* `AstNode`: contains a map from argument names to the values of each argument, which are of type AstArgument
* `AstArgument`: contains a list of `AstNode`. These are generally just joined up on the output, one after the other.
One important exception to this are plaintext nodes. These nodes contain just raw strings instead of a list of arguments. They are usually the leaf nodes.
We can easily observe the AST of an input document by using the \x[log] following log options:
``
ourbigbook --log=ast-simple input.bigb
ourbigbook --log=ast input.bigb
``
For example, the document:
``
= My title
{c}
A link to \x[another-title]{c}{p} and more text.
== Another title
``
produces with `--log=ast-simple` the following output:
``
ast Toplevel
arg content
ast H id="tmp"
arg c
arg level
ast plaintext "1"
arg numbered
ast plaintext "0"
arg scope
ast plaintext "0"
arg splitDefault
ast plaintext "0"
arg synonym
ast plaintext "0"
arg title
ast plaintext "My title"
ast P id="p-1"
arg content
ast plaintext "A link to "
ast x
arg c
arg child
ast plaintext "0"
arg full
ast plaintext "0"
arg href
ast plaintext "another-title"
arg p
arg parent
ast plaintext "0"
arg ref
ast plaintext "0"
ast plaintext " and more text."
ast Toc id="toc"
ast H id="another-title"
arg c
ast plaintext "0"
arg level
ast plaintext "2"
arg numbered
ast plaintext "0"
arg scope
ast plaintext "0"
arg splitDefault
ast plaintext "0"
arg synonym
ast plaintext "0"
arg title
ast plaintext "Another title"
``
= Autogenerated tests
{parent=conversion-process-overview}
The following scripts generate parametrized OurBigBook examples that can be used for performance or other types of interactive testing:
* \a[generate-deep-tree]:
``
./generate-deep-tree 2 5 > deep_tree.tmp.bigb
./ourbigbook deep_tree.tmp.bigb
``
Originally designed to be able to interactively play with a huge \x[table-of-contents] to streamline JavaScript open close interaction.
= generate-paragraphs
{parent=autogenerated-tests}
``
./generate-paragraphs 10 > main.bigb
``
Output:
``
0
1
2
3
4
5
6
7
8
9
``
= Performance
{parent=developing-ourbigbook}
To log some performance statistics, use:
= Speed comparison to other markup language implementations
{parent=performance}
One quick and dirty option is to use `generate-paragraphs` which generates output compatible for most markup languages:
``
./generate-paragraphs 100000 > tmp.bigb
``
On Ubuntu 20.04 \a[https://cirosantilli.com/linux-kernel-module-cheat/#p51][Lenovo ThinkPad P51] for example:
* OurBigBook 54ba49736323264a5c66aa5d419f8232b4ecf8d0 + 1, Node.js v12.18.1
``
time ./ourbigbook tmp.bigb
``
outputs:
``
real 0m5.104s
user 0m6.323s
sys 0m0.674s
``
* Asciidoctor 2.0.10, Ruby 2.6.0p0:
``
cp tmp.bigb tmp.adoc
time asciidoctor tmp.adoc
``
outputs:
``
real 0m1.911s
user 0m1.850s
sys 0m0.060s
``
* https://github.com/commonmark/cmark[cmark] 0.29.0:
``
cp tmp.bigb tmp.md
time cmark tmp.md > tmp.md.html
``
outputs:
``
real 0m0.091s
user 0m0.070s
sys 0m0.021s
``
Holy cow, it is 200x faster than Asciidoctor!
* https://github.com/markdown-it/markdown-it[markdown-it] at 5789a3fe9693aa3ef6aa882b0f57e0ea61efafc0 to get an idea of a JavaScript markdown implementation:
``
time markdown-it tmp.md > tmp.md.html
``
outputs:
``
real 0m0.361s
user 0m0.590s
sys 0m0.060s
``
* `cat` just to find the absolute floor:
``
time cat tmp.bigb > tmp.tmp
``
outputs:
``
real 0m0.006s
user 0m0.006s
sys 0m0.000s
``
= Performance log
{parent=developing-ourbigbook}
On P51:
* `time ./ourbigbook --no-render . && time ./ourbigbook -S --log=perf README.bigb`:
* ourbigbook 39e633f08b2abce10331b884c04d70dbe6d4565a before moving OurBigBook to Sequelize: 14s
``
convert README.bigb
perf start 248.0712810009718
perf tokenize_pre 248.36641899868846
perf tokenize_post 2027.6697090007365
perf parse_start 2028.678952999413
perf post_process_start 2684.3162699975073
perf post_process_end 5017.946601998061
perf split_render_pre 6572.925067000091
perf render_pre 5018.202895000577
perf render_2_pre undefined
perf render_post 13093.658641997725
perf end 13126.138450000435
perf convert_input_end 14281.568749997765
perf convert_path_pre_sqlite 14281.64151499793
perf convert_path_pre_sqlite_transaction 14281.835940998048
perf convert_path_post_sqlite_transaction 14551.673818998039
perf convert_path_end 14551.860617998987
convert README.bigb finished in 14309.703324001282 ms
perf convert_path_to_file_end 14552.230636000633
real 0m14.602s
user 0m16.500s
sys 0m1.832
``
`sqlite3 out/db.sqlite3 .schema`
``
CREATE TABLE IF NOT EXISTS 'ids' (
id TEXT PRIMARY KEY,
path TEXT,
ast_json TEXT
);
CREATE TABLE IF NOT EXISTS 'includes' (
from_id TEXT,
from_path TEXT,
to_id TEXT,
type TINYINT
);
CREATE INDEX includes_from_path
ON includes(from_path);
CREATE INDEX includes_from_id_type
ON includes(from_id, type);
CREATE TABLE IF NOT EXISTS 'files' (
path TEXT PRIMARY KEY,
toplevel_id TEXT UNIQUE
);
``
* ourbigbook 8e6a4311f7debd079721412e1ea5d647cc1c2941 after, OMG gotta debug perf now:
``
real 0m29.595s
user 0m34.095s
sys 0m4.427s
convert README.bigb
perf start_convert undefined
perf tokenize_pre 411.71094800531864
perf tokenize_post 2265.3646410033107
perf parse_start 2266.436312004924
perf post_process_start 2905.8304330036044
perf post_process_end 12113.761793002486
perf split_render_pre 16534.5258340016
perf render_pre 12114.092462003231
perf render_post 40937.611143000424
perf end_convert undefined
perf convert_input_end 42042.85608199984
perf convert_path_pre_sqlite 42042.92515899986
perf convert_path_pre_sqlite_transaction 42147.847070001066
perf convert_path_post_sqlite_transaction 42732.242132000625
perf convert_path_end 42732.35991900414
convert README.bigb finished in 42327.62727500498 ms
perf convert_path_to_file_end 42732.534088000655
real 0m42.779s
user 0m46.530s
sys 0m6.945s
``
`sqlite3 out/db.sqlite3 .schema`
``
CREATE TABLE `Files` (`id` INTEGER PRIMARY KEY AUTOINCREMENT, `path` TEXT NOT NULL UNIQUE, `toplevel_id` TEXT UNIQUE);
CREATE TABLE sqlite_sequence(name,seq);
CREATE INDEX `files_path` ON `Files` (`path`);
CREATE INDEX `files_toplevel_id` ON `Files` (`toplevel_id`);
CREATE TABLE `Ids` (`id` INTEGER PRIMARY KEY AUTOINCREMENT, `idid` TEXT NOT NULL UNIQUE, `path` TEXT NOT NULL, `ast_json` TEXT NOT NULL);
CREATE INDEX `ids_path` ON `Ids` (`path`);
CREATE TABLE `Refs` (`id` INTEGER PRIMARY KEY AUTOINCREMENT, `from_id` TEXT NOT NULL, `from_path` TEXT NOT NULL, `to_id` TEXT NOT NULL, `type` TINYINT NOT NULL);
CREATE INDEX `refs_from_path` ON `Refs` (`from_path`);
CREATE INDEX `refs_from_id_type` ON `Refs` (`from_id`, `type`);
CREATE INDEX `refs_to_id_type` ON `Refs` (`to_id`, `type`);
``
The question is: is it because we added `async` Everywhere, or is it because of changes in the database queries?
Answering the question: added old DB at: https://github.com/cirosantilli/ourbigbook/tree/async-slow-old-db and it is fast again. So DB debugging it is, hurray.
= Internals API
{parent=developing-ourbigbook}
Tokenized token stream and AST can be obtained as JSON from the API.
Errors can be obtained as JSON from the API.
Everything that you need to write OurBigBook tooling, is present in the main API.
All tooling will be merged into one single repo.
= The `\Toplevel` implicit macro
{id=toplevel}
{parent=internals-api}
Every OurBigBook document is implicitly put inside a `\Toplevel` document and:
* any optionally given arguments at the very beginning of the document will be treated as arguments of the `\Toplevel` macro
* anything else will be put inside the `content` argument of the `\Toplevel` macro
E.g., a OurBigBook document that contains:
``
{title=My favorite title.}
And now, some content!
``
is morally equivalent to:
``
\Toplevel{title=My favorite title.}
[
And now, some content!
]
``
In terms of HTML, the `\Toplevel` element corresponds to the ``, ``, `` and `` elements of a document.
Trying to use the `\Toplevel` macro explicitly in a document leads to an error.
= HTML document title
{c}
{parent=internals-api}
* if the `title` argument of `toplevel` is given, use that
* otherwise, if the document has a `\H[1]`, use the title of the first such header
* otherwise use a dummy value
= CSS
{c}
{parent=developing-ourbigbook}
Our CSS is located at \a[main.scss] and gets processed through https://sass-lang.com[Sass].
To generate the CSS during development after any changes to that file, you must run:
``
npm run sass
``
which generates the final CSS file:
``
main.css
``
You then need to explicitly include that `main.css` file in your \x[template]. For example, our \a[main.liquid.html] contains a line:
``
``
where `root_relpath` is explained under \x[template]{full}.
= `ourbigbook.common.scss`
{c}
{parent=css}
The file `ourbigbook.common.scss` contains stand-alone Sass definitions that can be used by third parties.
One use case is to factor out OurBigBook style with the site-specific boilerplate.
E.g. a website that stores its custom rules under \a[main.scss] can do stuff like:
``
@import 'ourbigbook/ourbigbook.common.scss';
``
= Mobile guidelines
{parent=css}
The main design goal on narrow screens is that there should never be horizontal scrolling enabled for the hole document, only on a per element basis.
= Formal grammar
{parent=developing-ourbigbook}
TODO. Describe OurBigBook's formal grammar, and classify it in the grammar hierarchy and parsing complexity.
= Release procedure
{parent=developing-ourbigbook}
= Do the release
{parent=release-procedure}
Before the first time you release, you need to login to NPM with:
``
npm login
``
Then, every new release can be done automatically with the \a[release] script, e.g. to release a version 0.7.2:
``
./release 0.7.2
``
or to just increment the minor version, e.g. from the current `0.7.1` to `0.7.2` you could can omit the version argument:
``
./release
``
That script does the following actions, aborting immediately if any of them fails:
* runs \x[test-system][the tests]
* publishes this documentation
* updates `version` in `package.json`
* creates a release commit and a git tag for it
* pushes the source code
* publishes the NPM package
= Out-of-box testing
{parent=release-procedure}
After publishing, a good minimal sanity check is to ensure that you can render the template as mentioned in \x[play-with-the-template]:
``
cd ~
# Get rid of the global npm link development version just to make sure it is not being used.
npm uninstall -g ourbigbook
git clone https://github.com/cirosantilli/ourbigbook-template
cd ourbigbook-template
npm install
npx ourbigbook .
firefox index.html
``