JSDoc parser/Sphinx extension for Odoo - Odoo 11.0

Why?

Spent about a week trying to coerce “standard” javascript tools (jsdoc with the hope of using sphinx-js for integration or documentation.js) and failed to ever get a sensible result: failed to get any result with the current state of the documentation, significant changes/additions/fixes to docstrings brought this up to “garbage output” level.

Bug reports and mailing list posts didn’t show any path to improvement on the ES5 codebase (if we ever go whole-hog on ES6 modules and classes things could be different, in fact most of JSDoc’s current effort seem focused on ES6/ES2015 features) but both experience and looking at the mailing lists told me that spending more time would be wasted.

Even more so as I was writing visitors/rewriters to generate documentation from our existing structure, which broadly speaking is relatively strict, and thus

What?

If it were possible to generate JSDoc annotations from our relatively well-defined code structures, it was obviously possible to extract documentary information directly from it, hence this Odoo-specific package/extension trying to do exactly that.

This package should eventually provide:

  • a command-line interface which can be invoked via -m autojsdoc (assuming your PYTHONPATH can find it) which should allow dumping the parsed AST in a convenient-ish form, possibly doing searches through the AST, a dependency graph extractor/analysis and a text dumper for the documentation.
  • a sphinx extension (autojsdoc.sphinx) which can be used to integrate the parsed JSDoc information into the Sphinx doc.

How?

Sphinx-aside, the package relies on 3 libraries:

  • pyjsparser, an Esprima-compliant ES5.1 parser (with bits of ES6 support), sadly it does not support comments in its current form so I had to fork it. Fed a javascript source file, pyjsparser simply generates a bunch of nested dicts representing an Esprima ast, ast-types does a reasonably good job of describing it once you understand that “bases” are basically just structural mixins.

    Because the original does not, this package provides a visitor module for pyjsparser ASTs.

  • pyjsdoc, a one-file “port” of jsdoc, can actually do much of the JS parsing (using string munging) but its core semantics don’t fit our needs so I’m only using it to parse the actual JSDoc content, and the jsdoc module contains some replacement classes, extensions & monkey patches for things pyjsdoc itself does not support, at the time of this writing:

    • a bug in FunctionDoc.return_val
    • a type on FunctionDoc so it’s compatible with ParamDoc
    • a more reliable comments-parsing function
    • a replacement ModuleDoc as the original does not materialise AMD modules
    • a ClassDoc extension to support mixins
    • two additional CommentDoc extensions for “namespaces” objects (bag of attributes without any more information) and mixin objects
  • pytest to configure and run the test suite, which you can run by invoking pytest doc/_extensions from the project top-level, the tests represent both “happy path” things we want to parse and various code patterns which tripped the happy path because e.g. they were matched and should not have, they were not matched and should have, or they were more complex than the happy path had expected