Literate AI

Repository documentation

DMN as a new specification_provider — design spike (MEDIUM(b))

This is not committed 0.5.0 scope

This document is the design spike 0.5.0-specification-hierarchy.md calls for under MEDIUM(b): a DMN (OMG Decision Model and Notation) specification_provider for rule/decision-table logic that OpenSpec's WHEN/THEN scenario grammar expresses badly. Nothing in this document is promised 0.5.0 delivery, and per that document's own instruction, no implementation should start until this spike's open questions are answered and reviewed. This is a design spike, not code — no source under src/literate_ai/ changes as part of this document.

Method

Every claim below is grounded in the actual specification_provider seam as it exists today on origin/release/0.5.0 (724add76), read directly rather than assumed. The seam lives in src/literate_ai/adapters/specifications/ (openspec.py, literate_markdown.py), src/literate_ai/contracts/specifications.py, and is consumed at three call sites: src/literate_ai/adapters/component_lock_planning.py, src/literate_ai/adapters/legacy_generation_catalog.py, and src/literate_ai/self_hosting.py. DMN itself (decision tables, hit policies, FEEL) is the OMG standard, current version DMN 1.5; concepts are named concretely below rather than hand-waved.

What the seam actually looks like (load-bearing finding)

ComponentDefinition.specification_provider (src/literate_ai/contracts/components.py:95) is a plain str, not a closed enum — string_value() validation only, no allow-list at the contract layer. That is genuinely good news for extensibility: the data model does not need to change to name a third provider.

But the two concrete call sites that actually dispatch on the string are hardcoded if/elif/else, not a plugin registry:

So "an extension of an established extension point" (the issue's framing, quoted in 0.5.0-specification-hierarchy.md) is correct in spirit but not literally true of the code today: adding DMN means adding a third arm to both of these functions, not registering a new implementation against an existing open interface. This is a small, mechanical change (two functions, symmetrical edits) — not a redesign — but it is not zero-touch. Flag this explicitly as scope for whoever implements this later.

More important finding — the shared data shape is WHEN/THEN-shaped, not neutral. LiterateMarkdownProvider.load() (literate_markdown.py:128-153) does not have an independently different specification shape from OpenSpecProvider: it constructs its own _loader = OpenSpecProvider(...) internally (literate_markdown.py:121) and returns the same LoadedOpenSpec wrapping the same SpecificationSet. The two existing providers differ only in authoring surface (raw ### Requirement: / #### Scenario: Markdown headings vs. a component.md-rooted node graph with YAML frontmatter) — both terminate in the identical SpecificationRequirement/ SpecificationScenario shape defined in src/literate_ai/contracts/specifications.py:

class SpecificationScenario:
    scenario_id: str
    title: str
    given: tuple[str, ...]
    when: tuple[str, ...]   # __post_init__ requires non-empty
    then: tuple[str, ...]   # __post_init__ requires non-empty

Every provider that exists today is, structurally, a WHEN/THEN provider. DMN would be the first provider whose native content doesn't decompose into given/when/then prose at all — a decision table's rows are input-column values → output-column values under a declared hit policy, not a scenario narrative. This is the single most important fact for everything below: DMN cannot be shoehorned into SpecificationSet without a lossy synthetic mapping, and the design should not attempt that mapping.

Generation-key binding is already provider-neutral, independent of the above. _specification_references() computes each ContentReference's content identity from the raw bytes of the files under specification_roots (_reference(root, "specification", uri, boundary=boundary)), before invoking the provider at all. The provider's .load() call is used only to (a) confirm the provider can parse the files without error and (b) cross-check that the content-identities the provider reports for its own artifacts (loaded.specification_set.artifacts) match the raw-byte-derived references exactly (component_lock_planning.py:1603-1609). If they don't match, lock planning fails closed (component_lock.specification_changed). Generation-key/content-hash binding therefore never depends on parsing the WHEN/THEN structure — it hashes opaque bytes either way. This means MEDIUM(b)'s "generation-key binding" open question has a concrete, already-true answer (see Q4 below): no change needed to the hashing mechanism itself, only to what "parse without error" and "artifacts match" mean for a DMN provider's own .load().

litai generate treats specification content as opaque context bytes, already provider-neutral. generation_preparation.py:454-471 (_documents_for_node) collects selected_node.revision.specifications and .acceptance_contracts as raw (uri, bytes) pairs handed to the code-generation skill as RecipeDocuments — there is no per-provider branch here at all. Whatever a provider's file format is, by the time generation runs it is just bytes in a prompt/context bundle. This is strong evidence for the "no DMN execution engine in litai" answer (Q3 below): litai's own generation path has never executed a specification, of any provider, at generation time — it only ever hands specification bytes to the generator and lets the generated code (and separately, acceptance_contracts) do the executing/testing.

assets: (MEDIUM(a), #116/#118) is the closest existing precedent for non-prose structured content, and its shape is instructive. ComponentAssetSelector (src/literate_ai/contracts/component_locking/authoring.py:65-85) is asset_id, source, path, role, media_type, pin: ContentIdentity | None, resolved at lock time into a ResolvedComponentAsset (.../locks.py:131-150) binding the selector to an exact BlobRef by content hash. src/literate_ai/source_to_specification/literal_data.py is the extraction half of this pattern for the migration pipeline: it deliberately does not ask a model to prose-summarize a literal Python dict/list/tuple/set constant table (issue #116 found 15 of 16 entries silently dropped when it tried), and instead extracts the literal values verbatim via ast.literal_eval, pins them as canonical JSON (canonical_literal_table_json), and registers a literal-data:<name> behavioral-surface suffix so spec-derivation coverage cannot silently treat the table as "covered ordinary behavior." The generalizable lesson: a lossy prose paraphrase of structured, valuable-when-exact data is worse than treating it as opaque, content-pinned, verbatim structured data — which is exactly why DMN decision tables (also dense, exact, and lossy when paraphrased) should follow the same family of treatment rather than being coerced into WHEN/THEN prose.

Open questions, answered

Q1 — What does a DMN-authored decision table look like end to end?

Proposed answer: author it as a .dmn file (the OMG DMN XML interchange format) or a constrained subset of it, declared in specification_roots like any other specification file, discovered the same way — not derived, hand-authored.

Q2 — What does "acceptance" mean for a decision table?

Proposed answer: acceptance for a DMN table is structural validation of the table itself (hit-policy consistency, input/output coverage, type conformance), not scenario pass/fail — and it happens at .load() time via the provider, not via a spec accept-shaped oracle borrowed from OpenSpec.

Concretely, a DMN decision table's own standard-defined hit policy governs what "the table is well-formed" means, and this is a genuinely different oracle shape than WHEN/THEN's boolean-per-scenario pass/fail:

Beyond hit-policy consistency, DMN completeness/consistency checking also covers: input coverage (does the table's rule set cover the full declared domain of each input, e.g. via a FEEL unary-test grammar like [18..65], "low","medium","high", or - for "don't care"?) and output-type conformance (does every rule's output cell conform to the output column's declared FEEL type?).

This is a genuinely different acceptance-oracle shape from OpenSpec's, and the right place for it is analogous to how OpenSpecProvider.snapshot() already validates structural well-formedness at parse time (unique requirement IDs, non-empty when/then, valid heading structure) and raises OpenSpecError subtypes on failure, before any scenario is ever "accepted" against generated code. A DmnProvider.load() should do the DMN-equivalent structural validation (hit-policy consistency, input/output coverage, type conformance) at the same point in the pipeline — lock-planning time, via require_unchanged()/.load() failing closed with a DmnError, not via a new CLI verb. This reuses the existing failure path (component_lock.specification_invalid, component_lock_planning.py:1598-1602, which already catches (OpenSpecError, OSError) and would need DmnError added to that tuple) rather than inventing a spec accept --dmn command that doesn't compose with anything else in the pipeline.

What "acceptance" does not mean for DMN, by design: whether the decision table produces the intended business answer for a given input (semantic correctness, as opposed to structural well-formedness) is not something litai's provider seam checks today for any provider — generation_preparation.py shows generation only ever consumes specification bytes as opaque context; nothing in the seam executes a scenario and compares an actual result to an expected one. OpenSpec's WHEN/THEN "acceptance" in this codebase is really "the generated code's own test suite, whose authoring is informed by the WHEN/THEN scenarios read as prose by the code-generation skill" — not a live comparator litai runs itself. DMN's structural acceptance (hit-policy/coverage validation) is stronger than what OpenSpec gets structurally (OpenSpec only checks heading well-formedness, not scenario semantic completeness), which is a genuine advantage DMN's own standard hands litai "for free," and worth calling out as a positive rather than a gap to fill.

Q3 — Minimum viable adapter: does litai need a DMN execution engine?

Proposed answer: no. Treat the DMN table as declarative structured data — closer to assets: than to a live spec engine — with execution deferred entirely to generated code, exactly as Q1/Q2's grounding shows generation already treats every provider's content today.

The minimum viable DmnProvider:

  1. Parses the .dmn XML file(s) far enough to extract: decision table structure (input/output column definitions with their FEEL type and label), hit policy, and the rule matrix (input-cell unary tests, output-cell literal expressions) — enough to run the Q2 structural validation and to compute stable per-artifact content identities.
  2. Does not implement a FEEL expression evaluator or execute any rule against a candidate input. Full FEEL (with its date/time arithmetic, list/context comprehensions, and built-in function library) is a substantial implementation in its own right — multiple open-source DMN engines (e.g. Camunda's dmn-scala, drools/kie-dmn) exist specifically because this is nontrivial; re-implementing it inside litai to make specs "runnable" would be exactly the kind of "invent a universal engine" scope the issue's own framing warns against, and duplicates what the target language's own decision-table library (if the generated code chooses to use one) already provides at the implementation layer.
  3. Returns a SpecificationSet-like structure — but not literally a SpecificationSet, because (per the seam finding above) that contract's SpecificationScenario hard-requires non-empty when/then tuples. A DMN rule row is not a scenario. Two structurally honest options:
    • (a) Add a DMN-native sibling type, e.g. DecisionTableSet/DecisionRule mirroring SpecificationRequirement/SpecificationScenario's shape (rule_id, input cell values, output cell values, hit policy) — a genuine data-model addition, not a reuse of the existing WHEN/THEN shape. This is honest about DMN being a different oracle shape, at the cost of component_lock_planning.py and legacy_generation_catalog.py needing to handle "the specification set might be one of two shapes" rather than one.
    • (b) Treat the whole .dmn file as a single opaque content-addressed blob, structurally identical to how assets:/ComponentAssetSelector already treats a JSON lookup table — no SpecificationRequirement/SpecificationScenario parsing at all, just a pinned ContentIdentity plus a provider-side structural validator that runs at lock time as a side effect rather than producing a parsed tree the rest of the pipeline consumes. This is the minimum-code path and the one this document recommends for a first version: it needs no new contract types, reuses assets:'s exact "declarative structured content, hash-pinned, execution deferred to generated code" precedent byte-for-byte, and defers (a)'s richer modeling until there's a concrete need (e.g. spec review wanting to render individual DMN rules — see Q5) that opaque-blob treatment can't satisfy.
  4. Generated code consumes the decision table however the target language/skill convention prescribes — e.g. a Python Component's generation skill reading the .dmn file and either hand-translating it to an if/match chain, or depending on a DMN execution library (pydmn-style, or hand-rolled) as an ordinary runtime dependency. This is exactly analogous to how assets:'s JSON lookup tables get read by generated code at runtime — litai never executes them itself.

Q4 — Generation-key binding: does a DMN table change content-hash computation?

Proposed answer: no change to the hashing mechanism itself; the existing provider-neutral raw-byte hashing already covers it, per the seam finding above. _specification_references() hashes specification_roots files as raw bytes before any provider-specific parsing happens, and only cross-checks that the provider's own reported artifact identities match those raw-byte hashes (component_lock_planning.py:1603-1609). A .dmn file's bytes hash exactly like a .md file's bytes — XML vs. Markdown is irrelevant to hashlib.sha256. The one DMN-specific nuance worth calling out explicitly (not a mechanism change, a modeling note): under the FIRST and PRIORITY hit policies, rule order is semantically significant (Q2), so a DmnProvider's validation must not, e.g., sort rules canonically before computing any digest it reports — the raw file bytes already preserve order faithfully, but if a future richer parsed representation (Q3 option (a)) is ever added, its own canonicalization must preserve rule order rather than alphabetizing or otherwise normalizing it away, unlike genuinely order-independent data (e.g. canonical_literal_table_json's sort_keys=True, which is safe only because dict key order is not semantically meaningful for a lookup table but would be wrong for a FIRST/PRIORITY DMN rule list).

Q5 — How would spec coverage/spec review need to treat DMN's output?

Grounding note: the spec derive/spec accept/spec coverage/spec review commands actually found in src/literate_ai/cli/source_to_specification.py are the source-to-specification migration pipeline (turning existing, undocumented source code into a draft specification via model-assisted derivation, then human review and promotion) — a different subsystem from the per-Component specification_provider seam this document is otherwise grounded in. Its coverage command (source_to_specification.py:3351-3359) reports result.coverage.entries, a behavioral-surface coverage count keyed by surface_id, and its review command (:2682-2730) walks result.draft.statements to build per-statement accept/reject decisions. Both are structured around the derivation pipeline's own SpecificationSet (defined independently in source_to_specification/review.py:105, not the same class as contracts/specifications.py's — a second, pipeline-local type with the same name, worth flagging as a minor naming collision risk for whoever touches this next) and its behavioral-surface/statement abstraction — not the Component-authoring provider seam.

This means MEDIUM(b)'s DMN provider and the spec derive/accept/coverage/ review pipeline are only loosely related, and the design should say so plainly rather than force a connection:

Q6 — Do litai lock/litai generate need changes for provider-neutrality?

Proposed answer: litai generate already generalizes (confirmed by direct grounding above — it treats specification bytes as opaque context regardless of provider). litai lock needs the two small, symmetrical hardcoded-dispatch edits identified at the top of this document, but no architectural change.

Concretely, for DMN to exist as a third provider:

  1. component_lock_planning.py:1581-1596 gains a third elif authoring.specification_provider == "dmn": arm calling DmnProvider().load(root, authoring.specification_roots).
  2. legacy_generation_catalog.py:598-614 gains the symmetrical third arm; its return type annotation (LoadedOpenSpec) needs to widen — either to a Protocol both LoadedOpenSpec and a DMN equivalent satisfy (if Q3 chooses option (a)'s richer modeling), or stays exactly LoadedOpenSpec unchanged (if Q3's recommended option (b), opaque-blob treatment, is chosen — a DmnProvider.load() could legitimately return a LoadedOpenSpec whose specification_set.requirements is a single synthetic placeholder entry satisfying the non-empty-tuple validation, with the real content living only in contents/artifacts; this needs its own small design decision at implementation time about whether a synthetic requirement is more honest than widening the type, but is not a blocking unknown — both are mechanically small).
  3. component_lock_planning.py:1598-1602's exception tuple except (OpenSpecError, OSError) needs DmnError added (or a shared base exception type both OpenSpecError and DmnError derive from, slightly cleaner long-term but not required for a first version).
  4. No changes identified anywhere in generation_preparation.py, component_lock_resolution.py, or locks.py beyond what already exists — these consume ContentReference/ContentIdentity generically today and were not written assuming exactly two providers.

This is a small, additive, well-bounded change surface once Q3's minimum-viable-adapter question is answered — the seam "already generalizes" in the sense that matters most (hashing, generation consumption) and needs mechanical, not architectural, extension for the two places that currently hardcode two provider names.

Follow-on prototype results (2026-08-19)

The three next-action items this document's original "Maturity / next-action assessment" section called for have now been executed as a narrow, throwaway prototype under docs/roadmap/0.6.0-dmn-prototype/, with its parser/checker script. None of this touches src/literate_ai/ — it is scoped exactly to proving out Q2/Q3 against real DMN XML, as instructed, not to implementation.

Item 1 — real .dmn file + minimal loader + UNIQUE overlap check, run end to end. docs/roadmap/0.6.0-dmn-prototype/risk-category.dmn is a hand-authored, valid DMN 1.3 XML file (https://www.omg.org/spec/DMN/20191111/MODEL/ namespace): one <decision>/<decisionTable>, hitPolicy="UNIQUE", 2 input columns (Age, Income, both typeRef="number"), 1 output column (RiskCategory, typeRef="string"), 5 rules using numeric-interval ([18..25]) and dontcare (-) FEEL unary tests. docs/roadmap/0.6.0-dmn-prototype/dmn_prototype.py parses it with xml.etree.ElementTree into a small DecisionTable/Rule dataclass structure and runs a UNIQUE-hit-policy overlap check (pairwise interval intersection across all input columns). Run against the file as authored:

$ python3 dmn_prototype.py risk-category.dmn
...
UNIQUE hit policy check: PASSED (no overlapping rule input conditions)

Then a rule was deliberately mutated (Rule5's Age interval widened from [56..120] to [50..120], which now overlaps Rule3's [26..55] and Rule4's [26..55]) and re-run against the mutated copy:

UNIQUE hit policy VIOLATED — 2 overlapping rule pair(s):
  - RiskCategoryTable_Rule3 inputs=('[26..55]', '[0..40000]') overlaps RiskCategoryTable_Rule5 inputs=('[50..120]', '-') on every input column
  - RiskCategoryTable_Rule4 inputs=('[26..55]', '[40001..999999]') overlaps RiskCategoryTable_Rule5 inputs=('[50..120]', '-') on every input column
exit=1

The mutated file was not kept (test performed against a /tmp copy, discarded after). This confirms end to end, against real DMN XML rather than the standard's prose: (a) a minimal loader can extract decision-table structure without a full DMN engine, and (b) the UNIQUE overlap check both passes clean tables and fails closed on a genuine overlap, which is exactly the "structural validation at load time" oracle shape Q2 proposed. Q2's confidence should move from Medium to High.

Item 2 — parsing approach: hand-rolled, stdlib-only, chosen over pyDMNrules. pyDMNrules was checked (pip3 show pyDMNrules on this machine — not installed; its published scope is a full DMN execution engine: S-FEEL/FEEL parsing via a generated ANTLR grammar, glossary-driven decision execution, and Excel-based authoring, not a bare structural parser). Pulling in a full execution engine as a dependency to do only structural validation is precisely the "invent/adopt a universal engine" scope Q3 already argued against for FEEL evaluation specifically — the same argument applies to the parsing dependency choice, not just execution. The prototype instead hand-rolls a parser using only xml.etree.ElementTree (stdlib, zero new dependency), scoped deliberately narrow: exactly one <decision>/<decisionTable> per file, and a FEEL unary-test subset limited to numeric closed intervals ([a..b]) and dontcare (-) — enough to validate this prototype's table, not the full FEEL grammar. This is a real, load-bearing scoping decision, not a permanent one: a production DmnProvider doing Q2's full hit-policy/coverage/type validation (string-list tests, open intervals, negation, FIRST/PRIORITY/COLLECT policies) will need a materially larger unary-test parser than this prototype's ~15-line _parse_numeric_interval/_intervals_overlap pair. The recommendation is still hand-rolled-and-scoped over a dependency (matches Q3's existing "no execution engine" reasoning and keeps the dependency surface at zero), but the cost estimate for "hand-rolled" should be revised upward from what this document originally implied — a real FEEL unary-test grammar (even execution-free) is more than a trivial parsing exercise. Flag this as the one place the original spike underestimated effort.

Item 3 — re-verify the two hardcoded dispatch sites (Q6). Ran the exact command the document specifies:

$ grep -rn "specification_provider ==" src/literate_ai/
src/literate_ai/adapters/legacy_generation_catalog.py:601:    if definition.specification_provider == "openspec":
src/literate_ai/adapters/legacy_generation_catalog.py:603:    if definition.specification_provider == "literate-markdown":
src/literate_ai/adapters/component_lock_planning.py:1581:        if authoring.specification_provider == "openspec":
src/literate_ai/adapters/component_lock_planning.py:1583:        elif authoring.specification_provider == "literate-markdown":
src/literate_ai/adapters/component_markdown.py:297:        authoring.specification_provider == "literate-markdown"

The two dispatch sites named in the original spike (component_lock_planning.py, legacy_generation_catalog.py) are unchanged and are still the only two that dispatch on specification_provider (choose which provider implementation to invoke). A third match has appeared since the spike was written: component_markdown.py:297. Reading it in context (src/literate_ai/adapters/component_markdown.py:284-309, render_component_markdown/frontmatter-serialization helper) shows it is not a dispatch site in Q6's sense — it is a single equality check deciding whether to omit the specification_provider/specification_roots frontmatter lines when serializing a Component's authoring metadata back to component.md (the omission is an optimization: don't write the field when it's already at its default, literate-markdown + ("component.md",)). It does not instantiate or call a provider and would not need a DMN-specific arm added — a DMN-authored Component would simply always have its specification_provider/specification_roots lines written explicitly (since "dmn" is never the default), which this code already handles correctly with no change needed. Net effect: Q6's implementation change-surface claim (component_lock_planning.py + legacy_generation_catalog.py, two symmetrical edits) is still accurate for what needs editing, but the re-verification command's raw hit count grew from 2 to 3 since the spike was written, and whoever implements this should re-run the same grep at implementation time rather than trusting either this document's or the original spike's count.

All three follow-on items are now satisfied. Combined with the unchanged Q1/Q4/Q6 findings above, this document's own closing claim — "no further wide-open design questions are expected to remain" once the three items are done — holds, with the one revision noted in item 2 (hand-rolled parsing is real, non-trivial work, not a quick throwaway) that a future implementer should account for in scoping estimates.

Maturity / next-action assessment

Not yet implementation-ready, but closer than a from-scratch design would be — a narrower follow-on step is needed, not another wide-open spike. Specifically:

What a follow-on prototype would need to establish, concretely, before code:

  1. Build one real, hand-authored .dmn file (a small decision table, 4-6 rules, UNIQUE hit policy, 2 input columns + 1 output column) and confirm a minimal DmnProvider.load() can parse its structure and run the UNIQUE-hit-policy overlap check end to end — proves out Q2/Q3 against real DMN XML rather than the standard's prose description of it.
  2. Pick and justify a DMN-XML parsing approach (dependency vs. hand-rolled minimal parser scoped to the exact subset litai needs) — this is squarely an implementation decision, not a design one, but the prototype should record which was chosen and why, since it materially affects Q3 option (a) vs (b)'s relative cost.
  3. Confirm the two hardcoded-dispatch edit sites (Q6) are still exactly these two functions and no others have appeared on main/release/0.5.0 by the time implementation starts — grep -rn "specification_provider ==" src/literate_ai/ is the exact re-verification command.

Once those three are done, this becomes implementation-ready in the normal sense — no further wide-open design questions are expected to remain.

Summary table

Question Answer reached here Confidence
Q1 — authoring/discovery shape .dmn XML file under specification_roots, discovered identically to existing providers High — directly grounded, no open sub-question
Q2 — acceptance-oracle shape Structural hit-policy/coverage/type validation at lock-planning time, not scenario pass/fail High — validated end to end against a real .dmn file and a deliberately-introduced overlap (see "Follow-on prototype results")
Q3 — minimum viable adapter No DMN execution engine; opaque content-pinned blob (assets:-style), execution deferred to generated code Medium — the "no execution engine, hand-roll narrowly" direction is confirmed, but the (a)/(b) shape choice is still open, and the hand-rolled-parser cost estimate was revised upward (see "Follow-on prototype results")
Q4 — generation-key binding No mechanism change; raw-byte hashing already provider-neutral; rule-order preservation is the only nuance High — directly grounded, no open sub-question
Q5 — spec coverage/spec review treatment Migration-pipeline commands are a different subsystem, largely orthogonal; DMN should not be a spec derive target High — directly grounded by code audit
Q6 — lock/generate provider-neutrality generate already neutral; lock needs two small symmetrical hardcoded-dispatch edits High — directly grounded, no open sub-question

0.6.0 implementation status (2026-08-23)

The narrow MVP is implemented under src/literate_ai/adapters/specifications/. Both Component lock planning and the legacy generation catalog now call one shared built-in provider registry. DmnProvider performs bounded file loading, rejects DTD/entity declarations before stdlib XML parsing, accepts only the explicit DMN 1.3 UNIQUE decision-table subset documented in the user guide, validates type/arity and pairwise overlap, projects one synthetic structural-validation requirement into the existing SpecificationSet, and retains exact source bytes for generation and drift checks.

This intentionally resolves Q3 with the opaque-artifact option. It is not a FEEL or DMN execution engine, does not broaden self-hosting beyond its existing OpenSpec-only path, and does not implement other hit policies or input syntax. Focused implementation tests are evidence for this blocker only; they are not full 0.6.0 release qualification, which remains owned by RELEASE-006.

Read the Markdown sourceConference homePublished 1.3.0 · Patch 1.3.1 in progress · Development 1.4