Guidelines for YANG Example Validation and Coverage Analysis in IETF Documents

Introduction Although the YANG data modeling language has been in use for over a decade, there remains a lack of tooling to help YANG module authors analyze and validate their YANG modules and example instances. People defining or evaluating YANG modules have limited guidance on how to include, extract, test, and evaluate the impact of example data for those modules. Example data instances are vital to illustrate how a module is intended to be used. provides guidelines for authors and reviewers of documents containing YANG data models, including recommendations for usage examples and validation of valid data instances. The YANG data modeling language itself is specified in . However, gaps still exist in defining conventions and tools that can automatically extract examples, validate them, and give authors feedback on what they might be missing in their modules. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in when, and only when, they appear in all capitals, as shown here. The objective of this document is to define guidelines that enable the development of tools which, given one or more IETF documents (RFCs or drafts), can extract YANG module examples, validate that they are correct, and analyze whether they sufficiently cover the module. To achieve this, we address two main aspects:

Guidelines for Including YANG Examples in RFCs: A clear method for authors to include example data instances of YANG modules in IETF documents in a way that tools can automatically extract and validate them. In addition, we outline how tools can perform this extraction and validation, and we propose a method to include extra YANG modules (when necessary) for validating more complex examples without affecting the main normative content.
Definition of YANG Module "Coverage": A new concept to measure how much of a YANG module's data nodes are exercised by the example instances in a document. By analyzing the examples, a tool can inform authors which parts of their YANG module are covered and which are still missing from the examples. A proof-of-concept implementation for computing such coverage is also considered.

Scope: The proposals in this document focus on typical use cases. It is expected that proof-of-concept tools will confirm the viability of these methods for moderately complex examples as this document progresses. If such automatic validation proves infeasible in practice, the approach may need adjustment. Extremely complex example scenarios (e.g., those involving schema mount or highly dynamic state) are outside the current scope and could be addressed in future work.

Including YANG Example Instances in IETF Documents When authors include example YANG instances in an RFC or Internet-Draft, those example instances are RECOMMENDED to be formatted in a way that allows automatic extraction and validation by tools. Specifically, example instances SHOULD be enclosed as code blocks using the CODE BEGINS / CODE ENDS convention in the rendered document. In RFCXML, this can be achieved with a sourcecode element that uses a name attribute and the markers attribute. They should be given a file name that clearly identifies them as examples, analogous to how YANG modules are named (see ). We propose that the name of valid example instances start with the prefix `yang/examples/valid-` and end with `.xml` for XML instances or `.json` for JSON instances. This naming convention makes it easy for tools to identify which code snippets are YANG instance examples that can be parsed and validated. The intent is not to replace the example-validation guidance in , but to make that guidance easier to apply automatically. This convention tries to align itself with the repository layout used by the "Internet-Draft with YANG Modules" template . Each example file name SHOULD be unique within a document (and ideally unique across related documents being analyzed together). Tools may complain or error out if they encounter duplicate example file names, since that could indicate conflicting or repeated content. For example, an XML instance might be included as: ... ]]> And a JSON instance as: By following this approach, a tool can scan the RFCXML source or the rendered document for code blocks whose file names are under "yang/examples/", start with "valid-", and end in ".xml" or ".json", extract their contents, and validate them against the appropriate YANG modules. In cases where an included code snippet is not meant to represent a valid YANG instance (for instance, it might be pseudocode, a partial fragment, or intentionally invalid to illustrate an error), the author can indicate this so that automated validators skip it. For example, an author might choose not to use the CODE BEGINS/CODE ENDS markers, might not place the snippet under "yang/examples/", or might not use the "valid-" prefix for such a snippet. Without the markers or filename convention, the tool will know not to treat that snippet as a YANG instance example to be validated. An author could also include an explicit note in the text indicating that a given snippet is not for validation. Using these conventions strikes a balance between making examples machine-checkable and allowing flexibility for authors to include non-valid illustrative snippets when necessary.

Types of Example Validation Not all example instances are the same in terms of validation. Many YANG examples in documentation are simple and only involve the module defined in the document, but in more complex scenarios an example might span multiple modules or need knowledge of deviations, features, or other context. For the latter, wrapping the example in an instance data file format (defined by ) can make explicit which modules and context are in use. This approach can handle more complicated example validation at the cost of a bit more verbosity. (If a given example scenario is too complex to easily capture even with the instance-data file format - for example, requiring extensive setup or dynamic state - it is considered out of scope for automatic validation in this document.) We distinguish between two categories of examples for validation purposes:

Simple Validation Examples These are straightforward examples that instantiate YANG modules defined in the same document, with a self-contained context. An extraction tool can load the YANG module(s) defined in the document (plus any standard modules they import) and then validate the example instance against those modules. For instance, if a draft defines a YANG module example-interfaces and provides a small XML snippet configuring an interface, the tool will use the example-interfaces module (and any of its dependencies) to verify that the XML conforms to the schema. In general, simple examples illustrate how to use the module(s) in the draft and can be validated with minimal additional information beyond what the document itself provides.

Complex Examples with External References These are examples that require additional context or external modules beyond those defined in the current document. For example, an instance data set may involve multiple modules (perhaps a module from this draft plus some from other RFCs), or it might describe operational state data that requires a specific datastore context or feature set. For such cases, we propose that the example be presented as a self-contained YANG instance data file using the standard format defined in (the YANG Instance Data File Format). In practice, this means the example should be wrapped in the instance-data-set structure which includes the necessary metadata (such as module names, revisions, feature toggle states, and target datastore) for validation. For an XML-formatted complex example, the first element of the snippet should be <instance-data-set xmlns="urn:ietf:params:xml:ns:yang:ietf-yang-instance-data">, with the content formatted according to . Inside this element, sub-elements specify what YANG content schema the instance data conforms to (including module names, revision dates, features, deviations, and the intended datastore). Similarly, for a JSON-formatted complex example, the top-level JSON object should have a single key "ietf-yang-instance-data:instance-data-set", whose value is an object providing the same kind of metadata and the actual data set. By using the instance-data-set wrapper, all the information needed to validate the instance is included within the text itself. A validation tool like yanglint (with appropriate support for ) can read the instance-data-set, automatically load the indicated modules and their revisions, apply any stated deviations or feature enables, and then verify the embedded data content against the assembled schema. Note that if a tool detects that an example instance uses the instance data format, it will attempt to validate it accordingly - and the example instance will fail validation if it does not conform to the declared schema or constraints.

Including Auxiliary YANG Modules for Examples Sometimes an example instance cannot be validated or understood without additional YANG definitions that are not part of the main module in the document. A common case is the use of YANG deviations or vendor-specific augmentations to adapt a base module. In principle, deviations are meant to be extraordinary measures (and their use is discouraged in final standards), but in practice they are sometimes used in industry deployments. To illustrate how a deviation or augmentation affects the data, an RFC might need to include a small auxiliary YANG module (for example, a module that contains deviation statements targeting the base module) purely for use in the example validation. The challenge is to do this without confusing readers or tools and without "contaminating" the main normative content of the document. We propose the following guidelines for including such auxiliary YANG modules in examples:

Keep Auxiliary Modules Separate: Include the auxiliary module in a way that makes it clear that it is not a normative YANG module from the document. This can be done with a dedicated directory (e.g. the file name starts with `auxiliary_modules/`), or a local meaninful extension. Any example that requires this auxiliary module would need to reference reference it by its full path when running the validation tool.
Clearly Identify the Module as Non-Normative: The auxiliary module should be clearly identified as an example-only, non-normative module. This can be achieved through both its naming and its content. For instance, the file name can include words like "example" or "dev" to signal its purpose. In the module source itself, use a module name and namespace that indicate it is not an official module. For example, use a namespace URI containing "example.com" or similar, and perhaps a module name ending in -example or with a clearly indicative name.
Explain in the Narrative: In the text surrounding the code, explicitly state that this module is an auxiliary construct provided for the sake of the example. For instance: "The following YANG module is an example showing how a deviation could be used for this scenario; it is not a normative part of this specification." This clarification is important for human readers and reviewers to avoid any confusion.

By following these practices, authors can include necessary YANG deviations or supplemental modules to support their examples while making it clear that these are not part of the official data model being defined. Tools that extract and compile YANG modules from the document can be instructed to either ignore these example-only modules or handle them separately. For example, a tool might compile them only in the context of validating the corresponding example, and not raise an error about their presence or treat them as missing normative modules in the draft.

YANG Module Coverage by Examples Providing examples in YANG module documents is not just helpful for readers, but can also improve the quality of the module itself. We introduce the concept of YANG module coverage as a way to measure how thoroughly the provided examples exercise the data model. By analogy to code coverage in software testing (which checks what fraction of code is executed by tests), YANG module coverage looks at what fraction of a module's data nodes are present in at least one of the example instances. For the baseline metric, coverage is defined over terminal data nodes, that is, leaf and leaf-list nodes from the YANG module that have a value in one or more example instances. A simple quantitative metric could be the percentage of the module's leaf and leaf-list nodes that appear in at least one example. For instance, if a YANG module defines 100 configurable or state leaf nodes, and the examples collectively include 45 of those leaves, one could say the example coverage is 45%. This number, while not telling the whole story, gives an indication of how much of the module's functionality is demonstrated by the examples. Low coverage might suggest that important parts of the module are never shown in any example. This could mean those parts are either less relevant or simply that the document might benefit from additional examples to cover them. Probably more helpful than the coverage percentage, we can also define an "uncovered-node tree" - essentially a view of the data model highlighting the nodes that were not covered by any example. This is perhaps the most direct way of showing the authors which portions of their module remain unused in the examples. For example, a tool could output a tree or list of all leaf and leaf-list nodes not present in any example instance, indicating potential gaps in example coverage. This kind of feedback can help authors identify gaps and improve their documents. If a particular feature or branch of the module has no example, the authors might choose to add one for completeness or clarity. In some cases, it might reveal that certain parts of the module are hard to exemplify or not yet implemented, which could prompt re-thinking that part of the design or at least adding an explanation for why it's not covered by any example. Coverage works directly on the data nodes from the YANG module. It should be as simple as possible, therefore, we will consider that any data leaf or leaf-list under a list is covered if there is at least an example with an instance of the list that includes it. That is, for a list with 2 different leaves, one leaf covered under one key and another under another one will still yield a 100% coverage of the list leaves.

Coverage Example Consider this simple YANG module: And this example instance: router1 eth0 true ]]> Coverage analysis shows:

Total terminal data nodes: 6 (hostname, location, admin-state, interface/name, interface/enabled, interface/other)
Covered nodes: 3 (hostname, interface/name, interface/enabled)
Coverage percentage: 50%

Uncovered-node tree (missing nodes):

This analysis helps authors identify that the 'location' and 'admin-state' leaves need examples.

Coverage Across Multiple Documents It is common that a YANG module defined in one document might be used, extended, or exemplified in another document. Examples could be spread across multiple RFCs or drafts (for instance, a base YANG module in an RFC and a usage guide in a separate draft that provides additional examples). While our primary focus is on coverage within a single document's scope (i.e. how well the examples in this document cover this document's module), one could imagine tooling that aggregates examples from multiple sources for a more holistic coverage analysis. For example, if module A is defined in RFC X, and RFC Y (perhaps a companion or an applicability statement) provides additional examples for module A, a combined coverage analysis across X and Y would give a better picture of how module A is covered in total. We will strive to define approaches that enable tooling to do this kind of multi-document coverage assessment automatically in the future. For the concept of YANG packages , the coverage across the entire package might be of interest. For the newly created ONSEN Working Group, the coverage across (service) abstractions might be of interest.

Open Issues and Future Work The baseline metric in this document is intentionally limited to leaf and leaf-list coverage. Feedback received on version -00 identified additional useful information that is not naturally represented by the baseline uncovered-node tree.

Union member types: If a covered leaf is of type union, a future version can define how to report which union member types were exercised by the example instances. The leaf itself may already be covered, while only some of its possible member types were used. Reporting this properly requires an uncovered tree format beyond the YANG tree definition from .
Enumeration values: If a covered leaf is of type enumeration, a future version can define whether tools should report which enumeration values were exercised by the example instances. Similar to the case of Union member types, we will need to expand the definition of YANG tree from to show these values.
Presence containers: A future version can define whether instantiating a presence container counts as coverage independently from the leaves below it. This is a separate question from the baseline leaf and leaf-list metric because a presence container can carry meaning by existing.
Groupings: The proof-of-concept implementation already handles the schema nodes created by uses of a grouping. Further work is needed to decide whether grouping-specific metrics are useful, and how they should be reported without confusing abstract grouping definitions with instantiated schema nodes.
Modules without data nodes: A future version can define clearer reporting for modules that primarily contain groupings, typedefs, identities, extensions, RPCs, actions, or notifications, where the baseline data-node coverage metric may not apply.

Implementation A proof-of-concept implementation has been developed in the yangson-coverage repository . The implementation uses Yangson as a library for loading YANG modules and validating instance data, and then implements the coverage logic described in this document. Yangson already exposes validation counters for schema nodes and can print a complete schema tree annotated with those counters. This is useful input, but it is not the report format defined here. The proof of concept tracks covered leaves across validated examples and generates the uncovered-node tree described in this document: covered terminal nodes are removed, and parent nodes are kept only as context. The proof of concept also provides a command-line interface to reset coverage state, validate examples while accumulating coverage, and print the uncovered-node report. It includes initial support for grouping-level reporting, while the exact grouping metrics remain future work. Other related tooling includes yanglint , example validation scripts such as yang-validation , the "Internet-Draft with YANG Modules" template , and the associated I-D-with-yang-tools repository . These tools and templates support validation of YANG modules, tree generation, and validation of examples.

Conclusion By standardizing how YANG examples are included and validated, we make it easier for implementers and reviewers to trust that the examples in a specification are correct and that a sufficient range of scenarios is covered. The concept of YANG module coverage, while exploratory, can further encourage authors to include more comprehensive examples and help pinpoint which parts of a data model might need additional illustration. Together, these measures aim to improve the overall quality and usability of YANG modules in IETF documents. Ultimately, the success of these proposals will depend on tool support. As a next step, developing proof-of-concept tools or integrations (for example, enhancements to existing YANG validators like yanglint, or new scripts to extract and analyze examples) will be crucial. If moderately complex examples can be automatically extracted, validated, and measured for coverage with such tools, it will validate the approach. On the other hand, if significant obstacles arise during implementation, the recommendations in this document may need to be refined. Our hope is that these initial guidelines spark discussion and experimentation, leading to more robust support for YANG examples and better validation practices in the near future.

IANA Considerations This document has no IANA actions.

Security Considerations This document defines guidelines for including and validating YANG examples in IETF documents. The recommendations do not introduce new security risks. However, implementers of validation tools should be aware of potential security considerations when processing example data, auxiliary modules, or repository contents from untrusted sources, such as denial of service attacks through maliciously crafted large or complex examples.