Menu
Article page

Article

The JSON Schema tester you really need

Test a folder of JSON data against a selected schema, retain the result, and use coverage evidence to improve the data contract.

Explore JSON Schema testingRead validation Help
JSONBuddy for Windows Test collections view listing JSON Schema test entries
The established Test collections view lists repeatable JSON Schema test entries and their data folders.

Test a data collection against its schema

A JSON Schema tester checks how a set of JSON data files validates against a selected schema resource. It is useful during schema design and when repeatable data checks need more context than a one-off online validator.

Create a repeatable test entry

  1. Open the Testing pane and select Create new.
  2. Name the entry so the schema and data set are clear to everyone using the test.
  3. Select the JSON Schema resource or the schema pool that provides it.
  4. Select the local folder containing the JSON test data, add the test, then run it from Test collections.

After a run, inspect the detailed validation errors, warnings, and processing information for each file.

Compare results and inspect coverage

The tester records a reference result that can be compared with later runs. It also exposes schema definition coverage, showing which parts of the selected schemas were exercised by the test data. Treat a coverage result as a review aid: confirm that the intended positive and negative cases are represented before relying on it as evidence of completeness.

Verify the test configuration

  • Open at least one failed and one expected-valid result to confirm the intended schema resolution context.
  • Keep the input folder and the selected schema version identifiable for a repeatable run.
  • Review changed results against the reference, then investigate the relevant data or schema change.

Continue with the closest workflow

Use the JSON Schema tester for the capability overview, the JSON Schema editor to change the contract, or validation Help for a focused correction sequence.

Keep the useful test evidence together

The tester describes a collection-level summary report with the overall status, error and message counts, and coverage. Keep that report with the schema revision and test-data revision it represents. A changed summary is a signal to inspect the per-file evidence; it does not establish whether a changed diagnostic is a regression, a deliberate schema update, or a modified fixture.

Use a complete set of expected cases

Include ordinary valid documents, invalid documents that each target one important constraint, and edge cases for optional fields, arrays, reused definitions, and resolved references. When a case is intentionally invalid, record the expected failure so later test runs distinguish a desired diagnostic from a newly introduced one.

Review the results as a change record

A repeatable schema test becomes more useful when its result can be compared meaningfully over time. Record the test-entry name, schema resource or pool, input-folder revision, run date, product build, and the accepted reference result. Keep a note of any intentionally changed result before updating the reference, so the history explains why the expected outcome moved.

Use detailed findings to locate the data instance and the governing schema definition. If a result changes unexpectedly, first confirm the selected schema and reference resolution context, then compare the changed fixture with the previous input. A coverage change can reveal that a new example reached a branch, but it does not remove the need to review the asserted result.

For a release gate, preserve the evidence needed to reproduce the collection and run it again after material changes. Do not promise unattended CI behavior, report formats, exit codes, or performance characteristics unless those capabilities are documented for the installed version.