#+TITLE: Test Coverage
#+SUBTITLE: Deterministic CLI, file output, and visual smoke tests for HasciiCam
#+DATE: 2026-05-26
#+KEYWORDS: hasciicam tests ctest cli html sdl synthetic camera

* DONE [#A] Define The Test Strategy And Boundaries
:PROPERTIES:
:Effort: 0.5 day
:Goal: Keep tests simple, deterministic, and C-first without adding a framework.
:Notes: Use CTest as the runner. Add plain C helpers only where direct core access is needed. Avoid new dependencies.
:END:

** DONE [#A] Document the test tiers in AGENTS.md
:PROPERTIES:
:Why: Future agents need to know which tests are required in CI, which are opt-in, and which are manual camera checks.
:Change: Update AGENTS.md with four tiers: unit/core C tests, deterministic CLI smoke tests, opt-in visual SDL tests, and opt-in real camera tests. State that no external test framework should be added until plain C + CTest becomes painful.
:Tests: Documentation-only; skip build.
:Done when: A future agent can decide where a new test belongs before editing code.
:END:

** DONE [#A] Keep CTest as the single runner
:PROPERTIES:
:Why: The project already uses CMake and CTest, so adding another runner would complicate cross-platform maintenance.
:Change: Make all automated tests visible through CTest. Use CMake labels such as unit, cli, visual, camera if useful.
:Tests: Configure with HASCIICAM_ENABLE_TESTS=ON and run ctest --output-on-failure.
:Done when: ctest lists all non-manual tests and no separate test command is required.
:END:

* DONE [#A] Add A Deterministic Synthetic CLI Source
:PROPERTIES:
:Effort: 1-2 days
:Goal: Let the real CLI run for a bounded number of frames without camera hardware.
:Notes: This is the key enabler for HTML/text/stdout/SDL smoke tests.
:END:

** DONE [#A] Add a synthetic capture backend
:PROPERTIES:
:Why: CLI output tests should not depend on camera permissions, hardware, or backend availability.
:Change: Add a capture backend selected by device "synthetic://" or a similarly explicit selector. It should generate deterministic grayscale frames with motion across frames. Keep it in src/capture and behind capture_ops like every other backend.
:Tests: Add a CTest that opens synthetic capture through capture_open_default and reads at least two frames with non-identical contents.
:Done when: The CLI can run without a physical camera by passing -d synthetic://.
:END:

** DONE [#A] Add a bounded frame-count option
:PROPERTIES:
:Why: CLI smoke tests need hasciicam to exit on its own after rendering a known number of frames.
:Change: Add a small option such as --frames N to hasciicam_config. Keep it documented as a test/smoke option. In the main loop, request session stop after N rendered frames, not N capture reads.
:Tests: Run hasciicam -d synthetic:// --frames 2 -O stdout and verify process exits with code 0.
:Done when: Every output mode can be exercised by CTest without killing a long-running process.
:END:

** DONE [#B] Keep synthetic source out of default real-camera selection
:PROPERTIES:
:Why: Synthetic capture is a test tool and should never hide a broken platform camera backend in normal use.
:Change: Only select synthetic capture when the device string is exactly synthetic://. Do not use it as fallback after real backends fail.
:Tests: Run a bad device selector and verify normal backend failure still reports real capture errors.
:Done when: Synthetic behavior is explicit and cannot mask production capture regressions.
:END:

* DONE [#A] Add Deterministic CLI Smoke Tests
:PROPERTIES:
:Effort: 1-2 days
:Goal: Verify the executable behavior users rely on: help, stdout, text, HTML, and clean exit.
:Notes: Prefer small CMake/CTest wrappers. Use PowerShell/cmd only where Windows shell semantics are unavoidable.
:END:

** DONE [#A] Add help and AA help tests
:PROPERTIES:
:Why: CLI option parsing is compatibility-sensitive and cheap to verify.
:Change: Add CTest tests for hasciicam -h and hasciicam -H. Check for expected strings such as "Usage: hasciicam" and "aalib options".
:Tests: Run ctest -R cli_help --output-on-failure.
:Done when: CLI help regressions fail in CI.
:END:

** DONE [#A] Add stdout output test
:PROPERTIES:
:Why: stdout is the simplest live-style output and exercises the real CLI render path without a window.
:Change: Run hasciicam -d synthetic:// --frames 2 -O stdout. Capture output to a file and assert it contains non-whitespace ASCII.
:Tests: Run ctest -R cli_stdout --output-on-failure.
:Done when: The CLI can render visible ASCII through stdout in an automated test.
:END:

** DONE [#A] Add text file output test
:PROPERTIES:
:Why: Text output is a supported non-visual mode and should not regress while changing output adapters.
:Change: Run hasciicam -d synthetic:// --frames 2 -m text -o <test-output>.txt. Assert file exists, is non-empty, and contains non-whitespace ASCII.
:Tests: Run ctest -R cli_text --output-on-failure.
:Done when: Text output is covered by deterministic CI.
:END:

** DONE [#A] Add HTML file output test
:PROPERTIES:
:Why: HTML mode has had regressions and is part of original HasciiCam behavior.
:Change: Run hasciicam -d synthetic:// --frames 2 -m html -o <test-output>.html. Assert file exists, contains the refresh meta tag, contains <PRE>, and contains non-whitespace ASCII inside the generated page.
:Tests: Run ctest -R cli_html --output-on-failure.
:Done when: Empty HTML pages fail deterministically in CI.
:END:

* DONE [#B] Add Opt-In Visual And Camera Tests
:PROPERTIES:
:Effort: 1-2 days
:Goal: Exercise SDL windows and real camera paths without making CI flaky.
:Notes: These tests should be off by default and enabled only on maintainer machines.
:END:

** DONE [#B] Add an opt-in SDL visual smoke test
:PROPERTIES:
:Why: SDL can fail in ways stdout/text/HTML cannot catch, but CI often lacks a display.
:Change: Add HASCIICAM_ENABLE_VISUAL_TESTS=OFF by default. When ON, add a CTest that runs hasciicam -d synthetic:// --frames 2 -O SDL and verifies the process exits cleanly.
:Tests: Configure with HASCIICAM_ENABLE_VISUAL_TESTS=ON and run ctest -R visual_sdl.
:Done when: Maintainers can validate SDL launch/render/quit with one opt-in test.
:END:

** DONE [#B] Add opt-in camera smoke tests
:PROPERTIES:
:Why: Real capture backends need manual/hardware validation, but should still have repeatable commands.
:Change: Add HASCIICAM_ENABLE_CAMERA_TESTS=OFF and a HASCIICAM_TEST_CAMERA_DEVICE cache variable. When enabled, run a bounded real-camera test with --frames 2, preferably to text or stdout.
:Tests: Configure with a real camera device/matcher and run ctest -R camera.
:Done when: Camera tests are available locally but never run accidentally in CI.
:END:

* DONE [#B] Add Small Reusable Test Helpers
:PROPERTIES:
:Effort: 1 day
:Goal: Avoid duplicating fragile file/string checks across CTest wrappers.
:Notes: Keep helpers plain C or CMake script. Do not add a test framework.
:END:

** DONE [#B] Add a file content assertion helper
:PROPERTIES:
:Why: HTML/text/stdout tests need repeated checks for file existence, non-empty content, and required substrings.
:Change: Add a small C helper or CMake script under tests/ that accepts a file path and required substrings/non-whitespace checks.
:Tests: Add one negative CTest case only if it does not complicate normal runs; otherwise rely on positive tests.
:Done when: CLI smoke tests use one shared assertion helper.
:END:

** DONE [#C] Add labels and output directories
:PROPERTIES:
:Why: Test output files should not clutter the source tree and labels make partial test runs easier.
:Change: Put generated files under the CTest build directory. Add CTest labels: unit, core, cli, visual, camera.
:Tests: Run ctest -L cli and ctest -L unit.
:Done when: Test artifacts stay under the build tree and labeled subsets work.
:END:

* DONE [#C] Update Documentation And CI
:PROPERTIES:
:Effort: 0.5-1 day
:Goal: Make the test workflow discoverable and keep CI deterministic.
:Notes: CI should run deterministic tests only.
:END:

** DONE [#A] Update AGENTS.md and docs/smoke-tests.md
:PROPERTIES:
:Why: Human maintainers and agents need the same test commands and opt-in rules.
:Change: Document synthetic://, --frames, deterministic CLI tests, visual tests, and camera tests. Include Windows vcvarsall guidance for running tests locally.
:Tests: Documentation-only; skip build unless code changes are in the same L2.
:Done when: A future agent can add a test without rediscovering the strategy.
:END:

** DONE [#B] Update CI to include deterministic CLI tests
:PROPERTIES:
:Why: CI should catch empty HTML/text/stdout regressions after synthetic CLI support lands.
:Change: Keep SDL and camera disabled in CI. Ensure CI runs ctest with deterministic unit/core/cli tests on Windows, Linux, and macOS.
:Tests: Local ctest; CI validates on push/PR.
:Done when: CI covers help, stdout, text, HTML, conversion, core link, and synthetic pipeline smoke.
:END:








