Test Trait Tagging

Test Trait Tagging

Analyze an existing test suite in any supported language and apply a standardized set of trait tags to each test method, giving teams visibility into their test distribution (positive vs. negative, critical-path coverage, smoke tests, etc.).

Language-specific guidance: Try test-analysis-extensions once. If it is unavailable, continue immediately with the built-in framework table below; never block tagging on the helper.

When to Use

• Auditing a test project to understand the mix of test types
• Adding trait attributes to untagged tests
• Generating a summary report of trait distribution across a test suite
• Reviewing whether critical paths have sufficient coverage

When Not to Use

• Writing new tests from scratch (use code-testing-agent for any language, or writing-mstest-tests for MSTest)
• Running or filtering tests (use run-tests for .NET; equivalent native runners elsewhere)
• Migrating between test frameworks
• General quality, smell, flakiness, or assertion audits (use test-anti-patterns or the matching analysis skill)
• Diagnostic .NET executed line/​branch/​Cobertura interpretation or project-wide CRAP risk (use coverage-analysis); raw coverage collection (use run-tests for .NET, native tooling otherwise)
• CRAP analysis for a named method, class, or file (use crap-score)
• Behavioral gaps where a test would survive broken production logic (use test-gap-analysis)

Inputs

Trait Taxonomy

Use exactly these trait names and values. Do not invent new trait values outside this table.

A single test may have multiple traits (e.g., both negative and boundary). At minimum, every test should receive one of positive or negative.

Workflow

Step 1: Detect the language, framework, and tagging capability

Resolve the requested test scope from the current workspace before asking for a path. The skill context's Base directory contains these instructions, not the user's repository. Always inspect the current working directory before claiming that repository files are unavailable. If the prompt's relative path is absent, search the workspace for the named project/​file and retry the exact result. A successful search proves that the target is present; if the normal reader then reports that same path missing, treat the contradiction as a reader path-normalization or transport failure rather than asking the user for files. Use a shell text reader (sed/cat on Unix, Get-Content on PowerShell) only for a confirmed reader availability, transport, or path-normalization failure and only after verifying the canonical path remains inside the current workspace. Stop on content-exclusion, permission/​policy, workspace-boundary, or unknown read failures. Never ask the user for a path or file contents after a workspace search found a readable target.

For an auto-edit framework, a failed patch/​editor call is not a stopping condition only when the failure is confirmed tool availability, transport, or path normalization. Do not bypass stale-context, concurrent-change, permission/​policy, or path-boundary errors. Before a shell fallback, resolve the canonical path inside the current workspace, freshly read the file, and use an anchored transformation that aborts unless the expected old text and exact match count are unchanged. Then re-open the complete file, inspect the diff, and run Step 6 validation. Do not report proposed attributes as completion when the user asked to apply them.

Identify the language and framework. Try the matching test-analysis-extensions guidance once. If unavailable, classify capability from the built-in rules below:

• auto-edit — framework has canonical tag syntax this skill can safely insert (.NET [TestCategory] / [Trait] / [Category] / [Property], pytest @pytest.mark.<name>, JUnit 5 @Tag("..."), TestNG groups = {"..."}, RSpec metadata it "..." , :tag => true, Pester -Tag '...', Kotest @Tags(...), Swift Testing @Tag(.tagName), Catch2 [tag], doctest * doctest::test_suite("tag") decorator).
• report-only — framework has no canonical, agreed-upon tag attribute; report tags in a Markdown table only and do not edit source (Go standard testing without build-tag conventions, Jest/​Vitest without consistent describe-prefix convention, Rust without project-specific cfg conventions, XCTest without a test plan, GoogleTest without test-name prefix conventions, Mocha without describe-prefix conventions).
• convention-based — framework uses naming or file conventions for tagging (Go //go:build integration build tags, file-name suffixes like *_integration_test.go, GoogleTest INTEGRATION_* filter prefix). Only emit canonical edits when the user has confirmed the project convention; otherwise treat as report-only.

Capture the capability before Step 4.

Also lock the requested mode before classification. Do not turn an audit into source edits because canonical attributes are available; edit only for an explicit tagging/​apply request.

Step 2: Scan existing traits

Check which tests already have trait attributes. Use the extension when loaded; otherwise use this built-in table as the source of truth:

Record which tests already have tags to avoid duplication.

Step 3: Classify each test method

Build one canonical inventory containing each discovered test exactly once. Record the test identifier, behavioral classification, and traits in that inventory; use the same rows for source edits, per-test reporting, totals, and distribution counts. Do not hand-count a separate denominator. Before publishing, reconcile the reported total with the number of inventory rows and verify that every row contributes to each displayed trait count.

For each test method without traits, analyze:

1. Method name -- names containing Invalid, Fail, Error, Throw, Reject, BadInput, Null, None, Nil, Negative, raises_, _throws_, _returns_error suggest negative
2. Assertion type -- Assert.ThrowsException / Assert.Throws / Should().Throw() / pytest.raises / expect(fn).toThrow / assertThrows / assert.Error(t, err) / expect { ... }.to raise_error / #[should_panic] / XCTAssertThrowsError / Should -Throw / EXPECT_THROW suggest negative
3. Input values -- null / None / nil / undefined, "", 0, -1, int.MaxValue / sys.maxsize / Number.MAX_SAFE_INTEGER / math.MaxInt64 / i32::MAX, empty collections suggest boundary
4. Setup complexity -- minimal setup with basic assertions suggests smoke; external dependencies (file/​db/​net/​env) suggest integration
5. Comments and names -- references to issue numbers or "regression" / "bug" / "fix for #..." suggest regression
6. Timing assertions -- Stopwatch, BenchmarkDotNet, elapsed-time checks; pytest-benchmark fixtures; benchmark.js; JMH @Benchmark; go test -bench; criterion.rs; XCTMetric; Google Benchmark; kotlinx-benchmark suggest performance
7. Feature centrality -- tests on primary public API entry points or critical user workflows suggest critical-path
8. Security patterns -- validates auth, checks permissions, sanitizes input, tests for injection, handles tokens/​secrets suggest security
9. Parallel/​async constructs -- per-language concurrency primitives (see Trait Taxonomy table) suggest concurrency
10. Fault injection -- simulates failures, tests retries, timeouts, or circuit breakers suggest resilience
11. State mutation -- deletes external records, drops resources, modifies shared/​global state suggest destructive
12. Full-stack flow -- test spans entry point through data layer to final response, covering a complete user scenario suggest end-to-end
13. Config/​settings -- loads configuration, tests missing keys, validates options, checks environment variables suggest configuration
14. Known instability -- test has skip / ignore annotations with comments about flakiness, or names contain "flaky" / "intermittent" suggest flaky
15. Default -- if the test verifies a normal success path, tag positive

When in doubt between positive and negative, read the assertion: if it asserts success -> positive; if it asserts failure -> negative.

For a requested distribution or coverage-shape audit, use available production code to map each test to the exact outcome it exercises before summarizing. Call out duplicated boundary coverage and whether the test inventory represents both sides of named thresholds and the observable collaborator outcomes on business-critical paths. Keep these as concise distribution observations, not new trait values. Do not perform mutation reasoning, prescribe new tests, or expand into the behavioral-gap audit owned by test-gap-analysis.

Step 4: Apply trait attributes (or report only)

Resolve the mode before applying the capability:

• Audit mode (audit, classify, report, or ambiguous intent): emit the per-test mapping and summary without modifying source, regardless of capability.
• Edit mode (tag, apply, or explicitly requested both): continue with the capability branch below.

In edit mode, if the resolved capability is auto-edit, add the appropriate attribute to each test method. Place trait attributes adjacent to the existing test attribute. Examples:

Apply traits at the individual test-method/​case level. Do not substitute one class-level category for method-level classification: different methods usually exercise different positive, negative, and boundary behavior.

MSTest:

[TestMethod]
[TestCategory("negative")]
[TestCategory("boundary")]
public void Parse_NullInput_ThrowsArgumentNullException() { ... }

xUnit:

[Fact]
[Trait("Category", "positive")]
[Trait("Category", "critical-path")]
public void CreateOrder_ValidItems_ReturnsConfirmation() { ... }

NUnit:

[Test]
[Category("regression")]
[Category("negative")]
public void Calculate_OverflowInput_ReturnsError() // Fix for #1234
{ ... }

pytest:

@pytest.mark.negative
@pytest.mark.boundary
def test_parse_none_input_raises_value_error():
    ...

JUnit 5:

@Test
@Tag("positive")
@Tag("critical-path")
void createOrder_validItems_returnsConfirmation() { ... }

TestNG:

@Test(groups = {"negative", "boundary"})
public void parse_nullInput_throwsIllegalArgumentException() { ... }

RSpec:

it "rejects null input", :negative, :boundary do
  ...
end

Pester:

It 'Rejects null input' -Tag 'negative','boundary' {
    ...
}

Kotest:

@Tags(Negative, Boundary)
class ParserSpec : StringSpec({
    "rejects null input" { ... }
})

Swift Testing:

@Test(.tags(.negative, .boundary))
func parseNullInputThrows() throws { ... }

Catch2:

TEST_CASE("Parse null input throws", "[negative][boundary]") { ... }

In any mode, if the resolved capability is report-only (Go standard testing, plain Jest/​Vitest without convention, Rust without project-specific cfg, plain XCTest, plain GoogleTest, plain Mocha), do NOT modify source files. Instead emit a concise mapping from each test to its suggested tags. Recommend a project-wide convention only when the user asks how to persist or filter those tags; an analysis-only request should report and stop.

In edit mode, if the resolved capability is convention-based (e.g., Go //go:build integration, *_integration_test.go, GoogleTest INTEGRATION_* prefix), only emit canonical edits when the user has confirmed the project's convention. Otherwise treat as report-only.

Step 5: Generate trait summary

After tagging, produce a summary table. Include only traits with a non-zero count unless the user asks for the full taxonomy; zero-filled rows obscure the suite's actual shape. For a small report-only suite, keep the per-test mapping and non-zero distribution together rather than expanding into a dashboard.

## Trait Distribution

| Trait         | Count | % of Total |
|---------------|-------|------------|
| positive      |    50 |      64.1% |
| negative      |    28 |      35.9% |
| boundary      |     8 |      10.3% |
| critical-path |    12 |      15.4% |
| **Total tests** | **78** | -- |

Note: Percentages exceed 100% because tests can have multiple traits.

Include observations such as:

• Ratio of positive to negative tests
• Whether critical-path tests exist for key public APIs
• Any tests that could not be confidently classified (list them for manual review)

boundary and every other specialized trait are additive. A boundary success case still counts as positive; a rejected boundary still counts as negative. Derive the positive/​negative distribution after applying this rule.

Step 6: Verify edits before reporting

For every auto-edit framework, run the narrowest command that compiles the edited attributes and confirms test discovery. This is required even when the user asks only to add tags: syntactically plausible attributes are not a completed edit.

If an edit or patch application was uncertain, re-open the complete edited file before verification and reconcile every inventory row with the actual attribute next to that test. Do not report success from a partial diff or from the intended patch. If verification fails, report the exact command and error; never publish a successful distribution handoff for uncompiled edits.

Validation

• Every test method has at least one trait classification (positive or negative at minimum) — in the report for report-only frameworks, or as an attribute for auto-edit frameworks
• The total equals the per-test inventory count, and displayed trait counts were derived from that inventory
• No invented trait values outside the taxonomy table
• Existing trait attributes were preserved, not duplicated
• The trait summary table was generated
• For auto-edit frameworks, the project still builds / tests still discover without executing unrequested tests (dotnet build plus list mode / pytest --collect-only / mvn test-compile / go vet ./​... / cargo check --tests / npm run test:list / equivalent)
• The final summary cites successful validation commands and the discovered test count when a discovery command is available
• For report-only frameworks, no source files were modified
• For convention-based frameworks, edits were applied ONLY when a project convention was confirmed

Common Pitfalls