DProvenanceKit: Reasoning observability for AI systems in Swift

Community Showcase
1

Hi everyone,

I've been working on a problem that I think many of us building AI in Swift are facing: how do you debug why an AI agent behaves differently between runs?

Traditional logging tells you what happened. But when a model skips a step, changes its reasoning order, or produces a different output with identical input — you're left staring at walls of logs with no clear answer to why.

I've released DProvenanceKit — a reasoning observability framework for Swift that lets you:

  • Record every reasoning step an agent takes (non-blocking, async-safe)
  • Query for reasoning patterns ("find runs where X happened but Y didn't")
  • Diff two executions to see structural differences
  • Detect regressions automatically with rule-based validation

Think of it as Git for AI logic.

Example:

Swift

// Record an execution
try await DProvenanceKit<MyAIDecision>.run(contextID: "case-123", store: store) {
    DProvenanceKit<MyAIDecision>.record(.documentEvaluated(documentID: "DocA", score: 0.95))
    DProvenanceKit<MyAIDecision>.record(.conflictDetected(reason: "timeline_inconsistency"))
    DProvenanceKit<MyAIDecision>.record(.finalDecisionMade(approved: false))
}

// Query for suspicious patterns
let suspiciousRuns = try await store.queryRuns(
    TraceQueryDSL<MyAIDecision>()
        .requiring(step: "conflictDetected")
        .missing(step: "documentEvaluated")  // Find runs where conflict was reported but no docs evaluated
)

// Diff two runs
let diff = engine.diff(base: runA, comparison: runB)
print(diff.changes)  // See exactly which steps appeared, disappeared, or moved

The design:

  • Built specifically for on-device AI (macOS/iOS) with Apple Foundation Models, MLX, or Core ML
  • Non-blocking recording (touches only in-memory buffer)
  • Durable, crash-safe persistence with SQLite WAL
  • Works with async/await context propagation

Status:

Experimental (core engine complete, actively evolving). Free for development/testing under BSL 1.1.

GitHub: GitHub - Therealdk8890/DProvenanceKit: Reasoning observability and regression testing for AI systems · GitHub

I'm curious if this resonates with anyone here building AI in Swift. What debugging/observability challenges are you facing with AI systems?

4 Likes
2

I don't think you meant to post in the category for discussing development of the Swift compiler. I've moved the post to "Community Showcase."

1 Like