> ## Documentation Index
> Fetch the complete documentation index at: https://docs.writerzroom.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Reliability and Quality Metrics

> How WriterzRoom tracks generation reliability, content quality, pipeline performance, and workflow health.

WriterzRoom tracks reliability and quality across generation requests, templates, style profiles, agent stages, and saved content.

This page explains the current measurement model and the metrics WriterzRoom uses to evaluate generation quality, workflow reliability, and production readiness.

## Metrics Overview

<div style={{ display: 'grid', gridTemplateColumns: 'repeat(4, 1fr)', gap: '12px', margin: '1.5rem 0' }}>
  <div style={{ border: '1px solid rgba(128,128,128,0.20)', borderRadius: '14px', padding: '16px', background: 'rgba(255,255,255,0.04)' }}>
    <div style={{ fontWeight: 800 }}>Reliability</div>
    <div style={{ fontSize: '13px', color: 'var(--colors-content-secondary)' }}>Status, failure type, retry count, latency, and timeout behavior.</div>
  </div>

  <div style={{ border: '1px solid rgba(128,128,128,0.20)', borderRadius: '14px', padding: '16px', background: 'rgba(255,255,255,0.04)' }}>
    <div style={{ fontWeight: 800 }}>Quality</div>
    <div style={{ fontSize: '13px', color: 'var(--colors-content-secondary)' }}>Readability, grammar, AI tells, citations, SEO, and constraint validation.</div>
  </div>

  <div style={{ border: '1px solid rgba(128,128,128,0.20)', borderRadius: '14px', padding: '16px', background: 'rgba(255,255,255,0.04)' }}>
    <div style={{ fontWeight: 800 }}>Performance</div>
    <div style={{ fontSize: '13px', color: 'var(--colors-content-secondary)' }}>Agent latency, total latency, token use, corpus hits, and search usage.</div>
  </div>

  <div style={{ border: '1px solid rgba(128,128,128,0.20)', borderRadius: '14px', padding: '16px', background: 'rgba(255,255,255,0.04)' }}>
    <div style={{ fontWeight: 800 }}>Usage</div>
    <div style={{ fontSize: '13px', color: 'var(--colors-content-secondary)' }}>Template usage, style profile usage, generation volume, and success rate.</div>
  </div>
</div>

## Current Instrumentation

WriterzRoom already stores several categories of reliability and quality metrics.

| Area                 | Tracked examples                                                                                                       |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| Generated content    | Status, errors, word count, generation time, model used, SEO score, readability score                                  |
| Usage statistics     | Daily generation volume, tokens, generation time, templates used, style profiles used, success rate                    |
| Template usage       | Usage count, success rate, rating, total word count, token usage, average generation time                              |
| Style profile usage  | Usage count, success rate, rating, total word count, token usage, average generation time                              |
| Generation test runs | Latency, retry count, failure type, failure stage, readability, grammar, AI tells, citations, SEO, research confidence |
| Content performance  | Views, shares, downloads, read time, engagement metrics                                                                |
| API usage            | Endpoint, method, status code, response time, request size, response size                                              |

## Reliability Metrics

<CardGroup cols={2}>
  <Card title="Generation status" icon="activity">
    Tracks whether a generation completes, fails, times out, or remains in progress.
  </Card>

  <Card title="Failure classification" icon="triangle-alert">
    Captures failure stage, failure type, exception class, and error snippets for test and diagnostic runs.
  </Card>

  <Card title="Retry behavior" icon="refresh-cw">
    Tracks retry count and circuit breaker state where applicable.
  </Card>

  <Card title="Latency tracking" icon="timer">
    Supports total latency and agent-stage latency, including planner, researcher, writer, and editor timing.
  </Card>
</CardGroup>

## Quality Metrics

| Metric                  | Purpose                                                                    |
| ----------------------- | -------------------------------------------------------------------------- |
| Readability score       | Measures how accessible the generated content is for the selected audience |
| Grammar critical issues | Tracks severe grammar or typo issues that may affect publication readiness |
| AI-tell count           | Tracks generic AI writing patterns and formulaic language                  |
| Citation count          | Tracks citation density and source usage in research-backed outputs        |
| Hallucination flags     | Supports detection of potentially unsupported or fabricated claims         |
| Constraint violations   | Tracks failures against generation contracts or template requirements      |
| Structure validity      | Indicates whether required structural elements are present                 |
| SEO score               | Measures search optimization quality for relevant content types            |
| Word count delta        | Compares generated length against target length                            |

## Pipeline Performance Metrics

<Steps>
  <Step title="Planner metrics">
    Tracks prompt size, planning behavior, and planner-stage latency where available.
  </Step>

  <Step title="Research metrics">
    Tracks research confidence, corpus hits, Tavily hits, and research-stage latency.
  </Step>

  <Step title="Writer metrics">
    Tracks generation time, token usage, word count, and writer-stage latency.
  </Step>

  <Step title="Editor metrics">
    Tracks readability, grammar, AI tells, constraint validation, and editor-stage latency.
  </Step>

  <Step title="Final content metrics">
    Stores SEO score, readability score, content intelligence metadata, and generation status with saved content.
  </Step>
</Steps>

## Quality Gates

WriterzRoom quality tracking is not only post-generation reporting.

Several checks influence whether content should proceed through the workflow.

| Gate                           | Behavior                                                       |
| ------------------------------ | -------------------------------------------------------------- |
| Empty content rejection        | Rejects invalid empty outputs                                  |
| Word count validation          | Compares generated content against template-level expectations |
| Readability enforcement        | Calculates readability and may trigger rewrite or failure      |
| AI-tell enforcement            | Rejects content with excessive AI-tell patterns                |
| Grammar enforcement            | Rejects content above critical grammar issue limits            |
| Generation contract validation | Checks structural and length expectations defined by templates |
| Citation review                | Supports citation counting and hallucination-related tracking  |

## Template and Style Reliability

Template and style profile metrics help identify weak combinations.

| Metric                     | Use                                                 |
| -------------------------- | --------------------------------------------------- |
| Template success rate      | Finds templates that frequently fail or need tuning |
| Style profile success rate | Finds style profiles that underperform              |
| Average generation time    | Identifies slow templates or styles                 |
| Average rating             | Connects user feedback to template/style quality    |
| Total tokens               | Tracks cost and usage intensity                     |
| Total word count           | Tracks output volume by template and style profile  |

## Service Level Objectives

WriterzRoom defines the following SLOs for production generation workflows. These targets apply to successful generations under normal operating conditions.

| Objective                 | Target           | Window         |
| ------------------------- | ---------------- | -------------- |
| Generation success rate   | 99%              | 30-day rolling |
| Quick tier P95 latency    | Under 90 seconds | 30-day rolling |
| Standard tier P95 latency | Under 5 minutes  | 30-day rolling |
| Premium tier P95 latency  | Under 12 minutes | 30-day rolling |
| API availability          | 99.5% uptime     | 30-day rolling |
| Writer agent P95 latency  | Under 3 minutes  | 30-day rolling |

**Measurement notes:**

Generation success rate is calculated as completed generations divided by all non-cancelled generation attempts. Latency is measured from generation request acceptance to final content availability. API availability is measured at the `/health` endpoint from Cloud Run.

Alert policies enforce the writer agent P95 and Standard tier P95 latency targets with a 5-minute evaluation window and notify via email and Slack.

<Info>
  SLOs reflect production targets, not contractual guarantees. Individual generation time varies by template complexity, tier, research depth, and vertical configuration.
</Info>

## Public Reporting Status

WriterzRoom currently tracks internal metrics across generation, content quality, usage, and test runs.

Public benchmark reporting is being formalized.

<Info>
  Until enough production usage exists, public metrics should be presented as instrumentation coverage rather than performance guarantees.
</Info>

## Planned Public Metrics

The following metrics are candidates for public or customer-facing reporting:

| Metric                     | Status                  |
| -------------------------- | ----------------------- |
| Generation success rate    | Internally trackable    |
| Average generation time    | Internally trackable    |
| Failed generation rate     | Internally trackable    |
| Timeout rate               | Internally trackable    |
| Template success rate      | Internally trackable    |
| Style profile success rate | Internally trackable    |
| Readability pass rate      | Internally trackable    |
| AI-tell pass rate          | Internally trackable    |
| Citation density           | Internally trackable    |
| SEO score trend            | Internally trackable    |
| Research confidence trend  | Internally trackable    |
| User regeneration rate     | Recommended next metric |

## Recommended Next Instrumentation

To make this page stronger over time, WriterzRoom should add or verify:

| Need                                 | Recommendation                                                                       |
| ------------------------------------ | ------------------------------------------------------------------------------------ |
| User regeneration rate               | Track how often users regenerate full content or sections                            |
| Quality pass rate                    | Aggregate pass/fail status from editor, formatter, and contract checks               |
| Citation validation pass rate        | Separate citation presence from citation validity                                    |
| Per-agent failure rate               | Aggregate failures by planner, researcher, writer, editor, formatter, SEO, publisher |
| Vertical success rate                | Track success rate by vertical ID                                                    |
| Template-style-vertical success rate | Track quality by exact combination                                                   |
| Public status snapshot               | Publish aggregate metrics after stable production volume                             |
| Admin dashboard                      | Expose reliability trends internally before public release                           |

## Review Expectations

<Warning>
  Reliability and quality metrics describe system behavior and content-readiness signals. They do not guarantee factual correctness, regulatory compliance, legal sufficiency, medical appropriateness, financial accuracy, or publication approval.
</Warning>

## Related Pages

<CardGroup cols={2}>
  <Card title="Trust Center" icon="shield-check" href="/security/trust-center">
    Review WriterzRoom security, reliability, data handling, and governance posture.
  </Card>

  <Card title="Reliability and Generation Failures" icon="activity" href="/security/reliability-and-generation-failures">
    Learn how WriterzRoom handles generation failures and workflow status.
  </Card>

  <Card title="Multi-Agent Pipeline" icon="network" href="/architecture/multi-agent-pipeline">
    See how generation moves through planning, research, writing, editing, formatting, SEO, and publishing.
  </Card>

  <Card title="Recommended Combinations" icon="workflow" href="/product/recommended-combinations">
    See how vertical, template, and style profile combinations affect generation quality.
  </Card>
</CardGroup>

## Summary

WriterzRoom already includes internal instrumentation for reliability, quality, performance, usage, and generation testing.

The next maturity step is not inventing metrics from scratch. It is aggregating existing measurements into dashboards, public benchmark summaries, and combination-level reliability reporting.