Pre-production validation gate for GCP stack (Cloud Run/Functions/App Engine, Firestore/Cloud SQL, Firebase Auth/Identity Platform) — generates test plans, e...
SKILL.md

---
name: qa-gate-gcp
description: Pre-production validation gate for GCP stack (Cloud Run/Functions/App Engine, Firestore/Cloud SQL, Firebase Auth/Identity Platform) — generates test plans, executes test suites, validates APIs, UI, toasts, LLM output quality, and produces go/no-go reports
user-invocable: true
---

# qa-gate-gcp: Pre-Production Validation Gate for Google Cloud Platform

You are a senior QA architect responsible for the final validation gate before production deployment on Google Cloud Platform. You do NOT write individual unit tests (that is test-sentinel's job). Instead, you orchestrate a comprehensive validation sweep: you generate a detailed test plan covering every critical surface, execute automated tests, validate API contracts, check UI/UX flows including toast notifications, assess LLM output quality using rule-based checks and LLM-as-judge, validate GCP infrastructure health (Cloud Run services, Cloud SQL instances, Firestore security rules, Secret Manager), and produce a structured go/no-go report. This skill creates test plan documents, validation scripts, and JSON reports. It never reads or modifies `.env`, `.env.local`, or credential files directly.

## Credential Scope

`OPENROUTER_API_KEY` is used in generated validation scripts to run LLM-as-judge evaluations on content quality. `GCP_PROJECT_ID` and `GCP_REGION` are referenced in generated infrastructure validation scripts. `GOOGLE_APPLICATION_CREDENTIALS` is used by `gcloud` CLI commands in generated scripts. All env vars are accessed via `process.env` or `os.environ.get()` in generated code only.

## Planning Protocol (MANDATORY)

Same structure as other skills:

1. **Understand the scope** — what is being validated (full app, specific feature, specific release)
2. **Survey the project** — detect test framework (Vitest/Jest/Playwright/Cypress), detect compute type (Cloud Run/Functions/App Engine), detect database (Firestore/Cloud SQL), check existing test coverage, read package.json, read app structure
3. **Identify all validation surfaces**: API routes/endpoints, Server Actions, database operations, auth flows (Firebase Auth or Identity Platform), UI pages, toast notifications, LLM-powered features, GCP service health
4. **Build the master test plan** (JSON document)
5. **Identify risks and blockers**
6. **Execute the validation pipeline**
7. **Produce the go/no-go report**

Do NOT skip this protocol. A rushed validation wastes tokens, misses critical failures, and gives false confidence before production.

---

## Part 1 — Test Plan Generation

The agent MUST generate a structured test plan before running anything. The plan is a JSON file saved to `qa-reports/test-plan.json`:

```json
{
  "project": "project-name",
  "version": "x.y.z",
  "date": "ISO-8601",
  "validator": "qa-gate-gcp",
  "stack": {
    "compute": "cloud-run | cloud-functions | app-engine",
    "database": "firestore | cloud-sql | both",
    "auth": "firebase-auth | identity-platform",
    "cdn": "cloudflare | cloud-cdn"
  },
  "surfaces": {
    "api_endpoints": [
      {
        "endpoint": "/api/entities",
        "methods": ["GET", "POST"],
        "auth_required": true,
        "compute_target": "cloud-run",
        "validations": ["status_codes", "response_schema", "error_handling", "cors", "auth_guard"]
      }
    ],
    "server_actions": [
      {
        "name": "createEntity",
        "file": "src/app/actions/entities.ts",
        "validations": ["input_validation", "auth_check", "db_write", "revalidation", "error_response"]
      }
    ],
    "ui_pages": [
      {
        "path": "/dashboard",
        "auth_required": true,
        "validations": ["renders_correctly", "responsive", "loading_states", "error_states", "accessibility"]
      }
    ],
    "toast_notifications": [
      {
        "trigger": "entity_created",
        "type": "success",
        "expected_message_pattern": "Entity .* created",
        "auto_dismiss": true,
        "validations": ["appears", "correct_type", "dismisses", "no_duplicate"]
      }
    ],
    "auth_flows": [
      {
        "flow": "email_login",
        "provider": "firebase-auth",
        "steps": ["navigate_to_login", "fill_form", "submit", "redirect_to_dashboard"],
        "error_cases": ["invalid_credentials", "unverified_email", "rate_limited"]
      }
    ],
    "llm_features": [
      {
        "feature": "content_generation",
        "endpoint": "/api/generate",
        "validations": ["response_format", "content_quality", "safety", "latency", "token_usage"]
      }
    ],
    "database_integrity": {
      "firestore": [
        {
          "collection": "entities",
          "validations": ["security_rules_enforced", "indexes_exist", "no_orphan_subcollections"]
        }
      ],
      "cloud_sql": [
        {
          "table": "entities",
          "validations": ["constraints_valid", "indexes_exist", "migrations_applied", "no_orphans"]
        }
      ]
    },
    "gcp_infrastructure": [
      {
        "service": "cloud-run",
        "name": "my-service",
        "region": "us-central1",
        "validations": ["service_running", "latest_revision_serving", "min_instances", "cpu_memory", "env_vars_set"]
      },
      {
        "service": "cloud-sql",
        "instance": "my-instance",
        "validations": ["instance_running", "connections_available", "storage_usage", "backup_enabled"]
      },
      {
        "service": "secret-manager",
        "validations": ["required_secrets_exist", "secret_versions_enabled"]
      }
    ]
  }
}
```

### How to discover surfaces:

- **API endpoints**: scan `src/app/api/**/route.ts` or framework-specific route files
- **Server Actions**: scan for `"use server"` directives
- **UI pages**: scan `src/app/**/page.tsx` or framework router files
- **Toast notifications**: grep for toast library usage (sonner, react-hot-toast, shadcn toast)
- **Auth flows**: check for Firebase Auth SDK usage, Identity Platform config
- **LLM features**: grep for OpenAI/OpenRouter/Anthropic/Vertex AI API calls
- **Database (Firestore)**: scan `firestore.rules`, check admin SDK usage
- **Database (Cloud SQL)**: check Prisma schema or migration files
- **GCP infra**: use `gcloud` CLI to inspect running services

## Part 2 — API Validation

### Framework Detection

```bash
# Detect test framework
if [ -f "vitest.config.ts" ] || [ -f "vitest.config.js" ]; then
  FRAMEWORK="vitest"
elif [ -f "jest.config.ts" ] || [ -f "jest.config.js" ]; then
  FRAMEWORK="jest"
else
  FRAMEWORK="vitest"  # default
fi

# Detect E2E framework
if [ -f "playwright.config.ts" ]; then
  E2E="playwright"
elif [ -f "cypress.config.ts" ] || [ -f "cypress.config.js" ]; then
  E2E="cypress"
else
  E2E="playwright"  # default
fi
```

### API Route Validation Template

```typescript
// qa-tests/api/entities.validation.test.ts
const BASE_URL = process.env.VALIDATION_BASE_URL || "http://localhost:3000";

describe("API Validation: /api/entities", () => {
  it("returns 200 for authenticated GET", async () => {
    const res = await fetch(`${BASE_URL}/api/entities`, {
      headers: { Authorization: `Bearer ${process.env.TEST_AUTH_TOKEN}` },
    });
    expect(res.status).toBe(200);
  });

  it("returns 401 for unauthenticated request", async () => {
    const res = await fetch(`${BASE_URL}/api/entities`);
    expect(res.status).toBe(401);
  });

  it("response matches expected schema", async () => {
    const res = await fetch(`${BASE_URL}/api/entities`, {
      headers: { Authorization: `Bearer ${process.env.TEST_AUTH_TOKEN}` },
    });
    const data = await res.json();
    expect(Array.isArray(data)).toBe(true);
    if (data.length > 0) {
      expect(data[0]).toHaveProperty("id");
      expect(data[0]).toHaveProperty("name");
    }
  });

  it("returns proper error for invalid input", async () => {
    const res = await fetch(`${BASE_URL}/api/entities`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.TEST_AUTH_TOKEN}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({}),
    });
    expect(res.status).toBe(400);
    const err = await res.json();
    expect(err).toHaveProperty("error");
  });

  it("CORS headers are present", async () => {
    const res = await fetch(`${BASE_URL}/api/entities`, {
      method: "OPTIONS",
    });
    expect(res.headers.get("access-control-allow-origin")).toBeTruthy();
  });
});
```

## Part 3 — UI & Toast Validation

### Playwright UI Validation Template

```typescript
// qa-tests/ui/dashboard.validation.spec.ts
import { test, expect } from "@playwright/test";

test.describe("UI Validation: /dashboard", () => {
  test.beforeEach(async ({ page }) => {
    await page.goto("/login");
    await page.fill('[name="email"]', process.env.TEST_USER_EMAIL!);
    await page.fill('[name="password"]', process.env.TEST_USER_PASSWORD!);
    await page.click('button[type="submit"]');
    await page.waitForURL("/dashboard");
  });

  test("page renders correctly", async ({ page }) => {
    await expect(page.locator("h1")).toBeVisible();
    await expect(page.locator("nav")).toBeVisible();
  });

  test("loading states display correctly", async ({ page }) => {
    await page.route("**/api/entities", async (route) => {
      await new Promise((r) => setTimeout(r, 2000));
      await route.continue();
    });
    await page.goto("/dashboard");
    await expect(page.locator('[data-testid="skeleton"]')).toBeVisible();
  });

  test("error states display correctly", async ({ page }) => {
    await page.route("**/api/entities", (route) =>
      route.fulfill({ status: 500, body: JSON.stringify({ error: "Server error" }) })
    );
    await page.goto("/dashboard");
    await expect(page.locator('[role="alert"]')).toBeVisible();
  });

  test("responsive layout at 375px, 768px, 1280px", async ({ page }) => {
    for (const width of [375, 768, 1280]) {
      await page.setViewportSize({ width, height: 720 });
      await expect(page.locator("nav")).toBeVisible();
    }
  });
});
```

### Toast Notification Validation

```typescript
// qa-tests/ui/toasts.validation.spec.ts
import { test, expect } from "@playwright/test";

test.describe("Toast Validation", () => {
  test("success toast appears on entity creation", async ({ page }) => {
    await page.goto("/entities/new");
    await page.fill('[name="name"]', "Test Entity");
    await page.click('button[type="submit"]');
    const toast = page.locator('[data-sonner-toast], [role="status"], .Toastify__toast');
    await expect(toast).toBeVisible({ timeout: 5000 });
    await expect(toast).toContainText(/created|success/i);
  });

  test("error toast appears on failed submission", async ({ page }) => {
    await page.route("**/api/entities", (route) =>
      route.fulfill({ status: 500, body: JSON.stringify({ error: "Failed" }) })
    );
    await page.goto("/entities/new");
    await page.fill('[name="name"]', "Test");
    await page.click('button[type="submit"]');
    const toast = page.locator('[data-sonner-toast][data-type="error"], .Toastify__toast--error, [role="alert"]');
    await expect(toast).toBeVisible({ timeout: 5000 });
  });

  test("toast auto-dismisses", async ({ page }) => {
    await page.goto("/entities/new");
    await page.fill('[name="name"]', "Test");
    await page.click('button[type="submit"]');
    const toast = page.locator('[data-sonner-toast], [role="status"]');
    await expect(toast).toBeVisible();
    await expect(toast).not.toBeVisible({ timeout: 10000 });
  });

  test("no duplicate toasts on rapid clicks", async ({ page }) => {
    await page.goto("/entities/new");
    await page.fill('[name="name"]', "Test");
    await page.click('button[type="submit"]');
    await page.click('button[type="submit"]');
    const toasts = page.locator('[data-sonner-toast], [role="status"]');
    expect(await toasts.count()).toBeLessThanOrEqual(1);
  });
});
```

## Part 4 — Auth Flow Validation

### Firebase Auth / Identity Platform Validation

```typescript
// qa-tests/auth/auth-flows.validation.spec.ts
import { test, expect } from "@playwright/test";

test.describe("Auth Flow Validation", () => {
  test("login with valid credentials redirects to dashboard", async ({ page }) => {
    await page.goto("/login");
    await page.fill('[name="email"]', process.env.TEST_USER_EMAIL!);
    await page.fill('[name="password"]', process.env.TEST_USER_PASSWORD!);
    await page.click('button[type="submit"]');
    await page.waitForURL("/dashboard", { timeout: 10000 });
    expect(page.url()).toContain("/dashboard");
  });

  test("login with invalid credentials shows error", async ({ page }) => {
    await page.goto("/login");
    await page.fill('[name="email"]', "wrong@example.com");
    await page.fill('[name="password"]', "wrongpass");
    await page.click('button[type="submit"]');
    await expect(page.locator('[role="alert"], .error, [data-testid="auth-error"]')).toBeVisible();
  });

  test("protected routes redirect unauthenticated users", async ({ page }) => {
    await page.goto("/dashboard");
    await page.waitForURL(/\/(login|auth)/);
  });

  test("logout clears session and redirects", async ({ page }) => {
    // Login first then logout
    await page.goto("/login");
    await page.fill('[name="email"]', process.env.TEST_USER_EMAIL!);
    await page.fill('[name="password"]', process.env.TEST_USER_PASSWORD!);
    await page.click('button[type="submit"]');
    await page.waitForURL("/dashboard");
    await page.click('[data-testid="logout"], button:has-text("Logout"), button:has-text("Sair")');
    await page.waitForURL(/\/(login|auth|$)/);
    await page.goto("/dashboard");
    await page.waitForURL(/\/(login|auth)/);
  });
});
```

## Part 5 — LLM Output Quality Validation

### Two-Layer Approach: Rule-Based + LLM-as-Judge

#### Layer 1: Rule-Based Checks

```typescript
// qa-tests/llm/rule-based-checks.ts
export interface LLMOutput {
  content: string;
  model: string;
  tokens_used: number;
  latency_ms: number;
}

export interface RuleCheckResult {
  rule: string;
  passed: boolean;
  details: string;
}

export function runRuleBasedChecks(output: LLMOutput, config: {
  maxTokens?: number;
  maxLatencyMs?: number;
  minLength?: number;
  maxLength?: number;
  requiredSections?: string[];
  forbiddenPatterns?: RegExp[];
  requiredFormat?: "json" | "markdown" | "plain";
}): RuleCheckResult[] {
  const results: RuleCheckResult[] = [];

  if (config.minLength) {
    results.push({
      rule: "min_length",
      passed: output.content.length >= config.minLength,
      details: `Content length: ${output.content.length}, minimum: ${config.minLength}`,
    });
  }

  if (config.maxLength) {
    results.push({
      rule: "max_length",
      passed: output.content.length <= config.maxLength,
      details: `Content length: ${output.content.length}, maximum: ${config.maxLength}`,
    });
  }

  if (config.maxTokens) {
    results.push({
      rule: "token_budget",
      passed: output.tokens_used <= config.maxTokens,
      details: `Tokens used: ${output.tokens_used}, budget: ${config.maxTokens}`,
    });
  }

  if (config.maxLatencyMs) {
    results.push({
      rule: "latency",
      passed: output.latency_ms <= config.maxLatencyMs,
      details: `Latency: ${output.latency_ms}ms, max: ${config.maxLatencyMs}ms`,
    });
  }

  if (config.requiredSections) {
    for (const section of config.requiredSections) {
      results.push({
        rule: `required_section:${section}`,
        passed: output.content.toLowerCase().includes(section.toLowerCase()),
        details: `Section "${section}" ${output.content.toLowerCase().includes(section.toLowerCase()) ? "found" : "missing"}`,
      });
    }
  }

  if (config.forbiddenPatterns) {
    for (const pattern of config.forbiddenPatterns) {
      const match = pattern.exec(output.content);
      results.push({
        rule: `forbidden_pattern:${pattern.source}`,
        passed: !match,
        details: match ? `Found forbidden pattern: "${match[0]}"` : "No forbidden patterns found",
      });
    }
  }

  if (config.requiredFormat === "json") {
    try {
      JSON.parse(output.content);
      results.push({ rule: "valid_json", passed: true, details: "Valid JSON" });
    } catch {
      results.push({ rule: "valid_json", passed: false, details: "Invalid JSON" });
    }
  }

  results.push({
    rule: "not_empty",
    passed: output.content.trim().length > 0,
    details: output.content.trim().length === 0 ? "Output is empty" : "Output has content",
  });

  results.push({
    rule: "not_truncated",
    passed: !output.content.endsWith("...") && !output.content.endsWith("..."),
    details: "Check for truncation markers",
  });

  return results;
}
```

#### Layer 2: LLM-as-Judge

```typescript
// qa-tests/llm/llm-judge.ts
export async function llmJudge(
  output: string,
  prompt: string,
  criteria: {
    relevance: boolean;
    accuracy: boolean;
    completeness: boolean;
    tone: boolean;
    safety: boolean;
  }
): Promise<{
  overall_score: number;
  criteria_scores: Record<string, number>;
  issues: string[];
  recommendation: "pass" | "review" | "fail";
}> {
  const OPENROUTER_API_KEY = process.env.OPENROUTER_API_KEY;
  if (!OPENROUTER_API_KEY) {
    return {
      overall_score: 0,
      criteria_scores: {},
      issues: ["OPENROUTER_API_KEY not set — skipping LLM judge"],
      recommendation: "review",
    };
  }

  const judgingPrompt = `You are a strict QA evaluator. Assess the following LLM output against the original prompt.

ORIGINAL PROMPT:
${prompt}

LLM OUTPUT:
${output}

Score each criterion from 1 (terrible) to 5 (excellent):
${criteria.relevance ? "- RELEVANCE: Does the output address the prompt?" : ""}
${criteria.accuracy ? "- ACCURACY: Is the information correct and not hallucinated?" : ""}
${criteria.completeness ? "- COMPLETENESS: Does the output cover all requested aspects?" : ""}
${criteria.tone ? "- TONE: Is the tone appropriate for the context?" : ""}
${criteria.safety ? "- SAFETY: Is the content free of harmful, biased, or inappropriate material?" : ""}

Respond in JSON:
{
  "criteria_scores": { "relevance": N, "accuracy": N, ... },
  "overall_score": N,
  "issues": ["list of specific issues found"],
  "recommendation": "pass|review|fail"
}

Rules:
- overall_score < 3 = "fail"
- overall_score >= 3 and < 4 = "review"
- overall_score >= 4 = "pass"
- Any safety score < 3 = automatic "fail"`;

  const response = await fetch("https://openrouter.ai/api/v1/chat/completions", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${OPENROUTER_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "google/gemini-flash-1.5",
      messages: [{ role: "user", content: judgingPrompt }],
      temperature: 0.1,
      response_format: { type: "json_object" },
    }),
  });

  const data = await response.json();
  return JSON.parse(data.choices[0].message.content);
}
```

### LLM Validation Test

```typescript
// qa-tests/llm/content-quality.validation.test.ts
describe("LLM Output Quality Validation", () => {
  it("content generation meets quality standards", async () => {
    const res = await fetch(`${BASE_URL}/api/generate`, {
      method: "POST",
      headers: { "Content-Type": "application/json", Authorization: `Bearer ${TOKEN}` },
      body: JSON.stringify({ prompt: "Describe the benefits of remote work" }),
    });
    const output = await res.json();

    const ruleResults = runRuleBasedChecks(output, {
      minLength: 100,
      maxLength: 5000,
      maxLatencyMs: 10000,
      forbiddenPatterns: [
        /\b(SSN|social security)\b/i,
        /\b(as an AI|I cannot)\b/i,
        /\b(undefined|null|NaN)\b/,
      ],
    });
    const ruleFailures = ruleResults.filter((r) => !r.passed);
    expect(ruleFailures).toHaveLength(0);

    const judgment = await llmJudge(output.content, "Describe the benefits of remote work", {
      relevance: true,
      accuracy: true,
      completeness: true,
      tone: true,
      safety: true,
    });
    expect(judgment.recommendation).not.toBe("fail");
    expect(judgment.overall_score).toBeGreaterThanOrEqual(3);
  });
});
```

## Part 6 — GCP Infrastructure Validation

This is the key differentiator from qa-gate-vercel. Validate GCP services using gcloud CLI.

### Cloud Run Validation

```bash
#!/bin/bash
# qa-tests/infra/cloud-run-validation.sh
set -euo pipefail

PROJECT_ID="${GCP_PROJECT_ID}"
REGION="${GCP_REGION:-us-central1}"
SERVICE_NAME="${1:-my-service}"

echo "=== Cloud Run Validation: $SERVICE_NAME ==="

# 1. Service exists and is serving
STATUS=$(gcloud run services describe "$SERVICE_NAME" \
  --project="$PROJECT_ID" --region="$REGION" \
  --format="value(status.conditions[0].status)" 2>/dev/null)
if [ "$STATUS" != "True" ]; then
  echo "FAIL: Service $SERVICE_NAME is not ready (status: $STATUS)"
  exit 1
fi
echo "PASS: Service is ready"

# 2. Latest revision is serving traffic
LATEST=$(gcloud run services describe "$SERVICE_NAME" \
  --project="$PROJECT_ID" --region="$REGION" \
  --format="value(status.latestReadyRevisionName)")
SERVING=$(gcloud run services describe "$SERVICE_NAME" \
  --project="$PROJECT_ID" --region="$REGION" \
  --format="value(status.traffic[0].revisionName)")
if [ "$LATEST" != "$SERVING" ]; then
  echo "WARN: Latest revision ($LATEST) != serving revision ($SERVING)"
else
  echo "PASS: Latest revision is serving"
fi

# 3. Health check (HTTP)
URL=$(gcloud run services describe "$SERVICE_NAME" \
  --project="$PROJECT_ID" --region="$REGION" \
  --format="value(status.url)")
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "$URL/api/health" 2>/dev/null || echo "000")
if [ "$HTTP_STATUS" = "200" ]; then
  echo "PASS: Health endpoint returns 200"
else
  echo "FAIL: Health endpoint returns $HTTP_STATUS"
fi

# 4. Min instances check
MIN_INSTANCES=$(gcloud run services describe "$SERVICE_NAME" \
  --project="$PROJECT_ID" --region="$REGION" \
  --format="value(spec.template.metadata.annotations['autoscaling.knative.dev/minScale'])")
echo "INFO: Min instances = ${MIN_INSTANCES:-0}"

# 5. Environment variables set (names only, not values)
echo "INFO: Checking required env vars..."
ENVS=$(gcloud run services describe "$SERVICE_NAME" \
  --project="$PROJECT_ID" --region="$REGION" \
  --format="value(spec.template.spec.containers[0].env.name)" 2>/dev/null)
for REQUIRED in "NODE_ENV" "DATABASE_URL"; do
  if echo "$ENVS" | grep -q "$REQUIRED"; then
    echo "PASS: $REQUIRED is set"
  else
    echo "WARN: $REQUIRED is NOT set"
  fi
done
```

### Cloud SQL Validation

```bash
#!/bin/bash
# qa-tests/infra/cloud-sql-validation.sh
set -euo pipefail

PROJECT_ID="${GCP_PROJECT_ID}"
INSTANCE="${1:-my-instance}"

echo "=== Cloud SQL Validation: $INSTANCE ==="

# 1. Instance running
STATE=$(gcloud sql instances describe "$INSTANCE" \
  --project="$PROJECT_ID" \
  --format="value(state)" 2>/dev/null)
if [ "$STATE" != "RUNNABLE" ]; then
  echo "FAIL: Instance state is $STATE (expected RUNNABLE)"
  exit 1
fi
echo "PASS: Instance is running"

# 2. Backup enabled
BACKUP=$(gcloud sql instances describe "$INSTANCE" \
  --project="$PROJECT_ID" \
  --format="value(settings.backupConfiguration.enabled)")
if [ "$BACKUP" = "True" ]; then
  echo "PASS: Automated backups enabled"
else
  echo "FAIL: Automated backups are DISABLED"
fi

# 3. Storage usage
STORAGE_USED=$(gcloud sql instances describe "$INSTANCE" \
  --project="$PROJECT_ID" \
  --format="value(currentDiskSize)")
STORAGE_MAX=$(gcloud sql instances describe "$INSTANCE" \
  --project="$PROJECT_ID" \
  --format="value(settings.dataDiskSizeGb)")
echo "INFO: Storage used = ${STORAGE_USED:-unknown}, max = ${STORAGE_MAX:-unknown}GB"

# 4. SSL required
SSL=$(gcloud sql instances describe "$INSTANCE" \
  --project="$PROJECT_ID" \
  --format="value(settings.ipConfiguration.requireSsl)")
if [ "$SSL" = "True" ]; then
  echo "PASS: SSL connections required"
else
  echo "WARN: SSL connections NOT required"
fi
```

### Firestore Security Rules Validation

```bash
#!/bin/bash
# qa-tests/infra/firestore-rules-validation.sh
set -euo pipefail

PROJECT_ID="${GCP_PROJECT_ID}"

echo "=== Firestore Security Rules Validation ==="

# 1. Check rules file exists locally
if [ -f "firestore.rules" ]; then
  echo "PASS: firestore.rules file found"

  # 2. Check for open rules (security risk)
  if grep -q "allow read, write: if true" firestore.rules; then
    echo "FAIL: CRITICAL — open read/write rules detected (allow if true)"
  elif grep -q "allow read, write" firestore.rules | grep -v "if request.auth"; then
    echo "WARN: Some rules may not check authentication"
  else
    echo "PASS: Rules appear to check authentication"
  fi

  # 3. Deploy rules to emulator for testing (if available)
  if command -v firebase &>/dev/null; then
    echo "INFO: Running Firestore rules emulator tests..."
    firebase emulators:exec --only firestore "npm run test:firestore-rules" 2>/dev/null || echo "WARN: Emulator test failed or not configured"
  fi
else
  echo "WARN: No firestore.rules file found locally"
fi
```

### Secret Manager Validation

```bash
#!/bin/bash
# qa-tests/infra/secret-manager-validation.sh
set -euo pipefail

PROJECT_ID="${GCP_PROJECT_ID}"

echo "=== Secret Manager Validation ==="

REQUIRED_SECRETS=("DATABASE_URL" "FIREBASE_PRIVATE_KEY" "OPENROUTER_API_KEY")

for SECRET in "${REQUIRED_SECRETS[@]}"; do
  EXISTS=$(gcloud secrets describe "$SECRET" \
    --project="$PROJECT_ID" \
    --format="value(name)" 2>/dev/null || echo "")
  if [ -n "$EXISTS" ]; then
    # Check that at least one version is enabled
    ENABLED=$(gcloud secrets versions list "$SECRET" \
      --project="$PROJECT_ID" \
      --filter="state=ENABLED" \
      --format="value(name)" --limit=1 2>/dev/null || echo "")
    if [ -n "$ENABLED" ]; then
      echo "PASS: Secret $SECRET exists with enabled version"
    else
      echo "FAIL: Secret $SECRET exists but has no enabled versions"
    fi
  else
    echo "FAIL: Secret $SECRET not found in Secret Manager"
  fi
done
```

## Part 7 — Database Integrity Validation (Firestore + Cloud SQL)

### Firestore Integrity

```typescript
// qa-tests/db/firestore-integrity.validation.test.ts
import { initializeApp, cert } from "firebase-admin/app";
import { getFirestore } from "firebase-admin/firestore";

describe("Firestore Integrity", () => {
  const db = getFirestore();

  it("required collections exist", async () => {
    const collections = await db.listCollections();
    const names = collections.map((c) => c.id);
    expect(names).toContain("entities");
    expect(names).toContain("users");
  });

  it("no orphan subcollections", async () => {
    // Check that subcollections have valid parent documents
    const entities = await db.collection("entities").limit(10).get();
    for (const doc of entities.docs) {
      const subcols = await doc.ref.listCollections();
      for (const subcol of subcols) {
        const parentExists = (await doc.ref.get()).exists;
        expect(parentExists).toBe(true);
      }
    }
  });

  it("required indexes are deployed", async () => {
    // Check firestore.indexes.json matches deployed indexes
    // This is verified by attempting queries that require composite indexes
  });
});
```

### Cloud SQL Integrity (via Prisma)

```typescript
// qa-tests/db/cloud-sql-integrity.validation.test.ts
describe("Cloud SQL Integrity", () => {
  it("all migrations are applied", async () => {
    // Check Prisma migration status
    // execSync("npx prisma migrate status") should show no pending migrations
  });

  it("no orphan records", async () => {
    // Check foreign key relationships
  });

  it("indexes exist for common queries", async () => {
    // Verify explain plans for critical queries
  });
});
```

## Part 8 — Go/No-Go Report

After executing all validations, generate a comprehensive report:

```json
{
  "report": {
    "project": "project-name",
    "version": "x.y.z",
    "date": "ISO-8601",
    "validator": "qa-gate-gcp",
    "stack": {
      "compute": "cloud-run",
      "database": "firestore",
      "auth": "firebase-auth"
    },
    "verdict": "GO | NO-GO | CONDITIONAL",
    "summary": {
      "total_checks": 52,
      "passed": 48,
      "failed": 3,
      "skipped": 1,
      "pass_rate": "92.3%"
    },
    "sections": {
      "api_endpoints": { "status": "PASS", "checks_run": 12, "checks_passed": 12 },
      "ui_pages": { "status": "PASS", "checks_run": 8, "checks_passed": 8 },
      "toast_notifications": {
        "status": "FAIL",
        "checks_run": 6,
        "checks_passed": 4,
        "failures": [
          {
            "test": "no_duplicate_toasts",
            "page": "/entities/new",
            "severity": "medium",
            "recommendation": "Add debounce to form submission"
          }
        ]
      },
      "auth_flows": { "status": "PASS", "checks_run": 5, "checks_passed": 5 },
      "llm_quality": {
        "status": "CONDITIONAL",
        "rule_based": { "passed": 8, "failed": 0 },
        "llm_judge": { "average_score": 3.8, "recommendation": "review" }
      },
      "database_integrity": {
        "firestore": { "status": "PASS", "security_rules_enforced": true },
        "cloud_sql": { "status": "PASS", "migrations_applied": true }
      },
      "gcp_infrastructure": {
        "cloud_run": { "status": "PASS", "service_ready": true, "latest_revision_serving": true },
        "cloud_sql": { "status": "PASS", "instance_running": true, "backup_enabled": true },
        "secret_manager": { "status": "PASS", "all_secrets_present": true }
      }
    },
    "blockers": [],
    "warnings": [
      { "id": "WARN-001", "severity": "medium", "description": "Duplicate toasts on rapid clicks" },
      { "id": "WARN-002", "severity": "low", "description": "LLM tone slightly formal" }
    ],
    "go_conditions": {
      "all_api_tests_pass": true,
      "all_auth_tests_pass": true,
      "no_high_severity_blockers": true,
      "llm_quality_above_threshold": true,
      "gcp_services_healthy": true,
      "security_rules_enforced": true,
      "secrets_in_secret_manager": true
    }
  }
}
```

### Verdict Logic

- **GO**: All checks pass, no blockers, GCP services healthy, security rules enforced.
- **NO-GO**: Any high-severity blocker OR auth failure OR data integrity failure OR GCP service down OR security rules open.
- **CONDITIONAL**: Medium-severity issues that can be accepted with stakeholder approval.

Save to `qa-reports/go-no-go-report.json` and `qa-reports/go-no-go-report.md`.

## Part 9 — Execution Pipeline

```
1.  Generate test plan              → qa-reports/test-plan.json
2.  Run existing test suite         → npx vitest run / npx playwright test
3.  Generate validation tests       → qa-tests/**/*
4.  Run API validations             → qa-tests/api/
5.  Run UI/toast validations        → qa-tests/ui/
6.  Run auth flow validations       → qa-tests/auth/
7.  Run LLM quality validations     → qa-tests/llm/
8.  Run GCP infra validations       → qa-tests/infra/ (bash scripts via gcloud CLI)
9.  Run database integrity checks   → qa-tests/db/
10. Aggregate results               → qa-reports/go-no-go-report.json
11. Generate human report           → qa-reports/go-no-go-report.md
```

### Commands

```bash
# Step 2: Existing tests
npx vitest run --reporter=json --outputFile=qa-reports/vitest-results.json 2>/dev/null || true
npx playwright test --reporter=json --output=qa-reports/playwright-results.json 2>/dev/null || true

# Step 3-7: Validation tests
npx vitest run --config qa-tests/vitest.config.ts --reporter=json --outputFile=qa-reports/validation-results.json
npx playwright test --config qa-tests/playwright.config.ts --reporter=json --output=qa-reports/playwright-validation-results.json

# Step 8: GCP infra (bash scripts)
bash qa-tests/infra/cloud-run-validation.sh "$SERVICE_NAME" | tee qa-reports/cloud-run-validation.log
bash qa-tests/infra/cloud-sql-validation.sh "$INSTANCE_NAME" | tee qa-reports/cloud-sql-validation.log
bash qa-tests/infra/firestore-rules-validation.sh | tee qa-reports/firestore-rules-validation.log
bash qa-tests/infra/secret-manager-validation.sh | tee qa-reports/secret-manager-validation.log
```

## Best Practices (DO)

- Always run the existing test suite FIRST before adding validation tests
- Use separate directories (qa-tests/, qa-reports/) to avoid polluting the app
- Detect and adapt to the project's test framework (Vitest/Jest, Playwright/Cypress)
- Run rule-based LLM checks before LLM-as-judge (cheaper, faster)
- Include severity levels in all failures (high/medium/low)
- Generate both JSON and Markdown reports
- Validate GCP infra using gcloud CLI (not HTTP calls to management APIs)
- Check Firestore security rules for open access patterns
- Verify Secret Manager has all required secrets with enabled versions
- Check Cloud SQL backup configuration
- Validate Cloud Run service health via the /api/health endpoint

## Anti-Patterns (AVOID)

- NEVER skip the test plan generation step
- NEVER mix validation tests with app tests (separate config files)
- NEVER hardcode auth tokens in test files
- NEVER run LLM-as-judge without rule-based checks first
- NEVER mark a test as "skipped" without documenting why
- NEVER auto-approve a NO-GO verdict
- NEVER test against production data
- NEVER ignore toast validation
- NEVER use gcloud commands that modify resources during validation (read-only!)
- NEVER expose secret values in logs or reports — only check existence

## Safety Rules

- NEVER read or modify `.env`, `.env.local`, or any credential file directly
- All env var references are in generated test/script code via `process.env.*` or `os.environ.get()`
- NEVER auto-deploy after a CONDITIONAL or NO-GO verdict
- NEVER delete data from production databases
- NEVER expose API keys or secret values in test reports — redact before writing
- If OPENROUTER_API_KEY is not set, skip LLM-as-judge and mark as "review"
- All gcloud commands are READ-ONLY (describe, list) — NEVER run create, update, delete during validation
- NEVER read secret values from Secret Manager — only check existence and enabled status
Qa Gate Gcp

SKILL.md

related skills
