Problem
Product teams tracked competitor Trust & Safety policies by hand, monthly at best.
My role
Senior PM. I scoped, designed, and built it end to end.
What I built
A monitor that checks 25+ policies every 6 hours and emails AI summaries of real changes.
Results
Alerts in under 6 hours on average, over 95% precision, 99.9% uptime, 10+ PM hours saved a month.
Trust & Safety Policy Watcher main view showing policies tracked, minutes since the last check, and time to the next check
The main view: policies tracked, minutes since the last check, and time to the next one.

The problem

In social commerce and live streaming, Trust & Safety policy is strategy. A new TikTok guideline or Meta commerce rule can show where the market is going. It can open an advantage or expose a risk.

Product teams tracked these changes by hand. They checked maybe once a month and missed most meaningful changes. They heard about updates weeks late, with no history to compare against. PM hours went to routine checking instead of analysis.

I had spent years as a Trust & Safety PM. This was work a machine should do.

What success looked like

One source of truth that runs with no upkeep. Three bars had to hold:

  • Fast: hear about a change within hours, not weeks
  • Precise: only real changes. One noisy alert and people stop reading.
  • Trusted: the system reports its own health, so silence means "nothing changed," not "something broke"

Key decisions

Precision over coverage

The hard part was telling real policy edits from cosmetic site changes. Ads, timestamps, and session IDs made pages look new on every run. YouTube looked changed daily while the text was identical.

I chose to filter aggressively. The pipeline strips 20+ kinds of dynamic content and compares both page structure and plain text. Missing a tiny edit costs less than a false alarm, because false alarms train people to ignore the tool.

Summaries about impact, not diffs

Raw diffs are useless to a busy PM. Each change goes to Google Gemini with one question: what changed, and why does it matter for product strategy? Alerts arrive as plain text email, which reads cleanly on a phone, where executives often check them.

Email alert on a phone with a short AI summary of a policy update
An alert on a phone: platform, what changed, and why it matters.

Two fetchers for bot-protected sites

TikTok, Meta, and Whatnot block ordinary scrapers. Plain HTTP requests handle simple pages. Playwright browser automation handles protected ones. Every site gets covered, and simple ones stay fast.

Zero maintenance as a requirement

It runs on GitHub Actions and recovers from errors on its own. A run log and health checks surface any failure, so problems get fixed before anyone notices stale data.

Policy Matrix listing platforms, tracked policies, status, last update, and quick actions
The Policy Matrix: every platform and policy, with status and when it last changed.

Results

  • 25+ policies tracked across TikTok, YouTube, Meta, Whatnot, and Twitch
  • Under 6 hours on average from a policy change to the alert, down from weeks
  • Over 95% precision telling real changes from cosmetic ones
  • 99.9% uptime with no silent failures
  • 10+ hours a month saved per PM, by estimate
Run log showing recent runs, with a failed run flagged
The run log shows failures with timestamps and error counts, so nothing fails quietly.

Takeaways

  • Internal tools need product rigor. Automating the checks was the easy part. Alert volume and trust decided whether anyone used it.
  • For alerting, precision beats recall. A tool that cries wolf gets muted.
  • Reliability is a feature. If people rely on it, uptime belongs on the roadmap.

What's next

More platforms (Discord, Snapchat, LinkedIn), policies outside the US, and trend analysis across platforms to flag competitive openings.

The Policy Watcher is live at ts-policy-watcher.vercel.app.