Two essays. One thesis at two altitudes. The first is what a company needs to ship AI without becoming the case study. The second is what a platform needs when anyone can publish software.
One thesis at two altitudes. Read either on its own — or scroll on for the full deep-read in sequence.
When generation gets cheap, evaluation becomes the edge.
The Taste Layer = Quality Gates for AI Outputs
Three technical capabilities: Correctness (accuracy, factuality) + Safety & Compliance (PII, regulations) + Brand Quality (tone, style, trust)
Modern AI excels at fluency—but fluency isn't accuracy.
Simone Leonelli — 16 years building quality systems for luxury brands where defects were unacceptable (Fendi, Bentley, Porsche). After seeing enterprises struggle with AI quality, I spent 2024-2025 bridging design systems thinking and ML evaluation.
Now building AI evaluation infrastructure for high-stakes enterprise applications.
CASE STUDY: Moffatt v. Air Canada (2024)
A simple customer support chatbot without guardrails promised a refund policy that didn't exist. When sued, the airline argued the AI was a "separate legal entity" responsible for its own actions.
The tribunal rejected this. The company was held liable for the hallucination—specifically for negligent misrepresentation via incorrect policy guidance.
Lesson: This case involved a basic chatbot without modern evaluation layers. It demonstrates what happens when AI outputs reach customers without systematic quality checks.
Source: McCarthy Tétrault Legal Analysis
I've analyzed 10+ enterprise AI deployments across healthcare, finance, and B2B services.
The majority lacked dedicated evaluation gates.
Observation from consulting engagements: "Evaluation layer" defined as systematic gating of outputs before customer delivery, with defined metrics, ownership, and escalation paths.
The Industry Shift: The bottleneck in AI development has shifted from "can we generate this?" to "can we trust this?" Evaluation infrastructure is the unsexy but critical layer that separates production-ready AI from expensive demos.
What "Taste Layer" Actually Means:
Taste Layer = Measurable Quality Constraints + Automated Gates + Audit Trails.
It is the infrastructure that makes AI outputs shippable—not subjective aesthetics, but rigorous adherence to policy, correctness, and brand compliance.
The Common Failure Mode: Ad-Hoc Evaluation
Many organizations rely on "vibes-based" spot checks rather than systematic gating. The AI team tracks velocity, but ownership of accuracy falls through the cracks between Engineering (who build pipelines) and Product/Legal (who define quality).
Without a dedicated evaluation layer, the customer becomes the QA team.
The AI industry delivered "generation at scale."
But without quality checks, volume creates liability, not value.
We are living through a constraint shift: generation got cheap and fast, evaluation stayed expensive and slow.
"Marginal cost of generation is collapsing" is directionally right, but technically sloppy.
The truth is scarier:
*Directional trends based on LLM API pricing (Epoch AI, 2020-2025) and enterprise evaluation infrastructure costs. Projections beyond 2025 are directional estimates, not forecasts. Actual costs vary by implementation.
Traditionally, judgment was artisanal — slow, expensive, unscalable. One expert could train maybe 10 apprentices in a lifetime.
In AI, we can BUILD judgment systems. Scale them. Prove them. Deploy them at machine scale.
This is the arbitrage.
The Methodology Transfer:
In high-stakes manufacturing, products pass through multiple independent quality gates—materials, construction, and final review. No single person can approve their own work.
This principle applies to AI: the system that generates should never be the system that judges. Separation of concerns isn't just good architecture—it's how you build trust at scale.
Technically: Judgment is a trained compression function over a domain. In enterprise terms: Domain-Aligned Quality.
"Judgment can be instant, but it is rarely free. Someone paid for it in years."
In markets, rents concentrate at constraints.
Adapted from quality engineering:
Engineering, Legal, and Product have different budgets and incentives.
| Stakeholder | What They Care About | Budget Source | Your Pitch |
|---|---|---|---|
| Engineering/CTO | Ship velocity, incident rate | Platform/Infra | "Reduce on-call incidents from AI failures" |
| Risk/Legal/CISO | Liability exposure, compliance | Risk budget | "Auditable defense against AI liability" |
| Product/Brand | Customer trust, NPS | Product budget | "Quality as differentiator, not cost center" |
Most companies treat evaluation like a post-ship audit. Winners build rejection gates before anything reaches production.
trace_9f8a7d2 (Stored for Audit)
This is not just a philosophy. It is a set of tangible artifacts that your engineering, legal, and product teams will own.
{
"CRITICAL": {
"HALLUCINATION_FACT": "Claims contradiction with retrieval ctx",
"UNSAFE_PII": "Contains email/phone/SSN",
"ILLEGAL_ADVICE": "Offers medical/legal judgment"
},
"WARN": {
"OFF_BRAND_TONE": "Tone mismatch score > 0.4",
"VERBOSITY": "Response > 150 words"
}
}
class FactJudge(dspy.Signature):
"""Judge if response is supported by context."""
context = dspy.InputField()
response = dspy.InputField()
# Output
is_supported = dspy.OutputField(desc="True/False")
citation_check = dspy.OutputField(desc="List of missing citations")
We treat the judge as a software product that requires its own testing. This is how we ensure reliability.
Companies deploying AI without systematic quality checks are accumulating hidden risk. The Taste Layer is how leading enterprises are building durable competitive advantage.
The Taste Layer is not universally required. Here's how to know if it's right for your use case.
The Taste Layer = 3 Technical Capabilities "Taste" isn't subjective—it's three measurable evaluation systems working together:
Factuality, task success, technical accuracy. Measurable via hallucination rate, accuracy metrics.
PII detection, regulatory alignment, policy adherence. Measurable via violation rates, audit results.
Tone, style, UX fit, customer experience. Measurable via brand alignment scores, trust metrics.
Each layer has distinct metrics, tooling, and funding sources (Engineering, Risk/Legal, Brand/Product). "Taste" is the brand wrapper for these three technical capabilities.
Concrete examples of evaluation gates by vertical:
Healthcare
Gate 1: Block medical advice outside scope of practice
Gate 2: Verify clinical claims against FDA-approved indications
Gate 3: Ensure HIPAA-compliant language
Finance
Gate 1: Detect and block PII leakage
Gate 2: Validate regulatory disclosures (FINRA, SEC)
Gate 3: Check investment advice against fiduciary standards
Retail/Brand
Gate 1: Filter offensive or off-brand language
Gate 2: Verify product claims against inventory/specs
Gate 3: Enforce brand voice guidelines and tone
What an enterprise buyer expects to see on Monday.
Core Infrastructure Components
⚙️ EVAL OPS IS REAL WORK:
Evaluation is a production system with ownership, on-call rotation, and versioning. It's not a script. Treat the judge like any critical service: monitor uptime, track drift, test with adversarial prompts, maintain human panels for calibration.
System Architecture: Component View
Input
User prompt
+ Context
+ History
Generator
LLM API
+ Retrieval
+ Tools
Eval Gates
Judge(s)
+ Policy filters
+ Golden set
Routing
Pass
Auto-fix loop
Human review
Supporting Infrastructure
Telemetry: Log all prompts, outputs, scores, model versions
Dashboard: Defect rate, coverage, latency, cost tracking
Human Queue: Escalated cases, calibration, golden set curation
→ Flow: Input passes through generator, eval gates score output, routing logic decides pass/fix/escalate, telemetry captures everything
Correctness: Factual error rate, task success, regression pass rate, citation validity
Safety/Compliance: PII leak rate, policy violation rate, refusal accuracy
Brand/Product: Tone match score, style violations, editor accept rate, complaint rate
Week 1: Collect 200-500 real outputs, label defects, build golden set v0, choose metrics, set SLOs
Week 2: Implement gates 1-2 (safety + correctness), dashboard defect rate, create escalation queue, run canary release
⚠️ Addressing the Objection "LLM-as-a-Judge is unreliable. Judges are biased, brittle, and easy to prompt-hack."
This is a valid concern. The solution is not to trust the judge blindly, but to treat it like any production system requiring testing, monitoring, and calibration.
Mitigation Strategies
Ownership & Monitoring
The judge is tested and monitored like any model. It's not magic—it's infrastructure.
While providers (Anthropic, OpenAI) solve safety and tooling platforms (LangSmith, Arize) solve tracing, domain-specific judgment remains the enterprise gap.
* Visual hypothesis illustrating the strategic shift from technical assembly to quality assurance.
Evaluation is a production system with real trade-offs.
Robust judges take time.
Verdict: Essential for async/email. Skip for real-time voice.
Strict gates can kill creative outliers.
Verdict: Tune "Judge Temperature" to match risk tolerance.
Golden sets rot as models change.
Verdict: Requires an "Eval Ops" owner, not just a script.
Generic checks (PII, Syntax) will become free/standard.
Verdict: The value is in bespoke Policy & Brand definitions.
| Indicator | Observed Trend |
|---|---|
| The Talent Filter | Ratio of "AI Engineer" roles requiring "Evaluation/Red-teaming" skills will flip from minority to majority by 2027. |
| The Compute Flip | Compute investment is shifting toward Inference-Time Reasoning as the primary lever for accuracy and reliability. |
| The New Category | Evaluation Infrastructure is emerging as a standalone category, independent of foundation model providers. |
Falsifiable Prediction By December 2026, at least one Fortune 500 company will list "AI Evaluation," "Model Risk," "AI Assurance," or "GenAI Quality" as a core competency in their 10-K filing, annual report, or earnings call transcript.
If I'm wrong, the manifesto ages poorly. If I'm right, you heard it here first. Hold me accountable.
What counts: Public disclosure in SEC filings (10-K), annual reports, risk sections, or earnings transcripts explicitly referencing evaluation capability as strategic or operational competency.
🔬 What Would Falsify This Thesis
Intellectual honesty requires stating what could prove me wrong. Here are concrete scenarios that would undermine the need for separate evaluation infrastructure:
Falsifier 1: Platform Commoditization
Foundation model providers ship end-to-end evaluation and governance that removes the need for separate infrastructure for most enterprises.
Falsifier 2: Near-Zero Eval Cost
Tooling advances reduce eval cost to near-zero even for domain-specific judgment, eliminating the economic moat.
Falsifier 3: Human-in-the-Loop Dominance
Enterprises shift to human-in-the-loop workflows at scale instead of automated gates because automated eval remains too brittle.
If any of these scenarios materialize, the strategic value of separate evaluation infrastructure diminishes significantly. I'm tracking these possibilities and will update this thesis accordingly.
The principles of quality assurance are universal, whether applied to physical goods or probabilistic software. Studio W230 translates rigorous quality methodologies—proven in high-stakes manufacturing—into the domain of AI evaluation. This is not about aesthetics; it is about systemizing judgment to prevent defects at scale.
⚡ Do This Tomorrow
The 5-minute test: Take your last 10 AI-generated customer communications. Have someone outside your team rate them for accuracy.
What's your defect rate?
If you don't know the answer, you're flying blind. If the answer makes you uncomfortable, that's the data you needed.
This manifesto represents ongoing research on The Taste Layer.
Working on AI evaluation challenges? I'd love to learn from your experience.
Simone Leonelli — Studio W230
A selection of foundational research and industry reports for those looking to build rigorous evaluation systems.
| Domain | Resource |
|---|---|
| Legal Liability | CRT Tribunal — Moffatt v. Air Canada (2024) |
| Cost Dynamics | Epoch AI — LLM Accuracy-Runtime Trade-offs (2025) |
| Market Research | NIQ — CMO Outlook: Guide to 2026 |
| AI Strategy | IBM — The AI-First CMO C-suite Study |
| Trust Economy | Salsify — Consumer Research 2025: The Value of Trust |
| Industry Forecast | Gartner — AI TRiSM (Trust, Risk & Security Management) |
| Risk Assessment | Stat News — UnitedHealth AI Lawsuit (Alleged 90% Error Rate) |
| Foundational | Simon (1971) — Designing Organizations for an Information-Rich World |
Chapter one asked: how does one company ship AI to its own customers without becoming the next case study? Chapter two asks the same question across a marketplace.
A new category is forming: user-generated software (UGS). Natural language to runnable apps, published into a shared catalog where discovery, ranking, and reuse are product primitives. The moment you add a marketplace feed, you inherit curation as an infrastructure problem.
UGS is distinct from internal one-off scripts, private automations without discovery, or traditional open-source repositories. It combines creation democratization with platform-mediated distribution.
The generation layer is converging fast. Prompt-to-app pipelines are becoming a commodity. The differentiators that matter in 2025 (speed, fidelity, native deployment) will be table stakes by 2027.
This table documents only publicly observable features as of February 2026. Internal roadmaps and private features are not assessed. Absence from this list does not mean absence from a platform's plans.
| Platform | Product Shape | Documented Curation Primitives |
|---|---|---|
| Wabi | Social-first UGS, persistent identity, remix culture | Algorithmic feed, remix as product feature, community signals |
| Anything | Full-stack code generation, native mobile + web, App Store deploy | Code linting, error recovery, user reviews, experts marketplace |
| Replit | Developer-adjacent platform, enterprise controls, SOC2 | Security screening, code review (enterprise tier), featured templates |
| Lovable | Design-forward code generation, visual quality emphasis | Template gallery, curated examples |
| GPT Store | Custom GPTs marketplace, conversational interface, no-code creation | Policy review, verified builder program, leaderboards, categories |
The pattern: every platform has invested heavily in generation infrastructure. Where curation primitives exist, they tend to be narrowly scoped — policy compliance, security screening, editorial picks — rather than systematic quality infrastructure across discovery and lifecycle.
The core thesis: In user-generated software marketplaces, the generation engine is converging toward commodity. The curation layer — the infrastructure that makes quality legible, discoverable, and trustworthy — is the durable competitive advantage. Elements of curation exist across platforms (editorial picks, reviews, health checks), but no platform has publicly named a coherent, end-to-end curation architecture as a product layer. The mechanism: quality tiers change user behavior (trust, reduced bounce), ranking changes creator incentives (polish yields visibility), and decay management changes retention (fewer broken first experiences).
YouTube in 2006: anyone can upload video. Massive volume. Discovery by views and virality. No quality signals beyond popularity. The platform was exciting and chaotic and largely unusable for anything serious.
YouTube in 2016: creator monetization, content policies, algorithmic ranking weighted by watch time and satisfaction, trust and safety teams, verified channels, copyright systems. A decade of quality infrastructure investment. [YouTube Partner Program History]
YouTube in 2025: professional-grade content coexists with amateur uploads. Quality tiers are legible. Creators have incentives to invest in production value. The platform succeeded not because it hosted the most video, but because it built the systems that made quality discoverable.
"Anyone can create." Platform celebrates quantity. Discovery is primitive (recency, popularity). Quality is vibes-based. Early adopters love the chaos.
Library grows past manual curation capacity. Broken, abandoned, and low-quality entries dominate discovery. New users bounce. Retention drops.
Platform builds quality signals: ratings, editorial picks, creator incentives, algorithmic quality weighting, trust tiers. Discovery improves. Quality becomes a platform feature, not an accident.
Quality infrastructure enables a creator economy. Top creators invest because the platform rewards quality. The flywheel works.
Every UGC platform, from YouTube to Etsy to the App Store, has traversed this arc. The ones that invested in quality infrastructure early (Airbnb's trust framework, Spotify's editorial playlists) compressed the timeline.
Luxury brands have solved the curation problem for physical goods across centuries. The methodology is precise, systematic, and transferable. It rests on three principles that map directly to software platform quality.
In luxury manufacturing, every production run is evaluated against a golden sample, a reference artifact that embodies the quality standard. Not a spec document. Not a checklist. A physical object you can hold, examine, and compare against.
For user-generated software: the equivalent is a curated set of reference apps that define what "excellent" looks like on the platform. Not just featured apps. Architecturally sound, well-designed, maintained, and deeply useful apps that serve as the evaluation benchmark.
At Fendi, the artisan who sews a bag never performs the final quality inspection. At Bentley, the engineer who builds a component is not the person who signs off on the finished vehicle. The system that generates should never be the system that judges.
You know a Michelin three-star restaurant is different from a one-star. You know a Hermès Birkin passed different gates than a department store handbag. The quality difference is visible and legible to the consumer before purchase.
Luxury brands don't rely on customers to discover quality through trial and error. They invest in making quality visible before the customer commits. This is the infrastructure that UGS platforms are missing.
Enterprise evaluation (as described in The Taste Layer) uses binary gates: pass or fail on correctness, safety, and brand compliance. Platform curation is structurally different. It operates on gradients, not binary gates, because the goal is ranking and discovery, not rejection.
The foundation layer. Every app carries a verified creator identity, a complete fork lineage, and a declared permission manifest. This enables governance (whose app is this?), trust transfer (verified creator badges), and fork safety (did this remix degrade or improve quality?).
Automated health checks at publish and continuously post-deploy. Does the app load? Do buttons trigger responses? Does it handle edge cases gracefully? Error rate monitoring catches decay before users do.
Measurable through automated heuristics: accessibility score (WCAG compliance), responsive breakpoint pass rate, contrast ratio checks, interaction completeness (percentage of UI elements that respond when activated), and layout anomaly detection. Not subjective taste — design signals that correlate with user trust and task completion.
Measured via cohort retention (D1, D7, D30 return rates), session depth, task completion rate, time-to-value (seconds from first open to first meaningful action), and category-normalized benchmarks. A personal CRM used daily by 200 people scores higher than a novelty app opened once by 10,000.
The composite layer. Weighted scores from L0–L3 produce a quality signal that powers discovery ranking, tier badges (Reference / Verified / Community / Unrated), and editorial featuring. This is where the Michelin model meets algorithmic scale.
L0 is marketplace-native infrastructure. In UGS platforms, provenance isn't decorative: it enables governance (whose app is this?), trust transfer (verified creator badges), and fork safety (did this remix degrade or improve quality?).
| Dimension | Enterprise (Taste Layer) | Platform (Curation Stack) |
|---|---|---|
| Decision type | Binary: ship or block | Gradient: rank, tier, feature |
| Quality owner | Internal team (Engineering + Legal + Brand) | Platform + creators + community |
| Evaluation timing | Pre-deployment gate | Continuous (creation, discovery, usage, decay) |
| Failure mode | Hallucination reaches customer | Low-quality apps dominate discovery feed |
| Economic incentive | Liability avoidance | Marketplace trust, retention, creator economy |
The traditional software quality hierarchy (code quality, test coverage, performance benchmarks) doesn't transfer cleanly to user-generated software. Most UGS apps have no tests, no CI/CD, no code review. The code is generated and opaque. Quality must be inferred from observable signals.
Discovery is a multi-objective ranking function: quality, freshness, personalization, and safety. The goal is not to block creation, it is to allocate attention under scarcity.
Load time, error rate, crash frequency, responsive layout, accessibility score, interaction completeness (do all buttons do something?), data persistence, session handling.
Return rate (used more than once), session depth, task completion, remix count, share rate, time-to-value (how fast does the user get utility?), active days since creation.
Quality score (weighted composite), editorial selection, creator reputation, category benchmark (best-in-class within type), freshness vs. durability balance.
The critical insight: popularity is not quality. A viral novelty app with 10,000 uses and a 5% return rate is less valuable to the platform than a personal CRM with 200 users and an 80% weekly return rate. Volume-weighted discovery repeats the YouTube 2006 mistake. Quality-weighted discovery builds the YouTube 2025 flywheel.
Most downloaded → most visible → more downloads. Novelty dominates. Utility dies in the long tail. Quality creators have no incentive to invest. Feed becomes noise. New users see broken apps, bounce, never return.
Highest utility → most visible → more sustained usage. Depth rewarded. Creators invest in polish because the platform rewards it. Feed feels curated. New users find useful tools immediately. Retention compounds.
Discovery is a multi-objective optimization. Here is the scoring function in sketch form:
This is the problem that separates user-generated software from user-generated content. A YouTube video from 2012 still plays. A blog post from 2015 still reads. But software decays: APIs change, dependencies break, design patterns age, data schemas drift.
In a UGS marketplace, apps are created in seconds and abandoned in hours. The platform accumulates digital debris at a rate proportional to creation velocity. Without active lifecycle management, the library becomes a graveyard with a fresh coat of paint.
Month 1: 1,000 apps created. Most functional. Feed is exciting.
Month 6: 50,000 apps. Dependency drift, API changes, and abandonment begin to degrade the library. Discovery surfaces abandoned apps alongside active ones. User frustration rises.
Month 12: 500,000 apps. A significant percentage are broken or abandoned. The "most popular" feed is dominated by novelty apps with declining utility. Quality apps are buried. Creator motivation drops because effort isn't rewarded. New user experience degrades.
Without quality infrastructure, creation velocity becomes a liability, not an asset.
A curation stack needs to track app health over time, not just at creation. Signals that matter: last updated, error rate trend (improving or degrading?), creator activity status, dependency health, user-reported issues. Apps should gracefully age out of discovery when they stop working, and resurface when maintained.
Michelin doesn't rate every restaurant. It rates the ones worth attention, using a consistent, independent, expert-driven framework. This is the model that UGS platforms need.
A quality tier system for user-generated software creates three platform-level benefits simultaneously: user trust (I know what I'm getting), creator incentive (investment in quality is rewarded with visibility), and marketplace efficiency (discovery surfaces the right apps to the right users).
The tier system is not gatekeeping. It is legibility. Users can still create, share, and use any app. But the discovery layer rewards quality, and the tier badges make quality visible before the user commits time and attention.
This is also the creator monetization path. In YouTube's model, monetization follows quality signals (subscriber count, watch hours, content policies). In UGS, monetization should follow quality tiers. Reference-tier creators earn from their work. Community-tier creators build reputation toward verified status.
One feature unique to UGS platforms (especially Wabi) is remix: the ability to fork, modify, and reshare apps. This creates a quality signal that doesn't exist in traditional software or content marketplaces.
Remix count is a proxy for architectural quality. An app that gets remixed frequently has a structure worth building on. An app that never gets remixed may be useful but is likely either too niche or too brittle to modify.
More importantly, remix creates quality compounding. Each remix potentially improves the original. The best apps become platforms within the platform, accumulating improvements from the community. This is open-source dynamics applied to personal software.
The platforms that treat remix as a quality mechanism (rather than just a social feature) will build compounding quality advantages. Remix genealogy, the ability to trace an app's lineage and identify which versions improved versus degraded quality, is a powerful curation signal that no platform currently tracks.
The default assumption in platform thinking: quality will emerge organically from community engagement. Upvotes, reviews, usage metrics, and social sharing will surface the best apps naturally.
This assumption has failed in every UGC marketplace that relied on it exclusively.
The failure pattern is consistent: at scale, volume drowns quality unless the platform actively invests in quality infrastructure. Community signals are a necessary input to curation, but they are not sufficient. Editorial intelligence, automated quality gates, and tier systems must layer on top.
When curation is under-resourced relative to creation velocity, predictable pathologies emerge:
| Failure Mode | Root Cause | Observable Signal | Mitigation Primitive |
|---|---|---|---|
| Broken apps dominate feed | No health gating at publish or decay detection | Load failure rate, crash rate trending up | Publish-time health checks, downrank on failure |
| Novelty beats utility | Popularity proxy (views, upvotes) over depth | Low weekly_return_ratio despite high creation volume | Quality-weighted ranking, return-rate boosting |
| Clone spam floods categories | Cheap generation, no uniqueness filter | High near-duplicate rate, keyword stuffing | Similarity detection, rate limits, trust tier gating |
| Rating inflation / Sybil attacks | Unverified upvotes, botted engagement | Spiky engagement patterns, low session depth | Verified user weighting, behavioral fraud detection |
| Badge laundering via forks | Reputation inheritance on remix | High abuse reports on forked verified apps | Badge doesn't transfer, fork must re-qualify |
| App rots trust | Dependency drift, no maintenance incentive | error_rate_7d_delta trending up, abandonment | Decay downrank, maintenance rewards |
The a16z thesis for Wabi explicitly expects "professionalization" to emerge organically, comparing to YouTube's arc. But YouTube's professionalization required a decade of active platform investment in quality systems. The organic part was the creator response to those systems, not the systems themselves.
A curation stack without a governance layer is incomplete architecture. Quality in a UGS marketplace is not only about surfacing good apps. It is also about preventing malicious, deceptive, or harmful ones from reaching users at all.
When any user can generate software through natural language, the attack surface expands beyond traditional app store threats. The generation layer itself becomes an abuse vector.
These threats are not hypothetical; every major UGC marketplace at scale has faced analogous attack patterns. The governance layer must operate as an independent system from the curation layer, with distinct ownership, escalation paths, and enforcement mechanisms. Curation answers "is this good?" Governance answers "is this safe?"
Based on publicly available positioning (checked public docs, marketing surfaces, and product UX as of Feb 2026), no major UGS platform appears to publicly present systematic curation infrastructure as a named product layer. [Inference] This is not because the problem is invisible. It is because the generation race is consuming all available engineering attention.
The window is specific: [Hypothesis] Between now and the point where library size exceeds manual curation capacity (plausibly within the next 1-2 growth cycles at current creation velocity), the curation architecture decisions will determine which platforms build durable marketplace trust and which drown in their own output.
By December 2027, at least one major UGS platform will have a named, public-facing quality tier system (analogous to Michelin stars, App Store editorial picks, or Airbnb Superhost) as a core product feature. The platform that builds it first will have measurably better D30 retention (percentage of new users active 30 days after first session) than competitors without it.
If this doesn't happen, the UGS category may not survive as a distinct product category, because undifferentiated app generation will commoditize into a feature of existing platforms (ChatGPT, Claude, general-purpose AI assistants).
Any UGS platform that adds publish-time health checks and decay-based downranking will see broken-app impressions in discovery drop within 30 days, with a corresponding improvement in D7 retention. This requires no algorithmic sophistication — just a load test on publish and an error rate monitor on a cron job.
For a UGS platform founder reading this on Monday morning, here is the minimum infrastructure that creates a quality advantage.
One app's journey through the stack, from creation to maturity:
Eight weeks. A small team. The foundation of a curation moat that compounds as the library grows. Every week of delay at current creation velocity adds thousands of uncurated apps to the discovery problem.
Abstract quality frameworks are useless without measurable telemetry. These are the signals a curation stack ingests, each mappable to a database column and a dashboard row.
Ten signals. Each observable, each automatable, each meaningful in isolation and composite. A platform team could instrument these in a sprint and begin building quality intelligence immediately.
The platforms that build curation infrastructure early are building durable competitive advantages. The ones that wait will spend 10x the resources retroactively sorting through a library that has already trained users to expect noise.
This is the same lesson luxury brands learned over centuries, compressed into a platform lifecycle. Quality systems don't scale retroactively. They must be architectural, designed into the platform from early days, not bolted on after the feed is already broken.
Generation is the cost of entry. Curation is the cost of survival. The platforms that confuse volume for value will not outlast the ones that invest in making quality legible, discoverable, and rewarding for creators.
The curation problem is not new. It's the same problem Hermès, Michelin, YouTube, and Airbnb solved in their respective domains. The tools are different. The principles are identical. Systematic evaluation, independent judgment, legible quality signals, and incentive alignment between platform, creator, and user.
The question isn't whether UGS platforms will need a curation stack. It's whether they'll build it before the library outgrows their ability to retrofit it.
| Platform | Resource |
|---|---|
| Wabi | wabi.ai | Personal software platform for mini‑apps |
| Anything | createanything.com | AI app builder, formerly Create.xyz |
| Replit | replit.com | Developer platform with AI generation |
| Lovable | lovable.dev | Design-forward AI code generation |
| GPT Store | openai.com/gpt-store | Custom GPT marketplace announcement |
| Topic | Resource |
|---|---|
| Wabi Funding | TechCrunch | "Replika founder raises $20M pre-seed for Wabi" (Nov 2025) |
| Anything Community | Product Hunt | Anything launch and community discussion |
| Domain | Resource |
|---|---|
| Enterprise Evaluation | The Taste Layer | Companion manifesto on enterprise AI evaluation infrastructure |
| YouTube Quality Arc | Based on publicly documented YouTube Partner Program history, Creator monetization policy changes (2007-2025), and YouTube Partner Program official documentation |
| Michelin Guide | Quality tier methodology synthesized from public Michelin Guide methodology descriptions and restaurant industry analysis |
| Luxury Manufacturing | Golden sample methodology derived from author's 16 years in luxury brand quality systems (Fendi, Bentley, Porsche) |
[Evidence] – Directly supported by linked public documentation, product UI, press coverage, or official platform materials.
[Inference] – Derived from multiple evidence points, observed product behavior, or publicly available positioning. Not explicitly stated by the platform but reasonably concluded from available information.
[Hypothesis] – Forward-looking claim, prediction, or proposed framework not yet validated. Presented as testable thesis.
[Unverified] – Specific data point (e.g., a number or date) that cannot be confirmed from a stable, citable source. Included for directional context only.
Platform assessments: Based on publicly available positioning, documentation, and press coverage as of February 2026. Internal product roadmaps may differ from public positioning. All comparative claims are labeled with evidence type to maintain analytical transparency.
Research method: Evidence sources include platform landing pages, press releases, public API documentation, observed UX behavior (as of Feb 2026), investor announcements, published user reviews, and publicly documented policy frameworks. Claims not directly verifiable from these sources are explicitly labeled [Inference] or [Hypothesis] throughout the document.
Generation is commoditizing. Evaluation is scarce. The question, at every altitude, is the same: can you tell the good from the bad, prove it, and keep doing it as the world drifts?