Here is a summary of our main points:

We defined this new paradigm as "Agent Experience" (AX), where the user’s role evolves from an active operator to a supervisor. Unlike simple assistants like Siri, true AI agents can independently manage complex, multi-step goals. This means users will no longer navigate intricate workflows but will instead state their objectives and oversee the AI's execution.

Our article posits that designers must now build "cockpits" or dashboards for monitoring and intervention, rather than step-by-step task flows. In this new world, natural language becomes the primary interface, and the core design challenge is to establish trust by ensuring the user remains the ultimate authority with clear pathways to step in and manage their AI counterparts. We concluded that the companies that succeed will be those that best empower users to supervise these increasingly autonomous systems confidently.

Read the Full Article Here

Then came the Generative AI explosion. Suddenly, the old playbook feels inadequate. We're no longer just predicting a "churn" vs. "no churn" label. We’re generating nuanced text for marketing, complex code for features, and intricate product designs. The very definition of a "good" output has become subjective and context-dependent.

This paradigm shift is forcing us to rethink evaluation from the ground up. For those on the front lines building these products, this isn't just an academic exercise; it's a critical bottleneck to shipping reliable software.

The Two Worlds of GenAI Evaluation: Closed and Open-Ended

The first step to building a coherent strategy around GenAI is to distinguish between the two fundamental types of tasks your system might be performing.

Closed-Ended Predictions:

This is familiar territory. The model's job is to produce a specific, constrained output. Because there's a definite "right" answer, we can lean on our traditional toolkit.

Example: You're building a feature to automatically categorize incoming support tickets ("Billing Issue", "Technical Glitch", "Feature Request").

Why it's closed-ended: There's a predefined, finite set of correct labels.

How you measure it: You can use precision (of all the tickets we labeled "Billing Issue," how many actually were?) and recall (of all the actual "Billing Issue" tickets, how many did we find?). These are clear, quantifiable KPIs you can build dashboards around and track release over release.

Open-Ended Predictions:

This is where the real challenge begins. Think of tasks where there are many possible "good" answers.

Example: You're launching an AI-powered feature that summarizes long customer feedback emails for internal teams.

Why it's open-ended: A 500-word email can be summarized effectively in dozens of different ways. Which one is best? It depends on what the reader needs to know.

The problem: Simple one-to-one comparisons with a "golden" summary in a test set are no longer sufficient. This is the core challenge for teams building the next generation of AI-powered features.

The Evaluation Toolkit for Open-Ended Generation

While the problem is complex, we're not flying completely blind. The field has developed several methods to bring structure to this ambiguity.

The Classics: BLEU and ROUGE

These metrics are your first-line, automated tools. They compare the words and phrases in the model's output to one or more human-written reference examples.

Let's take our feedback summarizer feature. Suppose the original feedback is: "The new dashboard is visually appealing, but the process to export reports is now much slower and requires three extra clicks. I also can't find the date filter easily."

Reference Summary: "User likes the new dashboard's look but finds report exporting slower and the date

The Modern Approaches: Semantic Similarity and LLM-as-a-Judge

To get closer to measuring actual quality, we need more sophisticated tools.

1. Semantic Evaluation: This measures if the meaning is the same, even if the words are different. In the example above, a good semantic similarity metric would score "Generated Summary B" highly against the reference because the core concepts are identical. This is a much better signal of true understanding.

2. LLM-as-a-Judge: This is a game-changer for iterating quickly. You use a powerful LLM as a tireless, automated evaluator. As a team, you define the rubric.

Example: To evaluate the summaries from your AI feature, you can use an API call to a "Judge" LLM with a prompt like this:

You are an expert editor. Please evaluate the following AI-generated summary based on the original customer feedback.

Original Feedback: [Insert original email here]

AI-Generated Summary: [Insert summary to be evaluated here]

Please score the summary on a scale of 1-5 for the following criteria:

1. Conciseness: Is the summary brief and to the point?

2. Factual Accuracy: Does the summary correctly represent all key points from the original feedback?

3. Clarity: Is the summary unambiguous and easy to understand?

Provide a score and a brief justification for each.

This approach is faster and cheaper than constant human review and allows you to scale your evaluation based on criteria that your team defines as important for the product.

Special Case: Evaluating RAG Systems

Retrieval-Augmented Generation (RAG) is the architecture powering most modern enterprise chatbots. It has two parts: finding the right info (Retrieval) and then using it to talk (Generation). To debug effectively, you must evaluate them separately.

• Example: A customer support chatbot that answers questions based on a company's knowledge base. A user asks, "How do I get a refund for my subscription?"

• 1. Evaluate the Retrieval: Did the system pull the correct document?

  • Success: The RAG system retrieves the "Refund Policy" article.

  • Failure: The system retrieves the "Subscription Upgrade" article.

  • The Metric: Use classic search metrics like NDCG or simple top-k hit rate (e.g., "was the correct document in the top 3 results 95% of the time?"). This tells you if your retrieval component—be it a vector database or a search API—is effective.

• 2. Evaluate the Generation: Assuming it found the right document, did it generate a good answer?

  • Success: "To get a refund, go to your Account Settings, click 'Subscription,' and follow the 'Cancel and Refund' link. You are eligible for a refund if you cancel within 14 days of purchase."

  • Failure: "The document mentions refunds." (Accurate but useless).

  • The Metric: Here you use the open-ended techniques: LLM-as-a-Judge to check for helpfulness, or human spot-checks.

By separating these, you can pinpoint the real problem. Is the chatbot failing because the knowledge base is bad (a content/retrieval problem) or because it's bad at explaining things (a generation problem)? This distinction is critical for assigning bugs and planning sprints.

It All Comes Down to Data and a Hybrid Approach

The uncomfortable truth for any team building with AI is that the biggest challenge isn't the metrics; it's the test data. Creating a high-quality, diverse evaluation set that reflects real-world usage is an expensive, ongoing engineering and product effort. I’ve seen many teams deprioritize this, only to build evaluation systems that show green lights while the actual user experience stagnates.

The most successful GenAI companies, like OpenAI and Anthropic, have a core competency in data curation for evaluation. This is not a coincidence.

An Action Plan for Teams:

1. Treat the Test Set as a Product: It needs to be versioned, maintained, and expanded alongside your main product. This is a shared responsibility: product managers define the key use cases to cover, and engineers build the infrastructure to test against them reliably.

2. Build a Hybrid Evaluation Dashboard: Don't rely on one number. Your team's dashboard should provide a complete picture:

   • Automated Metrics (ROUGE/Semantic Similarity): For a quick, directional pulse in your CI/CD pipeline.

   • LLM-as-a-Judge Scores: For scalable, criteria-based quality checks on nuanced attributes like "helpfulness" or "brand tone."

   • Human Feedback: A direct pipeline from a "thumbs up/down" button in the UI or periodic human reviews on critical flows.

3. Define "Quality" Holistically: Your team's definition of done must go beyond factual correctness. Your evaluation rubric should include:

   • Safety: Does it avoid harmful or inappropriate language?

   • Tone of Voice: Does it sound like your brand?

   • User Satisfaction: Is the user actually achieving their goal?

4. Deconstruct Your Systems: For multi-step systems like RAG, insist on component-level metrics. When a metric drops, you need to know which part of the system broke to debug efficiently.

Building in the GenAI era requires a new level of rigor in how we define and measure quality. Moving past simplistic scores to a holistic evaluation framework is no longer optional—it's the core work of building AI products that are not just impressive demos, but are also reliable, safe, and genuinely useful.

AI Is Becoming More Intuitive

The early days of LLMs often required users to experiment with different phrasings to get the best results. However, modern AI models are trained on vast amounts of data and improved architectures that enable them to interpret instructions more naturally. Instead of needing a carefully structured prompt, today’s models can process vague, casual, or even slightly ambiguous commands with ease.

For example, early models required precise formatting, explicit step-by-step breakdowns, and structured wording. Now, newer models can infer context, understand implied meaning, and generate useful outputs without the need for complex prompt tuning. This means that instead of focusing on how to "trick" the AI into giving the best answer, users can simply ask questions as they would to a knowledgeable human.

The Overhyped Industry of "Prompt Engineering"

As with any emerging technology, a subset of self-proclaimed experts have positioned themselves as gatekeepers, offering courses, guides, and consulting services on how to craft the perfect prompt. While some strategies may have been helpful in the past, the need for such expertise is rapidly fading.

Most prompt engineering advice boils down to common-sense practices like being clear, specifying output format, or providing context—all things that even casual users can figure out intuitively. The AI itself is improving at handling ambiguity, reducing the necessity for highly refined prompts. As a result, the idea that businesses need dedicated "prompt specialists" is becoming increasingly outdated.

The Future: Conversational AI, Not Manual Tweaking

Instead of relying on highly specific prompts, the future of LLMs is in their ability to engage in dynamic, natural conversations. AI systems are evolving to ask clarifying questions, refine their own outputs, and adapt based on user feedback. This means that rather than needing a human to master a rigid prompting technique, AI itself will adjust based on user intent.

Think about how we interact with human assistants: we don’t script perfect instructions in advance; we communicate, clarify, and refine our requests in real time. That’s exactly where AI is heading. The need to manually craft prompts will soon be seen as an unnecessary relic of early AI experimentation.

1. Define SLIs and SLOs Beyond Model Accuracy

Traditional ML metrics like accuracy or F1 scores are insufficient for production systems. SRE emphasizes Service Level Indicators (SLIs) and Service Level Objectives (SLOs) to quantify reliability. For ML systems, this includes:

• Latency: Response time for model inference.

• Availability: Uptime of ML APIs or services .

• Data Drift: Monitoring input distribution shifts that degrade model performance.
  By setting SLOs for these metrics, teams can prioritize reliability alongside accuracy.

2. Automate Deployment and Monitoring

SRE reduces manual toil through automation, a practice critical for ML workflows:

• CI/CD Pipelines: Automate model deployment with rollback capabilities to handle faulty updates.

• Self-Healing Systems: Use ML to detect anomalies (e.g., data pipeline failures) and trigger remediation.

• Testing: Integrate automated canary testing to validate model performance in staging before full rollout.
  Automation minimizes human error and accelerates iteration.

3. Prioritize Observability for Silent Failures

ML systems often fail silently (e.g., gradual accuracy decay). SRE-inspired observability includes:

• Model Metrics: Track precision/recall over time and correlate with infrastructure health.

• Data Lineage: Monitor data pipelines to catch preprocessing errors or missing features.

• Root Cause Analysis: Use tools like tracing to link model failures to specific code or data changes.
  Comprehensive observability helps detect issues before users are impacted.

4. Formalize Incident Response for Model Failures

Treat model failures like system outages using SRE incident management practices:

• Runbooks: Document steps to diagnose and resolve common issues (e.g., data drift).

• Blameless Postmortems: Analyze failures to improve processes rather than assign blame.

• Escalation Paths: Define roles for triaging severe incidents (e.g., automated rollbacks vs. human intervention).
  Proactive incident management reduces downtime and builds trust.

5. Design for Resilience

SRE emphasizes building systems that withstand failures. ML engineers should:

• Implement Fallbacks: Deploy simpler models (e.g., rule-based systems) as backups during outages.

• Redundancy: Replicate data pipelines and model servers to avoid single points of failure.

• Chaos Engineering: Test system resilience by intentionally injecting failures (e.g., synthetic data corruption).
  Resilient design ensures graceful degradation under stress.

Conclusion

Adopting SRE principles bridges the gap between experimental ML and production-grade systems. By focusing on reliability metrics, automation, observability, and resilience, ML engineers can create solutions that are not just innovative but also dependable at scale. As ML systems grow in complexity, the SRE mindset—proactive, data-driven, and iterative—will be indispensable for maintaining performance and user trust .

Key Provisions and Banned AI Applications

As of today, certain AI applications are outright banned to protect fundamental rights, privacy, and societal well-being. These include:

❌ Social Scoring Based on Personal Behavior: AI systems that rank individuals based on their social conduct, similar to systems used in some authoritarian regimes.

❌ Manipulative or Deceptive AI: Technologies designed to influence user decisions through covert manipulation or exploitation of psychological vulnerabilities.

❌ AI Exploiting Vulnerabilities: Systems targeting individuals based on specific vulnerabilities related to age, disability, or socioeconomic status.

❌ Crime Prediction Based on Appearance: AI models making predictions about criminal behavior based solely on physical traits.

❌ Biometric AI Inferring Personal Characteristics: Technologies that deduce sensitive personal information, such as sexual orientation, from biometric data.

❌ Real-Time Biometric Surveillance in Public Spaces: The use of AI for continuous biometric monitoring in public areas without stringent legal oversight.

❌ Emotion Recognition at Work or School: AI tools aimed at analyzing emotions in professional or educational environments, which can lead to invasive surveillance.

❌ Facial Recognition Databases from Online Scraping: Databases created by harvesting facial images from the internet without explicit consent.

What This Means for Businesses

Organizations operating within the EU or offering AI products and services in the region must ensure compliance with these regulations. Non-compliance can result in hefty fines and reputational damage. Businesses should:

• Conduct thorough audits of their AI systems.

• Eliminate or modify prohibited functionalities.

• Implement robust governance frameworks to oversee AI ethics and compliance.

Looking Ahead

The EU AI Act represents a significant shift towards ethical AI development and responsible deployment. As additional compliance deadlines approach, businesses must stay proactive, continuously adapting to meet evolving regulatory requirements. This new era of AI governance is not just about legal compliance—it’s about fostering trust, accountability, and fairness in technology.

Enter Deep Reinforcement Learning (DRL)

Unlike traditional models that optimize for short-term gains, such as achieving an immediate click, Deep Reinforcement Learning (DRL) takes a broader perspective. It focuses on maximizing user lifetime value (LTV), a metric that captures the long-term financial contribution of a user.

🔍 What Does This Mean in Practice?

🔥 1. Beyond the First Click

DRL models evaluate how each ad impression influences not just initial clicks but also downstream actions like sign-ups, purchases, repeat visits, and brand loyalty. It shifts the focus from “Did they click?” to “Did that click lead to meaningful engagement?” By analyzing long-term user behavior, DRL ensures that ads are optimized to attract users who are more likely to convert into loyal customers.

♻️ 2. Handling Delayed Rewards

In traditional models, the value of an ad is often judged immediately after the click. DRL, however, excels in environments where rewards are delayed. Even if a user doesn’t convert right away, DRL algorithms, such as Q-learning with discount factors, track the impact of that interaction over time. This approach allows advertisers to understand the cumulative value of each ad impression, accounting for both immediate responses and future actions.

🎯 3. Smarter Budget Allocation

When advertisers chase clicks, they risk overspending on ads that generate high CTR but low conversion rates. DRL changes the game by enabling smarter budget allocation. It helps advertisers identify and invest in strategies that nurture long-term customer relationships. This leads to maximized ROI, as funds are directed toward campaigns that drive sustainable growth rather than fleeting engagement.

The Bottom Line

While CTR can provide quick insights, it doesn’t capture the full story. Optimizing for long-term value through DRL allows advertisers to build deeper connections with their audience, enhance brand loyalty, and achieve greater financial returns. In the evolving landscape of AdTech, focusing on LTV isn’t just a smarter strategy—it’s the future of digital advertising.

Data size:

• For data sizes under 100K, a brute-force solution utilizing a FLAT index is sufficiently efficient.

• Advanced algorithms may not offer significant speed improvements in such scenarios.

Speed-Recall trade-off:

• When an exact match is important then brute force is the right solution

• Significant query latency reduction is achievable with a minor sacrifice in recall.

Memory limitation:

• Some algorithms like HNSW are memory-hungry.

• Scalar and Product Quantization significantly reduces storage consumption, at the expense of the Recall.

CPU vs GPU:

• Usually moving from CPU to GPU provides a speed boost

• Not all algorithms are optimized for GPU

Building/Indexing time:

• Sometimes the building/indexing time is crucial.

• IVF has a shorter indexing time compared to HNSW.

Garak:
It’s a LLM vulnerability scanner (nmap for LLMs). It supports:
- probes for hallucination
- data leakage
- prompt injection
- misinformation
- toxicity generation
- jailbreaks

Promptmap was developed by my colleague Utku Sen. It is a tool that automatically tests prompt injection attacks and supports the following attack types:
- Basic Injection
- Translation Injection
- Math Injection
- Context-Switch
- External Browsing