Guideline-Based AI Response Evaluation
Evaluation Type: AI Response Quality Review
Skills Demonstrated: Guideline Interpretation · Instruction Following · Accuracy Review · Relevance Assessment · Completeness · Quality Scoring · Written Rationale
Evaluation Scenario
An AI assistant is asked to answer a user question. The evaluator must review the response against the provided guidelines and determine whether it satisfies the user's request.
User Prompt
What are three simple ways a small business can improve customer retention without offering discounts? Keep the answer under 120 words.
Evaluation Guidelines
The response should:
-
Provide exactly three recommendations.
-
Focus specifically on customer retention.
-
Avoid recommending discounts, coupons, or price reductions.
-
Give practical recommendations appropriate for a small business.
-
Remain under 120 words.
-
Be clear, relevant, and easy to understand.
-
Avoid unnecessary or unrelated information.
AI Response
Small businesses can improve customer retention in several ways.
-
Personalize customer communication. Use customer names, remember preferences, and send relevant follow-up messages after purchases.
-
Create a loyalty discount program. Offering returning customers 10% off future purchases can encourage them to continue buying from the business.
-
Provide responsive customer service. Answer questions promptly, resolve problems professionally, and follow up to make sure customers are satisfied.
Businesses can also post regularly on social media, redesign their websites, attend networking events, and experiment with advertising to attract more customers.
Evaluation
Overall Rating
Partially Meets Guidelines
Guideline Review
1. Provides exactly three recommendations — PASS
The response includes three numbered recommendations.
2. Focuses on customer retention — PARTIAL PASS
Personalized communication and responsive customer service directly support customer retention. However, the final paragraph discusses social media, advertising, networking, and attracting new customers, which shifts away from the requested retention focus.
3. Avoids discounts or price reductions — FAIL
The second recommendation suggests a 10% loyalty discount, directly violating the instruction not to recommend discounts.
4. Provides practical advice for a small business — PASS
The suggestions involving personalized communication and responsive customer service are realistic and actionable for small businesses.
5. Remains under 120 words — FAIL
The response exceeds the requested word limit.
6. Clear and easy to understand — PASS
The response is organized, readable, and uses straightforward language.
7. Avoids unnecessary information — FAIL
The final paragraph introduces unrelated customer-acquisition activities that were not requested.
Evaluator Decision
Rating: 2 — Needs Improvement
The response addresses the general topic and includes some useful recommendations, but it fails several explicit instructions. Most importantly, it recommends a discount even though the user specifically prohibited discounts. It also exceeds the requested length and adds information about customer acquisition rather than staying focused on retention.
Because instruction-following is a core requirement, these issues prevent the response from receiving a fully satisfactory rating.
Corrected Response
-
Personalize follow-up communication. Thank customers after purchases, remember preferences, and send relevant updates that make interactions feel personal.
-
Provide consistently responsive service. Answer questions promptly, resolve problems carefully, and follow up after an issue is handled.
-
Give customers a reason to stay connected. Offer useful newsletters, early product information, helpful tips, or occasional customer-only updates.
Strong retention comes from making customers feel recognized, supported, and valued throughout their relationship with the business.
Final Evaluation Summary
Original AI Response: Needs Improvement
Primary Issue: Failure to follow explicit instructions
Secondary Issues: Exceeded length requirement and included irrelevant information
Corrected Response: Meets Guidelines
Portfolio Note
This is an independently created simulated guideline-based AI response evaluation designed to demonstrate rubric application, instruction-following assessment, quality scoring, factual accuracy review, and written evaluator rationale. The prompt, AI response, evaluation criteria, ratings, and analysis were created for portfolio demonstration purposes and do not represent proprietary materials or work completed for a specific employer, client, AI platform, or project.
NEXT SAMPLE: Comparative AI Response Evaluation -->