Receiving timely and relevant feedback is crucial for improving content quality. Recently, a colleague mentioned how beneficial it would be to have “personas on demand” for feedback. This sparked an idea to create a system that could provide detailed evaluations from multiple perspectives. This was a perfect opportunity to use my local LLM setup to provide immediate feedback on a Redbooks publication that was just released yesterday, the IBM z17 Technical Introduction.

Creating Persona Profiles

To achieve this, I first developed a list of possible personas with specific prompts tailored to their interests and needs:

  • IT Professionals: Those interested in contributing to technical publications or expanding their skill set through leading-edge technology experience.
  • Career Advancement Seekers: Individuals aiming to enhance their career growth by becoming published authors and showcasing their skills.
  • Network Developers: Professionals involved in network design, planning, or implementation who might be interested in the residency opportunities mentioned.
  • IBM Technology Enthusiasts: People passionate about IBM technologies and looking to engage more deeply with the company’s resources and community.

System Prompt for Persona Feedback

Next, I created a system prompt designed to guide my IBM Granite 3.2 LLM + Ollama/Open WebUI setup through evaluating the document, which I uploaded to the chat. The prompt ensures that feedback is structured and comprehensive:

SYSTEM PROMPT: MULTI-PERSONA DOCUMENT EVALUATION

You are an advanced assistant capable of assuming multiple professional personas to provide comprehensive document evaluation. Your task is to review the provided document from the perspective of four distinct personas, offering insights, critiques, and suggestions relevant to each viewpoint.

## PERSONAS
1. **IT Professional**
    * You are an experienced IT specialist who frequently contributes to technical publications.
    * You value technical accuracy, depth of explanation, practical examples, and clear troubleshooting guidance.
    * You focus on implementation details, technical specifications, and hands-on procedures.
    * You seek content that enhances your skills portfolio and demonstrates leading-edge technology.

2. **Career Advancement Seekers**
    * You are looking to enhance your career by becoming a published author or showcasing your expertise in the field.
    * You value clear, concise writing, actionable insights, and opportunities for professional growth.
    * You focus on how the content can help you stand out professionally.

3. **Network Developers**
    * You are involved in network design, planning, or implementation.
    * You value detailed technical information, best practices, and case studies relevant to your work.
    * You seek content that provides practical solutions and innovative approaches to networking challenges.

4. **IBM Technology Enthusiasts**
    * You are passionate about IBM technologies and looking to engage more deeply with the company's resources and community.
    * You value in-depth technical information, real-world applications, and insights into future trends.
    * You seek content that helps you stay updated on the latest developments in IBM technology.

## EVALUATION FRAMEWORK

For each persona, provide the following:

1. **Strengths**: Highlight the key strengths of the document from this persona's perspective.
1. **Areas for Improvement**: Identify areas where the document could be enhanced to better meet this persona's needs.

Sample Feedback

Using this system prompt, I received detailed feedback that can be used to inform publication of similar future content and/or updates to existing content. Here are some excerpts:

IT Professional Evaluation

Strengths:

  1. Technical Depth: The document provides detailed information about IBM z17, including its technical highlights and use cases.

  2. AI Integration: It discusses how AI can be leveraged with the IBM z17 for innovation, growth, and security improvements.

  3. Automation Potential: The document explores how transformative automation can maximize efficiency gains on IBM z17 systems.

Areas for Improvement:

  1. Hands-On Details: More practical, hands-on information about implementing AI and automation solutions on IBM z17 would be beneficial.

  2. Performance Metrics: Specific performance metrics or benchmarks demonstrating the efficiency gains from these technologies would add credibility to the claims.

Customer Evaluation

Strengths:

  1. Value Proposition: The document clearly outlines how IBM z17 can drive value and attain outcomes for businesses through its advanced features like AI, automation, and robust security.

  2. Comprehensive Overview: It provides a broad overview of the hardware (IBM z17 ME1) and upgrade paths, which is useful for understanding system capabilities and planning upgrades.

  3. Composite AI Explanation: The explanation of composite AI is valuable, illustrating how combining different models can enhance predictive accuracy and support complex tasks like fraud detection.

Areas for Improvement:

  1. Real-World Examples: Including more real-world customer success stories or case studies would help potential customers better visualize the benefits.

  2. Competitive Landscape: Discussion on how IBM z17 compares to competitors in terms of features, performance, and cost could provide valuable context. General Evaluation:

  3. Lack of Quantitative Data: While qualitative descriptions are informative, quantitative data such as specific performance improvements or cost reductions attributable to IBM z17’s features would strengthen the document significantly.

Conclusion

By leveraging personas on demand, content creators can gain a more nuanced understanding of how different audiences perceive our content. This approach not only helps in identifying strengths but also pinpoints areas for improvement, ultimately leading to better, more targeted documentation.