Do's and Don'ts

Prompt Engineering - Do’s and Don’ts

Effective prompt engineering is key to unlocking the best performance from Large Language Models (LLMs) and ensuring they generate contextually relevant, accurate, and useful responses. This chapter outlines essential do’s and don’ts to guide prompt engineers in crafting effective prompts and achieving desired outcomes.

Do’s of Prompt Engineering

• Do Be Clear and Specific: Clearly articulate the task, desired output format, and any constraints. The more precise your instructions, the better the LLM can understand and fulfill your request.
• Do Provide Sufficient Context: If the task requires background information, domain knowledge, or previous conversational turns, include it in the prompt or ensure the model has access to it.
• Do Use Role Prompting or Personas: Instruct the LLM to adopt a specific role or persona (e.g., “You are an expert biologist,” “Act as a travel guide”) to shape the tone, style, and expertise of its responses.
• Do Break Down Complex Tasks: For multi-step or complex problems, guide the LLM by breaking the task into smaller, manageable steps. Techniques like Chain-of-Thought (CoT) prompting can be very effective here.
• Do Use Examples (Few-Shot Prompting): For nuanced tasks or specific output formats, provide a few examples (input/output pairs) within the prompt to demonstrate what you want.
• Do Provide Structure and Formatting: Use clear delimiters (e.g., ###, ```, XML tags), headings, or bullet points within your prompt to help the LLM understand different parts of your instruction and to request structured output.
• Do Iterate and Refine: Prompt engineering is an iterative process. Start simple, test, analyze the output, and refine your prompt. Keep versions of your prompts and track what works best.
• Do Experiment with Model Parameters: Adjust inference parameters like temperature (for creativity vs. factuality), top_p, max_tokens, etc., in conjunction with your prompts to fine-tune the output.
• Do Consider the Target Audience of the Output: Tailor prompts to generate responses appropriate for the intended end-users’ language, knowledge level, and preferences.
• Do Monitor and Evaluate Performance: Regularly test your prompts against a defined set of criteria or an evaluation suite to ensure they consistently produce high-quality results.

Don’ts of Prompt Engineering

• Don’t Use Ambiguous or Vague Language: Avoid prompts that are open to multiple interpretations, as this can lead to inconsistent or irrelevant responses.
• Don’t Overgeneralize Prompts: If you need a specific answer, don’t ask a very broad question. This can cause the model to provide generic or unhelpful information.
• Don’t Embed Biased Language or Harmful Assumptions: Be mindful of the language used in your prompts. Avoid reinforcing stereotypes or introducing biases that the model might replicate or amplify.
• Don’t Ignore Ethical Considerations: Never overlook the ethical implications of your prompts, including potential for misuse, generation of harmful content, privacy violations, or unfair outcomes. Always design prompts responsibly.
• Don’t Neglect Domain Knowledge if Necessary: While LLMs have vast general knowledge, for specialized tasks, incorporating relevant domain-specific terminology or context into the prompt is crucial.
• Don’t Rely Solely on Automated Metrics for Evaluation: While metrics are useful, always include human evaluation to assess nuances of quality, relevance, and safety that automated tools might miss.
• Don’t Expect Perfection or Mind-Reading: LLMs are powerful but not infallible. They operate based on the patterns in their training data and the instructions you provide. They cannot infer intent beyond what is stated or clearly implied.
• Don’t Disregard Context Window Limitations: Be aware of the model’s maximum context length. Very long prompts or conversation histories can be truncated or lead to degraded performance. Manage context wisely.

Key Prompt Engineering Best Practices Summarized

• Start with a clear objective for your prompt.
• Be explicit in your instructions; don’t assume the LLM knows what you mean.
• Balance conciseness with providing sufficient detail and context.
• Use few-shot examples when precision or a specific format is critical.
• Test prompts thoroughly and iterate based on observed outputs.
• Always consider the ethical implications and potential for harm.
• For multilingual applications, design and test prompts that support diverse languages and cultural contexts appropriately.
• Regularly review and update your prompts, especially as models evolve or new techniques emerge.

Conclusion

Adhering to these do’s and don’ts forms a strong foundation for successful prompt engineering. Understanding task requirements, providing clear and contextually relevant instructions, iterating on prompts, and always keeping ethical considerations in mind are crucial.

By following best practices and incorporating user feedback, prompt engineers can create highly effective prompts that harness the full power of LLMs, leading to desirable outcomes and responsible AI applications.