A system prompt is a set of standing instructions, supplied outside the normal back-and-forth of a conversation, that shapes how an AI model behaves for the whole session: its role, its tone, its rules and the context it should rely on. To write one that works, give the model a specific role, state each rule together with the reason for it, describe the output you want in positive terms, and test the prompt against awkward inputs before trusting it. Most weak system prompts fail not because they are too short, but because they list prohibitions without explaining them.
This guide covers what a system prompt is and how each major API handles it, then builds a complete system prompt section by section, with a template you can copy and a testing routine for keeping behavior consistent over time.
What a system prompt is and why it matters more than any single message
A system prompt is the instruction layer an application developer sets before the user says anything. A user message affects one turn; the system prompt applies to every turn. That is why a vague system prompt does more damage than a vague user message: its weaknesses show up in every answer the application gives.
Typical things that belong in a system prompt are standing facts and rules: “You are a support agent for a project-management SaaS product”, “Reply in the language the user writes in”, “Never quote a price that is not in the pricing table below”. Things that change per request, such as the document to summarize or the user’s question, belong in the user turn.
How the major APIs name and prioritize it
Each provider exposes the same idea with slightly different names. The details below come from each provider’s own documentation.
| Provider | Where the system prompt goes | What the docs say about it |
|---|---|---|
| Anthropic (Claude) | The top-level system parameter of the Messages API | A role in the system prompt focuses behavior and tone; even a single sentence makes a difference |
| OpenAI | The instructions parameter, or messages with the developer role | Developer instructions take priority over user messages; instructions applies only to the current request |
| Google (Gemini) | The system_instruction parameter | System instructions guide the model’s behavior across the request |
One practical detail from OpenAI’s prompt engineering guide: when you chain requests with previous_response_id, the instructions from earlier turns are not carried over, so you must send them again each time. It is worth checking how your own stack handles this, because a system prompt that silently drops after the first turn looks exactly like a model that “forgets” its rules.
What a system prompt can and cannot guarantee
A system prompt makes behavior more consistent; it does not make it certain. Models weigh the system prompt heavily, but a long conversation, conflicting instructions or a hostile user can still push them off course. Anything that must never happen, such as leaking another customer’s data, needs enforcement in code as well: permissions, output validation, filters. The guide to prompt injection and how to defend against it explains why prompt-level rules alone are not a security boundary.
Building a system prompt, section by section
A reliable system prompt usually has five parts: role, context, rules with reasons, output format, and examples. OpenAI’s guide suggests a similar order (identity, instructions, examples, context) while noting the best arrangement can vary by model. The running example below is an illustrative support assistant for a fictional invoicing app called Ledgerly.
1. Role: one sentence, specific to the real task
Start with a sentence that names who the model is and what it is for. Anthropic’s prompting best practices use “You are a helpful coding assistant specializing in Python” as an example of how little it takes to focus tone and vocabulary. The role should match the job. “You are a helpful assistant” adds almost nothing. “You are a tax attorney” for a bot that answers billing questions is too narrow in the wrong direction and may make the model hedge or refuse ordinary requests. The deeper treatment is in role prompting.
You are the in-app support assistant for Ledgerly, an invoicing tool
for freelancers and small agencies. You help users create, send and
track invoices and fix problems with their account.2. Context: who the users are and what the model can rely on
Context is the background a new employee would need on day one. Describe the audience, the situation they are usually in, and the reference material the model should treat as true. Put large reference material (help articles, a product catalog) in its own clearly labelled section rather than mixing it into the rules. Anthropic’s guidance recommends XML-style tags such as <context> and <instructions> to keep different kinds of content unambiguous, and placing long documents above the instructions and the query.
3. Rules, each with its reason
This is where most system prompts go wrong. A rule without a reason forces the model to guess at the edges. A rule with a reason lets it generalize. Anthropic’s well-known example contrasts “NEVER use ellipses” with an explanation that the output will be read by a text-to-speech engine that cannot pronounce them; with the reason, the model can also avoid other unpronounceable symbols without being told.
Here is the difference in the Ledgerly example:
| Bare rule | Rule with reason |
|---|---|
| Never give tax advice. | Do not tell users how to treat an invoice for tax purposes. You are not a licensed advisor, rules differ by country, and a wrong answer could cost the user money. Suggest they check with their accountant. |
| Don’t discuss refunds. | Refund decisions are made by the billing team, not by you. If a user asks for one, collect the invoice number and the reason, and tell them billing will reply by email. |
| Be concise. | Users usually open this chat mid-task. Lead with the fix in one or two sentences, then add detail only if needed. |
Notice that every rewritten rule also tells the model what to do instead. That is the second habit worth building: state behavior positively wherever you can. “Write in flowing paragraphs” gives a target; “don’t use bullet points” only removes one option. The guide to negative prompting covers when a “don’t” is still the right call.
Separate hard rules (concrete, checkable, non-negotiable) from style guidance. Grouping them under their own heading makes them easier for the model to find and easier for your team to review. It also helps with a known failure: an important instruction buried in the middle of a long prompt can receive less attention, which is examined in why Claude can skip an instruction buried in a long system prompt.
4. Output format and action defaults
Say what a finished answer looks like: length, structure, whether markdown will render in your interface, and what language to reply in. If the model can take actions, say whether it should act or suggest. Anthropic’s guidance notes that a request phrased as “can you suggest some changes” may get suggestions, while “change this function” gets the change; set the default you want explicitly in the system prompt instead of hoping the user phrases things well.
5. Examples, used sparingly
One or two short examples of a good exchange often communicate tone better than adjectives. Wrap them in tags such as <example> so they are not mistaken for live conversation, and vary them, because models tend to copy examples closely. Examples that contradict your rules will usually win over the rules, so check them carefully.
A complete, copy-ready template
The template below assembles the five parts for the illustrative Ledgerly assistant. Replace the content, keep the structure.
<role>
You are the in-app support assistant for Ledgerly, an invoicing tool
for freelancers and small agencies. You help users create, send and
track invoices and fix problems with their account.
</role>
<context>
Users are usually in the middle of billing a client and want a quick
fix. Many are not native English speakers. The help articles in
<knowledge_base> are accurate and current; rely on them over your
general knowledge of invoicing software.
</context>
<hard_rules>
- Do not advise on tax treatment. You are not a licensed advisor,
rules vary by country, and errors cost users money. Suggest they
ask their accountant.
- Do not promise refunds or credits. Billing decides these. Collect
the invoice number and reason, and say billing will reply by email.
- Describe only features in <knowledge_base>. If something isn't
there, say Ledgerly doesn't support it today and offer to log a
feature request.
</hard_rules>
<style>
- Reply in the language the user writes in.
- Lead with the fix in one or two sentences, then add detail if needed.
- Use numbered steps for procedures of three or more steps; otherwise
write short paragraphs.
- Plain text only: the chat widget does not render markdown.
</style>
<example>
User: my invoice says overdue but the client paid yesterday
Assistant: You can mark it as paid manually: open the invoice, choose
"Record payment", and enter the date and amount. If the client paid
through Ledgerly's payment link, the status can take a short time to
update, so check again before recording it twice.
</example>
<knowledge_base>
...help articles...
</knowledge_base>Why it is built this way: the tags let the model, and your colleagues, find each kind of content instantly. Every hard rule carries its reason and a fallback action. The style section includes the reason for plain text (the widget), so the model avoids any formatting the widget would display as raw symbols. And the knowledge base is explicitly declared the source of truth, which reduces invented features. In production, you may prefer to place a very long knowledge base before the rules, following Anthropic’s long-context advice; test both positions.
Testing a system prompt before you trust it
A system prompt is only as good as its behavior on inputs you did not plan for. Typical questions will almost always look fine; the problems show up at the edges. Build a small test set with at least these categories:
- Typical requests to confirm the basic job works.
- Adjacent requests that are legitimate but slightly outside the core task, to check the role is not so narrow that the model refuses them.
- Rule boundaries, such as a question that is half product support and half tax question.
- Ambiguous inputs where two reasonable readings lead to different answers.
- Adversarial inputs, including requests to ignore the rules or reveal the prompt.
- Long conversations, to see whether rules still hold after many turns.
Judge the results against the intent behind each rule, not its literal wording. When the model’s behavior varies across valid readings of an instruction, the instruction is underspecified: add the reason, an example, or both. Rerun the full set after every change, because fixing one behavior often shifts another. A tool such as promptfoo makes this repeatable; see building a regression test suite for Claude prompts with promptfoo.
Keeping behavior consistent over time
Consistency problems usually come from changes around the prompt rather than the prompt itself. Four habits prevent most of them.
- Version the prompt like code. Keep it in source control, record why each change was made, and tie each deployed version to its test results.
- Re-test when you switch models. The structure (role, context, rules, examples) transfers well between providers, but the phrasing that works best can differ. Anthropic notes, for instance, that recent Claude models respond strongly to the system prompt, so wording like “CRITICAL: you MUST” written for older models can cause over-triggering and should be softened.
- Watch long sessions. In very long conversations, a large volume of recent text competes with the original instructions. Restating a key rule near where it matters, or summarizing and starting a fresh session, keeps behavior steady.
- Resolve conflicts explicitly. If one rule says “be thorough” and another says “keep it short”, the model will alternate. Decide which wins in which situation and write that down.
How long should a system prompt be?
A system prompt should be as long as it takes to state the role, the non-negotiable rules with their reasons, the output format and the necessary context, and no longer. A simple internal tool may need five lines. A customer-facing assistant with policies and a knowledge base can run to thousands of tokens, most of it reference material. Length becomes a problem only when it is padding, repetition or contradictory instructions; if the prompt is long because it carries a large knowledge base, prompt caching can reduce the cost of resending it on every request, depending on your provider.
