A system prompt is a set of instructions supplied by the application to shape how an AI assistant should behave. It can define the assistant’s role, priorities, boundaries, response style, and tool rules.
The name comes from chat APIs that label messages by role. A common request contains:
system: You are a support assistant. Use only the supplied product documentation.
user: How do I export my project?
The system instruction is not written by the end user in the normal interface. The application adds it before calling the model.
Where the system prompt fits
A model request can contain several instruction sources:
- Platform instructions added by the model provider.
- System or developer instructions added by the application.
- User instructions describing the current task.
- Tool results and retrieved documents that provide information but should not normally override instructions.
The exact role names and precedence rules differ between APIs. In general, higher-level instructions should win when lower-level content conflicts with them.
This does not make the system prompt a secure policy engine. Models can misunderstand instructions, and untrusted documents can contain prompt injection. The application must still enforce permissions and safety outside the model.
What belongs in a system prompt
A useful system prompt often includes stable rules such as:
- The assistant’s purpose.
- The audience it serves.
- The sources it may use.
- What it must not do.
- When it should ask a question.
- Required output formats.
- Rules for using tools.
- How to handle uncertainty.
Example:
You are a support assistant for Acme Cloud.
Answer using the supplied documentation.
If the documentation does not contain the answer, say that you do not know.
Never invent account status or billing details.
Before any action that changes customer data, ask for confirmation.
Use short paragraphs and numbered steps.
This is more testable than a vague instruction such as “Be helpful and accurate.”
What belongs elsewhere
Do not turn the system prompt into a storage bin for every piece of application data.
Keep these concerns separate:
- Put the current task in the user message.
- Put retrieved evidence in clearly labeled context sections.
- Put tool schemas in the API’s tool definition field when one exists.
- Keep durable workflow state in application data.
- Enforce permissions in code.
- Validate structured output with a schema.
This separation is part of context engineering. It makes each piece easier to update, test, and observe.
Write instructions the model can apply
Strong system instructions are:
- Specific: say what behavior is required.
- Prioritized: make conflicts easier to resolve.
- Bounded: define what the assistant should not assume or do.
- Operational: describe actions, not personality slogans.
- Consistent: avoid rules that contradict each other.
- Testable: make it possible to write examples that pass or fail.
Instead of:
Always provide a perfect answer.
Use:
Use the supplied sources. If evidence is missing or conflicting, explain the uncertainty and ask for the missing information.
A system prompt is not a database
Large reference documents, policy manuals, and changing product data usually do not belong directly in a permanent system prompt. They consume tokens on every request and become hard to keep current.
Use retrieval or tools to load the relevant information when needed. Keep the system prompt focused on how the assistant should use that information.
Long stable prefixes may benefit from prompt caching, but caching does not make irrelevant context useful.
A system prompt is not a security boundary
Suppose a system prompt says:
Never delete customer records without approval.
That rule is useful guidance, but the delete tool should also require real authorization and approval in the application. If the model ignores or misreads the instruction, the tool boundary must still prevent the action.
For agents, harness engineering provides these controls around the model. The system prompt explains expected behavior; the harness enforces what is possible.
Test the whole request
A system prompt can work well with one user message and fail with another. Test it with:
- Normal requests.
- Ambiguous requests.
- Requests that conflict with policy.
- Missing information.
- Malicious text inside retrieved documents.
- Tool errors.
- Requests for unsupported output formats.
Prompt evaluation should check behavior, not only wording. A shorter prompt that produces consistent results is often better than a long prompt full of repeated warnings.
Version system prompts like code. Record which version produced each trace so regressions can be reproduced.
Common mistakes
- Writing a long persona but leaving task rules vague.
- Repeating the same instruction in several conflicting ways.
- Including secrets in the prompt.
- Trusting the prompt to enforce permissions.
- Mixing user data into a shared cached prefix.
- Assuming the model will always follow instruction precedence correctly.
- Changing the prompt without replaying representative evaluations.
The key idea
A system prompt defines stable behavior for an AI assistant. Keep it clear, bounded, and testable, then use application code to enforce the permissions and guarantees that cannot depend on model judgment.