AI is reshaping how SAP consultants work. But the terminology can feel like a foreign language — even if you've been using SAP for 20 years. This module cuts through the jargon with plain explanations and real SAP examples.

What is a Large Language Model (LLM)?

A Large Language Model (LLM) is the AI system that understands and generates text. When you type a question into the AI Assistant on this platform, an LLM reads it, thinks about it, and writes a response.

Examples of LLMs you may have heard of: Claude (Anthropic), GPT-4 (OpenAI), Gemini (Google). SAP's Joule is built on top of LLMs — it's a product layer that sits on an LLM and connects it to SAP data and processes.

Prompts and System Prompts

A prompt is simply the input you give to an AI — your question or instruction. When you type "Explain document splitting in SAP FI" into the AI Assistant, that's your prompt.

A system prompt is a hidden set of instructions given to the AI before the conversation starts. It defines how the AI should behave — its role, its constraints, its tone. For example, the AI Assistant on this platform has a system prompt that says something like: "You are an expert SAP trainer. Answer only SAP-related questions. Be concise and practical."

SAP example: When Joule answers your question, SAP's engineers have written a system prompt that tells Joule to be an SAP assistant, to reference SAP Help documentation, and to refuse to answer off-topic questions.

Tokens — the AI's unit of measurement

Tokens are the chunks that LLMs use to read and write text. A token is roughly 3–4 characters or about ¾ of a word. The phrase "post a journal entry" is about 5 tokens.

Why does this matter? Because LLMs have a maximum number of tokens they can process in one conversation — this is called the context window (see below). It also affects cost — most AI APIs charge by the number of tokens processed.

SAP example: If you paste a 10,000-line ABAP dump into an AI Assistant, you may hit the token limit. The AI can't read the entire dump, so it will either truncate or refuse.

Context Window

The context window is the total amount of text an LLM can "hold in mind" at once — your entire conversation history, any documents you've shared, and the system prompt all count against this limit.

Think of it like working memory. Once you exceed the limit, earlier parts of the conversation get dropped — the AI "forgets" what you said at the start.

Modern context windows can be very large (100,000–200,000 tokens in some models). But they're not unlimited. For SAP consultants, this matters when:

• Debugging long ABAP programs
• Pasting large configuration tables or transport logs
• Having very long troubleshooting conversations