← All Google Cloud AI guidesGen AI · 1.2, 2.2, 3.2

The Google Cloud gen AI stack: Gemini, Model Garden and Agent Platform

How generative AI is built on Google Cloud in 2026: what Gemini Enterprise Agent Platform contains, choosing models in Model Garden (first-party, partner, open; serverless or self-deployed), prompt and context engineering, function calling and structured output, the adaptation ladder from prompting to tuning and distillation, agents (ADK, Agent Runtime, Memory Bank), and cost, latency and availability levers. Includes a token cost calculator.

1. The Agent Platform gen AI map

Gemini Enterprise Agent Platform (formerly Vertex AI) groups Google Cloud's gen AI pieces. Exam questions name the piece; you pick the one whose job matches.

Piece	Job	Old name
Model Garden	Catalogue of Google, partner and open models; deploy or call them	Same
Gemini API on Agent Platform	Generate content: text, images, audio, video, code; function calling; structured output	Vertex AI Gemini API
Agent Studio	Console playground: prompt design, compare, tune, save prompts	Vertex AI Studio
Agent Development Kit (ADK)	Open-source framework for building agents and multi-agent systems in code	Same
Agent Runtime	Managed runtime to deploy and scale agents, with Sessions, Memory Bank and Code Execution	Agent Engine
Agent Search	Managed enterprise search and RAG over your data stores	Vertex AI Search
RAG Engine	Managed RAG pipeline: ingest, chunk, embed, index, retrieve	Same
Agent Retrieval	Vector similarity search at scale (approximate nearest neighbour), hybrid search	Vector Search
Agent Platform Evals	Evaluate models, prompts and agents (pointwise, pairwise, rubrics, trajectories)	Gen AI evaluation service
Model Armor	Screens prompts and responses for injection, jailbreaks, sensitive data, harmful content, malicious URLs	Same
Tuning	Supervised fine-tuning, preference tuning, distillation	Same

SDKs and CLIs kept their names: google-cloud-aiplatform, the Google Gen AI SDK (google-genai), gcloud ai. Questions may use either the old or new product names.

2. Model Garden: which model, and how to run it

Model Garden is the catalogue of first-party (Gemini, Imagen, Veo, Gemma), partner (for example Anthropic Claude, Meta Llama, Mistral) and open models. Some are offered as models as a service (MaaS): a serverless API you pay per token. Others you deploy to your own endpoint with chosen hardware, paying for the nodes.

Need	Pick
Hardest reasoning, long documents, complex multimodal	Gemini Pro tier
Low latency and cost at high volume	Gemini Flash or Flash-Lite tier
Images / video generation	Imagen / Veo
Open weights, run in your VPC, customize freely	Gemma or another open model deployed from Model Garden to an endpoint or GKE
A specific partner model the team standardized on	That model through MaaS
Text embeddings for search or RAG	Gemini embedding / text-embedding models

Three ways to run a model

How you run it	What it means	Pay for	Choose when
Model as a service (MaaS)	Serverless API managed by Google; nothing to deploy	Tokens (or Provisioned Throughput)	Fastest start, spiky traffic, no infrastructure team
Self-deploy to an Agent Platform endpoint	One-click deploy of open weights onto GPUs/TPUs you choose	Node hours, even when idle	Open model, custom weights (LoRA), data in your VPC, steady high traffic
Deploy on GKE	Run the model yourself with vLLM, TGI or similar	Cluster nodes	Existing Kubernetes platform, maximum control, multi-model serving

Choosing between Gemini tiers

Pro: hardest reasoning, long-context analysis, complex coding and agents. Highest cost and latency.
Flash: balanced default for production at scale.
Flash-Lite: highest volume and lowest cost: classification, extraction, routing.
Choose by evaluating on your own task: start with the cheapest tier that meets the quality bar, not the biggest.
Pin a specific model version in production; aliases that auto-update can change behaviour under you.

3. Generation parameters

Parameter	Effect	Typical choice
`temperature`	Randomness of token choice; 0 is near-deterministic	0 to 0.3 for extraction and classification; higher for creative writing
`top_p` (nucleus)	Samples from the smallest set of tokens whose probability adds to p	Leave default unless tuning diversity
`top_k`	Samples from the k most likely tokens	Rarely changed
`max_output_tokens`	Caps response length and cost	Set it; truncated output ends with finish reason MAX_TOKENS
`stop_sequences`	Stop generating at given strings	Delimited formats
`candidate_count`	Several responses per request	Self-consistency or picking the best
Thinking controls (budget or level)	How much internal reasoning the model does before answering	More for multi-step problems; less for latency and cost
`seed`	More reproducible sampling	Testing

Temperature 0 reduces variation but doesn't guarantee identical outputs. For reliability, combine low temperature with structured output and evaluation.

4. Prompt and context engineering

Technique	What it does
System instructions	Role, rules, tone and constraints that apply to the whole conversation
Zero-shot vs few-shot	Instructions alone vs instructions plus worked examples; few-shot fixes format and edge cases
Clear structure	Delimit inputs (tags or headings), state the task, give the output format, say what to do when unsure
Chain-of-thought / thinking	Let the model reason step by step for multi-step problems (built-in thinking on newer models)
Decomposition and prompt chaining	Split a complex task into steps, each with its own prompt and checks
Self-consistency	Sample several answers and take the majority
Grounding	Supply facts in context (RAG, Google Search) instead of relying on memory
Context engineering	Deciding what goes in the context window: instructions, retrieved chunks, tool results, conversation history, memory. Less but relevant beats more

Long context vs RAG

	Put everything in a long context	Retrieve with RAG
Good for	A few large documents analysed as a whole; one-off analysis	Large or changing corpora; many users; citations; access control
Cost and latency	Every request pays for every token (context caching helps)	Only the relevant chunks are sent
Risk	Details "lost in the middle" of very long inputs	Retrieval misses the right chunk

5. Structured output and function calling

Structured output

Set response_mime_type="application/json" and a response_schema (OpenAPI-style or JSON Schema). The model is constrained to produce valid JSON matching the schema: enums for categories, required fields, types. This is the answer to "the app keeps failing to parse the model's JSON".

from google import genai
from google.genai import types

client = genai.Client(vertexai=True, project="PROJECT_ID", location="global")
resp = client.models.generate_content(
    model="gemini-flash-model-id",
    contents="Ticket: 'App crashes when I upload a photo.'",
    config=types.GenerateContentConfig(
        temperature=0,
        response_mime_type="application/json",
        response_schema={"type": "OBJECT",
                         "properties": {"category": {"type": "STRING", "enum": ["bug", "billing", "account", "other"]},
                                        "urgent": {"type": "BOOLEAN"}},
                         "required": ["category", "urgent"]}))
print(resp.text)   # {"category": "bug", "urgent": false}

Function calling (tool use)

You declare functions (name, description, parameter schema).
The model decides whether to call one and returns a function call with arguments. It doesn't execute anything.
Your code runs the function (an API, a database query) and sends back the result as a function response.
The model uses the result to answer, or calls another function.

Calling modes: AUTO (model decides), ANY (must call one; optionally restrict to allowed names), NONE.
Parallel function calls: several independent calls in one turn.
Built-in tools: Grounding with Google Search, code execution, URL context, enterprise search.
MCP (Model Context Protocol) servers expose tools to agents in a standard way; A2A (Agent2Agent) lets agents call other agents.

Validate function arguments and enforce authorization in your code. The model can be manipulated (prompt injection) into requesting calls a user shouldn't make; least-privilege tool credentials limit the blast radius.

6. Adapting a foundation model: the tuning ladder

Technique	What it changes	Use when
Prompt engineering	Instructions, examples (few-shot), system instructions, output schema	Always first: cheapest and fastest to iterate
Grounding / RAG (retrieval-augmented generation)	Adds retrieved facts to the prompt: Grounding with Google Search, Agent Search, RAG Engine, Vector Search	The model needs fresh or private facts, or must cite sources and hallucinate less
Supervised fine-tuning (parameter-efficient, adapter-based)	Model behaviour: style, format, domain task, classification labels	Prompting can't reach the quality, or prompts are getting long and costly; you have hundreds or more labelled examples
Preference tuning / reinforcement learning	Aligns with human preferences	You have preference pairs; quality judged by people
Distillation	Teaches a small model to imitate a large one	Need Pro-like quality at Flash-like cost and latency

Fine-tuning doesn't teach a model new facts reliably and doesn't keep them fresh. "The answers must reflect documents that change daily" is a grounding/RAG problem, not a tuning problem.

Supervised fine-tuning in practice

Topic	What to know
Data	JSONL of example conversations (user input, ideal model output), in Cloud Storage. Quality and consistency beat volume; start with ~100–500 good examples
Method	Parameter-efficient (adapter, LoRA-style) tuning on Gemini; full or LoRA tuning for open models from Model Garden
Hyperparameters	Epochs, adapter size, learning-rate multiplier; watch validation loss for overfitting
Output	A tuned model served from its own endpoint; call it like the base model
Evaluate	Against the base model with prompting, on a held-out set, with Agent Platform Evals
Cost	Tuning job cost plus serving the tuned model; re-tune when the base model version retires

Decision: knowledge problem (facts, freshness, citations) → grounding/RAG. Behaviour problem (format, style, a narrow task's accuracy, shorter prompts) → tuning. Cost/latency problem with a big model working well → distillation or a smaller tier.

7. Agents: ADK, Agent Runtime, Sessions and Memory Bank

An agent is a model in a loop: it plans, calls tools, reads results and continues until the task is done. Use agents when the steps aren't known in advance; use a fixed workflow (a chain or pipeline) when they are.

Piece	Role
ADK	Define agents, tools, sub-agents and workflow agents (sequential, parallel, loop) in Python or other languages; run locally
Agent Runtime	Deploy ADK (or LangGraph, LangChain and other framework) agents as a managed, autoscaling service with IAM, logging and tracing
Sessions	Stores conversation history and state for one interaction
Memory Bank	Long-term memory across sessions: extracts and recalls user facts and preferences
Code Execution	Sandboxed environment where an agent runs code it writes
Agent Garden	Samples and templates to start from
Gemini Enterprise	Employee-facing app where agents are shared and used across the company

Agent design choices the exam probes

Single agent with tools first; split into multiple agents only when roles, tools or prompts genuinely differ.
Short-term state in Sessions; durable user facts in Memory Bank; business data stays in your databases behind tools.
Guard tools: least-privilege service accounts, human approval for irreversible actions, Model Armor on inputs and outputs.
Evaluate trajectories (which tools were called, in what order), not only the final answer. See agent evaluation.

8. Safety settings and responses

Control	What it does
Safety filters	Harm categories (harassment, hate speech, sexually explicit, dangerous content) with blocking thresholds; non-configurable filters for the most severe content (such as CSAM)
Finish reason	`STOP` normal, `MAX_TOKENS` truncated, `SAFETY` blocked by filters, `RECITATION` too close to training data
Prompt feedback	A blocked prompt returns a block reason and no candidates
Model Armor	Templates that screen prompts and responses: prompt injection and jailbreak, sensitive data (via Sensitive Data Protection), malicious URLs, responsible-AI categories
System instructions	Scope and refusal rules ("only answer about our products")

Deeper coverage of security, responsible AI and monitoring is in Responsible AI and monitoring.

9. Cost, latency and availability

Lever	Effect
Smaller model tier (Flash, Flash-Lite)	Lower cost and latency; evaluate quality on your task first
Context caching	Reuse a large, repeated prefix (a manual, a codebase) across requests at a discount, with lower latency
Batch inference	Asynchronous jobs over BigQuery or Cloud Storage at roughly half the online price; minutes to hours of latency
Provisioned Throughput	Reserved capacity for predictable, production traffic; avoids 429s under shared quota
Global endpoint	Routes to available regions for higher availability (when data residency allows)
Retry with exponential backoff	Correct response to `429 Resource exhausted` under dynamic shared quota
Streaming responses	Better perceived latency (time to first token) for chat
Shorter prompts, `max_output_tokens`, structured output	Fewer tokens billed; parseable JSON without retries

Discount sizes and quota behaviour are set by current pricing pages. Learn the direction (batch and caching cost less; provisioned throughput buys reliability), not the exact percentages.

Context caching in detail

Implicit caching: automatic discount when requests share a long prefix; put the stable content (manual, instructions) first and the variable question last.
Explicit caching: create a cache object with a time to live; pay a storage fee per hour; reference it from many requests.
Worth it when the same large context is reused many times within the TTL.

10. Token cost calculator

Prices below are placeholders, not current list prices: enter the numbers from the pricing page. The point is how the levers compare.

Requests per day
Input tokens per request
Output tokens per request
Price per 1M input tokens
Price per 1M output tokens
Share of input that's a cacheable shared prefix (%)
Discount on cached input (%)
Batch discount (%)

11. Lab: a structured, tool-using Gemini app

In Agent Studio, write a system instruction for a support triage assistant. Try zero-shot, then add three examples; note which edge cases improve.
In a notebook with the Google Gen AI SDK, request structured output with an enum category and boolean urgent; run 50 tickets and count parse failures (should be zero).
Declare a function get_order_status(order_id); implement it as a stub; confirm the model returns a function call and that your code executes it.
Compare a Flash-Lite and a Flash tier on the same 50 tickets: accuracy, latency and token counts from usage metadata.
Put a 30-page policy document first in the prompt and send 20 questions; compare cached token counts and latency between the first and later requests.
Wrap the assistant as an ADK agent with the tool and deploy it to Agent Runtime; enable Sessions and check that a follow-up question uses earlier context.

Clean up: delete the Agent Runtime deployment, explicit caches and any tuned-model endpoints.

12. "Which gen AI option?" chooser

Pick an item to see what it means, why, and the tempting wrong choice.

Pick one above.

13. Sample outputs

What the tools return. Expand an item to see a typical result and what to read in it.

Illustrative output, not captured from a real project. IDs, numbers and some field names are invented or simplified, and formats change between releases. Learn the shape and the fields called out under each one.

Gemini response metadata (REST, simplified)Safety ratings and token usage

"candidates": [{
  "finishReason": "SAFETY",
  "safetyRatings": [
    {"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "probability": "MEDIUM", "blocked": true},
    {"category": "HARM_CATEGORY_HARASSMENT", "probability": "NEGLIGIBLE"}
  ]
}],
"usageMetadata": {"promptTokenCount": 8123, "cachedContentTokenCount": 7900,
                  "candidatesTokenCount": 0, "totalTokenCount": 8123}

What to look at: finishReason: SAFETY means a safety filter blocked the output; adjust the threshold for that category only if policy allows, or change the prompt. Most prompt tokens came from the context cache, which is cheaper than uncached input.

generate_content with tools (response, simplified)A function call the model wants you to run

{
  "candidates": [{
    "content": {"role": "model", "parts": [{
      "functionCall": {"name": "get_order_status", "args": {"order_id": "A-10442"}}
    }]},
    "finishReason": "STOP"
  }],
  "usageMetadata": {"promptTokenCount": 212, "candidatesTokenCount": 18}
}

What to look at: The model didn't fetch anything: it returned a functionCall with arguments. Your code must validate order_id, check the user is allowed to see it, call the API, and send the result back as a functionResponse part in the next turn.

14. Scenarios

Exam-style situations: how to diagnose each one, the likely cause and the fix.

15. Searchable reference

Item	Meaning

16. Flashcards (click to flip)

17. Common gotchas in one list

Tuning changes behaviour, not knowledge. Fresh or private facts need grounding.
The model never executes functions; your code does, and must check authorization.
Structured output beats "please return JSON" for reliable parsing.
Start with the cheapest tier that passes evaluation, not the biggest model.
MaaS for no-ops and spiky traffic; self-deploy for custom weights, VPC control or steady load.
Pin model versions and re-run evaluations before upgrading.
429s mean quota pressure: exponential backoff, then Provisioned Throughput for steady load.
Batch inference for big, non-urgent jobs: discounted and asynchronous.
Context caching needs the shared content at the start of the prompt.
Sessions are short-term; Memory Bank is long-term.
Prefer one agent with tools before multi-agent designs.
Temperature 0 isn't determinism; evaluate and constrain output.
Long context isn't free: every request pays for all tokens unless cached.

18. Practice quiz (24 questions)

Score: 0 / 0