Deploy, evaluate, and manage Foundry agents end-to-end: Docker build, ACR push, hosted/prompt agent create, container start, batch eval, continuous eval, prompt optimizer workflows, agent.yaml, dataset curation from traces. USE FOR: deploy agent to Foundry, hosted agent, create agent, invoke agent, evaluate agent, run batch eval, continuous eval, continuous monitoring, continuous eval status, optimize prompt, improve prompt, prompt optimizer, optimize agent instructions, improve agent instructions, optimize system prompt, deploy model, Foundry project, RBAC, role assignment, permissions, quota, capacity, region, troubleshoot agent, deployment failure, create dataset from traces, dataset versioning, eval trending, create AI Services, Cognitive Services, create Foundry resource, provision resource, knowledge index, agent monitoring, customize deployment, onboard, availability. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure-prepare).
Installation
Summary
Deploy, evaluate, and manage Foundry agents end-to-end: Docker build, ACR push, hosted/prompt agent create, container start, batch eval, continuous eval, prompt optimizer workflows, agent.yaml, dataset curation from traces. USE FOR: deploy agent to Foundry, hosted agent, create agent, invoke agent, evaluate agent, run batch eval, continuous eval, continuous monitoring, continuous eval status, optimize prompt, improve prompt, prompt optimizer, optimize agent instructions, improve agent instructions, optimize system prompt, deploy model, Foundry project, RBAC, role assignment, permissions, quota, capacity, region, troubleshoot agent, deployment failure, create dataset from traces, dataset versioning, eval trending, create AI Services, Cognitive Services, create Foundry resource, provision resource, knowledge index, agent monitoring, customize deployment, onboard, availability. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure-prepare).
agent-metadata.yaml is the preferred local/dev metadata file. Optional sidecar files such as agent-metadata.prod.yaml can hold a single prod or CI-targeted environment without mixing multiple environments in one file.
datasets/ and evaluators/ are local cache folders. Reuse them when they are current, and ask before refreshing or overwriting them.
See Agent Metadata Contract for the canonical schema and workflow rules.
SKILL.md
Microsoft Foundry Skill
This skill helps developers work with Microsoft Foundry resources, covering model discovery and deployment, complete dev lifecycle of AI agent, evaluation workflows, and troubleshooting.
Pre-Execution Requirements
MANDATORY: Before executing ANY workflow, you MUST first call the Azure MCP foundry tool and inspect the available Foundry MCP tools and related parameters. Treat this initial foundry call as a discovery/help step. For this skill, Azure MCP foundry is the required entry point for Foundry-related MCP operations.
Sub-Skills
MANDATORY: Before executing ANY workflow-specific steps, you MUST read the corresponding sub-skill document. Do not call workflow-specific MCP tools for a workflow without reading its skill document. This applies even if you already know the MCP tool parameters — the skill document contains required workflow steps, pre-checks, and validation logic that must be followed. This rule applies on every new user message that triggers a different workflow, even if the skill is already loaded.
This skill includes specialized sub-skills for specific workflows. Use these instead of the main skill when they match your task:
Sub-Skill
When to Use
Reference
deploy
Containerize, build, push to ACR, create/update/clone agent deployments
Evaluate agent quality, run batch evals, analyze failures, optimize prompts, improve agent instructions, compare versions, set up CI/CD monitoring, and enable continuous production evaluation
Create new hosted agent applications. Supports Microsoft Agent Framework, LangGraph, or custom frameworks in Python or C#, across responses or invocations protocols.
💡 Tip: For a complete onboarding flow: project/create (public) or private-network (VNet isolation) → models/deploy-model → agent workflows (create → deploy → invoke).
💡 Model Deployment: Use models/deploy-model for all deployment scenarios — it intelligently routes between quick preset deployment, customized deployment with full control, and capacity discovery across regions.
💡 Prompt Optimization: For requests like "optimize my prompt" or "improve my agent instructions," load observe and use the prompt_optimize MCP tool through that eval-driven workflow.
Infrastructure Lifecycle
Match user intent to the correct infrastructure workflow.
User Intent
Workflow
"Create Foundry" / "Set up Foundry" (ambiguous)
Use AskUserQuestion: (a) just an AI Services resource, (b) a project with public access, or (c) a project with network isolation? Route: (a) → resource/create, (b) → project/create, (c) → private-network
agent-metadata.yaml is the preferred local/dev metadata file. Optional sidecar files such as agent-metadata.prod.yaml can hold a single prod or CI-targeted environment without mixing multiple environments in one file.
datasets/ and evaluators/ are local cache folders. Reuse them when they are current, and ask before refreshing or overwriting them.
Standard Agent Setup - Standard capability-host setup with customer-managed data, search, and AI Services resources.
Agent: Project Context Resolution
Agent skills should run this step only when they need configuration values they don't already have. If a value (for example, agent root, environment, project endpoint, or agent name) is already known from the user's message or a previous skill in the same session, skip resolution for that value.
Step 1: Discover Agent Roots
Search the workspace for .foundry/ folders that contain agent-metadata.yaml or agent-metadata.<env>.yaml.
One match → use that agent root.
Multiple matches → require the user to choose the target agent folder.
No matches → for create/deploy workflows, seed a new .foundry/ folder during setup; for all other workflows, stop and ask the user which agent source folder to initialize.
After selecting an agent root, keep all local .foundry cache inspection, source inspection, evaluator suggestions, dataset suggestions, and prompt-optimization context inside that folder only. Do not scan sibling agent folders unless the user explicitly switches roots.
Step 2: Select Metadata File and Resolve Environment
Inside the selected agent root, choose the metadata file in this order:
Metadata filename or path explicitly provided by the user or workflow
If an explicit environment is already known and .foundry/agent-metadata.<env>.yaml exists, use that file
.foundry/agent-metadata.yaml
If multiple metadata files remain and no rule above selects one, prompt the user to choose
Read the selected metadata file and resolve the environment in this order:
Environment explicitly named by the user
If the selected metadata file defines exactly one environment, use it
Environment already selected earlier in the session
defaultEnvironment from metadata
If the selected metadata file still contains multiple environments and none of the rules above selects one, prompt the user to choose. Keep the selected agent root, metadata file, and environment visible in every workflow summary.
If the selected environment exposes older testSuites[] metadata but not evaluationSuites[], treat testSuites[] as the source for this session and normalize each entry in memory to the evaluationSuites[] shape before continuing. If the metadata is older still and only exposes legacy testCases[], normalize that list the same way. Preserve dataset and evaluator fields, keep any existing tags, and map legacy priority to tags.tier only when tags.tier is missing: P0 -> smoke, P1 -> regression, P2 -> coverage.
Step 3: Resolve Common Configuration
Use the selected environment in the selected metadata file as the primary source:
If create/deploy is initializing a new .foundry workspace and metadata fields are still missing, check if azure.yaml exists in the project root. If found, run azd env get-values and use it to seed agent-metadata.yaml by default, or agent-metadata.<env>.yaml when the workflow explicitly targets a separate environment-specific file.
On any metadata write (deploy, auto-setup, dataset refresh, or trace-to-dataset update), persist only evaluationSuites[] in the selected metadata file. If the selected file is a preferred single-environment file, rewrite only that one environment block. If the selected file is a legacy multi-environment file, rewrite only the selected environment block. Never copy or merge environments across sibling metadata files automatically. If the selected environment still uses older testSuites[] or legacy testCases[], rewrite it to evaluationSuites[] and remove migrated priority fields from the rewritten entries.
azd Variable
Seeds
AZURE_AI_PROJECT_ENDPOINT or AZURE_AIPROJECT_ENDPOINT
environments.<env>.projectEndpoint
AZURE_CONTAINER_REGISTRY_NAME or AZURE_CONTAINER_REGISTRY_ENDPOINT
environments.<env>.azureContainerRegistry
AZURE_SUBSCRIPTION_ID
Azure subscription for trace/troubleshoot lookups
Step 5: Collect Missing Values
Use the ask_user or askQuestions tool only for values not resolved from the user's message, session context, metadata, or azd bootstrap. Common values skills may need:
Convert existing Python agent code to a FAOS (Foundry Agent Optimization Service) optimization-ready version by wiring evaluator-targeted instructions/model/temperature knobs, then stop for review before deployment.
Harvest production traces into evaluation datasets, manage dataset versions and splits, track evaluation metrics over time, detect regressions, and maintain full lineage from trace to deployment. Use for: create dataset from traces, dataset versioning, evaluation trending, regression detection, dataset comparison, eval lineage.
Creating Azure AI Services multi-service resource (Foundry resource) using Azure CLI. Use when manually provisioning AI Services resources with granular control.
Managing quotas and capacity for Microsoft Foundry resources. Use when checking quota usage, troubleshooting deployment failures due to insufficient quota, requesting quota increases, or planning capacity.
Managing RBAC permissions, role assignments, managed identities, and service principals for Microsoft Foundry resources. Use for access control, auditing permissions, and CI/CD setup.