Piloting agents in GitHub Copilot - Christopher Harrison, Microsoft

GitHub Copilot is best understood as an “AI pair programmer.” This metaphor is key because, like a human collaborator, Copilot’s effectiveness is directly tied to the quality of the collaboration. It has strengths and weaknesses, and working with it involves a back-and-forth dialogue to arrive at the desired solution.

The Critical Role of Context

Providing clear and comprehensive context is the most important aspect of working effectively with any form of AI, including Copilot. Context is not just the prompt you write; it’s the entire environment Copilot observes.

• Beyond the Prompt: While a detailed prompt is crucial, Copilot also gains context from your existing code. Clean, readable code with clear variable names and a logical project structure is easier for Copilot to understand and build upon. Avoid single-letter variable names and overly abbreviated names.
• Comments: Well-placed comments that explain complex logic help not only human developers but also the “AI pair programmer.”
• Be Explicit: Don’t be “passive-aggressive” with Copilot. If you have specific requirements, constraints, or a particular approach in mind, state it clearly. The more ambiguity you leave, the higher the chance the generated output won’t meet your needs. Clarity, specificity, and verbosity are more effective than terse commands.

Copilot’s Different Workloads

Copilot offers several modes, or workloads, to assist developers in different scenarios:

• Code Completion: The original inline suggestion feature that proposes lines or entire blocks of code as you type. It’s ideal when you’re “in the zone” and know what you need to write.
• Chat and Edit Mode: Chat allows for single-shot requests like explaining code or generating a new file. Edit mode extends this by enabling you to guide Copilot through changes across multiple files simultaneously, which is common for most development tasks.
• Local Agent Mode: In this mode, Copilot takes a more proactive role. It can explore your project, understand the file structure, run external tasks like builds and tests, and even attempt to self-heal by fixing errors it encounters.
• Copilot Coding Agent: The newest capability, allowing you to assign a GitHub issue directly to Copilot. It works asynchronously, letting developers offload smaller tasks and focus on more complex problems.

Deep Dive: The Copilot Coding Agent

The Coding Agent is designed to operate with a high degree of autonomy, acting like a developer assigned to a task.

Providing Instructions

To ensure the agent performs its work correctly, you must provide robust context:

1. The Issue Body: The issue description serves as the primary prompt. It should be detailed, outlining the goal and, if necessary, the preferred implementation strategy.
2. Instruction Files (copilot-instructions.md): You can add a copilot-instructions.md file to your repository. This markdown file is used by both the Coding Agent and local chat sessions. It should contain high-level information such as:
   • An overview of the application’s purpose and architecture.
   • Team-wide coding standards and language guidance.
   • Information about available scripts.
   • Documentation of the repository’s structure.
3. Scoped Instruction Files: For more specific tasks, you can create instruction files that apply only to certain file paths (e.g., *.test.py). This allows you to enforce patterns for specific components, like React components or Flask endpoints.

The Execution Environment

The Coding Agent runs its tasks within a GitHub Actions environment. You can define this environment by creating a workflow file named copilot-setup-steps.yml. In this file, you specify all necessary setup steps, such as installing dependencies, frameworks, or running prerequisite scripts.

Security and Governance

The Coding Agent was built with security as a primary concern:

• Ephemeral Environment: It operates in an isolated, ephemeral space provided by GitHub Actions, which is torn down after the task is complete.
• Restricted Permissions: The agent has read-only access to the repository. The only write permission it has is to the specific branch it creates for its pull request (e.g., copilot-fix-<issue-number>).
• Locked-Down Network: By default, the agent has no access to external resources, secrets, or the internet. It cannot install packages unless you explicitly configure the environment and open firewall access.
• Human in the Loop: A pull request created by Copilot does not automatically trigger CI/CD workflows like testing. A developer must first review the code and manually approve the workflow run. This maintains the principle that core DevOps practices—code reviews, automated testing, and security scans—are non-negotiable, regardless of whether the code was written by a human or AI.

Extending Capabilities with MCP (Model Context Protocol)

The Model Context Protocol (MCP) is a framework for extending Copilot’s knowledge and capabilities. Since the underlying LLM is static, MCP allows Copilot to call out to external services to fetch real-time information or perform actions.

• How it Works: A request to Copilot can be routed to an MCP server, which can then access external resources like a database, an internal API, or a knowledge base.
• Use Cases: This allows Copilot to answer questions based on your database schema (by providing the DDL), search internal documentation, or create an issue in a project management tool.
• Trust and Security: When using third-party MCP servers, it’s crucial to ensure they are from a trusted source, as they can access data and perform actions on your behalf. You can also build your own MCP server for internal tools.

Key Q&A Insights

• Internal Libraries: To get Copilot to use your internal libraries correctly, show it examples of how they are used in your codebase. Copilot is a pattern-matcher; it doesn’t need to see the source code of the library, just how it’s consumed. You can also provide API documentation in instruction files.
• Fine-Tuning: Fine-tuning the model on a private codebase is not currently supported. The recommended approach for providing specialized, private context is to use instruction files and custom MCP servers.
• Issue Tracker Integration: The Coding Agent currently only integrates with GitHub Issues. For other trackers like Jira, a workaround would be to create a script that mirrors the issue from Jira to GitHub.

• Tags:
• Code Generation
• Software Engineering
• Devops
• Model Context Protocol
• AI Agent
• GitHub Copilot