Comprehensive guide for setting up Azure DevOps Services, self-hosted agents, CI/CD pipelines, and integration with various Azure services.
- Introduction
- Azure DevOps Services Setup
- Azure DevOps Server Installation
- Self-Hosted Agents Configuration
- CI/CD Pipeline Configuration
- Azure Repos Setup
- Azure Artifacts Configuration
- Azure Boards Setup
- Integration with Other Services
- Best Practices
- Troubleshooting
Azure DevOps is a suite of development tools provided by Microsoft for software development teams. It includes:
- Azure Boards: Agile planning, work item tracking, and visualization
- Azure Repos: Git repositories and code reviews
- Azure Pipelines: CI/CD for build, test, and deployment
- Azure Test Plans: Manual and exploratory testing
- Azure Artifacts: Package management
Azure DevOps Services (Cloud)
- Hosted by Microsoft
- No infrastructure management
- Automatic updates
- Free for up to 5 users
Azure DevOps Server (On-Premises)
- Self-hosted solution
- Full control over data
- Requires infrastructure maintenance
- formerly known as TFS (Team Foundation Server)
- Navigate to https://dev.azure.com
- Click on "Start free" or "Sign in"
- Sign in with your Microsoft account or create a new one
- Choose your preferred organization URL (e.g.,
https://dev.azure.com/yourcompany)
- After signing in, click "Create project"
- Fill in the project details:
- Project name: MyFirstProject
- Description: Description of your project
- Visibility: Private or Public
- Version control: Git or TFVC (Git recommended)
- Work item process: Agile, Scrum, or CMMI
- Click "Create"
- Go to Project Settings > Teams
- Click on your team
- Click Add members
- Enter email addresses and select access level (Basic, Stakeholder, etc.)
- Go to Project Settings > Permissions
- Configure access for different groups:
- Project Administrators: Full control
- Project Contributors: Can contribute code and work items
- Project Readers: Read-only access
- Project Valid Users: All users with access
Hardware Requirements
- CPU: 4-core processor (64-bit)
- RAM: 8 GB minimum (16 GB recommended)
- Disk: 100 GB for application tier, 500 GB for data tier
- Network: 1 Gbps
Software Requirements
- OS: Windows Server 2016/2019/2022
- SQL Server: SQL Server 2017/2019/2022 (Express/Standard/Enterprise)
- Browser: Latest version of Chrome, Edge, or Firefox
- Navigate to https://azure.microsoft.com/en-us/services/devops/server/
- Download the Azure DevOps Server installer
# Install SQL Server 2019 Express (if not already installed)
# Download from: https://www.microsoft.com/en-us/sql-server/sql-server-downloads- Run the Azure DevOps Server installer as Administrator
- Click on "Install Azure DevOps Server"
- Accept the license terms
- Choose installation type:
- Basic: Single-server installation
- Advanced: Custom configuration
- Configure the following:
- Application Tier: Configure web access URL
- Database: Connect to SQL Server instance
- Search: Configure search service (optional)
- Click "Install" and wait for completion
- Open the Azure DevOps Server Administration Console
- Create a new collection:
- Collection Name: DefaultCollection
- Description: Default project collection
- Configure the reporting and analysis services (optional)
# Download and install build agents
# Navigate to: https://dev.azure.com/{organization}/_admin/_AgentPool
# Download the agent package
# Extract and run the config.cmd- Go to Project Settings > Agent pools
- Click "Add pool"
- Select "Self-hosted"
- Name the pool (e.g., "Windows Agents")
- Click "Create"
# Create agent directory
mkdir C:\agent
cd C:\agent
# Download the agent
# Navigate to: https://dev.azure.com/{organization}/_settings/agentpools
# Download the Windows agent package
# Extract the agent
Add-Type -AssemblyName System.IO.Compression.FileSystem
[System.IO.Compression.ZipFile]::ExtractToDirectory("agent.zip", ".")
# Configure the agent
.\config.cmd
# Follow the prompts:
# - Enter server URL: https://dev.azure.com/{organization}
# - Enter authentication type: PAT
# - Enter personal access token
# - Enter agent pool name: Windows Agents
# - Enter agent name: (accept default or custom)
# - Enter work folder: _work (default)
# - Enter run as service: Y (recommended)
# - Enter service account: NETWORK SERVICE (default)
# Start the agent
.\run.cmd# Install required software on the agent
# Example: Install Docker
Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
Install-Package -Name docker -ProviderName DockerMsftProvider -Force
# Example: Install Node.js
choco install nodejs -y
# Example: Install Python
choco install python -y
# Example: Install .NET SDK
choco install dotnet-sdk -y# Update system
sudo apt-get update
sudo apt-get upgrade -y
# Install required packages
sudo apt-get install -y curl libunwind8 gettext
# Install .NET Core Runtime (for Azure DevOps agent)
wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
sudo dpkg -i packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install -y dotnet-runtime-6.0# Create agent directory
mkdir ~/agent
cd ~/agent
# Download the agent
# Navigate to: https://dev.azure.com/{organization}/_settings/agentpools
# Download the Linux agent package
# Extract the agent
tar zxvf agent.tar.gz
# Configure the agent
./config.sh
# Follow the prompts:
# - Enter server URL: https://dev.azure.com/{organization}
# - Enter authentication type: PAT
# - Enter personal access token
# - Enter agent pool name: Linux Agents
# - Enter agent name: (accept default or custom)
# - Enter work folder: _work (default)
# Install the agent as a service
sudo ./svc.sh install
sudo ./svc.sh startFROM ubuntu:22.04
# Install dependencies
RUN apt-get update && apt-get install -y \
curl \
libunwind8 \
gettext \
&& rm -rf /var/lib/apt/lists/*
# Install .NET Core Runtime
RUN wget https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
dpkg -i packages-microsoft-prod.deb && \
apt-get update && \
apt-get install -y dotnet-runtime-6.0 && \
rm packages-microsoft-prod.deb
# Install additional tools
RUN apt-get update && apt-get install -y \
docker.io \
git \
nodejs \
npm \
python3 \
python3-pip \
&& rm -rf /var/lib/apt/lists/*
# Create agent directory
WORKDIR /agent
# Copy agent files (downloaded separately)
COPY . .
# Configure and run agent
CMD ./config.sh --unattended --url ${AZP_URL} --auth PAT --token ${AZP_TOKEN} --pool ${AZP_POOL} --agent ${AZP_AGENT_NAME} --acceptTeeEula && ./run.sh# Build the image
docker build -t azure-devops-agent .
# Run the container
docker run -d \
-e AZP_URL=https://dev.azure.com/{organization} \
-e AZP_TOKEN={your-pat-token} \
-e AZP_POOL=Docker Agents \
-e AZP_AGENT_NAME=docker-agent-1 \
--name azure-devops-agent \
azure-devops-agent
# Mount Docker socket for Docker-in-Docker
docker run -d \
-e AZP_URL=https://dev.azure.com/{organization} \
-e AZP_TOKEN={your-pat-token} \
-e AZP_POOL=Docker Agents \
-e AZP_AGENT_NAME=docker-agent-1 \
-v /var/run/docker.sock:/var/run/docker.sock \
--name azure-devops-agent \
azure-devops-agentClassic Pipeline
- GUI-based configuration
- Easier for beginners
- Limited flexibility
- Not version controlled
YAML Pipeline (Recommended)
- Code-based configuration
- Version controlled
- More flexible and powerful
- Supports multi-stage pipelines
Create azure-pipelines.yml in your repository root:
# Basic pipeline structure
trigger:
branches:
include:
- main
- develop
paths:
exclude:
- docs/*
- README.md
pr:
branches:
include:
- main
- develop
pool:
vmImage: 'ubuntu-latest'
variables:
buildConfiguration: 'Release'
dotnetSdkVersion: '6.x'
stages:
- stage: Build
displayName: 'Build Stage'
jobs:
- job: Build
displayName: 'Build Job'
steps:
- task: UseDotNet@2
displayName: 'Install .NET SDK'
inputs:
packageType: 'sdk'
version: $(dotnetSdkVersion)
- task: DotNetCoreCLI@2
displayName: 'Restore Dependencies'
inputs:
command: 'restore'
projects: '**/*.csproj'
- task: DotNetCoreCLI@2
displayName: 'Build Project'
inputs:
command: 'build'
projects: '**/*.csproj'
arguments: '--configuration $(buildConfiguration)'
- task: DotNetCoreCLI@2
displayName: 'Run Tests'
inputs:
command: 'test'
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration) --no-build'
- task: PublishBuildArtifacts@1
displayName: 'Publish Artifacts'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: 'Container'
- stage: Deploy
displayName: 'Deploy Stage'
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: 'Deploy Job'
environment: 'Production'
strategy:
runOnce:
deploy:
steps:
- download: current
artifact: drop
- task: AzureWebApp@1
displayName: 'Deploy to Azure Web App'
inputs:
azureSubscription: 'Your Azure Service Connection'
appName: 'Your-Web-App-Name'
package: '$(Pipeline.Workspace)/drop/*.zip'trigger:
- main
pool:
vmImage: 'ubuntu-latest'
variables:
pythonVersion: '3.10'
stages:
- stage: Build
displayName: 'Build and Test'
jobs:
- job: Build
displayName: 'Build Job'
steps:
- task: UsePythonVersion@0
displayName: 'Install Python'
inputs:
versionSpec: $(pythonVersion)
- script: |
python -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
pip install pytest pytest-cov flake8
displayName: 'Setup Environment'
- script: |
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
displayName: 'Lint Code'
- script: |
pytest tests/ --cov=. --cov-report=xml --cov-report=html
displayName: 'Run Tests'
- task: PublishCodeCoverageResults@1
displayName: 'Publish Coverage'
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: '$(System.DefaultWorkingDirectory)/**/coverage.xml'
- script: |
python setup.py sdist bdist_wheel
displayName: 'Build Package'
- task: PublishBuildArtifacts@1
displayName: 'Publish Artifacts'
inputs:
PathtoPublish: '$(System.DefaultWorkingDirectory)/dist'
ArtifactName: 'python-package'
- stage: Deploy
displayName: 'Deploy to Azure Artifacts'
dependsOn: Build
condition: succeeded()
jobs:
- job: Deploy
displayName: 'Deploy Job'
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: $(pythonVersion)
- script: |
pip install twine
twine upload --repository-url $(AZURE_ARTIFACTS_URL) --skip-existing dist/*
displayName: 'Publish to Azure Artifacts'
env:
TWINE_USERNAME: $(AZURE_ARTIFACTS_USER)
TWINE_PASSWORD: $(AZURE_ARTIFACTS_PASSWORD)trigger:
- main
pool:
vmImage: 'ubuntu-latest'
variables:
dockerRegistryServiceConnection: 'Your Docker Registry Service Connection'
imageRepository: 'your-registry/your-image'
containerRegistry: 'your-registry.azurecr.io'
dockerfilePath: '$(Build.SourcesDirectory)/Dockerfile'
tag: '$(Build.BuildId)'
stages:
- stage: Build
displayName: 'Build Docker Image'
jobs:
- job: Build
displayName: 'Build Job'
steps:
- task: Docker@2
displayName: 'Build Image'
inputs:
repository: $(imageRepository)
command: build
Dockerfile: $(dockerfilePath)
tags: |
$(tag)
latest
- task: Docker@2
displayName: 'Push Image'
inputs:
containerRegistry: $(dockerRegistryServiceConnection)
repository: $(imageRepository)
command: push
tags: |
$(tag)
latest
- stage: Deploy
displayName: 'Deploy to Kubernetes'
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: 'Deploy to K8s'
environment: 'Production'
strategy:
runOnce:
deploy:
steps:
- task: KubernetesManifest@0
displayName: 'Deploy to K8s'
inputs:
action: deploy
kubernetesServiceConnection: 'Your K8s Service Connection'
manifests: |
$(Pipeline.Workspace)/manifests/deployment.yaml
$(Pipeline.Workspace)/manifests/service.yaml
containers: |
$(containerRegistry)/$(imageRepository):$(tag)- Navigate to Repos > Files
- Click "New repository"
- Choose repository type:
- Git (recommended)
- TFVC (legacy)
- Enter repository name
- Click "Create"
# Clone using HTTPS
git clone https://dev.azure.com/{organization}/{project}/_git/{repository}
# Clone using SSH (requires SSH key setup)
git clone git@ssh.dev.azure.com:v3/{organization}/{project}/{repository}git config --global user.name "Your Name"
git config --global user.email "your.email@example.com"# Generate SSH key
ssh-keygen -t rsa -b 4096 -C "your.email@example.com"
# Start SSH agent
eval $(ssh-agent -s)
# Add SSH key
ssh-add ~/.ssh/id_rsa
# Copy public key
cat ~/.ssh/id_rsa.pubAdd the public key to Azure DevOps:
- Go to User Settings > SSH public keys
- Click "New key"
- Paste the public key
- Click "Add"
- Navigate to Repos > Branches
- Click on the branch (e.g., main)
- Click "Branch policies"
- Configure policies:
- Require a minimum number of reviewers: Set to 2
- Check for linked work items: Enable
- Check for comment resolution: Enable
- Limit merge types: Enable "Squash merge"
- Build validation: Add pipeline validation
Create .azuredevops/pull_request_template.md:
## Description
<!-- Describe your changes here -->
## Type of Change
- [ ] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
## Testing
<!-- Describe how you tested your changes -->
## Checklist
- [ ] Code follows the style guidelines
- [ ] Self-review completed
- [ ] Comments added for complex code
- [ ] Unit tests added/updated
- [ ] Documentation updated- Navigate to Artifacts > Create feed
- Choose feed type:
- Project-scoped: Visible only within the project
- Organization-scoped: Visible across the organization
- Enter feed name and visibility
- Click "Create"
# Add NuGet source
dotnet nuget add source https://pkgs.dev.azure.com/{organization}/{project}/_packaging/{feed}/nuget/v3/index.json \
-n {feed} \
-u {username} \
-p {pat-token}# Pack the project
dotnet pack -c Release
# Push to Azure Artifacts
dotnet nuget push \
--source https://pkgs.dev.azure.com/{organization}/{project}/_packaging/{feed}/nuget/v3/index.json \
--api-key AzureDevOps \
bin/Release/*.nupkg# Install Azure Artifacts keyring
pip install keyring artifacts-keyring
# Add pip source
pip index versions \
--index-url https://pkgs.dev.azure.com/{organization}/{project}/_packaging/{feed}/pypi/simple/# Install twine
pip install twine
# Publish to Azure Artifacts
twine upload \
--repository-url https://pkgs.dev.azure.com/{organization}/{project}/_packaging/{feed}/pypi/upload/ \
dist/*# Create .npmrc file
echo "registry=https://pkgs.dev.azure.com/{organization}/{project}/_packaging/{feed}/npm/registry/" > .npmrc
echo "always-auth=true" >> .npmrc
# Set up authentication
npm config set //pkgs.dev.azure.com/{organization}/{project}/_packaging/{feed}/npm/registry/:_authToken {pat-token}# Publish to Azure Artifacts
npm publish- Navigate to Boards > Work Items
- Click "New Work Item"
- Select work item type:
- User Story: Feature requirements
- Bug: Defects and issues
- Task: Implementation tasks
- Epic: Large initiatives
- Feature: Group of user stories
- Go to Project Settings > Process
- Select your process (Agile, Scrum, or CMMI)
- Customize work item types and fields
- Navigate to Boards > Sprints
- Click "New iteration"
- Enter iteration name and dates
- Add to the iteration path
- Go to Boards > Boards
- Select your sprint
- Configure columns:
- New: New work items
- Active: Work in progress
- Resolved: Completed but not tested
- Closed: Completed and tested
- Navigate to Overview > Dashboards
- Click "New dashboard"
- Add widgets:
- Velocity Chart: Track sprint velocity
- Burndown Chart: Track sprint progress
- Work Item Summary: Overview of work items
- Build Summary: Build status
- Pull Request Status: PR status
- Go to Project Settings > Service connections
- Click "New service connection"
- Select "Azure Resource Manager"
- Choose authentication method:
- Service principal (automatic): Recommended
- Service principal (manual): For advanced scenarios
- Managed identity: For Azure DevOps Server
- task: AzureWebApp@1
displayName: 'Deploy to Azure Web App'
inputs:
azureSubscription: 'Your Azure Service Connection'
appName: 'Your-Web-App-Name'
package: '$(Pipeline.Workspace)/drop/*.zip'- task: KubernetesManifest@0
displayName: 'Deploy to AKS'
inputs:
action: deploy
kubernetesServiceConnection: 'Your K8s Service Connection'
manifests: |
$(Pipeline.Workspace)/manifests/deployment.yaml
$(Pipeline.Workspace)/manifests/service.yaml- Go to Project Settings > Service connections
- Click "New service connection"
- Select "GitHub"
- Authorize Azure DevOps to access your GitHub account
trigger:
- main
resources:
repositories:
- repository: mygithubrepo
type: github
name: YourOrg/YourRepo
endpoint: YourGitHubServiceConnection
pool:
vmImage: 'ubuntu-latest'
steps:
- checkout: self
- checkout: mygithubrepo- Install the Azure DevOps app in Slack
- In Azure DevOps, go to Project Settings > Service hooks
- Click "New subscription"
- Select "Slack"
- Configure triggers:
- Build completed
- Pull request created
- Work item created
- Use YAML pipelines for version control and flexibility
- Implement multi-stage pipelines for separation of build and deployment
- Use templates for reusable pipeline components
- Implement caching to speed up builds
- Use variables for configuration management
- Implement secrets management with Azure Key Vault
- Use pipeline artifacts for sharing data between stages
- Implement branch policies for main branches
- Use pull request templates for consistency
- Implement code reviews for all changes
- Use git-flow or trunk-based development
- Implement branch naming conventions
- Use .gitignore to exclude unnecessary files
- Use Personal Access Tokens (PATs) with minimal permissions
- Rotate PATs regularly
- Implement branch protection rules
- Use Azure Key Vault for secrets management
- Implement security scanning in pipelines
- Use managed identities for Azure resources
- Enable audit logs for compliance
- Use self-hosted agents for specific requirements
- Implement agent pools for different environments
- Use Docker agents for isolation
- Implement agent scaling for high-demand scenarios
- Keep agents updated with latest patches
- Monitor agent health and performance
Issue: Pipeline Fails with Authentication Error
# Solution: Check service connection and credentials
# Verify PAT token has required permissions
# Update service connection with correct credentialsIssue: Build Timeout
# Solution: Increase timeout in pipeline
timeoutInMinutes: 120
# Or use timeout at job level
jobs:
- job: Build
timeoutInMinutes: 60Issue: Agent Unavailable
# Solution: Check agent status
# Restart agent service
sudo ./svc.sh restart
# Reconfigure agent
./config.sh remove
./config.shIssue: Git Push Fails with Authentication Error
# Solution: Update git credentials
git config --global credential.helper store
git push
# Enter credentials when prompted
# Or use credential manager
git config --global credential.helper manager-coreIssue: Merge Conflicts
# Solution: Resolve conflicts
git pull origin main
# Resolve conflicts in files
git add .
git commit -m "Resolve conflicts"
git push origin feature-branchIssue: Agent Cannot Connect
# Solution: Check network connectivity
ping dev.azure.com
# Check agent configuration
cat .agent
# Reconfigure agent
./config.sh remove
./config.shIssue: Agent Out of Disk Space
# Solution: Clean agent work directory
rm -rf _work/*
# Configure work directory size limit
# In agent configuration, set work folder size limit# Clone Azure Repos repository
git clone https://dev.azure.com/{organization}/{project}/_git/{repository}
# Create Personal Access Token
# Navigate to: User Settings > Personal Access Tokens
# Configure Git credential helper
git config --global credential.helper manager-core
# View pipeline runs
# Navigate to: Pipelines > Pipelines > Select pipeline > Runs
# View agent status
# Navigate to: Project Settings > Agent Pools > Select pool > Agents# .NET Build
- task: DotNetCoreCLI@2
inputs:
command: 'build'
projects: '**/*.csproj'
# Node.js Build
- task: Npm@1
inputs:
command: 'install'
workingDir: '$(Build.SourcesDirectory)'
# Docker Build
- task: Docker@2
inputs:
repository: 'myimage'
command: 'build'
Dockerfile: '**/Dockerfile'
# Kubernetes Deploy
- task: KubernetesManifest@0
inputs:
action: 'deploy'
manifests: '**/deployment.yaml'- Azure DevOps Documentation
- Azure DevOps REST API
- Azure DevOps CLI
- Azure DevOps Blog
- Azure DevOps YouTube Channel
