Configuring CI Using Azure Pipelines and Nx

Nx is a smart, fast and extensible build system, and it works really well with monorepos. Monorepos provide a lot of advantages:

  • Everything at that current commit works together. Changes can be verified across all affected parts of the organization.
  • Easy to split code into composable modules
  • Easier dependency management
  • One toolchain setup
  • Code editors and IDEs are "workspace" aware
  • Consistent developer experience
  • And more ...

But they come with their own technical challenges. The more code you add into your repository, the slower the CI gets. Adding Nx to your CI pipeline makes this more efficient.

Setting up Azure Pipelines

Below is an example of an Azure Pipelines setup for an Nx workspace only building and testing what is affected.

Unlike GitHub Actions and CircleCI, you don't have the metadata to help you track the last successful run on main. In the example below, the base is set to HEAD~1 (for push) or branching point (for pull requests), but a more robust solution would be to tag a SHA in the main job once it succeeds, and then use this tag as a base. See the nx-tag-successful-ci-run and nx-set-shas (version 1 implements tagging mechanism) repos for more information.

We also have to set NX_BRANCH explicitly.

1trigger:
2  - main
3pr:
4  - main
5
6variables:
7  - name: NX_BRANCH
8    ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
9      value: $(System.PullRequest.PullRequestNumber)
10    ${{ if ne(variables['Build.Reason'], 'PullRequest') }}:
11      value: $(Build.SourceBranchName)
12  - name: TARGET_BRANCH
13    ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
14      value: $[replace(variables['System.PullRequest.TargetBranch'],'refs/heads/','origin/')]
15  - name: BASE_SHA
16    ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
17      value: $(git merge-base $(TARGET_BRANCH) HEAD)
18    ${{ if ne(variables['Build.Reason'], 'PullRequest') }}:
19      value: $(git rev-parse HEAD~1)
20  - name: HEAD_SHA
21    value: $(git rev-parse HEAD)
22
23jobs:
24  - job: main
25    pool:
26      vmImage: 'ubuntu-latest'
27    steps:
28      - script: npm ci
29
30      - script: npx nx workspace-lint
31      - script: npx nx format:check
32
33      - script: npx nx affected --base=$(BASE_SHA) --target=lint --parallel=3
34      - script: npx nx affected --base=$(BASE_SHA) --target=test --parallel=3 --ci --code-coverage
35      - script: npx nx affected --base=$(BASE_SHA) --target=build --parallel=3

The main job implements the CI workflow.

Distributed CI with Nx Cloud

A computation cache is created on your local machine to make the developer experience faster. This allows you to not waste time re-building, re-testing, re-linting, or any number of other actions you might take on code that hasn't changed. Because the cache is stored locally, you are the only member of your team that can take advantage of these instant commands. You can manage and share this cache manually.

Nx Cloud allows this cache to be shared across your entire organization, meaning that any cacheable operation completed on your workspace only needs to be run once. Nx Cloud also allows you to distribute your CI across multiple machines to make sure the CI is fast even for very large repos.

In order to use distributed task execution, we need to start agents and set the NX_CLOUD_DISTRIBUTED_EXECUTION flag to true. Once all tasks are finished, we must not forget to cleanup used agents.

1trigger:
2  - main
3pr:
4  - main
5
6variables:
7  - name: NX_CLOUD_DISTRIBUTED_EXECUTION
8    value: 'true'
9  - name: NX_BRANCH
10    ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
11      value: $(System.PullRequest.PullRequestNumber)
12    ${{ if ne(variables['Build.Reason'], 'PullRequest') }}:
13      value: $(Build.SourceBranchName)
14  - name: TARGET_BRANCH
15    ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
16      value: $[replace(variables['System.PullRequest.TargetBranch'],'refs/heads/','origin/')]
17  - name: BASE_SHA
18    ${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
19      value: $(git merge-base $(TARGET_BRANCH) HEAD)
20    ${{ if ne(variables['Build.Reason'], 'PullRequest') }}:
21      value: $(git rev-parse HEAD~1)
22  - name: HEAD_SHA
23    value: $(git rev-parse HEAD)
24
25jobs:
26  - job: agents
27    strategy:
28      parallel: 3
29    displayName: 'Agent $(imageName)'
30    pool:
31      vmImage: 'ubuntu-latest'
32    steps:
33      - script: npm ci
34      - script: npx nx-cloud start-agent
35
36  - job: main
37    pool:
38      vmImage: 'ubuntu-latest'
39    steps:
40      - script: npm ci
41      - script: npx nx-cloud start-ci-run
42
43      - script: npx nx workspace-lint
44      - script: npx nx format:check --base=$(BASE_SHA) --head=$(HEAD_SHA)
45      - script: npx nx affected --base=$(BASE_SHA) --head=$(HEAD_SHA) --target=lint --parallel=3
46      - script: npx nx affected --base=$(BASE_SHA) --head=$(HEAD_SHA) --target=test --parallel=3 --ci --code-coverage
47      - script: npx nx affected --base=$(BASE_SHA) --head=$(HEAD_SHA) --target=build --parallel=3
48
49      - script: npx nx-cloud stop-all-agents
50        condition: always()

You can also use our ci-workflow generator to generate the pipeline file.

Learn more about configuring your CI environment using Nx Cloud with Distributed Caching and Distributed Task Execution in the Nx Cloud docs.