Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce the dependency caching for Maven and Gradle #193

Merged
merged 26 commits into from Aug 19, 2021
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
fef7d58
implement a core logic to cache dependnecies
KengoTODA Jul 13, 2021
4a7bf99
integrate the cache logic to entry points
KengoTODA Jul 13, 2021
945940e
add a user doc about the dependency cache feature
KengoTODA Jul 13, 2021
0591f86
reflect changes to the dist dir
KengoTODA Jul 13, 2021
f537562
add a prefix to the cache key
KengoTODA Jul 14, 2021
b047f97
test: extract build.gradle to a file in __tests__ dir
KengoTODA Jul 14, 2021
3e2fde2
run the restore e2e test on the specified OS
KengoTODA Jul 14, 2021
a5a0c52
add an e2e test for maven
KengoTODA Jul 14, 2021
1872d8e
fix the dependency among workflows
KengoTODA Jul 14, 2021
313e1ad
stabilize the cache on the Windows in e2e test
KengoTODA Jul 15, 2021
8687e45
add .gitignore files to __tests__/cache directories
KengoTODA Jul 15, 2021
6977c03
try to run restore after the authentication
KengoTODA Jul 14, 2021
7fe6c4d
use the key in state to save caches in the post process
KengoTODA Jul 15, 2021
fae2927
suggest users to run without daemon if fail to save Gradle cache on W…
KengoTODA Jul 20, 2021
f6a3b97
add missing description in the README.md
KengoTODA Jul 20, 2021
5f3f74c
run clean-up tasks in serial
KengoTODA Aug 2, 2021
53f73ba
Add validation for post step (#3)
dmitry-shibanov Aug 18, 2021
862dedb
Merge remote-tracking branch 'origin/main' into introduce-cache
KengoTODA Aug 18, 2021
098a656
Update src/cleanup-java.ts
KengoTODA Aug 19, 2021
cb966d0
Update src/cache.ts
KengoTODA Aug 19, 2021
6edf849
style: put the name of input to the constants.ts
KengoTODA Aug 19, 2021
0082baa
format: run `npm run build` to reflect changes to the dist dir
KengoTODA Aug 19, 2021
ea721b3
chore: update licensed files by `licensed cache`
KengoTODA Aug 19, 2021
807e574
fix: rerun ncc on macOS with node v12
KengoTODA Aug 19, 2021
33dfe17
build: follow the suggestion at PR page
KengoTODA Aug 19, 2021
67b74a1
fix: throw error in case of no package manager file found
KengoTODA Aug 19, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
88 changes: 88 additions & 0 deletions .github/workflows/e2e-cache.yml
@@ -0,0 +1,88 @@
name: Validate cache
on:
push:
KengoTODA marked this conversation as resolved.
Show resolved Hide resolved
branches:
- main
- releases/*
paths-ignore:
- '**.md'
pull_request:
paths-ignore:
- '**.md'

defaults:
run:
shell: bash

jobs:
save:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [macos-latest, windows-latest, ubuntu-latest]
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Run setup-java with the cache for gradle
uses: ./
id: setup-java
with:
distribution: 'adopt'
java-version: '11'
cache: gradle
- name: Create files to cache
run: |
cat <<EOF > build.gradle
plugins { id 'java' }
repositories { mavenCentral() }
dependencies { implementation 'org.codehaus.groovy:groovy:1.8.6' }
tasks.register('downloadDependencies') { doLast {
def total = configurations.compileClasspath.inject (0) { sum, file ->
sum + file.length()
}
println total
}}
EOF
KengoTODA marked this conversation as resolved.
Show resolved Hide resolved
gradle downloadDependencies
if [ ! -d ~/.gradle/caches ]; then
echo "::error::The ~/.gradle/caches directory does not exist unexpectedly"
exit 1
fi
restore:
runs-on: ubuntu-latest
needs: save
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Create build.gradle
run: |
cat <<EOF > build.gradle
plugins { id 'java' }
repositories { mavenCentral() }
dependencies { implementation 'org.codehaus.groovy:groovy:1.8.6' }
tasks.register('downloadDependencies') { doLast {
def total = configurations.compileClasspath.inject (0) { sum, file ->
sum + file.length()
}
println total
}}
EOF
if [ -d ~/.gradle/caches ]; then
echo "::error::The ~/.gradle/caches directory exists unexpectedly"
exit 1
fi
KengoTODA marked this conversation as resolved.
Show resolved Hide resolved
- name: Run setup-java with the cache for gradle
uses: ./
id: setup-java
with:
distribution: 'adopt'
java-version: '11'
cache: gradle
- name: Confirm that ~/.gradle/caches directory has been made
run: |
if [ ! -d ~/.gradle/caches ]; then
echo "::error::The ~/.gradle/caches directory does not exist unexpectedly"
exit 1
fi
ls ~/.gradle/caches/
13 changes: 13 additions & 0 deletions README.md
Expand Up @@ -58,6 +58,19 @@ Currently, the following distributions are supported:

**NOTE:** The different distributors can provide discrepant list of available versions / supported configurations. Please refer to the official documentation to see the list of supported versions.

#### Supported cache types
Currently, `gradle` and `maven` are supported. You can set `cache` input like below:
```yaml
steps:
- uses: actions/checkout@v2
- uses: actions/setup-java@v2
with:
distribution: 'adopt'
java-version: '11'
cache: 'gradle' # will restore cache of dependencies and wrappers
- run: ./gradlew build
```

### Check latest
In the basic examples above, the `check-latest` flag defaults to `false`. When set to `false`, the action tries to first resolve a version of Java from the local tool cache on the runner. If unable to find a specific version in the cache, the action will download a version of Java. Use the default or set `check-latest` to `false` if you prefer a faster more consistent setup experience that prioritizes trying to use the cached versions at the expense of newer versions sometimes being available for download.

Expand Down
181 changes: 181 additions & 0 deletions __tests__/cache.test.ts
@@ -0,0 +1,181 @@
import { mkdtempSync } from 'fs';
import { tmpdir } from 'os';
import { join } from 'path';
import { restore, save } from '../src/cache';
import * as fs from 'fs';
import * as os from 'os';
import * as core from '@actions/core';
import * as cache from '@actions/cache';

describe('dependency cache', () => {
const ORIGINAL_RUNNER_OS = process.env['RUNNER_OS'];
const ORIGINAL_GITHUB_WORKSPACE = process.env['GITHUB_WORKSPACE'];
const ORIGINAL_CWD = process.cwd();
let workspace: string;
let spyInfo: jest.SpyInstance<void, Parameters<typeof core.info>>;
let spyWarning: jest.SpyInstance<void, Parameters<typeof core.warning>>;

beforeEach(() => {
workspace = mkdtempSync(join(tmpdir(), 'setup-java-cache-'));
switch (os.platform()) {
case 'darwin':
process.env['RUNNER_OS'] = 'macOS';
break;
case 'win32':
process.env['RUNNER_OS'] = 'Windows';
break;
case 'linux':
process.env['RUNNER_OS'] = 'Linux';
break;
default:
throw new Error(`unknown platform: ${os.platform()}`);
}
process.chdir(workspace);
// This hack is necessary because @actions/glob ignores files not in the GITHUB_WORKSPACE
// https://git.io/Jcxig
process.env['GITHUB_WORKSPACE'] = projectRoot(workspace);
});

beforeEach(() => {
spyInfo = jest.spyOn(core, 'info');
spyWarning = jest.spyOn(core, 'warning');
});

afterEach(() => {
process.chdir(ORIGINAL_CWD);
process.env['GITHUB_WORKSPACE'] = ORIGINAL_GITHUB_WORKSPACE;
process.env['RUNNER_OS'] = ORIGINAL_RUNNER_OS;
});

describe('restore', () => {
let spyCacheRestore: jest.SpyInstance<
ReturnType<typeof cache.restoreCache>,
Parameters<typeof cache.restoreCache>
>;

beforeEach(() => {
spyCacheRestore = jest
.spyOn(cache, 'restoreCache')
.mockImplementation((paths: string[], primaryKey: string) => Promise.resolve(undefined));
});

it('throws error if unsupported package manager specified', () => {
return expect(restore('ant')).rejects.toThrowError('unknown package manager specified: ant');
});

describe('for maven', () => {
it('warns if no pom.xml found', async () => {
await restore('maven');
expect(spyWarning).toBeCalledWith(
`No file in ${projectRoot(
workspace
)} matched to [**/pom.xml], make sure you have checked out the target repository`
);
});
it('downloads cache', async () => {
createFile(join(workspace, 'pom.xml'));

await restore('maven');
expect(spyCacheRestore).toBeCalled();
expect(spyWarning).not.toBeCalled();
expect(spyInfo).toBeCalledWith('maven cache is not found');
});
});
describe('for gradle', () => {
it('warns if no build.gradle found', async () => {
await restore('gradle');
expect(spyWarning).toBeCalledWith(
`No file in ${projectRoot(
workspace
)} matched to [**/*.gradle*,**/gradle-wrapper.properties], make sure you have checked out the target repository`
);
});
it('downloads cache based on build.gradle', async () => {
createFile(join(workspace, 'build.gradle'));

await restore('gradle');
expect(spyCacheRestore).toBeCalled();
expect(spyWarning).not.toBeCalled();
expect(spyInfo).toBeCalledWith('gradle cache is not found');
});
it('downloads cache based on build.gradle.kts', async () => {
createFile(join(workspace, 'build.gradle.kts'));

await restore('gradle');
expect(spyCacheRestore).toBeCalled();
expect(spyWarning).not.toBeCalled();
expect(spyInfo).toBeCalledWith('gradle cache is not found');
});
});
});
describe('save', () => {
let spyCacheSave: jest.SpyInstance<
ReturnType<typeof cache.saveCache>,
Parameters<typeof cache.saveCache>
>;

beforeEach(() => {
spyCacheSave = jest
.spyOn(cache, 'saveCache')
.mockImplementation((paths: string[], key: string) => Promise.resolve(0));
});

it('throws error if unsupported package manager specified', () => {
return expect(save('ant')).rejects.toThrowError('unknown package manager specified: ant');
});

describe('for maven', () => {
it('uploads cache even if no pom.xml found', async () => {
await save('maven');
expect(spyCacheSave).toBeCalled();
expect(spyWarning).not.toBeCalled();
expect(spyInfo).toBeCalledWith(expect.stringMatching(/^Cache saved with the key:.*/));
});
it('uploads cache', async () => {
createFile(join(workspace, 'pom.xml'));

await save('maven');
expect(spyCacheSave).toBeCalled();
expect(spyWarning).not.toBeCalled();
expect(spyInfo).toBeCalledWith(expect.stringMatching(/^Cache saved with the key:.*/));
});
});
describe('for gradle', () => {
it('uploads cache even if no build.gradle found', async () => {
await save('gradle');
expect(spyCacheSave).toBeCalled();
expect(spyWarning).not.toBeCalled();
expect(spyInfo).toBeCalledWith(expect.stringMatching(/^Cache saved with the key:.*/));
});
it('uploads cache based on build.gradle', async () => {
createFile(join(workspace, 'build.gradle'));

await save('gradle');
expect(spyCacheSave).toBeCalled();
expect(spyWarning).not.toBeCalled();
expect(spyInfo).toBeCalledWith(expect.stringMatching(/^Cache saved with the key:.*/));
});
it('uploads cache based on build.gradle.kts', async () => {
createFile(join(workspace, 'build.gradle.kts'));

await save('gradle');
expect(spyCacheSave).toBeCalled();
expect(spyWarning).not.toBeCalled();
expect(spyInfo).toBeCalledWith(expect.stringMatching(/^Cache saved with the key:.*/));
});
});
});
});

function createFile(path: string) {
core.info(`created a file at ${path}`);
fs.writeFileSync(path, '');
}

function projectRoot(workspace: string): string {
if (os.platform() === 'darwin') {
return `/private${workspace}`;
} else {
return workspace;
}
}
51 changes: 51 additions & 0 deletions __tests__/cleanup-java.test.ts
@@ -0,0 +1,51 @@
import { mkdtempSync } from 'fs';
import { tmpdir } from 'os';
import { join } from 'path';
import { restore, save } from '../src/cache';
import { run as cleanup } from '../src/cleanup-java';
import * as fs from 'fs';
import * as os from 'os';
import * as core from '@actions/core';
import * as cache from '@actions/cache';

describe('cleanup', () => {
let spyInfo: jest.SpyInstance<void, Parameters<typeof core.info>>;
let spyWarning: jest.SpyInstance<void, Parameters<typeof core.warning>>;

let spyCacheSave: jest.SpyInstance<
ReturnType<typeof cache.saveCache>,
Parameters<typeof cache.saveCache>
>;
beforeEach(() => {
spyInfo = jest.spyOn(core, 'info');
spyWarning = jest.spyOn(core, 'warning');
spyCacheSave = jest.spyOn(cache, 'saveCache');
});

it('does not fail nor warn even when the save provess throws a ReserveCacheError', async () => {
spyCacheSave.mockImplementation((paths: string[], key: string) =>
Promise.reject(
new cache.ReserveCacheError(
'Unable to reserve cache with key, another job may be creating this cache.'
)
)
);
jest.spyOn(core, 'getInput').mockImplementation((name: string) => {
return name === 'cache' ? 'gradle' : '';
});
await cleanup();
expect(spyCacheSave).toBeCalled();
expect(spyWarning).not.toBeCalled();
});

it('does not fail even though the save process throws error', async () => {
spyCacheSave.mockImplementation((paths: string[], key: string) =>
Promise.reject(new Error('Unexpected error'))
);
jest.spyOn(core, 'getInput').mockImplementation((name: string) => {
return name === 'cache' ? 'gradle' : '';
});
await cleanup();
expect(spyCacheSave).toBeCalled();
});
});
3 changes: 3 additions & 0 deletions action.yml
Expand Up @@ -53,6 +53,9 @@ inputs:
description: 'Environment variable name for the GPG private key passphrase. Default is
$GPG_PASSPHRASE.'
required: false
cache:
description: 'Name of the build platform to cache dependencies. It can be "maven" or "gradle".'
required: false
outputs:
distribution:
description: 'Distribution of Java that has been installed'
Expand Down