Skip to content

Commit

Permalink
feat: Test coverage support
Browse files Browse the repository at this point in the history
This commit lays the foundation for displaying coverage results
from `bazel coverage`.

For the time being, the functionality is only exposed through the
user-defined tasks in the `tasks.json`. It is thereby a bit hard to
discover. But this is fine for the time being, because coverage still
has a couple of rough edges anyway. As soon as it is more stable, we
should add builtin commands and expose coverage runs also in the "Bazel
Build Target" tree.

Changes in this commit:
* Bumps the VS Code version to 1.88, i.e. the first VS Code version
  which supports the test coverage API.
* Upgrades to ES2022.  I wanted to use `replaceAll` which was introduced
  in ES2021.
  VS Code 1.88 is based on Node 18 which in turn is based on V8 10.1. V8
  10.18 supports ECMA-262 also known as ES2023.  However, ES2023 is not
  yet available a target language in the `tsconfig.json`.  Furthermore,
  Firefox does not fully support ES2023, yet. While web browsers are
  currently not relevant, they might become so in the future if we want
  to turn this into a browser-enabled VSCode extension.
  An upgrade to ES2021 would have been sufficient, but I went directly
  to ES2022 because it might some of the other new features might also
  turn out useful.
* Introduces a custom LCOV parser. I could not find any other
  high-quality open-source parser. E.g., most other parser don't
  properly parse function names with `:` and / or `,` in them.
* Introduces test cases for that custom LCOV parser.

Future work:
* Support for branch coverage
* Demangling of function names
* Builtin commands to trigger coverage runs & offer them in the "Bazel
  Build Tree"

Tested with: Java, C++, Go, Rust
Untested: Python, Swift, Kotlin, Scala and many more

This is the first step towards bazel-contrib#362
  • Loading branch information
vogelsgesang committed Apr 5, 2024
1 parent 09daa3d commit d2c82d8
Show file tree
Hide file tree
Showing 18 changed files with 8,243 additions and 628 deletions.
3 changes: 3 additions & 0 deletions .bazelci/presubmit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,14 @@ tasks:
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | sudo tee /etc/apt/sources.list.d/nodesource.list
sudo apt-get update
sudo apt-get install nodejs -y
# Required for vscode-test
- sudo apt install -y libasound2 libgbm1 libgtk-3-0 libnss3 xvfb
# Install node_modules and then compile and check the code for lint
# errors.
- npm ci
- npm run compile
- npm run check-lint
- xvfb-run -a npm run test
# TODO(allevato): Add a prettier check to verify that *.ts files don't
# differ from their prettier output. We need `prettier-tslint` so that
# prettier will obey our tslint config, but it doesn't support
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
out
node_modules
.vscode-test
*.vsix

# Ignore the generated .d.ts and .js files for protos that end up in the src/
Expand Down
6 changes: 6 additions & 0 deletions .vscode-test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
const { defineConfig } = require('@vscode/test-cli');

module.exports = defineConfig({
files: "out/test/**/*.test.js",
mocha: { ui: "bdd" },
});
1 change: 1 addition & 0 deletions .vscodeignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ out/test/**
scripts/**
src/**
test/**
.vscode-test/

**/*.map

Expand Down
34 changes: 33 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ This extension provides support for Bazel in Visual Studio.
- **Buildifier** integration to lint and format your Bazel files (requires that
[Buildifier](https://github.com/bazelbuild/buildtools/releases) be installed)
- **Bazel Task** definitions for `tasks.json`
- **Coverage Support** showing coverage results from `bazel coverage` directly
in VS Code.
- Debug Starlark code in your `.bzl` files during a build (set breakpoints, step
through code, inspect variables, etc.)

Expand Down Expand Up @@ -61,7 +63,7 @@ This extension can use [Facebook's starlark project](https://github.com/facebook

Bazel tasks can be configured from the `tasks.json` using the following structure:

```json
```jsonc
{
// See https://go.microsoft.com/fwlink/?LinkId=733558
// for the documentation about the tasks.json format
Expand Down Expand Up @@ -89,6 +91,36 @@ Bazel tasks can be configured from the `tasks.json` using the following structur
}
```

## Coverage support (Experimental)

For all `coverage` tasks, the coverage results are automatically loaded into VS
Code upon completion of the task. E.g., you could define your own task to
display the coverage provided by your integration tests using the following task
definition:

```jsonc
{
"label": "Show test coverage from integration test",
"type": "bazel",
"command": "coverage",
"targets": ["//test/integration/...", "//cpp/test/integration/..."],
"options": ["--instrumentation_filter=.*"]
}
```

You might need additional Bazel `options` to get the intended coverage results.
In particular if are using remote builds, you might need to use the
`--experimental_split_coverage_postprocessing` and `--experimental_fetch_all_coverage_outputs`
options. See the documentation on [Code Coverage with Bazel](https://bazel.build/configure/coverage)
for more details.

Code coverage support in this extension is still rather fresh and might still
have rough edges. It was tested with the Java, C++, Go and Rust rules.
In case you are using the code coverage integration with any other language
(Python, Swift, Kotlin, Scala, ...), please let us know how things are going in
bazelbuild/vscode-bazel#367. Please share both positive and negative experiences
you might have.

## Contributing

If you would like to contribute to the Bazel Visual Studio extension, please
Expand Down
Loading

0 comments on commit d2c82d8

Please sign in to comment.