End to end tests
npm install @fluidframework/test-version-utilsThis is a package for writing and setting up Fluid end to end tests using mocha that will generate variants with
a specific driver and different version combinations of Fluid API between layers via TestObjectProvider provided
to the test. The different layers are loader, driver, container runtime and data runtime (includes DDS). Version
combinations and driver selection can be controlled via the mocha command line, assuming your test uses the provided
describe* functions. For advanced usage, a test can bypass this mechanism and directly call our
exports to get the versioned Fluid APIs.
Similar to mocha's describe, this package provide various describe* functions that will generate variants with
a specific driver and different version combinations of Fluid API between layers. All possible layer combinations that
are generated (empty entries are current versions):
| Compat Kind | Loader | Driver | Container Runtime | Data Runtime |
| ------------------- | ------ | ------ | ----------------- | ------------ |
| None | | | | |
| Loader | old | | | |
| Driver | | old | | |
| ContainerRuntime | | | old | |
| DataRuntime | | | | old |
| NewLoader | | old | old | old |
| NewDriver | old | | old | old |
| NewContainerRuntime | old | old | | old |
| NewDataRuntime | old | old | old | |
There are three compat describe* to generate different combinations, depending of the need of the tests
describeFullCompat: generate test variants with compat combinations that varies the version for all layers.
- Used for tests that exercise all layers and will benefits compat combinations of all layers.
describeLoaderCompat: generate test variants with compat combinations that only varies the loader version.
- Use for tests that targets the loader layer, and don't care about compat combinations of other layers.
- Test combination generated: [CompatKind.None, CompatKind.Loader]
describeNoCompat - generate one test variant that doesn't varies version of any layers.
- Use for tests that doesn't benefit or require any compat testing.
- Test combination generated: [CompatKind.None]
These compat describe* functions will also load the APIs with appropriate version and provide the test with aTestObjectProvider object, where the test can use to access Fluid functionality. Even when compat testing
is not necessary, TestObjectProvider provide functionalities that help writing Fluid tests, and it allows the test
to enable compat testing easily in the future just by changing the describe*.
By default, N-1, N-2, and LTS (hard coded) test variants are generated. The versions can be specified using command
line (see below) to run the test against any two versions. This package includes a mocha global hook that will
install legacy packages at the beginning of the package based on the compatVersion settings.
Tests using the compat describe* will be controllable using the command line options when running mocha on
driver selection, versions for compat testing, and compat kind combinations.
``text`
--compatKind
Default: undefined (no filter)
--compatVersion
Default: -1, -2, LTS (hard coded in src/compatConfig.ts)
--baseVersion
Default:
--driver
Default: "local"
--r11sEndpointName
Default: "r11s"
--tenantIndex
Default: 0
--reinstall - Force reinstall any required versions. Default: reuse if a version is already installed.
baseVersion is a semver or a semver range.
- If it is a semver range, the latest in that range will be picked.
compatVersion can be a semver or semver range or an integer apply to the minor version relative to baseVersion
- If it is a semver range, the latest in that range will be picked.
- If it is an integer, the value will be apply to the minor version of the baseVersion, and create a ^ rangebaseVersion
including prerelease versions. The latest in that range will be picked.
- i.e. if is 0.2.3, and compatVersion is -1, the resulting range^0.1.3-0
will be
We also accept some of the flags via environment variables.
| Environment Variables | Command line options |
| ----------------------------- | -------------------- |
| fluid__test__compatKind | --compatKind |
| fluid__test__compatVersion | --compatVersion |
| fluid__test__driver | --driver |
| fluid__test__r11sEndpointName | --r11sEndpointName |
| fluid__test__baseVersion | --baseVersion |
This bypasses any configuration of version used by the describe* functions and provides direct access to the versioned APIs.
First make sure to call ensurePackageInstalled before running the tests to make sure the necessary legacy version are
installed.
The main entry point is getVersionedTestObjectProvider to get a TestObjectProvider for a specific version combinations
and driver config. Additionally, you can get versioned API for specific layers using these API.
- getLoaderApi
-
-
-
All these API returns the current version by default if no arguments is passed.
If a number is provided, a relative version will be computed by adding the number to the minor version number
of the current version, and find the latest patch version. (^0.).
If a string is provided, then the string is treated as a specific version or a range of version, and it will
resolve the latest version that matches it.
OPEN ISSUE: while these API can be used directly, currently the default global mocha hook will still run and install the
default set of legacy versions whether it is necessary or not.
The legacy version are installed in their own version folder
./../node_modules/.legacy/ (current package root's node_module directory).
Legacy versions of all packages in all categories are installed regardless of what compat combination is requested.
(See packageList in src/testApi.ts`).
For now, the current version are statically bound to also provide type. Although it can be switch to
dynamic loading for consistency (or don't want to force the script to be loaded if they are not needed).
Currently, we don't have such scenario yet.