diff --git a/CHANGELOG.md b/CHANGELOG.md
index a3427cf937c4f96c777d614d8bb534f99ee73852..0f9a687d473dcebce8013d5e611e421852b2af97 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,12 @@
+## 0.12.25
+
+* Add a `override_platforms` configuration field which allows test platforms'
+ settings (such as browsers' executables) to be overridden by the user.
+
+* Add a `define_platforms` configuration field which makes it possible to define
+ new platforms that use the same logic as existing ones but have different
+ settings.
+
## 0.12.24+8
* `spawnHybridUri()` now interprets relative URIs correctly in browser tests.
diff --git a/doc/configuration.md b/doc/configuration.md
index 24cf7c038e8ad638c55b2e5687fbc56567160cbf..800cc5ddaee879085775553df7b48755980553a2 100644
--- a/doc/configuration.md
+++ b/doc/configuration.md
@@ -52,6 +52,10 @@ tags:
* [Configuring Platforms](#configuring-platforms)
* [`on_os`](#on_os)
* [`on_platform`](#on_platform)
+ * [`override_platforms`](#override_platforms)
+ * [`define_platforms`](#define_platforms)
+ * [Browser/Node.js Settings](#browser-and-node-js-settings)
+ * [`executable`](#executable)
* [Configuration Presets](#configuration-presets)
* [`presets`](#presets)
* [`add_preset`](#add_preset)
@@ -567,6 +571,126 @@ when running on a particular operating system, use [`on_os`](#on_os) instead.
This field counts as [test configuration](#test-configuration).
+### `override_platforms`
+
+This field allows you to customize the settings for built-in test platforms. It
+takes a map from platform identifiers to settings for those platforms. For example:
+
+```yaml
+override_platforms:
+ chrome:
+ # The settings to override for this platform.
+ settings:
+ executable: chromium
+```
+
+This tells the test runner to use the `chromium` executable for Chrome tests. It
+calls that executable with the same logic and flags it normally uses for Chrome.
+
+Each platform can define exactly which settings it supports. All browsers and
+Node.js support [the same settings](#browser-and-node-js-settings), but the VM
+doesn't support any settings and so can't be overridden.
+
+### `define_platforms`
+
+You can define new platforms in terms of old ones using the `define_platforms`
+field. This lets you define variants of existing platforms without overriding
+the old ones. This field takes a map from the new platform identifiers to
+definitions for those platforms. For example:
+
+```yaml
+define_platforms:
+ # This identifier is used to select the platform with the --platform flag.
+ chromium:
+ # A human-friendly name for the platform.
+ name: Chromium
+
+ # The identifier for the platform that this is based on.
+ extends: chrome
+
+ # Settings for the new child platform.
+ settings:
+ executable: chromium
+```
+
+Once this is defined, you can run `pub run test -p chromium` and it will run
+those tests in the Chromium browser, using the same logic it normally uses for
+Chrome. You can even use `chromium` in platform selectors; for example, you
+might pass `testOn: "chromium"` to declare that a test is Chromium-specific.
+User-defined platforms also count as their parents, so Chromium will run tests
+that say `testOn: "chrome"` as well.
+
+Each platform can define exactly which settings it supports. All browsers and
+Node.js support [the same settings](#browser-and-node-js-settings), but the VM
+doesn't support any settings and so can't be extended.
+
+This field is not supported in the
+[global configuration file](#global-configuration).
+
+### Browser and Node.js Settings
+
+All built-in browser platforms, as well as the built-in Node.js platform,
+provide the same settings that can be set using
+[`define_platforms`](#define_platforms), which control how their executables are
+invoked.
+
+#### `arguments`
+
+The `arguments` field provides extra arguments to the executable. It takes a
+string, and parses it in the same way as the POSIX shell:
+
+```yaml
+override_platforms:
+ firefox:
+ settings:
+ arguments: -headless
+```
+
+#### `executable`
+
+The `executable` field tells the test runner where to look for the executable to
+use to start the subprocess. It has three sub-keys, one for each supported
+operating system, which each take a path or an executable name:
+
+```yaml
+define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+
+ settings:
+ executable:
+ linux: chromium
+ mac_os: /Applications/Chromium.app/Contents/MacOS/Chromium
+ windows: Chromium\Application\chrome.exe
+```
+
+Executables can be defined in three ways:
+
+* As a plain basename, with no path separators. These executables are passed
+ directly to the OS, which looks them up using the `PATH` environment variable.
+
+* As an absolute path, which is used as-is.
+
+* **Only on Windows**, as a relative path. The test runner will look up this
+ path relative to the `LOCALAPPATA`, `PROGRAMFILES`, and `PROGRAMFILES(X86)`
+ environment variables, in that order.
+
+If a platform is omitted, it defaults to using the built-in executable location.
+
+As a shorthand, you can also define the same executable for all operating
+systems:
+
+```yaml
+define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+
+ settings:
+ executable: chromium
+```
+
## Configuration Presets
*Presets* are collections of configuration that can be explicitly selected on
diff --git a/lib/src/backend/declarer.dart b/lib/src/backend/declarer.dart
index 5432e9f128f1badd108485bac0ab075a121716f9..4e8851d4fde8c731e9ba7ac636bc2f387269bb14 100644
--- a/lib/src/backend/declarer.dart
+++ b/lib/src/backend/declarer.dart
@@ -4,6 +4,7 @@
import 'dart:async';
+import 'package:collection/collection.dart';
import 'package:stack_trace/stack_trace.dart';
import '../frontend/timeout.dart';
@@ -35,6 +36,10 @@ class Declarer {
/// and of the test suite.
final Metadata _metadata;
+ /// The set of variables that are valid for platform selectors, in addition to
+ /// the built-in variables that are allowed everywhere.
+ final Set<String> _platformVariables;
+
/// The stack trace for this group.
final Trace _trace;
@@ -84,17 +89,31 @@ class Declarer {
/// If [metadata] is passed, it's used as the metadata for the implicit root
/// group.
///
+ /// The [platformVariables] are the set of variables that are valid for
+ /// platform selectors in test and group metadata, in addition to the built-in
+ /// variables that are allowed everywhere.
+ ///
/// If [collectTraces] is `true`, this will set [GroupEntry.trace] for all
/// entries built by the declarer. Note that this can be noticeably slow when
/// thousands of tests are being declared (see #457).
///
/// If [noRetry] is `true` tests will be run at most once.
- Declarer({Metadata metadata, bool collectTraces: false, bool noRetry: false})
- : this._(null, null, metadata ?? new Metadata(), collectTraces, null,
+ Declarer(
+ {Metadata metadata,
+ Set<String> platformVariables,
+ bool collectTraces: false,
+ bool noRetry: false})
+ : this._(
+ null,
+ null,
+ metadata ?? new Metadata(),
+ platformVariables ?? const UnmodifiableSetView.empty(),
+ collectTraces,
+ null,
noRetry);
- Declarer._(this._parent, this._name, this._metadata, this._collectTraces,
- this._trace, this._noRetry);
+ Declarer._(this._parent, this._name, this._metadata, this._platformVariables,
+ this._collectTraces, this._trace, this._noRetry);
/// Runs [body] with this declarer as [Declarer.current].
///
@@ -111,13 +130,15 @@ class Declarer {
int retry}) {
_checkNotBuilt("test");
- var metadata = _metadata.merge(new Metadata.parse(
+ var newMetadata = new Metadata.parse(
testOn: testOn,
timeout: timeout,
skip: skip,
onPlatform: onPlatform,
tags: tags,
- retry: _noRetry ? 0 : retry));
+ retry: _noRetry ? 0 : retry);
+ newMetadata.validatePlatformSelectors(_platformVariables);
+ var metadata = _metadata.merge(newMetadata);
_entries.add(new LocalTest(_prefix(name), metadata, () async {
var parents = <Declarer>[];
@@ -151,17 +172,19 @@ class Declarer {
int retry}) {
_checkNotBuilt("group");
- var metadata = _metadata.merge(new Metadata.parse(
+ var newMetadata = new Metadata.parse(
testOn: testOn,
timeout: timeout,
skip: skip,
onPlatform: onPlatform,
tags: tags,
- retry: retry));
+ retry: _noRetry ? 0 : retry);
+ newMetadata.validatePlatformSelectors(_platformVariables);
+ var metadata = _metadata.merge(newMetadata);
var trace = _collectTraces ? new Trace.current(2) : null;
- var declarer = new Declarer._(
- this, _prefix(name), metadata, _collectTraces, trace, _noRetry);
+ var declarer = new Declarer._(this, _prefix(name), metadata,
+ _platformVariables, _collectTraces, trace, _noRetry);
declarer.declare(() {
// Cast to dynamic to avoid the analyzer complaining about us using the
// result of a void method.
diff --git a/lib/src/backend/metadata.dart b/lib/src/backend/metadata.dart
index 92a8ac81114b149df9a53ae7bce4bd34b0824571..7ff1a86246831b02377cbbb075d6bc8176493e2b 100644
--- a/lib/src/backend/metadata.dart
+++ b/lib/src/backend/metadata.dart
@@ -286,6 +286,17 @@ class Metadata {
"Dart identifiers.");
}
+ /// Throws a [FormatException] if any [PlatformSelector]s use any variables
+ /// that don't appear either in [validVariables] or in the set of variables
+ /// that are known to be valid for all selectors.
+ void validatePlatformSelectors(Set<String> validVariables) {
+ testOn.validate(validVariables);
+ onPlatform.forEach((selector, metadata) {
+ selector.validate(validVariables);
+ metadata.validatePlatformSelectors(validVariables);
+ });
+ }
+
/// Return a new [Metadata] that merges [this] with [other].
///
/// If the two [Metadata]s have conflicting properties, [other] wins. If
diff --git a/lib/src/backend/platform_selector.dart b/lib/src/backend/platform_selector.dart
index 590fc4e6addd1601713320330bfa697baad50492..d6da9f4246e678bfb474f40e1158a4b32e081609 100644
--- a/lib/src/backend/platform_selector.dart
+++ b/lib/src/backend/platform_selector.dart
@@ -8,10 +8,10 @@ import 'package:source_span/source_span.dart';
import 'operating_system.dart';
import 'test_platform.dart';
-/// The set of all valid variable names.
-final _validVariables =
+/// The set of variable names that are valid for all platform selectors.
+final _universalValidVariables =
new Set<String>.from(["posix", "dart-vm", "browser", "js", "blink"])
- ..addAll(TestPlatform.all.map((platform) => platform.identifier))
+ ..addAll(TestPlatform.builtIn.map((platform) => platform.identifier))
..addAll(OperatingSystem.all.map((os) => os.identifier));
/// An expression for selecting certain platforms, including operating systems
@@ -27,16 +27,46 @@ class PlatformSelector {
/// The boolean selector used to implement this selector.
final BooleanSelector _inner;
+ /// The source span from which this selector was parsed.
+ final SourceSpan _span;
+
/// Parses [selector].
///
- /// This will throw a [SourceSpanFormatException] if the selector is
- /// malformed or if it uses an undefined variable.
- PlatformSelector.parse(String selector)
- : _inner = new BooleanSelector.parse(selector) {
- _inner.validate(_validVariables.contains);
+ /// If [span] is passed, it indicates the location of the text for [selector]
+ /// in a larger document. It's used for error reporting.
+ PlatformSelector.parse(String selector, [SourceSpan span])
+ : _inner = _wrapFormatException(
+ () => new BooleanSelector.parse(selector), span),
+ _span = span;
+
+ const PlatformSelector._(this._inner) : _span = null;
+
+ /// Runs [body] and wraps any [FormatException] it throws in a
+ /// [SourceSpanFormatException] using [span].
+ ///
+ /// If [span] is `null`, runs [body] as-is.
+ static T _wrapFormatException<T>(T body(), SourceSpan span) {
+ if (span == null) return body();
+
+ try {
+ return body();
+ } on FormatException catch (error) {
+ throw new SourceSpanFormatException(error.message, span);
+ }
}
- const PlatformSelector._(this._inner);
+ /// Throws a [FormatException] if this selector uses any variables that don't
+ /// appear either in [validVariables] or in the set of variables that are
+ /// known to be valid for all selectors.
+ void validate(Set<String> validVariables) {
+ if (identical(this, all)) return;
+
+ _wrapFormatException(
+ () => _inner.validate((name) =>
+ _universalValidVariables.contains(name) ||
+ validVariables.contains(name)),
+ _span);
+ }
/// Returns whether the selector matches the given [platform] and [os].
///
@@ -46,6 +76,7 @@ class PlatformSelector {
return _inner.evaluate((variable) {
if (variable == platform.identifier) return true;
+ if (variable == platform.parent?.identifier) return true;
if (variable == os.identifier) return true;
switch (variable) {
case "dart-vm":
diff --git a/lib/src/backend/test_platform.dart b/lib/src/backend/test_platform.dart
index d387eba4987630015003a6f10390febca85b9398..e8ead67735b064bb90af5b106607218e33041fda 100644
--- a/lib/src/backend/test_platform.dart
+++ b/lib/src/backend/test_platform.dart
@@ -2,8 +2,6 @@
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
-import 'dart:collection';
-
// TODO(nweiz): support pluggable platforms.
/// An enum of all platforms on which tests can run.
class TestPlatform {
@@ -11,54 +9,55 @@ class TestPlatform {
// variable tests in test/backend/platform_selector/evaluate_test.
/// The command-line Dart VM.
- static const TestPlatform vm =
- const TestPlatform._("VM", "vm", isDartVM: true);
+ static const TestPlatform vm = const TestPlatform("VM", "vm", isDartVM: true);
/// Dartium.
- static const TestPlatform dartium = const TestPlatform._("Dartium", "dartium",
+ static const TestPlatform dartium = const TestPlatform("Dartium", "dartium",
isBrowser: true, isBlink: true, isDartVM: true);
/// Dartium content shell.
- static const TestPlatform contentShell = const TestPlatform._(
+ static const TestPlatform contentShell = const TestPlatform(
"Dartium Content Shell", "content-shell",
isBrowser: true, isBlink: true, isDartVM: true, isHeadless: true);
/// Google Chrome.
- static const TestPlatform chrome = const TestPlatform._("Chrome", "chrome",
+ static const TestPlatform chrome = const TestPlatform("Chrome", "chrome",
isBrowser: true, isJS: true, isBlink: true);
/// PhantomJS.
- static const TestPlatform phantomJS = const TestPlatform._(
+ static const TestPlatform phantomJS = const TestPlatform(
"PhantomJS", "phantomjs",
isBrowser: true, isJS: true, isBlink: true, isHeadless: true);
/// Mozilla Firefox.
static const TestPlatform firefox =
- const TestPlatform._("Firefox", "firefox", isBrowser: true, isJS: true);
+ const TestPlatform("Firefox", "firefox", isBrowser: true, isJS: true);
/// Apple Safari.
static const TestPlatform safari =
- const TestPlatform._("Safari", "safari", isBrowser: true, isJS: true);
+ const TestPlatform("Safari", "safari", isBrowser: true, isJS: true);
/// Microsoft Internet Explorer.
- static const TestPlatform internetExplorer = const TestPlatform._(
+ static const TestPlatform internetExplorer = const TestPlatform(
"Internet Explorer", "ie",
isBrowser: true, isJS: true);
/// The command-line Node.js VM.
static const TestPlatform nodeJS =
- const TestPlatform._("Node.js", "node", isJS: true);
-
- /// A list of all instances of [TestPlatform].
- static final UnmodifiableListView<TestPlatform> all =
- new UnmodifiableListView<TestPlatform>(_allPlatforms);
-
- /// Finds a platform by its identifier string.
- ///
- /// If no platform is found, returns `null`.
- static TestPlatform find(String identifier) =>
- all.firstWhere((platform) => platform.identifier == identifier,
- orElse: () => null);
+ const TestPlatform("Node.js", "node", isJS: true);
+
+ /// The platforms that are supported by the test runner by default.
+ static const List<TestPlatform> builtIn = const [
+ TestPlatform.vm,
+ TestPlatform.dartium,
+ TestPlatform.contentShell,
+ TestPlatform.chrome,
+ TestPlatform.phantomJS,
+ TestPlatform.firefox,
+ TestPlatform.safari,
+ TestPlatform.internetExplorer,
+ TestPlatform.nodeJS
+ ];
/// The human-friendly name of the platform.
final String name;
@@ -66,6 +65,13 @@ class TestPlatform {
/// The identifier used to look up the platform.
final String identifier;
+ /// The parent platform that this is based on, or `null` if there is no
+ /// parent.
+ final TestPlatform parent;
+
+ /// Returns whether this is a child of another platform.
+ bool get isChild => parent != null;
+
/// Whether this platform runs the Dart VM in any capacity.
final bool isDartVM;
@@ -81,45 +87,86 @@ class TestPlatform {
/// Whether this platform has no visible window.
final bool isHeadless;
- const TestPlatform._(this.name, this.identifier,
+ /// Returns the platform this is based on, or [this] if it's not based on
+ /// anything.
+ ///
+ /// That is, returns [parent] if it's non-`null` or [this] if it's `null`.
+ TestPlatform get root => parent ?? this;
+
+ const TestPlatform(this.name, this.identifier,
{this.isDartVM: false,
this.isBrowser: false,
this.isJS: false,
this.isBlink: false,
- this.isHeadless: false});
+ this.isHeadless: false})
+ : parent = null;
+
+ TestPlatform._child(this.name, this.identifier, this.parent)
+ : isDartVM = parent.isDartVM,
+ isBrowser = parent.isBrowser,
+ isJS = parent.isJS,
+ isBlink = parent.isBlink,
+ isHeadless = parent.isHeadless;
+
+ /// Converts a JSON-safe representation generated by [serialize] back into a
+ /// [TestPlatform].
+ factory TestPlatform.deserialize(Object serialized) {
+ if (serialized is String) {
+ return builtIn
+ .firstWhere((platform) => platform.identifier == serialized);
+ }
+
+ var map = serialized as Map;
+ var parent = map["parent"];
+ if (parent != null) {
+ // Note that the returned platform's [parent] won't necessarily be `==` to
+ // a separately-deserialized parent platform. This should be fine, though,
+ // since we only deserialize platforms in the remote execution context
+ // where they're only used to evaluate platform selectors.
+ return new TestPlatform._child(
+ map["name"], map["identifier"], new TestPlatform.deserialize(parent));
+ }
+
+ return new TestPlatform(map["name"], map["identifier"],
+ isDartVM: map["isDartVM"],
+ isBrowser: map["isBrowser"],
+ isJS: map["isJS"],
+ isBlink: map["isBlink"],
+ isHeadless: map["isHeadless"]);
+ }
+
+ /// Converts [this] into a JSON-safe object that can be converted back to a
+ /// [TestPlatform] using [new TestPlatform.deserialize].
+ Object serialize() {
+ if (builtIn.contains(this)) return identifier;
+
+ if (parent != null) {
+ return {
+ "name": name,
+ "identifier": identifier,
+ "parent": parent.serialize()
+ };
+ }
+
+ return {
+ "name": name,
+ "identifier": identifier,
+ "isDartVM": isDartVM,
+ "isBrowser": isBrowser,
+ "isJS": isJS,
+ "isBlink": isBlink,
+ "isHeadless": isHeadless
+ };
+ }
+
+ /// Returns a child of [this] that counts as both this platform's identifier
+ /// and the new [identifier].
+ ///
+ /// This may not be called on a platform that's already a child.
+ TestPlatform extend(String name, String identifier) {
+ if (parent == null) return new TestPlatform._child(name, identifier, this);
+ throw new StateError("A child platform may not be extended.");
+ }
String toString() => name;
}
-
-final List<TestPlatform> _allPlatforms = [
- TestPlatform.vm,
- TestPlatform.dartium,
- TestPlatform.contentShell,
- TestPlatform.chrome,
- TestPlatform.phantomJS,
- TestPlatform.firefox,
- TestPlatform.safari,
- TestPlatform.internetExplorer,
- TestPlatform.nodeJS
-];
-
-/// **Do not call this function without express permission from the test package
-/// authors**.
-///
-/// This constructs and globally registers a new TestPlatform with the provided
-/// details.
-TestPlatform registerTestPlatform(String name, String identifier,
- {bool isDartVM: false,
- bool isBrowser: false,
- bool isJS: false,
- bool isBlink: false,
- bool isHeadless: false}) {
- var platform = new TestPlatform._(name, identifier,
- isDartVM: isDartVM,
- isBrowser: isBrowser,
- isJS: isJS,
- isBlink: isBlink,
- isHeadless: isHeadless);
- _allPlatforms.add(platform);
- return platform;
-}
diff --git a/lib/src/executable.dart b/lib/src/executable.dart
index 1b2b1d772a2c3d12a18286dfbc1be55590a57f44..5fd756f6afa108d3a6920f958fce8ab87110800d 100644
--- a/lib/src/executable.dart
+++ b/lib/src/executable.dart
@@ -154,24 +154,30 @@ transformers:
return;
}
- var runner = new Runner(configuration);
-
+ Runner runner;
var signalSubscription;
close() async {
if (signalSubscription == null) return;
signalSubscription.cancel();
signalSubscription = null;
stdinLines.cancel(immediate: true);
- await runner.close();
+ await runner?.close();
}
signalSubscription = _signals.listen((_) => close());
try {
+ runner = new Runner(configuration);
exitCode = (await runner.run()) ? 0 : 1;
} on ApplicationException catch (error) {
stderr.writeln(error.message);
exitCode = exit_codes.data;
+ } on SourceSpanFormatException catch (error) {
+ stderr.writeln(error.toString(color: configuration.color));
+ exitCode = exit_codes.data;
+ } on FormatException catch (error) {
+ stderr.writeln(error.message);
+ exitCode = exit_codes.data;
} catch (error, stackTrace) {
stderr.writeln(getErrorMessage(error));
stderr.writeln(new Trace.from(stackTrace).terse);
diff --git a/lib/src/runner.dart b/lib/src/runner.dart
index 0b63e5ceae6f26d9c3c9c6ec910b5dedcf65e10e..91a444e971097a27504c65fe336f8435ec9aa231 100644
--- a/lib/src/runner.dart
+++ b/lib/src/runner.dart
@@ -128,7 +128,9 @@ class Runner {
if (testOn == PlatformSelector.all) return;
var unsupportedPlatforms = _config.suiteDefaults.platforms
- .where((platform) => !testOn.evaluate(platform, os: currentOS))
+ .map(_loader.findTestPlatform)
+ .where((platform) =>
+ platform != null && !testOn.evaluate(platform, os: currentOS))
.toList();
if (unsupportedPlatforms.isEmpty) return;
@@ -141,7 +143,7 @@ class Runner {
var unsupportedBrowsers =
unsupportedPlatforms.where((platform) => platform.isBrowser);
if (unsupportedBrowsers.isNotEmpty) {
- var supportsAnyBrowser = TestPlatform.all
+ var supportsAnyBrowser = _loader.allPlatforms
.where((platform) => platform.isBrowser)
.any((platform) => testOn.evaluate(platform));
@@ -351,7 +353,7 @@ class Runner {
/// Loads each suite in [suites] in order, pausing after load for platforms
/// that support debugging.
Future<bool> _loadThenPause(Stream<LoadSuite> suites) async {
- if (_config.suiteDefaults.platforms.contains(TestPlatform.vm)) {
+ if (_config.suiteDefaults.platforms.contains(TestPlatform.vm.identifier)) {
warn("Debugging is currently unsupported on the Dart VM.",
color: _config.color);
}
diff --git a/lib/src/runner/browser/browser.dart b/lib/src/runner/browser/browser.dart
index 79583b6b2f11590192286b8c7b67eb0729239809..e1602317e5976ebf3787acf3ecfb52e277b7e483 100644
--- a/lib/src/runner/browser/browser.dart
+++ b/lib/src/runner/browser/browser.dart
@@ -105,6 +105,7 @@ abstract class Browser {
_process.then((process) => process.kill());
if (stackTrace == null) stackTrace = new Trace.current();
+ if (_onExitCompleter.isCompleted) return;
_onExitCompleter.completeError(
new ApplicationException(
"Failed to run $name: ${getErrorMessage(error)}."),
diff --git a/lib/src/runner/browser/browser_manager.dart b/lib/src/runner/browser/browser_manager.dart
index 896672036d6c17be38c4f046070d14646463e393..9f9e3c9ad1595438bd164a6d7f6c5c9c63454081 100644
--- a/lib/src/runner/browser/browser_manager.dart
+++ b/lib/src/runner/browser/browser_manager.dart
@@ -15,6 +15,7 @@ import '../../util/stack_trace_mapper.dart';
import '../application_exception.dart';
import '../configuration/suite.dart';
import '../environment.dart';
+import '../executable_settings.dart';
import '../plugin/platform_helpers.dart';
import '../runner_suite.dart';
import 'browser.dart';
@@ -93,12 +94,14 @@ class BrowserManager {
/// [future]. If [debug] is true, starts the browser in debug mode, with its
/// debugger interfaces on and detected.
///
+ /// The [settings] indicate how to invoke this browser's executable.
+ ///
/// Returns the browser manager, or throws an [ApplicationException] if a
/// connection fails to be established.
- static Future<BrowserManager> start(
- TestPlatform platform, Uri url, Future<WebSocketChannel> future,
+ static Future<BrowserManager> start(TestPlatform platform, Uri url,
+ Future<WebSocketChannel> future, ExecutableSettings settings,
{bool debug: false}) {
- var browser = _newBrowser(url, platform, debug: debug);
+ var browser = _newBrowser(url, platform, settings, debug: debug);
var completer = new Completer<BrowserManager>();
@@ -128,26 +131,27 @@ class BrowserManager {
});
}
- /// Starts the browser identified by [browser] and has it load [url].
+ /// Starts the browser identified by [browser] using [settings] and has it load [url].
///
/// If [debug] is true, starts the browser in debug mode.
- static Browser _newBrowser(Uri url, TestPlatform browser,
+ static Browser _newBrowser(
+ Uri url, TestPlatform browser, ExecutableSettings settings,
{bool debug: false}) {
- switch (browser) {
+ switch (browser.root) {
case TestPlatform.dartium:
- return new Dartium(url, debug: debug);
+ return new Dartium(url, settings: settings, debug: debug);
case TestPlatform.contentShell:
- return new ContentShell(url, debug: debug);
+ return new ContentShell(url, settings: settings, debug: debug);
case TestPlatform.chrome:
- return new Chrome(url, debug: debug);
+ return new Chrome(url, settings: settings, debug: debug);
case TestPlatform.phantomJS:
- return new PhantomJS(url, debug: debug);
+ return new PhantomJS(url, settings: settings, debug: debug);
case TestPlatform.firefox:
- return new Firefox(url);
+ return new Firefox(url, settings: settings);
case TestPlatform.safari:
- return new Safari(url);
+ return new Safari(url, settings: settings);
case TestPlatform.internetExplorer:
- return new InternetExplorer(url);
+ return new InternetExplorer(url, settings: settings);
default:
throw new ArgumentError("$browser is not a browser.");
}
@@ -201,7 +205,8 @@ class BrowserManager {
///
/// If [mapper] is passed, it's used to map stack traces for errors coming
/// from this test suite.
- Future<RunnerSuite> load(String path, Uri url, SuiteConfiguration suiteConfig,
+ Future<RunnerSuite> load(
+ String path, Uri url, SuiteConfiguration suiteConfig, Object message,
{StackTraceMapper mapper}) async {
url = url.replace(
fragment: Uri.encodeFull(JSON.encode({
@@ -236,8 +241,8 @@ class BrowserManager {
});
try {
- controller = await deserializeSuite(
- path, _platform, suiteConfig, await _environment, suiteChannel,
+ controller = await deserializeSuite(path, _platform, suiteConfig,
+ await _environment, suiteChannel, message,
mapper: mapper);
_controllers.add(controller);
return controller.suite;
diff --git a/lib/src/runner/browser/chrome.dart b/lib/src/runner/browser/chrome.dart
index 7c5f481731980d9c0f1c8a0d2b26aa8c192b4f8f..e2150e240189c86762ca4c18c78e5fe0a2150136 100644
--- a/lib/src/runner/browser/chrome.dart
+++ b/lib/src/runner/browser/chrome.dart
@@ -5,10 +5,11 @@
import 'dart:async';
import 'dart:io';
-import 'package:path/path.dart' as p;
-
+import '../../backend/test_platform.dart';
import '../../util/io.dart';
+import '../executable_settings.dart';
import 'browser.dart';
+import 'default_settings.dart';
// TODO(nweiz): move this into its own package?
/// A class for running an instance of Chrome.
@@ -25,14 +26,10 @@ class Chrome extends Browser {
/// Starts a new instance of Chrome open to the given [url], which may be a
/// [Uri] or a [String].
- ///
- /// If [executable] is passed, it's used as the Chrome executable. Otherwise
- /// the default executable name for the current OS will be used.
- factory Chrome(url, {String executable, bool debug: false}) {
+ factory Chrome(url, {ExecutableSettings settings, bool debug: false}) {
+ settings ??= defaultSettings[TestPlatform.chrome];
var remoteDebuggerCompleter = new Completer<Uri>.sync();
return new Chrome._(() async {
- if (executable == null) executable = _defaultExecutable();
-
var tryPort = ([int port]) async {
var dir = createTempDir();
var args = [
@@ -45,7 +42,7 @@ class Chrome extends Browser {
"--no-default-browser-check",
"--disable-default-apps",
"--disable-translate",
- ];
+ ]..addAll(settings.arguments);
// Currently, Chrome doesn't provide any way of ensuring that this port
// was successfully bound. It produces an error if the binding fails,
@@ -54,7 +51,7 @@ class Chrome extends Browser {
// though.
if (port != null) args.add("--remote-debugging-port=$port");
- var process = await Process.start(executable, args);
+ var process = await Process.start(settings.executable, args);
if (port != null) {
remoteDebuggerCompleter.complete(
@@ -76,32 +73,4 @@ class Chrome extends Browser {
Chrome._(Future<Process> startBrowser(), this.remoteDebuggerUrl)
: super(startBrowser);
-
- /// Return the default executable for the current operating system.
- static String _defaultExecutable() {
- if (Platform.isMacOS) {
- return '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome';
- }
- if (!Platform.isWindows) return 'google-chrome';
-
- // Chrome could be installed in several places on Windows. The only way to
- // find it is to check.
- var prefixes = [
- Platform.environment['LOCALAPPDATA'],
- Platform.environment['PROGRAMFILES'],
- Platform.environment['PROGRAMFILES(X86)']
- ];
- var suffix = r'Google\Chrome\Application\chrome.exe';
-
- for (var prefix in prefixes) {
- if (prefix == null) continue;
-
- var path = p.join(prefix, suffix);
- if (new File(p.join(prefix, suffix)).existsSync()) return path;
- }
-
- // Fall back on looking it up on the path. This probably won't work, but at
- // least it will fail with a useful error message.
- return "chrome.exe";
- }
}
diff --git a/lib/src/runner/browser/content_shell.dart b/lib/src/runner/browser/content_shell.dart
index 5599a8dd5900af69535b75a4e1b691f2e9f54da6..e52bf1f20f0aafe5c357af30ffc73014f746f62b 100644
--- a/lib/src/runner/browser/content_shell.dart
+++ b/lib/src/runner/browser/content_shell.dart
@@ -5,10 +5,13 @@
import 'dart:async';
import 'dart:io';
+import '../../backend/test_platform.dart';
import '../../util/io.dart';
import '../../utils.dart';
import '../application_exception.dart';
+import '../executable_settings.dart';
import 'browser.dart';
+import 'default_settings.dart';
final _observatoryRegExp = new RegExp(r"^Observatory listening on ([^ ]+)");
final _errorTimeout = const Duration(seconds: 10);
@@ -27,18 +30,18 @@ class ContentShell extends Browser {
final Future<Uri> remoteDebuggerUrl;
- factory ContentShell(url, {String executable, bool debug: false}) {
+ factory ContentShell(url, {ExecutableSettings settings, bool debug: false}) {
+ settings ??= defaultSettings[TestPlatform.contentShell];
var observatoryCompleter = new Completer<Uri>.sync();
var remoteDebuggerCompleter = new Completer<Uri>.sync();
return new ContentShell._(() {
- if (executable == null) executable = _defaultExecutable();
-
var tryPort = ([int port]) async {
- var args = ["--dump-render-tree", url.toString()];
+ var args = ["--dump-render-tree", url.toString()]
+ ..addAll(settings.arguments);
if (port != null) args.add("--remote-debugging-port=$port");
- var process = await Process
- .start(executable, args, environment: {"DART_FLAGS": "--checked"});
+ var process = await Process.start(settings.executable, args,
+ environment: {"DART_FLAGS": "--checked"});
if (debug) {
observatoryCompleter.complete(lineSplitter
@@ -111,8 +114,4 @@ class ContentShell extends Browser {
ContentShell._(Future<Process> startBrowser(), this.observatoryUrl,
this.remoteDebuggerUrl)
: super(startBrowser);
-
- /// Return the default executable for the current operating system.
- static String _defaultExecutable() =>
- Platform.isWindows ? "content_shell.exe" : "content_shell";
}
diff --git a/lib/src/runner/browser/dartium.dart b/lib/src/runner/browser/dartium.dart
index aacf4140cbd79e82a3be605c58be9a47adde7e7e..e785231c0fd1c1d7921a65fa9d5343db93a28840 100644
--- a/lib/src/runner/browser/dartium.dart
+++ b/lib/src/runner/browser/dartium.dart
@@ -7,11 +7,13 @@ import 'dart:convert';
import 'dart:io';
import 'package:async/async.dart';
-import 'package:path/path.dart' as p;
+import '../../backend/test_platform.dart';
import '../../util/io.dart';
import '../../utils.dart';
+import '../executable_settings.dart';
import 'browser.dart';
+import 'default_settings.dart';
final _observatoryRegExp =
new RegExp(r'Observatory listening (?:on|at) ([^ "]+)');
@@ -30,12 +32,11 @@ class Dartium extends Browser {
final Future<Uri> remoteDebuggerUrl;
- factory Dartium(url, {String executable, bool debug: false}) {
+ factory Dartium(url, {ExecutableSettings settings, bool debug: false}) {
+ settings ??= defaultSettings[TestPlatform.dartium];
var observatoryCompleter = new Completer<Uri>.sync();
var remoteDebuggerCompleter = new Completer<Uri>.sync();
return new Dartium._(() async {
- if (executable == null) executable = _defaultExecutable();
-
var tryPort = ([int port]) async {
var dir = createTempDir();
var args = [
@@ -48,7 +49,7 @@ class Dartium extends Browser {
"--no-default-browser-check",
"--disable-default-apps",
"--disable-translate"
- ];
+ ]..addAll(settings.arguments);
if (port != null) {
args.add("--remote-debugging-port=$port");
@@ -62,8 +63,8 @@ class Dartium extends Browser {
args.add("--vmodule=startup_browser_creator_impl=1");
}
- var process = await Process
- .start(executable, args, environment: {"DART_FLAGS": "--checked"});
+ var process = await Process.start(settings.executable, args,
+ environment: {"DART_FLAGS": "--checked"});
if (port != null) {
// Dartium on Windows prints all standard IO to stderr, so we need to
@@ -124,47 +125,6 @@ class Dartium extends Browser {
this.remoteDebuggerUrl)
: super(startBrowser);
- /// Starts a new instance of Dartium open to the given [url], which may be a
- /// [Uri] or a [String].
- ///
- /// If [executable] is passed, it's used as the Dartium executable. Otherwise
- /// the default executable name for the current OS will be used.
-
- /// Return the default executable for the current operating system.
- static String _defaultExecutable() {
- var dartium = _executableInEditor();
- if (dartium != null) return dartium;
- return Platform.isWindows ? "dartium.exe" : "dartium";
- }
-
- static String _executableInEditor() {
- var dir = p.dirname(sdkDir);
-
- if (Platform.isWindows) {
- if (!new File(p.join(dir, "DartEditor.exe")).existsSync()) return null;
-
- var dartium = p.join(dir, "chromium\\chrome.exe");
- return new File(dartium).existsSync() ? dartium : null;
- }
-
- if (Platform.isMacOS) {
- if (!new File(p.join(dir, "DartEditor.app/Contents/MacOS/DartEditor"))
- .existsSync()) {
- return null;
- }
-
- var dartium =
- p.join(dir, "chromium/Chromium.app/Contents/MacOS/Chromium");
- return new File(dartium).existsSync() ? dartium : null;
- }
-
- assert(Platform.isLinux);
- if (!new File(p.join(dir, "DartEditor")).existsSync()) return null;
-
- var dartium = p.join(dir, "chromium", "chrome");
- return new File(dartium).existsSync() ? dartium : null;
- }
-
// TODO(nweiz): simplify this when sdk#23923 is fixed.
/// Returns the Observatory URL for the Dartium executable with the given
/// [stdout] stream, or `null` if the correct one couldn't be found.
diff --git a/lib/src/runner/browser/default_settings.dart b/lib/src/runner/browser/default_settings.dart
new file mode 100644
index 0000000000000000000000000000000000000000..600092694a64711660ea0d0c4e492ba8538ab90f
--- /dev/null
+++ b/lib/src/runner/browser/default_settings.dart
@@ -0,0 +1,37 @@
+// Copyright (c) 2017, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'dart:collection';
+
+import '../../backend/test_platform.dart';
+import '../executable_settings.dart';
+
+/// Default settings for starting browser executables.
+final defaultSettings = new UnmodifiableMapView({
+ TestPlatform.chrome: new ExecutableSettings(
+ linuxExecutable: 'google-chrome',
+ macOSExecutable:
+ '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
+ windowsExecutable: r'Google\Chrome\Application\chrome.exe'),
+ TestPlatform.contentShell: new ExecutableSettings(
+ linuxExecutable: 'content_shell',
+ macOSExecutable: 'content_shell',
+ windowsExecutable: 'content_shell.exe'),
+ TestPlatform.dartium: new ExecutableSettings(
+ linuxExecutable: 'dartium',
+ macOSExecutable: 'dartium',
+ windowsExecutable: 'dartium.exe'),
+ TestPlatform.firefox: new ExecutableSettings(
+ linuxExecutable: 'firefox',
+ macOSExecutable: '/Applications/Firefox.app/Contents/MacOS/firefox-bin',
+ windowsExecutable: r'Mozilla Firefox\firefox.exe'),
+ TestPlatform.internetExplorer: new ExecutableSettings(
+ windowsExecutable: r'Internet Explorer\iexplore.exe'),
+ TestPlatform.phantomJS: new ExecutableSettings(
+ linuxExecutable: 'phantomjs',
+ macOSExecutable: 'phantomjs',
+ windowsExecutable: 'phantomjs.exe'),
+ TestPlatform.safari: new ExecutableSettings(
+ macOSExecutable: '/Applications/Safari.app/Contents/MacOS/Safari')
+});
diff --git a/lib/src/runner/browser/firefox.dart b/lib/src/runner/browser/firefox.dart
index 5c9a016a3f7285e22e867ba113082a3b9aaca660..4e3ee8e1706e5f5ef3dd3ab100d9a56ad0dfedd3 100644
--- a/lib/src/runner/browser/firefox.dart
+++ b/lib/src/runner/browser/firefox.dart
@@ -7,8 +7,11 @@ import 'dart:io';
import 'package:path/path.dart' as p;
+import '../../backend/test_platform.dart';
import '../../util/io.dart';
+import '../executable_settings.dart';
import 'browser.dart';
+import 'default_settings.dart';
final _preferences = '''
user_pref("browser.shell.checkDefaultBrowser", false);
@@ -26,22 +29,20 @@ user_pref("dom.max_script_run_time", 0);
class Firefox extends Browser {
final name = "Firefox";
- Firefox(url, {String executable})
- : super(() => _startBrowser(url, executable));
+ Firefox(url, {ExecutableSettings settings})
+ : super(() => _startBrowser(url, settings));
/// Starts a new instance of Firefox open to the given [url], which may be a
/// [Uri] or a [String].
- ///
- /// If [executable] is passed, it's used as the Firefox executable.
- /// Otherwise the default executable name for the current OS will be used.
- static Future<Process> _startBrowser(url, [String executable]) async {
- if (executable == null) executable = _defaultExecutable();
-
+ static Future<Process> _startBrowser(url, ExecutableSettings settings) async {
+ settings ??= defaultSettings[TestPlatform.dartium];
var dir = createTempDir();
new File(p.join(dir, 'prefs.js')).writeAsStringSync(_preferences);
var process = await Process.start(
- executable, ["--profile", "$dir", url.toString(), "--no-remote"],
+ settings.executable,
+ ["--profile", "$dir", url.toString(), "--no-remote"]
+ ..addAll(settings.arguments),
environment: {"MOZ_CRASHREPORTER_DISABLE": "1"});
process.exitCode
@@ -49,31 +50,4 @@ class Firefox extends Browser {
return process;
}
-
- /// Return the default executable for the current operating system.
- static String _defaultExecutable() {
- if (Platform.isMacOS) {
- return '/Applications/Firefox.app/Contents/MacOS/firefox-bin';
- }
- if (!Platform.isWindows) return 'firefox';
-
- // Firefox could be installed in several places on Windows. The only way to
- // find it is to check.
- var prefixes = [
- Platform.environment['PROGRAMFILES'],
- Platform.environment['PROGRAMFILES(X86)']
- ];
- var suffix = r'Mozilla Firefox\firefox.exe';
-
- for (var prefix in prefixes) {
- if (prefix == null) continue;
-
- var path = p.join(prefix, suffix);
- if (new File(p.join(prefix, suffix)).existsSync()) return path;
- }
-
- // Fall back on looking it up on the path. This probably won't work, but at
- // least it will fail with a useful error message.
- return "firefox.exe";
- }
}
diff --git a/lib/src/runner/browser/internet_explorer.dart b/lib/src/runner/browser/internet_explorer.dart
index 8d20e29c6be1d1aa976079f67ad687ec1fb31cda..eee452478647b4cf61f1f7368b612c57da24d40c 100644
--- a/lib/src/runner/browser/internet_explorer.dart
+++ b/lib/src/runner/browser/internet_explorer.dart
@@ -5,9 +5,10 @@
import 'dart:async';
import 'dart:io';
-import 'package:path/path.dart' as p;
-
+import '../../backend/test_platform.dart';
+import '../executable_settings.dart';
import 'browser.dart';
+import 'default_settings.dart';
/// A class for running an instance of Internet Explorer.
///
@@ -15,39 +16,15 @@ import 'browser.dart';
class InternetExplorer extends Browser {
final name = "Internet Explorer";
- InternetExplorer(url, {String executable})
- : super(() => _startBrowser(url, executable));
+ InternetExplorer(url, {ExecutableSettings settings})
+ : super(() => _startBrowser(url, settings));
/// Starts a new instance of Internet Explorer open to the given [url], which
/// may be a [Uri] or a [String].
- ///
- /// If [executable] is passed, it's used as the Internet Explorer executable.
- /// Otherwise the default executable name for the current OS will be used.
- static Future<Process> _startBrowser(url, [String executable]) {
- if (executable == null) executable = _defaultExecutable();
- return Process.start(executable, ['-extoff', url.toString()]);
- }
-
- /// Return the default executable for the current operating system.
- static String _defaultExecutable() {
- // Internet Explorer could be installed in several places on Windows. The
- // only way to find it is to check.
- var prefixes = [
- Platform.environment['PROGRAMW6432'],
- Platform.environment['PROGRAMFILES'],
- Platform.environment['PROGRAMFILES(X86)']
- ];
- var suffix = r'Internet Explorer\iexplore.exe';
-
- for (var prefix in prefixes) {
- if (prefix == null) continue;
-
- var path = p.join(prefix, suffix);
- if (new File(p.join(prefix, suffix)).existsSync()) return path;
- }
+ static Future<Process> _startBrowser(url, ExecutableSettings settings) {
+ settings ??= defaultSettings[TestPlatform.dartium];
- // Fall back on looking it up on the path. This probably won't work, but at
- // least it will fail with a useful error message.
- return "iexplore.exe";
+ return Process.start(settings.executable,
+ ['-extoff', url.toString()]..addAll(settings.arguments));
}
}
diff --git a/lib/src/runner/browser/phantom_js.dart b/lib/src/runner/browser/phantom_js.dart
index 640841de2aba488acb951ca07093198a395d7f17..25e9b99ede14041d0307e060309cc3ce9e112039 100644
--- a/lib/src/runner/browser/phantom_js.dart
+++ b/lib/src/runner/browser/phantom_js.dart
@@ -7,10 +7,13 @@ import 'dart:io';
import 'package:path/path.dart' as p;
+import '../../backend/test_platform.dart';
import '../../util/exit_codes.dart' as exit_codes;
import '../../util/io.dart';
import '../application_exception.dart';
+import '../executable_settings.dart';
import 'browser.dart';
+import 'default_settings.dart';
/// The PhantomJS script that opens the host page.
final _script = """
@@ -39,26 +42,23 @@ class PhantomJS extends Browser {
final Future<Uri> remoteDebuggerUrl;
- factory PhantomJS(url, {String executable, bool debug: false}) {
+ factory PhantomJS(url, {ExecutableSettings settings, bool debug: false}) {
+ settings ??= defaultSettings[TestPlatform.phantomJS];
var remoteDebuggerCompleter = new Completer<Uri>.sync();
return new PhantomJS._(() async {
- if (executable == null) {
- executable = Platform.isWindows ? "phantomjs.exe" : "phantomjs";
- }
-
var dir = createTempDir();
var script = p.join(dir, "script.js");
new File(script).writeAsStringSync(_script);
var port = debug ? await getUnsafeUnusedPort() : null;
- var args = <String>[];
+ var args = settings.arguments.toList();
if (debug) {
args.addAll(
["--remote-debugger-port=$port", "--remote-debugger-autorun=yes"]);
}
args.addAll([script, url.toString()]);
- var process = await Process.start(executable, args);
+ var process = await Process.start(settings.executable, args);
// PhantomJS synchronously emits standard output, which means that if we
// don't drain its stdout stream it can deadlock.
diff --git a/lib/src/runner/browser/platform.dart b/lib/src/runner/browser/platform.dart
index 2268d2e4a61fa5f62c66decb537f885596d46e39..4b040c48e7bc67cee9e670253fbc4cf1a990ce86 100644
--- a/lib/src/runner/browser/platform.dart
+++ b/lib/src/runner/browser/platform.dart
@@ -19,6 +19,7 @@ import 'package:shelf_web_socket/shelf_web_socket.dart';
import 'package:shelf_packages_handler/shelf_packages_handler.dart';
import 'package:stream_channel/stream_channel.dart';
import 'package:web_socket_channel/web_socket_channel.dart';
+import 'package:yaml/yaml.dart';
import '../../backend/test_platform.dart';
import '../../util/io.dart';
@@ -29,13 +30,17 @@ import '../../utils.dart';
import '../compiler_pool.dart';
import '../configuration.dart';
import '../configuration/suite.dart';
+import '../executable_settings.dart';
import '../load_exception.dart';
+import '../plugin/customizable_platform.dart';
import '../plugin/platform.dart';
import '../runner_suite.dart';
import 'browser_manager.dart';
+import 'default_settings.dart';
import 'polymer.dart';
-class BrowserPlatform extends PlatformPlugin {
+class BrowserPlatform extends PlatformPlugin
+ implements CustomizablePlatform<ExecutableSettings> {
/// Starts the server.
///
/// [root] is the root directory that the server should serve. It defaults to
@@ -100,7 +105,13 @@ class BrowserPlatform extends PlatformPlugin {
/// [BrowserManager]s for those browsers, or `null` if they failed to load.
///
/// This should only be accessed through [_browserManagerFor].
- final _browserManagers = new Map<TestPlatform, Future<BrowserManager>>();
+ final _browserManagers = <TestPlatform, Future<BrowserManager>>{};
+
+ /// Settings for invoking each browser.
+ ///
+ /// This starts out with the default settings, which may be overridden by user settings.
+ final _browserSettings =
+ new Map<TestPlatform, ExecutableSettings>.from(defaultSettings);
/// A cascade of handlers for suites' precompiled paths.
///
@@ -198,13 +209,27 @@ class BrowserPlatform extends PlatformPlugin {
return new shelf.Response.notFound('Not found.');
}
+ ExecutableSettings parsePlatformSettings(YamlMap settings) =>
+ new ExecutableSettings.parse(settings);
+
+ ExecutableSettings mergePlatformSettings(
+ ExecutableSettings settings1, ExecutableSettings settings2) =>
+ settings1.merge(settings2);
+
+ void customizePlatform(TestPlatform platform, ExecutableSettings settings) {
+ var oldSettings =
+ _browserSettings[platform] ?? _browserSettings[platform.root];
+ if (oldSettings != null) settings = oldSettings.merge(settings);
+ _browserSettings[platform] = settings;
+ }
+
/// Loads the test suite at [path] on the browser [browser].
///
/// This will start a browser to load the suite if one isn't already running.
/// Throws an [ArgumentError] if [browser] isn't a browser platform.
- Future<RunnerSuite> load(
- String path, TestPlatform browser, SuiteConfiguration suiteConfig) async {
- assert(suiteConfig.platforms.contains(browser));
+ Future<RunnerSuite> load(String path, TestPlatform browser,
+ SuiteConfiguration suiteConfig, Object message) async {
+ assert(suiteConfig.platforms.contains(browser.identifier));
if (!browser.isBrowser) {
throw new ArgumentError("$browser is not a browser.");
@@ -281,7 +306,7 @@ class BrowserPlatform extends PlatformPlugin {
var browserManager = await _browserManagerFor(browser);
if (_closed || browserManager == null) return null;
- var suite = await browserManager.load(path, suiteUrl, suiteConfig,
+ var suite = await browserManager.load(path, suiteUrl, suiteConfig, message,
mapper: browser.isJS ? _mappers[path] : null);
if (_closed) return null;
return suite;
@@ -427,7 +452,8 @@ class BrowserPlatform extends PlatformPlugin {
'debug': _config.pauseAfterLoad.toString()
});
- var future = BrowserManager.start(platform, hostUrl, completer.future,
+ var future = BrowserManager.start(
+ platform, hostUrl, completer.future, _browserSettings[platform],
debug: _config.pauseAfterLoad);
// Store null values for browsers that error out so we know not to load them
diff --git a/lib/src/runner/browser/safari.dart b/lib/src/runner/browser/safari.dart
index ac1b7d5e7324ae66abd6ef0371cd43d0e994ed0b..2b35749c0df844bbdc651be9f004a6b70e9e76b2 100644
--- a/lib/src/runner/browser/safari.dart
+++ b/lib/src/runner/browser/safari.dart
@@ -8,8 +8,11 @@ import 'dart:io';
import 'package:path/path.dart' as p;
+import '../../backend/test_platform.dart';
import '../../util/io.dart';
+import '../executable_settings.dart';
import 'browser.dart';
+import 'default_settings.dart';
/// A class for running an instance of Safari.
///
@@ -17,19 +20,13 @@ import 'browser.dart';
class Safari extends Browser {
final name = "Safari";
- Safari(url, {String executable})
- : super(() => _startBrowser(url, executable));
+ Safari(url, {ExecutableSettings settings})
+ : super(() => _startBrowser(url, settings));
/// Starts a new instance of Safari open to the given [url], which may be a
/// [Uri] or a [String].
- ///
- /// If [executable] is passed, it's used as the content shell executable.
- /// Otherwise the default executable name for the current OS will be used.
- static Future<Process> _startBrowser(url, [String executable]) async {
- if (executable == null) {
- executable = '/Applications/Safari.app/Contents/MacOS/Safari';
- }
-
+ static Future<Process> _startBrowser(url, ExecutableSettings settings) async {
+ settings ??= defaultSettings[TestPlatform.safari];
var dir = createTempDir();
// Safari will only open files (not general URLs) via the command-line
@@ -39,7 +36,8 @@ class Safari extends Browser {
new File(redirect).writeAsStringSync(
"<script>location = " + JSON.encode(url.toString()) + "</script>");
- var process = await Process.start(executable, [redirect]);
+ var process = await Process.start(
+ settings.executable, settings.arguments.toList()..add(redirect));
process.exitCode
.then((_) => new Directory(dir).deleteSync(recursive: true));
diff --git a/lib/src/runner/configuration.dart b/lib/src/runner/configuration.dart
index e6a02b985da17b65d817bc199cecfb2293febdbd..873ef6a6159ccb719d2dcb2cbe3b50fb4c5572e5 100644
--- a/lib/src/runner/configuration.dart
+++ b/lib/src/runner/configuration.dart
@@ -8,13 +8,18 @@ import 'package:boolean_selector/boolean_selector.dart';
import 'package:collection/collection.dart';
import 'package:glob/glob.dart';
import 'package:path/path.dart' as p;
+import 'package:source_span/source_span.dart';
import '../backend/platform_selector.dart';
import '../backend/test_platform.dart';
import '../frontend/timeout.dart';
import '../util/io.dart';
+import '../utils.dart';
import 'configuration/args.dart' as args;
+import 'configuration/custom_platform.dart';
import 'configuration/load.dart';
+import 'configuration/platform_selection.dart';
+import 'configuration/platform_settings.dart';
import 'configuration/reporters.dart';
import 'configuration/suite.dart';
import 'configuration/values.dart';
@@ -169,6 +174,12 @@ class Configuration {
Set<String> _knownPresets;
+ /// Built-in platforms whose settings are overridden by the user.
+ final Map<String, PlatformSettings> overridePlatforms;
+
+ /// Platforms defined by the user in terms of existing platforms.
+ final Map<String, CustomPlatform> definePlatforms;
+
/// The default suite-level configuration.
final SuiteConfiguration suiteDefaults;
@@ -212,6 +223,8 @@ class Configuration {
Glob filename,
Iterable<String> chosenPresets,
Map<String, Configuration> presets,
+ Map<String, PlatformSettings> overridePlatforms,
+ Map<String, CustomPlatform> definePlatforms,
bool noRetry,
// Suite-level configuration
@@ -220,7 +233,7 @@ class Configuration {
Iterable<String> dart2jsArgs,
String precompiledPath,
Iterable<Pattern> patterns,
- Iterable<TestPlatform> platforms,
+ Iterable<PlatformSelection> platforms,
BooleanSelector includeTags,
BooleanSelector excludeTags,
Map<BooleanSelector, SuiteConfiguration> tags,
@@ -254,6 +267,8 @@ class Configuration {
filename: filename,
chosenPresets: chosenPresetSet,
presets: _withChosenPresets(presets, chosenPresetSet),
+ overridePlatforms: overridePlatforms,
+ definePlatforms: definePlatforms,
noRetry: noRetry,
suiteDefaults: new SuiteConfiguration(
jsTrace: jsTrace,
@@ -308,6 +323,8 @@ class Configuration {
Glob filename,
Iterable<String> chosenPresets,
Map<String, Configuration> presets,
+ Map<String, PlatformSettings> overridePlatforms,
+ Map<String, CustomPlatform> definePlatforms,
bool noRetry,
SuiteConfiguration suiteDefaults})
: _help = help,
@@ -328,6 +345,8 @@ class Configuration {
chosenPresets =
new UnmodifiableSetView(chosenPresets?.toSet() ?? new Set()),
presets = _map(presets),
+ overridePlatforms = _map(overridePlatforms),
+ definePlatforms = _map(definePlatforms),
_noRetry = noRetry,
suiteDefaults = pauseAfterLoad == true
? suiteDefaults?.change(timeout: Timeout.none) ??
@@ -384,6 +403,26 @@ class Configuration {
/// asynchronous callbacks transitively created by [body].
T asCurrent<T>(T body()) => runZoned(body, zoneValues: {_currentKey: this});
+ /// Throws a [FormatException] if [this] refers to any undefined platforms.
+ void validatePlatforms(List<TestPlatform> allPlatforms) {
+ // We don't need to verify [customPlatforms] here because those platforms
+ // already need to be verified and resolved to create [allPlatforms].
+
+ for (var settings in overridePlatforms.values) {
+ if (!allPlatforms
+ .any((platform) => platform.identifier == settings.identifier)) {
+ throw new SourceSpanFormatException(
+ 'Unknown platform "${settings.identifier}".',
+ settings.identifierSpan);
+ }
+ }
+
+ suiteDefaults.validatePlatforms(allPlatforms);
+ for (var config in presets.values) {
+ config.validatePlatforms(allPlatforms);
+ }
+ }
+
/// Merges this with [other].
///
/// For most fields, if both configurations have values set, [other]'s value
@@ -427,6 +466,14 @@ class Configuration {
filename: other._filename ?? _filename,
chosenPresets: chosenPresets.union(other.chosenPresets),
presets: _mergeConfigMaps(presets, other.presets),
+ overridePlatforms: mergeUnmodifiableMaps(
+ overridePlatforms, other.overridePlatforms,
+ value: (settings1, settings2) => new PlatformSettings(
+ settings1.identifier,
+ settings1.identifierSpan,
+ settings1.settings.toList()..addAll(settings2.settings))),
+ definePlatforms:
+ mergeUnmodifiableMaps(definePlatforms, other.definePlatforms),
noRetry: other._noRetry ?? _noRetry,
suiteDefaults: suiteDefaults.merge(other.suiteDefaults));
result = result._resolvePresets();
@@ -459,6 +506,8 @@ class Configuration {
Glob filename,
Iterable<String> chosenPresets,
Map<String, Configuration> presets,
+ Map<String, PlatformSettings> overridePlatforms,
+ Map<String, CustomPlatform> definePlatforms,
bool noRetry,
// Suite-level configuration
@@ -467,7 +516,7 @@ class Configuration {
Iterable<String> dart2jsArgs,
String precompiledPath,
Iterable<Pattern> patterns,
- Iterable<TestPlatform> platforms,
+ Iterable<PlatformSelection> platforms,
BooleanSelector includeTags,
BooleanSelector excludeTags,
Map<BooleanSelector, SuiteConfiguration> tags,
@@ -499,6 +548,8 @@ class Configuration {
filename: filename ?? _filename,
chosenPresets: chosenPresets ?? this.chosenPresets,
presets: presets ?? this.presets,
+ overridePlatforms: overridePlatforms ?? this.overridePlatforms,
+ definePlatforms: definePlatforms ?? this.definePlatforms,
noRetry: noRetry ?? _noRetry,
suiteDefaults: suiteDefaults.change(
jsTrace: jsTrace,
diff --git a/lib/src/runner/configuration/args.dart b/lib/src/runner/configuration/args.dart
index 83b0b05f261283467e64ddfffab0f750677df116..a3a84e7f8e1a28d79c3827654c553cd62b0283e6 100644
--- a/lib/src/runner/configuration/args.dart
+++ b/lib/src/runner/configuration/args.dart
@@ -10,6 +10,7 @@ import 'package:boolean_selector/boolean_selector.dart';
import '../../backend/test_platform.dart';
import '../../frontend/timeout.dart';
import '../configuration.dart';
+import 'platform_selection.dart';
import 'reporters.dart';
import 'values.dart';
@@ -17,7 +18,7 @@ import 'values.dart';
final ArgParser _parser = (() {
var parser = new ArgParser(allowTrailingOptions: true);
- var allPlatforms = TestPlatform.all.toList();
+ var allPlatforms = TestPlatform.builtIn.toList()..remove(TestPlatform.vm);
if (!Platform.isMacOS) allPlatforms.remove(TestPlatform.safari);
if (!Platform.isWindows) allPlatforms.remove(TestPlatform.internetExplorer);
@@ -62,9 +63,9 @@ final ArgParser _parser = (() {
parser.addSeparator("======== Running Tests");
parser.addOption("platform",
abbr: 'p',
- help: 'The platform(s) on which to run the tests.',
- defaultsTo: 'vm',
- allowed: allPlatforms.map((platform) => platform.identifier).toList(),
+ help: 'The platform(s) on which to run the tests.\n'
+ '[vm (default), '
+ '${allPlatforms.map((platform) => platform.identifier).join(", ")}]',
allowMultiple: true);
parser.addOption("preset",
abbr: 'P',
@@ -220,8 +221,9 @@ class _Parser {
totalShards: totalShards,
timeout: _parseOption('timeout', (value) => new Timeout.parse(value)),
patterns: patterns,
- platforms:
- (_ifParsed('platform') as List<String>)?.map(TestPlatform.find),
+ platforms: (_ifParsed('platform') as List<String>)
+ ?.map((platform) => new PlatformSelection(platform))
+ ?.toList(),
runSkipped: _ifParsed('run-skipped'),
chosenPresets: _ifParsed('preset') as List<String>,
paths: _options.rest.isEmpty ? null : _options.rest,
diff --git a/lib/src/runner/configuration/custom_platform.dart b/lib/src/runner/configuration/custom_platform.dart
new file mode 100644
index 0000000000000000000000000000000000000000..c4372de1414642e70b61f2fc6dc9667489a5d434
--- /dev/null
+++ b/lib/src/runner/configuration/custom_platform.dart
@@ -0,0 +1,34 @@
+// Copyright (c) 2017, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'package:source_span/source_span.dart';
+import 'package:yaml/yaml.dart';
+
+/// A user-defined test platform, based on an existing platform but with
+/// different configuration.
+class CustomPlatform {
+ /// The human-friendly name of the platform.
+ final String name;
+
+ /// The location that [name] was defined in the configuration file.
+ final SourceSpan nameSpan;
+
+ /// The identifier used to look up the platform.
+ final String identifier;
+
+ /// The location that [identifier] was defined in the configuration file.
+ final SourceSpan identifierSpan;
+
+ /// The identifier of the platform that this extends.
+ final String parent;
+
+ /// The location that [parent] was defined in the configuration file.
+ final SourceSpan parentSpan;
+
+ /// The user's settings for this platform.
+ final YamlMap settings;
+
+ CustomPlatform(this.name, this.nameSpan, this.identifier, this.identifierSpan,
+ this.parent, this.parentSpan, this.settings);
+}
diff --git a/lib/src/runner/configuration/load.dart b/lib/src/runner/configuration/load.dart
index 1fdfc002b1d64b1ef3ee5a67ee392eee7cbab945..cd9acf99c9ed9667514e5ac63450bf4ea2985c6e 100644
--- a/lib/src/runner/configuration/load.dart
+++ b/lib/src/runner/configuration/load.dart
@@ -13,12 +13,14 @@ import 'package:yaml/yaml.dart';
import '../../backend/operating_system.dart';
import '../../backend/platform_selector.dart';
-import '../../backend/test_platform.dart';
import '../../frontend/timeout.dart';
import '../../util/io.dart';
import '../../utils.dart';
import '../configuration.dart';
import '../configuration/suite.dart';
+import 'custom_platform.dart';
+import 'platform_selection.dart';
+import 'platform_settings.dart';
import 'reporters.dart';
/// A regular expression matching a Dart identifier.
@@ -94,7 +96,7 @@ class _ConfigurationLoader {
var onPlatform = _getMap("on_platform",
key: (keyNode) => _parseNode(keyNode, "on_platform key",
- (value) => new PlatformSelector.parse(value)),
+ (value) => new PlatformSelector.parse(value, keyNode.span)),
value: (valueNode) =>
_nestedConfig(valueNode, "on_platform value", runnerConfig: false));
@@ -155,8 +157,7 @@ class _ConfigurationLoader {
skip = true;
}
- var testOn =
- _parseValue("test_on", (value) => new PlatformSelector.parse(value));
+ var testOn = _parsePlatformSelector("test_on");
var addTags = _getList(
"add_tags", (tagNode) => _parseIdentifierLike(tagNode, "Tag name"));
@@ -193,6 +194,7 @@ class _ConfigurationLoader {
_disallow("plain_names");
_disallow("platforms");
_disallow("add_presets");
+ _disallow("override_platforms");
return Configuration.empty;
}
@@ -206,27 +208,50 @@ class _ConfigurationLoader {
var concurrency = _getInt("concurrency");
- var allPlatformIdentifiers =
- TestPlatform.all.map((platform) => platform.identifier).toSet();
- var platforms = _getList("platforms", (platformNode) {
- _validate(platformNode, "Platforms must be strings.",
- (value) => value is String);
- _validate(platformNode, 'Unknown platform "${platformNode.value}".',
- allPlatformIdentifiers.contains);
-
- return TestPlatform.find(platformNode.value);
- });
+ var platforms = _getList(
+ "platforms",
+ (platformNode) => new PlatformSelection(
+ _parseIdentifierLike(platformNode, "Platform name"),
+ platformNode.span));
var chosenPresets = _getList("add_presets",
(presetNode) => _parseIdentifierLike(presetNode, "Preset name"));
+ var overridePlatforms = _loadOverridePlatforms();
+
return new Configuration(
pauseAfterLoad: pauseAfterLoad,
runSkipped: runSkipped,
reporter: reporter,
concurrency: concurrency,
platforms: platforms,
- chosenPresets: chosenPresets);
+ chosenPresets: chosenPresets,
+ overridePlatforms: overridePlatforms);
+ }
+
+ /// Loads the `override_platforms` field.
+ Map<String, PlatformSettings> _loadOverridePlatforms() {
+ var platformsNode =
+ _getNode("override_platforms", "map", (value) => value is Map)
+ as YamlMap;
+ if (platformsNode == null) return const {};
+
+ var platforms = <String, PlatformSettings>{};
+ platformsNode.nodes.forEach((identifierNode, valueNode) {
+ var identifier =
+ _parseIdentifierLike(identifierNode, "Platform identifier");
+
+ _validate(valueNode, "Platform definition must be a map.",
+ (value) => value is Map);
+ var map = valueNode as YamlMap;
+
+ var settings = _expect(map, "settings");
+ _validate(settings, "Must be a map.", (value) => value is Map);
+
+ platforms[identifier] = new PlatformSettings(
+ identifier, identifierNode.span, [settings as YamlMap]);
+ });
+ return platforms;
}
/// Loads runner configuration that's not allowed in the global configuration
@@ -243,6 +268,7 @@ class _ConfigurationLoader {
_disallow("filename");
_disallow("include_tags");
_disallow("exclude_tags");
+ _disallow("define_platforms");
return Configuration.empty;
}
@@ -270,13 +296,16 @@ class _ConfigurationLoader {
var includeTags = _parseBooleanSelector("include_tags");
var excludeTags = _parseBooleanSelector("exclude_tags");
+ var definePlatforms = _loadDefinePlatforms();
+
return new Configuration(
pubServePort: pubServePort,
patterns: patterns,
paths: paths,
filename: filename,
includeTags: includeTags,
- excludeTags: excludeTags);
+ excludeTags: excludeTags,
+ definePlatforms: definePlatforms);
}
/// Returns a map representation of the `fold_stack_frames` configuration.
@@ -317,6 +346,43 @@ class _ConfigurationLoader {
});
}
+ /// Loads the `define_platforms` field.
+ Map<String, CustomPlatform> _loadDefinePlatforms() {
+ var platformsNode =
+ _getNode("define_platforms", "map", (value) => value is Map) as YamlMap;
+ if (platformsNode == null) return const {};
+
+ var platforms = <String, CustomPlatform>{};
+ platformsNode.nodes.forEach((identifierNode, valueNode) {
+ var identifier =
+ _parseIdentifierLike(identifierNode, "Platform identifier");
+
+ _validate(valueNode, "Platform definition must be a map.",
+ (value) => value is Map);
+ var map = valueNode as YamlMap;
+
+ var nameNode = _expect(map, "name");
+ _validate(nameNode, "Must be a string.", (value) => value is String);
+ var name = nameNode.value as String;
+
+ var parentNode = _expect(map, "extends");
+ var parent = _parseIdentifierLike(parentNode, "Platform parent");
+
+ var settings = _expect(map, "settings");
+ _validate(settings, "Must be a map.", (value) => value is Map);
+
+ platforms[identifier] = new CustomPlatform(
+ name,
+ nameNode.span,
+ identifier,
+ identifierNode.span,
+ parent,
+ parentNode.span,
+ settings as YamlMap);
+ });
+ return platforms;
+ }
+
/// Throws an exception with [message] if [test] returns `false` when passed
/// [node]'s value.
void _validate(YamlNode node, String message, bool test(value)) {
@@ -411,6 +477,14 @@ class _ConfigurationLoader {
BooleanSelector _parseBooleanSelector(String name) =>
_parseValue(name, (value) => new BooleanSelector.parse(value));
+ /// Parses [node]'s value as a platform selector.
+ PlatformSelector _parsePlatformSelector(String field) {
+ var node = _document.nodes[field];
+ if (node == null) return null;
+ return _parseNode(
+ node, field, (value) => new PlatformSelector.parse(value, node.span));
+ }
+
/// Asserts that [node] is a string, passes its value to [parse], and returns
/// the result.
///
@@ -485,6 +559,15 @@ class _ConfigurationLoader {
}
}
+ /// Asserts that [map] has a field named [field] and returns it.
+ YamlNode _expect(YamlMap map, String field) {
+ var node = map.nodes[field];
+ if (node != null) return node;
+
+ throw new SourceSpanFormatException(
+ 'Missing required field "$field".', map.span, _source);
+ }
+
/// Throws an error if a field named [field] exists at this level.
void _disallow(String field) {
if (!_document.containsKey(field)) return;
diff --git a/lib/src/runner/configuration/platform_selection.dart b/lib/src/runner/configuration/platform_selection.dart
new file mode 100644
index 0000000000000000000000000000000000000000..386ecfdd990e300726262207ff28f87b8e1dd6f3
--- /dev/null
+++ b/lib/src/runner/configuration/platform_selection.dart
@@ -0,0 +1,22 @@
+// Copyright (c) 2017, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'package:source_span/source_span.dart';
+
+/// A platform on which the user has chosen to run tests.
+class PlatformSelection {
+ /// The name of the platform.
+ final String name;
+
+ /// The location in the configuration file of this platform string, or `null`
+ /// if it was defined outside a configuration file (for example, on the
+ /// command line).
+ final SourceSpan span;
+
+ PlatformSelection(this.name, [this.span]);
+
+ bool operator ==(other) => other is PlatformSelection && other.name == name;
+
+ int get hashCode => name.hashCode;
+}
diff --git a/lib/src/runner/configuration/platform_settings.dart b/lib/src/runner/configuration/platform_settings.dart
new file mode 100644
index 0000000000000000000000000000000000000000..20390156467e5354bd553d6959441e24a01feec9
--- /dev/null
+++ b/lib/src/runner/configuration/platform_settings.dart
@@ -0,0 +1,26 @@
+// Copyright (c) 2017, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'package:source_span/source_span.dart';
+import 'package:yaml/yaml.dart';
+
+import '../plugin/customizable_platform.dart';
+
+/// User-defined settings for a built-in test platform.
+class PlatformSettings {
+ /// The identifier used to look up the platform being overridden.
+ final String identifier;
+
+ /// The location that [identifier] was defined in the configuration file.
+ final SourceSpan identifierSpan;
+
+ /// The user's settings for this platform.
+ ///
+ /// This is a list of settings, from most global to most specific, that will
+ /// eventually be merged using [CustomizablePlatform.mergePlatformSettings].
+ final List<YamlMap> settings;
+
+ PlatformSettings(this.identifier, this.identifierSpan, List<YamlMap> settings)
+ : settings = new List.unmodifiable(settings);
+}
diff --git a/lib/src/runner/configuration/suite.dart b/lib/src/runner/configuration/suite.dart
index c41f88c459c3404dd4403900f0f8aab984b53fec..6ed34586c9d6e45f31a35e354fc78a0aaaa9eb55 100644
--- a/lib/src/runner/configuration/suite.dart
+++ b/lib/src/runner/configuration/suite.dart
@@ -4,12 +4,14 @@
import 'package:boolean_selector/boolean_selector.dart';
import 'package:collection/collection.dart';
+import 'package:source_span/source_span.dart';
import '../../backend/metadata.dart';
import '../../backend/operating_system.dart';
import '../../backend/platform_selector.dart';
import '../../backend/test_platform.dart';
import '../../frontend/timeout.dart';
+import 'platform_selection.dart';
/// Suite-level configuration.
///
@@ -51,8 +53,10 @@ class SuiteConfiguration {
final Set<Pattern> patterns;
/// The set of platforms on which to run tests.
- List<TestPlatform> get platforms => _platforms ?? const [TestPlatform.vm];
- final List<TestPlatform> _platforms;
+ List<String> get platforms => _platforms == null
+ ? const ["vm"]
+ : new List.unmodifiable(_platforms.map((platform) => platform.name));
+ final List<PlatformSelection> _platforms;
/// Only run tests whose tags match this selector.
///
@@ -124,7 +128,7 @@ class SuiteConfiguration {
Iterable<String> dart2jsArgs,
String precompiledPath,
Iterable<Pattern> patterns,
- Iterable<TestPlatform> platforms,
+ Iterable<PlatformSelection> platforms,
BooleanSelector includeTags,
BooleanSelector excludeTags,
Map<BooleanSelector, SuiteConfiguration> tags,
@@ -172,7 +176,7 @@ class SuiteConfiguration {
Iterable<String> dart2jsArgs,
this.precompiledPath,
Iterable<Pattern> patterns,
- Iterable<TestPlatform> platforms,
+ Iterable<PlatformSelection> platforms,
BooleanSelector includeTags,
BooleanSelector excludeTags,
Map<BooleanSelector, SuiteConfiguration> tags,
@@ -249,7 +253,7 @@ class SuiteConfiguration {
Iterable<String> dart2jsArgs,
String precompiledPath,
Iterable<Pattern> patterns,
- Iterable<TestPlatform> platforms,
+ Iterable<PlatformSelection> platforms,
BooleanSelector includeTags,
BooleanSelector excludeTags,
Map<BooleanSelector, SuiteConfiguration> tags,
@@ -287,6 +291,32 @@ class SuiteConfiguration {
return config._resolveTags();
}
+ /// Throws a [FormatException] if [this] refers to any undefined platforms.
+ void validatePlatforms(List<TestPlatform> allPlatforms) {
+ var validVariables =
+ allPlatforms.map((platform) => platform.identifier).toSet();
+ _metadata.validatePlatformSelectors(validVariables);
+
+ if (_platforms != null) {
+ for (var selection in _platforms) {
+ if (!allPlatforms
+ .any((platform) => platform.identifier == selection.name)) {
+ if (selection.span != null) {
+ throw new SourceSpanFormatException(
+ 'Unknown platform "${selection.name}".', selection.span);
+ } else {
+ throw new FormatException('Unknown platform "${selection.name}".');
+ }
+ }
+ }
+ }
+
+ onPlatform.forEach((selector, config) {
+ selector.validate(validVariables);
+ config.validatePlatforms(allPlatforms);
+ });
+ }
+
/// Returns a copy of [this] with all platform-specific configuration from
/// [onPlatform] resolved.
SuiteConfiguration forPlatform(TestPlatform platform, {OperatingSystem os}) {
diff --git a/lib/src/runner/executable_settings.dart b/lib/src/runner/executable_settings.dart
new file mode 100644
index 0000000000000000000000000000000000000000..069fa39af745f99e424446a1498ade16fb4cb414
--- /dev/null
+++ b/lib/src/runner/executable_settings.dart
@@ -0,0 +1,162 @@
+// Copyright (c) 2017, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'dart:io';
+
+import 'package:io/io.dart';
+import 'package:path/path.dart' as p;
+import 'package:source_span/source_span.dart';
+import 'package:yaml/yaml.dart';
+
+/// User-provided settings for invoking an executable.
+class ExecutableSettings {
+ /// Additional arguments to pass to the executable.
+ final List<String> arguments;
+
+ /// The path to the executable on Linux.
+ ///
+ /// This may be an absolute path or a basename, in which case it will be
+ /// looked up on the system path. It may not be relative.
+ final String _linuxExecutable;
+
+ /// The path to the executable on Mac OS.
+ ///
+ /// This may be an absolute path or a basename, in which case it will be
+ /// looked up on the system path. It may not be relative.
+ final String _macOSExecutable;
+
+ /// The path to the executable on Windows.
+ ///
+ /// This may be an absolute path; a basename, in which case it will be looked
+ /// up on the system path; or a relative path, in which case it will be looked
+ /// up relative to the paths in the `LOCALAPPDATA`, `PROGRAMFILES`, and
+ /// `PROGRAMFILES(X64)` environment variables.
+ final String _windowsExecutable;
+
+ /// The path to the executable for the current operating system.
+ String get executable {
+ if (Platform.isMacOS) return _macOSExecutable;
+ if (!Platform.isWindows) return _linuxExecutable;
+ if (p.isAbsolute(_windowsExecutable)) return _windowsExecutable;
+ if (p.basename(_windowsExecutable) == _windowsExecutable) {
+ return _windowsExecutable;
+ }
+
+ var prefixes = [
+ Platform.environment['LOCALAPPDATA'],
+ Platform.environment['PROGRAMFILES'],
+ Platform.environment['PROGRAMFILES(X86)']
+ ];
+
+ for (var prefix in prefixes) {
+ if (prefix == null) continue;
+
+ var path = p.join(prefix, _windowsExecutable);
+ if (new File(path).existsSync()) return path;
+ }
+
+ // If we can't find a path that works, return one that doesn't. This will
+ // cause an "executable not found" error to surface.
+ return p.join(
+ prefixes.firstWhere((prefix) => prefix != null, orElse: () => '.'),
+ _windowsExecutable);
+ }
+
+ /// Parses settings from a user-provided YAML mapping.
+ factory ExecutableSettings.parse(YamlMap settings) {
+ List<String> arguments;
+ var argumentsNode = settings.nodes["arguments"];
+ if (argumentsNode != null) {
+ if (argumentsNode.value is String) {
+ try {
+ arguments = shellSplit(argumentsNode.value);
+ } on FormatException catch (error) {
+ throw new SourceSpanFormatException(
+ error.message, argumentsNode.span);
+ }
+ } else {
+ throw new SourceSpanFormatException(
+ "Must be a string.", argumentsNode.span);
+ }
+ }
+
+ String linuxExecutable;
+ String macOSExecutable;
+ String windowsExecutable;
+ var executableNode = settings.nodes["executable"];
+ if (executableNode != null) {
+ if (executableNode.value is String) {
+ // Don't check this on Windows because people may want to set relative
+ // paths in their global config.
+ if (!Platform.isWindows) _assertNotRelative(executableNode);
+
+ linuxExecutable = executableNode.value;
+ macOSExecutable = executableNode.value;
+ windowsExecutable = executableNode.value;
+ } else if (executableNode is YamlMap) {
+ linuxExecutable = _getExecutable(executableNode.nodes["linux"]);
+ macOSExecutable = _getExecutable(executableNode.nodes["mac_os"]);
+ windowsExecutable = _getExecutable(executableNode.nodes["windows"],
+ allowRelative: true);
+ } else {
+ throw new SourceSpanFormatException(
+ "Must be a map or a string.", executableNode.span);
+ }
+ }
+
+ return new ExecutableSettings(
+ arguments: arguments,
+ linuxExecutable: linuxExecutable,
+ macOSExecutable: macOSExecutable,
+ windowsExecutable: windowsExecutable);
+ }
+
+ /// Asserts that [executableNode] is a string or `null` and returns it.
+ ///
+ /// If [allowRelative] is `false` (the default), asserts that the value isn't
+ /// a relative path.
+ static String _getExecutable(YamlNode executableNode,
+ {bool allowRelative: false}) {
+ if (executableNode == null || executableNode.value == null) return null;
+ if (executableNode.value is! String) {
+ throw new SourceSpanFormatException(
+ "Must be a string.", executableNode.span);
+ }
+ if (!allowRelative) _assertNotRelative(executableNode);
+ return executableNode.value;
+ }
+
+ /// Throws a [SourceSpanFormatException] if [executableNode]'s value is a
+ /// relative POSIX path that's not just a plain basename.
+ ///
+ /// We loop up basenames on the PATH and we can resolve absolute paths, but we
+ /// have no way of interpreting relative paths.
+ static void _assertNotRelative(YamlScalar executableNode) {
+ var executable = executableNode.value as String;
+ if (!p.posix.isRelative(executable)) return;
+ if (p.posix.basename(executable) == executable) return;
+
+ throw new SourceSpanFormatException(
+ "Linux and Mac OS executables may not be relative paths.",
+ executableNode.span);
+ }
+
+ ExecutableSettings(
+ {Iterable<String> arguments,
+ String linuxExecutable,
+ String macOSExecutable,
+ String windowsExecutable})
+ : arguments =
+ arguments == null ? const [] : new List.unmodifiable(arguments),
+ _linuxExecutable = linuxExecutable,
+ _macOSExecutable = macOSExecutable,
+ _windowsExecutable = windowsExecutable;
+
+ /// Merges [this] with [other], with [other]'s settings taking priority.
+ ExecutableSettings merge(ExecutableSettings other) => new ExecutableSettings(
+ arguments: arguments.toList()..addAll(other.arguments),
+ linuxExecutable: other._linuxExecutable ?? _linuxExecutable,
+ macOSExecutable: other._macOSExecutable ?? _macOSExecutable,
+ windowsExecutable: other._windowsExecutable ?? _windowsExecutable);
+}
diff --git a/lib/src/runner/loader.dart b/lib/src/runner/loader.dart
index dc654f6295a6ca39e2fe9c7abd83ebaa5b650606..5f0e8ae530806a4421160f222af732aa93587677 100644
--- a/lib/src/runner/loader.dart
+++ b/lib/src/runner/loader.dart
@@ -8,6 +8,8 @@ import 'dart:io';
import 'package:analyzer/analyzer.dart' hide Configuration;
import 'package:async/async.dart';
import 'package:path/path.dart' as p;
+import 'package:source_span/source_span.dart';
+import 'package:yaml/yaml.dart';
import '../backend/group.dart';
import '../backend/invoker.dart';
@@ -21,12 +23,16 @@ import 'load_exception.dart';
import 'load_suite.dart';
import 'node/platform.dart';
import 'parse_metadata.dart';
+import 'plugin/customizable_platform.dart';
import 'plugin/environment.dart';
import 'plugin/hack_register_platform.dart';
import 'plugin/platform.dart';
import 'runner_suite.dart';
import 'vm/platform.dart';
+// TODO(nweiz): Use inline function types when sdk#30858 is fixed.
+typedef FutureOr<PlatformPlugin> _PlatformPluginFunction();
+
/// A class for finding test files and loading them into a runnable form.
class Loader {
/// The test runner configuration.
@@ -41,17 +47,46 @@ class Loader {
/// The functions to use to load [_platformPlugins].
///
/// These are passed to the plugins' async memoizers when a plugin is needed.
- final _platformCallbacks = <TestPlatform, AsyncFunction>{};
+ final _platformCallbacks = <TestPlatform, _PlatformPluginFunction>{};
+
+ /// A map of all platforms registered in [_platformCallbacks], indexed by
+ /// their string identifiers.
+ final _platformsByIdentifier = <String, TestPlatform>{};
+
+ /// The user-provided settings for platforms, as a list of settings that will
+ /// be merged together using [CustomizablePlatform.mergePlatformSettings].
+ final _platformSettings = <TestPlatform, List<YamlMap>>{};
+
+ /// The user-provided settings for platforms.
+ final _parsedPlatformSettings = <TestPlatform, Object>{};
+
+ /// All plaforms supported by this [Loader].
+ List<TestPlatform> get allPlatforms =>
+ new List.unmodifiable(_platformCallbacks.keys);
+
+ /// The platform variables supported by this loader, in addition the default
+ /// variables that are always supported.
+ Iterable<String> get _platformVariables =>
+ _platformCallbacks.keys.map((platform) => platform.identifier);
/// Creates a new loader that loads tests on platforms defined in
/// [Configuration.current].
///
/// [root] is the root directory that will be served for browser tests. It
/// defaults to the working directory.
- Loader({String root}) {
- registerPlatformPlugin([TestPlatform.vm], () => new VMPlatform());
- registerPlatformPlugin([TestPlatform.nodeJS], () => new NodePlatform());
- registerPlatformPlugin([
+ ///
+ /// The [plugins] register [PlatformPlugin]s that are associated with the
+ /// provided platforms. When the runner first requests that a suite be loaded
+ /// for one of the given platforms, the lodaer will call the associated
+ /// callback to load the platform plugin. That plugin is then preserved and
+ /// used to load all suites for all matching platforms. Platform plugins may
+ /// override built-in platforms.
+ Loader(
+ {String root,
+ Map<Iterable<TestPlatform>, _PlatformPluginFunction> plugins}) {
+ _registerPlatformPlugin([TestPlatform.vm], () => new VMPlatform());
+ _registerPlatformPlugin([TestPlatform.nodeJS], () => new NodePlatform());
+ _registerPlatformPlugin([
TestPlatform.dartium,
TestPlatform.contentShell,
TestPlatform.chrome,
@@ -62,27 +97,75 @@ class Loader {
], () => BrowserPlatform.start(root: root));
platformCallbacks.forEach((platform, plugin) {
- registerPlatformPlugin([platform], plugin);
+ _registerPlatformPlugin([platform], plugin);
});
+
+ plugins?.forEach(_registerPlatformPlugin);
+
+ _registerCustomPlatforms();
+
+ _config.validatePlatforms(allPlatforms);
+
+ _registerPlatformOverrides();
}
/// Registers a [PlatformPlugin] for [platforms].
- ///
- /// When the runner first requests that a suite be loaded for one of the given
- /// platforms, this will call [getPlugin] to load the platform plugin. It may
- /// return either a [PlatformPlugin] or a [Future<PlatformPlugin>]. That
- /// plugin is then preserved and used to load all suites for all matching
- /// platforms.
- ///
- /// This overwrites previous plugins for those platforms.
- void registerPlatformPlugin(Iterable<TestPlatform> platforms, getPlugin()) {
+ void _registerPlatformPlugin(
+ Iterable<TestPlatform> platforms, FutureOr<PlatformPlugin> getPlugin()) {
var memoizer = new AsyncMemoizer<PlatformPlugin>();
for (var platform in platforms) {
_platformPlugins[platform] = memoizer;
_platformCallbacks[platform] = getPlugin;
+ _platformsByIdentifier[platform.identifier] = platform;
}
}
+ /// Registers user-defined platforms from [Configuration.definePlatforms].
+ void _registerCustomPlatforms() {
+ for (var customPlatform in _config.definePlatforms.values) {
+ if (_platformsByIdentifier.containsKey(customPlatform.identifier)) {
+ throw new SourceSpanFormatException(
+ wordWrap(
+ 'The platform "${customPlatform.identifier}" already exists. '
+ 'Use override_platforms to override it.'),
+ customPlatform.identifierSpan);
+ }
+
+ var parent = _platformsByIdentifier[customPlatform.parent];
+ if (parent == null) {
+ throw new SourceSpanFormatException(
+ 'Unknown platform.', customPlatform.parentSpan);
+ }
+
+ var platform =
+ parent.extend(customPlatform.name, customPlatform.identifier);
+ _platformPlugins[platform] = _platformPlugins[parent];
+ _platformCallbacks[platform] = _platformCallbacks[parent];
+ _platformsByIdentifier[platform.identifier] = platform;
+
+ _platformSettings[platform] = [customPlatform.settings];
+ }
+ }
+
+ /// Registers users' platform settings from [Configuration.overridePlatforms].
+ void _registerPlatformOverrides() {
+ for (var settings in _config.overridePlatforms.values) {
+ var platform = _platformsByIdentifier[settings.identifier];
+
+ // This is officially validated in [Configuration.validatePlatforms].
+ assert(platform != null);
+
+ _platformSettings
+ .putIfAbsent(platform, () => [])
+ .addAll(settings.settings);
+ }
+ }
+
+ /// Returns the [TestPlatform] registered with this loader that's identified
+ /// by [identifier], or `null` if none can be found.
+ TestPlatform findTestPlatform(String identifier) =>
+ _platformsByIdentifier[identifier];
+
/// Loads all test suites in [dir] according to [suiteConfig].
///
/// This will load tests from files that match the global configuration's
@@ -117,8 +200,8 @@ class Loader {
Stream<LoadSuite> loadFile(
String path, SuiteConfiguration suiteConfig) async* {
try {
- suiteConfig = suiteConfig
- .merge(new SuiteConfiguration.fromMetadata(parseMetadata(path)));
+ suiteConfig = suiteConfig.merge(new SuiteConfiguration.fromMetadata(
+ parseMetadata(path, _platformVariables.toSet())));
} on AnalyzerErrorGroup catch (_) {
// Ignore the analyzer's error, since its formatting is much worse than
// the VM's or dart2js's.
@@ -137,7 +220,10 @@ class Loader {
return;
}
- for (var platform in suiteConfig.platforms) {
+ for (var platformName in suiteConfig.platforms) {
+ var platform = findTestPlatform(platformName);
+ assert(platform != null, 'Unknown platform "$platformName".');
+
if (!suiteConfig.metadata.testOn.evaluate(platform, os: currentOS)) {
continue;
}
@@ -163,7 +249,9 @@ class Loader {
try {
var plugin = await memo.runOnce(_platformCallbacks[platform]);
- var suite = await plugin.load(path, platform, platformConfig);
+ _customizePlatform(plugin, platform);
+ var suite = await plugin.load(path, platform, platformConfig,
+ {"platformVariables": _platformVariables.toList()});
if (suite != null) _suites.add(suite);
return suite;
} catch (error, stackTrace) {
@@ -175,6 +263,39 @@ class Loader {
}
}
+ /// Passes user-defined settings to [plugin] if necessary.
+ void _customizePlatform(PlatformPlugin plugin, TestPlatform platform) {
+ var parsed = _parsedPlatformSettings[platform];
+ if (parsed != null) {
+ (plugin as CustomizablePlatform).customizePlatform(platform, parsed);
+ return;
+ }
+
+ var settings = _platformSettings[platform];
+ if (settings == null) return;
+
+ if (plugin is CustomizablePlatform) {
+ parsed = settings
+ .map(plugin.parsePlatformSettings)
+ .reduce(plugin.mergePlatformSettings);
+ plugin.customizePlatform(platform, parsed);
+ _parsedPlatformSettings[platform] = parsed;
+ } else {
+ String identifier;
+ SourceSpan span;
+ if (platform.isChild) {
+ identifier = platform.parent.identifier;
+ span = _config.definePlatforms[platform.identifier].parentSpan;
+ } else {
+ identifier = platform.identifier;
+ span = _config.overridePlatforms[platform.identifier].identifierSpan;
+ }
+
+ throw new SourceSpanFormatException(
+ 'The "$identifier" platform can\'t be customized.', span);
+ }
+ }
+
Future closeEphemeral() async {
await Future.wait(_platformPlugins.values.map((memo) async {
if (!memo.hasRun) return;
diff --git a/lib/src/runner/node/platform.dart b/lib/src/runner/node/platform.dart
index 179edc41d7728abf683f94093f5e7aeb37bb45d8..c6e76334cc5b20913b688989a29f57859baac688 100644
--- a/lib/src/runner/node/platform.dart
+++ b/lib/src/runner/node/platform.dart
@@ -11,22 +11,27 @@ import 'package:node_preamble/preamble.dart' as preamble;
import 'package:package_resolver/package_resolver.dart';
import 'package:path/path.dart' as p;
import 'package:stream_channel/stream_channel.dart';
+import 'package:yaml/yaml.dart';
import '../../backend/test_platform.dart';
import '../../util/io.dart';
import '../../util/stack_trace_mapper.dart';
import '../../utils.dart';
+import '../application_exception.dart';
import '../compiler_pool.dart';
import '../configuration.dart';
import '../configuration/suite.dart';
+import '../executable_settings.dart';
import '../load_exception.dart';
+import '../plugin/customizable_platform.dart';
import '../plugin/environment.dart';
import '../plugin/platform.dart';
import '../plugin/platform_helpers.dart';
import '../runner_suite.dart';
/// A platform that loads tests in Node.js processes.
-class NodePlatform extends PlatformPlugin {
+class NodePlatform extends PlatformPlugin
+ implements CustomizablePlatform<ExecutableSettings> {
/// The test runner configuration.
final Configuration _config;
@@ -39,24 +44,43 @@ class NodePlatform extends PlatformPlugin {
/// The HTTP client to use when fetching JS files for `pub serve`.
final HttpClient _http;
- /// The Node executable to use.
- String get _executable => Platform.isWindows ? "node.exe" : "node";
+ /// Executable settings for [TestPlatform.nodeJS] and platforms that extend
+ /// it.
+ final _settings = {
+ TestPlatform.nodeJS: new ExecutableSettings(
+ linuxExecutable: "node",
+ macOSExecutable: "node",
+ windowsExecutable: "node.exe")
+ };
NodePlatform()
: _config = Configuration.current,
_http =
Configuration.current.pubServeUrl == null ? null : new HttpClient();
+ ExecutableSettings parsePlatformSettings(YamlMap settings) =>
+ new ExecutableSettings.parse(settings);
+
+ ExecutableSettings mergePlatformSettings(
+ ExecutableSettings settings1, ExecutableSettings settings2) =>
+ settings1.merge(settings2);
+
+ void customizePlatform(TestPlatform platform, ExecutableSettings settings) {
+ var oldSettings = _settings[platform] ?? _settings[platform.root];
+ if (oldSettings != null) settings = oldSettings.merge(settings);
+ _settings[platform] = settings;
+ }
+
StreamChannel loadChannel(String path, TestPlatform platform) =>
throw new UnimplementedError();
Future<RunnerSuite> load(String path, TestPlatform platform,
- SuiteConfiguration suiteConfig) async {
+ SuiteConfiguration suiteConfig, Object message) async {
assert(platform == TestPlatform.nodeJS);
- var pair = await _loadChannel(path, suiteConfig);
- var controller = await deserializeSuite(
- path, platform, suiteConfig, new PluginEnvironment(), pair.first,
+ var pair = await _loadChannel(path, platform, suiteConfig);
+ var controller = await deserializeSuite(path, platform, suiteConfig,
+ new PluginEnvironment(), pair.first, message,
mapper: pair.last);
return controller.suite;
}
@@ -65,9 +89,9 @@ class NodePlatform extends PlatformPlugin {
///
/// Returns that channel along with a [StackTraceMapper] representing the
/// source map for the compiled suite.
- Future<Pair<StreamChannel, StackTraceMapper>> _loadChannel(
- String path, SuiteConfiguration suiteConfig) async {
- var pair = await _spawnProcess(path, suiteConfig);
+ Future<Pair<StreamChannel, StackTraceMapper>> _loadChannel(String path,
+ TestPlatform platform, SuiteConfiguration suiteConfig) async {
+ var pair = await _spawnProcess(path, platform, suiteConfig);
var process = pair.first;
// Node normally doesn't emit any standard error, but if it does we forward
@@ -91,8 +115,8 @@ class NodePlatform extends PlatformPlugin {
///
/// Returns that channel along with a [StackTraceMapper] representing the
/// source map for the compiled suite.
- Future<Pair<Process, StackTraceMapper>> _spawnProcess(
- String path, SuiteConfiguration suiteConfig) async {
+ Future<Pair<Process, StackTraceMapper>> _spawnProcess(String path,
+ TestPlatform platform, SuiteConfiguration suiteConfig) async {
var dir = new Directory(_compiledDir).createTempSync('test_').path;
var jsPath = p.join(dir, p.basename(path) + ".node_test.dart.js");
@@ -122,7 +146,7 @@ class NodePlatform extends PlatformPlugin {
sdkRoot: p.toUri(sdkDir));
}
- return new Pair(await Process.start(_executable, [jsPath]), mapper);
+ return new Pair(await _startProcess(platform, jsPath), mapper);
}
var url = _config.pubServeUrl.resolveUri(
@@ -141,7 +165,22 @@ class NodePlatform extends PlatformPlugin {
sdkRoot: p.toUri('packages/\$sdk'));
}
- return new Pair(await Process.start(_executable, [jsPath]), mapper);
+ return new Pair(await _startProcess(platform, jsPath), mapper);
+ }
+
+ /// Starts the Node.js process for [platform] with [jsPath].
+ Future<Process> _startProcess(TestPlatform platform, String jsPath) async {
+ var settings = _settings[platform];
+ try {
+ return await Process.start(
+ settings.executable, settings.arguments.toList()..add(jsPath));
+ } catch (error, stackTrace) {
+ await new Future.error(
+ new ApplicationException(
+ "Failed to run ${platform.name}: ${getErrorMessage(error)}"),
+ stackTrace);
+ return null;
+ }
}
/// Runs an HTTP GET on [url].
diff --git a/lib/src/runner/parse_metadata.dart b/lib/src/runner/parse_metadata.dart
index 1ab4c30b842f53eedcd2d5cc43b3259c630d727a..e8aa1ae22417770e204531464816423cf91252b6 100644
--- a/lib/src/runner/parse_metadata.dart
+++ b/lib/src/runner/parse_metadata.dart
@@ -18,22 +18,31 @@ import '../utils.dart';
/// Parse the test metadata for the test file at [path].
///
+/// The [platformVariables] are the set of variables that are valid for platform
+/// selectors in suite metadata, in addition to the built-in variables that are
+/// allowed everywhere.
+///
/// Throws an [AnalysisError] if parsing fails or a [FormatException] if the
/// test annotations are incorrect.
-Metadata parseMetadata(String path) => new _Parser(path).parse();
+Metadata parseMetadata(String path, Set<String> platformVariables) =>
+ new _Parser(path, platformVariables).parse();
/// A parser for test suite metadata.
class _Parser {
/// The path to the test suite.
final String _path;
+ /// The set of variables that are valid for platform selectors, in addition to
+ /// the built-in variables that are allowed everywhere.
+ final Set<String> _platformVariables;
+
/// All annotations at the top of the file.
List<Annotation> _annotations;
/// All prefixes defined by imports in this file.
Set<String> _prefixes;
- _Parser(String path) : _path = path {
+ _Parser(String path, this._platformVariables) : _path = path {
var contents = new File(path).readAsStringSync();
var directives = parseDirectives(contents, name: path).directives;
_annotations = directives.isEmpty ? [] : directives.first.metadata;
@@ -105,9 +114,17 @@ class _Parser {
PlatformSelector _parseTestOn(Annotation annotation, String constructorName) {
_assertConstructorName(constructorName, 'TestOn', annotation);
_assertArguments(annotation.arguments, 'TestOn', annotation, positional: 1);
- var literal = _parseString(annotation.arguments.arguments.first);
+ return _parsePlatformSelector(annotation.arguments.arguments.first);
+ }
+
+ /// Parses an [expression] that should contain a string representing a
+ /// [PlatformSelector].
+ PlatformSelector _parsePlatformSelector(Expression expression) {
+ var literal = _parseString(expression);
return _contextualize(
- literal, () => new PlatformSelector.parse(literal.stringValue));
+ literal,
+ () => new PlatformSelector.parse(literal.stringValue)
+ ..validate(_platformVariables));
}
/// Parses a `@Retry` annotation.
@@ -217,9 +234,7 @@ class _Parser {
positional: 1);
return _parseMap(annotation.arguments.arguments.first, key: (key) {
- var selector = _parseString(key);
- return _contextualize(
- selector, () => new PlatformSelector.parse(selector.stringValue));
+ return _parsePlatformSelector(key);
}, value: (value) {
var expressions = [];
if (value is ListLiteral) {
diff --git a/lib/src/runner/plugin/customizable_platform.dart b/lib/src/runner/plugin/customizable_platform.dart
new file mode 100644
index 0000000000000000000000000000000000000000..e1ebd57e2e2e9bc3bae1cffd88f7860cc8aec198
--- /dev/null
+++ b/lib/src/runner/plugin/customizable_platform.dart
@@ -0,0 +1,56 @@
+// Copyright (c) 2017, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'package:yaml/yaml.dart';
+
+import '../../backend/test_platform.dart';
+import 'platform.dart';
+
+/// An interface for [PlatformPlugin]s that support per-platform customization.
+///
+/// If a [PlatformPlugin] implements this, the user will be able to override the
+/// [TestPlatform]s it supports using the
+/// [`override_platforms`][override_platforms] configuration field, and define
+/// new platforms based on them using the [`define_platforms`][define_platforms]
+/// field. The custom settings will be passed to the plugin using
+/// [customizePlatform].
+///
+/// [override_platforms]: https://github.com/dart-lang/test/blob/master/doc/configuration.md#override_platforms
+/// [define_platforms]: https://github.com/dart-lang/test/blob/master/doc/configuration.md#define_platforms
+///
+/// Plugins that implement this **must** support children of recognized
+/// platforms (created by [TestPlatform.extend]) in their [loadChannel] or
+/// [load] methods.
+abstract class CustomizablePlatform<T> extends PlatformPlugin {
+ /// Parses user-provided [settings] for a custom platform into a
+ /// plugin-defined format.
+ ///
+ /// The [settings] come from a user's configuration file. The parsed output
+ /// will be passed to [customizePlatform].
+ ///
+ /// Subclasses should throw [SourceSpanFormatException]s if [settings]
+ /// contains invalid configuration. Unrecognized fields should be ignored if
+ /// possible.
+ T parsePlatformSettings(YamlMap settings);
+
+ /// Merges [settings1] with [settings2] and returns a new settings object that
+ /// includes the configuration of both.
+ ///
+ /// When the settings conflict, [settings2] should take priority.
+ ///
+ /// This is used to merge global settings with local settings, or a custom
+ /// platform's settings with its parent's.
+ T mergePlatformSettings(T settings1, T settings2);
+
+ /// Defines user-provided [settings] for [platform].
+ ///
+ /// The [platform] is a platform this plugin was declared to accept when
+ /// registered with [Loader.registerPlatformPlugin], or a platform whose
+ /// [TestPlatform.parent] is one of those platforms. Subclasses should
+ /// customize the behavior for these platforms when [loadChannel] or [load] is
+ /// called with the given [platform], using the [settings] which are parsed by
+ /// [parsePlatformSettings]. This is guaranteed to be called before either
+ /// `load` method.
+ void customizePlatform(TestPlatform platform, T settings);
+}
diff --git a/lib/src/runner/plugin/platform.dart b/lib/src/runner/plugin/platform.dart
index f3e7eec008ba6cfd0d50d0216fa9d4dd8000d3e6..744c46347a644d2b03218ba9bc27cfb4725da101 100644
--- a/lib/src/runner/plugin/platform.dart
+++ b/lib/src/runner/plugin/platform.dart
@@ -23,8 +23,8 @@ import 'platform_helpers.dart';
/// In order to support interactive debugging, a plugin must override [load] as
/// well, which returns a [RunnerSuite] that can contain a custom [Environment]
/// and control debugging metadata such as [RunnerSuite.isDebugging] and
-/// [RunnerSuite.onDebugging]. To make this easier, implementations can call
-/// [deserializeSuite] in `platform_helpers.dart`.
+/// [RunnerSuite.onDebugging]. The plugin must create this suite by calling the
+/// [deserializeSuite] helper function.
///
/// A platform plugin can be registered with [Loader.registerPlatformPlugin].
abstract class PlatformPlugin {
@@ -52,16 +52,16 @@ abstract class PlatformPlugin {
/// fine-grained control over the [RunnerSuite], including providing a custom
/// implementation of [Environment].
///
- /// It's recommended that subclasses overriding this method call
- /// [deserializeSuite] in `platform_helpers.dart` to obtain a
- /// [RunnerSuiteController].
+ /// Subclasses overriding this method must call [deserializeSuite] in
+ /// `platform_helpers.dart` to obtain a [RunnerSuiteController]. They must
+ /// pass the opaque [message] parameter to the [deserializeSuite] call.
Future<RunnerSuite> load(String path, TestPlatform platform,
- SuiteConfiguration suiteConfig) async {
+ SuiteConfiguration suiteConfig, Object message) async {
// loadChannel may throw an exception. That's fine; it will cause the
// LoadSuite to emit an error, which will be presented to the user.
var channel = loadChannel(path, platform);
var controller = await deserializeSuite(
- path, platform, suiteConfig, new PluginEnvironment(), channel);
+ path, platform, suiteConfig, new PluginEnvironment(), channel, message);
return controller.suite;
}
diff --git a/lib/src/runner/plugin/platform_helpers.dart b/lib/src/runner/plugin/platform_helpers.dart
index 786acd69921fba21de1a235266ef4ca05ca1457a..b41e5ba8491486ad0953618911a87ac0b65a2c43 100644
--- a/lib/src/runner/plugin/platform_helpers.dart
+++ b/lib/src/runner/plugin/platform_helpers.dart
@@ -33,24 +33,25 @@ final _deserializeTimeout = new Duration(minutes: 8);
///
/// If the suite is closed, this will close [channel].
///
-/// If [mapTrace] is passed, it will be used to adjust stack traces for any
-/// errors emitted by tests.
+/// The [message] parameter is an opaque object passed from the runner to
+/// [PlatformPlugin.load]. Plugins shouldn't interact with it other than to pass
+/// it on to [deserializeSuite].
///
-/// If [asciiSymbols] is passed, it controls whether the `symbol` package is
-/// configured to use plain ASCII or Unicode symbols. It defaults to `true` on
-/// Windows and `false` elsewhere.
+/// If [mapper] is passed, it will be used to adjust stack traces for any errors
+/// emitted by tests.
Future<RunnerSuiteController> deserializeSuite(
String path,
TestPlatform platform,
SuiteConfiguration suiteConfig,
Environment environment,
StreamChannel channel,
+ Object message,
{StackTraceMapper mapper}) async {
var disconnector = new Disconnector();
var suiteChannel = new MultiChannel(channel.transform(disconnector));
suiteChannel.sink.add({
- 'platform': platform.identifier,
+ 'platform': platform.serialize(),
'metadata': suiteConfig.metadata.serialize(),
'os': platform == TestPlatform.vm ? currentOS.identifier : null,
'asciiGlyphs': Platform.isWindows,
@@ -60,7 +61,7 @@ Future<RunnerSuiteController> deserializeSuite(
'stackTraceMapper': mapper?.serialize(),
'foldTraceExcept': Configuration.current.foldTraceExcept.toList(),
'foldTraceOnly': Configuration.current.foldTraceOnly.toList(),
- });
+ }..addAll(message as Map));
var completer = new Completer();
@@ -74,7 +75,7 @@ Future<RunnerSuiteController> deserializeSuite(
// notify the user of the error.
loadSuiteZone.handleUncaughtError(error, stackTrace);
} else {
- completer.completeError(error);
+ completer.completeError(error, stackTrace);
}
}
diff --git a/lib/src/runner/remote_listener.dart b/lib/src/runner/remote_listener.dart
index a61fd0edf9dfdeab246fbb3bd9ccd458cc9fead3..883101fa498e14023139f42058bdb887e449d2de 100644
--- a/lib/src/runner/remote_listener.dart
+++ b/lib/src/runner/remote_listener.dart
@@ -79,6 +79,7 @@ class RemoteListener {
verboseChain = metadata.verboseTrace;
var declarer = new Declarer(
metadata: metadata,
+ platformVariables: message['platformVariables'].toSet(),
collectTraces: message['collectTraces'],
noRetry: message['noRetry']);
@@ -89,11 +90,13 @@ class RemoteListener {
await declarer.declare(main);
- var os =
- message['os'] == null ? null : OperatingSystem.find(message['os']);
- var platform = TestPlatform.find(message['platform']);
var suite = new Suite(declarer.build(),
- platform: platform, os: os, path: message['path']);
+ platform: new TestPlatform.deserialize(message['platform']),
+ os: message['os'] == null
+ ? null
+ : OperatingSystem.find(message['os']),
+ path: message['path']);
+
new RemoteListener._(suite, printZone)._listen(channel);
}, onError: (error, stackTrace) {
_sendError(channel, error, stackTrace, verboseChain);
diff --git a/lib/src/utils.dart b/lib/src/utils.dart
index f545e742563e81d522c447a6f3d7a75f2347dcab..2e06ef1c1a633fea41b31d20266c04b1472cabf4 100644
--- a/lib/src/utils.dart
+++ b/lib/src/utils.dart
@@ -8,6 +8,7 @@ import 'dart:math' as math;
import 'dart:typed_data';
import 'package:async/async.dart';
+import 'package:collection/collection.dart';
import 'package:matcher/matcher.dart';
import 'package:path/path.dart' as p;
import 'package:stream_channel/stream_channel.dart';
@@ -194,6 +195,17 @@ List flatten(Iterable nested) {
return result;
}
+/// Like [mergeMaps], but assumes both maps are unmodifiable and so avoids
+/// creating a new map unnecessarily.
+///
+/// The return value *may or may not* be unmodifiable.
+Map<K, V> mergeUnmodifiableMaps<K, V>(Map<K, V> map1, Map<K, V> map2,
+ {V value(V value1, V value2)}) {
+ if (map1.isEmpty) return map2;
+ if (map2.isEmpty) return map1;
+ return mergeMaps(map1, map2, value: value);
+}
+
/// Truncates [text] to fit within [maxLength].
///
/// This will try to truncate along word boundaries and preserve words both at
diff --git a/pubspec.yaml b/pubspec.yaml
index 375ca446b2438f570624c61edfd676995c8a32e8..c1c0441f885b229518f359734da6aab3255d014f 100644
--- a/pubspec.yaml
+++ b/pubspec.yaml
@@ -1,5 +1,5 @@
name: test
-version: 0.12.24+8
+version: 0.12.25-dev
author: Dart Team <misc@dartlang.org>
description: A library for writing dart unit tests.
homepage: https://github.com/dart-lang/test
@@ -14,6 +14,7 @@ dependencies:
collection: '^1.8.0'
glob: '^1.0.0'
http_multi_server: '>=1.0.0 <3.0.0'
+ io: '^0.3.0'
js: '^0.6.0'
meta: '^1.0.0'
node_preamble: '^1.3.0'
diff --git a/test/backend/metadata_test.dart b/test/backend/metadata_test.dart
index b32dfaf98895e6277571fbb2a28583403c08261a..35fc9ef183cb1cef4008ae6b4e090ba8f4b890eb 100644
--- a/test/backend/metadata_test.dart
+++ b/test/backend/metadata_test.dart
@@ -173,7 +173,7 @@ void main() {
test("refuses an invalid platform selector", () {
expect(() {
- new Metadata.parse(onPlatform: {"invalid": new Skip()});
+ new Metadata.parse(onPlatform: {"vm &&": new Skip()});
}, throwsFormatException);
});
@@ -194,6 +194,32 @@ void main() {
});
});
+ group("validatePlatformSelectors", () {
+ test("succeeds if onPlatform uses valid platforms", () {
+ new Metadata.parse(onPlatform: {"vm || browser": new Skip()})
+ .validatePlatformSelectors(new Set.from(["vm"]));
+ });
+
+ test("succeeds if testOn uses valid platforms", () {
+ new Metadata.parse(testOn: "vm || browser")
+ .validatePlatformSelectors(new Set.from(["vm"]));
+ });
+
+ test("fails if onPlatform uses an invalid platform", () {
+ expect(() {
+ new Metadata.parse(onPlatform: {"unknown": new Skip()})
+ .validatePlatformSelectors(new Set.from(["vm"]));
+ }, throwsFormatException);
+ });
+
+ test("fails if testOn uses an invalid platform", () {
+ expect(() {
+ new Metadata.parse(testOn: "unknown")
+ .validatePlatformSelectors(new Set.from(["vm"]));
+ }, throwsFormatException);
+ });
+ });
+
group("change", () {
test("preserves all fields if no parameters are passed", () {
var metadata = new Metadata(
diff --git a/test/runner/browser/chrome_test.dart b/test/runner/browser/chrome_test.dart
index 5f34c064c9b5994fe96f989767454fe21f5d179f..4ff242f356810c27b7fd75f23076d06e26c4e147 100644
--- a/test/runner/browser/chrome_test.dart
+++ b/test/runner/browser/chrome_test.dart
@@ -7,6 +7,7 @@
import 'package:test_descriptor/test_descriptor.dart' as d;
+import 'package:test/src/runner/executable_settings.dart';
import 'package:test/src/runner/browser/chrome.dart';
import 'package:test/test.dart';
@@ -42,8 +43,11 @@ webSocket.addEventListener("open", function() {
});
test("reports an error in onExit", () {
- var chrome =
- new Chrome("http://dart-lang.org", executable: "_does_not_exist");
+ var chrome = new Chrome("http://dart-lang.org",
+ settings: new ExecutableSettings(
+ linuxExecutable: "_does_not_exist",
+ macOSExecutable: "_does_not_exist",
+ windowsExecutable: "_does_not_exist"));
expect(
chrome.onExit,
throwsA(isApplicationException(
diff --git a/test/runner/browser/content_shell_test.dart b/test/runner/browser/content_shell_test.dart
index 2370cd472eedf2cbbda4fe55976d25f0d3329fda..f506f5d3d6418e4e17878f44146094e14730231e 100644
--- a/test/runner/browser/content_shell_test.dart
+++ b/test/runner/browser/content_shell_test.dart
@@ -8,6 +8,7 @@
import 'package:test_descriptor/test_descriptor.dart' as d;
import 'package:test/src/runner/browser/content_shell.dart';
+import 'package:test/src/runner/executable_settings.dart';
import 'package:test/test.dart';
import '../../io.dart';
@@ -40,8 +41,11 @@ webSocket.send("loaded!");
});
test("reports an error in onExit", () {
- var contentShell =
- new ContentShell("http://dart-lang.org", executable: "_does_not_exist");
+ var contentShell = new ContentShell("http://dart-lang.org",
+ settings: new ExecutableSettings(
+ linuxExecutable: "_does_not_exist",
+ macOSExecutable: "_does_not_exist",
+ windowsExecutable: "_does_not_exist"));
expect(
contentShell.onExit,
throwsA(isApplicationException(
diff --git a/test/runner/browser/dartium_test.dart b/test/runner/browser/dartium_test.dart
index 9991b9141a29106e799053ce34802c0b0af53693..0aac2a37f25199398993f2a205b32cef7c6cfff4 100644
--- a/test/runner/browser/dartium_test.dart
+++ b/test/runner/browser/dartium_test.dart
@@ -8,6 +8,7 @@
import 'package:test_descriptor/test_descriptor.dart' as d;
import 'package:test/src/runner/browser/dartium.dart';
+import 'package:test/src/runner/executable_settings.dart';
import 'package:test/test.dart';
import '../../io.dart';
@@ -40,8 +41,11 @@ webSocket.send("loaded!");
});
test("reports an error in onExit", () {
- var dartium =
- new Dartium("http://dart-lang.org", executable: "_does_not_exist");
+ var dartium = new Dartium("http://dart-lang.org",
+ settings: new ExecutableSettings(
+ linuxExecutable: "_does_not_exist",
+ macOSExecutable: "_does_not_exist",
+ windowsExecutable: "_does_not_exist"));
expect(
dartium.onExit,
throwsA(isApplicationException(
diff --git a/test/runner/browser/firefox_test.dart b/test/runner/browser/firefox_test.dart
index d18ac0b3ab9f719cf3fb6828932c3c1209819a3d..ccb7647751aadd42a73227867671395a6fd28672 100644
--- a/test/runner/browser/firefox_test.dart
+++ b/test/runner/browser/firefox_test.dart
@@ -8,6 +8,7 @@
import 'package:test_descriptor/test_descriptor.dart' as d;
import 'package:test/src/runner/browser/firefox.dart';
+import 'package:test/src/runner/executable_settings.dart';
import 'package:test/test.dart';
import '../../io.dart';
@@ -40,8 +41,11 @@ webSocket.addEventListener("open", function() {
});
test("reports an error in onExit", () {
- var firefox =
- new Firefox("http://dart-lang.org", executable: "_does_not_exist");
+ var firefox = new Firefox("http://dart-lang.org",
+ settings: new ExecutableSettings(
+ linuxExecutable: "_does_not_exist",
+ macOSExecutable: "_does_not_exist",
+ windowsExecutable: "_does_not_exist"));
expect(
firefox.onExit,
throwsA(isApplicationException(
diff --git a/test/runner/browser/internet_explorer_test.dart b/test/runner/browser/internet_explorer_test.dart
index 61a0248cb831990a68647a362bd92af872907d01..36f3c968a328d66861664f8a97aae3548179402e 100644
--- a/test/runner/browser/internet_explorer_test.dart
+++ b/test/runner/browser/internet_explorer_test.dart
@@ -8,6 +8,7 @@
import 'package:test_descriptor/test_descriptor.dart' as d;
import 'package:test/src/runner/browser/internet_explorer.dart';
+import 'package:test/src/runner/executable_settings.dart';
import 'package:test/test.dart';
import '../../io.dart';
@@ -41,7 +42,10 @@ webSocket.addEventListener("open", function() {
test("reports an error in onExit", () {
var ie = new InternetExplorer("http://dart-lang.org",
- executable: "_does_not_exist");
+ settings: new ExecutableSettings(
+ linuxExecutable: "_does_not_exist",
+ macOSExecutable: "_does_not_exist",
+ windowsExecutable: "_does_not_exist"));
expect(
ie.onExit,
throwsA(isApplicationException(startsWith(
diff --git a/test/runner/browser/loader_test.dart b/test/runner/browser/loader_test.dart
index b245176d2637ee7fd6b460cde9c1bb6c1ac0d43e..a2739c83d11713ca47145f38bf1b6a4382a36153 100644
--- a/test/runner/browser/loader_test.dart
+++ b/test/runner/browser/loader_test.dart
@@ -12,6 +12,7 @@ import 'package:test_descriptor/test_descriptor.dart' as d;
import 'package:test/src/backend/state.dart';
import 'package:test/src/backend/test.dart';
import 'package:test/src/backend/test_platform.dart';
+import 'package:test/src/runner/configuration/platform_selection.dart';
import 'package:test/src/runner/configuration/suite.dart';
import 'package:test/src/runner/loader.dart';
import 'package:test/test.dart';
@@ -21,7 +22,8 @@ import '../../utils.dart';
Loader _loader;
/// A configuration that loads suites on Chrome.
-final _chrome = new SuiteConfiguration(platforms: [TestPlatform.chrome]);
+final _chrome = new SuiteConfiguration(
+ platforms: [new PlatformSelection(TestPlatform.chrome.identifier)]);
void main() {
setUp(() async {
@@ -124,8 +126,10 @@ Future main() {
var suites = await _loader
.loadFile(
path,
- new SuiteConfiguration(
- platforms: [TestPlatform.vm, TestPlatform.chrome]))
+ new SuiteConfiguration(platforms: [
+ new PlatformSelection(TestPlatform.vm.identifier),
+ new PlatformSelection(TestPlatform.chrome.identifier)
+ ]))
.asyncMap((loadSuite) => loadSuite.getSuite())
.toList();
expect(suites[0].platform, equals(TestPlatform.vm));
diff --git a/test/runner/browser/phantom_js_test.dart b/test/runner/browser/phantom_js_test.dart
index fc1311bdde956a63e25b4445747bca75352c4a83..7441006f80e75e26ff4553a6b64e95598e8c3301 100644
--- a/test/runner/browser/phantom_js_test.dart
+++ b/test/runner/browser/phantom_js_test.dart
@@ -8,6 +8,7 @@
import 'package:test_descriptor/test_descriptor.dart' as d;
import 'package:test/src/runner/browser/phantom_js.dart';
+import 'package:test/src/runner/executable_settings.dart';
import 'package:test/test.dart';
import '../../io.dart';
@@ -40,8 +41,11 @@ webSocket.addEventListener("open", function() {
});
test("reports an error in onExit", () {
- var phantomJS =
- new PhantomJS("http://dart-lang.org", executable: "_does_not_exist");
+ var phantomJS = new PhantomJS("http://dart-lang.org",
+ settings: new ExecutableSettings(
+ linuxExecutable: "_does_not_exist",
+ macOSExecutable: "_does_not_exist",
+ windowsExecutable: "_does_not_exist"));
expect(
phantomJS.onExit,
throwsA(isApplicationException(
diff --git a/test/runner/browser/safari_test.dart b/test/runner/browser/safari_test.dart
index 82b475613458fd99a739aa25e68aaa4dfa32afe7..ac86d75dddc2fba0421c16418d31e3158fd577b7 100644
--- a/test/runner/browser/safari_test.dart
+++ b/test/runner/browser/safari_test.dart
@@ -8,6 +8,7 @@
import 'package:test_descriptor/test_descriptor.dart' as d;
import 'package:test/src/runner/browser/safari.dart';
+import 'package:test/src/runner/executable_settings.dart';
import 'package:test/test.dart';
import '../../io.dart';
@@ -40,8 +41,11 @@ webSocket.addEventListener("open", function() {
});
test("reports an error in onExit", () {
- var safari =
- new Safari("http://dart-lang.org", executable: "_does_not_exist");
+ var safari = new Safari("http://dart-lang.org",
+ settings: new ExecutableSettings(
+ linuxExecutable: "_does_not_exist",
+ macOSExecutable: "_does_not_exist",
+ windowsExecutable: "_does_not_exist"));
expect(
safari.onExit,
throwsA(isApplicationException(
diff --git a/test/runner/configuration/custom_platform_test.dart b/test/runner/configuration/custom_platform_test.dart
new file mode 100644
index 0000000000000000000000000000000000000000..6977e6d391e7478f2cdfb51b6757109c74f56b9c
--- /dev/null
+++ b/test/runner/configuration/custom_platform_test.dart
@@ -0,0 +1,869 @@
+// Copyright (c) 2017, the Dart project authors. Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+@TestOn("vm")
+
+import 'dart:io';
+
+import 'package:test_descriptor/test_descriptor.dart' as d;
+import 'package:test_process/test_process.dart';
+
+import 'package:test/src/backend/test_platform.dart';
+import 'package:test/src/runner/browser/default_settings.dart';
+import 'package:test/src/util/exit_codes.dart' as exit_codes;
+import 'package:test/test.dart';
+
+import '../../io.dart';
+
+void main() {
+ setUp(() async {
+ await d.file("test.dart", """
+ import 'package:test/test.dart';
+
+ void main() {
+ test("success", () {});
+ }
+ """).create();
+ });
+
+ group("override_platforms", () {
+ group("can override a browser", () {
+ test("without any changes", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome:
+ settings: {}
+ """).create();
+
+ var test = await runTest(["-p", "chrome", "test.dart"]);
+ expect(test.stdout, emitsThrough(contains("All tests passed!")));
+ await test.shouldExit(0);
+ }, tags: "chrome");
+
+ test("that's user-defined", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings: {}
+
+ override_platforms:
+ chromium:
+ settings: {}
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout, emitsThrough(contains("All tests passed!")));
+ await test.shouldExit(0);
+ }, tags: "chrome");
+
+ test("with a basename-only executable", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ dartium:
+ settings:
+ executable:
+ linux: dartium
+ mac_os: dartium
+ windows: dartium.exe
+ """).create();
+
+ var test = await runTest(["-p", "dartium", "test.dart"]);
+ expect(test.stdout, emitsThrough(contains("All tests passed!")));
+ await test.shouldExit(0);
+ }, tags: "dartium");
+
+ test("with an absolute-path executable", () async {
+ String path;
+ if (Platform.isLinux) {
+ var process = await TestProcess.start("which", ["google-chrome"]);
+ path = await process.stdout.next;
+ await process.shouldExit(0);
+ } else {
+ path = defaultSettings[TestPlatform.chrome].executable;
+ }
+
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome:
+ settings:
+ executable: $path
+ """).create();
+
+ var test = await runTest(["-p", "chrome", "test.dart"]);
+ expect(test.stdout, emitsThrough(contains("All tests passed!")));
+ await test.shouldExit(0);
+ }, tags: "chrome");
+ });
+
+ test("can override Node.js without any changes", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ node:
+ settings: {}
+ """).create();
+
+ var test = await runTest(["-p", "node", "test.dart"]);
+ expect(test.stdout, emitsThrough(contains("All tests passed!")));
+ await test.shouldExit(0);
+ }, tags: "node");
+
+ group("errors", () {
+ test("rejects a non-map value", () async {
+ await d.file("dart_test.yaml", "override_platforms: 12").create();
+
+ var test = await runTest([]);
+ expect(test.stderr,
+ containsInOrder(["override_platforms must be a map.", "^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("rejects a non-string key", () async {
+ await d
+ .file("dart_test.yaml", "override_platforms: {12: null}")
+ .create();
+
+ var test = await runTest([]);
+ expect(test.stderr,
+ containsInOrder(["Platform identifier must be a string.", "^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("rejects a non-identifier-like key", () async {
+ await d
+ .file("dart_test.yaml", "override_platforms: {foo bar: null}")
+ .create();
+
+ var test = await runTest([]);
+ expect(
+ test.stderr,
+ containsInOrder([
+ "Platform identifier must be an (optionally hyphenated) Dart "
+ "identifier.",
+ "^^^^^^^"
+ ]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("rejects a non-map definition", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome: 12
+ """).create();
+
+ var test = await runTest([]);
+ expect(test.stderr,
+ containsInOrder(["Platform definition must be a map.", "^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("requires a settings key", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome: {}
+ """).create();
+
+ var test = await runTest([]);
+ expect(test.stderr,
+ containsInOrder(['Missing required field "settings".', "^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("settings must be a map", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome:
+ settings: null
+ """).create();
+
+ var test = await runTest([]);
+ expect(test.stderr, containsInOrder(['Must be a map.', "^^^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("the overridden platform must exist", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chromium:
+ settings: {}
+ """).create();
+
+ var test = await runTest(["test.dart"]);
+ expect(test.stderr,
+ containsInOrder(['Unknown platform "chromium".', "^^^^^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("uncustomizable platforms can't be overridden", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ vm:
+ settings: {}
+ """).create();
+
+ var test = await runTest(["-p", "vm", "test.dart"]);
+ expect(test.stdout,
+ containsInOrder(['The "vm" platform can\'t be customized.', "^^"]));
+ await test.shouldExit(1);
+ });
+
+ group("when overriding browsers", () {
+ test("executable must be a string or map", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome:
+ settings:
+ executable: 12
+ """).create();
+
+ var test = await runTest(["-p", "chrome", "test.dart"]);
+ expect(test.stdout,
+ containsInOrder(['Must be a map or a string.', "^^"]));
+ await test.shouldExit(1);
+ });
+
+ test("executable string may not be relative on POSIX", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome:
+ settings:
+ executable: foo/bar
+ """).create();
+
+ var test = await runTest(["-p", "chrome", "test.dart"]);
+ expect(
+ test.stdout,
+ containsInOrder([
+ 'Linux and Mac OS executables may not be relative paths.',
+ "^^^^^^^"
+ ]));
+ await test.shouldExit(1);
+ },
+ // We allow relative executables for Windows so that Windows users
+ // can set a global executable without having to explicitly write
+ // `windows:`.
+ testOn: "!windows");
+
+ test("Linux executable must be a string", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome:
+ settings:
+ executable:
+ linux: 12
+ """).create();
+
+ var test = await runTest(["-p", "chrome", "test.dart"]);
+ expect(test.stdout, containsInOrder(['Must be a string.', "^^"]));
+ await test.shouldExit(1);
+ });
+
+ test("Linux executable may not be relative", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome:
+ settings:
+ executable:
+ linux: foo/bar
+ """).create();
+
+ var test = await runTest(["-p", "chrome", "test.dart"]);
+ expect(
+ test.stdout,
+ containsInOrder([
+ 'Linux and Mac OS executables may not be relative paths.',
+ "^^^^^^^"
+ ]));
+ await test.shouldExit(1);
+ });
+
+ test("Mac OS executable must be a string", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome:
+ settings:
+ executable:
+ mac_os: 12
+ """).create();
+
+ var test = await runTest(["-p", "chrome", "test.dart"]);
+ expect(test.stdout, containsInOrder(['Must be a string.', "^^"]));
+ await test.shouldExit(1);
+ });
+
+ test("Mac OS executable may not be relative", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome:
+ settings:
+ executable:
+ mac_os: foo/bar
+ """).create();
+
+ var test = await runTest(["-p", "chrome", "test.dart"]);
+ expect(
+ test.stdout,
+ containsInOrder([
+ 'Linux and Mac OS executables may not be relative paths.',
+ "^^^^^^^"
+ ]));
+ await test.shouldExit(1);
+ });
+
+ test("Windows executable must be a string", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome:
+ settings:
+ executable:
+ windows: 12
+ """).create();
+
+ var test = await runTest(["-p", "chrome", "test.dart"]);
+ expect(test.stdout, containsInOrder(['Must be a string.', "^^"]));
+ await test.shouldExit(1);
+ });
+
+ test("executable must exist", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ chrome:
+ settings:
+ executable: _does_not_exist
+ """).create();
+
+ var test = await runTest(["-p", "chrome", "test.dart"]);
+ expect(
+ test.stdout,
+ emitsThrough(
+ contains("Failed to run Chrome: $noSuchFileMessage")));
+ await test.shouldExit(1);
+ });
+
+ test("executable must exist for Node.js", () async {
+ await d.file("dart_test.yaml", """
+ override_platforms:
+ node:
+ settings:
+ executable: _does_not_exist
+ """).create();
+
+ var test = await runTest(["-p", "node", "test.dart"]);
+ expect(
+ test.stdout,
+ emitsThrough(
+ contains("Failed to run Node.js: $noSuchFileMessage")));
+ await test.shouldExit(1);
+ }, tags: "node");
+ });
+ });
+ });
+
+ group("define_platforms", () {
+ group("can define a new browser", () {
+ group("without any changes", () {
+ setUp(() async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings: {}
+ """).create();
+ });
+
+ test("that can be used to run tests", () async {
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout, emitsThrough(contains("All tests passed!")));
+ await test.shouldExit(0);
+ }, tags: "chrome");
+
+ test("that can be used in platform selectors", () async {
+ await d.file("test.dart", """
+ import 'package:test/test.dart';
+
+ void main() {
+ test("success", () {}, testOn: "chromium");
+ }
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout, emitsThrough(contains("All tests passed!")));
+ await test.shouldExit(0);
+
+ test = await runTest(["-p", "chrome", "test.dart"]);
+ expect(test.stdout, emitsThrough(contains("No tests ran.")));
+ await test.shouldExit(0);
+ }, tags: "chrome");
+
+ test("that counts as its parent", () async {
+ await d.file("test.dart", """
+ import 'package:test/test.dart';
+
+ void main() {
+ test("success", () {}, testOn: "chrome");
+ }
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout, emitsThrough(contains("All tests passed!")));
+ await test.shouldExit(0);
+ }, tags: "chrome");
+ });
+
+ test("with a basename-only executable", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ my-dartium:
+ name: My Dartium
+ extends: dartium
+ settings:
+ executable:
+ linux: dartium
+ mac_os: dartium
+ windows: dartium.exe
+ """).create();
+
+ var test = await runTest(["-p", "my-dartium", "test.dart"]);
+ expect(test.stdout, emitsThrough(contains("All tests passed!")));
+ await test.shouldExit(0);
+ }, tags: "dartium");
+
+ test("with an absolute-path executable", () async {
+ String path;
+ if (Platform.isLinux) {
+ var process = await TestProcess.start("which", ["google-chrome"]);
+ path = await process.stdout.next;
+ await process.shouldExit(0);
+ } else {
+ path = defaultSettings[TestPlatform.chrome].executable;
+ }
+
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ executable: $path
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout, emitsThrough(contains("All tests passed!")));
+ await test.shouldExit(0);
+ }, tags: "chrome");
+ });
+
+ group("errors", () {
+ test("rejects a non-map value", () async {
+ await d.file("dart_test.yaml", "define_platforms: 12").create();
+
+ var test = await runTest([]);
+ expect(test.stderr,
+ containsInOrder(["define_platforms must be a map.", "^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("rejects a non-string key", () async {
+ await d.file("dart_test.yaml", "define_platforms: {12: null}").create();
+
+ var test = await runTest([]);
+ expect(test.stderr,
+ containsInOrder(["Platform identifier must be a string.", "^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("rejects a non-identifier-like key", () async {
+ await d
+ .file("dart_test.yaml", "define_platforms: {foo bar: null}")
+ .create();
+
+ var test = await runTest([]);
+ expect(
+ test.stderr,
+ containsInOrder([
+ "Platform identifier must be an (optionally hyphenated) Dart "
+ "identifier.",
+ "^^^^^^^"
+ ]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("rejects a non-map definition", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium: 12
+ """).create();
+
+ var test = await runTest([]);
+ expect(test.stderr,
+ containsInOrder(["Platform definition must be a map.", "^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("requires a name key", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ extends: chrome
+ settings: {}
+ """).create();
+
+ var test = await runTest([]);
+ expect(
+ test.stderr,
+ containsInOrder(
+ ['Missing required field "name".', "^^^^^^^^^^^^^^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("name must be a string", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: null
+ extends: chrome
+ settings: {}
+ """).create();
+
+ var test = await runTest([]);
+ expect(test.stderr, containsInOrder(['Must be a string.', "^^^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("requires an extends key", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ settings: {}
+ """).create();
+
+ var test = await runTest([]);
+ expect(
+ test.stderr,
+ containsInOrder(
+ ['Missing required field "extends".', "^^^^^^^^^^^^^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("extends must be a string", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: null
+ settings: {}
+ """).create();
+
+ var test = await runTest([]);
+ expect(test.stderr,
+ containsInOrder(['Platform parent must be a string.', "^^^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("extends must be identifier-like", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: foo bar
+ settings: {}
+ """).create();
+
+ var test = await runTest([]);
+ expect(
+ test.stderr,
+ containsInOrder([
+ "Platform parent must be an (optionally hyphenated) Dart "
+ "identifier.",
+ "^^^^^^^"
+ ]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("requires a settings key", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ """).create();
+
+ var test = await runTest([]);
+ expect(
+ test.stderr,
+ containsInOrder(
+ ['Missing required field "settings".', "^^^^^^^^^^^^^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("settings must be a map", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings: null
+ """).create();
+
+ var test = await runTest([]);
+ expect(test.stderr, containsInOrder(['Must be a map.', "^^^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("the new platform may not override an existing platform", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chrome:
+ name: Chromium
+ extends: firefox
+ settings: {}
+ """).create();
+
+ await d.dir("test").create();
+
+ var test = await runTest([]);
+ expect(
+ test.stderr,
+ containsInOrder([
+ 'The platform "chrome" already exists. Use override_platforms to '
+ 'override it.',
+ "^^^^^^"
+ ]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("the new platform must extend an existing platform", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: foobar
+ settings: {}
+ """).create();
+
+ await d.dir("test").create();
+
+ var test = await runTest([]);
+ expect(test.stderr, containsInOrder(['Unknown platform.', "^^^^^^"]));
+ await test.shouldExit(exit_codes.data);
+ });
+
+ test("the new platform can't extend an uncustomizable platform",
+ () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ myvm:
+ name: My VM
+ extends: vm
+ settings: {}
+ """).create();
+
+ var test = await runTest(["-p", "myvm", "test.dart"]);
+ expect(test.stdout,
+ containsInOrder(['The "vm" platform can\'t be customized.', "^^"]));
+ await test.shouldExit(1);
+ });
+
+ group("when overriding browsers", () {
+ test("executable must be a string or map", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ executable: 12
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout,
+ containsInOrder(['Must be a map or a string.', "^^"]));
+ await test.shouldExit(1);
+ });
+
+ test("executable string may not be relative on POSIX", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ executable: foo/bar
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(
+ test.stdout,
+ containsInOrder([
+ 'Linux and Mac OS executables may not be relative paths.',
+ "^^^^^^^"
+ ]));
+ await test.shouldExit(1);
+ },
+ // We allow relative executables for Windows so that Windows users
+ // can set a global executable without having to explicitly write
+ // `windows:`.
+ testOn: "!windows");
+
+ test("Linux executable must be a string", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ executable:
+ linux: 12
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout, containsInOrder(['Must be a string.', "^^"]));
+ await test.shouldExit(1);
+ });
+
+ test("Linux executable may not be relative", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ executable:
+ linux: foo/bar
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(
+ test.stdout,
+ containsInOrder([
+ 'Linux and Mac OS executables may not be relative paths.',
+ "^^^^^^^"
+ ]));
+ await test.shouldExit(1);
+ });
+
+ test("Mac OS executable must be a string", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ executable:
+ mac_os: 12
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout, containsInOrder(['Must be a string.', "^^"]));
+ await test.shouldExit(1);
+ });
+
+ test("Mac OS executable may not be relative", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ executable:
+ mac_os: foo/bar
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(
+ test.stdout,
+ containsInOrder([
+ 'Linux and Mac OS executables may not be relative paths.',
+ "^^^^^^^"
+ ]));
+ await test.shouldExit(1);
+ });
+
+ test("Windows executable must be a string", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ executable:
+ windows: 12
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout, containsInOrder(['Must be a string.', "^^"]));
+ await test.shouldExit(1);
+ });
+
+ test("executable must exist", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ executable: _does_not_exist
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(
+ test.stdout,
+ emitsThrough(
+ contains("Failed to run Chrome: $noSuchFileMessage")));
+ await test.shouldExit(1);
+ });
+
+ test("arguments must be a string", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ arguments: 12
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout, containsInOrder(['Must be a string.', "^^"]));
+ await test.shouldExit(1);
+ });
+
+ test("arguments must be shell parseable", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ arguments: --foo 'bar
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout,
+ containsInOrder(['Unmatched single quote.', "^^^^^^^^^^"]));
+ await test.shouldExit(1);
+ });
+
+ test("with an argument that causes the browser to quit", () async {
+ await d.file("dart_test.yaml", """
+ define_platforms:
+ chromium:
+ name: Chromium
+ extends: chrome
+ settings:
+ arguments: --version
+ """).create();
+
+ var test = await runTest(["-p", "chromium", "test.dart"]);
+ expect(test.stdout,
+ emitsThrough(contains("Chromium exited before connecting.")));
+ await test.shouldExit(1);
+ }, tags: "chrome");
+ });
+ });
+ });
+}
diff --git a/test/runner/configuration/global_test.dart b/test/runner/configuration/global_test.dart
index 6590438a0d0f7b753598da89ece2e0e9ac5ee7af..67fc7245f841084a6ae2f9a28755d08b6825df64 100644
--- a/test/runner/configuration/global_test.dart
+++ b/test/runner/configuration/global_test.dart
@@ -94,9 +94,10 @@ void main() {
group("disallows local-only configuration:", () {
for (var field in [
"skip", "retry", "test_on", "paths", "filename", "names", "tags", //
- "plain_names", "include_tags", "exclude_tags", "pub_serve", "add_tags"
+ "plain_names", "include_tags", "exclude_tags", "pub_serve", "add_tags",
+ "define_platforms"
]) {
- test("rejects local-only configuration", () async {
+ test("for $field", () async {
await d.file("global_test.yaml", JSON.encode({field: null})).create();
await d.file("test.dart", """
diff --git a/test/runner/configuration/platform_test.dart b/test/runner/configuration/platform_test.dart
index 2bf1d6989789012572e95e3ca7c9ac4850f4429f..2d5c59dde19cb4f094f821ea59f3a68bd03e6b9e 100644
--- a/test/runner/configuration/platform_test.dart
+++ b/test/runner/configuration/platform_test.dart
@@ -116,11 +116,10 @@ void main() {
}))
.create();
+ await d.dir("test").create();
+
var test = await runTest([]);
- expect(
- test.stderr,
- containsInOrder(
- ["Invalid on_platform key: Undefined variable.", "^^^^^"]));
+ expect(test.stderr, containsInOrder(["Undefined variable.", "^^^^^"]));
await test.shouldExit(exit_codes.data);
});
diff --git a/test/runner/configuration/suite_test.dart b/test/runner/configuration/suite_test.dart
index 44cfd59369e02649d534b40b60ff8629234a09c0..a6ec33f9df1dd7dc9c61fa63cfbf439527c82d6f 100644
--- a/test/runner/configuration/suite_test.dart
+++ b/test/runner/configuration/suite_test.dart
@@ -8,6 +8,7 @@ import 'package:test/test.dart';
import 'package:test/src/backend/platform_selector.dart';
import 'package:test/src/backend/test_platform.dart';
+import 'package:test/src/runner/configuration/platform_selection.dart';
import 'package:test/src/runner/configuration/suite.dart';
void main() {
@@ -18,7 +19,7 @@ void main() {
expect(merged.jsTrace, isFalse);
expect(merged.runSkipped, isFalse);
expect(merged.precompiledPath, isNull);
- expect(merged.platforms, equals([TestPlatform.vm]));
+ expect(merged.platforms, equals([TestPlatform.vm.identifier]));
});
test("if only the old configuration's is defined, uses it", () {
@@ -26,12 +27,14 @@ void main() {
jsTrace: true,
runSkipped: true,
precompiledPath: "/tmp/js",
- platforms: [TestPlatform.chrome]).merge(new SuiteConfiguration());
+ platforms: [
+ new PlatformSelection(TestPlatform.chrome.identifier)
+ ]).merge(new SuiteConfiguration());
expect(merged.jsTrace, isTrue);
expect(merged.runSkipped, isTrue);
expect(merged.precompiledPath, equals("/tmp/js"));
- expect(merged.platforms, equals([TestPlatform.chrome]));
+ expect(merged.platforms, equals([TestPlatform.chrome.identifier]));
});
test("if only the new configuration's is defined, uses it", () {
@@ -39,12 +42,14 @@ void main() {
jsTrace: true,
runSkipped: true,
precompiledPath: "/tmp/js",
- platforms: [TestPlatform.chrome]));
+ platforms: [
+ new PlatformSelection(TestPlatform.chrome.identifier)
+ ]));
expect(merged.jsTrace, isTrue);
expect(merged.runSkipped, isTrue);
expect(merged.precompiledPath, equals("/tmp/js"));
- expect(merged.platforms, equals([TestPlatform.chrome]));
+ expect(merged.platforms, equals([TestPlatform.chrome.identifier]));
});
test(
@@ -54,18 +59,20 @@ void main() {
jsTrace: false,
runSkipped: true,
precompiledPath: "/tmp/js",
- platforms: [TestPlatform.chrome]);
+ platforms: [new PlatformSelection(TestPlatform.chrome.identifier)]);
var newer = new SuiteConfiguration(
jsTrace: true,
runSkipped: false,
precompiledPath: "../js",
- platforms: [TestPlatform.dartium]);
+ platforms: [
+ new PlatformSelection(TestPlatform.dartium.identifier)
+ ]);
var merged = older.merge(newer);
expect(merged.jsTrace, isTrue);
expect(merged.runSkipped, isFalse);
expect(merged.precompiledPath, equals("../js"));
- expect(merged.platforms, equals([TestPlatform.dartium]));
+ expect(merged.platforms, equals([TestPlatform.dartium.identifier]));
});
});
diff --git a/test/runner/configuration/top_level_error_test.dart b/test/runner/configuration/top_level_error_test.dart
index 712cf5ec9e3e7720c2bbc338b2626fcd0174b57c..0eb4a4449b2b35048d0461afb994dad6e28206af 100644
--- a/test/runner/configuration/top_level_error_test.dart
+++ b/test/runner/configuration/top_level_error_test.dart
@@ -281,7 +281,8 @@ void main() {
.create();
var test = await runTest(["test.dart"]);
- expect(test.stderr, containsInOrder(["Platforms must be strings", "^^"]));
+ expect(test.stderr,
+ containsInOrder(["Platform name must be a string", "^^"]));
await test.shouldExit(exit_codes.data);
});
@@ -294,7 +295,9 @@ void main() {
}))
.create();
- var test = await runTest(["test.dart"]);
+ await d.dir("test").create();
+
+ var test = await runTest([]);
expect(test.stderr, containsInOrder(['Unknown platform "foo"', "^^^^^"]));
await test.shouldExit(exit_codes.data);
});
diff --git a/test/runner/parse_metadata_test.dart b/test/runner/parse_metadata_test.dart
index 4da4ddcd9d4e58856fbab7214670a5087dfd7dd0..343307ec9b5f6bdc7ef9f6285ae12e6e858a423e 100644
--- a/test/runner/parse_metadata_test.dart
+++ b/test/runner/parse_metadata_test.dart
@@ -27,21 +27,21 @@ void main() {
test("returns empty metadata for an empty file", () {
new File(_path).writeAsStringSync("");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.testOn, equals(PlatformSelector.all));
expect(metadata.timeout.scaleFactor, equals(1));
});
test("ignores irrelevant annotations", () {
new File(_path).writeAsStringSync("@Fblthp\n@Fblthp.foo\nlibrary foo;");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.testOn, equals(PlatformSelector.all));
});
test("parses a prefixed annotation", () {
new File(_path).writeAsStringSync("@foo.TestOn('vm')\n"
"import 'package:test/test.dart' as foo;");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.testOn.evaluate(TestPlatform.vm), isTrue);
expect(metadata.testOn.evaluate(TestPlatform.chrome), isFalse);
});
@@ -49,54 +49,54 @@ void main() {
group("@TestOn:", () {
test("parses a valid annotation", () {
new File(_path).writeAsStringSync("@TestOn('vm')\nlibrary foo;");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.testOn.evaluate(TestPlatform.vm), isTrue);
expect(metadata.testOn.evaluate(TestPlatform.chrome), isFalse);
});
test("ignores a constructor named TestOn", () {
new File(_path).writeAsStringSync("@foo.TestOn('foo')\nlibrary foo;");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.testOn, equals(PlatformSelector.all));
});
group("throws an error for", () {
test("a named constructor", () {
new File(_path).writeAsStringSync("@TestOn.name('foo')\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("no argument list", () {
new File(_path).writeAsStringSync("@TestOn\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("an empty argument list", () {
new File(_path).writeAsStringSync("@TestOn()\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a named argument", () {
new File(_path)
.writeAsStringSync("@TestOn(expression: 'foo')\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("multiple arguments", () {
new File(_path)
.writeAsStringSync("@TestOn('foo', 'bar')\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a non-string argument", () {
new File(_path).writeAsStringSync("@TestOn(123)\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("multiple @TestOns", () {
new File(_path)
.writeAsStringSync("@TestOn('foo')\n@TestOn('bar')\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
});
});
@@ -113,7 +113,7 @@ void main() {
library foo;
""");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(
metadata.timeout.duration,
equals(new Duration(
@@ -130,7 +130,7 @@ library foo;
library foo;
""");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.timeout.scaleFactor, equals(1));
});
@@ -140,7 +140,7 @@ library foo;
library foo;
""");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.timeout.scaleFactor, equals(0.5));
});
@@ -150,95 +150,95 @@ library foo;
library foo;
""");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.timeout, same(Timeout.none));
});
test("ignores a constructor named Timeout", () {
new File(_path).writeAsStringSync("@foo.Timeout('foo')\nlibrary foo;");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.timeout.scaleFactor, equals(1));
});
group("throws an error for", () {
test("an unknown named constructor", () {
new File(_path).writeAsStringSync("@Timeout.name('foo')\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("no argument list", () {
new File(_path).writeAsStringSync("@Timeout\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("an empty argument list", () {
new File(_path).writeAsStringSync("@Timeout()\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("an argument list for Timeout.none", () {
new File(_path).writeAsStringSync("@Timeout.none()\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a named argument", () {
new File(_path).writeAsStringSync(
"@Timeout(duration: const Duration(seconds: 1))\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("multiple arguments", () {
new File(_path)
.writeAsStringSync("@Timeout.factor(1, 2)\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a non-Duration argument", () {
new File(_path).writeAsStringSync("@Timeout(10)\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a non-num argument", () {
new File(_path)
.writeAsStringSync("@Timeout.factor('foo')\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("multiple @Timeouts", () {
new File(_path).writeAsStringSync(
"@Timeout.factor(1)\n@Timeout.factor(2)\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
group("a Duration with", () {
test("a non-const constructor", () {
new File(_path)
.writeAsStringSync("@Timeout(new Duration(1))\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a named constructor", () {
new File(_path).writeAsStringSync(
"@Timeout(const Duration.name(seconds: 1))\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a positional argument", () {
new File(_path)
.writeAsStringSync("@Timeout(const Duration(1))\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("an unknown named argument", () {
new File(_path).writeAsStringSync(
"@Timeout(const Duration(name: 1))\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a duplicate named argument", () {
new File(_path).writeAsStringSync(
"@Timeout(const Duration(seconds: 1, seconds: 1))\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
});
});
@@ -247,54 +247,54 @@ library foo;
group("@Skip:", () {
test("parses a valid annotation", () {
new File(_path).writeAsStringSync("@Skip()\nlibrary foo;");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.skip, isTrue);
expect(metadata.skipReason, isNull);
});
test("parses a valid annotation with a reason", () {
new File(_path).writeAsStringSync("@Skip('reason')\nlibrary foo;");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.skip, isTrue);
expect(metadata.skipReason, equals('reason'));
});
test("ignores a constructor named Skip", () {
new File(_path).writeAsStringSync("@foo.Skip('foo')\nlibrary foo;");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.skip, isFalse);
});
group("throws an error for", () {
test("a named constructor", () {
new File(_path).writeAsStringSync("@Skip.name('foo')\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("no argument list", () {
new File(_path).writeAsStringSync("@Skip\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a named argument", () {
new File(_path).writeAsStringSync("@Skip(reason: 'foo')\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("multiple arguments", () {
new File(_path).writeAsStringSync("@Skip('foo', 'bar')\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a non-string argument", () {
new File(_path).writeAsStringSync("@Skip(123)\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("multiple @Skips", () {
new File(_path)
.writeAsStringSync("@Skip('foo')\n@Skip('bar')\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
});
});
@@ -302,13 +302,13 @@ library foo;
group("@Tags:", () {
test("parses a valid annotation", () {
new File(_path).writeAsStringSync("@Tags(const ['a'])\nlibrary foo;");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.tags, equals(["a"]));
});
test("ignores a constructor named Tags", () {
new File(_path).writeAsStringSync("@foo.Tags(const ['a'])\nlibrary foo;");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.tags, isEmpty);
});
@@ -316,34 +316,34 @@ library foo;
test("a named constructor", () {
new File(_path)
.writeAsStringSync("@Tags.name(const ['a'])\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("no argument list", () {
new File(_path).writeAsStringSync("@Tags\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a named argument", () {
new File(_path).writeAsStringSync("@Tags(tags: ['a'])\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("multiple arguments", () {
new File(_path)
.writeAsStringSync("@Tags(const ['a'], ['b'])\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a non-list argument", () {
new File(_path).writeAsStringSync("@Tags('a')\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("multiple @Tags", () {
new File(_path).writeAsStringSync(
"@Tags(const ['a'])\n@Tags(const ['b'])\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
});
});
@@ -356,7 +356,7 @@ library foo;
'vm': const [const Skip(), const Timeout.factor(3)]
})
library foo;""");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
var key = metadata.onPlatform.keys.first;
expect(key.evaluate(TestPlatform.chrome), isTrue);
@@ -374,7 +374,7 @@ library foo;""");
test("ignores a constructor named OnPlatform", () {
new File(_path).writeAsStringSync("@foo.OnPlatform('foo')\nlibrary foo;");
- var metadata = parseMetadata(_path);
+ var metadata = parseMetadata(_path, new Set());
expect(metadata.testOn, equals(PlatformSelector.all));
});
@@ -382,70 +382,70 @@ library foo;""");
test("a named constructor", () {
new File(_path)
.writeAsStringSync("@OnPlatform.name(const {})\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("no argument list", () {
new File(_path).writeAsStringSync("@OnPlatform\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("an empty argument list", () {
new File(_path).writeAsStringSync("@OnPlatform()\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a named argument", () {
new File(_path)
.writeAsStringSync("@OnPlatform(map: const {})\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("multiple arguments", () {
new File(_path)
.writeAsStringSync("@OnPlatform(const {}, const {})\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a non-map argument", () {
new File(_path)
.writeAsStringSync("@OnPlatform(const Skip())\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a non-const map", () {
new File(_path).writeAsStringSync("@OnPlatform({})\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a map with a non-String key", () {
new File(_path).writeAsStringSync(
"@OnPlatform(const {1: const Skip()})\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a map with a unparseable key", () {
new File(_path).writeAsStringSync(
"@OnPlatform(const {'invalid': const Skip()})\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a map with an invalid value", () {
new File(_path).writeAsStringSync(
"@OnPlatform(const {'vm': const TestOn('vm')})\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("a map with an invalid value in a list", () {
new File(_path).writeAsStringSync(
"@OnPlatform(const {'vm': [const TestOn('vm')]})\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
test("multiple @OnPlatforms", () {
new File(_path).writeAsStringSync(
"@OnPlatform(const {})\n@OnPlatform(const {})\nlibrary foo;");
- expect(() => parseMetadata(_path), throwsFormatException);
+ expect(() => parseMetadata(_path, new Set()), throwsFormatException);
});
});
});