Output Expectations
Smoke tests are useful for identifying if a program is broken, but they don't confirm correct functionality. When running commands manually in the terminal, the initial check for correct operation is through their output: does it match expectations, or are there error messages?
This is what output expectations are all about.
Output Expectation Types
The simplest variant of an output expectation was already demonstrated previously when the test for the jq --version command was created:
```scrut
$ jq --version
jq-1.7.1
```
The line that reads jq-1.7.1 is what Scrut calls a equal output expectation. It could also have been written like this:
```scrut
$ jq --version
jq-1.7.1 (equal)
```
The suffix (equal) here tells Scrut that the output is expected exactly as written before. There are other types, for example:
Glob: Match all
```scrut
$ jq --version
jq-* (glob)
```
The * wildcard in jq-* matches anything. Scrut would accept any string that starts with jq-.
Regex: Match precisely
```scrut
$ jq --version
jq-1\.\d+\.\d+ (regex)
```
The 1\.\d+\.\d+ regular expression matches any version number that starts with a one and is followed by two numbers, separated by a dot.
Learn more about output expectations in the Reference > Fundamentals > Output Expectations later.
Practical Example
Let's take a look at a practical example. Using jq some JSON input data is required. Following the same example as provided in the jq tutorial itself: Let's go with the Github API.
$ curl 'https://api.github.com/repos/jqlang/jq/commits?per_page=5'
[
{
"sha": "947fcbbb1fedbdd6021ef3f93782a500e32d5dcd",
"node_id": "C_kwDOAE3WVdoAKDk0N2ZjYmJiMWZlZGJkZDYwMjFlZjNmOTM3ODJhNTAwZTMyZDVkY2Q",
"commit": {
"author": {
"name": "dependabot[bot]",
"email": "49699333+dependabot[bot]@users.noreply.github.com",
"date": "2025-03-28T00:57:51Z"
},
"committer": {
"name": "GitHub",
"email": "noreply@github.com",
"date": "2025-03-28T00:57:51Z"
},
"message": "--%<--",
"tree": {
"sha": "8b30ae1036b74c4acf02c674f75db8f1ce014aa4",
"url": "https://api.github.com/repos/jqlang/jq/git/trees/8b30ae1036b74c4acf02c674f75db8f1ce014aa4"
},
"url": "https://api.github.com/repos/jqlang/jq/git/commits/947fcbbb1fedbdd6021ef3f93782a500e32d5dcd",
"comment_count": 0,
"verification": {
"verified": true,
"reason": "valid",
"signature": "--%<--",
"payload": "--%<--",
"verified_at": "2025-03-28T00:57:55Z"
}
},
"url": "https://api.github.com/repos/jqlang/jq/commits/947fcbbb1fedbdd6021ef3f93782a500e32d5dcd",
"html_url": "https://github.com/jqlang/jq/commit/947fcbbb1fedbdd6021ef3f93782a500e32d5dcd",
"comments_url": "https://api.github.com/repos/jqlang/jq/commits/947fcbbb1fedbdd6021ef3f93782a500e32d5dcd/comments",
"author": {
--%<--
That is a lot of output. Let's use jq to boil that down into something more manageable. Say, as CSV with the first column the commit date and the second column the author's name:
$ curl 'https://api.github.com/repos/jqlang/jq/commits?per_page=5' | \
jq -r '.[] | .commit.committer.date + "," + .commit.author.name'
2025-03-28T00:57:51Z,dependabot[bot]
2025-03-28T00:56:51Z,dependabot[bot]
2025-03-28T00:55:39Z,dependabot[bot]
2025-03-27T23:43:06Z,itchyny
2025-03-27T23:42:44Z,itchyny
The output you will see when executing the above curl command will contain more lines than are shown above:
$ curl 'https://api.github.com/repos/jqlang/jq/commits?per_page=5' | \
jq -r '.[] | .commit.committer.date + "," + .commit.author.name'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 28935 100 28935 0 0 250k 0 --:--:-- --:--:-- --:--:-- 250k
2025-03-28T00:57:51Z,dependabot[bot]
2025-03-28T00:56:51Z,dependabot[bot]
2025-03-28T00:55:39Z,dependabot[bot]
2025-03-27T23:43:06Z,itchyny
2025-03-27T23:42:44Z,itchyny
The first three lines above that curl prints are written to STDERR. Only the actual result content (i.e. the web request body) is printed to STDOUT and piped to jq which transforms them into five lines that are finally printed on STDOUT.
Scrut only considers STDOUT by default. More about how to change this behavior here.
To go from here to a test either use scrut create with the above command, or open a new file and add the commandline and output yourself:
# Output Expectations
```scrut
$ curl 'https://api.github.com/repos/jqlang/jq/commits?per_page=5' | \
> jq -r '.[] | .commit.committer.date + "," + .commit.author.name'
2025-03-28T00:57:51Z,dependabot[bot]
2025-03-28T00:56:51Z,dependabot[bot]
2025-03-28T00:55:39Z,dependabot[bot]
2025-03-27T23:43:06Z,itchyny
2025-03-27T23:42:44Z,itchyny
```
Shell expressions that span multiple lines need to be prefixed with a >, like so:
```scrut
$ line 1
> line 2
> line N
```
The whole expression will then be piped into a bash process and executed. If you do not concatenate the lines with something && or explicitly set -e, then the exit code will be from the last executed line.
```scrut
$ line 1 && \
> line 2 && \
> line N
```
Generalize Output Expectation
Running scrut test tests/expectations.md right after creating the file should succeed. Should, because the output is not stable. It is not guaranteed to be the same tomorrow, or even in a few minutes. To make it more stable the test can be changed:
- from with the JSON input from the github API this exact output is expected
- to with the JSON input from the github API 5 lines separated by a comma are expected
```scrut
$ curl 'https://api.github.com/repos/jqlang/jq/commits?per_page=5' | \
> jq -r '.[] | .commit.committer.date + "," + .commit.author.name'
*,* (glob)
*,* (glob)
*,* (glob)
*,* (glob)
*,* (glob)
```
Obviously this test lost precision compared to the previous variant, but on the plus side: it won't break as easy, it is still meaningful and it could break if, say, the jq concatenate operator + malfunctions. This could be made more precise using 20*T*Z,* (glob) to account for the date string, or even use matching regex rules.
Quantifiers for Expectations
The above test could be generalized further. While it probably would not make sense for this case the following would work as well:
```scrut
$ curl 'https://api.github.com/repos/jqlang/jq/commits?per_page=5' | \
> jq -r '.[] | .commit.committer.date + "," + .commit.author.name'
*,* (glob+)
```
Note the + behind the word glob. This is a quantifier. Quantifiers can be used with any output expectation. They make sense when a hard to predict amount of predictable formatted output needs to be accounted for.
Scrut currently understands three quantifiers:
?: Zero or one*: Any amount, including zero+: Any amount, but at least one
More detail in Reference > Fundamentals > Output Expectations > Quantifiers.