Compare commits

...

59 Commits

Author SHA1 Message Date
Adnan Hajdarevic 568c711625 wip 2021-04-03 18:02:40 +02:00
Adnan Hajdarević e329b6d9ff Merge pull request #518 from adhawkins/busybox-tests
Allow tests to run on systems that use busybox (such as Alpine)
2021-03-15 22:43:13 +01:00
Andy Hawkins 181672afcc Allow tests to run on systems that use busybox (such as Alpine) 2021-03-13 16:02:44 +00:00
Adnan Hajdarević d523af1b6c Fixes #497 2021-02-28 09:55:08 +01:00
Adnan Hajdarević 390e3bd772 Merge pull request #503 from TheCatLady/add-alt-docker-images
Add alternative Docker images
2021-01-29 20:08:46 +01:00
TheCatLady 21549749c0 Add alternative Docker images 2021-01-28 12:16:03 -05:00
Adnan Hajdarevic 6184509494 Add build directory to .gitignore 2021-01-26 20:53:07 +01:00
Adnan Hajdarevic b1f69564a3 Merge branch 'development' 2020-12-06 08:42:20 +01:00
Adnan Hajdarevic 159cb4a911 bump version to 2.8.0 2020-12-06 08:42:09 +01:00
Adnan Hajdarević b5af9a3968 Merge pull request #489 from moorereason/iss487-doc-string
Add string parameter example to docs
2020-12-06 08:40:31 +01:00
Adnan Hajdarević 2e4aea4cbc Merge pull request #486 from moorereason/iss439-raw-body
Add option to send raw request body to command
2020-12-06 08:39:55 +01:00
Adnan Hajdarević b6e5b11174 Merge pull request #485 from moorereason/iss234-soft-sig-errors
Add soft signature failure support
2020-12-06 08:39:31 +01:00
Adnan Hajdarević 9dec52c727 Merge pull request #484 from moorereason/iss421-slash-path
Add support for slashes in hook IDs
2020-12-06 08:38:53 +01:00
Cameron Moore f2b536dbad Add string parameter example to docs
Fixes #487
2020-12-05 16:34:49 -06:00
Cameron Moore 62f9c01cab Add option to send raw request body to command
The existing `entire-payload` option sends a JSON representation of the
parsed request body.  Add a new `raw-request-body` source to send the
raw request body.

Fixes #439
2020-11-25 10:20:10 -06:00
Cameron Moore 6d2f26d952 Add soft signature failure support
Add a new trigger-signature-soft-failures option to allow soft signature
failures in Or rules.

Fixes #234
2020-11-24 21:16:57 -06:00
Cameron Moore c2ffd465c4 Add support for slashes in hook IDs
When matching variables in routes, gorilla/mux uses a default pattern of
"[^/]+", thereby prohibiting slashes in variable matching.  Override the
default pattern to remove this restriction.

See https://github.com/gorilla/mux/blob/v1.8.0/regexp.go#L50

Fixes #421
2020-11-24 16:56:54 -06:00
Adnan Hajdarević 3e18a060ae Merge pull request #479 from moorereason/iss312-http-request
Add request source
2020-11-21 18:58:19 +01:00
Cameron Moore 6f5962f8f2 Use strings.ToLower on source name parameters 2020-11-21 10:00:03 -06:00
Cameron Moore 346c761ef6 Add request source
Add "request" source with support for "method" and "remote-addr"
parameters.  Both values are taken from the raw http.Request object.

Fixes #312
2020-11-20 16:32:55 -06:00
Adnan Hajdarević e513eb4bf4 Merge pull request #477 from moorereason/refactor-req-parsing
Move some request parsing into hook package
2020-11-19 19:44:33 +01:00
Cameron Moore 22c8a1670b Move some request parsing into hook package
Trying to simplify hookHandler.  No functional changes introduced.
2020-11-17 15:00:58 -06:00
Adnan Hajdarević 9c7f8c1ac4 Update README.md 2020-11-05 23:20:36 +01:00
Adnan Hajdarević 4fadb1171f Merge pull request #472 from moorereason/iss471-sc
Fix OrRule logic on parameter lookup failures
2020-10-26 14:09:19 +01:00
Cameron Moore dc184d2737 Fix OrRule logic on parameter lookup failures
Fixes #471
2020-10-24 11:40:27 -05:00
Adnan Hajdarević 7467933680 Merge pull request #469 from Maximization/patch-1
Add guide to the README
2020-10-14 15:31:17 +02:00
Maxim Orlov fd50118712 Add guide to the README 2020-10-13 16:12:01 +02:00
Adnan Hajdarević 67c317e741 Merge pull request #465 from moorereason/gh-action-tests
Add Github Action to build & run tests
2020-10-02 07:52:06 +02:00
Adnan Hajdarević ab3ff0343e Merge pull request #463 from moorereason/iss400
Document YAML support
2020-09-29 09:11:41 +02:00
Cameron Moore f007fa5280 Simplify build workflow 2020-09-28 21:18:38 -05:00
Cameron Moore a904537367 Add build badge to README.md 2020-09-28 20:48:39 -05:00
Cameron Moore 0814b10a16 Add Github Action to build & run tests 2020-09-28 17:20:22 -05:00
Cameron Moore d279505930 Document YAML support
Fixes #400
Updates #288
2020-09-28 14:44:21 -05:00
Adnan Hajdarević 0f4bbfac9f Merge pull request #461 from moorereason/iss289-hmac-cleanup
Transition payload hash option names to hmac
2020-09-28 04:47:24 +02:00
Cameron Moore 6bbf14f7d9 Transition payload hash option names to hmac
The payload-hash-* options are imprecisely named. Clarify their function
as HMAC validations by renaming them. The existing options will continue
to work but are deprecated.  Log a warning if the old options are used.

All tests, examples, and documentation are updated.

Fixes #289
2020-09-27 20:24:36 -05:00
Adnan Hajdarević 6797bf7cf7 Merge pull request #462 from moorereason/req-context
Add Request object to hook package to simplify API
2020-09-26 15:20:32 +02:00
Cameron Moore c6603894c1 Add Request object to hook package to simplify API
To avoid having to pass around so many parameters to the hook package,
create a Request object to store all request-specific data.  Update APIs
accordingly.
2020-09-25 19:46:06 -05:00
Adnan Hajdarević b8498c564d Merge pull request #460 from moorereason/iss456-log-exec-err
Log stdlib error on failed exec.LookPath
2020-09-25 07:13:35 +02:00
Cameron Moore dd5fa20415 Log stdlib error on failed exec.LookPath
The error returned by exec.LookPath was never surfaced to the user.
Without that detail, the user can't tell the difference between a
non-existent path and a permissions issue.

Additionally, when ExecuteCommand is an absolute path, we were still
attempting to prepend the CommandWorkingDirectory if the ExecuteCommand
was not found, which made it difficult to know which path the user
intended to execute.

This commit simplifies the logic to avoid multiple attempts with
ExecuteCommand is an absolute path and changes the error message from:

  error locating command: '/path/to/file'

to:

  error in exec: "/path/to/file": stat /path/to/file: no such file or directory
  error in exec: "/path/to/file": permission denied

Fixes #457
2020-09-24 21:02:07 -05:00
Adnan Hajdarević c7a8fbc929 Merge pull request #449 from moorereason/feature/448-return-json
Update ExtractParameterAsString to return JSON on complex types
2020-08-03 08:08:38 +02:00
Adnan Hajdarević 04ca211531 Merge pull request #446 from moorereason/feature/json-array
Add support for top-level JSON array in payload
2020-08-03 08:07:09 +02:00
Cameron Moore ae5e9e7894 Update ExtractParameterAsString to return JSON on complex types
Fixes #448
2020-07-31 11:58:12 -05:00
Adnan Hajdarević 47e5ae5527 Merge pull request #447 from moorereason/feature/docs-toc
Add a table of contents to some of the docs
2020-07-31 14:47:49 +02:00
Cameron Moore 534e99bf13 Add a table of contents to some of the docs 2020-07-29 17:23:23 -05:00
Cameron Moore 0e90ccb441 Add support for top-level JSON array in payload
Detect if leading character in JSON payload is an array bracket.  If
found, decode payload into an interface{} and then save the results into
payload["root"].  References to payload values would need to reference
the leading, "virtual" root node (i.e. "root.0.name").

Fixes #215
2020-07-29 16:56:25 -05:00
Adnan Hajdarević f692da2465 Merge pull request #445 from moorereason/bugfix/require-go14
Bugfix/require go14
2020-07-29 08:53:51 +02:00
Gabe Gałązka fb9b22a118 Change minimum golang version to 1.14 in README 2020-07-27 13:18:35 -05:00
Cameron Moore eefcd7f7d5 Require Go 1.14
When go.mod specifies go 1.14 or higher, the go tools now verify that
vendor/modules.txt is consistent with go.mod.  Fixed by running `go mod
vendor`.
2020-07-27 13:18:18 -05:00
Adnan Hajdarević c4f29b5d8b Merge pull request #432 from moorereason/bugfix/logging-gocritic
Fix issues in logging middleware
2020-05-29 09:43:32 +02:00
Adnan Hajdarević dd84a68483 Merge pull request #431 from moorereason/bugfix/pidfile-tighten
Tighten file permissions on pidfile creation
2020-05-29 09:42:55 +02:00
Cameron Moore c9199d62e4 Tighten file permissions on pidfile creation
Fixes report from gosec: "G306: Expect WriteFile permissions to be 0600
or less."  Also, use new octal number formatting.
2020-05-28 18:23:02 -05:00
Cameron Moore 3d824b47b7 Rename var to avoid shadowing bytes package
importShadow: shadow of imported package 'bytes' (gocritic)
2020-05-28 18:20:07 -05:00
Cameron Moore cc98de88ce Fix godoc comment on LogEntry.Panic 2020-05-28 18:17:41 -05:00
Adnan Hajdarević e71b45b28f Merge pull request #427 from moorereason/feature/empty-payload-signature
Warn on failed validate of empty payload signature
2020-05-23 09:28:09 +02:00
Cameron Moore 41ac427a89 Warn on failed validate of empty payload signature
If signature validation fails on an empty payload, append a note to the
end of the error message.

Updates #423
2020-05-22 14:02:12 -05:00
Adnan Hajdarević 7b3c5fd028 Merge pull request #426 from moorereason/bugfix/issue425
Fix request dumper
2020-05-22 07:15:43 +02:00
Cameron Moore 526c9a20ac Fix request dumper
The existing code had a bug in printing request params.  Simplify the
request logger by using httputil.DumpRequest.

Also print the request before handing it downstream.

Fixes #425
2020-05-21 17:47:55 -05:00
Adnan Hajdarević a75ab4f92f Merge pull request #420 from adnanh/master
Update README.md
2020-05-14 14:22:48 +02:00
Adnan Hajdarević 345bf3d409 Update README.md 2020-05-14 14:22:24 +02:00
160 changed files with 24593 additions and 90839 deletions
+23
View File
@@ -0,0 +1,23 @@
name: build
on: [push, pull_request]
jobs:
build:
strategy:
matrix:
go-version: [1.14.x, 1.15.x]
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v2
- uses: actions/setup-go@v2
with:
go-version: ${{ matrix.go-version }}
id: go
- name: Build
run: go build -v
- name: Test
run: go test -v ./...
+1
View File
@@ -3,3 +3,4 @@
coverage coverage
webhook webhook
/test/hookecho /test/hookecho
build
+1 -1
View File
@@ -1,7 +1,7 @@
language: go language: go
go: go:
- 1.13.x - 1.14.x
- master - master
os: os:
+25 -7
View File
@@ -1,4 +1,4 @@
# What is webhook? # What is webhook? ![build-status][badge]
<img src="https://github.com/adnanh/webhook/raw/development/docs/logo/logo-128x128.png" alt="Webhook" align="left" /> <img src="https://github.com/adnanh/webhook/raw/development/docs/logo/logo-128x128.png" alt="Webhook" align="left" />
@@ -22,15 +22,18 @@ Everything else is the responsibility of the command's author.
If you don't have time to waste configuring, hosting, debugging and maintaining your webhook instance, we offer a __SaaS__ solution that has all of the capabilities webhook provides, plus a lot more, and all that packaged in a nice friendly web interface. If you are interested, find out more at [hookdoo website](https://www.hookdoo.com/?ref=github-webhook-readme). If you have any questions, you can contact us at info@hookdoo.com If you don't have time to waste configuring, hosting, debugging and maintaining your webhook instance, we offer a __SaaS__ solution that has all of the capabilities webhook provides, plus a lot more, and all that packaged in a nice friendly web interface. If you are interested, find out more at [hookdoo website](https://www.hookdoo.com/?ref=github-webhook-readme). If you have any questions, you can contact us at info@hookdoo.com
#
<a href="https://www.hookdeck.io/?ref=adnanh-webhook"><img src="http://hajdarevic.net/hookdeck-logo.svg" height="17" alt="hookdeck" align="left" /></a> If you need a way of inspecting, monitoring and replaying webhooks without the back and forth troubleshooting, [give Hookdeck a try!](https://www.hookdeck.io/?ref=adnanh-webhook)
# Getting started # Getting started
## Installation ## Installation
### Building from source ### Building from source
To get started, first make sure you've properly set up your [Go](http://golang.org/doc/install) 1.12 or newer environment and then run To get started, first make sure you've properly set up your [Go](http://golang.org/doc/install) 1.14 or newer environment and then run
```bash ```bash
$ go get github.com/adnanh/webhook $ go build github.com/adnanh/webhook
``` ```
to get the latest version of the [webhook][w]. to build the latest version of the [webhook][w].
### Using package manager ### Using package manager
#### Snap store #### Snap store
@@ -46,7 +49,9 @@ If you are using Debian linux ("stretch" or later), you can install webhook usin
Prebuilt binaries for different architectures are available at [GitHub Releases](https://github.com/adnanh/webhook/releases). Prebuilt binaries for different architectures are available at [GitHub Releases](https://github.com/adnanh/webhook/releases).
## Configuration ## Configuration
Next step is to define some hooks you want [webhook][w] to serve. Begin by creating an empty file named `hooks.json`. This file will contain an array of hooks the [webhook][w] will serve. Check [Hook definition page](docs/Hook-Definition.md) to see the detailed description of what properties a hook can contain, and how to use them. Next step is to define some hooks you want [webhook][w] to serve.
[webhook][w] supports JSON or YAML configuration files, but we'll focus primarily on JSON in the following example.
Begin by creating an empty file named `hooks.json`. This file will contain an array of hooks the [webhook][w] will serve. Check [Hook definition page](docs/Hook-Definition.md) to see the detailed description of what properties a hook can contain, and how to use them.
Let's define a simple hook named `redeploy-webhook` that will run a redeploy script located in `/var/scripts/redeploy.sh`. Make sure that your bash script has `#!/bin/sh` shebang on top. Let's define a simple hook named `redeploy-webhook` that will run a redeploy script located in `/var/scripts/redeploy.sh`. Make sure that your bash script has `#!/bin/sh` shebang on top.
@@ -61,6 +66,13 @@ Our `hooks.json` file will now look like this:
] ]
``` ```
**NOTE:** If you prefer YAML, the equivalent `hooks.yaml` file would be:
```yaml
- id: redeploy-webhook
execute-command: "/var/scripts/redeploy.sh"
command-working-directory: "/var/webhook"
```
You can now run [webhook][w] using You can now run [webhook][w] using
```bash ```bash
$ /path/to/webhook -hooks hooks.json -verbose $ /path/to/webhook -hooks hooks.json -verbose
@@ -90,7 +102,7 @@ All files are ignored unless they match one of the following criteria:
In either case, the given file part will be parsed as JSON and added to the `payload` map. In either case, the given file part will be parsed as JSON and added to the `payload` map.
## Templates ## Templates
[webhook][w] can parse the `hooks.json` input file as a Go template when given the `-template` [CLI parameter](docs/Webhook-Parameters.md). See the [Templates page](docs/Templates.md) for more details on template usage. [webhook][w] can parse the hooks configuration file as a Go template when given the `-template` [CLI parameter](docs/Webhook-Parameters.md). See the [Templates page](docs/Templates.md) for more details on template usage.
## Using HTTPS ## Using HTTPS
[webhook][w] by default serves hooks using http. If you want [webhook][w] to serve secure content using https, you can use the `-secure` flag while starting [webhook][w]. Files containing a certificate and matching private key for the server must be provided using the `-cert /path/to/cert.pem` and `-key /path/to/key.pem` flags. If the certificate is signed by a certificate authority, the cert file should be the concatenation of the server's certificate followed by the CA's certificate. [webhook][w] by default serves hooks using http. If you want [webhook][w] to serve secure content using https, you can use the `-secure` flag while starting [webhook][w]. Files containing a certificate and matching private key for the server must be provided using the `-cert /path/to/cert.pem` and `-key /path/to/key.pem` flags. If the certificate is signed by a certificate authority, the cert file should be the concatenation of the server's certificate followed by the CA's certificate.
@@ -101,12 +113,16 @@ TLS version and cipher suite selection flags are available from the command line
If you want to set CORS headers, you can use the `-header name=value` flag while starting [webhook][w] to set the appropriate CORS headers that will be returned with each response. If you want to set CORS headers, you can use the `-header name=value` flag while starting [webhook][w] to set the appropriate CORS headers that will be returned with each response.
## Interested in running webhook inside of a Docker container? ## Interested in running webhook inside of a Docker container?
You can use [almir/webhook](https://hub.docker.com/r/almir/webhook/) docker image, or create your own (please read [this discussion](https://github.com/adnanh/webhook/issues/63)). You can use one of the following Docker images, or create your own (please read [this discussion](https://github.com/adnanh/webhook/issues/63)):
- [almir/webhook](https://github.com/almir/docker-webhook)
- [roxedus/webhook](https://github.com/Roxedus/docker-webhook)
- [thecatlady/webhook](https://github.com/thecatlady/docker-webhook)
## Examples ## Examples
Check out [Hook examples page](docs/Hook-Examples.md) for more complex examples of hooks. Check out [Hook examples page](docs/Hook-Examples.md) for more complex examples of hooks.
### Guides featuring webhook ### Guides featuring webhook
- [Plex 2 Telegram](https://gitlab.com/-/snippets/1972594) by [@psyhomb](https://github.com/psyhomb)
- [Webhook & JIRA](https://sites.google.com/site/mrxpalmeiras/notes/jira-webhooks) by [@perfecto25](https://github.com/perfecto25) - [Webhook & JIRA](https://sites.google.com/site/mrxpalmeiras/notes/jira-webhooks) by [@perfecto25](https://github.com/perfecto25)
- [Trigger Ansible AWX job runs on SCM (e.g. git) commit](http://jpmens.net/2017/10/23/trigger-awx-job-runs-on-scm-commit/) by [@jpmens](http://mens.de/) - [Trigger Ansible AWX job runs on SCM (e.g. git) commit](http://jpmens.net/2017/10/23/trigger-awx-job-runs-on-scm-commit/) by [@jpmens](http://mens.de/)
- [Deploy using GitHub webhooks](https://davidauthier.wearemd.com/blog/deploy-using-github-webhooks.html) by [@awea](https://davidauthier.wearemd.com) - [Deploy using GitHub webhooks](https://davidauthier.wearemd.com/blog/deploy-using-github-webhooks.html) by [@awea](https://davidauthier.wearemd.com)
@@ -120,6 +136,7 @@ Check out [Hook examples page](docs/Hook-Examples.md) for more complex examples
- [Adventures in webhooks](https://medium.com/@draketech/adventures-in-webhooks-2d6584501c62) by [Drake](https://medium.com/@draketech) - [Adventures in webhooks](https://medium.com/@draketech/adventures-in-webhooks-2d6584501c62) by [Drake](https://medium.com/@draketech)
- [GitHub pro tips](http://notes.spencerlyon.com/2016/01/04/github-pro-tips/) by [Spencer Lyon](http://notes.spencerlyon.com/) - [GitHub pro tips](http://notes.spencerlyon.com/2016/01/04/github-pro-tips/) by [Spencer Lyon](http://notes.spencerlyon.com/)
- [XiaoMi Vacuum + Amazon Button = Dash Cleaning](https://www.instructables.com/id/XiaoMi-Vacuum-Amazon-Button-Dash-Cleaning/) by [c0mmensal](https://www.instructables.com/member/c0mmensal/) - [XiaoMi Vacuum + Amazon Button = Dash Cleaning](https://www.instructables.com/id/XiaoMi-Vacuum-Amazon-Button-Dash-Cleaning/) by [c0mmensal](https://www.instructables.com/member/c0mmensal/)
- [Set up Automated Deployments From Github With Webhook](https://maximorlov.com/automated-deployments-from-github-with-webhook/) by [Maxim Orlov](https://twitter.com/_maximization)
- VIDEO: [Gitlab CI/CD configuration using Docker and adnanh/webhook to deploy on VPS - Tutorial #1](https://www.youtube.com/watch?v=Qhn-lXjyrZA&feature=youtu.be) by [Yes! Let's Learn Software Engineering](https://www.youtube.com/channel/UCH4XJf2BZ_52fbf8fOBMF3w) - VIDEO: [Gitlab CI/CD configuration using Docker and adnanh/webhook to deploy on VPS - Tutorial #1](https://www.youtube.com/watch?v=Qhn-lXjyrZA&feature=youtu.be) by [Yes! Let's Learn Software Engineering](https://www.youtube.com/channel/UCH4XJf2BZ_52fbf8fOBMF3w)
- ... - ...
- Want to add your own? Open an Issue or create a PR :-) - Want to add your own? Open an Issue or create a PR :-)
@@ -202,3 +219,4 @@ THE SOFTWARE.
[w]: https://github.com/adnanh/webhook [w]: https://github.com/adnanh/webhook
[wc]: https://github.com/adnanh/webhook-contrib [wc]: https://github.com/adnanh/webhook-contrib
[badge]: https://github.com/adnanh/webhook/workflows/build/badge.svg
Binary file not shown.
+3 -1
View File
@@ -1,5 +1,6 @@
# Hook definition # Hook definition
Hooks are defined as JSON objects. Please note that in order to be considered valid, a hook object must contain the `id` and `execute-command` properties. All other properties are considered optional.
Hooks are defined as objects in the JSON or YAML hooks configuration file. Please note that in order to be considered valid, a hook object must contain the `id` and `execute-command` properties. All other properties are considered optional.
## Properties (keys) ## Properties (keys)
@@ -21,6 +22,7 @@ Hooks are defined as JSON objects. Please note that in order to be considered va
* `pass-file-to-command` - specifies a list of entries that will be serialized as a file. Incoming [data](Referencing-Request-Values.md) will be serialized in a request-temporary-file (otherwise parallel calls of the hook would lead to concurrent overwritings of the file). The filename to be addressed within the subsequent script is provided via an environment variable. Use `envname` to specify the name of the environment variable. If `envname` is not provided `HOOK_` and the name used to reference the request value are used. Defining `command-working-directory` will store the file relative to this location, if not provided, the systems temporary file directory will be used. If `base64decode` is true, the incoming binary data will be base 64 decoded prior to storing it into the file. By default the corresponding file will be removed after the webhook exited. * `pass-file-to-command` - specifies a list of entries that will be serialized as a file. Incoming [data](Referencing-Request-Values.md) will be serialized in a request-temporary-file (otherwise parallel calls of the hook would lead to concurrent overwritings of the file). The filename to be addressed within the subsequent script is provided via an environment variable. Use `envname` to specify the name of the environment variable. If `envname` is not provided `HOOK_` and the name used to reference the request value are used. Defining `command-working-directory` will store the file relative to this location, if not provided, the systems temporary file directory will be used. If `base64decode` is true, the incoming binary data will be base 64 decoded prior to storing it into the file. By default the corresponding file will be removed after the webhook exited.
* `trigger-rule` - specifies the rule that will be evaluated in order to determine should the hook be triggered. Check [Hook rules page](Hook-Rules.md) to see the list of valid rules and their usage * `trigger-rule` - specifies the rule that will be evaluated in order to determine should the hook be triggered. Check [Hook rules page](Hook-Rules.md) to see the list of valid rules and their usage
* `trigger-rule-mismatch-http-response-code` - specifies the HTTP status code to be returned when the trigger rule is not satisfied * `trigger-rule-mismatch-http-response-code` - specifies the HTTP status code to be returned when the trigger rule is not satisfied
* `trigger-signature-soft-failures` - allow signature validation failures within Or rules; by default, signature failures are treated as errors.
## Examples ## Examples
Check out [Hook examples page](Hook-Examples.md) for more complex examples of hooks. Check out [Hook examples page](Hook-Examples.md) for more complex examples of hooks.
+107 -4
View File
@@ -1,5 +1,26 @@
# Hook examples # Hook Examples
This page is still work in progress. Feel free to contribute!
Hooks are defined in a hooks configuration file in either JSON or YAML format,
although the examples on this page all use the JSON format.
🌱 This page is still a work in progress. Feel free to contribute!
### Table of Contents
* [Incoming Github webhook](#incoming-github-webhook)
* [Incoming Bitbucket webhook](#incoming-bitbucket-webhook)
* [Incoming Gitlab webhook](#incoming-gitlab-webhook)
* [Incoming Gogs webhook](#incoming-gogs-webhook)
* [Incoming Gitea webhook](#incoming-gitea-webhook)
* [Slack slash command](#slack-slash-command)
* [A simple webhook with a secret key in GET query](#a-simple-webhook-with-a-secret-key-in-get-query)
* [JIRA Webhooks](#jira-webhooks)
* [Pass File-to-command sample](#pass-file-to-command-sample)
* [Incoming Scalr Webhook](#incoming-scalr-webhook)
* [Travis CI webhook](#travis-ci-webhook)
* [XML Payload](#xml-payload)
* [Multipart Form Data](#multipart-form-data)
* [Pass string arguments to command](#pass-string-arguments-to-command)
## Incoming Github webhook ## Incoming Github webhook
```json ```json
@@ -30,7 +51,7 @@ This page is still work in progress. Feel free to contribute!
{ {
"match": "match":
{ {
"type": "payload-hash-sha1", "type": "payload-hmac-sha1",
"secret": "mysecret", "secret": "mysecret",
"parameter": "parameter":
{ {
@@ -150,7 +171,7 @@ Values in the request body can be accessed in the command or to the match rule b
{ {
"match": "match":
{ {
"type": "payload-hash-sha256", "type": "payload-hmac-sha256",
"secret": "mysecret", "secret": "mysecret",
"parameter": "parameter":
{ {
@@ -425,6 +446,57 @@ Travis sends webhooks as `payload=<JSON_STRING>`, so the payload needs to be par
] ]
``` ```
## JSON Array Payload
If the JSON payload is an array instead of an object, `webhook` will process the payload and place it into a "root" object.
Therefore, references to payload values must begin with `root.`.
For example, given the following payload (taken from the Sendgrid Event Webhook documentation):
```json
[
{
"email": "example@test.com",
"timestamp": 1513299569,
"smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>",
"event": "processed",
"category": "cat facts",
"sg_event_id": "sg_event_id",
"sg_message_id": "sg_message_id"
},
{
"email": "example@test.com",
"timestamp": 1513299569,
"smtp-id": "<14c5d75ce93.dfd.64b469@ismtpd-555>",
"event": "deferred",
"category": "cat facts",
"sg_event_id": "sg_event_id",
"sg_message_id": "sg_message_id",
"response": "400 try again later",
"attempt": "5"
}
]
```
A reference to the second item in the array would look like this:
```json
[
{
"id": "sendgrid",
"execute-command": "{{ .Hookecho }}",
"trigger-rule": {
"match": {
"type": "value",
"parameter": {
"source": "payload",
"name": "root.1.event"
},
"value": "deferred"
}
}
}
]
```
## XML Payload ## XML Payload
Given the following payload: Given the following payload:
@@ -518,3 +590,34 @@ Content-Disposition: form-data; name="thumb"; filename="thumb.jpg"
``` ```
We key off of the `name` attribute in the `Content-Disposition` value. We key off of the `name` attribute in the `Content-Disposition` value.
## Pass string arguments to command
To pass simple string arguments to a command, use the `string` parameter source.
The following example will pass two static string parameters ("-e 123123") to the
`execute-command` before appending the `pusher.email` value from the payload:
```json
[
{
"id": "webhook",
"execute-command": "/home/adnan/redeploy-go-webhook.sh",
"command-working-directory": "/home/adnan/go",
"pass-arguments-to-command":
[
{
"source": "string",
"name": "-e"
},
{
"source": "string",
"name": "123123"
},
{
"source": "payload",
"name": "pusher.email"
}
]
}
]
```
+29 -13
View File
@@ -1,5 +1,20 @@
# Hook rules # Hook rules
### Table of Contents
* [And](#and)
* [Or](#or)
* [Not](#not)
* [Multi-level](#multi-level)
* [Match](#match)
* [Match value](#match-value)
* [Match regex](#match-regex)
* [Match payload-hmac-sha1](#match-payload-hmac-sha1)
* [Match payload-hmac-sha256](#match-payload-hmac-sha256)
* [Match payload-hmac-sha512](#match-payload-hmac-sha512)
* [Match Whitelisted IP range](#match-whitelisted-ip-range)
* [Match scalr-signature](#match-scalr-signature)
## And ## And
*And rule* will evaluate to _true_, if and only if all of the sub rules evaluate to _true_. *And rule* will evaluate to _true_, if and only if all of the sub rules evaluate to _true_.
```json ```json
@@ -95,7 +110,7 @@
"source": "header", "source": "header",
"name": "X-Hub-Signature" "name": "X-Hub-Signature"
}, },
"type": "payload-hash-sha1", "type": "payload-hmac-sha1",
"secret": "mysecret" "secret": "mysecret"
} }
}, },
@@ -135,9 +150,7 @@
*Please note:* Due to technical reasons, _number_ and _boolean_ values in the _match rule_ must be wrapped around with a pair of quotes. *Please note:* Due to technical reasons, _number_ and _boolean_ values in the _match rule_ must be wrapped around with a pair of quotes.
There are three different match rules: ### Match value
### 1. Match value
```json ```json
{ {
"match": "match":
@@ -153,7 +166,7 @@ There are three different match rules:
} }
``` ```
### 2. Match regex ### Match regex
For the regex syntax, check out <http://golang.org/pkg/regexp/syntax/> For the regex syntax, check out <http://golang.org/pkg/regexp/syntax/>
```json ```json
{ {
@@ -170,12 +183,13 @@ For the regex syntax, check out <http://golang.org/pkg/regexp/syntax/>
} }
``` ```
### 3. Match payload-hash-sha1 ### Match payload-hmac-sha1
Validate the HMAC of the payload using the SHA1 hash and the given *secret*.
```json ```json
{ {
"match": "match":
{ {
"type": "payload-hash-sha1", "type": "payload-hmac-sha1",
"secret": "yoursecret", "secret": "yoursecret",
"parameter": "parameter":
{ {
@@ -193,12 +207,13 @@ will be tried unless a match is found. For example:
X-Hub-Signature: sha1=the-first-signature,sha1=the-second-signature X-Hub-Signature: sha1=the-first-signature,sha1=the-second-signature
``` ```
### 4. Match payload-hash-sha256 ### Match payload-hmac-sha256
Validate the HMAC of the payload using the SHA256 hash and the given *secret*.
```json ```json
{ {
"match": "match":
{ {
"type": "payload-hash-sha256", "type": "payload-hmac-sha256",
"secret": "yoursecret", "secret": "yoursecret",
"parameter": "parameter":
{ {
@@ -216,12 +231,13 @@ will be tried unless a match is found. For example:
X-Hub-Signature: sha256=the-first-signature,sha256=the-second-signature X-Hub-Signature: sha256=the-first-signature,sha256=the-second-signature
``` ```
### 5. Match payload-hash-sha512 ### Match payload-hmac-sha512
Validate the HMAC of the payload using the SHA512 hash and the given *secret*.
```json ```json
{ {
"match": "match":
{ {
"type": "payload-hash-sha512", "type": "payload-hmac-sha512",
"secret": "yoursecret", "secret": "yoursecret",
"parameter": "parameter":
{ {
@@ -239,7 +255,7 @@ will be tried unless a match is found. For example:
X-Hub-Signature: sha512=the-first-signature,sha512=the-second-signature X-Hub-Signature: sha512=the-first-signature,sha512=the-second-signature
``` ```
### 6. Match Whitelisted IP range ### Match Whitelisted IP range
The IP can be IPv4- or IPv6-formatted, using [CIDR notation](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_blocks). To match a single IP address only, use `/32`. The IP can be IPv4- or IPv6-formatted, using [CIDR notation](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_blocks). To match a single IP address only, use `/32`.
@@ -253,7 +269,7 @@ The IP can be IPv4- or IPv6-formatted, using [CIDR notation](https://en.wikipedi
} }
``` ```
### 7. Match scalr-signature ### Match scalr-signature
The trigger rule checks the scalr signature and also checks that the request was signed less than 5 minutes before it was received. The trigger rule checks the scalr signature and also checks that the request was signed less than 5 minutes before it was received.
A unqiue signing key is generated for each webhook endpoint URL you register in Scalr. A unqiue signing key is generated for each webhook endpoint URL you register in Scalr.
+19 -3
View File
@@ -1,5 +1,5 @@
# Referencing request values # Referencing request values
There are three types of request values: There are four types of request values:
1. HTTP Request Header values 1. HTTP Request Header values
@@ -19,7 +19,23 @@ There are three types of request values:
} }
``` ```
3. Payload (JSON or form-value encoded) 3. HTTP Request parameters
```json
{
"source": "request",
"name": "method"
}
```
```json
{
"source": "request",
"name": "remote-addr"
}
```
4. Payload (JSON or form-value encoded)
```json ```json
{ {
"source": "payload", "source": "payload",
@@ -57,7 +73,7 @@ There are three types of request values:
If the payload contains a key with the specified name "commits.0.commit.id", then the value of that key has priority over the dot-notation referencing. If the payload contains a key with the specified name "commits.0.commit.id", then the value of that key has priority over the dot-notation referencing.
3. XML Payload 4. XML Payload
Referencing XML payload parameters is much like the JSON examples above, but XML is more complex. Referencing XML payload parameters is much like the JSON examples above, but XML is more complex.
Element attributes are prefixed by a hyphen (`-`). Element attributes are prefixed by a hyphen (`-`).
+4 -4
View File
@@ -1,12 +1,12 @@
# Templates in Webhook # Templates in Webhook
[`webhook`][w] can parse the `hooks.json` input file as a Go template when given the `-template` [CLI parameter](Webhook-Parameters.md). [`webhook`][w] can parse a hooks configuration file as a Go template when given the `-template` [CLI parameter](Webhook-Parameters.md).
In additional to the [built-in Go template functions and features][tt], `webhook` provides a `getenv` template function for inserting environment variables into a `hooks.json` file. In additional to the [built-in Go template functions and features][tt], `webhook` provides a `getenv` template function for inserting environment variables into a templated configuration file.
## Example Usage ## Example Usage
In the example `hooks.json` file below, the `payload-hash-sha1` matching rule looks up the secret hash from the environment using the `getenv` template function. In the example JSON template file below (YAML is also supported), the `payload-hmac-sha1` matching rule looks up the HMAC secret from the environment using the `getenv` template function.
Additionally, the result is piped through the built-in Go template function `js` to ensure that the result is a well-formed Javascript/JSON string. Additionally, the result is piped through the built-in Go template function `js` to ensure that the result is a well-formed Javascript/JSON string.
``` ```
@@ -44,7 +44,7 @@ Additionally, the result is piped through the built-in Go template function `js`
{ {
"match": "match":
{ {
"type": "payload-hash-sha1", "type": "payload-hmac-sha1",
"secret": "{{ getenv "XXXTEST_SECRET" | js }}", "secret": "{{ getenv "XXXTEST_SECRET" | js }}",
"parameter": "parameter":
{ {
+4 -4
View File
@@ -1,19 +1,19 @@
module github.com/adnanh/webhook module github.com/adnanh/webhook
go 1.13 go 1.14
require ( require (
github.com/antonmedv/expr v1.8.9
github.com/clbanning/mxj v1.8.4 github.com/clbanning/mxj v1.8.4
github.com/dustin/go-humanize v1.0.0 github.com/dustin/go-humanize v1.0.0
github.com/fsnotify/fsnotify v1.4.7 // indirect github.com/fsnotify/fsnotify v1.4.7 // indirect
github.com/ghodss/yaml v1.0.0 github.com/ghodss/yaml v1.0.0
github.com/go-chi/chi v4.0.2+incompatible github.com/go-chi/chi v4.0.2+incompatible
github.com/gofrs/uuid v3.2.0+incompatible github.com/gofrs/uuid v3.2.0+incompatible
github.com/google/cel-go v0.7.0 // indirect
github.com/gorilla/mux v1.7.3 github.com/gorilla/mux v1.7.3
github.com/kr/pretty v0.1.0 // indirect github.com/kr/pretty v0.1.0 // indirect
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553 // indirect golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 // indirect
gopkg.in/fsnotify.v1 v1.4.2 gopkg.in/fsnotify.v1 v1.4.2
gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7 // indirect
) )
+108
View File
@@ -1,15 +1,54 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/DATA-DOG/go-sqlmock v1.3.3/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM=
github.com/antlr/antlr4 v0.0.0-20200503195918-621b933c7a7f h1:0cEys61Sr2hUBEXfNV8eyQP01oZuBgoMeHunebPirK8=
github.com/antlr/antlr4 v0.0.0-20200503195918-621b933c7a7f/go.mod h1:T7PbCXFs94rrTttyxjbyT5+/1V8T2TYDejxUfHJjw1Y=
github.com/antonmedv/expr v1.8.9 h1:O9stiHmHHww9b4ozhPx7T6BK7fXfOCHJ8ybxf0833zw=
github.com/antonmedv/expr v1.8.9/go.mod h1:5qsM3oLGDND7sDmQGDXHkYfkjYMUX14qsgqmHhwGEk8=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/clbanning/mxj v1.8.4 h1:HuhwZtbyvyOw+3Z1AowPkU87JkJUSv751ELWaiTpj8I= github.com/clbanning/mxj v1.8.4 h1:HuhwZtbyvyOw+3Z1AowPkU87JkJUSv751ELWaiTpj8I=
github.com/clbanning/mxj v1.8.4/go.mod h1:BVjHeAH+rl9rs6f+QIpeRl0tfu10SXn1pUSa5PVGJng= github.com/clbanning/mxj v1.8.4/go.mod h1:BVjHeAH+rl9rs6f+QIpeRl0tfu10SXn1pUSa5PVGJng=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/davecgh/go-spew v0.0.0-20161028175848-04cdfd42973b/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo= github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo=
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I= github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/gdamore/encoding v1.0.0/go.mod h1:alR0ol34c49FCSBLjhosxzcPHQbf2trDkoo5dl+VrEg=
github.com/gdamore/tcell v1.3.0/go.mod h1:Hjvr+Ofd+gLglo7RYKxxnzCBmev3BzsS67MebKS4zMM=
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk= github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-chi/chi v4.0.2+incompatible h1:maB6vn6FqCxrpz4FqWdh4+lwpyZIQS7YEAUcHlgXVRs= github.com/go-chi/chi v4.0.2+incompatible h1:maB6vn6FqCxrpz4FqWdh4+lwpyZIQS7YEAUcHlgXVRs=
github.com/go-chi/chi v4.0.2+incompatible/go.mod h1:eB3wogJHnLi3x/kFX2A+IbTBlXxmMeXJVKy9tTv1XzQ= github.com/go-chi/chi v4.0.2+incompatible/go.mod h1:eB3wogJHnLi3x/kFX2A+IbTBlXxmMeXJVKy9tTv1XzQ=
github.com/gofrs/uuid v3.2.0+incompatible h1:y12jRkkFxsd7GpqdSZ+/KCs/fJbqpEXSGd4+jfEaewE= github.com/gofrs/uuid v3.2.0+incompatible h1:y12jRkkFxsd7GpqdSZ+/KCs/fJbqpEXSGd4+jfEaewE=
github.com/gofrs/uuid v3.2.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM= github.com/gofrs/uuid v3.2.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.3 h1:JjCZWpVbqXDqFVmTfYWEVTMIYrL/NPdPSCHPJ0T/raM=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/google/cel-go v0.7.0 h1:J0J8RSCJW+SdB53YPwPSm2m1Kfz1tqGXdwMuIAVRU9o=
github.com/google/cel-go v0.7.0/go.mod h1:4EtyFAHT5xNr0Msu0MJjyGxPUgdr9DlcaPyzLt/kkt8=
github.com/google/cel-spec v0.5.0/go.mod h1:Nwjgxy5CbjlPrtCWjeDjUyKMl8w41YBYGjsyDdqk0xA=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/mux v1.7.3 h1:gnP5JzjVOuiZD07fKKToCAOjS0yOpj/qPETTXCCS6hw= github.com/gorilla/mux v1.7.3 h1:gnP5JzjVOuiZD07fKKToCAOjS0yOpj/qPETTXCCS6hw=
github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs= github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI= github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
@@ -17,18 +56,87 @@ github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORN
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE= github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/lucasb-eyer/go-colorful v1.0.2/go.mod h1:0MS4r+7BZKSJ5mw4/S5MPN+qHFF1fYclkSPilDOKW0s=
github.com/lucasb-eyer/go-colorful v1.0.3/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-runewidth v0.0.8/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/pmezard/go-difflib v0.0.0-20151028094244-d8ed2627bdf0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/rivo/tview v0.0.0-20200219210816-cd38d7432498/go.mod h1:6lkG1x+13OShEf0EaOCaTQYyB7d5nSbb181KtjlS+84=
github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/sanity-io/litter v1.2.0/go.mod h1:JF6pZUFgu2Q0sBZ+HSV35P8TVPI1TTzEwyu9FXAw2W4=
github.com/stoewer/go-strcase v1.2.0 h1:Z2iHWqGXH00XYgqDmNgQbIBxf3wrNq0F3feEy0ainaU=
github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v0.0.0-20161117074351-18a02ba4a312/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553 h1:efeOvDhwQ29Dj3SdAV/MJf8oukgn+8D8WgaCaRMchF8= golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553 h1:efeOvDhwQ29Dj3SdAV/MJf8oukgn+8D8WgaCaRMchF8=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a h1:1BGLXjeY4akVXGgbC9HugT3Jv3hCI0z56oJR5vAMgBU= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a h1:1BGLXjeY4akVXGgbC9HugT3Jv3hCI0z56oJR5vAMgBU=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190626150813-e07cf5db2756/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8 h1:JA8d3MPx/IToSyXZG/RhwYEtfrKO1Fxrqe8KrkiLXKM= golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8 h1:JA8d3MPx/IToSyXZG/RhwYEtfrKO1Fxrqe8KrkiLXKM=
golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527 h1:uYVVQ9WP/Ds2ROhcaGPeIdVq0RIXVLwsHlnvJ+cT1So=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg= golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
google.golang.org/genproto v0.0.0-20201102152239-715cce707fb0 h1:d0rYPqjQfVuFe+tZgv4PHt2hNxK79MRXX7PaD/A5ynA=
google.golang.org/genproto v0.0.0-20201102152239-715cce707fb0/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
google.golang.org/protobuf v1.25.0 h1:Ejskq+SyPohKW+1uil0JJMtmHCgJPJ/qWTxr8qp+R4c=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15 h1:YR8cESwS4TdDjEe65xsg0ogRM/Nc3DYOhEAlW+xobZo=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/fsnotify.v1 v1.4.2 h1:AwZiD/bIUttYJ+n/k1UwlSUsM+VSE6id7UAnSKqQ+Tc= gopkg.in/fsnotify.v1 v1.4.2 h1:AwZiD/bIUttYJ+n/k1UwlSUsM+VSE6id7UAnSKqQ+Tc=
gopkg.in/fsnotify.v1 v1.4.2/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys= gopkg.in/fsnotify.v1 v1.4.2/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7 h1:+t9dhfO+GNOIGJof6kPOAenx7YgrZMTdRPV+EsnPabk= gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7 h1:+t9dhfO+GNOIGJof6kPOAenx7YgrZMTdRPV+EsnPabk=
gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74= gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+55
View File
@@ -0,0 +1,55 @@
[
{
"id": "webhook",
"execute-command": "/home/adnan/redeploy-go-webhook.sh",
"command-working-directory": "/home/adnan/go",
"response-message": "I got the payload!",
"response-headers":
[
{
"name": "Access-Control-Allow-Origin",
"value": "*"
}
],
"pass-arguments-to-command":
[
{
"source": "payload",
"name": "head_commit.id"
},
{
"source": "payload",
"name": "pusher.name"
},
{
"source": "payload",
"name": "pusher.email"
}
],
"trigger-rule":
{
"or":
[
{
"match":
{
"type": "expression",
"expression": "\"ada\" in Payload"
}
},
{
"match":
{
"type": "value",
"value": "refs/heads/master",
"parameter":
{
"source": "payload",
"name": "ref"
}
}
}
]
}
}
]
+1 -1
View File
@@ -33,7 +33,7 @@
{ {
"match": "match":
{ {
"type": "payload-hash-sha1", "type": "payload-hmac-sha1",
"secret": "mysecret", "secret": "mysecret",
"parameter": "parameter":
{ {
+1 -1
View File
@@ -33,7 +33,7 @@
{ {
"match": "match":
{ {
"type": "payload-hash-sha1", "type": "payload-hmac-sha1",
"secret": "{{ getenv "XXXTEST_SECRET" | js }}", "secret": "{{ getenv "XXXTEST_SECRET" | js }}",
"parameter": "parameter":
{ {
+1 -1
View File
@@ -15,7 +15,7 @@
trigger-rule: trigger-rule:
and: and:
- match: - match:
type: payload-hash-sha1 type: payload-hmac-sha1
secret: mysecret secret: mysecret
parameter: parameter:
source: header source: header
+1 -1
View File
@@ -15,7 +15,7 @@
trigger-rule: trigger-rule:
and: and:
- match: - match:
type: payload-hash-sha1 type: payload-hmac-sha1
secret: "{{ getenv "XXXTEST_SECRET" | js }}" secret: "{{ getenv "XXXTEST_SECRET" | js }}"
parameter: parameter:
source: header source: header
+184 -76
View File
@@ -26,19 +26,22 @@ import (
"text/template" "text/template"
"time" "time"
"github.com/antonmedv/expr"
"github.com/ghodss/yaml" "github.com/ghodss/yaml"
) )
// Constants used to specify the parameter source // Constants used to specify the parameter source
const ( const (
SourceHeader string = "header" SourceHeader string = "header"
SourceQuery string = "url" SourceQuery string = "url"
SourceQueryAlias string = "query" SourceQueryAlias string = "query"
SourcePayload string = "payload" SourcePayload string = "payload"
SourceString string = "string" SourceRawRequestBody string = "raw-request-body"
SourceEntirePayload string = "entire-payload" SourceRequest string = "request"
SourceEntireQuery string = "entire-query" SourceString string = "string"
SourceEntireHeaders string = "entire-headers" SourceEntirePayload string = "entire-payload"
SourceEntireQuery string = "entire-query"
SourceEntireHeaders string = "entire-headers"
) )
const ( const (
@@ -73,6 +76,8 @@ func IsParameterNodeError(err error) bool {
type SignatureError struct { type SignatureError struct {
Signature string Signature string
Signatures []string Signatures []string
emptyPayload bool
} }
func (e *SignatureError) Error() string { func (e *SignatureError) Error() string {
@@ -80,11 +85,26 @@ func (e *SignatureError) Error() string {
return "<nil>" return "<nil>"
} }
if e.Signatures != nil { var empty string
return fmt.Sprintf("invalid payload signatures %s", e.Signatures) if e.emptyPayload {
empty = " on empty payload"
} }
return fmt.Sprintf("invalid payload signature %s", e.Signature) if e.Signatures != nil {
return fmt.Sprintf("invalid payload signatures %s%s", e.Signatures, empty)
}
return fmt.Sprintf("invalid payload signature %s%s", e.Signature, empty)
}
// IsSignatureError returns whether err is of type SignatureError.
func IsSignatureError(err error) bool {
switch err.(type) {
case *SignatureError:
return true
default:
return false
}
} }
// ArgumentError describes an invalid argument passed to Hook. // ArgumentError describes an invalid argument passed to Hook.
@@ -160,21 +180,24 @@ func ValidateMAC(payload []byte, mac hash.Hash, signatures []string) (string, er
return "", err return "", err
} }
expectedMAC := hex.EncodeToString(mac.Sum(nil)) actualMAC := hex.EncodeToString(mac.Sum(nil))
for _, signature := range signatures { for _, signature := range signatures {
if hmac.Equal([]byte(signature), []byte(expectedMAC)) { if hmac.Equal([]byte(signature), []byte(actualMAC)) {
return expectedMAC, err return actualMAC, err
} }
} }
return expectedMAC, &SignatureError{ e := &SignatureError{Signatures: signatures}
Signatures: signatures, if len(payload) == 0 {
e.emptyPayload = true
} }
return actualMAC, e
} }
// CheckPayloadSignature calculates and verifies SHA1 signature of the given payload // CheckPayloadSignature calculates and verifies SHA1 signature of the given payload
func CheckPayloadSignature(payload []byte, secret string, signature string) (string, error) { func CheckPayloadSignature(payload []byte, secret, signature string) (string, error) {
if secret == "" { if secret == "" {
return "", errors.New("signature validation secret can not be empty") return "", errors.New("signature validation secret can not be empty")
} }
@@ -187,7 +210,7 @@ func CheckPayloadSignature(payload []byte, secret string, signature string) (str
} }
// CheckPayloadSignature256 calculates and verifies SHA256 signature of the given payload // CheckPayloadSignature256 calculates and verifies SHA256 signature of the given payload
func CheckPayloadSignature256(payload []byte, secret string, signature string) (string, error) { func CheckPayloadSignature256(payload []byte, secret, signature string) (string, error) {
if secret == "" { if secret == "" {
return "", errors.New("signature validation secret can not be empty") return "", errors.New("signature validation secret can not be empty")
} }
@@ -200,7 +223,7 @@ func CheckPayloadSignature256(payload []byte, secret string, signature string) (
} }
// CheckPayloadSignature512 calculates and verifies SHA512 signature of the given payload // CheckPayloadSignature512 calculates and verifies SHA512 signature of the given payload
func CheckPayloadSignature512(payload []byte, secret string, signature string) (string, error) { func CheckPayloadSignature512(payload []byte, secret, signature string) (string, error) {
if secret == "" { if secret == "" {
return "", errors.New("signature validation secret can not be empty") return "", errors.New("signature validation secret can not be empty")
} }
@@ -212,22 +235,26 @@ func CheckPayloadSignature512(payload []byte, secret string, signature string) (
return ValidateMAC(payload, hmac.New(sha512.New, []byte(secret)), signatures) return ValidateMAC(payload, hmac.New(sha512.New, []byte(secret)), signatures)
} }
func CheckScalrSignature(headers map[string]interface{}, body []byte, signingKey string, checkDate bool) (bool, error) { func CheckScalrSignature(r *Request, signingKey string, checkDate bool) (bool, error) {
// Check for the signature and date headers if r.Headers == nil {
if _, ok := headers["X-Signature"]; !ok {
return false, nil return false, nil
} }
if _, ok := headers["Date"]; !ok {
// Check for the signature and date headers
if _, ok := r.Headers["X-Signature"]; !ok {
return false, nil
}
if _, ok := r.Headers["Date"]; !ok {
return false, nil return false, nil
} }
if signingKey == "" { if signingKey == "" {
return false, errors.New("signature validation signing key can not be empty") return false, errors.New("signature validation signing key can not be empty")
} }
providedSignature := headers["X-Signature"].(string) providedSignature := r.Headers["X-Signature"].(string)
dateHeader := headers["Date"].(string) dateHeader := r.Headers["Date"].(string)
mac := hmac.New(sha1.New, []byte(signingKey)) mac := hmac.New(sha1.New, []byte(signingKey))
mac.Write(body) mac.Write(r.Body)
mac.Write([]byte(dateHeader)) mac.Write([]byte(dateHeader))
expectedSignature := hex.EncodeToString(mac.Sum(nil)) expectedSignature := hex.EncodeToString(mac.Sum(nil))
@@ -254,7 +281,7 @@ func CheckScalrSignature(headers map[string]interface{}, body []byte, signingKey
// CheckIPWhitelist makes sure the provided remote address (of the form IP:port) falls within the provided IP range // CheckIPWhitelist makes sure the provided remote address (of the form IP:port) falls within the provided IP range
// (in CIDR form or a single IP address). // (in CIDR form or a single IP address).
func CheckIPWhitelist(remoteAddr string, ipRange string) (bool, error) { func CheckIPWhitelist(remoteAddr, ipRange string) (bool, error) {
// Extract IP address from remote address. // Extract IP address from remote address.
// IPv6 addresses will likely be surrounded by []. // IPv6 addresses will likely be surrounded by [].
@@ -290,10 +317,33 @@ func CheckIPWhitelist(remoteAddr string, ipRange string) (bool, error) {
return false, nil return false, nil
} }
type ExpressionEnv struct {
Headers, Query, Payload map[string]interface{}
}
func CheckExpression(request *Request, expression string) (bool, error) {
program, err := expr.Compile(expression, expr.Env(ExpressionEnv{}))
if err != nil {
return false, err
}
out, err := expr.Run(program, ExpressionEnv{
Headers: request.Headers,
Query: request.Query,
Payload: request.Payload,
})
if err != nil {
return false, err
}
return out == true, nil
}
// ReplaceParameter replaces parameter value with the passed value in the passed map // ReplaceParameter replaces parameter value with the passed value in the passed map
// (please note you should pass pointer to the map, because we're modifying it) // (please note you should pass pointer to the map, because we're modifying it)
// based on the passed string // based on the passed string
func ReplaceParameter(s string, params interface{}, value interface{}) bool { func ReplaceParameter(s string, params, value interface{}) bool {
if params == nil { if params == nil {
return false return false
} }
@@ -382,14 +432,27 @@ func GetParameter(s string, params interface{}) (interface{}, error) {
return nil, &ParameterNodeError{s} return nil, &ParameterNodeError{s}
} }
// ExtractParameterAsString extracts value from interface{} as string based on the passed string // ExtractParameterAsString extracts value from interface{} as string based on
// the passed string. Complex data types are rendered as JSON instead of the Go
// Stringer format.
func ExtractParameterAsString(s string, params interface{}) (string, error) { func ExtractParameterAsString(s string, params interface{}) (string, error) {
pValue, err := GetParameter(s, params) pValue, err := GetParameter(s, params)
if err != nil { if err != nil {
return "", err return "", err
} }
return fmt.Sprintf("%v", pValue), nil switch v := reflect.ValueOf(pValue); v.Kind() {
case reflect.Array, reflect.Map, reflect.Slice:
r, err := json.Marshal(pValue)
if err != nil {
return "", err
}
return string(r), nil
default:
return fmt.Sprintf("%v", pValue), nil
}
} }
// Argument type specifies the parameter key name and the source it should // Argument type specifies the parameter key name and the source it should
@@ -403,41 +466,64 @@ type Argument struct {
// Get Argument method returns the value for the Argument's key name // Get Argument method returns the value for the Argument's key name
// based on the Argument's source // based on the Argument's source
func (ha *Argument) Get(headers, query, payload *map[string]interface{}) (string, error) { func (ha *Argument) Get(r *Request) (string, error) {
var source *map[string]interface{} var source *map[string]interface{}
key := ha.Name key := ha.Name
switch ha.Source { switch ha.Source {
case SourceHeader: case SourceHeader:
source = headers source = &r.Headers
key = textproto.CanonicalMIMEHeaderKey(ha.Name) key = textproto.CanonicalMIMEHeaderKey(ha.Name)
case SourceQuery, SourceQueryAlias: case SourceQuery, SourceQueryAlias:
source = query source = &r.Query
case SourcePayload: case SourcePayload:
source = payload source = &r.Payload
case SourceString: case SourceString:
return ha.Name, nil return ha.Name, nil
case SourceRawRequestBody:
return string(r.Body), nil
case SourceRequest:
if r == nil || r.RawRequest == nil {
return "", errors.New("request is nil")
}
switch strings.ToLower(ha.Name) {
case "remote-addr":
return r.RawRequest.RemoteAddr, nil
case "method":
return r.RawRequest.Method, nil
default:
return "", fmt.Errorf("unsupported request key: %q", ha.Name)
}
case SourceEntirePayload: case SourceEntirePayload:
r, err := json.Marshal(payload) res, err := json.Marshal(&r.Payload)
if err != nil { if err != nil {
return "", err return "", err
} }
return string(r), nil return string(res), nil
case SourceEntireHeaders: case SourceEntireHeaders:
r, err := json.Marshal(headers) res, err := json.Marshal(&r.Headers)
if err != nil { if err != nil {
return "", err return "", err
} }
return string(r), nil return string(res), nil
case SourceEntireQuery: case SourceEntireQuery:
r, err := json.Marshal(query) res, err := json.Marshal(&r.Query)
if err != nil { if err != nil {
return "", err return "", err
} }
return string(r), nil return string(res), nil
} }
if source != nil { if source != nil {
@@ -515,6 +601,7 @@ type Hook struct {
JSONStringParameters []Argument `json:"parse-parameters-as-json,omitempty"` JSONStringParameters []Argument `json:"parse-parameters-as-json,omitempty"`
TriggerRule *Rules `json:"trigger-rule,omitempty"` TriggerRule *Rules `json:"trigger-rule,omitempty"`
TriggerRuleMismatchHttpResponseCode int `json:"trigger-rule-mismatch-http-response-code,omitempty"` TriggerRuleMismatchHttpResponseCode int `json:"trigger-rule-mismatch-http-response-code,omitempty"`
TriggerSignatureSoftFailures bool `json:"trigger-signature-soft-failures,omitempty"`
IncomingPayloadContentType string `json:"incoming-payload-content-type,omitempty"` IncomingPayloadContentType string `json:"incoming-payload-content-type,omitempty"`
SuccessHttpResponseCode int `json:"success-http-response-code,omitempty"` SuccessHttpResponseCode int `json:"success-http-response-code,omitempty"`
HTTPMethods []string `json:"http-methods"` HTTPMethods []string `json:"http-methods"`
@@ -522,11 +609,11 @@ type Hook struct {
// ParseJSONParameters decodes specified arguments to JSON objects and replaces the // ParseJSONParameters decodes specified arguments to JSON objects and replaces the
// string with the newly created object // string with the newly created object
func (h *Hook) ParseJSONParameters(headers, query, payload *map[string]interface{}) []error { func (h *Hook) ParseJSONParameters(r *Request) []error {
errors := make([]error, 0) errors := make([]error, 0)
for i := range h.JSONStringParameters { for i := range h.JSONStringParameters {
arg, err := h.JSONStringParameters[i].Get(headers, query, payload) arg, err := h.JSONStringParameters[i].Get(r)
if err != nil { if err != nil {
errors = append(errors, &ArgumentError{h.JSONStringParameters[i]}) errors = append(errors, &ArgumentError{h.JSONStringParameters[i]})
} else { } else {
@@ -545,11 +632,11 @@ func (h *Hook) ParseJSONParameters(headers, query, payload *map[string]interface
switch h.JSONStringParameters[i].Source { switch h.JSONStringParameters[i].Source {
case SourceHeader: case SourceHeader:
source = headers source = &r.Headers
case SourcePayload: case SourcePayload:
source = payload source = &r.Payload
case SourceQuery, SourceQueryAlias: case SourceQuery, SourceQueryAlias:
source = query source = &r.Query
} }
if source != nil { if source != nil {
@@ -575,14 +662,14 @@ func (h *Hook) ParseJSONParameters(headers, query, payload *map[string]interface
// ExtractCommandArguments creates a list of arguments, based on the // ExtractCommandArguments creates a list of arguments, based on the
// PassArgumentsToCommand property that is ready to be used with exec.Command() // PassArgumentsToCommand property that is ready to be used with exec.Command()
func (h *Hook) ExtractCommandArguments(headers, query, payload *map[string]interface{}) ([]string, []error) { func (h *Hook) ExtractCommandArguments(r *Request) ([]string, []error) {
args := make([]string, 0) args := make([]string, 0)
errors := make([]error, 0) errors := make([]error, 0)
args = append(args, h.ExecuteCommand) args = append(args, h.ExecuteCommand)
for i := range h.PassArgumentsToCommand { for i := range h.PassArgumentsToCommand {
arg, err := h.PassArgumentsToCommand[i].Get(headers, query, payload) arg, err := h.PassArgumentsToCommand[i].Get(r)
if err != nil { if err != nil {
args = append(args, "") args = append(args, "")
errors = append(errors, &ArgumentError{h.PassArgumentsToCommand[i]}) errors = append(errors, &ArgumentError{h.PassArgumentsToCommand[i]})
@@ -602,11 +689,11 @@ func (h *Hook) ExtractCommandArguments(headers, query, payload *map[string]inter
// ExtractCommandArgumentsForEnv creates a list of arguments in key=value // ExtractCommandArgumentsForEnv creates a list of arguments in key=value
// format, based on the PassEnvironmentToCommand property that is ready to be used // format, based on the PassEnvironmentToCommand property that is ready to be used
// with exec.Command(). // with exec.Command().
func (h *Hook) ExtractCommandArgumentsForEnv(headers, query, payload *map[string]interface{}) ([]string, []error) { func (h *Hook) ExtractCommandArgumentsForEnv(r *Request) ([]string, []error) {
args := make([]string, 0) args := make([]string, 0)
errors := make([]error, 0) errors := make([]error, 0)
for i := range h.PassEnvironmentToCommand { for i := range h.PassEnvironmentToCommand {
arg, err := h.PassEnvironmentToCommand[i].Get(headers, query, payload) arg, err := h.PassEnvironmentToCommand[i].Get(r)
if err != nil { if err != nil {
errors = append(errors, &ArgumentError{h.PassEnvironmentToCommand[i]}) errors = append(errors, &ArgumentError{h.PassEnvironmentToCommand[i]})
continue continue
@@ -638,11 +725,11 @@ type FileParameter struct {
// ExtractCommandArgumentsForFile creates a list of arguments in key=value // ExtractCommandArgumentsForFile creates a list of arguments in key=value
// format, based on the PassFileToCommand property that is ready to be used // format, based on the PassFileToCommand property that is ready to be used
// with exec.Command(). // with exec.Command().
func (h *Hook) ExtractCommandArgumentsForFile(headers, query, payload *map[string]interface{}) ([]FileParameter, []error) { func (h *Hook) ExtractCommandArgumentsForFile(r *Request) ([]FileParameter, []error) {
args := make([]FileParameter, 0) args := make([]FileParameter, 0)
errors := make([]error, 0) errors := make([]error, 0)
for i := range h.PassFileToCommand { for i := range h.PassFileToCommand {
arg, err := h.PassFileToCommand[i].Get(headers, query, payload) arg, err := h.PassFileToCommand[i].Get(r)
if err != nil { if err != nil {
errors = append(errors, &ArgumentError{h.PassFileToCommand[i]}) errors = append(errors, &ArgumentError{h.PassFileToCommand[i]})
continue continue
@@ -749,16 +836,16 @@ type Rules struct {
// Evaluate finds the first rule property that is not nil and returns the value // Evaluate finds the first rule property that is not nil and returns the value
// it evaluates to // it evaluates to
func (r Rules) Evaluate(headers, query, payload *map[string]interface{}, body *[]byte, remoteAddr string) (bool, error) { func (r Rules) Evaluate(req *Request) (bool, error) {
switch { switch {
case r.And != nil: case r.And != nil:
return r.And.Evaluate(headers, query, payload, body, remoteAddr) return r.And.Evaluate(req)
case r.Or != nil: case r.Or != nil:
return r.Or.Evaluate(headers, query, payload, body, remoteAddr) return r.Or.Evaluate(req)
case r.Not != nil: case r.Not != nil:
return r.Not.Evaluate(headers, query, payload, body, remoteAddr) return r.Not.Evaluate(req)
case r.Match != nil: case r.Match != nil:
return r.Match.Evaluate(headers, query, payload, body, remoteAddr) return r.Match.Evaluate(req)
} }
return false, nil return false, nil
@@ -768,11 +855,11 @@ func (r Rules) Evaluate(headers, query, payload *map[string]interface{}, body *[
type AndRule []Rules type AndRule []Rules
// Evaluate AndRule will return true if and only if all of ChildRules evaluate to true // Evaluate AndRule will return true if and only if all of ChildRules evaluate to true
func (r AndRule) Evaluate(headers, query, payload *map[string]interface{}, body *[]byte, remoteAddr string) (bool, error) { func (r AndRule) Evaluate(req *Request) (bool, error) {
res := true res := true
for _, v := range r { for _, v := range r {
rv, err := v.Evaluate(headers, query, payload, body, remoteAddr) rv, err := v.Evaluate(req)
if err != nil { if err != nil {
return false, err return false, err
} }
@@ -790,13 +877,17 @@ func (r AndRule) Evaluate(headers, query, payload *map[string]interface{}, body
type OrRule []Rules type OrRule []Rules
// Evaluate OrRule will return true if any of ChildRules evaluate to true // Evaluate OrRule will return true if any of ChildRules evaluate to true
func (r OrRule) Evaluate(headers, query, payload *map[string]interface{}, body *[]byte, remoteAddr string) (bool, error) { func (r OrRule) Evaluate(req *Request) (bool, error) {
res := false res := false
for _, v := range r { for _, v := range r {
rv, err := v.Evaluate(headers, query, payload, body, remoteAddr) rv, err := v.Evaluate(req)
if err != nil { if err != nil {
return false, err if !IsParameterNodeError(err) {
if !req.AllowSignatureErrors || (req.AllowSignatureErrors && !IsSignatureError(err)) {
return false, err
}
}
} }
res = res || rv res = res || rv
@@ -812,25 +903,30 @@ func (r OrRule) Evaluate(headers, query, payload *map[string]interface{}, body *
type NotRule Rules type NotRule Rules
// Evaluate NotRule will return true if and only if ChildRule evaluates to false // Evaluate NotRule will return true if and only if ChildRule evaluates to false
func (r NotRule) Evaluate(headers, query, payload *map[string]interface{}, body *[]byte, remoteAddr string) (bool, error) { func (r NotRule) Evaluate(req *Request) (bool, error) {
rv, err := Rules(r).Evaluate(headers, query, payload, body, remoteAddr) rv, err := Rules(r).Evaluate(req)
return !rv, err return !rv, err
} }
// MatchRule will evaluate to true based on the type // MatchRule will evaluate to true based on the type
type MatchRule struct { type MatchRule struct {
Type string `json:"type,omitempty"` Type string `json:"type,omitempty"`
Regex string `json:"regex,omitempty"` Regex string `json:"regex,omitempty"`
Secret string `json:"secret,omitempty"` Expression string `json:"expression,omitempty"`
Value string `json:"value,omitempty"` Secret string `json:"secret,omitempty"`
Parameter Argument `json:"parameter,omitempty"` Value string `json:"value,omitempty"`
IPRange string `json:"ip-range,omitempty"` Parameter Argument `json:"parameter,omitempty"`
IPRange string `json:"ip-range,omitempty"`
} }
// Constants for the MatchRule type // Constants for the MatchRule type
const ( const (
MatchValue string = "value" MatchValue string = "value"
MatchRegex string = "regex" MatchRegex string = "regex"
MatchExpression string = "expression"
MatchHMACSHA1 string = "payload-hmac-sha1"
MatchHMACSHA256 string = "payload-hmac-sha256"
MatchHMACSHA512 string = "payload-hmac-sha512"
MatchHashSHA1 string = "payload-hash-sha1" MatchHashSHA1 string = "payload-hash-sha1"
MatchHashSHA256 string = "payload-hash-sha256" MatchHashSHA256 string = "payload-hash-sha256"
MatchHashSHA512 string = "payload-hash-sha512" MatchHashSHA512 string = "payload-hash-sha512"
@@ -839,15 +935,18 @@ const (
) )
// Evaluate MatchRule will return based on the type // Evaluate MatchRule will return based on the type
func (r MatchRule) Evaluate(headers, query, payload *map[string]interface{}, body *[]byte, remoteAddr string) (bool, error) { func (r MatchRule) Evaluate(req *Request) (bool, error) {
if r.Type == IPWhitelist { if r.Type == IPWhitelist {
return CheckIPWhitelist(remoteAddr, r.IPRange) return CheckIPWhitelist(req.RawRequest.RemoteAddr, r.IPRange)
} }
if r.Type == ScalrSignature { if r.Type == ScalrSignature {
return CheckScalrSignature(*headers, *body, r.Secret, true) return CheckScalrSignature(req, r.Secret, true)
}
if r.Type == MatchExpression {
return CheckExpression(req, r.Expression)
} }
arg, err := r.Parameter.Get(headers, query, payload) arg, err := r.Parameter.Get(req)
if err == nil { if err == nil {
switch r.Type { switch r.Type {
case MatchValue: case MatchValue:
@@ -855,13 +954,22 @@ func (r MatchRule) Evaluate(headers, query, payload *map[string]interface{}, bod
case MatchRegex: case MatchRegex:
return regexp.MatchString(r.Regex, arg) return regexp.MatchString(r.Regex, arg)
case MatchHashSHA1: case MatchHashSHA1:
_, err := CheckPayloadSignature(*body, r.Secret, arg) log.Print(`warn: use of deprecated option payload-hash-sha1; use payload-hmac-sha1 instead`)
fallthrough
case MatchHMACSHA1:
_, err := CheckPayloadSignature(req.Body, r.Secret, arg)
return err == nil, err return err == nil, err
case MatchHashSHA256: case MatchHashSHA256:
_, err := CheckPayloadSignature256(*body, r.Secret, arg) log.Print(`warn: use of deprecated option payload-hash-sha256: use payload-hmac-sha256 instead`)
fallthrough
case MatchHMACSHA256:
_, err := CheckPayloadSignature256(req.Body, r.Secret, arg)
return err == nil, err return err == nil, err
case MatchHashSHA512: case MatchHashSHA512:
_, err := CheckPayloadSignature512(*body, r.Secret, arg) log.Print(`warn: use of deprecated option payload-hash-sha512: use payload-hmac-sha512 instead`)
fallthrough
case MatchHMACSHA512:
_, err := CheckPayloadSignature512(req.Body, r.Secret, arg)
return err == nil, err return err == nil, err
} }
} }
+142 -64
View File
@@ -1,6 +1,7 @@
package hook package hook
import ( import (
"net/http"
"os" "os"
"reflect" "reflect"
"strings" "strings"
@@ -49,12 +50,14 @@ var checkPayloadSignatureTests = []struct {
{[]byte(`{"a": "z"}`), "secret", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", true}, {[]byte(`{"a": "z"}`), "secret", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", true},
{[]byte(`{"a": "z"}`), "secret", "sha1=b17e04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", true}, {[]byte(`{"a": "z"}`), "secret", "sha1=b17e04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", true},
{[]byte(`{"a": "z"}`), "secret", "sha1=XXXe04cbb22afa8ffbff8796fc1894ed27badd9e,sha1=b17e04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", true}, {[]byte(`{"a": "z"}`), "secret", "sha1=XXXe04cbb22afa8ffbff8796fc1894ed27badd9e,sha1=b17e04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", true},
{[]byte(``), "secret", "25af6174a0fcecc4d346680a72b7ce644b9a88e8", "25af6174a0fcecc4d346680a72b7ce644b9a88e8", true},
// failures // failures
{[]byte(`{"a": "z"}`), "secret", "XXXe04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", false}, {[]byte(`{"a": "z"}`), "secret", "XXXe04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", false},
{[]byte(`{"a": "z"}`), "secret", "sha1=XXXe04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", false}, {[]byte(`{"a": "z"}`), "secret", "sha1=XXXe04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", false},
{[]byte(`{"a": "z"}`), "secret", "sha1=XXXe04cbb22afa8ffbff8796fc1894ed27badd9e,sha1=XXXe04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", false}, {[]byte(`{"a": "z"}`), "secret", "sha1=XXXe04cbb22afa8ffbff8796fc1894ed27badd9e,sha1=XXXe04cbb22afa8ffbff8796fc1894ed27badd9e", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", false},
{[]byte(`{"a": "z"}`), "secreX", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", "900225703e9342328db7307692736e2f7cc7b36e", false}, {[]byte(`{"a": "z"}`), "secreX", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", "900225703e9342328db7307692736e2f7cc7b36e", false},
{[]byte(`{"a": "z"}`), "", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", "", false}, {[]byte(`{"a": "z"}`), "", "b17e04cbb22afa8ffbff8796fc1894ed27badd9e", "", false},
{[]byte(``), "secret", "XXXf6174a0fcecc4d346680a72b7ce644b9a88e8", "25af6174a0fcecc4d346680a72b7ce644b9a88e8", false},
} }
func TestCheckPayloadSignature(t *testing.T) { func TestCheckPayloadSignature(t *testing.T) {
@@ -80,11 +83,13 @@ var checkPayloadSignature256Tests = []struct {
{[]byte(`{"a": "z"}`), "secret", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", true}, {[]byte(`{"a": "z"}`), "secret", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", true},
{[]byte(`{"a": "z"}`), "secret", "sha256=f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", true}, {[]byte(`{"a": "z"}`), "secret", "sha256=f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", true},
{[]byte(`{"a": "z"}`), "secret", "sha256=XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89,sha256=f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", true}, {[]byte(`{"a": "z"}`), "secret", "sha256=XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89,sha256=f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", true},
{[]byte(``), "secret", "f9e66e179b6747ae54108f82f8ade8b3c25d76fd30afde6c395822c530196169", "f9e66e179b6747ae54108f82f8ade8b3c25d76fd30afde6c395822c530196169", true},
// failures // failures
{[]byte(`{"a": "z"}`), "secret", "XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", false}, {[]byte(`{"a": "z"}`), "secret", "XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", false},
{[]byte(`{"a": "z"}`), "secret", "sha256=XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", false}, {[]byte(`{"a": "z"}`), "secret", "sha256=XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", false},
{[]byte(`{"a": "z"}`), "secret", "sha256=XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89,sha256=XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", false}, {[]byte(`{"a": "z"}`), "secret", "sha256=XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89,sha256=XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", false},
{[]byte(`{"a": "z"}`), "", "XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "", false}, {[]byte(`{"a": "z"}`), "", "XXX7af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89", "", false},
{[]byte(``), "secret", "XXX66e179b6747ae54108f82f8ade8b3c25d76fd30afde6c395822c530196169", "f9e66e179b6747ae54108f82f8ade8b3c25d76fd30afde6c395822c530196169", false},
} }
func TestCheckPayloadSignature256(t *testing.T) { func TestCheckPayloadSignature256(t *testing.T) {
@@ -109,9 +114,11 @@ var checkPayloadSignature512Tests = []struct {
}{ }{
{[]byte(`{"a": "z"}`), "secret", "4ab17cc8ec668ead8bf498f87f8f32848c04d5ca3c9bcfcd3db9363f0deb44e580b329502a7fdff633d4d8fca301cc5c94a55a2fec458c675fb0ff2655898324", "4ab17cc8ec668ead8bf498f87f8f32848c04d5ca3c9bcfcd3db9363f0deb44e580b329502a7fdff633d4d8fca301cc5c94a55a2fec458c675fb0ff2655898324", true}, {[]byte(`{"a": "z"}`), "secret", "4ab17cc8ec668ead8bf498f87f8f32848c04d5ca3c9bcfcd3db9363f0deb44e580b329502a7fdff633d4d8fca301cc5c94a55a2fec458c675fb0ff2655898324", "4ab17cc8ec668ead8bf498f87f8f32848c04d5ca3c9bcfcd3db9363f0deb44e580b329502a7fdff633d4d8fca301cc5c94a55a2fec458c675fb0ff2655898324", true},
{[]byte(`{"a": "z"}`), "secret", "sha512=4ab17cc8ec668ead8bf498f87f8f32848c04d5ca3c9bcfcd3db9363f0deb44e580b329502a7fdff633d4d8fca301cc5c94a55a2fec458c675fb0ff2655898324", "4ab17cc8ec668ead8bf498f87f8f32848c04d5ca3c9bcfcd3db9363f0deb44e580b329502a7fdff633d4d8fca301cc5c94a55a2fec458c675fb0ff2655898324", true}, {[]byte(`{"a": "z"}`), "secret", "sha512=4ab17cc8ec668ead8bf498f87f8f32848c04d5ca3c9bcfcd3db9363f0deb44e580b329502a7fdff633d4d8fca301cc5c94a55a2fec458c675fb0ff2655898324", "4ab17cc8ec668ead8bf498f87f8f32848c04d5ca3c9bcfcd3db9363f0deb44e580b329502a7fdff633d4d8fca301cc5c94a55a2fec458c675fb0ff2655898324", true},
{[]byte(``), "secret", "b0e9650c5faf9cd8ae02276671545424104589b3656731ec193b25d01b07561c27637c2d4d68389d6cf5007a8632c26ec89ba80a01c77a6cdd389ec28db43901", "b0e9650c5faf9cd8ae02276671545424104589b3656731ec193b25d01b07561c27637c2d4d68389d6cf5007a8632c26ec89ba80a01c77a6cdd389ec28db43901", true},
// failures // failures
{[]byte(`{"a": "z"}`), "secret", "74a0081f5b5988f4f3e8b8dd34dadc6291611f2e6260635a7e1535f8e95edb97ff520ba8b152e8ca5760ac42639854f3242e29efc81be73a8bf52d474d31ffea", "4ab17cc8ec668ead8bf498f87f8f32848c04d5ca3c9bcfcd3db9363f0deb44e580b329502a7fdff633d4d8fca301cc5c94a55a2fec458c675fb0ff2655898324", false}, {[]byte(`{"a": "z"}`), "secret", "74a0081f5b5988f4f3e8b8dd34dadc6291611f2e6260635a7e1535f8e95edb97ff520ba8b152e8ca5760ac42639854f3242e29efc81be73a8bf52d474d31ffea", "4ab17cc8ec668ead8bf498f87f8f32848c04d5ca3c9bcfcd3db9363f0deb44e580b329502a7fdff633d4d8fca301cc5c94a55a2fec458c675fb0ff2655898324", false},
{[]byte(`{"a": "z"}`), "", "74a0081f5b5988f4f3e8b8dd34dadc6291611f2e6260635a7e1535f8e95edb97ff520ba8b152e8ca5760ac42639854f3242e29efc81be73a8bf52d474d31ffea", "", false}, {[]byte(`{"a": "z"}`), "", "74a0081f5b5988f4f3e8b8dd34dadc6291611f2e6260635a7e1535f8e95edb97ff520ba8b152e8ca5760ac42639854f3242e29efc81be73a8bf52d474d31ffea", "", false},
{[]byte(``), "secret", "XXX9650c5faf9cd8ae02276671545424104589b3656731ec193b25d01b07561c27637c2d4d68389d6cf5007a8632c26ec89ba80a01c77a6cdd389ec28db43901", "b0e9650c5faf9cd8ae02276671545424104589b3656731ec193b25d01b07561c27637c2d4d68389d6cf5007a8632c26ec89ba80a01c77a6cdd389ec28db43901", false},
} }
func TestCheckPayloadSignature512(t *testing.T) { func TestCheckPayloadSignature512(t *testing.T) {
@@ -130,7 +137,7 @@ func TestCheckPayloadSignature512(t *testing.T) {
var checkScalrSignatureTests = []struct { var checkScalrSignatureTests = []struct {
description string description string
headers map[string]interface{} headers map[string]interface{}
payload []byte body []byte
secret string secret string
expectedSignature string expectedSignature string
ok bool ok bool
@@ -169,7 +176,11 @@ var checkScalrSignatureTests = []struct {
func TestCheckScalrSignature(t *testing.T) { func TestCheckScalrSignature(t *testing.T) {
for _, testCase := range checkScalrSignatureTests { for _, testCase := range checkScalrSignatureTests {
valid, err := CheckScalrSignature(testCase.headers, testCase.payload, testCase.secret, false) r := &Request{
Headers: testCase.headers,
Body: testCase.body,
}
valid, err := CheckScalrSignature(r, testCase.secret, false)
if valid != testCase.ok { if valid != testCase.ok {
t.Errorf("failed to check scalr signature fot test case: %s\nexpected ok:%#v, got ok:%#v}", t.Errorf("failed to check scalr signature fot test case: %s\nexpected ok:%#v, got ok:%#v}",
testCase.description, testCase.ok, valid) testCase.description, testCase.ok, valid)
@@ -217,6 +228,9 @@ var extractParameterTests = []struct {
{"a.b.0", map[string]interface{}{"a": map[string]interface{}{"b": []interface{}{"x", "y", "z"}}}, "x", true}, {"a.b.0", map[string]interface{}{"a": map[string]interface{}{"b": []interface{}{"x", "y", "z"}}}, "x", true},
{"a.1.b", map[string]interface{}{"a": []interface{}{map[string]interface{}{"b": "y"}, map[string]interface{}{"b": "z"}}}, "z", true}, {"a.1.b", map[string]interface{}{"a": []interface{}{map[string]interface{}{"b": "y"}, map[string]interface{}{"b": "z"}}}, "z", true},
{"a.1.b.c", map[string]interface{}{"a": []interface{}{map[string]interface{}{"b": map[string]interface{}{"c": "y"}}, map[string]interface{}{"b": map[string]interface{}{"c": "z"}}}}, "z", true}, {"a.1.b.c", map[string]interface{}{"a": []interface{}{map[string]interface{}{"b": map[string]interface{}{"c": "y"}}, map[string]interface{}{"b": map[string]interface{}{"c": "z"}}}}, "z", true},
{"b", map[string]interface{}{"b": map[string]interface{}{"z": 1}}, `{"z":1}`, true},
{"c", map[string]interface{}{"c": []interface{}{"y", "z"}}, `["y","z"]`, true},
{"d", map[string]interface{}{"d": [2]interface{}{"y", "z"}}, `["y","z"]`, true},
// failures // failures
{"check_nil", nil, "", false}, {"check_nil", nil, "", false},
{"a.X", map[string]interface{}{"a": map[string]interface{}{"b": "z"}}, "", false}, // non-existent parameter reference {"a.X", map[string]interface{}{"a": map[string]interface{}{"b": "z"}}, "", false}, // non-existent parameter reference
@@ -233,60 +247,74 @@ func TestExtractParameter(t *testing.T) {
for _, tt := range extractParameterTests { for _, tt := range extractParameterTests {
value, err := ExtractParameterAsString(tt.s, tt.params) value, err := ExtractParameterAsString(tt.s, tt.params)
if (err == nil) != tt.ok || value != tt.value { if (err == nil) != tt.ok || value != tt.value {
t.Errorf("failed to extract parameter %q:\nexpected {value:%#v, ok:%#v},\ngot {value:%#v, err:%s}", tt.s, tt.value, tt.ok, value, err) t.Errorf("failed to extract parameter %q:\nexpected {value:%#v, ok:%#v},\ngot {value:%#v, err:%v}", tt.s, tt.value, tt.ok, value, err)
} }
} }
} }
var argumentGetTests = []struct { var argumentGetTests = []struct {
source, name string source, name string
headers, query, payload *map[string]interface{} headers, query, payload map[string]interface{}
request *http.Request
value string value string
ok bool ok bool
}{ }{
{"header", "a", &map[string]interface{}{"A": "z"}, nil, nil, "z", true}, {"header", "a", map[string]interface{}{"A": "z"}, nil, nil, nil, "z", true},
{"url", "a", nil, &map[string]interface{}{"a": "z"}, nil, "z", true}, {"url", "a", nil, map[string]interface{}{"a": "z"}, nil, nil, "z", true},
{"payload", "a", nil, nil, &map[string]interface{}{"a": "z"}, "z", true}, {"payload", "a", nil, nil, map[string]interface{}{"a": "z"}, nil, "z", true},
{"string", "a", nil, nil, &map[string]interface{}{"a": "z"}, "a", true}, {"request", "METHOD", nil, nil, map[string]interface{}{"a": "z"}, &http.Request{Method: "POST", RemoteAddr: "127.0.0.1:1234"}, "POST", true},
{"request", "remote-addr", nil, nil, map[string]interface{}{"a": "z"}, &http.Request{Method: "POST", RemoteAddr: "127.0.0.1:1234"}, "127.0.0.1:1234", true},
{"string", "a", nil, nil, map[string]interface{}{"a": "z"}, nil, "a", true},
// failures // failures
{"header", "a", nil, &map[string]interface{}{"a": "z"}, &map[string]interface{}{"a": "z"}, "", false}, // nil headers {"header", "a", nil, map[string]interface{}{"a": "z"}, map[string]interface{}{"a": "z"}, nil, "", false}, // nil headers
{"url", "a", &map[string]interface{}{"A": "z"}, nil, &map[string]interface{}{"a": "z"}, "", false}, // nil query {"url", "a", map[string]interface{}{"A": "z"}, nil, map[string]interface{}{"a": "z"}, nil, "", false}, // nil query
{"payload", "a", &map[string]interface{}{"A": "z"}, &map[string]interface{}{"a": "z"}, nil, "", false}, // nil payload {"payload", "a", map[string]interface{}{"A": "z"}, map[string]interface{}{"a": "z"}, nil, nil, "", false}, // nil payload
{"foo", "a", &map[string]interface{}{"A": "z"}, nil, nil, "", false}, // invalid source {"foo", "a", map[string]interface{}{"A": "z"}, nil, nil, nil, "", false}, // invalid source
} }
func TestArgumentGet(t *testing.T) { func TestArgumentGet(t *testing.T) {
for _, tt := range argumentGetTests { for _, tt := range argumentGetTests {
a := Argument{tt.source, tt.name, "", false} a := Argument{tt.source, tt.name, "", false}
value, err := a.Get(tt.headers, tt.query, tt.payload) r := &Request{
Headers: tt.headers,
Query: tt.query,
Payload: tt.payload,
RawRequest: tt.request,
}
value, err := a.Get(r)
if (err == nil) != tt.ok || value != tt.value { if (err == nil) != tt.ok || value != tt.value {
t.Errorf("failed to get {%q, %q}:\nexpected {value:%#v, ok:%#v},\ngot {value:%#v, err:%s}", tt.source, tt.name, tt.value, tt.ok, value, err) t.Errorf("failed to get {%q, %q}:\nexpected {value:%#v, ok:%#v},\ngot {value:%#v, err:%v}", tt.source, tt.name, tt.value, tt.ok, value, err)
} }
} }
} }
var hookParseJSONParametersTests = []struct { var hookParseJSONParametersTests = []struct {
params []Argument params []Argument
headers, query, payload *map[string]interface{} headers, query, payload map[string]interface{}
rheaders, rquery, rpayload *map[string]interface{} rheaders, rquery, rpayload map[string]interface{}
ok bool ok bool
}{ }{
{[]Argument{Argument{"header", "a", "", false}}, &map[string]interface{}{"A": `{"b": "y"}`}, nil, nil, &map[string]interface{}{"A": map[string]interface{}{"b": "y"}}, nil, nil, true}, {[]Argument{Argument{"header", "a", "", false}}, map[string]interface{}{"A": `{"b": "y"}`}, nil, nil, map[string]interface{}{"A": map[string]interface{}{"b": "y"}}, nil, nil, true},
{[]Argument{Argument{"url", "a", "", false}}, nil, &map[string]interface{}{"a": `{"b": "y"}`}, nil, nil, &map[string]interface{}{"a": map[string]interface{}{"b": "y"}}, nil, true}, {[]Argument{Argument{"url", "a", "", false}}, nil, map[string]interface{}{"a": `{"b": "y"}`}, nil, nil, map[string]interface{}{"a": map[string]interface{}{"b": "y"}}, nil, true},
{[]Argument{Argument{"payload", "a", "", false}}, nil, nil, &map[string]interface{}{"a": `{"b": "y"}`}, nil, nil, &map[string]interface{}{"a": map[string]interface{}{"b": "y"}}, true}, {[]Argument{Argument{"payload", "a", "", false}}, nil, nil, map[string]interface{}{"a": `{"b": "y"}`}, nil, nil, map[string]interface{}{"a": map[string]interface{}{"b": "y"}}, true},
{[]Argument{Argument{"header", "z", "", false}}, &map[string]interface{}{"Z": `{}`}, nil, nil, &map[string]interface{}{"Z": map[string]interface{}{}}, nil, nil, true}, {[]Argument{Argument{"header", "z", "", false}}, map[string]interface{}{"Z": `{}`}, nil, nil, map[string]interface{}{"Z": map[string]interface{}{}}, nil, nil, true},
// failures // failures
{[]Argument{Argument{"header", "z", "", false}}, &map[string]interface{}{"Z": ``}, nil, nil, &map[string]interface{}{"Z": ``}, nil, nil, false}, // empty string {[]Argument{Argument{"header", "z", "", false}}, map[string]interface{}{"Z": ``}, nil, nil, map[string]interface{}{"Z": ``}, nil, nil, false}, // empty string
{[]Argument{Argument{"header", "y", "", false}}, &map[string]interface{}{"X": `{}`}, nil, nil, &map[string]interface{}{"X": `{}`}, nil, nil, false}, // missing parameter {[]Argument{Argument{"header", "y", "", false}}, map[string]interface{}{"X": `{}`}, nil, nil, map[string]interface{}{"X": `{}`}, nil, nil, false}, // missing parameter
{[]Argument{Argument{"string", "z", "", false}}, &map[string]interface{}{"Z": ``}, nil, nil, &map[string]interface{}{"Z": ``}, nil, nil, false}, // invalid argument source {[]Argument{Argument{"string", "z", "", false}}, map[string]interface{}{"Z": ``}, nil, nil, map[string]interface{}{"Z": ``}, nil, nil, false}, // invalid argument source
} }
func TestHookParseJSONParameters(t *testing.T) { func TestHookParseJSONParameters(t *testing.T) {
for _, tt := range hookParseJSONParametersTests { for _, tt := range hookParseJSONParametersTests {
h := &Hook{JSONStringParameters: tt.params} h := &Hook{JSONStringParameters: tt.params}
err := h.ParseJSONParameters(tt.headers, tt.query, tt.payload) r := &Request{
Headers: tt.headers,
Query: tt.query,
Payload: tt.payload,
}
err := h.ParseJSONParameters(r)
if (err == nil) != tt.ok || !reflect.DeepEqual(tt.headers, tt.rheaders) { if (err == nil) != tt.ok || !reflect.DeepEqual(tt.headers, tt.rheaders) {
t.Errorf("failed to parse %v:\nexpected %#v, ok: %v\ngot %#v, ok: %v", tt.params, *tt.rheaders, tt.ok, *tt.headers, (err == nil)) t.Errorf("failed to parse %v:\nexpected %#v, ok: %v\ngot %#v, ok: %v", tt.params, tt.rheaders, tt.ok, tt.headers, (err == nil))
} }
} }
} }
@@ -294,19 +322,24 @@ func TestHookParseJSONParameters(t *testing.T) {
var hookExtractCommandArgumentsTests = []struct { var hookExtractCommandArgumentsTests = []struct {
exec string exec string
args []Argument args []Argument
headers, query, payload *map[string]interface{} headers, query, payload map[string]interface{}
value []string value []string
ok bool ok bool
}{ }{
{"test", []Argument{Argument{"header", "a", "", false}}, &map[string]interface{}{"A": "z"}, nil, nil, []string{"test", "z"}, true}, {"test", []Argument{Argument{"header", "a", "", false}}, map[string]interface{}{"A": "z"}, nil, nil, []string{"test", "z"}, true},
// failures // failures
{"fail", []Argument{Argument{"payload", "a", "", false}}, &map[string]interface{}{"A": "z"}, nil, nil, []string{"fail", ""}, false}, {"fail", []Argument{Argument{"payload", "a", "", false}}, map[string]interface{}{"A": "z"}, nil, nil, []string{"fail", ""}, false},
} }
func TestHookExtractCommandArguments(t *testing.T) { func TestHookExtractCommandArguments(t *testing.T) {
for _, tt := range hookExtractCommandArgumentsTests { for _, tt := range hookExtractCommandArgumentsTests {
h := &Hook{ExecuteCommand: tt.exec, PassArgumentsToCommand: tt.args} h := &Hook{ExecuteCommand: tt.exec, PassArgumentsToCommand: tt.args}
value, err := h.ExtractCommandArguments(tt.headers, tt.query, tt.payload) r := &Request{
Headers: tt.headers,
Query: tt.query,
Payload: tt.payload,
}
value, err := h.ExtractCommandArguments(r)
if (err == nil) != tt.ok || !reflect.DeepEqual(value, tt.value) { if (err == nil) != tt.ok || !reflect.DeepEqual(value, tt.value) {
t.Errorf("failed to extract args {cmd=%q, args=%v}:\nexpected %#v, ok: %v\ngot %#v, ok: %v", tt.exec, tt.args, tt.value, tt.ok, value, (err == nil)) t.Errorf("failed to extract args {cmd=%q, args=%v}:\nexpected %#v, ok: %v\ngot %#v, ok: %v", tt.exec, tt.args, tt.value, tt.ok, value, (err == nil))
} }
@@ -335,7 +368,7 @@ func TestHookExtractCommandArguments(t *testing.T) {
var hookExtractCommandArgumentsForEnvTests = []struct { var hookExtractCommandArgumentsForEnvTests = []struct {
exec string exec string
args []Argument args []Argument
headers, query, payload *map[string]interface{} headers, query, payload map[string]interface{}
value []string value []string
ok bool ok bool
}{ }{
@@ -343,14 +376,14 @@ var hookExtractCommandArgumentsForEnvTests = []struct {
{ {
"test", "test",
[]Argument{Argument{"header", "a", "", false}}, []Argument{Argument{"header", "a", "", false}},
&map[string]interface{}{"A": "z"}, nil, nil, map[string]interface{}{"A": "z"}, nil, nil,
[]string{"HOOK_a=z"}, []string{"HOOK_a=z"},
true, true,
}, },
{ {
"test", "test",
[]Argument{Argument{"header", "a", "MYKEY", false}}, []Argument{Argument{"header", "a", "MYKEY", false}},
&map[string]interface{}{"A": "z"}, nil, nil, map[string]interface{}{"A": "z"}, nil, nil,
[]string{"MYKEY=z"}, []string{"MYKEY=z"},
true, true,
}, },
@@ -358,7 +391,7 @@ var hookExtractCommandArgumentsForEnvTests = []struct {
{ {
"fail", "fail",
[]Argument{Argument{"payload", "a", "", false}}, []Argument{Argument{"payload", "a", "", false}},
&map[string]interface{}{"A": "z"}, nil, nil, map[string]interface{}{"A": "z"}, nil, nil,
[]string{}, []string{},
false, false,
}, },
@@ -367,7 +400,12 @@ var hookExtractCommandArgumentsForEnvTests = []struct {
func TestHookExtractCommandArgumentsForEnv(t *testing.T) { func TestHookExtractCommandArgumentsForEnv(t *testing.T) {
for _, tt := range hookExtractCommandArgumentsForEnvTests { for _, tt := range hookExtractCommandArgumentsForEnvTests {
h := &Hook{ExecuteCommand: tt.exec, PassEnvironmentToCommand: tt.args} h := &Hook{ExecuteCommand: tt.exec, PassEnvironmentToCommand: tt.args}
value, err := h.ExtractCommandArgumentsForEnv(tt.headers, tt.query, tt.payload) r := &Request{
Headers: tt.headers,
Query: tt.query,
Payload: tt.payload,
}
value, err := h.ExtractCommandArgumentsForEnv(r)
if (err == nil) != tt.ok || !reflect.DeepEqual(value, tt.value) { if (err == nil) != tt.ok || !reflect.DeepEqual(value, tt.value) {
t.Errorf("failed to extract args for env {cmd=%q, args=%v}:\nexpected %#v, ok: %v\ngot %#v, ok: %v", tt.exec, tt.args, tt.value, tt.ok, value, (err == nil)) t.Errorf("failed to extract args for env {cmd=%q, args=%v}:\nexpected %#v, ok: %v\ngot %#v, ok: %v", tt.exec, tt.args, tt.value, tt.ok, value, (err == nil))
} }
@@ -445,24 +483,30 @@ func TestHooksMatch(t *testing.T) {
var matchRuleTests = []struct { var matchRuleTests = []struct {
typ, regex, secret, value, ipRange string typ, regex, secret, value, ipRange string
param Argument param Argument
headers, query, payload *map[string]interface{} headers, query, payload map[string]interface{}
body []byte body []byte
remoteAddr string remoteAddr string
ok bool ok bool
err bool err bool
}{ }{
{"value", "", "", "z", "", Argument{"header", "a", "", false}, &map[string]interface{}{"A": "z"}, nil, nil, []byte{}, "", true, false}, {"value", "", "", "z", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": "z"}, nil, nil, []byte{}, "", true, false},
{"regex", "^z", "", "z", "", Argument{"header", "a", "", false}, &map[string]interface{}{"A": "z"}, nil, nil, []byte{}, "", true, false}, {"regex", "^z", "", "z", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": "z"}, nil, nil, []byte{}, "", true, false},
{"payload-hash-sha1", "", "secret", "", "", Argument{"header", "a", "", false}, &map[string]interface{}{"A": "b17e04cbb22afa8ffbff8796fc1894ed27badd9e"}, nil, nil, []byte(`{"a": "z"}`), "", true, false}, {"payload-hmac-sha1", "", "secret", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": "b17e04cbb22afa8ffbff8796fc1894ed27badd9e"}, nil, nil, []byte(`{"a": "z"}`), "", true, false},
{"payload-hash-sha256", "", "secret", "", "", Argument{"header", "a", "", false}, &map[string]interface{}{"A": "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89"}, nil, nil, []byte(`{"a": "z"}`), "", true, false}, {"payload-hash-sha1", "", "secret", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": "b17e04cbb22afa8ffbff8796fc1894ed27badd9e"}, nil, nil, []byte(`{"a": "z"}`), "", true, false},
{"payload-hmac-sha256", "", "secret", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89"}, nil, nil, []byte(`{"a": "z"}`), "", true, false},
{"payload-hash-sha256", "", "secret", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": "f417af3a21bd70379b5796d5f013915e7029f62c580fb0f500f59a35a6f04c89"}, nil, nil, []byte(`{"a": "z"}`), "", true, false},
// failures // failures
{"value", "", "", "X", "", Argument{"header", "a", "", false}, &map[string]interface{}{"A": "z"}, nil, nil, []byte{}, "", false, false}, {"value", "", "", "X", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": "z"}, nil, nil, []byte{}, "", false, false},
{"regex", "^X", "", "", "", Argument{"header", "a", "", false}, &map[string]interface{}{"A": "z"}, nil, nil, []byte{}, "", false, false}, {"regex", "^X", "", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": "z"}, nil, nil, []byte{}, "", false, false},
{"value", "", "2", "X", "", Argument{"header", "a", "", false}, &map[string]interface{}{"Y": "z"}, nil, nil, []byte{}, "", false, true}, // reference invalid header {"value", "", "2", "X", "", Argument{"header", "a", "", false}, map[string]interface{}{"Y": "z"}, nil, nil, []byte{}, "", false, true}, // reference invalid header
// errors // errors
{"regex", "*", "", "", "", Argument{"header", "a", "", false}, &map[string]interface{}{"A": "z"}, nil, nil, []byte{}, "", false, true}, // invalid regex {"regex", "*", "", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": "z"}, nil, nil, []byte{}, "", false, true}, // invalid regex
{"payload-hash-sha1", "", "secret", "", "", Argument{"header", "a", "", false}, &map[string]interface{}{"A": ""}, nil, nil, []byte{}, "", false, true}, // invalid hmac {"payload-hmac-sha1", "", "secret", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": ""}, nil, nil, []byte{}, "", false, true}, // invalid hmac
{"payload-hash-sha256", "", "secret", "", "", Argument{"header", "a", "", false}, &map[string]interface{}{"A": ""}, nil, nil, []byte{}, "", false, true}, // invalid hmac {"payload-hash-sha1", "", "secret", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": ""}, nil, nil, []byte{}, "", false, true}, // invalid hmac
{"payload-hmac-sha256", "", "secret", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": ""}, nil, nil, []byte{}, "", false, true}, // invalid hmac
{"payload-hash-sha256", "", "secret", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": ""}, nil, nil, []byte{}, "", false, true}, // invalid hmac
{"payload-hmac-sha512", "", "secret", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": ""}, nil, nil, []byte{}, "", false, true}, // invalid hmac
{"payload-hash-sha512", "", "secret", "", "", Argument{"header", "a", "", false}, map[string]interface{}{"A": ""}, nil, nil, []byte{}, "", false, true}, // invalid hmac
// IP whitelisting, valid cases // IP whitelisting, valid cases
{"ip-whitelist", "", "", "", "192.168.0.1/24", Argument{}, nil, nil, nil, []byte{}, "192.168.0.2:9000", true, false}, // valid IPv4, with range {"ip-whitelist", "", "", "", "192.168.0.1/24", Argument{}, nil, nil, nil, []byte{}, "192.168.0.2:9000", true, false}, // valid IPv4, with range
{"ip-whitelist", "", "", "", "192.168.0.1/24", Argument{}, nil, nil, nil, []byte{}, "192.168.0.2:9000", true, false}, // valid IPv4, with range {"ip-whitelist", "", "", "", "192.168.0.1/24", Argument{}, nil, nil, nil, []byte{}, "192.168.0.2:9000", true, false}, // valid IPv4, with range
@@ -481,7 +525,16 @@ var matchRuleTests = []struct {
func TestMatchRule(t *testing.T) { func TestMatchRule(t *testing.T) {
for i, tt := range matchRuleTests { for i, tt := range matchRuleTests {
r := MatchRule{tt.typ, tt.regex, tt.secret, tt.value, tt.param, tt.ipRange} r := MatchRule{tt.typ, tt.regex, tt.secret, tt.value, tt.param, tt.ipRange}
ok, err := r.Evaluate(tt.headers, tt.query, tt.payload, &tt.body, tt.remoteAddr) req := &Request{
Headers: tt.headers,
Query: tt.query,
Payload: tt.payload,
Body: tt.body,
RawRequest: &http.Request{
RemoteAddr: tt.remoteAddr,
},
}
ok, err := r.Evaluate(req)
if ok != tt.ok || (err != nil) != tt.err { if ok != tt.ok || (err != nil) != tt.err {
t.Errorf("%d failed to match %#v:\nexpected ok: %#v, err: %v\ngot ok: %#v, err: %v", i, r, tt.ok, tt.err, ok, err) t.Errorf("%d failed to match %#v:\nexpected ok: %#v, err: %v\ngot ok: %#v, err: %v", i, r, tt.ok, tt.err, ok, err)
} }
@@ -491,7 +544,7 @@ func TestMatchRule(t *testing.T) {
var andRuleTests = []struct { var andRuleTests = []struct {
desc string // description of the test case desc string // description of the test case
rule AndRule rule AndRule
headers, query, payload *map[string]interface{} headers, query, payload map[string]interface{}
body []byte body []byte
ok bool ok bool
err bool err bool
@@ -502,7 +555,8 @@ var andRuleTests = []struct {
{Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}}, {Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}},
{Match: &MatchRule{"value", "", "", "y", Argument{"header", "b", "", false}, ""}}, {Match: &MatchRule{"value", "", "", "y", Argument{"header", "b", "", false}, ""}},
}, },
&map[string]interface{}{"A": "z", "B": "y"}, nil, nil, []byte{}, map[string]interface{}{"A": "z", "B": "y"}, nil, nil,
[]byte{},
true, false, true, false,
}, },
{ {
@@ -511,7 +565,8 @@ var andRuleTests = []struct {
{Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}}, {Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}},
{Match: &MatchRule{"value", "", "", "y", Argument{"header", "b", "", false}, ""}}, {Match: &MatchRule{"value", "", "", "y", Argument{"header", "b", "", false}, ""}},
}, },
&map[string]interface{}{"A": "z", "B": "Y"}, nil, nil, []byte{}, map[string]interface{}{"A": "z", "B": "Y"}, nil, nil,
[]byte{},
false, false, false, false,
}, },
// Complex test to cover Rules.Evaluate // Complex test to cover Rules.Evaluate
@@ -537,7 +592,8 @@ var andRuleTests = []struct {
}, },
}, },
}, },
&map[string]interface{}{"A": "z", "B": "y", "C": "x", "D": "w", "E": "X", "F": "X"}, nil, nil, []byte{}, map[string]interface{}{"A": "z", "B": "y", "C": "x", "D": "w", "E": "X", "F": "X"}, nil, nil,
[]byte{},
true, false, true, false,
}, },
{"empty rule", AndRule{{}}, nil, nil, nil, nil, false, false}, {"empty rule", AndRule{{}}, nil, nil, nil, nil, false, false},
@@ -545,14 +601,20 @@ var andRuleTests = []struct {
{ {
"invalid rule", "invalid rule",
AndRule{{Match: &MatchRule{"value", "", "", "X", Argument{"header", "a", "", false}, ""}}}, AndRule{{Match: &MatchRule{"value", "", "", "X", Argument{"header", "a", "", false}, ""}}},
&map[string]interface{}{"Y": "z"}, nil, nil, nil, map[string]interface{}{"Y": "z"}, nil, nil, nil,
false, true, false, true,
}, },
} }
func TestAndRule(t *testing.T) { func TestAndRule(t *testing.T) {
for _, tt := range andRuleTests { for _, tt := range andRuleTests {
ok, err := tt.rule.Evaluate(tt.headers, tt.query, tt.payload, &tt.body, "") r := &Request{
Headers: tt.headers,
Query: tt.query,
Payload: tt.payload,
Body: tt.body,
}
ok, err := tt.rule.Evaluate(r)
if ok != tt.ok || (err != nil) != tt.err { if ok != tt.ok || (err != nil) != tt.err {
t.Errorf("failed to match %#v:\nexpected ok: %#v, err: %v\ngot ok: %#v, err: %v", tt.desc, tt.ok, tt.err, ok, err) t.Errorf("failed to match %#v:\nexpected ok: %#v, err: %v\ngot ok: %#v, err: %v", tt.desc, tt.ok, tt.err, ok, err)
} }
@@ -562,7 +624,7 @@ func TestAndRule(t *testing.T) {
var orRuleTests = []struct { var orRuleTests = []struct {
desc string // description of the test case desc string // description of the test case
rule OrRule rule OrRule
headers, query, payload *map[string]interface{} headers, query, payload map[string]interface{}
body []byte body []byte
ok bool ok bool
err bool err bool
@@ -573,7 +635,8 @@ var orRuleTests = []struct {
{Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}}, {Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}},
{Match: &MatchRule{"value", "", "", "y", Argument{"header", "b", "", false}, ""}}, {Match: &MatchRule{"value", "", "", "y", Argument{"header", "b", "", false}, ""}},
}, },
&map[string]interface{}{"A": "z", "B": "X"}, nil, nil, []byte{}, map[string]interface{}{"A": "z", "B": "X"}, nil, nil,
[]byte{},
true, false, true, false,
}, },
{ {
@@ -582,7 +645,8 @@ var orRuleTests = []struct {
{Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}}, {Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}},
{Match: &MatchRule{"value", "", "", "y", Argument{"header", "b", "", false}, ""}}, {Match: &MatchRule{"value", "", "", "y", Argument{"header", "b", "", false}, ""}},
}, },
&map[string]interface{}{"A": "X", "B": "y"}, nil, nil, []byte{}, map[string]interface{}{"A": "X", "B": "y"}, nil, nil,
[]byte{},
true, false, true, false,
}, },
{ {
@@ -591,23 +655,31 @@ var orRuleTests = []struct {
{Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}}, {Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}},
{Match: &MatchRule{"value", "", "", "y", Argument{"header", "b", "", false}, ""}}, {Match: &MatchRule{"value", "", "", "y", Argument{"header", "b", "", false}, ""}},
}, },
&map[string]interface{}{"A": "Z", "B": "Y"}, nil, nil, []byte{}, map[string]interface{}{"A": "Z", "B": "Y"}, nil, nil,
[]byte{},
false, false, false, false,
}, },
// failures // failures
{ {
"invalid rule", "missing parameter node",
OrRule{ OrRule{
{Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}}, {Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}},
}, },
&map[string]interface{}{"Y": "Z"}, nil, nil, []byte{}, map[string]interface{}{"Y": "Z"}, nil, nil,
false, true, []byte{},
false, false,
}, },
} }
func TestOrRule(t *testing.T) { func TestOrRule(t *testing.T) {
for _, tt := range orRuleTests { for _, tt := range orRuleTests {
ok, err := tt.rule.Evaluate(tt.headers, tt.query, tt.payload, &tt.body, "") r := &Request{
Headers: tt.headers,
Query: tt.query,
Payload: tt.payload,
Body: tt.body,
}
ok, err := tt.rule.Evaluate(r)
if ok != tt.ok || (err != nil) != tt.err { if ok != tt.ok || (err != nil) != tt.err {
t.Errorf("%#v:\nexpected ok: %#v, err: %v\ngot ok: %#v err: %v", tt.desc, tt.ok, tt.err, ok, err) t.Errorf("%#v:\nexpected ok: %#v, err: %v\ngot ok: %#v err: %v", tt.desc, tt.ok, tt.err, ok, err)
} }
@@ -617,18 +689,24 @@ func TestOrRule(t *testing.T) {
var notRuleTests = []struct { var notRuleTests = []struct {
desc string // description of the test case desc string // description of the test case
rule NotRule rule NotRule
headers, query, payload *map[string]interface{} headers, query, payload map[string]interface{}
body []byte body []byte
ok bool ok bool
err bool err bool
}{ }{
{"(a=z): !a=X", NotRule{Match: &MatchRule{"value", "", "", "X", Argument{"header", "a", "", false}, ""}}, &map[string]interface{}{"A": "z"}, nil, nil, []byte{}, true, false}, {"(a=z): !a=X", NotRule{Match: &MatchRule{"value", "", "", "X", Argument{"header", "a", "", false}, ""}}, map[string]interface{}{"A": "z"}, nil, nil, []byte{}, true, false},
{"(a=z): !a=z", NotRule{Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}}, &map[string]interface{}{"A": "z"}, nil, nil, []byte{}, false, false}, {"(a=z): !a=z", NotRule{Match: &MatchRule{"value", "", "", "z", Argument{"header", "a", "", false}, ""}}, map[string]interface{}{"A": "z"}, nil, nil, []byte{}, false, false},
} }
func TestNotRule(t *testing.T) { func TestNotRule(t *testing.T) {
for _, tt := range notRuleTests { for _, tt := range notRuleTests {
ok, err := tt.rule.Evaluate(tt.headers, tt.query, tt.payload, &tt.body, "") r := &Request{
Headers: tt.headers,
Query: tt.query,
Payload: tt.payload,
Body: tt.body,
}
ok, err := tt.rule.Evaluate(r)
if ok != tt.ok || (err != nil) != tt.err { if ok != tt.ok || (err != nil) != tt.err {
t.Errorf("failed to match %#v:\nexpected ok: %#v, err: %v\ngot ok: %#v, err: %v", tt.rule, tt.ok, tt.err, ok, err) t.Errorf("failed to match %#v:\nexpected ok: %#v, err: %v\ngot ok: %#v, err: %v", tt.rule, tt.ok, tt.err, ok, err)
} }
+119
View File
@@ -0,0 +1,119 @@
package hook
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
"net/url"
"unicode"
"github.com/clbanning/mxj"
)
// Request represents a webhook request.
type Request struct {
// The request ID set by the RequestID middleware.
ID string
// The Content-Type of the request.
ContentType string
// The raw request body.
Body []byte
// Headers is a map of the parsed headers.
Headers map[string]interface{}
// Query is a map of the parsed URL query values.
Query map[string]interface{}
// Payload is a map of the parsed payload.
Payload map[string]interface{}
// The underlying HTTP request.
RawRequest *http.Request
// Treat signature errors as simple validate failures.
AllowSignatureErrors bool
}
func (r *Request) ParseJSONPayload() error {
decoder := json.NewDecoder(bytes.NewReader(r.Body))
decoder.UseNumber()
var firstChar byte
for i := 0; i < len(r.Body); i++ {
if unicode.IsSpace(rune(r.Body[i])) {
continue
}
firstChar = r.Body[i]
break
}
if firstChar == byte('[') {
var arrayPayload interface{}
err := decoder.Decode(&arrayPayload)
if err != nil {
return fmt.Errorf("error parsing JSON array payload %+v", err)
}
r.Payload = make(map[string]interface{}, 1)
r.Payload["root"] = arrayPayload
} else {
err := decoder.Decode(&r.Payload)
if err != nil {
return fmt.Errorf("error parsing JSON payload %+v", err)
}
}
return nil
}
func (r *Request) ParseHeaders(headers map[string][]string) {
r.Headers = make(map[string]interface{}, len(headers))
for k, v := range headers {
if len(v) > 0 {
r.Headers[k] = v[0]
}
}
}
func (r *Request) ParseQuery(query map[string][]string) {
r.Query = make(map[string]interface{}, len(query))
for k, v := range query {
if len(v) > 0 {
r.Query[k] = v[0]
}
}
}
func (r *Request) ParseFormPayload() error {
fd, err := url.ParseQuery(string(r.Body))
if err != nil {
return fmt.Errorf("error parsing form payload %+v", err)
}
r.Payload = make(map[string]interface{}, len(fd))
for k, v := range fd {
if len(v) > 0 {
r.Payload[k] = v[0]
}
}
return nil
}
func (r *Request) ParseXMLPayload() error {
var err error
r.Payload, err = mxj.NewMapXmlReader(bytes.NewReader(r.Body))
if err != nil {
return fmt.Errorf("error parsing XML payload: %+v", err)
}
return nil
}
+30 -49
View File
@@ -8,13 +8,11 @@ import (
"bytes" "bytes"
"fmt" "fmt"
"io" "io"
"io/ioutil"
"net" "net"
"net/http" "net/http"
"net/http/httputil"
"sort" "sort"
"strings" "strings"
"github.com/gorilla/mux"
) )
// responseDupper tees the response to a buffer and a response writer. // responseDupper tees the response to a buffer and a response writer.
@@ -34,70 +32,53 @@ func Dumper(w io.Writer) func(http.Handler) http.Handler {
// Request ID // Request ID
rid := r.Context().Value(RequestIDKey) rid := r.Context().Value(RequestIDKey)
// Request URL // Dump request
buf.WriteString(fmt.Sprintf("> [%s] %s %s", rid, r.Method, r.URL.String()))
// Request Headers bd, err := httputil.DumpRequest(r, true)
keys := make([]string, len(r.Header))
i := 0
for k := range r.Header {
keys[i] = k
i++
}
sort.Strings(keys)
for _, k := range keys {
buf.WriteString(fmt.Sprintf("\n> [%s] %s: %s", rid, k, strings.Join(r.Header[k], ", ")))
}
// Request parameters
params := mux.Vars(r)
keys = make([]string, len(params))
i = 0
for k := range params {
keys[i] = k
i++
}
sort.Strings(keys)
for _, k := range keys {
buf.WriteString(fmt.Sprintf("\n> [%s] %s: %s", rid, k, strings.Join(r.Header[k], ", ")))
}
// Request body
b, err := ioutil.ReadAll(r.Body)
if err != nil { if err != nil {
b = []byte("failed to read body: " + err.Error()) buf.WriteString(fmt.Sprintf("[%s] Error dumping request for debugging: %s\n", rid, err))
} }
if len(b) > 0 {
buf.WriteByte('\n') sc := bufio.NewScanner(bytes.NewBuffer(bd))
lines := strings.Split(string(b), "\n") sc.Split(bufio.ScanLines)
for _, line := range lines { for sc.Scan() {
buf.WriteString(fmt.Sprintf("> [%s] %s\n", rid, line)) buf.WriteString(fmt.Sprintf("> [%s] ", rid))
} buf.WriteString(sc.Text() + "\n")
} }
r.Body = ioutil.NopCloser(bytes.NewBuffer(b))
w.Write(buf.Bytes())
buf.Reset()
// Dump Response
dupper := &responseDupper{ResponseWriter: rw, Buffer: &bytes.Buffer{}} dupper := &responseDupper{ResponseWriter: rw, Buffer: &bytes.Buffer{}}
h.ServeHTTP(dupper, r) h.ServeHTTP(dupper, r)
buf.WriteString(fmt.Sprintf("\n< [%s] %s", rid, http.StatusText(dupper.Status))) // Response Status
keys = make([]string, len(dupper.Header())) buf.WriteString(fmt.Sprintf("< [%s] %d %s\n", rid, dupper.Status, http.StatusText(dupper.Status)))
i = 0
// Response Headers
keys := make([]string, len(dupper.Header()))
i := 0
for k := range dupper.Header() { for k := range dupper.Header() {
keys[i] = k keys[i] = k
i++ i++
} }
sort.Strings(keys) sort.Strings(keys)
for _, k := range keys { for _, k := range keys {
buf.WriteString(fmt.Sprintf("\n< [%s] %s: %s", rid, k, strings.Join(dupper.Header()[k], ", "))) buf.WriteString(fmt.Sprintf("< [%s] %s: %s\n", rid, k, strings.Join(dupper.Header()[k], ", ")))
} }
// Response Body
if dupper.Buffer.Len() > 0 { if dupper.Buffer.Len() > 0 {
buf.WriteByte('\n') buf.WriteString(fmt.Sprintf("< [%s]\n", rid))
lines := strings.Split(dupper.Buffer.String(), "\n") sc = bufio.NewScanner(dupper.Buffer)
for _, line := range lines { sc.Split(bufio.ScanLines)
buf.WriteString(fmt.Sprintf("< [%s] %s\n", rid, line)) for sc.Scan() {
buf.WriteString(fmt.Sprintf("< [%s] ", rid))
buf.WriteString(sc.Text() + "\n")
} }
} }
buf.WriteByte('\n')
w.Write(buf.Bytes()) w.Write(buf.Bytes())
}) })
} }
+3 -3
View File
@@ -39,18 +39,18 @@ type LogEntry struct {
} }
// Write constructs and writes the final log entry. // Write constructs and writes the final log entry.
func (l *LogEntry) Write(status, bytes int, elapsed time.Duration) { func (l *LogEntry) Write(status, totalBytes int, elapsed time.Duration) {
rid := GetReqID(l.req.Context()) rid := GetReqID(l.req.Context())
if rid != "" { if rid != "" {
fmt.Fprintf(l.buf, "[%s] ", rid) fmt.Fprintf(l.buf, "[%s] ", rid)
} }
fmt.Fprintf(l.buf, "%03d | %s | %s | ", status, humanize.IBytes(uint64(bytes)), elapsed) fmt.Fprintf(l.buf, "%03d | %s | %s | ", status, humanize.IBytes(uint64(totalBytes)), elapsed)
l.buf.WriteString(l.req.Host + " | " + l.req.Method + " " + l.req.RequestURI) l.buf.WriteString(l.req.Host + " | " + l.req.Method + " " + l.req.RequestURI)
log.Print(l.buf.String()) log.Print(l.buf.String())
} }
/// Panic prints the call stack for a panic. // Panic prints the call stack for a panic.
func (l *LogEntry) Panic(v interface{}, stack []byte) { func (l *LogEntry) Panic(v interface{}, stack []byte) {
e := l.NewLogEntry(l.req).(*LogEntry) e := l.NewLogEntry(l.req).(*LogEntry)
fmt.Fprintf(e.buf, "panic: %#v", v) fmt.Fprintf(e.buf, "panic: %#v", v)
+2 -2
View File
@@ -35,10 +35,10 @@ func New(path string) (*PIDFile, error) {
return nil, err return nil, err
} }
// Note MkdirAll returns nil if a directory already exists // Note MkdirAll returns nil if a directory already exists
if err := MkdirAll(filepath.Dir(path), os.FileMode(0755)); err != nil { if err := MkdirAll(filepath.Dir(path), os.FileMode(0o755)); err != nil {
return nil, err return nil, err
} }
if err := ioutil.WriteFile(path, []byte(fmt.Sprintf("%d", os.Getpid())), 0644); err != nil { if err := ioutil.WriteFile(path, []byte(fmt.Sprintf("%d", os.Getpid())), 0o600); err != nil {
return nil, err return nil, err
} }
+296 -1
View File
@@ -31,7 +31,7 @@
{ {
"match": "match":
{ {
"type": "payload-hash-sha1", "type": "payload-hmac-sha1",
"secret": "mysecret", "secret": "mysecret",
"parameter": "parameter":
{ {
@@ -55,6 +55,149 @@
] ]
} }
}, },
{
"id": "github-multi-sig",
"execute-command": "{{ .Hookecho }}",
"command-working-directory": "/",
"http-methods": ["Post "],
"include-command-output-in-response": true,
"trigger-rule-mismatch-http-response-code": 400,
"trigger-signature-soft-failures": true,
"pass-environment-to-command":
[
{
"source": "payload",
"name": "head_commit.timestamp"
}
],
"pass-arguments-to-command":
[
{
"source": "payload",
"name": "head_commit.id"
},
{
"source": "payload",
"name": "head_commit.author.email"
}
],
"trigger-rule":
{
"and":
[
"or":
[
{
"match":
{
"type": "payload-hmac-sha1",
"secret": "mysecretFAIL",
"parameter":
{
"source": "header",
"name": "X-Hub-Signature"
}
}
},
{
"match":
{
"type": "payload-hmac-sha1",
"secret": "mysecret",
"parameter":
{
"source": "header",
"name": "X-Hub-Signature"
}
}
}
],
{
"match":
{
"type": "value",
"value": "refs/heads/master",
"parameter":
{
"source": "payload",
"name": "ref"
}
}
}
]
}
},
{
"id": "github-multi-sig-fail",
"execute-command": "{{ .Hookecho }}",
"command-working-directory": "/",
"http-methods": ["Post "],
"include-command-output-in-response": true,
"trigger-rule-mismatch-http-response-code": 400,
"pass-environment-to-command":
[
{
"source": "payload",
"name": "head_commit.timestamp"
}
],
"pass-arguments-to-command":
[
{
"source": "payload",
"name": "head_commit.id"
},
{
"source": "payload",
"name": "head_commit.author.email"
}
],
"trigger-rule":
{
"and":
[
"or":
[
{
"match":
{
"type": "payload-hmac-sha1",
"secret": "mysecretFAIL",
"parameter":
{
"source": "header",
"name": "X-Hub-Signature"
}
}
},
{
"match":
{
"type": "payload-hmac-sha1",
"secret": "mysecret",
"parameter":
{
"source": "header",
"name": "X-Hub-Signature"
}
}
}
],
{
"match":
{
"type": "value",
"value": "refs/heads/master",
"parameter":
{
"source": "payload",
"name": "ref"
}
}
}
]
}
},
{ {
"id": "bitbucket", "id": "bitbucket",
"execute-command": "{{ .Hookecho }}", "execute-command": "{{ .Hookecho }}",
@@ -168,6 +311,49 @@
], ],
} }
}, },
{
"id": "txt-raw",
"execute-command": "{{ .Hookecho }}",
"command-working-directory": "/",
"include-command-output-in-response": true,
"pass-arguments-to-command": [
{
"source": "raw-request-body"
}
]
},
{
"id": "sendgrid",
"execute-command": "{{ .Hookecho }}",
"command-working-directory": "/",
"response-message": "success",
"trigger-rule": {
"match": {
"type": "value",
"parameter": {
"source": "payload",
"name": "root.0.event"
},
"value": "processed"
}
}
},
{
"id": "sendgrid/dir",
"execute-command": "{{ .Hookecho }}",
"command-working-directory": "/",
"response-message": "success",
"trigger-rule": {
"match": {
"type": "value",
"parameter": {
"source": "payload",
"name": "root.0.event"
},
"value": "it worked!"
}
}
},
{ {
"id": "plex", "id": "plex",
"execute-command": "{{ .Hookecho }}", "execute-command": "{{ .Hookecho }}",
@@ -236,6 +422,21 @@
"include-command-output-in-response": true, "include-command-output-in-response": true,
"include-command-output-in-response-on-error": true "include-command-output-in-response-on-error": true
}, },
{
"id": "request-source",
"pass-arguments-to-command": [
{
"source": "request",
"name": "method"
},
{
"source": "request",
"name": "remote-addr"
}
],
"execute-command": "{{ .Hookecho }}",
"include-command-output-in-response": true
},
{ {
"id": "static-params-ok", "id": "static-params-ok",
"execute-command": "{{ .Hookecho }}", "execute-command": "{{ .Hookecho }}",
@@ -259,5 +460,99 @@
"name": "passed" "name": "passed"
} }
], ],
},
{
"id": "issue-471",
"execute-command": "{{ .Hookecho }}",
"response-message": "success",
"trigger-rule":
{
"or":
[
{
"match":
{
"parameter":
{
"source": "payload",
"name": "foo"
},
"type": "value",
"value": "bar"
}
},
{
"match":
{
"parameter":
{
"source": "payload",
"name": "exists"
},
"type": "value",
"value": 1
}
}
]
}
},
{
"id": "issue-471-and",
"execute-command": "{{ .Hookecho }}",
"response-message": "success",
"trigger-rule":
{
"and":
[
{
"match":
{
"parameter":
{
"source": "payload",
"name": "foo"
},
"type": "value",
"value": "bar"
}
},
{
"match":
{
"parameter":
{
"source": "payload",
"name": "exists"
},
"type": "value",
"value": 1
}
}
]
}
},
{
"id": "empty-payload-signature",
"execute-command": "{{ .Hookecho }}",
"command-working-directory": "/",
"include-command-output-in-response": true,
"trigger-rule":
{
"and":
[
{
"match":
{
"type": "payload-hmac-sha1",
"secret": "mysecret",
"parameter":
{
"source": "header",
"name": "X-Hub-Signature"
}
}
}
]
}
} }
] ]
+165 -1
View File
@@ -8,7 +8,82 @@
source: header source: header
name: X-Hub-Signature name: X-Hub-Signature
secret: mysecret secret: mysecret
type: payload-hash-sha1 type: payload-hmac-sha1
- match:
parameter:
source: payload
name: ref
type: value
value: refs/heads/master
include-command-output-in-response: true
trigger-rule-mismatch-http-response-code: 400
execute-command: '{{ .Hookecho }}'
pass-arguments-to-command:
- source: payload
name: head_commit.id
- source: payload
name: head_commit.author.email
pass-environment-to-command:
- source: payload
name: head_commit.timestamp
command-working-directory: /
- id: github-multi-sig
http-methods:
- "Post "
trigger-rule:
and:
- or:
- match:
parameter:
source: header
name: X-Hub-Signature
secret: mysecretFAIL
type: payload-hmac-sha1
- match:
parameter:
source: header
name: X-Hub-Signature
secret: mysecret
type: payload-hmac-sha1
- match:
parameter:
source: payload
name: ref
type: value
value: refs/heads/master
include-command-output-in-response: true
trigger-rule-mismatch-http-response-code: 400
trigger-signature-soft-failures: true
execute-command: '{{ .Hookecho }}'
pass-arguments-to-command:
- source: payload
name: head_commit.id
- source: payload
name: head_commit.author.email
pass-environment-to-command:
- source: payload
name: head_commit.timestamp
command-working-directory: /
- id: github-multi-sig-fail
http-methods:
- "Post "
trigger-rule:
and:
- or:
- match:
parameter:
source: header
name: X-Hub-Signature
secret: mysecretFAIL
type: payload-hmac-sha1
- match:
parameter:
source: header
name: X-Hub-Signature
secret: mysecret
type: payload-hmac-sha1
- match: - match:
parameter: parameter:
source: payload source: payload
@@ -97,6 +172,37 @@
name: "app.messages.message.#text" name: "app.messages.message.#text"
value: "Hello!!" value: "Hello!!"
- id: txt-raw
execute-command: '{{ .Hookecho }}'
command-working-directory: /
include-command-output-in-response: true
pass-arguments-to-command:
- source: raw-request-body
- id: sendgrid
execute-command: '{{ .Hookecho }}'
command-working-directory: /
response-message: success
trigger-rule:
match:
type: value
parameter:
source: payload
name: root.0.event
value: processed
- id: sendgrid/dir
execute-command: '{{ .Hookecho }}'
command-working-directory: /
response-message: success
trigger-rule:
match:
type: value
parameter:
source: payload
name: root.0.event
value: it worked!
- id: plex - id: plex
trigger-rule: trigger-rule:
match: match:
@@ -140,6 +246,15 @@
include-command-output-in-response: true include-command-output-in-response: true
include-command-output-in-response-on-error: true include-command-output-in-response-on-error: true
- id: request-source
pass-arguments-to-command:
- source: request
name: method
- source: request
name: remote-addr
execute-command: '{{ .Hookecho }}'
include-command-output-in-response: true
- id: static-params-ok - id: static-params-ok
execute-command: '{{ .Hookecho }}' execute-command: '{{ .Hookecho }}'
include-command-output-in-response: true include-command-output-in-response: true
@@ -150,3 +265,52 @@
- id: warn-on-space - id: warn-on-space
execute-command: '{{ .Hookecho }} foo' execute-command: '{{ .Hookecho }} foo'
include-command-output-in-response: true include-command-output-in-response: true
- id: issue-471
execute-command: '{{ .Hookecho }}'
response-message: success
trigger-rule:
or:
- match:
parameter:
source: payload
name: foo
type: value
value: bar
- match:
parameter:
source: payload
name: exists
type: value
value: 1
- id: issue-471-and
execute-command: '{{ .Hookecho }}'
response-message: success
trigger-rule:
and:
- match:
parameter:
source: payload
name: foo
type: value
value: bar
- match:
parameter:
source: payload
name: exists
type: value
value: 1
- id: empty-payload-signature
include-command-output-in-response: true
execute-command: '{{ .Hookecho }}'
command-working-directory: /
trigger-rule:
and:
- match:
parameter:
source: header
name: X-Hub-Signature
secret: mysecret
type: payload-hmac-sha1
+7
View File
@@ -0,0 +1,7 @@
*.exe
*.exe~
*.dll
*.so
*.dylib
*.test
*.out
+3
View File
@@ -0,0 +1,3 @@
language: go
go:
- 1.13.x
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2019 Anton Medvedev
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+169
View File
@@ -0,0 +1,169 @@
# Expr
[![Build Status](https://travis-ci.org/antonmedv/expr.svg?branch=master)](https://travis-ci.org/antonmedv/expr)
[![Go Report Card](https://goreportcard.com/badge/github.com/antonmedv/expr)](https://goreportcard.com/report/github.com/antonmedv/expr)
[![GoDoc](https://godoc.org/github.com/antonmedv/expr?status.svg)](https://godoc.org/github.com/antonmedv/expr)
<img src="docs/images/logo-small.png" width="150" alt="expr logo" align="right">
**Expr** package provides an engine that can compile and evaluate expressions.
An expression is a one-liner that returns a value (mostly, but not limited to, booleans).
It is designed for simplicity, speed and safety.
The purpose of the package is to allow users to use expressions inside configuration for more complex logic.
It is a perfect candidate for the foundation of a _business rule engine_.
The idea is to let configure things in a dynamic way without recompile of a program:
```coffeescript
# Get the special price if
user.Group in ["good_customers", "collaborator"]
# Promote article to the homepage when
len(article.Comments) > 100 and article.Category not in ["misc"]
# Send an alert when
product.Stock < 15
```
## Features
* Seamless integration with Go (no need to redefine types)
* Static typing ([example](https://godoc.org/github.com/antonmedv/expr#example-Env)).
```go
out, err := expr.Compile(`name + age`)
// err: invalid operation + (mismatched types string and int)
// | name + age
// | .....^
```
* User-friendly error messages.
* Reasonable set of basic operators.
* Builtins `all`, `none`, `any`, `one`, `filter`, `map`.
```coffeescript
all(Tweets, {.Size <= 280})
```
* Fast ([benchmarks](https://github.com/antonmedv/golang-expression-evaluation-comparison#readme)): uses bytecode virtual machine and optimizing compiler.
## Install
```
go get github.com/antonmedv/expr
```
## Documentation
* See [Getting Started](docs/Getting-Started.md) page for developer documentation.
* See [Language Definition](docs/Language-Definition.md) page to learn the syntax.
## Expr Code Editor
<a href="http://bit.ly/expr-code-editor">
<img src="https://antonmedv.github.io/expr/ogimage.png" align="center" alt="Expr Code Editor" width="1200">
</a>
Also, I have an embeddable code editor written in JavaScript which allows editing expressions with syntax highlighting and autocomplete based on your types declaration.
[Learn more →](https://antonmedv.github.io/expr/)
## Examples
[Play Online](https://play.golang.org/p/z7T8ytJ1T1d)
```go
package main
import (
"fmt"
"github.com/antonmedv/expr"
)
func main() {
env := map[string]interface{}{
"greet": "Hello, %v!",
"names": []string{"world", "you"},
"sprintf": fmt.Sprintf,
}
code := `sprintf(greet, names[0])`
program, err := expr.Compile(code, expr.Env(env))
if err != nil {
panic(err)
}
output, err := expr.Run(program, env)
if err != nil {
panic(err)
}
fmt.Println(output)
}
```
[Play Online](https://play.golang.org/p/4S4brsIvU4i)
```go
package main
import (
"fmt"
"github.com/antonmedv/expr"
)
type Tweet struct {
Len int
}
type Env struct {
Tweets []Tweet
}
func main() {
code := `all(Tweets, {.Len <= 240})`
program, err := expr.Compile(code, expr.Env(Env{}))
if err != nil {
panic(err)
}
env := Env{
Tweets: []Tweet{{42}, {98}, {69}},
}
output, err := expr.Run(program, env)
if err != nil {
panic(err)
}
fmt.Println(output)
}
```
## Contributing
**Expr** consist of a few packages for parsing source code to AST, type checking AST, compiling to bytecode and VM for running bytecode program.
Also expr provides powerful tool [exe](cmd/exe) for debugging. It has interactive terminal debugger for our bytecode virtual machine.
<p align="center">
<img src="docs/images/debug.gif" alt="debugger" width="605">
</p>
## Who is using Expr?
<table>
<tr>
<td>
<p align="center"><a href="https://aviasales.ru"><img alt="Aviasales Logo" height="100" src="https://cdn.worldvectorlogo.com/logos/aviasales-4.svg"></a></p>
<p><a href="https://aviasales.ru">Aviasales</a> are actively using Expr for different parts of the search engine.</p>
</td>
<td>
<p align="center"><a href="https://www.mysteryminds.com/en/"><img alt="Mystery Minds Logo" height="100" src="https://www.mysteryminds.com/wp-content/uploads/2018/07/MM_logo_colours.png"></a></p>
<p><a href="https://www.mysteryminds.com/en/">Mystery Minds</a> uses Expr to allow easy yet powerful customization of its matching algorithm.</p>
</td>
</tr>
</table>
[Add you company too](https://github.com/antonmedv/expr/edit/master/README.md)
## License
[MIT](LICENSE)
+168
View File
@@ -0,0 +1,168 @@
package ast
import (
"reflect"
"regexp"
"github.com/antonmedv/expr/file"
)
// Node represents items of abstract syntax tree.
type Node interface {
Location() file.Location
SetLocation(file.Location)
Type() reflect.Type
SetType(reflect.Type)
}
func Patch(node *Node, newNode Node) {
newNode.SetType((*node).Type())
newNode.SetLocation((*node).Location())
*node = newNode
}
type base struct {
loc file.Location
nodeType reflect.Type
}
func (n *base) Location() file.Location {
return n.loc
}
func (n *base) SetLocation(loc file.Location) {
n.loc = loc
}
func (n *base) Type() reflect.Type {
return n.nodeType
}
func (n *base) SetType(t reflect.Type) {
n.nodeType = t
}
type NilNode struct {
base
}
type IdentifierNode struct {
base
Value string
}
type IntegerNode struct {
base
Value int
}
type FloatNode struct {
base
Value float64
}
type BoolNode struct {
base
Value bool
}
type StringNode struct {
base
Value string
}
type ConstantNode struct {
base
Value interface{}
}
type UnaryNode struct {
base
Operator string
Node Node
}
type BinaryNode struct {
base
Operator string
Left Node
Right Node
}
type MatchesNode struct {
base
Regexp *regexp.Regexp
Left Node
Right Node
}
type PropertyNode struct {
base
Node Node
Property string
}
type IndexNode struct {
base
Node Node
Index Node
}
type SliceNode struct {
base
Node Node
From Node
To Node
}
type MethodNode struct {
base
Node Node
Method string
Arguments []Node
}
type FunctionNode struct {
base
Name string
Arguments []Node
Fast bool
}
type BuiltinNode struct {
base
Name string
Arguments []Node
}
type ClosureNode struct {
base
Node Node
}
type PointerNode struct {
base
}
type ConditionalNode struct {
base
Cond Node
Exp1 Node
Exp2 Node
}
type ArrayNode struct {
base
Nodes []Node
}
type MapNode struct {
base
Pairs []Node
}
type PairNode struct {
base
Key Node
Value Node
}
+59
View File
@@ -0,0 +1,59 @@
package ast
import (
"fmt"
"reflect"
"regexp"
)
func Dump(node Node) string {
return dump(reflect.ValueOf(node), "")
}
func dump(v reflect.Value, ident string) string {
if !v.IsValid() {
return "nil"
}
t := v.Type()
switch t.Kind() {
case reflect.Struct:
out := t.Name() + "{\n"
for i := 0; i < t.NumField(); i++ {
f := t.Field(i)
if isPrivate(f.Name) {
continue
}
s := v.Field(i)
out += fmt.Sprintf("%v%v: %v,\n", ident+"\t", f.Name, dump(s, ident+"\t"))
}
return out + ident + "}"
case reflect.Slice:
if v.Len() == 0 {
return "[]"
}
out := "[\n"
for i := 0; i < v.Len(); i++ {
s := v.Index(i)
out += fmt.Sprintf("%v%v,", ident+"\t", dump(s, ident+"\t"))
if i+1 < v.Len() {
out += "\n"
}
}
return out + "\n" + ident + "]"
case reflect.Ptr:
return dump(v.Elem(), ident)
case reflect.Interface:
return dump(reflect.ValueOf(v.Interface()), ident)
case reflect.String:
return fmt.Sprintf("%q", v)
default:
return fmt.Sprintf("%v", v)
}
}
var isCapital = regexp.MustCompile("^[A-Z]")
func isPrivate(s string) bool {
return !isCapital.Match([]byte(s))
}
+108
View File
@@ -0,0 +1,108 @@
package ast
import "fmt"
type Visitor interface {
Enter(node *Node)
Exit(node *Node)
}
type walker struct {
visitor Visitor
}
func Walk(node *Node, visitor Visitor) {
w := walker{
visitor: visitor,
}
w.walk(node)
}
func (w *walker) walk(node *Node) {
w.visitor.Enter(node)
switch n := (*node).(type) {
case *NilNode:
w.visitor.Exit(node)
case *IdentifierNode:
w.visitor.Exit(node)
case *IntegerNode:
w.visitor.Exit(node)
case *FloatNode:
w.visitor.Exit(node)
case *BoolNode:
w.visitor.Exit(node)
case *StringNode:
w.visitor.Exit(node)
case *ConstantNode:
w.visitor.Exit(node)
case *UnaryNode:
w.walk(&n.Node)
w.visitor.Exit(node)
case *BinaryNode:
w.walk(&n.Left)
w.walk(&n.Right)
w.visitor.Exit(node)
case *MatchesNode:
w.walk(&n.Left)
w.walk(&n.Right)
w.visitor.Exit(node)
case *PropertyNode:
w.walk(&n.Node)
w.visitor.Exit(node)
case *IndexNode:
w.walk(&n.Node)
w.walk(&n.Index)
w.visitor.Exit(node)
case *SliceNode:
if n.From != nil {
w.walk(&n.From)
}
if n.To != nil {
w.walk(&n.To)
}
w.visitor.Exit(node)
case *MethodNode:
w.walk(&n.Node)
for i := range n.Arguments {
w.walk(&n.Arguments[i])
}
w.visitor.Exit(node)
case *FunctionNode:
for i := range n.Arguments {
w.walk(&n.Arguments[i])
}
w.visitor.Exit(node)
case *BuiltinNode:
for i := range n.Arguments {
w.walk(&n.Arguments[i])
}
w.visitor.Exit(node)
case *ClosureNode:
w.walk(&n.Node)
w.visitor.Exit(node)
case *PointerNode:
w.visitor.Exit(node)
case *ConditionalNode:
w.walk(&n.Cond)
w.walk(&n.Exp1)
w.walk(&n.Exp2)
w.visitor.Exit(node)
case *ArrayNode:
for i := range n.Nodes {
w.walk(&n.Nodes[i])
}
w.visitor.Exit(node)
case *MapNode:
for i := range n.Pairs {
w.walk(&n.Pairs[i])
}
w.visitor.Exit(node)
case *PairNode:
w.walk(&n.Key)
w.walk(&n.Value)
w.visitor.Exit(node)
default:
panic(fmt.Sprintf("undefined node type (%T)", node))
}
}
+597
View File
@@ -0,0 +1,597 @@
package checker
import (
"fmt"
"reflect"
"github.com/antonmedv/expr/ast"
"github.com/antonmedv/expr/conf"
"github.com/antonmedv/expr/file"
"github.com/antonmedv/expr/parser"
)
func Check(tree *parser.Tree, config *conf.Config) (reflect.Type, error) {
v := &visitor{
collections: make([]reflect.Type, 0),
}
if config != nil {
v.types = config.Types
v.operators = config.Operators
v.expect = config.Expect
v.strict = config.Strict
v.defaultType = config.DefaultType
}
t := v.visit(tree.Node)
if v.expect != reflect.Invalid {
switch v.expect {
case reflect.Int64, reflect.Float64:
if !isNumber(t) {
return nil, fmt.Errorf("expected %v, but got %v", v.expect, t)
}
default:
if t.Kind() != v.expect {
return nil, fmt.Errorf("expected %v, but got %v", v.expect, t)
}
}
}
if v.err != nil {
return t, v.err.Bind(tree.Source)
}
return t, nil
}
type visitor struct {
types conf.TypesTable
operators conf.OperatorsTable
expect reflect.Kind
collections []reflect.Type
strict bool
defaultType reflect.Type
err *file.Error
}
func (v *visitor) visit(node ast.Node) reflect.Type {
var t reflect.Type
switch n := node.(type) {
case *ast.NilNode:
t = v.NilNode(n)
case *ast.IdentifierNode:
t = v.IdentifierNode(n)
case *ast.IntegerNode:
t = v.IntegerNode(n)
case *ast.FloatNode:
t = v.FloatNode(n)
case *ast.BoolNode:
t = v.BoolNode(n)
case *ast.StringNode:
t = v.StringNode(n)
case *ast.UnaryNode:
t = v.UnaryNode(n)
case *ast.BinaryNode:
t = v.BinaryNode(n)
case *ast.MatchesNode:
t = v.MatchesNode(n)
case *ast.PropertyNode:
t = v.PropertyNode(n)
case *ast.IndexNode:
t = v.IndexNode(n)
case *ast.SliceNode:
t = v.SliceNode(n)
case *ast.MethodNode:
t = v.MethodNode(n)
case *ast.FunctionNode:
t = v.FunctionNode(n)
case *ast.BuiltinNode:
t = v.BuiltinNode(n)
case *ast.ClosureNode:
t = v.ClosureNode(n)
case *ast.PointerNode:
t = v.PointerNode(n)
case *ast.ConditionalNode:
t = v.ConditionalNode(n)
case *ast.ArrayNode:
t = v.ArrayNode(n)
case *ast.MapNode:
t = v.MapNode(n)
case *ast.PairNode:
t = v.PairNode(n)
default:
panic(fmt.Sprintf("undefined node type (%T)", node))
}
node.SetType(t)
return t
}
func (v *visitor) error(node ast.Node, format string, args ...interface{}) reflect.Type {
if v.err == nil { // show first error
v.err = &file.Error{
Location: node.Location(),
Message: fmt.Sprintf(format, args...),
}
}
return interfaceType // interface represent undefined type
}
func (v *visitor) NilNode(*ast.NilNode) reflect.Type {
return nilType
}
func (v *visitor) IdentifierNode(node *ast.IdentifierNode) reflect.Type {
if v.types == nil {
return interfaceType
}
if t, ok := v.types[node.Value]; ok {
if t.Ambiguous {
return v.error(node, "ambiguous identifier %v", node.Value)
}
return t.Type
}
if !v.strict {
if v.defaultType != nil {
return v.defaultType
}
return interfaceType
}
return v.error(node, "unknown name %v", node.Value)
}
func (v *visitor) IntegerNode(*ast.IntegerNode) reflect.Type {
return integerType
}
func (v *visitor) FloatNode(*ast.FloatNode) reflect.Type {
return floatType
}
func (v *visitor) BoolNode(*ast.BoolNode) reflect.Type {
return boolType
}
func (v *visitor) StringNode(*ast.StringNode) reflect.Type {
return stringType
}
func (v *visitor) UnaryNode(node *ast.UnaryNode) reflect.Type {
t := v.visit(node.Node)
switch node.Operator {
case "!", "not":
if isBool(t) {
return boolType
}
case "+", "-":
if isNumber(t) {
return t
}
default:
return v.error(node, "unknown operator (%v)", node.Operator)
}
return v.error(node, `invalid operation: %v (mismatched type %v)`, node.Operator, t)
}
func (v *visitor) BinaryNode(node *ast.BinaryNode) reflect.Type {
l := v.visit(node.Left)
r := v.visit(node.Right)
// check operator overloading
if fns, ok := v.operators[node.Operator]; ok {
t, _, ok := conf.FindSuitableOperatorOverload(fns, v.types, l, r)
if ok {
return t
}
}
switch node.Operator {
case "==", "!=":
if isNumber(l) && isNumber(r) {
return boolType
}
if isComparable(l, r) {
return boolType
}
case "or", "||", "and", "&&":
if isBool(l) && isBool(r) {
return boolType
}
case "in", "not in":
if isString(l) && isStruct(r) {
return boolType
}
if isMap(r) {
return boolType
}
if isArray(r) {
return boolType
}
case "<", ">", ">=", "<=":
if isNumber(l) && isNumber(r) {
return boolType
}
if isString(l) && isString(r) {
return boolType
}
case "/", "-", "*":
if isNumber(l) && isNumber(r) {
return combined(l, r)
}
case "**":
if isNumber(l) && isNumber(r) {
return floatType
}
case "%":
if isInteger(l) && isInteger(r) {
return combined(l, r)
}
case "+":
if isNumber(l) && isNumber(r) {
return combined(l, r)
}
if isString(l) && isString(r) {
return stringType
}
case "contains", "startsWith", "endsWith":
if isString(l) && isString(r) {
return boolType
}
case "..":
if isInteger(l) && isInteger(r) {
return reflect.SliceOf(integerType)
}
default:
return v.error(node, "unknown operator (%v)", node.Operator)
}
return v.error(node, `invalid operation: %v (mismatched types %v and %v)`, node.Operator, l, r)
}
func (v *visitor) MatchesNode(node *ast.MatchesNode) reflect.Type {
l := v.visit(node.Left)
r := v.visit(node.Right)
if isString(l) && isString(r) {
return boolType
}
return v.error(node, `invalid operation: matches (mismatched types %v and %v)`, l, r)
}
func (v *visitor) PropertyNode(node *ast.PropertyNode) reflect.Type {
t := v.visit(node.Node)
if t, ok := fieldType(t, node.Property); ok {
return t
}
return v.error(node, "type %v has no field %v", t, node.Property)
}
func (v *visitor) IndexNode(node *ast.IndexNode) reflect.Type {
t := v.visit(node.Node)
i := v.visit(node.Index)
if t, ok := indexType(t); ok {
if !isInteger(i) && !isString(i) {
return v.error(node, "invalid operation: cannot use %v as index to %v", i, t)
}
return t
}
return v.error(node, "invalid operation: type %v does not support indexing", t)
}
func (v *visitor) SliceNode(node *ast.SliceNode) reflect.Type {
t := v.visit(node.Node)
_, isIndex := indexType(t)
if isIndex || isString(t) {
if node.From != nil {
from := v.visit(node.From)
if !isInteger(from) {
return v.error(node.From, "invalid operation: non-integer slice index %v", from)
}
}
if node.To != nil {
to := v.visit(node.To)
if !isInteger(to) {
return v.error(node.To, "invalid operation: non-integer slice index %v", to)
}
}
return t
}
return v.error(node, "invalid operation: cannot slice %v", t)
}
func (v *visitor) FunctionNode(node *ast.FunctionNode) reflect.Type {
if f, ok := v.types[node.Name]; ok {
if fn, ok := isFuncType(f.Type); ok {
inputParamsCount := 1 // for functions
if f.Method {
inputParamsCount = 2 // for methods
}
if !isInterface(fn) &&
fn.IsVariadic() &&
fn.NumIn() == inputParamsCount &&
fn.NumOut() == 1 &&
fn.Out(0).Kind() == reflect.Interface {
rest := fn.In(fn.NumIn() - 1) // function has only one param for functions and two for methods
if rest.Kind() == reflect.Slice && rest.Elem().Kind() == reflect.Interface {
node.Fast = true
}
}
return v.checkFunc(fn, f.Method, node, node.Name, node.Arguments)
}
}
if !v.strict {
if v.defaultType != nil {
return v.defaultType
}
return interfaceType
}
return v.error(node, "unknown func %v", node.Name)
}
func (v *visitor) MethodNode(node *ast.MethodNode) reflect.Type {
t := v.visit(node.Node)
if f, method, ok := methodType(t, node.Method); ok {
if fn, ok := isFuncType(f); ok {
return v.checkFunc(fn, method, node, node.Method, node.Arguments)
}
}
return v.error(node, "type %v has no method %v", t, node.Method)
}
// checkFunc checks func arguments and returns "return type" of func or method.
func (v *visitor) checkFunc(fn reflect.Type, method bool, node ast.Node, name string, arguments []ast.Node) reflect.Type {
if isInterface(fn) {
return interfaceType
}
if fn.NumOut() == 0 {
return v.error(node, "func %v doesn't return value", name)
}
if fn.NumOut() != 1 {
return v.error(node, "func %v returns more then one value", name)
}
numIn := fn.NumIn()
// If func is method on an env, first argument should be a receiver,
// and actual arguments less then numIn by one.
if method {
numIn--
}
if fn.IsVariadic() {
if len(arguments) < numIn-1 {
return v.error(node, "not enough arguments to call %v", name)
}
} else {
if len(arguments) > numIn {
return v.error(node, "too many arguments to call %v", name)
}
if len(arguments) < numIn {
return v.error(node, "not enough arguments to call %v", name)
}
}
offset := 0
// Skip first argument in case of the receiver.
if method {
offset = 1
}
for i, arg := range arguments {
t := v.visit(arg)
var in reflect.Type
if fn.IsVariadic() && i >= numIn-1 {
// For variadic arguments fn(xs ...int), go replaces type of xs (int) with ([]int).
// As we compare arguments one by one, we need underling type.
in = fn.In(fn.NumIn() - 1)
in, _ = indexType(in)
} else {
in = fn.In(i + offset)
}
if isIntegerOrArithmeticOperation(arg) {
t = in
setTypeForIntegers(arg, t)
}
if t == nil {
continue
}
if !t.AssignableTo(in) && t.Kind() != reflect.Interface {
return v.error(arg, "cannot use %v as argument (type %v) to call %v ", t, in, name)
}
}
return fn.Out(0)
}
func (v *visitor) BuiltinNode(node *ast.BuiltinNode) reflect.Type {
switch node.Name {
case "len":
param := v.visit(node.Arguments[0])
if isArray(param) || isMap(param) || isString(param) {
return integerType
}
return v.error(node, "invalid argument for len (type %v)", param)
case "all", "none", "any", "one":
collection := v.visit(node.Arguments[0])
if !isArray(collection) {
return v.error(node.Arguments[0], "builtin %v takes only array (got %v)", node.Name, collection)
}
v.collections = append(v.collections, collection)
closure := v.visit(node.Arguments[1])
v.collections = v.collections[:len(v.collections)-1]
if isFunc(closure) &&
closure.NumOut() == 1 &&
closure.NumIn() == 1 && isInterface(closure.In(0)) {
if !isBool(closure.Out(0)) {
return v.error(node.Arguments[1], "closure should return boolean (got %v)", closure.Out(0).String())
}
return boolType
}
return v.error(node.Arguments[1], "closure should has one input and one output param")
case "filter":
collection := v.visit(node.Arguments[0])
if !isArray(collection) {
return v.error(node.Arguments[0], "builtin %v takes only array (got %v)", node.Name, collection)
}
v.collections = append(v.collections, collection)
closure := v.visit(node.Arguments[1])
v.collections = v.collections[:len(v.collections)-1]
if isFunc(closure) &&
closure.NumOut() == 1 &&
closure.NumIn() == 1 && isInterface(closure.In(0)) {
if !isBool(closure.Out(0)) {
return v.error(node.Arguments[1], "closure should return boolean (got %v)", closure.Out(0).String())
}
if isInterface(collection) {
return arrayType
}
return reflect.SliceOf(collection.Elem())
}
return v.error(node.Arguments[1], "closure should has one input and one output param")
case "map":
collection := v.visit(node.Arguments[0])
if !isArray(collection) {
return v.error(node.Arguments[0], "builtin %v takes only array (got %v)", node.Name, collection)
}
v.collections = append(v.collections, collection)
closure := v.visit(node.Arguments[1])
v.collections = v.collections[:len(v.collections)-1]
if isFunc(closure) &&
closure.NumOut() == 1 &&
closure.NumIn() == 1 && isInterface(closure.In(0)) {
return reflect.SliceOf(closure.Out(0))
}
return v.error(node.Arguments[1], "closure should has one input and one output param")
case "count":
collection := v.visit(node.Arguments[0])
if !isArray(collection) {
return v.error(node.Arguments[0], "builtin %v takes only array (got %v)", node.Name, collection)
}
v.collections = append(v.collections, collection)
closure := v.visit(node.Arguments[1])
v.collections = v.collections[:len(v.collections)-1]
if isFunc(closure) &&
closure.NumOut() == 1 &&
closure.NumIn() == 1 && isInterface(closure.In(0)) {
if !isBool(closure.Out(0)) {
return v.error(node.Arguments[1], "closure should return boolean (got %v)", closure.Out(0).String())
}
return integerType
}
return v.error(node.Arguments[1], "closure should has one input and one output param")
default:
return v.error(node, "unknown builtin %v", node.Name)
}
}
func (v *visitor) ClosureNode(node *ast.ClosureNode) reflect.Type {
t := v.visit(node.Node)
return reflect.FuncOf([]reflect.Type{interfaceType}, []reflect.Type{t}, false)
}
func (v *visitor) PointerNode(node *ast.PointerNode) reflect.Type {
if len(v.collections) == 0 {
return v.error(node, "cannot use pointer accessor outside closure")
}
collection := v.collections[len(v.collections)-1]
if t, ok := indexType(collection); ok {
return t
}
return v.error(node, "cannot use %v as array", collection)
}
func (v *visitor) ConditionalNode(node *ast.ConditionalNode) reflect.Type {
c := v.visit(node.Cond)
if !isBool(c) {
return v.error(node.Cond, "non-bool expression (type %v) used as condition", c)
}
t1 := v.visit(node.Exp1)
t2 := v.visit(node.Exp2)
if t1 == nil && t2 != nil {
return t2
}
if t1 != nil && t2 == nil {
return t1
}
if t1 == nil && t2 == nil {
return nilType
}
if t1.AssignableTo(t2) {
return t1
}
return interfaceType
}
func (v *visitor) ArrayNode(node *ast.ArrayNode) reflect.Type {
for _, node := range node.Nodes {
_ = v.visit(node)
}
return arrayType
}
func (v *visitor) MapNode(node *ast.MapNode) reflect.Type {
for _, pair := range node.Pairs {
v.visit(pair)
}
return mapType
}
func (v *visitor) PairNode(node *ast.PairNode) reflect.Type {
v.visit(node.Key)
v.visit(node.Value)
return nilType
}
+349
View File
@@ -0,0 +1,349 @@
package checker
import (
"reflect"
"github.com/antonmedv/expr/ast"
)
var (
nilType = reflect.TypeOf(nil)
boolType = reflect.TypeOf(true)
integerType = reflect.TypeOf(int(0))
floatType = reflect.TypeOf(float64(0))
stringType = reflect.TypeOf("")
arrayType = reflect.TypeOf([]interface{}{})
mapType = reflect.TypeOf(map[string]interface{}{})
interfaceType = reflect.TypeOf(new(interface{})).Elem()
)
func typeWeight(t reflect.Type) int {
switch t.Kind() {
case reflect.Uint:
return 1
case reflect.Uint8:
return 2
case reflect.Uint16:
return 3
case reflect.Uint32:
return 4
case reflect.Uint64:
return 5
case reflect.Int:
return 6
case reflect.Int8:
return 7
case reflect.Int16:
return 8
case reflect.Int32:
return 9
case reflect.Int64:
return 10
case reflect.Float32:
return 11
case reflect.Float64:
return 12
default:
return 0
}
}
func combined(a, b reflect.Type) reflect.Type {
if typeWeight(a) > typeWeight(b) {
return a
} else {
return b
}
}
func dereference(t reflect.Type) reflect.Type {
if t == nil {
return nil
}
if t.Kind() == reflect.Ptr {
t = dereference(t.Elem())
}
return t
}
func isComparable(l, r reflect.Type) bool {
l = dereference(l)
r = dereference(r)
if l == nil || r == nil { // It is possible to compare with nil.
return true
}
if l.Kind() == r.Kind() {
return true
}
if isInterface(l) || isInterface(r) {
return true
}
return false
}
func isInterface(t reflect.Type) bool {
t = dereference(t)
if t != nil {
switch t.Kind() {
case reflect.Interface:
return true
}
}
return false
}
func isInteger(t reflect.Type) bool {
t = dereference(t)
if t != nil {
switch t.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
fallthrough
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return true
case reflect.Interface:
return true
}
}
return false
}
func isFloat(t reflect.Type) bool {
t = dereference(t)
if t != nil {
switch t.Kind() {
case reflect.Float32, reflect.Float64:
return true
case reflect.Interface:
return true
}
}
return false
}
func isNumber(t reflect.Type) bool {
return isInteger(t) || isFloat(t)
}
func isBool(t reflect.Type) bool {
t = dereference(t)
if t != nil {
switch t.Kind() {
case reflect.Bool:
return true
case reflect.Interface:
return true
}
}
return false
}
func isString(t reflect.Type) bool {
t = dereference(t)
if t != nil {
switch t.Kind() {
case reflect.String:
return true
case reflect.Interface:
return true
}
}
return false
}
func isArray(t reflect.Type) bool {
t = dereference(t)
if t != nil {
switch t.Kind() {
case reflect.Slice, reflect.Array:
return true
case reflect.Interface:
return true
}
}
return false
}
func isMap(t reflect.Type) bool {
t = dereference(t)
if t != nil {
switch t.Kind() {
case reflect.Map:
return true
case reflect.Interface:
return true
}
}
return false
}
func isStruct(t reflect.Type) bool {
t = dereference(t)
if t != nil {
switch t.Kind() {
case reflect.Struct:
return true
}
}
return false
}
func isFunc(t reflect.Type) bool {
t = dereference(t)
if t != nil {
switch t.Kind() {
case reflect.Func:
return true
}
}
return false
}
func fieldType(ntype reflect.Type, name string) (reflect.Type, bool) {
ntype = dereference(ntype)
if ntype != nil {
switch ntype.Kind() {
case reflect.Interface:
return interfaceType, true
case reflect.Struct:
// First check all struct's fields.
for i := 0; i < ntype.NumField(); i++ {
f := ntype.Field(i)
if f.Name == name {
return f.Type, true
}
}
// Second check fields of embedded structs.
for i := 0; i < ntype.NumField(); i++ {
f := ntype.Field(i)
if f.Anonymous {
if t, ok := fieldType(f.Type, name); ok {
return t, true
}
}
}
case reflect.Map:
return ntype.Elem(), true
}
}
return nil, false
}
func methodType(t reflect.Type, name string) (reflect.Type, bool, bool) {
if t != nil {
// First, check methods defined on type itself,
// independent of which type it is.
if m, ok := t.MethodByName(name); ok {
if t.Kind() == reflect.Interface {
// In case of interface type method will not have a receiver,
// and to prevent checker decreasing numbers of in arguments
// return method type as not method (second argument is false).
return m.Type, false, true
} else {
return m.Type, true, true
}
}
d := t
if t.Kind() == reflect.Ptr {
d = t.Elem()
}
switch d.Kind() {
case reflect.Interface:
return interfaceType, false, true
case reflect.Struct:
// First, check all struct's fields.
for i := 0; i < d.NumField(); i++ {
f := d.Field(i)
if !f.Anonymous && f.Name == name {
return f.Type, false, true
}
}
// Second, check fields of embedded structs.
for i := 0; i < d.NumField(); i++ {
f := d.Field(i)
if f.Anonymous {
if t, method, ok := methodType(f.Type, name); ok {
return t, method, true
}
}
}
case reflect.Map:
return d.Elem(), false, true
}
}
return nil, false, false
}
func indexType(ntype reflect.Type) (reflect.Type, bool) {
ntype = dereference(ntype)
if ntype == nil {
return nil, false
}
switch ntype.Kind() {
case reflect.Interface:
return interfaceType, true
case reflect.Map, reflect.Array, reflect.Slice:
return ntype.Elem(), true
}
return nil, false
}
func isFuncType(ntype reflect.Type) (reflect.Type, bool) {
ntype = dereference(ntype)
if ntype == nil {
return nil, false
}
switch ntype.Kind() {
case reflect.Interface:
return interfaceType, true
case reflect.Func:
return ntype, true
}
return nil, false
}
func isIntegerOrArithmeticOperation(node ast.Node) bool {
switch n := node.(type) {
case *ast.IntegerNode:
return true
case *ast.UnaryNode:
switch n.Operator {
case "+", "-":
return true
}
case *ast.BinaryNode:
switch n.Operator {
case "+", "/", "-", "*":
return true
}
}
return false
}
func setTypeForIntegers(node ast.Node, t reflect.Type) {
switch n := node.(type) {
case *ast.IntegerNode:
n.SetType(t)
case *ast.UnaryNode:
switch n.Operator {
case "+", "-":
setTypeForIntegers(n.Node, t)
}
case *ast.BinaryNode:
switch n.Operator {
case "+", "/", "-", "*":
setTypeForIntegers(n.Left, t)
setTypeForIntegers(n.Right, t)
}
}
}
+663
View File
@@ -0,0 +1,663 @@
package compiler
import (
"encoding/binary"
"fmt"
"math"
"reflect"
"github.com/antonmedv/expr/ast"
"github.com/antonmedv/expr/conf"
"github.com/antonmedv/expr/file"
"github.com/antonmedv/expr/parser"
. "github.com/antonmedv/expr/vm"
)
func Compile(tree *parser.Tree, config *conf.Config) (program *Program, err error) {
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("%v", r)
}
}()
c := &compiler{
index: make(map[interface{}]uint16),
locations: make(map[int]file.Location),
}
if config != nil {
c.mapEnv = config.MapEnv
c.cast = config.Expect
}
c.compile(tree.Node)
switch c.cast {
case reflect.Int64:
c.emit(OpCast, encode(0)...)
case reflect.Float64:
c.emit(OpCast, encode(1)...)
}
program = &Program{
Source: tree.Source,
Locations: c.locations,
Constants: c.constants,
Bytecode: c.bytecode,
}
return
}
type compiler struct {
locations map[int]file.Location
constants []interface{}
bytecode []byte
index map[interface{}]uint16
mapEnv bool
cast reflect.Kind
nodes []ast.Node
}
func (c *compiler) emit(op byte, b ...byte) int {
c.bytecode = append(c.bytecode, op)
current := len(c.bytecode)
c.bytecode = append(c.bytecode, b...)
var loc file.Location
if len(c.nodes) > 0 {
loc = c.nodes[len(c.nodes)-1].Location()
}
c.locations[current-1] = loc
return current
}
func (c *compiler) emitPush(value interface{}) int {
return c.emit(OpPush, c.makeConstant(value)...)
}
func (c *compiler) makeConstant(i interface{}) []byte {
hashable := true
switch reflect.TypeOf(i).Kind() {
case reflect.Slice, reflect.Map:
hashable = false
}
if hashable {
if p, ok := c.index[i]; ok {
return encode(p)
}
}
c.constants = append(c.constants, i)
if len(c.constants) > math.MaxUint16 {
panic("exceeded constants max space limit")
}
p := uint16(len(c.constants) - 1)
if hashable {
c.index[i] = p
}
return encode(p)
}
func (c *compiler) placeholder() []byte {
return []byte{0xFF, 0xFF}
}
func (c *compiler) patchJump(placeholder int) {
offset := len(c.bytecode) - 2 - placeholder
b := encode(uint16(offset))
c.bytecode[placeholder] = b[0]
c.bytecode[placeholder+1] = b[1]
}
func (c *compiler) calcBackwardJump(to int) []byte {
return encode(uint16(len(c.bytecode) + 1 + 2 - to))
}
func (c *compiler) compile(node ast.Node) {
c.nodes = append(c.nodes, node)
defer func() {
c.nodes = c.nodes[:len(c.nodes)-1]
}()
switch n := node.(type) {
case *ast.NilNode:
c.NilNode(n)
case *ast.IdentifierNode:
c.IdentifierNode(n)
case *ast.IntegerNode:
c.IntegerNode(n)
case *ast.FloatNode:
c.FloatNode(n)
case *ast.BoolNode:
c.BoolNode(n)
case *ast.StringNode:
c.StringNode(n)
case *ast.ConstantNode:
c.ConstantNode(n)
case *ast.UnaryNode:
c.UnaryNode(n)
case *ast.BinaryNode:
c.BinaryNode(n)
case *ast.MatchesNode:
c.MatchesNode(n)
case *ast.PropertyNode:
c.PropertyNode(n)
case *ast.IndexNode:
c.IndexNode(n)
case *ast.SliceNode:
c.SliceNode(n)
case *ast.MethodNode:
c.MethodNode(n)
case *ast.FunctionNode:
c.FunctionNode(n)
case *ast.BuiltinNode:
c.BuiltinNode(n)
case *ast.ClosureNode:
c.ClosureNode(n)
case *ast.PointerNode:
c.PointerNode(n)
case *ast.ConditionalNode:
c.ConditionalNode(n)
case *ast.ArrayNode:
c.ArrayNode(n)
case *ast.MapNode:
c.MapNode(n)
case *ast.PairNode:
c.PairNode(n)
default:
panic(fmt.Sprintf("undefined node type (%T)", node))
}
}
func (c *compiler) NilNode(node *ast.NilNode) {
c.emit(OpNil)
}
func (c *compiler) IdentifierNode(node *ast.IdentifierNode) {
v := c.makeConstant(node.Value)
if c.mapEnv {
c.emit(OpFetchMap, v...)
} else {
c.emit(OpFetch, v...)
}
}
func (c *compiler) IntegerNode(node *ast.IntegerNode) {
t := node.Type()
if t == nil {
c.emitPush(node.Value)
return
}
switch t.Kind() {
case reflect.Float32:
c.emitPush(float32(node.Value))
case reflect.Float64:
c.emitPush(float64(node.Value))
case reflect.Int:
c.emitPush(int(node.Value))
case reflect.Int8:
c.emitPush(int8(node.Value))
case reflect.Int16:
c.emitPush(int16(node.Value))
case reflect.Int32:
c.emitPush(int32(node.Value))
case reflect.Int64:
c.emitPush(int64(node.Value))
case reflect.Uint:
c.emitPush(uint(node.Value))
case reflect.Uint8:
c.emitPush(uint8(node.Value))
case reflect.Uint16:
c.emitPush(uint16(node.Value))
case reflect.Uint32:
c.emitPush(uint32(node.Value))
case reflect.Uint64:
c.emitPush(uint64(node.Value))
default:
c.emitPush(node.Value)
}
}
func (c *compiler) FloatNode(node *ast.FloatNode) {
c.emitPush(node.Value)
}
func (c *compiler) BoolNode(node *ast.BoolNode) {
if node.Value {
c.emit(OpTrue)
} else {
c.emit(OpFalse)
}
}
func (c *compiler) StringNode(node *ast.StringNode) {
c.emitPush(node.Value)
}
func (c *compiler) ConstantNode(node *ast.ConstantNode) {
c.emitPush(node.Value)
}
func (c *compiler) UnaryNode(node *ast.UnaryNode) {
c.compile(node.Node)
switch node.Operator {
case "!", "not":
c.emit(OpNot)
case "+":
// Do nothing
case "-":
c.emit(OpNegate)
default:
panic(fmt.Sprintf("unknown operator (%v)", node.Operator))
}
}
func (c *compiler) BinaryNode(node *ast.BinaryNode) {
l := kind(node.Left)
r := kind(node.Right)
switch node.Operator {
case "==":
c.compile(node.Left)
c.compile(node.Right)
if l == r && l == reflect.Int {
c.emit(OpEqualInt)
} else if l == r && l == reflect.String {
c.emit(OpEqualString)
} else {
c.emit(OpEqual)
}
case "!=":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpEqual)
c.emit(OpNot)
case "or", "||":
c.compile(node.Left)
end := c.emit(OpJumpIfTrue, c.placeholder()...)
c.emit(OpPop)
c.compile(node.Right)
c.patchJump(end)
case "and", "&&":
c.compile(node.Left)
end := c.emit(OpJumpIfFalse, c.placeholder()...)
c.emit(OpPop)
c.compile(node.Right)
c.patchJump(end)
case "in":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpIn)
case "not in":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpIn)
c.emit(OpNot)
case "<":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpLess)
case ">":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpMore)
case "<=":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpLessOrEqual)
case ">=":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpMoreOrEqual)
case "+":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpAdd)
case "-":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpSubtract)
case "*":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpMultiply)
case "/":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpDivide)
case "%":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpModulo)
case "**":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpExponent)
case "contains":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpContains)
case "startsWith":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpStartsWith)
case "endsWith":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpEndsWith)
case "..":
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpRange)
default:
panic(fmt.Sprintf("unknown operator (%v)", node.Operator))
}
}
func (c *compiler) MatchesNode(node *ast.MatchesNode) {
if node.Regexp != nil {
c.compile(node.Left)
c.emit(OpMatchesConst, c.makeConstant(node.Regexp)...)
return
}
c.compile(node.Left)
c.compile(node.Right)
c.emit(OpMatches)
}
func (c *compiler) PropertyNode(node *ast.PropertyNode) {
c.compile(node.Node)
c.emit(OpProperty, c.makeConstant(node.Property)...)
}
func (c *compiler) IndexNode(node *ast.IndexNode) {
c.compile(node.Node)
c.compile(node.Index)
c.emit(OpIndex)
}
func (c *compiler) SliceNode(node *ast.SliceNode) {
c.compile(node.Node)
if node.To != nil {
c.compile(node.To)
} else {
c.emit(OpLen)
}
if node.From != nil {
c.compile(node.From)
} else {
c.emitPush(0)
}
c.emit(OpSlice)
}
func (c *compiler) MethodNode(node *ast.MethodNode) {
c.compile(node.Node)
for _, arg := range node.Arguments {
c.compile(arg)
}
c.emit(OpMethod, c.makeConstant(Call{Name: node.Method, Size: len(node.Arguments)})...)
}
func (c *compiler) FunctionNode(node *ast.FunctionNode) {
for _, arg := range node.Arguments {
c.compile(arg)
}
op := OpCall
if node.Fast {
op = OpCallFast
}
c.emit(op, c.makeConstant(Call{Name: node.Name, Size: len(node.Arguments)})...)
}
func (c *compiler) BuiltinNode(node *ast.BuiltinNode) {
switch node.Name {
case "len":
c.compile(node.Arguments[0])
c.emit(OpLen)
c.emit(OpRot)
c.emit(OpPop)
case "all":
c.compile(node.Arguments[0])
c.emit(OpBegin)
var loopBreak int
c.emitLoop(func() {
c.compile(node.Arguments[1])
loopBreak = c.emit(OpJumpIfFalse, c.placeholder()...)
c.emit(OpPop)
})
c.emit(OpTrue)
c.patchJump(loopBreak)
c.emit(OpEnd)
case "none":
c.compile(node.Arguments[0])
c.emit(OpBegin)
var loopBreak int
c.emitLoop(func() {
c.compile(node.Arguments[1])
c.emit(OpNot)
loopBreak = c.emit(OpJumpIfFalse, c.placeholder()...)
c.emit(OpPop)
})
c.emit(OpTrue)
c.patchJump(loopBreak)
c.emit(OpEnd)
case "any":
c.compile(node.Arguments[0])
c.emit(OpBegin)
var loopBreak int
c.emitLoop(func() {
c.compile(node.Arguments[1])
loopBreak = c.emit(OpJumpIfTrue, c.placeholder()...)
c.emit(OpPop)
})
c.emit(OpFalse)
c.patchJump(loopBreak)
c.emit(OpEnd)
case "one":
count := c.makeConstant("count")
c.compile(node.Arguments[0])
c.emit(OpBegin)
c.emitPush(0)
c.emit(OpStore, count...)
c.emitLoop(func() {
c.compile(node.Arguments[1])
c.emitCond(func() {
c.emit(OpInc, count...)
})
})
c.emit(OpLoad, count...)
c.emitPush(1)
c.emit(OpEqual)
c.emit(OpEnd)
case "filter":
count := c.makeConstant("count")
c.compile(node.Arguments[0])
c.emit(OpBegin)
c.emitPush(0)
c.emit(OpStore, count...)
c.emitLoop(func() {
c.compile(node.Arguments[1])
c.emitCond(func() {
c.emit(OpInc, count...)
c.emit(OpLoad, c.makeConstant("array")...)
c.emit(OpLoad, c.makeConstant("i")...)
c.emit(OpIndex)
})
})
c.emit(OpLoad, count...)
c.emit(OpEnd)
c.emit(OpArray)
case "map":
c.compile(node.Arguments[0])
c.emit(OpBegin)
size := c.emitLoop(func() {
c.compile(node.Arguments[1])
})
c.emit(OpLoad, size...)
c.emit(OpEnd)
c.emit(OpArray)
case "count":
count := c.makeConstant("count")
c.compile(node.Arguments[0])
c.emit(OpBegin)
c.emitPush(0)
c.emit(OpStore, count...)
c.emitLoop(func() {
c.compile(node.Arguments[1])
c.emitCond(func() {
c.emit(OpInc, count...)
})
})
c.emit(OpLoad, count...)
c.emit(OpEnd)
default:
panic(fmt.Sprintf("unknown builtin %v", node.Name))
}
}
func (c *compiler) emitCond(body func()) {
noop := c.emit(OpJumpIfFalse, c.placeholder()...)
c.emit(OpPop)
body()
jmp := c.emit(OpJump, c.placeholder()...)
c.patchJump(noop)
c.emit(OpPop)
c.patchJump(jmp)
}
func (c *compiler) emitLoop(body func()) []byte {
i := c.makeConstant("i")
size := c.makeConstant("size")
array := c.makeConstant("array")
c.emit(OpLen)
c.emit(OpStore, size...)
c.emit(OpStore, array...)
c.emitPush(0)
c.emit(OpStore, i...)
cond := len(c.bytecode)
c.emit(OpLoad, i...)
c.emit(OpLoad, size...)
c.emit(OpLess)
end := c.emit(OpJumpIfFalse, c.placeholder()...)
c.emit(OpPop)
body()
c.emit(OpInc, i...)
c.emit(OpJumpBackward, c.calcBackwardJump(cond)...)
c.patchJump(end)
c.emit(OpPop)
return size
}
func (c *compiler) ClosureNode(node *ast.ClosureNode) {
c.compile(node.Node)
}
func (c *compiler) PointerNode(node *ast.PointerNode) {
c.emit(OpLoad, c.makeConstant("array")...)
c.emit(OpLoad, c.makeConstant("i")...)
c.emit(OpIndex)
}
func (c *compiler) ConditionalNode(node *ast.ConditionalNode) {
c.compile(node.Cond)
otherwise := c.emit(OpJumpIfFalse, c.placeholder()...)
c.emit(OpPop)
c.compile(node.Exp1)
end := c.emit(OpJump, c.placeholder()...)
c.patchJump(otherwise)
c.emit(OpPop)
c.compile(node.Exp2)
c.patchJump(end)
}
func (c *compiler) ArrayNode(node *ast.ArrayNode) {
for _, node := range node.Nodes {
c.compile(node)
}
c.emitPush(len(node.Nodes))
c.emit(OpArray)
}
func (c *compiler) MapNode(node *ast.MapNode) {
for _, pair := range node.Pairs {
c.compile(pair)
}
c.emitPush(len(node.Pairs))
c.emit(OpMap)
}
func (c *compiler) PairNode(node *ast.PairNode) {
c.compile(node.Key)
c.compile(node.Value)
}
func encode(i uint16) []byte {
b := make([]byte, 2)
binary.LittleEndian.PutUint16(b, i)
return b
}
func kind(node ast.Node) reflect.Kind {
t := node.Type()
if t == nil {
return reflect.Invalid
}
return t.Kind()
}
+44
View File
@@ -0,0 +1,44 @@
package compiler
import (
"github.com/antonmedv/expr/ast"
"github.com/antonmedv/expr/conf"
)
type operatorPatcher struct {
ops map[string][]string
types conf.TypesTable
}
func (p *operatorPatcher) Enter(node *ast.Node) {}
func (p *operatorPatcher) Exit(node *ast.Node) {
binaryNode, ok := (*node).(*ast.BinaryNode)
if !ok {
return
}
fns, ok := p.ops[binaryNode.Operator]
if !ok {
return
}
leftType := binaryNode.Left.Type()
rightType := binaryNode.Right.Type()
_, fn, ok := conf.FindSuitableOperatorOverload(fns, p.types, leftType, rightType)
if ok {
newNode := &ast.FunctionNode{
Name: fn,
Arguments: []ast.Node{binaryNode.Left, binaryNode.Right},
}
ast.Patch(node, newNode)
}
}
func PatchOperators(node *ast.Node, config *conf.Config) {
if len(config.Operators) == 0 {
return
}
patcher := &operatorPatcher{ops: config.Operators, types: config.Types}
ast.Walk(node, patcher)
}
+89
View File
@@ -0,0 +1,89 @@
package conf
import (
"fmt"
"reflect"
"github.com/antonmedv/expr/ast"
"github.com/antonmedv/expr/vm"
)
type Config struct {
Env interface{}
MapEnv bool
Types TypesTable
Operators OperatorsTable
Expect reflect.Kind
Optimize bool
Strict bool
DefaultType reflect.Type
ConstExprFns map[string]reflect.Value
Visitors []ast.Visitor
err error
}
func New(env interface{}) *Config {
var mapEnv bool
var mapValueType reflect.Type
if _, ok := env.(map[string]interface{}); ok {
mapEnv = true
} else {
if reflect.ValueOf(env).Kind() == reflect.Map {
mapValueType = reflect.TypeOf(env).Elem()
}
}
return &Config{
Env: env,
MapEnv: mapEnv,
Types: CreateTypesTable(env),
Optimize: true,
Strict: true,
DefaultType: mapValueType,
ConstExprFns: make(map[string]reflect.Value),
}
}
// Check validates the compiler configuration.
func (c *Config) Check() error {
// Check that all functions that define operator overloading
// exist in environment and have correct signatures.
for op, fns := range c.Operators {
for _, fn := range fns {
fnType, ok := c.Types[fn]
if !ok || fnType.Type.Kind() != reflect.Func {
return fmt.Errorf("function %s for %s operator does not exist in environment", fn, op)
}
requiredNumIn := 2
if fnType.Method {
requiredNumIn = 3 // As first argument of method is receiver.
}
if fnType.Type.NumIn() != requiredNumIn || fnType.Type.NumOut() != 1 {
return fmt.Errorf("function %s for %s operator does not have a correct signature", fn, op)
}
}
}
// Check that all ConstExprFns are functions.
for name, fn := range c.ConstExprFns {
if fn.Kind() != reflect.Func {
return fmt.Errorf("const expression %q must be a function", name)
}
}
return c.err
}
func (c *Config) ConstExpr(name string) {
if c.Env == nil {
c.Error(fmt.Errorf("no environment for const expression: %v", name))
return
}
c.ConstExprFns[name] = vm.FetchFn(c.Env, name)
}
func (c *Config) Error(err error) {
if c.err == nil {
c.err = err
}
}
+26
View File
@@ -0,0 +1,26 @@
package conf
import "reflect"
// OperatorsTable maps binary operators to corresponding list of functions.
// Functions should be provided in the environment to allow operator overloading.
type OperatorsTable map[string][]string
func FindSuitableOperatorOverload(fns []string, types TypesTable, l, r reflect.Type) (reflect.Type, string, bool) {
for _, fn := range fns {
fnType := types[fn]
firstInIndex := 0
if fnType.Method {
firstInIndex = 1 // As first argument to method is receiver.
}
firstArgType := fnType.Type.In(firstInIndex)
secondArgType := fnType.Type.In(firstInIndex + 1)
firstArgumentFit := l == firstArgType || (firstArgType.Kind() == reflect.Interface && (l == nil || l.Implements(firstArgType)))
secondArgumentFit := r == secondArgType || (secondArgType.Kind() == reflect.Interface && (r == nil || r.Implements(secondArgType)))
if firstArgumentFit && secondArgumentFit {
return fnType.Type.Out(0), fn, true
}
}
return nil, "", false
}
+100
View File
@@ -0,0 +1,100 @@
package conf
import "reflect"
type Tag struct {
Type reflect.Type
Method bool
Ambiguous bool
}
type TypesTable map[string]Tag
// CreateTypesTable creates types table for type checks during parsing.
// If struct is passed, all fields will be treated as variables,
// as well as all fields of embedded structs and struct itself.
//
// If map is passed, all items will be treated as variables
// (key as name, value as type).
func CreateTypesTable(i interface{}) TypesTable {
if i == nil {
return nil
}
types := make(TypesTable)
v := reflect.ValueOf(i)
t := reflect.TypeOf(i)
d := t
if t.Kind() == reflect.Ptr {
d = t.Elem()
}
switch d.Kind() {
case reflect.Struct:
types = FieldsFromStruct(d)
// Methods of struct should be gathered from original struct with pointer,
// as methods maybe declared on pointer receiver. Also this method retrieves
// all embedded structs methods as well, no need to recursion.
for i := 0; i < t.NumMethod(); i++ {
m := t.Method(i)
types[m.Name] = Tag{Type: m.Type, Method: true}
}
case reflect.Map:
for _, key := range v.MapKeys() {
value := v.MapIndex(key)
if key.Kind() == reflect.String && value.IsValid() && value.CanInterface() {
types[key.String()] = Tag{Type: reflect.TypeOf(value.Interface())}
}
}
// A map may have method too.
for i := 0; i < t.NumMethod(); i++ {
m := t.Method(i)
types[m.Name] = Tag{Type: m.Type, Method: true}
}
}
return types
}
func FieldsFromStruct(t reflect.Type) TypesTable {
types := make(TypesTable)
t = dereference(t)
if t == nil {
return types
}
switch t.Kind() {
case reflect.Struct:
for i := 0; i < t.NumField(); i++ {
f := t.Field(i)
if f.Anonymous {
for name, typ := range FieldsFromStruct(f.Type) {
if _, ok := types[name]; ok {
types[name] = Tag{Ambiguous: true}
} else {
types[name] = typ
}
}
}
types[f.Name] = Tag{Type: f.Type}
}
}
return types
}
func dereference(t reflect.Type) reflect.Type {
if t == nil {
return nil
}
if t.Kind() == reflect.Ptr {
t = dereference(t.Elem())
}
return t
}
+187
View File
@@ -0,0 +1,187 @@
package expr
import (
"fmt"
"github.com/antonmedv/expr/ast"
"github.com/antonmedv/expr/file"
"reflect"
"github.com/antonmedv/expr/checker"
"github.com/antonmedv/expr/compiler"
"github.com/antonmedv/expr/conf"
"github.com/antonmedv/expr/optimizer"
"github.com/antonmedv/expr/parser"
"github.com/antonmedv/expr/vm"
)
// Option for configuring config.
type Option func(c *conf.Config)
// Eval parses, compiles and runs given input.
func Eval(input string, env interface{}) (interface{}, error) {
if _, ok := env.(Option); ok {
return nil, fmt.Errorf("misused expr.Eval: second argument (env) should be passed without expr.Env")
}
tree, err := parser.Parse(input)
if err != nil {
return nil, err
}
program, err := compiler.Compile(tree, nil)
if err != nil {
return nil, err
}
output, err := vm.Run(program, env)
if err != nil {
return nil, err
}
return output, nil
}
// Env specifies expected input of env for type checks.
// If struct is passed, all fields will be treated as variables,
// as well as all fields of embedded structs and struct itself.
// If map is passed, all items will be treated as variables.
// Methods defined on this type will be available as functions.
func Env(env interface{}) Option {
return func(c *conf.Config) {
if _, ok := env.(map[string]interface{}); ok {
c.MapEnv = true
} else {
if reflect.ValueOf(env).Kind() == reflect.Map {
c.DefaultType = reflect.TypeOf(env).Elem()
}
}
c.Strict = true
c.Types = conf.CreateTypesTable(env)
c.Env = env
}
}
// AllowUndefinedVariables allows to use undefined variables inside expressions.
// This can be used with expr.Env option to partially define a few variables.
// Note what this option is only works in map environment are used, otherwise
// runtime.fetch will panic as there is no way to get missing field zero value.
func AllowUndefinedVariables() Option {
return func(c *conf.Config) {
c.Strict = false
}
}
// Operator allows to override binary operator with function.
func Operator(operator string, fn ...string) Option {
return func(c *conf.Config) {
c.Operators[operator] = append(c.Operators[operator], fn...)
}
}
// ConstExpr defines func expression as constant. If all argument to this function is constants,
// then it can be replaced by result of this func call on compile step.
func ConstExpr(fn string) Option {
return func(c *conf.Config) {
c.ConstExpr(fn)
}
}
// AsBool tells the compiler to expect boolean result.
func AsBool() Option {
return func(c *conf.Config) {
c.Expect = reflect.Bool
}
}
// AsInt64 tells the compiler to expect int64 result.
func AsInt64() Option {
return func(c *conf.Config) {
c.Expect = reflect.Int64
}
}
// AsFloat64 tells the compiler to expect float64 result.
func AsFloat64() Option {
return func(c *conf.Config) {
c.Expect = reflect.Float64
}
}
// Optimize turns optimizations on or off.
func Optimize(b bool) Option {
return func(c *conf.Config) {
c.Optimize = b
}
}
// Patch adds visitor to list of visitors what will be applied before compiling AST to bytecode.
func Patch(visitor ast.Visitor) Option {
return func(c *conf.Config) {
c.Visitors = append(c.Visitors, visitor)
}
}
// Compile parses and compiles given input expression to bytecode program.
func Compile(input string, ops ...Option) (*vm.Program, error) {
config := &conf.Config{
Operators: make(map[string][]string),
ConstExprFns: make(map[string]reflect.Value),
Optimize: true,
}
for _, op := range ops {
op(config)
}
if err := config.Check(); err != nil {
return nil, err
}
tree, err := parser.Parse(input)
if err != nil {
return nil, err
}
_, err = checker.Check(tree, config)
// If we have a patch to apply, it may fix out error and
// second type check is needed. Otherwise it is an error.
if err != nil && len(config.Visitors) == 0 {
return nil, err
}
// Patch operators before Optimize, as we may also mark it as ConstExpr.
compiler.PatchOperators(&tree.Node, config)
if len(config.Visitors) >= 0 {
for _, v := range config.Visitors {
ast.Walk(&tree.Node, v)
}
_, err = checker.Check(tree, config)
if err != nil {
return nil, err
}
}
if config.Optimize {
err = optimizer.Optimize(&tree.Node, config)
if err != nil {
if fileError, ok := err.(*file.Error); ok {
return nil, fileError.Bind(tree.Source)
}
return nil, err
}
}
program, err := compiler.Compile(tree, config)
if err != nil {
return nil, err
}
return program, nil
}
// Run evaluates given bytecode program.
func Run(program *vm.Program, env interface{}) (interface{}, error) {
return vm.Run(program, env)
}
+58
View File
@@ -0,0 +1,58 @@
package file
import (
"fmt"
"strings"
"unicode/utf8"
)
type Error struct {
Location
Message string
Snippet string
}
func (e *Error) Error() string {
return e.format()
}
func (e *Error) Bind(source *Source) *Error {
if snippet, found := source.Snippet(e.Location.Line); found {
snippet := strings.Replace(snippet, "\t", " ", -1)
srcLine := "\n | " + snippet
var bytes = []byte(snippet)
var indLine = "\n | "
for i := 0; i < e.Location.Column && len(bytes) > 0; i++ {
_, sz := utf8.DecodeRune(bytes)
bytes = bytes[sz:]
if sz > 1 {
goto noind
} else {
indLine += "."
}
}
if _, sz := utf8.DecodeRune(bytes); sz > 1 {
goto noind
} else {
indLine += "^"
}
srcLine += indLine
noind:
e.Snippet = srcLine
}
return e
}
func (e *Error) format() string {
if e.Location.Empty() {
return e.Message
}
return fmt.Sprintf(
"%s (%d:%d)%s",
e.Message,
e.Line,
e.Column+1, // add one to the 0-based column for display
e.Snippet,
)
}
+10
View File
@@ -0,0 +1,10 @@
package file
type Location struct {
Line int // The 1-based line of the location.
Column int // The 0-based column number of the location.
}
func (l Location) Empty() bool {
return l.Column == 0 && l.Line == 0
}
+95
View File
@@ -0,0 +1,95 @@
package file
import (
"encoding/json"
"strings"
"unicode/utf8"
)
type Source struct {
contents []rune
lineOffsets []int32
}
func NewSource(contents string) *Source {
s := &Source{
contents: []rune(contents),
}
s.updateOffsets()
return s
}
func (s *Source) MarshalJSON() ([]byte, error) {
return json.Marshal(s.contents)
}
func (s *Source) UnmarshalJSON(b []byte) error {
contents := make([]rune, 0)
err := json.Unmarshal(b, &contents)
if err != nil {
return err
}
s.contents = contents
s.updateOffsets()
return nil
}
func (s *Source) Content() string {
return string(s.contents)
}
func (s *Source) Snippet(line int) (string, bool) {
charStart, found := s.findLineOffset(line)
if !found || len(s.contents) == 0 {
return "", false
}
charEnd, found := s.findLineOffset(line + 1)
if found {
return string(s.contents[charStart : charEnd-1]), true
}
return string(s.contents[charStart:]), true
}
// updateOffsets compute line offsets up front as they are referred to frequently.
func (s *Source) updateOffsets() {
lines := strings.Split(string(s.contents), "\n")
offsets := make([]int32, len(lines))
var offset int32
for i, line := range lines {
offset = offset + int32(utf8.RuneCountInString(line)) + 1
offsets[int32(i)] = offset
}
s.lineOffsets = offsets
}
// findLineOffset returns the offset where the (1-indexed) line begins,
// or false if line doesn't exist.
func (s *Source) findLineOffset(line int) (int32, bool) {
if line == 1 {
return 0, true
} else if line > 1 && line <= len(s.lineOffsets) {
offset := s.lineOffsets[line-2]
return offset, true
}
return -1, false
}
// findLine finds the line that contains the given character offset and
// returns the line number and offset of the beginning of that line.
// Note that the last line is treated as if it contains all offsets
// beyond the end of the actual source.
func (s *Source) findLine(characterOffset int32) (int32, int32) {
var line int32 = 1
for _, lineOffset := range s.lineOffsets {
if lineOffset > characterOffset {
break
} else {
line++
}
}
if line == 1 {
return line, 0
}
return line, s.lineOffsets[line-2]
}
+10
View File
@@ -0,0 +1,10 @@
module github.com/antonmedv/expr
go 1.13
require (
github.com/gdamore/tcell v1.3.0
github.com/rivo/tview v0.0.0-20200219210816-cd38d7432498
github.com/sanity-io/litter v1.2.0
github.com/stretchr/testify v1.5.1
)
+37
View File
@@ -0,0 +1,37 @@
github.com/DATA-DOG/go-sqlmock v1.3.3/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM=
github.com/davecgh/go-spew v0.0.0-20161028175848-04cdfd42973b/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/gdamore/encoding v1.0.0 h1:+7OoQ1Bc6eTm5niUzBa0Ctsh6JbMW6Ra+YNuAtDBdko=
github.com/gdamore/encoding v1.0.0/go.mod h1:alR0ol34c49FCSBLjhosxzcPHQbf2trDkoo5dl+VrEg=
github.com/gdamore/tcell v1.3.0 h1:r35w0JBADPZCVQijYebl6YMWWtHRqVEGt7kL2eBADRM=
github.com/gdamore/tcell v1.3.0/go.mod h1:Hjvr+Ofd+gLglo7RYKxxnzCBmev3BzsS67MebKS4zMM=
github.com/lucasb-eyer/go-colorful v1.0.2/go.mod h1:0MS4r+7BZKSJ5mw4/S5MPN+qHFF1fYclkSPilDOKW0s=
github.com/lucasb-eyer/go-colorful v1.0.3 h1:QIbQXiugsb+q10B+MI+7DI1oQLdmnep86tWFlaaUAac=
github.com/lucasb-eyer/go-colorful v1.0.3/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-runewidth v0.0.8 h1:3tS41NlGYSmhhe/8fhGRzc+z3AYCw1Fe1WAyLuujKs0=
github.com/mattn/go-runewidth v0.0.8/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/pmezard/go-difflib v0.0.0-20151028094244-d8ed2627bdf0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rivo/tview v0.0.0-20200219210816-cd38d7432498 h1:4CFNy7/q7P06AsIONZzuWy7jcdqEmYQvOZ9FAFZdbls=
github.com/rivo/tview v0.0.0-20200219210816-cd38d7432498/go.mod h1:6lkG1x+13OShEf0EaOCaTQYyB7d5nSbb181KtjlS+84=
github.com/rivo/uniseg v0.1.0 h1:+2KBaVoUmb9XzDsrx/Ct0W/EYOSFf/nWTauy++DprtY=
github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/sanity-io/litter v1.2.0 h1:DGJO0bxH/+C2EukzOSBmAlxmkhVMGqzvcx/rvySYw9M=
github.com/sanity-io/litter v1.2.0/go.mod h1:JF6pZUFgu2Q0sBZ+HSV35P8TVPI1TTzEwyu9FXAw2W4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v0.0.0-20161117074351-18a02ba4a312/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
golang.org/x/sys v0.0.0-20190626150813-e07cf5db2756/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4 h1:sfkvUWPNGwSV+8/fNqctR5lS2AqCSqYwXdrjCxp/dXo=
golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+77
View File
@@ -0,0 +1,77 @@
package optimizer
import (
"fmt"
. "github.com/antonmedv/expr/ast"
"github.com/antonmedv/expr/file"
"reflect"
"strings"
)
type constExpr struct {
applied bool
err error
fns map[string]reflect.Value
}
func (*constExpr) Enter(*Node) {}
func (c *constExpr) Exit(node *Node) {
defer func() {
if r := recover(); r != nil {
msg := fmt.Sprintf("%v", r)
// Make message more actual, it's a runtime error, but at compile step.
msg = strings.Replace(msg, "runtime error:", "compile error:", 1)
c.err = &file.Error{
Location: (*node).Location(),
Message: msg,
}
}
}()
patch := func(newNode Node) {
c.applied = true
Patch(node, newNode)
}
switch n := (*node).(type) {
case *FunctionNode:
fn, ok := c.fns[n.Name]
if ok {
in := make([]reflect.Value, len(n.Arguments))
for i := 0; i < len(n.Arguments); i++ {
arg := n.Arguments[i]
var param interface{}
switch a := arg.(type) {
case *NilNode:
param = nil
case *IntegerNode:
param = a.Value
case *FloatNode:
param = a.Value
case *BoolNode:
param = a.Value
case *StringNode:
param = a.Value
case *ConstantNode:
param = a.Value
default:
return // Const expr optimization not applicable.
}
if param == nil && reflect.TypeOf(param) == nil {
// In case of nil value and nil type use this hack,
// otherwise reflect.Call will panic on zero value.
in[i] = reflect.ValueOf(&param).Elem()
} else {
in[i] = reflect.ValueOf(param)
}
}
out := fn.Call(in)
constNode := &ConstantNode{Value: out[0].Interface()}
patch(constNode)
}
}
}
+33
View File
@@ -0,0 +1,33 @@
package optimizer
import (
. "github.com/antonmedv/expr/ast"
)
type constRange struct{}
func (*constRange) Enter(*Node) {}
func (*constRange) Exit(node *Node) {
switch n := (*node).(type) {
case *BinaryNode:
if n.Operator == ".." {
if min, ok := n.Left.(*IntegerNode); ok {
if max, ok := n.Right.(*IntegerNode); ok {
size := max.Value - min.Value + 1
// In this case array is too big. Skip generation,
// and wait for memory budget detection on runtime.
if size > 1e6 {
return
}
value := make([]int, size)
for i := range value {
value[i] = min.Value + i
}
Patch(node, &ConstantNode{
Value: value,
})
}
}
}
}
}
+133
View File
@@ -0,0 +1,133 @@
package optimizer
import (
"math"
"reflect"
. "github.com/antonmedv/expr/ast"
"github.com/antonmedv/expr/file"
)
type fold struct {
applied bool
err *file.Error
}
func (*fold) Enter(*Node) {}
func (fold *fold) Exit(node *Node) {
patch := func(newNode Node) {
fold.applied = true
Patch(node, newNode)
}
// for IntegerNode the type may have been changed from int->float
// preserve this information by setting the type after the Patch
patchWithType := func(newNode Node, leafType reflect.Type) {
patch(newNode)
newNode.SetType(leafType)
}
switch n := (*node).(type) {
case *UnaryNode:
switch n.Operator {
case "-":
if i, ok := n.Node.(*IntegerNode); ok {
patchWithType(&IntegerNode{Value: -i.Value}, n.Node.Type())
}
case "+":
if i, ok := n.Node.(*IntegerNode); ok {
patchWithType(&IntegerNode{Value: i.Value}, n.Node.Type())
}
}
case *BinaryNode:
switch n.Operator {
case "+":
if a, ok := n.Left.(*IntegerNode); ok {
if b, ok := n.Right.(*IntegerNode); ok {
patchWithType(&IntegerNode{Value: a.Value + b.Value}, a.Type())
}
}
if a, ok := n.Left.(*StringNode); ok {
if b, ok := n.Right.(*StringNode); ok {
patch(&StringNode{Value: a.Value + b.Value})
}
}
case "-":
if a, ok := n.Left.(*IntegerNode); ok {
if b, ok := n.Right.(*IntegerNode); ok {
patchWithType(&IntegerNode{Value: a.Value - b.Value}, a.Type())
}
}
case "*":
if a, ok := n.Left.(*IntegerNode); ok {
if b, ok := n.Right.(*IntegerNode); ok {
patchWithType(&IntegerNode{Value: a.Value * b.Value}, a.Type())
}
}
case "/":
if a, ok := n.Left.(*IntegerNode); ok {
if b, ok := n.Right.(*IntegerNode); ok {
if b.Value == 0 {
fold.err = &file.Error{
Location: (*node).Location(),
Message: "integer divide by zero",
}
return
}
patchWithType(&IntegerNode{Value: a.Value / b.Value}, a.Type())
}
}
case "%":
if a, ok := n.Left.(*IntegerNode); ok {
if b, ok := n.Right.(*IntegerNode); ok {
if b.Value == 0 {
fold.err = &file.Error{
Location: (*node).Location(),
Message: "integer divide by zero",
}
return
}
patch(&IntegerNode{Value: a.Value % b.Value})
}
}
case "**":
if a, ok := n.Left.(*IntegerNode); ok {
if b, ok := n.Right.(*IntegerNode); ok {
patch(&FloatNode{Value: math.Pow(float64(a.Value), float64(b.Value))})
}
}
}
case *ArrayNode:
if len(n.Nodes) > 0 {
for _, a := range n.Nodes {
if _, ok := a.(*IntegerNode); !ok {
goto string
}
}
{
value := make([]int, len(n.Nodes))
for i, a := range n.Nodes {
value[i] = a.(*IntegerNode).Value
}
patch(&ConstantNode{Value: value})
}
string:
for _, a := range n.Nodes {
if _, ok := a.(*StringNode); !ok {
return
}
}
{
value := make([]string, len(n.Nodes))
for i, a := range n.Nodes {
value[i] = a.(*StringNode).Value
}
patch(&ConstantNode{Value: value})
}
}
}
}
+65
View File
@@ -0,0 +1,65 @@
package optimizer
import (
"reflect"
. "github.com/antonmedv/expr/ast"
)
type inArray struct{}
func (*inArray) Enter(*Node) {}
func (*inArray) Exit(node *Node) {
switch n := (*node).(type) {
case *BinaryNode:
if n.Operator == "in" || n.Operator == "not in" {
if array, ok := n.Right.(*ArrayNode); ok {
if len(array.Nodes) > 0 {
t := n.Left.Type()
if t == nil || t.Kind() != reflect.Int {
// This optimization can be only performed if left side is int type,
// as runtime.in func uses reflect.Map.MapIndex and keys of map must,
// be same as checked value type.
goto string
}
for _, a := range array.Nodes {
if _, ok := a.(*IntegerNode); !ok {
goto string
}
}
{
value := make(map[int]struct{})
for _, a := range array.Nodes {
value[a.(*IntegerNode).Value] = struct{}{}
}
Patch(node, &BinaryNode{
Operator: n.Operator,
Left: n.Left,
Right: &ConstantNode{Value: value},
})
}
string:
for _, a := range array.Nodes {
if _, ok := a.(*StringNode); !ok {
return
}
}
{
value := make(map[string]struct{})
for _, a := range array.Nodes {
value[a.(*StringNode).Value] = struct{}{}
}
Patch(node, &BinaryNode{
Operator: n.Operator,
Left: n.Left,
Right: &ConstantNode{Value: value},
})
}
}
}
}
}
}
+41
View File
@@ -0,0 +1,41 @@
package optimizer
import (
. "github.com/antonmedv/expr/ast"
)
type inRange struct{}
func (*inRange) Enter(*Node) {}
func (*inRange) Exit(node *Node) {
switch n := (*node).(type) {
case *BinaryNode:
if n.Operator == "in" || n.Operator == "not in" {
if rng, ok := n.Right.(*BinaryNode); ok && rng.Operator == ".." {
if from, ok := rng.Left.(*IntegerNode); ok {
if to, ok := rng.Right.(*IntegerNode); ok {
Patch(node, &BinaryNode{
Operator: "and",
Left: &BinaryNode{
Operator: ">=",
Left: n.Left,
Right: from,
},
Right: &BinaryNode{
Operator: "<=",
Left: n.Left,
Right: to,
},
})
if n.Operator == "not in" {
Patch(node, &UnaryNode{
Operator: "not",
Node: *node,
})
}
}
}
}
}
}
}
+37
View File
@@ -0,0 +1,37 @@
package optimizer
import (
. "github.com/antonmedv/expr/ast"
"github.com/antonmedv/expr/conf"
)
func Optimize(node *Node, config *conf.Config) error {
Walk(node, &inArray{})
for limit := 1000; limit >= 0; limit-- {
fold := &fold{}
Walk(node, fold)
if fold.err != nil {
return fold.err
}
if !fold.applied {
break
}
}
if config != nil && len(config.ConstExprFns) > 0 {
for limit := 100; limit >= 0; limit-- {
constExpr := &constExpr{
fns: config.ConstExprFns,
}
Walk(node, constExpr)
if constExpr.err != nil {
return constExpr.err
}
if !constExpr.applied {
break
}
}
}
Walk(node, &inRange{})
Walk(node, &constRange{})
return nil
}
+205
View File
@@ -0,0 +1,205 @@
package lexer
import (
"fmt"
"strings"
"unicode/utf8"
"github.com/antonmedv/expr/file"
)
func Lex(source *file.Source) ([]Token, error) {
l := &lexer{
input: source.Content(),
tokens: make([]Token, 0),
}
l.loc = file.Location{1, 0}
l.prev = l.loc
l.startLoc = l.loc
for state := root; state != nil; {
state = state(l)
}
if l.err != nil {
return nil, l.err.Bind(source)
}
return l.tokens, nil
}
type lexer struct {
input string
state stateFn
tokens []Token
start, end int // current position in input
width int // last rune with
startLoc file.Location // start location
prev, loc file.Location // prev location of end location, end location
err *file.Error
}
const eof rune = -1
func (l *lexer) next() rune {
if l.end >= len(l.input) {
l.width = 0
return eof
}
r, w := utf8.DecodeRuneInString(l.input[l.end:])
l.width = w
l.end += w
l.prev = l.loc
if r == '\n' {
l.loc.Line++
l.loc.Column = 0
} else {
l.loc.Column++
}
return r
}
func (l *lexer) peek() rune {
r := l.next()
l.backup()
return r
}
func (l *lexer) backup() {
l.end -= l.width
l.loc = l.prev
}
func (l *lexer) emit(t Kind) {
l.emitValue(t, l.word())
}
func (l *lexer) emitValue(t Kind, value string) {
l.tokens = append(l.tokens, Token{
Location: l.startLoc,
Kind: t,
Value: value,
})
l.start = l.end
l.startLoc = l.loc
}
func (l *lexer) emitEOF() {
l.tokens = append(l.tokens, Token{
Location: l.prev, // Point to previous position for better error messages.
Kind: EOF,
})
l.start = l.end
l.startLoc = l.loc
}
func (l *lexer) word() string {
return l.input[l.start:l.end]
}
func (l *lexer) ignore() {
l.start = l.end
l.startLoc = l.loc
}
func (l *lexer) accept(valid string) bool {
if strings.ContainsRune(valid, l.next()) {
return true
}
l.backup()
return false
}
func (l *lexer) acceptRun(valid string) {
for strings.ContainsRune(valid, l.next()) {
}
l.backup()
}
func (l *lexer) acceptWord(word string) bool {
pos := l.end
loc := l.loc
prev := l.prev
for _, ch := range word {
if l.next() != ch {
l.end = pos
l.loc = loc
l.prev = prev
return false
}
}
return true
}
func (l *lexer) error(format string, args ...interface{}) stateFn {
if l.err == nil { // show first error
l.err = &file.Error{
Location: l.loc,
Message: fmt.Sprintf(format, args...),
}
}
return nil
}
func digitVal(ch rune) int {
switch {
case '0' <= ch && ch <= '9':
return int(ch - '0')
case 'a' <= lower(ch) && lower(ch) <= 'f':
return int(lower(ch) - 'a' + 10)
}
return 16 // larger than any legal digit val
}
func lower(ch rune) rune { return ('a' - 'A') | ch } // returns lower-case ch iff ch is ASCII letter
func (l *lexer) scanDigits(ch rune, base, n int) rune {
for n > 0 && digitVal(ch) < base {
ch = l.next()
n--
}
if n > 0 {
l.error("invalid char escape")
}
return ch
}
func (l *lexer) scanEscape(quote rune) rune {
ch := l.next() // read character after '/'
switch ch {
case 'a', 'b', 'f', 'n', 'r', 't', 'v', '\\', quote:
// nothing to do
ch = l.next()
case '0', '1', '2', '3', '4', '5', '6', '7':
ch = l.scanDigits(ch, 8, 3)
case 'x':
ch = l.scanDigits(l.next(), 16, 2)
case 'u':
ch = l.scanDigits(l.next(), 16, 4)
case 'U':
ch = l.scanDigits(l.next(), 16, 8)
default:
l.error("invalid char escape")
}
return ch
}
func (l *lexer) scanString(quote rune) (n int) {
ch := l.next() // read character after quote
for ch != quote {
if ch == '\n' || ch == eof {
l.error("literal not terminated")
return
}
if ch == '\\' {
ch = l.scanEscape(quote)
} else {
ch = l.next()
}
n++
}
return
}
+134
View File
@@ -0,0 +1,134 @@
package lexer
import (
"strings"
)
type stateFn func(*lexer) stateFn
func root(l *lexer) stateFn {
switch r := l.next(); {
case r == eof:
l.emitEOF()
return nil
case IsSpace(r):
l.ignore()
return root
case r == '\'' || r == '"':
l.scanString(r)
str, err := unescape(l.word())
if err != nil {
l.error("%v", err)
}
l.emitValue(String, str)
case '0' <= r && r <= '9':
l.backup()
return number
case strings.ContainsRune("([{", r):
l.emit(Bracket)
case strings.ContainsRune(")]}", r):
l.emit(Bracket)
case strings.ContainsRune("#,?:%+-/", r): // single rune operator
l.emit(Operator)
case strings.ContainsRune("&|!=*<>", r): // possible double rune operator
l.accept("&|=*")
l.emit(Operator)
case r == '.':
l.backup()
return dot
case IsAlphaNumeric(r):
l.backup()
return identifier
default:
return l.error("unrecognized character: %#U", r)
}
return root
}
func number(l *lexer) stateFn {
if !l.scanNumber() {
return l.error("bad number syntax: %q", l.word())
}
l.emit(Number)
return root
}
func (l *lexer) scanNumber() bool {
digits := "0123456789_"
// Is it hex?
if l.accept("0") {
// Note: Leading 0 does not mean octal in floats.
if l.accept("xX") {
digits = "0123456789abcdefABCDEF_"
} else if l.accept("oO") {
digits = "01234567_"
} else if l.accept("bB") {
digits = "01_"
}
}
l.acceptRun(digits)
loc, prev, end := l.loc, l.prev, l.end
if l.accept(".") {
// Lookup for .. operator: if after dot there is another dot (1..2), it maybe a range operator.
if l.peek() == '.' {
// We can't backup() here, as it would require two backups,
// and backup() func supports only one for now. So, save and
// restore it here.
l.loc, l.prev, l.end = loc, prev, end
return true
}
l.acceptRun(digits)
}
if l.accept("eE") {
l.accept("+-")
l.acceptRun(digits)
}
// Next thing mustn't be alphanumeric.
if IsAlphaNumeric(l.peek()) {
l.next()
return false
}
return true
}
func dot(l *lexer) stateFn {
l.next()
if l.accept("0123456789") {
l.backup()
return number
}
l.accept(".")
l.emit(Operator)
return root
}
func identifier(l *lexer) stateFn {
loop:
for {
switch r := l.next(); {
case IsAlphaNumeric(r):
// absorb
default:
l.backup()
switch l.word() {
case "not":
return not
case "in", "or", "and", "matches", "contains", "startsWith", "endsWith":
l.emit(Operator)
default:
l.emit(Identifier)
}
break loop
}
}
return root
}
func not(l *lexer) stateFn {
if l.acceptWord(" in") {
l.emit(Operator)
} else {
l.emit(Operator)
}
return root
}
+47
View File
@@ -0,0 +1,47 @@
package lexer
import (
"fmt"
"github.com/antonmedv/expr/file"
)
type Kind string
const (
Identifier Kind = "Identifier"
Number = "Number"
String = "String"
Operator = "Operator"
Bracket = "Bracket"
EOF = "EOF"
)
type Token struct {
file.Location
Kind Kind
Value string
}
func (t Token) String() string {
if t.Value == "" {
return string(t.Kind)
}
return fmt.Sprintf("%s(%#v)", t.Kind, t.Value)
}
func (t Token) Is(kind Kind, values ...string) bool {
if len(values) == 0 {
return kind == t.Kind
}
for _, v := range values {
if v == t.Value {
goto found
}
}
return false
found:
return kind == t.Kind
}
+194
View File
@@ -0,0 +1,194 @@
package lexer
import (
"fmt"
"strings"
"unicode"
"unicode/utf8"
)
func IsSpace(r rune) bool {
return unicode.IsSpace(r)
}
func IsAlphaNumeric(r rune) bool {
return IsAlphabetic(r) || unicode.IsDigit(r)
}
func IsAlphabetic(r rune) bool {
return r == '_' || r == '$' || unicode.IsLetter(r)
}
var (
newlineNormalizer = strings.NewReplacer("\r\n", "\n", "\r", "\n")
)
// Unescape takes a quoted string, unquotes, and unescapes it.
func unescape(value string) (string, error) {
// All strings normalize newlines to the \n representation.
value = newlineNormalizer.Replace(value)
n := len(value)
// Nothing to unescape / decode.
if n < 2 {
return value, fmt.Errorf("unable to unescape string")
}
// Quoted string of some form, must have same first and last char.
if value[0] != value[n-1] || (value[0] != '"' && value[0] != '\'') {
return value, fmt.Errorf("unable to unescape string")
}
value = value[1 : n-1]
// The string contains escape characters.
// The following logic is adapted from `strconv/quote.go`
var runeTmp [utf8.UTFMax]byte
buf := make([]byte, 0, 3*n/2)
for len(value) > 0 {
c, multibyte, rest, err := unescapeChar(value)
if err != nil {
return "", err
}
value = rest
if c < utf8.RuneSelf || !multibyte {
buf = append(buf, byte(c))
} else {
n := utf8.EncodeRune(runeTmp[:], c)
buf = append(buf, runeTmp[:n]...)
}
}
return string(buf), nil
}
// unescapeChar takes a string input and returns the following info:
//
// value - the escaped unicode rune at the front of the string.
// multibyte - whether the rune value might require multiple bytes to represent.
// tail - the remainder of the input string.
// err - error value, if the character could not be unescaped.
//
// When multibyte is true the return value may still fit within a single byte,
// but a multibyte conversion is attempted which is more expensive than when the
// value is known to fit within one byte.
func unescapeChar(s string) (value rune, multibyte bool, tail string, err error) {
// 1. Character is not an escape sequence.
switch c := s[0]; {
case c >= utf8.RuneSelf:
r, size := utf8.DecodeRuneInString(s)
return r, true, s[size:], nil
case c != '\\':
return rune(s[0]), false, s[1:], nil
}
// 2. Last character is the start of an escape sequence.
if len(s) <= 1 {
err = fmt.Errorf("unable to unescape string, found '\\' as last character")
return
}
c := s[1]
s = s[2:]
// 3. Common escape sequences shared with Google SQL
switch c {
case 'a':
value = '\a'
case 'b':
value = '\b'
case 'f':
value = '\f'
case 'n':
value = '\n'
case 'r':
value = '\r'
case 't':
value = '\t'
case 'v':
value = '\v'
case '\\':
value = '\\'
case '\'':
value = '\''
case '"':
value = '"'
case '`':
value = '`'
case '?':
value = '?'
// 4. Unicode escape sequences, reproduced from `strconv/quote.go`
case 'x', 'X', 'u', 'U':
n := 0
switch c {
case 'x', 'X':
n = 2
case 'u':
n = 4
case 'U':
n = 8
}
var v rune
if len(s) < n {
err = fmt.Errorf("unable to unescape string")
return
}
for j := 0; j < n; j++ {
x, ok := unhex(s[j])
if !ok {
err = fmt.Errorf("unable to unescape string")
return
}
v = v<<4 | x
}
s = s[n:]
if v > utf8.MaxRune {
err = fmt.Errorf("unable to unescape string")
return
}
value = v
multibyte = true
// 5. Octal escape sequences, must be three digits \[0-3][0-7][0-7]
case '0', '1', '2', '3':
if len(s) < 2 {
err = fmt.Errorf("unable to unescape octal sequence in string")
return
}
v := rune(c - '0')
for j := 0; j < 2; j++ {
x := s[j]
if x < '0' || x > '7' {
err = fmt.Errorf("unable to unescape octal sequence in string")
return
}
v = v*8 + rune(x-'0')
}
if v > utf8.MaxRune {
err = fmt.Errorf("unable to unescape string")
return
}
value = v
s = s[2:]
multibyte = true
// Unknown escape sequence.
default:
err = fmt.Errorf("unable to unescape string")
}
tail = s
return
}
func unhex(b byte) (rune, bool) {
c := rune(b)
switch {
case '0' <= c && c <= '9':
return c - '0', true
case 'a' <= c && c <= 'f':
return c - 'a' + 10, true
case 'A' <= c && c <= 'F':
return c - 'A' + 10, true
}
return 0, false
}
+579
View File
@@ -0,0 +1,579 @@
package parser
import (
"fmt"
"regexp"
"strconv"
"strings"
"unicode/utf8"
. "github.com/antonmedv/expr/ast"
"github.com/antonmedv/expr/file"
. "github.com/antonmedv/expr/parser/lexer"
)
type associativity int
const (
left associativity = iota + 1
right
)
type operator struct {
precedence int
associativity associativity
}
type builtin struct {
arity int
}
var unaryOperators = map[string]operator{
"not": {50, left},
"!": {50, left},
"-": {500, left},
"+": {500, left},
}
var binaryOperators = map[string]operator{
"or": {10, left},
"||": {10, left},
"and": {15, left},
"&&": {15, left},
"==": {20, left},
"!=": {20, left},
"<": {20, left},
">": {20, left},
">=": {20, left},
"<=": {20, left},
"not in": {20, left},
"in": {20, left},
"matches": {20, left},
"contains": {20, left},
"startsWith": {20, left},
"endsWith": {20, left},
"..": {25, left},
"+": {30, left},
"-": {30, left},
"*": {60, left},
"/": {60, left},
"%": {60, left},
"**": {70, right},
}
var builtins = map[string]builtin{
"len": {1},
"all": {2},
"none": {2},
"any": {2},
"one": {2},
"filter": {2},
"map": {2},
"count": {2},
}
type parser struct {
tokens []Token
current Token
pos int
err *file.Error
depth int // closure call depth
}
type Tree struct {
Node Node
Source *file.Source
}
func Parse(input string) (*Tree, error) {
source := file.NewSource(input)
tokens, err := Lex(source)
if err != nil {
return nil, err
}
p := &parser{
tokens: tokens,
current: tokens[0],
}
node := p.parseExpression(0)
if !p.current.Is(EOF) {
p.error("unexpected token %v", p.current)
}
if p.err != nil {
return nil, p.err.Bind(source)
}
return &Tree{
Node: node,
Source: source,
}, nil
}
func (p *parser) error(format string, args ...interface{}) {
if p.err == nil { // show first error
p.err = &file.Error{
Location: p.current.Location,
Message: fmt.Sprintf(format, args...),
}
}
}
func (p *parser) next() {
p.pos++
if p.pos >= len(p.tokens) {
p.error("unexpected end of expression")
return
}
p.current = p.tokens[p.pos]
}
func (p *parser) expect(kind Kind, values ...string) {
if p.current.Is(kind, values...) {
p.next()
return
}
p.error("unexpected token %v", p.current)
}
// parse functions
func (p *parser) parseExpression(precedence int) Node {
nodeLeft := p.parsePrimary()
token := p.current
for token.Is(Operator) && p.err == nil {
if op, ok := binaryOperators[token.Value]; ok {
if op.precedence >= precedence {
p.next()
var nodeRight Node
if op.associativity == left {
nodeRight = p.parseExpression(op.precedence + 1)
} else {
nodeRight = p.parseExpression(op.precedence)
}
if token.Is(Operator, "matches") {
var r *regexp.Regexp
var err error
if s, ok := nodeRight.(*StringNode); ok {
r, err = regexp.Compile(s.Value)
if err != nil {
p.error("%v", err)
}
}
nodeLeft = &MatchesNode{
Regexp: r,
Left: nodeLeft,
Right: nodeRight,
}
nodeLeft.SetLocation(token.Location)
} else {
nodeLeft = &BinaryNode{
Operator: token.Value,
Left: nodeLeft,
Right: nodeRight,
}
nodeLeft.SetLocation(token.Location)
}
token = p.current
continue
}
}
break
}
if precedence == 0 {
nodeLeft = p.parseConditionalExpression(nodeLeft)
}
return nodeLeft
}
func (p *parser) parsePrimary() Node {
token := p.current
if token.Is(Operator) {
if op, ok := unaryOperators[token.Value]; ok {
p.next()
expr := p.parseExpression(op.precedence)
node := &UnaryNode{
Operator: token.Value,
Node: expr,
}
node.SetLocation(token.Location)
return p.parsePostfixExpression(node)
}
}
if token.Is(Bracket, "(") {
p.next()
expr := p.parseExpression(0)
p.expect(Bracket, ")") // "an opened parenthesis is not properly closed"
return p.parsePostfixExpression(expr)
}
if p.depth > 0 {
if token.Is(Operator, "#") || token.Is(Operator, ".") {
if token.Is(Operator, "#") {
p.next()
}
node := &PointerNode{}
node.SetLocation(token.Location)
return p.parsePostfixExpression(node)
}
} else {
if token.Is(Operator, "#") || token.Is(Operator, ".") {
p.error("cannot use pointer accessor outside closure")
}
}
return p.parsePrimaryExpression()
}
func (p *parser) parseConditionalExpression(node Node) Node {
var expr1, expr2 Node
for p.current.Is(Operator, "?") && p.err == nil {
p.next()
if !p.current.Is(Operator, ":") {
expr1 = p.parseExpression(0)
p.expect(Operator, ":")
expr2 = p.parseExpression(0)
} else {
p.next()
expr1 = node
expr2 = p.parseExpression(0)
}
node = &ConditionalNode{
Cond: node,
Exp1: expr1,
Exp2: expr2,
}
}
return node
}
func (p *parser) parsePrimaryExpression() Node {
var node Node
token := p.current
switch token.Kind {
case Identifier:
p.next()
switch token.Value {
case "true":
node := &BoolNode{Value: true}
node.SetLocation(token.Location)
return node
case "false":
node := &BoolNode{Value: false}
node.SetLocation(token.Location)
return node
case "nil":
node := &NilNode{}
node.SetLocation(token.Location)
return node
default:
node = p.parseIdentifierExpression(token)
}
case Number:
p.next()
value := strings.Replace(token.Value, "_", "", -1)
if strings.ContainsAny(value, ".eE") {
number, err := strconv.ParseFloat(value, 64)
if err != nil {
p.error("invalid float literal: %v", err)
}
node := &FloatNode{Value: number}
node.SetLocation(token.Location)
return node
} else if strings.Contains(value, "x") {
number, err := strconv.ParseInt(value, 0, 64)
if err != nil {
p.error("invalid hex literal: %v", err)
}
node := &IntegerNode{Value: int(number)}
node.SetLocation(token.Location)
return node
} else {
number, err := strconv.ParseInt(value, 10, 64)
if err != nil {
p.error("invalid integer literal: %v", err)
}
node := &IntegerNode{Value: int(number)}
node.SetLocation(token.Location)
return node
}
case String:
p.next()
node := &StringNode{Value: token.Value}
node.SetLocation(token.Location)
return node
default:
if token.Is(Bracket, "[") {
node = p.parseArrayExpression(token)
} else if token.Is(Bracket, "{") {
node = p.parseMapExpression(token)
} else {
p.error("unexpected token %v", token)
}
}
return p.parsePostfixExpression(node)
}
func (p *parser) parseIdentifierExpression(token Token) Node {
var node Node
if p.current.Is(Bracket, "(") {
var arguments []Node
if b, ok := builtins[token.Value]; ok {
p.expect(Bracket, "(")
// TODO: Add builtins signatures.
if b.arity == 1 {
arguments = make([]Node, 1)
arguments[0] = p.parseExpression(0)
} else if b.arity == 2 {
arguments = make([]Node, 2)
arguments[0] = p.parseExpression(0)
p.expect(Operator, ",")
arguments[1] = p.parseClosure()
}
p.expect(Bracket, ")")
node = &BuiltinNode{
Name: token.Value,
Arguments: arguments,
}
node.SetLocation(token.Location)
} else {
arguments = p.parseArguments()
node = &FunctionNode{
Name: token.Value,
Arguments: arguments,
}
node.SetLocation(token.Location)
}
} else {
node = &IdentifierNode{Value: token.Value}
node.SetLocation(token.Location)
}
return node
}
func (p *parser) parseClosure() Node {
token := p.current
p.expect(Bracket, "{")
p.depth++
node := p.parseExpression(0)
p.depth--
p.expect(Bracket, "}")
closure := &ClosureNode{
Node: node,
}
closure.SetLocation(token.Location)
return closure
}
func (p *parser) parseArrayExpression(token Token) Node {
nodes := make([]Node, 0)
p.expect(Bracket, "[")
for !p.current.Is(Bracket, "]") && p.err == nil {
if len(nodes) > 0 {
p.expect(Operator, ",")
if p.current.Is(Bracket, "]") {
goto end
}
}
node := p.parseExpression(0)
nodes = append(nodes, node)
}
end:
p.expect(Bracket, "]")
node := &ArrayNode{Nodes: nodes}
node.SetLocation(token.Location)
return node
}
func (p *parser) parseMapExpression(token Token) Node {
p.expect(Bracket, "{")
nodes := make([]Node, 0)
for !p.current.Is(Bracket, "}") && p.err == nil {
if len(nodes) > 0 {
p.expect(Operator, ",")
if p.current.Is(Bracket, "}") {
goto end
}
if p.current.Is(Operator, ",") {
p.error("unexpected token %v", p.current)
}
}
var key Node
// a map key can be:
// * a number
// * a string
// * a identifier, which is equivalent to a string
// * an expression, which must be enclosed in parentheses -- (1 + 2)
if p.current.Is(Number) || p.current.Is(String) || p.current.Is(Identifier) {
key = &StringNode{Value: p.current.Value}
key.SetLocation(token.Location)
p.next()
} else if p.current.Is(Bracket, "(") {
key = p.parseExpression(0)
} else {
p.error("a map key must be a quoted string, a number, a identifier, or an expression enclosed in parentheses (unexpected token %v)", p.current)
}
p.expect(Operator, ":")
node := p.parseExpression(0)
pair := &PairNode{Key: key, Value: node}
pair.SetLocation(token.Location)
nodes = append(nodes, pair)
}
end:
p.expect(Bracket, "}")
node := &MapNode{Pairs: nodes}
node.SetLocation(token.Location)
return node
}
func (p *parser) parsePostfixExpression(node Node) Node {
token := p.current
for (token.Is(Operator) || token.Is(Bracket)) && p.err == nil {
if token.Value == "." {
p.next()
token = p.current
p.next()
if token.Kind != Identifier &&
// Operators like "not" and "matches" are valid methods or property names.
(token.Kind != Operator || !isValidIdentifier(token.Value)) {
p.error("expected name")
}
if p.current.Is(Bracket, "(") {
arguments := p.parseArguments()
node = &MethodNode{
Node: node,
Method: token.Value,
Arguments: arguments,
}
node.SetLocation(token.Location)
} else {
node = &PropertyNode{
Node: node,
Property: token.Value,
}
node.SetLocation(token.Location)
}
} else if token.Value == "[" {
p.next()
var from, to Node
if p.current.Is(Operator, ":") { // slice without from [:1]
p.next()
if !p.current.Is(Bracket, "]") { // slice without from and to [:]
to = p.parseExpression(0)
}
node = &SliceNode{
Node: node,
To: to,
}
node.SetLocation(token.Location)
p.expect(Bracket, "]")
} else {
from = p.parseExpression(0)
if p.current.Is(Operator, ":") {
p.next()
if !p.current.Is(Bracket, "]") { // slice without to [1:]
to = p.parseExpression(0)
}
node = &SliceNode{
Node: node,
From: from,
To: to,
}
node.SetLocation(token.Location)
p.expect(Bracket, "]")
} else {
// Slice operator [:] was not found, it should by just index node.
node = &IndexNode{
Node: node,
Index: from,
}
node.SetLocation(token.Location)
p.expect(Bracket, "]")
}
}
} else {
break
}
token = p.current
}
return node
}
func isValidIdentifier(str string) bool {
if len(str) == 0 {
return false
}
h, w := utf8.DecodeRuneInString(str)
if !IsAlphabetic(h) {
return false
}
for _, r := range str[w:] {
if !IsAlphaNumeric(r) {
return false
}
}
return true
}
func (p *parser) parseArguments() []Node {
p.expect(Bracket, "(")
nodes := make([]Node, 0)
for !p.current.Is(Bracket, ")") && p.err == nil {
if len(nodes) > 0 {
p.expect(Operator, ",")
}
node := p.parseExpression(0)
nodes = append(nodes, node)
}
p.expect(Bracket, ")")
return nodes
}
+3247
View File
File diff suppressed because it is too large Load Diff
+53
View File
@@ -0,0 +1,53 @@
package vm
const (
OpPush byte = iota
OpPop
OpRot
OpFetch
OpFetchMap
OpTrue
OpFalse
OpNil
OpNegate
OpNot
OpEqual
OpEqualInt
OpEqualString
OpJump
OpJumpIfTrue
OpJumpIfFalse
OpJumpBackward
OpIn
OpLess
OpMore
OpLessOrEqual
OpMoreOrEqual
OpAdd
OpSubtract
OpMultiply
OpDivide
OpModulo
OpExponent
OpRange
OpMatches
OpMatchesConst
OpContains
OpStartsWith
OpEndsWith
OpIndex
OpSlice
OpProperty
OpCall
OpCallFast
OpMethod
OpArray
OpMap
OpLen
OpCast
OpStore
OpLoad
OpInc
OpBegin
OpEnd // This opcode must be at the end of this list.
)
+216
View File
@@ -0,0 +1,216 @@
package vm
import (
"encoding/binary"
"fmt"
"regexp"
"github.com/antonmedv/expr/file"
)
type Program struct {
Source *file.Source
Locations map[int]file.Location
Constants []interface{}
Bytecode []byte
}
func (program *Program) Disassemble() string {
out := ""
ip := 0
for ip < len(program.Bytecode) {
pp := ip
op := program.Bytecode[ip]
ip++
readArg := func() uint16 {
if ip+1 >= len(program.Bytecode) {
return 0
}
i := binary.LittleEndian.Uint16([]byte{program.Bytecode[ip], program.Bytecode[ip+1]})
ip += 2
return i
}
code := func(label string) {
out += fmt.Sprintf("%v\t%v\n", pp, label)
}
jump := func(label string) {
a := readArg()
out += fmt.Sprintf("%v\t%v\t%v\t(%v)\n", pp, label, a, ip+int(a))
}
back := func(label string) {
a := readArg()
out += fmt.Sprintf("%v\t%v\t%v\t(%v)\n", pp, label, a, ip-int(a))
}
argument := func(label string) {
a := readArg()
out += fmt.Sprintf("%v\t%v\t%v\n", pp, label, a)
}
constant := func(label string) {
a := readArg()
var c interface{}
if int(a) < len(program.Constants) {
c = program.Constants[a]
}
if r, ok := c.(*regexp.Regexp); ok {
c = r.String()
}
out += fmt.Sprintf("%v\t%v\t%v\t%#v\n", pp, label, a, c)
}
switch op {
case OpPush:
constant("OpPush")
case OpPop:
code("OpPop")
case OpRot:
code("OpRot")
case OpFetch:
constant("OpFetch")
case OpFetchMap:
constant("OpFetchMap")
case OpTrue:
code("OpTrue")
case OpFalse:
code("OpFalse")
case OpNil:
code("OpNil")
case OpNegate:
code("OpNegate")
case OpNot:
code("OpNot")
case OpEqual:
code("OpEqual")
case OpEqualInt:
code("OpEqualInt")
case OpEqualString:
code("OpEqualString")
case OpJump:
jump("OpJump")
case OpJumpIfTrue:
jump("OpJumpIfTrue")
case OpJumpIfFalse:
jump("OpJumpIfFalse")
case OpJumpBackward:
back("OpJumpBackward")
case OpIn:
code("OpIn")
case OpLess:
code("OpLess")
case OpMore:
code("OpMore")
case OpLessOrEqual:
code("OpLessOrEqual")
case OpMoreOrEqual:
code("OpMoreOrEqual")
case OpAdd:
code("OpAdd")
case OpSubtract:
code("OpSubtract")
case OpMultiply:
code("OpMultiply")
case OpDivide:
code("OpDivide")
case OpModulo:
code("OpModulo")
case OpExponent:
code("OpExponent")
case OpRange:
code("OpRange")
case OpMatches:
code("OpMatches")
case OpMatchesConst:
constant("OpMatchesConst")
case OpContains:
code("OpContains")
case OpStartsWith:
code("OpStartsWith")
case OpEndsWith:
code("OpEndsWith")
case OpIndex:
code("OpIndex")
case OpSlice:
code("OpSlice")
case OpProperty:
constant("OpProperty")
case OpCall:
constant("OpCall")
case OpCallFast:
constant("OpCallFast")
case OpMethod:
constant("OpMethod")
case OpArray:
code("OpArray")
case OpMap:
code("OpMap")
case OpLen:
code("OpLen")
case OpCast:
argument("OpCast")
case OpStore:
constant("OpStore")
case OpLoad:
constant("OpLoad")
case OpInc:
constant("OpInc")
case OpBegin:
code("OpBegin")
case OpEnd:
code("OpEnd")
default:
out += fmt.Sprintf("%v\t%#x\n", pp, op)
}
}
return out
}
+343
View File
@@ -0,0 +1,343 @@
package vm
//go:generate go run ./generate
import (
"fmt"
"math"
"reflect"
)
type Call struct {
Name string
Size int
}
type Scope map[string]interface{}
func fetch(from interface{}, i interface{}) interface{} {
v := reflect.ValueOf(from)
kind := v.Kind()
// Structures can be access through a pointer or through a value, when they
// are accessed through a pointer we don't want to copy them to a value.
if kind == reflect.Ptr && reflect.Indirect(v).Kind() == reflect.Struct {
v = reflect.Indirect(v)
kind = v.Kind()
}
switch kind {
case reflect.Array, reflect.Slice, reflect.String:
value := v.Index(toInt(i))
if value.IsValid() && value.CanInterface() {
return value.Interface()
}
case reflect.Map:
value := v.MapIndex(reflect.ValueOf(i))
if value.IsValid() {
if value.CanInterface() {
return value.Interface()
}
} else {
elem := reflect.TypeOf(from).Elem()
return reflect.Zero(elem).Interface()
}
case reflect.Struct:
value := v.FieldByName(reflect.ValueOf(i).String())
if value.IsValid() && value.CanInterface() {
return value.Interface()
}
}
panic(fmt.Sprintf("cannot fetch %v from %T", i, from))
}
func slice(array, from, to interface{}) interface{} {
v := reflect.ValueOf(array)
switch v.Kind() {
case reflect.Array, reflect.Slice, reflect.String:
length := v.Len()
a, b := toInt(from), toInt(to)
if b > length {
b = length
}
if a > b {
a = b
}
value := v.Slice(a, b)
if value.IsValid() && value.CanInterface() {
return value.Interface()
}
case reflect.Ptr:
value := v.Elem()
if value.IsValid() && value.CanInterface() {
return slice(value.Interface(), from, to)
}
}
panic(fmt.Sprintf("cannot slice %v", from))
}
func FetchFn(from interface{}, name string) reflect.Value {
v := reflect.ValueOf(from)
// Methods can be defined on any type.
if v.NumMethod() > 0 {
method := v.MethodByName(name)
if method.IsValid() {
return method
}
}
d := v
if v.Kind() == reflect.Ptr {
d = v.Elem()
}
switch d.Kind() {
case reflect.Map:
value := d.MapIndex(reflect.ValueOf(name))
if value.IsValid() && value.CanInterface() {
return value.Elem()
}
case reflect.Struct:
// If struct has not method, maybe it has func field.
// To access this field we need dereference value.
value := d.FieldByName(name)
if value.IsValid() {
return value
}
}
panic(fmt.Sprintf(`cannot get "%v" from %T`, name, from))
}
func in(needle interface{}, array interface{}) bool {
if array == nil {
return false
}
v := reflect.ValueOf(array)
switch v.Kind() {
case reflect.Array, reflect.Slice:
for i := 0; i < v.Len(); i++ {
value := v.Index(i)
if value.IsValid() && value.CanInterface() {
if equal(value.Interface(), needle).(bool) {
return true
}
}
}
return false
case reflect.Map:
n := reflect.ValueOf(needle)
if !n.IsValid() {
panic(fmt.Sprintf("cannot use %T as index to %T", needle, array))
}
value := v.MapIndex(n)
if value.IsValid() {
return true
}
return false
case reflect.Struct:
n := reflect.ValueOf(needle)
if !n.IsValid() || n.Kind() != reflect.String {
panic(fmt.Sprintf("cannot use %T as field name of %T", needle, array))
}
value := v.FieldByName(n.String())
if value.IsValid() {
return true
}
return false
case reflect.Ptr:
value := v.Elem()
if value.IsValid() && value.CanInterface() {
return in(needle, value.Interface())
}
return false
}
panic(fmt.Sprintf(`operator "in"" not defined on %T`, array))
}
func length(a interface{}) int {
v := reflect.ValueOf(a)
switch v.Kind() {
case reflect.Array, reflect.Slice, reflect.Map, reflect.String:
return v.Len()
default:
panic(fmt.Sprintf("invalid argument for len (type %T)", a))
}
}
func negate(i interface{}) interface{} {
switch v := i.(type) {
case float32:
return -v
case float64:
return -v
case int:
return -v
case int8:
return -v
case int16:
return -v
case int32:
return -v
case int64:
return -v
case uint:
return -v
case uint8:
return -v
case uint16:
return -v
case uint32:
return -v
case uint64:
return -v
default:
panic(fmt.Sprintf("invalid operation: - %T", v))
}
}
func exponent(a, b interface{}) float64 {
return math.Pow(toFloat64(a), toFloat64(b))
}
func makeRange(min, max int) []int {
size := max - min + 1
rng := make([]int, size)
for i := range rng {
rng[i] = min + i
}
return rng
}
func toInt(a interface{}) int {
switch x := a.(type) {
case float32:
return int(x)
case float64:
return int(x)
case int:
return x
case int8:
return int(x)
case int16:
return int(x)
case int32:
return int(x)
case int64:
return int(x)
case uint:
return int(x)
case uint8:
return int(x)
case uint16:
return int(x)
case uint32:
return int(x)
case uint64:
return int(x)
default:
panic(fmt.Sprintf("invalid operation: int(%T)", x))
}
}
func toInt64(a interface{}) int64 {
switch x := a.(type) {
case float32:
return int64(x)
case float64:
return int64(x)
case int:
return int64(x)
case int8:
return int64(x)
case int16:
return int64(x)
case int32:
return int64(x)
case int64:
return x
case uint:
return int64(x)
case uint8:
return int64(x)
case uint16:
return int64(x)
case uint32:
return int64(x)
case uint64:
return int64(x)
default:
panic(fmt.Sprintf("invalid operation: int64(%T)", x))
}
}
func toFloat64(a interface{}) float64 {
switch x := a.(type) {
case float32:
return float64(x)
case float64:
return x
case int:
return float64(x)
case int8:
return float64(x)
case int16:
return float64(x)
case int32:
return float64(x)
case int64:
return float64(x)
case uint:
return float64(x)
case uint8:
return float64(x)
case uint16:
return float64(x)
case uint32:
return float64(x)
case uint64:
return float64(x)
default:
panic(fmt.Sprintf("invalid operation: float64(%T)", x))
}
}
func isNil(v interface{}) bool {
if v == nil {
return true
}
r := reflect.ValueOf(v)
switch r.Kind() {
case reflect.Chan, reflect.Func, reflect.Map, reflect.Ptr, reflect.Interface, reflect.Slice:
return r.IsNil()
default:
return false
}
}
+439
View File
@@ -0,0 +1,439 @@
package vm
import (
"fmt"
"reflect"
"regexp"
"strings"
"github.com/antonmedv/expr/file"
)
var (
MemoryBudget int = 1e6
)
func Run(program *Program, env interface{}) (interface{}, error) {
if program == nil {
return nil, fmt.Errorf("program is nil")
}
vm := VM{}
return vm.Run(program, env)
}
type VM struct {
stack []interface{}
constants []interface{}
bytecode []byte
ip int
pp int
scopes []Scope
debug bool
step chan struct{}
curr chan int
memory int
limit int
}
func Debug() *VM {
vm := &VM{
debug: true,
step: make(chan struct{}, 0),
curr: make(chan int, 0),
}
return vm
}
func (vm *VM) Run(program *Program, env interface{}) (out interface{}, err error) {
defer func() {
if r := recover(); r != nil {
f := &file.Error{
Location: program.Locations[vm.pp],
Message: fmt.Sprintf("%v", r),
}
err = f.Bind(program.Source)
}
}()
vm.limit = MemoryBudget
vm.ip = 0
vm.pp = 0
if vm.stack == nil {
vm.stack = make([]interface{}, 0, 2)
} else {
vm.stack = vm.stack[0:0]
}
if vm.scopes != nil {
vm.scopes = vm.scopes[0:0]
}
vm.bytecode = program.Bytecode
vm.constants = program.Constants
for vm.ip < len(vm.bytecode) {
if vm.debug {
<-vm.step
}
vm.pp = vm.ip
vm.ip++
op := vm.bytecode[vm.pp]
switch op {
case OpPush:
vm.push(vm.constant())
case OpPop:
vm.pop()
case OpRot:
b := vm.pop()
a := vm.pop()
vm.push(b)
vm.push(a)
case OpFetch:
vm.push(fetch(env, vm.constant()))
case OpFetchMap:
vm.push(env.(map[string]interface{})[vm.constant().(string)])
case OpTrue:
vm.push(true)
case OpFalse:
vm.push(false)
case OpNil:
vm.push(nil)
case OpNegate:
v := negate(vm.pop())
vm.push(v)
case OpNot:
v := vm.pop().(bool)
vm.push(!v)
case OpEqual:
b := vm.pop()
a := vm.pop()
vm.push(equal(a, b))
case OpEqualInt:
b := vm.pop()
a := vm.pop()
vm.push(a.(int) == b.(int))
case OpEqualString:
b := vm.pop()
a := vm.pop()
vm.push(a.(string) == b.(string))
case OpJump:
offset := vm.arg()
vm.ip += int(offset)
case OpJumpIfTrue:
offset := vm.arg()
if vm.current().(bool) {
vm.ip += int(offset)
}
case OpJumpIfFalse:
offset := vm.arg()
if !vm.current().(bool) {
vm.ip += int(offset)
}
case OpJumpBackward:
offset := vm.arg()
vm.ip -= int(offset)
case OpIn:
b := vm.pop()
a := vm.pop()
vm.push(in(a, b))
case OpLess:
b := vm.pop()
a := vm.pop()
vm.push(less(a, b))
case OpMore:
b := vm.pop()
a := vm.pop()
vm.push(more(a, b))
case OpLessOrEqual:
b := vm.pop()
a := vm.pop()
vm.push(lessOrEqual(a, b))
case OpMoreOrEqual:
b := vm.pop()
a := vm.pop()
vm.push(moreOrEqual(a, b))
case OpAdd:
b := vm.pop()
a := vm.pop()
vm.push(add(a, b))
case OpSubtract:
b := vm.pop()
a := vm.pop()
vm.push(subtract(a, b))
case OpMultiply:
b := vm.pop()
a := vm.pop()
vm.push(multiply(a, b))
case OpDivide:
b := vm.pop()
a := vm.pop()
vm.push(divide(a, b))
case OpModulo:
b := vm.pop()
a := vm.pop()
vm.push(modulo(a, b))
case OpExponent:
b := vm.pop()
a := vm.pop()
vm.push(exponent(a, b))
case OpRange:
b := vm.pop()
a := vm.pop()
min := toInt(a)
max := toInt(b)
size := max - min + 1
if vm.memory+size >= vm.limit {
panic("memory budget exceeded")
}
vm.push(makeRange(min, max))
vm.memory += size
case OpMatches:
b := vm.pop()
a := vm.pop()
match, err := regexp.MatchString(b.(string), a.(string))
if err != nil {
panic(err)
}
vm.push(match)
case OpMatchesConst:
a := vm.pop()
r := vm.constant().(*regexp.Regexp)
vm.push(r.MatchString(a.(string)))
case OpContains:
b := vm.pop()
a := vm.pop()
vm.push(strings.Contains(a.(string), b.(string)))
case OpStartsWith:
b := vm.pop()
a := vm.pop()
vm.push(strings.HasPrefix(a.(string), b.(string)))
case OpEndsWith:
b := vm.pop()
a := vm.pop()
vm.push(strings.HasSuffix(a.(string), b.(string)))
case OpIndex:
b := vm.pop()
a := vm.pop()
vm.push(fetch(a, b))
case OpSlice:
from := vm.pop()
to := vm.pop()
node := vm.pop()
vm.push(slice(node, from, to))
case OpProperty:
a := vm.pop()
b := vm.constant()
vm.push(fetch(a, b))
case OpCall:
call := vm.constant().(Call)
in := make([]reflect.Value, call.Size)
for i := call.Size - 1; i >= 0; i-- {
param := vm.pop()
if param == nil && reflect.TypeOf(param) == nil {
// In case of nil value and nil type use this hack,
// otherwise reflect.Call will panic on zero value.
in[i] = reflect.ValueOf(&param).Elem()
} else {
in[i] = reflect.ValueOf(param)
}
}
out := FetchFn(env, call.Name).Call(in)
vm.push(out[0].Interface())
case OpCallFast:
call := vm.constant().(Call)
in := make([]interface{}, call.Size)
for i := call.Size - 1; i >= 0; i-- {
in[i] = vm.pop()
}
fn := FetchFn(env, call.Name).Interface()
vm.push(fn.(func(...interface{}) interface{})(in...))
case OpMethod:
call := vm.constants[vm.arg()].(Call)
in := make([]reflect.Value, call.Size)
for i := call.Size - 1; i >= 0; i-- {
param := vm.pop()
if param == nil && reflect.TypeOf(param) == nil {
// In case of nil value and nil type use this hack,
// otherwise reflect.Call will panic on zero value.
in[i] = reflect.ValueOf(&param).Elem()
} else {
in[i] = reflect.ValueOf(param)
}
}
out := FetchFn(vm.pop(), call.Name).Call(in)
vm.push(out[0].Interface())
case OpArray:
size := vm.pop().(int)
array := make([]interface{}, size)
for i := size - 1; i >= 0; i-- {
array[i] = vm.pop()
}
vm.push(array)
vm.memory += size
if vm.memory >= vm.limit {
panic("memory budget exceeded")
}
case OpMap:
size := vm.pop().(int)
m := make(map[string]interface{})
for i := size - 1; i >= 0; i-- {
value := vm.pop()
key := vm.pop()
m[key.(string)] = value
}
vm.push(m)
vm.memory += size
if vm.memory >= vm.limit {
panic("memory budget exceeded")
}
case OpLen:
vm.push(length(vm.current()))
case OpCast:
t := vm.arg()
switch t {
case 0:
vm.push(toInt64(vm.pop()))
case 1:
vm.push(toFloat64(vm.pop()))
}
case OpStore:
scope := vm.Scope()
key := vm.constant().(string)
value := vm.pop()
scope[key] = value
case OpLoad:
scope := vm.Scope()
key := vm.constant().(string)
vm.push(scope[key])
case OpInc:
scope := vm.Scope()
key := vm.constant().(string)
i := scope[key].(int)
i++
scope[key] = i
case OpBegin:
scope := make(Scope)
vm.scopes = append(vm.scopes, scope)
case OpEnd:
vm.scopes = vm.scopes[:len(vm.scopes)-1]
default:
panic(fmt.Sprintf("unknown bytecode %#x", op))
}
if vm.debug {
vm.curr <- vm.ip
}
}
if vm.debug {
close(vm.curr)
close(vm.step)
}
if len(vm.stack) > 0 {
return vm.pop(), nil
}
return nil, nil
}
func (vm *VM) push(value interface{}) {
vm.stack = append(vm.stack, value)
}
func (vm *VM) current() interface{} {
return vm.stack[len(vm.stack)-1]
}
func (vm *VM) pop() interface{} {
value := vm.stack[len(vm.stack)-1]
vm.stack = vm.stack[:len(vm.stack)-1]
return value
}
func (vm *VM) arg() uint16 {
b0, b1 := vm.bytecode[vm.ip], vm.bytecode[vm.ip+1]
vm.ip += 2
return uint16(b0) | uint16(b1)<<8
}
func (vm *VM) constant() interface{} {
return vm.constants[vm.arg()]
}
func (vm *VM) Stack() []interface{} {
return vm.stack
}
func (vm *VM) Scope() Scope {
if len(vm.scopes) > 0 {
return vm.scopes[len(vm.scopes)-1]
}
return nil
}
func (vm *VM) Step() {
if vm.ip < len(vm.bytecode) {
vm.step <- struct{}{}
}
}
func (vm *VM) Position() chan int {
return vm.curr
}
+11
View File
@@ -149,6 +149,17 @@ To add a constant, add the header that includes it to the appropriate variable.
Then, edit the regex (if necessary) to match the desired constant. Avoid making Then, edit the regex (if necessary) to match the desired constant. Avoid making
the regex too broad to avoid matching unintended constants. the regex too broad to avoid matching unintended constants.
### mkmerge.go
This program is used to extract duplicate const, func, and type declarations
from the generated architecture-specific files listed below, and merge these
into a common file for each OS.
The merge is performed in the following steps:
1. Construct the set of common code that is idential in all architecture-specific files.
2. Write this common code to the merged file.
3. Remove the common code from all architecture-specific files.
## Generated files ## Generated files
-7
View File
@@ -23,10 +23,6 @@ TEXT ·SyscallNoError(SB),NOSPLIT,$0-48
MOV a1+8(FP), A0 MOV a1+8(FP), A0
MOV a2+16(FP), A1 MOV a2+16(FP), A1
MOV a3+24(FP), A2 MOV a3+24(FP), A2
MOV $0, A3
MOV $0, A4
MOV $0, A5
MOV $0, A6
MOV trap+0(FP), A7 // syscall entry MOV trap+0(FP), A7 // syscall entry
ECALL ECALL
MOV A0, r1+32(FP) // r1 MOV A0, r1+32(FP) // r1
@@ -44,9 +40,6 @@ TEXT ·RawSyscallNoError(SB),NOSPLIT,$0-48
MOV a1+8(FP), A0 MOV a1+8(FP), A0
MOV a2+16(FP), A1 MOV a2+16(FP), A1
MOV a3+24(FP), A2 MOV a3+24(FP), A2
MOV ZERO, A3
MOV ZERO, A4
MOV ZERO, A5
MOV trap+0(FP), A7 // syscall entry MOV trap+0(FP), A7 // syscall entry
ECALL ECALL
MOV A0, r1+32(FP) MOV A0, r1+32(FP)
+1 -1
View File
@@ -486,7 +486,7 @@ ccflags="$@"
$2 ~ /^TCSET/ || $2 ~ /^TCSET/ ||
$2 ~ /^TC(FLSH|SBRKP?|XONC)$/ || $2 ~ /^TC(FLSH|SBRKP?|XONC)$/ ||
$2 !~ "RTF_BITS" && $2 !~ "RTF_BITS" &&
$2 ~ /^(IFF|IFT|NET_RT|RTM|RTF|RTV|RTA|RTAX)_/ || $2 ~ /^(IFF|IFT|NET_RT|RTM(GRP)?|RTF|RTV|RTA|RTAX)_/ ||
$2 ~ /^BIOC/ || $2 ~ /^BIOC/ ||
$2 ~ /^RUSAGE_(SELF|CHILDREN|THREAD)/ || $2 ~ /^RUSAGE_(SELF|CHILDREN|THREAD)/ ||
$2 ~ /^RLIMIT_(AS|CORE|CPU|DATA|FSIZE|LOCKS|MEMLOCK|MSGQUEUE|NICE|NOFILE|NPROC|RSS|RTPRIO|RTTIME|SIGPENDING|STACK)|RLIM_INFINITY/ || $2 ~ /^RLIMIT_(AS|CORE|CPU|DATA|FSIZE|LOCKS|MEMLOCK|MSGQUEUE|NICE|NOFILE|NPROC|RSS|RTPRIO|RTTIME|SIGPENDING|STACK)|RLIM_INFINITY/ ||
+17
View File
@@ -510,6 +510,23 @@ func SysctlRaw(name string, args ...int) ([]byte, error) {
return buf[:n], nil return buf[:n], nil
} }
func SysctlClockinfo(name string) (*Clockinfo, error) {
mib, err := sysctlmib(name)
if err != nil {
return nil, err
}
n := uintptr(SizeofClockinfo)
var ci Clockinfo
if err := sysctl(mib, (*byte)(unsafe.Pointer(&ci)), &n, nil, 0); err != nil {
return nil, err
}
if n != SizeofClockinfo {
return nil, EIO
}
return &ci, nil
}
//sys utimes(path string, timeval *[2]Timeval) (err error) //sys utimes(path string, timeval *[2]Timeval) (err error)
func Utimes(path string, tv []Timeval) error { func Utimes(path string, tv []Timeval) error {
-17
View File
@@ -155,23 +155,6 @@ func getAttrList(path string, attrList attrList, attrBuf []byte, options uint) (
//sys getattrlist(path *byte, list unsafe.Pointer, buf unsafe.Pointer, size uintptr, options int) (err error) //sys getattrlist(path *byte, list unsafe.Pointer, buf unsafe.Pointer, size uintptr, options int) (err error)
func SysctlClockinfo(name string) (*Clockinfo, error) {
mib, err := sysctlmib(name)
if err != nil {
return nil, err
}
n := uintptr(SizeofClockinfo)
var ci Clockinfo
if err := sysctl(mib, (*byte)(unsafe.Pointer(&ci)), &n, nil, 0); err != nil {
return nil, err
}
if n != SizeofClockinfo {
return nil, EIO
}
return &ci, nil
}
//sysnb pipe() (r int, w int, err error) //sysnb pipe() (r int, w int, err error)
func Pipe(p []int) (err error) { func Pipe(p []int) (err error) {
-6
View File
@@ -529,12 +529,6 @@ func PtraceGetRegs(pid int, regsout *Reg) (err error) {
return ptrace(PTRACE_GETREGS, pid, uintptr(unsafe.Pointer(regsout)), 0) return ptrace(PTRACE_GETREGS, pid, uintptr(unsafe.Pointer(regsout)), 0)
} }
func PtraceIO(req int, pid int, addr uintptr, out []byte, countin int) (count int, err error) {
ioDesc := PtraceIoDesc{Op: int32(req), Offs: (*byte)(unsafe.Pointer(addr)), Addr: (*byte)(unsafe.Pointer(&out[0])), Len: uint(countin)}
err = ptrace(PTRACE_IO, pid, uintptr(unsafe.Pointer(&ioDesc)), 0)
return int(ioDesc.Len), err
}
func PtraceLwpEvents(pid int, enable int) (err error) { func PtraceLwpEvents(pid int, enable int) (err error) {
return ptrace(PTRACE_LWPEVENTS, pid, 0, enable) return ptrace(PTRACE_LWPEVENTS, pid, 0, enable)
} }
+6
View File
@@ -54,3 +54,9 @@ func sendfile(outfd int, infd int, offset *int64, count int) (written int, err e
} }
func Syscall9(num, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r1, r2 uintptr, err syscall.Errno) func Syscall9(num, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r1, r2 uintptr, err syscall.Errno)
func PtraceIO(req int, pid int, addr uintptr, out []byte, countin int) (count int, err error) {
ioDesc := PtraceIoDesc{Op: int32(req), Offs: (*byte)(unsafe.Pointer(addr)), Addr: (*byte)(unsafe.Pointer(&out[0])), Len: uint32(countin)}
err = ptrace(PTRACE_IO, pid, uintptr(unsafe.Pointer(&ioDesc)), 0)
return int(ioDesc.Len), err
}
+6
View File
@@ -54,3 +54,9 @@ func sendfile(outfd int, infd int, offset *int64, count int) (written int, err e
} }
func Syscall9(num, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r1, r2 uintptr, err syscall.Errno) func Syscall9(num, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r1, r2 uintptr, err syscall.Errno)
func PtraceIO(req int, pid int, addr uintptr, out []byte, countin int) (count int, err error) {
ioDesc := PtraceIoDesc{Op: int32(req), Offs: (*byte)(unsafe.Pointer(addr)), Addr: (*byte)(unsafe.Pointer(&out[0])), Len: uint64(countin)}
err = ptrace(PTRACE_IO, pid, uintptr(unsafe.Pointer(&ioDesc)), 0)
return int(ioDesc.Len), err
}
+6
View File
@@ -54,3 +54,9 @@ func sendfile(outfd int, infd int, offset *int64, count int) (written int, err e
} }
func Syscall9(num, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r1, r2 uintptr, err syscall.Errno) func Syscall9(num, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r1, r2 uintptr, err syscall.Errno)
func PtraceIO(req int, pid int, addr uintptr, out []byte, countin int) (count int, err error) {
ioDesc := PtraceIoDesc{Op: int32(req), Offs: (*byte)(unsafe.Pointer(addr)), Addr: (*byte)(unsafe.Pointer(&out[0])), Len: uint32(countin)}
err = ptrace(PTRACE_IO, pid, uintptr(unsafe.Pointer(&ioDesc)), 0)
return int(ioDesc.Len), err
}
+6
View File
@@ -54,3 +54,9 @@ func sendfile(outfd int, infd int, offset *int64, count int) (written int, err e
} }
func Syscall9(num, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r1, r2 uintptr, err syscall.Errno) func Syscall9(num, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r1, r2 uintptr, err syscall.Errno)
func PtraceIO(req int, pid int, addr uintptr, out []byte, countin int) (count int, err error) {
ioDesc := PtraceIoDesc{Op: int32(req), Offs: (*byte)(unsafe.Pointer(addr)), Addr: (*byte)(unsafe.Pointer(&out[0])), Len: uint64(countin)}
err = ptrace(PTRACE_IO, pid, uintptr(unsafe.Pointer(&ioDesc)), 0)
return int(ioDesc.Len), err
}
+26 -2
View File
@@ -1555,8 +1555,8 @@ func Sendfile(outfd int, infd int, offset *int64, count int) (written int, err e
//sys Acct(path string) (err error) //sys Acct(path string) (err error)
//sys AddKey(keyType string, description string, payload []byte, ringid int) (id int, err error) //sys AddKey(keyType string, description string, payload []byte, ringid int) (id int, err error)
//sys Adjtimex(buf *Timex) (state int, err error) //sys Adjtimex(buf *Timex) (state int, err error)
//sys Capget(hdr *CapUserHeader, data *CapUserData) (err error) //sysnb Capget(hdr *CapUserHeader, data *CapUserData) (err error)
//sys Capset(hdr *CapUserHeader, data *CapUserData) (err error) //sysnb Capset(hdr *CapUserHeader, data *CapUserData) (err error)
//sys Chdir(path string) (err error) //sys Chdir(path string) (err error)
//sys Chroot(path string) (err error) //sys Chroot(path string) (err error)
//sys ClockGetres(clockid int32, res *Timespec) (err error) //sys ClockGetres(clockid int32, res *Timespec) (err error)
@@ -1654,6 +1654,30 @@ func Setgid(uid int) (err error) {
return EOPNOTSUPP return EOPNOTSUPP
} }
// SetfsgidRetGid sets fsgid for current thread and returns previous fsgid set.
// setfsgid(2) will return a non-nil error only if its caller lacks CAP_SETUID capability.
// If the call fails due to other reasons, current fsgid will be returned.
func SetfsgidRetGid(gid int) (int, error) {
return setfsgid(gid)
}
// SetfsuidRetUid sets fsuid for current thread and returns previous fsuid set.
// setfsgid(2) will return a non-nil error only if its caller lacks CAP_SETUID capability
// If the call fails due to other reasons, current fsuid will be returned.
func SetfsuidRetUid(uid int) (int, error) {
return setfsuid(uid)
}
func Setfsgid(gid int) error {
_, err := setfsgid(gid)
return err
}
func Setfsuid(uid int) error {
_, err := setfsuid(uid)
return err
}
func Signalfd(fd int, sigmask *Sigset_t, flags int) (newfd int, err error) { func Signalfd(fd int, sigmask *Sigset_t, flags int) (newfd int, err error) {
return signalfd(fd, sigmask, _C__NSIG/8, flags) return signalfd(fd, sigmask, _C__NSIG/8, flags)
} }
+2 -2
View File
@@ -70,8 +70,8 @@ func Pipe2(p []int, flags int) (err error) {
//sys Pwrite(fd int, p []byte, offset int64) (n int, err error) = SYS_PWRITE64 //sys Pwrite(fd int, p []byte, offset int64) (n int, err error) = SYS_PWRITE64
//sys Renameat(olddirfd int, oldpath string, newdirfd int, newpath string) (err error) //sys Renameat(olddirfd int, oldpath string, newdirfd int, newpath string) (err error)
//sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) = SYS_SENDFILE64 //sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) = SYS_SENDFILE64
//sys Setfsgid(gid int) (err error) = SYS_SETFSGID32 //sys setfsgid(gid int) (prev int, err error) = SYS_SETFSGID32
//sys Setfsuid(uid int) (err error) = SYS_SETFSUID32 //sys setfsuid(uid int) (prev int, err error) = SYS_SETFSUID32
//sysnb Setregid(rgid int, egid int) (err error) = SYS_SETREGID32 //sysnb Setregid(rgid int, egid int) (err error) = SYS_SETREGID32
//sysnb Setresgid(rgid int, egid int, sgid int) (err error) = SYS_SETRESGID32 //sysnb Setresgid(rgid int, egid int, sgid int) (err error) = SYS_SETRESGID32
//sysnb Setresuid(ruid int, euid int, suid int) (err error) = SYS_SETRESUID32 //sysnb Setresuid(ruid int, euid int, suid int) (err error) = SYS_SETRESUID32
+2 -2
View File
@@ -55,8 +55,8 @@ func Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err
} }
//sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) //sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error)
//sys Setfsgid(gid int) (err error) //sys setfsgid(gid int) (prev int, err error)
//sys Setfsuid(uid int) (err error) //sys setfsuid(uid int) (prev int, err error)
//sysnb Setregid(rgid int, egid int) (err error) //sysnb Setregid(rgid int, egid int) (err error)
//sysnb Setresgid(rgid int, egid int, sgid int) (err error) //sysnb Setresgid(rgid int, egid int, sgid int) (err error)
//sysnb Setresuid(ruid int, euid int, suid int) (err error) //sysnb Setresuid(ruid int, euid int, suid int) (err error)
+2 -2
View File
@@ -98,8 +98,8 @@ func Seek(fd int, offset int64, whence int) (newoffset int64, err error) {
//sys Renameat(olddirfd int, oldpath string, newdirfd int, newpath string) (err error) //sys Renameat(olddirfd int, oldpath string, newdirfd int, newpath string) (err error)
//sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) = SYS_SENDFILE64 //sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) = SYS_SENDFILE64
//sys Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err error) = SYS__NEWSELECT //sys Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err error) = SYS__NEWSELECT
//sys Setfsgid(gid int) (err error) = SYS_SETFSGID32 //sys setfsgid(gid int) (prev int, err error) = SYS_SETFSGID32
//sys Setfsuid(uid int) (err error) = SYS_SETFSUID32 //sys setfsuid(uid int) (prev int, err error) = SYS_SETFSUID32
//sysnb Setregid(rgid int, egid int) (err error) = SYS_SETREGID32 //sysnb Setregid(rgid int, egid int) (err error) = SYS_SETREGID32
//sysnb Setresgid(rgid int, egid int, sgid int) (err error) = SYS_SETRESGID32 //sysnb Setresgid(rgid int, egid int, sgid int) (err error) = SYS_SETRESGID32
//sysnb Setresuid(ruid int, euid int, suid int) (err error) = SYS_SETRESUID32 //sysnb Setresuid(ruid int, euid int, suid int) (err error) = SYS_SETRESUID32
+2 -2
View File
@@ -42,8 +42,8 @@ func Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err
} }
//sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) //sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error)
//sys Setfsgid(gid int) (err error) //sys setfsgid(gid int) (prev int, err error)
//sys Setfsuid(uid int) (err error) //sys setfsuid(uid int) (prev int, err error)
//sysnb Setregid(rgid int, egid int) (err error) //sysnb Setregid(rgid int, egid int) (err error)
//sysnb Setresgid(rgid int, egid int, sgid int) (err error) //sysnb Setresgid(rgid int, egid int, sgid int) (err error)
//sysnb Setresuid(ruid int, euid int, suid int) (err error) //sysnb Setresuid(ruid int, euid int, suid int) (err error)
+6 -2
View File
@@ -36,8 +36,8 @@ func Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err
} }
//sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) //sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error)
//sys Setfsgid(gid int) (err error) //sys setfsgid(gid int) (prev int, err error)
//sys Setfsuid(uid int) (err error) //sys setfsuid(uid int) (prev int, err error)
//sysnb Setregid(rgid int, egid int) (err error) //sysnb Setregid(rgid int, egid int) (err error)
//sysnb Setresgid(rgid int, egid int, sgid int) (err error) //sysnb Setresgid(rgid int, egid int, sgid int) (err error)
//sysnb Setresuid(ruid int, euid int, suid int) (err error) //sysnb Setresuid(ruid int, euid int, suid int) (err error)
@@ -216,6 +216,10 @@ func (cmsg *Cmsghdr) SetLen(length int) {
cmsg.Len = uint64(length) cmsg.Len = uint64(length)
} }
func InotifyInit() (fd int, err error) {
return InotifyInit1(0)
}
//sys poll(fds *PollFd, nfds int, timeout int) (n int, err error) //sys poll(fds *PollFd, nfds int, timeout int) (n int, err error)
func Poll(fds []PollFd, timeout int) (n int, err error) { func Poll(fds []PollFd, timeout int) (n int, err error) {
+2 -2
View File
@@ -31,8 +31,8 @@ func Syscall9(trap, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r1, r2 uintptr,
//sys Renameat(olddirfd int, oldpath string, newdirfd int, newpath string) (err error) //sys Renameat(olddirfd int, oldpath string, newdirfd int, newpath string) (err error)
//sys Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err error) = SYS__NEWSELECT //sys Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err error) = SYS__NEWSELECT
//sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) = SYS_SENDFILE64 //sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) = SYS_SENDFILE64
//sys Setfsgid(gid int) (err error) //sys setfsgid(gid int) (prev int, err error)
//sys Setfsuid(uid int) (err error) //sys setfsuid(uid int) (prev int, err error)
//sysnb Setregid(rgid int, egid int) (err error) //sysnb Setregid(rgid int, egid int) (err error)
//sysnb Setresgid(rgid int, egid int, sgid int) (err error) //sysnb Setresgid(rgid int, egid int, sgid int) (err error)
//sysnb Setresuid(ruid int, euid int, suid int) (err error) //sysnb Setresuid(ruid int, euid int, suid int) (err error)
+2 -2
View File
@@ -34,8 +34,8 @@ package unix
//sys Seek(fd int, offset int64, whence int) (off int64, err error) = SYS_LSEEK //sys Seek(fd int, offset int64, whence int) (off int64, err error) = SYS_LSEEK
//sys Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err error) = SYS__NEWSELECT //sys Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err error) = SYS__NEWSELECT
//sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) //sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error)
//sys Setfsgid(gid int) (err error) //sys setfsgid(gid int) (prev int, err error)
//sys Setfsuid(uid int) (err error) //sys setfsuid(uid int) (prev int, err error)
//sysnb Setregid(rgid int, egid int) (err error) //sysnb Setregid(rgid int, egid int) (err error)
//sysnb Setresgid(rgid int, egid int, sgid int) (err error) //sysnb Setresgid(rgid int, egid int, sgid int) (err error)
//sysnb Setresuid(ruid int, euid int, suid int) (err error) //sysnb Setresuid(ruid int, euid int, suid int) (err error)
+2 -2
View File
@@ -41,8 +41,8 @@ func Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err
} }
//sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) //sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error)
//sys Setfsgid(gid int) (err error) //sys setfsgid(gid int) (prev int, err error)
//sys Setfsuid(uid int) (err error) //sys setfsuid(uid int) (prev int, err error)
//sysnb Setregid(rgid int, egid int) (err error) //sysnb Setregid(rgid int, egid int) (err error)
//sysnb Setresgid(rgid int, egid int, sgid int) (err error) //sysnb Setresgid(rgid int, egid int, sgid int) (err error)
//sysnb Setresuid(ruid int, euid int, suid int) (err error) //sysnb Setresuid(ruid int, euid int, suid int) (err error)
+2 -2
View File
@@ -34,8 +34,8 @@ import (
//sys Seek(fd int, offset int64, whence int) (off int64, err error) = SYS_LSEEK //sys Seek(fd int, offset int64, whence int) (off int64, err error) = SYS_LSEEK
//sys Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err error) //sys Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err error)
//sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) //sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error)
//sys Setfsgid(gid int) (err error) //sys setfsgid(gid int) (prev int, err error)
//sys Setfsuid(uid int) (err error) //sys setfsuid(uid int) (prev int, err error)
//sysnb Setregid(rgid int, egid int) (err error) //sysnb Setregid(rgid int, egid int) (err error)
//sysnb Setresgid(rgid int, egid int, sgid int) (err error) //sysnb Setresgid(rgid int, egid int, sgid int) (err error)
//sysnb Setresuid(ruid int, euid int, suid int) (err error) //sysnb Setresuid(ruid int, euid int, suid int) (err error)
+2 -2
View File
@@ -30,8 +30,8 @@ package unix
//sys Seek(fd int, offset int64, whence int) (off int64, err error) = SYS_LSEEK //sys Seek(fd int, offset int64, whence int) (off int64, err error) = SYS_LSEEK
//sys Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err error) //sys Select(nfd int, r *FdSet, w *FdSet, e *FdSet, timeout *Timeval) (n int, err error)
//sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error) //sys sendfile(outfd int, infd int, offset *int64, count int) (written int, err error)
//sys Setfsgid(gid int) (err error) //sys setfsgid(gid int) (prev int, err error)
//sys Setfsuid(uid int) (err error) //sys setfsuid(uid int) (prev int, err error)
//sysnb Setregid(rgid int, egid int) (err error) //sysnb Setregid(rgid int, egid int) (err error)
//sysnb Setresgid(rgid int, egid int, sgid int) (err error) //sysnb Setresgid(rgid int, egid int, sgid int) (err error)
//sysnb Setresuid(ruid int, euid int, suid int) (err error) //sysnb Setresuid(ruid int, euid int, suid int) (err error)
+3 -19
View File
@@ -106,23 +106,6 @@ func direntNamlen(buf []byte) (uint64, bool) {
return readInt(buf, unsafe.Offsetof(Dirent{}.Namlen), unsafe.Sizeof(Dirent{}.Namlen)) return readInt(buf, unsafe.Offsetof(Dirent{}.Namlen), unsafe.Sizeof(Dirent{}.Namlen))
} }
func SysctlClockinfo(name string) (*Clockinfo, error) {
mib, err := sysctlmib(name)
if err != nil {
return nil, err
}
n := uintptr(SizeofClockinfo)
var ci Clockinfo
if err := sysctl(mib, (*byte)(unsafe.Pointer(&ci)), &n, nil, 0); err != nil {
return nil, err
}
if n != SizeofClockinfo {
return nil, EIO
}
return &ci, nil
}
//sysnb pipe() (fd1 int, fd2 int, err error) //sysnb pipe() (fd1 int, fd2 int, err error)
func Pipe(p []int) (err error) { func Pipe(p []int) (err error) {
if len(p) != 2 { if len(p) != 2 {
@@ -270,6 +253,7 @@ func Statvfs(path string, buf *Statvfs_t) (err error) {
//sys Close(fd int) (err error) //sys Close(fd int) (err error)
//sys Dup(fd int) (nfd int, err error) //sys Dup(fd int) (nfd int, err error)
//sys Dup2(from int, to int) (err error) //sys Dup2(from int, to int) (err error)
//sys Dup3(from int, to int, flags int) (err error)
//sys Exit(code int) //sys Exit(code int)
//sys ExtattrGetFd(fd int, attrnamespace int, attrname string, data uintptr, nbytes int) (ret int, err error) //sys ExtattrGetFd(fd int, attrnamespace int, attrname string, data uintptr, nbytes int) (ret int, err error)
//sys ExtattrSetFd(fd int, attrnamespace int, attrname string, data uintptr, nbytes int) (ret int, err error) //sys ExtattrSetFd(fd int, attrnamespace int, attrname string, data uintptr, nbytes int) (ret int, err error)
@@ -295,7 +279,7 @@ func Statvfs(path string, buf *Statvfs_t) (err error) {
//sys Fpathconf(fd int, name int) (val int, err error) //sys Fpathconf(fd int, name int) (val int, err error)
//sys Fstat(fd int, stat *Stat_t) (err error) //sys Fstat(fd int, stat *Stat_t) (err error)
//sys Fstatat(fd int, path string, stat *Stat_t, flags int) (err error) //sys Fstatat(fd int, path string, stat *Stat_t, flags int) (err error)
//sys Fstatvfs1(fd int, buf *Statvfs_t) (err error) = SYS_FSTATVFS1 //sys Fstatvfs1(fd int, buf *Statvfs_t, flags int) (err error) = SYS_FSTATVFS1
//sys Fsync(fd int) (err error) //sys Fsync(fd int) (err error)
//sys Ftruncate(fd int, length int64) (err error) //sys Ftruncate(fd int, length int64) (err error)
//sysnb Getegid() (egid int) //sysnb Getegid() (egid int)
@@ -352,7 +336,7 @@ func Statvfs(path string, buf *Statvfs_t) (err error) {
//sysnb Settimeofday(tp *Timeval) (err error) //sysnb Settimeofday(tp *Timeval) (err error)
//sysnb Setuid(uid int) (err error) //sysnb Setuid(uid int) (err error)
//sys Stat(path string, stat *Stat_t) (err error) //sys Stat(path string, stat *Stat_t) (err error)
//sys Statvfs1(path string, buf *Statvfs_t) (err error) = SYS_STATVFS1 //sys Statvfs1(path string, buf *Statvfs_t, flags int) (err error) = SYS_STATVFS1
//sys Symlink(path string, link string) (err error) //sys Symlink(path string, link string) (err error)
//sys Symlinkat(oldpath string, newdirfd int, newpath string) (err error) //sys Symlinkat(oldpath string, newdirfd int, newpath string) (err error)
//sys Sync() (err error) //sys Sync() (err error)
+8 -20
View File
@@ -55,23 +55,6 @@ func direntNamlen(buf []byte) (uint64, bool) {
return readInt(buf, unsafe.Offsetof(Dirent{}.Namlen), unsafe.Sizeof(Dirent{}.Namlen)) return readInt(buf, unsafe.Offsetof(Dirent{}.Namlen), unsafe.Sizeof(Dirent{}.Namlen))
} }
func SysctlClockinfo(name string) (*Clockinfo, error) {
mib, err := sysctlmib(name)
if err != nil {
return nil, err
}
n := uintptr(SizeofClockinfo)
var ci Clockinfo
if err := sysctl(mib, (*byte)(unsafe.Pointer(&ci)), &n, nil, 0); err != nil {
return nil, err
}
if n != SizeofClockinfo {
return nil, EIO
}
return &ci, nil
}
func SysctlUvmexp(name string) (*Uvmexp, error) { func SysctlUvmexp(name string) (*Uvmexp, error) {
mib, err := sysctlmib(name) mib, err := sysctlmib(name)
if err != nil { if err != nil {
@@ -89,16 +72,20 @@ func SysctlUvmexp(name string) (*Uvmexp, error) {
return &u, nil return &u, nil
} }
//sysnb pipe(p *[2]_C_int) (err error)
func Pipe(p []int) (err error) { func Pipe(p []int) (err error) {
return Pipe2(p, 0)
}
//sysnb pipe2(p *[2]_C_int, flags int) (err error)
func Pipe2(p []int, flags int) error {
if len(p) != 2 { if len(p) != 2 {
return EINVAL return EINVAL
} }
var pp [2]_C_int var pp [2]_C_int
err = pipe(&pp) err := pipe2(&pp, flags)
p[0] = int(pp[0]) p[0] = int(pp[0])
p[1] = int(pp[1]) p[1] = int(pp[1])
return return err
} }
//sys Getdents(fd int, buf []byte) (n int, err error) //sys Getdents(fd int, buf []byte) (n int, err error)
@@ -248,6 +235,7 @@ func Uname(uname *Utsname) error {
//sys Close(fd int) (err error) //sys Close(fd int) (err error)
//sys Dup(fd int) (nfd int, err error) //sys Dup(fd int) (nfd int, err error)
//sys Dup2(from int, to int) (err error) //sys Dup2(from int, to int) (err error)
//sys Dup3(from int, to int, flags int) (err error)
//sys Exit(code int) //sys Exit(code int)
//sys Faccessat(dirfd int, path string, mode uint32, flags int) (err error) //sys Faccessat(dirfd int, path string, mode uint32, flags int) (err error)
//sys Fchdir(fd int) (err error) //sys Fchdir(fd int) (err error)
+2453
View File
File diff suppressed because it is too large Load Diff
+462 -2863
View File
File diff suppressed because it is too large Load Diff
+462 -2863
View File
File diff suppressed because it is too large Load Diff
+468 -2869
View File
File diff suppressed because it is too large Load Diff
+455 -2856
View File
File diff suppressed because it is too large Load Diff
+464 -2865
View File
File diff suppressed because it is too large Load Diff
+464 -2865
View File
File diff suppressed because it is too large Load Diff
+464 -2865
View File
File diff suppressed because it is too large Load Diff
+464 -2865
View File
File diff suppressed because it is too large Load Diff
+524 -2924
View File
File diff suppressed because it is too large Load Diff
+524 -2924
View File
File diff suppressed because it is too large Load Diff
+449 -2850
View File
File diff suppressed because it is too large Load Diff
+522 -2923
View File
File diff suppressed because it is too large Load Diff
+513 -2913
View File
File diff suppressed because it is too large Load Diff
+1825
View File
File diff suppressed because it is too large Load Diff
+6 -1819
View File
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More