Sample project to explore an interactive shell for hypermedia services
npm install @mamund/hyper_Exploring an interactive REPL/shell for interacting with HTTP-based hypermedia services_
Along with HTTP- and mediatype-aware commands, hyper also supports some convience functionality like SHELL commands, configuration file management, and a LIFO stack to handle local memory variabes.
Importantly, hyper is not just a shell/REPL, it is a hypermedia DSL. It encourages users to think' in hypermedia. Rather than writing complex HTTP queries that look like this (an example that works fine in hyper):
`
ACTIVATE http://localhost:8181/task/
WITH-METHOD PUT
WITH-BODY title=testing&tags=hyper&completeFlag=false
WITH-ENCODING application/x-www-form-urlencoded
WITH-HEADERS {"if-none-match":"*"}
The hyper shell can also use mediatype-aware convience commands to locate, parse, fill, and execute inline hypermedia controls. This results in a much more readable hyper exeperience:
`
STACK PUSH {
"title":"testing",
"tags":"hyper",
"completeFlag":"false"
}
ACTIVATE http://locahost:8181/home/
ACTIVATE WITH-FORM taskFormAdd WITH-STACK
`
In both cases, the same work is completed. In the first example, a human can read all the docs and examples and craft a successful HTTP PUT request. This works until the server changes a parameter (e.g. moves from PUT to POST).
In the second example, the hyper engine loads available data (it could have been from disk using STACK LOAD task-record.txt) and uses identified hypermedia controls (in this case the taskFormAdd control) to complete the request. This will continue to work even if HTTP details are changed (like changing PUT to POST)-- as long as the hypermedia form taskFormAdd is included in the response.
and command-line tools like curl. You can start a stateful client session by typing hyper at the command line. Then you can make an HTTP request (ACTIVATE) and manipulate the responses. You can also write hyper commands in a file and pipe this file into hyper for a scripted experience: (hyper < scripts/sample.txt > scripts/sample.log).Hyper is "mediatype-aware" -- that is, it recognizes well-known media types and offers convience methods for dealing with them. For example after loading a SIREN response, you can use the following commands:
`
SIREN example
GOTO http://rwcbook10.herokuapp.com
SIREN LINKS
SIREN ENTITIES
SIREN ACTIONSGOTO WITH-REL taskFormListByUser WITH-QUERY {"assignedUser" : "alice"}
That last command 1) uses the
associated with the SIREN action element identified by the rel:taskFormListByUser, 2) supplies a querystring argument and 3) makes the HTTP request.Another way to use hyper is to load the data stack with some name/value pairs and then use a named form within the response to execute an action. Like this:
`
read list
GOTO http://rwcbook10.herokuapp.comadd data to the stack and execute the write operation
STACK PUSH {"title":"just another one","tags":"with-test","completeFlag":"false"}
GOTO WITH-FORM taskFormAdd WITH-STACK check the write results using the same stack data
GOTO WITH-FORM taskFormListByTag WITH-STACK
SIREN PATH $..*[?(@property==='tags'&&@.match(/with-test/i))]^
NOTE: Spaces are significant in HYPER commands. The above example shows spaces are properly handed within embeded quotes (STACK PUSH {"title":"just another one","tags":"with-test","completeFlag":"false"}). However, there must be no spaces between elements in the JSON segments.
Note that the client will use whatever URL, HTTP method, and body encoding the server indicates. Also, notice that the client will automatically match up any form fields on the stack to fill in the form. Even when the server changes details (new URL, different method, etc.), the client will be able to handle the write operation without changes.
You can also use JSONPath to query responses:
SIREN PATH $.entities.*[?(@property==='id'&&@.match(/rmqzgqfq3d/i))]^.[id,title,href,type]
You can also use hyper to program a modificaiton of existing records:
REQUEST WITH-URL http://rwcbook10.herokuapp.com WITH-ACCEPT application/vnd.siren+json
REQUEST WITH-PATH $.entities[0].href WITH-ACCEPT application/vnd.siren+json
STACK PUSH WITH-PATH $.properties
STACK SET {"tags":"fishing,skiing,hiking"}
REQUEST WITH-FORM taskFormEdit WITH-STACK WITH-ACCEPT application/vnd.siren+json
EXITIn the above example, the hyper :
* Calls the root resource of the serivce asking for a SIREN formatted response
* Locates the first item in the collection, pulls its HREF value and calls that record
* Pushes the item properties of the record onto the local stack
* Updates the
value on the stack to reflect the change
* Uses the taskFormEdit form, fills it with values from the stack and makes the request
* Once all is done, the script exits
Similar options exist for HAL, CollectionJSON, JSON+FORMS, and other formats. These various format types are defined using external plug-ins that can be created and just dropped into the /plugins/ folder to be loaded at runtime.Examples
See the scripts folder for lots of working examples.Futures
Some notes on future enhancementsFeature tracking
This is a work in progress and totally unstable/unreliable. Here the current workplan and status for this project: - [x] : Initial CLI loop
- [x] : support for piped scripts (in and out)
- [x] : support for # - comment lines
- [x] : support for VERSION - returns version info
- [x] : support for EXIT|STOP - halt and exit with 0
- [ ] : support for EXIT-ERR - halt and exit with 1
- [x] : support for EXIT-IF - halt and exit with 1 if simple condition is met
- [x] : support for .. INVALID-URL __ : returns TRUE if the string is NOT a valid URL
- [x] : support for .. STACK-EMPTY : returns TRUE if there is nothing on the internal stack
- [x] : support for CLEAR - clears the console
- [x] : support for SHELL simple SHELL (bash/dos) support
- [x] : support for .. LS|DIR _[folder/path]_
- [x] : support for PLUGINS returns list of loaded external plug-ins
- [x] : support for CONFIG (READ) returns NVP of saved config data
- [x] : support for .. FILE|LOAD _[filename]_ loads config file (defaults to "hyper.config")
- [x] : support for .. SAVE|WRITE _[filename]_ loads config file (defaults to "hyper.config")
- [x] : support for .. SET _<{n:v,...}>_ shared config file write
- [x] : support for .. CLEAR removes all settings
- [x] : support for .. RESET resets to default settings
- [x] : support for .. REMOVE __ removes the named item
- [x] : support for STACK JSON object LIFO stack
- [x] : support for .. CLEAR|FLUSH clears all the items from the stack
- [x] : support for .. PEEK displays the JSON object at the stop of the stack
- [x] : support for .. PUSH _<{n:v,...}>_ adds a new JSON object to the stack
- [x] : support for .. PUSH WITH-REPONSE adds a new item on the stack from the top of the _response_ stack
- [x] : support for .. PUSH WITH-PATH __ adds a new item on the stack which is the result of the JSONPATH
- [x] : support for .. EXPAND-ARRAY expands the array on the top of the stack into n-items on the stack.
- [x] : support for .. POP removes the top item from the stack
- [x] : support for .. LEN|LENGTH returns depth of the stack
- [x] : support for .. SET _<{"n":"v",...}>_ update the JSON object on the top of the stack
- [x] : support for .. LOAD|FILE _[filename]_ reads a single JSON object from disk onto the stack (defaults to hyper.stack)
- [x] : support for .. SAVE|WRITE _[filename]_ writes the top item on the stack to disk (defaults to hyper.stack)
- [x] : support for .. DUMP _[filename]_ writes the full stack to disk (defaults to hyper.dump)
- [x] : support for .. FILL _[filename]_ replaces the current stack with contents in disk file (defaults to hyper.dump)
- [x] : support for OAUTH OAuth 2.0 support
- [x] : support for .. LOAD _[filename]_ loads OAuth config file (defaults to "oauth.env")
- [x] : support for .. SAVE _[filename]_ loads OAuth config file (defaults to "oauth.env")
- [x] : support for .. DEFINE __ _<{n:v,...}>_ Creates an entry in the OAuth configuration
- [x] : support for .. UPDATE __ _<{n:v,...}>_ Modifies settings in an existing OAuth configuration
- [x] : support for .. GENERATE|GEN __ Generates an access token using the configuration data
- [x] : support for .. REMOVE __ removes the named item
- [x] : support for ACTIVATE|CALL|GOTO|GO - makes an HTTP request
- [x] : support for .. WITH-URL __ - uses URL to make the request
- [x] : support for .. WITH-REL __ - uses HREF value on the associated in-doc element
- [x] : support for .. WITH-ID __ - uses HREF value on the associated in-doc element
- [x] : support for .. WITH-NAME __ - uses HREF value on the associated in-doc element
- [x] : support for .. WITH-PATH __ - uses value from JSONPath result as the URL
- [x] : support for .. WITH-ACCEPT _string|$#>_ - sets the accept header directly
- [x] : support for .. WITH-HEADERS _<{n:v,...}|$#>_ - request headers
- [x] : support for .. WITH-OAUTH __ - Sets the
authorization header using the named OAUTH token
- [x] : support for .. WITH-BASIC __ - Sets the authorization header using the named BASIC config
- [x] : support for .. WITH-QUERY _<{n:v,...}|$#>_ - query string args as JSON nvps
- [x] : support for .. WITH-BODY __ - for POST/PUT/PATCH (defaults to app/form-urlencoded)
- [x] : support for .. WITH-METHOD __ - to set HTTP method (defaults to GET)
- [x] : support for .. WITH-ENCODING __ - to set custom encoding for POST/PUT/PATCH
- [x] : support for .. WITH-FORMAT - sets accept header w/ config value
- [x] : support for .. WITH-PROFILE - sets link profile header w/ config value
- [x] : support for .. WITH-FORM __ - uses the metadata of the named form (URL, METHOD, ENCODING, FIELDS) to construct an HTTP request (SIREN-ONLY)
- [x] : support for .. WITH-STACK - uses the top level STACK item as a set of vars for other operations (e.g. to fill in forms, supply querystring values, headers, etc.
- [x] : support for .. WITH-DATA __ - for use fill in forms with ad-hoc data
- [x] : support for DISPLAY|SHOW - show saved reponse (from top of the LIFO stack)
- [x] : support for .. ALL - returns the complete interaction (request, response metadata, response body)
- [x] : support for .. REQUEST - returns request info (URL, method, querystring, body, headers)
- [x] : support for .. METADATA|META - returns the response metadata (URL, status, & headers)
- [x] : support for .. RESPONSE|PEEK - returns response body from the top of the stack
- [x] : support for .. URL|HREF - returns actual URL of the response
- [x] : support for .. STATUS|STATUS-CODE - returns HTTP status of the response
- [x] : support for .. CONTENT-TYPE - returns HTTP content-type of the response
- [x] : support for .. HEADERS - returns the complete HTTP header collection of the response
- [x] : support for .. POP remove response from top of the stack
- [x] : support for .. CLEAR|FLUSH - remove all responses from the stack
- [x] : support for .. LEN|LENGTH - returns length of saved stack
- [x] : support for .. PATH __ returns results of a query (xml or json)from top-of-stack response
- [x] : support for .. JPATH __ returns results of an JSONPath query from top-of-stack response
- [x] : support for .. XPATH __ returns results of an XPath query from top-of-stack response
- [x] : support for CJ returns a strong-typed version of response from top of the stack ( vnd.collection+json)
- [x] : support for .. METADATA returns metadata array from a collection+JSON response
- [x] : support for .. LINKS returns links array from a collection+JSON response
- [x] : support for .. ITEMS returns items array from a collection+JSON response
- [x] : support for .. QUERIES returns queries array from a collection+JSON response
- [x] : support for .. TEMPLATE returns template collection from a collection+JSON response
- [x] : support for .. ERROR|ERRORS returns error object from a collection+JSON response
- [x] : support for .. RELATED returns the related object from a collection+JSON response
- [x] : support for .. ID|NAME|REL|TAG __ returns a single node
- [ ] : support for .. IDS|RELS|NAMES|TAGS returns a simple list
- [x] : support for .. PATH __ returns results of a JSONPath query from a collection+JSON response
- [x] : support for HAL returns a strong-typed version of response from top of the stack ( vnd.hal+json)
- [x] : support for .. LINKS returns links array from a HAL response
- [x] : support for .. EMBEDDED returns items array from a HAL response
- [x] : support for .. ID|REL|KEY|NAME|TAG __ returns a single node
- [ ] : support for .. IDS|RELS|KEYSTAGS returns a simple list
- [x] : support for .. PATH __ returns results of a JSONPath query from a HAL response
- [x] : support for SIREN returns a strong-typed version of response from top of the stack ( vnd.siren+json)
- [x] : support for .. LINKS returns links array from a SIREN response
- [x] : support for .. ACTIONS|FORMS returns actions array from a SIREN response
- [x] : support for .. ENTITIES returns entities array from a SIREN response
- [x] : support for .. PROPERTIES returns properties array from a SIREN response
- [x] : support for .. TAG|CLASS __ returns nodes associated with the CLASS value
- [x] : support for .. ID|ENTITY __ returns an entity associated with the ID
- [x] : support for .. REL|LINK __ returns a link associated with the REL
- [x] : support for .. NAME|FORM|ACTION __ returns an action associated with the NAME
- [ ] : support for .. IDS|RELS|NAMES|FORMS|TAGS|CLASSES returns a simple list
- [x] : support for .. PATH __ returns results of a JSONPath query from a SIREN response
- [x] : support for WSTL returns a strong-typed version of response from top of the stack ( vnd.wstl+json)
- [x] : support for .. TITLE returns title string from a WSTL response
- [x] : support for .. ACTIONS returns actions array from a WSTL response
- [x] : support for .. DATA returns entities array from a WSTL response
- [x] : support for .. RELATED returns related object from a WSTL response
- [x] : support for .. CONTENT returns content object from a WSTL response
- [x] : support for .. ID|REL|NAME|FORM|TAG|TARGET __ returns a single node
- [ ] : support for .. IDS|RELS|NAMES|FORMS|TAGS|TARGETS returns a simple list
- [x] : support for .. PATH __ returns results of a JSONPath query from a WSTL response
- [x] : support for FJ returns a strong-typed version of JSON+FORMS response from top of the stack ( forms+json)
- [x] : support for .. METADATA returns metadata array from a response
- [x] : support for .. LINKS returns links array from a response
- [x] : support for .. ITEMS returns items array from a response
- [x] : support for .. ID __ returns an element (metadata, link, item) associated with the ID
- [x] : support for .. TAG __ returns matching nodes
- [x] : support for .. REL __ returns a link associated with the REL
- [x] : support for .. NAME __ returns an element (metadata, link, property) associated with the NAME
- [x] : support for .. IDS|NAMES|RELS|FORMS|TAGS returns a simple list
- [x] : support for .. PATH __ returns results of a JSONPath query from a response
- [ ] : support for MASH returns a strong-typed version of response from top of the stack ( vnd.mash+json)
- [ ] : support for .. METADATA returns metadata array from a response
- [ ] : support for .. LINKS returns links array from a response
- [ ] : support for .. ITEMS returns items array from a response
- [ ] : support for .. TAG __ returns matching nodes
- [ ] : support for .. ID __ returns an element (metadata, link, item) associated with the ID
- [ ] : support for .. REL __ returns a link associated with the REL
- [ ] : support for .. NAME __ returns an element (metadata, link, property) associated with the NAME
- [ ] : support for .. IDS|NAMES|RELS|FORMS|TAGS returns a simple list
- [ ] : support for .. PATH __ returns results of a JSONPath query from a SIREN response
- [ ] : support for PRAG returns a strong-typed version of response from top of the stack ( vnd.prag+json)
- [ ] : support for .. METADATA returns metadata array from a PRAG response
- [ ] : support for .. LINKS returns links array from a PRAG response
- [ ] : support for .. ITEMS returns items array from a PRAG response
- [ ] : support for .. ID __ returns an element (metadata, link, item) associated with the ID
- [ ] : support for .. TAG __ returns matching nodes
- [ ] : support for .. REL __ returns a link associated with the REL
- [ ] : support for .. NAME __ returns an element (metadata, link, property) associated with the NAME
- [ ] : support for .. IDS|NAMES|RELS|FORMS|TAGS returns a simple list
- [ ] : support for .. PATH __ returns results of a JSONPath query from a SIREN response
TODO Items
Here's a list of things I think need to be done before #HyperLang (as I like to call it) is "complete": - [ ] : improved support for VERBOSE setting, possibly levels. See noting (0), see status (1), see errors (2)
- [ ] : support for URITemplates - required for HAL (and other formats?)
- [ ] : support for HAL-FORMS - could greatly enhance HAL support
- [ ] : support for IF-ERROR - error checking (
4xx, 5xx)
- [ ] : support for JUMP _{label}_ - jump to defined label in the script (might be forward-only jumping)
- [ ] : Loops : Do we really want to implement loops? If yes, then it MUST be a single line element. Like list comprehensions in Python. For example WHILE STACK NOT EMPTY ACTIVATE WITH-FORM taskAddForm WITH-STACK POP.
- [ ] : Branching : Would like to avoid branching but might consider JUMP EXIT or JUMP :label|line (much harder)
Dependencies
These modules are used in the hyper app. * https://github.com/ForbesLindesay/sync-request
* https://www.npmjs.com/package/jsonpath-plus
* https://www.npmjs.com/package/stack-lifo
* https://www.npmjs.com/package/html2json
* https://www.npmjs.com/package/glob
* https://www.npmjs.com/package/valid-url
* https://www.npmjs.com/package/xmldom
* https://www.npmjs.com/package/xpath
Plug-ins Support
You an author your own hyper plug-in and place it in the /plugins/` folder of the project. It will be automatically loaded at runtime. See Plug-In Authoring for details.WARNING: This is all proof-of-concept code and it's pretty messy. I spend time exploring new features much more than I do properly grooming the source code. If you're offended by this kind of behavior, don't look behind the curtain on this one -- I'll only disappoint you[grin]. -- @mamund