Dark-Alex-17/kapow
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

CLOSES: #77

Browse Source
This commit is contained in:
Héctor Hurtado
2019-10-31 15:22:47 +01:00
parent 84a106db19
commit 7c48eb730d
9 changed files with 319 additions and 186 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.rst
+58 -1
View File
@@ -3,7 +3,7 @@
.. image:: https://circleci.com/gh/BBVA/kapow/tree/master.svg?style=svg .. image:: https://circleci.com/gh/BBVA/kapow/tree/master.svg?style=svg
:target: https://circleci.com/gh/BBVA/kapow/tree/master :target: https://circleci.com/gh/BBVA/kapow/tree/master
**Kapow!** If you can script it, you can HTTP it. **Kapow!** If you can script it, you can HTTP it.
@@ -15,6 +15,63 @@ the provided code is a Proof of Concept and the final version will not even
share programming language. Ye be warned. share programming language. Ye be warned.
What is?
==============
Kapow! is an adapter between the world of Pure UNIX® Shell and a HTTP service.
Some tasks are more convenient in the shell, like cloud interactions, or some
administrative tools. On the other hand, some tasks are more convenient as a
service, like DevSecOps tooling.
Kapow! lies between these two worlds, making your life easier. Maybe you wonder
about how this kind of magic can happen; if you want to know the nitty-gritty
details, just read our `specification </spec>`_;. Or, if you want to know how
Kapow! can help you first, let's start with a common situation.
Think about that awesome command that you use every day, something very
familiar, like ``cloudx storage ls /backups``. Then someone asks you for an
specific backup, so you ``ssh`` into the host, execute your command, possibly
``grepping`` through its output, copy the result and send it back to him.
And that's fine... for the 100 first times.
Then you decide, let's use an API for this and generate an awesome web server
with it. So, you create a project, manage its dependencies, code the server,
parse the request, learn how to use the API, call the API and deploy it
somewhere. And that's fine... until you find yourself again in the same
situation with another awesome command.
The awesomeness of UNIX® commands is infinite, so you'll be in this situation
an infinite number of times! Instead, let's put Kapow! into action.
With Kapow! you just need to create a ``.pow`` file named ``backups.pow`` that
contains:
```Shell
kapow route add /backups \
-c 'cloudx storage ls /backups | grep $(kapow get /request/params/query) | kapow set /response/body'
```
and execute it in the remote host with the command:
```Shell
kapow server backups.pow
```
and that's it. Done. You have a web server that people can use to request
their backups every time they need only by invoking the URL
`http://remotehost/backups?query=project`
Do you like it? yes? Then let's start learning a little more, you can access
the `documentation </doc>`_; section to find installation instructions and some
examples.
How it was born How it was born
--------------- ---------------
doc/README.md
+247
View File
@@ -0,0 +1,247 @@
# Installing Kapow!
Kapow! has a reference implementation in Go that is under active develpment right
now. If you want to start using kapow you can:
* Download a binary (linux, at this moment) from our
[releases](https://github.com/BBVA/kapow/releases) section
* Install the package with the get command (you need the Go runtime installed
and [configured](https://golang.org/cmd/go/))
```Shell
go get -u github.com/BBVA/kapow
```
# Examples
Below are some examples on how to define and invoke routes in Kapow!
As you can see Kapow! binary is a server and a CLI that you can use to configure
a running server. The server exposes an [API](/spec#http-control-api) that you
can invoke directly if you want.
In order to get information from the request that fired the scrip execution and
to help you in composing the response, the server exposes
some [resources](/spec#handlers) to interact with from the script.
## The mandatory Hello World (for WWW fans)
First you create a pow file named `greet.pow` with the following contents:
```Shell
kapow route add /greet -c 'name=$(kapow get /request/params/name); echo Hello ${name:-World} | kapow set /response/body'
```
note you have to escape as the command will run on a shell itself. Then, you
execute:
```Shell
kapow server greet.pow
```
to start a Kapow! server exposing your service. Now you can check that it works
as intended with good ole' ``curl``:
```Shell
curl localhost:8080/greet
Hello World
curl localhost:8080/greet?name=friend
Hello friend
```
If you want to work with JSON you can use this version of the pow
`greet-json.pow`
```Shell
kapow route add -X POST /greet -c 'kapow route add -X POST /greet -c 'who=$(kapow get /request/body | jq -r .name); kapow set /response/status 201; jq --arg value "${who:-World}" -n \{name:\$value\} | kapow set /response/body''
```
that uses [jq](https://stedolan.github.io/jq/) to allow you working with json
from the command line. Check that it works with
```Shell
curl -X POST -H "Content-Type: application/json" -d '{"name": "friend"}' localhost:8080/greet
{"name": "friend" }
curl -X POST -H "Content-Type: application/json" -d '' localhost:8080/greet
{"name": "World"}
```
## The mandatory Echo (for UNIX fans)
First you create a pow file named `echo.pow` with the following contents:
```Shell
kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/body'
```
then, you execute:
```Shell
kapow server echo.pow
```
and you can check that it works as intended with good ole' `curl`:
```Shell
curl -X POST -d '1,2,3... testing' localhost:8080/echo
1, 2, 3, 4, 5, 6, 7, 8, 9, testing
```
If you send a big file and want to see the content back as a real-time stream
you can use this version `echo-stream.pow`
```Shell
kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/stream'
```
## The multiline fun
Unless you're a hardcore Perl hacker, you'll probably need to write your stuff
over more than one line in order to avoid the mess we saw on our json greet
version.
Don't worry, we need to write several lines, too. Bash, in its magnificent
UNIX® style, provides us with the
[here-documents](https://www.gnu.org/software/bash/manual/bash.html#Here-Documents)
mechanism that we can leverage precisely for this purpose.
Imagine we want to return both the standard output and a generated file from a
command execution. Let's write a `log-and-stuff.pow` file with the following content:
```Shell
kapow route add /log_and_stuff - <<- 'EOF'
echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body
cat log.txt | kapow set /response/body
EOF
```
then we serve it with `kapow`:
```Shell
kapow server log-and-stuff.pow
```
Yup. As simple as that. You can check it.
```Shell
curl localhost:8080/log_and_stuff
this is a quite long sentence and other stuff
this is a quite long sentence and other stuff
```
## Interact with other systems
You can leverage all the power of the shell in your scripts and interact with
other systems by using all the available tools. Write a
`log-and-stuff-callback.pow` file with the following content:
```Shell
kapow route add /log_and_stuff - <<- 'EOF'
callback_url=$(kapow get /request/params/callback)
echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body
echo sending to $callback_url | kapow set /response/body
curl -X POST --data-binary @log.txt $callback_url | kapow set /response/body
EOF
```
serve it with `kapow`:
```Shell
kapow server log-and-stuff-callback.pow
```
and finally check it.
```Shell
curl localhost:8080/log_and_stuff?callback=nowhere.com
this is a quite long sentence and other stuff
sending to nowhere.com
<html>
<head><title>405 Not Allowed</title></head>
<body>
<center><h1>405 Not Allowed</h1></center>
<hr><center>nginx</center>
</body>
</html>
```
You must be aware that you must have all the dependencies you use in your
scripts installed in the host that will run the Kapow! server.
In addition, a pow file can contain as much routes as you like so you can start
a server with several routes configured in one shoot.
# Sample Docker usage
## Clone the project
```Shell
git clone https://github.com/BBVA/kapow.git
```
## Build the kapow! docker image
```Shell
make docker
```
Now you have a container image with all the above pow files copied in /tmp so
you can start each example by running
```Shell
docker run --rm -p 8080:8080 docker server example.pow
```
## Build a docker image for running the nmap example
```Shell
cd /path/to/kapow/poc/examples/nmap; docker build -t kapow-nmap .
```
## Run kapow
```Shell
docker run \
-d \
-p 8080:8080 \
kapow-nmap
```
which will output something like this:
```Shell
e7da20c7d9a39624b5c56157176764671e5d2d8f1bf306b3ede898d66fe3f4bf
```
## Test /list endpoint
In another terminal, try running:
```Shell
curl http://localhost:8080/list/github.com
```
which will respond something like:
```Shell
Starting Nmap 7.70 ( https://nmap.org ) at 2019-05-10 14:01 UTC
Nmap scan report for github.com (140.82.118.3)
rDNS record for 140.82.118.3: lb-140-82-118-3-ams.github.com
Nmap done: 1 IP address (0 hosts up) scanned in 0.04 seconds
```
et voilà !
# License
This project is distributed under the [Apache License 2.0](/LICENSE).
doc/echo-stream.pow
+1
View File
@@ -0,0 +1 @@
kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/stream'
doc/echo.pow
+1
View File
@@ -0,0 +1 @@
kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/body'
doc/greet-json.pow
+1
View File
@@ -0,0 +1 @@
kapow route add -X POST /greet -c 'who=$(kapow get /request/body | jq -r .name); kapow set /response/status 201; jq --arg value "${who:-World}" -n \{name:\$value\} | kapow set /response/body'
doc/greet.pow
+1
View File
@@ -0,0 +1 @@
kapow route add /greet -c 'name=$(kapow get /request/params/name); echo Hello ${name:-World} | kapow set /response/body'
doc/log-and-stuff-callback.pow
+6
View File
@@ -0,0 +1,6 @@
kapow route add /log_and_stuff - <<- 'EOF'
callback_url=$(kapow get /request/params/callback)
echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body
echo sending to $callback_url | kapow set /response/body
curl -X POST --data-binary @log.txt $callback_url | kapow set /response/body
EOF
doc/log-and-stuff.pow
+4
View File
@@ -0,0 +1,4 @@
kapow route add /log_and_stuff - <<- 'EOF'
echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body
cat log.txt | kapow set /response/body
EOF
doc/readme.rst
-185
View File
@@ -1,185 +0,0 @@
What is Kapow!
==============
Kapow! is an adapter between the world of Pure UNIX® Shell and an HTTP service.
Some tasks are more convenient in the shell, like cloud interactions, or some
administrative tools. On the other hand, some tasks are more convenient as a
service, like DevSecOps tooling.
Kapow! lies between these two worlds, making your life easier. Maybe you wonder
about how this kind of magic can happen; if you want to know the nitty-gritty
details, just read our [spec](/spec/). Or, if you want to know how Kapow! can
help you first, let's start with a common situation.
Think about that awesome command that you use every day, something very
familiar, like ``cloudx storage ls /backups``. Then someone asks you for an
specific backup, so you ``ssh`` into the host, execute your command, possibly
``grepping`` through its output, copy the result and send it back to him.
And that's fine... for the 100 first times.
Then you decide, let's use an API for this and generate an awesome web server
with it. So, you create a project, manage its dependencies, code the server,
parse the request, learn how to use the API, call the API and deploy it
somewhere. And that's fine... until you find yourself again in the same
situation with another awesome command.
The awesomeness of UNIX® commands is infinite, so you'll be in this situation
an infinite number of times! Instead, let's put Kapow! into action.
With Kapow!, when someone asks you for an specific backup (remember your
familiar command?) you just need to create a ``.pow`` file named ``backups.pow``
that contains:
.. code-block:: bash
kapow route add /backups \
-c 'cloudx storage ls /backups | grep $(kapow get /request/params/query) | kapow set /response/body'
and execute it in the host with the command:
.. code-block:: bash
kapow server backups.pow
and that's it. Done. Do you like it? yes? Then let's start learning a little
more.
The mandatory Hello World (for WWW fans)
----------------------------------------
First you must create a pow file named ``hello.pow`` with the following contents:
.. code-block:: bash
kapow route add /greet -c "echo 'hello world' | kapow set /response/body"
then, you must execute:
.. code-block:: bash
kapow server hello.pow
and you can check that it works as intended with good ole' ``curl``:
.. code-block:: bash
curl localhost:8080/greet
The mandatory Echo (for UNIX fans)
----------------------------------
First you must create a pow file named ``echo.pow`` with the following contents:
.. code-block:: bash
kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/body'
then, you must execute:
.. code-block:: bash
kapow server echo.pow
and you can check that it works as intended with good ole ``curl``:
.. code-block:: bash
curl -X POST -d '1,2,3... testing' localhost:8080/echo
The multiline fun
-----------------
Unless you're a hardcore Perl hacker, you'll probably need to write your stuff
over more than one line.
Don't worry, we need to write several lines, too. Bash, in its magnificent
UNIX® style, provides us with the
`here-documents`_ mechanism that we can leverage precisely for this purpose.
.. _here-documents: https://www.gnu.org/software/bash/manual/bash.html#Here-Documents
Let's write a ``multiline.pow`` file with the following content:
.. code-block:: bash
kapow route add /log_and_love - <<- 'EOF'
echo "[$(date)] and stuff" >> stuff.log
echo love | kapow set /response/body
EOF
and then we serve it with ``kapow``:
.. code-block:: bash
kapow server multiline.pow
Yup. As simple as that.
Sample Docker usage
===================
Clone the project
-----------------
.. code-block:: bash
# clone this project
Build the kapow! docker image
-----------------------------
.. code-block:: bash
cd /path/to/kapow/poc; docker build -t bbva/kapow:0.1 .
Build a docker image for running the nmap example
.. code-block:: bash
cd /path/to/kapow/poc/examples/nmap; docker build -t kapow-nmap .
Run kapow
---------
.. code-block:: bash
docker run \
-d \
-p 8080:8080 \
kapow-nmap
which will output something like this:
.. code-block:: bash
e7da20c7d9a39624b5c56157176764671e5d2d8f1bf306b3ede898d66fe3f4bf
Test /list endpoint
-------------------
In another terminal, try running:
.. code-block:: bash
curl http://localhost:8080/list/github.com
which will respond something like:
.. code-block:: bash
Starting Nmap 7.70 ( https://nmap.org ) at 2019-05-10 14:01 UTC
Nmap scan report for github.com (140.82.118.3)
rDNS record for 140.82.118.3: lb-140-82-118-3-ams.github.com
Nmap done: 1 IP address (0 hosts up) scanned in 0.04 seconds
et voilà !
License
=======
This project is distributed under the [Apache License 2.0](/LICENSE).