diff --git a/README.rst b/README.rst index 4c6fca0..1c11d43 100644 --- a/README.rst +++ b/README.rst @@ -3,7 +3,7 @@ .. image:: https://circleci.com/gh/BBVA/kapow/tree/master.svg?style=svg :target: https://circleci.com/gh/BBVA/kapow/tree/master - + **Kapow!** If you can script it, you can HTTP it. @@ -15,6 +15,63 @@ the provided code is a Proof of Concept and the final version will not even share programming language. Ye be warned. +What is? +============== + +Kapow! is an adapter between the world of Pure UNIX® Shell and a HTTP service. + +Some tasks are more convenient in the shell, like cloud interactions, or some +administrative tools. On the other hand, some tasks are more convenient as a +service, like DevSecOps tooling. + +Kapow! lies between these two worlds, making your life easier. Maybe you wonder +about how this kind of magic can happen; if you want to know the nitty-gritty +details, just read our `specification `_;. Or, if you want to know how +Kapow! can help you first, let's start with a common situation. + +Think about that awesome command that you use every day, something very +familiar, like ``cloudx storage ls /backups``. Then someone asks you for an +specific backup, so you ``ssh`` into the host, execute your command, possibly +``grepping`` through its output, copy the result and send it back to him. +And that's fine... for the 100 first times. + +Then you decide, let's use an API for this and generate an awesome web server +with it. So, you create a project, manage its dependencies, code the server, +parse the request, learn how to use the API, call the API and deploy it +somewhere. And that's fine... until you find yourself again in the same +situation with another awesome command. + +The awesomeness of UNIX® commands is infinite, so you'll be in this situation +an infinite number of times! Instead, let's put Kapow! into action. + +With Kapow! you just need to create a ``.pow`` file named ``backups.pow`` that +contains: + +```Shell + kapow route add /backups \ + -c 'cloudx storage ls /backups | grep $(kapow get /request/params/query) | kapow set /response/body' +``` + +and execute it in the remote host with the command: + +```Shell + kapow server backups.pow +``` + +and that's it. Done. You have a web server that people can use to request +their backups every time they need only by invoking the URL +`http://remotehost/backups?query=project` + +Do you like it? yes? Then let's start learning a little more, you can access +the `documentation `_; section to find installation instructions and some +examples. + + + + + + + How it was born --------------- diff --git a/doc/README.md b/doc/README.md new file mode 100644 index 0000000..33b0ebb --- /dev/null +++ b/doc/README.md @@ -0,0 +1,247 @@ +# Installing Kapow! + +Kapow! has a reference implementation in Go that is under active develpment right +now. If you want to start using kapow you can: +* Download a binary (linux, at this moment) from our +[releases](https://github.com/BBVA/kapow/releases) section +* Install the package with the get command (you need the Go runtime installed +and [configured](https://golang.org/cmd/go/)) +```Shell + go get -u github.com/BBVA/kapow +``` + + +# Examples + +Below are some examples on how to define and invoke routes in Kapow! + +As you can see Kapow! binary is a server and a CLI that you can use to configure +a running server. The server exposes an [API](/spec#http-control-api) that you +can invoke directly if you want. + +In order to get information from the request that fired the scrip execution and +to help you in composing the response, the server exposes +some [resources](/spec#handlers) to interact with from the script. + + +## The mandatory Hello World (for WWW fans) + +First you create a pow file named `greet.pow` with the following contents: + +```Shell + kapow route add /greet -c 'name=$(kapow get /request/params/name); echo Hello ${name:-World} | kapow set /response/body' +``` + +note you have to escape as the command will run on a shell itself. Then, you +execute: + +```Shell + kapow server greet.pow +``` + +to start a Kapow! server exposing your service. Now you can check that it works +as intended with good ole' ``curl``: + +```Shell + curl localhost:8080/greet + Hello World + + curl localhost:8080/greet?name=friend + Hello friend +``` + +If you want to work with JSON you can use this version of the pow +`greet-json.pow` + +```Shell + kapow route add -X POST /greet -c 'kapow route add -X POST /greet -c 'who=$(kapow get /request/body | jq -r .name); kapow set /response/status 201; jq --arg value "${who:-World}" -n \{name:\$value\} | kapow set /response/body'' +``` + +that uses [jq](https://stedolan.github.io/jq/) to allow you working with json +from the command line. Check that it works with + +```Shell + curl -X POST -H "Content-Type: application/json" -d '{"name": "friend"}' localhost:8080/greet + {"name": "friend" } + + curl -X POST -H "Content-Type: application/json" -d '' localhost:8080/greet + {"name": "World"} +``` + + +## The mandatory Echo (for UNIX fans) + +First you create a pow file named `echo.pow` with the following contents: + +```Shell + kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/body' +``` + +then, you execute: + +```Shell + kapow server echo.pow +``` + +and you can check that it works as intended with good ole' `curl`: + +```Shell + curl -X POST -d '1,2,3... testing' localhost:8080/echo + 1, 2, 3, 4, 5, 6, 7, 8, 9, testing +``` + +If you send a big file and want to see the content back as a real-time stream +you can use this version `echo-stream.pow` + +```Shell + kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/stream' +``` + + +## The multiline fun + +Unless you're a hardcore Perl hacker, you'll probably need to write your stuff +over more than one line in order to avoid the mess we saw on our json greet +version. + +Don't worry, we need to write several lines, too. Bash, in its magnificent +UNIX® style, provides us with the +[here-documents](https://www.gnu.org/software/bash/manual/bash.html#Here-Documents) +mechanism that we can leverage precisely for this purpose. + +Imagine we want to return both the standard output and a generated file from a +command execution. Let's write a `log-and-stuff.pow` file with the following content: + +```Shell + kapow route add /log_and_stuff - <<- 'EOF' + echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body + cat log.txt | kapow set /response/body + EOF +``` + +then we serve it with `kapow`: + +```Shell + kapow server log-and-stuff.pow +``` + +Yup. As simple as that. You can check it. + +```Shell + curl localhost:8080/log_and_stuff + this is a quite long sentence and other stuff + this is a quite long sentence and other stuff +``` + + +## Interact with other systems + +You can leverage all the power of the shell in your scripts and interact with +other systems by using all the available tools. Write a +`log-and-stuff-callback.pow` file with the following content: + +```Shell + kapow route add /log_and_stuff - <<- 'EOF' + callback_url=$(kapow get /request/params/callback) + echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body + echo sending to $callback_url | kapow set /response/body + curl -X POST --data-binary @log.txt $callback_url | kapow set /response/body + EOF +``` + +serve it with `kapow`: + +```Shell + kapow server log-and-stuff-callback.pow +``` + +and finally check it. + +```Shell + curl localhost:8080/log_and_stuff?callback=nowhere.com + this is a quite long sentence and other stuff + sending to nowhere.com + + 405 Not Allowed + +

405 Not Allowed

+
nginx
+ + +``` + +You must be aware that you must have all the dependencies you use in your +scripts installed in the host that will run the Kapow! server. + +In addition, a pow file can contain as much routes as you like so you can start +a server with several routes configured in one shoot. + +# Sample Docker usage + + +## Clone the project + +```Shell + git clone https://github.com/BBVA/kapow.git +``` + + +## Build the kapow! docker image + +```Shell + make docker +``` + +Now you have a container image with all the above pow files copied in /tmp so +you can start each example by running + +```Shell + docker run --rm -p 8080:8080 docker server example.pow +``` + + +## Build a docker image for running the nmap example + +```Shell + cd /path/to/kapow/poc/examples/nmap; docker build -t kapow-nmap . +``` + +## Run kapow + +```Shell + docker run \ + -d \ + -p 8080:8080 \ + kapow-nmap +``` + +which will output something like this: + +```Shell + e7da20c7d9a39624b5c56157176764671e5d2d8f1bf306b3ede898d66fe3f4bf +``` + + +## Test /list endpoint + +In another terminal, try running: + +```Shell + curl http://localhost:8080/list/github.com +``` + +which will respond something like: + +```Shell + Starting Nmap 7.70 ( https://nmap.org ) at 2019-05-10 14:01 UTC + Nmap scan report for github.com (140.82.118.3) + rDNS record for 140.82.118.3: lb-140-82-118-3-ams.github.com + Nmap done: 1 IP address (0 hosts up) scanned in 0.04 seconds +``` + +et voilà ! + + +# License + +This project is distributed under the [Apache License 2.0](/LICENSE). diff --git a/doc/echo-stream.pow b/doc/echo-stream.pow new file mode 100644 index 0000000..a98ed1e --- /dev/null +++ b/doc/echo-stream.pow @@ -0,0 +1 @@ +kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/stream' diff --git a/doc/echo.pow b/doc/echo.pow new file mode 100644 index 0000000..a164a74 --- /dev/null +++ b/doc/echo.pow @@ -0,0 +1 @@ +kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/body' diff --git a/doc/greet-json.pow b/doc/greet-json.pow new file mode 100644 index 0000000..18671c0 --- /dev/null +++ b/doc/greet-json.pow @@ -0,0 +1 @@ +kapow route add -X POST /greet -c 'who=$(kapow get /request/body | jq -r .name); kapow set /response/status 201; jq --arg value "${who:-World}" -n \{name:\$value\} | kapow set /response/body' diff --git a/doc/greet.pow b/doc/greet.pow new file mode 100644 index 0000000..f423a62 --- /dev/null +++ b/doc/greet.pow @@ -0,0 +1 @@ +kapow route add /greet -c 'name=$(kapow get /request/params/name); echo Hello ${name:-World} | kapow set /response/body' diff --git a/doc/log-and-stuff-callback.pow b/doc/log-and-stuff-callback.pow new file mode 100644 index 0000000..372ce0e --- /dev/null +++ b/doc/log-and-stuff-callback.pow @@ -0,0 +1,6 @@ +kapow route add /log_and_stuff - <<- 'EOF' + callback_url=$(kapow get /request/params/callback) + echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body + echo sending to $callback_url | kapow set /response/body + curl -X POST --data-binary @log.txt $callback_url | kapow set /response/body +EOF diff --git a/doc/log-and-stuff.pow b/doc/log-and-stuff.pow new file mode 100644 index 0000000..6cc0b89 --- /dev/null +++ b/doc/log-and-stuff.pow @@ -0,0 +1,4 @@ +kapow route add /log_and_stuff - <<- 'EOF' + echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body + cat log.txt | kapow set /response/body +EOF diff --git a/doc/readme.rst b/doc/readme.rst deleted file mode 100644 index 15a761e..0000000 --- a/doc/readme.rst +++ /dev/null @@ -1,185 +0,0 @@ -What is Kapow! -============== - -Kapow! is an adapter between the world of Pure UNIX® Shell and an HTTP service. - -Some tasks are more convenient in the shell, like cloud interactions, or some -administrative tools. On the other hand, some tasks are more convenient as a -service, like DevSecOps tooling. - -Kapow! lies between these two worlds, making your life easier. Maybe you wonder -about how this kind of magic can happen; if you want to know the nitty-gritty -details, just read our [spec](/spec/). Or, if you want to know how Kapow! can -help you first, let's start with a common situation. - -Think about that awesome command that you use every day, something very -familiar, like ``cloudx storage ls /backups``. Then someone asks you for an -specific backup, so you ``ssh`` into the host, execute your command, possibly -``grepping`` through its output, copy the result and send it back to him. -And that's fine... for the 100 first times. - -Then you decide, let's use an API for this and generate an awesome web server -with it. So, you create a project, manage its dependencies, code the server, -parse the request, learn how to use the API, call the API and deploy it -somewhere. And that's fine... until you find yourself again in the same -situation with another awesome command. - -The awesomeness of UNIX® commands is infinite, so you'll be in this situation -an infinite number of times! Instead, let's put Kapow! into action. - -With Kapow!, when someone asks you for an specific backup (remember your -familiar command?) you just need to create a ``.pow`` file named ``backups.pow`` -that contains: - -.. code-block:: bash - - kapow route add /backups \ - -c 'cloudx storage ls /backups | grep $(kapow get /request/params/query) | kapow set /response/body' - -and execute it in the host with the command: - -.. code-block:: bash - - kapow server backups.pow - -and that's it. Done. Do you like it? yes? Then let's start learning a little -more. - - -The mandatory Hello World (for WWW fans) ----------------------------------------- - -First you must create a pow file named ``hello.pow`` with the following contents: - -.. code-block:: bash - - kapow route add /greet -c "echo 'hello world' | kapow set /response/body" - -then, you must execute: - -.. code-block:: bash - - kapow server hello.pow - -and you can check that it works as intended with good ole' ``curl``: - -.. code-block:: bash - - curl localhost:8080/greet - - -The mandatory Echo (for UNIX fans) ----------------------------------- - -First you must create a pow file named ``echo.pow`` with the following contents: - -.. code-block:: bash - - kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/body' - -then, you must execute: - -.. code-block:: bash - - kapow server echo.pow - -and you can check that it works as intended with good ole ``curl``: - -.. code-block:: bash - - curl -X POST -d '1,2,3... testing' localhost:8080/echo - - -The multiline fun ------------------ - -Unless you're a hardcore Perl hacker, you'll probably need to write your stuff -over more than one line. - -Don't worry, we need to write several lines, too. Bash, in its magnificent -UNIX® style, provides us with the -`here-documents`_ mechanism that we can leverage precisely for this purpose. - -.. _here-documents: https://www.gnu.org/software/bash/manual/bash.html#Here-Documents - -Let's write a ``multiline.pow`` file with the following content: - -.. code-block:: bash - - kapow route add /log_and_love - <<- 'EOF' - echo "[$(date)] and stuff" >> stuff.log - echo love | kapow set /response/body - EOF - -and then we serve it with ``kapow``: - -.. code-block:: bash - - kapow server multiline.pow - -Yup. As simple as that. - - -Sample Docker usage -=================== - -Clone the project ------------------ - -.. code-block:: bash - - # clone this project - - -Build the kapow! docker image ------------------------------ - -.. code-block:: bash - - cd /path/to/kapow/poc; docker build -t bbva/kapow:0.1 . - -Build a docker image for running the nmap example - -.. code-block:: bash - - cd /path/to/kapow/poc/examples/nmap; docker build -t kapow-nmap . - -Run kapow ---------- -.. code-block:: bash - - docker run \ - -d \ - -p 8080:8080 \ - kapow-nmap - -which will output something like this: - -.. code-block:: bash - - e7da20c7d9a39624b5c56157176764671e5d2d8f1bf306b3ede898d66fe3f4bf - - -Test /list endpoint -------------------- -In another terminal, try running: - -.. code-block:: bash - - curl http://localhost:8080/list/github.com - -which will respond something like: - -.. code-block:: bash - - Starting Nmap 7.70 ( https://nmap.org ) at 2019-05-10 14:01 UTC - Nmap scan report for github.com (140.82.118.3) - rDNS record for 140.82.118.3: lb-140-82-118-3-ams.github.com - Nmap done: 1 IP address (0 hosts up) scanned in 0.04 seconds - -et voilà ! - -License -======= - -This project is distributed under the [Apache License 2.0](/LICENSE).