Dark-Alex-17/kapow
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'feature-new-doc'

Browse Source
This commit is contained in:
Roberto Abdelkader Martínez Pérez
2019-11-19 16:37:22 +01:00
parent 83e4523dc7 12106338b0
commit 5462dd4323
17 changed files with 280 additions and 352 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+3
View File
@@ -3,3 +3,6 @@ __pycache__
output
build
.vscode
docs/build
docs/Pipfile.lock
README.rst
+36 -126
View File
@@ -1,134 +1,44 @@
.. image:: https://trello-attachments.s3.amazonaws.com/5c824318411d973812cbef67/5ca1af818bc9b53e31696de3/f51eb40412bf09c8c800511d7bbe5634/kapow-1601675_480.png
:alt: Kapow!
.. image:: https://circleci.com/gh/BBVA/kapow/tree/master.svg?style=svg
:target: https://circleci.com/gh/BBVA/kapow/tree/master
Welcome to *Kapow!*
===================
.. image:: https://goreportcard.com/badge/github.com/bbva/kapow
:target: https://goreportcard.com/report/github.com/bbva/kapow
**Kapow!** If you can script it, you can HTTP it.
With *Kapow!* you can publish simple **shell scripts** as **HTTP services** easily.
*Kapow!* with an example
------------------------
**Goal**
We want users on the Internet to be able to ``ping`` an *Internal Host*
which is inside a private network.
.. image:: https://github.com/BBVA/kapow/raw/feature-new-doc/docs/source/_static/network.png
**Limitations**
- We can't allow users to log into any host.
- We need to have full control over the precise command is run as
well as the parameters used.
**Solution**
With a *Kapow!* one-liner you can allow your users to run a command inside
*External Host* through an HTTP call.
.. image:: https://github.com/BBVA/kapow/raw/feature-new-doc/docs/source/_static/sequence.png
This is the only line you'll need:
.. code-block:: bash
$ kapow route add /ping -c 'ping -c1 10.10.10.100 | kapow set /response/body'
CAVEAT EMPTOR
=============
.. todo::
**Warning!!! Kapow!** is under **heavy development** and `specification </spec>`_;
the provided code is a Proof of Concept and the final version will not even
share programming language. Ye be warned.
Mention license and contributing
What is it?
===========
Kapow! is an adapter between the world of Pure UNIX® Shell and a HTTP service.
Some tasks are more convenient in the shell, like cloud interactions, or some
administrative tools. On the other hand, some tasks are more convenient as a
service, like DevSecOps tooling.
Kapow! lies between these two worlds, making your life easier. Maybe you wonder
about how this kind of magic can happen; if you want to know the nitty-gritty
details, just read our `specification </spec>`_;. Or, if you want to know how
Kapow! can help you first, let's start with a common situation.
Think about that awesome command that you use every day, something very
familiar, like ``cloudx storage ls /backups``. Then someone asks you for an
specific backup, so you ``ssh`` into the host, execute your command, possibly
``grepping`` through its output, copy the result and send it back to him.
And that's fine... for the 100 first times.
Then you decide, let's use an API for this and generate an awesome web server
with it. So, you create a project, manage its dependencies, code the server,
parse the request, learn how to use the API, call the API and deploy it
somewhere. And that's fine... until you find yourself again in the same
situation with another awesome command.
The awesomeness of UNIX® commands is infinite, so you'll be in this situation
an infinite number of times! Instead, let's put Kapow! into action.
With Kapow! you just need to create a ``.pow`` file named ``backups.pow`` that
contains:
.. code-block:: sh
kapow route add /backups \
-c 'cloudx storage ls /backups | grep "$(kapow get /request/params/query)" | kapow set /response/body'
and execute it in the remote host with the command:
.. code-block:: sh
kapow server backups.pow
and that's it. Done. You have a web server that people can use to request
their backups every time they need only by invoking the URL
`http://remotehost/backups?query=project`
Do you like it? yes? Then let's start learning a little more, you can access
the `documentation </doc>`_; section to find installation instructions and some
examples.
How it was born
---------------
Some awesome history is coming.
Kapow! for the impatient
========================
When you need to **share** a ``command`` but **not** a complete remote ``ssh
access``, Kapow! will help you with the power of HTTP:
.. image:: https://trello-attachments.s3.amazonaws.com/5c824318411d973812cbef67/5ca1af818bc9b53e31696de3/784a183fba3f24872dd97ee28e765922/Kapow!.png
:alt: Where Kapow! lives
Kapow! allows you to write a litte script that will **serve an executable as REST
service**. This script will let you define how to connect HTTP and the Shell
using Kapow!'s shell abstractions to the HTTP world. See it to believe:
.. image:: resources/kapow.gif?raw=true
:alt: Kapow! in action
Superpowers
-----------
Kapow! gives you:
* A very simple way to turn any shell **executable into an API**
* A **remote administration** API
* A way to define the integration in you own terms, obligations-free!
Curses
------
Kapow! can't help when:
-----------------------
* You need high throughput: Kapow! spawns a new executable for every HTTP call
* You must perform complex logic to attend the request: never use Kapow! if
your executables don't perform al least 90% of the hard work
* You are building a huge application
When it is your best friend:
--------------------------
* Easy command + Hard API = Kapow! to the rescue
* SSH for one command? Kapow! allows you to share only that command
* Remote instrumentation of several machines? Make it easy with Kapow!
The more you know
=================
If you want to know more, please follow our `documentation </doc>`_.
doc/README.md
-226
View File
@@ -1,226 +0,0 @@
# Installing Kapow!
Kapow! has a reference implementation in Go that is under active development right
now. If you want to start using Kapow! you can:
* Download a binary (linux, at this moment) from our
[releases](https://github.com/BBVA/kapow/releases) section
* Install the package with the `get` command (you need the Go runtime installed
and [configured](https://golang.org/cmd/go/))
```sh
go get -u github.com/BBVA/kapow
```
# Examples
Below are some examples on how to define and invoke routes in Kapow!
As you will see `kapow` binary is both a server and a CLI that you can use to configure
a running server. The server exposes an [API](/spec#http-control-api) that you
can use directly if you want.
In order to get information from the request that fired the script execution and
to help you compose the response, the server exposes
some [resources](/spec#handlers) to interact with from the script.
## The mandatory Hello World (for WWW fans)
First, you create a pow file named `greet.pow` with the following contents:
```sh
kapow route add /greet -c 'name=$(kapow get /request/params/name); echo Hello ${name:-World} | kapow set /response/body'
```
note that you have to escape it as the command will run on a shell itself. Then, you
execute:
```sh
kapow server greet.pow
```
to start a Kapow! server exposing your service. Now you can check that it works
as intended with good ole’ `curl`:
```sh
curl localhost:8080/greet
Hello World
curl localhost:8080/greet?name=friend
Hello friend
```
If you want to work with JSON you can use this version of the pow
`greet-json.pow`
```sh
kapow route add -X POST /greet -c 'who=$(kapow get /request/body | jq -r .name); kapow set /response/status 201; jq --arg value "${who:-World}" -n \{name:\$value\} | kapow set /response/body'
```
that uses [jq](https://stedolan.github.io/jq/) to allow you to work with JSON
from the command line. Check that it works with
```sh
curl -X POST -H 'Content-Type: application/json' -d '{"name": "friend"}' localhost:8080/greet
{"name": "friend" }
curl -X POST -H 'Content-Type: application/json' -d '' localhost:8080/greet
{"name": "World"}
```
## The mandatory Echo (for UNIX fans)
First, you create a pow file named `echo.pow` with the following contents:
```sh
kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/body'
```
then, you execute:
```sh
kapow server echo.pow
```
and you can check that it works as intended with good ole’ `curl`:
```sh
curl -X POST -d '1,2,3... testing' localhost:8080/echo
1, 2, 3, 4, 5, 6, 7, 8, 9, testing
```
If you send a big file and want to see the content back as a real-time stream
you can use this version `echo-stream.pow`
```sh
kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/stream'
```
## The multiline fun
Unless you're a hardcore Perl golfer, you'll probably need to write your stuff
over more than one line in order to avoid the mess we saw on our JSON greet
version.
Don't worry, we need to write several lines, too. Bash, in its magnificent
UNIX® style, provides us with the
[here-documents](https://www.gnu.org/software/bash/manual/bash.html#Here-Documents)
mechanism that we can leverage precisely for this purpose.
Imagine that we want to return both the standard output and a generated file from a
command execution. Let's write a `log-and-stuff.pow` file with the following content:
```sh
kapow route add /log_and_stuff - <<-'EOF'
echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body
cat log.txt | kapow set /response/body
EOF
```
then we serve it with `kapow`:
```sh
kapow server log-and-stuff.pow
```
Yup. As simple as that. You can check it.
```sh
curl localhost:8080/log_and_stuff
this is a quite long sentence and other stuff
this is a quite long sentence and other stuff
```
## Interact with other systems
You can leverage all the power of the shell in your scripts and interact with
other systems by using all the available tools. Write a
`log-and-stuff-callback.pow` file with the following content:
```sh
kapow route add /log_and_stuff - <<-'EOF'
callback_url="$(kapow get /request/params/callback)"
echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body
echo sending to $callback_url | kapow set /response/body
curl -X POST --data-binary @log.txt $callback_url | kapow set /response/body
EOF
```
serve it with `kapow`:
```sh
kapow server log-and-stuff-callback.pow
```
and finally check it.
```sh
curl localhost:8080/log_and_stuff?callback=nowhere.com
this is a quite long sentence and other stuff
sending to nowhere.com
<html>
<head><title>405 Not Allowed</title></head>
<body>
<center><h1>405 Not Allowed</h1></center>
<hr><center>nginx</center>
</body>
</html>
```
You must be aware that you must have all the dependencies you use in your
scripts installed in the host that will run the Kapow! server.
In addition, a pow file can contain as many routes as you like, so you can start
a server with several routes configured in one shot.
# Sample Docker usage
## Clone the project
```sh
git clone https://github.com/BBVA/kapow.git
```
## Build the kapow! docker image
```sh
make docker
```
Now you have a container image with all the above pow files copied in /tmp so
you can start each example by running
```sh
docker run --rm -p 8080:8080 docker server example.pow
```
## Build a docker image for running the nmap example
```sh
cd /path/to/kapow/poc/examples/nmap; docker build -t kapow-nmap .
```
## Run kapow
```sh
docker run \
-d \
-p 8080:8080 \
kapow-nmap
```
which will output something like this:
```sh
e7da20c7d9a39624b5c56157176764671e5d2d8f1bf306b3ede898d66fe3f4bf
```
## Test /list endpoint
In another terminal, try running:
```sh
curl http://localhost:8080/list/github.com
```
which will respond something like:
```sh
Starting Nmap 7.70 ( https://nmap.org ) at 2019-05-10 14:01 UTC
Nmap scan report for github.com (140.82.118.3)
rDNS record for 140.82.118.3: lb-140-82-118-3-ams.github.com
Nmap done: 1 IP address (0 hosts up) scanned in 0.04 seconds
```
et voilà !
# License
This project is distributed under the [Apache License 2.0](/LICENSE).
doc/working_with_forms.rst
+16
View File
@@ -0,0 +1,16 @@
Working with Forms
==================
When a browser submits a form to a server all the values included in the
form are sent to the server in an HTTP call.
Kapow! handles the form decoding for you, the only thing you need to
know is the **name** of the *field* or *fields*.
In this example we respond back with the content of the form field
``myfield``:
.. code-block:: bash
kapow get /request/form/myfield | kapow set /response/body
docs/Makefile
+20
View File
@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line, and also
# from the environment for the first two.
SPHINXOPTS ?=
SPHINXBUILD ?= sphinx-build
SOURCEDIR = source
BUILDDIR = build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
docs/Pipfile
+13
View File
@@ -0,0 +1,13 @@
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
[packages]
sphinx = "*"
sphinx-rtd-theme = "*"
[requires]
python_version = "3.8"
docs/make.bat
+35
View File
@@ -0,0 +1,35 @@
@ECHO OFF
pushd %~dp0
REM Command file for Sphinx documentation
if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=source
set BUILDDIR=build
if "%1" == "" goto help
%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)
%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
goto end
:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
:end
popd
docs/source/_static/browser.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

docs/source/_static/logo.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 75 KiB

docs/source/_static/network.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 95 KiB

docs/source/_static/sequence.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 180 KiB

docs/source/cheatsheet.rst
View File
docs/source/conf.py
+62
View File
@@ -0,0 +1,62 @@
# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# https://www.sphinx-doc.org/en/master/usage/configuration.html
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
# -- Project information -----------------------------------------------------
project = 'Kapow!'
copyright = '2019, BBVA Innovation Labs'
author = 'BBVA Innovation Labs'
# -- General configuration ---------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.todo',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = []
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = "sphinx_rtd_theme"
html_logo = "_static/logo.png"
html_theme_options = {
'logo_only': True,
'collapse_navigation': False,
'navigation_depth': 4,
'includehidden': True,
'titles_only': False
}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
docs/source/index.rst
+23
View File
@@ -0,0 +1,23 @@
.. include:: ../../README.rst
.. toctree::
:maxdepth: 2
:caption: TODO
install
quickstart
.. toctree::
:maxdepth: 2
:caption: Tutorial
tutorial
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
docs/source/install.rst
+17
View File
@@ -0,0 +1,17 @@
Installation
============
Precompiled Binaries
--------------------
1. Get a precompiled static binary for your system from our `releases section <https://github.com/BBVA/kapow/releases/latest>`_.
2. Put it in your ``$PATH`` (Linux example):
.. code-block:: bash
sudo install kapow1.0.0-rc1_linux_amd64 /usr/bin/kapow
.. todo::
Describe all the options available.
docs/source/quickstart.rst
+27
View File
@@ -0,0 +1,27 @@
Quickstart Guide
================
Hello World! -- *Kapow!* style
------------------------------
In a shell, the traditional `Hello World!` program would be ``echo "Hello World!"``.
Let's publish it through HTTP using *Kapow!*
- First you need a *script file* with the route that will publish your command, lets's call the file ``greet.sh`` and should contain the following code:
.. code-block:: bash
kapow route add /greet -c 'echo "Hello World!" | kapow set /response/body'
- Start the *Kapow!* server with your script as an argument
.. code-block:: bash
kapow server greet.sh
- Finally check that all is working as intended:
.. image:: _static/browser.png
docs/source/tutorial.rst
+28
View File
@@ -0,0 +1,28 @@
.. todo::
Redo
Hello World! -- *Kapow!* style
------------------------------
In a shell, the traditional `Hello World!` program would be ``echo "Hello World!"``.
Let's publish it through HTTP using *Kapow!*
- First you need a *script file* with the route that will publish your command, lets's call the file ``greet.sh`` and should contain the following code:
.. code-block:: bash
kapow route add /greet -c 'echo "Hello World!" | kapow set /response/body'
- Start the *Kapow!* server with your script as an argument
.. code-block:: bash
kapow server greet.sh
- Finally check that all is working as intended:
.. image:: _static/browser.png