diff --git a/.gitignore b/.gitignore index f6d3a04..372dd5d 100644 --- a/.gitignore +++ b/.gitignore @@ -3,3 +3,6 @@ __pycache__ output build .vscode + +docs/build +docs/Pipfile.lock diff --git a/README.rst b/README.rst index b0723f7..49c0219 100644 --- a/README.rst +++ b/README.rst @@ -1,134 +1,44 @@ -.. image:: https://trello-attachments.s3.amazonaws.com/5c824318411d973812cbef67/5ca1af818bc9b53e31696de3/f51eb40412bf09c8c800511d7bbe5634/kapow-1601675_480.png - :alt: Kapow! - -.. image:: https://circleci.com/gh/BBVA/kapow/tree/master.svg?style=svg - :target: https://circleci.com/gh/BBVA/kapow/tree/master +Welcome to *Kapow!* +=================== .. image:: https://goreportcard.com/badge/github.com/bbva/kapow :target: https://goreportcard.com/report/github.com/bbva/kapow -**Kapow!** If you can script it, you can HTTP it. + +With *Kapow!* you can publish simple **shell scripts** as **HTTP services** easily. + +*Kapow!* with an example +------------------------ + +**Goal** + +We want users on the Internet to be able to ``ping`` an *Internal Host* +which is inside a private network. + +.. image:: https://github.com/BBVA/kapow/raw/feature-new-doc/docs/source/_static/network.png + +**Limitations** + +- We can't allow users to log into any host. + +- We need to have full control over the precise command is run as + well as the parameters used. + +**Solution** + +With a *Kapow!* one-liner you can allow your users to run a command inside +*External Host* through an HTTP call. + +.. image:: https://github.com/BBVA/kapow/raw/feature-new-doc/docs/source/_static/sequence.png + +This is the only line you'll need: + +.. code-block:: bash + + $ kapow route add /ping -c 'ping -c1 10.10.10.100 | kapow set /response/body' -CAVEAT EMPTOR -============= +.. todo:: -**Warning!!! Kapow!** is under **heavy development** and `specification `_; -the provided code is a Proof of Concept and the final version will not even -share programming language. Ye be warned. + Mention license and contributing - -What is it? -=========== - -Kapow! is an adapter between the world of Pure UNIX® Shell and a HTTP service. - -Some tasks are more convenient in the shell, like cloud interactions, or some -administrative tools. On the other hand, some tasks are more convenient as a -service, like DevSecOps tooling. - -Kapow! lies between these two worlds, making your life easier. Maybe you wonder -about how this kind of magic can happen; if you want to know the nitty-gritty -details, just read our `specification `_;. Or, if you want to know how -Kapow! can help you first, let's start with a common situation. - -Think about that awesome command that you use every day, something very -familiar, like ``cloudx storage ls /backups``. Then someone asks you for an -specific backup, so you ``ssh`` into the host, execute your command, possibly -``grepping`` through its output, copy the result and send it back to him. -And that's fine... for the 100 first times. - -Then you decide, let's use an API for this and generate an awesome web server -with it. So, you create a project, manage its dependencies, code the server, -parse the request, learn how to use the API, call the API and deploy it -somewhere. And that's fine... until you find yourself again in the same -situation with another awesome command. - -The awesomeness of UNIX® commands is infinite, so you'll be in this situation -an infinite number of times! Instead, let's put Kapow! into action. - -With Kapow! you just need to create a ``.pow`` file named ``backups.pow`` that -contains: - -.. code-block:: sh - - kapow route add /backups \ - -c 'cloudx storage ls /backups | grep "$(kapow get /request/params/query)" | kapow set /response/body' - -and execute it in the remote host with the command: - -.. code-block:: sh - - kapow server backups.pow - -and that's it. Done. You have a web server that people can use to request -their backups every time they need only by invoking the URL -`http://remotehost/backups?query=project` - -Do you like it? yes? Then let's start learning a little more, you can access -the `documentation `_; section to find installation instructions and some -examples. - - - - - - - -How it was born ---------------- - -Some awesome history is coming. - - -Kapow! for the impatient -======================== - -When you need to **share** a ``command`` but **not** a complete remote ``ssh -access``, Kapow! will help you with the power of HTTP: - -.. image:: https://trello-attachments.s3.amazonaws.com/5c824318411d973812cbef67/5ca1af818bc9b53e31696de3/784a183fba3f24872dd97ee28e765922/Kapow!.png - :alt: Where Kapow! lives - -Kapow! allows you to write a litte script that will **serve an executable as REST -service**. This script will let you define how to connect HTTP and the Shell -using Kapow!'s shell abstractions to the HTTP world. See it to believe: - -.. image:: resources/kapow.gif?raw=true - :alt: Kapow! in action - - -Superpowers ------------ - -Kapow! gives you: - -* A very simple way to turn any shell **executable into an API** -* A **remote administration** API -* A way to define the integration in you own terms, obligations-free! - - -Curses ------- - -Kapow! can't help when: ------------------------ - -* You need high throughput: Kapow! spawns a new executable for every HTTP call -* You must perform complex logic to attend the request: never use Kapow! if - your executables don't perform al least 90% of the hard work -* You are building a huge application - - -When it is your best friend: --------------------------- - -* Easy command + Hard API = Kapow! to the rescue -* SSH for one command? Kapow! allows you to share only that command -* Remote instrumentation of several machines? Make it easy with Kapow! - - -The more you know -================= - -If you want to know more, please follow our `documentation `_. diff --git a/doc/README.md b/doc/README.md deleted file mode 100644 index acc8529..0000000 --- a/doc/README.md +++ /dev/null @@ -1,226 +0,0 @@ -# Installing Kapow! - -Kapow! has a reference implementation in Go that is under active development right -now. If you want to start using Kapow! you can: -* Download a binary (linux, at this moment) from our -[releases](https://github.com/BBVA/kapow/releases) section -* Install the package with the `get` command (you need the Go runtime installed -and [configured](https://golang.org/cmd/go/)) -```sh -go get -u github.com/BBVA/kapow -``` - - -# Examples - -Below are some examples on how to define and invoke routes in Kapow! - -As you will see `kapow` binary is both a server and a CLI that you can use to configure -a running server. The server exposes an [API](/spec#http-control-api) that you -can use directly if you want. - -In order to get information from the request that fired the script execution and -to help you compose the response, the server exposes -some [resources](/spec#handlers) to interact with from the script. - - -## The mandatory Hello World (for WWW fans) - -First, you create a pow file named `greet.pow` with the following contents: -```sh -kapow route add /greet -c 'name=$(kapow get /request/params/name); echo Hello ${name:-World} | kapow set /response/body' -``` - -note that you have to escape it as the command will run on a shell itself. Then, you -execute: -```sh -kapow server greet.pow -``` - -to start a Kapow! server exposing your service. Now you can check that it works -as intended with good ole’ `curl`: -```sh -curl localhost:8080/greet -Hello World - -curl localhost:8080/greet?name=friend -Hello friend -``` - -If you want to work with JSON you can use this version of the pow -`greet-json.pow` -```sh -kapow route add -X POST /greet -c 'who=$(kapow get /request/body | jq -r .name); kapow set /response/status 201; jq --arg value "${who:-World}" -n \{name:\$value\} | kapow set /response/body' -``` - -that uses [jq](https://stedolan.github.io/jq/) to allow you to work with JSON -from the command line. Check that it works with -```sh -curl -X POST -H 'Content-Type: application/json' -d '{"name": "friend"}' localhost:8080/greet -{"name": "friend" } - -curl -X POST -H 'Content-Type: application/json' -d '' localhost:8080/greet -{"name": "World"} -``` - - -## The mandatory Echo (for UNIX fans) - -First, you create a pow file named `echo.pow` with the following contents: -```sh -kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/body' -``` - -then, you execute: -```sh -kapow server echo.pow -``` - -and you can check that it works as intended with good ole’ `curl`: -```sh -curl -X POST -d '1,2,3... testing' localhost:8080/echo -1, 2, 3, 4, 5, 6, 7, 8, 9, testing -``` - -If you send a big file and want to see the content back as a real-time stream -you can use this version `echo-stream.pow` -```sh -kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/stream' -``` - - -## The multiline fun - -Unless you're a hardcore Perl golfer, you'll probably need to write your stuff -over more than one line in order to avoid the mess we saw on our JSON greet -version. - -Don't worry, we need to write several lines, too. Bash, in its magnificent -UNIX® style, provides us with the -[here-documents](https://www.gnu.org/software/bash/manual/bash.html#Here-Documents) -mechanism that we can leverage precisely for this purpose. - -Imagine that we want to return both the standard output and a generated file from a -command execution. Let's write a `log-and-stuff.pow` file with the following content: -```sh -kapow route add /log_and_stuff - <<-'EOF' - echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body - cat log.txt | kapow set /response/body -EOF -``` - -then we serve it with `kapow`: -```sh -kapow server log-and-stuff.pow -``` - -Yup. As simple as that. You can check it. -```sh -curl localhost:8080/log_and_stuff -this is a quite long sentence and other stuff -this is a quite long sentence and other stuff -``` - - -## Interact with other systems - -You can leverage all the power of the shell in your scripts and interact with -other systems by using all the available tools. Write a -`log-and-stuff-callback.pow` file with the following content: -```sh -kapow route add /log_and_stuff - <<-'EOF' - callback_url="$(kapow get /request/params/callback)" - echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body - echo sending to $callback_url | kapow set /response/body - curl -X POST --data-binary @log.txt $callback_url | kapow set /response/body -EOF -``` - -serve it with `kapow`: -```sh -kapow server log-and-stuff-callback.pow -``` - -and finally check it. -```sh -curl localhost:8080/log_and_stuff?callback=nowhere.com -this is a quite long sentence and other stuff -sending to nowhere.com - -405 Not Allowed - -

405 Not Allowed

-
nginx
- - -``` - -You must be aware that you must have all the dependencies you use in your -scripts installed in the host that will run the Kapow! server. - -In addition, a pow file can contain as many routes as you like, so you can start -a server with several routes configured in one shot. - - -# Sample Docker usage - -## Clone the project - -```sh -git clone https://github.com/BBVA/kapow.git -``` - - -## Build the kapow! docker image - -```sh -make docker -``` - -Now you have a container image with all the above pow files copied in /tmp so -you can start each example by running -```sh -docker run --rm -p 8080:8080 docker server example.pow -``` - - -## Build a docker image for running the nmap example -```sh -cd /path/to/kapow/poc/examples/nmap; docker build -t kapow-nmap . -``` - - -## Run kapow -```sh -docker run \ - -d \ - -p 8080:8080 \ - kapow-nmap -``` -which will output something like this: -```sh -e7da20c7d9a39624b5c56157176764671e5d2d8f1bf306b3ede898d66fe3f4bf -``` - - -## Test /list endpoint - -In another terminal, try running: -```sh -curl http://localhost:8080/list/github.com -``` - -which will respond something like: -```sh -Starting Nmap 7.70 ( https://nmap.org ) at 2019-05-10 14:01 UTC -Nmap scan report for github.com (140.82.118.3) -rDNS record for 140.82.118.3: lb-140-82-118-3-ams.github.com -Nmap done: 1 IP address (0 hosts up) scanned in 0.04 seconds -``` - -et voilà ! - - -# License - -This project is distributed under the [Apache License 2.0](/LICENSE). diff --git a/doc/working_with_forms.rst b/doc/working_with_forms.rst new file mode 100644 index 0000000..41d3946 --- /dev/null +++ b/doc/working_with_forms.rst @@ -0,0 +1,16 @@ +Working with Forms +================== + +When a browser submits a form to a server all the values included in the +form are sent to the server in an HTTP call. + +Kapow! handles the form decoding for you, the only thing you need to +know is the **name** of the *field* or *fields*. + +In this example we respond back with the content of the form field +``myfield``: + +.. code-block:: bash + + kapow get /request/form/myfield | kapow set /response/body + diff --git a/docs/Makefile b/docs/Makefile new file mode 100644 index 0000000..d0c3cbf --- /dev/null +++ b/docs/Makefile @@ -0,0 +1,20 @@ +# Minimal makefile for Sphinx documentation +# + +# You can set these variables from the command line, and also +# from the environment for the first two. +SPHINXOPTS ?= +SPHINXBUILD ?= sphinx-build +SOURCEDIR = source +BUILDDIR = build + +# Put it first so that "make" without argument is like "make help". +help: + @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) + +.PHONY: help Makefile + +# Catch-all target: route all unknown targets to Sphinx using the new +# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). +%: Makefile + @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) diff --git a/docs/Pipfile b/docs/Pipfile new file mode 100644 index 0000000..7c726e3 --- /dev/null +++ b/docs/Pipfile @@ -0,0 +1,13 @@ +[[source]] +name = "pypi" +url = "https://pypi.org/simple" +verify_ssl = true + +[dev-packages] + +[packages] +sphinx = "*" +sphinx-rtd-theme = "*" + +[requires] +python_version = "3.8" diff --git a/docs/make.bat b/docs/make.bat new file mode 100644 index 0000000..6247f7e --- /dev/null +++ b/docs/make.bat @@ -0,0 +1,35 @@ +@ECHO OFF + +pushd %~dp0 + +REM Command file for Sphinx documentation + +if "%SPHINXBUILD%" == "" ( + set SPHINXBUILD=sphinx-build +) +set SOURCEDIR=source +set BUILDDIR=build + +if "%1" == "" goto help + +%SPHINXBUILD% >NUL 2>NUL +if errorlevel 9009 ( + echo. + echo.The 'sphinx-build' command was not found. Make sure you have Sphinx + echo.installed, then set the SPHINXBUILD environment variable to point + echo.to the full path of the 'sphinx-build' executable. Alternatively you + echo.may add the Sphinx directory to PATH. + echo. + echo.If you don't have Sphinx installed, grab it from + echo.http://sphinx-doc.org/ + exit /b 1 +) + +%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% +goto end + +:help +%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% + +:end +popd diff --git a/docs/source/_static/browser.png b/docs/source/_static/browser.png new file mode 100644 index 0000000..a466a63 Binary files /dev/null and b/docs/source/_static/browser.png differ diff --git a/docs/source/_static/logo.png b/docs/source/_static/logo.png new file mode 100644 index 0000000..148bd92 Binary files /dev/null and b/docs/source/_static/logo.png differ diff --git a/docs/source/_static/network.png b/docs/source/_static/network.png new file mode 100644 index 0000000..3f5d222 Binary files /dev/null and b/docs/source/_static/network.png differ diff --git a/docs/source/_static/sequence.png b/docs/source/_static/sequence.png new file mode 100644 index 0000000..6cdaa82 Binary files /dev/null and b/docs/source/_static/sequence.png differ diff --git a/docs/source/cheatsheet.rst b/docs/source/cheatsheet.rst new file mode 100644 index 0000000..e69de29 diff --git a/docs/source/conf.py b/docs/source/conf.py new file mode 100644 index 0000000..8020ecb --- /dev/null +++ b/docs/source/conf.py @@ -0,0 +1,62 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) + + +# -- Project information ----------------------------------------------------- + +project = 'Kapow!' +copyright = '2019, BBVA Innovation Labs' +author = 'BBVA Innovation Labs' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'sphinx.ext.todo', +] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = [] + + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = "sphinx_rtd_theme" +html_logo = "_static/logo.png" +html_theme_options = { + 'logo_only': True, + 'collapse_navigation': False, + 'navigation_depth': 4, + 'includehidden': True, + 'titles_only': False + +} + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] diff --git a/docs/source/index.rst b/docs/source/index.rst new file mode 100644 index 0000000..b39d9b3 --- /dev/null +++ b/docs/source/index.rst @@ -0,0 +1,23 @@ +.. include:: ../../README.rst + +.. toctree:: + :maxdepth: 2 + :caption: TODO + + install + quickstart + +.. toctree:: + :maxdepth: 2 + :caption: Tutorial + + tutorial + + + +Indices and tables +================== + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` diff --git a/docs/source/install.rst b/docs/source/install.rst new file mode 100644 index 0000000..a6670d1 --- /dev/null +++ b/docs/source/install.rst @@ -0,0 +1,17 @@ +Installation +============ + +Precompiled Binaries +-------------------- + +1. Get a precompiled static binary for your system from our `releases section `_. + +2. Put it in your ``$PATH`` (Linux example): + + .. code-block:: bash + + sudo install kapow1.0.0-rc1_linux_amd64 /usr/bin/kapow + +.. todo:: + + Describe all the options available. diff --git a/docs/source/quickstart.rst b/docs/source/quickstart.rst new file mode 100644 index 0000000..c4c3cb2 --- /dev/null +++ b/docs/source/quickstart.rst @@ -0,0 +1,27 @@ +Quickstart Guide +================ + + +Hello World! -- *Kapow!* style +------------------------------ + +In a shell, the traditional `Hello World!` program would be ``echo "Hello World!"``. +Let's publish it through HTTP using *Kapow!* + +- First you need a *script file* with the route that will publish your command, lets's call the file ``greet.sh`` and should contain the following code: + +.. code-block:: bash + + kapow route add /greet -c 'echo "Hello World!" | kapow set /response/body' + +- Start the *Kapow!* server with your script as an argument + +.. code-block:: bash + + kapow server greet.sh + +- Finally check that all is working as intended: + +.. image:: _static/browser.png + + diff --git a/docs/source/tutorial.rst b/docs/source/tutorial.rst new file mode 100644 index 0000000..b40ad1e --- /dev/null +++ b/docs/source/tutorial.rst @@ -0,0 +1,28 @@ + +.. todo:: + + Redo + +Hello World! -- *Kapow!* style +------------------------------ + +In a shell, the traditional `Hello World!` program would be ``echo "Hello World!"``. +Let's publish it through HTTP using *Kapow!* + +- First you need a *script file* with the route that will publish your command, lets's call the file ``greet.sh`` and should contain the following code: + +.. code-block:: bash + + kapow route add /greet -c 'echo "Hello World!" | kapow set /response/body' + +- Start the *Kapow!* server with your script as an argument + +.. code-block:: bash + + kapow server greet.sh + +- Finally check that all is working as intended: + +.. image:: _static/browser.png + +