Merge branch 'feature-new-doc'

2019-11-19 16:37:22 +01:00
parent 83e4523dc7 12106338b0
commit 5462dd4323
17 changed files with 280 additions and 352 deletions
@@ -3,3 +3,6 @@ __pycache__
 output
 build
 .vscode
 docs/build
 docs/Pipfile.lock
@@ -1,134 +1,44 @@
-.. image:: https://trello-attachments.s3.amazonaws.com/5c824318411d973812cbef67/5ca1af818bc9b53e31696de3/f51eb40412bf09c8c800511d7bbe5634/kapow-1601675_480.png
+Welcome to *Kapow!*
-    :alt: Kapow!
+===================
 .. image:: https://circleci.com/gh/BBVA/kapow/tree/master.svg?style=svg
    :target: https://circleci.com/gh/BBVA/kapow/tree/master
 .. image:: https://goreportcard.com/badge/github.com/bbva/kapow
    :target: https://goreportcard.com/report/github.com/bbva/kapow
-**Kapow!** If you can script it, you can HTTP it.
+
 With *Kapow!* you can publish simple **shell scripts** as **HTTP services** easily.
 *Kapow!* with an example
 ------------------------
 **Goal**
 We want users on the Internet to be able to ``ping`` an *Internal Host*
 which is inside a private network.
 .. image:: https://github.com/BBVA/kapow/raw/feature-new-doc/docs/source/_static/network.png
 **Limitations**
 - We can't allow users to log into any host. 
 - We need to have full control over the precise command is run as
  well as the parameters used.
 **Solution**
 With a *Kapow!* one-liner you can allow your users to run a command inside
 *External Host* through an HTTP call.
 .. image:: https://github.com/BBVA/kapow/raw/feature-new-doc/docs/source/_static/sequence.png
 This is the only line you'll need:
 .. code-block:: bash
   $ kapow route add /ping -c 'ping -c1 10.10.10.100 | kapow set /response/body'
-CAVEAT EMPTOR
+.. todo::
 =============
-**Warning!!! Kapow!** is under **heavy development** and `specification </spec>`_;
+   Mention license and contributing
 the provided code is a Proof of Concept and the final version will not even
 share programming language.  Ye be warned.
 What is it?
 ===========
 Kapow! is an adapter between the world of Pure UNIX® Shell and a HTTP service.
 Some tasks are more convenient in the shell, like cloud interactions, or some
 administrative tools.  On the other hand, some tasks are more convenient as a
 service, like DevSecOps tooling.
 Kapow! lies between these two worlds, making your life easier.  Maybe you wonder
 about how this kind of magic can happen; if you want to know the nitty-gritty
 details, just read our `specification </spec>`_;.  Or, if you want to know how
 Kapow! can help you first, let's start with a common situation.
 Think about that awesome command that you use every day, something very
 familiar, like ``cloudx storage ls /backups``.  Then someone asks you for an
 specific backup, so you ``ssh`` into the host, execute your command, possibly
 ``grepping`` through its output, copy the result and send it back to him.
 And that's fine... for the 100 first times.
 Then you decide, let's use an API for this and generate an awesome web server
 with it.  So, you create a project, manage its dependencies, code the server,
 parse the request, learn how to use the API, call the API and deploy it
 somewhere.  And that's fine... until you find yourself again in the same
 situation with another awesome command.
 The awesomeness of UNIX® commands is infinite, so you'll be in this situation
 an infinite number of times!  Instead, let's put Kapow! into action.
 With Kapow! you just need to create a ``.pow`` file named ``backups.pow`` that
 contains:
 .. code-block:: sh
    kapow route add /backups \
        -c 'cloudx storage ls /backups | grep "$(kapow get /request/params/query)" | kapow set /response/body'
 and execute it in the remote host with the command:
 .. code-block:: sh
    kapow server backups.pow
 and that's it.  Done.  You have a web server that people can use to request
 their backups every time they need only by invoking the URL
 `http://remotehost/backups?query=project`
 Do you like it? yes?  Then let's start learning a little more, you can access
 the `documentation </doc>`_; section to find installation instructions and some
 examples.
 How it was born
 ---------------
 Some awesome history is coming.
 Kapow! for the impatient
 ========================
 When you need to **share** a ``command`` but **not** a complete remote ``ssh
 access``, Kapow!  will help you with the power of HTTP:
 .. image:: https://trello-attachments.s3.amazonaws.com/5c824318411d973812cbef67/5ca1af818bc9b53e31696de3/784a183fba3f24872dd97ee28e765922/Kapow!.png
    :alt: Where Kapow! lives
 Kapow! allows you to write a litte script that will **serve an executable as REST
 service**.  This script will let you define how to connect HTTP and the  Shell
 using Kapow!'s shell abstractions to the HTTP world. See it to believe:
 .. image:: resources/kapow.gif?raw=true
    :alt: Kapow! in action
 Superpowers
 -----------
 Kapow! gives you:
 * A very simple way to turn any shell **executable into an API**
 * A **remote administration** API
 * A way to define the integration in you own terms, obligations-free!
 Curses
 ------
 Kapow! can't help when:
 -----------------------
 * You need high throughput: Kapow! spawns a new executable for every HTTP call
 * You must perform complex logic to attend the request: never use Kapow! if
  your executables don't perform al least 90% of the hard work
 * You are building a huge application
 When it is your best friend:
 --------------------------
 * Easy command + Hard API = Kapow! to the rescue
 * SSH for one command?  Kapow! allows you to share only that command
 * Remote instrumentation of several machines?  Make it easy with Kapow!
 The more you know
 =================
 If you want to know more, please follow our `documentation </doc>`_.
@@ -1,226 +0,0 @@
 # Installing Kapow!
 Kapow! has a reference implementation in Go that is under active development right
 now.  If you want to start using Kapow! you can:
 * Download a binary (linux, at this moment) from our
 [releases](https://github.com/BBVA/kapow/releases) section
 * Install the package with the `get` command  (you need the Go runtime installed
 and [configured](https://golang.org/cmd/go/))
 ```sh
 go get -u github.com/BBVA/kapow
 ```
 # Examples
 Below are some examples on how to define and invoke routes in Kapow!
 As you will see `kapow` binary is both a server and a CLI that you can use to configure
 a running server.  The server exposes an [API](/spec#http-control-api) that you
 can use directly if you want.
 In order to get information from the request that fired the script execution and
 to help you compose the response, the server exposes
 some [resources](/spec#handlers) to interact with from the script.
 ## The mandatory Hello World (for WWW fans)
 First, you create a pow file named `greet.pow` with the following contents:
 ```sh
 kapow route add /greet -c 'name=$(kapow get /request/params/name); echo Hello ${name:-World} | kapow set /response/body'
 ```
 note that you have to escape it as the command will run on a shell itself.  Then, you
 execute:
 ```sh
 kapow server greet.pow
 ```
 to start a Kapow! server exposing your service.  Now you can check that it works
 as intended with good ole’ `curl`:
 ```sh
 curl localhost:8080/greet
 Hello World
 curl localhost:8080/greet?name=friend
 Hello friend
 ```
 If you want to work with JSON you can use this version of the pow
 `greet-json.pow`
 ```sh
 kapow route add -X POST /greet -c 'who=$(kapow get /request/body | jq -r .name); kapow set /response/status 201; jq --arg value "${who:-World}" -n \{name:\$value\} | kapow set /response/body'
 ```
 that uses [jq](https://stedolan.github.io/jq/) to allow you to work with JSON
 from the command line.  Check that it works with
 ```sh
 curl -X POST -H 'Content-Type: application/json' -d '{"name": "friend"}' localhost:8080/greet
 {"name": "friend" }
 curl -X POST -H 'Content-Type: application/json' -d '' localhost:8080/greet
 {"name": "World"}
 ```
 ## The mandatory Echo (for UNIX fans)
 First, you create a pow file named `echo.pow` with the following contents:
 ```sh
 kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/body'
 ```
 then, you execute:
 ```sh
 kapow server echo.pow
 ```
 and you can check that it works as intended with good ole’ `curl`:
 ```sh
 curl -X POST -d '1,2,3... testing' localhost:8080/echo
 1, 2, 3, 4, 5, 6, 7, 8, 9, testing
 ```
 If you send a big file and want to see the content back as a real-time stream
 you can use this version `echo-stream.pow`
 ```sh
 kapow route add -X POST /echo -c 'kapow get /request/body | kapow set /response/stream'
 ```
 ## The multiline fun
 Unless you're a hardcore Perl golfer, you'll probably need to write your stuff
 over more than one line in order to avoid the mess we saw on our JSON greet
 version.
 Don't worry, we need to write several lines, too.  Bash, in its magnificent
 UNIX® style, provides us with the
 [here-documents](https://www.gnu.org/software/bash/manual/bash.html#Here-Documents)
 mechanism that we can leverage precisely for this purpose.
 Imagine that we want to return both the standard output and a generated file from a
 command execution.  Let's write a `log-and-stuff.pow` file with the following content:
 ```sh
 kapow route add /log_and_stuff - <<-'EOF'
 	echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body
 	cat log.txt | kapow set /response/body
 EOF
 ```
 then we serve it with `kapow`:
 ```sh
 kapow server log-and-stuff.pow
 ```
 Yup.  As simple as that.  You can check it.
 ```sh
 curl localhost:8080/log_and_stuff
 this is a quite long sentence and other stuff
 this is a quite long sentence and other stuff
 ```
 ## Interact with other systems
 You can leverage all the power of the shell in your scripts and interact with
 other systems by using all the available tools.  Write a
 `log-and-stuff-callback.pow` file with the following content:
 ```sh
 kapow route add /log_and_stuff - <<-'EOF'
 	callback_url="$(kapow get /request/params/callback)"
 	echo this is a quite long sentence and other stuff | tee log.txt | kapow set /response/body
 	echo sending to $callback_url | kapow set /response/body
 	curl -X POST --data-binary @log.txt $callback_url | kapow set /response/body
 EOF
 ```
 serve it with `kapow`:
 ```sh
 kapow server log-and-stuff-callback.pow
 ```
 and finally check it.
 ```sh
 curl localhost:8080/log_and_stuff?callback=nowhere.com
 this is a quite long sentence and other stuff
 sending to nowhere.com
 <html>
 <head><title>405 Not Allowed</title></head>
 <body>
 <center><h1>405 Not Allowed</h1></center>
 <hr><center>nginx</center>
 </body>
 </html>
 ```
 You must be aware that you must have all the dependencies you use in your
 scripts installed in the host that will run the Kapow! server.
 In addition, a pow file can contain as many routes as you like, so you can start
 a server with several routes configured in one shot.
 # Sample Docker usage
 ## Clone the project
 ```sh
 git clone https://github.com/BBVA/kapow.git
 ```
 ## Build the kapow! docker image
 ```sh
 make docker
 ```
 Now you have a container image with all the above pow files copied in /tmp so
 you can start each example by running
 ```sh
 docker run --rm -p 8080:8080 docker server example.pow
 ```
 ## Build a docker image for running the nmap example
 ```sh
 cd /path/to/kapow/poc/examples/nmap; docker build -t kapow-nmap .
 ```
 ## Run kapow
 ```sh
 docker run \
 	-d \
 	-p 8080:8080 \
 	kapow-nmap
 ```
 which will output something like this:
 ```sh
 e7da20c7d9a39624b5c56157176764671e5d2d8f1bf306b3ede898d66fe3f4bf
 ```
 ## Test /list endpoint
 In another terminal, try running:
 ```sh
 curl http://localhost:8080/list/github.com
 ```
 which will respond something like:
 ```sh
 Starting Nmap 7.70 ( https://nmap.org ) at 2019-05-10 14:01 UTC
 Nmap scan report for github.com (140.82.118.3)
 rDNS record for 140.82.118.3: lb-140-82-118-3-ams.github.com
 Nmap done: 1 IP address (0 hosts up) scanned in 0.04 seconds
 ```
 et voilà !
 # License
 This project is distributed under the [Apache License 2.0](/LICENSE).
@@ -0,0 +1,16 @@
 Working with Forms
 ==================
 When a browser submits a form to a server all the values included in the
 form are sent to the server in an HTTP call.
 Kapow! handles the form decoding for you, the only thing you need to
 know is the **name** of the *field* or *fields*.
 In this example we respond back with the content of the form field
 ``myfield``:
 .. code-block:: bash
   kapow get /request/form/myfield | kapow set /response/body
@@ -0,0 +1,20 @@
 # Minimal makefile for Sphinx documentation
 #
 # You can set these variables from the command line, and also
 # from the environment for the first two.
 SPHINXOPTS    ?=
 SPHINXBUILD   ?= sphinx-build
 SOURCEDIR     = source
 BUILDDIR      = build
 # Put it first so that "make" without argument is like "make help".
 help:
 	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
 .PHONY: help Makefile
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile
 	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
@@ -0,0 +1,13 @@
 [[source]]
 name = "pypi"
 url = "https://pypi.org/simple"
 verify_ssl = true
 [dev-packages]
 [packages]
 sphinx = "*"
 sphinx-rtd-theme = "*"
 [requires]
 python_version = "3.8"
@@ -0,0 +1,35 @@
@ECHO OFF
 pushd %~dp0
 REM Command file for Sphinx documentation
 if "%SPHINXBUILD%" == "" (
 	set SPHINXBUILD=sphinx-build
 )
 set SOURCEDIR=source
 set BUILDDIR=build
 if "%1" == "" goto help
 %SPHINXBUILD% >NUL 2>NUL
 if errorlevel 9009 (
 	echo.
 	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
 	echo.installed, then set the SPHINXBUILD environment variable to point
 	echo.to the full path of the 'sphinx-build' executable. Alternatively you
 	echo.may add the Sphinx directory to PATH.
 	echo.
 	echo.If you don't have Sphinx installed, grab it from
 	echo.http://sphinx-doc.org/
 	exit /b 1
 )
 %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
 goto end
 :help
 %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
 :end
 popd
@@ -0,0 +1,62 @@
 # Configuration file for the Sphinx documentation builder.
 #
 # This file only contains a selection of the most common options. For a full
 # list see the documentation:
 # https://www.sphinx-doc.org/en/master/usage/configuration.html
 # -- Path setup --------------------------------------------------------------
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
 # import os
 # import sys
 # sys.path.insert(0, os.path.abspath('.'))
 # -- Project information -----------------------------------------------------
 project = 'Kapow!'
 copyright = '2019, BBVA Innovation Labs'
 author = 'BBVA Innovation Labs'
 # -- General configuration ---------------------------------------------------
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
    'sphinx.ext.todo',
 ]
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
 exclude_patterns = []
 # -- Options for HTML output -------------------------------------------------
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
 html_theme = "sphinx_rtd_theme"
 html_logo = "_static/logo.png"
 html_theme_options = {
    'logo_only': True,
    'collapse_navigation': False,
    'navigation_depth': 4,
    'includehidden': True,
    'titles_only': False
 }
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ['_static']
@@ -0,0 +1,23 @@
 .. include:: ../../README.rst
 .. toctree::
   :maxdepth: 2
   :caption: TODO
   install
   quickstart
 .. toctree::
   :maxdepth: 2
   :caption: Tutorial
   tutorial
 Indices and tables
 ==================
 * :ref:`genindex`
 * :ref:`modindex`
 * :ref:`search`
@@ -0,0 +1,17 @@
 Installation
 ============
 Precompiled Binaries
 --------------------
 1. Get a precompiled static binary for your system from our `releases section <https://github.com/BBVA/kapow/releases/latest>`_.
 2. Put it in your ``$PATH`` (Linux example):
   .. code-block:: bash
      sudo install kapow1.0.0-rc1_linux_amd64 /usr/bin/kapow
 .. todo::
   Describe all the options available.
@@ -0,0 +1,27 @@
 Quickstart Guide
 ================
 Hello World! -- *Kapow!* style
 ------------------------------
 In a shell, the traditional `Hello World!` program would be ``echo "Hello World!"``.
 Let's publish it through HTTP using *Kapow!*
 - First you need a *script file* with the route that will publish your command, lets's call the file ``greet.sh`` and should contain the following code:
 .. code-block:: bash
    kapow route add /greet -c 'echo "Hello World!" | kapow set /response/body'
 - Start the *Kapow!* server with your script as an argument
 .. code-block:: bash
    kapow server greet.sh
 - Finally check that all is working as intended:
 .. image:: _static/browser.png
@@ -0,0 +1,28 @@
 .. todo::
   Redo 
 Hello World! -- *Kapow!* style
 ------------------------------
 In a shell, the traditional `Hello World!` program would be ``echo "Hello World!"``.
 Let's publish it through HTTP using *Kapow!*
 - First you need a *script file* with the route that will publish your command, lets's call the file ``greet.sh`` and should contain the following code:
 .. code-block:: bash
    kapow route add /greet -c 'echo "Hello World!" | kapow set /response/body'
 - Start the *Kapow!* server with your script as an argument
 .. code-block:: bash
    kapow server greet.sh
 - Finally check that all is working as intended:
 .. image:: _static/browser.png