From 5e9437fa412c82c78a153d359b0f0f76ca254df5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Abdelkader=20Mart=C3=ADnez=20P=C3=A9rez?= Date: Tue, 26 Nov 2019 07:41:43 +0100 Subject: [PATCH 1/4] Use code-blocks for bash fragments --- docs/source/tutorial/_brainstorm.rst | 115 +++++++++++++++++---------- 1 file changed, 72 insertions(+), 43 deletions(-) diff --git a/docs/source/tutorial/_brainstorm.rst b/docs/source/tutorial/_brainstorm.rst index 34c9eb4..ad4518b 100644 --- a/docs/source/tutorial/_brainstorm.rst +++ b/docs/source/tutorial/_brainstorm.rst @@ -45,21 +45,27 @@ User Journey - Problem/Motivation: Each time an ACME project is finished it is desirable to make a backup of the entire database. Given that the database server is a critical machine we don't want to grant SSH - access to lowly developers. The script is very fast because the + access to lowly developers. The script is fast because the database is small (for now). - pre-Kapow! solution: Launching the script via SSH shell. - ``` - ssh user@server - $ ./backup_db.sh - ``` + + .. code-block:: console + + $ ssh user@server + Password: + (server)$ ./backup_db.sh + - Kapow!-enabled solution: Provide an HTTP endpoint that when accessed triggers the run of the backup script. - ``` - curl -X PUT http://server:8080/db/backup - ``` - ``` - kapow route add -X PUT /db/backup -e ./backup_db.sh - ``` + + .. code-block:: console + + $ curl -X PUT http://server:8080/db/backup + + + .. code-block:: console + + $ kapow route add -X PUT /db/backup -e ./backup_db.sh #. Basic server monitoring @@ -71,9 +77,10 @@ User Journey - pre-Kapow! solution: SSH into the host + cat /tmp/backup_db.log. - Kapow!-enabled solution: Provide an endpoint that returns the contents of /tmp/backup_db.log. - ``` - cat /var/log/backup_db.log | kapow set /response/body - ``` + + .. code-block:: console + + $ cat /var/log/backup_db.log | kapow set /response/body #. Filter over basic monitoring @@ -89,13 +96,14 @@ User Journey this task. - Kapow!-enabled solution: - ``` - LINES="$(kapow get /request/params/lines)" - FILTER="$(kapow get /request/params/filter)" - grep "$FILTER" /var/log/backup_db.log \ - | tail -n"$LINES" \ - | kapow set /response/body - ``` + + .. code-block:: sh + + LINES="$(kapow get /request/params/lines)" + FILTER="$(kapow get /request/params/filter)" + grep "$FILTER" /var/log/backup_db.log \ + | tail -n"$LINES" \ + | kapow set /response/body #. Advanced database monitoring @@ -104,36 +112,57 @@ User Journey - Problem/Motivation: - pre-Kapow! solution: - Kapow!-enabled solution: - ``` - { - echo Memory: - free -m - echo ================================================================================ - echo Load: - uptime - echo ================================================================================ - echo Disk: - df -h - } | kapow set /response/body - ``` + + From this: + + .. code-block:: sh + + echo Date: | kapow set /response/body + echo ======...==== | kapow set /response/body + echo Memory | kapow set /response/body + # ... + + + To this: + + .. code-block:: sh + + { + echo Date: + date + echo ================================================================================ + echo Memory: + free -m + echo ================================================================================ + echo Load: + uptime + echo ================================================================================ + echo Disk: + df -h + } | kapow set /response/body #. Share your achievements - User Learns: Format a complex HTTP response with JSON format to feed the corporate dashboard. - Kapow! Concepts: backtick interpolation and `kapow set /response/headers` - Problem/Motivation: - - pre-Kapow! solution: + - pre-Kapow! solution: Write a php/perl/python script to serve this - Kapow!-enabled solution: - ``` DON'T HANDWRITE JSON - echo "{memory: `free -m`, ...uups..}' | kapow set /response/body - ``` - ``` USE JQ - MEMORY=$(free -m) - LOAD=$(uptime) - DISK=$(df -h) - jq -nc --arg memory "$MEMORY" '{"memory": $memory}' - ``` + Don't handwrite JSON + + .. code-block:: sh + + echo "{memory: `free -m`, ...uups..}" | kapow set /response/body + + Use jq + + .. code-block:: sh + + MEMORY=$(free -m) + LOAD=$(uptime) + DISK=$(df -h) + jq -nc --arg memory "$MEMORY" '{"memory": $memory}' Ideas ----- From 68bb365a78ce6494ef3f2367cb0f9f14bab8d1cd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Abdelkader=20Mart=C3=ADnez=20P=C3=A9rez?= Date: Tue, 26 Nov 2019 09:07:16 +0100 Subject: [PATCH 2/4] Removing missing files --- docs/source/index.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index 7dab6da..f242060 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -9,8 +9,6 @@ :caption: Examples examples/index - examples/greet-json - examples/upload .. toctree:: :maxdepth: 2 @@ -27,7 +25,7 @@ :maxdepth: 2 :caption: Tutorial - tutorial/_brianstorm + tutorial/_brainstorm Indices and tables ================== From 33b315338dc892cbfaa47f00494269531739d40b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Abdelkader=20Mart=C3=ADnez=20P=C3=A9rez?= Date: Tue, 26 Nov 2019 10:28:54 +0100 Subject: [PATCH 3/4] First attempt at tutorial. Co-authored-by: Hector Hurtado --- docs/source/index.rst | 2 + docs/source/tutorial/_brainstorm.rst | 65 +++++++++++++++++---- docs/source/tutorial/tutorial00.rst | 37 ++++++++++++ docs/source/tutorial/tutorial01.rst | 85 ++++++++++++++++++++++++++++ 4 files changed, 178 insertions(+), 11 deletions(-) create mode 100644 docs/source/tutorial/tutorial00.rst create mode 100644 docs/source/tutorial/tutorial01.rst diff --git a/docs/source/index.rst b/docs/source/index.rst index f242060..6256870 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -25,6 +25,8 @@ :maxdepth: 2 :caption: Tutorial + tutorial/tutorial00 + tutorial/tutorial01 tutorial/_brainstorm Indices and tables diff --git a/docs/source/tutorial/_brainstorm.rst b/docs/source/tutorial/_brainstorm.rst index ad4518b..3050bf7 100644 --- a/docs/source/tutorial/_brainstorm.rst +++ b/docs/source/tutorial/_brainstorm.rst @@ -35,6 +35,12 @@ ACME's Infrastructure - Corporate Server - Backup Server +Characters +---------- + +- Seasoned Ops +- Junior Ops + User Journey ------------ @@ -60,12 +66,11 @@ User Journey .. code-block:: console - $ curl -X PUT http://server:8080/db/backup - + $ kapow route add -X PUT /db/backup -e ./backup_db.sh .. code-block:: console - $ kapow route add -X PUT /db/backup -e ./backup_db.sh + $ curl -X PUT http://server:8080/db/backup #. Basic server monitoring @@ -80,7 +85,7 @@ User Journey .. code-block:: console - $ cat /var/log/backup_db.log | kapow set /response/body + $ cat /tmp/backup_db.log | kapow set /response/body #. Filter over basic monitoring @@ -109,8 +114,11 @@ User Journey - User Learns: Compose complex HTTP responses with more than one local command. - Kapow! Concepts: HEREDOC and subshells - - Problem/Motivation: - - pre-Kapow! solution: + - Problem/Motivation: The OPs manager needs to have information about + the health status of our servers. And she is always asking to the + team to write a report that involves calling several commands. + - pre-Kapow! solution: SSH into the server and manually execute the + commands, collect the output and write the report. - Kapow!-enabled solution: From this: @@ -127,6 +135,7 @@ User Journey .. code-block:: sh + kapow set /response/headers/Content-Type text/plain { echo Date: date @@ -145,24 +154,29 @@ User Journey - User Learns: Format a complex HTTP response with JSON format to feed the corporate dashboard. - Kapow! Concepts: backtick interpolation and `kapow set /response/headers` - - Problem/Motivation: + - Problem/Motivation: The OPs manager wants to create a dashboard to + see the server health information in real time. She hired a fronted + developer to make a nice dashboard application and we need to + provide him with the information in a format suitable for display. - pre-Kapow! solution: Write a php/perl/python script to serve this - Kapow!-enabled solution: - Don't handwrite JSON + Don't handwrite `JSON` .. code-block:: sh - echo "{memory: `free -m`, ...uups..}" | kapow set /response/body + kapow set /response/body application/json + echo "{memory: `free -m`, ...uups...}" | kapow set /response/body - Use jq + Use ``jq`` .. code-block:: sh MEMORY=$(free -m) LOAD=$(uptime) DISK=$(df -h) - jq -nc --arg memory "$MEMORY" '{"memory": $memory}' + kapow set /response/body application/json + jq -nc --arg memory "$MEMORY" '{"memory": $memory}' | kapow set /response/body Ideas ----- @@ -176,3 +190,32 @@ Ideas Add this to serve the webpage that uses the implemented HTTP API kapow route add / -c 'kapow set /resonse/headers/Content-Type text/html ; curl --output - http:// | kapow set /response/body' + + +Test +---- + +**User** + + Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod + tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At + vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, + no sea takimata sanctus est Lorem ipsum dolor sit amet. + +**Admin** + + Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod + tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At + vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, + no sea takimata sanctus est Lorem ipsum dolor sit amet. + +**User** + + Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod + tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. + + .. code-block:: console + + $ cat something.txt + + Right? diff --git a/docs/source/tutorial/tutorial00.rst b/docs/source/tutorial/tutorial00.rst new file mode 100644 index 0000000..b5d426f --- /dev/null +++ b/docs/source/tutorial/tutorial00.rst @@ -0,0 +1,37 @@ +Your First Day at Work +====================== + +**Senior** + + Welcome to ACME Inc. This is your first day here, right? + +**Junior** + + Hi! I am excited to start working. What will be my first task? + +**Senior** + + First let me explain to you what is our infrastructure. + +**Junior** + + Ok. + +**Senior** + + We have two Linux machines that provide services to our employees. + + 1. The Corporate Server: Provides email, database and web services. + + 2. The Backup Server: It is used to store backup of the important + company data. + +**Junior** + + That's it? Ok, just like Google. + +**Senior** + + Well, I think is time for you to start with your first task. It just + so happens that we received another request to backup the database + from the projects team. diff --git a/docs/source/tutorial/tutorial01.rst b/docs/source/tutorial/tutorial01.rst new file mode 100644 index 0000000..7909556 --- /dev/null +++ b/docs/source/tutorial/tutorial01.rst @@ -0,0 +1,85 @@ +Backup that Database! +===================== + +**Junior** + + A Backup? Don't you have this kind of things already automated? + +**Senior** + + Well, is not that simple. We of course have periodic backups. But, our + project team ask us for a backup every time a project is finished. + + I've already prepared a script to do the task. Before executing it in + production download it and test it in your own machine. + + .. todo:: + + - Link backup script from Github. + +**Junior** + + Ok, done! When I executed it the output says: + + .. code-block:: console + + $ ./backup_db.sh + Backup done! + Your log file is at /tmp/backup_db.log + +**Senior** + + That's right. That script performed the backup and stored it into the + **Backup Server** and appended some information into the backup log + file at ``/tmp/backup_db.log``. + + Now you can SSH into the **Corporate Server** and make the real + backup. + + +**Junior** + + Wait, wait... how long have you been doing this? + + +**Senior** + + This procedure was already here when I arrived. + +**Junior** + + And why don't they do it themselves? I mean, what do you contribute + to the process? + +**Senior** + + I am the only allowed to SSH into the **Corporate Server** for obvious + reasons. + +**Junior** + + Why do you need to SSH in the first place? Couldn't it be done + without SSH? + +**Senior** + + Actually it could be done with a promising new tool I've just found... + Kapow! + + Is a tool that allows you to publish scripts as HTTP services. If we + use it here we can give them the ability to do the backup whenever + they want. + +**Junior** + + Sounds like less work for me. I like it. + +**Senior** + + Ok then, let's try on your laptop first. + + First of all you have to follow the installation instructions XXX. + +**Junior** + + ... From f276417fd56350db2a475885dfcf416571c6a973 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roberto=20Abdelkader=20Mart=C3=ADnez=20P=C3=A9rez?= Date: Tue, 26 Nov 2019 11:44:21 +0100 Subject: [PATCH 4/4] Finish tutorial01.rst Co-authored-by: Hector Hurtado --- docs/source/tutorial/tutorial01.rst | 103 +++++++++++++++++++++++++++- 1 file changed, 102 insertions(+), 1 deletion(-) diff --git a/docs/source/tutorial/tutorial01.rst b/docs/source/tutorial/tutorial01.rst index 7909556..2a1bd7b 100644 --- a/docs/source/tutorial/tutorial01.rst +++ b/docs/source/tutorial/tutorial01.rst @@ -82,4 +82,105 @@ Backup that Database! **Junior** - ... + I've just installed it in my laptop, but I don't understand how all of + this is going to work. + +**Senior** + + Don't worry it is pretty easy. Basically we will provide an HTTP + endpoint managed by Kapow! at the **Corporate Server**; when the + project team wants to perform a backup they only need to call the + endpoint and Kapow! will call the backup script. + +**Junior** + + It seems pretty easy. How can I create the endpoint? + +**Senior** + + First you have to start a fresh server. Please run this in your laptop: + + .. code-block:: console + + $ kapow server + + .. warning:: + + It is important that you run this command in the same directory + in which you downloaded ``backup_db.sh``. + +**Junior** + + Done! But it doesn't do anything. + +**Senior** + + Now you have the port 8080 open but don't have any endpoints defined. + To define our endpoint you have to run this in another terminal: + + .. code-block:: console + + $ kapow route add -X PUT /db/backup -e ./backup_db.sh + + This will create an endpoint accessible via + ``http://localhost:8080/db/backup``. This endpoint have to be invoked + with the ``PUT`` method to prevent accidental calls. + +**Junior** + + Cool! Do we need to do all this stuff every time we start the + **Corporate Server**? + +**Senior** + + Not at all. The have thought of everything. You can put all your route + definitions in a special script file and pass it to the server on + startup. They call those files `POW` files and have ``.pow`` + extension. + + It should look something like: + + .. code-block:: console + + $ cat backup.pow + kapow route add -X PUT /db/backup -e ./backup_db.sh + + And then you can start Kapow! with it: + + .. code-block:: console + + $ kapow server backup.pow + +**Junior** + + Great! Now it says: + + .. code-block:: console + + $ kapow server backup.pow + 2019/11/26 11:40:01 Running powfile: "backup.pow" + {"id":"19bb4ac7-1039-11ea-aa00-106530610c4d","method":"PUT","url_pattern":"/db/backup","entrypoint":"./backup_db.sh","command":"","index":0} + 2019/11/26 11:40:01 Done running powfile: "backup.pow" + + I understand that this is proof that we have the endpoint available. + +**Senior** + + That appears to be the case, but better we check it. + + Call it with ``curl``: + + .. code-block:: console + + $ curl -X PUT http://localhost:8080/db/backup + +**Junior** + + Yay! I can see the log file at ``/tmp/backup_db.log`` + +**Senior** + + That's great. I am going to install all this in the *Corporate Server* + and forget about the old procedure. + + That enough for your first day! You can go home.