Merge remote-tracking branch 'origin/master'
This commit is contained in:
@@ -71,7 +71,9 @@ Table of content
|
|||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
:caption: Tutorial
|
:caption: Tutorial
|
||||||
|
|
||||||
tutorial/_brianstorm
|
tutorial/tutorial00
|
||||||
|
tutorial/tutorial01
|
||||||
|
tutorial/_brainstorm
|
||||||
|
|
||||||
Indices and tables
|
Indices and tables
|
||||||
==================
|
==================
|
||||||
|
|||||||
@@ -35,6 +35,12 @@ ACME's Infrastructure
|
|||||||
- Corporate Server
|
- Corporate Server
|
||||||
- Backup Server
|
- Backup Server
|
||||||
|
|
||||||
|
Characters
|
||||||
|
----------
|
||||||
|
|
||||||
|
- Seasoned Ops
|
||||||
|
- Junior Ops
|
||||||
|
|
||||||
User Journey
|
User Journey
|
||||||
------------
|
------------
|
||||||
|
|
||||||
@@ -45,21 +51,26 @@ User Journey
|
|||||||
- Problem/Motivation: Each time an ACME project is finished it is
|
- Problem/Motivation: Each time an ACME project is finished it is
|
||||||
desirable to make a backup of the entire database. Given that the
|
desirable to make a backup of the entire database. Given that the
|
||||||
database server is a critical machine we don't want to grant SSH
|
database server is a critical machine we don't want to grant SSH
|
||||||
access to lowly developers. The script is very fast because the
|
access to lowly developers. The script is fast because the
|
||||||
database is small (for now).
|
database is small (for now).
|
||||||
- pre-Kapow! solution: Launching the script via SSH shell.
|
- pre-Kapow! solution: Launching the script via SSH shell.
|
||||||
```
|
|
||||||
ssh user@server
|
.. code-block:: console
|
||||||
$ ./backup_db.sh
|
|
||||||
```
|
$ ssh user@server
|
||||||
|
Password:
|
||||||
|
(server)$ ./backup_db.sh
|
||||||
|
|
||||||
- Kapow!-enabled solution: Provide an HTTP endpoint that when accessed
|
- Kapow!-enabled solution: Provide an HTTP endpoint that when accessed
|
||||||
triggers the run of the backup script.
|
triggers the run of the backup script.
|
||||||
```
|
|
||||||
curl -X PUT http://server:8080/db/backup
|
.. code-block:: console
|
||||||
```
|
|
||||||
```
|
$ kapow route add -X PUT /db/backup -e ./backup_db.sh
|
||||||
kapow route add -X PUT /db/backup -e ./backup_db.sh
|
|
||||||
```
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ curl -X PUT http://server:8080/db/backup
|
||||||
|
|
||||||
#. Basic server monitoring
|
#. Basic server monitoring
|
||||||
|
|
||||||
@@ -71,9 +82,10 @@ User Journey
|
|||||||
- pre-Kapow! solution: SSH into the host + cat /tmp/backup_db.log.
|
- pre-Kapow! solution: SSH into the host + cat /tmp/backup_db.log.
|
||||||
- Kapow!-enabled solution: Provide an endpoint that returns the contents of
|
- Kapow!-enabled solution: Provide an endpoint that returns the contents of
|
||||||
/tmp/backup_db.log.
|
/tmp/backup_db.log.
|
||||||
```
|
|
||||||
cat /var/log/backup_db.log | kapow set /response/body
|
.. code-block:: console
|
||||||
```
|
|
||||||
|
$ cat /tmp/backup_db.log | kapow set /response/body
|
||||||
|
|
||||||
#. Filter over basic monitoring
|
#. Filter over basic monitoring
|
||||||
|
|
||||||
@@ -89,51 +101,82 @@ User Journey
|
|||||||
this task.
|
this task.
|
||||||
|
|
||||||
- Kapow!-enabled solution:
|
- Kapow!-enabled solution:
|
||||||
```
|
|
||||||
LINES="$(kapow get /request/params/lines)"
|
.. code-block:: sh
|
||||||
FILTER="$(kapow get /request/params/filter)"
|
|
||||||
grep "$FILTER" /var/log/backup_db.log \
|
LINES="$(kapow get /request/params/lines)"
|
||||||
| tail -n"$LINES" \
|
FILTER="$(kapow get /request/params/filter)"
|
||||||
| kapow set /response/body
|
grep "$FILTER" /var/log/backup_db.log \
|
||||||
```
|
| tail -n"$LINES" \
|
||||||
|
| kapow set /response/body
|
||||||
|
|
||||||
#. Advanced database monitoring
|
#. Advanced database monitoring
|
||||||
|
|
||||||
- User Learns: Compose complex HTTP responses with more than one local command.
|
- User Learns: Compose complex HTTP responses with more than one local command.
|
||||||
- Kapow! Concepts: HEREDOC and subshells
|
- Kapow! Concepts: HEREDOC and subshells
|
||||||
- Problem/Motivation:
|
- Problem/Motivation: The OPs manager needs to have information about
|
||||||
- pre-Kapow! solution:
|
the health status of our servers. And she is always asking to the
|
||||||
|
team to write a report that involves calling several commands.
|
||||||
|
- pre-Kapow! solution: SSH into the server and manually execute the
|
||||||
|
commands, collect the output and write the report.
|
||||||
- Kapow!-enabled solution:
|
- Kapow!-enabled solution:
|
||||||
```
|
|
||||||
{
|
From this:
|
||||||
echo Memory:
|
|
||||||
free -m
|
.. code-block:: sh
|
||||||
echo ================================================================================
|
|
||||||
echo Load:
|
echo Date: | kapow set /response/body
|
||||||
uptime
|
echo ======...==== | kapow set /response/body
|
||||||
echo ================================================================================
|
echo Memory | kapow set /response/body
|
||||||
echo Disk:
|
# ...
|
||||||
df -h
|
|
||||||
} | kapow set /response/body
|
|
||||||
```
|
To this:
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
kapow set /response/headers/Content-Type text/plain
|
||||||
|
{
|
||||||
|
echo Date:
|
||||||
|
date
|
||||||
|
echo ================================================================================
|
||||||
|
echo Memory:
|
||||||
|
free -m
|
||||||
|
echo ================================================================================
|
||||||
|
echo Load:
|
||||||
|
uptime
|
||||||
|
echo ================================================================================
|
||||||
|
echo Disk:
|
||||||
|
df -h
|
||||||
|
} | kapow set /response/body
|
||||||
|
|
||||||
#. Share your achievements
|
#. Share your achievements
|
||||||
|
|
||||||
- User Learns: Format a complex HTTP response with JSON format to feed the corporate dashboard.
|
- User Learns: Format a complex HTTP response with JSON format to feed the corporate dashboard.
|
||||||
- Kapow! Concepts: backtick interpolation and `kapow set /response/headers`
|
- Kapow! Concepts: backtick interpolation and `kapow set /response/headers`
|
||||||
- Problem/Motivation:
|
- Problem/Motivation: The OPs manager wants to create a dashboard to
|
||||||
- pre-Kapow! solution:
|
see the server health information in real time. She hired a fronted
|
||||||
|
developer to make a nice dashboard application and we need to
|
||||||
|
provide him with the information in a format suitable for display.
|
||||||
|
- pre-Kapow! solution: Write a php/perl/python script to serve this
|
||||||
- Kapow!-enabled solution:
|
- Kapow!-enabled solution:
|
||||||
``` DON'T HANDWRITE JSON
|
|
||||||
echo "{memory: `free -m`, ...uups..}' | kapow set /response/body
|
|
||||||
```
|
|
||||||
|
|
||||||
``` USE JQ
|
Don't handwrite `JSON`
|
||||||
MEMORY=$(free -m)
|
|
||||||
LOAD=$(uptime)
|
.. code-block:: sh
|
||||||
DISK=$(df -h)
|
|
||||||
jq -nc --arg memory "$MEMORY" '{"memory": $memory}'
|
kapow set /response/body application/json
|
||||||
```
|
echo "{memory: `free -m`, ...uups...}" | kapow set /response/body
|
||||||
|
|
||||||
|
Use ``jq``
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
MEMORY=$(free -m)
|
||||||
|
LOAD=$(uptime)
|
||||||
|
DISK=$(df -h)
|
||||||
|
kapow set /response/body application/json
|
||||||
|
jq -nc --arg memory "$MEMORY" '{"memory": $memory}' | kapow set /response/body
|
||||||
|
|
||||||
Ideas
|
Ideas
|
||||||
-----
|
-----
|
||||||
@@ -147,3 +190,32 @@ Ideas
|
|||||||
|
|
||||||
Add this to serve the webpage that uses the implemented HTTP API
|
Add this to serve the webpage that uses the implemented HTTP API
|
||||||
kapow route add / -c 'kapow set /resonse/headers/Content-Type text/html ; curl --output - http:// | kapow set /response/body'
|
kapow route add / -c 'kapow set /resonse/headers/Content-Type text/html ; curl --output - http:// | kapow set /response/body'
|
||||||
|
|
||||||
|
|
||||||
|
Test
|
||||||
|
----
|
||||||
|
|
||||||
|
**User**
|
||||||
|
|
||||||
|
Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod
|
||||||
|
tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At
|
||||||
|
vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren,
|
||||||
|
no sea takimata sanctus est Lorem ipsum dolor sit amet.
|
||||||
|
|
||||||
|
**Admin**
|
||||||
|
|
||||||
|
Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod
|
||||||
|
tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At
|
||||||
|
vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren,
|
||||||
|
no sea takimata sanctus est Lorem ipsum dolor sit amet.
|
||||||
|
|
||||||
|
**User**
|
||||||
|
|
||||||
|
Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod
|
||||||
|
tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ cat something.txt
|
||||||
|
|
||||||
|
Right?
|
||||||
|
|||||||
@@ -0,0 +1,37 @@
|
|||||||
|
Your First Day at Work
|
||||||
|
======================
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
Welcome to ACME Inc. This is your first day here, right?
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
Hi! I am excited to start working. What will be my first task?
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
First let me explain to you what is our infrastructure.
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
Ok.
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
We have two Linux machines that provide services to our employees.
|
||||||
|
|
||||||
|
1. The Corporate Server: Provides email, database and web services.
|
||||||
|
|
||||||
|
2. The Backup Server: It is used to store backup of the important
|
||||||
|
company data.
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
That's it? Ok, just like Google.
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
Well, I think is time for you to start with your first task. It just
|
||||||
|
so happens that we received another request to backup the database
|
||||||
|
from the projects team.
|
||||||
@@ -0,0 +1,186 @@
|
|||||||
|
Backup that Database!
|
||||||
|
=====================
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
A Backup? Don't you have this kind of things already automated?
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
Well, is not that simple. We of course have periodic backups. But, our
|
||||||
|
project team ask us for a backup every time a project is finished.
|
||||||
|
|
||||||
|
I've already prepared a script to do the task. Before executing it in
|
||||||
|
production download it and test it in your own machine.
|
||||||
|
|
||||||
|
.. todo::
|
||||||
|
|
||||||
|
- Link backup script from Github.
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
Ok, done! When I executed it the output says:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ ./backup_db.sh
|
||||||
|
Backup done!
|
||||||
|
Your log file is at /tmp/backup_db.log
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
That's right. That script performed the backup and stored it into the
|
||||||
|
**Backup Server** and appended some information into the backup log
|
||||||
|
file at ``/tmp/backup_db.log``.
|
||||||
|
|
||||||
|
Now you can SSH into the **Corporate Server** and make the real
|
||||||
|
backup.
|
||||||
|
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
Wait, wait... how long have you been doing this?
|
||||||
|
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
This procedure was already here when I arrived.
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
And why don't they do it themselves? I mean, what do you contribute
|
||||||
|
to the process?
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
I am the only allowed to SSH into the **Corporate Server** for obvious
|
||||||
|
reasons.
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
Why do you need to SSH in the first place? Couldn't it be done
|
||||||
|
without SSH?
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
Actually it could be done with a promising new tool I've just found...
|
||||||
|
Kapow!
|
||||||
|
|
||||||
|
Is a tool that allows you to publish scripts as HTTP services. If we
|
||||||
|
use it here we can give them the ability to do the backup whenever
|
||||||
|
they want.
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
Sounds like less work for me. I like it.
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
Ok then, let's try on your laptop first.
|
||||||
|
|
||||||
|
First of all you have to follow the installation instructions XXX.
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
I've just installed it in my laptop, but I don't understand how all of
|
||||||
|
this is going to work.
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
Don't worry it is pretty easy. Basically we will provide an HTTP
|
||||||
|
endpoint managed by Kapow! at the **Corporate Server**; when the
|
||||||
|
project team wants to perform a backup they only need to call the
|
||||||
|
endpoint and Kapow! will call the backup script.
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
It seems pretty easy. How can I create the endpoint?
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
First you have to start a fresh server. Please run this in your laptop:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ kapow server
|
||||||
|
|
||||||
|
.. warning::
|
||||||
|
|
||||||
|
It is important that you run this command in the same directory
|
||||||
|
in which you downloaded ``backup_db.sh``.
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
Done! But it doesn't do anything.
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
Now you have the port 8080 open but don't have any endpoints defined.
|
||||||
|
To define our endpoint you have to run this in another terminal:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ kapow route add -X PUT /db/backup -e ./backup_db.sh
|
||||||
|
|
||||||
|
This will create an endpoint accessible via
|
||||||
|
``http://localhost:8080/db/backup``. This endpoint have to be invoked
|
||||||
|
with the ``PUT`` method to prevent accidental calls.
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
Cool! Do we need to do all this stuff every time we start the
|
||||||
|
**Corporate Server**?
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
Not at all. The have thought of everything. You can put all your route
|
||||||
|
definitions in a special script file and pass it to the server on
|
||||||
|
startup. They call those files `POW` files and have ``.pow``
|
||||||
|
extension.
|
||||||
|
|
||||||
|
It should look something like:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ cat backup.pow
|
||||||
|
kapow route add -X PUT /db/backup -e ./backup_db.sh
|
||||||
|
|
||||||
|
And then you can start Kapow! with it:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ kapow server backup.pow
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
Great! Now it says:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ kapow server backup.pow
|
||||||
|
2019/11/26 11:40:01 Running powfile: "backup.pow"
|
||||||
|
{"id":"19bb4ac7-1039-11ea-aa00-106530610c4d","method":"PUT","url_pattern":"/db/backup","entrypoint":"./backup_db.sh","command":"","index":0}
|
||||||
|
2019/11/26 11:40:01 Done running powfile: "backup.pow"
|
||||||
|
|
||||||
|
I understand that this is proof that we have the endpoint available.
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
That appears to be the case, but better we check it.
|
||||||
|
|
||||||
|
Call it with ``curl``:
|
||||||
|
|
||||||
|
.. code-block:: console
|
||||||
|
|
||||||
|
$ curl -X PUT http://localhost:8080/db/backup
|
||||||
|
|
||||||
|
**Junior**
|
||||||
|
|
||||||
|
Yay! I can see the log file at ``/tmp/backup_db.log``
|
||||||
|
|
||||||
|
**Senior**
|
||||||
|
|
||||||
|
That's great. I am going to install all this in the *Corporate Server*
|
||||||
|
and forget about the old procedure.
|
||||||
|
|
||||||
|
That enough for your first day! You can go home.
|
||||||
Reference in New Issue
Block a user