Dark-Alex-17/kapow
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

doc: tutorial style fixes

Browse Source
This commit is contained in:
pancho horrillo
2019-12-10 18:01:29 +01:00
parent 94b284f625
commit ae68cf8fed
6 changed files with 233 additions and 211 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/source/tutorial/tutorial00.rst
+18 -10
View File
@@ -3,35 +3,43 @@ Your First Day at Work
**Senior**
Welcome to ACME Inc. This is your first day here, right?
Welcome to *ACME Inc.* This is your first day here, right?
**Junior**
Hi! I am excited to start working. What will be my first task?
Hi! Yes! And I am eager to start working. What will be my first task?
**Senior**
First let me explain to you what is our infrastructure.
First, let me help you get acquainted with our infrastructure.
**Junior**
Ok.
OK.
**Senior**
We have two Linux machines that provide services to our employees.
We have two Linux boxen that provide services to our employees.
1. The Corporate Server: Provides email, database and web services.
2. The Backup Server: It is used to store backup of the important
company data.
company data.
**Junior**
That's it? Ok, just like Google.
That's it? OK, just like Google, then.
**Senior**
Well, I think is time for you to start with your first task. It just
so happens that we received another request to backup the database
from the projects team.
Smartass...
**Junior**
(chuckles nervously).
**Senior**
Well, I think is time for you to start with your first task. It just so
happens that we received another request to backup the database from the
projects team.
docs/source/tutorial/tutorial01.rst
+39 -37
View File
@@ -7,20 +7,22 @@ Backup that Database!
**Senior**
Well, is not that simple. We of course have periodic backups. But, our
Well, is not that simple. We of course have periodic backups. But, our
project team ask us for a backup every time a project is finished.
I've already prepared a script to do the task. Before executing it in
I've already prepared a script to do the task. Before executing it in
production download it and test it in your own machine.
.. code-block:: console
$ curl --output backup_db.sh 'https://raw.githubusercontent.com/BBVA/kapow/master/docs/source/tutorial/materials/backup_db.sh'
$ curl --output backup_db.sh https://raw.githubusercontent.com/BBVA/kapow/master/docs/source/tutorial/materials/backup_db.sh
$ chmod u+x backup_db.sh
**Junior**
Ok, done! When I executed it the output says:
(after a few minutes)
OK, done! When I executed it the output says:
.. code-block:: console
@@ -30,11 +32,11 @@ Backup that Database!
**Senior**
That's right. That script performed the backup and stored it into the
**Backup Server** and appended some information into the backup log
file at ``/tmp/backup_db.log``.
That's right. That script performed the backup and stored it into the
**Backup Server** and appended some information into the backup log file at
``/tmp/backup_db.log``.
Now you can SSH into the **Corporate Server** and make the real
Now you can ``ssh`` into the **Corporate Server** and make the real
backup.
@@ -54,32 +56,33 @@ Backup that Database!
**Senior**
I am the only allowed to SSH into the **Corporate Server** for obvious
reasons.
I am the only one allowed to ``ssh`` into the **Corporate Server**, for
obvious reasons.
**Junior**
Why do you need to SSH in the first place? Couldn't it be done
without SSH?
Why do you need to ``ssh`` in the first place? Couldn't it be done without
``ssh``?
**Senior**
Actually it could be done with a promising new tool I've just found...
Kapow!
Actually, it could be done with a promising new tool I've just found...
*Kapow!*
Is a tool that allows you to publish scripts as HTTP services. If we
Is a tool that allows you to publish scripts as ``HTTP`` services. If we
use it here we can give them the ability to do the backup whenever
they want.
**Junior**
Sounds like less work for me. I like it.
Sounds like less work for me. I like it!
**Senior**
Ok then, let's try on your laptop first.
OK then, let's it try on your laptop first.
First of all you have to follow the `installation instructions </the_project/install_and_configure>`_.
First of all, you have to follow the
`installation instructions </the_project/install_and_configure>`_.
**Junior**
@@ -88,10 +91,10 @@ Backup that Database!
**Senior**
Don't worry it is pretty easy. Basically we will provide an HTTP
endpoint managed by *Kapow!* at the **Corporate Server**; when the
project team wants to perform a backup they only need to call the
endpoint and *Kapow!* will call the backup script.
Don't worry, it is pretty easy. Basically we will provide an ``HTTP``
endpoint managed by *Kapow!* at the **Corporate Server**; when the project
team wants to perform a backup they only need to call the endpoint and
*Kapow!* will call the backup script.
**Junior**
@@ -99,7 +102,7 @@ Backup that Database!
**Senior**
First you have to start a fresh server. Please run this in your laptop:
First you have to start a fresh server. Please run this in your laptop:
.. code-block:: console
@@ -107,8 +110,8 @@ Backup that Database!
.. warning::
It is important that you run this command in the same directory
in which you downloaded ``backup_db.sh``.
It is important that you run this command in the same directory in which
you downloaded ``backup_db.sh``.
**Junior**
@@ -116,7 +119,7 @@ Backup that Database!
**Senior**
Now you have the port 8080 open but don't have any endpoints defined.
Now you have the port ``8080`` open but don't have any endpoints defined.
To define our endpoint you have to run this in another terminal:
.. code-block:: console
@@ -124,8 +127,8 @@ Backup that Database!
$ kapow route add -X PUT /db/backup -e ./backup_db.sh
This will create an endpoint accessible via
``http://localhost:8080/db/backup``. This endpoint have to be invoked
with the ``PUT`` method to prevent accidental calls.
``http://localhost:8080/db/backup``. This endpoint has to be invoked with the
``PUT`` method to prevent accidental calls.
**Junior**
@@ -134,10 +137,9 @@ Backup that Database!
**Senior**
Not at all. The have thought of everything. You can put all your route
definitions in a special script file and pass it to the server on
startup. They call those files `POW` files and have ``.pow``
extension.
Not at all. The creators of *Kapow!* have thought of everything. You can put
all your route definitions in a special script file and pass it to the server
on startup. They call those files `pow` files and they have ``.pow`` extension.
It should look something like:
@@ -167,7 +169,7 @@ Backup that Database!
**Senior**
That appears to be the case, but better we check it.
That appears to be the case, but we better check it.
Call it with ``curl``:
@@ -177,11 +179,11 @@ Backup that Database!
**Junior**
Yay! I can see the log file at ``/tmp/backup_db.log``
Yay! I can see the log file at ``/tmp/backup_db.log``
**Senior**
That's great. I am going to install all this in the *Corporate Server*
and forget about the old procedure.
That's great. I am going to install all this in the *Corporate Server* and
forget about the old procedure.
That enough for your first day! You can go home.
That enough for your first day! Go home now and get some rest.
docs/source/tutorial/tutorial02.rst
+16 -16
View File
@@ -1,44 +1,44 @@
What we've done?
================
What have we done?
==================
**Senior**
Ey, I come from seeing our project team mates. They're delighted with their
new toy, but they miss something.
I forgot to tell you that after the backup run they need to review the log
file to check that all went ok.
I forgot to tell you that after the backup is run they need to review the log
file to check that all went OK.
**Junior**
Makes sense. Do you think that *Kapow!* can help with this? I feel that this is
the way to do it.
Makes sense. Do you think that *Kapow!* can help with this? I have the
feeling that this is the right way to do it...
**Senior**
Sure! Let's take a look at the documentation to see how can we tweak with the
Sure! Let's take a look at the documentation to see how we can tweak the
logic of the request.
**Junior**
Got it! There're a
`lot of resources to work with </theory/resource_tree.rst>`_, I see that we
can write to the response, do you think this will work for us?
can write to the response. Do you think this will work for us?
**Senior**
Yeah, they used to cat the log file contents to see what happened in the
last execution:
Yeah, the team is used to ``cat`` the log file contents to see what happened
in the last execution:
.. code-block:: console
$ cat /tmp/backup_db.log
I've make it easy for you, do you dare to do it?
I've made it easy for you. Are you up to it?
**Junior**
Let me try add this to our ``POW`` file:
Let me try add this to our ``pow`` file:
.. code-block:: console
@@ -47,13 +47,13 @@ What we've done?
**Senior**
Looks good to me, clean and simple, and it is a very good idea to use ``GET``
here as it wont change anything in the server. Restart *Kapow!* and try it.
here as it wont change anything in the server. Restart *Kapow!* and try it.
**Junior**
Wooow! I get back the content of the file. If they liked the first one they're
going to love this.
Wooow! I get back the content of the file. If they liked the first one
they're going to loooove this.
**Senior**
Agree. we've done the day with this...
Agree. We are done for the day with this...
docs/source/tutorial/tutorial03.rst
+25 -24
View File
@@ -3,43 +3,44 @@ We need to filter
**Senior**
Hi guy! How're you doing this morning? I've got a new challenge from our
Hiya! How're you doing this morning? I've got a new challenge from our
grateful mates.
As time goes on from the last rotation the size of the log file gets bigger.
Furthermore, they want to limit the output of the file to pick only some
records and only from the end of the file. We need to do something to help
them as they waste a lot of time reviewing the output.
As time goes on from the last log rotation, the size of the log file gets
bigger and bigger. Furthermore, they want to limit the output of the file to
pick only some records, and only from the end of the file. We need to do
something to help them as they are wasting a lot of time reviewing the output.
**Junior**
My two cents is that this time is going to be more of a deep bash knowledge.
Do you agree?
I have a feeling that this is going to entail some serious *bash-foo*. What
do you think?
**Senior**
By sure, but in addition to some good shell plumbing we're going to squeeze
Sure! But in addition to some good shell plumbing we're going to squeeze
*Kapow!*'s superpowers a litle bit more to get a really good solution.
Can you take a look at *Kapow!*'s documentation to see if something can be done?
Can you take a look at *Kapow!*'s documentation to see if something can be
done?
**Junior**
I've seen in the documentation and there is a way to get access to the data
comming in the request. Do you think we can use this to let them choose how
I've read in the documentation that there is a way to get access to the data
coming in the request. Do you think we can use this to let them choose how
to do the filtering?
**Senior**
Sounds great! How we have lived without *Kapow!* all this time?
Sounds great! How have we lived without *Kapow!* all this time?
As they requested, we can offer them with a parameter to filter the registers
they want to pick and another parameter to limit the output size in lines.
As they requested, we can offer them a parameter to filter the registers
they want to pick, and another parameter to limit the output size in lines.
**Junior**
Sounds that will be enough. Now we have to make some modifications to our
last endpoint definition to add this new feature. Let's start working...
Sounds about right. Now we have to make some modifications to our last
endpoint definition to add this new feature. Let's get cracking!
**Senior**
@@ -51,20 +52,20 @@ We need to filter
| tail -n "$(kapow get /request/params/lines)" \
| kapow set /response/body'
It looks a bit weird but we'll have time to re-styling later. Please make
some tests on your laptop before to publish on the *Corporate Server*.
Remember to send them an example URL with the parameters the can use to
filter.
It looks a bit weird, but we'll have time to revise the style later. Please
make some tests on your laptop before we publish it on the *Corporate Server*.
Remember to send them an example URL with the parameters they can use to
filter and limit the amount of lines they get.
**Junior**
Ok, Should look like this, isn't it?
OK, should look like this, doesn't it?
.. code-block:: console
$ curl http://localhost:8080/db/backup_logs?filter=rows%20inserted&lines=200
$ curl 'http://localhost:8080/db/backup_logs?filter=rows%20inserted&lines=200'
**Senior**
Exactly. Another great day helping the company to advance. Let's go for a
beer for celebrating!
Exactly. Another great day helping the company advance. Let's go grab a
beer to celebrate!
docs/source/tutorial/tutorial04.rst
+68 -59
View File
@@ -1,23 +1,22 @@
I Need My Report
I Need my Report
================
**Junior**
Good morning!
You look very busy, what's the matter?
You look very busy, what's going on?
**Senior**
I am finishing the capacity planning report. Let me just
mail it... Done!
I am finishing the capacity planning report. Let me just mail it... Done!
Today I am going to teach you how to do this report so we can split
the workload.
Today I am going to teach you how to write this report so we can split the
workload.
**Junior**
Oh. That sounds... fun. Ok, tell me about this report.
Oh. That sounds... fun. OK, tell me about this report.
**Senior**
@@ -30,37 +29,44 @@ I Need My Report
**Junior**
I see this company scales just like Google.
I see this company scales up just like Google...
**Senior**
Smartass...
**Junior**
(chuckles)
**Senior**
We have a procedure:
1. SSH into the machine.
2. Execute the following commands copying its output for later fill in
the report:
1. ``ssh`` into the machine.
2. Execute the following commands copying its output for later filling in the
report:
- ``hostname`` and ``date``: To include in the report.
- ``free -m``: To know if we have to buy more RAM.
- ``uptime``: To see the load of the system.
- ``df -h``: Just in case we need another hard disk.
- ``df -h``: Just in case we need another hard disk drive.
3. Copy all this in a mail and send it to *Susan*, the operations
manager.
3. Copy all this in an email and send it to *Susan*, the operations manager.
**Junior**
And why *Susan* don't enter the server herself to see all of this?
And why *Susan* can't ``ssh`` into the server herself to see all of this?
**Senior**
She doesn't have time for this. She is a manager, she is very busy!
She doesn't have time for this. She is a manager, and she is very busy!
**Junior**
Well, I guess we can make a *Kapow!* endpoint to let her see all this
information from the browser. This way she doesn't need to waste any
time asking us.
information from the browser. This way she doesn't need to waste any time
asking us.
I started to write it already:
@@ -70,10 +76,10 @@ I Need My Report
**Senior**
That is preposterous!
Not good enough!
First of all that code is not readable. And the output would be
something like.
First of all, that code is not readable. And the output would be something
like:
.. code-block:: text
@@ -89,7 +95,7 @@ I Need My Report
Which is also very difficult to read!
What *Susan* is used to see is more like:
What *Susan* is used to see is more like this:
.. code-block:: text
@@ -114,9 +120,9 @@ I Need My Report
**Senior**
That fix the issue for *Susan* but make it worst for us.
That fixes the issue for *Susan*, but makes it worse for us.
What about a HEREDOC to help us make the code more readable.
What about a HEREDOC to help us make the code more readable?
**Junior**
@@ -132,9 +138,9 @@ I Need My Report
.. code-block:: console
$ cat <<-'EOF'
you can put
more than one line
here
you can put
more than one line
here
EOF
The shell will put the data between the first ``EOF`` and the second
@@ -142,43 +148,44 @@ I Need My Report
**Junior**
If I want to use this with *Kapow!* I have to make it read the script
from ``stdin``. To do this I know that I have to put a ``-`` at the
end.
OK, I understand. That's cool, by the way.
So, if I want to use this with *Kapow!*, I have to make it read the script
from ``stdin``. To do this I know that I have to put a ``-`` at the end.
Let me try:
.. code-block:: bash
kapow route add /capacityreport - <<-'EOF'
hostname | kapow set /response/body
echo ================================================================================ | kapow set /response/body
date | kapow set /response/body
echo ================================================================================ | kapow set /response/body
free -m | kapow set /response/body
echo ================================================================================ | kapow set /response/body
uptime | kapow set /response/body
echo ================================================================================ | kapow set /response/body
df -h | kapow set /response/body
echo ================================================================================ | kapow set /response/body
hostname | kapow set /response/body
echo ================================================================================ | kapow set /response/body
date | kapow set /response/body
echo ================================================================================ | kapow set /response/body
free -m | kapow set /response/body
echo ================================================================================ | kapow set /response/body
uptime | kapow set /response/body
echo ================================================================================ | kapow set /response/body
df -h | kapow set /response/body
echo ================================================================================ | kapow set /response/body
EOF
**Senior**
That would work. Nevertheless I am not satisfied.
That would work. Nevertheless I am not yet satisfied.
What about all the repeated ``kapow set /response/body`` statements?
Could we do any better?
Do you think we could do any better?
**Junior**
Maybe we can redirect all to a file and use the file as the input of
Maybe we can redirect all output to a file and use the file as the input of
``kapow set /response/body``.
**Senior**
There is a better way. You can make use of another neat ``bash``
feature: **group commands**.
There is a better way. You can make use of another neat ``bash`` feature:
**group commands**.
Group commands allows you to execute several commands treating the
group as one single command.
@@ -196,24 +203,26 @@ I Need My Report
.. code-block:: bash
kapow route add /capacityreport - <<-'EOF'
{
hostname
echo ================================================================================
date
echo ================================================================================
free -m
echo ================================================================================
uptime
echo ================================================================================
df -h
echo ================================================================================
} | kapow set /response/body
{
hostname
echo ================================================================================
date
echo ================================================================================
free -m
echo ================================================================================
uptime
echo ================================================================================
df -h
echo ================================================================================
} | kapow set /response/body
EOF
**Senior**
I am not worried about maintaining that script. Good job!
Nice! Now I am not worried about maintaining that script. Good job!
**Junior**
You know me. Whatever it takes to avoid writing reports ;)
You know me. Whatever it takes to avoid writing reports ;-)
(both chuckle).
docs/source/tutorial/tutorial05.rst
+67 -65
View File
@@ -9,23 +9,23 @@ Sharing the Stats
Just about time... We are in trouble!
The report stuff was a complete success, so much that now *Susan* has
hired a frontend developer to create a custom dashboard to see the
stats in real time.
The report stuff was a complete success, so much so that now *Susan* has hired
a frontend developer to create a custom dashboard to see the stats in real
time.
Now we have to provide the backend for the solution.
**Junior**
And whats the problem?
And what's the problem?
**Senior**
We are not developers what are we doing writing backend?
We are not developers! What are we doing writing a backend?
**Junior**
Chill out, man. Can't be that difficult? What they need exactly?
Just chill out. Can't be that difficult... What do they need, exactly?
**Senior**
@@ -34,20 +34,19 @@ Sharing the Stats
**Junior**
So we have half of the work already done.
So, we have half of the work done already!
What about this?
.. code-block:: bash
kapow route add /capacitystats - <<-'EOF'
echo "{\"memory\": \"`free -m`\"}" | kapow set /response/body
echo "{\"memory\": \"`free -m`\"}" | kapow set /response/body
EOF
**Senior**
For starters that's not valid JSON. The output would be something
like:
For starters, that's not valid ``JSON``. The output would be something like:
.. code-block:: console
@@ -56,9 +55,8 @@ Sharing the Stats
Mem: 31967 3121 21680 980 7166 27418
Swap: 0 0 0"}
You can't add new lines inside a JSON string that way, you have to
encode with ``\n``.
You can't add new lines inside a ``JSON`` string that way, you have to escape
the new line characters as ``\n``.
**Junior**
@@ -66,7 +64,7 @@ Sharing the Stats
**Senior**
See it by yourself.
See it for yourself.
.. code-block:: console
@@ -75,23 +73,23 @@ Sharing the Stats
**Junior**
``jq``? What is that command?
``jq``? What is that command?
**Senior**
``jq`` is a wonderful tool for working with JSON data from the command
line. With you ``jq`` you can extract data from JSON and also
generate well-formed JSON.
``jq`` is a wonderful tool for working with ``JSON`` data from the command
line. With ``jq`` you can extract data from a ``JSON`` document and it also
allows you to generate a well-formed ``JSON`` document.
**Junior**
Let's use it then!
Let's use it, then!
How can we generate a JSON document with ``jq``?
How can we generate a ``JSON`` document with ``jq``?
**Senior**
To generate a document we use the ``-n`` argument:
To generate a document we use the ``-n`` option:
.. code-block:: console
@@ -102,11 +100,12 @@ Sharing the Stats
**Junior**
That is not very useful. The output is the same.
That does not seem very useful. The output is just the same.
**Senior**
It get's better. You can add variables to the JSON and ``jq`` will escape them for you.
Bear with me, it gets better. You can add variables to the ``JSON`` and
``jq`` will escape them for you.
.. code-block:: console
@@ -117,7 +116,9 @@ Sharing the Stats
**Junior**
That's just what I need.
Sweet! That's just what I need.
(hacks away for a few minutes).
What do you think of this?
@@ -134,8 +135,8 @@ Sharing the Stats
**Senior**
That is the data we have to produce. But the code is far from readable. And
you also forgot about adding the endpoint.
That is the output we have to produce, right. But the code is far from
readable. And you also forgot about adding the endpoint.
Can we do any better?
@@ -146,60 +147,61 @@ Sharing the Stats
.. code-block:: bash
kapow route add /capacitystats - <<-'EOF'
jq -n \
--arg hostname "$(hostname)" \
--arg date "$(date)" \
--arg memory "$(free -m)" \
--arg load "$(uptime)" \
--arg disk "$(df -h)" \
'{"hostname": $hostname, "date": $date, "memory": $memory, "load": $load, "disk": $disk}' \
| kapow set /response/body
jq -n \
--arg hostname "$(hostname)" \
--arg date "$(date)" \
--arg memory "$(free -m)" \
--arg load "$(uptime)" \
--arg disk "$(df -h)" \
'{"hostname": $hostname, "date": $date, "memory": $memory, "load": $load, "disk": $disk}' \
| kapow set /response/body
EOF
What do you think?
**Senior**
You forgot one more thing.
I'm afraid you forgot an important detail.
**Junior**
I think you are wrong, the JSON is well-formed and it contains all the
required data. Also the code is very readable.
I don't think so! the ``JSON`` is well-formed and it contains all the required
data. And the code is quite readable.
**Senior**
You are right but, you are not using HTTP correctly. You have to set the
You are right, but you are not using ``HTTP`` correctly. You have to set the
``Content-Type`` header to let your client know the format of the data you are
outputting.
**Junior**
Ok, let me try:
Oh, I see. Let me try again:
.. code-block:: bash
kapow route add /capacitystats - <<-'EOF'
jq -n \
--arg hostname "$(hostname)" \
--arg date "$(date)" \
--arg memory "$(free -m)" \
--arg load "$(uptime)" \
--arg disk "$(df -h)" \
'{"hostname": $hostname, "date": $date, "memory": $memory, "load": $load, "disk": $disk}' \
| kapow set /response/body
echo application/json | kapow set /response/headers/Content-Type
jq -n \
--arg hostname "$(hostname)" \
--arg date "$(date)" \
--arg memory "$(free -m)" \
--arg load "$(uptime)" \
--arg disk "$(df -h)" \
'{"hostname": $hostname, "date": $date, "memory": $memory, "load": $load, "disk": $disk}' \
| kapow set /response/body
echo application/json | kapow set /response/headers/Content-Type
EOF
**Senior**
Just a couple of details.
Better. Just a couple of details.
1. You have to set the headers **before** the body. This is because the body
can be so big that *Kapow!* is forced to start sending it out.
2. In cases where you want to set a small piece of data (like the header) is
better to not use the ``stdin``. *Kapow!* provides a secondary syntax for these
cases:
1. You have to set the headers **before** writing to the body. This is
because the body can be so big that *Kapow!* is forced to start sending it
out.
2. In cases where you want to set a small piece of data (like the header), it
is better not to use ``stdin``. *Kapow!* provides a secondary syntax
for these cases:
.. code-block:: console
@@ -212,18 +214,18 @@ Sharing the Stats
.. code-block:: bash
kapow route add /capacitystats - <<-'EOF'
kapow set /response/headers/Content-Type application/json
jq -n \
--arg hostname "$(hostname)" \
--arg date "$(date)" \
--arg memory "$(free -m)" \
--arg load "$(uptime)" \
--arg disk "$(df -h)" \
'{"hostname": $hostname, "date": $date, "memory": $memory, "load": $load, "disk": $disk}' \
| kapow set /response/body
kapow set /response/headers/Content-Type application/json
jq -n \
--arg hostname "$(hostname)" \
--arg date "$(date)" \
--arg memory "$(free -m)" \
--arg load "$(uptime)" \
--arg disk "$(df -h)" \
'{"hostname": $hostname, "date": $date, "memory": $memory, "load": $load, "disk": $disk}' \
| kapow set /response/body
EOF
**Senior**
That's perfect! Let's upload this to the *Corporate Server* and tell the
frontend developer.
That's perfect! Now, let's upload this to the *Corporate Server* and tell the
frontend developer about it.