831 lines
40 KiB
HTML
831 lines
40 KiB
HTML
|
||
<!DOCTYPE html>
|
||
|
||
<html>
|
||
<head>
|
||
<meta charset="utf-8" />
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
|
||
|
||
<title>Khéops 101 — Khéops 0.1 documentation</title>
|
||
<link rel="stylesheet" type="text/css" href="../_static/pygments.css" />
|
||
<link rel="stylesheet" type="text/css" href="../_static/alabaster.css" />
|
||
<script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
|
||
<script src="../_static/jquery.js"></script>
|
||
<script src="../_static/underscore.js"></script>
|
||
<script src="../_static/doctools.js"></script>
|
||
<link rel="index" title="Index" href="../genindex.html" />
|
||
<link rel="search" title="Search" href="../search.html" />
|
||
<link rel="next" title="Introduction" href="../docs/learn/100.html" />
|
||
<link rel="prev" title="Learn" href="../docs/learn/index.html" />
|
||
|
||
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
|
||
|
||
|
||
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
|
||
|
||
</head><body>
|
||
|
||
|
||
<div class="document">
|
||
<div class="documentwrapper">
|
||
<div class="bodywrapper">
|
||
|
||
|
||
<div class="body" role="main">
|
||
|
||
<section class="tex2jax_ignore mathjax_ignore" id="kheops-101">
|
||
<h1>Khéops 101<a class="headerlink" href="#kheops-101" title="Permalink to this headline">¶</a></h1>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nb">cd</span> /home/jez/volumes/data/prj/bell/training/tiger-ansible/ext/kheops
|
||
<span class="nb">echo</span> <span class="nv">$PWD</span>
|
||
<span class="nb">export</span> <span class="nv">KHEOPS_NAMESPACE</span><span class="o">=</span>ex1_enc
|
||
<span class="nb">export</span> <span class="nv">KHEOPS_CONFIG</span><span class="o">=</span>examples/kheops.yml
|
||
rm -rf <span class="s2">"examples/</span><span class="nv">$KHEOPS_NAMESPACE</span><span class="s2">"</span>
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>/home/jez/volumes/data/prj/bell/training/tiger-ansible/ext/kheops
|
||
</pre></div>
|
||
</div>
|
||
<section id="command-line">
|
||
<h2>Command line<a class="headerlink" href="#command-line" title="Permalink to this headline">¶</a></h2>
|
||
<p>Let’s check first that kheops works correclty, and start to read the manual.</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kheops --help
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>usage: kheops [-h] [-v] [-c CONFIG]
|
||
{schema,gen_doc,lookup,demo,subcommand2} ...
|
||
|
||
Kheops, hierarchical data lookup tool
|
||
|
||
options:
|
||
-h, --help show this help message and exit
|
||
-v, --verbose Increase verbosity (KHEOPS_VERBOSE) (default: 0)
|
||
-c CONFIG, --config CONFIG
|
||
Kheops configuration file (KHEOPS_CONFIG) (default:
|
||
examples/kheops.yml)
|
||
|
||
subcommands:
|
||
valid subcommands
|
||
|
||
{schema,gen_doc,lookup,demo,subcommand2}
|
||
</pre></div>
|
||
</div>
|
||
<p>So we have a working <code class="docutils literal notranslate"><span class="pre">kheops</span></code> command, and we will focus on the <code class="docutils literal notranslate"><span class="pre">lookup</span></code> command. On it’s simplest form, a lookup consists in querying a <code class="docutils literal notranslate"><span class="pre">key</span></code> for a given <code class="docutils literal notranslate"><span class="pre">scope</span></code>. The output of the <code class="docutils literal notranslate"><span class="pre">key</span></code> will change depending the <code class="docutils literal notranslate"><span class="pre">scope</span></code> value. A <code class="docutils literal notranslate"><span class="pre">key</span></code> is in simple word.</p>
|
||
</section>
|
||
<section id="defining-a-hierarchy">
|
||
<h2>Defining a hierarchy<a class="headerlink" href="#defining-a-hierarchy" title="Permalink to this headline">¶</a></h2>
|
||
<p>To illustrate how Khéops works, let’s start with a simple example, we will try to lookup the <code class="docutils literal notranslate"><span class="pre">profile</span></code> key of the following two (fictive) servers:</p>
|
||
<ul class="simple">
|
||
<li><p>web.domain.org: which act as a webserver role</p></li>
|
||
<li><p>mysql.domain.org: which act as mysql role</p></li>
|
||
</ul>
|
||
<p>But first we need to create our hierarchy. It’s as simple as creating directories and put some json or yaml data into different files. Let’s create our hierarchy. We will first create the default profile:</p>
|
||
<p>From our use case, we will build a lookup tree. We want to be able to organise data depending the 3 criterias:</p>
|
||
<ul class="simple">
|
||
<li><p>node: name of the node</p></li>
|
||
<li><p>role: assigned role to the node</p></li>
|
||
<li><p>environment: it can either be dev or prod</p></li>
|
||
</ul>
|
||
<p>Let’s create our lookup hierarchy:</p>
|
||
<p>default:
|
||
lookups:
|
||
- path: default # Simplest form, just a path
|
||
- path: “roles/{role}” # If list, it’s auto expanded like in bash
|
||
- path: “env_{env}” # If list, it’s auto expanded like in bash
|
||
- path: “nodes/{node}”</p>
|
||
<p>So for a given key, Khéops will iterate all over those paths to find the requested <code class="docutils literal notranslate"><span class="pre">key</span></code> , and then it will merge all results. Some paths are variabilized, and those variable comes from the scope. The scope come along the <code class="docutils literal notranslate"><span class="pre">key</span></code>, it’s can be any metadata. For complex metadata you may want to store those in a file and load your scope with the <code class="docutils literal notranslate"><span class="pre">-f</span> <span class="pre"><yaml_scope_file></span></code> option:</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">kheops</span> <span class="n">lookup</span> <span class="o">-</span><span class="n">e</span> <span class="o"><</span><span class="n">var1</span><span class="o">=</span><span class="n">val1</span><span class="o">></span> <span class="o">-</span><span class="n">e</span> <span class="o"><</span><span class="n">var2</span><span class="o">=</span><span class="n">val2</span><span class="o">></span> <span class="o"><</span><span class="n">key</span><span class="o">></span>
|
||
</pre></div>
|
||
</div>
|
||
<p>A scope is completely optional while keys are required.</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>
|
||
</pre></div>
|
||
</div>
|
||
</section>
|
||
<section id="basic-hierarchy">
|
||
<h2>Basic hierarchy<a class="headerlink" href="#basic-hierarchy" title="Permalink to this headline">¶</a></h2>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="c1"># We create a fresh hierachy</span>
|
||
mkdir -p examples/<span class="nv">$KHEOPS_NAMESPACE</span>
|
||
|
||
<span class="c1"># We create a profile key, which is a dict</span>
|
||
cat > examples/<span class="nv">$KHEOPS_NAMESPACE</span>/default.yml <span class="s"><<EOF</span>
|
||
<span class="s">---</span>
|
||
<span class="s">profile:</span>
|
||
<span class="s"> env: "NO_ENV"</span>
|
||
<span class="s"> product: "NO_PRODUCT"</span>
|
||
<span class="s"> </span>
|
||
<span class="s">EOF</span>
|
||
|
||
<span class="c1"># Let's inspect our hierarchy</span>
|
||
tree examples/<span class="nv">$KHEOPS_NAMESPACE</span>
|
||
cat examples/<span class="nv">$KHEOPS_NAMESPACE</span>/default.yml
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>[01;34mexamples/ex1_enc[00m
|
||
└── default.yml
|
||
|
||
0 directories, 1 file
|
||
---
|
||
profile:
|
||
env: "NO_ENV"
|
||
product: "NO_PRODUCT"
|
||
</pre></div>
|
||
</div>
|
||
<p>From this point, we defined our profile with two attribute, <code class="docutils literal notranslate"><span class="pre">team</span></code> and <code class="docutils literal notranslate"><span class="pre">product</span></code>. As it’s the default case, we set them both unconfigured.</p>
|
||
<p>You are now already able to query your hierarchy:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kheops lookup profile
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>env: NO_ENV
|
||
product: NO_PRODUCT
|
||
</pre></div>
|
||
</div>
|
||
<p>Good, no surprise. But, we mentionned we wanted to get the profile of two instances, this how would do that:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kheops lookup -e <span class="nv">node</span><span class="o">=</span>web.infra.net profile
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>env: NO_ENV
|
||
product: NO_PRODUCT
|
||
</pre></div>
|
||
</div>
|
||
</section>
|
||
<section id="roles">
|
||
<h2>Roles<a class="headerlink" href="#roles" title="Permalink to this headline">¶</a></h2>
|
||
<p>However, same result as before, which is expected as we did not finished to configure our hierarchy. Among our instances, we identified 2 roles: web and mysql. Let’s create those two roles:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>mkdir -p examples/<span class="nv">$KHEOPS_NAMESPACE</span>/roles
|
||
|
||
<span class="c1"># We create a new web role</span>
|
||
cat > examples/<span class="nv">$KHEOPS_NAMESPACE</span>/roles/web.yml <span class="s"><<EOF</span>
|
||
<span class="s">---</span>
|
||
<span class="s">profile:</span>
|
||
<span class="s"> product: "httpd_server"</span>
|
||
|
||
<span class="s"> web_top_domain: ""</span>
|
||
<span class="s"> web_app: "NO_APP"</span>
|
||
<span class="s"> web_port: 80</span>
|
||
<span class="s"> web_user_list:</span>
|
||
<span class="s"> - sysadmins</span>
|
||
<span class="s"> </span>
|
||
<span class="s">EOF</span>
|
||
|
||
<span class="c1"># We create a new mysql role</span>
|
||
cat > examples/<span class="nv">$KHEOPS_NAMESPACE</span>/roles/mysql.yml <span class="s"><<EOF</span>
|
||
<span class="s">---</span>
|
||
<span class="s">profile:</span>
|
||
<span class="s"> product: "mysql_server"</span>
|
||
|
||
<span class="s"> mysql_database: "NO_DATABASE"</span>
|
||
<span class="s"> mysql_users:</span>
|
||
<span class="s"> - "sysadmin@10.0.42%"</span>
|
||
<span class="s"> mysql_port: 3306</span>
|
||
<span class="s"> mysql_cluster: False</span>
|
||
<span class="s"> </span>
|
||
<span class="s">EOF</span>
|
||
|
||
<span class="c1"># Let's inspect our hierarchy</span>
|
||
tree examples/<span class="nv">$KHEOPS_NAMESPACE</span>
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>[01;34mexamples/ex1_enc[00m
|
||
├── default.yml
|
||
└── [01;34mroles[00m
|
||
├── mysql.yml
|
||
└── web.yml
|
||
|
||
1 directory, 3 files
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>tail -n <span class="m">999</span> examples/<span class="nv">$KHEOPS_NAMESPACE</span>/<span class="o">{</span>*.yml,*/*.yml<span class="o">}</span>
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>==> examples/ex1_enc/default.yml <==
|
||
---
|
||
profile:
|
||
env: "NO_ENV"
|
||
product: "NO_PRODUCT"
|
||
|
||
|
||
==> examples/ex1_enc/roles/mysql.yml <==
|
||
---
|
||
profile:
|
||
product: "mysql_server"
|
||
|
||
mysql_database: "NO_DATABASE"
|
||
mysql_users:
|
||
- "sysadmin@10.0.42%"
|
||
mysql_port: 3306
|
||
mysql_cluster: False
|
||
|
||
|
||
==> examples/ex1_enc/roles/web.yml <==
|
||
---
|
||
profile:
|
||
product: "httpd_server"
|
||
|
||
web_top_domain: ""
|
||
web_app: "NO_APP"
|
||
web_port: 80
|
||
web_user_list:
|
||
- sysadmins
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kheops lookup -e <span class="nv">node</span><span class="o">=</span>web.infra.net -e <span class="nv">role</span><span class="o">=</span>web profile
|
||
kheops lookup -e <span class="nv">node</span><span class="o">=</span>mysql.infra.net -e <span class="nv">role</span><span class="o">=</span>mysql profile
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>env: NO_ENV
|
||
product: httpd_server
|
||
web_top_domain: ''
|
||
web_app: NO_APP
|
||
web_port: 80
|
||
web_user_list:
|
||
- sysadmins
|
||
|
||
env: NO_ENV
|
||
product: mysql_server
|
||
mysql_database: NO_DATABASE
|
||
mysql_users:
|
||
- sysadmin@10.0.42%
|
||
mysql_port: 3306
|
||
mysql_cluster: false
|
||
</pre></div>
|
||
</div>
|
||
</section>
|
||
<section id="per-node-override">
|
||
<h2>Per node override<a class="headerlink" href="#per-node-override" title="Permalink to this headline">¶</a></h2>
|
||
<p>It’s getting better, we can see that the profile key has been merged with the key values, across the different locations.</p>
|
||
<p>However, we will have those placeholders, and we want to have personalized value, depending if it’s aweb server, it need an unique domain and some unique parameters. So let’s create a <code class="docutils literal notranslate"><span class="pre">nodes</span></code> directory and place some data inside.</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>mkdir -p examples/<span class="nv">$KHEOPS_NAMESPACE</span>/nodes
|
||
|
||
<span class="c1"># We create a new web role</span>
|
||
cat > examples/<span class="nv">$KHEOPS_NAMESPACE</span>/nodes/web.infra.net.yml <span class="s"><<EOF</span>
|
||
<span class="s">---</span>
|
||
<span class="s">profile:</span>
|
||
<span class="s"> web_app: 'myapp'</span>
|
||
<span class="s"> web_user_list:</span>
|
||
<span class="s"> - domain_org</span>
|
||
<span class="s"> - domain_org_external</span>
|
||
<span class="s"> </span>
|
||
<span class="s">EOF</span>
|
||
|
||
<span class="c1"># We create a new mysql role</span>
|
||
cat > examples/<span class="nv">$KHEOPS_NAMESPACE</span>/nodes/mysql.infra.net.yml <span class="s"><<EOF</span>
|
||
<span class="s">---</span>
|
||
<span class="s">profile: </span>
|
||
<span class="s"> mysql_database: "app_domain_org"</span>
|
||
<span class="s"> mysql_users:</span>
|
||
<span class="s"> - "app_domain_org@10.0.51%"</span>
|
||
<span class="s"> </span>
|
||
<span class="s">EOF</span>
|
||
|
||
<span class="c1"># Let's inspect our hierarchy</span>
|
||
tree examples/<span class="nv">$KHEOPS_NAMESPACE</span>
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>[01;34mexamples/ex1_enc[00m
|
||
├── default.yml
|
||
├── [01;34mnodes[00m
|
||
│ ├── mysql.infra.net.yml
|
||
│ └── web.infra.net.yml
|
||
└── [01;34mroles[00m
|
||
├── mysql.yml
|
||
└── web.yml
|
||
|
||
2 directories, 5 files
|
||
</pre></div>
|
||
</div>
|
||
<p>And we try again:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kheops lookup -e <span class="nv">node</span><span class="o">=</span>web.infra.net -e <span class="nv">role</span><span class="o">=</span>web profile
|
||
kheops lookup -e <span class="nv">node</span><span class="o">=</span>mysql.infra.net -e <span class="nv">role</span><span class="o">=</span>mysql profile
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>env: NO_ENV
|
||
product: httpd_server
|
||
web_top_domain: ''
|
||
web_app: myapp
|
||
web_port: 80
|
||
web_user_list:
|
||
- sysadmins
|
||
- domain_org
|
||
- domain_org_external
|
||
|
||
env: NO_ENV
|
||
product: mysql_server
|
||
mysql_database: app_domain_org
|
||
mysql_users:
|
||
- sysadmin@10.0.42%
|
||
- app_domain_org@10.0.51%
|
||
mysql_port: 3306
|
||
mysql_cluster: false
|
||
</pre></div>
|
||
</div>
|
||
</section>
|
||
<section id="environment-override">
|
||
<h2>Environment override<a class="headerlink" href="#environment-override" title="Permalink to this headline">¶</a></h2>
|
||
<p>Let’s say you want to support environment, it’s the same:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="c1"># We create a new dev environment</span>
|
||
cat > examples/<span class="nv">$KHEOPS_NAMESPACE</span>/env_dev.yml <span class="s"><<EOF</span>
|
||
<span class="s">---</span>
|
||
<span class="s">profile:</span>
|
||
<span class="s"> env: dev</span>
|
||
<span class="s"> </span>
|
||
<span class="s"> # We change the top domain for dev environment, and reduce the cache</span>
|
||
<span class="s"> web_top_domain: dev.infra.net</span>
|
||
<span class="s"> web_cache: 1m</span>
|
||
<span class="s"> </span>
|
||
<span class="s"> # We want a debug users</span>
|
||
<span class="s"> web_user_list:</span>
|
||
<span class="s"> - debug_user</span>
|
||
<span class="s"> mysql_users:</span>
|
||
<span class="s"> - debug@10.0.%</span>
|
||
|
||
<span class="s"> debug: true</span>
|
||
|
||
<span class="s">EOF</span>
|
||
|
||
<span class="c1"># We create a new mysql role</span>
|
||
cat > examples/<span class="nv">$KHEOPS_NAMESPACE</span>/env_prod.yml <span class="s"><<EOF</span>
|
||
<span class="s">---</span>
|
||
<span class="s">profile:</span>
|
||
<span class="s"> env: prod</span>
|
||
<span class="s"> </span>
|
||
<span class="s"> # On production environment, we always want to use public faced domain and 12 hour cache.</span>
|
||
<span class="s"> web_top_domain: infra.com</span>
|
||
<span class="s"> web_cache: 12h</span>
|
||
<span class="s"> </span>
|
||
<span class="s">EOF</span>
|
||
|
||
<span class="c1"># Let's inspect our hierarchy</span>
|
||
tree examples/<span class="nv">$KHEOPS_NAMESPACE</span>
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>[01;34mexamples/ex1_enc[00m
|
||
├── default.yml
|
||
├── env_dev.yml
|
||
├── env_prod.yml
|
||
├── [01;34mnodes[00m
|
||
│ ├── mysql.infra.net.yml
|
||
│ └── web.infra.net.yml
|
||
└── [01;34mroles[00m
|
||
├── mysql.yml
|
||
└── web.yml
|
||
|
||
2 directories, 7 files
|
||
</pre></div>
|
||
</div>
|
||
<p>So it’s become quite easy to compare the difference between environment, with a simple variable switch:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kheops lookup -e <span class="nv">node</span><span class="o">=</span>web.infra.net -e <span class="nv">role</span><span class="o">=</span>web -e <span class="nv">env</span><span class="o">=</span>prod profile
|
||
kheops lookup -e <span class="nv">node</span><span class="o">=</span>web.infra.net -e <span class="nv">role</span><span class="o">=</span>web -e <span class="nv">env</span><span class="o">=</span>dev profile
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>env: prod
|
||
product: httpd_server
|
||
web_top_domain: infra.com
|
||
web_app: myapp
|
||
web_port: 80
|
||
web_user_list:
|
||
- sysadmins
|
||
- domain_org
|
||
- domain_org_external
|
||
web_cache: 12h
|
||
|
||
env: dev
|
||
product: httpd_server
|
||
web_top_domain: dev.infra.net
|
||
web_app: myapp
|
||
web_port: 80
|
||
web_user_list:
|
||
- sysadmins
|
||
- debug_user
|
||
- domain_org
|
||
- domain_org_external
|
||
web_cache: 1m
|
||
mysql_users:
|
||
- debug@10.0.%
|
||
debug: true
|
||
</pre></div>
|
||
</div>
|
||
<p>Same for mysql:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kheops lookup -e <span class="nv">node</span><span class="o">=</span>mysql.infra.net -e <span class="nv">role</span><span class="o">=</span>mysql -e <span class="nv">env</span><span class="o">=</span>prod profile
|
||
kheops lookup -e <span class="nv">node</span><span class="o">=</span>mysql.infra.net -e <span class="nv">role</span><span class="o">=</span>mysql -e <span class="nv">env</span><span class="o">=</span>dev profile
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>env: prod
|
||
product: mysql_server
|
||
mysql_database: app_domain_org
|
||
mysql_users:
|
||
- sysadmin@10.0.42%
|
||
- app_domain_org@10.0.51%
|
||
mysql_port: 3306
|
||
mysql_cluster: false
|
||
web_top_domain: infra.com
|
||
web_cache: 12h
|
||
|
||
env: dev
|
||
product: mysql_server
|
||
mysql_database: app_domain_org
|
||
mysql_users:
|
||
- sysadmin@10.0.42%
|
||
- debug@10.0.%
|
||
- app_domain_org@10.0.51%
|
||
mysql_port: 3306
|
||
mysql_cluster: false
|
||
web_top_domain: dev.infra.net
|
||
web_cache: 1m
|
||
web_user_list:
|
||
- debug_user
|
||
debug: true
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>
|
||
</pre></div>
|
||
</div>
|
||
<p>You have to keep in mind you can query the key with a different scope, and get different views:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kheops lookup profile
|
||
|
||
<span class="nb">echo</span> <span class="s2">"==> Per environment view"</span>
|
||
kheops lookup -e <span class="nv">env</span><span class="o">=</span>prod profile
|
||
kheops lookup -e <span class="nv">env</span><span class="o">=</span>dev profile
|
||
|
||
<span class="nb">echo</span> <span class="s2">"==> Per role and environment view"</span>
|
||
kheops lookup -e <span class="nv">role</span><span class="o">=</span>mysql -e <span class="nv">env</span><span class="o">=</span>prod profile
|
||
kheops lookup -e <span class="nv">role</span><span class="o">=</span>web -e <span class="nv">env</span><span class="o">=</span>prod profile
|
||
|
||
<span class="nb">echo</span> <span class="s2">"==> Per node view"</span>
|
||
kheops lookup -e <span class="nv">node</span><span class="o">=</span>web.infra.net -e <span class="nv">role</span><span class="o">=</span>web -e <span class="nv">env</span><span class="o">=</span>dev profile
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>env: NO_ENV
|
||
product: NO_PRODUCT
|
||
|
||
==> Per environment view
|
||
env: prod
|
||
product: NO_PRODUCT
|
||
web_top_domain: infra.com
|
||
web_cache: 12h
|
||
|
||
env: dev
|
||
product: NO_PRODUCT
|
||
web_top_domain: dev.infra.net
|
||
web_cache: 1m
|
||
web_user_list:
|
||
- debug_user
|
||
mysql_users:
|
||
- debug@10.0.%
|
||
debug: true
|
||
|
||
==> Per role and environment view
|
||
env: prod
|
||
product: mysql_server
|
||
mysql_database: NO_DATABASE
|
||
mysql_users:
|
||
- sysadmin@10.0.42%
|
||
mysql_port: 3306
|
||
mysql_cluster: false
|
||
web_top_domain: infra.com
|
||
web_cache: 12h
|
||
|
||
env: prod
|
||
product: httpd_server
|
||
web_top_domain: infra.com
|
||
web_app: NO_APP
|
||
web_port: 80
|
||
web_user_list:
|
||
- sysadmins
|
||
web_cache: 12h
|
||
|
||
==> Per node view
|
||
env: dev
|
||
product: httpd_server
|
||
web_top_domain: dev.infra.net
|
||
web_app: myapp
|
||
web_port: 80
|
||
web_user_list:
|
||
- sysadmins
|
||
- debug_user
|
||
- domain_org
|
||
- domain_org_external
|
||
web_cache: 1m
|
||
mysql_users:
|
||
- debug@10.0.%
|
||
debug: true
|
||
</pre></div>
|
||
</div>
|
||
<p>Even if somwaht clunky, this method can help to troubleshoot wrong data by dichotomy.</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>
|
||
</pre></div>
|
||
</div>
|
||
</section>
|
||
<section id="tooling-and-applications">
|
||
<h2>Tooling and applications<a class="headerlink" href="#tooling-and-applications" title="Permalink to this headline">¶</a></h2>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>
|
||
</pre></div>
|
||
</div>
|
||
</section>
|
||
<section id="troubleshooting">
|
||
<h2>Troubleshooting<a class="headerlink" href="#troubleshooting" title="Permalink to this headline">¶</a></h2>
|
||
<p>Sometimes, it can may be hard to navigate across file and hierachy, but GNU Utils are here to help. There is a selection of small tips:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nb">set</span> -x
|
||
|
||
: Find where a key has been defined
|
||
grep -r <span class="s1">'^profile:'</span> examples/<span class="nv">$KHEOPS_NAMESPACE</span>
|
||
|
||
: Find where a key has been defined and <span class="m">5</span> first lines
|
||
grep -r -A <span class="m">5</span> <span class="s1">'web_user_list:'</span> examples/<span class="nv">$KHEOPS_NAMESPACE</span>
|
||
|
||
: Search from anything related to database
|
||
grep -R -C <span class="m">3</span> <span class="s1">'database'</span> examples/<span class="nv">$KHEOPS_NAMESPACE</span>
|
||
|
||
<span class="nb">set</span> +x
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>+ : Find where a key has been defined
|
||
+ grep --colour=auto -r '^profile:' examples/ex1_enc
|
||
[35m[Kexamples/ex1_enc/env_prod.yml[m[K[36m[K:[m[K[01;31m[Kprofile:[m[K
|
||
[35m[Kexamples/ex1_enc/roles/mysql.yml[m[K[36m[K:[m[K[01;31m[Kprofile:[m[K
|
||
[35m[Kexamples/ex1_enc/roles/web.yml[m[K[36m[K:[m[K[01;31m[Kprofile:[m[K
|
||
[35m[Kexamples/ex1_enc/nodes/mysql.infra.net.yml[m[K[36m[K:[m[K[01;31m[Kprofile:[m[K
|
||
[35m[Kexamples/ex1_enc/nodes/web.infra.net.yml[m[K[36m[K:[m[K[01;31m[Kprofile:[m[K
|
||
[35m[Kexamples/ex1_enc/default.yml[m[K[36m[K:[m[K[01;31m[Kprofile:[m[K
|
||
[35m[Kexamples/ex1_enc/env_dev.yml[m[K[36m[K:[m[K[01;31m[Kprofile:[m[K
|
||
+ : Find where a key has been defined and 5 first lines
|
||
+ grep --colour=auto -r -A 5 web_user_list: examples/ex1_enc
|
||
[35m[Kexamples/ex1_enc/roles/web.yml[m[K[36m[K:[m[K [01;31m[Kweb_user_list:[m[K
|
||
[35m[Kexamples/ex1_enc/roles/web.yml[m[K[36m[K-[m[K - sysadmins
|
||
[35m[Kexamples/ex1_enc/roles/web.yml[m[K[36m[K-[m[K
|
||
[36m[K--[m[K
|
||
[35m[Kexamples/ex1_enc/nodes/web.infra.net.yml[m[K[36m[K:[m[K [01;31m[Kweb_user_list:[m[K
|
||
[35m[Kexamples/ex1_enc/nodes/web.infra.net.yml[m[K[36m[K-[m[K - domain_org
|
||
[35m[Kexamples/ex1_enc/nodes/web.infra.net.yml[m[K[36m[K-[m[K - domain_org_external
|
||
[35m[Kexamples/ex1_enc/nodes/web.infra.net.yml[m[K[36m[K-[m[K
|
||
[36m[K--[m[K
|
||
[35m[Kexamples/ex1_enc/env_dev.yml[m[K[36m[K:[m[K [01;31m[Kweb_user_list:[m[K
|
||
[35m[Kexamples/ex1_enc/env_dev.yml[m[K[36m[K-[m[K - debug_user
|
||
[35m[Kexamples/ex1_enc/env_dev.yml[m[K[36m[K-[m[K mysql_users:
|
||
[35m[Kexamples/ex1_enc/env_dev.yml[m[K[36m[K-[m[K - debug@10.0.%
|
||
[35m[Kexamples/ex1_enc/env_dev.yml[m[K[36m[K-[m[K
|
||
[35m[Kexamples/ex1_enc/env_dev.yml[m[K[36m[K-[m[K debug: true
|
||
+ : Search from anything related to database
|
||
+ grep --colour=auto -R -C 3 database examples/ex1_enc
|
||
[35m[Kexamples/ex1_enc/roles/mysql.yml[m[K[36m[K-[m[Kprofile:
|
||
[35m[Kexamples/ex1_enc/roles/mysql.yml[m[K[36m[K-[m[K product: "mysql_server"
|
||
[35m[Kexamples/ex1_enc/roles/mysql.yml[m[K[36m[K-[m[K
|
||
[35m[Kexamples/ex1_enc/roles/mysql.yml[m[K[36m[K:[m[K mysql_[01;31m[Kdatabase[m[K: "NO_DATABASE"
|
||
[35m[Kexamples/ex1_enc/roles/mysql.yml[m[K[36m[K-[m[K mysql_users:
|
||
[35m[Kexamples/ex1_enc/roles/mysql.yml[m[K[36m[K-[m[K - "sysadmin@10.0.42%"
|
||
[35m[Kexamples/ex1_enc/roles/mysql.yml[m[K[36m[K-[m[K mysql_port: 3306
|
||
[36m[K--[m[K
|
||
[35m[Kexamples/ex1_enc/nodes/mysql.infra.net.yml[m[K[36m[K-[m[K---
|
||
[35m[Kexamples/ex1_enc/nodes/mysql.infra.net.yml[m[K[36m[K-[m[Kprofile:
|
||
[35m[Kexamples/ex1_enc/nodes/mysql.infra.net.yml[m[K[36m[K:[m[K mysql_[01;31m[Kdatabase[m[K: "app_domain_org"
|
||
[35m[Kexamples/ex1_enc/nodes/mysql.infra.net.yml[m[K[36m[K-[m[K mysql_users:
|
||
[35m[Kexamples/ex1_enc/nodes/mysql.infra.net.yml[m[K[36m[K-[m[K - "app_domain_org@10.0.51%"
|
||
[35m[Kexamples/ex1_enc/nodes/mysql.infra.net.yml[m[K[36m[K-[m[K
|
||
+ set +x
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>
|
||
</pre></div>
|
||
</div>
|
||
<p>The tail/head command is quite usefull to look at multiple files at the same time, it add a nice header for each file:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>head -n <span class="m">999</span> examples/ex1_enc/roles/*
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>==> examples/ex1_enc/roles/mysql.yml <==
|
||
---
|
||
profile:
|
||
product: "mysql_server"
|
||
|
||
mysql_database: "NO_DATABASE"
|
||
mysql_users:
|
||
- "sysadmin@10.0.42%"
|
||
mysql_port: 3306
|
||
mysql_cluster: False
|
||
|
||
|
||
==> examples/ex1_enc/roles/web.yml <==
|
||
---
|
||
profile:
|
||
product: "httpd_server"
|
||
|
||
web_top_domain: ""
|
||
web_app: "NO_APP"
|
||
web_port: 80
|
||
web_user_list:
|
||
- sysadmins
|
||
</pre></div>
|
||
</div>
|
||
<p>You can also have a view of all files with this command:</p>
|
||
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">find</span> <span class="o">.</span> <span class="o">-</span><span class="nb">type</span> <span class="n">f</span><span class="o">|</span> <span class="n">xargs</span> <span class="n">head</span> <span class="o">-</span><span class="n">n</span> <span class="mi">999</span> <span class="o">|</span> <span class="n">less</span>
|
||
</pre></div>
|
||
</div>
|
||
<p>From there, you will be able to have a nice overview of your data.</p>
|
||
<p>You can even diff your change with this command. There is this simple trick to compare the data difference between 2 lookups:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>diff -u <span class="se">\</span>
|
||
<<span class="o">(</span>kheops lookup -e <span class="nv">node</span><span class="o">=</span>web.infra.net -e <span class="nv">role</span><span class="o">=</span>web -e <span class="nv">env</span><span class="o">=</span>prod profile<span class="o">)</span> <span class="se">\</span>
|
||
<<span class="o">(</span>kheops lookup -e <span class="nv">node</span><span class="o">=</span>web.infra.net -e <span class="nv">role</span><span class="o">=</span>web -e <span class="nv">env</span><span class="o">=</span>dev profile<span class="o">)</span>
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span>--- /dev/fd/63 2022-02-01 20:10:53.094525316 -0500
|
||
+++ /dev/fd/62 2022-02-01 20:10:53.094525316 -0500
|
||
@@ -1,11 +1,15 @@
|
||
-env: prod
|
||
+env: dev
|
||
product: httpd_server
|
||
-web_top_domain: infra.com
|
||
+web_top_domain: dev.infra.net
|
||
web_app: myapp
|
||
web_port: 80
|
||
web_user_list:
|
||
- sysadmins
|
||
+- debug_user
|
||
- domain_org
|
||
- domain_org_external
|
||
-web_cache: 12h
|
||
+web_cache: 1m
|
||
+mysql_users:
|
||
+- debug@10.0.%
|
||
+debug: true
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>
|
||
</pre></div>
|
||
</div>
|
||
<p>You can also ask Kheops to explain you how he built the result, you can use the <code class="docutils literal notranslate"><span class="pre">-x</span></code> flag:</p>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kheops lookup -e <span class="nv">role</span><span class="o">=</span>web profile -x
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-none notranslate"><div class="highlight"><pre><span></span> INFO: Explain lookups:
|
||
+------------------------+------------------------------+
|
||
| Config | Runtime |
|
||
+------------------------+------------------------------+
|
||
| | |
|
||
| Config:{ | Runtime:{ |
|
||
| "path": "default", | "scope": { |
|
||
| "backend": "file", | "role": "web" |
|
||
| "continue": true | }, |
|
||
| } | "key": "profile", |
|
||
| | "conf": { |
|
||
| | "index": 0 |
|
||
| | }, |
|
||
| | "raw_path": "default" |
|
||
| | } |
|
||
| | |
|
||
| Config:{ | Runtime:{ |
|
||
| "path": "roles/web", | "scope": { |
|
||
| "backend": "file", | "role": "web" |
|
||
| "continue": true | }, |
|
||
| } | "key": "profile", |
|
||
| | "conf": { |
|
||
| | "index": 1 |
|
||
| | }, |
|
||
| | "raw_path": "roles/{role}" |
|
||
| | } |
|
||
+------------------------+------------------------------+
|
||
INFO: Explain candidates:
|
||
+----------------------------------------------------------------------------------+-------------------------------+------------------------------+
|
||
| Status | Runtime | Data |
|
||
+----------------------------------------------------------------------------------+-------------------------------+------------------------------+
|
||
| | | |
|
||
| Status:{ | Runtime:{ | Data:{ |
|
||
| "path": "/home/jez/volumes/data/prj/bell/training/tiger-ansible/ext/kheops ... | "scope": { | "env": "NO_ENV", |
|
||
| "status": "found", | "role": "web" | "product": "NO_PRODUCT" |
|
||
| "rel_path": "examples/ex1_enc/default.yml" | }, | } |
|
||
| } | "key": "profile", | |
|
||
| | "conf": { | |
|
||
| | "index": 0 | |
|
||
| | }, | |
|
||
| | "raw_path": "default", | |
|
||
| | "backend_index": 0 | |
|
||
| | } | |
|
||
| | | |
|
||
| Status:{ | Runtime:{ | Data:{ |
|
||
| "path": "/home/jez/volumes/data/prj/bell/training/tiger-ansible/ext/kheops ... | "scope": { | "product": "httpd_server", |
|
||
| "status": "found", | "role": "web" | "web_top_domain": "", |
|
||
| "rel_path": "examples/ex1_enc/roles/web.yml" | }, | "web_app": "NO_APP", |
|
||
| } | "key": "profile", | "web_port": 80, |
|
||
| | "conf": { | "web_user_list": [ |
|
||
| | "index": 1 | "sysadmins" |
|
||
| | }, | ] |
|
||
| | "raw_path": "roles/{role}", | } |
|
||
| | "backend_index": 1 | |
|
||
| | } | |
|
||
+----------------------------------------------------------------------------------+-------------------------------+------------------------------+
|
||
env: NO_ENV
|
||
product: httpd_server
|
||
web_top_domain: ''
|
||
web_app: NO_APP
|
||
web_port: 80
|
||
web_user_list:
|
||
- sysadmins
|
||
</pre></div>
|
||
</div>
|
||
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>
|
||
</pre></div>
|
||
</div>
|
||
</section>
|
||
</section>
|
||
|
||
|
||
</div>
|
||
|
||
</div>
|
||
</div>
|
||
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
|
||
<div class="sphinxsidebarwrapper">
|
||
<h1 class="logo"><a href="../index.html">Khéops</a></h1>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
<h3>Navigation</h3>
|
||
<p class="caption" role="heading"><span class="caption-text">Contents:</span></p>
|
||
<ul class="current">
|
||
<li class="toctree-l1"><a class="reference internal" href="../docs/app/index.html">Application</a></li>
|
||
<li class="toctree-l1 current"><a class="reference internal" href="../docs/learn/index.html">Learn</a><ul class="current">
|
||
<li class="toctree-l2 current"><a class="current reference internal" href="#">Khéops 101</a><ul>
|
||
<li class="toctree-l3"><a class="reference internal" href="#command-line">Command line</a></li>
|
||
<li class="toctree-l3"><a class="reference internal" href="#defining-a-hierarchy">Defining a hierarchy</a></li>
|
||
<li class="toctree-l3"><a class="reference internal" href="#basic-hierarchy">Basic hierarchy</a></li>
|
||
<li class="toctree-l3"><a class="reference internal" href="#roles">Roles</a></li>
|
||
<li class="toctree-l3"><a class="reference internal" href="#per-node-override">Per node override</a></li>
|
||
<li class="toctree-l3"><a class="reference internal" href="#environment-override">Environment override</a></li>
|
||
<li class="toctree-l3"><a class="reference internal" href="#tooling-and-applications">Tooling and applications</a></li>
|
||
<li class="toctree-l3"><a class="reference internal" href="#troubleshooting">Troubleshooting</a></li>
|
||
</ul>
|
||
</li>
|
||
<li class="toctree-l2"><a class="reference internal" href="../docs/learn/100.html">Introduction</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="../docs/learn/101.html">Learn 101</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="../docs/learn/102.html">Learn 102</a></li>
|
||
<li class="toctree-l2"><a class="reference internal" href="../docs/learn/105.html">Learn 105</a></li>
|
||
</ul>
|
||
</li>
|
||
<li class="toctree-l1"><a class="reference internal" href="../docs/guide/index.html">Guides</a></li>
|
||
<li class="toctree-l1"><a class="reference internal" href="../api/modules.html">Python API</a></li>
|
||
</ul>
|
||
|
||
<div class="relations">
|
||
<h3>Related Topics</h3>
|
||
<ul>
|
||
<li><a href="../index.html">Documentation overview</a><ul>
|
||
<li><a href="../docs/learn/index.html">Learn</a><ul>
|
||
<li>Previous: <a href="../docs/learn/index.html" title="previous chapter">Learn</a></li>
|
||
<li>Next: <a href="../docs/learn/100.html" title="next chapter">Introduction</a></li>
|
||
</ul></li>
|
||
</ul></li>
|
||
</ul>
|
||
</div>
|
||
<div id="searchbox" style="display: none" role="search">
|
||
<h3 id="searchlabel">Quick search</h3>
|
||
<div class="searchformwrapper">
|
||
<form class="search" action="../search.html" method="get">
|
||
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
|
||
<input type="submit" value="Go" />
|
||
</form>
|
||
</div>
|
||
</div>
|
||
<script>$('#searchbox').show(0);</script>
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
</div>
|
||
</div>
|
||
<div class="clearer"></div>
|
||
</div>
|
||
<div class="footer">
|
||
©2022, Barbu IT.
|
||
|
||
|
|
||
Powered by <a href="http://sphinx-doc.org/">Sphinx 4.4.0</a>
|
||
& <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
|
||
|
||
|
|
||
<a href="../_sources/learn/learn101.md.txt"
|
||
rel="nofollow">Page source</a>
|
||
</div>
|
||
|
||
|
||
|
||
|
||
</body>
|
||
</html> |