ansible+vault

content/slides/cri/devops.html

@@ -66,7 +66,7 @@
        </section>
       </div>
       <div class="slides">
-        <section data-markdown="md/stack.md"
+        <section data-markdown="md/terraform.md"
          data-separator="^\n\n\n"
          data-separator-vertical="^\n\n"
          data-separator-notes="^Note:"
@@ -74,7 +74,7 @@
        </section>
       </div>
       <div class="slides">
-        <section data-markdown="md/terraform.md"
+        <section data-markdown="md/stack.md"
          data-separator="^\n\n\n"
          data-separator-vertical="^\n\n"
          data-separator-notes="^Note:"

content/slides/cri/md/ansible.md

@@ -432,72 +432,131 @@ ignore_errors: True
 ```
 
 
-## task
+## [<i class="fa fa-book" aria-hidden="true"></i> set_fact](https://docs.ansible.com/ansible/latest/modules/set_fact_module.html) & pre_task
 
-[<i class="fa fa-book" aria-hidden="true"></i> set_fact](https://docs.ansible.com/ansible/latest/modules/set_fact_module.html)
+```yaml
+criprod:
+  pvecriprod1:
+    api_users:
+      - proxmoxapi
+      - vimazeno
+```
+
+```yaml
+- name: provisionner l'environnement du noeud (pour y accéder plus facilement dans les roles)
+  set_fact:
+    _pve: "{ 'cluster': '{{ lookup('env','PVE_CLUSTER') }}', 'node': '{{ lookup('env','PVE_NODE') }}', 'host': '{{ lookup('env','PVE_HOST') }}'}"
 
-pre_task
+- name: provisionner les utilisateurs d'api pve uniquement du noeud (pour y accéder plus facilement dans les roles)
+  set_fact:
+    api_users: "{ 'api_users': {{ hostvars[inventory_hostname][_pve.cluster][_pve.node]['api_users'] }}}"
+
+- name: fusionner l'environnement du noeud (pour y accéder plus facilement dans les roles)
+  set_fact:
+    pve: "{{ _pve | combine(api_users) }}"
+```
 
 [<i class="fa fa-gitlab" aria-hidden="true"></i> exemple permettant de réorganiser les variables](https://gitlab.isima.fr/cri/stack/blob/master/ansible/pre-tasks/set-pve-vars.yml)
 
 
 ## tags
 
-* `always`
-  * tag spécial exécuté à tous les coups
-* tags au niveau tâches
-* tags au niveau roles
+tags au niveau tâches
+
+```yaml
+- name: s'assurer que le fichier user.cfg existe
+  file:
+    dest: /etc/pve/user.cfg
+    state: touch
+  tags: [pve-users]
+```
+
+tags au niveau roles à l'inclusion dans le playbook
+
+```yaml
+roles:
+
+  - role: debug
+    tags: debug
+```
+
+`always` tag spécial exécuté à tous les coups
 
 
 ## modules
 
-* file
-* infile
-* copy
+* [<i class="fa fa-book" aria-hidden="true"></i> file](https://docs.ansible.com/ansible/latest/modules/file_module.html)
+* [<i class="fa fa-book" aria-hidden="true"></i> lineinfile](https://docs.ansible.com/ansible/latest/modules/lineinfile_module.html)
+* [<i class="fa fa-book" aria-hidden="true"></i> copy](https://docs.ansible.com/ansible/latest/modules/copy_module.html)
 * [<i class="fa fa-book" aria-hidden="true"></i> template](https://docs.ansible.com/ansible/latest/modules/template_module.html)
-* package
-* stat
-* get_url
-* unarchive
-* user
-* systemd
-* pip
-* pexpect (require pexpect)
+* [<i class="fa fa-book" aria-hidden="true"></i> stat](https://docs.ansible.com/ansible/latest/modules/stat_module.html)
+* [<i class="fa fa-book" aria-hidden="true"></i> get_url](https://docs.ansible.com/ansible/latest/modules/get_url_module.html)
+* [<i class="fa fa-book" aria-hidden="true"></i> unarchive](https://docs.ansible.com/ansible/latest/modules/unarchive_module.html)
+
+
+## modules
+
+* [<i class="fa fa-book" aria-hidden="true"></i> package](https://docs.ansible.com/ansible/latest/modules/package_module.html)
+* [<i class="fa fa-book" aria-hidden="true"></i> user](https://docs.ansible.com/ansible/latest/modules/user_module.html)
+* [<i class="fa fa-book" aria-hidden="true"></i> systemd](https://docs.ansible.com/ansible/latest/modules/systemd_module.html)
+* [<i class="fa fa-book" aria-hidden="true"></i> pip](https://docs.ansible.com/ansible/latest/modules/pip_module.html)
+* [<i class="fa fa-book" aria-hidden="true"></i> expect](https://docs.ansible.com/ansible/latest/modules/exepect_module.html)
+  * `pip install pexpect`
+* [<i class="fa fa-book" aria-hidden="true"></i> ...](https://docs.ansible.com/ansible/latest/modules/modules_by_category.html)
 
 
 ## roles
 
+[<i class="fa fa-book" aria-hidden="true"></i> debops](https://docs.debops.org/en/master/)
+
 * [<i class="fa fa-github" aria-hidden="true"></i> bau-sec/ansible-openvpn-hardened](https://github.com/bau-sec/ansible-openvpn-hardened)
+* [<i class="fa fa-github" aria-hidden="true"></i> ...](https://github.com/)
 
+### skeleton
 
-## ansible.cfg
+```
+ansible-galaxy init --role-skeleton /path/to/stack/ansible/roles/skeletons/role-with-vagrant gitlab
+```
+[<i class="fa fa-gitlab" aria-hidden="true"></i>ansible/roles/skeletons/role-with-vagrant](https://gitlab.isima.fr/cri/stack/tree/master/ansible/roles/skeletons/role-with-vagrant)
 
 
 ## remote roles
 
-* mettre à jour `requirements.yml` avec les rôles nécessaires
-
-* récupérer les rôles
+`requirements.yml`
 
+```yaml
+- name: vault-server
+  src: git+ssh://git@gitlab.isima.fr/cri/ansible-playbook-vault.git
+  path: ./ansible/roles/remotes
 ```
-ansible-galaxy install -f -r requirements.yml -p ansible/roles/public
-```
-ou
+
 ```
-ansible-galaxy install -f -r requirements.yml
+$ ansible-galaxy install -f -r requirements.yml
 ```
-en affectant le path dans le fichier requirements.yml au niveau du role distant
 
 
-## skeleton
+## ansible.cfg
 
+```ini
+[defaults]
+roles_path = ./ansible/roles/remotes:./ansible/roles/apps:./ansible/roles/commons:./ansible/roles/services
+inventory = ./ansible/inventory.ini
+filter_plugins = ./ansible/plugins/filter:
+lookup_plugins = ./ansible/plugins/lookup:
+callback_plugins = ./ansible/plugins/callback:
+module_utils = ./ansible/module_utils:
+stdout_callback = anstomlog
+deprecation_warnings = False
+
+[privilege_escalation]
+become: yes
+become_user: root
+become_method: sudo
 ```
-ansible-galaxy init --role-skeleton /path/to/stack/ansible/roles/skeletons/role-with-vagrant gitlab
-```
-* [<i class="fa fa-github" aria-hidden="true"></i> mrjk/ansible-skel](https://github.com/mrjk/ansible-skel)
 
 
 ## extend
 
-* https://docs.ansible.com/ansible/latest/dev_guide/developing_plugins.html
-* https://docs.ansible.com/ansible/latest/dev_guide/developing_modules.html
+[<i class="fa fa-book" aria-hidden="true"></i> developing plugins](https://docs.ansible.com/ansible/latest/dev_guide/developing_plugins.html)
+
+[<i class="fa fa-book" aria-hidden="true"></i> developing modules](https://docs.ansible.com/ansible/latest/dev_guide/developing_modules.html)

content/slides/cri/md/vault.md

@@ -9,10 +9,13 @@
 
 * téléchargement d'un binaire
 
-  * https://releases.hashicorp.com/vault/
+  * [https://releases.hashicorp.com/vault/](https://releases.hashicorp.com/vault/)
   * décompresser dans /usr/local/bin
   * configurer les permissions
-  * créer un service systemd
+  * serveur
+    * créer un service systemd
+  * cli
+    * `vault`
 
 
 ## Configuration
@@ -32,20 +35,58 @@ listener "tcp" {
 ```
 
 
-## Secret engine
+## [<i class="fa fa-book" aria-hidden="true"></i> Secret engine](https://www.vaultproject.io/docs/secrets/)
 
+* [<i class="fa fa-book" aria-hidden="true"></i> Secrets Engines - getting started](https://learn.hashicorp.com/vault/getting-started/dynamic-secrets)
+* [<i class="fa fa-book" aria-hidden="true"></i> AWS Secrets Engine](https://www.vaultproject.io/docs/secrets/aws/index.html)
+* [<i class="fa fa-book" aria-hidden="true"></i> Active Directory Secrets Engine](https://www.vaultproject.io/docs/secrets/aws/index.html)
+* [<i class="fa fa-book" aria-hidden="true"></i> SSH Secrets Engine](https://www.vaultproject.io/docs/secrets/ssh/index.html)
+* [<i class="fa fa-book" aria-hidden="true"></i> KV Secrets Engine](https://www.vaultproject.io/docs/secrets/kv/index.html)
 
-## KV2
 
+## KV
+
+```shell
+$ vault kv get secret/test
+====== Data ======
+Key          Value
+---          -----
+password1    secret$
+
+$ vault kv put secret/test password2=secret!
+Success! Data written to: secret/test
+
+$ vault kv get secret/test
+====== Data ======
+Key          Value
+---          -----
+password2    secret!
 ```
+
+
+## KV2
+
+```shell
 vault login token=<root-token>
 vault secrets enable -path=cri kv
-vault kv enable-versioning cri/ # kv2
+vault kv enable-versioning secret/ # kv2
 ```
 
 * les secrets sont versionnés
 * il est possible d'utiliser PATCH et pas seulement PUT
 
+```shell
+$ vault kv patch secret/test password1=secret$
+Success! Data written to: secret/test
+
+$ vault kv get secret/test
+====== Data ======
+Key          Value
+---          -----
+password1    secret$
+password2    secret!
+```
+
 
 ## Authentification
 
@@ -59,7 +100,7 @@ vault kv enable-versioning cri/ # kv2
 
 ## LDAP
 
-```
+```shell
 $ vault write auth/ldap/config \
     url="ldaps://samantha.local.isima.fr" \
     userattr="sAMAccountName" \
@@ -73,6 +114,8 @@ $ vault write auth/ldap/config \
     starttls="true"
 ```
 
+[<i class="fa fa-book" aria-hidden="true"></i> LDAP Auth Method](https://www.vaultproject.io/docs/auth/ldap.html)
+
 
 ## Policy
 
@@ -81,7 +124,7 @@ $ vault write auth/ldap/config \
 ```
 # Write and manage secrets in key-value secret engine
 path "secret/*" {
-  capabilities = ["create", "read", "update", "delete", "list"]
+  capabilities = ["create", "read", "update", "delete", "list", "sudo"]
 }
 
 # To enable secret engines
@@ -89,89 +132,123 @@ path "sys/mounts/*" {
   capabilities = [ "create", "read", "update", "delete" ]
 }
 
-path "secret/data/cri/*" {
-  capabilities = ["create", "read", "update", "delete", "list", "sudo"]
-}
-
 path "cubbyhole/*" {
   capabilities = ["create", "read", "update", "delete", "list"]
 }
 
 ```
 
-```
+```shell
 $ vault policy write cri /etc/vault/cri.hcl
 ```
 
 
-## Map policy and ldap group
+## appliquer une policy à un groupe ldap
 
-```
+```shell
 $ vault write auth/ldap/groups/cri policies=cri
 ```
 
 
 ## Utilisation
 
-* le même binaire à télécharger
-  * cross plateforme
-* deux variables d'environnement
-  * $VAULT_ADDR=https://vault.isima.fr
-  * $VAULT_TOKEN
+* [binaire à télécharger](https://releases.hashicorp.com/vault/)
+  * cross plateform
+  * deux variables d'environnement
+    * $VAULT_ADDR=https://vault.isima.fr
+    * $VAULT_TOKEN ou authentification ldap
 ou
-* l'[api](https://www.vaultproject.io/api/overview)
+* l'[<i class="fa fa-book" aria-hidden="true"></i> api](https://www.vaultproject.io/api/overview)
 
 
 ## Workflow
 
-```
+```shell
 $ vault login -method=ldap username=vimazeno
 $ vault secrets list
-$ vault list cri/
-$ vault kv get  cri/tokens
-$ vault kv get  cri/tokens # à chaque put on écrase les entrées qu'on ne réécrit pas
-$ vault kv get  cri/tokens -format=json
-$ vault kv get  cri/tokens -format=json | jq .data
-$ vault kv get cri/tokens -format=json | jq .data.password
-$ vault kv put cri/tokens root=pipo2
-$ vault kv patch cri/tokens root1=pipo1
-$ vault delete cri/tokens
+$ vault list secret/
+$ vault kv get  secret/tokens
+$ vault kv get  secret/tokens # à chaque put on écrase les entrées qu'on ne réécrit pas
+$ vault kv get  secret/tokens -format=json
+$ vault kv get  secret/tokens -format=json | jq .data
+$ vault kv get secret/tokens -format=json | jq .data.password
+$ vault kv put secret/tokens root=$(openssl rand -base64 25)
+$ vault kv patch secret/tokens root1=$(openssl rand -base64 25)
+$ vault delete secret/tokens
 ```
 
 
 ## création de token
 
+my.hcl
+
 ```
+path "secret/data/cri/apps/my" {
+  capabilities = ["create", "read", "update", "delete", "list"]
+}
+```
+
+```shell
 $ vault policy write vault/apps/my.hcl
 $ vault token create -policy=my
 ```
 
 
-## token et ci/cd
-
-* on teste en local avec ses droits via ldaps
-* on génère un token en variable de CI/CD pour autoriser l'appli à lire des secrets
+## vault/ci/cd
 
+### en local
 
-## Générer un secret
+authentification ldap
 
-Un secret robuste en ligne de commande se génère avec la commande `openssl` comme suit
+### <i class="fa fa-gitlab" aria-hidden="true"></i> CI / CD
 
-```
-$ openssl rand -base64 256
-```
-
-Stocker le secret directement dans hashicorp vault
+![vault CI](images/vault-ci.png)
 
-```
-$ vault kv put cri/tokens root=$(openssl rand -base64 25)
-```
 
+## bin/setup
 
-## SEE ALSO
-
-* [cri/ansible-playbook-vault](https://gitlab.isima.fr/cri/ansible-playbook-vault)
-
-* [Vault - Getting started](https://learn.hashicorp.com/vault/?track=getting-started#getting-started)
-
-* [consul](https://www.consul.io/)
+<small>
+```bash
+command -v "vault" >/dev/null 2>&1 || {
+  echo >&2 "I require vault to run see stack"
+  exit 1
+}
+if [[ -z "${VAULT_ADDR}" ]] ; then
+  export VAULT_ADDR=https://vault.isima.fr
+fi
+if [[ -z "${VAULT_TOKEN}" ]] ; then
+  if [[ -z "${VAULT_USERNAME}" ]] ; then
+    echo uca username
+    read username
+    export VAULT_USERNAME=${username}
+  fi
+  vault login -method=ldap username=$VAULT_USERNAME > /dev/null
+  echo " export VAULT_TOKEN=$(cat ~/.vault-token)"
+else
+  vault login token=${VAULT_TOKEN} > /dev/null
+fi
+```
+</small>
+
+
+## bin/configure
+
+<small>
+```bash
+# lecture des clés vault avec python: la sortie est une liste python UTF8 (u'value')
+KV=$(vault read cri/my -format=json | python -c "import sys, json; print json.load(sys.stdin)['data'].keys()")
+# converison de la liste python en liste bash
+VAULT_KEYS=( $(echo ${KV} | sed -r "s/', u'/' '/g" | sed -r "s/\[u'/'/g" | sed -r "s/\]//g") )
+# copie du ttemplatye de configuration en fichier de configuration
+cp config.sample.py config.py
+# itération sur les clés vault
+for i in "${VAULT_KEYS[@]}"
+do
+  # enlève le permier '
+  i=${i%\'}
+  # enlève le dernier '
+  i=${i#\'}
+  sed -i "s|$i|$(vault read cri/my -format=json | jq -r .data.$i | sed -r "s/\n//g")|g" config.py 2>/dev/null
+done
+```
+</small>