.. -*- mode: rst -*- .. _server-plugins-generators-cfg: === Cfg === The Cfg plugin provides a repository to describe configuration file contents for clients. In its simplest form, the Cfg repository is just a directory tree modeled off of the directory tree on your client machines. The Cfg Repository ================== The Cfg plugin is enabled by including **Cfg** on the **plugins** line of the **[server]** section of your Bcfg2 server config file. The repository itself lives in ``/var/lib/bcfg2/Cfg``, assuming you are using the default repository location of ``/var/lib/bcfg2``. The contents of this directory are a series of directories corresponding to the real-life locations of the files on your clients, starting at the root level. For example:: % ls Cfg bin/ boot/ etc/ opt/ root/ usr/ var/ Specific config files go in like-named directories in this heirarchy. For example the password file, ``/etc/passwd``, goes in ``Cfg/etc/passwd/passwd``, while the ssh pam module config file, ``/etc/pam.d/sshd``, goes in ``Cfg/etc/pam.d/sshd/sshd``. The reason for the like-name directory is to allow multiple versions of each file to exist, as described below. Note that these files are exact copies of what will appear on the client machine (except when using Genshi or Cheetah templating -- see below). Group-Specific Files ==================== It is often the case that you want one version of a config file for all of your machines except those in a particular group. For example, ``/etc/fstab`` should look alike on all of your desktop machines, but should be different on your file servers. Bcfg2 can handle this case through use of group-specific files. As mentioned above, all Cfg entries live in like-named directories at the end of their directory tree. In the case of fstab, the file at ``Cfg/etc/fstab/fstab`` will be handed out by default to any client that asks for a copy of ``/etc/fstab``. Group-specific files are located in the same directory and are named with the following syntax:: /path/to/filename/filename.GNN_groupname **NN** is a priority number where **00** is lowest and **99** is highest, and **groupname** is the name of a group defined in ``Metadata/groups.xml``. Back to our fstab example, we might have a ``Cfg/etc/fstab/`` directory that looks like this:: fstab fstab.G50_server fstab.G99_fileserver By default, clients will receive the plain fstab file when they request ``/etc/fstab``. Any machine that is in the **server** group, however, will instead receive the ``fstab.G50_server`` file. Finally, any machine that is in the **fileserver** group will receive the ``fstab.G99_fileserver`` file, even if they are also in the **server** group. Host-Specific Files =================== Similar to the case with group-specific files, there are cases where a specific machine should have a different version of a file than all others. This can be accomplished with host-specific files. The format of a host-specific file name is:: /path/to/filename/filename.H_host.example.com Host-specific files have a higher priority than group specific files. Again, the fstab example:: fstab fstab.G50_server fstab.G99_fileserver fstab.H_host.example.com In this case, *host.example.com* will always get the host-specific version, even if it is part of the **server** or **fileserver** (or both) classes. .. note:: If you have the ability to choose between using a group-specific and a host-specific file, it is almost always best to use a group-specific one. That way if a hostname changes or an extra copy of a particular client is built, it will get the same changes as the original. Templates ========= .. _server-plugins-generators-cfg-genshi: Genshi Templates ---------------- Genshi templates allow you to use the `Genshi `_ templating system. This is similar to the deprecated :ref:`server-plugins-generators-tgenshi-index` plugin. Genshi templates should be named with a ``.genshi`` extension, e.g.:: % ls Cfg/etc/motd info.xml motd.genshi See the genshi `documentation `_ for examples of Genshi syntax. Troubleshooting ~~~~~~~~~~~~~~~ When developing a template, you can see what the template would generate on a client with :ref:`bcfg2-info `:: bcfg2-info buildfile E.g.:: bcfg2-info buildfile /etc/foo.conf foo.example.com To generate a file with an :ref:`altsrc ` attribute, you can run:: bcfg2-info buildfile /etc/foo/foo.conf --altsrc=/etc/foo.conf \ foo.example.com Sometimes, it's useful to be able to do more in-depth troubleshooting by running the template manually. To do this, run ``bcfg2-info debug``, and, once in the Python interpreter, run:: metadata = self.build_metadata("") source_path = "" name = source_path[len(self.setup['repo']):] Then, run:: import os from genshi.template import TemplateLoader, NewTextTemplate template = TemplateLoader().load(source_path, cls=NewTextTemplate) data = dict(metadata=metadata, source_path=source_path, path=source_path, name=name, repo=self.setup['repo']) print(template.generate(**data).render()) This gives you more fine-grained control over how your template is rendered. E.g., you can tweak the values of the variables passed to the template, or evaluate the template manually, line-by-line, and so on. You can also use this approach to render templates that depend on :ref:`altsrc ` tags by setting ``source_path`` to the path to the template, and setting ``name`` to the path to the file to be generated, e.g.:: metadata = self.build_metadata("foo.example.com") source_path = "/Cfg/etc/sysconfig/network-scripts/ifcfg-template/ifcfg-template.genshi" name = "/etc/sysconfig/network-scripts/ifcfg-bond0" Error handling ~~~~~~~~~~~~~~ Situations may arise where a templated file cannot be generated due to missing or incomplete information. A TemplateError can be raised to force a bind failure and prevent sending an incomplete file to the client. For example, this template:: {% python from genshi.template import TemplateError grp = None for g in metadata.groups: if g.startswith('ganglia-gmond-'): grp = g break else: raise TemplateError, "Missing group" %}\ will fail to bind if the client is not a member of a group starting with "ganglia-gmond-". The syslogs on the server will contain this message:: bcfg2-server[5957]: Genshi template error: Missing group bcfg2-server[5957]: Failed to bind entry: Path /etc/ganglia/gmond.conf ...indicating the bind failure and message raised with the TemplateError. Handling Dollar Signs ~~~~~~~~~~~~~~~~~~~~~ In a Genshi template, ``$`` is a special character and must be escaped by doubling, i.e., ``$$``. For instance, to embed the Subversion ``$Id$`` keyword in a Genshi template, you would have to do ``$$Id$$``. Examples ~~~~~~~~ .. toctree:: :glob: :maxdepth: 1 examples/genshi/* .. _server-plugins-generators-cfg-cheetah: Cheetah Templates ----------------- Cheetah templates allow you to use the `cheetah templating system `_. This is similar to the deprecated :ref:`server-plugins-generators-tcheetah` plugin. Cheetah templates should be named with a ``.cheetah`` extension, e.g.:: % ls Cfg/etc/motd info.xml motd.cheetah Examples ~~~~~~~~ .. toctree:: :glob: :maxdepth: 1 examples/cheetah/* Comments and Cheetah ~~~~~~~~~~~~~~~~~~~~ As Cheetah processes your templates it will consider hash "#" style comments to be actual comments in the template and will strip them from the final config file. If you would like to preserve the comment in the final config file you need to escape the hash character '\#' which will tell Cheetah (and Python) that you do in fact want the comment to appear in the final config file.:: # This is a comment in my template which will be stripped when it's processed through Cheetah \# This comment will appear in the generated config file. Inside Templates ---------------- Several variables are pre-defined inside templates: +-------------+--------------------------------------------------------+ | Name | Description | +=============+========================================================+ | metadata | :ref:`Client metadata | | | ` | +-------------+--------------------------------------------------------+ | name | The value of the ``name`` attribute as specified in | | | the Path entry in Bcfg2. If an :ref:`altsrc | | | ` attribute is used, | | | then ``name`` will be the value of that attribute. | +-------------+--------------------------------------------------------+ | source_path | The path to the template file on the filesystem | +-------------+--------------------------------------------------------+ | repo | The path to the Bcfg2 repository on the filesystem | +-------------+--------------------------------------------------------+ | path | In Genshi templates, ``path`` is a synonym for | | | ``source_path``. In Cheetah templates, it's a synonym | | | for ``name``. For this reason, use of ``path`` is | | | discouraged, and it may be deprecated in a future | | | release. | +-------------+--------------------------------------------------------+ To access these variables in a Genshi template, you can simply use the name, e.g.:: Path to this file: ${name} In a Cheetah template, the variables are properties of ``self``, e.g.:: Path to this file: $self.name Notes on Using Templates ------------------------ Templates can be host and group specific as well. Deltas will not be processed for any Genshi or Cheetah base file. .. note:: If you are using templating in combination with host-specific or group-specific files, you will need to ensure that the ``.genshi`` or ``.cheetah`` extension is at the **end** of the filename. Using the examples from above for *host.example.com* and group *server* you would have the following:: Cfg/etc/fstab/fstab.H_host.example.com.genshi Cfg/etc/fstab/fstab.G50_server.cheetah You can mix Genshi and Cheetah when using different host-specific or group-specific files. For example:: Cfg/etc/fstab/fstab.H_host.example.com.genshi Cfg/etc/fstab/fstab.G50_server.cheetah .. _server-plugins-generators-cfg-encryption: Encrypted Files =============== .. versionadded:: 1.3.0 Bcfg2 allows you to encrypt files stored in ``Cfg/`` to protect the data in them from other people who need access to the repository. See also :ref:`server-plugins-connectors-properties-encryption` for information on encrypting elements in Properties files, which is often more friendly for tracking changes in a VCS. .. note:: This feature is *not* intended to secure the files against a malicious attacker who has gained access to your Bcfg2 server, as the encryption passphrases are held in plaintext in ``bcfg2.conf``. This is only intended to make it easier to use a single Bcfg2 repository with multiple admins who should not necessarily have access to each other's sensitive data. See :ref:`server-encryption` for more details on encryption in Bcfg2 in general. Encrypting Files ---------------- An encrypted file should end with ``.crypt``, e.g.:: Cfg/etc/foo.conf Cfg/etc/foo.conf/foo.conf.crypt Cfg/etc/foo.conf/foo.conf.G10_foo.crypt Encrypted Genshi or Cheetah templates can have the extensions in either order, e.g.:: Cfg/etc/foo.conf/foo.conf.crypt.genshi Cfg/etc/foo.conf/foo.conf.G10_foo.genshi.crypt Cfg/etc/foo.conf/foo.conf.H_bar.example.com.crypt.cheetah To encrypt or decrypt a file, use :ref:`bcfg2-crypt`. .. _server-plugins-generators-cfg-sshkeys: SSH Keys ======== .. versionadded:: 1.3.0 Cfg can also be used to automatically create and distribute SSH key pairs and the ``authorized_keys`` file. Keys can be created one of two ways: * Host-specific keys, where each client has its own key pair. This is the default. * Group-specific keys. To do this, you must set ``category`` in either ``bcfg2.conf`` (see "Configuration" below) or in ``privkey.xml``. Keys created for a given client will be specific to that client's group in the specified category. Group-specific keys are useful if, for instance, you have multiple distinct environments (development, testing, production, for example) and want to maintain separate keys for each environment. This feature actually creates static keys, much like the :ref:`server-plugins-generators-sshbase` plugin creates SSH certificates. It doesn't generate them on the fly for each request; it generates the key once, then saves it to the filesystem. Creating key pairs ------------------ To create an SSH key pair, you need to define how the private key will be created in ``privkey.xml``. For instance, to create ``/home/foo/.ssh/id_rsa``, you would create ``/var/lib/bcfg2/Cfg/home/foo/.ssh/id_rsa/privkey.xml``. This will create *both* the private key and the public key; the latter is created by appending ``.pub`` to the private key filename. It is not possible to change the public key filename. You may *optionally* also create a corresponding ``pubkey.xml``, which will allow the key pair to be created when the public key is requested. (For the example above, you'd create ``/var/lib/bcfg2/Cfg/home/foo/.ssh/id_rsa.pub/pubkey.xml``. This can speed up the propagation of SSH keys throughout your managed systems, particularly if you use the ``authorized_keys`` generation feature. ``privkey.xml`` ~~~~~~~~~~~~~~~ ``privkey.xml`` contains a top-level ``PrivateKey`` element, and is structured as follows: .. xml:element:: PrivateKey :linktotype: See :ref:`server-encryption` for more details on encryption in Bcfg2 in general. ``pubkey.xml`` ~~~~~~~~~~~~~~~ ``pubkey.xml`` only ever contains a single line: .. code-block:: xml .. xml:element:: PublicKey It acts only as a flag to Bcfg2 that a key pair should be generated, if none exists, using the associated ``privkey.xml`` file. The path to ``privkey.xml`` is determined by removing ``.pub`` from the directory containing ``pubkey.xml``. I.e., if you create ``/var/lib/bcfg2/Cfg/home/foo/.ssh/id_rsa.pub/pubkey.xml``, then Bcfg2 will use ``/var/lib/bcfg2/Cfg/home/foo/.ssh/id_rsa/privkey.xml`` to create the key pair. Use of ``pubkey.xml`` is optional, but is recommended. If you do not use ``pubkey.xml`` files, you may encounter two problems: * On the first Bcfg2 client run on a given client, the private keys may be present but the public keys may not be. This will be fixed by running ``bcfg2`` again. * If you are including an automatically created public key in ``authorized_keys``, it will not be created until the client the key is for requests the key pair. As an example of this latter scenario, suppose that your ``authorized_keys.xml`` allows access to foo.example.com from ``/root/.ssh/id_rsa.pub`` for bar.example.com. If bar.example.com has not run the Bcfg2 client, then no key pair will have been generated, and generating the foo.example.com ``authorized_keys`` file will create a warning. But if you create ``Cfg/root/.ssh/id_rsa.pub/pubkey.xml``, then building ``authorized_keys`` for foo.example.com will create root's keypair for bar.example.com. .. note:: In order to use ``pubkey.xml``, there *must* be a corresponding ``privkey.xml``. You cannot, for instance, populate a directory with manually-generated private SSH keys, drop ``pubkey.xml`` in the related public key directory, and expect Bcfg2 to generate the public keys. It will not. Examples ~~~~~~~~ ``privkey.xml`` can, at its simplest, be very simple indeed: .. code-block:: xml This will create a private key with all defaults. Or it can be more complex: .. code-block:: xml U2FsdGVkX19xACol83uyPELP94s4CmngD12oU6PLLuE= This creates a 1024-bit DSA key for each group in the ``environment`` category, and keys for clients in the ``secure`` group will be protected with the given (encrypted) passphrase. To complete the example, assume that this file was saved at ``/var/lib/bcfg2/Cfg/home/foo/.ssh/id_rsa/privkey.xml``. If a client in the ``development`` group, which is a group in the ``environment`` category, requests the private key, then the following files would be created:: /var/lib/bcfg2/Cfg/home/foo/.ssh/id_rsa/id_rsa.G50_development /var/lib/bcfg2/Cfg/home/foo/.ssh/id_rsa.pub/id_rsa.pub.G50_development ``/var/lib/bcfg2/Cfg/home/foo/.ssh/id_rsa.pub`` would be created if it did not exist. Subsequent clients that were also members of the ``development`` environment would get the keys that have already been generated. ``pubkey.xml`` always contains a single empty tag: .. code-block:: xml Generating ``authorized_keys`` ------------------------------ ``authorized_keys`` can be automatically generated from public SSH keys that exist in the Cfg tree. The keys in question can be generated from ``privkey.xml``, or they can be manually created. If a key doesn't exist when ``authorized_keys`` is generated, the key will only be created if ``pubkey.xml`` exists. If that is not the case, a warning will be produced. To generate ``authorized_keys``, create ``authorized_keys.xml``, e.g.: ``/var/lib/bcfg2/Cfg/root/.ssh/authorized_keys/authorized_keys.xml``. ``authorized_keys.xml`` ~~~~~~~~~~~~~~~~~~~~~~~ ``authorized_keys.xml`` is structured as follows: .. xml:element:: AuthorizedKeys :linktotype: Example ~~~~~~~ .. code-block:: xml ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDw/rgKQeARRAHK5bQQhAAe1b+gzdtqBXWrZIQ6cIaLgxqj76TwZ3DY4A6aW9RgC4zzd0p4a9MfsScUIB4+UeZsx9GopUj4U6H8Vz7S3pXxrr4E9logVLuSfOLFbI/wMWNRuOANqquLYQ+JYWKeP4kagkVp0aAWp7mH5IOI0rp0A6qE2you4ep9N/nKvHDrtypwhYBWprsgTUXXMHnAWGmyuHGYWxNYBV9AARPdAvZfb8ggtuwibcOULlyK4DdVNbDTAN1/BDBE1ve6WZDcrc386KhqUGj/yoRyPjNZ46uZiOjRr3cdY6yUZoCwzzxvm5vle6mEbLjHgjGEMQMArzM9 vendor@example.com .. note:: ``authorized_keys.xml`` allows you to specify the group whose public key should be allowed. This retrieves the public key specific to that group (if it exists), *not* the public key for all hosts in that group. This is due to the performance penalties that would be imposed by that approach. Similarly, it is not possible to allow access from all keys for a given user (i.e., at a given path). Hopefully, the performance concerns can be resolved in a future release and these features can be added. Configuration ------------- In addition to ``privkey.xml`` and ``authorized_keys.xml``, described above, the behavior of the SSH key generation feature can be influenced by several options in the ``[sshkeys]`` section of ``bcfg2.conf``: +----------------+---------------------------------------------------------+-----------------------+------------+ | Option | Description | Values | Default | +================+=========================================================+=======================+============+ | ``passphrase`` | Use the named passphrase to encrypt private keys on the | String | None | | | filesystem. The passphrase must be defined in the | | | | | ``[encryption]`` section. See :ref:`server-encryption` | | | | | for more details on encryption in Bcfg2 in general. | | | +----------------+---------------------------------------------------------+-----------------------+------------+ | ``category`` | Generate keys specific to groups in the given category. | String | None | | | It is best to pick a category that all clients have a | | | | | group from. | | | +----------------+---------------------------------------------------------+-----------------------+------------+ Deltas ====== .. note:: In Bcfg2 1.3 and newer, deltas are deprecated. It is recommended that you use templates instead. The :ref:`TemplateHelper plugin ` comes with an example helper that can be used to include other files easily, a subset of cat file functionality. ``bcfg2-lint`` checks for deltas and warns about them. .. warning:: In Bcfg2 1.3, deltas **do not** work with `SSH key or authorized_keys generation `_. Bcfg2 has finer grained control over how to deliver configuration files to a host. Let's say we have a Group named file-server. Members of this group need the exact same ``/etc/motd`` as all other hosts except they need one line added. We could copy motd to ``motd.G01_file-server``, add the one line to the Group specific version and be done with it, but we're duplicating data in both files. What happens if we need to update the motd? We'll need to remember to update both files then. Here's where deltas come in. A delta is a small change to the base file. There are two types of deltas: cats and diffs. The cat delta simply adds or removes lines from the base file. The diff delta is more powerful since it can take a unified diff and apply it to the base configuration file to create the specialized file. Diff deltas should be used very sparingly. Cat Files --------- Continuing our example for cat files, we would first create a file named ``motd.G01_file-server.cat``. The .cat suffix designates that the file is a diff. We would then edit that file and add the following line:: +This is a file server The **+** at the begining of the file tells Bcfg2 that the line should be appended to end of the file. You can also start a line with **-** to tell Bcfg2 to remove that exact line wherever it might be in the file. How do we know what base file Bcfg2 will choose to use to apply a delta? The same rules apply as before: Bcfg2 will choose the highest priority, most specific file as the base and then apply deltas in the order of most specific and then increasing in priority. What does this mean in real life. Let's say our machine is a web server, mail server, and file server and we have the following configuration files:: motd motd.G01_web-server motd.G01_mail-server.cat motd.G02_file-server.cat motd.H_bar.example.com motd.H_foo.example.com.cat If our machine isn't *foo.example.com* or *bar.example.com*, but is a web server, then Bcfg2 would choose ``motd.G01_web-server`` as the base file. It is the most specific base file for this host. Bcfg2 would apply the ``motd.G01_mail-server.cat`` delta to the ``motd.G01_web-server`` base file. It is the least specific delta. Bcfg2 would then apply the ``motd.G02_file-server.cat`` delta to the result of the delta before it. If our machine is *foo.example.com* and a web server, then Bcfg2 would choose ``motd.G01_web-server`` as the base file. It is the most specific base file for this host. Bcfg2 would apply the ``motd.H_foo.example.com.cat`` delta to the ``motd.G01_web-server`` base file. The reason the other deltas aren't applied to *foo.example.com* is because a **.H_** delta is more specific than a **.G##_** delta. Bcfg2 applies all the deltas at the most specific level. If our machine is *bar.example.com*, then Bcfg2 would chose ``motd.H_foo.example.com`` as the base file because it is the most specific base file for this host. Regardless of the groups *bar.example.com* is a member of, **no cat files** would be applied, because only cat files as specific or more specific than the base file are applied. (In other words, if a group-specific base file is selected, only group- or host-specific cat files can be applied; if a host-specific base file is selected, only host-specific cat files can be applied.) .. _server-plugins-generators-cfg-validation: Content Validation ================== To ensure that files with invalid content are not pushed out, you can provide a content validation script that will be run against each file. Create a file called ``:test`` inside the directory for the file you want to test. For example:: Cfg/etc/sudoers/:test You can also create host- and group-specific validators:: Cfg/etc/sudoers/:test.G80_foogroup Cfg/etc/sudoers/:test.H_bar.example.com A validator script has the following attributes: * It must be executable, or specify a valid bangpath; * The entire content of the file is passed to the validator on stdin; * The validator is not called with any flags or arguments; * The validator must return 0 on success and non-zero on failure; and * The validator must output a sensible error message on failure. For ``sudoers``, a very simple validator is:: #!/bin/sh visudo -cf - This uses the ``visudo`` command's built-in validation. If you wish to disable validation, this can be done with the following setting in ``bcfg2.conf``:: [cfg] validation=no If you have a very large number of validators, you may wish to disable validation by default to avoid slowing down the generation of configurations on the server, and use ``bcfg2-test`` (for instance, as a post-commit hook or as part of a code review process) to run validation. You can do this by setting ``validation=no`` in ``bcfg2.conf`` as described above, and then calling ``bcfg2-test`` with the ``--cfg-validation`` flag. File permissions ================ File permissions for entries handled by Cfg are controlled via the use of :ref:`server-info` files. Note that you **cannot** use both a Permissions entry and a Path entry to handle the same file.