The goal of this driver is to resolve the issues that exist with the RPM and Yum client tool drivers.
For the most part, the issues are due to RPM being able to have multiple packages of the same name installed. This is an issue on all Red Hat and SUSE based distributions.
Examples of this are:
The new Pkgmgr format files with Instances are therefore the only way to accurately describe an RPM based system. It is recommended that all RPM based systems be changed to use the new format configuration files and the RPMng driver. Alternatively, you can use the newer Packages plugin.
Initial development of the drivers was done on Centos 4.4 x86_64, with testing on openSUSE 10.2 x86_64. Centos has been tested with a new style Pkgmgr file and openSUSE with an old style file (see the Configuration section below for what this means). Testing has now moved to Centos 5 x86_64 and old style files are no longer being tested.
RPMng/YUMng are the default RPM drivers.
The RPMng driver uses a mixture of rpm commands and rpm-python as detailed in the sections below.
The rpmtools module conatins most of the rpm-python code and is imported by RPMng.py and YUMng.py.
The RPMng.RefreshPackages method generates the installed dict using rpm-python code from the rpmtools module. Full name, epoch, version, release and arch information is stored.
The RPMng.VerifyPackages method generates a number of structures that record the state of the of the system compared to the Bcfg2 literal configuration retrieved from the server. These structures are mainly used by the RPMng.Install method.
AS part of the verification process an rpm package level verification is carried out using rpm-python code from the rpmtools module. Full details of the failures are returned in a complicated dict/list structure for later use.
The RPMng.Install method attempts to fix what the RPMng.VerifyPackages method found wrong. It does this by installing, reinstalling, deleting and upgrading RPMs. RPMng.Install does not use rpm-python. It does use the following rppm commands as appropriate:
rpm -install
rpm --import
rpm -upgrade
A method (RPMng.to reinstall_check()) to decide whether to do a reinstall of a package instance or not has been added, but is very simple at this stage. Currently it will prevent a reinstall if the only reason for a verification failure was due to an RPM configuration (%config) file. A package reinstall will not replace these, so there is no point reinstalling.
The RPMng.Remove method is written using rpm-python code in the rpmtools module. Full nevra information is used in the selection of the package removal.
This is a Python C extension module that checks to see if a file has been prelinked or not. It should be built and installed on systems that have the prelink package installed (only Red Hat family systems as far as I can tell). rpmtools will function without the isprelink module, but performance is not good.
Source can be found here ftp://ftp.mcs.anl.gov/pub/bcfg/isprelink-0.1.2.tar.gz
To compile and install prelink, execute:
python setup.py install
in the rpmtools directory. The elfutils-libelf-devel package is required for the compilation.
There are Centos x86_64 RPMs here ftp://ftp.mcs.anl.gov/pub/bcfg/archive/redhat/
The RPMng driver can be loaded by command line options, client configuration file options or as the default driver for RPM packages.
From the command line:
bcfg2 -n -v -d -D Action,POSIX,Chkconfig,RPMng
This produces quite a bit of output so you may want to redirect the output to a file for review.
In the bcfg2.conf file:
[client]
#drivers = Action,Chkconfig,POSIX,YUMng
drivers = Action,Chkconfig,POSIX,RPMng
Note
Note that loading this driver will unload the RPM driver, so the Yum driver will not work.
A number of paramters can be set in the client configuration for both the RPMng and YUMng drivers. Each driver has its own section. A full client configuration file with all the options specified is below:
[communication]
protocol = xmlrpc/ssl
password = xxxxxx
user = yyyyyyy
[components]
bcfg2 = https://bcfg2:6789
[client]
#drivers = Action,Chkconfig,POSIX,YUMng
drivers = Action,Chkconfig,POSIX,RPMng
[RPMng]
pkg_checks = true
pkg_verify = true
erase_flags = allmatches
installonlypackages = kernel, kernel-bigmem, kernel-enterprise, kernel-smp, kernel-modules, kernel-debug, kernel-unsupported, kernel-source, kernel-devel, kernel-default, kernel-largesmp-devel, kernel-largesmp, kernel-xen, gpg-pubkey
install_action = install
version_fail_action = upgrade
verify_fail_action = reinstall
[YUMng]
pkg_checks = True
pkg_verify = true
erase_flags = allmatches
autodep = true
installonlypackages = kernel, kernel-bigmem, kernel-enterprise, kernel-smp, kernel-modules, kernel-debug, kernel-unsupported, kernel-source, kernel-devel, kernel-default, kernel-largesmp-devel, kernel-largesmp, kernel-xen, gpg-pubkey
install_action = install
version_fail_action = upgrade
verify_fail_action = reinstall
Install only packages are packages that should only ever be installed or deleted, not upgraded.
The only packages for which this is an absolute on, are the gpg-pubkey packages. It is however ‘best’ practice to only ever install/delete kernel packages. The wisdom being that the package for the currently running kernel should always be installed. Doing an upgrade would delete the running kernel package.
The RPMng driver follows the YUM practice of having a list of install only packages. A default list is hard coded in RPMng.py. This maybe over ridden in the client configuration file.
Note that except for gpg-pubkey packages (which are always added to the list by the driver) the list in the client configuration file completely replaces the default list. An empty list means that there are no install only packages (except for gpg-pubkey), which is the behaviour of the old RPM driver.
Example - an empty list:
[RPMng]
installonlypackages =
Example - The default list:
[RPMng]
installonlypackages = kernel, kernel-bigmem, kernel-enterprise, kernel-smp, kernel-modules, kernel-debug, kernel-unsupported, kernel-source, kernel-devel, kernel-default, kernel-largesmp-devel, kernel-largesmp, kernel-xen, gpg-pubkey
erase_flags are rpm options used by ‘rpm -erase’ in the client Remove() method. The RPMng erase is written using rpm-python and does not use the rpm command.
The erase flags are specified in the client configuration file as a comma separated list and apply to all RPM erase operations. The default is:
[RPMng]
erase_flags = allmatches
The following rpm erase options are supported, see the rpm man page for details.:
noscripts
notriggers
repackage
allmatches
nodeps
Note
Note that specifying erase_flags in the configuration file completely replaces the default.
The RPMng/YUMng drivers do the following three checks/status:
Setting pkg_checks = true (the default) in the client configuration file means that all three checks will be done for all packages.
Setting pkg_checks = false in the client configuration file means that only the Installed check will be done for all packages.
The true/false value can be any combination of upper and lower case.
Note
The RPMng/YUMng drivers do the following three checks/status:
Setting pkg_verify = true (the default) in the client configuration file means that all three checks will be done for all packages as long as pkg_checks = true.
Setting pkg_verify = false in the client configuration file means that the rpm verify wil not be done for all packages on the client.
The true/false value can be any combination of upper and lower case.
Note
The RPMng/YUMng drivers do the following three checks/status:
install_action controls whether or not a package instance will be installed if the installed check fails (i.e. if the package instance isn’t installed).
If install_action = install then the package instance is installed. If install_action = none then the package instance is not installed.
Note
The RPMng/YUMng drivers do the following three checks/status:
version_fail_action controls whether or not a package instance will be updated if the version check fails (i.e. if the installed package instance isn’t the same version as specified in the configuration).
If version_fail_action = upgrade then the package instance is upgraded (or downgraded).
If version_fail_action = none then the package instance is not upgraded (or downgraded).
Note
The RPMng/YUMng drivers do the following three checks/status:
verify_fail_action controls whether or not a package instance will be reinstlled if the version check fails (i.e. if the installed package instance isn’t the same version as specified in the configuration).
If verify_fail_action = reinstall then the package instance is reinstalled. If verify_fail_action = none then the package instance is not reinstalled.
Note
Running the client in interactive mode (-I) prompts for the actions to be taken as before. Prompts are per package and may apply to multiple instances of that package. Each per package prompt will contain a list of actions per instance.
Actions are encoded as
D - Delete
I - Install
R - Reinstall
U - Upgrade/Downgrade
An example is below. The example is from a system that is still using the old Pkgmgr format, so the epoch and arch appear as ‘*’.:
Install/Upgrade/delete Package aaa_base instance(s) - R(*:10.2-38.*) (y/N)
Install/Upgrade/delete Package evms instance(s) - R(*:2.5.5-67.*) (y/N)
Install/Upgrade/delete Package gpg-pubkey instance(s) - D(*:9c800aca-40d8063e.*) D(*:0dfb3188-41ed929b.*) D(*:7e2e3b05-44748aba.*) D(*:a1912208-446a0899.*) D(*:9c777da4-4515b5fd.*) D(*:307e3d54-44201d5d.*) (y/N)
Install/Upgrade/delete Package module-init-tools instance(s) - R(*:3.2.2-62.*) (y/N)
Install/Upgrade/delete Package multipath-tools instance(s) - R(*:0.4.7-29.*) (y/N)
Install/Upgrade/delete Package pam instance(s) - R(*:0.99.6.3-29.1.*) (y/N)
Install/Upgrade/delete Package perl-AppConfig instance(s) - U(None:1.52-4.noarch -> *:1.63-17.*) (y/N)
Install/Upgrade/delete Package postfix instance(s) - R(*:2.3.2-28.*) (y/N)
Install/Upgrade/delete Package sysconfig instance(s) - R(*:0.60.4-3.*) (y/N)
Install/Upgrade/delete Package udev instance(s) - R(*:103-12.*) (y/N)
GPG is used by RPM to ‘sign’ packages. All vendor packages are signed with the vendors GPG key. Additional signatures maybe added to the rpm file at the users discretion.
It is normal to have multiple GPG keys installed. For example, SLES10 out of the box has six GPG keys installed.
To the RPM database all GPG ‘packages’ have the name ‘gpg-pubkey’, which may be nothing like the name of the file specified in the rpm -import command. For example on Centos 4 the file name is RPM-GPG-KEY-centos4. For SLES10 this means that there are six packages with the name ‘gpg-pubkey’ installed.
RPM does not check GPG keys at package installation, YUM does.
RPMng uses the rpm command for installation and does not therefore check GPG signatures at package install time. RPMng uses rpm-python for verification and does by default do signature checks as part of the client Inventory process. To do the signature check the appropriate GPG keys must be installed. rpm-python is not very friendly if the required key(s) is not installed (it crashes the client).
The RPMng driver detects, on a per package instance basis, if the appropriate key is installed. If it is not, a warning message is printed and the signature check is disabled for that package instance, for that client run only.
GPG keys can be installed and removed by the RPMng driver. To install a GPG key configure it in Pkgmgr/Rules as a package and add gpg-pubkey to the clients abstract configuration. The gpg-pubkey package/instance is treated as an install only package. gpg-pubkey packages are installed by the RPMng driver with the rpm -import command.
gpg-pubkey packages will be removed by bcfg2 -r packages if they are not in the clients configuration.
<PackageList uri='http://fortress/' priority='0' type='rpm'>
<Group name='Centos4.4-Standard'>
<Group name='x86_64'>
<Package name='gpg-pubkey' type='rpm'>
<Instance simplefile='mrepo/Centos44-x86_64/disc1/RPM-GPG-KEY-centos4' version='443e1821' release='421f218f'/>
<Instance simplefile='RPM-GPG-KEY-mbrady' version='9c777da4' release='4515b5fd'/>
</Package>
</Group>
</Group>
</PackageList>
Example gpg-pubkey Pkgmgr configuration file.
Also see the general Pkgmgr and altsrc pages.
Old style (meaning no Instance tag) Pkgmgr files have limited support. Specifically the multiarch and verify attributes are ignored.
If multiarch type support is needed a new style format file must be used.
If some control over the verification is needed, replace the verify attribute with the pkg_checks or verify_flags attributes. The pkg_checks and verify_flags attributes are detailed under the Instance tag heading.
The new style package tag supports the name and pkg_checks attributes and requires the use of Instance tag entries.
New style configuration files must be generated from the RPM headers. Either from RPM files or from the RPM DB.
The included pkgmgr_gen.py can be used as a starting point for generating configuration files from directories of RPM package files. pkgmgr_gen.py –help for the options.
The included pkgmgr_update.py can be used to update the package instance versions in configuration files from directories of package files. pkgmgr_update.py –help for the options.
| Attribute | Description | Values |
|---|---|---|
| name | Package name. | String |
| pkg_checks | Do the version and rpm verify checks. | true(default) or false |
The instance tag supports the following attributes:
| Attribute | Description | Values |
|---|---|---|
| simplefile | Package file name. | String (see Notes below) |
| epoch | Package epoch. | String (numeric only) (optional) |
| version | Package version. | String |
| release | Package release. | String |
| arch | Package architecture. | Architecture String e.g. (i386|i586|i686|x86_64) |
| verify_flags | Comma separated list of rpm –verify options. See the rpm man page for their details. | nodeps, nodigest, nofiles, noscripts, nosignature, nolinkto, nomd5, nosize, nouser, nogroup, nomtime, nomode, nordev |
| pkg_verify | Do the rpm verify | true(default) or false |
| install_action | Install package instance if it is not installed. | install(default) or none |
| version_fail_action | Upgrade package if the incorrect version is installed. | upgrade(default) or none |
| verify_fail_action | Reinstall the package instance if the rpm verify failed | reinstall(default) or none |
Note
The simplefile attribute doesn’t need to be just the filename, meaning the basename. It is joined with the uri attribute from the PackageList Tag to form the URL that the client will use to download the package. So the uri could just be the host portion of the url and simple file could be the directory path.
e.g.
<PackageList uri='http://fortress/' priority='0' type='rpm'>
<Group name='Centos4.4-Standard'>
<Group name='x86_64'>
<Package name='gpg-pubkey' type='rpm'>
<Instance simplefile='mrepo/Centos44-x86_64/disc1/RPM-GPG-KEY-centos4' version='443e1821' release='421f218f'/>
<Instance simplefile='RPM-GPG-KEY-mbrady' version='9c777da4' release='4515b5fd'/>
</Package>
</Group>
</Group>
</PackageList>
The values for epoch, version, release and arch attributes must come from the RPM header, not the RPM file name.
Epoch is a strange thing. In short:
epoch not set == epoch=None < epoch='0' < epoch='1'
and it is an int, but elementtree attributes have to be str or unicode, so the driver is constantly converting.
The Ignore tag is used to “mask out” individual files from the RPM verification. This is done by comparing the verification failure results with the Ignore tag name. If there is a match, that entry is not used by the client to determine if a package has failed verification.
Ignore tag entries can be specified at both the Package level, in which case they apply to all Instances, and/or at the Instance level, in which case they only apply to that instance.
Ignore tag entries are used by the RPMng driver. They can be specified in both old and new style Pkgmgr files.
The Ignore Tag supports the following attributes:
| Attribute | Description | Values |
|---|---|---|
| name | File name. | String |
Example
<Package name='glibc' type='rpm'>
<Ignore name='/etc/rpc'/>
<Instance simplefile='glibc-2.3.4-2.25.x86_64.rpm' version='2.3.4' release='2.25' arch='x86_64'/>
</Package>
The YUMng analog to the Ignore Tag used by RPMng is the use of Path entries of type ‘ignore’. The following shows an example for the centos-release package which doesn’t verify if you remove the default repos and replace them with a custom repo.
<!-- Ignore verification failures for centos-release -->
<BoundPath name='/etc/yum.repos.d/CentOS-Base.repo' type='ignore'/>
<BoundPath name='/etc/yum.repos.d/CentOS-Media.repo' type='ignore'/>
The two utilities detailed below are provided in the tools directory of the source tarball.
Also see the general Pkgmgr and altsrc pages.
pkgmgr_gen will generate a Pkgmgr file from a list of directories containing RPMs or from a list of YUM repositories.:
[root@bcfg2 Pkgmgr]# pkgmgr_gen.py --help usage: pkgmgr_gen.py
[options]
options:
-h, --help show this help message and exit
-aARCHS, --archs=ARCHS
Comma separated list of subarchitectures to include.
The highest subarichitecture required in an
architecture group should specified. Lower
subarchitecture packages will be loaded if that
is all that is available. e.g. The higher of i386,
i486 and i586 packages will be loaded if -a i586
is specified. (Default: all).
-dRPMDIRS, --rpmdirs=RPMDIRS
Comma separated list of directories to scan for RPMS.
Wilcards are permitted.
-eENDDATE, --enddate=ENDDATE
End date for RPM file selection.
-fFORMAT, --format=FORMAT
Format of the Output. Choices are yum or rpm.
(Default: yum)
-gGROUPS, --groups=GROUPS
List of comma separated groups to nest Package
entities in.
-iINDENT, --indent=INDENT
Number of leading spaces to indent nested entries in
the output.
(Default:4)
-oOUTFILE, --outfile=OUTFILE
Output file name.
-P, --pkgmgrhdr Include PackageList header in output.
-pPRIORITY, --priority=PRIORITY
Value to set priority attribute in the PackageList Tag.
(Default: 0)
-rRELEASE, --release=RELEASE
Which releases to include in the output. Choices are
all or latest. (Default: latest).
-sSTARTDATE, --startdate=STARTDATE
Start date for RPM file selection.
-uURI, --uri=URI URI for PackageList header required for RPM format
ouput.
-v, --verbose Enable verbose output.
-yYUMREPOS, --yumrepos=YUMREPOS
Comma separated list of YUM repository URLs to load.
NOTE: Each URL must end in a '/' character.
Note
The startdate and enddate options are not yet implemented.
pkgmgr_update will update the release (meaning the epoch, version and release) information in an existing Pkgrmgr file from a list of directories containing RPMs or from a list of YUM repositories. All Tags and other attributes in the existing file will remain unchanged.:
[root@bcfg2 Pkgmgr]# pkgmgr_update.py --help
usage: pkgmgr_update.py [options]
options:
-h, --help show this help message and exit
-cCONFIGFILE, --configfile=CONFIGFILE
Existing Pkgmgr configuration file name.
-dRPMDIRS, --rpmdirs=RPMDIRS
Comma separated list of directories to scan for RPMS.
Wilcards are permitted.
-oOUTFILE, --outfile=OUTFILE
Output file name or new Pkgrmgr file.
-v, --verbose Enable verbose output.
-yYUMREPOS, --yumrepos=YUMREPOS
Comma separated list of YUM repository URLs to load.
NOTE: Each URL must end in a '/' character.
This entry was used for the Centos test client used during RPMng development.
<Package name='bcfg2' type='rpm'>
<Instance simplefile='bcfg2-0.9.3-0.0pre5.noarch.rpm' version='0.9.3' release='0.0pre5' arch='noarch' verify_flags='nomd5,nosize,nomtime'/>
</Package>
<Package name='beecrypt' type='rpm'>
<Instance simplefile='beecrypt-3.1.0-6.x86_64.rpm' version='3.1.0' release='6' arch='x86_64'/>
<Instance simplefile='beecrypt-3.1.0-6.i386.rpm' version='3.1.0' release='6' arch='i386'/>
</Package>
Note
Multiple instances with the same architecture must be in the installOnlyPkgs list.
<Package name='kernel' type='rpm'>
<Instance simplefile='kernel-2.6.9-42.0.8.EL.x86_64.rpm' version='2.6.9' release='42.0.8.EL' arch='x86_64'/>
<Instance simplefile='kernel-2.6.9-42.0.10.EL.x86_64.rpm' version='2.6.9' release='42.0.10.EL' arch='x86_64'/>
</Package>
Note
In this case a per instance ignore is actually a bad idea as the verify failure is because of multiarch issues where the last package installed wins. So this would be better as a Package level ignore.
Ignore tag entries only work with the RPMng driver. They do not appear to be supported in YUMng as of 1.0pre5.
<Package name='glibc' type='rpm'>
<Instance simplefile='glibc-2.3.4-2.25.x86_64.rpm' version='2.3.4' release='2.25' arch='x86_64'>
<Ignore name='/etc/rpc'/>
</Instance>
<Instance simplefile='glibc-2.3.4-2.25.i686.rpm' version='2.3.4' release='2.25' arch='i686'/>
</Package>
If pkg_checks = false the version information is not required. If pkg_checks = true the full information is needed as normal.
For YUMng a minimal entry is
<Package name="bcfg2" type="yum" pkg_checks="False"/>
In fact for YUMng, with pkg_checks = false, any combination of the nevra attributes that will build a valid yum package name (see the Misc heading on the yum man page) is valid.
<Package name="bcfg2" type="yum" pkg_checks="False" arch="x86_64"/>
For RPMng a minimal entry is
<Package name="bcfg2" type="rpm" pkg_checks="False" simplefile="bcfg2-0.9.4-0.0pre1.noarch.rpm"/>
The way I have Bcfg2 configured for my development systems. This way it reports bad, but doesn’t do anything about it.
<Package name='bcfg2' type='rpm'>
<Instance simplefile='bcfg2-0.9.3-0.0pre5.noarch.rpm' version='0.9.3' release='0.0pre5' arch='noarch' verify_fail_action='none'/>
</Package>