New in version 1.3.0.
Bcfg2 1.3 added a pluggable server core system so that the server core itself can be easily swapped out to use different technologies. It currently ships with several backends: a builtin core written from scratch using the various server tools in the Python standard library; a variant on the builtin core that uses Python 2.6’s multiprocessing library to process requests in parallel; and an experimental CherryPy based core. This page documents the server core interface so that other cores can be written to take advantage of other technologies, e.g., Tornado or Twisted.
A core implementation needs to:
A core that wants to use the network (i.e., a core that isn’t used entirely for introspection, as in bcfg2-info, or other local tasks) should inherit from Bcfg2.Server.Core.NetworkCore, and must also override Bcfg2.Server.Core.NetworkCore._daemonize() to handle daemonization, writing the PID file, and dropping privileges.
Nearly all XML-RPC handling is delegated entirely to the core implementation. It needs to:
Additionally, running and configuring the server is delegated to the core. It needs to honor the configuration options that influence how and where the server runs, including the server location (host and port), listening interfaces, and SSL certificate and key.
Bcfg2.Server.Core provides the base core object that server core implementations inherit from.
Bases: object
The server core is the container for all Bcfg2 server logic and modules. All core implementations must inherit from Core.
Start up the server; this method should return immediately. This must be overridden by a core implementation.
Enter the infinite loop. This method should not return until the server is killed. This must be overridden by a core implementation.
The thread that runs the Bcfg2.Server.FileMonitor.FileMonitor. This also queries Bcfg2.Server.Plugin.interfaces.Version plugins for the current revision of the Bcfg2 repo.
Set profile for a client.
Parameters: | address (tuple) – Client (address, port) pair |
---|---|
Returns: | bool - True on success |
Raises : | xmlrpclib.Fault |
Bind a single entry using the appropriate generator.
Parameters: |
|
---|
Bind all elements in a single structure (i.e., bundle).
Parameters: |
|
---|
Given a list of structures (i.e. bundles), bind all the entries in them and add the structures to the config.
Parameters: |
|
---|
Build the complete configuration for a client.
Parameters: | client (string) – The hostname of the client to build the configuration for |
---|---|
Returns: | lxml.etree._Element - A complete Bcfg2 configuration document |
Declare the client version.
Parameters: | |
---|---|
Returns: | bool - True on success |
Raises : | xmlrpclib.Fault |
Build config for a client by calling BuildConfiguration().
Parameters: | address (tuple) – Client (address, port) pair |
---|---|
Returns: | lxml.etree._Element - The full configuration document for the client |
Raises : | xmlrpclib.Fault |
Get the decision list for the client with GetDecisions().
Parameters: | address (tuple) – Client (address, port) pair |
---|---|
Returns: | list of decision tuples |
Raises : | xmlrpclib.Fault |
Get the decision list for a client.
Parameters: |
|
---|---|
Returns: | list of Decision tuples (<entry tag>, <entry name>) |
Fetch probes for the client.
Parameters: | address (tuple) – Client (address, port) pair |
---|---|
Returns: | lxml.etree._Element - XML tree describing probes for this client |
Raises : | xmlrpclib.Fault |
Get all structures (i.e., bundles) for the given client
Parameters: | metadata (Bcfg2.Server.Plugins.Metadata.ClientMetadata) – Client metadata to get structures for |
---|---|
Returns: | list of lxml.etree._Element objects |
Handle a change in the Bcfg2 config file.
Parameters: | event (Bcfg2.Server.FileMonitor.Event) – The event to handle |
---|
Receive probe data from clients.
Parameters: | address (tuple) – Client (address, port) pair |
---|---|
Returns: | bool - True on success |
Raises : | xmlrpclib.Fault |
Act on statistics upload with process_statistics().
Parameters: | address (tuple) – Client (address, port) pair |
---|---|
Returns: | bool - True on success |
Raises : | xmlrpclib.Fault |
Block until all fam events have been handleed, optionally handling events as well. (Setting handle_events=True is useful for local server cores that don’t spawn an event handling thread.)
Build initial client metadata for a client
Parameters: | client_name (string) – The name of the client to build metadata for |
---|---|
Returns: | Bcfg2.Server.Plugins.Metadata.ClientMetadata |
Path to bcfg2.conf
Check client IP address and metadata object against all Bcfg2.Server.Plugin.interfaces.ClientACLs plugins. If any ACL plugin denies access, then access is denied. ACLs are checked in two phases: First, with the client IP address; and second, with the client metadata object. This lets an ACL interface do a quick rejection based on IP before metadata is ever built.
Parameters: |
|
---|---|
Returns: | bool |
Invoke hooks from Bcfg2.Server.Plugin.interfaces.ClientRunHooks plugins for a given stage.
Parameters: |
|
---|
Log an error with its traceback and return an XML-RPC fault to the client.
Parameters: | message (string) – The message to log and return to the client |
---|---|
Raises : | xmlrpclib.Fault |
RLock to be held on writes to the backend db
Used to keep track of the current debug state of the core.
Expire the metadata cache for one or all clients
Parameters: | hostnames (None or list of strings) – A list of hostnames to expire the metadata cache for or None. If None the cache of all clients will be expired. |
---|
The Bcfg2.Server.FileMonitor.FileMonitor object used by the core to monitor for Bcfg2 data changes.
The FAM threading.Thread, _file_monitor_thread()
Get current statistics about component execution from Bcfg2.Server.Statistics.stats.
Returns: | dict - The statistics data as returned by Bcfg2.Server.Statistics.Statistics.display() |
---|
Import and instantiate a single plugin. The plugin is stored to plugins.
Parameters: | plugin (type) – The plugin class to load. |
---|---|
Returns: | None |
List all exposed methods, including plugin RMI.
Parameters: | address (tuple) – Client (address, port) pair |
---|---|
Returns: | list of exposed method names |
Load all plugins, setting Bcfg2.Server.Core.BaseCore.plugins and Bcfg2.Server.Core.BaseCore.metadata as side effects. This does not start plugin threads; that is done later, in Bcfg2.Server.Core.BaseCore.run()
A threading.Lock() for use by Bcfg2.Server.FileMonitor.FileMonitor.handle_event_set()
A logging.Logger object for use by the core
The Metadata plugin
A Bcfg2.Server.Cache.Cache object for caching client metadata
Get the client metadata_cache mode. Options are off, initial, cautious, aggressive, on (synonym for cautious). See Server-side Caching for more details.
Get help from the docstring of an exposed method
Parameters: | |
---|---|
Returns: | string - The help message from the method’s docstring |
The name of this server core. This can be overridden by core implementations to provide a more specific name.
The threading.Thread that reports performance statistics to syslog.
Blacklist of plugins that conflict with enabled plugins. If two plugins are loaded that conflict with each other, the first one loaded wins.
Dict of plugins that are enabled. Keys are the plugin names (just the plugin name, in the correct case; e.g., “Cfg”, not “Bcfg2.Server.Plugins.Cfg”), and values are plugin objects.
Return a list of loaded plugins that match the passed type.
The returned list is sorted in ascending order by the plugins’ sort_order value. The Bcfg2.Server.Plugin.base.Plugin.sort_order defaults to 500, but can be overridden by individual plugins. Plugins with the same numerical sort_order value are sorted in alphabetical order by their name.
Parameters: | base_cls (type) – The base plugin interface class to match (see Bcfg2.Server.Plugin.interfaces) |
---|---|
Returns: | list of Bcfg2.Server.Plugin.base.Plugin objects |
Process uploaded statistics for client.
Parameters: |
|
---|
Given a client address, get the client hostname and optionally metadata.
Parameters: |
|
---|---|
Returns: | tuple - If metadata is False, returns (<canonical hostname>, None); if metadata is True, returns (<canonical hostname>, <client metadata object>) |
Revision of the Bcfg2 specification. This will be sent to the client in the configuration, and can be set by a Bcfg2.Server.Plugin.interfaces.Version plugin.
Run the server core. This calls _run(), starts the fam_thread, and calls _block(), but note that it is the responsibility of the server core implementation to call shutdown() under normal operation. This also handles creation of the directory containing the pidfile, if necessary.
Explicity set debug status of the server core
Parameters: | debug (bool or string) – The new debug status. This can either be a boolean, or a string describing the state (e.g., “true” or “false”; case-insensitive) |
---|---|
Returns: | bool - The new debug state of the FAM |
Explicitly set debug status of the FAM and all plugins
Parameters: |
|
---|---|
Returns: | bool - The new debug state |
Explicitly set debug status of the FAM
Parameters: | debug (bool or string) – The new debug status of the FAM. This can either be a boolean, or a string describing the state (e.g., “true” or “false”; case-insensitive) |
---|---|
Returns: | bool - The new debug state of the FAM |
Threading event to signal worker threads (e.g., fam_thread) to shutdown
Toggle debug status of the server core
Parameters: | address (tuple) – Client (address, hostname) pair |
---|---|
Returns: | bool - The new debug state of the FAM |
Toggle debug status of the FAM and all plugins
Parameters: | address (tuple) – Client (address, port) pair |
---|---|
Returns: | bool - The new debug state of the FAM |
Toggle debug status of the FAM
Returns: | bool - The new debug state of the FAM |
---|
Checks that the config matches the goals enforced by Bcfg2.Server.Plugin.interfaces.GoalValidator plugins by calling Bcfg2.Server.Plugin.interfaces.GoalValidator.validate_goals().
Parameters: |
|
---|
Checks the data structures by calling the Bcfg2.Server.Plugin.interfaces.StructureValidator.validate_structures() method of Bcfg2.Server.Plugin.interfaces.StructureValidator plugins.
Parameters: |
|
---|
Bases: exceptions.Exception
Raised when the server core cannot be initialized.
Bases: Bcfg2.Server.Plugin.base.Plugin, Bcfg2.Server.Plugin.interfaces.ClientACLs
Default ACL ‘plugin’ that provides security by default. This is only loaded if no other ClientACLs plugin is enabled.
Bases: Bcfg2.Server.Core.Core
A server core that actually listens on the network, can be daemonized, etc.
Daemonize the server and write the pidfile. This must be overridden by a core implementation.
Authenticate a client connection with Bcfg2.Server.Plugin.interfaces.Metadata.AuthenticateConnection().
Parameters: | |
---|---|
Returns: | bool - True if the authenticate succeeds, False otherwise |
The CA that signed the server cert
Run the server core. This calls _daemonize() before calling Bcfg2.Server.Core.Core.run() to run the server core.
Bases: exceptions.Exception
Raised when an XML-RPC method is called, but there is no method exposed with the given name.
Decorator that closes the Django database connection at the end of the function. This should decorate any exposed function that might open a database connection.
Decorator that sets the exposed attribute of a function to True expose it via XML-RPC. This currently works for both the builtin and CherryPy cores, although if other cores are added this may need to be made a core-specific function.
Parameters: | func (callable) – The function to decorate |
---|---|
Returns: | callable - the decorated function |
Recursively sort an XML document in a deterministic fashion. This shouldn’t be used to perform a useful sort, merely to put XML in a deterministic, replicable order. The document is sorted in-place.
Parameters: |
|
---|---|
Returns: | None |
The builtin server core consists of the core implementation (Bcfg2.Server.BuiltinCore.Core) and the XML-RPC server implementation (Bcfg2.Server.SSLServer).
The core of the builtin Bcfg2 server.
Bases: Bcfg2.Server.Core.NetworkCore
The built-in server core
Open context to drop privileges, write the PID file, and daemonize the server core.
The daemon.DaemonContext used to drop privileges, write the PID file (with PidFile), and daemonize this core.
The Bcfg2.Server.SSLServer.XMLRPCServer instance powering this server core
Bcfg2 SSL server used by the builtin server core (Bcfg2.Server.BuiltinCore). This needs to be documented better.
Bases: SocketServer.TCPServer, object
TCP server supporting SSL encryption.
Parameters: |
|
---|
Bases: exceptions.Exception
Raised when ACL checks fail on an RPC request
Bases: SimpleXMLRPCServer.SimpleXMLRPCDispatcher
An XML-RPC dispatcher.
Bases: SimpleXMLRPCServer.SimpleXMLRPCRequestHandler
XML-RPC request handler.
Adds support for HTTP authentication.
Bases: SocketServer.ThreadingMixIn, Bcfg2.Server.SSLServer.SSLServer, Bcfg2.Server.SSLServer.XMLRPCDispatcher, object
Component XMLRPCServer.
Parameters: |
|
---|
The multiprocessing server core is a reimplementation of the Bcfg2.Server.BuiltinCore that uses the Python multiprocessing library to offload work to multiple child processes. As such, it requires Python 2.6+.
The parent communicates with the children over multiprocessing.Queue objects via a Bcfg2.Server.MultiprocessingCore.RPCQueue object.
A method being called via the RPCQueue must be exposed by the child by decorating it with Bcfg2.Server.Core.exposed().
Bases: Bcfg2.Server.Core.Core
A child process for Bcfg2.MultiprocessingCore.Core. This core builds configurations from a given multiprocessing.Pipe. Note that this is a full-fledged server core; the only input it gets from the parent process is the hostnames of clients to render. All other state comes from the FAM. However, this core only is used to render configs; it doesn’t handle anything else (authentication, probes, etc.) because those are all much faster. There’s no reason that it couldn’t handle those, though, if the pipe communication “protocol” were made more robust.
Parameters: |
|
---|
The name of this child
How long to wait while polling for new RPC commands. This doesn’t affect the speed with which a command is processed, but setting it too high will result in longer shutdown times, since we only check for the termination event from the main process every poll_wait seconds.
The queue used for RPC communication
The multiprocessing.Event that will be monitored to determine when this child should shut down.
Bases: object
DualEvent is a clone of threading.Event that internally implements both threading.Event and multiprocessing.Event.
Return true if and only if the internal flag is true.
Bases: Bcfg2.Server.BuiltinCore.BuiltinCore
A multiprocessing core that delegates building the actual client configurations to Bcfg2.Server.MultiprocessingCore.ChildCore objects. The parent process doesn’t build any children itself; all calls to GetConfig() are delegated to children. All other calls are handled by the parent process.
A queue that keeps track of which children are available to render a configuration. A child is popped from the queue when it starts to render a config, then it’s pushed back on when it’s done. This lets us use a blocking call to Queue.Queue.get() when waiting for an available child.
An iterator that each child will be taken from in sequence, to provide a round-robin distribution of render requests
A dict of child name -> one end of the multiprocessing.Pipe object used to communicate with that child. (The child is given the other end of the Pipe.)
A Bcfg2.Server.MultiprocessingCore.RPCQueue object used to send or publish commands to children.
How long to wait for a child process to shut down cleanly before it is terminated.
The flag that indicates when to stop child threads and processes
Bases: Bcfg2.Logger.Debuggable
An implementation of a multiprocessing.Queue designed for several additional use patterns:
The subscribers can deal with this as a normal Queue with no special handling.
Add a subscriber to the queue. This returns the multiprocessing.Queue object that the subscriber should read from.
Publish an RPC call to the queue for consumption by all subscribers.
Make an RPC call to the named subscriber, expecting a response. This opens a multiprocessing.connection.Listener and passes the Listener address to the child as part of the RPC call, so that the child can connect to the Listener to submit its results.
The core of the CherryPy-powered server.
Bases: Bcfg2.Server.Core.NetworkCore
The CherryPy-based server core.
- _block()[source]¶
Enter the blocking infinite server loop. Bcfg2.Server.Core.NetworkCore.shutdown() is called on exit by a subscription on the top-level CherryPy engine.
Drop privileges, daemonize with cherrypy.process.plugins.Daemonizer and write a PID file with cherrypy.process.plugins.PIDFile.
Handle all XML-RPC calls. It was necessary to make enough changes to the stock CherryPy cherrypy._cptools.XMLRPCController to support plugin RMI and prepending the client address that we just rewrote it. It clearly wasn’t written with inheritance in mind.
Perform authentication by calling Bcfg2.Server.Core.NetworkCore.authenticate(). This is implemented as a CherryPy tool.
List of exposed plugin RMI
CherryPy error handler that handles xmlrpclib.Fault objects and so allows for the possibility of returning proper error codes. This obviates the need to use cherrypy.lib.xmlrpc.on_error(), the builtin CherryPy xmlrpc tool, which does not handle xmlrpclib.Fault objects and returns the same error code for every error.