Scope

  • All sorts of parameters which can't be hardcoded in the software, but should be modifiable by users (= system administrators)

  • The configuration system is not a replacement for various "data files". For example resource bundles, templates, user management, ... should generally not be stored as configuration information. We can of course not prevent this, and it is up to the individual developer to judge what is appropriate use.

  • We want to avoid that module owners each need to invent their own way of getting configuration information.

Examples of configuration needs

  • the i18n systeem needs configuration for “default language” and for how a “date”, “time” etc. should by default be rendered

  • an e-mailing module might need things like an smtp server, a from address, ...

  • a database persistence module might need database connection settings

  • an advanced widget (like google maps) might need a 3rd party API key

  • a standard template layout.xml might allow to inject a configurable logo or css file reference...

References

Configuration service versus Spring bean configuration

  • To some extent, configuration information can be injected into beans from a Spring configuration.

  • Some problems:

    • This only works for Spring beans, not for arbitrary Java classes, scripted application parts, templates, etc. that want some configuration.

    • Spring bean configurations might be too complex (= contain too much irrelevant things) for sysadmins.

    • Users can do more harm if they can change the bean-level wiring of the application

    • Upgrading will be harder if configuration is not manageable declarative-style data.

  • Sometimes letting users supply a Spring configuration can be useful too. Especially if users need to make decissions about implementation classes and wiring of them. In other words, where you really want to enable them to write some spring config. The contract is then typically that the user-written spring conf should contains beans with a specific ID and supporting a specific interface.

    • Maybe for such cases, we could allow import of spring containers stored in the config directories, supporting fallback logic etc.

  • So to summarize, there is a need for a configuration system, which might also have the ability to manage snippets of user-defined spring configs.

Desired features

  • Configuration format should support hierarchy and repeating. Something like XML or JSON.

  • Configuration should be fast to access/read: caching, efficient data structures

  • Configuration changes should be able to have effect during runtime, not requiring a restart:

    • it should be possible to be notified of changes

      • warning: components receiving these notifications should return quickly, if they need to do intensive work they need to do it on a separate thread, in order not to block the other components from receiving notifications

    • whether configuration changes will have effect without restarting Kauri will depend on the components that consume the configuration data.

  • There should be fallback between multiple locations of configuration information:

    • default config built into the module (e.g. at KAURI-INF/conf)

    • an external location

      • also multiple levels: e.g. imagine different configs for “dev” and “deploy”, with common things in a “common/shared” configuration

    • The fallback should be able to merge configuration entities (with those higher-up in the fallback hierarchy), possibly based on instructions in those files (see config format elsewhere)

    • Smart fallback/merge however can hide the actual end-value or the path leading up to it. Some trace for each setting or else some debug-output during build up is  desirable: in case of mis-configuration people should easily find the causing file.
  • It should be easily possible to launch a Kauri-hosted application with a different configuration, e.g. by specifying a CLI argument.

    • TODO: think about the higher level: “data dir” containing conf, other data, logs, ... http://www.kauriproject.org/trac/ticket/122

    • slightly less flexible but maybe acceptible: a reference from the kauri.xml file (keeps it as the single starting-point for the app, avoids forgetting additional CLI args)..
  • Configuration should be accessible:  (NOTE: readonly)

    • in code

    • in routing

    • in templates

    • in spring config (possibly via property resolver)

    • at certain public URI's: in produced json/xml snippets towards the browser (or alternative?)
  • The API for the configuration data should preferably be independent of the storage format. Thus not expose things that are too XML-, JSON-, YAML-, whatever-specific.

  • The config data should not be modifiable, certainly not if the same instance is delivered to multiple configuration consumers.

  • Retrieving a non-existing configuration should throw an exception (possibly a variant with a boolean to control 'silent' behaviour)

    • another variant might allow programmatical defaults in the call

  • Ideally a module should only be able to access configuration information intended for that module

    • could take this further: a bean should only be able to access config info intended for it (somewhat achieved if injected into bean via spring and no 'parent' navigation possible)

    • for some special kinds of modules, it might be interesting if they could read a piece of config from each module (don't have a use case at hand)

    • not sure how easy it would be to prevent this. It is not really needed for security, but more to prevent stupid uninformed users?

    • maybe similar access as to the browser-clients can be foreseen?
  • [low] configuration should also be updateable. This in order to support admin GUIs. Brings issues as to how a merged config is stored (in its entirety at the lowest level in the fallback chain)

    • This opens a broader discussion about these admin GUIs: extremely spoken enabling a remotely-manageable kauri instance will require write-access (PUT?) to all local resources on the system that make up the kauri-instance. Next to config files this might include kauri.xml itself.
  • Be able to serialize a configuration entity as XML for streaming into an XML pipeline.

  • Be able to serialize a configuration entity as JSON to provide to the browser client (e.g. include in template).

Module or runtime feature

Preferably the configuration should be a module?

  • if it is a module, it can be replaced, can have multiple instances, ...

  • if it is a module, it will require wiring, there can be multiple instances, ... which could be avoided if it is a built-in runtime feature

  • since most modules will dependent on the configuration system, not using it will likely not be an option. Though if interfaces are well-defined, the implementation could be replaced.

Configuration structure

The configuration system manages “configuration entities” that are identified by a path (a slash-separated string).

This maps nicely to a filesystem directory structure with configuration files in it (though it might be stored in other ways). So there would definitely not be one big configuration file.

It should be possible to list the configurations available below a certain path. (what is the use-case for this?)

Configuration entity API and storage format

API

Configuration.get(“...”)

Configuration.getAsInteger(“...”)

Configuration.getArray(“...”) : String[]

Configuration.getArrayAsInteger(“...”) : int[]

Configuration.getChildArray(“...”) : Configuration[]

Configuration.getChild(“...”) : Configuration

If you try to get something but the type does not match the kind of getter, an exception will be thrown.

Format

Still undecided. Currently leaning towards XML.

JSON:

  • no mixed content

  • arrays

  • more datatyping

XML:

  • elements and attributes

  • has schema support

  • better readability because of its verbosity: if you nest some levels of JSON objects and arrays, it can be hard to find out what a ] or } closes.

The configuration format should allow to specify how merging with the base configuration needs to happend. Default is no merging. For example, in JSON this could be:

{
   "config:inherit": "shallow|deep",
   "dingen" : {default: "ding1", values: [ "ding1", "ding2"] }
   
   "foo" : {
      "config:inherit": "deep",
      ...
   }
}

The shallow inherit would only take properties from the base config which are missing in the json object containing the config:inherit property. The deep inherit would also do the inherit for each of the child json objects.

A re-use syntax like supported by YAML would also be cool to have, but will not be considered at this time.

Getting a handle to the configuration

If we go for the normal module approach, it would look something like:

<kauri:import-service id=”configManager” service=”...”/>
<bean id="config" factory-bean="configManager" factory-method="getConfig">
  <constructor-arg ref="module"/>
</bean>

If it would be a core Kauri feature, it could be:

<kauri:module [...] configId=”config”/>

Passing it on to a bean:

<bean id=”...” class=”...”>
  <constructor-arg>
    <bean factory-bean="config" factory-method="get">
      <constructor-arg ref=”/path/to/config"/>
    </bean>
  </constructor-arg>
</bean>

Or with specific support (could be either Kauri feature or provided by module – see Daisy repo):

<bean id=”...” class=”...”>
  <constructor-arg><config:get path=”/path/to/config” source="configManager"/></constructor-arg>
</bean>

The Daisy Wiki use case

The Daisy Wiki has a need for site-specific configuration, next to the normal configuration.

Note that since the Daisy Wiki will be redesigned, we won't do any effort in implementing anything that would be only/mainly useful for that case, but still it is interesting to take the requirements into consideration.

The site-level configurations are not part of the normal configuration tree. They are literally stored within the “site directory”. For the user, it is better to have everything related to a site in one place (encapsulated by a directory).

We could enable site-level configuration by enabling the configuration system to manage multiple parallel “configuration spaces” or “configuration trees”. Each such configuration space would have some logical name and a corresponding directory, and possibly a fallback to another configuration space.

kauri.xml as configuration?

We could drop the kauri.xml and make it into a configuration object like any other.

In that case, the configuration system would need to be part of the runtime itself.

When starting the runtime, the user specifies the location of the configuration directory rather than the kauri.xml.

What's currently in the kauri.xml could be made available as:

{conf-root}/runtime/wiring.xml
{conf-root}/runtime/client-connectors.xml
{conf-root}/runtime/server-connectors.xml

{conf-root}/{moduleId}/{module-specific-conf-name}.xml

The authentication-specific configuration file can also move in this scheme.

Pro's for one config syntax:

  • one location to look for all configuration
  • common syntax and common rules for all configuration
  • enhancements to the configuration system will apply to everything (e.g. support for macro's, configuration inheritance, runtime-mode-specific configs, ...)

Con's:

  • config system will be part of the core, so replacing it with custom implementation will not be possible (unless we make this possible in some way, but this becomes less of a goal)