A Universal Configuration System for More Secure and Portable Code

Problems

UMN Libraries had much legacy, web application code with hard-coded, environment-specific information, e.g. database usernames and passwords, file system paths, etc. This makes code difficult to move among different environments, like development, staging, and production, because, for example, each environment typically uses different databases, each with different usernames and passwords. Using hard-coding in such a situation effectively requires different versions of each source code file for each environment, which leads to multiple copies of almost identical code, that developers must manually update in each environment. Manual updates of multiple copies of essentially the same code leads to errors. Worse yet, it is almost impossible to use a revision control system to track changes to such code, which would help to prevent those errors as well as provide many other benefits. Perhaps worst of all, this web application code was all in the web server document root, where it is all too easy to mistakenly expose source code files containing sensitive information, like database credentials, to web browsers, a huge security risk.

Solution: A Universal Configuration System

To solve these problems, we designed a universal configuration system. These were our requirements:

• Portable, environment-agnostic. Must support personal laptops, OIT virtual machines, OIT hardware, development/staging/production environments, etc.
• Must not assume use by web applications, command line applications, or any specific kind of software.
• Common convention for configuration location, in order to bootstrap configuration. Allow overriding of the convention.
• Does not expose sensitive credentials or other information to non-privileged systems or users, e.g. web application configurations must be outside of the web server document root, production database credentials must not be exposed to a development environment.
• Support for different sets of configuration values for different applications, as well as common configuration values for shared among applications.
• Framework- and language-agnostic configuration formats.
• Support for a hierarchy/tree of configuration values.
• Easily switchable configurations, e.g. allow for switching the configuration of a production application to use a development database instead of the production database, for debugging purposes.
• Common libraries for all programming languages necessary, deployable via a dependency management system for that language.
• Support for caching, to minimize loading of configurations.

Implementation

Formats

We prefer the YAML format for configuration data and use it wherever possible. The benefits of YAML are many:

• Popular format, with parser libraries available for every popular programming language.
• Terse, easily-to-understand syntax.
• Support for hierarchies/trees of data.
• Support for comments.

Our first programming language library to support this system, \UmnLib\Core\Config for PHP, also supports JSON and INI formats, and we plan to support XML, though none of those has all the advantages of YAML.

Location Convention

On all recent machines, OIT restricts almost all customer files to the /swadm directory, providing very little space anywhere else. To minimize confusion about the purpose of sub-directories, we emulate a Linux root file system under /swadm. For example, we put applications in /swadm/bin, website document roots in /swadm/var/www, and configuration files in /swadm/etc. Following this convention allows configuration library code to be simpler and more portable, because we can count on configuration files to always be in the same place. This location also puts configuration files outside the webserver document root, preventing its exposure to web browsers, thus making our code more secure. A common location also encourages pulling configuration data out of application- or framework-specific configuration files, e.g. Drupal's settings.php, so that other applications can use Drupal's database credentials without having to parse PHP or without developers having to copy the credentials to a different file in a different format.

Location Convention Problems

While we can count on having a /swadm directory on all new OIT machines, the UMN Libraries main web site and most other web applications currently reside on older machines where this is not the case. Instead, we have different directory structures in each environment, i.e. /umnlibdev for development, /umnlibstage for staging, and /umnlib for production. Further complicating matters, these customer directories are cross-mounted such that all of them exist simultaneously on a single machine, with their web application code running inside the same instance of Apache. Therefore, it's impossible to have one /swadm/etc to rule them all on those machines. Also, some people like to run multiple development environments on their laptops or local machines, in which cases a single /swadm/etc also would be impossible. For these and other reasons, we needed a way to override the location convention.

Location Convention Solution: Overriding with Environment Variables

We use environment variables to override the location convention, so that any application written in any language, running inside a web server or on the command line, can override configurations and access overridden configurations. This still leaves unsolved the problem of the cross-mounted environments using a single instance of Apache on a single machine. Michael Berkowski provided a clever solution to this problem: Use Apache rewrite rules to conditionally set environment variables depending on the domain. We use the RewriteRule flag S|skip to tie multiples rewrite rules to a single negated RewriteCond. Example:

RewriteCond %{SERVER_NAME} !^wwwstage\.lib\.umn\.edu$
RewriteRule .? - [S=2]
RewriteRule .? - [E=UMNLIB_COMPOSER_AUTOLOADER:/umnlibstage/usr/share/php/composer/vendor/autoload.php]
RewriteRule .? - [E=UMNLIB_CONFIG_PATHS:/umnlibstage/etc]
RewriteCond %{SERVER_NAME} !^www\.lib\.umn\.edu$
RewriteRule .? - [S=4]
RewriteRule .? - [E=UMNLIB_COMPOSER_AUTOLOADER:/umnlib/usr/share/php/composer/vendor/autoload.php]
RewriteRule .? - [E=UMNLIB_CONFIG_PATHS:/umnlib/etc]
RewriteRule .? - [E=UMNLIB_CONFIG_SHOULD_CACHE:true]
RewriteRule .? - [E=UMNLIB_ENABLE_MEMCACHE:true]

view raw
.htaccess
This Gist brought to you by GitHub.

So if the condition is false, we skip the number of rules indicated by the S=, in the second rule. Notice also that the second rule is effectively a no-op, since it doesn't match anything. Its only function is to specify how many of the following rules to skip.

Solution Pitfalls

We encountered a couple of pitfalls in implementing support for the environment variable solution.

Use SERVER_NAME, not HTTP_HOST

In addition to the fact that HTTP_HOST may not always contain a value, relying on HTTP_HOST introduces an XSS security risk. See Why HTTP_HOST is evil for details. We also fell into the pitfall that SERVER_NAME may not contain a value unless you set UseCanonicalName On.

Rewrite Rules Can Override Each Other

Without RewriteOptions inherit, Apache ignores all rewrite directives in all .htaccess files in all parent directories:

• What is the precedence of multiple mod_rewrite rules in multiple .htaccess files?
• RewriteOptions Directive