Viper syntax reference

This page should be used as a reference on available Viper backend syntax that you can use within LDAP attribute values to implement dynamic behavior.

Configuration

All dynamic features must be configured in slapd.conf, according to syntax documented in Configuration file reference.

After specific features are enabled by appropriate slapd.conf configuration, the rules explained here apply.

The basics

Any attribute value that wants to take advantage of dynamic behavior will have to have at least one '$' character in its value.

When '$' is present in the value, the value is split into "components". After split, each component will contain one type of data: regular text, name of the overlay to run, or a parameter for the overlay.

In each overlay, regular text is passed through verbatim, while overlay name and parameters are replaced with the generated value. Then, the whole value is put back together, honoring exact whitespace and other characteristics as before the processing.

Here are two examples, showing the specification and resulting value, to help you put things in perspective: 

exp $ macAddress
00:18:51:58:36:89

domain-name "$ exp $ .. domainName $"
domain-name "c1.com"

Alright. Now we can take a tour of all the dynamic options.

Overlays

Here's the list of available overlays in order of execution, soonest-first.

File expansion

file [prefix PREFIX/] $ FILENAME [SPEC]

Expand into values of on-disk file, relative to suffix data directory.

prefix - prefix to insert between suffix data directory and FILENAME. It is not assumed that prefix will be a directory, so use the trailing "/" when appropriate.

FILENAME - filename to load

SPEC - when omitted, whole file is returned. When number, specific line number is returned. When regex without parentheses, line matching the regex is returned. When regex with parentheses, $1 is returned.

Examples: 

file $ myfile
Test data from 'myfile'

file prefix link. $ myfile
Test data from 'link.myfile'

file prefix link. $ uptime
386008.70 380334.60

file prefix link. $ vmstat nr_mapped (\d+)
27953

file prefix link. $ motd 0
Linux sl1 2.6.26-2-openvz-amd64 #1 SMP Sun Jun 21 06:01:29 UTC 2009 x86_64

Attribute value expansion

exp $ [DN] ATTR [VALX]

Expand into values of other attributes.

DN - target entry's DN. When unspecified, the current entry is assumed. Dots (".") can be used alone or at the end of DN to fill in that many DN components from the current entry.

ATTR - attribute to expand to.

VALX - value to extract from multi-valued attributes. When unspecified, defaults to the first value. When a number, specifies the value ID. When a string, specifies the joiner on which to join all values (\x20 can be used for space, but also all other hex chars are supported).

Examples: 

exp $ macAddress
00:18:51:58:36:89

routers $ exp $ cn=default,ou=networks.. router
routers 10.0.1.1

nis-domain "$ exp $ .. domainName $"
nis-domain "c1.com"

filename "http://$ exp $ cn=default,ou=networks.. configServer $/cgi-bin/preseed.cfg"
filename "http://10.0.1.1/cgi-bin/preseed.cfg"

in-target sh -c "echo $ exp $ cn=default,ou=networks.. configServer $ puppet >>/etc/hosts"
in-target sh -c "echo 10.0.1.1 puppet >>/etc/hosts"

Subsearch value expansion

find [if [!]ATTRIBUTE PATTERN] $ BASEDN SCOPE DEREF SIZE TIME FILTER ATTRONLY ATTR [VALX [GVALX]]

Perform a subsearch using full LDAP SEARCH specification, retrieve specified attribute from all entries found, and construct the final value based on some combination of all attribute values.

This feature arose from the need to be able to search and extract data from the directory, without knowing the data location specifically (opposite of what would be the case when using 'exp' overlay).

For example, this feature allows you to find all NTP servers configured for a client, then construct a NTP client config file containing line "server IP.ADD.RE.SS" for each of the servers found.

People familiar with Puppet's exported resources may notice that this feature can perform the same function and elegantly provide data in a Puppet array variable, without the need for Puppet trickery and using stored configs.

if - process and include this value in output only if at least one of the ATTRIBUTE values matches PATTERN. The exclamation mark ("!") can be used to invert test logic.

BASEDN - starting point in the tree for the search. Dots (".") can be used alone or at the end of DN to fill in that many DN components from the current entry.

SCOPE - search scope, 0= base, 1= one, 2= sub, 3= children. In general, you want to include the basedn in the search, and also search through all the tree beneath it, so use scope 2 (sub)

DEREF - dereference, 0= never, 1= always, 2= search, 3= find. Currently unused, but must specify something, so use 0

SIZE - limit in number of entries returned, 0 = unlimited, "max" = maximum

TIME - time limit in seconds, 0 = unlimited, "max" = maximum

FILTER - complete, valid LDAP filter to match against, the default matches all and is (objectClass=*)

ATTRONLY - return attribute names only. Currently unused, but must specify something, so use 0

ATTR - the attribute to retrieve from each entry found

VALX - either a numerical ID of the value you want, or a string which is used to join eventual multiple values of each individual entry in a single string. When omitted, defaults to 0 (first value). When \x20, joins values in a single string with whitespace (but also any other hex char can be used). When \x0, does not join the values but takes them as multiple values for an attribute.

GVALX - either a numerical ID of the value you want, or a string which is used to join values from all entries found in a single string. When omitted, defaults to 0 (first value). When \x20, joins values in a single string with whitespace (but also any other hex char can be used). When \x0, does not join the values but takes them as multiple values.

Note that joining on GVALX operates after joining on VALX, so any combination of joins is possible. For example, you may join values from each individual entry in a single string using say, a pipe ("|"), but leave all strings as multiple values (valx=|, gvalx=\x0).
Or, you may choose to leave all values from individual entries as multiple values, then join them all together in a big string using say, a pipe (VALX=\x0, GVALX=|).
Or for example, with valx="|" and gvalx=", ", one might get a result such as "e1v1|e1v2, e2v1, e3v1|e3v2|e3v3".

Examples: 

find $ ... 2 0 0 0 (puppetClass=ntp::server*) 0 ipHostNumber \x0 \x0
10.0.0.7
10.0.0.8
10.0.0.9

In the above example, notice that we literally use "..." as BASEDN specification, which expands to the last three components of the original search base. If someone requested LDAP key cn=ntp/servers,cn=h2,ou=hosts,o=c1.com,ou=clients, the last three components would be ou=hosts,o=c1.com,ou=clients, and that would be the starting point for the search.

Also, keep in mind that the search filter is not a regex but a fully valid LDAP search filter. Furthermore, we match "ntp::server*" and not just "ntp::server", because the module might have subclasses (NTP module has a subclass ntp::server::stock which we also want to match in our search).

Perl evaluation

perl $ SPEC

Pass SPEC through Perl evaluator and return the result.

This is a dangerous feature, and is disabled by default. To enable it, you need to edit Viper.pm and enable constant PERLEVAL.

Examples: 

perl $ 3 + 4
7

perl $ "string."
string.

Overlays - specifying options

Each overlay may support overlay-specific options, and generic options applicable to all overlays.

Options, if any, need to be specified in the same component as the overlay, such as: 

OVERLAY [OPTION VALUE ...] $ PARAMETERS

file prefix link. $ myfile

exp cache 10s $ cn=default,ou=networks.. configServer $

Options parser supports quoting using double quotes, you do not have to worry about that, but do pay attention to separate all fields using exactly one space character.

Overlay-specific options are documented in individual overlay descriptions. Generic options are documented here:

  • cache - dynamically produced values might take a long time to compute. Viper supports a generic cache mechanism which allows caching of any overlay's result, with variable validity period, specifiable as time interval, number of accesses, or number of LDAP operations.

    Examples:
     
    1. Time interval: 5 (implies seconds), 5s, 10m, 2h, 2d, 1w. The letters stand for seconds, minutes, hours days and weeks respectively.
       
    2. Number of accesses: specify how many times the value should be retrieved internally from cache before being rebuilt. This is the least used method for specifying cache validity, as you do not know and cannot control how many times a value will be accessed during processing (it may happen 1 or N times during each request). To specify such validity period, use specifier 'u', i.e. 20u. Note that 20u implies the value will be rebuilt on 20th access, giving out cached value 19 times.
       
    3. Number of LDAP operations: specify how many operations should LDAP perform using the cached value. This is a better approach than using number of accesses, because LDAP operations are quantifiable and under your control. To specify such validity period, use specifier 'o', such as 1o.
       
    Expiry setting of 1o is by far best overall setting that minimizes or eliminates risk of stale data getting served, while still producing enormous performance benefits (multiple times).