FTimes

Location:

Home

FTimes

Man Pages

ftimes

Man Pages
ftimes
ftimes-bimvl
ftimes-cat
ftimes-cmp2dbi
ftimes-cmp2diff
ftimes-crv2dbi
ftimes-crv2raw
ftimes-dbm-bash
ftimes-dbm-dump
ftimes-dbm-find
ftimes-dbm-make
ftimes-dbm-reap
ftimes-dbm-weed
ftimes-dig2ctx
ftimes-dig2dbi
ftimes-encoder
ftimes-grabber
ftimes-map2dbi
ftimes-map2mac
ftimes-proximo
ftimes-sizimus
ftimes-sortini
ftimes-srm
ftimes-xformer
ftimes-xpa
hashcp
hashdig-bash
hashdig-bind
hashdig-dump
hashdig-filter
hashdig-find
hashdig-harvest
hashdig-harvest-sunsolve
hashdig-make
hashdig-resolve-sunsolve
hashdig-stat
hashdig-weed
hipdig
tarmap
zipmap

FTimes
ChangeLog
Cookbook
Download
FTimes in Action
GnuPG
HashDig
History
Man Pages
News
Papers
Project Summary
XMagic

Home
FTimes
WebJob

NAME

ftimes - A system baselining and evidence collection tool.

SYNOPSIS

ftimes --cfgtest file mode [-s]

ftimes --compare mask baseline snapshot [-l {0-6}]

ftimes --decode snapshot [-l {0-6}]

ftimes --dig file [-l {0-6}] [target [...]]

ftimes --digauto file [-l {0-6}] [target [...]]

ftimes --get file [-l {0-6}]

ftimes --mad file [-l {0-6}] [target [...]]

ftimes --map file [-l {0-6}] [target [...]]

ftimes --mapauto mask [-l {0-6}] [target [...]]

ftimes {-v|--version}

DESCRIPTION

FTimes is a system baselining and evidence collection tool. The primary purpose of FTimes is to gather and/or develop information about specified directories and files in a manner conducive to intrusion analysis.

FTimes is a lightweight tool in the sense that it doesn't need to be "installed" on a given system to work on that system, it is small enough to fit on a single floppy, and it provides only a command line interface.

Preserving records of all activity that occurs during a snapshot is important for intrusion analysis and evidence admissibility. For this reason, FTimes was designed to log four types of information: configuration settings, progress indicators, metrics, and errors. Output produced by FTimes is delimited text, and therefore, is easily assimilated by a wide variety of existing tools.

FTimes basically implements two general capabilities: file topography and string search. File topography is the process of mapping key attributes of directories and files on a given file system. String search is the process of digging through directories and files on a given file system while looking for a specific sequence of bytes. Respectively, these capabilities are referred to as map mode and dig mode.

FTimes supports two operating environments: workbench and client-server. In the workbench environment, the operator uses FTimes to do things such as examine evidence (e.g., a disk image or files from a compromised system), analyze snapshots for change, search for files that have specific attributes, verify file integrity, and so on. In the client-server environment, the focus shifts from what the operator can do locally to how the operator can efficiently monitor, manage, and aggregate snapshot data for many hosts. In the client-server environment, the primary goal is to move collected data from the host to a centralized system, known as an Integrity Server, in a secure and authenticated fashion. An Integrity Server is a hardened system that has been configured to handle FTimes GET, PING, and PUT HTTP/S requests.

The FTimes distribution contains a script called nph-ftimes.cgi that may be used in conjunction with a Web server to implement a public Integrity Server interface. Deeper topics such as the construction and internal mechanics of an Integrity Server are not addressed in this document.

FTimes provides several modes of operation that either implement its basic capabilities or support them in some way. These modes are described in the MODES OF OPERATION section of this document and are outlined here:

cfgtest - check config file syntax for a given file and mode
compare - compare two map snapshots to detect change
decode - decode a compressed map snapshot
dig - search for strings in files using a specified configuration
digauto - search for strings in files using a default configuration
get - download a dig or map config file from an Integrity Server
mad - map and dig using a specified configuration
map - collect directory and file attributes using a specified configuration
mapauto - collect directory and file attributes using a default configuration
version - display version information and exit

FTimes also has many controls which dictate how it will execute. Some modes support very few controls while others support quite a few. The following table summarizes what controls apply to each mode of operation. An 'R', 'O', or 'C' indicates that the given control applies to the selected mode and whether it's required, optional, or conditionally required, respectively.

                       =======   MODES   =======
                       c  c  d  d  g  m  m  m  v
                       f  o  i  i  e  a  a  a  e
                       g  m  g  g  t  d  p  p  r
                       t  p  a     m     a     s
                       e  a  u     o     u     i
                       s  r  t     d     t     o
                       t  e  o     e     o     n
   ======   CONTROL    =========================
   AnalyzeBlockSize    .  .  .  O  .  O  .  O  .
   AnalyzeByteCount    .  .  .  O  .  O  .  O  .
   AnalyzeCarrySize    .  .  .  O  .  O  .  .  .
   AnalyzeDeviceFiles  .  .  .  O  .  O  .  O  .
   AnalyzeMaxDepth     .  .  .  O  .  O  .  O  .
   AnalyzeMaxDps       .  .  .  O  .  O  .  O  .
   AnalyzeRemoteFiles  .  .  .  O  .  O  .  O  .
   AnalyzeStartOffset  .  .  .  O  .  O  .  O  .
   AnalyzeStepSize     .  .  .  O  .  O  .  .  .
   BaseName            .  .  .  R  R  R  .  R  .
   BaseNameSuffix      .  .  .  O  .  O  .  O  .
   Compress            .  .  .  .  .  .  .  O  .
   DigString           .  .  C  C  .  C  .  .  .
   DigStringNoCase     .  .  C  C  .  C  .  .  .
   DigStringNormal     .  .  C  C  .  C  .  .  .
   DigStringRegExp     .  .  C  C  .  C  .  .  .
   DigStringXMagic     .  .  C  C  .  C  .  .  .
   EnableRecursion     .  .  .  O  .  O  .  O  .
   Exclude             .  .  .  O  .  O  .  O  .
   ExcludeFilter       .  .  .  O  .  O  .  O  .
   ExcludeFilterKlel   .  .  .  O  .  O  .  O  .
   ExcludeFilterMd5    .  .  .  .  .  O  .  O  .
   ExcludeFilterSha1   .  .  .  .  .  O  .  O  .
   ExcludeFilterSha256 .  .  .  .  .  O  .  O  .
   ExcludesMustExist   .  .  .  O  .  O  .  O  .
   FieldMask           .  R  .  .  .  R  R  R  .
   FileHook            .  .  .  O  .  O  .  O  .
   FileSizeLimit       .  .  .  O  .  O  .  O  .
   GetAndExec          .  .  .  .  O  .  .  .  .
   GetFileName         .  .  .  .  R  .  .  .  .
   HashDirectories     .  .  .  .  .  O  .  O  .
   HashSymbolicLinks   .  .  .  .  .  O  .  O  .
   Import              .  .  .  O  O  O  .  O  .
   Include             .  .  .  O  .  O  .  O  .
   IncludeFilter       .  .  .  O  .  O  .  O  .
   IncludeFilterKlel   .  .  .  O  .  O  .  O  .
   IncludeFilterMd5    .  .  .  .  .  O  .  O  .
   IncludeFilterSha1   .  .  .  .  .  O  .  O  .
   IncludeFilterSha256 .  .  .  .  .  O  .  O  .
   IncludesMustExist   .  .  .  O  .  O  .  O  .
   LogDigStrings       .  .  .  O  .  O  .  .  .
   LogDir              .  .  .  O  .  O  .  O  .
   MagicFile           .  .  .  .  .  O  .  O  .
   MapRemoteFiles      .  .  .  O  .  O  .  O  .
   MatchLimit          .  .  .  O  .  O  .  .  .
   NewLine             .  .  .  O  .  O  .  O  .
   OutDir              .  .  .  R  .  R  .  R  .
   Priority            .  O  O  O  .  O  O  O  .
   RequirePrivilege    .  .  .  O  .  O  .  O  .
   RunType             .  .  .  O  .  O  .  O  .
   StrictControls      .  .  .  O  O  O  .  O  .
   SSLBundledCAsFile   .  .  .  C  C  C  .  C  .
   SSLExpectedPeerCN   .  .  .  C  C  C  .  C  .
   SSLMaxChainLength   .  .  .  O  O  O  .  O  .
   SSLPassPhrase       .  .  .  O  O  O  .  O  .
   SSLPrivateKeyFile   .  .  .  C  C  C  .  C  .
   SSLPublicCertFile   .  .  .  C  C  C  .  C  .
   SSLUseCertificate   .  .  .  O  O  O  .  O  .
   SSLVerifyPeerCert   .  .  .  O  O  O  .  O  .
   URLAuthType         .  .  .  O  O  O  .  O  .
   URLGetRequest       .  .  .  .  R  .  .  .  .
   URLGetURL           .  .  .  .  R  .  .  .  .
   URLPassword         .  .  .  O  O  O  .  O  .
   URLPutSnapshot      .  .  .  O  .  O  .  O  .
   URLPutURL           .  .  .  O  .  O  .  O  .
   URLUnlinkOutput     .  .  .  O  .  O  .  O  .
   URLUsername         .  .  .  O  O  O  .  O  .

MODES OF OPERATION

The modes of operation described in this section are mutually exclusive. In other words, only one mode may be specified per invocation. Unless otherwise stated, the value for the baseline, snapshot, and file arguments may be the name of a regular file or '-'. If the latter form is given, FTimes expects to read the equivalent input from stdin. Note, however, that the baseline and snapshot arguments can not be '-' simultaneously. The elements and syntax rules of for all configuration files are described in the CONFIGURATION CONTROLS section of this document. The -l option is used to set the log level, and it is described in the OPTIONS section of this document. The target arguments identify directories and/or files that are to be scanned. Collectively, these targets represent an Include list. See the Include control for more information.

--cfgtest {file|-} mode [-s]

Verify the syntax of a given configuration file in the context of a specified mode where mode can be one of: digauto, dig, get, mad, or map. The given configuration file is parsed with the same methods that would be used if FTimes had been invoked in that particular run mode. By default, directories and files are not checked for existence. This allows config files to be tested in a separate environment from where they will be used. Strict testing (i.e., directories and files must exist) may be enabled with the -s option.

The value 'Syntax Passed' is written to stdout, if all syntax checks are satisfied. Otherwise, the value 'Syntax Failed' and a description of the failure will be written to stdout.

Note: The fact that a given file passes all syntax checks does not guarantee that its use will lead to a successful outcome. It merely ensures that specified controls are valid for a given mode, and the values for those controls meet basic syntax requirements.

--compare mask {baseline|-} {snapshot|-} [-l {0-6}]

Compare baseline and snapshot data according to the specified compare mask, which identifies the attributes to be analyzed. Output is written to stdout and has the following format:

    category|name|changed|unknown|records

Note: Legacy versions (3.8.0 and below) use the following format:

    category|name|changed|unknown

The category field indicates what type of change has occurred. It can have one of the following values:

    C - Changed
    M - Missing
    N - New
    U - Unknown (i.e., one or both fields were NULL)
    X - Cross (i.e., Changed and Unknown)

The changed field contains a comma-separated list of attributes that have changed. This field will be NULL if the category is New or Missing.

The unknown field contains a comma-separated list of attributes that could not be compared due to lack of information. This field will be NULL if the category is New or Missing.

The records field contains a comma-separated pair of record numbers. Each number indicates the line in the corresponding baseline or snapshot containing the attributes that were analyzed. A value of zero means that no record was present in the corresponding file, which is expected when the category is New or Missing.

The specified compare mask must comply with the syntax rules set forth for the FieldMask control.

Note: The baseline and snapshot arguments can not be '-' simultaneously.

Note: Support for comparing compressed snapshots was added in version 3.6.0.

--decode {snapshot|-} [-l {0-6}]

Decode a compressed snapshot. A compressed snapshot can be created by running FTimes in map mode with Compress enabled. Output is written to stdout.

--dig {file|-} [-l {0-6}] [target [...]]

Use the configuration settings in file to search an Include list for a set of user defined strings. The Include list may be specified by a combination of Include controls and target arguments. If an Include list is not specified, FTimes will search the entire system. Remote shares or mount points will only be searched if AnalyzeRemoteFiles is enabled. Device files specifically included (i.e., by name) will only be searched if AnalyzeDeviceFiles is enabled.

--digauto {file|-} [-l {0-6}] [target [...]]

Use default configuration settings to search an Include list for a set of user defined strings. These strings are defined in file according to the syntax for the DigStringNormal, DigStringNoCase, and DigStringRegExp controls. If no targets are specified, FTimes will search the entire system including remote shares or mount points. Device files specifically included (i.e., by name) will be searched (i.e., AnalyzeDeviceFiles is always enabled in this mode of operation). Output is written to stdout and has the following format.

    name|type|tag|offset|string

The offset field, represented as a decimal value, contains the location in the file identified by name where the specified string was found.

--get {file|-}

Use the configuration settings in file to download dig or map configuration information. One of three possible actions, depending on how get mode is configured, will take place once the download is complete:

FTimes writes the downloaded information to stdout,
FTimes writes the downloaded information to the file specified by GetFileName, or
FTimes restarts in dig, or map mode using the downloaded information as its new configuration file

The first action is effected when GetAndExec is disabled and GetFileName is not specified. The second action is effected when GetAndExec is disabled and GetFileName is specified. The third action is effected when GetAndExec is enabled and GetFileName is specified.

--mad {file|-} [-l {0-6}] [target [...]]

Use configuration settings in file to map and dig an Include list. The Include list may be specified by a combination of Include controls and target arguments. If an Include list is not specified, FTimes will process the entire system. Remote shares or mount points will only be processed if AnalyzeRemoteFiles is enabled. Device files specifically included (i.e., by name) will only be processed if AnalyzeDeviceFiles is enabled.

--map {file|-} [-l {0-6}] [target [...]]

Use configuration settings in file to map an Include list. The Include list may be specified by a combination of Include controls and target arguments. If an Include list is not specified, FTimes will map the entire system. Remote shares or mount points will only be mapped if AnalyzeRemoteFiles is enabled. Device files specifically included (i.e., by name) will only be mapped if AnalyzeDeviceFiles is enabled.

--mapauto mask [-l {0-6}] [target [...]]

Use default configuration settings to map an Include list according to the specified mask where mask identifies the attributes to be collected. If no targets are specified, FTimes will map the entire system including remote shares or mount points. Device files specifically included (i.e., by name) will be mapped (i.e., AnalyzeDeviceFiles is always enabled in this mode of operation). Output is written to stdout, and its format depends on the value of mask.

The specified mask must comply with the syntax rules set forth for the FieldMask control.

{-v|--version}

Display version information and exit.

OPTIONS

-l {0-6}

The LogLevel option controls the amount of log output. As the level decreases, the amount of output increases. The range of values that may be assigned is stated below. In cases where evidence collection is of primary concern, LogLevel should be no higher than Landmark. The default LogLevel is Landmark.

    6 - Critical
    5 - Failure
    4 - Warning
    3 - Information
    2 - Landmark
    1 - Waypoint
    0 - Debug

-s

Enforce strict testing. This requires that specified directories and files exist on the system running the test. Controls affected by this option include: LogDir, OutDir, SSLPublicCertFile, SSLPrivateKeyFile, and SSLBundledCAsFile.

CONFIGURATION CONTROLS

This section describes the various controls that FTimes recognizes. In general, controls either shape runtime behavior or provide information needed by the application to perform a specific function. Controls and their values, one pair/line, are written to a file having the following format.

    <control> = <value>

All controls are case insensitive, but, in general, their values are not. Comments may occur anywhere on a given line, and must begin with a pound character (i.e., '#'). In any given line, all text to the right of the first comment will be ignored. White space surrounding controls and values is ignored.

CONTROL DESCRIPTIONS

This section describes each control that may be specified, defines what values it may have, and states which modes of operation recognize the control.

AnalyzeBlockSize: [1-1048576]

Applies to dig, mad, and map.

AnalyzeBlockSize is optional. It instructs the analysis engine to use the specified block size (in bytes) when reading and processing file data. The default value for this control is 32768 (32 KB).

AnalyzeByteCount: [0-18446744073709551615]

Applies to dig, mad, and map.

AnalyzeByteCount is optional. It instructs the analysis engine to stop processing once the specified amount of data (in bytes) has been read/analyzed. The default value for this control is zero, which means do not impose a byte count limit.

Note: When this control is set, map mode hash values and XMagic file types may be different from those computed during a traditional snapshot (i.e., one where all bytes in each file are processed). Other than the information recorded in the log file, there is no convenient way to distinguish between traditional snapshots and those where a byte count limit. This could lead to unexpected compare mode results.

Note: Support for this control was added in version 3.9.0.

AnalyzeCarrySize: [1-1048576]

Applies to dig, and mad.

AnalyzeCarrySize is optional. It instructs the analysis engine to use the specified block size (in bytes) when saving (or carrying) data from one dig operation to the next. The default value for this control is 1024 (1 KB).

Note: The value for this control must not exceed AnalyzeBlockSize, and it must be equal to or larger than the maximum string length for normal and case insenstive strings. If either condition is not met, the program will abort.

AnalyzeDeviceFiles: [Y|N]

Applies to dig, mad, and map.

AnalyzeDeviceFiles is optional. When enabled ('Y' or 'y'), it instructs FTimes to analyze block/character device files that have been specifically included by name on the command line or through an Include (e.g., Include=/dev/ad0). Device files that reside in an included directory (e.g., Include=/dev) are not analyzed simply because their parent was included -- you must specifically call them out. Also, any device files that were specifically included will be pruned if their parent or any higher level directory was included too. The default value is 'N'.

Note: Analyzing block/character device files can take a very long time or forever (e.g., /dev/zero).

AnalyzeMaxDepth: [0-65536]

Applies to dig, mad, and map.

AnalyzeMaxDepth is optional. It instructs the analysis engine to stop descending a tree once the specified depth, relative to a given Include path, has been reached. The default value is zero, which means maximum depth enforcement is disabled.

Note: Support for this control was added in version 3.12.0.

AnalyzeMaxDps: [0-2097152]

Applies to dig, mad, and map.

AnalyzeMaxDps is optional. It instructs the analysis engine to throttle the Data Processing Speed (DPS) to the specified number of KB per second. This capability is useful for systems that are sensitive to the load imposed by running this utility. The default value is zero, which means do not throttle the DPS.

Note: This is a best-effort control. In other words, there is no guarantee that the final DPS will precisely match the specified limit.

Note: This control has no influence over the priority of the executing process.

Note: The upper limit is fixed according to following relation: 2^31/1024, which guarantees that a signed integer can represent the maximum DPS as either KB/s or B/s.

Note: Support for this control was added in version 3.9.0.

AnalyzeRemoteFiles: [Y|N]

Applies to dig, mad, and map.

AnalyzeRemoteFiles is optional. When enabled ('Y' or 'y'), it instructs FTimes to ignore the fact that a given Include does not reside on the local system. The result is that FTimes will attempt to analyze remote files. The default value is 'N'.

Note: Analyzing remote file systems can create large amounts of network traffic. Just remember that you may be mapping an entire disk.

AnalyzeStartOffset: [0-18446744073709551615]

Applies to dig, mad, and map.

AnalyzeStartOffset is optional. It instructs the analysis engine to seek to the specified offset (in bytes) before processing any data. The default value for this control is zero.

Note: To map or dig through a supported block/character device file, AnalyzeDeviceFiles must be enabled and the device file must be explicitly included on the command line or in the config file. FTimes refuses to read device files by default since that could result in hung process.

Note: Device files may require you to specify an offset that is aligned to a block boundary. Your mileage may vary.

Note: When this control is set, map mode hash values and XMagic file types will be different from those computed during a traditional snapshot (i.e., one where the start offset was zero). Other than the information recorded in the log file, there is no convenient way to distinguish between traditional snapshots and those where an offset was set. This could lead to unexpected compare mode results.

Note: Support for this control was added in version 3.9.0.

AnalyzeStepSize: [1-1048576]

Applies to dig, and mad.

AnalyzeStepSize is optional. It instructs the analysis engine to step through the search buffer in increments of the specified size. Currently, this control may only be used in conjunction with the DigStringXMagic control. Internally, AnalyzeStepSize is capped to prevent its value from exceeding the value for AnalyzeBlockSize. The default value for this control is 32768 (32 KB).

Note: This control is only available if XMagic support was compiled into the binary.

BaseName: <name|->

Applies to dig, get, mad, and map.

BaseName is required. It specifies the name prefix that will be attached to the various output files. It also serves as the CLIENTID parameter in GET/PING/PUT requests. The recommended name format is one that matches the following regular expression:

    ^[.0-9A-Za-z_-]{1,64}$

This is because nph-ftimes.cgi uses that expression to validate the CLIENTID parameter in GET/PING/PUT requests. Typically, BaseName and URLUsername will be the same when basic authentication is enabled, but this is not a requirement.

A good naming convention would be to use the hostname of the system being baselined. Also, you may specify a BaseName value of '-'. This causes FTimes to write its output to stdout/stderr.

BaseNameSuffix: [datetime|none|pid]

Applies to dig, mad, and map.

BaseNameSuffix is optional. It specifies the type of suffix that will be attached to the BaseName. If BaseNameSuffix is set to datetime, a suffix of the following format will be appended to the BaseName: YYYYMMDDHHMMSS. If it is set to none, no suffix will be appended, and if it is set to pid, the value of the current process ID will be appended. The default value is none.

Compress: [Y|N]

Applies to mad and map.

Compress is optional. When enabled ('Y' or 'y'), it activates a form of lossless ASCII compression. This yields a compression ratio that can be as good as three to one. The default value is 'N'.

As a side note, compressing compressed snapshots with a program like gzip(1) yields better compression than if gzip(1) was used alone on the same data in its uncompressed form.

DigString: <string> [<tag>]

Applies to digauto, dig, and mad.

DigString is an alias for DigStringNormal, and it is being phased out. Please use DigStringNormal instead.

DigStringNoCase: <string> [<tag>]

Applies to digauto, dig, and mad.

DigStringNoCase is conditionally required. It specifies a case insensitive search string. This string must be URL encoded in the same manner as a normal dig string -- refer to the DigStringNormal control description for the details. Internally, all alpha characters (i.e., [A-Za-z]) are converted to lower case.

An optional tag may specified. Tags can be used to identify a given dig string or set of strings -- the same tag may be assigned to multiple strings. This can make the process of analyzing dig output much easier. However, tags must not contain embedded white space, and they must be less than 64 bytes long.

DigStringNormal: <string> [<tag>]

Applies to digauto, dig, and mad.

DigStringNormal is conditionally required. It specifies a search string. This string must be URL encoded if it contains '%', '+', ' ', or any non-printable characters. When in doubt about whether or not a value should be encoded, encode it. To encode a character, convert its hex value according to the format %HH where H is a hex digit. Spaces may alternatively be encoded as '+'.

DigStringRegExp: <regexp> [<tag>]

Applies to digauto, dig, and mad.

DigStringRegExp is conditionally required. It specifies a Perl compatible regular expression. Unlike the strings specified in the DigStringNormal and DigStringNoCase controls, this string must not be URL encoded. With DigStringRegExp patterns, you must specify no more than nine (9) capturing '()' subpatterns. You can use '(?:)' if you require additional parentheses for grouping purposes. If you do not specify a capturing subpattern, the entire match will be captured and used as the output value. Otherwise, the output value is taken from \1 -- (i.e., the first capturing subpattern).

Note: This control is only available if PCRE support was compiled into the binary. As of version 3.5.0, PCRE support is enabled by default.

Note: Embedded white space is not currently allowed in the regular expression. Use '\x20' as a substitute for a single space.

DigStringXMagic: <file> [<tag>]

Applies to digauto, dig, and mad.

DigStringXMagic is conditionally required. It specifies the name of a file that contains one or more XMagic incantations. The XMagic file may be specified as a relative path.

Note: This control is only available if XMagic support was compiled into the binary.

EnableRecursion: [Y|N]

Applies to dig, mad, and map.

EnableRecursion is optional. When enabled ('Y' or 'y'), it instructs FTimes to recursively process directories. The default value is 'Y'.

Exclude: [directory|file|link]

Applies to dig, mad, and map.

Exclude controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one Exclude control/value pair per line. It is not necessary to explicitly exclude special file systems such as PROCFS as FTimes will detect their presence and automatically exclude them. Exclude values must be specified as a fully qualified path (see Include control). If ExcludesMustExist is enabled, then each Exclude must reference an existing file, directory, or symbolic link. Otherwise, FTimes will abort.

If you need to exclude paths that contain special or non-printable characters, you must specify the path in the form of a file URL. Special characters are defined (in square brackets) as [`'"|%+#] or any non-printable character except for the NULL byte (%00), which is not allowed for excludes. For example, the path '/tmp/foo#bar' would need to be specified as 'file:///tmp/foo%23bar'.

Note: Symbolic links are not supported in WIN32-based file systems.

Note: The exlude mechanism works on an exact match basis, but it can be used to produce a recursive effect. For example, if you include '/' and exclude '/etc', then '/etc' and anything below it will not be processed. However, if you include '/etc/hosts' and exclude '/etc', then '/etc/hosts' will be processed because the recursive effect would not be in play, and there is no Exclude that exactly matches it.

ExcludeFilter: <pcre-expr>

Applies to dig, mad, and map.

ExcludeFilter controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one control/value pair per line.

Each exclude filter must be specified as a Perl compatible regular expression with the caveat that no more than nine capturing '()' subpatterns for backreferences are allowed. Use '(?:)' if you require parentheses for grouping purposes.

Any object matched by a given exclude filter will be rejected. All other objects will be mapped subject to include filter processing. If no exclude filters are specified, exclude filter processing is implicitly disabled.

Note: This control is only available if PCRE support was compiled into the binary.

ExcludeFilterKlel: <klel-expr>

Applies to mad and map. Conditionally applies to dig (expressions may not utilize hash or magic attributes).

ExcludeFilterKlel controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one control/value pair per line.

Each include filter must be specified as a KLEL expression. Refer to the VALUE EXPRESSIONS section of klellang(3) for details. Information pertaining to the attributes available for use on UNIX/WINX platforms is given in the IncludeFilterKlel section of this document.

Note: This control is only available if KLEL filters were enabled and KLEL support was compiled into the binary.

Note: Support for this control was added in version 3.13.0.

ExcludeFilterMd5: <pcre-expr>

Applies to mad, and map.

ExcludeFilterMd5 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one control/value pair per line.

Note: This control is only available if PCRE support was compiled into the binary.

Note: Support for this control was added in version 3.9.0.

ExcludeFilterSha1: <pcre-expr>

Applies to mad, and map.

ExcludeFilterSha1 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one control/value pair per line.

Note: This control is only available if PCRE support was compiled into the binary.

Note: Support for this control was added in version 3.9.0.

ExcludeFilterSha256: <pcre-expr>

Applies to mad, and map.

ExcludeFilterSha256 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one control/value pair per line.

Note: This control is only available if PCRE support was compiled into the binary.

Note: Support for this control was added in version 3.9.0.

ExcludesMustExist: [Y|N]

Applies to dig, mad, and map.

ExcludesMustExist is optional. When enabled ('Y' or 'y'), it instructs FTimes to check that every Exclude exists prior to mapping or digging. If this control is enabled and any Exclude does not exist, FTimes will abort. The default value is 'N'.

FieldMask: <mask>

Applies to compare, mapauto, mad, and map.

FieldMask is required in compare. Its value indicates what fields are to be compared for change.

FieldMask is required in mapauto, mad, and map. Its value dictates what attributes get collected or derived during a scan.

There can be no embedded white space in a given mask specification, and it must comply with the following case insensitive syntax:

    ALL[<+|-><field>...]

    NONE<+|-><field>[<+|-><field>...]

The following fields may be specified on WINX platforms with two caveats: (1) chtime is only available on Windows NT or higher and (2) altstreams is only available if the target file system is NTFS.

    volume     - Volume serial number
    findex     - File serial number
    attributes - File attributes
    atime      - Time of last file access
    mtime      - Time of last file modification
    ctime      - Creation time
    chtime     - Change time (undocumented)
    size       - File size in bytes
    altstreams - Number of alternate or named streams
    md5        - MD5 digest of the file's data stream
    sha1       - SHA1 digest of the file's data stream
    sha256     - SHA256 digest of the file's data stream
    magic      - File type
    osid       - Owner SID
    gsid       - Group SID
    dacl       - Discretionary access control list

The following fields may be specified on UNIX platforms:

    dev        - Device identification number
    inode      - File identification number
    mode       - File attributes and permissions
    nlink      - Number of hard links
    uid        - User identification number
    gid        - Group identification number
    rdev       - Device type (contains major/minor numbers)
    atime      - Time of last file access
    mtime      - Time of last file modification
    ctime      - Time of last file status change
    size       - File size in bytes
    md5        - MD5 digest of the file's data stream
    sha1       - SHA1 digest of the file's data stream
    sha256     - SHA256 digest of the file's data stream
    magic      - File type

In addition to the standard fields, the following group fields may be specified:

    hashes     - Compute all supported hash values
    times      - Collect all supported time values

For example, the "hashes" field is equivalent to "md5+sha1+sha256". Similarly, the "times" field is equivalent to "atime+mtime+ctime" and "atime+mtime+ctime+chtime" on UNIX and WINX platforms, respectively.

The "attributes" field is a collection of flags stored as a bitmap. The breakdown for these flags is as follows:

    ------------------------
    bit  0     - 0x00001 Read Only
    bit  1     - 0x00002 Hidden
    bit  2     - 0x00004 System
    bit  3     - 0x00008 Volume Label (?)
    bit  4     - 0x00010 Directory
    bit  5     - 0x00020 Archive
    bit  6     - 0x00040 Device
    bit  7     - 0x00080 Normal
    ------------------------
    bit  8     - 0x00100 Temporary
    bit  9     - 0x00200 Sparse
    bit 10     - 0x00400 Reparse Point
    bit 11     - 0x00800 Compressed
    bit 12     - 0x01000 Offline
    bit 13     - 0x02000 Not Content Indexed
    bit 14     - 0x04000 Encrypted
    bit 15     - 0x08000 (?)
    ------------------------
    bit 16     - 0x10000 Virtual
    ------------------------

Additional file attribute details are documented here:

    msdn.microsoft.com/en-us/library/ee332330(VS.85).aspx

The "mode" field is a collection of flags stored as a bitmap. The breakdown for these flags is as follows:

    ------------------------
    bit  0     - X for other
    bit  1     - W for other
    bit  2     - R for other
    ------------------------
    bit  3     - X for group
    bit  4     - W for group
    bit  5     - R for group
    ------------------------
    bit  6     - X for owner
    bit  7     - W for owner
    bit  8     - R for owner
    ------------------------
    bit  9     - sticky bit
    bit 10     - set group ID on execution
    bit 11     - set owner ID on execution
    ------------------------
    bits 12-15 - file type
    ------------------------

To isolate the file type portion of the mode, AND the mode with octal 0170000. The breakdown for file types is as follows:

    0010000    - named pipe (fifo)
    0020000    - character special
    0040000    - directory
    0060000    - block special
    0100000    - regular
    0120000    - symbolic link
    0140000    - socket
    0160000    - whiteout

FileHook: <klel-expression>

Applies to dig, mad, and map.

FileHook controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one FileHook control/value pair per line. Each hook must be specified as a special type of Kl-El expression known as a guarded command.

Any path matched by a hook will cause the specified command to be executed. All digging/mapping operations will cease until this command completes.

Note: This control is only available if file hooks were enabled at compile time.

FileSizeLimit: <integer>

Applies to dig, mad, and map.

FileSizeLimit is optional. It instructs the analysis engine to skip files that are larger than the specified size limit. The default value is zero, which means do not impose a limit.

GetAndExec: [Y|N]

Applies to get.

GetAndExec is optional. When enabled ('Y' or 'y'), it causes FTimes to start a new snapshot once the download is complete. This is accomplished through an exec() call. GetAndExec depends on GetFileName. The default value is 'N'.

Note: Take care when specifying GetFileName. If you choose a location that is writeable by other processes, FTimes may not read the config file you intended it to. That is to say, some other process may have modified or replaced the origianl file.

GetFileName: <file>

Applies to get.

GetFileName is required if GetAndExec is enabled. Its value is the name of the file in which the downloaded configuration information is to be stored. GetFileName may be specified as a relative path.

HashDirectories: [Y|N]

Applies to mad, and map.

HashDirectories is optional. When enabled ('Y' or 'y'), it instructs FTimes to compute digests for directories. This is done by hashing the digests of all files and directories contained in a given directory in the order they are encountered. Thus, if a directory has the following structure where D{1|2} and F{1|2} represent directories and files respectively,

  D1
   |
   - F1
   + D2
      |
      - F2

then, assuming that F1 is mapped before D2, D1 and D2 have the following hashes:

  Hash(D2) = H(H(F2))
  Hash(D1) = H(H(F1), Hash(D2))

where H represents the hash algorithm (e.g., MD5, SHA1, etc.).

If an entry within the directory is a special file (e.g., a device) or cannot be opened/hashed, a 16 byte string consisting of all zeros is used for the computation. The default value is 'N'.

HashSymbolicLinks: [Y|N]

Applies to mad, and map.

HashSymbolicLinks is optional. When enabled ('Y' or 'y'), it instructs FTimes to compute digests for symbolic links. This is done by hashing the data returned by readlink(). The default value is 'Y'.

Import: <file>

Applies to dig, mad, map, and get.

Import is optional. When specified, the directives contained in the file referenced by this control are included in the current configuration. Multiple instances of this control are allowed per file, and recursion is permitted up to three levels deep. Imports may be specified using a relative path.

Include: [directory|file|link]

Applies to dig, mad, and map.

Include controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one Include control/value pair per line. If no Include controls are specified, FTimes will attempt to map the entire system. If IncludesMustExist is enabled, then each Include must reference an existing file, directory, or symbolic link. Otherwise, FTimes will abort.

Include values must be a regular file, directory, or symbolic link specified as a fully qualified path. For WIN32 file systems, this means that each Include must begin with a drive designator ([A-Za-z]:) and be followed by the target path (e.g., 'c:\temp'). For UNIX file systems, each Include must begin with a forward slash (e.g., '/tmp').

If you need to include paths that contain special or non-printable characters, you must specify the path in the form of a file URL. Special characters are defined (in square brackets) as [`'"|%+#] or any non-printable character except for the NULL byte (%00), which is not allowed for includes. For example, the path '/tmp/foo#bar' would need to be specified as 'file:///tmp/foo%23bar'.

Note: Symbolic links are not supported in WIN32-based file systems.

Note: Take care when including file systems that reside on remote shares because FTimes may attempt to map them. To prevent this from happening, you can either exclude the remote file system or disable AnalyzeRemoteFiles.

Note: Directory hashing only applies to directories. This means that each file Include is treated as an isolated object apart from any specific tree. The exception to this is any Include that gets automatically pruned because it's part of a larger branch.

IncludeFilter: <pcre-expr>

Applies to dig, mad, and map.

IncludeFilter controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one control/value pair per line.

Each include filter must be specified as a Perl compatible regular expression with the caveat that no more than nine capturing '()' subpatterns for backreferences are allowed. Use '(?:)' if you require parentheses for grouping purposes.

Any object matched by a given include filter will be recorded in the output stream. All other objects are implicitly excluded. Note that include filter processing occurs after exclude filter processing, and once an object has been explicitly excluded, it will not be recorded in the output stream (even if a matching include filter exists). If no include filters are specified, include filter processing is implicitly disabled.

Include filters are applied at various points in the mapping process, but they only interrupt that process at the point where sufficient information is available to make a determination. Consequently, directory traversal and/or attribute production may be required. This has several implications:

    - directory/file access times may be altered
    - more data than anticipated may need to be processed
    - run times may be significantly slower than expected

To illustrate the case where directory traversal is required, suppose that you want to map all files whose filename ends with the string: '.pdf'. The corresponding include filter would be specified as follows:

    IncludeFilter=[.]pdf$

Since there is no way to know, a priori, where all such files are located, it becomes necessary to traverse all directories applying the filter to each path encountered.

Note: This control is only available if PCRE support was compiled into the binary.

IncludeFilterKlel: <klel-expr>

Applies to mad and map. Conditionally applies to dig (expressions may not utilize hash or magic attributes).

IncludeFilterKlel controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one control/value pair per line.

Each include filter must be specified as a KLEL expression. Refer to the VALUE EXPRESSIONS section of klellang(3) for details. Attribute values are strongly typed. Possible types include boolean (KLEL_TYPE_BOOLEAN), integer (KLEL_TYPE_INT64), real (KLEL_TYPE_REAL), and string (KLEL_TYPE_STRING). The following attributes are available for use on UNIX/WINX platforms (as noted below).

f_altstreams: KLEL_TYPE_INT64 (WINX): Number of alternate or named streams (NTFS only).
f_atime: KLEL_TYPE_INT64 (UNIX/WINX): Time of last file access in seconds since the UNIX epoch (1970-01-01 00:00:00 UTC).
f_attributes: KLEL_TYPE_INT64 (WINX): File attributes expressed as a 20-bit hexadecimal mask.
f_attributes_archive: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x00020) indicating whether or not the file or directory has been marked as archive.
f_attributes_compressed: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x00800) indicating whether or not the file or directory has been compressed.
f_attributes_device: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x00040) indicating whether or not the file or directory has been marked as a device. According to MSDN, this value is reserved for system use.
f_attributes_directory: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x00010) indicating that the object is a directory.
f_attributes_encrypted: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x04000) indicating whether or not the file or directory has been encrypted.
f_attributes_hidden: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x00002) indicating whether or not the file or directory has been marked as hidden.
f_attributes_normal: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x00080) indicating whether or not the file has other attributes set. According to MSDN, this attribute is valid only when used alone.
f_attributes_not_content_indexed: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x02000) indicating that the file or directory is not to be indexed by the content indexing service.
f_attributes_offline: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x01000) indicating that the data of the file are not immediately available. According to MSDN, this attribute indicates that the file's data are physically moved to offline storage.
f_attributes_read_only: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x00001) indicating whether or not the file has been marked as read only.
f_attributes_reparse_point: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x00400) indicating whether or not the file or directory has an associated reparse point. If the object is a file, it may also indicate that the file, itself, is a symbolic link.
f_attributes_sparse_file: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x00200) indicating whether or not the file is sparse.
f_attributes_system: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x00004) indicating whether or not the file or directory has been marked as being for system use either in part or whole.
f_attributes_temporary: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x00100) indicating whether or not the file is temporary. According to MSDN, the data for such files are not written to mass storage if sufficient cache memory is available.
f_attributes_unknown_1: KLEL_TYPE_BOOLEAN (WINX): Undocumented file attribute (0x00008).
f_attributes_unknown_2: KLEL_TYPE_BOOLEAN (WINX): Undocumented file attribute (0x08000).
f_attributes_virtual: KLEL_TYPE_BOOLEAN (WINX): File attribute (0x10000) indicating whether or not the file or directory has been marked as virtual. According to MSDN, this value is reserved for system use.
f_chtime: KLEL_TYPE_INT64 (WINX): Undocumented time of last file change in seconds since the UNIX epoch (1970-01-01 00:00:00 UTC).
f_ctime: KLEL_TYPE_INT64 (UNIX/WINX): Time of last file status change (UNIX) or creation time (WINX) in seconds since the UNIX epoch (1970-01-01 00:00:00 UTC).
f_dacl: KLEL_TYPE_STRING (WINX): Discretionary access control list expressed using Security Descriptor Definition Language (SDDL).
f_dev: KLEL_TYPE_INT64 (UNIX): Device identification number.
f_exists: KLEL_TYPE_BOOLEAN (UNIX/WINX): Flag indicating whether or not a given object (file or directory) exists.
f_findex: KLEL_TYPE_INT64 (WINX): File serial number.
f_fstype: KLEL_TYPE_STRING (UNIX): File system type on which the object is stored.
f_gid: KLEL_TYPE_INT64 (UNIX): File's group ownership ID.
f_gsid: KLEL_TYPE_STRING (WINX): File's group ownership SID.
f_inode: KLEL_TYPE_INT64 (UNIX): File identification number.
f_magic: KLEL_TYPE_STRING (UNIX/WINX): File type based on XMagic analysis.
f_md5: KLEL_TYPE_STRING (UNIX/WINX): MD5 digest of the file's data stream.
f_mode: KLEL_TYPE_INT64 (UNIX): File attributes and permissions expressed as a 16-bit octal mask.
f_mode_gr: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o000040) indicating file permissions grant group read (gr).
f_mode_gw: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o000020) indicating file permissions grant group write (gw).
f_mode_gx: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o000010) indicating file permissions grant group execute (gx).
f_mode_or: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o000004) indicating file permissions grant other (i.e., world) read (or).
f_mode_ow: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o000002) indicating file permissions grant other (i.e., world) write (ow).
f_mode_ox: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o000001) indicating file permissions grant other (i.e., world) execute (ox).
f_mode_sg: KLEL_TYPE_BOOLEAN (UNIX): File mode for set group ID privilege on execution (sg).
f_mode_st: KLEL_TYPE_BOOLEAN (UNIX): File mode for sticky access (st).
f_mode_su: KLEL_TYPE_BOOLEAN (UNIX): File mode for set user ID privilege on execution (su).
f_mode_tb: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o060000) indicating file type is a block device (tb).
f_mode_tc: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o020000) indicating file type is a character device (tc).
f_mode_td: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o040000) indicating file type is a directory (td).
f_mode_tl: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o120000) indicating file type is a symbolic link (tl).
f_mode_tp: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o010000) indicating file type is a named pipe (tp).
f_mode_tr: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o100000) indicating file type is a regular file (tr).
f_mode_ts: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o140000) indicating file type is a socket (ts).
f_mode_tw: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o160000) indicating file type a whiteout (tw).
f_mode_ur: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o000400) indicating file permissions grant user (i.e., owner) read (ur).
f_mode_uw: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o000200) indicating file permissions grant user (i.e., owner) write (uw).
f_mode_ux: KLEL_TYPE_BOOLEAN (UNIX): File mode (0o000100) indicating file permissions grant user (i.e., owner) execute (ux).
f_mtime: KLEL_TYPE_INT64 (UNIX/WINX): Time of last file modification in seconds since the UNIX epoch (1970-01-01 00:00:00 UTC).
f_name: KLEL_TYPE_STRING (UNIX/WINX): Filename (full path).
f_nlink: KLEL_TYPE_INT64 (UNIX): Number of hard links.
f_osid: KLEL_TYPE_STRING (WINX): File's user ownership SID.
f_rdev: KLEL_TYPE_INT64 (UNIX): Device type (contains major/minor numbers).
f_sha1: KLEL_TYPE_STRING (UNIX/WINX): SHA1 digest of the file's data stream.
f_sha256: KLEL_TYPE_STRING (UNIX/WINX): SHA256 digest of the file's data stream.
f_size: KLEL_TYPE_INT64 (UNIX/WINX): File size in bytes.
f_uid: KLEL_TYPE_INT64 (UNIX): File's user ownership ID.
f_volume: KLEL_TYPE_INT64 (WINX): Volume serial number.

    - directory/file access times may be altered
    - more data than anticipated may need to be processed
    - run times may be significantly slower than expected

    IncludeFilterKlel=f_name =~ "[.]pdf$"

Since there is no way to know, a priori, where all such files are located, it becomes necessary to traverse all directories applying the filter to each path encountered.

To illustrate the attribute production requirement, suppose that you want to map only those files having a particular MD5 message digest (e.g., "d41d8cd98f00b204e9800998ecf8427e"). The corresponding include filter would be specified as follows:

    IncludeFilterKlel=f_md5 == "d41d8cd98f00b204e9800998ecf8427e"

Since a given object's MD5 hash can't be produced without digesting its content, all object content (i.e., all files and directories) must be processed.

Note: This control is only available if KLEL filters were enabled and KLEL support was compiled into the binary.

Note: Support for this control was added in version 3.13.0.

IncludeFilterMd5: <pcre-expr>

Applies to mad, and map.

IncludeFilterMd5 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one control/value pair per line.

Each include filter value must be specified as a Perl compatible regular expression with the exception that capturing '()' subpatterns and back references are not allowed. You can use '(?:)' if you require parentheses for grouping purposes.

Note: This control is only available if PCRE support was compiled into the binary.

Note: Support for this control was added in version 3.9.0.

IncludeFilterSha1: <pcre-expr>

Applies to mad, and map.

IncludeFilterSha1 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one control/value pair per line.

Note: This control is only available if PCRE support was compiled into the binary.

Note: Support for this control was added in version 3.9.0.

IncludeFilterSha256: <pcre-expr>

Applies to mad, and map.

IncludeFilterSha256 controls are optional, and there is no predefined limit on the number that may be specified. However, there can be only one control/value pair per line.

Note: This control is only available if PCRE support was compiled into the binary.

Note: Support for this control was added in version 3.9.0.

IncludesMustExist: [Y|N]

Applies to dig, mad, and map.

IncludesMustExist is optional. When enabled ('Y' or 'y'), it instructs FTimes to check that every Include exists prior to mapping or digging. If this control is enabled and any Include does not exist, FTimes will abort. The default value is 'N'.

LogDigStrings: [Y|N]

Applies to dig, and mad.

LogDigStrings is optional. When enabled ('Y' or 'y'), it instructs FTimes to record each dig string in the log file. Preserving this information along side the dig output is often convenient and useful, but in some cases (e.g., searches for sensitive data), it's better to suppress it. The default value is 'N'.

LogDir: <directory>

Applies to dig, mad, and map.

LogDir is optional. It tells FTimes where to write log data. If not specified, the value of OutDir will be used as the value for LogDir. LogDir may be specified as a relative path.

MagicFile: <file>

Applies to mad, and map.

MagicFile is optional. If the magic field in FieldMask is set and MagicFile references a valid XMagic file, FTimes attempts to determine the type of each file it maps. If the magic field is not set, this control is ignored. MagicFile may be specified as a relative path. Additional information regaring XMagic can be found here:

  http://ftimes.sourceforge.net/FTimes/XMagic.shtml

Note: XMagic is not built into FTimes by default. If your version of FTimes does not have this support and the magic field is requested, FTimes will produce null fields.

Note: XMagic is automatically disabled if compression is enabled.

MapRemoteFiles: [Y|N]

Applies to dig, mad, and map.

MapRemoteFiles is an alias for AnalyzeRemoteFiles, and it is being phased out. Please use AnalyzeRemoteFiles instead.

MatchLimit: <integer>

Applies to dig, and mad.

MatchLimit is optional. It instructs the search engine to stop searching for a particular pattern within a file once the specified match limit has been reached. The default value is zero, which means do not impose a limit.

Note: Searching for a string such as 'A' with a MatchLimit of zero may produce a tremendous amount of output.

NewLine: [LF|CRLF]

Applies to dig, mad, and map.

NewLine is optional. When set to the value CRLF, it generates WINX style line feeds (i.e., carriage return and linefeed). When NewLine is set to the value LF, it generates UNIX style line feeds (i.e., linefeed). This control is useful if you review/analyze data sets from different platforms on one particular computer. The default value is the native value for the operating system running the program.

OutDir: <directory>

Applies to dig, mad, and map.

OutDir is required. It tells FTimes where to write output data. If not specified, the program will abort. OutDir may be specified as a relative path.

Priority: [low|below_normal|normal|above_normal|high]

Applies to compare, digauto, dig, mad, mapauto, and map.

Priority is optional. Use it to set the priority of the running program. The environment variable, FTIMES_PRIORITY, trumps the value of this control when set. No priority is set by default.

On UNIX platforms, the priority values listed above correspond to the following priority levels:

    low .............. 20
    below_normal ..... 10
    normal ...........  0
    above_normal .....-10
    high .............-20

On WINX platforms, the priority values listed above correspond to the following priority classes:

    low .............. IDLE_PRIORITY_CLASS
    below_normal ..... BELOW_NORMAL_PRIORITY_CLASS
    normal ........... NORMAL_PRIORITY_CLASS
    above_normal ..... ABOVE_NORMAL_PRIORITY_CLASS
    high ............. HIGH_PRIORITY_CLASS

Note: If the specified priority can't be set, the program will abort.

Note: If an environment variable is defined but exceeds the maximum allowed length, it will be rejected.

Note: The priority must be set via the environment for the compare and mapauto modes of operation.

Note: Support for this control was added in version 3.9.0.

RequirePrivilege: [Y|N]

Applies to dig, mad, and map.

RequirePrivilege is optional. When enabled ('Y' or 'y'), it indicates that the operator wants to ensure that the snapshot is run from a privileged account. On UNIX systems this means that FTimes must be run from an account that has a real user id of zero (i.e., root). On NT/2K systems this means that it must be run from an account that has the backup and restore user rights. The default value is 'N'.

Note: FTimes will work without privilege, but it's likely to generate more errors due to permission problems.

RunType: [baseline|linktest|snapshot]

Applies to dig, mad, and map.

RunType is optional. This control sets a corresponding flag in the log file that classifies output data as baseline, snapshot, or linktest. The value of this control does not affect the format or content of the output. It simply classifies the data so that automated analysis applications can process it accordingly. The default value is baseline.

SSLBundledCAsFile: <file>

Applies to dig, mad, map, and get.

SSLBundledCAsFile is required when SSLVerifyPeerCert is enabled. This control specifies the name of a PEM (Privacy Enhanced Mail) encoded file that contains a bundled set of Certificate Authority (CA) certificates. Any validated peer certificate that is signed by one of these certificate authorities will be accepted provided that the SSLMaxChainLength and SSLExpectedPeerCN checks are also satisfied. SSLBundledCAsFile may be specified as a relative path.

SSLExpectedPeerCN: <name>

Applies to dig, mad, map, and get.

SSLExpectedPeerCN is required when SSLVerifyPeerCert is enabled. The value of this control represents the peer's expected Common Name (CN). Conventionally, CNs are specified as fully qualified domain names. This control eliminates the need to perform a DNS lookup at the time of certificate validation. This, in turn, may help to prevent attacks involving DNS spoofing.

SSLMaxChainLength: [1-10]

Applies to dig, mad, map, and get.

SSLMaxChainLength is optional when SSLVerifyPeerCert is enabled. The value of this control determines how deep a certificate chain may be before it is considered invalid. The default value is one.

SSLPassPhrase: <passphrase>

Applies to dig, mad, map, and get.

SSLPassPhrase is optional when SSLUseCertificate is enabled. Its value, if specified, is used to decrypt the contents of the client's private key file (see SSLPrivateKeyFile).

SSLPrivateKeyFile: <file>

Applies to dig, mad, map, and get.

SSLPrivateKeyFile is required when SSLUseCertificate is enabled. This control specifies the name of a PEM (Privacy Enhanced Mail) encoded key file that can be used to sign SSL certificates. SSLPrivateKeyFile may be specified as a relative path.

SSLPublicCertFile: <file>

Applies to dig, mad, map, and get.

SSLPublicCertFile is required when SSLUseCertificate is enabled. This control specifies the name of a PEM (Privacy Enhanced Mail) encoded certificate that will be provided during SSL handshakes. SSLPublicCertFile may be specified as a relative path.

SSLUseCertificate: [Y|N]

Applies to dig, mad, map, and get.

SSLUseCertificate is optional. When enabled ('Y' or 'y'), it instructs the application to provide client side certificate authentication, if necessary. The default value is 'N'.

SSLVerifyPeerCert: [Y|N]

Applies to dig, mad, map, and get.

SSLVerifyPeerCert is optional. When enabled ('Y' or 'y'), it instructs the application to verify the credentials of the peer server. The default value is 'N'.

StrictControls: [Y|N]

Applies to dig, mad, map, and get.

StrictControls is optional. When enabled ('Y' or 'y'), any unknown or unsupported configuration controls will cause the program to abort. When disabled ('N' or 'n'), any unknown or unsupported controls will be silently ignored. The default value is 'N'.

Note: If you want to enforce strict controls within a single config file (or chain of config files), StrictControls must be the first control processed. You may also enable/disable this control multiple times. This allows you to control the strictness of certain sections or starting at a particular point in the entire configuration. Once set, this control takes effect immediately.

Note: For versions 3.8.0 and below, this control was not supported, and the default behavior was to abort on any unknown or unsupported configuration control.

URLAuthType: [basic|none]

Applies to dig, mad, map, and get.

URLAuthType is optional. It identifies what authentication scheme to use when issuing HTTP/HTTPS requests. The value specified by this control applies to any requests involving URLGetURL or URLPutURL. When URLAuthType is set to basic, user credentials are base 64 encoded and incorporated into the request header. User credentials specified in the URL take precedence over credentials specified in the URLUsername and URLPassword controls. The default value is none.

URLGetRequest: [DigConfig|MapConfig]

Applies to get.

URLGetRequest is required. This control specifies what kind of config file the client is requesting when it issues a GET request. It also determines the next runmode when GetAndExec is enabled. Thus, values of DigConfig will cause FTimes to restart in dig mode, and values of MapConfig will cause FTimes to restart in map mode.

URLGetURL: <url>

Applies to get.

URLGetURL is required. It defines the scheme, user credentials, host address, port, and CGI application to be used when making requests. If a username/password pair is specified in the URL, that pair takes precedence over the values specified by URLUsername/URLPassword, if any. URLs must use a scheme of http or https and satisfy the following regular expression:

scheme://(user(:pass)?@)?host(:port)?/(path(\?query)?)?

URLPassword: <password>

Applies to dig, mad, map, and get.

URLPassword is optional. It identifies the password to use when accessing an Integrity Server. The value specified by this control is used in conjunction with URLGetURL and URLPutURL unless those controls supply their own username/password pair.

URLPutSnapshot: [Y|N]

Applies to dig, mad, and map.

URLPutSnapshot is optional. When enabled ('Y' or 'y'), FTimes attempts to post the snapshot to an Integrity Server. Prior to starting its scan, FTimes will transmit an application layer PING to the server to verify that it is accessible and functional. If the remote CGI application is functioning properly, the extended HTTP response code 250 (Ping Received) is returned. URLPutSnapshot depends on URLPutURL. If basic authentication is required, the controls URLUsername, URLPassword, and URLAuthType may need to be specified as well. The default value is 'N'.

URLPutURL: <url>

Applies to dig, mad, and map.

URLPutURL is optional. It defines the scheme, user credentials, host address, port, and CGI application to be used when making PUT requests. If a username/password pair is specified in the URL, that pair takes precedence over the values specified by URLUsername/URLPassword, if any. In any event, user credentials are only sent when basic authentication has been requested (See URLAuthType). URLPutURL uses the same syntax as URLGetURL.

URLUnlinkOutput: [Y|N]

Applies to dig, mad, and map.

URLUnlinkOutput is optional when URLPutSnapshot is enabled. When enabled ('Y' or 'y'), any output files are overwritten and unlinked before the program exits. The default value is 'N'.

URLUsername: <username>

Applies to dig, mad, map, and get.

URLUsername is optional. It identifies the username to use when accessing an Integrity Server. The value specified by this control is used in conjunction with URLGetURL and URLPutURL unless those controls supply their own username/password pair.

RETURN VALUES

Upon successful completion, a value of XER_OK (0) is returned. If the program encountered a critical error and had to abort immediately, the value XER_Abort (1) is returned. If the command line does not conform to the required syntax, a value of XER_Usage (2) is returned. Otherwise, one of the following error codes is returned. These codes indicate which subsystem encountered the fatal error.

XER_BootStrap (3)
XER_ProcessArguments (4)
XER_Initialize (5)
XER_CheckDependencies (6)
XER_Finalize (7)
XER_WorkHorse (8)
XER_FinishUp (9)
XER_FinalStage (10)

FILES

Several different files may be required as input or produced as output in the course of running FTimes. These files are generically described below.

BaseName(_BaseNameSuffix)?.log: The main log file. This file contains a record of the activities that took place during a dig or map run provided that LogLevel was set at an appropriate level.
BaseName(_BaseNameSuffix)?.{dig,map}: The main output file. This file contains possibly compressed map or dig data collected during a scan. The first line in this file contains a header that specifies the name of each field collected.
Bundled Certificate Authorities: When SSLUseCertificate has been enabled, FTimes expects to find a bundled certificate authorities file in the location specified by SSLBundledCAs control. If this file does not exist or have the proper format, FTimes will abort.
Config: This file contains directives used to configure FTimes. In general a particular config file applies to a single mode of operation. This is because different modes support different controls. Refer to the CONFIGURATION CONTROLS section of this document to determine what controls apply to each mode.
Public Certificate and Private Key: When SSLUseCertificate has been enabled, FTimes expects to find certificate and key files in the locations specified by SSLPublicCert and SSLPrivateKey controls, respectively. If these files do not exist or have the proper format, FTimes will abort.
Snapshot: This is a previously created and possibly compressed map file that is being supplied as input. In compare mode it represents an uncompressed map file and applies to both the baseline and snapshot arguments. In decode mode it represents a compressed map file.
XMagic: This file contains magic tests and descriptions. The magic format used by FTimes is XMagic. FTimes searches up to three locations for magic: (1) the location specified by the MagicFile control, (2) /usr/local/ftimes/etc/xmagic or c:\ftimes\etc\xmagic, and (3) the current working directory. FTimes will not abort, if a suitable file is not found, but magic output will be limited.

NOTES

The name attribute may be partially encoded if it contains any special characters. Special characters are defined as [`'"|%+#] or any non-printable character. If a name contains Unicode characters, the high byte is always encoded, and the low byte is encoded as described above. To decode an encoded name, first convert all pluses to spaces (i.e., '+' -> ' '). Then, moving from left to right, convert each %HH sequence, to its byte value. Here 'H' denotes a hex digit.

As a matter of security and good configuration control, you should create a baseline of your entire system on a periodic basis and whenever major events occur such as the addition/removal of software packages or the application of system/security patches.

In critical situations it is best to enable RequirePrivilege and run FTimes from a privileged account. This will avoid common hang-ups associated with having too little privilege. Backup and restore rights are required to satisfy RequirePrivilege on NT systems, and root privilege is required for UNIX systems. If you are simply scanning your own files, you probably do not need to enable RequirePrivilege. On NT systems the operator may need the bypass traverse checking privilege in addition to backup and restore rights. Also, make sure that domain policies aren't rendering the operator's privileges ineffective.

In general, use the default LogLevel and review the log data after each invocation as it may reveal errors or issues that need to be addressed.

In general, create distinct config files for each profile you intend to maintain. Here, profile refers to the set of files and directories within a given host that you intend to scan on a regular basis. If you have a large number of profiles that share a common set of controls, you may want to move these controls to a single file. Then, you may simply refer to this file using the Import control.

Set OutDir to a directory that has the capacity to hold all resultant data. If external media such as jaz, zip, or floppy are available, consider using them. However, using a floppy is not generally recommended unless you know that it has the capacity to hold the data. You may also want to consider writing output to an NFS, Samba, or Windows share. A third option is to use FTimes' built-in upload capability, and post your data directly to an Integrity Server.

Setting LogDir to a path that represents a floppy device may be useful if data integrity is a concern and there is no removable or protectible media available to store all of the anticipated output. The reason for this is that the log file will contain an MD5 digest of the output file upon the completion of the map. Because the log data was written to floppy, it would be possible to remove and write protect this information for safe keeping. This would provide additional time to offload the results of the map, which can be quite large. Obviously, both files can still be altered by a skilled attacker positioned to take advantage of the situation.

Manage dig and map config files on your Integrity Server, and use FTimes' get mode (i.e., --get) to automatically retrieve them at runtime. This facilitates centralized management and helps to protect potentially sensitive configuration information.

Use the URLPutSnapshot, URLPutURL, URLUsername, URLPassword, and URLAuthType controls when you want to automatically upload data to an Integrity Server that is configured to handle FTimes' PUT requests.

If you have an SSL enabled version of FTimes, you'll need to ensure that OpenSSL and/or its libraries are installed and accessible to FTimes on the target platform.

When possible, mount target file systems as read only. This will help to ensure minimal time stamp perturbation. In any event, FTimes accurately records time stamp information before it is modified by any subsequent access.

EXAMPLES

The following examples are intended to demonstrate different ways of configuring and using FTimes. Any text encapsulated between '--- XXX ---' delimiters represents a file's content.

Example 0. Basic mapping/digging

To map the entire system with default settings:

    ftimes --mapauto all /

To map the entire system with default settings, but disable log output, exclude some unwanted path, and omit MD5/SHA1 hashes:

    ftimes --map - -l 6 / << EOF
    BaseName=-
    Exclude=/some/unwanted/path
    FieldMask=all-md5-sha1
    EOF

To search the entire system for the string "marbles" with default settings:

    echo "DigStringNormal=marbles" | ftimes --digauto - /

To map the entire system and simultaneously search for the regular expression "[0-9]{3}-[0-9]{2}-[0-9]{4}":

    ftimes --mad - / << EOF
    BaseName=-
    DigStringRegExp=[0-9]{3}-[0-9]{2}-[0-9]{4}
    FieldMask=all
    EOF

Example 1. Using map mode to baseline and upload

This example demonstrates how to baseline a system and upload the resulting snapshot to an Integrity Server.

The first thing that needs to be done is to obtain the necessary upload information. Assume that an Integrity Server has already been configured to receive snapshots. Also, assume that the following information has been provided:

    URL = https://192.168.1.50:443/cgi-client/nph-ftimes.cgi
    Authentication Type = Basic
    Username/ClientID = client_1
    Password = password
    AllowedFieldMask = ALL-magic
    AllowedDataType = map
    Server Validates Client Certificates = N

Observe that remote server speaks HTTPS. Therefore, an SSL enabled version of FTimes is needed. To determine if FTimes has SSL support, run the following command:

    ftimes --version

If FTimes has SSL support, you will see the string 'ssl' in the output.

The next item to tackle is the creation of a suitable config file. The following file contains the necessary directives to complete this task.

    --- example1.cfg ---
    BaseName=client_1
    OutDir=.
    RunType=baseline
    FieldMask=ALL-magic
    URLPutSnapshot=Y
    URLPutURL=https://192.168.1.50:443/cgi-client/nph-ftimes.cgi
    URLAuthType=basic
    URLUsername=client_1
    URLPassword=password
    Compress=Y
    --- example1.cfg ---

Note that there are no Include directives. This omission causes FTimes to map everything (i.e., the entire system).

Also, note that OutDir has been set to the current directory (i.e., '.'). This, while entirely legal, assumes that (1) '.' is writeable and (2) there is enough disk space to hold all generated output.

Compression was enabled to expedite data transfer.

The last step is to run FTimes and review the corresponding log output to ensure that snapshot was uploaded successfully.

    ftimes --map example1.cfg

Example 2. Using dig mode to search for strings

This example demonstrates how to to search files and directories for a set of HEX/ASCII and regular expression strings.

Given that the target file system is /disk1, and the list of strings to to be sought are:

    String1 = /dev/ptyqa
    String2 = 0xda 0xbe
    String3 = 0x00 A 0x00 A 0x00 A 0x00 A
    String4 = A date of the form <YYYY-MM-DD HH:MM:SS>
    String5 = Line oriented IP addresses of the form <BOL|tab><192.168.1.*><tab|EOL>

where

    BOL = Beginning Of Line
    EOL = End Of Line

Strings 1-3 are expressed as (ignore whitespace) ASCII, HEX, and HEX/ASCII, respectively. Strings 4 and 5, on the other hand, are textual descriptions that must be translated into regular expressions. Assume that the required MatchLimit is three. That is to say, no more than three matches per file of any single string should be recorded.

The following config file contains those directives required to search all files in /disk1.

    --- example2.cfg ---
    BaseName=digger
    OutDir=.
    MatchLimit=3
    DigStringNormal=/dev/ptyqa
    DigStringNormal=%da%be
    DigStringNoCase=%00A%00A%00A%00A
    DigStringRegExp=\d{4}-\d{2}-\d{2}\x20\d{2}:\d{2}:\d{2}
    DigStringRegExp=(?m)(?:^|\t)(192\.168\.1\.\d{1,3})(?=\t|$)
    Include=/disk1
    --- example2.cfg ---

Observe that strings two and three are URL encoded and that string three will be processed in a case insensitive manner. Read the DigStringNormal control description for more details on URL encoding. Also notice that the fourth and fifth strings correspond to Perl compatible regular expressions -- these types of dig strings are available if you compiled FTimes with PCRE support. With DigStringRegExp patterns, however, the number of capturing '()' subpatterns is limited (see the description of that control for details). You can use '(?:)' if you require additional parentheses for grouping purposes. If you do not specify a capturing subpattern, the entire match will be captured and used as the output value. Otherwise, the output value is taken from \1 -- (i.e., the first capturing subpattern). In this particular case, only the IP address will be captured. Also note the use of '\x20' to represent a single space. This translation is necessary because dig strings may not contain embedded white space.

At this point FTimes may be run as follows:

    ftimes --dig example2.cfg

If spontaneity is desired over strict configuration, switch to the digauto mode of operation. In this mode elaborate config files aren't necessary, and output is written to stdout. Also, no match limit is imposed. The following example shows how to search an image file (e.g., example2.image) for a set of specific strings. The output, if any, can be piped into other tools that take their input on stdin.

    --- strings.cfg ---
    DigStringNormal=This+box+is+0wn3d    attack_program
    DigStringNormal=3l33t                attack_program
    DigStringNormal=175.20.1.7           attack_ip
    DigStringNormal=hacklist@foo.bar.com
    --- strings.cfg ---

    ftimes --digauto strings.cfg example2.image | some-other-tool

Just remember that dig strings must be URL encoded with no embedded white space. That's why the spaces in the first string have been replaced with '+'. Also, note that first three dig strings are tagged. In other words, each string has been assigned a value that gives it a special meaning or identity. Because the tag for first two dig strings is the same, it can be used to classify both strings as belonging to the same group.

Example 3. Change analysis

This example demonstrates how to detect change between two snapshots.

Given that the following files are valid, uncompressed snapshots:

    Baseline = daily_20001230203000.map
    Snapshot = daily_20001231203000.map

Compare MD5 digests to determine (C)hanged, (M)issing, and (N)ew files.

The only critical observation needed here is that daily_20001230203000.map is considered to be the baseline. In other words, a baseline is a snapshot taken at an arbitrary point in time to which subsequent snapshots are compared.

This comparison is carried out with the following command:

    ftimes --compare none+md5 daily_20001230203000.map daily_20001231203000.map

To compare multiple attributes at the same time, simply specify the additional fields in the FieldMask. For example, the following mask would compare MD5, SHA1, and size attributes:

    none+md5+sha1+size

To compare all attributes at the same time, use a FieldMask of 'all'.

Example 4. Using get mode to download a config file

This example demonstrates how to download a config file from an Integrity Server.

The first thing that needs to be done is to obtain the necessary download information. Assume that an Integrity Server has already been configured to serve config files. Also, assume that the following information has been provided:

    URL = https://www.integrity.net:443/cgi-client/nph-ftimes.cgi
    Authentication Type = Basic
    Username/ClientID = client_1
    Password = password
    Server Validates Client Certificates = Y
    Server Common Name = www.integrity.net
    Maximum Certificate Chain Length = 2

Observe that remote server requires certificates. This means that you'll need three additional PEM encoded files: Public Certificate, Private Key, and Bundled Certificate Authorities. Assume that these files are located on the target system as follows:

    /usr/local/ftimes/etc/PublicCert.pem
    /usr/local/ftimes/etc/PrivateKey.pem
    /usr/local/ftimes/etc/BundledCAs.pem

Armed with that information, the following config file may be constructed.

    --- example5.cfg ---
    BaseName=client_1
    URLGetURL=https://www.integrity.net:443/cgi-client/nph-ftimes.cgi
    URLGetRequest=MapConfig
    GetAndExec=N
    URLAuthType=basic
    URLUsername=client_1
    URLPassword=password
    SSLUseCertificate=Y
    SSLPublicCertFile=/usr/local/ftimes/etc/PublicCert.pem
    SSLPrivateKeyFile=/usr/local/ftimes/etc/PrivateKey.pem
    SSLPassPhrase=passphrase
    SSLVerifyPeerCert=Y
    SSLBundledCAsFile=/usr/local/ftimes/etc/BundledCAs.pem
    SSLExpectedPeerCN=www.integrity.net
    SSLMaxChainLength=2
    --- example5.cfg ---

The following command will attempt to download a map config file from the specified Integrity Server. If successful, the contents of the config file will be written to stdout.

    ftimes --get example5.cfg

If you want to download directly to a file, you can redirect the output to a file or add the GetFileName control to your config file. Then, if you wanted to download a config file and take a snapshot in one operation, simply enable GetAndExec. This causes FTimes to restart in --map mode. If URLGetRequest was set to DigConfig, then FTimes would request a dig config file and subsequently restart in --dig mode.

Another way to achieve the same effect is to use the original config file and construct the following pipeline:

    ftimes --get example5.cfg -l 6 | ftimes --map -

This has the benefit that potentially sensitive configuration information is not specifically written to a file on disk.

Finally, note that the LogLevel for the first command was set to its highest value. This was done simply to reduce log output from that process.

AUTHOR

Klayton Monroe

HISTORY

The official or given name of this utility is FTimes, but operationally it is referred to as ftimes. In the fledgling stages of development, the name FTimes was short for File Times. This was because its primary purpose was to collect timestamp information from systems suspected of having been compromised.

The first version of FTimes was written in the Spring of 1998 to support a forensic examination of a compromised system. At that time, FTimes was primarily a workbench tool for the forensic practitioner. As time passed, its audience expanded to include system administrators and computer security professionals who needed tools to monitor the integrity of their systems. Eventually, it became a core component in Exodus' Content Integrity Monitoring Service (CIMS).

In the Fall of 2001, Exodus Communications, Inc. approved my request to make FTimes Open Source. At that time version two was very stable, but I was in the throes of creating version three which was a major restructuring of the code base. Consequently, I decided to wait until the new version was complete before releasing it to the community.

On January 29, 2002, the first Open Source version of FTimes was released from SourceForge.

FTimes continues to support the workbench environment, but has evolved into a tool that can be utilized to support Integrity Monitoring in large, diverse, and distributed environments. Today, FTimes more appropriately equates to File Topography and Integrity Monitoring on an Enterprise Scale.

The original design objectives for FTimes were to build a tool that:

does one thing extremely well (i.e., collect file attributes),
utilizes a simple, effective, and well understood algorithm that can be applied equally well to different operating systems,
generates output that is easily assimilated by a wide variety of existing tools,
has built-in logging that is complete, precise, and useful for analysis purposes,
is accurate, efficient, and minimally invasive,
doesn't need to be "installed" on a given system to work on that system,
is small enough to run from floppy even if statically compiled, and
provides only a command line interface.

NAME

SYNOPSIS

DESCRIPTION

MODES OF OPERATION

OPTIONS

CONFIGURATION CONTROLS

CONTROL DESCRIPTIONS

RETURN VALUES

FILES

NOTES

EXAMPLES

Example 0. Basic mapping/digging

Example 1. Using map mode to baseline and upload

Example 2. Using dig mode to search for strings

Example 3. Change analysis

Example 4. Using get mode to download a config file

SEE ALSO

AUTHOR

HISTORY