Our proposed method is based on mathematical and computational concepts (e.g., measures). We show here that, at least in some cases in sequence analysis, the proposed measure is useful in analyzing protein families with interpretable experimental results.

Abel and Trevors have delineated three qualitative aspects of linear digital sequence complexity [2,3], Random Sequence Complexity (RSC), Ordered Sequence Complexity (OSC) and Functional Sequence Complexity (FSC). RSC corresponds to stochastic ensembles with minimal physicochemical bias and little or no tendency toward functional free-energy binding. OSC is usually patterned either by the natural regularities described by physical laws or by statistically weighted means. For example, a physico-chemical self-ordering tendency creates redundant patterns such as highly-patterned polysaccharides and the polyadenosines adsorbed onto montmorillonite [4]. Repeating motifs, with or without biofunction, result in observed OSC in nucleic acid sequences. The redundancy in OSC can, in principle, be compressed by an algorithm shorter than the sequence itself. As Abel and Trevors have pointed out, neither RSC nor OSC, or any combination of the two, is sufficient to describe the functional complexity observed in living organisms, for neither includes the additional dimension of functionality, which is essential for life [5]. FSC includes the dimension of functionality [2,3]. Szostak [6] argued that neither Shannon's original measure of uncertainty [7] nor the measure of algorithmic complexity [8] are sufficient. Shannon's classical information theory does not consider the meaning, or function, of a message. Algorithmic complexity fails to account for the observation that 'different molecular structures may be functionally equivalent'. For this reason, Szostak suggested that a new measure of information–functional information–is required [6]. Chiu, Wong, and Cheung also discussed the insufficiency of Shannon uncertainty [9,10] when applied to measuring outcomes of variables. The differences between RSC, OSC and FSC in living organisms are necessary and useful in describing biosequences of living organisms.

Consider two main uses for the proposed method for measuring FSC, which incorporates functionality: 1) comparative analysis of biosequence subgroups when explicit time lag is known; and 2) typicality analysis between biosequence subgroups when there is no explicit time lag. In the first case, such as an evolutionary time scale, an increase or decrease in FSC between an earlier gene or protein and a later gene or protein can be measured, evaluating its possible degradations and/or functional effects due to various changes such as insertions, deletions, mutations and rearrangements. In the second case, a large set of aligned sequences representing a protein family can be subdivided according to phylogenetic relationships derived from typicality of species groupings and the FSC for each subgroup measured. This is important when evaluating the emergence or evolution of viral or microbial strains with novel functions such as in the comparisons of the Chlamydia family genomes [11]. An analysis may reveal the extent of the difference in FSC between one functional group and the other, as well as the modular interactions of the internal relationship structure of the sequences [12].

The ability to measure FSC would be a significant advance in the ability to identify, analyze, compare, and predict the metabolic utility of biopolymeric sequences. Mutational drift, emerging pathogenic viral and microbial species/strains, generated mutations, acquired heritable diseases and mutagenic effects could all be evaluated quantitatively. Furthermore, In vitro experiments using SELEX [13-15] to study transitions in possible early ribozyme family growth could then be evaluated in a quantitative, as well as qualitative and intuitive fashion. Evolutionary changes, both actual and theoretical, can also be evaluated using FSC.

It is known that the variability of data can be measured using Shannon uncertainty [16]. However, Shannon's original formulation when applied to biological sequences does not express variations related to biological functionality such as metabolic utility. Shannon uncertainty, however, can be extended to measure the joint variable (X, F), where X represents the variability of data, and F functionality. This explicitly incorporates empirical knowledge of metabolic function into the measure that is usually important for evaluating sequence complexity. This measure of both the observed data and a conceptual variable of function jointly can be called Functional Uncertainty (H_f) [17], and is defined by the equation:

where X_f denotes the conditional variable of the given sequence data (X) on the described biological function f which is an outcome of the variable (F). For example, a set of 2,442 aligned sequences of proteins belonging to the ubiquitin protein family (used in the experiment later) can be assumed to satisfy the same specified function f, where f might represent the known 3-D structure of the ubiquitin protein family, or some other function common to ubiquitin. The entire set of aligned sequences that satisfies that function, therefore, constitutes the outcomes of X_f. Here, functionality relates to the whole protein family which can be inputted from a database. The advantage of using H(X_f(t)) is that changes in the functionality characteristics can be incorporated and analyzed. Furthermore, the data can be a single monomer, or a biosequence, or an entire set of aligned sequences all having the same common function. The significance of the statistical variations can then be evaluated if necessary [8]. The state variable t, representing time or a sequence of ordered events, can be fixed, discrete, or continuous. Discrete changes may be represented as discrete time states.

Functional bioinformatics is emerging as an important area of research [18-20]. Even though the term 'biological function' has been freely used for specific experimentation, there is no generally consistent usage of the term. According to Karp [21], biological functionality can refer to biochemical specified reactions, cellular responses, and structural properties of proteins and nucleic acids. It can be defined or specified at the global level (i.e., the entire organism), locally at the sub-molecular level, or applicable to the whole molecule. Hence confusion exists in interpreting its meaning. Karp [21] recognized that biological function is a complex concept. Using Webster, he refers to 'specially fitted' action or 'normal and specific contribution' of a part to the economy of the whole. Function can be related to cellular components (e.g., macromolecules, proteins or small molecules) that interact and catalyze biochemical transformations. In specific applications, it can be a local function of an enzyme such as the substrate that is acted on, or the ligands that activate or inhibit the enzyme. In more systemic integrated functions, it may refer to pathways, single or multiple, in a hierarchical, nested scope [22,23]. In general, it is a challenge 'to define a single best set of biologically acceptable rules for performing this decomposition' [21].

In our approach, we leave the specific defined meaning of functionality as an input to the application, in reference to the whole sequence family. It may represent a particular domain, or the whole protein structure, or any specified function with respect to the cell. Mathematically, it is defined precisely as an outcome of a discrete-valued variable, denoted as F={f}. The set of outcomes can be thought of as specified biological states. They are presumed non-overlapping, but can be extended to be fuzzy elements. In order to get a meaningful calculation in measuring FSC, the measure should be statistically significant in practice [7,10], much larger than zero when relating to the sequences. When sequences are chosen that are unrelated to the function to be analyzed, or are simply arbitrarily ordered or randomly generated sequences, then the measure of FSC will be small and statistically not significant. For example, if many sequences that do not share the same function f, are mistakenly included within an aligned set representing some particular function, we should expect the measure of FSC of that set to be degraded, possibly even to a very small value. However, when the specified functionality is chosen meaningfully (even in part), then FSC can be interpreted.

Consider further, when a biosequence is mutated, the mutating sequence can be compared at two different time states going from t_i to t_j. For example, t_i could represent an ancestral gene and t_j a current mutant allele. Different sequences sharing the same function f (as outcomes of the variables denoted respectively as X_f, Y_f) can also be compared at the same time t. Within a sequence, any change of a monomer in the sequence represents a single step change that may or may not affect the overall function. Sequence reversals, gene splits, lateral transfers, and multiple point mutations can also be quantified between the two states t_i, t_j. The limits of the change in functional uncertainty between the two states can then be evaluated at t = t_i and t = t_j.

The change in functional uncertainty (denoted as ΔH_f) between two states can be defined as

where X_f (t_i) and X_g (t_j) can be applied to the same sequence at two different times or to two different sequences at the same time. ΔH_f can then quantify the change in functional uncertainty between two biopolymeric states with regard to biological functionality. Unrelated biomolecules with the same function or the same sequence evolving a new or additional function through genetic drift can be compared and analyzed. A measure of ΔH_f can increase, decrease, or remain unchanged.

Biological function is mostly, though not entirely determined by the organism's genetic instructions [24-26]. The function could theoretically arise stochastically through mutational changes coupled with selection pressure, or through human experimenter involvement [13-15]. A time limit can be set in some situations to evaluate what changes to X_f(t_i) might be possible within that limit. For example, an estimation of the evolutionary limits projected over the next 10 years could be computed in this approach for any particular strain of the HIV virus. The specifics of the function (as an outcome of the function variable) can remain constant, or it can be permitted to vary within a range of efficiency. The limit may be determined by what is permitted metabolically. There is often a minimum limit of catalytic efficiency required by the organism for a given function.

The ground state g (an outcome of F) of a system is the state of presumed highest uncertainty (not necessarily equally probable) permitted by the constraints of the physical system, when no specified biological function is required or present. Certain physical systems may constrain the number of options in the ground state so that not all possible sequences are equally probable [27]. An example of a highly constrained ground state resulting in a highly ordered sequence occurs when the phosphorimidazolide of adenosine is added daily to a decameric primer bound to montmorillonite clay, producing a perfectly ordered, 50-mer sequence of polyadenosine [3]. In this case, the ground state permits only one single possible sequence. Since the ground state represents the state of presumed highest uncertainty permitted by the physical constraints of the system, the set of functional options, if there are any, will therefore be a subset of the permitted options, assuming the constraints for the physical system remain constant. If the ground state permits only one sequence, then there is no possibility of change in the functional uncertainty of the system.

The null state, a possible outcome of F denoted as ø, is defined here as a special case of the ground state of highest uncertainly when the physical system imposes no constraints at all, resulting in the equi-probability of all possible sequences or options. Such sequencing has been called "dynamically inert, dynamically decoupled, or dynamically incoherent" [28,29]. For example, the ground state of a 300 amino acid protein family can be represented by a completely random 300 amino acid sequence where functional constraints have been loosened such that any of the 20 amino acids will suffice at any of the 300 sites. From Eqn. (1) the functional uncertainty of the null state is represented as

where (X_ø(t_i)) is the conditional variable for all possible equiprobable sequences. Consider the number of all possible sequences is denoted by W. Letting the length of each sequence be denoted by N and the number of possible options at each site in the sequence be denoted by m, W = m^N. For example, for a protein of length N = 257 and assuming that the number of possible options at each site is m = 20, W = 20²⁵⁷. Since, for the null state, we are requiring that there are no constraints and all possible sequences are equally probable, P(X_ø(t_i)) = 1/W and

The change in functional uncertainty from the null state is, therefore,

First, as observed in Table 1, although we might expect larger proteins to have a higher FSC, that is not always the case. For example, 342-residue SecY has a FSC of 688 Fits, but the smaller 240-residue RecA actually has a larger FSC of 832 Fits. The Fit density (Fits/amino acid) is, therefore, lower in SecY than in RecA. This indicates that RecA is likely more functionally complex than SecY. The results for the array of random sequences and for a 50-mer polyadenosine sequence formed on Montmorillonite show that ΔH_f distinguishes FSC from RSC and OSC. The results for the array of random sequences are shown in the second from the last row of Table 1, and indicate that random sequences, which are an example of RSC, tend to have an FSC of approximately 0 Fits. The results of the highly ordered 50-mer polyadenosine, which is an example of OSC, are shown in the last row of Table 1, and indicate an FSC of approximately 0 Fits. This is consistent with Abel and Trevors' prediction that neither OSC nor RSC can contain the functional sequence complexity observed in biosequences.

A plot of FSC vs. protein length for the 35 selected protein families is shown in Figure Figure2.2. The x-intercept was found to be approximately 23 amino acids. For the 35 protein families analyzed, there were no points corresponding to any protein families in the lower right area of the plot. The right hand plot in Figure Figure22 shows the average number of Fits/site for the 35 protein families analyzed. For our small sample of 35 protein families, we found no points between 0 and 1.3 Fits/site.

To demonstrate some ways in which our approach can be applied to proteins, ubiquitin was chosen. A sample plot of FITs/site and amino acid conservation for each site, is shown in Figure Figure33 for Ubiquitin. Data for the first 5 sites in the aligned set was not available from PFAM. The conservation value was obtained by subtracting the total number of different options observed at a given site, from the total possible options. In this case, 20 different amino acids. If all 20 amino acids were permitted at a site, then the conservation value was 0. A maximum value of 19 would obtain if only 1 amino acid was observed at the site. From Figure Figure3,3, it can be observed that a high conservation value usually corresponded to a high measured FSC value. The measurement is also affected by the number of amino acids observed which could be different for different sites. For example, site 37 shows 20 observed amino acids, but still a relatively high value of 2.9 Fits. Conservation of a site reflects the degree of variations which is affected by both the number of observed amino acids and their frequency of observation in the alignment [12]. For example, if a site is observed to be dominated by only a few amino acids even though all the amino acid types are observed, its measured FSC could still be high.

From the plot in Figure Figure3,3, one can observe which sites have higher measured FSC value. An arbitrary lower cutoff limit of 3.32 Fits/site was chosen to indicate high measured FSC sites, presumed to be significantly associated with the functionality of the whole molecule. A more rigorous statistical analysis can also be used as well as other types of measures based on dependency [12]. All sites exceeding that lower cutoff value were examined. A total of seven sites were found that had a high value that was between 3.32 Fits/site and the maximum of 4.32 Fits/site. These sites were located on a 3-D model of ubiquitin (1AAR.pdb) as shown in Figure Figure4.4. Out of the 7 sites, 6 where located on the binding domain containing the binding site Lys-48 [34]. The site with the maximum FIT value was site 47, immediately next to the binding site. Surprisingly, the binding site itself was poorly conserved, with an amino acid conservation value of only 1, albeit a relatively high value of 2.91 Fits. Five of the remaining 6 sites were found to be clustered in the area of the binding site, with bonds between Leu-50 and Leu-43, as well as between Gly-41, Lys-27, and Asp-52, as shown in Figure Figure5.5. Since these sites had the highest FSC values in ubiquitin, we infer that they play a critical role in either binding, or in the structure of the binding site domain for that protein. The fact that of the top seven sites, one was immediately adjacent to the binding site, and five others were located on the structure supporting the binding site lends support for our hypothesis that high FIT values can be used to locate key functional components of a protein family.

In this paper, we have presented an important advance in the measurement of the FSC of biopolymers. It was assumed that aligned sequences from the same Pfam family, could be assigned the same functionality label. Even though the same functionality may not be applicable to individual sites, site independence and significance was assumed and the measured FSC of each site was summed. However, further extension of the method should be considered [12,35]. For example, if dependency of joint occurrences is detected between the outcomes of two variables X₃ and X₄ in the aligned sequences, then the N-tuple representation of the sequences could be transformed into a new R-tuple Y_R where these outcomes of X₃ and X₄ are represented as outcome by a single variable Y₃ as shown in Figure Figure6.6. An outcome of the two variables in X₃ and X₄ correspond to a hypercell in Y_R. A more accurate estimate of FSC could then be calculated. We are currently considering this more general scenario.