Research

Financial Literacy Education
* The Incharge Financial Distress/Financial Well-Being Scale: Establishing Validity and Reliability Proceedings of the Association for Financial Counseling and Planning Education, 2006, Prawitz, Garman, Sorhaindo, O'Neill, Kim, & Drenteas

Click here to downlaod this article (PDF).

The purpose of this article was to detail the process used to establish validity and reliability for a recently developed instrument measuring financial distress/financial well-being. Using methods and tests that included (a) a Delphi study, (b) Pearson Product Moment correlations, (c) t tests for differences between financially distressed consumers and the general population on both IFDFW scores and bill-paying behaviors, (d) factor analysis, and (e) the Cronbach's alpha statistic, the developers have given evidence that the IFDFW Scale is both valid and reliable. The instrument is appropriate for use with both the general population and financially distressed consumers.

Introduction

Most social scientists would agree that in order to be useful in measuring a theoretical construct, a measurement tool must be valid and reliable. In this sense, validity refers to the matter of whether the instrument measures what it is supposed to measure (Babbie, 2004; Becker, 1999; Litwin, 1995; Rosnow & Rosenthal, 2005; Trochim, 2000). Reliability estimates the consistency of an instrument, including not only the consistency of its results across items within the measure, but also with the instrument's results overall when it is used over time, with different observers, and/ or when two different versions of the measure are constructed using the same content (Trochim).

The purpose of this article was to detail the process used to establish validity and reliability for a recently developed instrument measuring financial distress/ financial well-being. Knowing that the establishment of validity and reliability of an instrument is an ongoing process, and keeping in mind that construct validity can be established only after years of use with different samples and in different settings (Litwin, 1995), the developers of the InCharge Financial Distress/Financial Well-Being (IFDFW) Scale have made strides in establishing that the measurement tool is valid and reliable. That is, the tool has been shown to measure the construct accurately and consistently through assessment methods set up to test for validity and reliability.

The developers have defined financial distress/ financial well-being as the level of stress and well-being emanating from one's personal financial situation. The construct represents a continuum extending from negative to positive feelings about and reactions to one's financial condition (Prawitz, Garman, Sorhaindo, O'Neill, & Kim, 2006). This article describes the efforts made to establish validity and reliability of the IFDFW Scale in measuring the construct, financial distress/ financial well-being.

Establishing Validity and Reliability

Over the past decade, methodologists have differed in the ways they have defined the specific types of validity used in the development of measurement tools. While there has been general agreement about the labeling of face validity and content validity, methodologists have diverged on the labeling of criterion validity and construct validity. They have differed, too, on the definition of specific validity types encompassed by criterion validity and construct validity. For example, in most of the methodological literature, overarching categories have been termed face validity, content validity, criterion validity, and construct validity (Babbie, 2004; Litwin, 1995; Rosnow & Rosenthal, 2005). Litwin as well as Rosnow and Rosenthal have further divided criterion validity into concurrent and predictive validity, and have separated construct validity into convergent and discriminant validity. Babbie, on the other hand, has not subdivided criterion validity and construct validity, and has used the terms "criterion" and "predictive" validity interchangeably.

Trochim (2000) has taken a different approach, and has both labeled and subdivided the types of validity in ways that differ from the terminology of other methodologists. Having argued that the establishment of validity in the operationalization of any construct implies the general term, "construct validity," Trochim has eliminated this term entirely in labeling the different types of validity.Trochim has organized the types of validity under the overall term, construct validity, as follows: (a) translation validity (face and content validity), and (b) criterion validity (predictive, concurrent, convergent, and discriminant validity). (The term, "translation validity" emerged from Trochim's search for a label to describe the extent to which the operationalization reflects the definition of the construct.) Thus, according to Trochim, construct validity is the overarching term, and translation validity and criterion validity encompass the two major approaches of researchers in the establishment of validity for a measurement tool.

In addition to differences in the organization and labeling of validity terms, methodologists have differed somewhat in their definition of the terms. There has been general agreement that for face validity and content validity, the criteria against which the instrument is evaluated are internal-- the researcher checks the operationalization of the construct by comparing the items included in the measurement tool against a wellconstructed definition of the construct (Babbie, 2004; Litwin, 1995; Rosnow & Rosenthal, 2005; Trochim, 2000). For all other types of validity, external criteria help establish that the operationalization of the construct behaves as it should, based on the theoretical meaning of the construct (Babbie; Litwin; Rosnow & Rosenthal; Trochim). Methodologists agree, for example, that predictive validity is based on tests of whether the instrument can be used to predict outcomes for some other variable or criteria in the future which it logically should be able to predict (Babbie; Litwin; Rosnow & Rosenthal; Trochim). Precise definitions for other external criterion-based validity subcategories, however, have presented a point of departure among methodologists.

Litwin (1995), for example, has defined concurrent validity as a statistical comparison of the new instrument's results with results from an accepted standard measure of the same construct. If both measures were administered concurrently, a positive correlation between the two would support the concurrent validity of the new measure (Litwin). Rosnow and Rosenthal (2005) have explained concurrent validity to mean comparison with any reasonable criterion representing measurement in the present (concurrently). Researchers, they have admitted, have no real basis upon which to determine what constitutes a comparable set of reasonable criteria.

Trochim (2000) has taken a completely different approach to the meaning of concurrent validity. Rather than a comparison of measures, Trochim has defined concurrent validity to mean a comparison of groups. According to Trochim, an instrument with concurrent validity would have the ability to differentiate between groups (samples) that should be distinct from one another based on the theory underlying the construct. Becker (1999), however, describes this contrasted groups approach as a way of measuring convergent validity rather than concurrent validity. But, the interpretation of the definition and measurement indicators for convergent validity differ among methodologists as well. According to Becker (1999), in addition to the contrasted groups approach, researchers can use correlations between the instrument and other measures of the same construct (concurrent validity) to help establish convergent validity. Garson (2006) has claimed that convergent validity refers not only to the convergence of similar scales, but also to the correlation among the indicators making up the instrument being validated. Garson has stated that Cronbach's alpha, a statistic used to establish internal consistency reliability, also helps to establish convergent validity. Litwin has maintained that convergent validity, although more theoretical and labor-intensive, is similar to alternateform reliability.

Discriminant validity represents another point of departure for methodologists. Garson (2006) has described discriminant validity as the degree to which indicators making up the instrument can demonstrate that they are sufficiently different from one another. In other words, indicators should be correlated with one another, but not perfectly correlated. Garson has pointed out that factor analysis frequently has been conducted in the establishment of discriminant validity. Litwin (1995), Rosnow and Rosenthal (2004), and Trochim (2000), however, have described discriminant validity differently. They have stated that the measure itself, rather than the indicators within the instrument, must demonstrate that it measures something that is different from similar yet distinct constructs. Clearly, methodologists have differing views on the meanings attached to the specific types of validity involved in the development of a measurement tool.

There is considerably less controversy over what constitutes reliability for a measurement tool. Methodologists agree that reliability estimates the consistency of an instrument, and most agree on subdivisions of the term. Reliability includes consistency (a) of the items making up the instrument with the items themselves, (b) of results resulting from use of the instrument over time, (c) of use with different observers, and (d) when two different versions of the instrument are constructed using the same content (Litwin, 1995; Trochim, 2000).

Methodology

Definition of Terms Used in the Development of the IFDFW Scale
Regardless of how they have defined and organized the terms related to validity, methodologists have always agreed that establishment of validity and reliability for an instrument is extremely important. The critical factor, then, is not how the validity terminology has been delineated, but rather that assessment of the validity and reliability of the instrument has been done in a thorough and purposeful way. In the development of the IFDFW Scale, 12 criteria guided the establishment of validity and reliability for the instrument. See Table 1 for a list of the criteria and assessment methods used.

For the purpose of this article, it is important that the validity and reliability terminology used in the development of the InCharge Financial Distress/ Financial Well-Being (IFDFW) Scale be detailed carefully. Validity and reliability in the development of the IFDFW Scale has been defined and organized as follows.

Face Validity
Face validity is an informal judgment of the appropriateness of the items included in the instrument. The assessment represents the degree to which the measurement tool, on its face, appears to measure what it is supposed to measure (Litwin, 1995).

Content Validity
Content validity is an assessment of whether the items included in the instrument encompass all of the major aspects reflecting the conceptualization of the construct (Litwin, 1995; Rosnow & Rosenthal, 2005). An assumption is that the researcher has formulated a good working definition of the construct within which to frame the assessment of the appropriateness of the content of the instrument (Trochim, 2000). Content validity is assessed by experts with knowledge of the subject matter (Litwin; Rosnow & Rosenthal).

Criterion Validity
Criterion validity is determined through comparison of the operationalization of the construct with some external yardstick or criterion. Statistical tests for correlation help determine criterion validity, or the degree to which the instrument is correlated with other outcome variables with which it should be correlated (Litwin, 1995; Rosnow & Rosenthal, 2005).

Concurrent criterion validity.
Concurrent validity represents the correlation of outcomes from the new instrument with outcomes on an accepted standard measure of the same construct (Litwin, 1995) or with any reasonable criterion indicating presence of the construct (Rosnow & Rosenthal, 2005). Researchers consider the validity of the criterion itself, selecting for comparison the most sensitive and meaningful measurement available (Litwin; Rosnow & Rosenthal). Finding a correlation after administering both instruments concurrently (e.g., including both instruments on the same questionnaire) helps to establish concurrent validity (Becker, 1999; Rosnow & Rosenthal, 2005).

Predictive criterion validity.
Predictive validity is an assessment of the instrument's accuracy in predicting something it logically should be able to predict based on the conceptualization of the construct (Litwin, 1995; Rosnow & Rosenthal, 2005; Trochim, 2000). That is, an instrument high in predictive validity should be able to forecast some observable behavior common to groups of people who fit in a specific way on the construct being measured.

Construct Validity
Construct validity is less straightforward than face, content, and criterion validity. Litwin (1995) contends that construct validity is the most valuable way to measure the usefulness of an instrument, but at the same time represents the most difficult and elusive type of validity to comprehend or to measure. Basically, construct validity assesses the degree to which the operationalization of the construct reflects its theoretical meaning (Becker, 1999; Litwin; Rosnow & Rosenthal, 2005; Trochim, 2000). There is no best or single way to measure construct validity; it represents a gestalt, or accumulation of knowledge over time and repeated use with different groups and in multiple settings (Becker; Litwin).

Convergent construct validity - Convergent validity, a type of construct validity, generally is described as convergence across different measures, implying that different measures of the same construct produce similar results (Litwin, 1995; Rosnow & Rosenthal, 2005). For the purposes of this article, convergent validity means the degree of convergence of concepts making up the instrument as well as convergence of the instrument itself with different measures of the same construct. For the IFDFW Scale, convergent validity has been assessed using the Pearson Product Moment correlation of all indicators making up the instrument (Becker, 1999), and t tests for differences between groups that logically should produce different outcomes on the measure (Becker).

Table 1
Validity and Reliability: Criteria and assessment methods
Item# Item Description Assessment Methods
Face Validity
1 Each concept must have face validity with people in the general adult population. They would logically consider each concept as important to an individual’s financial distress/financial well-being and recognize that each had the properties ascribed to it. In essence, each item must be perceived on the face of it as adequately covering the ideas people associate with the terms financial distress and/ or financial well-being. Adults untrained in measurement would perceive that the instrument measures what it is intended to measure. Further, each concept must fit the subject of financial distress and/or financial well-being and be a meaningful descriptor of some aspect of that content. Application of four refinement criteria prior to Delphi study data collection; use of similar items in previous research; survey of personal finance college professors and financial education experts in business
2 While the subject of personal finance certainly includes consumer credit (e.g., credit cards, installment loans), no specific item should cover that specific topic, since many adults do not use credit cards.  
Content Validity
3 Each personal finance concept denoted in an item must have been used in previous conceptual frameworks and/or research. Review of literature for identification of concepts
4 Each item must have been highly ranked by the personal finance experts in the Delphi study. Conclusions on the content validity of each item can be deduced using insights from focus groups, individuals interviewed, statistical analysis, and experts in personal finance. Delphi study of experts
5 The list of personal finance concepts comprising the items should be a representative sample of concepts in the total construct of financial distress/ financial well-being, and sufficient in number to assure content validity. Delphi study of experts; Beta version of the instrument
Concurrent criterion validity
6 The IFDFW Scale scores for the lower rankings on the instrument should distinguish varying degrees of financial distress/financial well-being among a population of initially financially distressed adults (i.e., those who have contacted a consumer credit counseling agency). Use of “contacting of a consumer credit counseling agency” as a criterion to indicate financial distress Predictive criterion validity
7 The scale items must exhibit predictive validity with adults exhibiting varying levels of financial distress/financial well-being. T test for differences in billpaying behavior of financially distressed group and general population
Convergent construct validity
8 Each item must correlate well with other individual concepts associated with personal financial distress or financial well-being; therefore, the collective concepts must stand as an adequate measure of financial distress/financial well-being. Pearson Product Moment correlation matrix
9 The summative total scores on the scale should identify widely varying degrees of the financial distress/financial well-being of the individuals responding to the survey items, and scores should discriminate readily between those with more financial distress/less financial well-being and those with less financial distress/ more financial well-being. Establishment of norms for IFDFW Scale; T test for differences in IFDFW mean scores between financially distressed group and general population
Discriminant construct validity
10 Each personal finance concept item must have construct validity, both logical and factorial. It is rationally hypothesized that measures of financial distress and financial well-being are correlated. Similarly, the scale items measure different aspects of the qualities that make up the construct of financial distress, financial well-being, or a combination of both. Factor analysis
11 Each item must contribute to factor analysis results that suggest a single, rather than multiple, factors.  
Reliability (Internal consistency)
12 Each item must contribute to a robust Cronbach’s Alpha score. Cronbach’s alpha statistic

Discriminant construct validity.
Discriminant validity, a second type of construct validity, has been defined differently by different methodologists (Garson, 2006; Litwin, 1995; Rosnow & Rosenthal, 2005; Trochim, 2002). In the development of the IFDFW Scale, discriminant validity represented the degree of discrimination among indicators used to measure the construct, and was assessed using factor analysis (Garson).

Reliability
Reliability has to do with the consistency of a measure, both internally and with repeated usage (Trochim, 2000). In the development of the IFDFW Scale, reliability referred to the internal consistency of the instrument, an estimate of how reliable the indicators were in their measurement of the same construct. Cronbach's alpha was used to determine internal consistency of items making up the measure.

Establishing Validity for the IFDFW Scale

Content and Face Validity
A number of criteria were used to assess content validity for the instrument. First, each personal finance concept denoted in an item must have been used in previous conceptual frameworks and/or research. To assure that this was done, developers of the IFDFW Scale reviewed the work of personal finance researchers spanning over four decades (Aldana & Liljenquist, 1998; Bailey, Woodiel, Turner, & Young, 1998; Beutler & Mason, 1987; Blumstein & Schwartz, 1983; Danes & Rettig, 1993; Davis & Schumm, 1987; Drentea, 2000; Drentea & Lavrakas, 2000; Freeman, Carlson, & Sperry, 1993; Garman, Leech, & Grable, 1996; Godwin & Carroll, 1986; Hafstrom & Dunsing, 1973; Joo & Garman, 1998; Kim, 1999; Lawrence, Carter & Verma, 1987; Mills, Grasmick, Morgan, & Wenk, 1992; Mirowsky and Ross, 2003; Pittman & Lloyd, 1988; Porter & Garman, 1993; Prochaska-Cue, 1993; Ross & Huber, 1985; Strumpel, 1976; Voydanoff, 1984; Walson & Fitzsimmons, 1993). Those studies collectively referred to 58 concepts reflecting some behavior, experience, perception, or personal judgment about personal finance topics. It became clear that there have been many approaches to and perspectives on the concepts surrounding financial distress and financial well-being.

Using as a starting point the 58 concepts determined to be representative of different aspects of financial distress/financial well-being, the developers of the IFDFW Scale initiated a modified Delphi study to further establish content validity for the instrument. A Delphi study solicits input multiple times from a panel of experts on the topic under review, eventually establishing consensus (Custer, Scarcella, & Stewart, 1999). The modification consisted of presenting the panel with pre-selected concepts to provide guidance based on the literature, rather than relying solely on input from the experts (Custer et al.).

Prior to the start of data collection in the Delphi study, the developers of the IFDFW Scale set up four criteria for refinement of the list of concepts in order to begin establishing face validity for the instrument. To be included in the list sent to the panel of experts, a concept had to (a) clearly describe a distinct aspect of financial distress and/or financial well-being; (b) be different enough to avoid being confused with other concepts; (c) be likely to occur in a substantive proportion of the population; and (d) be likely to occur with adults whether or not they utilized credit cards and installment loans/leases. Following the criteria check, the resulting list of financial distress/financial well-being concepts consisted of 20 items. See Table 2 for a complete list of the original 20 concepts used in the Delphi study. The three-phase Delphi process reduced the list of items through expert consensus from 20 to 10. All 10 of the items making up the final list had been ranked consistently in the top 10 by the experts during all three phases of the Delphi data collection process. See (Garman & Sorhaindo, 2005) for a detailed description of the Delphi study. Table 3 contains the ranking of the final 10 concepts.

A preliminary form of the scale, referred to as the Beta version, represented an attempt to further clarify the content validity of the instrument. The idea was to verify whether the list of personal finance concepts making up the instrument worked well together to represent the construct of financial distress/financial well-being. The Beta version, published in 2004 (Garman, Sorhaindo, Kim et al.), represented a preliminary attempt to use a set of items together to measure the concept, financial distress/financial well-being. It included six items, four of which subsequently were retained on the final version of the IFDFW Scale. The four items retained for use in the final version represented two items on financial distress (stress "today" and stress "in general") and two on financial well-being (satisfaction with and feelings about one's current financial situation). Variations of these four items had been tested over time in 10 different data collection efforts; items not highly correlated with these four eventually were dropped, and others more highly correlated were added for additional testing. The purpose of the Beta version, then, was to assess the usefulness of specific items in combination with one another to help establish content validity. Table 4 contains the six items used in the Beta version.

Table 2
Twenty concepts making up the list for Delphi study of experts
Item Number Concepts (in alphabetical order)
1. Ability to handle $1,000 financial emergency
2. Ability to manage money
3. Assessment of quality of personal financial behaviors
4. Availability of savings to pay for 3 months’ living expenses
5. Availability of money to go out for entertainment
6. Availability of money to pay for minor emergency
7. Confidence about a plan to reach financial goals
8. Confidence about long-term financial future
9. Confidence about being on track for a financially successful retirement
10. Feelings about level of financial stress today
11. Feelings about one’s current financial condition
12.

How good or bad finances are likely to be a year from now

13. How well off financially
14. Knowledge of personal finances
15. Living today on a paycheck-to-paycheck basis
16. Satisfaction with present financial situation
17. Secure about one’s personal finances for retirement
18. Spend some time at work on personal financial concerns
19. Stressed about one’s personal finances in general
20. Worry about being able to meet normal monthly living expenses

 

Table 3
Rankings of 10 concepts emerging from final phase of Delphi study
Item Number Item Description Item ranka
1 Worry about being able to meet normal monthly living expenses 1.47
2 Living today on a paycheck-to-paycheck basis 2.24
3 Feeling about one's current financial situation 3.06
4 Stressed about one's personal finances in general 3.23
5 Feelings about level of financial stress today 3.27
6 Satisfaction with present financial situation 3.38
7 Ability to handle $1,000 financial emergency 4.00
8 Availability of money to pay for a minor emergency 4.18
9 Knowledge of personal finances 4.27
10 Ability to manage money 4.62
a Lower numbers indicate higher rankings

 

Table 4
Items making up the Beta version of the instrument
Item Item description
1 What do you feel is the level of your financial stress today? a
2 On the stair steps below, mark how satisfied you are with your present financial situation. a
3 How well off are you financially?
4 How do you feel about your current financial situation? a
5 How secure do you feel about your personal finances for retirement?
6 How stressed do you feel about your personal finances in general? a
a Retained in the final version

Criterion Validity

Concurrent criterion validity
One method of establishing concurrent criterion validity for an instrument is to compare outcomes on the new measure with outcomes of another reasonable measure expected to indicate similar outcomes. To establish concurrent criterion validity for the IFDFW Scale, the reasonable measure chosen was having contacted a consumer counseling credit agency for assistance with financial problems. The fact that a consumer had made such a contact indicated the likelihood that financial distress was high and financial well-being was low for that individual, thus constituting a criterion that could be used for comparison with outcomes from the IFDFW Scale. The mean score on the IFDFW Scale for a separate sample of consumers who previously had contacted a consumer credit counseling agency was M = 3.42 (a score that denotes high financial distress, low financial well-being), indicating that the concurrent assessment on the measure of financial distress/financial well-being produced similar outcomes.

Predictive criterion validity
Predictive validity assesses whether an instrument is useful in predicting behaviors it logically should be able to predict. To establish predictive validity for the IFDFW Scale, a comparison was made between two samples of consumers. The first was a sample of 590 distressed consumers who had contacted a consumer credit counseling agency for assistance; the other was a representative sample of 1,298 consumers from the general population. The groups first were compared on their scores on the IFDFW Scale to test the hypothesis that financially distressed consumers would report more financial distress and less financial well-being than would the general public. The hypothesis was supported, as there was a significant difference in the mean scores of the two groups on the IFDFW Scale, with the financially distressed group scoring significantly lower (M = 3.42, SD = 1.64) than the general population (M = 5.72, SD = 2.41), t (1,886) = -24.17, p < .0001. These results were used as the basis for comparing the two groups on bill-paying behaviors to establish predictive validity.

Total mean scores were computed for occurrences of the following bill-paying behaviors: (a) paying a credit card bill late, (b) paying only the minimum amount due on monthly credit card bill, (c) paying utility bills late (beyond the due date), (d) receipt of notices from creditors about overdue bills, and (e) receipt of letters or telephone calls from creditors or collection agencies. It was hypothesized that there would be a difference in mean scores for the two groups, with the financially distressed group displaying more negative bill-paying behaviors (i. e., paying bills late). T-test results indicated that this was the case, with lower mean scores reported for the financially distressed consumers (M = 4.66, SD = 2.11) than for the general population (M = 8.10, SD = 2.37), t(1,865) = -31.41, p < .0001. The results contributed to the establishment of the predictive validity of the instrument.

Construct Validity

Convergent construct validity
Convergent construct validity represents the degree to which the concepts making up an instrument converge, as well as the degree of convergence of the instrument itself with other measures of the same construct. To begin establishing convergent validity for the IFDFW Scale, the developers of the instrument, using a data set from the general population (N = 1,300), obtained Pearson Product Moment correlation coefficients for all of the indicators included in the instrument (Becker, 1999). As can be seen in Table 5, the Pearson Product Moment correlation matrix demonstrated that all of the indicators were correlated with one another, indicating convergence of the indicators making up the instrument.

A second method used to establish convergent validity for the instrument was to test for differentiation between groups hypothesized to report different levels of financial distress/financial well-being. Consumers who had contacted a consumer credit counseling agency were expected to report more financial distress and less financial well-being than were consumers from the general population. Results from a t test comparing IFDFW scores from a sample of 590 financially

Table 5
Pearson Product Moment correlations for the indicators making up the IFDFW Scale
Item Statistics Q-A Q-B Q-C Q-D Q-E Q-F Q-G Q-H
Q-A Pearson correlation 1              
Sig. (2-tailed)                
N 1127              
Q-B Pearson correlation .788** 1            
Sig. (2-tailed) .000              
N 1124 1291            
Q-C Pearson correlation .725** .644** 1          
Sig. (2-tailed) .000 .000            
N 1126 1291 1293          
Q-D Pearson correlation .704** .818** .825** 1        
Sig. (2-tailed) .000 .000 .000          
N 1125 1290 1292 1292        
Q-E Pearson correlation .718** .824** .826** .927** 1      
Sig. (2-tailed) .000 .000 .000 .000        
N 1126 1291 1293 1292 1293      
Q-F Pearson correlation .651** .725** .745** .684** .690** 1    
Sig. (2-tailed) .000 .000 .000 .000 .000      
N 1124 1289 1291 1290 1291 1291    
Q-G Pearson correlation .650** .738** .759** .682** .699** .751** 1  
Sig. (2-tailed) .000 .000 .000 .000 .000 .000    
N 1113 1270 1272 1271 1272 1271 1276  
Q-H Pearson correlation .689** .772** .795** .714** .717** .795** .828** 1
Sig. (2-tailed) .000 .000 .000 .000 .000 .000 .000  
N 1104 1261 1263 1262 1263 1262 1262 1267
Q-A On the stair steps below, mark how satisfied you are your present financial situation.
Q-B How do you feel about your current financial situation?
Q-C How often do you worry about being able to meet normal monthly living expenses?
Q-D What do you feel is the level of your financial stress today?
Q-E How stressed do you feel about your personal finances in general?
Q-F How confident are you that you could find the money to pay for a financial emergency that costs about $1,000?
Q-G How often does this happen to you? You want to go out to eat, go to a movie or do something else and don’t go because you can’t afford it?
Q-H How frequently do you find yourself just getting by financially and living paycheck to paycheck?

distressed consumers (M = 3.42, SD = 1.64) and a sample of 1,298 consumers representing the general population (M = 5.72, SD = 2.41), indicated that the IFDFW Scale was able to differentiate between groups that logically should produce different results on the measure, t(1,886) = -24.17, p < .0001. These results contributed to the establishment of the convergent validity of the IFDFW Scale.

Discriminant construct validity
Discriminant construct validity has been defined differently by different methodologists (Garson, 2006; Litwin, 1995; Rosnow & Rosenthal, 2005; Trochim, 2000), but basically it represents the degree to which the instrument can demonstrate differentiation among constructs similar to the one being measured. According to Garson, factor analysis sometimes is used to assess discriminant validity, as it allows the researcher to discriminate among subscales representing similar yet distinct variables within the same instrument. The factor analysis conducted for the IFDFW Scale using a sample of the general population (N = 1,097) produced one factor, indicating that the instrument represented measurement of a single, rather than several constructs. See Table 6 for factor loadings of IFDFW Scale items.

Establishing Reliability for the IFDFW Scale

Cronbach's alpha helped establish the internal consistency reliability of the set of indicators representing the IFDFW Scale. According to Nunnally and Bernstein (1994), the Cronbach's alpha statistic provides a calculation of the ability of a group of items to measure a unidimensional construct. A high score (closer to 1.0) indicates that the indicators as a group represent a unidimensional construct. A low score (distant from 1.0) indicates that the indicators are pointing to different constructs rather than working together to represent the same construct. Nunnally and Bernstein further contend that, while a Cronbach's alpha of 0.60 or higher is considered acceptable for group scores, the minimum acceptable Cronbach's alpha for use with individual scores should not fall below 0.90. An alpha of 0.95 or higher is the desirable standard (Nunnally & Bernstein); the Cronbach's alpha for the IFDFW Scale was 0.956, indicating excellent internal consistency/reliability. The eight items making up the IFDFW Scale, when used together, contribute to a consistent measurement of the construct, financial distress/financial well-being.

Summary

The developers of the IFDFW Scale have carried out purposeful assessment procedures to establish the validity and reliability of the instrument. Using multiple methods and tests, the developers have provided evidence that the IFDFW Scale is both valid and reliable.

Face and content validity were established first. To assure face validity of the instrument, defined as an informal judgment of the appropriateness of the items included in the instrument (Litwin, 1995), the researchers applied four refinement criteria to each item, used items similar to those used in previous research, and surveyed personal finance professors, educators, and experts in business. A review of the literature for identification of concepts, a Delphi study of experts, and the development of the Beta version of the instrument helped determine content validity, an assessment of whether the items included encompassed all of the major aspects reflecting the conceptualization of the construct (Litwin; Rosnow & Rosenthal, 2005).

To establish concurrent criterion validity, or the correlation of outcomes from the new instrument with any reasonable criterion indicating presence of the construct (Rosnow & Rosenthal, 2005), the developers of the IFDFW Scale chose the contacting of a consumer credit counseling agency as a criterion to indicate financial distress. As hypothesized, a sample of consumers who had made such a contact scored in the range of high financial distress/low financial well-being on the IFDFW Scale.

The developers of the IFDFW Scale defined predictive criterion validity as the ability of the instrument to make an accurate prediction of something it logically should be able to predict based on the conceptualization of the construct (Litwin, 1995; Rosnow & Rosenthal, 2005; Trochim, 2000). A t test for differences in the billpaying behavior of financially distressed consumers and that of the general population provided evidence of predictive criterion validity.

Convergent construct validity implies that different measures of the same construct produce similar results (Litwin, 1995; Rosnow & Rosenthal, 2005). Three separate assessment measures confirmed convergent construct validity for the IFDFW Scale: a) generation of a Pearson Product Moment correlation matrix, b) establishment of norms for the IFDFW Scale, and c) a t test for differences in IFDFW mean scores between financially distressed consumers and the general population.

Discriminant construct validity represents the degree to which indicators can demonstrate that they are sufficiently different from one another (Garson, 2006). A good balance is indicated when an instrument's items are correlated with one another, but not perfectly

Table 6
Factor loadings for the eight items making up the IFDFW Scale (General pop., N = 1,097)
Item # Item description Factor loading
1 What do you feel is the level of your financial stress today? .905
2 On the stair steps below, mark how satisfied you are with your present financial situation. .833
3 How do you feel about your current financial situation? .921
4 How often do you worry about being able to meet normal monthly living expenses? .926
5 How confident are you that you could find the money to pay for a financial emergency that costs about $1,000? .857
6 How often does this happen to you? You want to go out to eat, go to a movie or do something else and don’t go because you can’t afford to. .861
7 How frequently do you find yourself just getting by financially and living paycheck to paycheck? .891
8 How stressed do you feel about your personal finances in general? .909
  Eigenvalue 6.314
  Proportion of variance explained .789

correlated. Factor analysis gave clear evidence that the indicators making up the IFDFW Scale, when used together, measured one single construct rather than separate variables.

In the development of the IFDFW Scale, reliability was defined as internal consistency of the measure. That is, the indicators making up the instrument should each contribute to the measurement of the same construct. Cronbach's alpha provided evidence of the reliability of instrument.

In every single assessment method used to determine the validity of the IFDFW Scale, there was clear evidence that the instrument was measuring what it was supposed to measure. The test for reliability indicated that there was internal consistency among the various indicators of the construct. The IFDFW Scale, then, represents a valid and reliable measurement tool to assess levels of financial distress/financial well-being in both the general population and groups of financially distressed consumers. Based on the evidence provided during development of the instrument over a multi-year period, researchers, financial educators, practitioners, and employers using the IFDFW Scale can be confident that the instrument consistently will provide accurate and appropriate measurement of financial distress/financial well-being in a variety of settings and across multiple populations. According to Nunnally and Bernstein(1994), the Cronbach's alpha level produced by the IFDFW Scale indicates that practitioners also can use the instrument with confidence in assessing financial distress/financial well-being of individual clients.

correlated. Factor analysis gave clear evidence that the indicators making up the IFDFW Scale, when used together, measured one single construct rather than separate variables.

In the development of the IFDFW Scale, reliability was defined as internal consistency of the measure. That is, the indicators making up the instrument should each contribute to the measurement of the same construct. Cronbach's alpha provided evidence of the reliability of instrument.

In every single assessment method used to determine the validity of the IFDFW Scale, there was clear evidence that the instrument was measuring what it was supposed to measure. The test for reliability indicated that there was internal consistency among the various indicators of the construct. The IFDFW Scale, then, represents a valid and reliable measurement tool to assess levels of financial distress/financial well-being in both the general population and groups of financially distressed consumers. Based on the evidence provided during development of the instrument over a multi-year period, researchers, financial educators, practitioners, and employers using the IFDFW Scale can be confident that the instrument consistently will provide accurate and appropriate measurement of financial distress/financial well-being in a variety of settings and across multiple populations. According to Nunnally and Bernstein (1994), the Cronbach's alpha level produced by the IFDFW Scale indicates that practitioners also can use the instrument with confidence in assessing financial distress/financial well-being of individual clients.

Case Examples of Organizations Using the IFDFW Scale

The IFDFW Scale has a number of implications for use; following are four illustrative case examples of organizations using the IFDFW. TwoMedicine Health and Financial Fitness has integrated the IFDFW into the annual Mayo Clinic Health Risk Assessment that is completed annually by thousands of Mayo patients, thus obtaining information on financial distress levels, perceptions of financial well-being, effects on health, and demand data on specific areas of financial planning for individuals and organizations. The Foundation for Financial Literacy uses the IFDFW to assess the financial stress and well-being of thousands of employees of 15 Texas corporations who accept the foundation's "Fiscal Fitness Challenge" and participate in its online course, "Passport to Fiscal Fitness." Before and after data collection tracks changes, advances, and progress. The Pennsylvania Office of Financial Education uses the IFDFW in its workplace financial literacy program to help judge the effectiveness of its presentations to employees of the Commonwealth as well as to help private employers train their employees in ways that both complement their business objectives and boost their bottom lines. The University of Minnesota's Latino Financial Literacy Program has translated the IFDFW into Spanish and compares participants' pre- and post-class participation scores. This information is being used to help Extension better meet client's educational needs.

For additional suggestions for use of the IFDFW Scale in research and the workplace, and for complete instructions for administering the instrument and interpreting results, see (Prawitz et al., 2006). The IFDFW Scale is copyrighted, but is available for use. Approval for use may be obtained by contacting bsorhain@incharge.org or ethomasgarman@pfeef.org. The Appendix contains a copy of the IFDFW Scale.

Sincere appreciation is extended to the InCharge Education Foundation and the InCharge Institute of America for their cooperation and support of this research.

Aldana, S. G., & Liljenquist, W. (1998). Validity and reliability of a financial strain survey. Financial Counseling and Planning, 9(2), 11-18.

Babbie, E. (2004). The practice of social research (10th ed.). Belmont, CA: Wadsworth.

Bailey, W. C., Woodiel, D. K., Turner, M. J., & Young, J. (1998). The relationship of financial stress to overall stress and satisfaction. Personal Finances and Worker Productivity, 2(2), 198-207.

Beutler, I. F., & Mason, J. W. (1987). Family cash flow budgeting. Home Economics Research Journal, 16, 3-12.

Becker, L. A. (1999). Reliability and validity, Part II. Retrieved May 24, 2006 from http:// web.uccs.edu/lbecker/Psy590/relval_II.htm

Blumstein, P. W., & Schwartz, P. (1983). American couples: Money, work, sex. New York: William Morrow.

Danes, S. M., & Rettig, K. D. (1993). The role of perception in the intention to change the family financial situation. Journal of Family and Economic Issues, 14(4), 365-389.

Davis, P., & Schumm, W. R. (1987). Family financial satisfaction: The impact of reference points. Home Economics Research Journal, 14, 123-131.

Drentea, P. (2000). Age, debt, and anxiety. Journal of Health and Social Behavior, 41, 437-450.

Drentea, P., & Lavrakas, P. J. (2000). Over the limit: The association among health status, race and debt. Social Science & Medicine, 50, 517-529.

Freeman, C., Carlson, J., & Sperry, L. (1993). Adlerian marital therapy strategies with middle income couples facing financial stress. American Journal of Family Therapy, 21(4), 324-332.

Garman, E. T., Leech, I. E., & Grable, J. E. (1996). The negative impact of employee poor personal financial behaviors on employers. Financial Counseling and Planning, 7, 157-168.

Garman, E. T., & Sorhaindo, B. (2005). Delphi study of experts' rankings of personal finance concepts important in the development of the InCharge financial distress/financial well-being scale. Consumer Interests Annual, 51, 184-194.

Garman, E. T., Sorhaindo, B., Kim, J., Xiao, J. J., Bailey, W., & Prawitz, A. (2004). The evolution of the Beta Version of the InCharge Financial Distress Scale. Consumer Interests Annual, 50, 134-144.

Garson, G. D. (2006). Validity. Retrieved May 24, 2006 from http://www2.chass.ncsu.edu/garson/PA765/ validity.htm

Godwin, D. D., & Carroll, D. D. (1986). Financial management attitudes and behaviors of husbands and wives. Journal of Consumer Studies and Home Economics, 10, 77-96.

Hafstrom, J. L., & Dunsing, M. M. (1973). Level of living: Factors influencing the homemakers' satisfaction. Home Economics Research Journal, 2(2), 119-132.

Joo, S., Garman, E. T. (1998). The relationship between personal financial wellness and employee productivity: A conceptual model. Personal Finances and Worker Productivity, 2(2), 162-171.

Kim, J. (1999). Financial satisfaction, personal financework conflict, and work outcomes: Pay satisfaction, organizational commitment, and productivity. Proceedings of the Association for Financial Counseling and Planning Education, 16, 38-45.

Lawrence, F. C., Carter, C. G., & Verma, S. (1987). Financial satisfaction in relation to financial management. Proceedings of the American Council on Consumer Interests, 337.

Litwin, M. S. (1995). How to measure survey reliability and validity. Thousand Oaks, CA: Sage.

Mills, R. J., Grasmick, H. G., Morgan, C. S., & Wenk, D. (1992). The effects of gender, family satisfaction, and economic strain on psychological well-being. Family Relations, 41, 440-446.

Mirowsky, J., & Ross, C. (2003). Social causes of psychological distress (2nd ed). New York: Aldine de Gruyter.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric Theory (3rd ed). New York: McGraw Hill.

Pittman, J. F., & Lloyd, S. A. (1988). Quality of family life, social support, and stress. Journal of Marriage and the Family, 50, 53-67.

Porter, N. M., & Garman, E. T. (1993). Testing a conceptual model of financial well-being. Financial Counseling and Planning, 4, 135-164.

Prawitz, A. D., Garman, E. T., Sorhaindo, B., O'Neill, B., Kim, J., & Drentea, P. (2006). The InCharge financial distress/financial well-being scale: Development, administration, and score interpretation. Financial Counseling and Planning, 17(1), 34-50.

Prochaska-Cue, K. (1993). An exploratory study for a model of personal financial management style. Financial Counseling and Planning, 4, 111-134.

Rosnow, R. L., & Rosenthal, R. (2005). Beginning behavioral research: A conceptual primer (5th ed). Upper Saddle River, NJ: Pearson Prentice Hall.

Ross, C., & Huber, J. (1985). Hardship and depression. Journal of Health and Social Behavior, 26, 312- 27.

Strumpel, B. (Ed.) (1976). Economic Means for Human Needs. Ann Arbor, MI: Institute for Social Research.

Trochim, W. (2000). The research methods knowledge base (2nd ed). Cincinnati, OH: Atomic Dog Publishing.

Voydanoff, P. (1984). Economic distress and families. Journal of Family Issues, 5(2), 273-288.

Walson, C. O., & Fitzsimmons, V. S. (1993). Financial manager's perception of rural household economic well-being: Development and testing of a composite measure. Journal of Family and Economic Issues, 14(3), 193-215. 1 School of Family, Consumer, and Nutrition Sciences, DeKalb, IL 60115, 815.753.6344 Fax: 815.753.1321, aprawitz@niu.edu

Click here to downlaod this article (PDF).

 

Copyright 2006, 2007, 2008
Privacy  |  Site Map  |  Terms of Use