Research
Financial Literacy Education
* The Incharge Financial Distress/Financial Well-Being Scale:
Establishing Validity and Reliability Proceedings of the Association for Financial Counseling and Planning Education, 2006, Prawitz, Garman, Sorhaindo, O'Neill, Kim, & Drenteas
Click here to downlaod this article (PDF).
The purpose of this article was to detail the process used to establish validity and reliability for a recently developed
instrument measuring financial distress/financial well-being. Using methods and tests that included (a) a Delphi study,
(b) Pearson Product Moment correlations, (c) t tests for differences between financially distressed consumers and the
general population on both IFDFW scores and bill-paying behaviors, (d) factor analysis, and (e) the Cronbach's alpha
statistic, the developers have given evidence that the IFDFW Scale is both valid and reliable. The instrument is appropriate
for use with both the general population and financially distressed consumers.
Introduction
Most social scientists would agree that in order to be
useful in measuring a theoretical construct, a
measurement tool must be valid and reliable. In this
sense, validity refers to the matter of whether the
instrument measures what it is supposed to measure
(Babbie, 2004; Becker, 1999; Litwin, 1995; Rosnow &
Rosenthal, 2005; Trochim, 2000). Reliability estimates
the consistency of an instrument, including not only the
consistency of its results across items within the
measure, but also with the instrument's results overall
when it is used over time, with different observers, and/
or when two different versions of the measure are
constructed using the same content (Trochim).
The purpose of this article was to detail the process used
to establish validity and reliability for a recently
developed instrument measuring financial distress/
financial well-being. Knowing that the establishment of
validity and reliability of an instrument is an ongoing
process, and keeping in mind that construct validity can
be established only after years of use with different
samples and in different settings (Litwin, 1995), the
developers of the InCharge Financial Distress/Financial
Well-Being (IFDFW) Scale have made strides in
establishing that the measurement tool is valid and
reliable. That is, the tool has been shown to measure the
construct accurately and consistently through assessment
methods set up to test for validity and reliability.
The developers have defined financial distress/ financial
well-being as the level of stress and well-being
emanating from one's personal financial situation. The
construct represents a continuum extending from
negative to positive feelings about and reactions to one's financial condition (Prawitz, Garman, Sorhaindo,
O'Neill, & Kim, 2006). This article describes the efforts
made to establish validity and reliability of the IFDFW
Scale in measuring the construct, financial distress/
financial well-being.
Establishing Validity and Reliability
Over the past decade, methodologists have differed in
the ways they have defined the specific types of validity
used in the development of measurement tools. While
there has been general agreement about the labeling of
face validity and content validity, methodologists have
diverged on the labeling of criterion validity and
construct validity. They have differed, too, on the
definition of specific validity types encompassed by
criterion validity and construct validity. For example, in
most of the methodological literature, overarching
categories have been termed face validity, content
validity, criterion validity, and construct validity
(Babbie, 2004; Litwin, 1995; Rosnow & Rosenthal,
2005). Litwin as well as Rosnow and Rosenthal have
further divided criterion validity into concurrent and
predictive validity, and have separated construct validity
into convergent and discriminant validity. Babbie, on the
other hand, has not subdivided criterion validity and
construct validity, and has used the terms "criterion" and
"predictive" validity interchangeably.
Trochim (2000) has taken a different approach, and has
both labeled and subdivided the types of validity in ways
that differ from the terminology of other methodologists.
Having argued that the establishment of validity in the
operationalization of any construct implies the general
term, "construct validity," Trochim has eliminated this
term entirely in labeling the different types of validity.Trochim has organized the types of validity under the
overall term, construct validity, as follows: (a)
translation validity (face and content validity), and (b)
criterion validity (predictive, concurrent, convergent,
and discriminant validity). (The term, "translation
validity" emerged from Trochim's search for a label to
describe the extent to which the operationalization
reflects the definition of the construct.) Thus, according
to Trochim, construct validity is the overarching term,
and translation validity and criterion validity encompass
the two major approaches of researchers in the
establishment of validity for a measurement tool.
In addition to differences in the organization and
labeling of validity terms, methodologists have differed
somewhat in their definition of the terms. There has
been general agreement that for face validity and content
validity, the criteria against which the instrument is
evaluated are internal-- the researcher checks the
operationalization of the construct by comparing the
items included in the measurement tool against a wellconstructed
definition of the construct (Babbie, 2004;
Litwin, 1995; Rosnow & Rosenthal, 2005; Trochim,
2000). For all other types of validity, external criteria
help establish that the operationalization of the construct
behaves as it should, based on the theoretical meaning of
the construct (Babbie; Litwin; Rosnow & Rosenthal;
Trochim). Methodologists agree, for example, that
predictive validity is based on tests of whether the
instrument can be used to predict outcomes for some
other variable or criteria in the future which it logically
should be able to predict (Babbie; Litwin; Rosnow &
Rosenthal; Trochim). Precise definitions for other
external criterion-based validity subcategories, however,
have presented a point of departure among
methodologists.
Litwin (1995), for example, has defined concurrent
validity as a statistical comparison of the new
instrument's results with results from an accepted
standard measure of the same construct. If both
measures were administered concurrently, a positive
correlation between the two would support the
concurrent validity of the new measure (Litwin).
Rosnow and Rosenthal (2005) have explained
concurrent validity to mean comparison with any
reasonable criterion representing measurement in the
present (concurrently). Researchers, they have admitted,
have no real basis upon which to determine what
constitutes a comparable set of reasonable criteria.
Trochim (2000) has taken a completely different
approach to the meaning of concurrent validity. Rather
than a comparison of measures, Trochim has defined
concurrent validity to mean a comparison of groups. According to Trochim, an instrument with concurrent
validity would have the ability to differentiate between
groups (samples) that should be distinct from one
another based on the theory underlying the construct.
Becker (1999), however, describes this contrasted
groups approach as a way of measuring convergent
validity rather than concurrent validity. But, the
interpretation of the definition and measurement
indicators for convergent validity differ among
methodologists as well. According to Becker (1999), in
addition to the contrasted groups approach, researchers
can use correlations between the instrument and other
measures of the same construct (concurrent validity) to
help establish convergent validity. Garson (2006) has
claimed that convergent validity refers not only to the
convergence of similar scales, but also to the correlation
among the indicators making up the instrument being
validated. Garson has stated that Cronbach's alpha, a
statistic used to establish internal consistency reliability,
also helps to establish convergent validity. Litwin has
maintained that convergent validity, although more
theoretical and labor-intensive, is similar to alternateform
reliability.
Discriminant validity represents another point of
departure for methodologists. Garson (2006) has
described discriminant validity as the degree to which
indicators making up the instrument can demonstrate
that they are sufficiently different from one another. In
other words, indicators should be correlated with one
another, but not perfectly correlated. Garson has pointed
out that factor analysis frequently has been conducted in
the establishment of discriminant validity. Litwin
(1995), Rosnow and Rosenthal (2004), and Trochim
(2000), however, have described discriminant validity
differently. They have stated that the measure itself,
rather than the indicators within the instrument, must
demonstrate that it measures something that is different
from similar yet distinct constructs. Clearly,
methodologists have differing views on the meanings
attached to the specific types of validity involved in the
development of a measurement tool.
There is considerably less controversy over what
constitutes reliability for a measurement tool.
Methodologists agree that reliability estimates the
consistency of an instrument, and most agree on
subdivisions of the term. Reliability includes
consistency (a) of the items making up the instrument
with the items themselves, (b) of results resulting from
use of the instrument over time, (c) of use with different
observers, and (d) when two different versions of the
instrument are constructed using the same content
(Litwin, 1995; Trochim, 2000).
Methodology
Definition of Terms Used in the Development of the
IFDFW Scale
Regardless of how they have defined and organized the
terms related to validity, methodologists have always
agreed that establishment of validity and reliability for
an instrument is extremely important. The critical factor,
then, is not how the validity terminology has been
delineated, but rather that assessment of the validity and
reliability of the instrument has been done in a thorough
and purposeful way. In the development of the IFDFW
Scale, 12 criteria guided the establishment of validity
and reliability for the instrument. See Table 1 for a list
of the criteria and assessment methods used.
For the purpose of this article, it is important that the
validity and reliability terminology used in the
development of the InCharge Financial Distress/
Financial Well-Being (IFDFW) Scale be detailed
carefully. Validity and reliability in the development of
the IFDFW Scale has been defined and organized as
follows.
Face Validity
Face validity is an informal judgment of the
appropriateness of the items included in the instrument.
The assessment represents the degree to which the
measurement tool, on its face, appears to measure what
it is supposed to measure (Litwin, 1995).
Content Validity
Content validity is an assessment of whether the items
included in the instrument encompass all of the major
aspects reflecting the conceptualization of the construct
(Litwin, 1995; Rosnow & Rosenthal, 2005). An
assumption is that the researcher has formulated a good
working definition of the construct within which to
frame the assessment of the appropriateness of the
content of the instrument (Trochim, 2000). Content
validity is assessed by experts with knowledge of the
subject matter (Litwin; Rosnow & Rosenthal).
Criterion Validity
Criterion validity is determined through comparison of
the operationalization of the construct with some
external yardstick or criterion. Statistical tests for
correlation help determine criterion validity, or the
degree to which the instrument is correlated with other
outcome variables with which it should be correlated
(Litwin, 1995; Rosnow & Rosenthal, 2005).
Concurrent criterion validity.
Concurrent validity
represents the correlation of outcomes from the new
instrument with outcomes on an accepted standard
measure of the same construct (Litwin, 1995) or with any reasonable criterion indicating presence of the
construct (Rosnow & Rosenthal, 2005). Researchers
consider the validity of the criterion itself, selecting for
comparison the most sensitive and meaningful
measurement available (Litwin; Rosnow & Rosenthal).
Finding a correlation after administering both
instruments concurrently (e.g., including both
instruments on the same questionnaire) helps to establish
concurrent validity (Becker, 1999; Rosnow &
Rosenthal, 2005).
Predictive criterion validity.
Predictive validity is an assessment of the instrument's
accuracy in predicting something it logically should be
able to predict based on the conceptualization of the
construct (Litwin, 1995; Rosnow & Rosenthal, 2005;
Trochim, 2000). That is, an instrument high in predictive
validity should be able to forecast some observable
behavior common to groups of people who fit in a
specific way on the construct being measured.
Construct Validity
Construct validity is less straightforward than face,
content, and criterion validity. Litwin (1995) contends
that construct validity is the most valuable way to
measure the usefulness of an instrument, but at the same
time represents the most difficult and elusive type of
validity to comprehend or to measure. Basically,
construct validity assesses the degree to which the
operationalization of the construct reflects its theoretical
meaning (Becker, 1999; Litwin; Rosnow & Rosenthal,
2005; Trochim, 2000). There is no best or single way to
measure construct validity; it represents a gestalt, or
accumulation of knowledge over time and repeated use
with different groups and in multiple settings (Becker;
Litwin).
Convergent construct validity - Convergent validity, a
type of construct validity, generally is described as
convergence across different measures, implying that
different measures of the same construct produce similar
results (Litwin, 1995; Rosnow & Rosenthal, 2005). For
the purposes of this article, convergent validity means
the degree of convergence of concepts making up the
instrument as well as convergence of the instrument
itself with different measures of the same construct. For
the IFDFW Scale, convergent validity has been assessed
using the Pearson Product Moment correlation of all
indicators making up the instrument (Becker, 1999), and
t tests for differences between groups that logically
should produce different outcomes on the measure
(Becker).
| Table 1 |
| Validity and Reliability: Criteria and assessment methods |
| Item# |
Item Description |
Assessment Methods |
| Face Validity |
| 1 |
Each concept must have face validity with people in the general adult population.
They would logically consider each concept as important to an individual’s
financial distress/financial well-being and recognize that each had the properties
ascribed to it. In essence, each item must be perceived on the face of it as
adequately covering the ideas people associate with the terms financial distress and/
or financial well-being. Adults untrained in measurement would perceive that the
instrument measures what it is intended to measure. Further, each concept must fit
the subject of financial distress and/or financial well-being and be a meaningful
descriptor of some aspect of that content. |
Application of four refinement
criteria prior to Delphi study data
collection; use of similar items in
previous research; survey of
personal finance college
professors and financial
education experts in business |
| 2 |
While the subject of personal finance certainly includes consumer credit (e.g., credit
cards, installment loans), no specific item should cover that specific topic, since
many adults do not use credit cards. |
|
| Content Validity |
| 3 |
Each personal finance concept denoted in an item must have been used in previous
conceptual frameworks and/or research. |
Review of literature for
identification of concepts |
| 4 |
Each item must have been highly ranked by the personal finance experts in the
Delphi study. Conclusions on the content validity of each item can be deduced
using insights from focus groups, individuals interviewed, statistical analysis, and
experts in personal finance. |
Delphi study of experts |
| 5 |
The list of personal finance concepts comprising the items should be a
representative sample of concepts in the total construct of financial distress/
financial well-being, and sufficient in number to assure content validity. |
Delphi study of experts; Beta
version of the instrument |
| Concurrent criterion validity |
| 6 |
The IFDFW Scale scores for the lower rankings on the instrument should
distinguish varying degrees of financial distress/financial well-being among a
population of initially financially distressed adults (i.e., those who have contacted a
consumer credit counseling agency). |
Use of “contacting of a consumer
credit counseling agency” as a
criterion to indicate financial
distress
Predictive criterion validity |
| 7 |
The scale items must exhibit predictive validity with adults exhibiting varying
levels of financial distress/financial well-being. |
T test for differences in billpaying
behavior of financially
distressed group and general
population |
| Convergent construct validity |
| 8 |
Each item must correlate well with other individual concepts associated with
personal financial distress or financial well-being; therefore, the collective concepts
must stand as an adequate measure of financial distress/financial well-being. |
Pearson Product Moment
correlation matrix |
| 9 |
The summative total scores on the scale should identify widely varying degrees of
the financial distress/financial well-being of the individuals responding to the
survey items, and scores should discriminate readily between those with more
financial distress/less financial well-being and those with less financial distress/
more financial well-being. |
Establishment of norms for
IFDFW Scale; T test for
differences in IFDFW mean
scores between financially
distressed group and general
population |
| Discriminant construct validity |
| 10 |
Each personal finance concept item must have construct validity, both logical and
factorial. It is rationally hypothesized that measures of financial distress and
financial well-being are correlated. Similarly, the scale items measure different
aspects of the qualities that make up the construct of financial distress, financial
well-being, or a combination of both. |
Factor analysis |
| 11 |
Each item must contribute to factor analysis results that suggest a single, rather than
multiple, factors. |
|
| Reliability (Internal consistency) |
| 12 |
Each item must contribute to a robust Cronbach’s Alpha score. |
Cronbach’s alpha statistic |
Discriminant construct validity.
Discriminant validity, a
second type of construct validity, has been defined
differently by different methodologists (Garson, 2006;
Litwin, 1995; Rosnow & Rosenthal, 2005; Trochim,
2002). In the development of the IFDFW Scale,
discriminant validity represented the degree of
discrimination among indicators used to measure the
construct, and was assessed using factor analysis
(Garson).
Reliability
Reliability has to do with the consistency of a measure,
both internally and with repeated usage (Trochim,
2000). In the development of the IFDFW Scale,
reliability referred to the internal consistency of the
instrument, an estimate of how reliable the indicators
were in their measurement of the same construct.
Cronbach's alpha was used to determine internal
consistency of items making up the measure.
Establishing Validity for the IFDFW Scale
Content and Face Validity
A number of criteria were used to assess content validity
for the instrument. First, each personal finance concept
denoted in an item must have been used in previous
conceptual frameworks and/or research. To assure that
this was done, developers of the IFDFW Scale reviewed
the work of personal finance researchers spanning over
four decades (Aldana & Liljenquist, 1998; Bailey,
Woodiel, Turner, & Young, 1998; Beutler & Mason,
1987; Blumstein & Schwartz, 1983; Danes & Rettig,
1993; Davis & Schumm, 1987; Drentea, 2000; Drentea
& Lavrakas, 2000; Freeman, Carlson, & Sperry, 1993;
Garman, Leech, & Grable, 1996; Godwin & Carroll,
1986; Hafstrom & Dunsing, 1973; Joo & Garman, 1998;
Kim, 1999; Lawrence, Carter & Verma, 1987; Mills,
Grasmick, Morgan, & Wenk, 1992; Mirowsky and Ross,
2003; Pittman & Lloyd, 1988; Porter & Garman, 1993;
Prochaska-Cue, 1993; Ross & Huber, 1985; Strumpel,
1976; Voydanoff, 1984; Walson & Fitzsimmons, 1993).
Those studies collectively referred to 58 concepts
reflecting some behavior, experience, perception, or
personal judgment about personal finance topics. It
became clear that there have been many approaches to
and perspectives on the concepts surrounding financial
distress and financial well-being.
Using as a starting point the 58 concepts determined to
be representative of different aspects of financial
distress/financial well-being, the developers of the
IFDFW Scale initiated a modified Delphi study to
further establish content validity for the instrument. A
Delphi study solicits input multiple times from a panel
of experts on the topic under review, eventually
establishing consensus (Custer, Scarcella, & Stewart,
1999). The modification consisted of presenting the
panel with pre-selected concepts to provide guidance
based on the literature, rather than relying solely on
input from the experts (Custer et al.).
Prior to the start of data collection in the Delphi study,
the developers of the IFDFW Scale set up four criteria
for refinement of the list of concepts in order to begin
establishing face validity for the instrument. To be
included in the list sent to the panel of experts, a concept
had to (a) clearly describe a distinct aspect of financial
distress and/or financial well-being; (b) be different
enough to avoid being confused with other concepts; (c)
be likely to occur in a substantive proportion of the
population; and (d) be likely to occur with adults
whether or not they utilized credit cards and installment
loans/leases. Following the criteria check, the resulting
list of financial distress/financial well-being concepts
consisted of 20 items. See Table 2 for a complete list of
the original 20 concepts used in the Delphi study. The
three-phase Delphi process reduced the list of items
through expert consensus from 20 to 10. All 10 of the
items making up the final list had been ranked
consistently in the top 10 by the experts during all three
phases of the Delphi data collection process. See
(Garman & Sorhaindo, 2005) for a detailed description
of the Delphi study. Table 3 contains the ranking of the
final 10 concepts.
A preliminary form of the scale, referred to as the Beta
version, represented an attempt to further clarify the
content validity of the instrument. The idea was to verify
whether the list of personal finance concepts making up
the instrument worked well together to represent the
construct of financial distress/financial well-being. The
Beta version, published in 2004 (Garman, Sorhaindo,
Kim et al.), represented a preliminary attempt to use a
set of items together to measure the concept, financial
distress/financial well-being. It included six items, four
of which subsequently were retained on the final version
of the IFDFW Scale. The four items retained for use in
the final version represented two items on financial
distress (stress "today" and stress "in general") and two
on financial well-being (satisfaction with and feelings
about one's current financial situation). Variations of
these four items had been tested over time in 10
different data collection efforts; items not highly
correlated with these four eventually were dropped, and
others more highly correlated were added for additional
testing. The purpose of the Beta version, then, was to
assess the usefulness of specific items in combination
with one another to help establish content validity. Table
4 contains the six items used in the Beta version.
| Table 2 |
| Twenty concepts making up the list for Delphi study of experts |
| Item Number |
Concepts (in alphabetical order) |
| 1. |
Ability to handle $1,000 financial emergency |
| 2. |
Ability to manage money |
| 3. |
Assessment of quality of personal financial behaviors |
| 4. |
Availability of savings to pay for 3 months’ living expenses |
| 5. |
Availability of money to go out for entertainment |
| 6. |
Availability of money to pay for minor emergency |
| 7. |
Confidence about a plan to reach financial goals |
| 8. |
Confidence about long-term financial future |
| 9. |
Confidence about being on track for a financially successful retirement |
| 10. |
Feelings about level of financial stress today |
| 11. |
Feelings about one’s current financial condition |
| 12. |
How good or bad finances are likely to be a year from now |
| 13. |
How well off financially |
| 14. |
Knowledge of personal finances |
| 15. |
Living today on a paycheck-to-paycheck basis |
| 16. |
Satisfaction with present financial situation |
| 17. |
Secure about one’s personal finances for retirement |
| 18. |
Spend some time at work on personal financial concerns |
| 19. |
Stressed about one’s personal finances in general |
| 20. |
Worry about being able to meet normal monthly living expenses |
| Table 3 |
| Rankings of 10 concepts emerging from final phase of Delphi study |
| Item Number |
Item Description |
Item ranka |
| 1 |
Worry about being able to meet normal monthly living expenses |
1.47 |
| 2 |
Living today on a paycheck-to-paycheck basis |
2.24 |
| 3 |
Feeling about one's current financial situation |
3.06 |
| 4 |
Stressed about one's personal finances in general |
3.23 |
| 5 |
Feelings about level of financial stress today |
3.27 |
| 6 |
Satisfaction with present financial situation |
3.38 |
| 7 |
Ability to handle $1,000 financial emergency |
4.00 |
| 8 |
Availability of money to pay for a minor emergency |
4.18 |
| 9 |
Knowledge of personal finances |
4.27 |
| 10 |
Ability to manage money |
4.62 |
| a Lower numbers indicate higher rankings |
| Table 4 |
| Items making up the Beta version of the instrument |
| Item |
Item description |
| 1 |
What do you feel is the level of your
financial stress today? a |
| 2 |
On the stair steps below, mark how
satisfied you are with your present
financial situation. a |
| 3 |
How well off are you financially? |
| 4 |
How do you feel about your current
financial situation? a |
| 5 |
How secure do you feel about your
personal finances for retirement? |
| 6 |
How stressed do you feel about your
personal finances in general? a |
| a Retained in the final version |
Criterion Validity
Concurrent criterion validity
One method of establishing concurrent criterion validity
for an instrument is to compare outcomes on the new
measure with outcomes of another reasonable measure
expected to indicate similar outcomes. To establish
concurrent criterion validity for the IFDFW Scale, the
reasonable measure chosen was having contacted a
consumer counseling credit agency for assistance with
financial problems. The fact that a consumer had made
such a contact indicated the likelihood that financial
distress was high and financial well-being was low for
that individual, thus constituting a criterion that could be
used for comparison with outcomes from the IFDFW
Scale. The mean score on the IFDFW Scale for a
separate sample of consumers who previously had
contacted a consumer credit counseling agency was M =
3.42 (a score that denotes high financial distress, low
financial well-being), indicating that the concurrent
assessment on the measure of financial distress/financial
well-being produced similar outcomes.
Predictive criterion validity
Predictive validity assesses whether an instrument is
useful in predicting behaviors it logically should be able
to predict. To establish predictive validity for the
IFDFW Scale, a comparison was made between two
samples of consumers. The first was a sample of 590
distressed consumers who had contacted a consumer
credit counseling agency for assistance; the other was a
representative sample of 1,298 consumers from the
general population. The groups first were compared on
their scores on the IFDFW Scale to test the hypothesis
that financially distressed consumers would report more
financial distress and less financial well-being than
would the general public. The hypothesis was supported,
as there was a significant difference in the mean scores
of the two groups on the IFDFW Scale, with the
financially distressed group scoring significantly lower
(M = 3.42, SD = 1.64) than the general population (M =
5.72, SD = 2.41), t (1,886) = -24.17, p < .0001. These
results were used as the basis for comparing the two
groups on bill-paying behaviors to establish predictive
validity.
Total mean scores were computed for occurrences of
the following bill-paying behaviors: (a) paying a credit
card bill late, (b) paying only the minimum amount due
on monthly credit card bill, (c) paying utility bills late
(beyond the due date), (d) receipt of notices from
creditors about overdue bills, and (e) receipt of letters or
telephone calls from creditors or collection agencies. It
was hypothesized that there would be a difference in
mean scores for the two groups, with the financially
distressed group displaying more negative bill-paying
behaviors (i. e., paying bills late). T-test results indicated
that this was the case, with lower mean scores reported
for the financially distressed consumers (M = 4.66, SD =
2.11) than for the general population (M = 8.10, SD =
2.37), t(1,865) = -31.41, p < .0001. The results
contributed to the establishment of the predictive
validity of the instrument.
Construct Validity
Convergent construct validity
Convergent construct validity represents the degree to
which the concepts making up an instrument converge,
as well as the degree of convergence of the instrument
itself with other measures of the same construct. To
begin establishing convergent validity for the IFDFW
Scale, the developers of the instrument, using a data set
from the general population (N = 1,300), obtained
Pearson Product Moment correlation coefficients for all
of the indicators included in the instrument (Becker,
1999). As can be seen in Table 5, the Pearson Product
Moment correlation matrix demonstrated that all of the
indicators were correlated with one another, indicating
convergence of the indicators making up the instrument.
A second method used to establish convergent validity
for the instrument was to test for differentiation between
groups hypothesized to report different levels of
financial distress/financial well-being. Consumers who
had contacted a consumer credit counseling agency were
expected to report more financial distress and less
financial well-being than were consumers from the
general population. Results from a t test comparing
IFDFW scores from a sample of 590 financially
| Table 5 |
| Pearson Product Moment correlations for the indicators making up the IFDFW Scale |
| Item |
Statistics |
Q-A |
Q-B |
Q-C |
Q-D |
Q-E |
Q-F |
Q-G |
Q-H |
| Q-A |
Pearson correlation |
1 |
|
|
|
|
|
|
|
| Sig. (2-tailed) |
|
|
|
|
|
|
|
|
| N |
1127 |
|
|
|
|
|
|
|
| Q-B |
Pearson correlation |
.788** |
1 |
|
|
|
|
|
|
| Sig. (2-tailed) |
.000 |
|
|
|
|
|
|
|
| N |
1124 |
1291 |
|
|
|
|
|
|
| Q-C |
Pearson correlation |
.725** |
.644** |
1 |
|
|
|
|
|
| Sig. (2-tailed) |
.000 |
.000 |
|
|
|
|
|
|
| N |
1126 |
1291 |
1293 |
|
|
|
|
|
| Q-D |
Pearson correlation |
.704** |
.818** |
.825** |
1 |
|
|
|
|
| Sig. (2-tailed) |
.000 |
.000 |
.000 |
|
|
|
|
|
| N |
1125 |
1290 |
1292 |
1292 |
|
|
|
|
| Q-E |
Pearson correlation |
.718** |
.824** |
.826** |
.927** |
1 |
|
|
|
| Sig. (2-tailed) |
.000 |
.000 |
.000 |
.000 |
|
|
|
|
| N |
1126 |
1291 |
1293 |
1292 |
1293 |
|
|
|
| Q-F |
Pearson correlation |
.651** |
.725** |
.745** |
.684** |
.690** |
1 |
|
|
| Sig. (2-tailed) |
.000 |
.000 |
.000 |
.000 |
.000 |
|
|
|
| N |
1124 |
1289 |
1291 |
1290 |
1291 |
1291 |
|
|
| Q-G |
Pearson correlation |
.650** |
.738** |
.759** |
.682** |
.699** |
.751** |
1 |
|
| Sig. (2-tailed) |
.000 |
.000 |
.000 |
.000 |
.000 |
.000 |
|
|
| N |
1113 |
1270 |
1272 |
1271 |
1272 |
1271 |
1276 |
|
| Q-H |
Pearson correlation |
.689** |
.772** |
.795** |
.714** |
.717** |
.795** |
.828** |
1 |
| Sig. (2-tailed) |
.000 |
.000 |
.000 |
.000 |
.000 |
.000 |
.000 |
|
| N |
1104 |
1261 |
1263 |
1262 |
1263 |
1262 |
1262 |
1267 |
| Q-A |
On the stair steps below, mark how satisfied you are your present financial situation. |
| Q-B |
How do you feel about your current financial situation? |
| Q-C |
How often do you worry about being able to meet normal monthly living expenses? |
| Q-D |
What do you feel is the level of your financial stress today? |
| Q-E |
How stressed do you feel about your personal finances in general? |
| Q-F |
How confident are you that you could find the money to pay for a financial emergency that costs about
$1,000? |
| Q-G |
How often does this happen to you? You want to go out to eat, go to a movie or do something else and don’t
go because you can’t afford it? |
| Q-H |
How frequently do you find yourself just getting by financially and living paycheck to paycheck? |
distressed consumers (M = 3.42, SD = 1.64) and a
sample of 1,298 consumers representing the general
population (M = 5.72, SD = 2.41), indicated that the
IFDFW Scale was able to differentiate between groups
that logically should produce different results on the
measure, t(1,886) = -24.17, p < .0001. These results
contributed to the establishment of the convergent
validity of the IFDFW Scale.
Discriminant construct validity
Discriminant construct validity has been defined
differently by different methodologists (Garson, 2006;
Litwin, 1995; Rosnow & Rosenthal, 2005; Trochim,
2000), but basically it represents the degree to which the
instrument can demonstrate differentiation among
constructs similar to the one being measured. According
to Garson, factor analysis sometimes is used to assess
discriminant validity, as it allows the researcher to
discriminate among subscales representing similar yet
distinct variables within the same instrument. The factor
analysis conducted for the IFDFW Scale using a sample
of the general population (N = 1,097) produced one
factor, indicating that the instrument represented
measurement of a single, rather than several constructs.
See Table 6 for factor loadings of IFDFW Scale items.
Establishing Reliability for the IFDFW Scale
Cronbach's alpha helped establish the internal
consistency reliability of the set of indicators
representing the IFDFW Scale. According to Nunnally
and Bernstein (1994), the Cronbach's alpha statistic
provides a calculation of the ability of a group of items
to measure a unidimensional construct. A high score
(closer to 1.0) indicates that the indicators as a group
represent a unidimensional construct. A low score
(distant from 1.0) indicates that the indicators are
pointing to different constructs rather than working
together to represent the same construct. Nunnally and
Bernstein further contend that, while a Cronbach's alpha
of 0.60 or higher is considered acceptable for group
scores, the minimum acceptable Cronbach's alpha for
use with individual scores should not fall below 0.90.
An alpha of 0.95 or higher is the desirable standard
(Nunnally & Bernstein); the Cronbach's alpha for the
IFDFW Scale was 0.956, indicating excellent internal
consistency/reliability. The eight items making up the
IFDFW Scale, when used together, contribute to a
consistent measurement of the construct, financial
distress/financial well-being.
Summary
The developers of the IFDFW Scale have carried out
purposeful assessment procedures to establish the
validity and reliability of the instrument. Using multiple
methods and tests, the developers have provided evidence that the IFDFW Scale is both valid and
reliable.
Face and content validity were established first. To
assure face validity of the instrument, defined as an
informal judgment of the appropriateness of the items
included in the instrument (Litwin, 1995), the
researchers applied four refinement criteria to each item,
used items similar to those used in previous research,
and surveyed personal finance professors, educators, and
experts in business. A review of the literature for
identification of concepts, a Delphi study of experts, and
the development of the Beta version of the instrument
helped determine content validity, an assessment of
whether the items included encompassed all of the major
aspects reflecting the conceptualization of the construct
(Litwin; Rosnow & Rosenthal, 2005).
To establish concurrent criterion validity, or the
correlation of outcomes from the new instrument with
any reasonable criterion indicating presence of the
construct (Rosnow & Rosenthal, 2005), the developers
of the IFDFW Scale chose the contacting of a consumer
credit counseling agency as a criterion to indicate
financial distress. As hypothesized, a sample of
consumers who had made such a contact scored in the
range of high financial distress/low financial well-being
on the IFDFW Scale.
The developers of the IFDFW Scale defined predictive
criterion validity as the ability of the instrument to make
an accurate prediction of something it logically should
be able to predict based on the conceptualization of the
construct (Litwin, 1995; Rosnow & Rosenthal, 2005;
Trochim, 2000). A t test for differences in the billpaying
behavior of financially distressed consumers and
that of the general population provided evidence of
predictive criterion validity.
Convergent construct validity implies that different
measures of the same construct produce similar results
(Litwin, 1995; Rosnow & Rosenthal, 2005). Three
separate assessment measures confirmed convergent
construct validity for the IFDFW Scale: a) generation of
a Pearson Product Moment correlation matrix, b)
establishment of norms for the IFDFW Scale, and c) a t
test for differences in IFDFW mean scores between
financially distressed consumers and the general
population.
Discriminant construct validity represents the degree to
which indicators can demonstrate that they are
sufficiently different from one another (Garson, 2006).
A good balance is indicated when an instrument's items
are correlated with one another, but not perfectly
| Table 6 |
| Factor loadings for the eight items making up the IFDFW Scale (General pop., N = 1,097) |
| Item # |
Item description |
Factor loading |
| 1 |
What do you feel is the level of your financial stress today? |
.905 |
| 2 |
On the stair steps below, mark how satisfied you are with your present financial
situation. |
.833 |
| 3 |
How do you feel about your current financial situation? |
.921 |
| 4 |
How often do you worry about being able to meet normal monthly living expenses? |
.926 |
| 5 |
How confident are you that you could find the money to pay for a financial
emergency that costs about $1,000? |
.857 |
| 6 |
How often does this happen to you? You want to go out to eat, go to a movie or do
something else and don’t go because you can’t afford to. |
.861 |
| 7 |
How frequently do you find yourself just getting by financially and living paycheck
to paycheck? |
.891 |
| 8 |
How stressed do you feel about your personal finances in general? |
.909 |
| |
Eigenvalue |
6.314 |
| |
Proportion of variance explained |
.789 |
correlated. Factor analysis gave clear evidence that the
indicators making up the IFDFW Scale, when used
together, measured one single construct rather than
separate variables.
In the development of the IFDFW Scale, reliability was
defined as internal consistency of the measure. That is,
the indicators making up the instrument should each
contribute to the measurement of the same construct.
Cronbach's alpha provided evidence of the reliability of
instrument.
In every single assessment method used to determine the
validity of the IFDFW Scale, there was clear evidence
that the instrument was measuring what it was supposed
to measure. The test for reliability indicated that there
was internal consistency among the various indicators of
the construct. The IFDFW Scale, then, represents a valid
and reliable measurement tool to assess levels of
financial distress/financial well-being in both the general
population and groups of financially distressed
consumers. Based on the evidence provided during
development of the instrument over a multi-year period,
researchers, financial educators, practitioners, and
employers using the IFDFW Scale can be confident that
the instrument consistently will provide accurate and
appropriate measurement of financial distress/financial
well-being in a variety of settings and across multiple
populations. According to Nunnally and Bernstein(1994), the Cronbach's alpha level produced by the
IFDFW Scale indicates that practitioners also can use
the instrument with confidence in assessing financial
distress/financial well-being of individual clients.
correlated. Factor analysis gave clear evidence that the
indicators making up the IFDFW Scale, when used
together, measured one single construct rather than
separate variables.
In the development of the IFDFW Scale, reliability was
defined as internal consistency of the measure. That is,
the indicators making up the instrument should each
contribute to the measurement of the same construct.
Cronbach's alpha provided evidence of the reliability of
instrument.
In every single assessment method used to determine the
validity of the IFDFW Scale, there was clear evidence
that the instrument was measuring what it was supposed
to measure. The test for reliability indicated that there
was internal consistency among the various indicators of
the construct. The IFDFW Scale, then, represents a valid
and reliable measurement tool to assess levels of
financial distress/financial well-being in both the general
population and groups of financially distressed
consumers. Based on the evidence provided during
development of the instrument over a multi-year period,
researchers, financial educators, practitioners, and employers using the IFDFW Scale can be confident that
the instrument consistently will provide accurate and
appropriate measurement of financial distress/financial
well-being in a variety of settings and across multiple
populations. According to Nunnally and Bernstein
(1994), the Cronbach's alpha level produced by the
IFDFW Scale indicates that practitioners also can use
the instrument with confidence in assessing financial
distress/financial well-being of individual clients.
Case Examples of Organizations Using the
IFDFW Scale
The IFDFW Scale has a number of implications for use;
following are four illustrative case examples of
organizations using the IFDFW. TwoMedicine Health
and Financial Fitness has integrated the IFDFW into the
annual Mayo Clinic Health Risk Assessment that is
completed annually by thousands of Mayo patients, thus
obtaining information on financial distress levels,
perceptions of financial well-being, effects on health,
and demand data on specific areas of financial planning
for individuals and organizations. The Foundation for
Financial Literacy uses the IFDFW to assess the
financial stress and well-being of thousands of
employees of 15 Texas corporations who accept the
foundation's "Fiscal Fitness Challenge" and participate
in its online course, "Passport to Fiscal Fitness." Before
and after data collection tracks changes, advances, and
progress. The Pennsylvania Office of Financial
Education uses the IFDFW in its workplace financial
literacy program to help judge the effectiveness of its
presentations to employees of the Commonwealth as
well as to help private employers train their employees
in ways that both complement their business objectives
and boost their bottom lines. The University of
Minnesota's Latino Financial Literacy Program has
translated the IFDFW into Spanish and compares
participants' pre- and post-class participation scores.
This information is being used to help Extension better
meet client's educational needs.
For additional suggestions for use of the IFDFW Scale
in research and the workplace, and for complete
instructions for administering the instrument and
interpreting results, see (Prawitz et al., 2006). The
IFDFW Scale is copyrighted, but is available for use.
Approval for use may be obtained by contacting
bsorhain@incharge.org or ethomasgarman@pfeef.org.
The Appendix contains a copy of the IFDFW Scale.
Sincere appreciation is extended to the InCharge
Education Foundation and the InCharge Institute of
America for their cooperation and support of this
research.
Aldana, S. G., & Liljenquist, W. (1998). Validity and
reliability of a financial strain survey. Financial
Counseling and Planning, 9(2), 11-18.
Babbie, E. (2004). The practice of social research (10th
ed.). Belmont, CA: Wadsworth.
Bailey, W. C., Woodiel, D. K., Turner, M. J., & Young,
J. (1998). The relationship of financial stress to
overall stress and satisfaction. Personal Finances
and Worker Productivity, 2(2), 198-207.
Beutler, I. F., & Mason, J. W. (1987). Family cash flow
budgeting. Home Economics Research Journal,
16, 3-12.
Becker, L. A. (1999). Reliability and validity, Part II.
Retrieved May 24, 2006 from http://
web.uccs.edu/lbecker/Psy590/relval_II.htm
Blumstein, P. W., & Schwartz, P. (1983). American
couples: Money, work, sex. New York: William
Morrow.
Danes, S. M., & Rettig, K. D. (1993). The role of
perception in the intention to change the family
financial situation. Journal of Family and
Economic Issues, 14(4), 365-389.
Davis, P., & Schumm, W. R. (1987). Family financial
satisfaction: The impact of reference points.
Home Economics Research Journal, 14, 123-131.
Drentea, P. (2000). Age, debt, and anxiety. Journal of
Health and Social Behavior, 41, 437-450.
Drentea, P., & Lavrakas, P. J. (2000). Over the limit:
The association among health status, race and
debt. Social Science & Medicine, 50, 517-529.
Freeman, C., Carlson, J., & Sperry, L. (1993). Adlerian
marital therapy strategies with middle income
couples facing financial stress. American Journal
of Family Therapy, 21(4), 324-332.
Garman, E. T., Leech, I. E., & Grable, J. E. (1996). The
negative impact of employee poor personal
financial behaviors on employers. Financial
Counseling and Planning, 7, 157-168.
Garman, E. T., & Sorhaindo, B. (2005). Delphi study of
experts' rankings of personal finance concepts
important in the development of the InCharge
financial distress/financial well-being scale.
Consumer Interests Annual, 51, 184-194.
Garman, E. T., Sorhaindo, B., Kim, J., Xiao, J. J.,
Bailey, W., & Prawitz, A. (2004). The evolution
of the Beta Version of the InCharge Financial
Distress Scale. Consumer Interests Annual, 50,
134-144.
Garson, G. D. (2006). Validity. Retrieved May 24, 2006
from http://www2.chass.ncsu.edu/garson/PA765/
validity.htm
Godwin, D. D., & Carroll, D. D. (1986). Financial
management attitudes and behaviors of husbands
and wives. Journal of Consumer Studies and
Home Economics, 10, 77-96.
Hafstrom, J. L., & Dunsing, M. M. (1973). Level of
living: Factors influencing the homemakers'
satisfaction. Home Economics Research Journal,
2(2), 119-132.
Joo, S., Garman, E. T. (1998). The relationship between
personal financial wellness and employee
productivity: A conceptual model. Personal
Finances and Worker Productivity, 2(2), 162-171.
Kim, J. (1999). Financial satisfaction, personal financework
conflict, and work outcomes: Pay
satisfaction, organizational commitment, and
productivity. Proceedings of the Association for
Financial Counseling and Planning Education,
16, 38-45.
Lawrence, F. C., Carter, C. G., & Verma, S. (1987).
Financial satisfaction in relation to financial
management. Proceedings of the American
Council on Consumer Interests, 337.
Litwin, M. S. (1995). How to measure survey reliability
and validity. Thousand Oaks, CA: Sage.
Mills, R. J., Grasmick, H. G., Morgan, C. S., & Wenk,
D. (1992). The effects of gender, family
satisfaction, and economic strain on psychological
well-being. Family Relations, 41, 440-446.
Mirowsky, J., & Ross, C. (2003). Social causes of
psychological distress (2nd ed). New York: Aldine
de Gruyter.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric
Theory (3rd ed). New York: McGraw Hill.
Pittman, J. F., & Lloyd, S. A. (1988). Quality of family
life, social support, and stress. Journal of
Marriage and the Family, 50, 53-67.
Porter, N. M., & Garman, E. T. (1993). Testing a
conceptual model of financial well-being.
Financial Counseling and Planning, 4, 135-164.
Prawitz, A. D., Garman, E. T., Sorhaindo, B., O'Neill,
B., Kim, J., & Drentea, P. (2006). The InCharge
financial distress/financial well-being scale:
Development, administration, and score
interpretation. Financial Counseling and
Planning, 17(1), 34-50.
Prochaska-Cue, K. (1993). An exploratory study for a
model of personal financial management style.
Financial Counseling and Planning, 4, 111-134.
Rosnow, R. L., & Rosenthal, R. (2005). Beginning
behavioral research: A conceptual primer (5th
ed). Upper Saddle River, NJ: Pearson Prentice
Hall.
Ross, C., & Huber, J. (1985). Hardship and depression.
Journal of Health and Social Behavior, 26, 312-
27.
Strumpel, B. (Ed.) (1976). Economic Means for Human
Needs. Ann Arbor, MI: Institute for Social
Research.
Trochim, W. (2000). The research methods knowledge
base (2nd ed). Cincinnati, OH: Atomic Dog
Publishing.
Voydanoff, P. (1984). Economic distress and families.
Journal of Family Issues, 5(2), 273-288.
Walson, C. O., & Fitzsimmons, V. S. (1993). Financial
manager's perception of rural household
economic well-being: Development and testing of
a composite measure. Journal of Family and
Economic Issues, 14(3), 193-215.
1 School of Family, Consumer, and Nutrition Sciences, DeKalb, IL 60115, 815.753.6344 Fax: 815.753.1321,
aprawitz@niu.edu
Click here to downlaod this article (PDF).
|
|