Department of Social Security Community Research Project · PDF fileDepartment of Social...
96
• • • • • • • • • • Department of Social Security Community Research Project Pre & post-test eval nation of Action Research Proj ects: Analysis of living standards change David de Vaus & DianaAmos 5th June 1997 Australian Institute of Family Studies AIFS 339.47 i 0994 DEV
Transcript of Department of Social Security Community Research Project · PDF fileDepartment of Social...
•
•
•
•
•
•
•
•
•
•
Department of Social Security Community Research Project
Pre & post-test eval nation of Action Research Proj ects: Analysis of living standards change
David de Vaus &
DianaAmos
5th June 1997
Australian Institute of Family Studies
AIFS 339.47 i
0994 DEV
•
•
•
•
•
•
•
•
•
•
TABLE OF CONTENTS
TABLE OF CONTENTS ......................................................................................................................... i
LISIOFTABLES .•................•.......................... .................................................................................. iv
LISIOFFIGURES ............................................. ................................................................................. V
ExECUTIVE SUMMARY ... ................................................................................................................... vi
1. BACKGROUND TO THE CRP .................................................................................................. 1
1.1 LIVING STANDARDS FRAMEWORK ............................................................................................... 2
1.2 COMMUNITY BASED SERVICE DEUVERY FRAMEWORK ................................................................. 3
1.3 DEVELOPMENT OF THE CRP WITHIN AN ACTION RESEARCH APPROACH ........................................ 4
1.4 DESIGN OF THE QUANTITATIVE EVALUATION .............................................................................. 5
1.4.1 The research design .......................................................................................................... 5
1.4.2 The pre and post-tests ....................................................................................................... 6
1.4.3 Administration of the data collection ................................................................................ 8
1.4.4 The questionnaire .............................................................................................................. 9
1.4.5 Living standards areas to be examined ........................................................................... 10
1.4.6 The questions .............................................................................................. ................... 10
1.5 PuRPOSE OF THIS REPORT ........................................................................................................ 13
2. WHY SO LITTLE CHANGE? •••........•••••••••••••.••••••••••••.••••••••••••••••••••••• ~ ••••••••••••••••••••••••••••..•••••• 14
2.1 THEORY .................................................................................................................................. 15
2.2 DESIGN AND METHOD .............................................................................................................. 18
2.2.1 Design ............................................................................................................................ 18
2.2.2 Method ........................................................................................................................... 21
2.2.3 Sample size ..................................................................................................................... 23
2.3 MEASURES ....................................................................................................................... : ...... 25
2.3.1 Standardised measures ................................................................................................... 25
2.3.2 Questionnaire length ......... ~ ............................................................................................ 26
2.3.3 Language ....................................................................................................................... 26
2.3.4 Layout/method of administration .................................................................................... 27
2.3.5 Measures of living standards .......................................................................................... 27
2.4 WHY THE DIFFERENCE FROM THE QUALITATIVE RESEARCH? ...................................................... 28
2.5 METHODOLOGICAL RECOMMENDATIONS .................................................................................. 30
3. METHOD OF ANAL YSIS ••••••••••••••••••.•••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• 32
3.1 PROJECT CHARACTERISTIC ANALYSIS .................................. : .................................................... 32
3.1.1 Computing change scores ............................................................................................... 32
3.1.2 Group level change: the use of means ............................................................................ 33
3.1.3 The impact of project characteristics ......... ................ LlBRAR.Y ....................................... 33 AUSTRALIAN INSTITUTE OF FAMILY STUDIES
tE'o'l!t 20. 485 LA i ROBE S i REE , MELBOURNE 3000 Ph: 039214 7888
i
• 3.1.4 Which groups will be examined? .................................................................................... 34
3.1.5 Project characteristics ................................................................................................... 34
3.2 INDIVIDUAL CHARACTERISTICS ANALYSIS ................................................................................ 37
• 3.3 SIGNIFICANCE LEVELS .....................................................................................•....................... 39
4. THE SAMPLE ••••.••••••••••••••....•••••••••••••••.•••••..•••••••••.••••.••••••••••••.••••••.••••••••••••••••••••••••••••••••••••••••••• 40
5. OVERALL CHANGE: PROJECT PARTICIPANTS AND CONTROL GROUP COMPARED
• ........................................................................................................................................................ 41
6. THE IMPACT OF PROJECT CHARACTERISTICS ON LIVING STANDARDS CHANGE43
6.1 'SUCCESSFUL PROJECT?' .......................................................................................................... 44
6.1.1 Hypothesis: .................................................................................................................... 44
• 6.1.2 Results ........................................................................................................................... 44
6.2 FIRST ORDER OUTCOMES AND SECOND ORDER OPPORTUNmES ................................................... 47
6.2.1 Hypothesis ...................................................................................................................... 48
6.2.2 Results ........................................................................................................................... 48
• 6.3 DID THE PROJECT TARGET THE PARTICULAR LSA? .................................................................... 52
6.3.1 Hypothesis: .................................................................................................................... 52
6.3.2 Results ........................................................................................................................... 52
6.4 LOCALLY ASSESSED NEEDS? .................................................................................................... 54
6.4.1 Hypothesis .............. : ....................................................................................................... 54
• 6.4.2 Results ........................................................................................................................... 55
6.5 SERVICE DELIVERY ORIENTATION ............................................................. : .............................. 59
6.5. 1 Hypothesis ................................................................................. : .................................... 59
6.5.2 Results ............................................................................................................................ 60
• 6.6 EXISTING CORE GROUP? .......................................................................................................... 62
6.6.1 Hypothesis .... .................................................................................................................. 6r
6.6.2 Results ........................................................................................................................... 62
6.7 ACTIVITY RANGE OF GROUP ..................................................................................................... 64
6. 7.1 Hypothesis ......... ............................................................................................................. 64
• 6.7.2 Results ........................................................................................................................... 65
6.8 AoAPTABILITY/FLEXIBILITY OF THE PROJECT ........................................................................... 67
6.8.1 Hypothesis ...................................................................................................................... 67
6.8.2 Results ........................................................................................................................... 67
6.9 TYPE OF ACTIVITY: FIVEFOLD PROJECT CLASSIFICATION ............................................................ 69
• 6.10 SUMMARY OF THE IMPACT OF GROUP CHARACTERISTICS .......................................................... 74
7. INDIVIDUAL CHANGE ........................................................................................................... 75
7.1 SOCIAL CHARACTERISTICS OF PARTICIPANTS ............................................................................ 78
• ii
• 7.1.1 Education level .............................................................................................................. 78
7. 1.2 Income level .................................................... ; .............................................................. 78
7.1.3 Home ownership ................................................. ~ ........................................................... 78
• 7.1.4 Ethnic background ......................................................................................................... 79
7.1.5 Employment status ........................................................................................................ 79
7. 1. 6 Marital status ................................................................................................................. 81
7.1.7 Gender ........................................................................................................................... 81
• 7.1.8Age ................................................................................................................................. 82
7.1.9 Attendance frequency ........ ............................................................................................. 83
7.1.10 Sociable participants .................................................................................................... 84
7.2 SUMMARY ............................................................••.............•.....................•............................ 85
References 87
•
•
•
•
•
•
• iii
•
•
•
•
•
•
•
•
•
•
List of Tables
TABLE 1 ALPHA COEFFICIENTS FOR BASE MODULE AND SPECIAL MODULE SCALES USED IN ANALYSIS OF
DSS CRP DATA. .............................................................................•... : ..................................•. 12
TABLE 2 LEVEL AND DURATION OF PARTICIPATION IN PROJECTS .......................................................... 17
TABLE 3: PROJECT AND RESPONDENT NUMBERS AND RETENTION RATES .................•....•.•.....................• 40
TABLE 4: MEAN CHANGE ON 31 LIVING STANDARDS MEASURES: A COMPARISON OF PROJECT
PARTICIPANTS AND CONTROL GROUP RESPONDENTS ......................................•.....•...................... 41
TABLE 5: LIVING STANDARDS CHANGE ACCORDING TO THE RATED SUCCESS OF THE PROJECT ........•....... 44
TABLE 6: LIVING STANDARDS CHANGE BY ACHIEVEMENT OF FIRST ORDER OUTCOMES AND SECOND ORDER
OPPORTIJNmES ...............................................................................................•............•.....•..... 49
TABLE 7: LIVING STANDARDS CHANGE BY PROJECT TARGETING ...........................•....•......•.............•.••.. 52
TABLE 8: LIVING STANDARDS CHANGE BY TYPE OF LOCAL NEEDS ASSESSMENT .............•................•...... 55
TABLE 9: LIVING STANDARDS CHANGE BY SERVICE DELIVERY ORIENTATION ........................................ 60
TABLE 10: LIVING STANDARDS CHANGE BY TYPE OF CORE GROUP ........................................................ 62
TABLE 11: LIVING STANDARDS CHANGE BY PROJECT ACTIVITY RANGE .....•...........•............................... 65
TABLE 12: LIVING STANDARDS CHANGE BY PROJECT FLEXIBILITY ........................................................ 67
TABLE 13: LIVING STANDARDS CHANGE BY TYPE OF PROJECT ACTIVITY .....................•••......................• 71
TABLE 14: LIVING STANDARDS CHANGE AMONG INDIVIDUALS: CONTROL GROUP AND PROJECT
PARTICIPANTS COMPARED (PER CENT HAVING PosmVE CHANGE) .....••..........••............................• 75
TABLE 15: PosmVE LIVING STANDARDS CHANGE BY TENURE TYPE (PER CENT PosmVE CHANGE) ......... 78
TABLE 16: PosmVE LIVING STANDARDS CHANGE BY EMPLOYMENT STATUS (PER CENT PosmVE CHANGE)80
TABLE 17: PosmVE LIVING STANDARDS CHANGE BY MARITAL STATUS (PER CENT POsmVE CHANGE) ... 81
TABLE 18: PosmVE LIVING STANDARDS CHANGE BY GENDER (PER CENT POsmVE CHANGE) ................. 82
TABLE 19: MEAN AGE (IN YEARS) BY LIVING STANDARDS CHANGE .................................•.................... 83
TABLE 20: PER CENT PosmVE CHANGE BY FREQUENCY OF ATTENDANCE AT THE PROJECT .•........•......... 84
TABLE 21: PosmVE CHANGE IN LIVING STANDARDS BY PROPENSITY TO JOIN COMMUNITY GROUPS •...... 8r
iv
• List of Figures
FIGURE 1: RESEARCH DESIGN OF THE QUANTITATIVE EVALUATION .•.....................•..........•...................... 6
• FIGURE 2: MEASURES FOR EACH OF THE EIGHT LIVING STANDARDS AREAS ............................................ 10
•
•
•
•
•
•
•
• v
•
•
•
•
•
•
•
•
•
•
Executive summary
This report analyses the findings of the quantitative pre-test and post-test evaluation of
the impact of the Action Research Projects component of the Community Research
Project on eight dimensions of the living standards of project participants. Overall the
level of change was modest and restricted to a limited number of areas of living
standards. Although the way in which services were delivered had some of the
anticipated impacts on living standards these impacts were modest and partial. An
examination of living standards change of individuals indicated that a range of the
living standards of participants , especially younger participants, were more likely to
improve than were those of non participants. However these changes were generally
small and limited. Overall, the pre-post test analysis revealed less evidence of living
standards improvement than was suggested by the qualitative evaluation of the ARPs.
Reasons for this difference and for the limited level of observed change are discussed
and recommendations regarding alternative ways of evaluating this type of community
intervention are made.
vi
•
•
•
•
•
•
•
•
•
•
The Action Research Projects (ARPs) which are the subject of this evaluation report
were part of the Department of Social Security (DSS) Community Research Project
(CRP). The purpose of the CRP was to examine new ways of improving the Living
Standards of DSS clients and to identify effective modes of service delivery of such
programs. The following description of the CRP draws on the overview of the CRP
by Maher and Smith (1996)
1. Background to the CRP
The CRP was based on the recognition that income support alone represents a limited
way of improving the economic and social well being of DSS clients and low income
citizens.
Labour market restructuring, demographic change and changes in family and
household structures have important implications for Australia's social security system.
Growth in long-term unemployment, the aging of the population and the growth of
sole parent households mean that the number and diversity of people requiring income
support has increased. Long term welfare dependency has prompted social policy
analysts to re-evaluate the methods of supporting those who require it (Cass, 1986).
Increasingly policy analysts (DECD, 1988a; 1988b) are arguing that income support
can be usefully supplemented by measures designed to enable welfare recipients to be
less dependent and to take action themselves to improve aspects of their living
standards.
There is a growing recognition (Travers and Richardson, 1993; Browniee, 1990) that
material living standards are dependent on more than income and can be improved by
means such as skills development, pooling of resources with others, local self-help
initiatives and cash income. Social and psychological well-being can be improved
through facilitating social participation.
DSS has responded to this thinking by initiating social policy research into the
feasibility of creating a resource, opportunity and information rich environment, to help
DSS clients improve their living standards. Furthermore, in recognition of the
importance of both cash and non cash resources and of the importance ofDSS clients
Page 1
•
•
•
•
•
•
•
•
•
•
being active participants in improving their living standards DSS has given increased
emphasis to the development of programs to support these outcomes.
The CRP was designed to facilitate these outcomes further by providing individuals
with a resource, opportunity and information rich environment. The structure of the
CRP was designed to determine the utility of a living standards framework and a
community based service delivery framework as an effective means of achieving the
CRP goals
1.1 Living standards framework
Until the 1970s, living standards were generally considered to be primarily a function
of a person's or household's 'real' disposable income and was the main method of
measuring changes in living standards. More recently however it has been recognised
that ' ... a person's living standard is multi-dimensional and cahriot be measured simply
in terms of cash income'. (McDonald et aI, 1995:30; see also Travers and Richardson,
1993; Brownlee, 1990; Townsend, 1979). Among these non cash income dimensions
of living standards are health and educational status, opportunities for social and
political participation, access to information, family relationships and support
networks, recreational activity and environmental conditions (Brownlee, 1990).
To the extent that living standards are multi-dimensional it follows that there may also
be a range of ways of improving living standards. To the extent that some of these
dimensions of living standards are independent of income then non income based
methods need to be adopted to enhance these aspects of living standards. Indeed tluL
components of the 'social wage' - public health, education, <;:hildcare, housing and
labour market assistance reflects an acceptance by· Australian society that some
support services are best provided in non-cash forms. As McDonald et al (1995:30)
noted:
' ... the proposition has been put, especially in the White Paper, Working Nation,
that provision in the form of cash alone is marginalising. That is, societal
responsibility goes beyond merely providing a subsistence income to people and
then leaving them to their own ends. Social responsibility in a civil society extends
to the provision of rights and resources to citizens so that they have control over
Page 2
•
•
•
•
•
•
•
•
•
•
their own lives and to ensuring that all citizens are full participants in the broader
life of the community.'
A central goal of the CRP was to explore additional ways in which non-cash assistance
in the form of public or community sector service provision, could enhance the living
standards of people on low incomes.
1.2 Community based service delivery framework
Community based, self help approaches to improving the well being of low income
people through client empowerment are not new to Australia (e.g. the Brotherhood of
StLaurence Family Centre Project (Gilley, 1990) and the Australian Assistance Plan).
Currently, individuals and communities are engaged in a wide range of community
activities such as Local Employment and Trading Systems (LETS), food co
operatives, small business co-operatives, community and neighbourhood centres with
varying degrees of government support. This community based mode of service
provision is based on the view that community organisations are closer to the clients
and therefore result in more effectively targeted programs than distant government and
bureaucratically structured agencies can provide.
Community based and self-help approaches are consistent with the themes of the
current debate about citizenship, social participation and inclusion. Broadening the
social security agenda to include ways of assisting social security recipients to become
full members of the community is consistent with the thrust of the current citizenship
debate. One way of fostering social inclusion and social participation is by adopting .L
community development approach which:
, ., .involves the establishment of supportive communities, which are based on
mutual support and self-help and in which people identify, develop and share
resources. Underlying the community development approach is a belief that
people who are directly affected by a condition are those who know most about
it, and in general they are the people who are best able to make a major
contribution to doing something about it - in short that people can and should
take more responsibility for identifying their own needs and managing their own
welfare.' (Hartley, 1995:13).
Page 3
•
•
•
•
•
•
•
•
•
•
It was for these reasons that the CRP adopted a community based model of service
delivery as the means of improving non cash based dimensions of living standards.
1.3 Development of the CRP within an action research approach
The CRP research proposal was structured on the assumption that living standards can
be improved by giving individuals the chance to make choices and participate in their
social environment. It did not aim to address all dimensions of living standards. In
selecting living standards areas to be targeted by the CRP weight was given to
constraints on the Commonwealth's and the DSS role in service provision. The focus
of assistance was on those living standards areas that could be integrated with the
delivery of income support, drew on DSS strengths and did not duplicate service
provision in other areas of Government. Among those products and services that
satisfied these criteria were those that improve client material well-being and social
inclusion. Products and services that improve material well being focuses on access to
non-cash income, reduced costs and increased spending power, participation in the
informal labour market and in alternative aspects of the formal labour market. Products
and services that improve social inclusion stressed, as goals, social participation, access
to information, improvement offamily relationships and personal well-being.
Within the context of the CRP an action research approach was adopted to test the
potential of local, community based activities and services for improving living
standards of low income earners. The action research was undertaken in selected
locations: Gympie, Nundah and Chermside in Queensland; Modbury, Salisbury and
Elizabeth in South Australia; and numerous locations throughout Tasmania. In
selecting these locations consideration was given to:
• testing the activities in a variety of urban and rural locations, and across States
• customer need;
• community and staff support
• accessibility and ease of administration .
In October 1994, interested organisations, community, trade union, local government
or private sector organisations. were invited to submit proposals for community based
action research projects CARPs). These proposals were required to identify:
Page 4
•
•
• an innovative product or service to improve the living standards of people on low
incomes.
• how the project would result in improved living standards.
• which living standards would be targeted by the project.
• the methods by which the project organisers proposed to develop, deliver and
• measure the service outcomes.
•
•
•
•
•
•
•
From 335 expressions of interest 245 applications to participate in the CRP as ARPs
were received. Eighty of these projects were selected for funding. Generally projects
were funded for up to $20,000 for approximately 12 months and the funds were mainly
intended to cover data collection and operational costs.
Examples of the type of projects that were funded include:
• co--operatives - including food, garden, furniture and savings and loans~
• information and support networks - including a network of isolated farming
communities and people with disabilities;
• resource sharing networks - including a transport network, accommodation
network and tool libraries
• training and support services.
1.4 Design of the quantitative evaluation
The ARPs were evaluated in a number of ways including case studies and er
quantitative survey. This report provides the results of the quantitative survey
component of the ARP evaluation.
1.4.1 The research design
The evaluation adopted a field experiment design (Figure 1). This involved the
administration of a pre-test questionnaire to project participants at the beginning of
their involvement in their ARP and a post test questionnaire near the end of their
involvement. The questionnaires were also given to a control group of DSS clients
Page 5
•
•
•
•
•
•
•
•
•
•
who were not involved in an ARP to provide a frame of reference against which to
compare changes in living standards of ARP participants.
Figure 1: Research design of the quantitative evaluation
Pre-test Involvement in ARP Post-test
ARP ~ participants
1 ~ Projectchange=X2'"X1
Control ~ group i CS] Controlchange=Y 2'" Y 1
No involvement in
ARP
The logic of this design was that change was to be measured among project
participants at both the pre-test and post test phases. Analysis would measure the
amount of change in living standards from pre to post test phases (projectchange=X2-
Xl). Since measured change could be due to random measurement error or to other
social and economic changes over the study period that were not due to the ARP
projects it was necessary to measure the amount of change over a comparable period
for a comparable population (DSS clients). This change is designated in the above
diagram as Controlchange (Controlchange=Y2-YI). The logic of the analysis that
flows from this design is that participation in the ARP can only be said to have created
change if Projectchange is greater (at a statistically significant level) than
Controlchange. If Projectchange is > 0 but is not greater than Controlchange then
Projectchange will not be attributed to participation in the ARP but is likely to be due
to measurement error or to factors in society or the individuals lives separate from the
project.
1.4.2 The pre and post-tests
The quantitative evaluation is based on a pre-test and a post test survey of project
participants. The purpose of the pre-and post-test surveys was to measure the net
change in living standards of individuals over the term of the project. Project
Page 6
•
•
•
•
•
•
•
•
•
•
participants were asked to complete a pre-test questionnaire at the beginning of their
involvement in their ARP. This questionnaire sought basic socio-demographic
information on the participant and information about their level of living standards. In
addition to the eight core living standards areas described above the survey also sought
basic information about access to health services, housing, community support services
income, employment and other factors which contribute to an individual's overall living
standard.
Towards the end of their involvement with the ARP participants were asked to
complete the post-test questionnaire. This questionnaire asked ~imilar questions to the
pre-test questionnaire and enabled an analysis of the extent to which the participant's
level of living standards had changed over the duration of the project.
As indicated previously (section on research design) the base module questionnaire
was also administered to a sample ofDSS clients who were not involved in' an ARP.
These respondents will be referred to in this report as the control group. The base
module of both the pre and post test questionnaires was administered to the control
group six months apart. The purpose of the control group was to provide a frame of
reference against which to compare the changes among ARP participants and to try to
isolate from the results the effects of any major economic or political factors which
may have had a general impact on the living standards of individuals in the research
projects. Before we could be reasonably confident that projects had an effect on a
living standards area we would need to be able to demonstrate that participants in the
project experienced a greater change in living standards than those in the contrd
group.
Although not part of the quantitative evaluation of the ARPs a qualitative evaluation
was also undertaken. This involved case studies of each project. These case studies
were undertaken to examine links between participant characteristics, project
service/product features, processes and outcomes for participants. The case studies
were designed to enable an assessment of which service types, processes and delivery
combinations produced the best outcomes in each of the 8 living standard areas
targeted by the CRP, and to determine those that impact on the broadest range of
living standards areas. This assessment focused particularly on the way in which
Page 7
•
•
•
•
•
•
•
•
•
•
projects operated and how the services were delivered. These case studies do not
form part of the ARP evaluation reported in this
1.4.3 Administration of the data collection
1.4.3.1 Project participants
The administration of the pre and post test questionnaires to project participants was
the responsibility of the ARP organisers. Although participation in the survey was not
compulsory, ARP organisers were responsible, through a contractual agreement, for
ensuring that a viable sample for testing the living standards outcomes of their project
was achieved.
Project participants were defined as those whose living standards were to be improved
through active involvement in the projects. It excludes those whose involvement was in
a professional or managerial capacity only. For example, in a craft cooperative, which
was organised to produce and market goods, all cooperative members were invited to
complete a questionnaire, irrespective of whether they produced the goods or
participated in the cooperative. However, if the project operated as a direct service, for
example delivering counselling services or training courses, then administrators or
deliverers of these service were not be invited to complete the questionnaire.
An initial target sample of at least 1500 participants was set. Privacy and
confidentiality was ensured by self completion and the use of sealed. envelope
arrangements similar to the ABS Census. Respondents were followed up, and Post-test
questionnaires were distributed by ARP project organisers, between 6 and 12 months
after the commencement of the project.
1.4.3.2 Control group respondents
The field work for the recruitment ofa stratified random sample of 1851 DSS payment
recipients for pre-testing, and the distribution of questionnaires, was conducted by Roy
Morgan Research using a positive random recruitment strategy. The recruitment
strategy involved telephone screening of households in the target regions (i.e. the same
regions as the ARPs were located) to identify a DSS payment recipient willing to
Page 8
•
•
•
•
•
•
•
•
•
participate in the survey. The self completion questionnaires were delivered personally,
using the ABS Census of Population and Housing methodology. At least two
telephone contacts and a personal follow up were undertaken, to maximise response
rates to the pre-test questionnaires.
The sample was stratified in order to gain adequate representation of sub-groups
across gender and age groups. Maher and Smith (1996) provide details regarding the
distribution of the final control group sample across regions.
To maximise responses for the post-test phase, Morgan Research followed up
respondents periodically. Follow-up was undertaken by mailing out a postcard
reminding respondents about the survey. The post-test questionnaires were
administered approximately six months after the pre-test using the same field work
approaches as the pre-test phase. At this post-test stage; respondents who had
participated in an ARP since the pre-test phase were excluded.
1.4.4 The questionnaire
The instrument was developed by the Australian Institute of Family Studies in
cooperation with the CRP Evaluation te~fl1. The instrument was pilot tested with
approximately 80 people from diverse backgrounds and with differing skills and
capacities. Minor changes were made to the instrument after the pilot.
The pre and post test questionnaires consisted of a Base Module (BM) consisting of
socio-demographic questions and questions on each of 14 living standards areas
including the 8 core living standards areas (see below). This base module was
administered to all ARP participants in the survey and to all control group respondents.
In addition, Special Modules (SM) for each of the eight core living standards areas
were developed. Participants in projects targeting particular core living standards were
given appropriate special modules in addition to the base module.
Page 9
•
•
•
•
•
•
•
•
•
•
1.4.5 Living standards areas to be examined
Each ARP was required to target one or more of the eight core living standards areas
that was identified as being relevant to the scope of the CRP. The analysis in this
report focuses on eight core living standards areas which are:
1. F ormallabour market participation
2. Reduced costs
3. Personal well being
4. Social Participation
5. Non cash income
6. Informal labour market participation
7. Family relationships
1.4.6 The questions
To measure these living standards areas the following analysis uses 31 measures of
these areas. Twenty one of these measures which cover all eight living standards areas
were contained in the Base Module (BM) questionnaire. In addition 10 measures from
the Special Modules (SM) were used but since these measures were only used in
projects that were especially targeted to particular living standards areas and were not
used for the control group respondents the number of respondents answering these
special module questions was often quite limited.
Figure 2 lists the measures used to measure each of the eight living standards areas.
Figure 2: Measures for each of the eight living standards areas
Living standards area and items
Formal Labour Market:
Work aspect pre's" (BM)
"FLM1"(SM)
Content of measures
Importance placed on a range of aspects of paid work including the work environment, location of workplace, sense of satisfaction, opportunities for advancement, relationships with co-workers, pay, and job security
Attitudes to working and seeking work in the formal labour market
Page 10
•
•
•
•
•
•
•
•
•
•
Living standards area and items
"FLM2"(SM)
Reduced Costs:
"Self help" (BM)
Access to Infonnation:
"Info satisfaction" (BM)
"AI1" (SM)
Personal Well Being:
"Life satisfaction" (BM)
"Mastery" (BM)
'WB1"(SM)
'WB2"(SM)
Social Participation:
"Sociable"(BM)
"SP1"(SM)
"SP2"(SM)
Non cash Income:
"NCI1" (SM)
"NCI3" (SM)
Q6a
Q6b
Q6c
Q6d
Q6e
Content of measures
The extent of involvement in a range of more innovative money earning activities
How often the respondent had done such things as growing food, making clothing and carrying out own house repairs, because of shortage of money in the past 12 months
Respondent's level of satisfaction with the information available on several aspects of living including job opportunities, community services, where to buy cheap goods, financial management, and education/training
Respondent's 'ratings of their .knowledge of how to find out about job opportunities, low priced goods, legal advice and a range of other aspects of living
An overall measure of satisfaction with a range of aspects of the respondents' life
Respondent's sense of control over their life
Feelings of emotional and physical well being over the previous two weeks
Satisfaction with feelings about self and othe"s perceptions of self and personal opportunities
Degree of attachment to, and interest in, people around respondent
Respondent's level of social participation in neighbourhood
Amount of contact with various community groups and networks
How often respondents received help or used several strategies to help them get by finanCially in the past year
How often the respondents or members of their household had done, a range of things, including car repairs, making toys, repairing household furniture, and renovating their own home in the past year
Make things to swap or trade (not for money)
Grow things to swap or trade (not for money)
Dependence on welfare agencies to obtain goods ("+" = less dependence)
Receive goods (not money) from friends/relatives C'+" = receive goods more often)
Have friends/relatives give you a hand C'+" = receive help more often)
Page 11
•
•
•
•
•
•
•
•
•
•
Living standards area and items Content of measures
Informal Labour Market:
"ILM" (SM)
Q7a
Q7b
Q7c
Q7d
Q7e
Family Relationships:
"FR2" (SM)
Q4a
Q4b
Q4c
Q4d
Q4e
How often respondents did arrange of activities apart from regular work, including participating in collectives, dOing voluntary work, selling or bartering home made goods and doing odd jobs
Works for someone in exchange for goods/services (not cash)
Sells things you have made or grown at local markets/fairs/craft shows
Does volunteer work in a community group/co-op
Does other voluntary work
Has unpaid work experience
The likelihood of respondents seeking emotional support from a range of family members and friends (NOTE - not used because of very low n)
Satisfaction - well being of kids
Satisfaction - relationship with partner
Satisfaction - relationship with kids
Satisfaction - how well kids get along
Satisfaction - partner's relationship with your kids
For the following scales and items a "+" sign indicates a change in a positive direction (or increase) following project participation. For items 6d and 6e interpretation is tricky. See description below for interpretation of change.
A number of these measure are scales. Table 1 below details the items used in these
scales and reports their reliability. Only scales that had a reliability coefficient (ex ~
0.65) were used as scales. Where a ·set of items did not meet this scaling criterion
individual items were used.
Table 1 Alpha coefficients for base module and special module scales used in
analysis of DSS CRP data.
Targeted living standards area
Non-cash income
Formal labour market
Scale
NCI1 (SM)
NCI3 (SM)
"Work aspect preferences" (BM)
FLM1 (SM)
FL M2 (SM)
Items contributing to Alpha scale
NCI1a to NCI1n 0.77
NCI3a to NCI3h 0.80
Q21a to Q21j 0.84
FLM1 a,b,d,e,f,h,j,k,l, 0.70 m,n
FLM2a to FLM2g 0.79
Page 12
•
•
•
•
•
•
•
•
•
•
Targeted living standards Scale Items contributing to Alpha area scale
Informal labour market ILM (SM) ILM1a to ILM1i 0.70
Reduced Costs "Self help" (BM) 05d to 5f 0.65
Access to information "Info satisfaction" (BM) 018a to 018e 0.80
AI1 (SM) AI1a to AI1q 0.93
Personal well being "Life satisfaction" (BM) 019a to 019k 0.87
"Mastery" (BM) 020 b,d,g,h,k 0.69
WB1 (SM) WB1a to WB11 0.92
WB2 (SM) WB2a to WB2d 0.80
Social participation "Sociable"(BM) 020a,c .. e,f,i,j 0.67
SP1 (SM) SP1atoSP1i 0.84
SP2 (SM) SP2a to SP2j 0.73
Family relationships FR2 (SM) FR2a to FR2g 0.76
FR4 (SM) FR4a to FR4e 0.76
BM =base module scale SM =special module scale
1.5 Purpose of this report
The purpose of this report is to evaluate, from a quantitative, survey based perspective
the effectiveness of the ARPs in producing positive change in eight selected living
standards areas. The report will:
1. Provide an appraisal of the value of the quantitative pre-post test approach to
evaluating the impact of CRP participation on living standards.
2. Examine the extent to which change in each of the eight living standards areas was
evident.
3. Assess whether the change in these living standards areas was any different from
the change exhibited among people who did not participate in ARPs.
4. Identify whether the way in which ARP projects were developed and run
contributed to positive living standards change.
5. Ascertain the characteristics of individual project participants that appeared to
promote positive living standards change.
Page 13
•
•
•
•
•
•
•
•
•
•
2. Why so little change?
Taken overall this quantitative evaluation detected less change in living standards than
might have been anticipated by some of those involved in the ARPs or the CRP in
general. (see sections 5 and 6).
The analysis based on mean change took into account both positive and negative
changes in living standards. This indicated that although there were cases where
individuals had positive change scores just as many had negative change scores of a
similar magnitude. The analysis using mean change scores i~dicated that on most
measures living standards declined as much as they increased and that generally there
was little net effect.
The analysis based on simply designating individuals as having changed in a positive
direction or not having changed in a positive direction showed that more individuals in
projects than in controls displayed positive change in living standards. However, care
must be taken with this finding. These positive changes were generally modest and
mask the substantial group of individuals who had negative change scores.
The purpose of the ARPs was to investigate whether these locally delivered
community based projects would lead to an improvement in selected areas of living
standards. It was always recognised that the projects might not be effective or have
limited and specific effects and it was the purpose of the evaluation to determine the
effectiveness. Early analysis of some of the case study material and other qualitative
data suggested that the projects were effective in some respects - especially in
improving social participation and personal well being and participation in the formal
labour market (Smith, Herbert and Roche, 1997).
The findings from this analysis generally did not support this early qualitative analysis
or the hypotheses that lay behind the development of the ARPs. What is to be made of
this?
Page 14
•
•
•
•
•
•
•
•
•
•
There are basically four possible explanations for the failure to detect change in living
standards.
1. The theory was wrong and living standards are not amenable to improvement by
participation in the type of community based groups that were part of this research
project.
2. The design of the study was wrong and did not allow sufficient time to detect living
standards change or the methods of administration were such that it did not
encourage project leaders or participants to approach the quantitative evaluation
with sufficient commitment or care.
3. The measurement of living standards was inadequate. The measures were either
not appropriate for the types of participants, were too standardised for the range of
participants or were insufficiently sensitive to detect living standards change that
occurred.
4. A combination of these factors.
2.1 Theory
The dimensions of living standards that were the subject of the CRP in many cases
represent core dimensions of a person's life. Levels of social participation, the quality
of family relationships and feelings of personal well being as well as participation in the
labour market are complex phenomena that will be the outcome of a complex set of
factors. A person's living standards will be the outcome of factors that include the
psychological make up of the person, their personality, structural factors, their life
stage, the characteristics of other people in their environment disposable income and
the like.
The complexity and the fact that a person's current level ofliving standards will result
from long term factors that are not readily changed will mean that for many people
certain aspects of their living standards will be quite resistant to change. Research into
inequality and disadvantage has documented the resilience of disadvantage and the
importance of the interaction of complex set of factors in sustaining disadvantage.
Page 15
•
•
•
•
•
•
•
•
•
•
Case work practice both in social work and the various therapeutic professions has
also demonstrated the difficulty of producing enduring and substantial change even
with highly personalised and sustained interventions.
Given the resilience and complexity of disadvantage and of living standards it is
ambitious to expect that participation in projects will have a major and general impact
on the lives of participants. To expect this is to underestimate the impact of the
constraints and structures within which most people live their lives and of the lifelong
learning and patterns that people have established. It assumes a highly adaptable and
malleable person that underestimates the constraints· of structure, life history and
personality.
The likelihood of not producing change is reinforced when the nature of some of the
projects is considered. For example, how much change to core areas oflife is likely to
be produced by access to a tool library. Even where such access is aimed at reducing
costs there are so many other areas of expenditure in a person's life that such access,
while useful, has little material impact on this general domain of the cost of living.
Individuals who are struggling to make ends meet will probably still struggle regardless
of whether they have access to a tool library. A limited targeted intervention will not
necessarily be experienced as bringing about a measurable or notable improvement in a
general domain of living standards. Access to some of these community based services
will, no doubt, make people's lives easier in very specific way~ but to expect this to
have a generalised result or even an impact on a targeted living standards area (which
is still general) is optimistic.
In this context we need to be mindful of the level of participation in project groups.
As Table 2 below indicates the level of participation was often very modest and for a
short period. Although about two thirds were involved in projects for six months or
more half the respondents were involved in the project on only an occasional or even
less frequent basis. To expect relatively infrequent group involvement over what is, in
the scheme of things, a relatively brief period of time, to produce real change in core
areas of a person's life is to underestimate the resilience of disadvantage and an
oversimplification of the factors that contribute to a person's living standards.
Page 16
•
•
•
•
•
•
•
•
•
•
Table 2 Level and duration of participation in projects
Level % Duration %
Frequent 49 More than one year 18
Occasional 26 6 months to a year 53
Infrequent 14 5 to 6 months 18
Once only 11 1-4 months 9
Less than a month 2
N 447 N 433
Note: there were no responses for 148 participants regarding level of participation and
from 162 regarding duration of participation. These data were provided by project
leaders for each responding participant.
The notion that living standards, low income and disadvantage are the outcome of
multiple causes must raise questions about how likely it is that intervening on one
aspect ofa person's life or a single intervention (participation in a project) will make a
material difference to core aspects of a person's life. In other words the concept of
multiple causation forces us to seriously question whether a single intervention can be
reasonably expected to have an observable impact.
Furthermore, given that living standards and disadvantage are complex, multi-causal
phenomena it is highly probable that many causal factors will, in fact, be the result of
an interaction of several causal variables. That is, it is only when a number of factors
are present together and interact with one another that they have an impact.
Accordingly, it is probable that effective interventions will require multiple and
simultaneous interventions so that the interaction of these will produce effective
change (e.g. increase disposable income as well as increase opportunities for social
participation as well as assist with communication skills as well as help with family
relationships).
Page 17
•
•
•
•
•
•
•
•
•
•
2.2 Design and method
While it may have been overly optimistic to expect the ARPs to produce measurable or
sustainable changes in various dimensions of living standards the design and methods
employed in the quantitative evaluation are likely to have exacerbated the difficulty of
detecting living standards change.
2.2.1 Design
2.2.1.1 Use of the experimental design
The use of a field based experimental design with the pre and post test and the control
was an excellent design for measuring the impact of an intervention. There were,
however, a number of aspects relating to the implementation of this design that were
less than optimal.
Practical constraints and funding limitations meant that the time period allowed for the
projects to have an impact was limited effectively to between 6 to 12 months. As
previously argued, given the nature of living standards, the resilience of disadvantage
and the fact that projects were but one of many influences on a person's life this is a
very short time period in which to bring about change in core areas of a person's life.
No doubt there will be individual cases where people change over this period but to
expect generalised change at a project level is optimistic.
While the experimental model is highly desirable it was not suited to all projects .....
Given that it required a pre and post test there was always going to be a problem with
distortions due to participants using the project on only a limited number of occasions
or on only one occasion. It was quite inappropriate for services where single uses by
the community may be common. The questionnaires were long and many project
leaders noted the difficulty of asking a person who had simply come into the project
once for a piece of advice (in the case of CABs) or to borrow a piece of equipment (as
in a tool library) to complete a detailed, lengthy and, what would appear to participants
as, an irrelevant questionnaire given the purpose of their use of the service. One co
ordinator commented that:
Page 18
•
•
•
•
•
•
•
•
•
•
'It's embarrassing asking people to spend all that time filling in a pre-test when
they only had an inquiry that took a few minutes to deal with'
To do this twice (pre and post test) was seen, by project leaders and participants to be
burdensome, intrusive and irrelevant to their needs. One project leader commented
that:
'The service was not suited to this type of process. Many users would have
used the service on a one off contact of a few minutes ... A questionnaire as
long and as intrusive as the one used would be inappropriate [to give to these
users]. Over time, a number of users became repeat users but there was no way
of predicting this on initial contact. By the time the relationship with the
service had reached a level where a pre-test could be appropriately explained
and administered the "pre-test" quality would have been lost.'
Another commented that:
'Due to the transient nature of people involved in the project, difficulties
surrounded securing contact details of respondents to the pre-test questionnaire
so they could be followed up to complete the post-test questionnaire.'
The use of a control group was also both desirable and informative for the analysis and
its interpretation. Reasonable efforts were made to select a c.ontrol group that was
comparable in certain respects at least to the project participants. They came from the
same communities and were low income people in receipt of DSS benefits. In a true
experimental design it is necessary that the control group and the experimental group
be the identical as possible at the beginning of the process. This is normally achieved
by random assignment of people to the control and experimental groups or by
matching. In this case limited matching was used but the fact that this was limited
means that the control and experimental groups may not have been strictly comparable
to start with. Participation in the projects will have been affected by self selection and
the fact that the projects were targeted at particular types of users.
The fact that the control and experimental groups were probably not strictly
comparable may have led to the masking of the impact of projects. If the types of
Page 19
•
•
•
•
•
•
•
•
•
•
people who participated in the projects made them more resistant to change then the
comparisons between control group change and project participant change will have
been compromised to an unknown extent.
2.2.1.2 Focus and hypothesis development
The questionnaire and the analysis would have benefited from a more systematic
development of research hypotheses in the early stages of the evaluation. This would
have led to a more focussed and tighter questionnaire. The survey instruments were
designed relatively early in the development of the ARPs. They were developed at a
time when the theorising in relation to the factors that might contribute to the
effectiveness of projects was still under development. As a result insufficient attention
was given to the hypotheses that were being tested. In particular insufficient attention
had been given at that stage to which aspects of service delivery, which aspects of
project activity and which participant characteristics might affect living standards
outcomes. This meant that there was no systematic measurement of these service
delivery aspects from the perspective of participants. In certain respects this meant
that some information was not collected that should have been but it also meant that
too much unnecessary information was collected.
Partly because of a lack of staff continuity within DSS it did not prove possible to
remedy these difficulties in the development of the post-test q~estionnaire. Although
scheduling did not make it possible it would have been desirable to have had some of
the tentative findings from the qualitative study available before the finalisation of the
post test questionnaire. If this had been possible measures of project style could have
been built into that questionnaire.
2.2.1.3 Sponsorship of the study
It was evident from the comments of some project leaders that some participants were
suspicious about the confidentiality of the information they provided and the uses to
which it would be put. Although participants were assured of the confidentiality of the
process and despite DSS being extremely careful to respect this confidentiality this
suspicion may well have affected response rates and contaminated the responses.
Page 20
•
•
•
•
•
•
•
•
•
•
Project leaders commented that
'There were some comments by youth which suggested concern (despite
assurances) about confidentiality of the information they provided via the
questionnaires'
'Despite efforts by the Department to assure project participants of
anonymity ... there was still a degree of suspicion'
'There was resistance to 'form filling' amongst participants as this had
unpleasant associations with former crisis periods requiring approach to
government agencies (including DSS).'
'Many of the people using the centre were DSS recipients and perhaps
unwilling to provide information to a department responsible for providing
their entitlement or benefit. '
It is difficult, however, to envisage how this problem could have been alleviated.
Assurances were given and DSS did undertake the data collection and analysis at arms
length.
2.2.2 Method
2.2.2.1 Standardised questionnaire
The quantitative evaluation relied on a standardised questionnaire consisting almost
entirely of closed choice questions and was administered to project participants
regardless of project or participant characteristics. Given the wide diversity of both
project types, activities and participants this was probably an ill-advised strategy. The
highly specific nature of the projects and services meant that the way in which living
standards improvement would be manifested would vary from one project to another.
The use of 'one size fits all' indicators is likely to have been insensitive to important
but subtle changes in the lives of some individuals.
Page 21
•
•
•
•
•
•
•
•
•
•
A related issue was the method by which the questionnaire was constructed. It would
have benefited from input from project leaders to adapt questions and measures for the
project (see below). As it was there was no input from the particular projects.
2.2.2.2 Questionnaire administration
The way in which questionnaires were administered produced another set of problems
which affected response rates and probably affected data quality. Questionnaires were
administered by project leaders at the local level. Some leaders used creative strategies
to gain co-operation. One gained co-operation by using completion as 'payment' for
furniture. Despite the committed efforts of some leaders others evidently regarded the
exercise as an imposition, as irrelevant and displayed limited commitment or
understanding of the exercise. This was no doubt compounded by the length and
perceived inappropriateness of the questionnaire in some projects.
The whole process of the quantitative evaluation was too remote from project
participants and leaders and they were 'not brought on board'. Far more work and
opportunity was needed to convince projects of the value of the evaluation and the
need for it to be carefully completed.
The failure to achieve a higher level of understanding and commitment led to some
data contamination. There was evidence of leaders completing the questionnaires
themselves on 'behalf' of participants.
A greater problem (as far as it can be assessed) was created by leaving the allocation
and recording of ID numbers with project leaders. This led to considerable
misallocation, mismatching and loss of participant ID numbers. In a project measuring
change among individuals it is critical that there is accurate matching of IDs and that
change is not simply a function of mismatched questionnaires.
At the pre-test phase a group of 40 respondents were identified who shared and ID
number with at least one other person within their project. By the post test phase these
duplicates had still not been sorted out so they were eliminated from the sample for
calculation of change scores and for subsequent analyses.
Page 22
•
•
•
•
•
•
•
•
•
•
There were also anomalies between pre and post-test data. One respondent had been
reported as a 16 year old male on unemployment benefits at pre test, but at post-test
was reported as having two children (one aged 1 year and one aged 6 years), and being
in receipt of a sole parent pension. The handwriting style on the two questionnaires
was also obviously different. It is difficult to assess the extent of this problem, which
appeared to arise from some confusion at the project level about allocation of correct
identification numbers for individuals at pre and post test..
A further consequence of leaving all the administration of the questionnaires to project
leaders was that it did not enable the evaluators to obtain feedback from the project
sites. Had this been possible or had the evaluators had the advantage of some site
visits it is likely that a more refined instrument could have been developed.
2.2.2.3 Questionnaire structure
The questionnaire consisted of the base module which contained measures of 14 living
standards areas including the 8 core living standards areas and special modules which
were administered only to projects specifically targeting this living standards area and
not to the control group. The effect of the selective administration of the special
modules was twofold.
It led to a small number of respondents for many of the modules (especially after the
high drop out at the post-test) which in turn made it difficult to achieve statistical
significance in the analysis.
In addition, the absence of measures for the control group on the special module
measures meant that it was not possible to compare patterns of change among project
participants with a control group to see if the pattern for change among project
participants was any different on these measures from non participants.
2.2.3 Sample size
The number of cases available for analysis was less than desirable and led to less than
optimal data analysis strategies. Numbers at the pre-test stage were lower than initially
planned. This led to a decision not to do any analysis at the specific project level but
Page 23
•
•
•
•
•
•
•
•
•
•
to group projects into broad categories which did not necessarily share important
attributes. Combining essentially dissimilar projects together could have the effect of
masking true inter-project differences in living standards change.
The problem of a smaller than desirable sample size was compounded by a high drop
out rate between the pre and post tests with just over half the pre-test sample
providing useable post-test questionnaires. The high drop out rate was due to:
• drop out from the project (moved, project not appropriate.etc)
• transient use of project
• resistance from project participants to complete a long questionnaire twice.
• difficulties some project leaders had in tracking participants
• poor recording of ID numbers which meant that 72 post test questionnaires were
unusable. In a further 19 cases project leaders used the pre-test questionnaire at
the post-test phase
The effect of small numbers was threefold:
1. It skewed the sample which may in turn led to some unrepresentative results. The
drop out rate was especially high among young, unmarried, Australian-born males.
2. Made it harder to achieve statistical significance.
3. Necessitated that categories be combined in the analysis to overcome difficulties
caused by small numbers. One effect of combining categories (eg. project type plus
the project characteristics) is that unlike groups or people can be treated in the
analysis as though they are the same. One effect of this can be to lose fine but
important distinctions that can in turn mask meaningful differences in living
standards change of different projects and service delivery styles.
Page 24
•
•
•
•
•
•
•
•
•
•
2.3 Measures
2.3.1 Standardised measures
Having reliable, valid and sensitive measures of living standards is fundamental to
measuring living standards change. It has already been suggested that the use of
standardised measures for all participants and groups may have seriously blunted the
sensitivity of the indicators of living standards. A number of project leaders alluded to
this problem and the shortcomings of standardised measures commenting that many
questions were irrelevant to their project, that they seemed more appropriate to urban
participants. Accordingly, they anticipated that the pre-post test questionnaire would
not be successful in identifying change among their participants.
The particular indicators of living standards will depend in large measure on the age of
the person, their stage of life, their family circumstances, personality, ethnicity gender
and the like. For example, for one person in particular circumstances their capacity to
have ready access to affordable, flexible child care in which they had confidence could
be a critical measure of well being. For another person in different circumstances it
would be irrelevant.
Apart form the difficulty of deve~oping living standards measures appropriate to all
respondents it is also evident that when measuring the impact of a particular project on
living standards that measures relevant to the very specific goals, activities and nature
of that group need to be developed. For example, a project might be targeted
specifically at fostering co-operative efforts to grow vegetables for trading (non-cash
income). If most of the measures of improvement in the non cash income living
standards area relate to something other than this specific activity it is unlikely that we
will detect much change for participants of this group (except perhaps on a very
specific question relating to growing things to swap or trade). By the same token,
designing general measures of living standards change makes it probable that real, but
project specific, living standards change will have remained undetected.
This suggests that it would have been preferable (but not necessarily practicable) to
have developed project specific indicators for each living standards area. This would
Page 25
•
•
•
•
•
•
•
•
•
•
have been a complex task but it may have been desirable to have done this and to have
done so in conjunction with project leaders.
2.3.2 Questionnaire length
In pilot testing a sample of 80 DSS clients took approximately 35 minutes to complete
the questionnaire. It was considered at the time that this was at the upper time limit
that would be appropriate to ask of project participants to spend completing the
questionnaire. Comments from project leaders indicated that .in many cases project
participants took considerably longer completing the questionnaire and required help in
doing so. Not only did this affect the response rate, some project leaders felt that it
compromised the quality of responses. One commented that
Many of the respondents complained about the length. of the questionnaire, the
time it took to complete them was always in excess of 20 minutes. Some
respondents took theirs home to complete when they realised it would take
some time and very few returned them ... '
Questionnaire length became a particular issue because of the ambivalence about the
pre-test and post-test among both some project leaders and participants. There clearly
was a need to shorten the questionnaire and to make it more focussed. The earlier
comments about designing the questionnaire to test specific hypotheses would have
assisted in this as would have assisted designing measures that focussed on specific
aspects of the particular projects.
2.3.3 Language
Despite careful pilot testing some project participants encountered difficulties with
language. In some cases this arose from literacy problems with some participants and
a non English speaking background in other cases. Understandably, project leaders did
not have the time to provide extensive assistance with questionnaire completion.
Another probable reason for difficulties with the language of the questionnaire
stemmed from its standardisation. The more standardised the questionnaire the
broader the range of particular circumstances that questions have to cater for.
Page 26
•
•
•
•
•
•
•
•
•
•
Adopting a higher level of generality can lead to more abstract terms and concepts
which will cause difficulty for respondents who are more comfortable operating at a
more concrete level.
2.3.4 Layout/method of administration
Although the questionnaire employed normal questionnaire layout and avoided a
complex 'skip to question X' format it nevertheless proved difficult for some
respondents. The questionnaire was self administered and would have involved a
longer reading and writing task than that with which some project participants would
be familiar. The result of this was undoubtedly a lower than desirable response rate,
some bias in the response rate and a higher level of item non response than expected.
2.3.5 Measures of living standards
The CRP and the ARP evaluation was an ambitious, exploratory research project that
made use of some relatively new and complex concepts in the conceptualisation of
living standards. Because of the innovative nature of the study new measures were
required for complex new concepts. The development of new measures for complex
concepts is always a long term process where measures are refined over time in a
number of different contexts. The measures used here were in some cases a 'first cut'
at measuring these concepts. Because of this there is bound to ~e room for improving
and refining the measures of living standards. It is· entirely possible that lack of
refinement will have contributed to difficulties in measuring living standards changtr
that may have occurred among project participants.
As well as the measures measuring what they are meant to measure (validity) they need
to be reliable. That is, a respondent should answer a question on two occasions in the
same way (unless of course there has been genuine change between the two
occasions). Reliability refers to the stability of a measurement. If a measure elicits
different responses on two occasions simply because of random factors or because of .
the way in which the question is constructed (e.g. ambiguity of words, confusing
structure, etc) it will be unreliable. In a study of change it is especially important that
the measures are reliable (i.e. stable) since it is critical that any observed change
Page 27
•
•
.' •
•
•
•
•
•
•
reflects real change rather than measurement error (the unreliability of the
instruments).
Unfortunately there is some evidence of instrument unreliability in this study and we
therefore cannot be entirely confident that the change we have measured reflects real
change or that the absence of change reflects a real absence of change. An
examination of the distribution of change scores, in most cases, forms a pattern that is
close to a normal distribution. That is, most change scores cluster around the zero
point and 'fan out' evenly each side of the zero point. The riumber of respondents
displaying slight positive change is matched by the number displaying slight negative
change; the number displaying moderate positive change is less but is matched by the
number displaying moderate negative change; and the number s displaying high
positive change is less again but is matched by the number displaying high levels of
negative change. With change scores this pattern is consistent with a pattern that
would result from unreliable instruments .
In conclusion, the novelty of the measurement instruments used in this study means
that they were liable to encounter shortcomings in terms of reliability and validity.
Furthermore, the resilience of disadvantage and the multi-causal foundations of living
standards together with the relatively short time period over which change was to be
measured means that very sensitive measurement instruments were required. The
exploratory nature of the study and the novelty of the measures makes it likely that the
measures and the design will only been able to detect some of the change - and the
grossest changes at that - in living standards among project participants.
2.4 Why the difference from the qualitative research?
The results from the quantitative analysis diverge from those in the early qualitative
analysis. The early qualitative analysis noted improvements in social participation,
personal well being and participation in the formal labour market. The quantitative
analysis found little to support these early observations. The. quantitative analysis
generally found modest levels of change that was limited to particular living standards
areas. The most notable areas in which there appeared to be a positive impact on
living standards was in the tangible and focussed outcomes of aspects of generating
Page 28
•
•
•
•
•
•
•
•
•
•
non cash income and participating in the informal labour market. Projects that
targeted relatively tangible outcomes in these living standards areas appeared to meet
with some success.
The different findings may reflect an number of factors and they certainly point to the I
benefits of a triangulated research methodology. The different pattern of results may
be due to:
1. The use of different indicators in the different studies. The pre post-test survey
used quite specific measures of concepts such as social participation and personal
well being. The survey is therefore necessarily restricted to measuring these
concepts as operationaised by these specific measures. The qualitative research is
not confined to these pre-established measures and can be responsive to other cues
that indicate change in these areas. To the extent that the qualitative and
quantitative studies used different indicators of concepts it is entirely conceivable
that they will find different patterns.
2. The qualitative studies were less bound by a standardised procedure or instrument.
They may therefore have been be more sensitive to the context of the particular
project and the nuances and meanings provided by that context.
3. The pre and post-test survey obtained the data directly frQm project participants
without any filtering via the perceptions of the project leaders or other project
'staff' .
4. Not only might project staff mis-perceive the effects of the project on the
participants they may also have an interest in presenting the project as being more
successful than it may in fact have been. This is not to imply deliberate
misrepresentation but is simply to acknowledge that the generation of data is rarely
free from the interests of those collecting the data or those from whom it is
collected.
5. There is always a danger in qualitative research that the experiences of particular
individuals are given undue weight in arriving at overall assessments. In the case of
projects it is conceivable that success stories with individual participants stand out
Page 29
•
•
•
•
•
•
•
•
•
•
and lead assessors to overgeneralise from a successful case at the expense of cases
where there were less successful outcomes.
6. The qualitative research had no comparison group against which to measure
whether change among participants was any greater than among non participants.
The pre-post test analysis indicated positive change among project participants but
in many cases this was no more than among the control group respondents. The
change observed in some groups in the qualitative research may have been nothing
more than the level of change that would have occurred even without participation
in the projects.
2.5 Methodological recommendations
In our view the field experimental design of the pre post-test survey was an
appropriate way of systematically measuring the effectiveness of most projects and
should be used again for similar evaluations. However a different strategy would need
to be employed for those projects where usage is transitory and one-off rather than
sustained use is typical.
While this was an appropriate design the ARP quantitative evaluation had a number of
shortcomings which makes it difficult to ascertain whether the modest extent of living
standards change was due to the ineffectiveness of the projects or to an inability of the
methodology to detect change that actually took place ..
For the reasons outlined earlier an improved methodology would require:
• A considerably longer time between pre and post-tests.
• A much shorter instrument.
• This instrument might deal only with living standards areas at which the project
was targeted.
• More focussed hypotheses.
Page 30
•
•
•
•
•
•
•
•
•
•
• More time spent with project leaders and participants to ensure full understanding
and co-operation
• Greater opportunities for project leaders to have significant input into designing
questions for their particular project.
•
•
Allocating and controlling ID numbers centrally rather than at project level by
project co-ordinators.
Considerably less reliance on standardised questions across projects and greater
use of project specific questions (or at least modules).
• Less reliance on closed questions and more use of open ended questions.
• Abandoning the special modules.
In summary, an improved methodology would generally retain the experimental design
used in this study. However, it would develop project specific questionnaires to
measure living standards change within the context of the project itself, the type of
participants and its specific goals. This questionnaire would be developed in a
consultative mode with project leaders who would need to have greater involvement
and greater ownership of the process. Ideally the questionnaire would be relatively
short and provide either open ended questions or very speCific questions directly
relevant to that project. Participants would need to see much more clearly how the
questions are relevant to the project in which they are participating.
The remainder of this report provides a description of the main findings relating to the
extent to which living standards changed between the pre and post tests. The results
elaborate on the core findings outlined already.
Page 31
•
•
•
•
•
•
•
•
•
•
3. Method of analysis
The analysis will consist of three parts.
1. A description of the overall level of change on each of the living standards measures
and a comparison of this change with that among the control group respondents.
2. A comparison of the extent of change in projects with particular characteristics
(e.g. type of project activity, management style, project flexibility etc).
3. Change among individuals with particular personal 'and social characteristics (e.g.
age, gender, marital status etc).
3.1 Project characteristic analysis
3.1.1 Computing change scores
Throughout the analysis the first step was to determine whether individual participants
showed any change on the various measures of living standards. Given the pre-test
post-test design change scores were computed for each participant and these change
scores were designed to reflect the amount and direction of change of each individual
on each living standards measure. Because different measures had varying numbers of
categories we adopted a method of standardising change scores across all measures.
This was achieved by expressing the change score as a percentage of the possible
amount of change on that scale or variable.
For example, on the Social Participation scale respondents could achieve a score of
anywhere between 9 and 45. If Person A scored 15 at the pre-test and 20 at the post
test their raw change score was +5. Given that the maximum possible change on the
scale was 36 (45-9) person A's change score of 5 represents a +14% change (i.e.
5/36 * 100/1). Conversely, person B scored 16 at the pre-test and 10 at the post-test
yielding a raw change score of -6.' This represents a percentage change score of -17%.
Page 32
•
•
•
•
•
•
•
•
•
•
3.1.2· Group level change: the use of means
The analysis of change in the section project characteristics will compare the mean
change on a given living standards measure for a given category of project.
Differences in the mean change in different groupings of projects will be explored to
help assess whether, on average, projects with particular characteristics change more
or less than projects without those characteristics or more or less than the control
group.
As a hypothetical example we might wish to examine· changes in social participation
and we may wish to see whether projects that specially targeted social participation
produced more improvement in social participation than projects which did not
specifically target social participation. We would calculate the mean change for all
individuals in projects involved in projects targeting social participation (e.g. on
average social participation scores were 10% higher at the post test compared to the
pre test phase). We would also calculate the mean change in social participation
scores for those in other project groups (say they were 5% higher) and also for the
control group respondents (say they were the same, on average, at the post test as the
pre test phase). Statistical analysis would then be performed to estimate whether these
differences in the means (10% change cif 5% change cif 0% change) is likely to be due
to chance or represent 'real' change.
Relying on mean change is the most appropriate way ·of summarising the amount of
change at a group level. Of course it can mask the range of direction and magnitude oL
change of individuals.
3.1.3 The impact of project characteristics
The purpose of section 4 and 6 of the report is to evaluate the impact of the way
projects were structured and organised on the eight living standards areas .. The impact
of nine project characteristics will be examined:
1. The success of the project overall as assessed by the qualitative research
2. Did the project target the particular living standards area?
Page 33
•
•
•
•
•
•
•
•
•
•
3. Whether the project had first order outcomes and created second order
opportunities.
4. Locally assessed needs: The extent to which the development of the project was
based on a proper assessment of the needs in the community in which it operated.
5. The service delivery orientation of the group.
6. Origin of the group: did it build on an existing core group?
7. The activity range of the group: was it a single activity or a multiple activity
group?
8. The adaptability of the group: its capacity to adapt to the needs of users.
9. The project category.
3.1.4 Which groups will be examined?
Not all projects were successful. With the exception of the analysis designed to
explore whether the success of the group was associated with living standards change
the analysis focuses on groups classified as very successful or moderately successful.
This enables us to ask the question 'Do the characteristics of su~cessful groups have an
impact on the living standards of project participants?' It was inappropriate to explore
whether groups that failed provided a helpful way of improving the living standards o!.
participants.
In addition, on the advice of the project director the CIN projects and the Mininet
projects were not included in these analyses. The number of projects and participants
from each category of project is provided in Table 3
3.1.5 Project characteristics
The project director provided the AIFS with a classification of each project in terms of
the project characteristics that relate to the way in which ARP projects was developed
and run. The classification of each project on each of these aspects of project style was
Page 34
•
•
•
•
•
•
•
•
•
•
based on the qualitative evaluations of the projects also underta~en by the DSS. These
characteristics and their categories are:
1. Project success. Projects were classified as:
• Highly successful (33 projects and 384 individuals)
.• Moderately successful (16 projects and 147 individuals)
• Failure (9 projects and 64 individuals)
2. Targeted group: When projects were established they specified which of the eight
living standards areas they were designed to improve. On the basis of this each
project was classified for each living standards area as:
• Targeted
• Non targeted
3. First order outcomes and second order opportunities
• medium first order and limited second order ( 64 respondents)
• Medium first order and medium second order (119 respondents)
• significant first order and medium second order (32 respondents)
• medium first order and significant second order (143 respondents)
• medium first order only (109 respondents)
• significant first order and limited second order (4 respondents)·
. • limited first order and medium second order (1 respondent)·
• significant first order and significant second order (18 respondents)·
• limited first order only (11 respondents)·
"'Because of the very small numbers those with this combination of outcomes were not
included in the analysis for this variable
Page 35
•
•
•
•
•
•
•
•
•
•
4. Locally assessed needs: The extent to which the development of the project was
based on a proper assessment of the needs in the community in which it operated.
Six categories were used:
• Needs assessed (4 projects and 74 individuals)
• Needs expressed (4 projects and 62 individuals)
• Needs guessed (20 projects and 165 individuals)
• Needs assessed/expressed (15 projects and 150 individuals)
• Needs assessed/guessed (7 projects and 61 individuals)
• Needs guessed/assessed (8 projects and 83 individuals) .
5. The service delivery orientation of the group was classified as:
• Producer/participant orientation (23 projects and 234 individuals)
• Consumer/client orientation (22 projects and 206 individuals)
• Mixed orientation (23 projects and 155 individuals)
6. Origin of the group. Projects were classified as building on:
• an existing provider group (36 projects and 302 individuals)
• an existing user group (1 projects and 23 individuals)
• an existing provider and user group (16 projects and 220 individuals)
• no existing group (5 projects and 50 individuals)
7. The activity range of the group. Projects were classified as having the following
activity ranges
• single activity (12 projects and 104 individuals)
Page 36
•
•
•
•
•
•
•
•
•
•
•. two activities (26 projects and 254 individuals)
• . Three or more activities (20 projects and 237 individuals)
S. The adaptability of the group. Projects were classified as
• Flexible (25 projects and 294 individuals)
• Having some flexibility (23 projects and 209 individuals)
• Inflexible (10 projects and SO individuals)
9. The project category. Projects were classified into one of five project categories
based on the nature of their activity
• Community education and development (30 projects, 352 respondents)
• Collectives and cooperatives (14 projects, 13Srespondents)
• LETS (4 projects, 57 respondents) !i
• Support networks (5 projects, 26 respondents)
• Individual case management (5 projects, 22 respondents)
3.2 Individual characteristics analysis
This part of the analysis (section 7) will explore whether some types of individuals are
more likely than others to experience positive change in living standards. Rather than
relying on mean change individuals will simply be classified according to whether or
not, on any particular measure, their standard of livi~g improved. We will not be
measuring the amount of change. However, to avoid counting trivial change which
could easily be due to measurement error, we will only classify as'improvers' those
individuals whose score on a particular variable changed by at least half a standard
deviation unit on that measure.
Page 37
•
•
•
•
•
•
•
•
•
•
The impact of the following individual and personal characteristics of project
participants was examined
1. Educational level
• pnmary
• secondary
• trade
• TAPE
• tertiary
2. Family Income (fortnightly)
• $<192
• $193-308
• $309-460
• $461-616
• $617-770
• $771-962
• >$962
3. Home ownership
• purchaser
• rent er
• neither owned nor rented by self (e.g. lives with parents)
4. Ethnicity
• Australian born
• other English speaking background
• non English Speaking Background
5. Employment status
• full time
• part time
• job seeker
• not working & not looking for work
6. Age
• In years
Page 38
•
•
•
•
•
•
•
•
•
7. Gender
• male
• female
8. Marital status
• never married
• married/de facto
• separated/divorced
9. Level of involvement in the project
• frequent
• occasional
• infrequent
• once only
3.3 Significance levels
In the analysis of the impact of project characteristics and individual characteristics
tests of significance will be used. The purpose to which these will be put needs to be
clear as they are frequently misunderstood. To say that a relationship or a pattern is
statistically significant is simply to say that the differences between groups (whether
they be groups or individuals) are unlikely to be due to chance or sampling error. To
say that a relationship or difference is statistically significant is to say that it is likely to
reflect a real difference rather than being the result of statistical chance. To say that a
relationship is significant at the 0.05 level is simply to say that there is only a 5%
probability that the relationship is caused by chance r~ther than reflecting something
real. In this report a significance level of<0.05 is signified with .; a level of<O.Ol with
•• ; and a level of <0.001 with .... The lower the significance level the more likely
the differences are to reflect real differences rather than sampling error.
To say that a relationship or a difference is statistically significant however, is to say
nothing about either the size or importance of the relationship. Statistically significant
patterns can be both small and unimportant.
Page 39
•
•
•
•
•
•
•
•
•
•
4. The sample
The sample in this analysis consists of two sets of respondents:
1. Control group participants. This is a sample of 1276 people who did not participate
in any ARP but completed the base module questionnaire on two occasions six
months apart. This sample was recruited by Roy Morgan Research who
administered the questionnaire.
2. Participants of ARP projects described above who completed both the pre-test and
post-test questionnaires. Although 1105 completed the pre-test questionnaire there
were only 595 useable, completed post test questionnaires. In most of the analysis
we have excluded participants in projects that the qualitative research evaluated as
having failed. In many of these cases we received few useable questionnaires at the
post test phase.
Table 3 provides details of the response rates according to project category.
Table 3: Project and respondent numbers and retention rates
Project category N of Projects N of projects N of N of N of Retention in Category returning Pre projects Participants Participants rate
test data returning in Pre test in Post test Post test Category Category
data
1. Community 33 32 30 599 352 59% education & development
2. Collectives & 20 18 14 284 138 49% co-operatives
3 .. LETS 5 4 4 109 57 52%
4. Support 5 5 5 52 26 50% networks
5. Individual 5 5 5 61 22 36% case management services
Total 68 64 58 1105 595 54%
Page 40
• 5. Overall change: project participants and control group compared
Table 4 shows that, overall, the living standards change of project participants is both
• very slight and does not differ from the level or direction of change in the control
group.
Table 4: Mean change on 31 living standards measures: a comparison of project
• participants and control group respondents
Item/Scale Controls All project Statistical
participants significance
Formal Labour Market
• "Work preferences" + 2.1 +1.9 ns
"FLM1" Attitude to seeking work N/A +1.4
"FLM2" Money earning activities N/A +3.4
• Reduced Costs
"Self help" - 4.6 -2.5 ns
Access to Information
"Information satisfaction" - 2.1 -.4 ns • "AI1" N/A +2.6
Personal Well Being
"Life satisfaction" - 0.1 +.7 ns
• "Mastery" + 0.3 +.3 Il£.
"WB 1" Emotional/physical w/b N/A +2.7
"WB2" Satisfaction with self & personal N/A +4.8
opportunities
• Social Participation
"Sociability" + 0.3 -.26 ns
"SP1" Participation in neighbourhood N/A +1.0
• "SP2" Amount of contact - community groups N/A +2.1
• Page 41
• Item/Scale Controls All project Statistical
participants significance
Non-Cash Income
• q6a Make things to swapltrade + 0.6 +2.7 ns
q6b Grow things to swap/trade - 0.1 -2.8 -q6c Less need for welfare handouts + 0.5 +1.0 ns
• q6d Receive goods from friends/relatives - 3.0 -1.9 ns
q6e Receive help from friends/relatives - 3.0 -1.1 ns
"NCI1" Frequency strategies used to cope N/A -0.1
financially
• "NCI3" Own repairs, maintenance N/A -5.8
Informal Labour Market
q7a Work in exchange for goods/services + 0.5 +1.6 ns
• q7b Sell made/grown goods + 0.2 +2.7 ns
q7c Volunteer, Community group/Co-op - 1.3 -1.3 ns
q7d Other voluntary work - 0.3 -1.9 ns
q7e Unpaid work experience + 0.9 +1.8 ns
• "ILM" Participation in non-work activities N/A +2.3
Family Relationships -Satisfaction
q4a Well being of children - 0.1 -1.6 ns
• q4b Rlship with partner -1.5 -1.2 ns
q4c Rlship with children - 1.3 -1.3 ns
q4d How well children get along - 1.7 -.3 ns
q4e Partner's r/ship with children - 1.0 -1.7 ns • ns = no statistical significant differences between control group respondents and project participants;
- = differences are statistically significant at the .01 level (Le. p<.01).
The figures in Table 4 show that on most measures respondents change less than 3 per
• cent on the possible amount they could change. They also show that for the project
participants the change was negative on 14 measures. However, these negative
changes, like the positive changes are so small that they can be regarded as
• Page 42
•
•
•
•
•
•
•
•
•
•
representing no change at all. Table 4 also shows that on only one measure did project
participants change more than the control group and in this case they did worse than
the control group.
However, combining all the project participants regardless of differences between
projects and regardless of the characteristics of individuals may well mask living
standards change in particular types of projects and for particular types of participants.
The remainder of the analysis in this report focuses on:
• The extent to which different types of projects have different outcomes in terms
of living standards change.
• The extent to which some types of project participants display living standards
change.
6. The impact of project characteristics on living standards change
Early analysis of the qualitative evaluation of the projects suggested a number of
characteristics of projects that may be associated with improvement in living standards
(Smith, Herbert & Roche, 1997) This qualitative analysis suggested that projects will
be more successful when the project has particular features including:
• a clearly identified local need to be addressed.
• flexible approaches to changing target group need over the life of the initiative.
• a range of concrete activities with observable outcomes.
• the existence of a 'core human resource base' .
• transparent and participative management structures.
Page 43
•
•
•
•
•
•
•
•
•
•
It is on the basis of these findings that many of the hypotheses below are based.
6.1 'Successful project?'
6.1.1 Hypothesis:
The assessment of a group as successful or unsuccessful will be based on a wide
variety of criteria some of which will be captured by the range of group characteristics
to be examined below. However, given that the purpose of the projects was to
improve specified living standards of project participants we· would anticipate that
groups classified as successful will produce greater positive change in living
standards than either the control group or the groups classified as failures.
6.1.2 Results
Table 5: Living standards change according to the rated success of the project
Group success Statistical significance
Item/Scale Controls High Moderate Failure sig of F K-W
Fonnal Labour Market
"Work preferences" + 2.1 +1.5 +4.0 -0.8 ns
"FLM1" Attitude to seeking work N/A -0.6 -1.3 -5.4 ns
"FLM2" Money earning activities N/A +4.6 +0.9 +1.4 ns
Reduced Costs
"Self help" - 4.6 -2.8 -2.6 -1.0 ns
Access to Infonnation
"Information satisfaction" - 2.1 -1.8 -0.8 +8.9 -.
"AI1" N/A +1.6 +0.4 +13.5 -Personal Well Being
"Life satisfaction" - 0.1 +0.6 -0.4 +3.4 ns
"Mastery" + 0.3 +0.4 -0.5 +1.7 ns
"WB1" Emotional/physical N/A +2.6 -3.5 +6.9 ns wellbeing
"WB2" Satisfaction with self & N/A +4.0 -1.4 +12.0 ns personal opportunities
Page 44
• Group success Statistical
significance
Item/Scale Controls High Moderate Failure sig of F K-W
• Social Participation
"Sociability" + 0.3 -0.3 -0.1 -0.3 ns
·SP 18 Participation in N/A +1.2 -1.2 +5.3 ns neighbourhood
"SP2" Amount of contact - N/A +3.1 +0.4 -0.1 ns
• community groups
Non-Cash Income
q6a Make things to swapltrade + 0.6 +1.2 +5.2 +6.3 *
q6b Grow things to swap/trade - 0.1 +2.5 +3.0 +3.9 -q6c Less need for welfare + 0.5 +2.3 +0.5 -5.5 --
• handouts
q6d Receive goods from - 3.0 -2.9 -3.0 +6.6 * friends/relatives
q6e Receive help from - 3.0 -1.7 -3.1 +7.0 * friends/relatives
"NCI1" Frequency strategies N/A -0.2 -2.5 +3.2 ns
• used to cope financially
"NCI3" Own repairs, N/A -6.0 -4.4 -2.5 ns maintenance
Informal Labour Market
q7a Work in exchange for + 0.5 +1.6 -0.9 +7.1 ns
• goods/services
q7b Sell made/grown goods + 0.2 +5.0 -3.5 +3.6 --q7c Volunteer, Community -1.3 -2.5 +1.6 +7.7 * group/Co-op
• q7d Other voluntary work - 0.3 -2.6 -1.2 +0.8 ns
q7e Unpaid work experience + 0.9 +2.8 -2.8 +6.0 *
"ILM" Participation in non-work N/A +2.18 +3.6 ns activities
Family Relationships -Satisfaction
• q4a Well being of children - 0.1 -3.5 -1.7 +8.9 ns
q4b Rlship with partner - 1.5 -1.9 0.0 0.0 ns
q4c Rlship with children - 1.3 -2.3 -1.9 +5.3 ns
q4d How well children get along - 1.7 -0.3 -0.7 0.0 ns
q4e Partner's r/ship with children - 1.0 -3.8 +0.6 +3.5 ns
• ns= differences are attributable to sampling error (Le. statistically not significant); *= only a 5% chance
that differences are due to sampling error (Le. p<.05); -=p<.01; ***=p<.OO1
• Page 45
•
•
•
•
•
•
•
•
•
•
Table 5 indicates the mean percentage change on each living standards measure
according to whether the group was considered to be successful or not. To assist in
the interpretation of these figures it is helpful to examine the work preferences measure
as an example of how to read the figures. The figure +2.1 for the control group means
that for all the control group respondents the average improvement in work
preferences was 2.1 per cent. The figure + 1.5 for the Highly Successful group means
that for projects considered to be highly successful the average improvement in the
work preferences score was 1.5%. For the moderately successful groups it was a 4%
improvement and for the failure groups the scores showed an average decline of-
0.8%. All these changes are very n:.odest and the measure of statistical significance
means that any of the differences in the change for the four groupings (control, high
success, moderate success and failures) were not statistically significant - that is, any
differences between groupings is probably due to chance alone.
Two main points stand out from Table 5. The first is that on all living standards areas
the amount of change (improvement or decline in living standards) is slight. While it
may not be reasonable to expect substantial change over a relatively short period the
modest levels of change observed are such that they may well be attributable to
random measurement error as much as representing real change.
Secondly, the hypothesis that successful projects will lead to greater improvement in
living standards was not supported
• On 21 out of 31 measures there were no statistically significant differences in
living standards change between the control group, and successful or
unsuccessful groups.
• On most measures the control group changed in a similar way to the highly
successful groups.
• On these 21 measures the change in the successful groups was no different to the
change in the groups that the DSS qualitative analysis classified as failures.
Page 46
•
•
•
•
•
•
•
•
•
•
• More surprisingly, where there were differences in living standards change
according to the success of the group it was the failure groups in which there
was greatest improvement in living standards.
• This counter intuitive pattern was evident in relation to the access to infonnation
areas, some infonnal labour market variables and some non cash income
measures where those in failed groups indicated that they were doing better in
these areas.
These results must be treated with a degree of cau~ion. In particular the better
outcomes among those from failed groups is difficult to understand or accept. It is
likely that the quality of data from these failed groups is contributing to these patterns.
It can be argued that participants in failed groups may have taken the task of
completing the questionnaires less seriously than those from successful and positive
groups .
6.2 First order outcomes and second order opportunities
Projects can be distinguished in tenns of whether they achieve first or second order
outcomes. That is a group may meet the immediate needs of the target group (first
order outcomes). It may also achieve second order opportunities through the teaching
of new skills , linking into networks or by providing access to new or additional
opportunities. Typically second order outcomes will build on' first order outcomes .
The distinction between first and second order outcomes
'highlights an important difference between the provision of a program or
direct service by government or the community sector and the process of
equipping communities to respond to changing needs and enhance social
capital. To provide first order outcomes resolves concrete needs but could
result in welfare dependency if provided alone and isolated from new
opportunities; to achieve first order outcomes and facilitate second order
opportunities could provide people with an important means to achieve greater
self reliance' (Smith, Herbert & Roche, 1997; 16) .
Page 47
•
•
•
•
•
•
•
•
•
•
6.2.1 Hypothesis
Projects that were judged to be very successful in achieving both first order outcomes
and second order opportunities would be expected to have a greater impact on living
standards than those that achieved first order outcomes only or achieved the outcomes
to only a limited extent.
6.2.2 Results
• On 28 of 31 measures the classification of groups according to types of outcomes
was not linked to changes in living standards.
• The three measures on which the outcome classification was linked to living
standards change were two measures of informal labour market participation and
one measure of the capacity to generate non-cash income. .
• In each of these cases the projects that led to the greatest improvement in these
living standards measures had medium second order opportunities and at least
medium first order outcomes.
Page 48
• • • • • • •
Table 6: Living standards change by achievement of first order outcomes and second order opportunities
Item/Scale
Formal Labour Market
"Work preferences"
"FLM1" Attitude to seeking work
"FLM2" Money eaming activities
Reduced Costs
·Self help"
Access to Information
"Information satisfaction"
"AI1"
Personal Well Being
~t.g!.f~(§i.n§f~'gIQnf;;it\!i':;:!:!!:!\;;;;f;;;'!;;};il~;m~!.~@;:l
"Mastery"
"WB1" Emotional/physical w/b
"WB2" Satisfaction with self & personal opportunities
Controls
Base N (max)
+ 2.1
N/A
N/A
- 4.6
- 2.1
N/A
- 0.1
+ 0.3
N/A
N/A
medium 1s1 & limited 2nd
64
+1.6
-7.8
+18.2
-4.8
-3.0
-2.6
-2.0
-3.6
-9.4
-1.0
Outcomes
medium 1 si & Sig 1 si & medium 1 si & medium 1 si
medium 2nd medium signif 2nd only
119
-1.8
+1.2
-0.4
-6.1
2nd
32
+4.2
-1.8
-2.3 +3.7
~fltl;ifrl'id~il~lrl + 3.3
+0.5 -3.2
-0.6 +5.0
+1.8
+4.2
143
+3.8
+1.1
+3.3
-0.1
-1.4
+2.9
+2.1
. +1.5
+5.6
+0.9
109
+4.3
+0.2
+0.3
-2.0
-1.4
+0.2
-.-0.3
-3.5
-1.4
• • •
statistical significance
sig of F K-W
ns
ns
ns
ns
ns
ns
ns
ns
ns
ns
Page 49
• •
Item/Scale
Social Participation
"Sociability"
"SP1" Participation in neighbourhood
"SP2" Amount of contact -community groups
Non-Cash Income
q6a Make things to swapltrade
q6b Grow things to swapltrade
•
;g§~U'~~$lo~~~tQr;w.~if~re.:!fl~'n~Qq!i;
q6d Receive goods from friends/relatives
q6e Receive help from friends/relatives
"NCI1" Frequency strategies used to cope financially.
"NCI3" Own repairs, maintenance
•
Controls medium 1st & limited 2nd
+ 0.3 -3.5
N/A -1.8
N/A +4.2
+ 0.6
- 0.1 0.0
+ 0.5 +3.1
- 3.0 -3.1
- 3.0 -1.2
N/A -2.8
N/A -17.4
• •
Outcomes
medium 1St & Sig 1st & medium 2nd medium
2nd
-1.0 +2.0
+0.3
-2.5
+4.2 --+1.6 0.0
+2.4 -0.8
'IBIt2qi. -1.5
.-3.0 +3.0
0.0
-8.2
• • • •
statistical significance
medium 1St & medium 1st sig of F K-W signif 2nd only
+1.1 0.0 ns
+3.3 -1.6 ns
+5.8 +0.4 ns
-0.3 +6.2 *
+3.8 _11\till~ ns
+1.9 +0.2 ns
-3.6 8%·,,1 ns
-1.7 -3.5 ns
-0.9 ns
-6.4 ns
Page 50
• • • • • • • • • •
Outcomes statistical significance
Item/Scale Controls medium 1st & medium 1st & Sig 1st & medium 1st & medium 1st sig of F K-W limited 2nd medium 2nd medium signif 2nd only
2nd
Infonnal Labour Market
q7a Work in exchange for + 0.5 -0.8 +5.5 0.0 +0.3 -1.3 ns goods/services
q7b Sell made/grown goods + 0.2 -0.8 +5.0 +7.6 +7.6 -3.1 ***
q7c Volunteer, Community - 1.3 -0.8 -2.4 +3.8 -3.8 +2.0 ns group/Co-op
q7d Other voluntary work - 0.3 +1.2 -4.8 -2.3 -1.4 +0.6 ns
q7e Unpaid work experience + 0.9 -4.2 +7.7 -6.8 +4.1 -2.9 -"ILM" Participation in non-work N/A -1.9 +1.0 +4.9 ns activities
Family Relationships -Satisfaction
q4a Well being of children - 0.1 -3.2 -7.4 -6.0 +1.0 -2.8 ns
q4b Rlship with. partner - 1.5 +2.0 .-2.0 -8.0 -1.2 -1.4 ns
q4c Rlship with children - 1.3 -2.3 -5.5 -1.0 +0.7 -3.6 ns
q4d How well children get along - 1.7 4.0 -4.4 -5.6 2.1 -1.8 ns
q4e Partner's r/ship with children - 1.0 -3.6 -5.7 -1.4 -3.7 +0.8 ns
ns= differences are attributable to sampling error (i.e. statistically not significant); *= only a 5% chance that differences are due to sampling error (Le. p<.05); **=p<.01;
***=p<.OO1
Page 51
• 6.3 Did the project target the particular LSA?
• 6.3.1 Hypothesis:
Since projects were designed to improve specific living standards areas we would
expect to see greater improvement in living standards where the project was
specifically targeting that living standards area.
• 6.3.2 Results
Table 7: Living standards change by project targeting
• Group targeted LSA Statistical significance
Item/Scale Controls Yes No sig of F K-W
FOnDal Labour Market
"Work preferences" + 2.1 +2.2 +2.2 ns
• Reduced Costs
"Self help" - 4.6 -3.2 -2.4 ns
Access to InfonDation
"Information satisfaction" - 2.1 -1.4 -1.8 ns
Personal Well Being
• "Life satisfaction" - 0.1 +1.1 +0.03 ns
"Mastery" + 0.3 +0.9 -0.2 ns
Social PartiCipation
"Sociability" + 0.3 -0.7 +0.2 ns
• Non-Cash Income
q6a Make things to swap/trade + 0.6 +6.1 +1.0 ns
q6b Grow things to swap/trade - 0.1 +5.2 +1.8 -q6c Less need for welfare + 0.5 +1.5 +1.9 ns handouts
• q6d Receive goods from - 3.0 -1.3 -3.5 ns friends/relatives
q6e Receive help from - 3.0 -0.4 -2.7 friends/relatives
•
• Page 52
•
•
•
•
•
•
•
•
•
•
Group targeted LSA Statistical significance
Item/Scale Controls Yes No sig of F K-W
Infonnal Labour Market
q7a Work in exchange for + 0.5 +8.7 -1.4 ***
goods/services
q7b Sell made/grown goods + 0.2 +10.5 +0.3 ***
q7c Volunteer, Community -1.3 +1.3 -2.2 ns group/Co-op
q7d Other voluntary work - 0.3 -2.5 -2.1 ns
q7e Unpaid work experience + 0.9 +2.9 +0.7 ns
Family Relationships -Satisfaction
q4a Well being of children - 0.1 +0.8 -3.4 ns
q4b Rlship with partner - 1.5 0.0 -1.4 ns
q4c Rlship with children - 1.3 +3.8 -2.8 ns
q4d How well children get along - 1.7 +3.4 -0.9 ns
q4e Partner's r/ship with children - 1.0 -7.4 -2.0 ns
ns= differences are attributable to sampling error (Le. statistically not significant); *= only a 5% chance
that differences are due to sampling error (Le. p<.05); **=p<.01; "'=p<.OO1
Since the special modules were only administered to participants in groups where the
particular living standards area was targeted we do not have responses from the
control group or from non targeting projects for these special modules. Consequently
we can only make comparisons of change for the measures contained in the base
module.
Table 7 provides the comparisons between living standards change in the control group
and projects that did and did not target that living standards area. In general the results
failed to support the hypothesis that targeting will make a difference. The following
main points stand out:
• Targeting in the areas of growing things to swap or trade (not cash), working
for exchange (but not cash) and selling things at local markets did lead to a
significant improvement in these areas.
Page 53
•
•
•
•
•
•
•
•
•
•
• . projects that targeted making things to swap or trade (not for money) and
improvement in family relationships showed some lightly better outcomes but
these were slight and not statistically significant.
• On all the other measures there was just as much change where the living
standards area was not targeted as when it was. That is, the unintended
outcomes were just as great (or small) as the intended outcomes.
• With the exception noted above the control group exhibited as much living
standards change as did projects targeted at a particular living standards area.
6.4 Locally assessed needs?
6.4.1 Hypothesis
To the extent that a project meets the needs of the community it is reasonable to
expect that it is more likely to be successful in achieving its goals. Since the stated
goals were to improve living standards we would anticipate that projects that were able
to identify those needs accurately would have the greater chance of success. On the
assumption that assessing n~eds will produce better outcomes ~han guessing them we
anticipate that projects based on a needs assessment will do better than those where a
needs assessment was not undertaken
Page 54
• • • • • • • • • •
6.4.2 Results
Table 8: Living standards change by type of local needs assessment
Type of local needs assessment Statistical significance
Item/Scale Controls Assessedl Assessedl Guessedl Guessed Expressed Assessed sig K Expressed Guessed Expressed
W
Formal Labour Market
"Work preferences" + 2.1 +0.5 +3.0 +4.4 +3.3 +1.4 +1.4 n s
"FLM1" Attitude to seeking N/A +2.2 -6.4 -2.7 -1.0 ns work
"FLM2" Money earning N/A +3.2 +12.8 --- +0.9 -1.4 n activities s
Reduced Costs
"Self help" - 4.6 -1.5 -4.5 -0.5 -6.3 -4.0 -0.6 n s
Access to Information
"Information satisfaction" - 2.1 -2.6 -4.1 -2.4 +1.5 -2.2 +0.2 n s
"AI1" N/A +4.0 -1.0 +1.1 +6.4 +1.5 -3.7 ns
Page 55
• • • • • • • • • •
Type of local needs assessment Statistical significance
Item/Scale Controls Assessed/ Assessed/ Guessed/ Guessed Expressed Assessed sig K Expressed Guessed Expressed
W
Personal Well Being
"Life satisfaction" - 0.1 +0.8 -3.9 -2.3 +0.1 +2.7 +4.0 ns
"Mastery" + 0.3 +2.6 -3.1 -1.0 -1.5 -0.6 +2.0 n s
"ws 1" Emotional/physical N/A +0.3 -3.0 +4.2 +4.8 +13.3 ns w/b
"WS2" Satisfaction with self N/A +4.8 -5.0 +2.1 +11.2 +9.6 ns & personal opportunities
Social Participation
"SOCiability" + 0.3 +0.1 -0.3 -3.1 +0.04 +1.0 +1.0 n s
"SP1" Participation in NiA +2.9 -1.6 +1.0 +4.0 +2.0 -6.0 ns neighbourhood
"SP2" Amount of contact - N/A' -0.1 +6.0 +1.6 +1.8 +0.1 +5.0 ns community groups
Page 56
• • • • • • e· • • •
Page 57
• • • • • • • • • •
Type of local needs assessment Statistical significance
Item/Scale Controls Assessedl Assessedl Guessedl Guessed Expressed Assessed sig K Expressed Guessed Expressed
W
*
q7c Volunteer, Community - 1.3 -4.0 -O.B -1.6 -1.0 -1.6 +3.1 n group/Co-op s
q7d Other voluntary work - 0.3 -2.4 -4.2 +2.3 -0.2 -2.B -6.B n s
q7e Unpaid work experience + 0.9 +3.2 -2.1 -1.6 -2.5 +B.6 +2.0 n s
"ILM" PartiCipation in non- N/A +9.2 -3.3 +5.0 +6.4 +0.4 ns work activities
Family Relationships -Satisfaction
q4a Well being of children - 0.1 -4;3 +5.0 -2.2 -2.1 -6.1 -3.9 n s
q4b Rlship with partner - 1".5 +0.3 +4.5 -B.5 -1.7 -0.7 0.0 n s
q4c Rlship with children -1.3 -3.0 0.0 -5.6 +1.1 +1.4 -6.0 n s
q4d How well children get - 1.7 -2.5 +2.0 0.0 +4.6 +1.2 -7.0 n along s
q4e Partne(s rlship with - 1.0 -2.5 0.0 -12.B 5.5 -3.7 0.0 * children
ns= differences are attributable to sampling error (Le. statistically not significant); *= only a 5% chance that differences are due to sampling error (Le. p<.05); -=p<.01;
***=p<.OO1
Page 58
•
•
•
•
•
•
•
•
•
•
Table 8 reports the outcomes according to whether there was a needs assessment
involved. The results are by no means clear cut but in general there was little evidence
to support the hypothesis.
• On 24 out of 30 measures l there were no differences in living standards change
according to whether a needs assessment had been undertaken or the form of
that assessment.
• Where there were statistically significant differences in outcomes the more
positive outcomes were likely to be evident where there was at least some
'assessed' element as opposed to reliance purely on guessipg or expressed needs.
• An element of assessment of needs led to statistically significant improvement on
some of the measures of improvement in non cash income and in working for
someone in exchange for goods and services (not cash).
6.5 Service delivery orientation
6.5.1 Hypothesis
The extent to which projects adopted a producer/participant orientation as opposed to
a consumer/client orientation may be expected to affect the. outcomes of a group. To
the extent that a producerlparticipant orientation fosters a sense of responsibility for
oneself, a sense of agency and a feeling of competence it can be expected that this
should have a positive outcome for that person's living standards.
I No measure was used for 'FR2' as there were too few cases in various classifications of needs
assessment.
Page 59
• 6.5.2 Results
Table 9: Living standards change by service delivery orientation
• Service delivery orientation Statistical significance
Item/Scale Controls Mixed COnsumer Producer- sig of K-W - Client PartiCipant F
Formal Labour Market • "Work preferences" + 2.1 +0.6 +3.3 +3.7 ns
"FLM1" Attitude to seeking work N/A -1.8 -2.7 +3.9 ns
"FLM2" Money earning activities N/A +3.5 +0.9 +9.3 ns
Reduced Costs
• "Self help" - 4.6 -3.8 -3.4 -0.4 ns
Access to Information
"Information satisfaction" - 2.1 -2.3 -0.3 -1.6 ns
"AI1" N/A +1.6 +3.2 +3.1 ns
Personal Well Being
• "Life satisfaction" - 0.1 -1.2 +0.2 +2.8 ns
"Mastery" + 0.3 -1.1 +0.2 +2.0 ns
"WB 1" Emotional/physical w/b N/A +2.1 +5.7 +2.0 ns
"WB2" Satisfaction with self & N/A +2.8 +11.2 +4.7 ns personal opportunities
• Social Participation
"Sociability" + 0.3 -2.1 +0.5 +1.9 -"SP1" Participation in N/A -0.5 +1.6 +4.0 ns neighbourhood
"SP2" Amount of contact - N/A +2.7 +2.2 +0.4 ns • community groups
Non-Cash Income
q6a Make things to swapltrade + 0.6 -2.1 +4.0 +7.3 -q6b Grow things to swap/trade - 0.1 +2.4 +0.2 +5.5 -q6c Less need for welfare + 0.5 +1.6 +1.5 +2.4 ns • handouts
q6d Receive goods from - 3.0 -3.2 -5.0 -0.3 ns friends/relatives
q6e Receive help from - 3.0 -1.5 -2.5 -2.7 ns friends/relatives
"NCI1" Frequency strategies used N/A +0.4 -0.3 ns • to cope finanCially
"NCI3" Own repairs, maintenance N/A -9.2 -4.6 ns
• Page 60
•
•
•
•
•
•
•
•
•
•
Service delivery orientation Statistical significance
Item/Scale Controls Mixed Consumer Producer- sig of - Client Participant F
Informal Labour Market
q7a Work in exchange for + 0.5 -1.5 -2.3 +7.9 goods/services
q7b Sell made/grown goods + 0.2 +1.5 -2.2 +9.3
q7c Volunteer, Community - 1.3 -2.8 -0.3 -0.3 group/Co-op
q7d Other voluntary work - 0.3 -2.8 -0.7 -2.8
q7e Unpaid work experience + 0.9 +4.0 -3.5 +1.8
"ILM" Participation in non-work N/A +5.6 -3.8 +3.8 activities
Family Relationships -Satisfaction
q4a Well being of children - 0.1 -3.1 -2.1 -3.6
q4b Rlship with partner -1.5 -1.2 . +1.1 -4.2
q4c Rlship with children - 1.3 -4.2 -1.0 -0.5
q4d How well children get along - 1.7 -1.8 +0.4 +1.4 ns
q4e Partner's r/ship with children -1.0 -6.0 +3.2 -2.8
ns = no statistical significant differences between control group respondents and project participants;
- = differences are statistically significant at the .01 level (Le. p<.01).
Table 9 provides comparisons in living standards change between the control group,
projects that were consumer/client oriented and those that were producer/participant
oriented. Again there was very limited support for the hypothesis.
On 25 out of 31 measures there were no statistically significant differences according
to service delivery orientation. That is, on 25 out of 31 measures projects with a
producer/participant orientation did not lead to any more improvement in living
standards than did the control group or the consumer/client oriented groups.
However, where there were significant outcomes it was in the direction of
producer/participant oriented projects leading to greater improvement in living
standards.
A producer/participant orientation was associated with an improvement in the areas of
sociability (a slight improvement), participation in the informal labour market
Page 61
K-W
--ns
ns
ns
*
ns
ns
ns
ns
•
•
•
•
•
•
•
•
•
•
(working in exchange for goods and services (not cash), selling their own produce,
"ILM", and in generating non cash income (making or growing things to swap or
trade but not for money).
6.6 Existing core group?
6.6.1 Hypothesis
It was anticipated that projects that built on existing community groups would be
more effective than groups that were developed just for this exercise. There are
several reasons for expecting this. Given the short time over which living standards
change was to be measured it was anticipated that groups that were already up and
running would be able to have a more immediate impact.· Furthermore it was
anticipated that existing groups were more likely to be attuned to the needs of the
community and had targeted and appropriate group of participants.
6.6.2 Results
Table 10: Living standards change by type of core group
Type of existing group Statistical significance
Item/Scale Controls Providers Users Providers & Nothing sig of K-W Users F
Formal Labour Market
"Work preferences" + 2.1 +0.7 +5.7 +3.7 +1.8 ns
"FLM1" Attitude to seeking work N/A -3.1 +2.2 -4.5 ns
"FLM2" Money earning activities N/A +3.4 +3.2 +4.3 ns
Reduced Costs
"Self help" -4.6 -3.8 +1.9 -1.4 -7.0 ns
Access to Information
"Information satisfaction" - 2.1
"AI1" N/A +4.9 +1.7 +1.1 +0.8 ns
Personal Well Being
"Life satisfaction" - 0.1 -0.5 -0.3 +1.6 +1.8 ns
"Mastery" + 0.3 -0.3 -2.8 +1.0 +0.4 ns
"WB1" Emotional/physical w/b N/A +3.2 +2.8 -0.8 ns
"WB2" Satisfaction with self & N/A +5.2 +4.8 +3.1 ns personal opportunities
Page 62
• Type of existing group Statistical
significance
Item/Scale Controls Providers Users Providers & Nothing sig of K-W Users F • Social Participation
"Sociability" + 0.3 -1.2 +2.5 +0.4 +0.6 ns
"SP1" Participation in N/A +0.4 +0.9 +0.9 +10.4 ns neighbourhood
• "SP2" Amount of contact - N/A +2.5 +4.4 +1.2 +4.2 ns community groups
Non-Cash Income
q6a Make things to swapltrade + 0.6 +1.3 +5.4 +3.1 +2.3 ns
q6b Grow things to swapltrade - 0.1 +3.7 +1.1 +2.7 -4.5 -• q6c Less need for welfare + 0.5 +1.6 -1.1 +2.3 +1.5 ns
handouts
q6d Receive goods from - 3.0 -3.9 -7.6 -1:3 -3.0 ns friends/relatives
q6e Receive help from - 3.0 -4.4 -2.2 +0.1 +0.8 ns friends/relatives
• "NCI1" Frequency strategies N/A -1.5 -2.0 +0.8 ns used to cope financially
"NCI3" Own repairs, N/A -2.4 -5.2 -7.1 ns maintenance
Infoonal Labour Market
• q7a Work in exchange for + 0.5 -2.5 +2.2 +5.0 -0.8 -goods/services
q7b Sell made/grown goods + 0.2 -3.1 +16.3 +7.6 +4.5 ***
q7c Volunteer, Community -1.3 +1.2 +4.3 -5.3 0.0 ns group/Co-op
q7d Other voluntary work - 0.3 +1.3 +2.2 -6.4 -4.7 * • q7e Unpaid work experience + 0.9 -1.5 +12.5 +3.8 -2.3 *
"ILM" Participation in non-work N/A +0.2 +6.4 +2.7 -3.7 ns activities
Family Relationships -Satisfaction
• q4a Well being of children - 0.1 -1.4 -9.7 -3.4 -6.0 ns
q4b Rlship with partner - 1.5 +2.6 0.0 -4.2 -9.1 ns
q4c Rlship with children - 1.3 -1.9 +6.9 -3.7 -1.9 ns
q4d How well children get along - 1.7 +1.9 +3.6 -2.6 -4.4 ns
q4e Partner's r/ship with children - 1.0 +2.0 -4.7 -6.5 -1.5 ns
• ns = no statistical significant differences between control group respondents and project participants;
- = differences are statistically significant at the .01 level (Le. p<.01).
• Page 63
•
•
•
•
•
•
•
•
•
•
Table 10 compares the extent of living standards change between the control group
and projects that built on existing provider and user groups and projects that were not
based on an existing core group.
There was only very limited support for the hypothesis that building on existing groups
will lead to greater positive living standards change.
• On 26 of 31 measures there was no difference in living standards change between
the control group, projects that were based on an existing core groups and brand
new groups.
• However where there were statistically significant effects the important
difference was not whether a project was based on an exi·sting group or not but
the type of existing group on which it was based. In, these cases the projects
which were based solely on existing user groups did better than those that
involved pre-existing provider groups.
• On all the measures of informal labour market the projects based on existing
user groups showed positive improvement. In these same areas provider based
projects did not do so well.
6.7 Activity range of group
6.7.1 Hypothesis
The range of activities of a group may be expected to be related to the effectiveness of
a group. Since individuals will differ in their circu,mstances and needs one may
anticipate that projects that have diverse activities will better be able to cater for the
range of individual needs. To the extent that this occurs one would expect that groups
that have several activities will be more successful in improving the living standards
of participants.
Page 64
• 6.7.2 Results
Table 11: Living standards change by project activity range
• N of activities Statistical significance
Item/Scale Controls One Two 3 or more sig of F K-W
Fonnal Labour Market
"Work preferences" + 2.1 +4.2 -1.1 +4.8 -• "FLM1" Attitude to seeking work N/A -1.0 -2.5 -0.7 ns
"FLM2" Money earning activities N/A -1.5 +8.0 +2.4 ns
Reduced Costs
"Self help" - 4.6 -4.4 -4.9 -0.1 ns
• Access to Infonnation
"Information satisfaction" - 2.1 +0.2 -0.7 +3.0 ns
"AI1" N/A +0.1 +4.2 +2.6 ns
Personal Well Being
• "Life satisfaction" - 0.1 +0.2 +0.1 +0.6 ns
"Mastery" + 0.3 -1.2 -0.7 +1.4 ns
"WB1" Emotional/physical w/b N/A -0.5 +6.8 +0.7 ns
"WB2" Satisfaction with self & N/A +4.3 +9.0 +2.5 ns personal opportunities
• Social Participation
"Sociability" + 0.3 -1.6 -0.4 +0.3 ns
"SP1" Participation in neighbourhood N/A +3.9 -0.7 +1.9 ns
"SP2" Amount of contact - N/A +4.0 +3.5 +0.8 ns community groups
• Non-Cash Income
q6a Make things to swapltrade + 0.6 +1.3 +3.8 +1.2 ns
q6b Grow things to swapltrade - 0.1 +1.6 +3.1 +2.5 ** q6c Less need for welfare handouts + 0.5 +2.6 +1.8 +1.4 ns
q6d Receive goods from - 3.0 -3.2 -2.7 -3.1 ns
• friends/relatives
q6e Receive help from - 3.0 +0.3 -4.0 -1.1 ns friends/relatives
"NCI1" Frequency strategies used to N/A -0.9 +0.5 ns cope financially
"NCI3" Own repairs, maintenance N/A -6.3 -5.3 ns
•
• Page 65
•
•
•
•
•
•
•
•
•
•
N of activities Statistical significance
Item/Scale Controls One Two 3 or more sig of F K-W
Informal Labour Market
q7a Work in exchange for + 0.5 -1.6 +3.6 -0.8 goods/services
q7b Sell made/grown goods + 0.2 +0.3 +2.8 +3.3
q7c Volunteer, Community group/Co- - 1.3 +0.7 -2.2 -1.3 op
q7d Other voluntary work - 0.3 +0.7 -3.4 -1.9
q7e Unpaid work experience + 0.9 -2.3 +1.2 +2.5
"ILM" Participation in non-work N/A activities
Family Relationships -Satisfaction
q4a Well being of children - 0.1 -0.6 . -3.0 -3.6
q4b Rlship with partner - 1.5 -3.2 -1.8 -0.4
q4c Rlship with children -1.3 +1.0 -4.4 -1.2
q4d How well children get along -1.7 0.0 +0.7 -1.4
q4e Partner's r/ship with children -1.0 +3.9 -0.7 -5.7
ns = no statistical significant differences between control group respondents and project participants;
- = differences are statistically significant at the .01 level (Le. p<.01).
Table 11 compares the level of living standards change of the control group
participants with that of those in single, dual and multiple activity groups. Again there
was little support for the hypothesis that activity level would affect the level of livin~
standards change.
• On 27 of 31 measures there was no difference in the living standards change of
the control group or the projects with different range of activities.
• Three of the four measures in which activity range is linked to living standards
improvement are informal labour market measures non cash income.
• On the four measures on which activity range does make a difference the effect is
small and not especially meaningful. On three of the measures projects with two
activities do better than those with one or three activities. On the other measure
Page 66
*
*
ns
ns
ns
ns
ns
ns
ns
ns
•
•
•
•
•
•
•
•
•
•
the projects with three or more activities did best (but only marginally better than
single activity projects).
6.8 Adaptability/flexibility of the project
6.8.1 Hypothesis
It is reasonable to expect that projects that are flexible and responsive to the needs of
participants will have a greater positive impact and greater success in achieving their
goals than will inflexible groups that are not able to readily adapt to participant
needs.
6.8.2 Results
Table 12: Living standards change by project flexibility
Flexibility Statistical significance
Item/Scale Controls High Some Low sig of F K-W
Formal Labour Market
"Work preferences" + 2.1 +1.7 +2.8 +6.5 ns
"FLM1" Attitude to seeking work N/A +0.8 -1.9 -5.4 ns
"FLM2" Money earning activities N/A +2.7 +4.7 +1.4 ns
Reduced Costs
"Self help" - 4.6 -1.0 -4.5 -6.7 ns
Access to Information
"Information satisfaction" - 2.1 -1.4 -2.7 +1.5 ns
"AI1" N/A +3.0 -1.2 +11.2 -Personal Well Being
"Ufe satisfaction" - 0.1 +1.0 -1.3 +2.3 ns
"Mastery" + 0.3 +1.2 -1.4 +1.4 ns
"WB1" Emotional/physical w/b N/A +2.4 -1.7 +8.7 ns
"WB2" Satisfaction with self & N/A +3.0 +3.4 +11.8 ns personal opportunities
Social Participation
"Sociability" + 0.3 +0.4 -1.7 +1.9 ns
"SP1" Participation in N/A +1.7 -1.4 +5.2 ns neighbourhood
"SP2" Amount of contact - N/A +3.0 +3.1 -1.5 ns community groups
Non-Cash Income
Page 67
•
..
•
•
•
•
•
•
•
•
Flexibility Statistical significance
Item/Scale Controls High Some Low sig of F
q6a Make things to swap/trade + 0.6 +2.6 +2.3 +6.7
q6b Grow things to swap/trade - 0.1 +4.0 0.0 +8.6
q6c Less need for welfare handouts + 0.5 +1.8 +1.7 -1.9
q6d Receive.goods from - 3.0 -3.6 -2.6 -2.9 friends/relatives
q6e Receive help from - 3.0 -1.6 -2.6 -4.8 ns friends/relatives
"NCI1" Frequency strategies used N/A -0.2 +0.4 ns to cope financially
"NCI3" Own repairs, maintenance N/A -6.0 -3.4
Informal Labour Market
q7a Work in exchange for + 0.5 +1.5 -0.3· +5.8 goods/services
q7b Sell made/grown goods + 0.2 +5.6 -0.9 . -3.8
q7c Volunteer, Community -1.3 -3.7 +2.9 -2.9 group/Co-operative
q7d Other voluntary work - 0.3 -3.4 +1.6 -7.7
q7e Unpaid work experience + 0.9 +1.4 +2.1 -1.9
"ILM" Participation in non-work N/A +3.4 +0.4 -3.7 activities
Family Relationships -Satisfaction
q4a Well being of children - 0.1 -3.5 -2.1 +3.3
q4b Rlship with partner - 1.5 -1.9 -2.6 +19.6
q4c Rlship with children - 1.3 -3.1 -2.1 +8.3
q4d How well children get along - 1.7 -1.0 -1.1 +15.9 ns
q4e Partner's r/ship with children - 1.0 -3.7 -3.5 +25.0
ns = no statistical significant differences between control group respondents and project participants;
- = differences are statistically significant at the .01 level (Le. p<.01).
Table 12 compares the extent of living standards change among the control group and
projects according to the level of flexibility of the project group. There was virtually
no support for the proposition that flexible and adaptable groups produce more
positive living standards outcomes.
• On 26 of 31 measures there were no statistically significant differences between
the outcomes of flexible and inflexible projects.
Page 68
K-W
ns
***
ns
ns
ns
ns
***
ns
ns
ns
ns
*
ns
***
•
•
•
•
•
•
•
•
•
•
• The control group did not differ from project outcomes.
• On four measures where the degree of project flexibility was related to living
standards change the change was in the opposite direction to that which was
anticipated.
• That is, inflexible projects exhibited greater change than did flexible groups and
greater change than the control groups. This pattern was most evident in terms
of improvement in family relationships (especially with partner and partner's
relationship with children) and in the non cash income area of growing things to
swap and trade .
Although the pattern was counter to that which was anticipated these results may
reflect the greater focus and direction of the more inflexible groups. It is conceivable
that the highly adaptable groups lost direction and thus did not achieve the goals for
which they were established.
6.9 Type of activity: fivefold project classification
Table 13 reports the extent of living standards change for the control group and for
project groups according to the type of activity in which the group was engaged.
• On 23 of 31 living standards measures the type of activity in the group bore no
relation to the extent or direction ofliving standards change. Nor did the control
group differ from any of the project categories on any of these living standards
measures.
• On eight measures there were statistically significant differences In living
standards change.
• However there is not particularly clear pattern to these changes which may be
summarised as:
Page 69
•
•
•
•
•
•
•
•
•
•
o Those involved in support networks showed a significant increase in life
satisfaction while those in individual case management projects showed a
significant decline.
o Those in LETS showed a significant improvement in the satisfaction with their
access to information, their informal labour market activity (not voluntary
work) and their capacity to generate non cash income (especially making
things to trade).
o Those in individual case management projects showed· statistically significant
improvement in several measures of their capacity to generate non cash
income including trading and getting things from friends and relatives.
o Those involved in collectives and co-operatives showed a decline in their
satisfaction with their access to information, their receipt of goods from
friends and relatives and, relative to others, did less well in terms of selling
things they had produced.
o Community development and education projects displayed almost no
distinctive changes on any measure - either in terms of a decline or
improvement in living standards.
Page 70
• • • • • • • • • •
Table 13: Living standards change by type of project activity
Project category statistical significance
Item/Scale Controls Comm. Collective & LETS Support Individual. sig of F K-W Education & Co-operatives Networks Case
Development. Management
Formal Labour Market
"Work preferences" + 2.1 +2.4 -0.5 +1.6 +6.4 +4.3 ns
"FLM 1" Attitude to seeking work N/A -1.4 +1.3 -10.8 ns
"FLM2" Money earning activities N/A +3.4 +1.9 +10.7 ns
Reduced Costs
"Self help" - 4.6 -3.0 -2.1 -3.5 -2.1 +4.4 ns
Access to Information
"Information satisfaction" - 2.1 +0.2 -4.3 +3.8 -0.2 +2.4 *
"AI1" N/A +3.0 -2.8 -1.4 +18.8 *
Personal Well Being
"Life satisfaction" - 0.1 +0.4 -1.2 +5.8 +8.1 -7.0 ***
"Mastery" + 0.3 +0.3 -0.9 +1.7 +0.6 +4.8 ns
"WB 1" Emotional/physical w/b N/A +3.9 -0.4 +7.3 -5.0 ns
"WB2" Satisfaction with self & N/A +2.9 +7.6 +12.1 -2.7 ns personal opportunities
Page 71
• • • • • • • • • •
Project category statistical significance
Item/Scale Controls Comm. Collective & LETS Support Individual. sig of F K-W Education & Co-operatives Networks Case
Development. Management
Social Participation
"Sociability" + 0.3 +0.1 -2.3 +0.7 +2.8 +0.4 ns
"SP1" Participation in N/A +1.1 -2.3 +6.1 +3.8 ns neighbourhood
"SP2" Amount of contact - N/A +1.6 +5.6 +1.4 -4.4 1ns community groups
Non-Cash Income
q6a Make things to swapltrade + 0.6 +0.5 +2.8 +13.8 +1.9 +10.7 -q6b Grow things to swapltrade - 0.1 +0.4 +5.4 +8.5 +2.0 +11.4 ***
q6c Less need for welfare handouts + 0.5 +0.5 +1.7 +3.5 +2.9 -4.5 ns
q6d Receive goods from - 3.0 -2.8 -5.2 +5.3 -1.9 +13.6 -friends/relatives
q6e Receive help from - 3.0 -1.5 -3.1 +4.4 -6.7 +10.7 ns friends/relatives
"NCI1" Frequency strategies used N/A -0.3 +1.4 -0.7 ns to cope financially
"NCI3" Own repairs, maintenance N/A -7.8 +0.1 -6.5 ns
Page 72
• • • • • • • • • •
Project category statistical significance
Item/Scale Controls Comm. Collective & LETS Support Individual. sig of F K-W Education & Co-operatives Networks Case
Development. Management
Infonnal Labour Market
q7a Work in exchange for + 0.5 -0.1 -1.7 +20.5 -1.9 +3.6 *** goods/services
q7b Sell made/grown goods + 0.2 +2.3 -0.8 +13.6 +2.9 +2.4 ***
q7c Volunteer, Community - 1.3 -1.9 +2.7 +0.4 -3.8 +6.0 ns group/Co-op
q7d Other voluntary work - 0.3 -4.2 +4.9 -5.3 -1.0 +1.2 ns
q7e Unpaid work experience + 0.9 +2.5 +1.7 0.0 +5.8 -10.7 ns
"ILM" Participation in non-work N/A +3.0 +5.6 1.4 -3.7 ns activities
Family Relationships -Satisfaction
q4a Well being of children - 0.1 -1.6 -3.8 0.0 +3.4 -1.6 ns
q4b Rlship with partner - 1.5 -1.0 -3.5 -2.1 +12.5 +2.1 ns
q4c Rlship with children -1.3 -0.3 -5.8 -3.4 +8.3 -1.6 ns
q4d How well children get along -1.7 +0.1 -2.5 0.0 _ +1.3 0.0 ns
q4e Partner's r/ship with children -1.0 -1.7 -2.5 +1.0 0.0 -4.2 ns
ns = no statistical significant differences between control group respondents and project participants; ** = differences are statistically significant at the .01 level (Le. p<.01).
Page 73
•
•
•
•
•
•
•
•
•
•
6.10 Summary of the impact of group characteristics
• Taken overall, the impact of the project characteristics had, at best, a modest
impact on a very limited set of measures of living standards.
• On most measures of living standards project characteristics made no difference
to either the extent or direction of living standards change. Similarly on these
same measures project participants did not change any differently from the
control group respondents.
• Two living standards areas (or some measures of these areas) stood out as being
linked with various characteristics of the projects. These living standards areas
were participation in the informal labour market and the capacity to
generate non cash income.
• The project characteristics that were associated with improvement in one or both
these living standards areas were:
o This living standards area was specifically targeted by the project.
o Having a local needs assessment before establishing the project.
o Having a producer/participant service delivery orientation.
o Building on an existing user group.
o Having more than one activity.
o Being a LETS type project.
• There was virtually no evidence of any project characteristics having any impact
on
o social participation.
o personal well being.
Page 74
•
•
•
•
•
•
•
•
•
•
o formal labour market orientation.
o reduced costs.
o access to information (with the exception of participation in LETS).
o family relationships (except for a non explicable link with the inflexibility of
the project).
7. Individual change
There was evidence of living standards change among individuals and evidence that
individuals in projects were more likely than control group respondents to exhibit
positive change. This evidence is provided in Table 14.
Table 14: Living standards change among individuals: Control group and
project participants compared (per cent having positive change)
Item/Scale control project statistical
group participants Significance
Fonnal Labour Market
"Work preferences" 27.1% 27.3% ns
Reduced Costs
"Self help" 27.3% 32.7% *
Access to Infonnation
"Information satisfaction" 22.6% 31.6% ***
Personal Well Being
"Ufe satisfaction" 25.8% 31.6% -"Mastery" . 26.2% 26.8% ns
Social Participation
"Sociability" 24.3% 24.9% ns
Non-Cash Income
q6a Make things to swap/trade 12.2% 26.2% ***
q6b Grow things to swap/trade 7.7% 21.5% ***
q6c Less need for welfare handouts 4.8% 13.4% ***
q6d Receive goods from 17.1% 25.9% *** friends/relatives
Page 75
•
•
•
•
•
•
•
•
•
•
Item/Scale control project statistical
group participants significance
q6e Receive help from 21.4% 26.3% * friends/relatives
Infonnal Labour Market
q7a Work in exchange for 12.7% 24.7% -goods/services
q7b Sell made/grown goods 5.9% 19.1% -q7c Volunteer, Community group/Co- 15.4% 24.9% -operatives
q7d Other voluntary work 18.5% 28.0% -q7e Unpaid work experience 8.7% 22.4% -
Family Relationships -Satisfaction
q4a Well being of children 22.0% 23.5% ns
q4b Rlship with partner 14.3% 20.5% -q4c Rlship with children 15.7% 20.3% *
q4d How well children get along 17.7% 25.8% -q4e Partner's r/ship with children 16.1% 19.9% ns
ns = no statistical significant differences between control group respondents and project participants;
- = differences are statistically significant at the .01 level (Le. p<.01).
• These results indicate that on many measures of living standards more project
participants than control group respondents showed positive living standards
change.
• On 16 of 21 measures more project participants than control group participants ha~
positive living standards change.
• The differences between control group and project participant change was greatest
in relation to non-cash income and informal labour market. In these living
standards areas all measures showed control group and project differences. There
were also notable differences between the controls and participants in improvement
of access to information.
• The greatest differences between controls and participants related to measures such
as making or growing things to swap or trade, working in exchange for goods or
Page 76
•
•
•
•
•
•
services or selling goods· that the respondent has made or grown. All these were
very specific goals of particular projects and reflect the success of these groups in
achieving these specific and concrete outcomes.
• There were more modest differences between controls and participants in relation
to reduced costs (5.5%), Life satisfaction (5.8%) and aspects of family
relationships (about 5%).
• There were no differences in how many controls and participants showed positive
change in relation to social participation, aspects of family relationships, aspects of
personal well being and labour market participation. As a general observation the
differences between controls and participants were less or absent for the less
concrete dimension of living standards (personal well being, social participation,
work preferences, and aspects offamily relationships).
• This latter pattern is opposite to the initial findings of the qualitative research which
indicated some improvement in the areas of social participation, formal labour
market and personal well being. Of course these figures reflect improvement in
these areas among participants but show no greater change than controls.
Care must be taken in interpreting these figures
• The criterion for change was modest. Using a change score· of 0.5 above the mean
• did not mean that individuals had to change a great deal to be classified as positiv:,.
changers.
• Given the general lack of mean differences between controls and participants (see
• earlier analysis) the fact that more participants changed positively is
counterbalanced by a greater extent of negative change among participants as well.
•
•
• While the percentage of participants that changed positively was greater than
among the controls the differences, while statistically significant, were not great.
Page 77
•
•
•
•
•
•
•
•
•
•
7.1 Social characteristics of participants
The analysis in this section focuses on the characteristics of those whose living
standards scores improved. The question being asked throughout this section is 'Do
the social and personal characteristics of respondents increase the likelihood of
exhibiting positive changes in their living standards after participation in and
ARP?' Or put differently 'Are some types of people (e.g. younger, females,
Australian born, those with more education?) more iikely to benefit from these type of
projects?'
7.1.1 Education level
The educational level of project participants made no difference to the number of
project participants that experienced living standards change.
7.1.2 Income level
The level of family income made no difference to the number of project participants
who exhibited.
7.1.3 Home ownership
Home ownership was related to improvement in a number of areas of living standards
(see Table 15)
Table 15: Positive living standards change by tenure type (per cent positive
change)
Social Participation
"SP1" Social participation in neighbourhood
"SP2" Contact with community groups
Home Ownership
Owner! Renting Not renting purchaser or buying
18% 31% 45%
14% 25% 35%
statistical significance
**
*
Page 78
•
•
•
•
•
•
•
•
•
•
"Sociability"
Non-Cash Income
q6c Less need for welfare handouts
Formal Labour Market
"FLM2" Involved in money earning activity
Home Ownership
Owner/ Renting Not renting purchaser or buying
20% 30% 30%
6% 24% 12%
10% 19% 57%
statistical significance
*
***
**
ns = no statistical significant differences between control group respondents and project participants;
.. = differences are statistically significant at the .01 level (Le. p<.01).
• More of those who were not renting or buying (Le. usually living with parents)
gained on all three measures of social participation over the life of the project.
More of these participants increased their level of social participation than those
owning or purchasing or renting their home.
• More of those living with their parents than home purchasers or renters improved
in their involvement in the formal labour market after project involvement.
7.1.4 Ethnic background
In terms of living standards improvements participants born in Australia were
indistinguishable from those born overseas - both those from English speaking
countries or those from non English speaking backgrounds. The projects were neither
more nor less effective for those from NES backgrounds than for any other group.
The only minor exception to this was a slight tendency for participants who were
English speaking migrants, more than other people, to increase their involvement in
volunteer groups, community groups and co-operatives.
7.1.5 Employment status
Employment status was linked to living standards change but not in any particularly
clear or meaningful pattern (Table 16)
Page 79
•
•
•
•
•
•
•
Table 16: Positive living standards change by employment status (per cent
positive change)
Employment Status Statistical significance
Full Part Not working, Not working, time time but looking not looking
Reduced Costs
"Self help" Frequency -growing, 41% 22% 33% 34% * making, repairing to save money
Infonnal Labour Market
q7a Work in exchange for 37% 29% 29% 16% *** goods/services
q7c Volunteer, Community 16% 28% 32% 21% * group/Co-op
q7e Unpaid work experience 24% 24% 31% 16% **
Family Relationships -Satisfaction
q4c Rlship with children 6% 26% 23% 21% *
Fonnal Labour Market
"FLM1" Attitude to seeking work 14% 44% 16% *
ns = no statistical significant differences between control group respondents and project participants;
*" = differences are statistically significant at the .01 level (Le. p<.01).
• More full-time workers than others, especially part time workers improved their
levels of self help.
• More full time workers than others (especially those not even looking for workL
increased their level of work for goods and services (informal labour market).
• Job seekers exhibited the greatest improvement in volunteer work and involvement
• with community groups and co-operatives while full time workers showed the least
change.
•
•
• Job seekers were also the most likely to show an improvement in their attitude
towards seeking work and gaining unpaid work experience.
Page 80
•
•
•
•
•
•
•
•
•
•
.• Project participants not working full time showed the greatest improvement in
relationships with their children while full time workers showed the least
improvement.
7.1.6 Marital status
Marital status was linked to only two improvements in living standards among project
participants. (Table 17) In each case those who were married showed less
improvement than the never married or the separated/divorced.
Table 17: Positive living standards change by marital status (per cent positive
change)
Social Participation
"SP1" Participation in neighbourhood
Non-Cash Income
q6c Less need for welfare handouts
Never married
36%
16%
Marital Status
Married,! de facto
17%
9%
statistical Significance.
Divorcedl separated
38% **
21% **
ns = no statistical significant differences between control group respondents and project participants;
- = differences are statistically significant at the .01 level (Le. p<.01).
• Fewer married than non married . participants improved 10 their level of
neighbourhood participation.
• Fewer married than non married people reduced their level of reliance on welfare
handouts.
7.1.7 Gender
Gender was generally unrelated to living standards change. Just as many male as
female participants improved their living standards. The only two measures where
there was a gender difference was in relation to:
Page 81
•
•
•
•
•
•
•
•
•
•
• Selling goods that they had made or grown
• growing things to swap or trade
where a greater proportion of females than males improved their living standards in
this regard.
Table 18: Positive living standards change by gender (per cent positive change)
Gender statistical significance
Informal Labour Market
q7b Sell made/grown goods
Non-Cash Income
q6b Grow things to swap/trade
Male . Female
14% 22% *
16% 24% *
ns = no statistical significant differences between control group respondents and project participants;
.. = differences are statistically significant at the .01 level (Le. p<.01).
7.1.8 Age
Age was a factor in whether or not a number of living standards areas improved.
(Table 19). For each of these living standards areas the positive changers were
significantly younger than those who did not experience positive change.
This advantage of being younger was evident in the following living standards domains_
• Social participation (all measures).
• Family relationships (improved relationships with children).
• Informal labour market participation (Work in exchange for goods/services).
• Non cash income (reduced reliance on welfare handouts).
• Personal well being (emotional and physical).
• Formal labour market participation.
Page 82
•
•
•
•
•
•
•
•
•
•
Table 19: Mean age (in years) by living standards change
No positive Positive Statistical change change significance
Formal labour market
"FLM2- Money eaming activities 37.4 years 26.1 years ***
Social Participation
"sociability- 40.0 36.3 -"SP1- Participation in neighbourhood 38.4 34.2 *
·SP2- Amount of contact with community 39.1 31.9 *** groups
Personal well being
"W81- Emotional/physical wellbeing 42.6 35.0 -Informal labour market
q7a Work in exchange for goods/services 39.8 36.9 *
Non cash income
q6c Less need for welfare handouts 40.0 32.7 ***
Family relationships
Relationship with children 42.5 38.0 -ns = no statistical significant differences between control group respondents and project participants;
- = differences are statistically significant at the .01 level (Le. p<.01).
7.1.9 Attendance frequency
To test the reasonable proposition that those who wer~ most involved in the projects
should show the greatest likelihood of living standards improvement we examined..
living standards change by level of involvement.
The level of participation in projects was generally unrelated to living standards
improvement. On only four measures did level of involvement affect outcomes (Table
20). In these cases the occasional attenders usually did better than the regular
attenders or those who rarely used the service and the effect was linked. In three of the
four measures improvement was evident in relation to participation in the informal
labour market.
Page 83
•
•
•
•
•
•
•
•
•
•
Table 20: Per cent positive change by frequency of attendance at the project
Infonnal Labour Market
q7a Work in exchange for goods/services
q7d Other voluntary work
q7e Unpaid work experience
Non cash income
q6b Grow things to swap/trade
Attendance Frequency
Frequent Occasional Infrequent! once
27% 38% 16%
28% 39% 22%
32% 22% 11%
24% 27% 13%
statistical significance
***
*
***
*
ns = no statistical significant differences between control group respondents and project participants;
.. = differences are statistically significant at the .01 level (Le. p<.01).
The duration of involvement in the project made no difference to improvement in living
standards.
7.1.10 Sociable participants
The proposition that people who feel more comfortable in groups - those who feel at
ease in groups and are joiners of groups - would be more likely than others to benefit
from a group approach to improving living standards was tested ..
Table 21: Positive change in living standards by propensity to join community
groups
Infonnal Labour Market
q7b Sell made/grown things at markets
Non-Cash Income
q6b Grow things to Swap/trade
Personal Well Being
I get involved in community group
Very Fairly Not Not at all true true very true
true
20% 24% 17% 7%
23% 26% 20% 9%
Significance
*
*
Page 84
•
•
•
•
•
•
•
•
•
•
"Life satisfaction"
Social Participation
"SP2" Contact with community groups
Fonnal Labour Market
"FLM1" Attitudes to work
Family Relationships
q4a Satisfaction with childrens wellbeing
I get involved in community group Significance
Very Fairly Not Not at all true true very true
true
42% 29% 25% 33%
25% 13%
37% 17%
30% 14%
*
*
*
***
ns = no statistical significant differences between control group respondents and project participants;
.. = differences are statistically significant at the .01 level (Le. p<.01).
Table 21 shows that being a JOiner or a non Jomer made little difference to
improvement in many of the living standards areas. However, on six measures it made
a difference with the pattern in these cases being that joiners benefited more than non
JOiners. The more sociable were, not surprisingly, able to benefit more from a
community group approach to improving living standards.
7.2 Summary
Using the particular measure of change adopted in this ·section (Le. any change greater
than 0.5 of a standard deviation greater than the mean change for that variable) there
was evidence that more project participants displayed improvement on a number of
measures of living standards than did control group respondents. However the earlier
analysis of mean change showed that the extent of change barely differed between
project and control groups. In other words, while more project participants than
control group members displayed living standards improvement these changes were
small and would be counterbalanced by greater negative change among the project
participants as well. The greater extent of negative change among project participants
will have cancelled out the impact of the greater number of project participants whose
living standards improved.
Page 85
•
•
•
•
•
•
•
•
•
•
An examination of the social and personal characteristics of individuals was undertaken
to determine whether these characteristics predicted improvement in living standards.
This analysis showed that:
• Education and income levels and ethnicity had no impact on the capacity of
individuals to benefit from participation in projects.
• Gender made almost no difference to the capacity of individuals to benefit from
participation in projects except that females showed most. improvement in their
capacity manage by making or growing things to sell or trade.
• Employment status made little difference to the living standards change of
participants.
• Housing tenure type made a difference to the likelihood of living standards
improvement in the areas of social participation and aspects of participation in the
formal labour market and the capacity to generate non cash income. Those living
with their parents showed greatest improvement in these areas - a pattern that
partly reflects the young age of these participants.
• Those who benefited from projects were more likely to be younger than those who
did not benefit.
• The level of involvement in projects had only limited impact on the benefit
participants received. Where it made a difference it was in favour of the middling
rather than frequent or rare attenders.
Page 86
•
•
•
•
•
•
•
•
•
•
References
Brownlee, H. (1990), Measuring Living Standards; Australian Living Standards Study
Paper No. 1, Australian Institute of Family Studies, Melbourne.
Cass, B (1986), The Case for Review of Aspects of the Australian Social Security
System, Social Security Review, AGPS, Canberra.
Commonwealth of Australia (1994), Working Nation - Policies ~d Programs, AGPS,
Canberra
DSS (1995), Developing a Framework of Benchmarks of Adequacy for Social
Security Payments, Policy Discussion Paper No. 6, AGPS, Canberra
GilIey, T. (1990), Empowering Poor People, Brotherhood of St Laurence, Melbourne
Hardey, R. (1995), Social Participation and Inclusion, DSS Community Research
Project Discussion Paper No 2 (Draft May 1995), unpublished paper
Maher, J and Smith, B. (1996) Future Social Provision: The DSS Community
Research Project. DSS Research Paper No. 69, AGPS, Canberra
McDonald,P., Moyle, H. and Gibson, D. (1995) Contributions of Cash and Non-Cash
Elements to the Enhancement of Living . Standards, DSS Community
Research Project Discussion Paper No. 1 (Draft, June 1995), unpublishe~
paper
DECD (1988a), The Future of Social Protection, in DECD Social Policy Studies No.
6, DECD, Paris
OECD (1988b), OECD Employment Outlook, OECD, Paris.
Salvaris, M. (1995), Citizenship, social rights and constitutional reform, in P Smyth
(ed), An Australian Republic: Participation and Citizenship, Discussion Paper
No. 7, Uniya, Sydney.
Page 87
•
•
•
•
•
•
•
•
•
•
Smith, B, Herbert, J. & Roche, S. (1997) 'Future Social ProvisIon: Community Based
Initiatives Gateways to Opportunities', Paper presented to Beyond
Dependency Conference, Aukland, New Zealand, March 1997
Townsend, P. (1979), Poverty in the United Kingdom: A Survey of Household
Resources and Standard of Living, Penguin, Harmondsworth
Travers, P. and Richardson, S. (1993), Living Decently - Material Well-Being in
Australia, Oxford University Press, Melbourne
Page 88
![Η ΧΡΥΣΗ ΤΟΜΗ Ο ΑΡΙΘΜΟΣ ΤΟΥ ΦΕΙ∆ΙΑ Φ ΚΑΙ Η ...users.sch.gr/elscardan/Usefully/xr_tomh.pdfΟ ΑΡΙΘΜΟΣ ΤΟΥ ΦΕΙ∆ΙΑ [Φ] ΚΑΙ Η ΑΚΟΛΟΥΘΙΑ](https://static.fdocument.pub/doc/165x107/5e34481829f5d93c544304e6/-oe-oe-a-usersschgrelscardanusefullyxrtomhpdf.jpg)
