Dr. Bernie Pauly and colleagues at the University of Victoria in British Columbia have published a highly readable paper about research methods in homelessness programs. In it, they talk about three things: a case study evaluation of a homeless transitional shelter in Victoria, other evaluations of homelessness services, and the contribution that research can make to ending homelessness.
Using a case study to evaluate a homeless transitional shelter
The Victoria transitional shelter was a new program that provided temporary housing and support. The support included case management to assist residents to move from the shelter to permanent housing. Victoria had (and continues to have) an expensive rental market with few vacancies. The agency operating the program wanted an evaluation that would help them (and their funders) understand how and why the transitional shelter worked in that environment.
A case study methodology was used to complete the evaluation. In the Pauly research paper, the authors explain that using a case study approach to evaluate a program involves making a number of decisions. As one example, the evaluators had to decide whether to include more than one site besides the Victoria transitional shelter where programs were offered to people who were homeless. In this evaluation, they decided to look at the Victoria transitional shelter on its own.
The research team designed their evaluation using four building blocks: affordable housing, adequate income, community supports and health care supports. The evaluation was organized to assess how the activities of the program would affect success for their clients in each of these blocks.
Data from multiple sources was collected to provide a rich understanding of how the transitional shelter and its supports helped the clients who used the program. The data included the outcomes for individual clients, client surveys, a demographic profile of people using homelessness services in Victoria, and vacancy and rental data for the area.
The study involved 150 clients and ran for five years. The evaluation was able to demonstrate how it helped the individual people who used the program. At the front end, some of the people in the program were able to avoid becoming homeless because the program included transitional housing. Some of the people who used the program moved on to permanent housing. In the group that moved to housing, most reported they had more income to pay for their living expenses than when they entered the program. Despite having higher incomes, the clients generally did not have enough income to pay the rent in the most affordable units in the private market.
The evaluation was also able to demonstrate that more affordable housing was needed and that basic income supports needed to rise. The evaluation finished up with recommendations of what needed to change to be able to have better success in moving to stable, permanent housing from homelessness. For more on this evaluation try: Researchers Issue A Challenge To Make Housing First Work Better
Evaluations of other homelessness programs
The Pauly research paper also looks at the research methodologies used to evaluate ‘Housing First’ and ‘transitional housing’ programs.
Evaluations of Housing First zero in on changes at the individual client level. The Housing First studies have tended to be randomized control trials with many study subjects. Some look at one location, while others look at several.
The At Home/Chez Soi project is one example. It evaluated the use of Housing First in five different Canadian communities, with over 2,000 study subjects. It was able to demonstrate that individual clients moved to permanent housing with the support of the program. For more on the At Home/Chez Soi project, see at the Mental Health Commission of Canada: National At Home/Chez Soi Final Report
The Pauly research paper also reports on several evaluations of other transitional housing programs. These evaluations tended to look at a single site, just as the evaluation of the transitional shelter in Victoria did. However, the evaluations of other transitional housing programs zeroed in on the outcomes for individual clients to evaluate success. In this they were similar to Housing First evaluations.
Why use different research methods?
The Pauly paper describes how different research methods contribute to our knowledge of homelessness programs in different ways.
The results of the Housing First research have been able to focus attention on the benefit of a community adopting a Housing First program. Once housed, some people are able to maintain housing on their own. Others will be able to maintain their housing on their own after a period of receiving temporary support. A third group will require support on an ongoing basis. The results of Housing First evaluations have persuaded governments to make Housing First a central focus of public funding for homelessness services. This is very important because it directed attention to a strategy that has been proven to work.
The case study of the Victoria transitional shelter illustrates why Housing First won’t always be able to end homelessness on its own. There wasn’t any affordable housing in the private market that clients could rent at their level of income. When Housing First programs face the same roadblock, attention also needs to focus on changing the income support programs and housing programs.
Finally, the Pauly research paper discusses the potential for case studies to explain what is observed. Perhaps appropriately, this is called realist theory. Other people are working on developing realist theory and have identified concepts that form its backbone. The authors of the Pauly research paper describe five of the realist theory concepts: embeddedness, mechanisms, contexts, regularities and change. They also demonstrate how the concepts could be examined in the case study evaluation.
Why does all this matter?
Here are some thoughts about why a paper about homelessness research matters in a blog about affordable housing.
Homelessness research has helped direct funding to programs that actually help people. To some extent, it has also helped to increase the amount of money spent on those programs. Homelessness research also points out how gaps in the housing market can limit the success of programs
In the field of affordable housing, we haven’t come across research about how affordable housing could be delivered in significant quantity to make a difference. Some would argue that its complexity makes it impossible to evaluate. But Homelessness is also a complex issue. We might learn something about how to evaluate housing interventions from homelessness research.
Beyond that, if you’re interested in the limits of different kinds of research approaches, this paper could be considered a must read. It doesn’t cover every kind of research, but you’ll come away with a much better understanding of some of the ways an issue can be investigated.
For research skeptics who cleave to the expression: there are three kinds of lies: lies, damn lies and statistics, this article provides thoughtful insight.
Here is a link to the abstract of the article, which is published in Housing Care and Support: Approaches to evaluation of homelessness interventions. For more details (and don’t have access to the journal) you can reach out to Dr. Pauly at bpauly@uvic.ca
