Research on and with the Outcomes Framework and MARA
There is much work to be done to develop the Outcomes Framework and MARA as assessment tools that have the highest levels of validity and reliability and are most useful for professionals and those who fund them. Contributions from DM therapists and researchers internationally are welcomed on all of these projects. Contact Kim Dunphy, firstname.lastname@example.org to express your interest.
The project currently has ethics clearance from the University of Melbourne (Number 1647380.3) that covers most of the topics below.
Current research projects:
- Development of the Outcomes Framework: content, definitions
Researchers: Kim Dunphy (Australia) with DM therapists internationally.
- Reliability testing of the Outcomes Framework
Researchers: Kim Dunphy (Australia) and Sabine Koch (Germany), February 2020 with DM therapists internationally.
How to contribute: respond to our survey: https://melbourneuni.au1.qualtrics.com/jfe/form/SV_ai7uceR3rYjEQuh
Timing: current data gathering round closes 23 February, 2020.
- Development of a short version of the Framework, that will include only outcomes that all DM therapists might expect for their clients. This will open the possibility that outcome data about DMT could be gathered and shared across contexts and sites.
Researchers: Kim Dunphy and Sabine Koch,
Timing: February-June 2020.
- Development of a Portuguese version of the Framework and MARA,
including validity testing of the translation.
Researchers: Kim Dunphy (Australia) and Paula Lebre (Portugal).
Timing: January-June 2020.
Research projects as yet to be undertaken:
- Validation tests
- Content validation: experts scoring on of the Framework’s assessment elements: relevance, clarity, simplicity, ambiguity
- Intra-rater agreement of scores: How much consistency is there in ratings given by the same person across multiple instances
- Inter-rater reliability and intra-rater agreement of scores: How much agreement is there in the scores of different DM therapists ? people in different professions (DM therapist, psychomotor therapists, student, untrained assistant)?
- Cultural differences in scoring: How do the scores of therapists from different cultures, or with clients from different cultures, align or not?
- Impact of timing on assessment: Is assessment scoring impacted by its timing: such as a controlled situation of watching video clip? In normal situation during a session?
- Impact of background knowledge of client on assessment: are assessment scores impacted by assessors’ background knowledge of clients?
- Alignment with other assessment tools: How do the elements in the Framework align with assessment parameters from other relevant tools? Can the Framework be validated against them
- DMT Program Planning:
Group profile: when planning a program, it is possible to average intake scores of individual clients to create a group profile that help the leader to decide on objectives. What is the implication of this process? Does it support effective planning?
Setting objectives for different client groups: how are these decided upon? Which Framework elements are chosen for which clients, in which settings? How do these align with objectives of host organisation/funder (or not)?
- Assessing: process
Non-norm referenced scoring: MARA’s assessment process requires the therapists to estimate what the client’s potential is on every element includes the possibility of changing the scale of clients’ maximum capacity over period of time. Is this a reasonable consideration?
Length of observations needed: how long should observations and /or video used for assessment observations be? MARA currently enables video clips of 15 seconds..
Developing client perspective in assessment: can MARA also be used to stimulate clients’ engagement and agency in the therapeutic process?
For clients who have communication preferences that are non-verbal and may not involve reading and writing, how can media options of photos, videos and drawings potentially support their engagement in assessment of their own progress?
Eco-sytematic approach to assessment: how can MARA support the contribution of other stakeholders, including other staff in agencies and families, in the assessment process?
Reflective cycle in DMT: What are the implications of using such an assessment tool? For practice, does it become part of a reflective cycle? What mights its contribution be to decision-making for funding?
- Assessment findings:
What are outcomes of DMT programs for various client groups, as assessed using MARA and the Framework, tested in an RCT?
What would a report from MARA look like that was useful for funders, host agency and clients and families of DMT programs? How would reporting using data such as gathered through MARA, impact agencies’ understanding and valuing of DMT programs?
- Training of therapists:
What are the implications of the use of such an instrument on student learning?
What does a process of assessing using a formal tool such as MARA contribute to students’ capacity: for observation, for planning, for sensitive leadership?
- Other languages
Translation of the Outcomes Framework and MARA into Korean and Chinese, including validity testing of the translation.
- Other professions
Trialling of the Outcomes Framework and MARA app as assessment tools that could be useful for other therapy professions: drama therapy, art therapy, music therapy, physiotherapy, occupational therapy, psychomotor therapy, psychotherapists and counsellors.
- Other platforms
Development of MARA on other platforms: tablets, computers, phones.
- Client specific software
The development of a version of MARA that is for therapy clients, that could be used in or between sessions to advance therapeutic objectives. This might involve communication with the therapist, between client and caregivers/families, or documentation of self-directed activities that can be reflected on in therapy sessions.
- Client videos:
Development of a library of videos of clients, offering examples of assessment scoring, to help improved therapists assessment capacity and inter-rater reliability
- Literature bank:
Development of a bank of literature to substantiate all the elements in the Framework
- Bank of activities:
Development of a bank of data about activities that contribute to outcomes
- Transferability of outcomes from DMT: How could outcomes of DMT evidenced in sessions, as assessed by MARA, transfer into everyday life? What can a therapist do to advance transference?